mirror of
https://github.com/zulip/zulip.git
synced 2025-11-03 21:43:21 +00:00
docs: Split /production/deployment.html into multiple pages.
- Makes "Deployment options" easier to navigate by splitting the
"Reverse proxies" and "System configuration" sections out into
dedicated pages.
Fixes #28928.
(cherry picked from commit 2b95068406)
This commit is contained in:
David Rosa
Tim Abbott
2
docs/_templates/layout.html
vendored
2
docs/_templates/layout.html
vendored
@@ -6,7 +6,7 @@
|
||||
# version e.g. to say that something is likely to have changed.
|
||||
# For more info see: https://www.sphinx-doc.org/en/master/templating.html
|
||||
#}
|
||||
{% if pagename in [] and release.endswith('+git') %}
|
||||
{% if pagename in ["production/system-configuration", "production/reverse-proxies"] and release.endswith('+git') %}
|
||||
{#
|
||||
# This page doesn't exist in the stable documentation yet.
|
||||
# This temporary workaround prevents test failures and should be removed after the next release.
|
||||
|
||||
@@ -335,7 +335,7 @@ _Released 2023-08-25_
|
||||
- Fixed a bug, introduced in Zulip Server 7.2, when the
|
||||
[email gateway](../production/email-gateway.md)
|
||||
was used in conjunction with a
|
||||
[reverse proxy](../production/deployment.md#putting-the-zulip-application-behind-a-reverse-proxy).
|
||||
[reverse proxy](../production/reverse-proxies.md).
|
||||
- Improved the performance of
|
||||
[resolving](https://zulip.com/help/resolve-a-topic) or
|
||||
[moving](https://zulip.com/help/move-content-to-another-topic) long topics.
|
||||
@@ -394,7 +394,7 @@ _Released 2023-07-05_
|
||||
`X-Forwarded-Proto` is also necessary.
|
||||
|
||||
- Removed [reverse proxy][proxies] nginx configuration files when the
|
||||
[`loadbalancer.ips`](../production/deployment.md#ips)
|
||||
[`loadbalancer.ips`](../production/system-configuration.md#ips)
|
||||
setting has been unset.
|
||||
- Improved error-handling of scheduled emails, so they cannot attempt infinite
|
||||
deliveries of a message with no recipients.
|
||||
@@ -412,10 +412,10 @@ _Released 2023-07-05_
|
||||
[import](https://zulip.com/help/import-from-slack#export-your-slack-data),
|
||||
such as a token having too few permissions.
|
||||
- Added support for IPv6
|
||||
[nameservers in the nginx configuration](../production/deployment.md#nameserver).
|
||||
[nameservers in the nginx configuration](../production/system-configuration.md#nameserver).
|
||||
- Updated translations.
|
||||
|
||||
[proxies]: ../production/deployment.md#configuring-zulip-to-trust-proxies
|
||||
[proxies]: ../production/reverse-proxies.md#configuring-zulip-to-trust-proxies
|
||||
|
||||
### Zulip Server 7.1
|
||||
|
||||
@@ -583,7 +583,7 @@ _Released 2023-05-31_
|
||||
- High volume log files like `server.log` are now by default retained
|
||||
for 14 days, configured via the `access_log_retention_days`
|
||||
[deployment
|
||||
option](../production/deployment.md#system-and-deployment-configuration). This
|
||||
option](../production/system-configuration.md). This
|
||||
replaces a harder to understand size-based algorithm that was not
|
||||
easily configurable.
|
||||
- The URL patterns for
|
||||
@@ -603,8 +603,8 @@ _Released 2023-05-31_
|
||||
- Zulip's Twitter preview integration has been disabled due to Twitter
|
||||
desupporting the API that it relied on.
|
||||
|
||||
[reverse-proxy-docs]: ../production/deployment.md#putting-the-zulip-application-behind-a-reverse-proxy
|
||||
[loadbalancer-ips]: ../production/deployment.md#configuring-zulip-to-trust-proxies
|
||||
[reverse-proxy-docs]: ../production/reverse-proxies.md
|
||||
[loadbalancer-ips]: ../production/reverse-proxies.md#configuring-zulip-to-trust-proxies
|
||||
|
||||
## Zulip Server 6.x series
|
||||
|
||||
@@ -1762,7 +1762,7 @@ _Released 2021-05-13_
|
||||
codebase with Prettier.
|
||||
- Migrated testing from CircleCI to GitHub Actions.
|
||||
|
||||
[zulip-conf-settings]: ../production/deployment.md#system-and-deployment-configuration
|
||||
[zulip-conf-settings]: ../production/system-configuration.md
|
||||
|
||||
## Zulip Server 3.x series
|
||||
|
||||
|
||||
@@ -95,7 +95,7 @@ as well as those mentioned in the
|
||||
- `--no-overwrite-settings`: This option preserves existing
|
||||
`/etc/zulip` configuration files.
|
||||
|
||||
[missing-dicts]: #missing_dictionaries
|
||||
[missing-dicts]: system-configuration.md#missing_dictionaries
|
||||
|
||||
## Installing on an existing server
|
||||
|
||||
@@ -212,6 +212,9 @@ directory if you want to customize. A good example of doing this is
|
||||
in the [zulip_ops Puppet configuration][zulipchat-puppet] that we use
|
||||
as part of managing chat.zulip.org and zulip.com.
|
||||
|
||||
[standalone.pp]: https://github.com/zulip/zulip/blob/main/puppet/zulip/manifests/profile/standalone.pp
|
||||
[zulipchat-puppet]: https://github.com/zulip/zulip/tree/main/puppet/zulip_ops/manifests
|
||||
|
||||
### Using Zulip with Amazon RDS as the database
|
||||
|
||||
You can use DBaaS services like Amazon RDS for the Zulip database.
|
||||
@@ -317,7 +320,7 @@ configure that as follows:
|
||||
We also have documentation for a Zulip server [using HTTP][using-http] for use
|
||||
behind reverse proxies.
|
||||
|
||||
[using-http]: #configuring-zulip-to-allow-http
|
||||
[using-http]: reverse-proxies.md#configuring-zulip-to-allow-http
|
||||
|
||||
## Customizing the outgoing HTTP proxy
|
||||
|
||||
@@ -365,7 +368,7 @@ In Zulip 4.7 and older, to enable SSRF protection via Smokescreen, you
|
||||
will need to explicitly add the `zulip::profile::smokescreen` Puppet
|
||||
class, and configure the `[http_proxy]` block as above.
|
||||
|
||||
[proxy.enable_for_camo]: #enable_for_camo
|
||||
[proxy.enable_for_camo]: system-configuration.md#enable_for_camo
|
||||
[smokescreen]: https://github.com/stripe/smokescreen
|
||||
[smokescreen-acls]: https://github.com/stripe/smokescreen#acls
|
||||
[ssrf]: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery
|
||||
@@ -383,284 +386,6 @@ some other proxy, you can override this default by setting
|
||||
|
||||
[s3]: upload-backends.md#s3-backend-configuration
|
||||
|
||||
## Putting the Zulip application behind a reverse proxy
|
||||
|
||||
Zulip is designed to support being run behind a reverse proxy server.
|
||||
This section contains notes on the configuration required with
|
||||
variable reverse proxy implementations.
|
||||
|
||||
### Installer options
|
||||
|
||||
If your Zulip server will not be on the public Internet, we recommend,
|
||||
installing with the `--self-signed-cert` option (rather than the
|
||||
`--certbot` option), since Certbot requires the server to be on the
|
||||
public Internet.
|
||||
|
||||
#### Configuring Zulip to allow HTTP
|
||||
|
||||
Zulip requires clients to connect to Zulip servers over the secure
|
||||
HTTPS protocol; the insecure HTTP protocol is not supported. However,
|
||||
we do support using a reverse proxy that speaks HTTPS to clients and
|
||||
connects to the Zulip server over HTTP; this can be secure when the
|
||||
Zulip server is not directly exposed to the public Internet.
|
||||
|
||||
After installing the Zulip server as [described
|
||||
above](#installer-options), you can configure Zulip to accept HTTP
|
||||
requests from a reverse proxy as follows:
|
||||
|
||||
1. Add the following block to `/etc/zulip/zulip.conf`:
|
||||
|
||||
```ini
|
||||
[application_server]
|
||||
http_only = true
|
||||
```
|
||||
|
||||
1. As root, run
|
||||
`/home/zulip/deployments/current/scripts/zulip-puppet-apply`. This
|
||||
will convert Zulip's main `nginx` configuration file to allow HTTP
|
||||
instead of HTTPS.
|
||||
|
||||
1. Finally, restart the Zulip server, using
|
||||
`/home/zulip/deployments/current/scripts/restart-server`.
|
||||
|
||||
Note that Zulip must be able to accurately determine if its connection to the
|
||||
client was over HTTPS or not; if you enable `http_only`, it is very important
|
||||
that you correctly configure Zulip to trust the `X-Forwarded-Proto` header from
|
||||
its proxy (see the next section), or clients may see infinite redirects.
|
||||
|
||||
#### Configuring Zulip to trust proxies
|
||||
|
||||
Before placing Zulip behind a reverse proxy, it needs to be configured to trust
|
||||
the client IP addresses that the proxy reports via the `X-Forwarded-For` header,
|
||||
and the protocol reported by the `X-Forwarded-Proto` header. This is important
|
||||
to have accurate IP addresses in server logs, as well as in notification emails
|
||||
which are sent to end users. Zulip doesn't default to trusting all
|
||||
`X-Forwarded-*` headers, because doing so would allow clients to spoof any IP
|
||||
address, and claim connections were over a secure connection when they were not;
|
||||
we specify which IP addresses are the Zulip server's incoming proxies, so we
|
||||
know which `X-Forwarded-*` headers to trust.
|
||||
|
||||
1. Determine the IP addresses of all reverse proxies you are setting up, as seen
|
||||
from the Zulip host. Depending on your network setup, these may not be the
|
||||
same as the public IP addresses of the reverse proxies. These can also be IP
|
||||
address ranges, as expressed in CIDR notation.
|
||||
|
||||
1. Add the following block to `/etc/zulip/zulip.conf`.
|
||||
|
||||
```ini
|
||||
[loadbalancer]
|
||||
# Use the IP addresses you determined above, separated by commas.
|
||||
ips = 192.168.0.100
|
||||
```
|
||||
|
||||
1. Reconfigure Zulip with these settings. As root, run
|
||||
`/home/zulip/deployments/current/scripts/zulip-puppet-apply`. This will
|
||||
adjust Zulip's `nginx` configuration file to accept the `X-Forwarded-For`
|
||||
header when it is sent from one of the reverse proxy IPs.
|
||||
|
||||
1. Finally, restart the Zulip server, using
|
||||
`/home/zulip/deployments/current/scripts/restart-server`.
|
||||
|
||||
### nginx configuration
|
||||
|
||||
Below is a working example of a full nginx configuration. It assumes
|
||||
that your Zulip server sits at `https://10.10.10.10:443`; see
|
||||
[above](#configuring-zulip-to-allow-http) to switch to HTTP.
|
||||
|
||||
1. Follow the instructions to [configure Zulip to trust
|
||||
proxies](#configuring-zulip-to-trust-proxies).
|
||||
|
||||
1. Configure the root `nginx.conf` file. We recommend using
|
||||
`/etc/nginx/nginx.conf` from your Zulip server for our recommended
|
||||
settings. E.g. if you don't set `client_max_body_size`, it won't be
|
||||
possible to upload large files to your Zulip server.
|
||||
|
||||
1. Configure the `nginx` site-specific configuration (in
|
||||
`/etc/nginx/sites-available`) for the Zulip app. The following
|
||||
example is a good starting point:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
listen [::]:80;
|
||||
location / {
|
||||
return 301 https://$host$request_uri;
|
||||
}
|
||||
}
|
||||
|
||||
server {
|
||||
listen 443 ssl http2;
|
||||
listen [::]:443 ssl http2;
|
||||
server_name zulip.example.com;
|
||||
|
||||
ssl_certificate /etc/letsencrypt/live/zulip.example.com/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/zulip.example.com/privkey.pem;
|
||||
|
||||
location / {
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_set_header Host $host;
|
||||
proxy_http_version 1.1;
|
||||
proxy_buffering off;
|
||||
proxy_read_timeout 20m;
|
||||
proxy_pass https://10.10.10.10:443;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Don't forget to update `server_name`, `ssl_certificate`,
|
||||
`ssl_certificate_key` and `proxy_pass` with the appropriate values
|
||||
for your deployment.
|
||||
|
||||
[nginx-proxy-longpolling-config]: https://github.com/zulip/zulip/blob/main/puppet/zulip/files/nginx/zulip-include-common/proxy_longpolling
|
||||
[standalone.pp]: https://github.com/zulip/zulip/blob/main/puppet/zulip/manifests/profile/standalone.pp
|
||||
[zulipchat-puppet]: https://github.com/zulip/zulip/tree/main/puppet/zulip_ops/manifests
|
||||
|
||||
### Apache2 configuration
|
||||
|
||||
Below is a working example of a full Apache2 configuration. It assumes
|
||||
that your Zulip server sits at `https://internal.zulip.hostname:443`.
|
||||
Note that if you wish to use SSL to connect to the Zulip server,
|
||||
Apache requires you use the hostname, not the IP address; see
|
||||
[above](#configuring-zulip-to-allow-http) to switch to HTTP.
|
||||
|
||||
1. Follow the instructions to [configure Zulip to trust
|
||||
proxies](#configuring-zulip-to-trust-proxies).
|
||||
|
||||
1. Set `USE_X_FORWARDED_HOST = True` in `/etc/zulip/settings.py` and
|
||||
restart Zulip.
|
||||
|
||||
1. Enable some required Apache modules:
|
||||
|
||||
```bash
|
||||
a2enmod ssl proxy proxy_http headers rewrite
|
||||
```
|
||||
|
||||
1. Create an Apache2 virtual host configuration file, similar to the
|
||||
following. Place it the appropriate path for your Apache2
|
||||
installation and enable it (E.g. if you use Debian or Ubuntu, then
|
||||
place it in `/etc/apache2/sites-available/zulip.example.com.conf`
|
||||
and then run
|
||||
`a2ensite zulip.example.com && systemctl reload apache2`):
|
||||
|
||||
```apache
|
||||
<VirtualHost *:80>
|
||||
ServerName zulip.example.com
|
||||
RewriteEngine On
|
||||
RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
|
||||
</VirtualHost>
|
||||
|
||||
<VirtualHost *:443>
|
||||
ServerName zulip.example.com
|
||||
|
||||
RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
|
||||
|
||||
RewriteEngine On
|
||||
RewriteRule /(.*) https://internal.zulip.hostname:443/$1 [P,L]
|
||||
|
||||
<Location />
|
||||
Require all granted
|
||||
ProxyPass https://internal.zulip.hostname:443/ timeout=1200
|
||||
</Location>
|
||||
|
||||
SSLEngine on
|
||||
SSLProxyEngine on
|
||||
SSLCertificateFile /etc/letsencrypt/live/zulip.example.com/fullchain.pem
|
||||
SSLCertificateKeyFile /etc/letsencrypt/live/zulip.example.com/privkey.pem
|
||||
# This file can be found in ~zulip/deployments/current/puppet/zulip/files/nginx/dhparam.pem
|
||||
SSLOpenSSLConfCmd DHParameters "/etc/nginx/dhparam.pem"
|
||||
SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
|
||||
SSLCipherSuite ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
|
||||
SSLHonorCipherOrder off
|
||||
SSLSessionTickets off
|
||||
Header set Strict-Transport-Security "max-age=31536000"
|
||||
</VirtualHost>
|
||||
```
|
||||
|
||||
Don't forget to update `ServerName`, `RewriteRule`, `ProxyPass`,
|
||||
`SSLCertificateFile`, and `SSLCertificateKeyFile` as are
|
||||
appropriate for your deployment.
|
||||
|
||||
### HAProxy configuration
|
||||
|
||||
Below is a working example of a HAProxy configuration. It assumes that
|
||||
your Zulip server sits at `https://10.10.10.10:443`; see
|
||||
[above](#configuring-zulip-to-allow-http) to switch to HTTP.
|
||||
|
||||
1. Follow the instructions to [configure Zulip to trust
|
||||
proxies](#configuring-zulip-to-trust-proxies).
|
||||
|
||||
1. Configure HAProxy. The below is a minimal `frontend` and `backend`
|
||||
configuration:
|
||||
|
||||
```text
|
||||
frontend zulip
|
||||
mode http
|
||||
bind *:80
|
||||
bind *:443 ssl crt /etc/ssl/private/zulip-combined.crt
|
||||
http-request redirect scheme https code 301 unless { ssl_fc }
|
||||
http-request set-header X-Forwarded-Proto http unless { ssl_fc }
|
||||
http-request set-header X-Forwarded-Proto https if { ssl_fc }
|
||||
default_backend zulip
|
||||
|
||||
backend zulip
|
||||
mode http
|
||||
timeout server 20m
|
||||
server zulip 10.10.10.10:443 check ssl ca-file /etc/ssl/certs/ca-certificates.crt
|
||||
```
|
||||
|
||||
Don't forget to update `bind *:443 ssl crt` and `server` as is
|
||||
appropriate for your deployment.
|
||||
|
||||
### Other proxies
|
||||
|
||||
If you're using another reverse proxy implementation, there are few
|
||||
things you need to be careful about when configuring it:
|
||||
|
||||
1. Configure your reverse proxy (or proxies) to correctly maintain the
|
||||
`X-Forwarded-For` HTTP header, which is supposed to contain the series
|
||||
of IP addresses the request was forwarded through. Additionally,
|
||||
[configure Zulip to respect the addresses sent by your reverse
|
||||
proxies](#configuring-zulip-to-trust-proxies). You can verify
|
||||
your work by looking at `/var/log/zulip/server.log` and checking it
|
||||
has the actual IP addresses of clients, not the IP address of the
|
||||
proxy server.
|
||||
|
||||
1. Configure your reverse proxy (or proxies) to correctly maintain the
|
||||
`X-Forwarded-Proto` HTTP header, which is supposed to contain either `https`
|
||||
or `http` depending on the connection between your browser and your
|
||||
proxy. This will be used by Django to perform CSRF checks regardless of your
|
||||
connection mechanism from your proxy to Zulip. Note that the proxies _must_
|
||||
set the header, overriding any existing values, not add a new header.
|
||||
|
||||
1. Configure your proxy to pass along the `Host:` header as was sent
|
||||
from the client, not the internal hostname as seen by the proxy.
|
||||
If this is not possible, you can set `USE_X_FORWARDED_HOST = True`
|
||||
in `/etc/zulip/settings.py`, and pass the client's `Host` header to
|
||||
Zulip in an `X-Forwarded-Host` header.
|
||||
|
||||
1. Ensure your proxy doesn't interfere with Zulip's use of
|
||||
long-polling for real-time push from the server to your users'
|
||||
browsers. This [nginx code snippet][nginx-proxy-longpolling-config]
|
||||
does this.
|
||||
|
||||
The key configuration options are, for the `/json/events` and
|
||||
`/api/1/events` endpoints:
|
||||
|
||||
- `proxy_read_timeout 1200;`. It's critical that this be
|
||||
significantly above 60s, but the precise value isn't important.
|
||||
- `proxy_buffering off`. If you don't do this, your `nginx` proxy may
|
||||
return occasional 502 errors to clients using Zulip's events API.
|
||||
|
||||
1. The other tricky failure mode we've seen with `nginx` reverse
|
||||
proxies is that they can load-balance between the IPv4 and IPv6
|
||||
addresses for a given hostname. This can result in mysterious errors
|
||||
that can be quite difficult to debug. Be sure to declare your
|
||||
`upstreams` equivalent in a way that won't do load-balancing
|
||||
unexpectedly (e.g. pointing to a DNS name that you haven't configured
|
||||
with multiple IPs for your Zulip machine; sometimes this happens with
|
||||
IPv6 configuration).
|
||||
|
||||
## PostgreSQL warm standby
|
||||
|
||||
Zulip's configuration allows for [warm standby database
|
||||
@@ -689,341 +414,3 @@ If you are using password authentication, you can set a
|
||||
|
||||
[warm-standby]: https://www.postgresql.org/docs/current/warm-standby.html
|
||||
[wal-g]: export-and-import.md#database-only-backup-tools
|
||||
|
||||
## System and deployment configuration
|
||||
|
||||
The file `/etc/zulip/zulip.conf` is used to configure properties of
|
||||
the system and deployment; `/etc/zulip/settings.py` is used to
|
||||
[configure the application itself](settings.md). The `zulip.conf`
|
||||
sections and settings are described below.
|
||||
|
||||
When a setting refers to "set to true" or "set to false", the values
|
||||
`true` and `false` are canonical, but any of the following values will
|
||||
be considered "true", case-insensitively:
|
||||
|
||||
- 1
|
||||
- y
|
||||
- t
|
||||
- yes
|
||||
- true
|
||||
- enable
|
||||
- enabled
|
||||
|
||||
Any other value (including the empty string) is considered false.
|
||||
|
||||
### `[machine]`
|
||||
|
||||
#### `puppet_classes`
|
||||
|
||||
A comma-separated list of the Puppet classes to install on the server.
|
||||
The most common is **`zulip::profile::standalone`**, used for a
|
||||
stand-alone single-host deployment.
|
||||
[Components](../overview/architecture-overview.md#components) of
|
||||
that include:
|
||||
|
||||
- **`zulip::profile::app_frontend`**
|
||||
- **`zulip::profile::memcached`**
|
||||
- **`zulip::profile::postgresql`**
|
||||
- **`zulip::profile::redis`**
|
||||
- **`zulip::profile::rabbitmq`**
|
||||
|
||||
If you are using a [Apache as a single-sign-on
|
||||
authenticator](authentication-methods.md#apache-based-sso-with-remote_user),
|
||||
you will need to add **`zulip::apache_sso`** to the list.
|
||||
|
||||
#### `pgroonga`
|
||||
|
||||
Set to true if enabling the [multi-language PGroonga search
|
||||
extension](../subsystems/full-text-search.md#multi-language-full-text-search).
|
||||
|
||||
#### `timesync`
|
||||
|
||||
What time synchronization daemon to use; defaults to `chrony`, but also supports
|
||||
`ntpd` and `none`. Installations should not adjust this unless they are aligning
|
||||
with a fleet-wide standard of `ntpd`. `none` is only reasonable in containers
|
||||
like LXC which do not allow adjustment of the clock; a Zulip server will not
|
||||
function correctly without an accurate clock.
|
||||
|
||||
### `[deployment]`
|
||||
|
||||
#### `deploy_options`
|
||||
|
||||
Options passed by `upgrade-zulip` and `upgrade-zulip-from-git` into
|
||||
`upgrade-zulip-stage-2`. These might be any of:
|
||||
|
||||
- **`--skip-puppet`** skips doing Puppet/apt upgrades. The user will need
|
||||
to run `zulip-puppet-apply` manually after the upgrade.
|
||||
- **`--skip-migrations`** skips running database migrations. The
|
||||
user will need to run `./manage.py migrate` manually after the upgrade.
|
||||
- **`--skip-purge-old-deployments`** skips purging old deployments;
|
||||
without it, only deployments with the last two weeks are kept.
|
||||
|
||||
Generally installations will not want to set any of these options; the
|
||||
`--skip-*` options are primarily useful for reducing upgrade downtime
|
||||
for servers that are upgraded frequently by core Zulip developers.
|
||||
|
||||
#### `git_repo_url`
|
||||
|
||||
Default repository URL used when [upgrading from a Git
|
||||
repository](upgrade.md#upgrading-from-a-git-repository).
|
||||
|
||||
### `[application_server]`
|
||||
|
||||
#### `http_only`
|
||||
|
||||
If set to true, [configures Zulip to allow HTTP access][using-http];
|
||||
use if Zulip is deployed behind a reverse proxy that is handling
|
||||
SSL/TLS termination.
|
||||
|
||||
#### `nginx_listen_port`
|
||||
|
||||
Set to the port number if you [prefer to listen on a port other than
|
||||
443](#using-an-alternate-port).
|
||||
|
||||
#### `nginx_worker_connections`
|
||||
|
||||
Adjust the [`worker_connections`][nginx_worker_connections] setting in
|
||||
the nginx server. This defaults to 10000; increasing it allows more
|
||||
concurrent connections per CPU core, at the cost of more memory
|
||||
consumed by NGINX. This number, times the number of CPU cores, should
|
||||
be more than twice the concurrent number of users.
|
||||
|
||||
[nginx_worker_connections]: http://nginx.org/en/docs/ngx_core_module.html#worker_connections
|
||||
|
||||
#### `queue_workers_multiprocess`
|
||||
|
||||
By default, Zulip automatically detects whether the system has enough
|
||||
memory to run Zulip queue processors in the higher-throughput but more
|
||||
multiprocess mode (or to save 1.5GiB of RAM with the multithreaded
|
||||
mode). The calculation is based on whether the system has enough
|
||||
memory (currently 3.5GiB) to run a single-server Zulip installation in
|
||||
the multiprocess mode.
|
||||
|
||||
Set explicitly to true or false to override the automatic
|
||||
calculation. This override is useful both Docker systems (where the
|
||||
above algorithm might see the host's memory, not the container's)
|
||||
and/or when using remote servers for postgres, memcached, redis, and
|
||||
RabbitMQ.
|
||||
|
||||
#### `rolling_restart`
|
||||
|
||||
If set to true, when using `./scripts/restart-server` to restart
|
||||
Zulip, restart the uwsgi processes one-at-a-time, instead of all at
|
||||
once. This decreases the number of 502's served to clients, at the
|
||||
cost of slightly increased memory usage, and the possibility that
|
||||
different requests will be served by different versions of the code.
|
||||
|
||||
#### `service_file_descriptor_limit`
|
||||
|
||||
The number of file descriptors which [Supervisor is configured to allow
|
||||
processes to use][supervisor-minfds]; defaults to 40000. If your Zulip deployment
|
||||
is very large (hundreds of thousands of concurrent users), your Django processes
|
||||
hit this limit and refuse connections to clients. Raising it above this default
|
||||
may require changing system-level limits, particularly if you are using a
|
||||
virtualized environment (e.g. Docker, or Proxmox LXC).
|
||||
|
||||
[supervisor-minfds]: http://supervisord.org/configuration.html?highlight=minfds#supervisord-section-values
|
||||
|
||||
#### `s3_memory_cache_size`
|
||||
|
||||
Used only when the [S3 storage backend][s3-backend] is in use.
|
||||
Controls the in-memory size of the cache _index_; the default is 1MB,
|
||||
which is enough to store about 8 thousand entries.
|
||||
|
||||
#### `s3_disk_cache_size`
|
||||
|
||||
Used only when the [S3 storage backend][s3-backend] is in use.
|
||||
Controls the on-disk size of the cache _contents_; the default is
|
||||
200MB.
|
||||
|
||||
#### `s3_cache_inactive_time`
|
||||
|
||||
Used only when the [S3 storage backend][s3-backend] is in use.
|
||||
Controls the longest amount of time an entry will be cached since last
|
||||
use; the default is 30 days. Since the contents of the cache are
|
||||
immutable, this serves only as a potential additional limit on the
|
||||
size of the contents on disk; `s3_disk_cache_size` is expected to be
|
||||
the primary control for cache sizing.
|
||||
|
||||
#### `nameserver`
|
||||
|
||||
When the [S3 storage backend][s3-backend] is in use, downloads from S3 are
|
||||
proxied from nginx, whose configuration requires an explicit value of a DNS
|
||||
nameserver to resolve the S3 server's hostname. Zulip defaults to using the
|
||||
resolver found in `/etc/resolv.conf`; this setting overrides any value found
|
||||
there.
|
||||
|
||||
[s3-backend]: upload-backends.md
|
||||
|
||||
#### `uwsgi_listen_backlog_limit`
|
||||
|
||||
Override the default uwsgi backlog of 128 connections.
|
||||
|
||||
#### `uwsgi_processes`
|
||||
|
||||
Override the default `uwsgi` (Django) process count of 6 on hosts with
|
||||
more than 3.5GiB of RAM, 4 on hosts with less.
|
||||
|
||||
#### `access_log_retention_days`
|
||||
|
||||
Number of days of access logs to keep, for both nginx and the application.
|
||||
Defaults to 14 days.
|
||||
|
||||
### `[postfix]`
|
||||
|
||||
#### `mailname`
|
||||
|
||||
The hostname that [Postfix should be configured to receive mail
|
||||
at](email-gateway.md#local-delivery-setup), as well as identify itself as for
|
||||
outgoing email.
|
||||
|
||||
### `[postgresql]`
|
||||
|
||||
#### `effective_io_concurrency`
|
||||
|
||||
Override PostgreSQL's [`effective_io_concurrency`
|
||||
setting](https://www.postgresql.org/docs/current/runtime-config-resource.html#GUC-EFFECTIVE-IO-CONCURRENCY).
|
||||
|
||||
#### `listen_addresses`
|
||||
|
||||
Override PostgreSQL's [`listen_addresses`
|
||||
setting](https://www.postgresql.org/docs/current/runtime-config-connection.html#GUC-LISTEN-ADDRESSES).
|
||||
|
||||
#### `random_page_cost`
|
||||
|
||||
Override PostgreSQL's [`random_page_cost`
|
||||
setting](https://www.postgresql.org/docs/current/runtime-config-query.html#GUC-RANDOM-PAGE-COST)
|
||||
|
||||
#### `replication_primary`
|
||||
|
||||
On the [warm standby replicas](#postgresql-warm-standby), set to the
|
||||
hostname of the primary PostgreSQL server that streaming replication
|
||||
should be done from.
|
||||
|
||||
#### `replication_user`
|
||||
|
||||
On the [warm standby replicas](#postgresql-warm-standby), set to the
|
||||
username that the host should authenticate to the primary PostgreSQL
|
||||
server as, for streaming replication. Authentication will be done
|
||||
based on the `pg_hba.conf` file; if you are using password
|
||||
authentication, you can set a `postgresql_replication_password` secret
|
||||
for authentication.
|
||||
|
||||
#### `skip_backups`
|
||||
|
||||
If set to as true value, inhibits the nightly [`wal-g` backups][wal-g] which
|
||||
would be taken on all non-replicated hosts and [all warm standby
|
||||
replicas](#postgresql-warm-standby). This is generally only set if you have
|
||||
multiple warm standby replicas, in order to avoid taking multiple backups, one
|
||||
per replica.
|
||||
|
||||
#### `backups_disk_concurrency`
|
||||
|
||||
Number of concurrent disk reads to use when taking backups. Defaults to 1; you
|
||||
may wish to increase this if you are taking backups on a replica, so can afford
|
||||
to affect other disk I/O, and have an SSD which is good at parallel random
|
||||
reads.
|
||||
|
||||
#### `backups_storage_class`
|
||||
|
||||
What [storage class](https://aws.amazon.com/s3/storage-classes/) to use when
|
||||
uploading database backups. Defaults to `STANDARD`, meaning "[S3
|
||||
standard][s3-standard]", but many deployments will have overall lower costs if
|
||||
"[S3 Standard - Infrequent Access][s3-ia]" is used, via the `STANDARD_IA`
|
||||
value. Also supported is "[S3 Reduced Redundancy][s3-rr]", by setting
|
||||
`REDUCED_REDUNDANCY`, but this is not suggested for production use.
|
||||
|
||||
[s3-standard]: https://aws.amazon.com/s3/storage-classes/#General_purpose
|
||||
[s3-ia]: https://aws.amazon.com/s3/storage-classes/#Infrequent_access
|
||||
[s3-rr]: https://aws.amazon.com/s3/reduced-redundancy/
|
||||
|
||||
#### `missing_dictionaries`
|
||||
|
||||
If set to a true value during initial database creation, uses PostgreSQL's
|
||||
standard `pg_catalog.english` text search configuration, rather than Zulip's
|
||||
improved set of stopwords. Has no effect after initial database construction.
|
||||
|
||||
#### `ssl_ca_file`
|
||||
|
||||
Set to the path to the PEM-encoded certificate authority used to
|
||||
authenticate client connections.
|
||||
|
||||
#### `ssl_cert_file`
|
||||
|
||||
Set to the path to the PEM-encoded public certificate used to secure
|
||||
client connections.
|
||||
|
||||
#### `ssl_key_file`
|
||||
|
||||
Set to the path to the PEM-encoded private key used to secure client
|
||||
connections.
|
||||
|
||||
#### `ssl_mode`
|
||||
|
||||
The mode that should be used to verify the server certificate. The
|
||||
PostgreSQL default is `prefer`, which provides no security benefit; we
|
||||
strongly suggest setting this to `require` or better if you are using
|
||||
certificate authentication. See the [PostgreSQL
|
||||
documentation](https://www.postgresql.org/docs/current/libpq-ssl.html#LIBPQ-SSL-SSLMODE-STATEMENTS)
|
||||
for potential values.
|
||||
|
||||
#### `version`
|
||||
|
||||
The version of PostgreSQL that is in use. Do not set by hand; use the
|
||||
[PostgreSQL upgrade tool](upgrade.md#upgrading-postgresql).
|
||||
|
||||
### `[memcached]`
|
||||
|
||||
#### `memory`
|
||||
|
||||
Override the number of megabytes of memory that memcached should be
|
||||
configured to consume; defaults to 1/8th of the total server memory.
|
||||
|
||||
#### `max_item_size`
|
||||
|
||||
Override the maximum size that an item in memcached can store. This defaults to
|
||||
1m; adjusting it should only be necessary if your Zulip server has organizations
|
||||
which have more than 20k users.
|
||||
|
||||
### `[loadbalancer]`
|
||||
|
||||
#### `ips`
|
||||
|
||||
Comma-separated list of IP addresses or netmasks of external load balancers
|
||||
whose `X-Forwarded-For` and `X-Forwarded-Proto` should be respected. These can
|
||||
be individual IP addresses, or CIDR IP address ranges.
|
||||
|
||||
### `[http_proxy]`
|
||||
|
||||
#### `host`
|
||||
|
||||
The hostname or IP address of an [outgoing HTTP `CONNECT`
|
||||
proxy](#customizing-the-outgoing-http-proxy). Defaults to `localhost`
|
||||
if unspecified.
|
||||
|
||||
#### `port`
|
||||
|
||||
The TCP port of the HTTP `CONNECT` proxy on the host specified above.
|
||||
Defaults to `4750` if unspecified.
|
||||
|
||||
#### `listen_address`
|
||||
|
||||
The IP address that Smokescreen should bind to and listen on.
|
||||
Defaults to `127.0.0.1`.
|
||||
|
||||
#### `enable_for_camo`
|
||||
|
||||
Because Camo includes logic to deny access to private subnets, routing
|
||||
its requests through Smokescreen is generally not necessary. Set to
|
||||
true or false to override the default, which uses the proxy only if
|
||||
it is not the default of Smokescreen on a local host.
|
||||
|
||||
### `[sentry]`
|
||||
|
||||
#### `organization`
|
||||
|
||||
The Sentry organization used for the [Sentry deploy hook](#sentry-deploy-hook).
|
||||
|
||||
#### `project`
|
||||
|
||||
The Sentry project used for the [Sentry deploy hook](#sentry-deploy-hook).
|
||||
|
||||
@@ -101,7 +101,7 @@ using an [HTTP reverse proxy][reverse-proxy]).
|
||||
|
||||
Congratulations! The integration should be fully operational.
|
||||
|
||||
[reverse-proxy]: deployment.md#putting-the-zulip-application-behind-a-reverse-proxy
|
||||
[reverse-proxy]: reverse-proxies.md
|
||||
|
||||
## Polling setup
|
||||
|
||||
|
||||
@@ -10,6 +10,7 @@ install
|
||||
troubleshooting
|
||||
management-commands
|
||||
settings
|
||||
system-configuration
|
||||
mobile-push-notifications
|
||||
upgrade
|
||||
modify
|
||||
@@ -21,6 +22,7 @@ upload-backends
|
||||
ssl-certificates
|
||||
email
|
||||
deployment
|
||||
reverse-proxies
|
||||
multiple-organizations
|
||||
email-gateway
|
||||
video-calls
|
||||
|
||||
@@ -119,7 +119,7 @@ access to incoming port 22 for SSH access for remote access.
|
||||
|
||||
[ssrf]: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery
|
||||
[smokescreen-proxy]: deployment.md#customizing-the-outgoing-http-proxy
|
||||
[reverse-proxy]: deployment.md#putting-the-zulip-application-behind-a-reverse-proxy
|
||||
[reverse-proxy]: reverse-proxies.md
|
||||
|
||||
## Credentials needed
|
||||
|
||||
|
||||
275
docs/production/reverse-proxies.md
Normal file
275
docs/production/reverse-proxies.md
Normal file
@@ -0,0 +1,275 @@
|
||||
## Reverse proxies
|
||||
|
||||
Zulip is designed to support being run behind a reverse proxy server.
|
||||
This section contains notes on the configuration required with
|
||||
variable reverse proxy implementations.
|
||||
|
||||
### Installer options
|
||||
|
||||
If your Zulip server will not be on the public Internet, we recommend,
|
||||
installing with the `--self-signed-cert` option (rather than the
|
||||
`--certbot` option), since Certbot requires the server to be on the
|
||||
public Internet.
|
||||
|
||||
#### Configuring Zulip to allow HTTP
|
||||
|
||||
Zulip requires clients to connect to Zulip servers over the secure
|
||||
HTTPS protocol; the insecure HTTP protocol is not supported. However,
|
||||
we do support using a reverse proxy that speaks HTTPS to clients and
|
||||
connects to the Zulip server over HTTP; this can be secure when the
|
||||
Zulip server is not directly exposed to the public Internet.
|
||||
|
||||
After installing the Zulip server as [described
|
||||
above](#installer-options), you can configure Zulip to accept HTTP
|
||||
requests from a reverse proxy as follows:
|
||||
|
||||
1. Add the following block to `/etc/zulip/zulip.conf`:
|
||||
|
||||
```ini
|
||||
[application_server]
|
||||
http_only = true
|
||||
```
|
||||
|
||||
1. As root, run
|
||||
`/home/zulip/deployments/current/scripts/zulip-puppet-apply`. This
|
||||
will convert Zulip's main `nginx` configuration file to allow HTTP
|
||||
instead of HTTPS.
|
||||
|
||||
1. Finally, restart the Zulip server, using
|
||||
`/home/zulip/deployments/current/scripts/restart-server`.
|
||||
|
||||
Note that Zulip must be able to accurately determine if its connection to the
|
||||
client was over HTTPS or not; if you enable `http_only`, it is very important
|
||||
that you correctly configure Zulip to trust the `X-Forwarded-Proto` header from
|
||||
its proxy (see the next section), or clients may see infinite redirects.
|
||||
|
||||
#### Configuring Zulip to trust proxies
|
||||
|
||||
Before placing Zulip behind a reverse proxy, it needs to be configured to trust
|
||||
the client IP addresses that the proxy reports via the `X-Forwarded-For` header,
|
||||
and the protocol reported by the `X-Forwarded-Proto` header. This is important
|
||||
to have accurate IP addresses in server logs, as well as in notification emails
|
||||
which are sent to end users. Zulip doesn't default to trusting all
|
||||
`X-Forwarded-*` headers, because doing so would allow clients to spoof any IP
|
||||
address, and claim connections were over a secure connection when they were not;
|
||||
we specify which IP addresses are the Zulip server's incoming proxies, so we
|
||||
know which `X-Forwarded-*` headers to trust.
|
||||
|
||||
1. Determine the IP addresses of all reverse proxies you are setting up, as seen
|
||||
from the Zulip host. Depending on your network setup, these may not be the
|
||||
same as the public IP addresses of the reverse proxies. These can also be IP
|
||||
address ranges, as expressed in CIDR notation.
|
||||
|
||||
1. Add the following block to `/etc/zulip/zulip.conf`.
|
||||
|
||||
```ini
|
||||
[loadbalancer]
|
||||
# Use the IP addresses you determined above, separated by commas.
|
||||
ips = 192.168.0.100
|
||||
```
|
||||
|
||||
1. Reconfigure Zulip with these settings. As root, run
|
||||
`/home/zulip/deployments/current/scripts/zulip-puppet-apply`. This will
|
||||
adjust Zulip's `nginx` configuration file to accept the `X-Forwarded-For`
|
||||
header when it is sent from one of the reverse proxy IPs.
|
||||
|
||||
1. Finally, restart the Zulip server, using
|
||||
`/home/zulip/deployments/current/scripts/restart-server`.
|
||||
|
||||
### nginx configuration
|
||||
|
||||
Below is a working example of a full nginx configuration. It assumes
|
||||
that your Zulip server sits at `https://10.10.10.10:443`; see
|
||||
[above](#configuring-zulip-to-allow-http) to switch to HTTP.
|
||||
|
||||
1. Follow the instructions to [configure Zulip to trust
|
||||
proxies](#configuring-zulip-to-trust-proxies).
|
||||
|
||||
1. Configure the root `nginx.conf` file. We recommend using
|
||||
`/etc/nginx/nginx.conf` from your Zulip server for our recommended
|
||||
settings. E.g. if you don't set `client_max_body_size`, it won't be
|
||||
possible to upload large files to your Zulip server.
|
||||
|
||||
1. Configure the `nginx` site-specific configuration (in
|
||||
`/etc/nginx/sites-available`) for the Zulip app. The following
|
||||
example is a good starting point:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
listen [::]:80;
|
||||
location / {
|
||||
return 301 https://$host$request_uri;
|
||||
}
|
||||
}
|
||||
|
||||
server {
|
||||
listen 443 ssl http2;
|
||||
listen [::]:443 ssl http2;
|
||||
server_name zulip.example.com;
|
||||
|
||||
ssl_certificate /etc/letsencrypt/live/zulip.example.com/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/zulip.example.com/privkey.pem;
|
||||
|
||||
location / {
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_set_header Host $host;
|
||||
proxy_http_version 1.1;
|
||||
proxy_buffering off;
|
||||
proxy_read_timeout 20m;
|
||||
proxy_pass https://10.10.10.10:443;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Don't forget to update `server_name`, `ssl_certificate`,
|
||||
`ssl_certificate_key` and `proxy_pass` with the appropriate values
|
||||
for your deployment.
|
||||
|
||||
### Apache2 configuration
|
||||
|
||||
Below is a working example of a full Apache2 configuration. It assumes
|
||||
that your Zulip server sits at `https://internal.zulip.hostname:443`.
|
||||
Note that if you wish to use SSL to connect to the Zulip server,
|
||||
Apache requires you use the hostname, not the IP address; see
|
||||
[above](#configuring-zulip-to-allow-http) to switch to HTTP.
|
||||
|
||||
1. Follow the instructions to [configure Zulip to trust
|
||||
proxies](#configuring-zulip-to-trust-proxies).
|
||||
|
||||
1. Set `USE_X_FORWARDED_HOST = True` in `/etc/zulip/settings.py` and
|
||||
restart Zulip.
|
||||
|
||||
1. Enable some required Apache modules:
|
||||
|
||||
```bash
|
||||
a2enmod ssl proxy proxy_http headers rewrite
|
||||
```
|
||||
|
||||
1. Create an Apache2 virtual host configuration file, similar to the
|
||||
following. Place it the appropriate path for your Apache2
|
||||
installation and enable it (E.g. if you use Debian or Ubuntu, then
|
||||
place it in `/etc/apache2/sites-available/zulip.example.com.conf`
|
||||
and then run
|
||||
`a2ensite zulip.example.com && systemctl reload apache2`):
|
||||
|
||||
```apache
|
||||
<VirtualHost *:80>
|
||||
ServerName zulip.example.com
|
||||
RewriteEngine On
|
||||
RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
|
||||
</VirtualHost>
|
||||
|
||||
<VirtualHost *:443>
|
||||
ServerName zulip.example.com
|
||||
|
||||
RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
|
||||
|
||||
RewriteEngine On
|
||||
RewriteRule /(.*) https://internal.zulip.hostname:443/$1 [P,L]
|
||||
|
||||
<Location />
|
||||
Require all granted
|
||||
ProxyPass https://internal.zulip.hostname:443/ timeout=1200
|
||||
</Location>
|
||||
|
||||
SSLEngine on
|
||||
SSLProxyEngine on
|
||||
SSLCertificateFile /etc/letsencrypt/live/zulip.example.com/fullchain.pem
|
||||
SSLCertificateKeyFile /etc/letsencrypt/live/zulip.example.com/privkey.pem
|
||||
# This file can be found in ~zulip/deployments/current/puppet/zulip/files/nginx/dhparam.pem
|
||||
SSLOpenSSLConfCmd DHParameters "/etc/nginx/dhparam.pem"
|
||||
SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
|
||||
SSLCipherSuite ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
|
||||
SSLHonorCipherOrder off
|
||||
SSLSessionTickets off
|
||||
Header set Strict-Transport-Security "max-age=31536000"
|
||||
</VirtualHost>
|
||||
```
|
||||
|
||||
Don't forget to update `ServerName`, `RewriteRule`, `ProxyPass`,
|
||||
`SSLCertificateFile`, and `SSLCertificateKeyFile` as are
|
||||
appropriate for your deployment.
|
||||
|
||||
### HAProxy configuration
|
||||
|
||||
Below is a working example of a HAProxy configuration. It assumes that
|
||||
your Zulip server sits at `https://10.10.10.10:443`; see
|
||||
[above](#configuring-zulip-to-allow-http) to switch to HTTP.
|
||||
|
||||
1. Follow the instructions to [configure Zulip to trust
|
||||
proxies](#configuring-zulip-to-trust-proxies).
|
||||
|
||||
1. Configure HAProxy. The below is a minimal `frontend` and `backend`
|
||||
configuration:
|
||||
|
||||
```text
|
||||
frontend zulip
|
||||
mode http
|
||||
bind *:80
|
||||
bind *:443 ssl crt /etc/ssl/private/zulip-combined.crt
|
||||
http-request redirect scheme https code 301 unless { ssl_fc }
|
||||
http-request set-header X-Forwarded-Proto http unless { ssl_fc }
|
||||
http-request set-header X-Forwarded-Proto https if { ssl_fc }
|
||||
default_backend zulip
|
||||
|
||||
backend zulip
|
||||
mode http
|
||||
timeout server 20m
|
||||
server zulip 10.10.10.10:443 check ssl ca-file /etc/ssl/certs/ca-certificates.crt
|
||||
```
|
||||
|
||||
Don't forget to update `bind *:443 ssl crt` and `server` as is
|
||||
appropriate for your deployment.
|
||||
|
||||
### Other proxies
|
||||
|
||||
If you're using another reverse proxy implementation, there are few
|
||||
things you need to be careful about when configuring it:
|
||||
|
||||
1. Configure your reverse proxy (or proxies) to correctly maintain the
|
||||
`X-Forwarded-For` HTTP header, which is supposed to contain the series
|
||||
of IP addresses the request was forwarded through. Additionally,
|
||||
[configure Zulip to respect the addresses sent by your reverse
|
||||
proxies](#configuring-zulip-to-trust-proxies). You can verify
|
||||
your work by looking at `/var/log/zulip/server.log` and checking it
|
||||
has the actual IP addresses of clients, not the IP address of the
|
||||
proxy server.
|
||||
|
||||
1. Configure your reverse proxy (or proxies) to correctly maintain the
|
||||
`X-Forwarded-Proto` HTTP header, which is supposed to contain either `https`
|
||||
or `http` depending on the connection between your browser and your
|
||||
proxy. This will be used by Django to perform CSRF checks regardless of your
|
||||
connection mechanism from your proxy to Zulip. Note that the proxies _must_
|
||||
set the header, overriding any existing values, not add a new header.
|
||||
|
||||
1. Configure your proxy to pass along the `Host:` header as was sent
|
||||
from the client, not the internal hostname as seen by the proxy.
|
||||
If this is not possible, you can set `USE_X_FORWARDED_HOST = True`
|
||||
in `/etc/zulip/settings.py`, and pass the client's `Host` header to
|
||||
Zulip in an `X-Forwarded-Host` header.
|
||||
|
||||
1. Ensure your proxy doesn't interfere with Zulip's use of
|
||||
long-polling for real-time push from the server to your users'
|
||||
browsers. This [nginx code snippet][nginx-proxy-longpolling-config]
|
||||
does this.
|
||||
|
||||
The key configuration options are, for the `/json/events` and
|
||||
`/api/1/events` endpoints:
|
||||
|
||||
- `proxy_read_timeout 1200;`. It's critical that this be
|
||||
significantly above 60s, but the precise value isn't important.
|
||||
- `proxy_buffering off`. If you don't do this, your `nginx` proxy may
|
||||
return occasional 502 errors to clients using Zulip's events API.
|
||||
|
||||
1. The other tricky failure mode we've seen with `nginx` reverse
|
||||
proxies is that they can load-balance between the IPv4 and IPv6
|
||||
addresses for a given hostname. This can result in mysterious errors
|
||||
that can be quite difficult to debug. Be sure to declare your
|
||||
`upstreams` equivalent in a way that won't do load-balancing
|
||||
unexpectedly (e.g. pointing to a DNS name that you haven't configured
|
||||
with multiple IPs for your Zulip machine; sometimes this happens with
|
||||
IPv6 configuration).
|
||||
|
||||
[nginx-proxy-longpolling-config]: https://github.com/zulip/zulip/blob/main/puppet/zulip/files/nginx/zulip-include-common/proxy_longpolling
|
||||
@@ -260,7 +260,7 @@ strength allowed is controlled by two settings in
|
||||
[go-camo]: https://github.com/cactus/go-camo
|
||||
[ssrf]: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery
|
||||
[smokescreen-setup]: deployment.md#customizing-the-outgoing-http-proxy
|
||||
[proxy.enable_for_camo]: deployment.md#enable_for_camo
|
||||
[proxy.enable_for_camo]: system-configuration.md#enable_for_camo
|
||||
|
||||
## Rate limiting
|
||||
|
||||
|
||||
@@ -33,7 +33,7 @@ to each new major release.
|
||||
|
||||
Since Zulip’s settings file is a Python script, there are a number of
|
||||
other things that one can configure that are documented in
|
||||
[System and deployment configuration](deployment.md#system-and-deployment-configuration).
|
||||
[System and deployment configuration](system-configuration.md).
|
||||
Otherwise, ask in [the Zulip development community](https://zulip.com/development-community/)
|
||||
if there’s something you’d like to do but can’t figure out how to.
|
||||
|
||||
|
||||
337
docs/production/system-configuration.md
Normal file
337
docs/production/system-configuration.md
Normal file
@@ -0,0 +1,337 @@
|
||||
## System configuration
|
||||
|
||||
The file `/etc/zulip/zulip.conf` is used to configure properties of
|
||||
the system and deployment; `/etc/zulip/settings.py` is used to
|
||||
[configure the application itself](settings.md). The `zulip.conf`
|
||||
sections and settings are described below.
|
||||
|
||||
When a setting refers to "set to true" or "set to false", the values
|
||||
`true` and `false` are canonical, but any of the following values will
|
||||
be considered "true", case-insensitively:
|
||||
|
||||
- 1
|
||||
- y
|
||||
- t
|
||||
- yes
|
||||
- true
|
||||
- enable
|
||||
- enabled
|
||||
|
||||
Any other value (including the empty string) is considered false.
|
||||
|
||||
### `[machine]`
|
||||
|
||||
#### `puppet_classes`
|
||||
|
||||
A comma-separated list of the Puppet classes to install on the server.
|
||||
The most common is **`zulip::profile::standalone`**, used for a
|
||||
stand-alone single-host deployment.
|
||||
[Components](../overview/architecture-overview.md#components) of
|
||||
that include:
|
||||
|
||||
- **`zulip::profile::app_frontend`**
|
||||
- **`zulip::profile::memcached`**
|
||||
- **`zulip::profile::postgresql`**
|
||||
- **`zulip::profile::redis`**
|
||||
- **`zulip::profile::rabbitmq`**
|
||||
|
||||
If you are using a [Apache as a single-sign-on
|
||||
authenticator](authentication-methods.md#apache-based-sso-with-remote_user),
|
||||
you will need to add **`zulip::apache_sso`** to the list.
|
||||
|
||||
#### `pgroonga`
|
||||
|
||||
Set to true if enabling the [multi-language PGroonga search
|
||||
extension](../subsystems/full-text-search.md#multi-language-full-text-search).
|
||||
|
||||
#### `timesync`
|
||||
|
||||
What time synchronization daemon to use; defaults to `chrony`, but also supports
|
||||
`ntpd` and `none`. Installations should not adjust this unless they are aligning
|
||||
with a fleet-wide standard of `ntpd`. `none` is only reasonable in containers
|
||||
like LXC which do not allow adjustment of the clock; a Zulip server will not
|
||||
function correctly without an accurate clock.
|
||||
|
||||
### `[deployment]`
|
||||
|
||||
#### `deploy_options`
|
||||
|
||||
Options passed by `upgrade-zulip` and `upgrade-zulip-from-git` into
|
||||
`upgrade-zulip-stage-2`. These might be any of:
|
||||
|
||||
- **`--skip-puppet`** skips doing Puppet/apt upgrades. The user will need
|
||||
to run `zulip-puppet-apply` manually after the upgrade.
|
||||
- **`--skip-migrations`** skips running database migrations. The
|
||||
user will need to run `./manage.py migrate` manually after the upgrade.
|
||||
- **`--skip-purge-old-deployments`** skips purging old deployments;
|
||||
without it, only deployments with the last two weeks are kept.
|
||||
|
||||
Generally installations will not want to set any of these options; the
|
||||
`--skip-*` options are primarily useful for reducing upgrade downtime
|
||||
for servers that are upgraded frequently by core Zulip developers.
|
||||
|
||||
#### `git_repo_url`
|
||||
|
||||
Default repository URL used when [upgrading from a Git
|
||||
repository](upgrade.md#upgrading-from-a-git-repository).
|
||||
|
||||
### `[application_server]`
|
||||
|
||||
#### `http_only`
|
||||
|
||||
If set to true, [configures Zulip to allow HTTP access][using-http];
|
||||
use if Zulip is deployed behind a reverse proxy that is handling
|
||||
SSL/TLS termination.
|
||||
|
||||
#### `nginx_listen_port`
|
||||
|
||||
Set to the port number if you [prefer to listen on a port other than
|
||||
443](deployment.md#using-an-alternate-port).
|
||||
|
||||
#### `nginx_worker_connections`
|
||||
|
||||
Adjust the [`worker_connections`][nginx_worker_connections] setting in
|
||||
the nginx server. This defaults to 10000; increasing it allows more
|
||||
concurrent connections per CPU core, at the cost of more memory
|
||||
consumed by NGINX. This number, times the number of CPU cores, should
|
||||
be more than twice the concurrent number of users.
|
||||
|
||||
[nginx_worker_connections]: http://nginx.org/en/docs/ngx_core_module.html#worker_connections
|
||||
|
||||
#### `queue_workers_multiprocess`
|
||||
|
||||
By default, Zulip automatically detects whether the system has enough
|
||||
memory to run Zulip queue processors in the higher-throughput but more
|
||||
multiprocess mode (or to save 1.5GiB of RAM with the multithreaded
|
||||
mode). The calculation is based on whether the system has enough
|
||||
memory (currently 3.5GiB) to run a single-server Zulip installation in
|
||||
the multiprocess mode.
|
||||
|
||||
Set explicitly to true or false to override the automatic
|
||||
calculation. This override is useful both Docker systems (where the
|
||||
above algorithm might see the host's memory, not the container's)
|
||||
and/or when using remote servers for postgres, memcached, redis, and
|
||||
RabbitMQ.
|
||||
|
||||
#### `rolling_restart`
|
||||
|
||||
If set to true, when using `./scripts/restart-server` to restart
|
||||
Zulip, restart the uwsgi processes one-at-a-time, instead of all at
|
||||
once. This decreases the number of 502's served to clients, at the
|
||||
cost of slightly increased memory usage, and the possibility that
|
||||
different requests will be served by different versions of the code.
|
||||
|
||||
#### `service_file_descriptor_limit`
|
||||
|
||||
The number of file descriptors which [Supervisor is configured to allow
|
||||
processes to use][supervisor-minfds]; defaults to 40000. If your Zulip deployment
|
||||
is very large (hundreds of thousands of concurrent users), your Django processes
|
||||
hit this limit and refuse connections to clients. Raising it above this default
|
||||
may require changing system-level limits, particularly if you are using a
|
||||
virtualized environment (e.g. Docker, or Proxmox LXC).
|
||||
|
||||
[supervisor-minfds]: http://supervisord.org/configuration.html?highlight=minfds#supervisord-section-values
|
||||
|
||||
#### `s3_memory_cache_size`
|
||||
|
||||
Used only when the [S3 storage backend][s3-backend] is in use.
|
||||
Controls the in-memory size of the cache _index_; the default is 1MB,
|
||||
which is enough to store about 8 thousand entries.
|
||||
|
||||
#### `s3_disk_cache_size`
|
||||
|
||||
Used only when the [S3 storage backend][s3-backend] is in use.
|
||||
Controls the on-disk size of the cache _contents_; the default is
|
||||
200MB.
|
||||
|
||||
#### `s3_cache_inactive_time`
|
||||
|
||||
Used only when the [S3 storage backend][s3-backend] is in use.
|
||||
Controls the longest amount of time an entry will be cached since last
|
||||
use; the default is 30 days. Since the contents of the cache are
|
||||
immutable, this serves only as a potential additional limit on the
|
||||
size of the contents on disk; `s3_disk_cache_size` is expected to be
|
||||
the primary control for cache sizing.
|
||||
|
||||
#### `nameserver`
|
||||
|
||||
When the [S3 storage backend][s3-backend] is in use, downloads from S3 are
|
||||
proxied from nginx, whose configuration requires an explicit value of a DNS
|
||||
nameserver to resolve the S3 server's hostname. Zulip defaults to using the
|
||||
resolver found in `/etc/resolv.conf`; this setting overrides any value found
|
||||
there.
|
||||
|
||||
[s3-backend]: upload-backends.md
|
||||
|
||||
#### `uwsgi_listen_backlog_limit`
|
||||
|
||||
Override the default uwsgi backlog of 128 connections.
|
||||
|
||||
#### `uwsgi_processes`
|
||||
|
||||
Override the default `uwsgi` (Django) process count of 6 on hosts with
|
||||
more than 3.5GiB of RAM, 4 on hosts with less.
|
||||
|
||||
#### `access_log_retention_days`
|
||||
|
||||
Number of days of access logs to keep, for both nginx and the application.
|
||||
Defaults to 14 days.
|
||||
|
||||
### `[postfix]`
|
||||
|
||||
#### `mailname`
|
||||
|
||||
The hostname that [Postfix should be configured to receive mail
|
||||
at](email-gateway.md#local-delivery-setup), as well as identify itself as for
|
||||
outgoing email.
|
||||
|
||||
### `[postgresql]`
|
||||
|
||||
#### `effective_io_concurrency`
|
||||
|
||||
Override PostgreSQL's [`effective_io_concurrency`
|
||||
setting](https://www.postgresql.org/docs/current/runtime-config-resource.html#GUC-EFFECTIVE-IO-CONCURRENCY).
|
||||
|
||||
#### `listen_addresses`
|
||||
|
||||
Override PostgreSQL's [`listen_addresses`
|
||||
setting](https://www.postgresql.org/docs/current/runtime-config-connection.html#GUC-LISTEN-ADDRESSES).
|
||||
|
||||
#### `random_page_cost`
|
||||
|
||||
Override PostgreSQL's [`random_page_cost`
|
||||
setting](https://www.postgresql.org/docs/current/runtime-config-query.html#GUC-RANDOM-PAGE-COST)
|
||||
|
||||
#### `replication_primary`
|
||||
|
||||
On the [warm standby replicas](deployment.md#postgresql-warm-standby), set to the
|
||||
hostname of the primary PostgreSQL server that streaming replication
|
||||
should be done from.
|
||||
|
||||
#### `replication_user`
|
||||
|
||||
On the [warm standby replicas](deployment.md#postgresql-warm-standby), set to the
|
||||
username that the host should authenticate to the primary PostgreSQL
|
||||
server as, for streaming replication. Authentication will be done
|
||||
based on the `pg_hba.conf` file; if you are using password
|
||||
authentication, you can set a `postgresql_replication_password` secret
|
||||
for authentication.
|
||||
|
||||
#### `skip_backups`
|
||||
|
||||
If set to as true value, inhibits the nightly [`wal-g` backups][wal-g] which
|
||||
would be taken on all non-replicated hosts and [all warm standby
|
||||
replicas](deployment.md#postgresql-warm-standby). This is generally only set if you have
|
||||
multiple warm standby replicas, in order to avoid taking multiple backups, one
|
||||
per replica.
|
||||
|
||||
#### `backups_disk_concurrency`
|
||||
|
||||
Number of concurrent disk reads to use when taking backups. Defaults to 1; you
|
||||
may wish to increase this if you are taking backups on a replica, so can afford
|
||||
to affect other disk I/O, and have an SSD which is good at parallel random
|
||||
reads.
|
||||
|
||||
#### `backups_storage_class`
|
||||
|
||||
What [storage class](https://aws.amazon.com/s3/storage-classes/) to use when
|
||||
uploading database backups. Defaults to `STANDARD`, meaning "[S3
|
||||
standard][s3-standard]", but many deployments will have overall lower costs if
|
||||
"[S3 Standard - Infrequent Access][s3-ia]" is used, via the `STANDARD_IA`
|
||||
value. Also supported is "[S3 Reduced Redundancy][s3-rr]", by setting
|
||||
`REDUCED_REDUNDANCY`, but this is not suggested for production use.
|
||||
|
||||
[s3-standard]: https://aws.amazon.com/s3/storage-classes/#General_purpose
|
||||
[s3-ia]: https://aws.amazon.com/s3/storage-classes/#Infrequent_access
|
||||
[s3-rr]: https://aws.amazon.com/s3/reduced-redundancy/
|
||||
|
||||
#### `missing_dictionaries`
|
||||
|
||||
If set to a true value during initial database creation, uses PostgreSQL's
|
||||
standard `pg_catalog.english` text search configuration, rather than Zulip's
|
||||
improved set of stopwords. Has no effect after initial database construction.
|
||||
|
||||
#### `ssl_ca_file`
|
||||
|
||||
Set to the path to the PEM-encoded certificate authority used to
|
||||
authenticate client connections.
|
||||
|
||||
#### `ssl_cert_file`
|
||||
|
||||
Set to the path to the PEM-encoded public certificate used to secure
|
||||
client connections.
|
||||
|
||||
#### `ssl_key_file`
|
||||
|
||||
Set to the path to the PEM-encoded private key used to secure client
|
||||
connections.
|
||||
|
||||
#### `ssl_mode`
|
||||
|
||||
The mode that should be used to verify the server certificate. The
|
||||
PostgreSQL default is `prefer`, which provides no security benefit; we
|
||||
strongly suggest setting this to `require` or better if you are using
|
||||
certificate authentication. See the [PostgreSQL
|
||||
documentation](https://www.postgresql.org/docs/current/libpq-ssl.html#LIBPQ-SSL-SSLMODE-STATEMENTS)
|
||||
for potential values.
|
||||
|
||||
#### `version`
|
||||
|
||||
The version of PostgreSQL that is in use. Do not set by hand; use the
|
||||
[PostgreSQL upgrade tool](upgrade.md#upgrading-postgresql).
|
||||
|
||||
### `[memcached]`
|
||||
|
||||
#### `memory`
|
||||
|
||||
Override the number of megabytes of memory that memcached should be
|
||||
configured to consume; defaults to 1/8th of the total server memory.
|
||||
|
||||
#### `max_item_size`
|
||||
|
||||
Override the maximum size that an item in memcached can store. This defaults to
|
||||
1m; adjusting it should only be necessary if your Zulip server has organizations
|
||||
which have more than 20k users.
|
||||
|
||||
### `[loadbalancer]`
|
||||
|
||||
#### `ips`
|
||||
|
||||
Comma-separated list of IP addresses or netmasks of external load balancers
|
||||
whose `X-Forwarded-For` and `X-Forwarded-Proto` should be respected. These can
|
||||
be individual IP addresses, or CIDR IP address ranges.
|
||||
|
||||
### `[http_proxy]`
|
||||
|
||||
#### `host`
|
||||
|
||||
The hostname or IP address of an [outgoing HTTP `CONNECT`
|
||||
proxy](deployment.md#customizing-the-outgoing-http-proxy). Defaults to
|
||||
`localhost` if unspecified.
|
||||
|
||||
#### `port`
|
||||
|
||||
The TCP port of the HTTP `CONNECT` proxy on the host specified above.
|
||||
Defaults to `4750` if unspecified.
|
||||
|
||||
#### `listen_address`
|
||||
|
||||
The IP address that Smokescreen should bind to and listen on.
|
||||
Defaults to `127.0.0.1`.
|
||||
|
||||
#### `enable_for_camo`
|
||||
|
||||
Because Camo includes logic to deny access to private subnets, routing
|
||||
its requests through Smokescreen is generally not necessary. Set to
|
||||
true or false to override the default, which uses the proxy only if
|
||||
it is not the default of Smokescreen on a local host.
|
||||
|
||||
### `[sentry]`
|
||||
|
||||
#### `organization`
|
||||
|
||||
The Sentry organization used for the [Sentry deploy hook](deployment.md#sentry-deploy-hook).
|
||||
|
||||
#### `project`
|
||||
|
||||
The Sentry project used for the [Sentry deploy hook](deployment.md#sentry-deploy-hook).
|
||||
@@ -95,7 +95,7 @@ servers; Zulip defaults this value to the first nameserver found in
|
||||
will need to run `/home/zulip/deployments/current/scripts/zulip-puppet-apply` to
|
||||
update the nginx configuration for the new value.
|
||||
|
||||
[s3-resolver]: deployment.md#nameserver
|
||||
[s3-resolver]: system-configuration.md#nameserver
|
||||
|
||||
## S3 bucket policy
|
||||
|
||||
|
||||
Reference in New Issue
Block a user