diff --git a/docs/production/postgresql.md b/docs/production/postgresql.md index ddd4d0a5db..1943866c6d 100644 --- a/docs/production/postgresql.md +++ b/docs/production/postgresql.md @@ -170,16 +170,15 @@ su zulip -c '/home/zulip/deployments/current/manage.py generate_realm_creation_l ## PostgreSQL warm standby -Zulip's configuration allows for [warm standby database -replicas][warm-standby] as a disaster recovery solution; see the -linked PostgreSQL documentation for details on this type of -deployment. Zulip's configuration builds on top of `wal-g`, our -[streaming database backup solution][wal-g], and thus requires that it -be configured for the primary and all secondary warm standby replicas. +Zulip's configuration allows for [warm standby database replicas][warm-standby] +as a disaster recovery solution; see the linked PostgreSQL documentation for +details on this type of deployment. Zulip's configuration can, but is not +required to, build on top of `wal-g`, our [streaming database backup +solution][wal-g], for ease of establishing base images without incurring load on +the primary. -In addition to having `wal-g` backups configured, warm standby -replicas should configure the hostname of their primary replica, and -username to use for replication, in `/etc/zulip/zulip.conf`: +Warm standby replicas should configure the hostname of their primary replica, +and username to use for replication, in `/etc/zulip/zulip.conf`: ```ini [postgresql] @@ -187,13 +186,22 @@ replication_user = replicator replication_primary = hostname-of-primary.example.com ``` -The `postgres` user on the replica will need to be able to -authenticate as the `replication_user` user, which may require further -configuration of `pg_hba.conf` and client certificates on the replica. -If you are using password authentication, you can set a -`postgresql_replication_password` secret in +The `postgres` user on the replica will need to be able to authenticate as the +`replication_user` user, which may require further configuration of +`pg_hba.conf` and client certificates on the replica. If you are using password +authentication, you can set a `postgresql_replication_password` secret in `/etc/zulip/zulip-secrets.conf`. +Use `zulip-puppet-apply` to rebuild the PostgreSQL configuration with those +values. If [wal-g][wal-g] is used, use `env-wal-g backup-fetch` to fetch the +latest copy of the base backup; otherwise, use [`pg_basebackup`][pg_basebackup]. +The PostgreSQL replica should then begin live-replicating from the primary. + +In the event of a primary failure, use [`pg_ctl promote`][promote] on the warm +standby to promote it to primary. As with all database promotions, care should +be taken to ensure that the old primary cannot come back online, and leave the +cluster with two diverging timelines. + To make fail-over to the warm-standby faster, without requiring a restart of Zulip services, you can configure Zulip with a comma-separated list of remote PostgreSQL servers to connect to; it will choose the first which accepts writes @@ -207,6 +215,8 @@ REMOTE_POSTGRES_HOST = 'primary-database-host,warm-standby-host' [warm-standby]: https://www.postgresql.org/docs/current/warm-standby.html [wal-g]: export-and-import.md#database-only-backup-tools +[pg_basebackup]: https://www.postgresql.org/docs/current/app-pgbasebackup.html +[promote]: https://www.postgresql.org/docs/current/app-pg-ctl.html ## PostgreSQL vacuuming alerts