paulmataruso/zulip
1
0
Fork 0
mirror of https://github.com/zulip/zulip.git synced 2025-11-03 21:43:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

wal-g: Add support for incremental backups.

Browse Source 
This only defaults to on for local-disk backups, since they are more
disk-size-sensitive, and local accesses are quite cheap compared to
loading multiple incremental backups from S3.
This commit is contained in:
Alex Vandiver
2024-03-18 20:22:31 +00:00
parent e200f4d233
commit 323e1e92b7
3 changed files with 20 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
docs/production/export-and-import.md
View File

@@ -556,8 +556,9 @@ day.
``` ```
You may also want to adjust the You may also want to adjust the
[concurrency](system-configuration.md#backups_disk_concurrency) or [S3 storage [concurrency](system-configuration.md#backups_disk_concurrency), [S3 storage
class](system-configuration.md#backups_storage_class). class](system-configuration.md#backups_storage_class), or [incremental
backups][incremental] configuration.
### Streaming backups to local disk ### Streaming backups to local disk
@@ -584,6 +585,10 @@ analysis of recent application-level data changes.
/home/zulip/deployments/current/scripts/zulip-puppet-apply /home/zulip/deployments/current/scripts/zulip-puppet-apply
``` ```
You may also want to adjust the [incremental backups][incremental]
configuration.
[wal]: https://www.postgresql.org/docs/current/wal-intro.html [wal]: https://www.postgresql.org/docs/current/wal-intro.html
[archive-timeout]: https://www.postgresql.org/docs/current/runtime-config-wal.html#GUC-ARCHIVE-TIMEOUT [archive-timeout]: https://www.postgresql.org/docs/current/runtime-config-wal.html#GUC-ARCHIVE-TIMEOUT
[mobile-push]: ../production/mobile-push-notifications.md [mobile-push]: ../production/mobile-push-notifications.md
[incremental]: system-configuration.md#backups_incremental

5
docs/production/system-configuration.md
View File

@@ -269,6 +269,11 @@ on a NAS mountpoint, or if some other process copies this data off the disk; or
if backups are purely for point-in-time historical analysis of recent if backups are purely for point-in-time historical analysis of recent
application-level data changes. application-level data changes.
#### `backups_incremental`
The number of delta (incremental) database backups to take between full backups.
Defaults to 0 for S3 backups, and 6 for local-disk backups.
#### `backups_storage_class` #### `backups_storage_class`
What [storage class](https://aws.amazon.com/s3/storage-classes/) to use when What [storage class](https://aws.amazon.com/s3/storage-classes/) to use when

8
puppet/zulip/files/postgresql/env-wal-g
View File

@@ -5,6 +5,9 @@ fi
export PGHOST=/var/run/postgresql/ export PGHOST=/var/run/postgresql/
WALG_DELTA_MAX_STEPS=$(crudini --get /etc/zulip/zulip.conf postgresql backups_incremental 2>/dev/null)
export WALG_DELTA_MAX_STEPS
s3_backups_bucket=$(crudini --get "$ZULIP_SECRETS_CONF" secrets s3_backups_bucket 2>/dev/null) s3_backups_bucket=$(crudini --get "$ZULIP_SECRETS_CONF" secrets s3_backups_bucket 2>/dev/null)
if [ "$s3_backups_bucket" != "" ]; then if [ "$s3_backups_bucket" != "" ]; then
@@ -30,6 +33,11 @@ else
WALG_FILE_PREFIX=$(crudini --get /etc/zulip/zulip.conf postgresql backups_directory 2>/dev/null) WALG_FILE_PREFIX=$(crudini --get /etc/zulip/zulip.conf postgresql backups_directory 2>/dev/null)
if [ "$WALG_FILE_PREFIX" != "" ]; then if [ "$WALG_FILE_PREFIX" != "" ]; then
export WALG_FILE_PREFIX export WALG_FILE_PREFIX
if [ "$WALG_DELTA_MAX_STEPS" = "" ]; then
# Default to only taking a full backup every week
export WALG_DELTA_MAX_STEPS=6
fi
else else
echo "Could not determine where to back up data to!" echo "Could not determine where to back up data to!"
exit 1 exit 1