1596 Commits

Author SHA1 Message Date
Alex Vandiver
a8f0cb2cf9 restart-server: All realm Tornado ports need a restart when they change. 2025-10-21 11:38:03 -07:00
Alex Vandiver
21f08265de restart-server: --tornado-reshard implies --skip-client-reloads.
The flags are marked mutually exclusive, so don't pass both; have the
former imply the latter.
2025-10-21 11:38:03 -07:00
Anders Kaseorg
4192d46a0e upgrade-zulip-from-git: Fix fully broken refname parsing.
Commit c903128eb7 (#36214) was evidently
never tested.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-10-12 16:47:53 -07:00
Alex Vandiver
c903128eb7 upgrade-zulip-from-git: Provide better error message on a bad refname. 2025-10-07 22:44:53 -07:00
Anders Kaseorg
88a7fde8bd install-node: Upgrade Node.js from 22.19.0 to 22.20.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-10-07 22:42:50 -07:00
Alex Vandiver
01664a1a10 sharding: Swap new config into place during restart-server.
This allows restart-server, before moving the new config into place,
to perform a diff and only restart the affected Tornado ports.
2025-10-07 13:19:05 -07:00
Alex Vandiver
e18b0fcd16 sharding: Use restart-server after setting up new sharding config.
restart-server has grown more complex since
efresh-sharding-and-restart was written -- use that complexity.  For
instance, workers are stopped one-at-a-time, rather than requiring
that they all stop sequentially before being started again.

It also means that Django restarts will be rolling, which trades off
500s during login for realms which are moving, in exchange for leaving
other realms unaffected.
2025-10-07 13:19:05 -07:00
Alex Vandiver
2bd4c07fdb sharding: Move chown/chmod into where the files are written. 2025-10-07 13:19:05 -07:00
Alex Vandiver
6249c2f970 sharding: Make nginx shard map owned by zulip:zulip.
This will make it easier to manipulate in restart-server.
2025-10-07 13:19:05 -07:00
Alex Vandiver
970ff73c5f restart-server: Fix comment to be sensical. 2025-10-07 09:40:53 -07:00
Anders Kaseorg
890ccec8d4 timestamp: Use localized formatting in format_datetime_to_string.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-10-03 10:43:16 -07:00
Anders Kaseorg
0b400a66cc install-uv: Upgrade uv from 0.8.15 to 0.8.22.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-30 16:47:54 -07:00
Anders Kaseorg
270c50651c install-uv: Upgrade uv from 0.8.10 to 0.8.15.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-04 19:48:55 -07:00
Anders Kaseorg
fc32adaa38 install-node: Upgrade Node.js from 22.18.0 to 22.19.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-04 15:01:37 -07:00
Alex Vandiver
fc962b8b0e run-hooks: Handle hooks after OS upgrade on fresh install. 2025-09-03 15:20:53 -07:00
Alex Vandiver
bef864251b reload-clients: Ensure that Smokescreen does not interfere with reloads.
When run from cron, reload-server (and thus reload-clients) picks up
the `HTTP_proxy` environment variable, which redirects HTTP requests
through Smokescreen -- which prevents localhost requests.  This
results in clients never getting sent reload events.

Explicitly unset proxies when talking to localhost in reload-clients.
2025-08-19 23:39:38 -07:00
Anders Kaseorg
8de55e7cc1 install-node: Upgrade Node.js from 22.17.1 to 22.18.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-08-14 21:37:50 -07:00
Anders Kaseorg
26dc355bf5 install-uv: Upgrade uv from 0.7.21 to 0.8.10.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-08-13 23:40:33 -07:00
Anders Kaseorg
ff15d746c3 install: Support Debian 13.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2025-08-13 14:11:05 -07:00
Anders Kaseorg
fd1543a67c apt-repos: Use signed-by instead of globally trusted keys.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2025-08-13 14:11:05 -07:00
Anders Kaseorg
60fe9d438f build-pgroonga: Use msgpack-c on Debian 13.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2025-08-13 14:11:05 -07:00
Anders Kaseorg
2e97637d8e build-pgroonga: Upgrade PGroonga from 3.1.8 to 4.0.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-08-13 14:11:05 -07:00
Alex Vandiver
e132af28fc start-server: Warn on deploy mismatch if it is a noop. 2025-08-12 10:50:30 -07:00
Alex Vandiver
f6eba9c396 log-search: Add --extra filter, for [dm] and similar filters. 2025-08-12 10:25:44 -07:00
Shubham Padia
3e60b16ac1 starlight_help: Move help-beta over to starlight_help.
We are starting the cutover process and starlight_help is the directory
we have agreed on to place our new help center project. We do not want
to use `starlight_help` as the URL for the project, but this commit
changes the url from `help-beta` to `starlight_help` temporarily since
we can only change URL once we get rid of the current help center
project. That will be done in a future commit.
2025-08-08 11:59:29 -07:00
Alex Vandiver
fd390b9eb1 settings: Enable Altcha in development if on localhost.
We only do this on localhost, because users of remote development
servers will run afoul of "Secure context is required"[^1].

[^1]: https://altcha.org/docs/troubleshooting/#secure-context
2025-07-25 22:29:51 -07:00
Anders Kaseorg
c911dcf279 install-node: Upgrade Node.js from 22.17.0 to 22.17.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-07-22 14:19:56 -07:00
Alex Vandiver
43034a79b6 run_hooks: Make post-deploy hooks be best-effort. 2025-07-16 18:00:47 -07:00
Alex Vandiver
e9a2ee56c3 log-search: Add --stats flag. 2025-07-16 09:22:02 -07:00
Alex Vandiver
2616b7d030 log-search: Add a missing new filter flag to -L. 2025-07-16 09:22:02 -07:00
Alex Vandiver
6d80d4c77f restart-server: Skip weekly restart-server if a deploy is in process.
This is unlikely, but cron running `restart-server` at the random
wrong time mid-deploy could be bad.
2025-07-15 09:07:36 -07:00
Anders Kaseorg
996eb72e2a install-uv: Upgrade uv from 0.7.15 to 0.7.21.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-07-15 07:32:44 -07:00
Prakhar Pratyush
8b3cef554b settings: Add push_registration_encryption_keys map.
The `push_registration_encryption_keys` map stores the
assymetric key pair generated on bouncer.

The public key will be used by the client to encrypt
registration data and the bouncer will use the corresponding
private key to decrypt.

- Updated the `generate_secrets.py` script to generate the map
during installation in dev environment.
- Added a management command to add / remove key i.e. use it
for key rotation while retaining the older key-pair for a period
of time.
2025-07-06 21:11:26 -07:00
Anders Kaseorg
7959a1853c install-node: Upgrade Node.js from 22.16.0 to 22.17.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-06-25 16:53:09 -07:00
Anders Kaseorg
9f8f6e60d9 install-uv: Upgrade uv from 0.7.11 to 0.7.15.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-06-25 16:52:39 -07:00
Anders Kaseorg
cdbe2d157f flush_memcached: Respect DJANGO_SETTINGS_MODULE.
We don’t need to flush anything for zproject.test_settings, which
disables memcached.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-06-25 11:49:02 -07:00
Alex Vandiver
b924169d17 setup-apt-repo: Add libheif PPA, and debian bookworm backport.
libheif 1.18 is required to be able to parse images generated by iOS
18; none of Zulip's supported distributions package libheif 1.18, so
we pull new version of the package from PPA (Ubuntu) or backports
(Debian).
2025-06-25 11:39:18 -07:00
Alex Vandiver
a0683927ef check_rabbitmq_queue: Relax paging thresholds for email_senders. 2025-06-18 12:29:57 -07:00
Anders Kaseorg
acd6c51b6f manage: Delete custom PYTHONSTARTUP.
In Django 5.2, manage.py shell automatically imports models.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-06-12 09:28:00 -07:00
Anders Kaseorg
927ea011d3 upgrade-postgresql: Get PostgreSQL version without manage.py shell.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-06-11 16:25:25 -07:00
Alex Vandiver
6f1950ac0e restart-server: Send client reload events in the background.
For deploys with --skip-puppet, this makes the output visible much
more promptly.
2025-06-11 10:16:46 -07:00
Anders Kaseorg
56470bba8d install-uv: Upgrade uv from 0.7.2 to 0.7.11.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-06-06 11:35:32 -07:00
Tim Abbott
0ec07fe4c8 queue: Allow sharding user_activity worker.
This follows the existing patterns for the sharded mobile
notifications worker.
2025-06-06 10:33:20 -07:00
Anders Kaseorg
f6be163bcc install-node: Upgrade Node.js from 22.15.0 to 22.16.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-06-04 16:24:47 -07:00
Alex Vandiver
c6e0f0b436 email-mirror: Remove HTTP interface. 2025-05-19 16:39:44 -07:00
Alex Vandiver
7a62a9b509 upgrade: Swap postfix_localmail for local_mailserver. 2025-05-19 16:39:44 -07:00
Alex Vandiver
1f0cfd4662 email-mirror: Add a standalone server that processes incoming email.
Using postfix to handle the incoming email gateway complicates things
a great deal:

- It cannot verify that incoming email addresses exist in Zulip before
  accepting them; it thus accepts mail at the `RCPT TO` stage which it
  cannot handle, and thus must reject after the `DATA`.

- It is built to handle both incoming and outgoing email, which
  results in subtle errors (1c17583ad5, 79931051bd, a53092687e,
  #18600).

- Rate-limiting happens much too late to avoid denial of
  service (#12501).

- Mis-configurations of the HTTP endpoint can break incoming
  mail (#18105).

Provide a replacement SMTP server which accepts incoming email on port
25, verifies that Zulip can accept the address, and that no
rate-limits are being broken, and then adds it directly to the
relevant queue.

Removes an incorrect comment which implied that missed-message
addresses were only usable once.  We leave rate-limiting to only
channel email addresses, since missed-message addresses are unlikely
to be placed into automated systems, as channel email addresses are.

Also simplifies #7814 somewhat.
2025-05-19 16:39:44 -07:00
Alex Vandiver
0442bb6f0e upgrade-postgresql: Slightly better error-proof post-upgrade scripts. 2025-05-16 11:33:20 -07:00
Alex Vandiver
3ab6be650b upgrade-postgresql: Explicitly ask to not start the new cluster.
Recent versions of postgresql-common's `pg_upgradecluster`, starting
with version 254, (i.e. on Ubuntu 24.04, but not 22.04) will not just
_suggest_ running the analyze, but will do so automatically.  While
somewhat helpful, it always does so with `--analyze-in-stages`, which
as noted in f77bbd3323, is actually the incorrect choice for us.
Passing `--no-start` ensures that `pg_upgradecluster` consistently
does not do any analyzing, allowing us to start the cluster manually
and then perform the analyze correctly ourselves.
2025-05-16 11:33:20 -07:00
Alex Vandiver
e13f82f048 upgrade-postgresql: Use tags to partially-apply configuration.
This uses the same technique used in 840884ec89, to only apply select
parts of the Puppet configuration.  This is more correct, and simpler,
than attempting to chop out some base puppet roles, and hack around
the `purge => true` supervisor.d configuration.
2025-05-16 11:33:20 -07:00