Commit Graph

11 Commits

Author SHA1 Message Date
Alex Vandiver
01664a1a10 sharding: Swap new config into place during restart-server.
This allows restart-server, before moving the new config into place,
to perform a diff and only restart the affected Tornado ports.
2025-10-07 13:19:05 -07:00
Alex Vandiver
e18b0fcd16 sharding: Use restart-server after setting up new sharding config.
restart-server has grown more complex since
efresh-sharding-and-restart was written -- use that complexity.  For
instance, workers are stopped one-at-a-time, rather than requiring
that they all stop sequentially before being started again.

It also means that Django restarts will be rolling, which trades off
500s during login for realms which are moving, in exchange for leaving
other realms unaffected.
2025-10-07 13:19:05 -07:00
Alex Vandiver
2bd4c07fdb sharding: Move chown/chmod into where the files are written. 2025-10-07 13:19:05 -07:00
Alex Vandiver
6249c2f970 sharding: Make nginx shard map owned by zulip:zulip.
This will make it easier to manipulate in restart-server.
2025-10-07 13:19:05 -07:00
Anders Kaseorg
7666ff603d sharding: Configure Tornado sharding with nginx map.
https://nginx.org/en/docs/http/ngx_http_map_module.html

Since Puppet doesn’t manage the contents of nginx_sharding.conf after
its initial creation, it needs to be renamed so we can give it
different default contents.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-15 16:07:50 -07:00
Anders Kaseorg
ea6f18bb46 refresh-sharding-and-restart: Quote to prevent shell glob expansion.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-14 09:35:12 -07:00
Alex Vandiver
d51272cc3d puppet: Remove zulip_deliver_scheduled_* from zulip-workers:*.
Staging and other hosts that are `zulip::app_frontend_base` but not
`zulip::app_frontend_once` do not have a
/etc/supervisor/conf.d/zulip/zulip-once.conf and as such do not have
`zulip_deliver_scheduled_emails` or `zulip_deliver_scheduled_messages`
and thus supervisor will fail to reload.

Making the contents of `zulip-workers` contingent on if the server is
_also_ a `-once` server is complicated, and would involve using Concat
fragments, which severely limit readability.

Instead, expel those two from `zulip-workers`; this is somewhat
reasonable, since they are use an entirely different codepath from
zulip_events_*, using the database rather than RabbitMQ for their
queuing.
2021-06-14 17:12:59 -07:00
Alex Vandiver
8adf530400 puppet: Generate sharding in puppet, then refresh-sharding-and-restart.
This supports running puppet to pick up new sharding changes, which
will warn of the need to finalize them via
`refresh-sharding-and-restart`, or simply running that directly.
2020-09-14 16:27:15 -07:00
Alex Vandiver
bf029d99f1 sharding: Also mark sharding.json 644 for consistency.
There is no reason to limit this to 640; mark it 644 for consistency
with the other file.
2020-09-14 16:27:15 -07:00
Alex Vandiver
b5bcff04e5 sharding: Consistent mode for nginx sharding file.
This disagreed between `tornado_sharding.pp` in puppet and
`scripts/refresh-sharding-and-restart`.
2020-09-14 16:27:15 -07:00
Tim Abbott
220620e7cf sharding: Add basic sharding configuration for Tornado.
This allows straight-forward configuration of realm-based Tornado
sharding through simply editing /etc/zulip/zulip.conf to configure
shards and running scripts/refresh-sharding-and-restart.

Co-Author-By: Mateusz Mandera <mateusz.mandera@zulip.com>
2020-05-20 13:47:20 -07:00