We also make sure that /help/ is accessible when using the dev server.
We add vite as a dependency. v7 of vite is the latest version but using
that causes type problems for PluginOptions since astro uses vite v6.
Therefore we pin vite to v6 for now.
This commit also runs `pnpm dedupe` for `tinyglobby` pinning it to
0.2.15 instead of the previous 0.2.14 in the lockfile.
The upstream PostgreSQL default is 4, which is more appropriate for
spinning disks. In general, production deploys almost always use
SSDs; adjust the Zulip default value to a better value for those.
This fixes behaviour where a Docker image with a memory limit would
immediately OOM, as it would not detect the container's memory limit,
only the host's.
Fixes: #29794
Pagefind files with the extensions `pf_fragment`, `pf_mindex` and
`pf_meta` are hashed in pagefind and thus we can cache it agressively.
Other files in pagefind are not hashed, although they only change
between different versions, we can think about a caching strategy for it
later.
Having a separate block was redundant since moving /help/_astro/ inside
location /help means that it can inherit its alias and we don't have to
re-declare it.
Default installs will re-examine every stream, since the process is
relatively fast. Large installs have the option to only examine
streams which have the potential to have changed subscriber counts.
We have a copy of help center with relative links disabled which is
reserved for root domains without an organisation on the root domain.
Ideally, we should have some logic to determine whether we are on such
a root domain or not. For practical short term purposes, since this
type of documentation is mainly useful for zulip.com, we add an
exception for zulip.com.
Fixes#35131.
Most of the nginx config is copied from
https://docs.astro.build/en/recipes/docker/#nginx.
We ideally should be generating two builds for Zulip cloud and serve a
different one depending on whether the subdomain is same as the root
domain or not. We can look into this as an immediate followup while this
commit helps things get ready for a test deploy on CZO.
This is done synchronously, despite taking ~60s. We can move it to a
background thread later if that's an issue, but generally Prometheus
is tolerant to exporters taking a while to come back with results.
This is most important for `send_zulip_update_announcements`, which
can race with the version run as a post-deploy hook. However, all of
these crons can tolerate being slightly delayed, and there's little
benefit to them taking CPU or possibly hitting odd borderline race
conditions when the deploy is in progress.
For safety, we only trust the deploy lockfile if it was created
within the last hour -- deploys should not take more than an hour, and
failing to ever run hourly crons is much worse than perhaps running
them during a real very-long deploy.
Although this code was not actually vulnerable as written, we never
want to be disabling this Ruff rule, in order to discourage later
introduction of vulnerabilities.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Using postfix to handle the incoming email gateway complicates things
a great deal:
- It cannot verify that incoming email addresses exist in Zulip before
accepting them; it thus accepts mail at the `RCPT TO` stage which it
cannot handle, and thus must reject after the `DATA`.
- It is built to handle both incoming and outgoing email, which
results in subtle errors (1c17583ad5, 79931051bd, a53092687e,
#18600).
- Rate-limiting happens much too late to avoid denial of
service (#12501).
- Mis-configurations of the HTTP endpoint can break incoming
mail (#18105).
Provide a replacement SMTP server which accepts incoming email on port
25, verifies that Zulip can accept the address, and that no
rate-limits are being broken, and then adds it directly to the
relevant queue.
Removes an incorrect comment which implied that missed-message
addresses were only usable once. We leave rate-limiting to only
channel email addresses, since missed-message addresses are unlikely
to be placed into automated systems, as channel email addresses are.
Also simplifies #7814 somewhat.
This provides at least a 10x speedup.
We could use a CTE to fetch from, and lock, `fts_update_log` at the
same time as performing the `zerver_message` update -- however, this
would mean that the number of returned rows would be less than
BATCH_SIZE if some rows in `zerver_message` no longer existed, causing
premature termination of the outer loop.
We lock the `zerver_message` rows in ascending ID order, to prevent
deadlocks. We use `ctid` values[^1] to transfer the rows from the
inner query to the outer update; this is safe, as the rows are locked
from updates.
[^1]: https://www.postgresql.org/docs/current/ddl-system-columns.html#DDL-SYSTEM-COLUMNS-CTID
Every insert into `fts_update_log` triggers a `NOTIFY`; processing a
batch of 1000 for every NOTIFY that we get thus results in a lot of
extra processing of no rows.
Instead, we clear all notifies, and repeat the pattern of processing
rows until the batch comes up short. This may still result in extra
wake-ups, in the event that a new NOTIFY occurs after we clear them,
and that new row is processed by us in `update_all_rows`. However,
these wake-ups are necessary for correctness, and this change will
result in drastically fewer extra checks.
This code dates back to 57b52310639a; however, this has been handled
by `postgresql-common` adding a post-install trigger to call
`pg_updatedicts` for each new PostgreSQL version, since
`postgresql-common` version 153 (February 2014).
This uses the same technique used in 840884ec89, to only apply select
parts of the Puppet configuration. This is more correct, and simpler,
than attempting to chop out some base puppet roles, and hack around
the `purge => true` supervisor.d configuration.
The value, if unset, defaults to[^1]:
> a size equal to 1/32nd (about 3%) of shared_buffers, but not less
> than 64kB nor more than the size of one WAL segment, typically 16MB
Letting it choose this, by default, but allowing an override via
`zulip.conf`, seems preferable to hard-coding an arbitrary value.
[^1]: https://www.postgresql.org/docs/current/runtime-config-wal.html#GUC-WAL-BUFFERS
The previous thresholds were set based on the assumption that 50% of
the memory on the host was taken up by Django and friends, leaving
only 50% for the database host.
On separate database hosts, this leads to a too-small default value
for shared_buffers, which is generally suggested to be 25% of your
database-dedicated memory.
Check if there is a `zulip::app_frontend_base` as part of the deploy,
and use that to adjust if we halve `$zulip::common::total_memory_mb`
or not for purposes of calculating PostgreSQL memory settings. We
also provide a `postgresql.memory` override, if necessary.
