Combine nginx and Django middlware to stop putting misleading warnings
about `CSRF_TRUSTED_ORIGINS` when the issue is untrusted proxies.
This attempts to, in the error logs, diagnose and suggest next steps
to fix common proxy misconfigurations.
See also #24599 and zulip/docker-zulip#403.
(cherry picked from commit 8a77cca341)
Having exactly 17 or 18 middlewares, on Python 3.11.0 and above,
causes python to segfault when running tests with coverage; see
https://github.com/python/cpython/issues/106092
Work around this by adding one or two no-op middlewares if we would
hit those unlucky numbers. We only add them in testing, since
coverage is a requirement to trigger it, and there is no reason to
burden production with additional wrapping.
(cherry picked from commit cf0b803d50)
Simplebar seems unaware of the `max-height: 1000px` on
`.subscriptions-container`, and therefore does not properly provide
a scrollbar when it's needed.
This commit adds a `max-height` to the stream Simplebar container,
ensuring that otherwise hidden content that Simplebar believes to
be visible can be scrolled to.
Finally, rather than rely on magic numbers or math done in comments,
this commit establishes CSS variables for all relevant modal-element
heights, doing the math inline using CSS calc().
Fixes#26107.
(cherry picked from commit 0c55fb7e89)
This is a follow-up to 4c8915c8e4, for
the case when the `team:read` permission is missing, which causes the
`team.info` call itself to fail. The error message supplies
information about the provided and missing permissions -- but it also
still sends the `X-OAuth-Scopes` header which we normall read, so we can
use that as normal.
(cherry picked from commit 21aeb4a040)
Updating the pgroonga package is not sufficient to upgrade the
extension in PostgreSQL -- an `ALTER EXTENSION pgroonga UPDATE` must
explicitly be run[^1]. Failure to do so can lead to unexpected behavior,
including crashes of PostgreSQL.
Expand on the existing `pgroonga_setup.sql.applied` file, to track
which version of the PostgreSQL extension has been configured. If the
file exists but is empty, we run `ALTER EXTENSION pgroonga UPDATE`
regardless -- if it is a no-op, it still succeeds with a `NOTICE`:
```
zulip=# ALTER EXTENSION pgroonga UPDATE;
NOTICE: version "3.0.8" of extension "pgroonga" is already installed
ALTER EXTENSION
```
The simple `ALTER EXTENSION` is sufficient for the
backwards-compatible case[^1] -- which, for our usage, is every
upgrade since 0.9 -> 1.0. Since version 1.0 was released in 2015,
before pgroonga support was added to Zulip in 2016, we can assume for
the moment that all pgroonga upgrades are backwards-compatible, and
not bother regenerating indexes.
Fixes: #25989.
[^1]: https://pgroonga.github.io/upgrade/
(cherry picked from commit c8ec3dfcf6)
This was only necessary for PGroonga 1.x, and the `pgroonga` schema
will most likely be removed at some point inthe future, which will
make this statement error out.
Drop the unnecessary statement.
(cherry picked from commit dc2726c814)
In commit #25837, we added in a check for the user's mark as read
policy in the frontend for `by_topic` and `by_recipient` narrowing.
In that change, the assumption was that for both functions, it was
sufficient to check only for whether the user policy was to never
mark as read.
But because the `by_recipient` function may narrow to an interleaved
stream view, it is possible that message will be marked as read
when the user did not expect it to be (e.g. they marked all the
messages in a topic narrow as unread and then used the `S` key
shortcut to navigate back to the stream view) when they have
conversation views only as their mark as read policy.
Here we move the check for the user's mark as read policy to be in
the two cases for `by_recipient` so that the mark as read behavior
here matches the user's setting.
(cherry picked from commit c5fbd3f085)
The syntax in `/etc/resolv.conf` does not include any brackets:
```
nameserver 2001:db8::a3
```
However, the format of the nginx `resolver` directive[^1] requires that
IPv6 addresses be enclosed in brackets.
Adjust the `resolver_ip` puppet function to surround any IPv6
addresses extracted from `/etc/resolv.conf` with square brackets, and
any addresses from `application_server.resolver` to gain brackets if
necessary.
Fixes: #26013.
[^1]: http://nginx.org/en/docs/http/ngx_http_core_module.html#resolver
(cherry picked from commit 7ef05316d5)
Previously, we used to have top and bottom paddings of 4px to
the select elements but it was removed in a208da9c4d to make
sure that text for the selected option is aligned properly.
All other select elements have height set to 30px, but the
select elements in stream settings page had height set to
"fit-content" and so they looked ugly after removing the
padding.
This commit sets the height of select elements in stream
settings to 30px.
(cherry picked from commit b119ff68c3)
Previously, `X-Forwarded-Proto` did not need to be set, and failure to
set `loadbalancer.ips` would merely result in bad IP-address
rate-limiting and incorrect access logs; after 0935d388f0, however,
failure to do either of those, if Zulip is deployed with `http_only`,
will lead to infinite redirect loops after login. These are
accompanied by a misleading error, from Tornado, of:
Forbidden (Origin checking failed - https://zulip.example.com does not match any trusted origins.): /json/events
This is most common with Docker deployments, where deployments use
another docker container, such as nginx or Traefik, to do SSL
termination. See zulip/docker-zulip#403.
Update the documentation to reinforce that `loadbalancer.ips` also
controls trust of `X-Forwarded-Proto`, and that failure to set it will
cause the application to not function correctly.
(cherry picked from commit d46279c41e)
9d97af6ebb addressed the one major source of inconsistent data which
would be solved by simply re-attempting the ScheduledEmail row. Every
other instance that we have seen since then has been a corrupt or
modified database in some way, which does not self-resolve. This
results in an endless stream of emails to the administrator, and no
forward progress.
Drop this to a warning, and make it remove the offending row. This
ensures we make forward progress.
(cherry picked from commit 77c146b8b0)
This PR creates templates for filing issues. The templates are
intentionally quite light-weight. Note that I'm specifically not using
forms for creating issues, as the UI for filling out such a form does
not include GitHub's helpful formatting buttons and preview mode.
Follow-up to #25998, pushed as a separate PR so that the original one can pass CI.
This PR creates templates for filing issues. The templates are
intentionally quite light-weight. Note that I'm specifically not using
forms for creating issues, as the UI for filling out such a form does
not include GitHub's helpful formatting buttons and preview mode.
A major goal is to guide users towards starting a CZO conversation
when that's more appropriate than filing a GitHub issue.
Note that the config makes it possible to create a blank issue, which
should be handy for:
* Issues filed by maintainers
* Issues for tracking follow-ups from merged PRs
* Probably some other situations
Because the blank issue option is easy to miss, it should probably be
documented somewhere, but I'm not sure where. We can perhaps start
with a note on CZO.
Relevant CZO threads:
https://chat.zulip.org/#narrow/stream/137-feedback/topic/issues.20link.20in.20description/near/1561110)
https://chat.zulip.org/#narrow/stream/2-general/topic/bug.20report.20management/near/1589141
Also provide external documentation links for situations where
filing an issue is not the best approach.
(cherry picked from commit 0adcc2a1df)
- Create a dedicated "Reporting bugs" page to learly document
where and how bugs should be reported.
- Drop "Reporting issues" section from the Contributing guide.
- Delete "Bug report guidelines" page.
(cherry picked from commit 052a109ba4)
04cf68b45e make nginx responsible for downloading (and caching)
files from S3. As noted in that commit, nginx implements its own
non-blocking DNS resolver, since the base syscall is blocking, so
requires an explicit nameserver configuration. That commit used
127.0.0.53, which is provided by systemd-resolved, as the resolver.
However, that service may not always be enabled and running, and may
in fact not even be installed (e.g. on Docker). Switch to parsing
`/etc/resolv.conf` and using the first-provided nameserver. In many
deployments, this will still be `127.0.0.53`, but for others it will
provide a working DNS server which is external to the host.
In the event that a server is misconfigured and has no resolvers in
`/etc/resolv.conf`, it will error out:
```console
Error: Evaluation Error: Error while evaluating a Function Call, No nameservers found in /etc/resolv.conf! Configure one by setting application_server.nameserver in /etc/zulip/zulip.conf (file: /home/zulip/deployments/current/puppet/zulip/manifests/app_frontend_base.pp, line: 76, column: 70) on node example.zulipdev.org
```
(cherry picked from commit bd217ad31b)
Failing to remove all of the rules which were added causes action at a
distance with other tests. The two methods were also only used by
test code, making their existence in zerver.lib.rate_limiter clearly
misplaced.
This fixes one instance of a mis-balanced add/remove, which caused
tests to start failing if run non-parallel and one more anonymous
request was added within a rate-limit-enabled block.
(cherry picked from commit 0dbe111ab3)
The user group depedency graph should always be a DAG.
This commit adds code to make sure we keep the graph DAG
while adding subgroups to a user group.
Fixes#25913.
We want to make sure that the system groups, once created, will always
have the GroupGroupMemberships fully set up.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
These changes appear to correct the keyboard-navigation repro
from #25907, and it makes it possible for users without the
permission to create streams to exit the streams modal by
hitting Esc.
This reorganizes logic within the Tippy `onShow` method to
ensure that nothing is set or called for those users without
stream-creation privileges.
These changes probably require broader testing to determine
whether the fix addresses only that specific reproducer, or
the broader problems #25907 addresses with malfunctioning
j, k, Esc, and Return keys (when Ctrl + Return to send is
enabled).
Fixes a part of #25907.
If the `postgresql.version` in `/etc/zulip/zulip.conf` is out of date
or wrong, upgrading to the actual current version would drop your
production database without prompting. While we do document taking a
Zulip backup (which includes a database backup) before running
`upgrade-postgresql`[^1], not everyone does so, with possibly
catastrophic consequences.
Do a true end-to-end check of the version in `/etc/zulip/zulip.conf`
by asking Django to query the database for its version, checking that
against the configured value, and aborting if there is any
disagreement.
[^1]: https://zulip.readthedocs.io/en/latest/production/upgrade.html#upgrading-postgresql
It turns out that for some some deployments, there exists a second,
duplicate, foreign key constraint for user_profile_id. The logic below
would try to rename both to the same name, which would fail on the
second:
```
psycopg2.errors.DuplicateObject: constraint "zerver_userpresenceo_user_profile_id_d75366d6_fk_zerver_us" for relation "zerver_userpresence" already exists
```
Eliminate the duplicate constraint, rather than attempting to rename
it. Also add a block, in case of future reuse of this pattern, which
caveats that this approach will not work in the presence of
explicitly-named indexes. UserPresence happens to not have any, so
this technique is safe in this instance.
Co-authored-by: Alex Vandiver <alexmv@zulip.com>
It was throwing error while schudiling a message having wildcard mention,
because the function `open_send_later_menu` was using param instance to track down
interval, but the parametere instance was not passed from when it was
called from warning banner action. This commit removes the instance
param as it is of no use, and uses a variable to track interval.
