Commit Graph

47026 Commits

Author SHA1 Message Date
Alex Vandiver
85e2738a00 version: Update version after 5.7 release. 2022-11-16 16:01:34 +00:00
Alex Vandiver
6a67c5b852 Release Zulip Server 5.7. 5.7 2022-11-16 15:44:17 +00:00
Anders Kaseorg
59edbfa411 scim: Check SCIM tokens using constant-time comparison.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-16 15:41:32 +00:00
Anders Kaseorg
1dc094dded requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit b9dfa23666)
2022-11-15 15:28:42 -08:00
Anders Kaseorg
62730cde13 openapi: Fix “contatining” typo.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit c6f6df986c)
2022-11-15 15:28:42 -08:00
Anders Kaseorg
5ffd941f09 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 3417bb4c28)
2022-11-15 15:28:42 -08:00
Anders Kaseorg
d509cd0a0f markdown: Store ZulipMarkdown in members with the right type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 8230324068)
2022-11-15 15:28:42 -08:00
Anders Kaseorg
07e1e47db3 fenced_code: Avoid sloppy AttributeError handler.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 2bd81dd5c9)
2022-11-15 15:28:42 -08:00
Anders Kaseorg
5983770d7b markdown: Rename our Markdown subclass to ZulipMarkdown.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 3cf91e9e45)
2022-11-15 15:28:42 -08:00
Anders Kaseorg
86bfc7067c markdown: Remove Optional from zulip_rendering_result type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 97be895cf0)
2022-11-15 15:28:42 -08:00
Anders Kaseorg
140fcee48c markdown: Add missing None check in InlineInterestingLinkProcessor.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit d01c99d2ee)
2022-11-15 15:28:42 -08:00
Anders Kaseorg
864649aa3e subdomains: Fix realm=None case for is_static_or_current_realm_url.
Fixes #22636.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit eb9957aa35)
2022-11-15 15:28:42 -08:00
Anders Kaseorg
fe372a47c7 python: Mark dict parameters with defaults as read-only.
Found by semgrep 0.115 more accurately applying the rule added in
commit 0d6c771baf (#15349).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 47c5deeccd)
2022-11-15 15:28:42 -08:00
Mateusz Mandera
e730a25754 scim: Upgrade scim2-filter-parser to 0.4.0 to fix case-sensitivity.
userName lookups are supposed to be case-insensitive per the SCIM2 RFC
and this was fixed upstream in
https://github.com/15five/scim2-filter-parser/pull/31
and included in 0.4.0 release.

(cherry picked from commit 43107fcdc3)
2022-11-15 15:28:42 -08:00
Zixuan James Li
9f3802f5bf requirements: Upgrade types-stripe to 3.5.1.
This includes the change from 28fde2ee27.
Only a minor bump is required because it has no effect on type
checking yet before django-stubs gets integrated.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
(cherry picked from commit b5f1134172)
2022-11-15 15:28:42 -08:00
Alex Vandiver
75af21ac98 release: Support -rc1 style suffixes for releases.
These suffixes suppress some checks in the process, but still generate
and upload a tarball, push a tag, and make a Github prerelease.
`upload-release` already understands that anything with a suffix never
becomes the "latest" release.

(cherry picked from commit f3fd0b6975)
2022-11-15 14:12:53 -08:00
Alex Vandiver
d7cd2a3079 build-release-tarball: Override merge-base for full official releases.
(cherry picked from commit 1f1e1e4ec2)
2022-11-15 14:12:53 -08:00
Alex Vandiver
b7e15df30b release: Be explicit about which branch was expected.
(cherry picked from commit e95e8b985f)
2022-11-15 14:12:53 -08:00
Alex Vandiver
5f8ee7a15f release: Be consistent about TZ-setting.
(cherry picked from commit abb7a63b62)
2022-11-15 14:12:53 -08:00
Tim Abbott
d383b2ff99 tools: Don't display release candidate as forks.
(cherry picked from commit 931ed069b3)
2022-11-15 14:12:53 -08:00
Anders Kaseorg
d91e397d5c tornado: Fix Tornado CSRF check with X-Forwarded-Proto.
Since Django factors request.is_secure() into its CSRF check, we need
this to tell it to consider requests forwarded from nginx to Tornado
as secure.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit ce9ceb7f9f)
2022-11-15 10:13:00 -08:00
Alex Vandiver
001e37a34f ci: Do not pre-install rabbitmq-server in Docker images.
Before Zulip 4.9, the Zulip install process left any already-installed
rabbitmq with whatever nodename it had previously configured.  Wince
this encodes the name of the host when it was installed, this does not
function well with containers.

Leave rabbitmq-server uninstalled, which lets the Zulip installation
process set the nodename to `localhost`, which ensures that it is
usable across container restarts.

(cherry picked from commit 63d2565467)
2022-11-15 08:42:23 -05:00
Alex Vandiver
2fdf0f60ca ci: Backport two patches to the 3.4 prod install process.
(cherry picked from commit b161cba078)
2022-11-15 08:42:23 -05:00
Alex Vandiver
de76c3d498 ci: Stop rabbitmq when making CI images.
The production CI image starts `rabbitmq-server` but does not stop it,
which leaves a stale `/var/run/rabbitmq/pid` file in the image.

`rabbitmqctl wait --timeout 600 /var/run/rabbitmq/pid`, which is run
after starting the rabbitmq node, reads the PID file and waits for the
PID to be running, and for rabbitmq's port to be responding to pings.

If it reads an old PID file before the new PID is written, it
aborts (all but the first and last lines are output from `rabbitmqctl
wait` that is hidden by `/etc/init.d/rabbitmq-server`):

```
 * Starting RabbitMQ Messaging Server rabbitmq-server
Waiting for pid file '/var/run/rabbitmq/pid' to appear
pid is 341
Waiting for erlang distribution on node 'rabbit@fc8f64d6acdb' while OS process '341' is running
Error:
process_not_running
 * FAILED - check /var/log/rabbitmq/startup_\{log, _err\}
 ```

If it failed, the `production-upgrade` script tried to start
`rabbitmq` again -- despite it already still starting in the
background.  These two attempts conflicted, and often one or both
failed.

Stop `rabbitmq-server` when building the image, which removes the
stale PID file.

(cherry picked from commit fb338f22d7)
2022-11-15 08:42:23 -05:00
Anders Kaseorg
95e0c0313d settings_config: Add specialize_stream_notification_setting table.
This fixes a bug in commit 513207523c
(#21284) where handle_global_notification_updates would throw an error
on wildcard_mentions_notify because our API isn’t as symmetric as it
should be.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit d123056000)
2022-11-14 16:08:18 -08:00
Anders Kaseorg
0015d546f4 settings_config: Add generalize_stream_notification_setting table.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 5387e56da7)
2022-11-14 16:08:18 -08:00
Anders Kaseorg
827ef3fef4 sub_store: Extract StreamSpecificNotificationSettings type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 77599d4db1)
2022-11-14 16:08:18 -08:00
Anders Kaseorg
b48ba7de1d user_settings: Extract {Stream,Pm}NotificationSettings types.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 4240b0400b)
2022-11-14 16:08:18 -08:00
Anders Kaseorg
3581f957c5 settings_config: Remove incorrect ‘Type’ suffix on type names.
‘Type’ only makes sense as a suffix on a type *of types*.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 804be2f2a5)
2022-11-14 16:08:18 -08:00
Mateusz Mandera
43606b6e39 do_deactivate_user: Use .on_commit around send_event calls.
The previous commit did this for revoking sessions. send_events should
be handled similarly too, to correctly handle calling do_deactivate_user
inside a transaction.

(cherry picked from commit 470c0458e6)
2022-11-14 16:06:25 -08:00
Mateusz Mandera
a3d9297c37 do_deactivate_user: Revoke sessions in transaction.on_commit().
Fixes #21709.

(cherry picked from commit a94b2572be)
2022-11-14 16:06:25 -08:00
Anders Kaseorg
7f6bd415f0 timeout: Correct thread id type passed to PyThreadState_SetAsyncExc.
This type changed in Python 3.7:
https://github.com/python/cpython/pull/781

Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 89e4233b68)
2022-11-14 16:05:05 -08:00
Anders Kaseorg
f14dde1a64 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit a8d640a5d6)
2022-11-14 17:11:39 -05:00
Anders Kaseorg
917b2ca1b6 scripts: Help mypy resolve the psycopg2.connect overload.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 5d77d50423)
2022-11-14 17:11:39 -05:00
Anders Kaseorg
88b6f1fe14 run-codespell: Update codespell exclude list.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 4fb97576e9)
2022-11-14 17:11:39 -05:00
Anders Kaseorg
c61799aef9 test_message_flags: Fix “accessibility” spelling.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 019c9527c1)
2022-11-14 17:11:39 -05:00
Anders Kaseorg
596057dd0a populate_db: Spell “stickiness” correctly.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit f27a709a5e)
2022-11-14 17:11:39 -05:00
Mateusz Mandera
897476d4fb middleware: Log 5xx json_errors in JsonErrorHandler.
django.request logs responses with 5xx response codes (our configuration
of the logger prevents it from logging 4xx as well which it normally
does too). However, it does it without the traceback which results in
quite unhelpful log message that look like
"Bad Gateway:/api/v1/users/me/apns_device_token" - particularly
confusing when sent via email to server admins.

The solution here is to do the logging ourselves, using Django's
log_response() (which is meant for this purpose), and including the
traceback. Django tracks (via response._has_been_logged attribute) that
the response has already been logged, and knows to not duplicate that
action. See log_response() in django's codebase for these details.

Fixes #19596.
2022-10-17 17:45:44 -04:00
Mateusz Mandera
cdd01b9f8e send_analytics_to_remote_server: Log connection errors with traceback.
It seems helpful for this to get logged with the traceback rather than
just the general
"<exception name>  while trying to connect to push notification bouncer."
2022-10-17 17:45:44 -04:00
Anders Kaseorg
55c0a15f1c middleware: Pass unhandled API exceptions through to the test suite.
This results in more useful stack traces in failing tests.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-17 17:45:44 -04:00
Alex Vandiver
4139427c41 js: Only trigger reload-on-focus once.
We have observed infrequent storms of accesses (tens of thousands of
requests to minute) to `/` after an event queue expires.  The current
best theory is that the act of reloading the page itself triggers a
focus event, which itself triggers a reload before the prior one had
had time to do anything but send the network request.

Since the `focus` event here is merely as a backstop in case the
synchronous reloading and deferred reloading fail, we need only run it
once.

(cherry picked from commit e7ff4afc36)
2022-10-17 17:45:14 -04:00
Alex Vandiver
09319a1be0 js: Prevent scheduling a non-immediate reload while one is happening.
Prevent a non-immediate reload from being scheduled while an immediate
reload is already in progress.  This is highly unlikely in practice,
but is a reasonable safeguard.

(cherry picked from commit f8c9d60d33)
2022-10-17 17:45:14 -04:00
Alex Vandiver
ee88d4b4fa js: Ensure that reload.initiate cannot fall through.
A `reload.initiate({immediate: true, ...})` *should* not return, as it
should trigger a `window.location.reload` and stop execution.

In the event that it continues execution and returns (for instance,
due to being in the background and reloads being suppressed for
power-saving -- see #6821), there is no need to fall through and
potentially schedule a 90-second-later retry.

(cherry picked from commit ffadf82f8c)
2022-10-17 17:45:14 -04:00
Varun Sharma
385a408be5 ci: Limit GitHub token permissions for workflows.
This limits the ability for an Action to do mischief with this token.

Fixes #22786.

Signed-off-by: Varun Sharma <varunsh@stepsecurity.io>
2022-08-30 10:06:44 -07:00
Tim Abbott
e469cf4bc9 release: Don't remove OUTPUT_DIR on exit.
This ends up deleting your local copy of the release tarball, which
can be annoying if you need to upload it manually to the GitHub
releases page.
2022-08-25 12:51:41 -07:00
Tim Abbott
5823cc79f1 release: Fix pushing new release tags. 2022-08-25 12:51:41 -07:00
Tim Abbott
97efbf8590 version: Update version after 5.6 release. 2022-08-24 16:34:43 -07:00
Tim Abbott
d6533973b6 Release Zulip Server 5.6. 5.6 2022-08-24 13:10:05 -07:00
Anders Kaseorg
b12a5be4a0 CVE-2022-36048: Rewrite only specific local links to relative.
Due to mismatches between the URL parsers in Python and browsers, it
was possible to hoodwink rewrite_local_links_to_relative into
generating links that browsers would interpret as absolute.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-08-24 13:07:42 -07:00
Anders Kaseorg
09fb71f95a requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 74f7b6e4fd)
2022-08-23 15:25:01 -07:00