9481 Commits

Author SHA1 Message Date
Prakhar Pratyush
1468c8c160 push_notification: Prepare payload only if registered device exists.
Earlier, we were constructing the APNs & FCM payloads for legacy &
E2EE push notifications even if the user didn't have such a registered
device to send notifications to.

This commit makes changes to construct:
* apns_payload only if the user has an apple device registered
* fcm_payload only if the user has an android device registered
* payload_to_encrypt only if the user has push device registered
  which supports E2EE.

Also, now we perform one db query instead of two to calculate
`apple_devices` and `android_devices` for the legacy case.

Overall, this helps to avoid unnecessary compute.

Signed-off-by: Prakhar Pratyush <prakhar@zulip.com>
2025-10-28 13:29:02 -07:00
Prakhar Pratyush
140e575ddc push_notification: Use durable=True for the outermost transaction.
This commit updates the `transaction.atomic` context manager
used to take a lock on Message row in `handle_push_notification`
to use `durable=True` instead of `savepoint=False`. It helps to
avoid introducing an outer transaction by mistake - resulting in
locking row for a longer duration.

Signed-off-by: Prakhar Pratyush <prakhar@zulip.com>
2025-10-28 13:29:02 -07:00
Alex Vandiver
c6a60fd533 exceptions: Add link to rate-limiting docs in 429 response. 2025-10-27 16:22:13 -07:00
Alex Vandiver
17fd249a62 email_mirror_server: Fix the logfile path which is checked. 2025-10-23 11:57:30 -07:00
Niloth P
f317af2e1d integration-docs: Add Git integration screenshot config and image.
Temporarily exclude Git from `test_integration_doc_endpoints`,
until the doc is updated.

The screenshot for this integration is added separately since this
adds a new screenshot for an integration whose doc is in the
python-zulip-api repo.
Since this breaks the sync between the docs and the screenshots, an
exclusion is added to make the tests pass.
2025-10-20 10:24:49 -07:00
Niloth P
f6e847d026 integration-docs: Add jira-plugin screenshot config and image.
Temporarily exclude jira-plugin from `test_integration_doc_endpoints`,
until the doc is updated.

The screenshot for this integration is added separately since this
adds a new screenshot for an integration whose doc is in the
python-zulip-api repo.
Since this breaks the sync between the docs and the screenshots, an
exclusion is added to make the tests pass.
2025-10-20 10:24:49 -07:00
Niloth P
7b5e4db7ee integrations: Add a set NO_SCREENSHOT_CONFIG.
to track the integrations without a screenshot config.

This will be used for automated testing.
2025-10-20 10:24:49 -07:00
Niloth P
e7922e2472 integrations: Add a new category "Video calling".
For video and audio call integrations.

The new category is a subset of the "Communication" category, as all
"Video calling" integrations are also listed under "Communication".
2025-10-20 10:13:31 -07:00
Alex Vandiver
9815db9811 upload: Use normpath when comparing to LOCAL_UPLOADS_DIR.
This prevents a development-mode-only directory traversal attack,
where the Django development server could be made to respond to
requests for `/user_avatars/../../../../../../etc/passwd`.

The production server is not affected by this vulnerability, as
nginx's configuration sets `PATH_INFO` to `$document_uri`, which is
normalized[^1] -- that is, by the time uwsgi and Django see it, the path
has been percent-decoded once, and all `../` path components have been
applied[^2].

Close this by explicitly normalizing the paths before comparing; the
`LOCAL_UPLOADS_DIR` side is unlikely to require normalization as well,
but is also normalized for consistency.  The failure here is left as
an assertion failure, and not a JsonableError, because it only affects
the development server.

[^1]: https://nginx.org/en/docs/http/ngx_http_core_module.html#var_uri
[^2]: https://nginx.org/en/docs/http/ngx_http_core_module.html#location
2025-10-14 12:56:10 -07:00
Niloth P
ad122af6f8 integrations: Use dir_name instead of name for default view fn.
Use the directory name as the template literal in the default view
function path.
2025-10-14 12:17:05 -07:00
Niloth P
23740c97a4 integrations: Remove redundant arguments. 2025-10-14 12:17:05 -07:00
Niloth P
f33ef8f206 integrations: Rename IFTTT view function to match conventions. 2025-10-14 12:17:05 -07:00
Alya Abbott
aac651fa39 updates: Add update message (level=22). 2025-10-13 23:18:00 -07:00
Alex Vandiver
79e718ed3a send_email: Move break into custom_email_sender. 2025-10-09 12:10:15 -07:00
Anders Kaseorg
b8f8056f56 markdown: Use re-exported guess_type to ensure adding EXTRA_MIME_TYPES.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-10-08 23:36:15 -07:00
Anders Kaseorg
b05b5e72f5 test_helpers: Use care in load_subdomain_token URL parsing.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-10-08 23:33:47 -07:00
Tim Abbott
9361a3cffe test_classes: Give useful error when using /json/ URLs wrong. 2025-10-03 15:49:20 -07:00
Niloth P
1f4267f3e0 integrations: Add screenshot configs for monitoring integrations.
that do not have fixtures.
2025-10-03 15:38:03 -07:00
Niloth P
d70b099644 integrations: Add screenshot configs for productivity integrations.
that do not have fixtures.
2025-10-03 15:38:03 -07:00
Niloth P
433f43a9b4 integrations: Add screenshot configs for communication integrations.
that do not have fixtures.
2025-10-03 15:38:03 -07:00
Niloth P
f5e925b26b integrations: Add screenshot configs for the project management type.
of integrations that do not have fixtures.
2025-10-03 15:38:03 -07:00
Niloth P
24eab61ab0 integrations: Add screenshot configs for deployment integrations.
that do not have fixtures.
2025-10-03 15:38:03 -07:00
Niloth P
7f2161bffa integrations: Add screenshot configs for fixtureless CI integrations. 2025-10-03 15:38:03 -07:00
Niloth P
65d01b553b integrations: Add screenshot configs for fixtureless VCS integrations.
Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2025-10-03 15:38:03 -07:00
Niloth P
23a8e26741 integrations: Load optional fields for fixtureless screenshot configs.
The hardcoded content (topic and message) for the screenshot configs are
loaded from `fixtureless_integrations.py`. Add a new
FIXTURELESS_SCREENSHOT_CONFIG_OPTIONAL_FIELDS dict to maintain the
optional fields of fixtureless screenshot configs, and load both into
FIXTURELESS_SCREENSHOT_CONFIG.

This commit does not set any screenshot configs, but only adds the
framework.
2025-10-03 15:38:03 -07:00
Niloth P
c7a6fcaf88 integrations: Add new file for fixtureless screenshot configs registry.
This does not include any screenshot configs, just defining the format,
and loading FIXTURELESS_SCREENSHOT_CONFIG from the file.

Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2025-10-03 15:38:03 -07:00
Anders Kaseorg
890ccec8d4 timestamp: Use localized formatting in format_datetime_to_string.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-10-03 10:43:16 -07:00
Aman Agrawal
024559e1b9 email_notifications: Convert datetime to local date string. 2025-10-01 11:25:00 -07:00
Aman Agrawal
a694fe1865 signals: Extract function to convert datetime to readable string. 2025-10-01 11:25:00 -07:00
Alex Vandiver
755cb7d854 export: Move all queries, when possible, to iterators.
This reduces overall memory usage for large exports.
2025-10-01 11:21:34 -07:00
Alex Vandiver
67743d150a export: Support streaming iterators when writing JSON. 2025-10-01 11:21:34 -07:00
Alex Vandiver
78bc17ecbe export: Switch chunked() to the itertools.batched definition.
We only require Python 3.10, so we cannot use `itertools.batched`
directly, which came in 3.12.  Include and use a short version of it.
2025-10-01 11:21:34 -07:00
Alex Vandiver
cf33119348 export: Remove export-most-recent symlink.
The only callsite of do_export_realm calls `rmtree` on the output
path, which means this symlink is always dangling.  Since realms can
also be exported by end-users, following it would always be a race
condition, anyways.

Remove it.
2025-10-01 11:21:34 -07:00
Alex Vandiver
6a8deb7115 export: We expect that zerver_client is written late.
This silences a misleading warning; the behaviour changed in
f2024ad625.
2025-10-01 11:21:34 -07:00
Alex Vandiver
bea7d6ff93 cache_helpers: Use iterators, so we do not keep the full list in memory. 2025-10-01 10:36:37 -07:00
Anders Kaseorg
c1a4dcf33d requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-30 16:47:54 -07:00
Anders Kaseorg
2eb2d5b6de user_groups: Fix union typing.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-30 16:47:54 -07:00
Anders Kaseorg
1d694aeff4 mypy: Fix no-redef errors from django-stubs.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-30 16:47:54 -07:00
Anders Kaseorg
f24a0a6b81 ruff: Fix RUF059 Unpacked variable is never used.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-30 16:47:54 -07:00
Anders Kaseorg
fa4d680851 ruff: Fix PYI059 Generic[] should always be the last base class.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-30 16:47:54 -07:00
Anders Kaseorg
e9abaacd5c ruff: Fix PYI061 Use None rather than Literal[None].
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-30 16:47:54 -07:00
Alex Vandiver
3538455ca8 email_mirror: Drop missed-message emails which are autogenerated.
Emails to missed-message email addressees which are marked
"auto-replied" are clearly auto-replies, and will not contribute
usefully to the conversation.  We also ignore "auto-generated" emails
to missed-message addresses, as they must actually be auto-replies
which are misclassifying themselves, as missed-message addresses are
not meant to be targets for any auto-generated emails.

We accept auto-generated and auto-replied emails to stream incoming
email addresses, as auto-generated emails to those are clearly useful,
and auto-replied emails are unexpected enough to allow (given that
Zulip does not produce outgoing emails From: stream email addresses).
2025-09-26 09:18:13 -07:00
Sahil Batra
de5a78344a user_groups: Remove unnecessary select_related.
There is no need for 'select_related("usergroup_ptr")' in queries
for NamedUserGroup table because Django always does a join against
base UserGroup table.
2025-09-23 12:15:53 -07:00
Sahil Batra
764f4aa2e0 groups: Use realm_for_sharding for limiting NamedUserGroup queries.
For get and filter queries of NamedUserGroup, realm_for_sharding
field is used instead of realm field, as directly using
realm_for_sharding field on NamedUserGroup makes the query faster
than using realm present on the base UserGroup table.
2025-09-23 12:15:53 -07:00
Sahil Batra
d4d7a8fe2a streams: Make sure that unused anonymous groups are not created.
Anonymous groups were being created for stream permission
settings when calling the subscriptions endpoint without
any streams data or when calling it only for subscribing
users to streams and not for creating any new streams.

This commit makes sure that no such unused anonymous groups
are created.
2025-09-23 12:15:51 -07:00
Sahil Batra
1e21f7b224 register: Rename stream_creator_or_nobody to channel_creator.
This commit renames `stream_creator_or_nobody` value for
`default_group_name` field in `server_supported_permission_settings`
object to `channel_creator`.
2025-09-23 12:12:23 -07:00
Prakhar Pratyush
d78f338916 event_queue: Add compatibility code for push_device_registered_user_ids.
In #35965 we added `push_device_registered_user_ids` to message &
update_message event.

Zulip servers with such events in their event queues when upgraded
to the new version set push_device_registered_user_ids to empty list,
which is incorrect - it leads to no push notification sent.

This commit adds compatibility code to handle such events. The newly
introduced `push_device_registered` check is used only for events
with `push_device_registered_user_id` present in them.
2025-09-19 09:19:36 -07:00
Alex Vandiver
3064939ff7 email_mirror_server: chown the logfile before dropping privileges.
This prevents the logger from failing due to an old, root-owned,
logfile already existing.
2025-09-18 09:04:08 -07:00
Alex Vandiver
b39188f9cf email_mirror_server: Drop privileges before attaching file handler. 2025-09-17 13:23:48 -07:00
Shubham Padia
468a7a82dd help: Do not treat Help Center as a proper noun.
Fixes https://chat.zulip.org/#narrow/channel/101-design/topic/help.20center.20capitalization
2025-09-17 13:20:25 -07:00