Commit Graph

20261 Commits

Author SHA1 Message Date
Alex Vandiver
0c6b3373be upload: Ensure that we always have a content-type.
Old uploads are stored without a content-type, and guess_type may not
provide a type for them.
2025-07-29 10:01:40 -07:00
Alya Abbott
1bc7696c24 updates: Add Zulip update (level=20). 2025-07-28 23:52:56 -07:00
Alya Abbott
641a436b44 preferences: Update inactive channels preference setting for new UI.
"Hide" is a better verb with the channel folders UI.

Also automatically hide inactive channels at 20+ susbscribed channels,
as we have more content in the left sidebar now.
2025-07-28 23:52:55 -07:00
Sahil Batra
8ae9cca3c9 channel_folders: Check max length restrictions while updating.
This commit updates code to make sure we check maximum allowed
length for name and description for channel folders when updating
them.
2025-07-28 23:12:58 -07:00
Sahil Batra
6186df4f72 channel_folders: Update MAX_NAME_LENGTH to 60. 2025-07-28 23:12:58 -07:00
Sahil Batra
db00ceaeb2 register: Pass channel folder max name and description length.
This commit adds "max_channel_folder_name" and
"max_channel_folder_description" fields to response.

Also updated webapp to use these fields instead of hardcoding
the values.
2025-07-28 23:12:58 -07:00
Sahil Batra
945e5553e4 test_channel_folders: Add test for long name and description.
This commit add tests for name and description being too long
when creating channel folders.
2025-07-28 23:12:58 -07:00
Anders Kaseorg
b404ca1a02 makemessages: Remove vestigial custom JavaScript extraction.
Commit d8a36d0702 (#18075) should have
removed this.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-07-28 17:44:23 -07:00
Prakhar Pratyush
d972bb1ca9 push_notifications: Redact content for older clients if E2EE required.
This commit replaces the `PUSH_NOTIFICATION_REDACT_CONTENT` server
setting with `require_e2ee_push_notifications` realm setting.

If `require_e2ee_push_notifications` set to True:
* Older clients: Content redacted
* Updated clients: Encrypted content

If `require_e2ee_push_notifications` set to False:
* Older clients: Content NOT redacted
* Updated clients: Encrypted content

Note: Older clients refers to clients that don't support E2EE.

Fixes part of #35370.
2025-07-28 17:06:59 -07:00
Prakhar Pratyush
fc6cd9a966 settings: Add require_e2ee_push_notifications realm setting.
This commit adds a realm setting:
"Require end-to-end encryption for push notification content".

This commit just involves backend & UI changes to add the setting,
the impact of this setting is in the next commit.

Fixes part of #35370.
2025-07-28 17:06:59 -07:00
Rohan Gudimetla
cb01bdf0c4 topics: Follow topics that are merged into even after merge.
Previously, when there were two topics -- A and B, and a user
follows topic A, then when messages of topic B are merged into
topic A then topic A loses its follow status.

This is fixed by updating the algorithm for getting the visibility
policy after merge to be followed in the above case.

Fixes #35226
2025-07-28 12:35:30 -07:00
Rohan Gudimetla
8330e7c283 topics: Refactor method that decides topic visibility after merge.
This commit refactors the method
which determines the visibility of two topics merged.

This is a preparatory commit for #35226.
2025-07-28 12:35:30 -07:00
Aman Agrawal
f4dd3aa066 auth: Pass None as default_subdomain to verify the host.
We were not verifying correctly if the host is in the same
subdomain as we never returned None for get_subdomain_from_hostname.
2025-07-28 10:59:38 -07:00
Prakhar Pratyush
5616be4afa test_e2ee_push_notification: Improve tests to cover more cases.
This commit adds a test and updates a few existing tests to
cover more cases related to send push notifications.

* We no longer mock the 'send_push_notifications_legacy' function
  while testing 'send_push_notifications' codepath and vice-versa.
  This makes the tests more realistic as both functions gets called
  in 'handle_push_notification'.

  This covers the case when only old clients (which don't support
  E2EE) exists for a user. Or only updated clients (which supports
  E2EE) exist.

* Adds a test 'test_both_old_and_new_client_coexists' for the case
  when a user has both type of clients at an instant i.e. they have
  updated a few devices only.
2025-07-28 10:39:48 -07:00
Prakhar Pratyush
d91a6be3f1 push_notification: Return early if no device registered - w/o bouncer.
In 'send_push_notifications_legacy', when a user has no registered
devices:
* `uses_notification_bouncer()`=True: we log "Skipping..." and return.

* `uses_notification_bouncer()`=False: we make some function calls,
  which effectively does nothing.

It's better to have a common check (and log) early in the codepath.
This commit makes that change.
2025-07-28 10:39:48 -07:00
Lauryn Menard
7cbc321704 help: Remove "Reading direct messages" article.
In #27186, we updated the help center documentation on
reading conversations and removed the article from the
left sidebar.

Removes remaining links to the article in "Printing
messages" and adds a redirect to go to the main article
on "Direct messages".
2025-07-28 09:50:14 -07:00
Prakhar Pratyush
36a2093585 push_notification: Remove dev guard from send E2EE notif codepath.
This commit removes the DEVELOPMENT guard from the new
'send_push_notifications' function in 'handle_push_notification'
and 'handle_remove_push_notification'.
2025-07-27 20:47:16 -07:00
Alex Vandiver
74675fae94 settings: Use get_secret for altcha_hmac.
When building tarballs, we do not have development secrets.
2025-07-25 22:29:51 -07:00
Alex Vandiver
8af90294d8 altcha: More error-proofing around settings validation. 2025-07-25 22:29:51 -07:00
Prakhar Pratyush
6ab6df96c8 push_notification: Send a list of push requests.
Earlier, we were passing a map `device_id_to_encrypted_data`
and http headers as separate fields to bouncer.

The downside of that approach is it restricts the bouncer to
process only one type of notice i.e. either notification for
a new message or removal of sent notification, because it
used to receive a fixed priority and push_type for all the
entries in the map.

Also, using map restricts the bouncer to receive only one
request per device_id. Server can't send multiple notices
to a device in a single call to bouncer.

Currently, the server isn't modelled in a way to make a
single call to the bouncer with:
* Both send-notification & remove-notification request data.
* Multiple send-notification request data to the same device.

This commit replaces the old protocol of sending data with
a list of objects where each object has the required data
for bouncer to send it to FCM or APNs.

This makes things a lot flexible and opens possibility for
server to batch requests in a different way if we'd like to.
2025-07-25 12:47:55 -07:00
Prakhar Pratyush
3d3f4d5e62 push_notification: Remove 'get_apns_payload_data_to_encrypt'.
'get_apns_payload_data_to_encrypt' was added in commit
0ae34ddb65, in parallel
to 'get_message_payload_apns' - to use in E2EE codepath.

The intent was to avoid nesting in the payload returned
by 'get_message_payload_apns' function, just like FCM
payload returned by 'get_message_payload_gcm'.

Turned out, the nesting is helpful in APNs case for various
reasons. So, this commit reverts that function and we'll
continue to use the older structure returned by the function
'get_message_payload_apns'.
2025-07-25 12:47:55 -07:00
Lauryn Menard
07ff9ec4f9 help: Add mobile tab for "Rename a topic" instructions. 2025-07-25 10:09:56 -07:00
Alex Vandiver
25fbb05fea upload: Attempt to guess character set text/* which are served inline.
This is only `text/plain`, currently.  In such cases where the
client-provided content-type also does not specify a `charset`, we use
`chardet` to make a guess, and store that guess to provide later when
serving the content.  The detection is done in a streaming fashion,
and thus should not require re-downloading the full content.
2025-07-24 22:40:12 -07:00
Alex Vandiver
ae001dfa96 upload: Allow uploads to set charset.
Previously, the `user_file.content_type` only contained the MIME type
of the uploaded file; no other parameters were included, meaning that
a file a client specified as `content-type: text/plain; charset=big5`
would be stored with an `Attachment.content_type` of `text/plain`.

Re-construct the full content-type header from `content_type_extra`,
which includes those parameters.

We do not include a test because Django does not support specifying
such parameters in the upload path.
2025-07-24 22:40:12 -07:00
Alex Vandiver
edb5943d8b upload: Use content_type instead of mimetype for consistency. 2025-07-24 22:40:12 -07:00
Alex Vandiver
183da665ac upload: Rename .source to .vips_source. 2025-07-24 22:40:12 -07:00
Anders Kaseorg
fa23059f17 event_schema: Fix Enum validation in check_realm_update and friends.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-07-24 22:35:05 -07:00
Anders Kaseorg
ba53ec5712 event_types: Add missing default EventPushDevice.error_code = None.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-07-24 22:34:20 -07:00
Anders Kaseorg
94ed87129f event_types: Remove defunct PersonIsBillingAdmin.
Commit c049259d07 (#33739) should have
removed this completely, rather than merely removing its
is_billing_admin member.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-07-24 22:34:06 -07:00
Anders Kaseorg
87e866d23e event_types: Unmark typing_edit_message recipient fields as optional.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-07-24 22:33:49 -07:00
Anders Kaseorg
4603472fe3 event_types: Combine typing_edit_message variants.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-07-24 22:33:49 -07:00
Anders Kaseorg
30f93fa563 event_types: Remove nonexistent realm_billing event.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-07-24 16:50:39 -07:00
Mohammad Reza Kianifar
015f674520 notification: Use existing email format for missed 1:1 DM via DM group.
To maintain API compatibility, we render the email notification for
missed 1:1 direct messages using DirectMessageGroup with the same
format as messages sent to a Personal recipient.
2025-07-24 16:41:52 -07:00
Prakhar Pratyush
2dbc17b453 push_notification: Revoke push notifications using encrypted payload.
This commit updates 'handle_remove_push_notification' function
to use the new 'send_push_notifications' function.

It leads to encrypt the removal payload before sending it to bouncer.

Fixes part of #35368.
2025-07-24 16:13:00 -07:00
Prakhar Pratyush
84db492dfb push_notification: Reuse 'send_push_notification_legacy'.
This commit refactors 'handle_remove_push_notification'
to reuse 'send_push_notification_legacy' function.
2025-07-24 16:12:59 -07:00
Prakhar Pratyush
29ea8a07c2 push_notification: Add support to configure priority and push_type.
This commit adds support to let server configure:
* fcm_priority
* apns_priority
* apns_push_type

while sending E2EE push notifications.

The values of these fields will vary depending on whether the
send request is to send push notification for a message or
revoke an already sent notification.

Since, the bouncer receives encrypted data so it can't inspect
the payload to determine whether it is a removal request or not,
hence can't configure priority on its own.

The server needs to specify explicitly.

We're not simply sending a single 'is_removal' flag because
allowing the server to configure them separately will help in
future to support other types of notifications with a different
combination of priority and push_type, like whose aim is to notify
user about information other than a new message or removal request.

Fixes part of #35368.
2025-07-24 11:56:42 -07:00
Prakhar Pratyush
945f27f099 push_notification: Remove redundant @activate_push_notification_service.
In 'test_e2ee_push_notifications', the `SendPushNotificationTest` class
is already decorated with `@activate_push_notification_service()`,
so individual test methods do not need to repeat it.
2025-07-24 11:56:42 -07:00
Pratik Chanda
e7629ef07d message_flags: Fix message flags not updating for wildcard mentions.
Earlier, editing a message with wildcard mention and removing the
wildcard mention didn't properly remove the corresponding flag for
it in the message object. It was only updating flags when mentions
were present in new message but not the other way around.

This commit fixes this behaviour by removing flags if the new message
removed mention from it.
2025-07-24 11:52:20 -07:00
Lauryn Menard
95e6af8c64 api-docs: Revise feature level 404 and 392 changes entries.
Updates the changelog entries for feature levels 404 and 392, and
revises the descriptive text for the topics_policy channel and
realm level settings.

The feature level 404 changes were originally added in commit
a77fc6aa79.

The feature level 392 changes were originally added in commit
deaa43c7e6.
2025-07-23 11:49:55 -07:00
Prakhar Pratyush
1c7dd83dbe register_push_device: Add an error code to push-not-configured error.
This commit adds an error code to make it easier for clients
to identify when push device registration is rejected due to
server not being configured to use push notification service.

Fixes part of 35368.
2025-07-23 11:11:46 -07:00
Alex Vandiver
5319a2f33e narrow: Skip topic mutes for DM narrows. 2025-07-22 20:57:36 -07:00
Alex Vandiver
83acf70d0f models: Add a partial index on unread DMs.
This helps the find_first_unread_anchor call for DMs with a user.
2025-07-22 20:57:36 -07:00
Prakhar Pratyush
7e1afa0e8a push_notification: Send end-to-end encrypted push notifications.
This commit adds support to send encrypted push notifications
to devices registered to receive encrypted notifications.

URL: `POST /api/v1/remotes/push/e2ee/notify`
payload: `realm_uuid` and `device_id_to_encrypted_data`

The POST request needs to be authenticated with the server’s
API key.

Note: For Zulip Cloud, a background fact about the push bouncer is
that it runs on the same server and database as the main application;
it’s not a separate service. So, as an optimization we directly call
'send_e2ee_push_notifications' function and skip the HTTP request.
2025-07-22 17:08:55 -07:00
Sahil Batra
eb57fe10a2 streams: Handle empty topic only streams being used for announcements.
This commit updates code to send messages to "general chat" topic
if streams used for announcements for "New user signups",
"New created streams", "Moderation requests" and "Zulip updates"
have topics policy set to allow only "general chat" messages.
2025-07-22 16:59:47 -07:00
Mateusz Mandera
639972b753 get_recursive_group_members: Use DISTINCT in the query.
Having repetitions in the result can cause subtle bugs in callers that
don't expect it.
2025-07-22 16:21:05 -07:00
Alex Vandiver
2f4dd72076 push_notifications: Adjust APNs tokens to be case-insensitive in the database.
APNs apparently treats its tokens case-insensitively; FCM does not.
Adjust the `unique_together` to instead be separate partial
constraints, keyed on the `kind` of the PushDeviceToken.
2025-07-22 14:30:13 -07:00
Evy Kassirer
935a8b9278 events: Consider partial_subscribers key in apply_event.
Fixing bug reported here:
https://chat.zulip.org/#narrow/channel/343-kandra-errors/topic/KeyError.3A.20'subscribers'/near/2226586
2025-07-22 12:49:28 -07:00
Alex Vandiver
dbe1f778cf grafana: Be more forgiving of missing alert keys. 2025-07-22 12:20:04 -07:00
Mateusz Mandera
25a75b9ed7 scim_filter: Remove unnecessary join.
This join is completely redundant. We don't need to join the tables to
add filtering by realm id in the db queries for SCIM user lookup.
2025-07-22 12:02:06 -07:00
Mateusz Mandera
3052d0cca7 scim: Make "id" a string in our responses.
In SCIM, this is always a string, even if numerical. While SCIM clients
seem to handle our prior behavior, this was a bug nonetheless.
2025-07-22 12:02:06 -07:00