Commit Graph

20457 Commits

Author SHA1 Message Date
Alex Vandiver
b39188f9cf email_mirror_server: Drop privileges before attaching file handler. 2025-09-17 13:23:48 -07:00
Shubham Padia
468a7a82dd help: Do not treat Help Center as a proper noun.
Fixes https://chat.zulip.org/#narrow/channel/101-design/topic/help.20center.20capitalization
2025-09-17 13:20:25 -07:00
Prakhar Pratyush
421637ce31 push_notifications: Skip enqueuing notif for users without push devices.
Previously, we enqueued events to "missedmessage_mobile_notifications"
even for users who had no registered push devices.
'handle_push_notification' later used to perform the check & skip
if there were no registered devices.

This commit avoids putting such events into the
"missedmessage_mobile_notifications" queue at all. By doing so,
we reduce unnecessary churn.
2025-09-17 12:32:49 -07:00
Lauryn Menard
0af56bb80b integrations: Update GitLab doc to direct to create webhook.
GitLab has both integrations and webhooks, and our documentation
should direct users to the webhooks set up for integrating with
Zulip.

Adds a link to GitLab's create a webhook documentation as well.
2025-09-17 09:39:48 -07:00
Sai
eaa2ece4e1 slack: Fix bugs during import of thread messages.
Consider the following messages JSON (taken from real data, redacted):
```json
[
    {
        "subtype": "bot_message",
        "text": "",
        "attachments": [
            {
                "fallback": "Open Slack to cast your vote in this Simple Poll",
                "title": "Should we do a thing?",
                "id": 1,
                "color": "6ecadc",
                "fields": [
                    {
                        "title": "",
                        "value": "1️⃣ Yes 👍\n\n"",
                        "short": false
                    },
                    {
                        "title": "",
                        "value": "2️⃣ No 👎\n\n",
                        "short": false
                    },
                    {
                        "title": "",
                        "value": "3️⃣ Abstain :spock-hand:\n\n",
                        "short": false
                    }
                ],
                "mrkdwn_in": [
                    "fields"
                ]
            },
            {
                "callback_id": "12345678-1234-1234-1234-123456789abc",
                "fallback": "Open Slack to cast your vote in this Simple Poll",
                "id": 2,
                "color": "6ecadc",
                "actions": [
                    {
                        "id": "1",
                        "name": "vote",
                        "text": "1️⃣",
                        "type": "button",
                        "value": "1",
                        "style": ""
                    },
                    {
                        "id": "2",
                        "name": "vote",
                        "text": "2️⃣",
                        "type": "button",
                        "value": "2",
                        "style": ""
                    },
                    {
                        "id": "3",
                        "name": "vote",
                        "text": "3️⃣",
                        "type": "button",
                        "value": "3",
                        "style": ""
                    },
                    {
                        "id": "4",
                        "name": "delete-v2",
                        "text": "Delete Poll",
                        "type": "button",
                        "value": "",
                        "style": "danger",
                        "confirm": {
                            "text": "Are you sure you want to delete the Poll?",
                            "title": "Delete Poll?",
                            "ok_text": "Yes",
                            "dismiss_text": "No"
                        }
                    }
                ]
            },
            {
                "callback_id": "12345678-1234-1234-1234-123456789abc",
                "fallback": "Open Slack to cast your vote in this Simple Poll",
                "footer": "Simple Poll        <https:\/\/simplepoll.rocks\/dashboard\/redacted\/settings\/|Edit Settings>",
                "id": 3,
                "footer_icon": "https:\/\/simplepoll.rocks\/static\/main\/favicon.png",
                "color": "6ecadc"
            }
        ],
        "type": "message",
        "ts": "1234567890.123456",
        "bot_id": "B1ABCDEF1",
        "thread_ts": "1234567890.123456",
        "reply_count": 1,
        "reply_users_count": 1,
        "latest_reply": "1234567890.765432",
        "reply_users": [
            "U1ABC1234"
        ],
        "replies": [
            {
                "user": "U1ABC1234",
                "ts": "1234567890.765432"
            }
        ],
        "is_locked": false,
        "subscribed": false
    },
    {
        "user": "U1ABC1234",
        "type": "message",
        "ts": "1234567890.765432",
        "text": "Maybe do qux instead",
        "team": "T1AB23456",
        "user_team": "T1AB23456",
        "source_team": "T1AB23456",
        "user_profile": {
            "avatar_hash": "a123456789ab",
            "image_72": "https:\/\/avatars.slack-edge.com\/2017-01-01\/123456789abc_def123456789abcdef12_72.jpg",
            "first_name": "Alice",
            "real_name": "Alice Smith",
            "display_name": "a.smith",
            "team": "T1AB23456",
            "name": "a.smith",
            "is_restricted": false,
            "is_ultra_restricted": false
        },
        "thread_ts": "1234567890.123456",
        "blocks": [
            {
                "type": "rich_text",
                "block_id": "EoBdt",
                "elements": [
                    {
                        "type": "rich_text_section",
                        "elements": [
                            {
                                "type": "text",
                                "text": "Maybe do qux instead"
                            }
                        ]
                    }
                ]
            }
        ]
    }
]
```

533f177175/zerver/data_import/slack.py (L922-L924)
fails for the first message, because it lacks a 'user' key. It should fall back to the bot_id.

533f177175/zerver/data_import/slack.py (L925-L926)
fails for the second message, because it lacks a 'parent_user_id' key.
However, the thread root will have been processed earlier, so
memoization of thread parents fixes this issue. Because the original
message may not be in the same file, the memoization needs to be global
rather than in `channel_message_to_zerver_message`.
2025-09-16 11:35:53 -07:00
Anders Kaseorg
1fdb81b6f8 streams: Remove is_in_zephyr_realm field from database.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-16 11:23:11 -07:00
Anders Kaseorg
8a35871217 events: Remove realm_is_zephyr_mirror_realm from API.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-16 11:23:10 -07:00
Anders Kaseorg
40a022dcc3 zephyr: Remove Zephyr mirroring support.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-16 11:18:18 -07:00
bedo
bc1ed82620 rate_limiter: Block IPv6 by /64 instead of per-single-IP.
Fixes #21544.

The network prefix (e.g. 64) identifies the network portion
which determines which bucket should that IPv6 belong to.

Implemented as a generic approach, so we have the option
to use different prefix sizes (leading to different buckets)
in the future.

Add comprehensive assertions for which IPv6s
are expected to belong to the same bucket and which are not.
2025-09-12 13:06:29 -07:00
Alex Vandiver
8c37e9c7a6 update_subscriber_counts: Fix calling without any flags. 2025-09-12 12:59:46 -07:00
Alex Vandiver
f915498e48 update_subscriber_counts: Add tool to re-sync subscriber counts. 2025-09-10 12:07:42 -07:00
Alex Vandiver
169daa31a0 settings: Derive RUNNING_INSIDE_TORNADO based on sys.argv.
This allows it to be used at application startup time.
2025-09-10 10:21:16 -07:00
Alex Vandiver
db89874bb0 cache: Write cache_key_prefixes without any compression.
This ensures that the new choice of zstd compression does not break
the previous, still-running, install during upgrades when the list of
cache_key_prefixes is updated.
2025-09-09 19:28:52 -07:00
Lauryn Menard
1030f41884 test-subs: Make new channel name consistent in notification bot test.
Updates final check in test_notification_bot_dm_on_subscription to
use the same pattern for the new channel name, and adds a comment
to clarify what that final check is for since it doesn't look at
the Notification Bot DM or channel messages for that case.
2025-09-08 16:52:37 -07:00
Lauryn Menard
14581de0de test-subs: Fix comment in test_notification_bot_dm_on_subscription. 2025-09-08 16:52:37 -07:00
Alex Vandiver
839135d007 users: Allow 1-character names.
Real people have 1-character names.  As seen by the previous attempt
to enforce length, in b7779667fc, real
people have names with all sorts of lengths.  All that we should care
about is that they're non-empty.
2025-09-08 14:30:59 -07:00
Evy Kassirer
31505e90f7 subscription_info: Improve subscriber fetch preformance.
Discussed here:
https://chat.zulip.org/#narrow/channel/141-kandra-ops/topic/partial.20subscribers.20query.20does.20users.20sequential.20scan/near/2255421
2025-09-08 11:47:59 -07:00
Alex Vandiver
74b07740b0 openapi: Make example UUID be a UUID. 2025-09-08 11:47:03 -07:00
Prakhar Pratyush
271aba001c api_docs: Fix 500 error on /api/register-remote-push-device.
`settings.ZULIP_SERVICES_URL` is used to construct curl
example for `/api/register-remote-push-device`.

On zulip cloud `settings.ZULIP_SERVICES_URL=None`, it resulted
in 500 error on visiting that page.

This commit fixes the bug by using "https://push.zulipchat.com"
as the default value if `settings.ZULIP_SERVICES_URL=None`.

The doc already mentions that the endpoint is meant to be used
by self-hosted servers, so the default value makes sense if
ZULIP_SERVICES_URL=None.
2025-09-05 11:01:05 -07:00
Alex Vandiver
257cfad667 outgoing_webhook: Lump chunked-encoding errors with connection errors. 2025-09-05 10:35:01 -07:00
Anders Kaseorg
1a05b2c10f tabbed-instructions: Delete activate_correct_tab.
We no longer have any OS-specific tabs; we can just activate the first
one on the server side.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-04 09:37:34 -07:00
Anders Kaseorg
35ce90d4f8 help_relative_links: Delete.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-04 09:37:34 -07:00
Prakhar Pratyush
e64be0c48b api_docs: Deprecate 'remove-fcm-token' & 'remove-apns-token'.
Endpoints marked deprecated:
* `DELETE /users/me/apns_device_token`
* `DELETE /users/me/android_gcm_reg_id`

These endpoints were for non-E2EE push notification case.

Fixes #35213.
2025-09-04 09:20:02 -07:00
Shubham Padia
012115a2bc documentation: Make sure article is secure to use.
The `article` variable being passed to `get_path` can also be a relative
path, which has it's security implications. By using `secure_filename`,
we mitigate that risk. We don't need to check if `/` exists in article
anymore since `secure_filename` will do so on it's own.
2025-09-03 09:28:15 -07:00
Shubham Padia
a766c092fc help: Restore broken link checks for help center documentation.
We add a step to build help center and then test the broken links as we
used to before removing the test temporarily.

This commit focuses on just adding back the broken link checks for the
help center. We skip the fragment check since that is in-built in
starlight and starlight tests account for that already. For the image
check we can add it back in a followup issue.
2025-09-03 09:28:15 -07:00
Shubham Padia
d333ddb961 help: Remove current help center.
This commit removes the current help center markdown files and any logic
that was used to host those files at help/.

We also remove a bunch of tests, we should the equivalent of those tests
for the new help center. Issues to track: #35649, #35647. These issues
track adding back tests for redirects and broken links.

We had a symlink from templates/zerver/integrations/email.md pointing to
help/using-zulip-via-email.md. We can no longer have that symlink since
the latter has been converted to an MDX file. We have deleted the
symlink and put a markdown file in it's place. Both the files have
comments to edit the other in case of changes.

This commit also makes changes in astro config, astro component paths
and other places to move the starlight help center docs base path from
/starlight_help to /help.

The change to rename /starlight_help/ to /help/ in MDX files is done in
the next commit. If we squash these commits, this line should be
removed.

`./tools/build-help-center` no longer does the conversion step.

We also remove some dead code related to the old help center in
documentation.py.
2025-09-03 09:28:15 -07:00
Shubham Padia
8f16f1139e help: Move URL redirects from Django to Astro.
Fixes #35649.

We do not serve the astro build via Django and thus we have to move the
help center redirects. We get a warning by pagefind on all these
redirects that they will be ignored in the search results as they don't
have an HTML body. We can ignore this warning till the cutover and
create a followup issue to solve that warning if this commit goes
through main.
2025-09-03 09:28:15 -07:00
Mateusz Mandera
6ab30fcced import: Set Stream.subscriber_count for imported channels.
The import code wasn't setting subscriber_count at all, resulting in a
value of 0 when dealing with imports from 3rd party apps.
We run this unconditionally, also for imports from Zulip, since this
ensures we won't inherit incorrect values, if the data we're import has
them - e.g. due to some bugs that affected the server the data came from.
2025-08-29 11:58:17 -07:00
Karl Stolley
41e8d79e00 api_docs: Update get-file-temporary-url endpoint and add to sidebar.
Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2025-08-29 10:21:42 -07:00
Sahil Batra
6e17b05156 openapi: Mark server_supported_permission_settings field as stable.
We have mostly decided on the structure of the group permission
setting object in server_supported_permission_settings field of
register response, so this commit removes the comment mentioning
it as unstable and adds the corresponding "Changes" entry.
2025-08-29 10:20:28 -07:00
Lauryn Menard
f5587e9ffe api-docs: Revise update_message changes notes for feature level 114. 2025-08-29 10:00:44 -07:00
Prakhar Pratyush
e8c9f4a811 api_docs: Deprecate the non-E2EE register push device endpoints.
Endpoints marked deprecated:
* /users/me/apns_device_token
* /users/me/android_gcm_reg_id

The older endpoints were for non-E2EE push notification case.

Fixes part of #35213.
2025-08-28 16:45:51 -07:00
Prakhar Pratyush
b157b14d76 api_docs: Mark /mobile_push/test_notification as deprecated.
The older endpoint was for non-E2EE push notification case.

Fixes part of #35213.
2025-08-28 16:45:51 -07:00
Prakhar Pratyush
f52533795b api_docs: Document /remotes/push/e2ee/register endpoint.
This commit documents the `/remotes/push/e2ee/register` endpoint.

We use auth_email="ZULIP_ORG_ID" and auth_api_key="ZULIP_ORG_KEY"
instead of "BOT_EMAIL_ADDRESS" and "BOT_API_KEY".
2025-08-28 11:46:09 -07:00
Prakhar Pratyush
062a736097 api_docs: Prep work to document zilencer endpoints.
Until now we were not documenting bouncer's REST API endpoints.

We plan to document the newly introduced "remotes/push/e2ee/register"
and "remotes/push/e2ee/notify" endpoints.

This commit does the prep work for documenting bouncer endpoints:
* mark the older endpoints related to sending non-E2EE push
  notifications as "intentionally_undocumented" - we'll remove
  them in future.
* the remaining endpoints are marked pending-to-document with
  helpful comments.
2025-08-28 11:46:09 -07:00
Prakhar Pratyush
9785cb4a09 render_curl_example: Remove unused 'auth_email' & 'auth_api_key'.
The 'function' variable in 'APICodeExamplesPreprocessor.generate_text'
matches the pattern of API_ENDPOINT_NAME = r"/[a-z_\-/-{}]+:[a-z]+"

It is always of the form 'endpoint_path:endpoint_method'.

There's no possibility of multiple ':'.

This commit removes the unused code block in `render_curl_example`,
which was trying to get 'auth_email' & 'auth_api_key' from 'function'.
2025-08-28 11:46:09 -07:00
Prakhar Pratyush
c604ecb902 push_notifications: Log roundtrip time since worker decided to send.
Adds a log for the end-to-end latency from when the worker decided
to send push notifications & received a success response from bouncer.

Fixes part of #35368.
2025-08-28 11:08:02 -07:00
Lauryn Menard
e5514d575d create-channel: Add test for invalid user ID in subscribers param. 2025-08-28 10:13:02 -07:00
Lauryn Menard
8ee59e9277 api-docs: Revise create channel endpoint documentation. 2025-08-28 10:13:02 -07:00
Lauryn Menard
910ca975ab api-docs: Improve rendering of create channel python example. 2025-08-28 10:13:02 -07:00
Lauryn Menard
224499b58f api-docs: Revise folder_id parameter for adding/updating channels.
Revises the description of the `folder_id` parameters in the
api/create-channel, api/subscribe and api/update-stream endpoints.
2025-08-28 10:13:02 -07:00
Mohammad Reza Kianifar
e2d1c233ba get_recipient_ids: Include user ID for self-messages using DM group.
When using direct message groups as the preferred method for 1:1 or
self-messages, the get_recipient_ids function should include the
user ID for self-messages. Without this, the UI cannot detect who
was the recipient of the message.
2025-08-28 10:04:37 -07:00
Vector73
92fbe47a3b events: Add support for processing modern presence event in web client. 2025-08-27 17:16:31 -07:00
Kislay Verma
396f4d004a api_docs: Clarify description for send_new_subscription_messages.
The same subscription request can contain both existing channels
and newly created channels, so we clarify here that direct message
notifications are only sent when users are subscribed to existing
channels.

Also, updates this text to use "direct message" instead of "DM".

Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2025-08-27 17:06:31 -07:00
Kislay Verma
9b7d6335c1 test_subs: Add send_new_subscription_messages parameter false test case.
For the sake of completion, we add a test case ensuring that
the response does not contain `new_subscription_messages_sent` if
the parameter `send_new_subscription_messages` is `false`,
as mentioned in the API documentation.
2025-08-27 17:06:31 -07:00
Kislay Verma
18d9b686b7 add_subscriptions: Only exclude DMs and not new channel notifications.
When the number of new subscriptions crosses a threshold, we wish to
avoid sending DMs to the user as implemented in #31206. But there was
a bug causing announcement channel and new channel notification
messages to also not be sent.

This commit fixes that bug by ensuring that we only exclude direct
message notications.
2025-08-27 17:06:31 -07:00
Kislay Verma
06d2ecb482 test_subs: Add test for all notification bot messages.
In #31206, we changed `add_subscriptions_backend` to exclude the
notification bot DMs when subscribing users to a channel when the
number of subscriptions exceeds MAX_BULK_NEW_SUBSCRIPTION_MESSAGES.

This caused a bug where new channel announcement and creation
notification messages were not being sent. Here we add a test that
confirmes the current behavior, which will be corrected in a
subsequent commit.

Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2025-08-27 17:06:31 -07:00
Alex Vandiver
575dd10f99 reminders: Stop interpolating user-provided values in format string.
We must not intermix Markdown strings which are ready for the message,
with format strings which we intend to interpolate on.
2025-08-27 12:07:21 -07:00
Lauryn Menard
699c0c6200 create-channel: Remove send_new_subscription_messages parameter.
Removes send_new_subscription_messages parameter from
`POST channel/create` endpoint as Notification Bot DMs
are never sent when users are subscribed as part of
creating a new channel.

Not considered a documentable API change, since the previous parameter
never had any effect, so this is effectively just a documentation and
error-handling bug fix, not an API change relevant to implementors.
2025-08-27 11:12:05 -07:00
Lauryn Menard
dda2c6e285 create-channel: Clarify flow of new channel and subscription notices.
Updates comments in send_user_subscribed_and_new_channel_notifications
so that it is clearer what notification each "if" block of the function
is generating.

Also, moves variables that are only used in the user subscribed DMs to
that "if" block.
2025-08-27 11:12:05 -07:00