Commit Graph

19329 Commits

Author SHA1 Message Date
Steve Howell
3f0658803c event tests: Improve debugging output for events. 2025-01-04 08:53:44 -08:00
Niloth P
09f47d5a0a webhooks/github: Suppress "comment edited" events when body is same.
GitHub generates spurious edited events on hiding/unhiding a comment,
and on clicking "edit"/"update" without any changes, when the
comment "body" contains certain specific characters.

Instead of sending a notification message, we suppress those edge cases
by comparing the "body" content of the comment and changes in the event
payload.

Fixes #31042, #26136.
2025-01-03 15:18:39 -08:00
Aman Agrawal
0a6579640f message_send: Fix stream still inactive after sending message.
We updated the stream property but forgot to inform the client.
2025-01-03 14:14:39 -08:00
sanchi-t
0188e5c2ac test_subs: Fix test_unarchive_stream test.
Replaced list with set to ensure order-independent
comparison of IDs during assertions.
2025-01-03 08:59:06 -08:00
Steve Howell
9e14e69112 event_schema: Use correct mypy type for EventModel. 2025-01-03 08:45:02 -08:00
PieterCK
639b291f30 import: Sort realms migrations status before checking.
This commit sorts the list of migrations to ensure the same
sets of migrations don't somehow ordered differently. This
has been reported to happen when importing Zulip Cloud to
self-host (zulip-cloud-current).

The order of migrations listed in `migration_status.json`
don't actually matter, migrations specify which other
migrations they depend on.

Fixes #32826.
2025-01-02 18:43:16 -08:00
Alex Vandiver
d2464ff52b markdown: Catch OverflowErrors in global times.
This also matches the except in the block above.
2025-01-02 17:31:36 -08:00
Anders Kaseorg
7ce49a709e check-schemas: Remove unnecessary special case for saved_snippets.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-01-02 17:28:17 -08:00
Prakhar Pratyush
891a35570b migrations: Update active status of Channels based on recent activity.
This commit adds a database migration to set is_recently_active=False
for channels based on the last time a message was sent.

The migration follows the same logic used in the
`update_channel_recently_active_status` management command.

This change is particularly useful for self-hosted servers, as it
avoids the need to wait a week for the
`update_channel_recently_active_status` management command to run
via cron.
2025-01-02 16:37:34 -08:00
Steve Howell
5fd24bc9a2 event checkers: Replace data_types.py with pydantic.
This commit completely eliminates the home-grown
data_types.py system for checking types.

Now, in our unit tests and schema checkers, we use
pydantic types to check the shape of the event
objects that are either in our JSON fixtures or
that get captured by test_events.py.

The code in event_types.py was completely
code-generated from the legacy version of
event_schema.py, so there shouldn't be any
manual errors in how I ported the types over.
(The code generation was non-trivial, though,
as I had to account for changes from
snake_case to CamelCase and deal with some
other anomalies.)
2025-01-02 12:23:35 -08:00
Niloth P
7261898d89 integrations: Remove Desk.com integration.
Desk.com has been discontinued by Salesforce in March 2020.
2024-12-31 16:31:48 -08:00
Anders Kaseorg
9ba1763855 event_schema: Add local_message_id to message_event.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-12-30 09:42:26 -08:00
Anders Kaseorg
d512290b15 event_schema: Use event_dict_type for user_topic_event.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-12-30 09:42:26 -08:00
Anders Kaseorg
58822372d5 typos: Fix typos caught by typos and mwic.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-12-24 19:15:36 -08:00
Anders Kaseorg
63aaafb94a send_email: Parse emails in a way mypy 1.14 understands.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-12-21 21:06:53 -08:00
Anders Kaseorg
0bb0d3598b scim: Add None check for email_new_value.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-12-21 21:06:53 -08:00
Anders Kaseorg
19b8cde27f ruff: Fix PLC0206 Extracting value from dictionary without calling .items().
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-12-21 21:06:53 -08:00
Anders Kaseorg
5bad79dd5b openapi: Depluralize Subscription schema name.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-12-20 16:15:45 -08:00
Anders Kaseorg
c1abeb050f openapi: Depluralize Attachment schema name.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-12-20 16:15:45 -08:00
opmkumar
5b0c55fda3 realm: Add option to schedule data deletion while deactivating.
Introduce a feature to schedule realm data deletion time during realm
deactivation. This includes a server-level setting to configure the
minimum and maximum allowed deletion days.

Co-authored-by: Ujjawal Modi <umodi2003@gmail.com>
Co-authored-by: Lauryn Menard <lauryn@zulip.com>

Fixes #24677.
2024-12-18 23:06:12 -08:00
PieterCK
c6fd0ba424 integrations: Clean up duplicate functions.
This commit makes the Slack incoming webhook use the original
`convert_slack_formatting` and `convert_slack_workspace_mentions` from
`slack_message_conversion.py`. Previously those were not refactored out
into two smaller functions yet.

Fixes part of #30827.
2024-12-18 16:14:20 -08:00
PieterCK
a194d62093 integration-doc: Fix typo/wrong in Slack token scope list.
This commit deletes the duplicate `users:read` token scope from the list
of token scopes.
2024-12-18 16:14:20 -08:00
PieterCK
f2599bf33d integrations: Do check_token_access only initially.
Previously the `check_token_access` is called for every request we get
from Slack webhook, this may introduce significant latency.

This commit moves `check_token_access` to the same condition for when we
need to handle Slack challenge handshake so that we only do API token
check once per URL registered.

Additionally, we now check for the specific scopes that we need to run
the Slack webhook integration (SLACK_INTEGRATION_TOKEN_SCOPES).

Fixes part of #30827.
2024-12-18 16:14:19 -08:00
PieterCK
a746be807f slack_import: Make check_token_access more flexible.
Previously, the `check_token_access` function had a hardcoded
`required_parameters` variable because it was only used in the Slack
data importer. This commit refactors `required_parameters` into a
function parameter, enabling the function to check a Slack token’s scope
for other purposes, such as Slack webhook integration.

Additionally, this commit changes the Slack API call in
`check_token_access` from `teams.info` to `api.test`. The endpoint is
better suited for this purpose since we're only checking a token’s scope
using the response header here.
2024-12-18 16:11:31 -08:00
PieterCK
dd306e8d80 channels: Prevent moderation_request_channel to be public.
This commit ensures that the channel for `moderation_request_channel`
cannot be set to public when updating it's settings.

Fixes part of #20047.
2024-12-18 11:36:22 -08:00
PieterCK
943fbb76fe settings: Add realm-level setting moderation_request_channel.
This commit introduces a new non-operational
`moderation_request_channel` field to the server/API. This setting will
support a feature allowing users to flag or report abusive content
(harassment, spam, etc.).

Fixes part of #20047.
2024-12-18 11:36:19 -08:00
Anders Kaseorg
3a33060891 message_cache: Handle Django/Tornado skew for sender_recipient_id.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-12-18 08:55:52 -08:00
Harsh Bansal
4d3cc2c5a5 portico: Improve layout of /go page.
Made the input field of the page consistent with
the other similar pages. Here are I have done the following things:
1. Changed text 'Enter your organization's Zulip URL:'->'Organization URL'.
2. Left aligned the label and button to make it consistent.
3. Moved 'Don't know your organization URL? Find your organization.' to be
just below the URL field.
4. Changed the placeholder 'your-organization-url' -> 'your-organization'

Fixes #32198
2024-12-17 23:24:00 -08:00
Alya Abbott
e05f66b311 portico: Add Rush Stack case study. 2024-12-17 11:44:32 -08:00
Sahil Batra
a9bd614df3 default_streams: Remove get_default_streams_for_realm_as_dicts.
Since get_default_streams_for_realm_as_dicts function was only
used in tests, this commit removes it and updates the test to
use the function which returns Stream objects instead of dicts.

This commit also removes Stream.to_dict function which is no
longer used.
2024-12-17 09:40:10 -08:00
Sahil Batra
1182e7e350 streams: Pass only stream IDs for default stream groups data.
Clients do not use default stream groups feature now but they
should only require IDs when they use them and not the full
stream data since that is already available in streams and
subscriptions data.
2024-12-17 09:40:10 -08:00
Sahil Batra
1c933dd231 streams: Include only IDs in default streams data.
Clients do not need complete stream objects in the default
streams data since they already have that via streams and
subscriptions data.
2024-12-17 09:40:10 -08:00
Anders Kaseorg
a9b9ed6c5b run-dev: Support extra MIME types from zerver.lib.mime_types.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-12-17 09:34:29 -08:00
PieterCK
f988412394 slack_data_import: Support converting integration bot users.
Currently, we're unable to convert messages from Slack's integration
bots because this message subtype doesn't come from a Slack "user", that
is they don't have a Slack user profile.

This is a preparatory change to support converting Slack's integration
bot messages. This commit artificially creates Slack user data from the
integration bot's "profile" so that we can create a corresponding Zulip
user for them.

Part of #31311.
2024-12-16 13:09:57 -08:00
PieterCK
10946caa3d slack_data_import: Update how Slack user avatars are processed.
Previously, the Slack export converter can only process Slack's avatar
URL from Slack's "ca.slack-edge.com" server, which looks like this:

https://ca.slack-edge.com/T0CDRA6HM3P-U06NABE26M9-1173e04f818e-512

This commit adds support for converting any public downloadable image
URLs.

This is done to support importing Slack's integration bots and their
messages, which typically have PNG type file url:

https://avatars.slack-edge.com/2024-05-01/7057208497908_a4351f6deb91094eac4c_72.png
2024-12-16 13:09:57 -08:00
Mateusz Mandera
94cc413016 docs: Renaming AzureAD to Entra ID, part two.
This is a follow-up to #32693, which missed a bunch of places due to
varying ways of typing Azure AD. (Azure AD, Azure Active Directory etc.)

In the diff there's an apparent inconsistency with renaming, in some
places renaming to just "Microsoft Entra ID" and in others
"Microsoft Entra ID (AzureAD)".
The idea is to add (AzureAD) in ordinary documentation, as it can help
clarity since many people still think of this as AzureAD, while sticking
with brevity and cleanliness of just using the official name in
marketing focused material.
2024-12-15 08:22:30 -08:00
Tim Abbott
6e58982d4b api_docs: Improve upload file API description. 2024-12-13 17:01:43 -08:00
Alya Abbott
8c712838aa updates: Add update message (level=11). 2024-12-13 15:35:44 -08:00
Kislay Verma
00df8a593d user_settings: Add field web_suggest_update_timezone.
This field tracks whether the user should be shown an
alert offering to update their profile time zone to the
time zone of the browser in case they differ.

The field is added to RealmUserDefault and UserProfile
models with a default value of True.

Fixes part of #16957
2024-12-13 12:58:07 -08:00
Vector73
143ca5065e api: Remove user object from reactions objects in the API.
Removes deprecated `user` object from reactions objects returned by
the API as it is redundant because of the presence of `user_id` field in
the API and is not used by any clients now.
2024-12-13 09:58:25 -08:00
Mateusz Mandera
81c2c0b328 docs: Finish renaming AzureAD to Entra ID.
Microsoft has been renaming AzureAD to Entra ID. Though both names still
seem to function, even if unofficially, so this mostly renames to `Entra
ID (AzureAD)` to reference both for clarity.
2024-12-12 17:48:54 -08:00
Niloth P
0d14ae1acf integration-docs: Remove the second screenshot from CircleCI doc.
The file had already been deleted, leaving behind its entry in the doc.
2024-12-12 15:04:02 -08:00
PieterCK
82e0468071 slack-integration: Update Slack integration documentation.
The doc now shows instructions to setup the integration using
Slack's Events API instead of the legacy Outgoing webhook
service.

Co-authored-by: Alya Abbott <alya@zulip.com>
Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2024-12-11 13:06:52 -08:00
PieterCK
42a22e6aaa slack-integration: Block requests from Slack retries.
A Slack fail condition occurs when we don't respond with HTTP 200 within
3 seconds after Slack calls our endpoint. If this happens, Slack will
retry sending the same payload. This is often triggered because we need
to perform callbacks when converting messages. To avoid sending the same
message multiple times, we block subsequent retry calls from Slack.

This commit returns early HTTP 200 response as soon as we get any retry
calls from Slack.

Part of #30465.
2024-12-11 13:06:52 -08:00
PieterCK
c54d90e5c7 slack-integration: Add callback function to Slack API.
Payloads from Slack's Events API don't include human-readbale
information for a Slack username and for Slack channels.

This commit makes the Slack integration callback to the relevant
Slack API endpoints for that information.

Fixes #30074.
2024-12-11 13:06:52 -08:00
PieterCK
1ebbe2fd99 slack-integration: Add test fixtures for extra Slack payloads.
This commit adds 4 new payload fixtures to the Slack
integration for messages we don't process properly yet:

- Messages with code blocks.
- Messages with overlapping text formatting (e.g, bold
  and italic).
- Messages with formatted mentions.
- Messages with quote blocks.

The follow-up plan to add support for these kind of
Slack messages is tracked in #30827.

Part of #30465.
2024-12-11 13:06:52 -08:00
PieterCK
f29312ce03 slack-integration: Update Slack integration to handle Events API.
This updates the Slack webhook integration to handle the Slack Events
API[1], while maintaining backwards compatibility with Slack's legacy
Outgoing Webhook service.

The Events API introduces the "challenge" handshake[2] to verify and
add a new webhook URL for them to call. This commit adds a handler for
the challenge handshake.

Additionally, this commit reformats incoming payloads using the Slack
text reformatter from `slack_message_conversion.py`. There is some
duplicative code here because of the difference in Slack export data
and Slack's webhook payload.

Part of #30465

[1]: https://api.slack.com/apis/events-api#using-events-api
[2]: https://api.slack.com/apis/events-api#handshake
2024-12-11 13:06:52 -08:00
Alex Vandiver
06d04b4291 migrations: Change migration 0639's dependency for backporting to 9.x.
This moves it "earlier" in the migrations graph, onto the last
migration in 9.x, and adds a merge migration to fold it back into
main's current migrations tip.
2024-12-11 12:21:25 -08:00
Sahil Batra
143bda8a4b test_user_groups: Use "123456" as invalid group ID.
Since we have many group based settings and many of them
use anonymous groups as default, there might be a group
with ID "1111" as group IDs keep increasing across a test
suite run and it is no longer an invalid group ID.

This commit updates the tests to use a large enough
value like "123456" as invalid group ID.
2024-12-11 12:15:02 -08:00
Sahil Batra
1153d088b9 tests: Fix flaky behavior of test to update group permissions.
The test for updating group permissions when passing old value
used "1111" as an invalid group ID passed as new value as well
as old value. But now since we have many group based settings
and many of them use anonymous groups as default, there might
be a group with ID "1111" as group IDs keep increasing across
a test suite run and it is no longer an invalid group ID.

So, updated the test to -
- Use a large enough value like "123456" as new value for
testing invalid group ID case.
- Remove the test passing invalid value as "old" because we
do not check validity for old values and just compare it
with the actual current value of the setting.
2024-12-11 12:15:02 -08:00