Commit Graph

1996 Commits

Author SHA1 Message Date
Sahil Batra
9fd9c5f1d2 actions: Pass realm_playgrounds as argument to notify_realm_playgrounds.
This commit modifies the notify_realm_playgrounds function to accept
realm_playgrounds as argument from the caller instead of computing it
in the function to avoid duplicate queries since the realm playgrounds
list will be required in its caller functions as well in further commits.
2022-03-15 15:58:36 -07:00
Alex Vandiver
62ba8e455d CVE-2022-24751: Clear sessions outside of the transaction.
Clearing the sessions inside the transaction makes Zulip vulnerable to
a narrow window where the deleted session has not yet been committed,
but has been removed from the memcached cache.  During this window, a
request with the session-id which has just been deleted can
successfully re-fill the memcached cache, as the in-database delete is
not yet committed, and thus not yet visible.  After the delete
transaction commits, the cache will be left with a cached session,
which allows further site access until it expires (after
SESSION_COOKIE_AGE seconds), is ejected from the cache due to memory
pressure, or the server is upgraded.

Move the session deletion outside of the transaction.

Because the testsuite runs inside of a transaction, it is impossible
to test this is CI; the testsuite uses the non-caching
`django.contrib.sessions.backends.db` backend, regardless.  The test
added in this commit thus does not fail before this commit; it is
merely a base expression that the session should be deleted somehow,
and does not exercise the assert added in the previous commit.
2022-03-15 13:52:15 -07:00
Sahil Batra
f0606b34ad user_groups: Add cron job for adding users to full members system group.
This commit adds a cron job which runs every hour to add the users to
full members system group if user is promoted to a full member.

This should ensure that full member status is available no more than
an hour after configuration suggests it should be.
2022-03-14 18:53:47 -07:00
Sahil Batra
bbf4c25553 user_groups: Update user groups on changing waiting_period_threshold.
The members of "Full members" user group are updated with change in
waiting_period_threshold setting.
2022-03-14 18:53:47 -07:00
Sahil Batra
9345c344d7 user_groups: Add members to the System groups created.
This commit adds users to the appropriate system user group
based on their role. We also change the user groups when
changing role of the user.

We also add migration to add existing users to the appropriate
user groups.

This commit adds update_users_in_full_members_system_group which
is currently used to update the full members group on changing
role of a user. This function will be modified in next commit such
that it can be used to update full members group on changing
waiting_period_threshold setting of realm.
2022-03-14 18:53:46 -07:00
Sahil Batra
c33ba4ed6e realm: Create role-based system user groups on creating realm.
We create system user groups for following roles - owners,
admins, moderators, members and guests. Full members user
group will be handled separately.
2022-03-14 18:52:47 -07:00
Sahil Batra
f31bb6754b user_groups: Pass list of user ids to remove_members_from_user_group.
We pass list of user ids instead of user profile objects to
remove_members_from_user_group. We still need to call user_id_to_users
in the views function instead of directly passing the ids to
remove_members_from_user_group to make sure we check whether all
ids are valid or not.
2022-03-14 18:52:47 -07:00
Sahil Batra
303fca2ce7 user_groups: Pass list of user ids to bulk_add_members_to_user_group.
We pass list of user ids instead of user profile objects to
bulk_add_members_to_user_group. We still need to call user_id_to_users
in the views function instead of directly passing the ids to
bulk_add_members_to_user_group to make sure we check whether all
ids are valid or not.
2022-03-14 18:52:47 -07:00
Sahil Batra
b48b98f1d9 user_groups: Use atomic transaction for adding/removing members.
We execute code for bulk adding and removing members from a user
group inside a transaction.
2022-03-14 18:52:47 -07:00
Steve Howell
f9ddd80122 subscriptions: Improve types for subscription info.
The newly extracted module here has significant changes
to all the functions, but they achieve the same thing
as before.
2022-03-14 18:01:39 -07:00
Steve Howell
8f99894302 streams: Extract stream_color library.
This is a pure code move.
2022-03-14 18:01:36 -07:00
Steve Howell
200a89cef0 streams: Extract stream_traffic library.
This is a pure code move.
2022-03-14 18:01:36 -07:00
Kartik Srivastava
eefaa9120f user_topic: Rename topic_mutes.py to user_topics.py. 2022-03-11 14:26:55 -08:00
Sahil Batra
57f01e0727 actions: Use transaction.atomic for do_remove_realm_domain. 2022-03-10 17:48:02 -08:00
Sahil Batra
5999dcd316 actions: Use transaction.atomic for do_change_realm_domain. 2022-03-10 17:48:02 -08:00
Sahil Batra
9b9931df7f actions: Use transaction.atomic for do_add_realm_domain. 2022-03-10 17:48:02 -08:00
Sahil Batra
07352271b9 realm: Create RealmAuditLog entry for removing a realm domain.
Fixes a part of #21268.
2022-03-10 17:48:02 -08:00
Sahil Batra
ab5567e8c5 realm: Create RealmAuditLog entry while changing an allowed domain.
This commit also adds 'acting_user' argument for do_change_realm_domain
function.

Fixes a part of #21268.
2022-03-10 17:48:02 -08:00
Sahil Batra
5ef8da40a9 realm: Create RealmAuditLog entry while adding a new allowed domain.
This commit also adds 'acting_user' argument for do_add_realm_domain
function.

Fixes a part of #21268.
2022-03-10 17:48:02 -08:00
Steve Howell
822c232e37 message flags: Extract create_historical_user_messages.
A user can subscribe to a stream and sometimes (depending
on stream permissions) see messages from the stream
that were sent before they subscribed, and that user
won't have a UserMessage row for that message.

In order to do things like star a message, we need
to create UserMessage records on the fly.

In the past we wisely constrained this logic to the
specific use cases. But I think we can generalize
the logic now.  For example, we are now building a
feature to mark messages as unread, and it motivates
the same need to auto-create UserMessage rows.

So now we handle this in a more generalized fashion.
2022-03-10 13:21:02 -08:00
Steve Howell
a90d9ef536 unread: Remove unused client parameter. 2022-03-10 10:04:51 -05:00
Tim Abbott
d693a6717c i18n: Remove quote syntax from stream description notification.
Translators found it confusing, since it's not at all obvious that the
word "quote" should not be translated.

I'm not altogether happy with the code formatting for this.

While we're changing this, also standardize on the "```` quote" style
of quote blocks to ensure code/quote blocks in stream descriptions are
unlikely to conflict with this syntax.
2022-03-09 15:22:57 -08:00
Dinesh
9f8d60cd5a actions: Mention that stream can also be changed with do_update_message(). 2022-03-07 12:44:36 -08:00
Tim Abbott
a3a9adbfa9 edit_history: Stop writing prev_subject to the database.
Now that we have code to support reading prev_topic, this is no longer
necessary.

We'll want to deploy this change to production before running the
migration to remove prev_subject from edit history entries, so that
prev_subject can be fully purged from the database.
2022-03-04 10:25:48 -08:00
Tim Abbott
a410eee2e1 edit_history: Store additional fields on edit history events.
We modify the message_edit_history marshalling code so that this
commit does not change the API, since we haven't backfilled the data
yet.

FormattedEditHistoryEvent, introduced in the previous commit, doesn't
directly inherit fields from EditHistoryEvent, so no changes are
required there.
2022-03-04 10:25:48 -08:00
Tim Abbott
f1e5ed91a1 types: Add EditHistoryEvent and APIEditHistoryEvent types.
These types will help make iteration on this code easier.

Note that `user_id` can be null due to the fact that
edit history entries before March 2017 did not log
the user that made the edit, which was years after
supporting topic edits (discovered in test deployment
of migration on chat.zulip.org).

Co-authored-by: Lauryn Menard <lauryn.menard@gmail.com>
2022-03-04 10:25:48 -08:00
Sahil Batra
392b17da5f invite: Add backend support for "Never expires" option.
The database value for expiry_date is None for the invite
that will never expire and the clients send -1 as value
in the API similar to the message retention setting.

Also, when passing invite_expire_in_days as an argument
in various functions, invite_expire_in_days is passed as
-1 for "Never expires" option since invite_expire_in_days
is an optional argument in some functions and thus we cannot
pass "None" value.
2022-02-24 16:32:19 -08:00
Anders Kaseorg
21cd1c10b3 docs: Add missing space in “time zone”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-24 14:05:12 -08:00
Shantanu
4371705a4c streams: Update formatting of retention change notification.
This changes the notifications to be more readable like our other
similar stream events notifications.

Fixes #20969.
2022-02-18 11:11:38 -08:00
Alex Vandiver
a40b3e1118 realm_emoji: Stop swallowing all exceptions from upload_emoji_image.
Putting all of the logic in a `finally` block is equivalent to a bare
`except` block, which silently consumes all exceptions.

Move only the most-necessary parts into the except; this lets
`BadImageError` exceptions from `zerver/lib/upload.py` to escape,
allowing better the generic "Image file upload failed" to be replaced
with a more specific message.

It also allows unexpected exceptions, as the previous commit resolved,
to escape and 500.  This lets them be detected and resolved, rather
than give users a silently bad experience.
2022-02-17 12:19:47 -08:00
Anders Kaseorg
961633fcec timezone: List only canonical timezone identifiers.
For aliases that will no longer be listed, see the third column of

grep '^L ' zulip-py3-venv/lib/python3.*/site-packages/pytz/zoneinfo/tzdata.zi

Time zones previously set to an alias will be canonicalized on demand.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-11 17:38:57 -08:00
Tim Abbott
12ed0c3104 message edit: Improve logic for detecting whole topic moves.
The new logic better matches reasonable user expectations, that if you
move all the messages, that's a whole-topic move, regardless of which
propagation mode you selected.
2022-02-09 15:13:00 -08:00
Shubh Gupta
79069b5dec message edit: Indicate how many messages were moved in notificactions.
When moving only part of a topic, it's useful to display that
information to users in these notifications so that it's clear what's
happening.

The most important consequence is actually just increasing confidence
that when you see that the whole topic was moved, that's accurate.

Substantially modified by tabbott.

Fixes #20575.
2022-02-09 15:08:56 -08:00
Anders Kaseorg
b0ce4f1bce docs: Fix many spelling mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-07 18:51:06 -08:00
Sahil Batra
9f0d498994 message: Check mandatory_topics setting in backend as well.
Previously, we only checked mandatory_topics setting before
sending message in frontend and there was no restriction in
backend. This commit adds the check in backend also making
sure messages without topic cannot be sent through API as
well if mandatory_topics setting is set to True.
2022-02-02 10:11:35 -08:00
Tim Abbott
438af33b19 create_user_messages: Rename mark_as_read_user_ids.
This should use a name that follows the naming pattern of adjacent
fields.
2022-02-01 12:01:03 -08:00
Tim Abbott
1a46df40c3 resolve topic: Limit unread flag on automated notifications.
Previously, users found it annoying that the automated "Resolve topic"
notifications triggered an unread for everyone in the stream; this
discouraged some users from using the feature on older threads for
fear of being annoying. We change this to a better default, of only
users who participated in the topic (via either messages or reactions)
being eligible for the new message being unread.

We will likely want to create global and stream-level notifications
settings to control this behavior as a follow-up -- some users, like
me, might prefer the simpler "Always unread" behavior in some streams.

Note that the automated notifications that a topic was resolved will
still result in the topic being moved to the top of the left sidebar.
This would be somewhat difficult to change, since the left sidebar
algorithm just looks at the highest message ID in the topic.

Fixes #19709.

Tests added by Aman Agrawal (amanagr@zulip.com).
2022-02-01 11:35:50 -08:00
Anders Kaseorg
90e202cd38 docs: Consistently hyphenate “web-public”.
In English, compound adjectives should essentially always be
hyphenated.  This makes them easier to parse, especially for users who
might not recognize that the words “web public” go together as a
phrase.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-28 17:45:45 -08:00
Mateusz Mandera
364139feec actions: Add appropriate argument to transaction.atomic decorators.
These were missing an argument to the decorator - and we generally want
to use either savepoint=False or durable=True.
2022-01-28 13:03:40 -08:00
Mateusz Mandera
120de1db19 do_deactivate_stream: Use transaction.atomic. 2022-01-28 13:03:39 -08:00
Mateusz Mandera
e025e85b77 do_remove_default_stream: Use transaction.atomic.
on_commit on the event will make it easier to use transaction.atomic in
do_deactivate_stream, which calls this function.
2022-01-28 13:03:39 -08:00
Mateusz Mandera
0dd97eeaab do_set_realm_property: Use transaction.atomic. 2022-01-28 13:03:39 -08:00
Sahil Batra
5e506a833f actions: Use transaction.atomic for do_change_avatar_fields. 2022-01-27 10:33:55 -08:00
Sahil Batra
a2df7470d5 actions: Use transaction.atomic for do_change_realm_plan_type. 2022-01-27 10:33:55 -08:00
Sahil Batra
f5ea13eea8 actions: Use transaction.atomic for do_change_user_delivery_email. 2022-01-27 10:33:55 -08:00
Lauryn Menard
aaa627229e api: Update update_message event required fields.
Makes `edit_timestamp` and `user_id` required fields for all
`update_message` events.

Adds `rendering_only` as another required field to signal if
events are only updating the rendered content of the message,
which is currently the case for adding inline url previews.

Updates `test_event.py` so that `do_update_message` and
`do_update_embedded_data` refer to the same testing schema
for `update_message` events, and therefore reflect the same
required fields for the `update_message` event.

The OpenAPI definition for `update_message` events is also
updated to reflect the required field and descriptions of
various properties are updated for the addition of the
`rendering_only` property.
2022-01-26 13:11:26 -08:00
Eeshan Garg
aa8b3f9729 streams: Add RealmAuditLog entries for permission changes. 2022-01-21 13:59:35 -08:00
Eeshan Garg
0d99809fd3 streams: Add notifications for permission policy changes.
The change to curl_param_value_generators.py warrants a brief
explanation. Stream permission changes now generate a notification
message. Our curl example test for removing a reaction comes after
the two tests for updating the stream permission changes, thus the
hardcoded message ID in that test needs to be incremented by 2 to
account for the two notification messages that now come before it.

This is a part of #20289.
2022-01-21 13:59:34 -08:00
Eeshan Garg
fab1b7f5d5 actions: Refactor functions for stream permission changes.
do_make_stream_web_public and do_change_stream_invite_only seem
to contain very similar logic that could just live inside the
do_change_stream_permission function that handles all permission
changes in one place.
2022-01-21 13:59:34 -08:00
Eeshan Garg
f0ee065292 streams: Use bulleted format for description change notifications.
We want the format for our description change notifications to be
consistent with the format of our stream posting policy change
notifications.
2022-01-21 13:59:34 -08:00