Sending emails synchronously is useful because it reports
configuration errors -- but it also means that occasional failures can
result in ugly 500's, since those don't retry.
Add a setting which forces all email to go through the `emil_senders`
queue, so it can be retried as needed.
This is a prep commit which moves the `PushNotificationTest` class
from `test_push_notifications.py` to `test_classes.py` as we plan to
import it and reuse it in other files.
This commit also renames `PushNotificationTest` to
`PushNotificationTestCase` to align with our naming convention
in `test_classes.py` to name classes, which act as a parent class
with helper methods and no actual test within itself, with a
"TestCase" suffix.
Similar to group based setting values, we expect
the client to send the previous content alongwith
the edited content to the edit message endpoint.
We reject the request incase the previous content
doesn't match the current message content, which
could happen in case two users simultaneously edit
a message - which will be implemented in #33051.
This class will be inherited by all errors related to
some sort of mismatch from the expected value.
This is a prep commit for #33051, as a part of which we
add a new exception class for message content mismatch.
This is a prep commit for #30212. It will allow us to compare the key
and not translated policy name.
Co-authored-by: Tanmay Kumar <tnmdotkr@gmail.com>
Previously, `hot_conversations` was incorrectly passed as a string to the
`enough_traffic` function. This commit fixes the issue by passing the length
of `hot_conversations` as a parameter instead.
This commit updates code to not prefetch can_access_all_users_group
and can_access_all_users_group__named_user_group fields using
select_related. We can just use get_realm_system_groups_name_dict
function to check if setting is set to "Everyone" group when
needed and can avoid unnecessarily fetching groups for every user
query.
This commit refactors user_has_permission_for_group_setting
to accept setting group ID instead of UserGroup object.
We only need ID in checking the permission and this helps in
further commit to avoid prefetching can_access_all_users_group
setting.
For `check_user_has_permission_by_role`, we were using
`user.is_moderator` by default to check whether the user had those
priviliges. But that specific function returns false if the user is an
admin or an owner. So we check `is_realm_admin` too in that case.
"first_message_id" field for subscription objects needs
to be updated when archiving a stream as we send a
notification message, but first_message_id will only
change if the stream did not have any messages previously.
This commit updates the code to update first_message_id
only when required.
When checking DM permissions, instead of using list of
users, we now use set of users to check if any user is
in direct_message_permission_group because there can be
case when sender can also be one of the recipients.
This commit updates code to not pre-fetch DM permission
group settings using select_related and instead just
fetch the required data from DB when checking permission.
This will increase one query but will help in pre-fetching
the settings for all users and for all type of messages.
Fixes part of #33677.
This commit updates is_user_in_group and is_any_user_in_group
to accept group ID as parameter instead of UserGroup object.
This is a prep commit for updating code to not prefetch
direct message permissions group.
This commit removes the `/try-zulip` landing page.
The URLs are replaced with `chat.zulip.org/?show_try_zulip_modal`,
which leads to display a modal for spectators.
Fixes#34181.
This commit adds a modal which will be displayed when
a spectator visits `/?show_try_zulip_modal`.
When a user visits `/?show_try_zulip_modal` and is a spectator,
we set a new `show_try_zulip_modal` field in `page_params` to
`true` (in all other cases, it's `false`).
Based on the `show_try_zulip_modal` page param, the web client
shows the modal.
Fixes part of #34181.
We're doing this so that the client can keep track of which channels
it might need to request full subscriber data from, and which already
have full subscriber data.
This is private information, as by inspecting the DirectMessageGroup
objects and their associated Subscription objects, you could determine
which users conversed with each other in a DM group.
This did *not* leak any actual message - only the fact that at least one
of the users in the group sent a group DM.
The prior significantly restricted what data gets exported from
non-consented users. The last thing we're missing is to fix the logic
to work correctly for public exports.
Prior commits focused on addressing exports with consent. This commit
adapts it to work with public exports.:
- Do not turn user accounts into mirror dummies in the public export - or
after export->import you'll end up with a realm with no functional
accounts; as every user is non-consented and the original logic added in
the prior commits will turn them into mirror dummies.
- Some of the custom fetch/process functions were changed without
considering public exports - now they work correctly, by setting
consenting_user_ids to an empty set.
The Subscription Config is constructed in a bit of a strange way, that's
not compatible with defining a custom_fetch function.
Instead we have to extend the system to support passing a custom
function for processing just the final list of rows right before it's
returned for writing to export files.
As explained in the comment, if we turn a non-consented deactivated user
into a mirror dummy, this will violate the rule that a deactivated user
cannot restore their account by themselves after an export->import
cycle.
As explained in the comment added to the function, in terms of privacy
concerns, it is fine to export all data for these accounts. And it is
important to do - so that exporting an organization which was originally
imported e.g. from Slack doesn't result in excessively limited data for
accounts that were mirror dummies and never "activated" themselves.
Now that we severely limited the way that non-consenting users get
exported, we need to start to consider bots as consenting when
appropriate - otherwise the exported bot accounts will be unusable after
importing.
An administrator shouldn't be able to bypass a user's setting to hide
their email address from everyone, including admins.
Therefore, we should overwrite the delivery_email for such users during
export - unless the user consented to have their private data exported.
The notable consequence of this is that such user accounts will become
completely inaccessible after importing this data to a new server, due
to not having a functional email address on record.
These accounts will only be possible to reclaim via a manual
intervention to change the email address on the `UserProfile` by server
administrators.
This allows us to get rid of the call to `get_consented_user_ids` in
`fetch_usermessages`. Now it's only called at the beginning of the
export, eliminating the redundant db query and also resolving the
potential for data consistency issues, if some users change their
consent setting after the export starts.
Now the full export process operates with a single snapshot of these
consenting user ids.
These ids need to be plumbed through via a file rather than normal arg
passing, because this is a separate management command, run in
subprocesses during the export.
These users didn't consent to having their private data exported.
Therefore, correct handling of these users should involve scrubbing
their settings to just match the realm defaults.
Instead of making repeated calls to get_consented_user_ids, we can just
fetch it (mostly) once and put it in
`context["exportable_user_ids"]`. This is essentially what the
(unused until now) exportable_user_ids logic was added for after all.
The added, intended, effect of this is that non-consenting users will
now get exported as mirror dummy accounts, due to the handling of
non-exportable users in `custom_fetch_user_profile`.
The remaining additional call to `get_consented_user_ids` is in
`fetch_usermessages`. This one is tricky as this function gets called
in subprocesses via
`zerver/management/commands/export_usermessage_batch.py` management
command invoked by the export process.
It requires passing the `exportable_user_ids` in some other way. This
can be dealt with in upcoming commits.
We shouldn't export the entire Client table - it includes Clients for
all the realms on the server, completely unrelated to the realm we're
exporting. Since these contain parts of the UserAgents used by the
users, we should treat these as private data and only export the Clients
that the specific data we're exporting "knows" about.
This reduces confusion amount users when they download the
intel version and it works super slow. Downloading the arm64
version on an intel mac would just not work.
Users who use intel macs have a habit now (atleast me) to look
for `Intel` version of software when downloading an app. So, made
`Intel` bold to help that process.
To zulip/python-zulip-api, to keep them closer to their source code.
- Renamed the generate_zulip_bots_static_files to
generate_bots_integrations_static_files to accomodate the new function.
- Added a new function to
tools/setup/generate_bots_integrations_static_files to copy the
integration docs into static/generated/integrations.
- Updated integrations.py and computed_settings.py to use the new doc
paths.
- Deleted the affected integration docs.
- Updated the dependency URL.
The Jira plugin integration doc will be moved in the next commit, to
"static/generated/integrations/jira/doc.md", as the directory name of
the integration is "jira" in zulip/python-zulip-api.
- Jira integration doc (previously) - "zerver/webhooks/jira/doc.md"
- Jira plugin integration doc (next commit) -
"static/generated/integrations/jira/doc.md"
Both of these will use the same path "jira/doc.md" as their
integration.doc value, and the actual file is loaded based on the order
of template directories listed in computed_settings.py.
Hence, use a custom path for the Jira integration doc to avoid this
collision.
