For get and filter queries of NamedUserGroup, realm_for_sharding
field is used instead of realm field, as directly using
realm_for_sharding field on NamedUserGroup makes the query faster
than using realm present on the base UserGroup table.
Previously, we enqueued events to "missedmessage_mobile_notifications"
even for users who had no registered push devices.
'handle_push_notification' later used to perform the check & skip
if there were no registered devices.
This commit avoids putting such events into the
"missedmessage_mobile_notifications" queue at all. By doing so,
we reduce unnecessary churn.
This avoids a potential unnecessary message.recipient fetch required by
is_stream_message(). is_stream_message() methods precedes the addition
of the denormalized is_channel_message column and is now unnecessary.
In practice, we usually fetch Message objects with `.recipient` already,
so I don't expect any notable performance impact here - but it's still a
useful change to make.
This commit adds a test and updates a few existing tests to
cover more cases related to send push notifications.
* We no longer mock the 'send_push_notifications_legacy' function
while testing 'send_push_notifications' codepath and vice-versa.
This makes the tests more realistic as both functions gets called
in 'handle_push_notification'.
This covers the case when only old clients (which don't support
E2EE) exists for a user. Or only updated clients (which supports
E2EE) exist.
* Adds a test 'test_both_old_and_new_client_coexists' for the case
when a user has both type of clients at an instant i.e. they have
updated a few devices only.
This commit adds support to send encrypted push notifications
to devices registered to receive encrypted notifications.
URL: `POST /api/v1/remotes/push/e2ee/notify`
payload: `realm_uuid` and `device_id_to_encrypted_data`
The POST request needs to be authenticated with the server’s
API key.
Note: For Zulip Cloud, a background fact about the push bouncer is
that it runs on the same server and database as the main application;
it’s not a separate service. So, as an optimization we directly call
'send_e2ee_push_notifications' function and skip the HTTP request.
APNs apparently treats its tokens case-insensitively; FCM does not.
Adjust the `unique_together` to instead be separate partial
constraints, keyed on the `kind` of the PushDeviceToken.
APNs tokens are provided by the client in hex, and we store them in
hex. The existing code which attempts to "validate" them by parsing
them as base64 only works because base64 is a superset of hex.
Enforce that APNs tokens are hex, and remove all of the pieces of test
code which were incorrectly passing them in as base64 strings.
This is a preparatory refactor for migrating the internal structure of
Recipient objects for group DMs to use the DirectMessageGroup type,
not the legacy PERSONAL type. This step has the message-sending code
path check if a DirectMessageGroupe exists and prefer it if available.
It should have no effect in production other than doing a useless
database query for each outgoing DM, since we do not at present ever
create such DirectMessageGroup objects. (It will not add a marginal
database query once the migration is complete, just during this
transition).
This commit adds two functions in `zerver/lib/test_classes.py`. The
first function returns the `user_ids` of users for whom the message
is marked as unread. The second function returns the `user_ids` of
users for whom the message is marked read, based on the `message_id`.
This was broken, due the mechanism simply using our
is_guest/is_realm_admin/etc. role setters, but failing to adjust system
group memberships - resulting in corrupted database state.
We need to ensure that change_user_role is called for setting user role.
There are two relevant codepaths that run the sync based on
AUTH_LDAP_USER_FLAGS_BY_GROUP and thus need to get this right:
1. manage.py sync_ldap_user_data
2. Just-in-time user creation when a user without a Zulip account logs
in for the first using their ldap credentials. After
get_or_build_user returns, django-auth-ldap sees that the user
account has just been created, and proceeds to run ._populate_user().
Now that both user.save() and do_change_user_realm will be getting
called together, we need to ensure this always happens atomically.
This imposes the need to override _get_or_create_user to put it in a
transaction. The troublesome consequence is that this new
`atomic(savepoint=False)` causes the usual type of issue, where tests
testing error get their transaction rolled back and cannot continue
executing.
To get around that, we add a test helper
`artificial_transaction_savepoint` which allows these tests to wrap
their problematic blocks in an artificial transaction which provides a
savepoint, thus preventing the full test transaction rollback derailing
the rest of the test.
This is a prep commit which moves the `PushNotificationTest` class
from `test_push_notifications.py` to `test_classes.py` as we plan to
import it and reuse it in other files.
This commit also renames `PushNotificationTest` to
`PushNotificationTestCase` to align with our naming convention
in `test_classes.py` to name classes, which act as a parent class
with helper methods and no actual test within itself, with a
"TestCase" suffix.
This is valuable so that one is forced to explicitly make a decision
on what is correct when adding new callers. Past experience tells us that
not having to explicitly show the decision leads to people introducing
security bugs in PRs that the maintainer has to catch in review, and our
goal for access control code should be that security bugs are hard to write.
Fixes#33688.
We remove `invite_to_stream_policy` from the backend wherever applicable
except deleting the field. We have just ported the existing behaviour of
`invite_to_stream_policy` to `can_add_subscribers_group` except one
change. We have added an explicit exception for admins to have this
permission whether they are part of this group or not. The reason for
this is we are adding `stream.can_add_susbcribers_group` in the future
which will grant all admins permission to subscribe other users to a
channel given they have access to a channel. So it makes sense that we
add this exception to the realm level property also.
See https://chat.zulip.org/#narrow/channel/101-design/topic/Can.20subscribe.20other.20users.20on.20user.20profile/near/2039825
b4fedaa765 introduced
this helper, and I assume that the weird loop over
zip made sense at the time.
The assertEqual approach on the whole
set gives nice messages in modern Python.
Earlier, in 'handle_missedmessage_emails' codepath we were using
'queue_json_publish' which can lead to a situation where we enqueue
events but the transaction fails at a later stage.
Events should not be published until we know we're not rolling back.
Now a failure from assert_json_success and friends shows the full
server-side traceback from the JsonableError that caused the failure,
not just the test-side traceback for `AssertionError: 400 != 200 :
{message}`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
We were using admins group as a hardcoded value for the default of
`can_remove_subscribers_group`, now we use a function to get the value
of the default group.
We create savepoint at a couple of places in backend tests
to avoid rollback due to error raised within test's transaction.
This commit explicitly adds 'savepoint=True' at those places.
This commit renames the 'send_event' function to
'send_event_rollback_unsafe' to reflect the fact that it doesn't
wait for the db transaction (within which it gets called, if any)
to commit and sends event irrespective of commit or rollback.
In most of the cases we don't want to send event in the case of
rollbacks, so the caller should be aware that calling the function
directly is rollback unsafe.
Currently, we want to ask users if they would like to delete their
attachments after they have removed the attachments while editing. These
changes are preparatory changes on the backend to return a list of removed
attachments after the user has removed attachments while editing.
Fixes part of #25525.
Instead of the PUSH_NOTIFICATIONS_BOUNCER_URL and
SUBMIT_USAGE_STATISTICS settings, we want servers to configure
individual ZULIP_SERVICE_* settings, while maintaining backward
compatibility with the old settings. Thus, if all the new
ZULIP_SERVICE_* are at their default False value, but the legacy
settings are activated, they need to be translated in computed_settings
to the modern way.
