Earlier, we were passing a map `device_id_to_encrypted_data`
and http headers as separate fields to bouncer.
The downside of that approach is it restricts the bouncer to
process only one type of notice i.e. either notification for
a new message or removal of sent notification, because it
used to receive a fixed priority and push_type for all the
entries in the map.
Also, using map restricts the bouncer to receive only one
request per device_id. Server can't send multiple notices
to a device in a single call to bouncer.
Currently, the server isn't modelled in a way to make a
single call to the bouncer with:
* Both send-notification & remove-notification request data.
* Multiple send-notification request data to the same device.
This commit replaces the old protocol of sending data with
a list of objects where each object has the required data
for bouncer to send it to FCM or APNs.
This makes things a lot flexible and opens possibility for
server to batch requests in a different way if we'd like to.
'get_apns_payload_data_to_encrypt' was added in commit
0ae34ddb65, in parallel
to 'get_message_payload_apns' - to use in E2EE codepath.
The intent was to avoid nesting in the payload returned
by 'get_message_payload_apns' function, just like FCM
payload returned by 'get_message_payload_gcm'.
Turned out, the nesting is helpful in APNs case for various
reasons. So, this commit reverts that function and we'll
continue to use the older structure returned by the function
'get_message_payload_apns'.
This commit updates 'handle_remove_push_notification' function
to use the new 'send_push_notifications' function.
It leads to encrypt the removal payload before sending it to bouncer.
Fixes part of #35368.
This commit adds support to let server configure:
* fcm_priority
* apns_priority
* apns_push_type
while sending E2EE push notifications.
The values of these fields will vary depending on whether the
send request is to send push notification for a message or
revoke an already sent notification.
Since, the bouncer receives encrypted data so it can't inspect
the payload to determine whether it is a removal request or not,
hence can't configure priority on its own.
The server needs to specify explicitly.
We're not simply sending a single 'is_removal' flag because
allowing the server to configure them separately will help in
future to support other types of notifications with a different
combination of priority and push_type, like whose aim is to notify
user about information other than a new message or removal request.
Fixes part of #35368.
This commit adds support to send encrypted push notifications
to devices registered to receive encrypted notifications.
URL: `POST /api/v1/remotes/push/e2ee/notify`
payload: `realm_uuid` and `device_id_to_encrypted_data`
The POST request needs to be authenticated with the server’s
API key.
Note: For Zulip Cloud, a background fact about the push bouncer is
that it runs on the same server and database as the main application;
it’s not a separate service. So, as an optimization we directly call
'send_e2ee_push_notifications' function and skip the HTTP request.
APNs apparently treats its tokens case-insensitively; FCM does not.
Adjust the `unique_together` to instead be separate partial
constraints, keyed on the `kind` of the PushDeviceToken.
APNs tokens are provided by the client in hex, and we store them in
hex. The existing code which attempts to "validate" them by parsing
them as base64 only works because base64 is a superset of hex.
Enforce that APNs tokens are hex, and remove all of the pieces of test
code which were incorrectly passing them in as base64 strings.
This commit adds a `push_devices` dictionary to
`POST /register` response, keyed with push account ID,
where each entry describes the user's push device's
registration status and error code (if registration failed).
This commit adds a zilencer endpoint to let self-hosted
servers register push devices to whom mobile push notifications
will be sent.
POST "/api/v1/remotes/push/e2ee/register"
Payload: realm_uuid, push_account_id, encrypted_push_registration,
bouncer_public_key
The post request needs to be authenticated with the server’s API key.
Note: For Zulip Cloud, a background fact about the push bouncer is
that it runs on the same server and database as the main application;
it’s not a separate service.
So, as an optimization, we plan to directly call the
`do_register_remote_push_device` function and skip the HTTP request.
Fixeszulip/zulip-flutter#1617.
It turns out that an APNs token (which is a hex string) is equally
valid in lower or upper case. The old app would send the server
the lower-case form of the token, but the new app sends the
upper-case form.
Because we've been treating tokens case-sensitively, if the user
upgrades from the old app to the new, that results in the server
and bouncer each having two copies of the token (one lower-case and
one upper-case), and therefore sending that device two copies of
each notification: zulip/zulip-flutter#1617.
To fix that immediately, have the bouncer drop duplicate tokens
before sending the notifications to APNs.
Work is also in progress on fixing this in a better-structured way,
by having the database correctly treat tokens as the same when they
differ only in case.
To maintain API compatibility during migration to DirectMessageGroup
for 1:1 DMs, generate notification payloads for such messages in the
same format as those sent to a Personal recipient.
Fixes: part of issue #25713.
In aioapns 4.0, which we upgraded to in acd7353538, the `key`
parameter is treated as the ASCII-armored string contents of the key,
not the path to the file with that content.
Read the file ourselves, and pass the string it expects.
This is valuable so that one is forced to explicitly make a decision
on what is correct when adding new callers. Past experience tells us that
not having to explicitly show the decision leads to people introducing
security bugs in PRs that the maintainer has to catch in review, and our
goal for access control code should be that security bugs are hard to write.
Fixes#33688.
Now that we introduced an URL for serving permalinks redirecting to
docs in #33444, the docs_url mechanism is no longer needed, as we can
have a URL that's safe to hard-code in register_server.py.
The HostnameAlreadyInUseBouncerError.docs_url has been merged in main
briefly enough, that this should be safe to remove.
This commit adds support to display `Message.EMPTY_TOPIC_FALLBACK_NAME`
value (translated) in the push notifications for topics having the
actual value of empty string.
Fixes part of #32996.
This means that the URL is only hard-coded on the bouncer side. That's
useful, because now we'll be able to change the URL and only need a
bouncer deployment for users to get the new URL when they encounter
HostnameAlreadyInUseBouncerError. As opposed to self-hosted servers
being stuck with an outdated docs link hardcoded in their
register_server.py.
An even better way than the current json error message recommending the
--registration-transfer option is to return an appropriate error code
and have that get picked up by the register_server command.
The register_server command can then display a more comprehensive,
better formatted error message with proper whitespaces and a pointer to
the documentation.
Instead of the PUSH_NOTIFICATIONS_BOUNCER_URL and
SUBMIT_USAGE_STATISTICS settings, we want servers to configure
individual ZULIP_SERVICE_* settings, while maintaining backward
compatibility with the old settings. Thus, if all the new
ZULIP_SERVICE_* are at their default False value, but the legacy
settings are activated, they need to be translated in computed_settings
to the modern way.
This commit performs a sweep on the first batch of non API
files to rename "huddle" to "direct_message_group`.
It also renames variables and methods of type -
"huddle_message" to "group_direct_message".
This is a part of #28640
We believe this to already be obsolete and dead code and is about to be
removed with the migration to the FCM HTTP v1 API, where the concept
doesn't exist anymore.
This commit replaces the local 'is_same_server_message_link'
function used in 'get_mobile_push_content' with the
'is_same_server_message_link' lib function.
The lib function is the same logically but uses urllib instead
of regex for parsing and is backed by tests, hence more robust.
To make better use of the limited characters in mobile push
notifications for messages quoting another message, we compress
the blockquotes and "user said" paragraphs to make space for the
actual message.
Fixes#28951.
4430ab9cbe changed this, assuming that all servers would send
`realm_url` -- however, only servers running that commit do. Update
to accept either `realm_url` or `realm_uri` payload properties.
The naming `uri` is deprecated while `url` should be used in order to
satisfy URL standards. For this reason, four endpoints are affected:
* The response content of three endpoints `/server_settings`,
`/register` and `/realm` that contain a field `realm_uri` is
changed to `realm_url`.
* In one of the common fields for all mobile push notifications payloads,
`realm_url` field is now added as an alias to `realm_uri`.
For backwards compatibility, we keep the field `realm_uri` and add
an alias `realm_url`.
Co-authored-by: Junyao Chen <junyao.chen@socitydao.org>
In #23380, we are changing all occurrences of uri with url in order to
follow the latest URL standard. Previous PRs #25038 and #25045 has
replaced the occurences of uri that has no direct relation with realm.
This commit changes just the model property, which has no API
compatibility concerns.
Earlier, we didn't soft-reactivate users for group mentions
at all because it wasn't easy to calculate group size.
Now, we will soft reactivate if the user group mentions has
less than 12 members.
We don't reactivate all users because a user group can have a
very large size, which can lead to large backlogs in the
deferred-work queue.
Fixes part of #27586.
This commit updates the code in 'handle_push_notifications'
to use a common lib function 'get_mentioned_user_group'.
The code logically does the same thing in both the places,
hence the change.
Fixes#28403
Uses redis to remember the last time push notifications were experienced
working. This needs to work across processes, so can't be done just in
memory.
As this is transient data that's fairly harmless to lose and thus
doesn't require the persistence benefits of the database, and we're
keeping a single "row", so don't need an entire new db table, we settle
on using redis instead of postgres. This is also consistent with how we
store other kinds of such transient data.
f92d43c690 added uses of `@overload` to probide multiple type
signatures for `access_message`, based on the `get_user_message`
parameter. Unfortunately, mypy does not check the function body
against overload signatures, so it allows type errors to go
undetected.
Replace the overloads with two functions, for one of which also
returns the usermessage. The third form, of only returning if the
usermessage exists, is not in a high-enough performance endpoint that
a third form is worth maintaining; it uses the usermessage form.
The IntegrityError shows up in the database logs, which looks
unnecessarily concerning. Use `ON CONFLICT IGNORE` to mark this as
expected, especially since the return value is never used.
Replace the long string for organisations that have notification
body/content disabled (settings.PUSH_NOTIFICATION_REDACT_CONTENT
set to true) with "New message".
This allows more of the limited space on the mobile device screen to
be used for additional messages rather than this verbose content.
Fixes#29152
