APNs apparently treats its tokens case-insensitively; FCM does not.
Adjust the `unique_together` to instead be separate partial
constraints, keyed on the `kind` of the PushDeviceToken.
APNs tokens are provided by the client in hex, and we store them in
hex. The existing code which attempts to "validate" them by parsing
them as base64 only works because base64 is a superset of hex.
Enforce that APNs tokens are hex, and remove all of the pieces of test
code which were incorrectly passing them in as base64 strings.
This commit adds a `push_devices` dictionary to
`POST /register` response, keyed with push account ID,
where each entry describes the user's push device's
registration status and error code (if registration failed).
This commit adds a zilencer endpoint to let self-hosted
servers register push devices to whom mobile push notifications
will be sent.
POST "/api/v1/remotes/push/e2ee/register"
Payload: realm_uuid, push_account_id, encrypted_push_registration,
bouncer_public_key
The post request needs to be authenticated with the server’s API key.
Note: For Zulip Cloud, a background fact about the push bouncer is
that it runs on the same server and database as the main application;
it’s not a separate service.
So, as an optimization, we plan to directly call the
`do_register_remote_push_device` function and skip the HTTP request.
Fixeszulip/zulip-flutter#1617.
It turns out that an APNs token (which is a hex string) is equally
valid in lower or upper case. The old app would send the server
the lower-case form of the token, but the new app sends the
upper-case form.
Because we've been treating tokens case-sensitively, if the user
upgrades from the old app to the new, that results in the server
and bouncer each having two copies of the token (one lower-case and
one upper-case), and therefore sending that device two copies of
each notification: zulip/zulip-flutter#1617.
To fix that immediately, have the bouncer drop duplicate tokens
before sending the notifications to APNs.
Work is also in progress on fixing this in a better-structured way,
by having the database correctly treat tokens as the same when they
differ only in case.
To maintain API compatibility during migration to DirectMessageGroup
for 1:1 DMs, generate notification payloads for such messages in the
same format as those sent to a Personal recipient.
Fixes: part of issue #25713.
In aioapns 4.0, which we upgraded to in acd7353538, the `key`
parameter is treated as the ASCII-armored string contents of the key,
not the path to the file with that content.
Read the file ourselves, and pass the string it expects.
This is valuable so that one is forced to explicitly make a decision
on what is correct when adding new callers. Past experience tells us that
not having to explicitly show the decision leads to people introducing
security bugs in PRs that the maintainer has to catch in review, and our
goal for access control code should be that security bugs are hard to write.
Fixes#33688.
Now that we introduced an URL for serving permalinks redirecting to
docs in #33444, the docs_url mechanism is no longer needed, as we can
have a URL that's safe to hard-code in register_server.py.
The HostnameAlreadyInUseBouncerError.docs_url has been merged in main
briefly enough, that this should be safe to remove.
This commit adds support to display `Message.EMPTY_TOPIC_FALLBACK_NAME`
value (translated) in the push notifications for topics having the
actual value of empty string.
Fixes part of #32996.
This means that the URL is only hard-coded on the bouncer side. That's
useful, because now we'll be able to change the URL and only need a
bouncer deployment for users to get the new URL when they encounter
HostnameAlreadyInUseBouncerError. As opposed to self-hosted servers
being stuck with an outdated docs link hardcoded in their
register_server.py.
An even better way than the current json error message recommending the
--registration-transfer option is to return an appropriate error code
and have that get picked up by the register_server command.
The register_server command can then display a more comprehensive,
better formatted error message with proper whitespaces and a pointer to
the documentation.
Instead of the PUSH_NOTIFICATIONS_BOUNCER_URL and
SUBMIT_USAGE_STATISTICS settings, we want servers to configure
individual ZULIP_SERVICE_* settings, while maintaining backward
compatibility with the old settings. Thus, if all the new
ZULIP_SERVICE_* are at their default False value, but the legacy
settings are activated, they need to be translated in computed_settings
to the modern way.
This commit performs a sweep on the first batch of non API
files to rename "huddle" to "direct_message_group`.
It also renames variables and methods of type -
"huddle_message" to "group_direct_message".
This is a part of #28640
We believe this to already be obsolete and dead code and is about to be
removed with the migration to the FCM HTTP v1 API, where the concept
doesn't exist anymore.
This commit replaces the local 'is_same_server_message_link'
function used in 'get_mobile_push_content' with the
'is_same_server_message_link' lib function.
The lib function is the same logically but uses urllib instead
of regex for parsing and is backed by tests, hence more robust.
To make better use of the limited characters in mobile push
notifications for messages quoting another message, we compress
the blockquotes and "user said" paragraphs to make space for the
actual message.
Fixes#28951.
4430ab9cbe changed this, assuming that all servers would send
`realm_url` -- however, only servers running that commit do. Update
to accept either `realm_url` or `realm_uri` payload properties.
The naming `uri` is deprecated while `url` should be used in order to
satisfy URL standards. For this reason, four endpoints are affected:
* The response content of three endpoints `/server_settings`,
`/register` and `/realm` that contain a field `realm_uri` is
changed to `realm_url`.
* In one of the common fields for all mobile push notifications payloads,
`realm_url` field is now added as an alias to `realm_uri`.
For backwards compatibility, we keep the field `realm_uri` and add
an alias `realm_url`.
Co-authored-by: Junyao Chen <junyao.chen@socitydao.org>
In #23380, we are changing all occurrences of uri with url in order to
follow the latest URL standard. Previous PRs #25038 and #25045 has
replaced the occurences of uri that has no direct relation with realm.
This commit changes just the model property, which has no API
compatibility concerns.
Earlier, we didn't soft-reactivate users for group mentions
at all because it wasn't easy to calculate group size.
Now, we will soft reactivate if the user group mentions has
less than 12 members.
We don't reactivate all users because a user group can have a
very large size, which can lead to large backlogs in the
deferred-work queue.
Fixes part of #27586.
This commit updates the code in 'handle_push_notifications'
to use a common lib function 'get_mentioned_user_group'.
The code logically does the same thing in both the places,
hence the change.
Fixes#28403
Uses redis to remember the last time push notifications were experienced
working. This needs to work across processes, so can't be done just in
memory.
As this is transient data that's fairly harmless to lose and thus
doesn't require the persistence benefits of the database, and we're
keeping a single "row", so don't need an entire new db table, we settle
on using redis instead of postgres. This is also consistent with how we
store other kinds of such transient data.
f92d43c690 added uses of `@overload` to probide multiple type
signatures for `access_message`, based on the `get_user_message`
parameter. Unfortunately, mypy does not check the function body
against overload signatures, so it allows type errors to go
undetected.
Replace the overloads with two functions, for one of which also
returns the usermessage. The third form, of only returning if the
usermessage exists, is not in a high-enough performance endpoint that
a third form is worth maintaining; it uses the usermessage form.
The IntegrityError shows up in the database logs, which looks
unnecessarily concerning. Use `ON CONFLICT IGNORE` to mark this as
expected, especially since the return value is never used.
Replace the long string for organisations that have notification
body/content disabled (settings.PUSH_NOTIFICATION_REDACT_CONTENT
set to true) with "New message".
This allows more of the limited space on the mobile device screen to
be used for additional messages rather than this verbose content.
Fixes#29152
For the common case of not needing to reference the UserMessage row
later, and for being a stream without private history, the UserMessage
row is irrelevant. Convert `has_user_message` to a thunk, and defer
loading it unless necessary.
Replaced HUDDLE attribute with DIRECT_MESSAGE_GROUP using VS Code search,
part of a general renaming of the object class.
Fixes part of #28640.
Co-authored-by: JohnLu2004 <JohnLu10212004@gmail.com>
The problem was that earlier this was just an uncaught JsonableError,
leading to a full traceback getting spammed to the admins.
The prior commit introduced a clear .code for this error on the bouncer
side, meaning the self-hosted server can now detect that and handle it
nicely, by just logging.error about it and also take the opportunity to
adjust the realm.push_notifications_... flags.
