UTF-8 is far and wide the dominant encoding, so we assume that
encoding if we have more than 66% confidence -- this allows it to be
guessed even for short files. ISO-8859-1 caps at 73% confidence, so
we assume that if it's the best option and it hits that max.
Otherwise, we still require 90% confidence.
This commit updates the data that gets encrypted to be
the same on both android and iOS.
The data and its format is almost the same as what we send
as FCM payload to android clients with no E2EE support,
changes are:
For send push notification payload:
* 'realm_id`, 'server', 'sender_email', and 'realm_uri' fields
don't exist in the new payload.
* 'event' field renamed to 'type'
* 'stream' and 'stream_id' fields renamed to 'channel_name'
and 'channel_id' respectively.
* The value of 'recipient_type' will be 'channel' & 'direct'
instead of 'stream' & 'private' respectively.
* 'zulip_message_id' field renamed to 'message_id'
For remove push notification payload:
* 'realm_id`, 'server', and 'realm_uri' fields don't exist
in the new payload.
* 'event' field renamed to 'type'
* 'zulip_message_ids' field renamed to 'message_ids' and it's
value will be a JSON array instead of a string.
In the existing iOS client, we have no code of our own involved
in constructing the notifications in the UI, and instead we
leave it to the iOS SDK to do so.
Since, for clients with E2EE support the data is going to be
interpreted by our own code, not by the iOS SDK - we are free
to keep the same data and format.
Co-authored-by: Tim Abbott <tabbott@zulip.com>
This commit removes the 'zulip_message_id' field from the FCM
payload for remove push notification.
Clients that use `zulip_message_ids` instead have been out for
years and we no longer support older client versions that don't.
Fixes#20028.
There's no reason to have a special `RealmCreationKey` class - the
`Confirmation` system already does this job.
This is somewhat complicated by the need to write a migration for
`RealmCreationKey`->`Confirmation` for pre-existing, valid objects, to
avoid breaking realm creation links that haven't been used yet.
Clears up some naming confusion - `REALM_CREATION_LINK_VALIDITY_DAYS` is
a setting which describes the validaty period of RealmCreationKey, not
of Confirmation with REALM_CREATION type. Now that the latter is called
NEW_REALM_USER_REGISTRATION, there should be no confusion about this.
This commit updates code to require unique names for only
non-archived folders in a realm, which means a folder can
use a name which is being used by an archived folder.
The order of token values returned by the database query isn't
guaranteed, so 'assertEqual' fails in a flaky way.
This commit uses 'assertCountEqual' to fix that.
This reverts 1862c3d9ab.
The "time" field was added in the APNs payload to log
"Remote queuing latency".
The client doesn't use this field. Ideally, we should have
removed the field from the payload before sending it to APNs.
We no longer log "Remote queuing latency", so it's safe to
remove it from the APNs payload.
Only FCM payload will have this field as it is used by android
clients.
The function was used to reformat apns payload sent
by servers older than 613d093d7.
Since we no longer need to worry about servers older
than 1.6.0, it's safe to remove that function.
"Hide" is a better verb with the channel folders UI.
Also automatically hide inactive channels at 20+ susbscribed channels,
as we have more content in the left sidebar now.
This commit adds "max_channel_folder_name" and
"max_channel_folder_description" fields to response.
Also updated webapp to use these fields instead of hardcoding
the values.
This commit replaces the `PUSH_NOTIFICATION_REDACT_CONTENT` server
setting with `require_e2ee_push_notifications` realm setting.
If `require_e2ee_push_notifications` set to True:
* Older clients: Content redacted
* Updated clients: Encrypted content
If `require_e2ee_push_notifications` set to False:
* Older clients: Content NOT redacted
* Updated clients: Encrypted content
Note: Older clients refers to clients that don't support E2EE.
Fixes part of #35370.
This commit adds a realm setting:
"Require end-to-end encryption for push notification content".
This commit just involves backend & UI changes to add the setting,
the impact of this setting is in the next commit.
Fixes part of #35370.
Previously, when there were two topics -- A and B, and a user
follows topic A, then when messages of topic B are merged into
topic A then topic A loses its follow status.
This is fixed by updating the algorithm for getting the visibility
policy after merge to be followed in the above case.
Fixes#35226
This commit adds a test and updates a few existing tests to
cover more cases related to send push notifications.
* We no longer mock the 'send_push_notifications_legacy' function
while testing 'send_push_notifications' codepath and vice-versa.
This makes the tests more realistic as both functions gets called
in 'handle_push_notification'.
This covers the case when only old clients (which don't support
E2EE) exists for a user. Or only updated clients (which supports
E2EE) exist.
* Adds a test 'test_both_old_and_new_client_coexists' for the case
when a user has both type of clients at an instant i.e. they have
updated a few devices only.
In 'send_push_notifications_legacy', when a user has no registered
devices:
* `uses_notification_bouncer()`=True: we log "Skipping..." and return.
* `uses_notification_bouncer()`=False: we make some function calls,
which effectively does nothing.
It's better to have a common check (and log) early in the codepath.
This commit makes that change.
In #27186, we updated the help center documentation on
reading conversations and removed the article from the
left sidebar.
Removes remaining links to the article in "Printing
messages" and adds a redirect to go to the main article
on "Direct messages".
This commit removes the DEVELOPMENT guard from the new
'send_push_notifications' function in 'handle_push_notification'
and 'handle_remove_push_notification'.
Earlier, we were passing a map `device_id_to_encrypted_data`
and http headers as separate fields to bouncer.
The downside of that approach is it restricts the bouncer to
process only one type of notice i.e. either notification for
a new message or removal of sent notification, because it
used to receive a fixed priority and push_type for all the
entries in the map.
Also, using map restricts the bouncer to receive only one
request per device_id. Server can't send multiple notices
to a device in a single call to bouncer.
Currently, the server isn't modelled in a way to make a
single call to the bouncer with:
* Both send-notification & remove-notification request data.
* Multiple send-notification request data to the same device.
This commit replaces the old protocol of sending data with
a list of objects where each object has the required data
for bouncer to send it to FCM or APNs.
This makes things a lot flexible and opens possibility for
server to batch requests in a different way if we'd like to.
'get_apns_payload_data_to_encrypt' was added in commit
0ae34ddb65, in parallel
to 'get_message_payload_apns' - to use in E2EE codepath.
The intent was to avoid nesting in the payload returned
by 'get_message_payload_apns' function, just like FCM
payload returned by 'get_message_payload_gcm'.
Turned out, the nesting is helpful in APNs case for various
reasons. So, this commit reverts that function and we'll
continue to use the older structure returned by the function
'get_message_payload_apns'.
This is only `text/plain`, currently. In such cases where the
client-provided content-type also does not specify a `charset`, we use
`chardet` to make a guess, and store that guess to provide later when
serving the content. The detection is done in a streaming fashion,
and thus should not require re-downloading the full content.
Previously, the `user_file.content_type` only contained the MIME type
of the uploaded file; no other parameters were included, meaning that
a file a client specified as `content-type: text/plain; charset=big5`
would be stored with an `Attachment.content_type` of `text/plain`.
Re-construct the full content-type header from `content_type_extra`,
which includes those parameters.
We do not include a test because Django does not support specifying
such parameters in the upload path.
