An even better way than the current json error message recommending the
--registration-transfer option is to return an appropriate error code
and have that get picked up by the register_server command.
The register_server command can then display a more comprehensive,
better formatted error message with proper whitespaces and a pointer to
the documentation.
This is the final naming that we want, compared to the naming we merged
in #32399.
Includes renaming the API endpoints, but that should be fine as the
original PR was just merged and this isn't deployed anywhere.
If the server controls the registration's hostname, it can reclaim its
registration credentials. This is useful, because self-hosted admins
frequently lose the credentials when moving their Zulip server to a
different machine / deployment method.
The flow is the following:
1. The host sends a POST request to
/api/v1/remotes/server/register/takeover.
2. The bouncer responds with a signed token.
3. The host prepares to serve this token at /api/v1/zulip-services/verify and
sends a POST to /remotes/server/register/verify_challenge endpoint of
the bouncer.
4. Upon receiving the POST request, the bouncer GETS
https://{hostname}/api/v1/zulip-services/verify, verifies the secret and
responds to the original POST with the registration credentials.
5. The host can now save these credentials to it zulip-secrets.conf file
and thus regains its push notifications registration.
Includes a global rate limit on the usage of the /verify_challenge
endpoint, as it causes us to make outgoing requests.
This moves `get_migration_status` to its own file in
zerver/lib/migration_status.py. This is a prep work to refactor the
check migration function of import/export later on.
Some of the imports are moved into `get_migration_status` because we're
planning to share this file with `check-database-compatibility` which is
also called when one does `production-upgrade`, so we'd want to avoid
doing file-wide import on certain types of modules because it will fail
under that scenario.
In `test_fixtures.py`, `get_migration_status` is imported within
`Database.what_to_do_with_migrations` so that it is called after
`cov.start()` in `test-backend`. This is to avoid wierd interaction with
coverage, see more details in #33063.
Fixes#33063.
This commit adds a `sender_id` parameter to the
`GET /streams/{stream_id}/email_address` endpoint to specify the
ID of a user or bot which should appear as the sender when messages
are sent to a channel using the channel email address.
Earlier, Email gateway bot was always the sender.
Fixes part of #31566.
This commit extracts the logic for creating or retrieving a channel
email token into a dedicated `get_channel_email_token` function.
This improves code clarity by decoupling token generation from
the email encoding process.
This commit renames the 'queue_json_publish' function to
'queue_json_publish_rollback_unsafe' to reflect the fact that it doesn't
wait for the db transaction (within which it gets called, if any)
to commit and sends event irrespective of commit or rollback.
In most of the cases we don't want to send event in the case of
rollbacks, so the caller should be aware that calling the function
directly is rollback unsafe.
Fixes part of #30489.
After #28478 we no longer change the name when deactivating a channel,
so this is particularly needed - now there's always a "collision" on
new_name.
Also fixes a second detail - the name should be matched
case-insensitively.
orjson’s use of Unicode is more consistent with what we get from
Transifex. (We could alternatively use json’s ensure_ascii=False
flag.)
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This commit adds `durable=True` to the outermost db transactions
created in the following:
* confirm_email_change
* handle_upload_pre_finish_hook
* deliver_scheduled_emails
* restore_data_from_archive
* do_change_realm_subdomain
* do_create_realm
* do_deactivate_realm
* do_reactivate_realm
* do_delete_user
* do_delete_user_preserving_messages
* create_stripe_customer
* process_initial_upgrade
* do_update_plan
* request_sponsorship
* upload_message_attachment
* register_remote_server
* do_soft_deactivate_users
* maybe_send_batched_emails
It helps to avoid creating unintended savepoints in the future.
This is as a part of our plan to explicitly mark all the
transaction.atomic calls with either 'savepoint=False' or
'durable=True' as required.
* 'savepoint=True' is used in special cases.
This commit adds a --reset-level optional argument to
send_zulip_update_announcements management command to
reset all the active realms to a given level.
This commit adds 'durable=True' to the outermost transactions
of the following functions:
* do_create_multiuse_invite_link
* do_revoke_user_invite
* do_revoke_multi_use_invite
* sync_ldap_user_data
* do_reactivate_remote_server
* do_deactivate_remote_server
* bulk_handle_digest_email
* handle_customer_migration_from_server_to_realm
* add_reaction
* remove_reaction
* deactivate_user_group
It helps to avoid creating unintended savepoints in the future.
This is as a part of our plan to explicitly mark all the
transaction.atomic decorators with either 'savepoint=False' or
'durable=True' as required.
* 'savepoint=True' is used in special cases.
This adds a new special UserProfile flag can_change_user_emails(disabled
by default) and the ability for changing the email address of users in
the realm via update_user_backend. This is useful for allowing
organizations to update user emails without needing to set up a SCIM
integration, but since it gives the ability to hijack user accounts, it
needs to be behind this additional permission and can't be just given to
organization owners by default. Analogical to how the
create_user_backend endpoint works.
Earlier, we used to store the key data related to realm exports
in RealmAuditLog. This commit adds a separate table to store
those data.
It includes the code to migrate the concerned existing data in
RealmAuditLog to RealmExport.
Fixes part of #31201.
In docker-zulip installs, /etc/zulip/zulip.conf,
/etc/zulip/zulip-secrets.conf, and /home/zulip/uploads are all
symlinks into the `/data` directory which is mounted as a Docker
Volume. By default, `tar` does not dereference symlinks, leading to
backups that are missing these critical pieces.
Add `-h` to the `tar` invocation, to follow symlinks, so backups in
Docker have all of their pieces. Since none of the contents of the
backup intentionally use symlinks, this is safe.
Co-authored-by: Alex Vandiver <alexmv@zulip.com>
For exporting full with consent:
* Earlier, a message advertising users to react with thumbs up
was sent and later used to determine the users who consented.
* Now, we no longer need to send such a message. This commit
updates the logic to use `allow_private_data_export` user-setting
to determine users who consented.
Fixes part of #31201.
Adds a check for changing an existing guest user's role before
calling do_update_user in the case that a realm has a current
paid plan with manual license management.
Currently, it handles two hook types: 'pre-create' (to verify that the
user is authenticated and the file size is within the limit) and
'pre-finish' (which creates an attachment row).
No secret is shared between Django and tusd for authentication of the
hooks endpoints, because none is necessary -- tusd forwards the
end-user's credentials, and the hook checks them like it would any
end-user request. An end-user gaining access to the endpoint would be
able to do no more harm than via tusd or the normal file upload API.
Regardless, the previous commit has restricted access to the endpoint
at the nginx layer.
Co-authored-by: Brijmohan Siyag <brijsiyag@gmail.com>
Renamed event types below in the enum class to use channel instead of
stream.
Event types moved: STREAM_CREATED, STREAM_DEACTIVATED, STREAM_NAME_CHANGED
STREAM_REACTIVATED, STREAM_MESSAGE_RETENTION_DAYS_CHANGED
STREAM_PROPERTY_CHANGED, STREAM_GROUP_BASED_SETTING_CHANGED
A utility command to enable or disable certain authentication backends
for a realm from the command line. Can be helpful e.g. if the
administrator accidentally disables some auth methods in the UI leaving
themselves with none remaining that they could actually use to log back
into the organization.
Example usage:
```
(zulip-py3-venv) vagrant@c32c137f59a0:/srv/zulip$ ./manage.py change_auth_backends -r zulip --show
Current authentication backends for the realm:
Enabled backends:
Dev
Email
GitHub
GitLab
Google
Apple
SAML
OpenID Connect
(zulip-py3-venv) vagrant@c32c137f59a0:/srv/zulip$ ./manage.py change_auth_backends -r zulip --disable GitHub
Disabling GitHub backend for realm Zulip Dev
Updated authentication backends for the realm:
Enabled backends:
Dev
Email
GitLab
Google
Apple
SAML
OpenID Connect
Disabled backends:
GitHub
Done!
(zulip-py3-venv) vagrant@c32c137f59a0:/srv/zulip$ ./manage.py change_auth_backends -r zulip --enable GitHub
Enabling GitHub backend for realm Zulip Dev
Updated authentication backends for the realm:
Enabled backends:
Dev
Email
GitHub
GitLab
Google
Apple
SAML
OpenID Connect
Done!
```
Instead of the PUSH_NOTIFICATIONS_BOUNCER_URL and
SUBMIT_USAGE_STATISTICS settings, we want servers to configure
individual ZULIP_SERVICE_* settings, while maintaining backward
compatibility with the old settings. Thus, if all the new
ZULIP_SERVICE_* are at their default False value, but the legacy
settings are activated, they need to be translated in computed_settings
to the modern way.
Tornado ≥ 5 uses AsyncIOLoop and AsyncIOMainLoop automatically when
appropriate. Fixes “DeprecationWarning: make_current is deprecated;
start the event loop first” with warnings enabled.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
A new table is created to track which path_id attachments are images,
and for those their metadata, and which thumbnails have been created.
Using path_id as the effective primary key lets us ignore if the
attachment is archived or not, saving some foreign key messes.
A new worker is added to observe events when rows are added to this
table, and to generate and store thumbnails for those images in
differing sizes and formats.
This is needed for the command to work in docker-zulip, where the zulip
user may not have write permissions to the parent directory of the
secrets file. By default, crudini creates a temporary file, which makes
the command fail under those conditions.
With --inplace, the secret file gets written to directly without
creation of temporary files.
This commit renames the "Huddle" Django model class to
"DirectMessageGroup", while maintaining the same table --
"zerver_huddle".
Fixes part of #28640.
This commit performs a sweep on the first batch of non API
files to rename "huddle" to "direct_message_group`.
It also renames variables and methods of type -
"huddle_message" to "group_direct_message".
This is a part of #28640
