When these exceptions are thrown from the request-to-bouncer functions
inside of manage.py register_server/update_analytics_counts, they
shouldn't be silenced, merely calling maybe_mark_pushes_disabled in the
background.
This results in the occurrence of the error not being shown to the
user. Failure to upload analytics data when running these commands
should result in a loud, obvious error.
Failure of running register_server before this change:
```
./manage.py register_server
This command registers your server for the Mobile Push Notifications Service.
Doing so will share basic metadata with the service's maintainers:
* This server's configured hostname: zulipdev.com:9991
* This server's configured contact email address: desdemona+admin@zulip.com
* Metadata about each organization hosted by the server; see:
<https://zulip.com/doc-permalinks/basic-metadata>
Use of this service is governed by the Zulip Terms of Service:
<https://zulip.com/policies/terms>
Do you want to agree to the Zulip Terms of Service and proceed? [Y/n]
Mobile Push Notification Service registration successfully updated!
```
The occurrence of the error is not revealed to the user. Same concern
applies to the update_analytics_counts command.
After this change:
```
./manage.py register_server
This command registers your server for the Mobile Push Notifications Service.
Doing so will share basic metadata with the service's maintainers:
<...>
Do you want to agree to the Zulip Terms of Service and proceed? [Y/n]
Traceback (most recent call last):
File "/srv/zulip/./manage.py", line 150, in <module>
execute_from_command_line(sys.argv)
File "/srv/zulip/./manage.py", line 115, in execute_from_command_line
utility.execute()
File "/srv/zulip-venv-cache/bb36fc1fcb6d8c70a9a0bcb7bac45d78623a9ff4/zulip-py3-venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 436, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/srv/zulip-venv-cache/bb36fc1fcb6d8c70a9a0bcb7bac45d78623a9ff4/zulip-py3-venv/lib/python3.10/site-packages/django/core/management/base.py", line 413, in run_from_argv
self.execute(*args, **cmd_options)
File "/srv/zulip/zerver/lib/management.py", line 97, in execute
super().execute(*args, **options)
File "/srv/zulip-venv-cache/bb36fc1fcb6d8c70a9a0bcb7bac45d78623a9ff4/zulip-py3-venv/lib/python3.10/site-packages/django/core/management/base.py", line 459, in execute
output = self.handle(*args, **options)
File "/srv/zulip/zerver/management/commands/register_server.py", line 137, in handle
send_server_data_to_push_bouncer(consider_usage_statistics=False, raise_on_error=True)
File "/srv/zulip/zerver/lib/remote_server.py", line 453, in send_server_data_to_push_bouncer
response = send_to_push_bouncer(
File "/srv/zulip/zerver/lib/remote_server.py", line 233, in send_to_push_bouncer
raise JsonableError(msg)
zerver.lib.exceptions.JsonableError: Duplicate registration detected.
```
Now that we introduced an URL for serving permalinks redirecting to
docs in #33444, the docs_url mechanism is no longer needed, as we can
have a URL that's safe to hard-code in register_server.py.
The HostnameAlreadyInUseBouncerError.docs_url has been merged in main
briefly enough, that this should be safe to remove.
This means that the URL is only hard-coded on the bouncer side. That's
useful, because now we'll be able to change the URL and only need a
bouncer deployment for users to get the new URL when they encounter
HostnameAlreadyInUseBouncerError. As opposed to self-hosted servers
being stuck with an outdated docs link hardcoded in their
register_server.py.
An even better way than the current json error message recommending the
--registration-transfer option is to return an appropriate error code
and have that get picked up by the register_server command.
The register_server command can then display a more comprehensive,
better formatted error message with proper whitespaces and a pointer to
the documentation.
This is the final naming that we want, compared to the naming we merged
in #32399.
Includes renaming the API endpoints, but that should be fine as the
original PR was just merged and this isn't deployed anywhere.
If the server controls the registration's hostname, it can reclaim its
registration credentials. This is useful, because self-hosted admins
frequently lose the credentials when moving their Zulip server to a
different machine / deployment method.
The flow is the following:
1. The host sends a POST request to
/api/v1/remotes/server/register/takeover.
2. The bouncer responds with a signed token.
3. The host prepares to serve this token at /api/v1/zulip-services/verify and
sends a POST to /remotes/server/register/verify_challenge endpoint of
the bouncer.
4. Upon receiving the POST request, the bouncer GETS
https://{hostname}/api/v1/zulip-services/verify, verifies the secret and
responds to the original POST with the registration credentials.
5. The host can now save these credentials to it zulip-secrets.conf file
and thus regains its push notifications registration.
Includes a global rate limit on the usage of the /verify_challenge
endpoint, as it causes us to make outgoing requests.
Instead of the PUSH_NOTIFICATIONS_BOUNCER_URL and
SUBMIT_USAGE_STATISTICS settings, we want servers to configure
individual ZULIP_SERVICE_* settings, while maintaining backward
compatibility with the old settings. Thus, if all the new
ZULIP_SERVICE_* are at their default False value, but the legacy
settings are activated, they need to be translated in computed_settings
to the modern way.
This is needed for the command to work in docker-zulip, where the zulip
user may not have write permissions to the parent directory of the
secrets file. By default, crudini creates a temporary file, which makes
the command fail under those conditions.
With --inplace, the secret file gets written to directly without
creation of temporary files.
Using --rotate-key without write access to the secrets file is currently
quite painful, since you end up rotating your registration's secret with
no local record of it; so effectively you lose your registration and
need help from support. We should just prevent this failure mode.
We call 'send_server_data_to_push_bouncer' just after registering
server for push notification.
This helps to have a current state of the user counts when first
logging in after the RemoteRealm flow.
This adds the X-Smokescreen-Role header to proxy connections, to track
usage from various codepaths, and enforces a timeout. Timeouts were
kept consistent with their previous values, or set to 5s if they had
none previously.
We're migrating to using the cleaner zulip.com domain, which involves
changing all of our links from ReadTheDocs and other places to point
to the cleaner URL.
This is a preparatory commit for using isort for sorting all of our
imports, merging changes to files where we can easily review the
changes as something we're happy with.
These are also files with relatively little active development, which
means we don't expect much merge conflict risk from these changes.
This fixes exceptions when sending PMs in development (where we were
trying to connect to the localhost push bouncer, which we weren't
authorized for, but even if we were, it wouldn't work, since there's
no APNS/GCM certs).
At the same time, we also set and order of operations that ensures one
has the opportunity to adjust the server URL before submitting
anything to us.
