Refactors Realm.VIDEO_CHAT_PROIVDERS to have all the possible options
for video chat integrations, and use get_enabled_video_chat_providers
to compute the enabled options for the realm.
Prep for adding Zoom server to server video chat integration.
`zulip_update_announcements_stream` should be set to None when its
channel is deactivated. This commit adds a safeguard to explicitly
raise an error if it is found in a deactivated state, indicating a
potential bug that needs investigation.
Without this commit, we won't face any runtime error but the error
raised would be at a much later stage i.e. while sending message,
with a generic error message i.e. "Not authorized to send to channel "
which would require further investigation to determine the root cause.
This commit helps with an early return and better error message
to debug.
When a stream configured for any of these settings is deactivated,
the corresponding realm settings should be set to NULL:
* new_stream_announcements_stream
* signup_announcements_stream
* zulip_update_announcements_stream
* moderation_request_channel
Earlier, we were not updating those realm settings to NULL.
We had helper functions like 'get_new_stream_announcements_stream'
to return None if the configured stream was deactivated.
But it makes more sense to just set them to NULL in DB.
This commit also includes a migration to clear those fields
if the configured channels are deactivated.
This commit adds support to display `Message.EMPTY_TOPIC_FALLBACK_NAME`
value (translated) in the push notifications for topics having the
actual value of empty string.
Fixes part of #32996.
This commit converts the links generated by the markdown
of the "#-mention" of topics to permalinks -- the links containing
the "with" narrow operator, the operand being the last message
of the channel and topic of the mention.
Fixes part of #21505
The integrations - GoSquared, Greenhouse, OpsGenie and Zendesk - have
been using `000.png` as their example screenshots, renamed them to
match the default name `001.png`, to avoid passing them as arguments.
This removes a race which parallels 6f20c15ae9, but in the import
path; thumbnails could have been generated while the message content
was being rendered, leaving them permanently with spinners.
The act of rendering the message markdown is what enqueues the
thumbnailing of referenced images; previously, images were enqueued at
least _twice_: once when the row was created, and again for every
message they were referenced in.
This means that only ImageAttachment row needs to be fetched, and
removes the need to pass around an extra parameter. This
denormalization is safe, since in general Attachment rows are
read-only, so we are not concerned with drift between the Attachment
and ImageAttachment tables.
We cannot make content_type non-null, since while the both the
`content_type` column in Attachment and populating that from requests
predates the ImageAttachment table, we have both backfilled
ImageAttachment rows to consider, and imports may also leave files
with no `content_type`. Any backfill of currently-null `content_type`
values will thus need to update both tables.
This change fixes a race condition when importing. ImageAttachment
rows are imported before rendering Messages, which are both before
importing Attachment rows; if the thumbnailing finished after the
Message was imported but before Attachment rows were imported, then
the re-rendering step would not know the image's content-type.
This commit adds support for empty string as a valid topic name
in syntax for linking to channel messages.
The server stores it after empty string is replaced with
`realm_empty_topic_display_name` and wrapped with an <em> tag.
The web client parses the rendered_content and updates
the topic_name part in the HTML with topic_name in user's language
+ wraps it in a <span> tag with 'empty-topic-display' css class.
This commit adds support for empty string as a valid topic name
in syntax for linking to topics.
The server stores it after empty string is replaced with
`realm_empty_topic_display_name` and wrapped with an <em> tag.
The web client parses the rendered_content and updates
the topic_name part in the HTML with topic_name in user's language
+ wraps it in a <span> tag with 'empty-topic-display' css class.
This means that the URL is only hard-coded on the bouncer side. That's
useful, because now we'll be able to change the URL and only need a
bouncer deployment for users to get the new URL when they encounter
HostnameAlreadyInUseBouncerError. As opposed to self-hosted servers
being stuck with an outdated docs link hardcoded in their
register_server.py.
The functionality of gravatar can break anonymity if
the user has had a gravatar account set up previously.
This option allows specifically cloud instances to
have gravatar disabled selectively.
An even better way than the current json error message recommending the
--registration-transfer option is to return an appropriate error code
and have that get picked up by the register_server command.
The register_server command can then display a more comprehensive,
better formatted error message with proper whitespaces and a pointer to
the documentation.
This is the final naming that we want, compared to the naming we merged
in #32399.
Includes renaming the API endpoints, but that should be fine as the
original PR was just merged and this isn't deployed anywhere.
Added `can_create_bots_group` setting which controls who can
create any type of bots in the organization.
Added `can_create_write_only_bots_group` setting which controls
who can create incoming webhooks in the organization in additon
to those who are in `can_create_bots_group`.
Made maybe_negated a required parameter in
check_not_both_channel_and_dm_narrow so that it doesn't raise error
when -is:dm is called with channels:public or
when -channels:public is called with is:dm
Fixes#33033
This commit adds hardening such that if the invariant "no usermessage
row corresponding to a message exists if the user loses access to the
message" is violated due to some bug, user can't access the attachment
if they are not subscribed.
If the server controls the registration's hostname, it can reclaim its
registration credentials. This is useful, because self-hosted admins
frequently lose the credentials when moving their Zulip server to a
different machine / deployment method.
The flow is the following:
1. The host sends a POST request to
/api/v1/remotes/server/register/takeover.
2. The bouncer responds with a signed token.
3. The host prepares to serve this token at /api/v1/zulip-services/verify and
sends a POST to /remotes/server/register/verify_challenge endpoint of
the bouncer.
4. Upon receiving the POST request, the bouncer GETS
https://{hostname}/api/v1/zulip-services/verify, verifies the secret and
responds to the original POST with the registration credentials.
5. The host can now save these credentials to it zulip-secrets.conf file
and thus regains its push notifications registration.
Includes a global rate limit on the usage of the /verify_challenge
endpoint, as it causes us to make outgoing requests.
Fixes: #32934
context:
Fetching all users who are members (directly or via sub-groups)
of groups mentioned in one message.
Reduce O(n) queries, where n is the number
of mentioned groups, to a constant of 1 query.
Extend "get_recursive_subgroups_for_groups" functionality to
"get_root_id_annotated_recursive_subgroups_for_groups"
which is the same but keeps track of each group root_id and
annotates it to each group.
Then in init_user_group_data(), we only fetch
each group's root_id along with
active direct members.
This commit adds support to display `Message.EMPTY_TOPIC_FALLBACK_NAME`
value (translated) in the missedmessage email body and subject for
topics having the actual value of empty string.
Fixes part of #32996.
We've added a comment highlighting that the function does not check
whether a user has access to the channel or not. Adding `accessible` to
the function name further emphasises that.
Rename `can_subscribe_others_to_all_streams` to
`can_subscribe_others_to_all_accessible_streams` so it's clear that we
are not attempting to check basic access in this function.
value can be None when property is jitsi_server_url,
message_content_edit_limit_seconds,
message_content_delete_limit_seconds,
move_messages_between_streams_limit_seconds, or
move_messages_within_stream_limit_seconds.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Fixes#32369
Migrate stream delete event to include only stream ids in the form of
"stream_ids": [1,...], because clients only need the ids.
While keep sending ids in the form of "streams": [{stream_id: 1},...]
for compatibility with all clients other than web.
This commit updates outgoing bots to mark messages they process as read.
Since the service bots have their own `UserMessage` rows, this change
enables us to track whether the bot has in fact processed the message by
adding the `read` flag to their `UserMessage`.
The current `get_migration_by_app` has a rather naive approach to
compiling the migration status of a realm, which has led to issues like
#32826. Specifically, those flaws are:
- it does not report the complete state of the migration status of the
exporting servers, only the applied migration.
- it shows both the replaced and the squashed migrations. This would be
a problem if we decide to clean up old migration files we've
squashed(replaced) and import a slightly older realm with those still in
disk. `check_migration_status` would complain of incompatibility even
though those migration files don't matter (they are replaced, after
all).
- it does not clean up ancient/stale applied migrations (for reference,
see how `check-database-compatibility` cleans those)
This commit attempts to write a better `migration_status.json` by
parsing the output of `showmigrations` instead.
This is because Django's `showmigrations` has a lot more logic and
validations baked into it than previously thought. Ones that we care
about are:
- it does validations to make sure app names are valid
- it doesn't list replaced migrations and only squashed one
- it takes into account migrations in disk(`MigrationsLoader`) vs
applied migrations (`MigrationsRecorder`)
Which would resolve the first two points highlighted above.
This commit adds `parse_migration_status`, which takes in the string
output of `showmigrations` and parse it into key-value pair of installed
apps and a list of its migration status.
This is a prep commit to rework the check migrations function of
import/export which will parse the output of `showmigrations` to write
the `migration_status.json` file.
This consolidates the list of stale migration to
`lib/migration_status.py` as `STALE_MIGRATIONS`.
This is a prep work to make the migration status tool at
`migration_status.py` be able to clean its output of these migrations
too.
in `get_migrations_status`, we clean up the printed output of any ANSI
codes used to format the output. Currently the regex only cleans up bold
ANSI escape code (\x1b[1m) and style reset code (\x1b[0m). So it won't
be able to clean up basic ANSI escape codes such as "\x1b\31;1m" which
is used to format `showmigrations` output for apps with no migrations.
e.g, "\x1b\31;1m (no migrations)"
This commit updates the regex to catch a wider range of basic ANSI
codes.
The `get_migration_status` command calls `connections.close_all()` when
its done and it was previously only called when we need to rebuild the
dev or test database and when running the `get_migration_status`
command.
This commit moves the `connections.close_all()` call out of the function
and into `test_fixtures.py` directly, making sure it will only be called
when we are rebuilding the dev/test database. This is a prep work to
refactor the check migration function of import/export later on which
plans to use `get_migration_status`.
This moves `get_migration_status` to its own file in
zerver/lib/migration_status.py. This is a prep work to refactor the
check migration function of import/export later on.
Some of the imports are moved into `get_migration_status` because we're
planning to share this file with `check-database-compatibility` which is
also called when one does `production-upgrade`, so we'd want to avoid
doing file-wide import on certain types of modules because it will fail
under that scenario.
In `test_fixtures.py`, `get_migration_status` is imported within
`Database.what_to_do_with_migrations` so that it is called after
`cov.start()` in `test-backend`. This is to avoid wierd interaction with
coverage, see more details in #33063.
Fixes#33063.
