These two endpoints make up ~85% of requests to Zulip servers; since
presence is also a performance-critical endpoint, having them share
the same cache increases how hot it stays in memcached, in addition to
making the presence endpoint faster.
This comes at the very slightly cost of one extra field. Checks
for the `is_bot` column are switched to the equivalent `bot_type`
check, since the columns are slightly duplicative, and we can get away
with only checking bot_type.
Tornado requests try hard to not make SQL queries -- and if they're
necessary, to minimize the number of them. Specifically, both session
objects and user objects are cached in memcached, and we expect that
both of them will have been filled there by Django before any requests
are made to Tornado.
In the event that memcached is flushed, or data is otherwise evicted,
we perform two database queries -- one for the session, and one for
the user. However, the *width* of the latter query has grown
significantly over time, as the Realm object grew more fields, and
recently with the addition of role groups, which require multiple
joins each. This leads to a query which is over 12k of text long, and
results in 319 columns. In the event of a memcached flush, this can
result in a *significant* amount of SQL traffic, as nearly every
active Tornado request will make that query.
We do not wish to narrow the default query for Django; we instead tag
the request in the REST wrapper, and use that to use a much narrower
user cache entry. That narrower cache entry is filled before the
queue is created in Django; we also use it to explicitly set the log
data, so the second "half" of the continued Tornado request does not
need to fetch any user data either when writing its log line.
Because they use different cache keys, this only affects the
session-based `/json/events` endpoint, which caches by user-id; the
`/api/v1/events` endpoint, which uses an API-key cache, keeps its wide
user object. The former is 50% of the total request volume, whereas
the latter is only 2%, so adding an additional cache for it is
unnecessary complexity.
The previous logic over-wrote the requester with the old value if it
had been set to anything in the new request, which it never could have
been. This logic likely stems from confusion in the hasattr
introduced in `89394fc1ebee`.
Always copy the `requester_for_logs` from the first half of the
request.
1ea2f188ce mistakenly introduced a `_rate_limit` member of the
request, which was dutifully transcribed in 3f9a5e1e17. However,
`_rate_limit` was never read from, nor written to -- `_ratelimit` (with
no middle `_`) was the dict that contained rate-limiting data. This
`_ratelimit` dict was later renamed to the `_ratelimits_applied` list,
in e86cfbdbd7, which became the `ratelimits_applied` request note
field in 03693cd27e.
Remove the entirely unused `rate_limit` note, and properly copy the
`ratelimits_applied` data into the new request.
Now that we introduced an URL for serving permalinks redirecting to
docs in #33444, the docs_url mechanism is no longer needed, as we can
have a URL that's safe to hard-code in register_server.py.
The HostnameAlreadyInUseBouncerError.docs_url has been merged in main
briefly enough, that this should be safe to remove.
This commit adds typing indicators for message editing in stream
as well as in dm, if the send typing notification
for corresponding is enabled.
Based on earlier work in #28585.
Co-authored-by: Rohan Gudimetla <rohan.gudimetla07@gmail.com>
Fixes#25719.
The comment was incorrectly placed and misleadingly suggested that
`long_term_idle` subscribers were excluded from `subscription_query`,
whereas they were not.
Previously, when description for a channel -- either during its
creating or when we change its description contained a topic
permalink (through #-mention), then it was not rendered. This
is because of lack of authorization to access the channel.
This is fixed by passing the acting_user through the methods
which update or add the description, so that permissions
of the acting_user could be used to determine whether to
render the #-mention in stream description or not.
Previously when system bots used to `#-mention` a private
channel's topics in cases like moving messages of a private
channel, then the #-mentions were not rendered by the
markdown. This was because the system bots did not have
authorization to mention these channels.
This is fixed by passing down an `acting_user` parameter
in code paths involving sending these move message
notifications so that permission of acting_user to mention
the topic is verified for rendering the markdown, rather
than that of the system bot.
This will gives us the flexibility to edit our documentation, including
section names, without worrying about breaking links to docs hard-coded
in older releases / deployed self-hosted servers.
When a user gets access to a private channel, they get a stream creation
event. Channel admins and users in `can_add_subscribers_group` already
have metadata access to a private channel and therefore do not need to
be notified.
Previously, we needed to pass the group to the function, which sometimes
meant having 1 extra query to fetch the user group when we just needed
the group id for this function.
We were not able to use
get_user_ids_with_metadata_access_via_permission_groups in the function
in question due to a cyclic dependency to `lib/streams.py`.
Fixes https://chat.zulip.org/#narrow/channel/101-design/topic/permissions.20for.20admin.20to.20unsubscribe.20others/near/2060197
Non realm admin users were not able to view private channels they were
an administrator of but not subscribed to it. This commit changes that.
We also made changes for those users to be able to see the subscribers
list.
The increase in query count in test_home and test_event_system can be
mitigated by only fetching recursive user group ids when needed within
the `validate_user_access_to_subscribers_helper` function. But that
would require refactoring that function to handle multiple streams and
subscriptions at once, along with changing how that function is used at
different places, which might be an exercise better left as a follow up.
We have optimised the code a little bit by not fetching the group ids in
case the current user is a realm admin.
We are fetching channel_admin_ids and users belonging to
can_add_subscribers_group directly in stream_subscription.py without
using the helper function
`get_user_ids_with_metadata_access_via_permission_groups`. This is due
to a cyclic dependency and we will move `bulk_get_subscriber_peer_info`
to another file in the next commit.
Users in `can_administer_channel_group` and `can_add_subscribers_group`
have access to private channel metadata. They should be notified of
relevant events.
We've only made relevant changes to lib/streams.py in this commit to
make the changes small and reviewable.
Previously, realm and channel admins were not able to change settings
for a private channel they were not subscribed to. This commit changes
that.
We have only added the exception for can_add_subscribers_group
and not privacy settings.
We also need proper functions with proper terminologies for content
and metadata access.
We've also converted the function to check for permission to unsubscribe
others to accept a list of streams instead of checking each stream one
by one.
Earlier, we were passing the whole subscription object to the function
in order to check if the user was subscribed or not. In the future
commits, we want to check that without fetching and passing the complete
subscription object and this commit will help us do that.
Instead of referring to users being added or removed from channels,
we now use subscribe or unsubscribed from channels.
Splits the article for adding and removing users from a channel
into separate articles: /help/subscribe-users-to-a-channel and
/help/unsubscribe-users-from-a-channel.
The URL redirects for the combined add/remove articles (for both
channel and stream terminology) go to the subscribe users to a
channel article.
For some cases, if the function called for testing events
changes some realm property using the realm object other
than the one derived from self.user_profile, then while
matching the states, self.user_profile still has the
old realm object.
Added `enable_guest_user_dm_warning` setting to decide whether
clients should show a warning when a user is composing to a guest
user in the organization.
Fixes#30078.
Co-authored-by: adnan-td <generaladnan139@gmail.com>
