mirror of
https://github.com/zulip/zulip.git
synced 2025-10-23 04:52:12 +00:00
tornado: Limit the width of the user queries, when they're needed.
Tornado requests try hard to not make SQL queries -- and if they're necessary, to minimize the number of them. Specifically, both session objects and user objects are cached in memcached, and we expect that both of them will have been filled there by Django before any requests are made to Tornado. In the event that memcached is flushed, or data is otherwise evicted, we perform two database queries -- one for the session, and one for the user. However, the *width* of the latter query has grown significantly over time, as the Realm object grew more fields, and recently with the addition of role groups, which require multiple joins each. This leads to a query which is over 12k of text long, and results in 319 columns. In the event of a memcached flush, this can result in a *significant* amount of SQL traffic, as nearly every active Tornado request will make that query. We do not wish to narrow the default query for Django; we instead tag the request in the REST wrapper, and use that to use a much narrower user cache entry. That narrower cache entry is filled before the queue is created in Django; we also use it to explicitly set the log data, so the second "half" of the continued Tornado request does not need to fetch any user data either when writing its log line. Because they use different cache keys, this only affects the session-based `/json/events` endpoint, which caches by user-id; the `/api/v1/events` endpoint, which uses an API-key cache, keeps its wide user object. The former is 50% of the total request volume, whereas the latter is only 2%, so adding an additional cache for it is unnecessary complexity.
This commit is contained in:
committed by
Tim Abbott
parent
7c20f1d3ea
commit
58bf2a7935
@@ -523,7 +523,11 @@ v1_api_and_json_patterns = [
|
||||
# used to register for an event queue in tornado
|
||||
rest_path("register", POST=(events_register_backend, {"allow_anonymous_user_web"})),
|
||||
# events -> zerver.tornado.views
|
||||
rest_path("events", GET=get_events, DELETE=cleanup_event_queue),
|
||||
rest_path(
|
||||
"events",
|
||||
GET=(get_events, {"narrow_user_session_cache"}),
|
||||
DELETE=(cleanup_event_queue, {"narrow_user_session_cache"}),
|
||||
),
|
||||
# Used to generate a Zoom video call URL
|
||||
rest_path("calls/zoom/create", POST=make_zoom_video_call),
|
||||
# Used to generate a BigBlueButton video call URL
|
||||
|
Reference in New Issue
Block a user