07c4291749
The previous query suffered from bad corner cases when the user had received a large number of direct messages but sent very few, comparatively. This mean that the first half of the UNION would retrieve a very large number of UserMessage rows, requiring fetching a large number of Message rows, merely to throw them away upon determining that the recipient was the current user. Instead of merging two queries of "last 1k received" + "last 1k sent", we instead make better use of the UserMessage rows to find "last 1k sent or received." This may change the list of recipients, as large disparities in sent/received messages may result in pushing the most-recently-sent users off of the list. These are likely uncommon edge cases, however -- and the disparity is the whole reason for the performance problem. This also provides more correct answers. In the case where a user's 1001'th message sent was to person A today, but my most recent message received was from them yesterday, the previous plan would show the message I received yesterday message-id as the max, and not the more recent message I sent today. While we could theoretically raise the `RECENT_CONVERSATIONS_LIMIT` to more frequently match the same recipient list as previously, this increases the cost of the most common cases unreasonably. With a 1000-message limit, the common cases are slightly faster, and the tail latencies are very much improved; raising `RECENT_CONVERSATIONS_LIMIT` would increase the result similarity to the old algorithm, at the cost of the p50 and p75. | | Old | New | | ------ | ------- | ------- | | Mean | 0.05287 | 0.02520 | | p50 | 0.00695 | 0.00556 | | p75 | 0.05592 | 0.03351 | | p90 | 0.14645 | 0.08026 | | p95 | 0.20181 | 0.10906 | | p99 | 0.30691 | 0.16014 | | p99.9 | 0.57894 | 0.19521 | | max | 22.0610 | 0.22184 | On the whole, however, the much more bounded worst case are worth the small changes to the resultset.
Zulip overview
Zulip is an open-source team collaboration tool with unique topic-based threading that combines the best of email and chat to make remote work productive and delightful. Fortune 500 companies, leading open source projects, and thousands of other organizations use Zulip every day. Zulip is the only modern team chat app that is designed for both live and asynchronous conversations.
Zulip is built by a distributed community of developers from all around the world, with 74+ people who have each contributed 100+ commits. With over 1000 contributors merging over 500 commits a month, Zulip is the largest and fastest growing open source team chat project.
Come find us on the development community chat!
Getting started
-
Contributing code. Check out our guide for new contributors to get started. We have invested in making Zulip’s code highly readable, thoughtfully tested, and easy to modify. Beyond that, we have written an extraordinary 150K words of documentation for Zulip contributors.
-
Contributing non-code. Report an issue, translate Zulip into your language, or give us feedback. We'd love to hear from you, whether you've been using Zulip for years, or are just trying it out for the first time.
-
Checking Zulip out. The best way to see Zulip in action is to drop by the Zulip community server. We also recommend reading about Zulip's unique approach to organizing conversations.
-
Running a Zulip server. Self-host Zulip directly on Ubuntu or Debian Linux, in Docker, or with prebuilt images for Digital Ocean and Render. Learn more about self-hosting Zulip.
-
Using Zulip without setting up a server. Learn about Zulip Cloud hosting options. Zulip sponsors free Zulip Cloud Standard for hundreds of worthy organizations, including fellow open-source projects.
-
Participating in outreach programs like Google Summer of Code and Outreachy.
-
Supporting Zulip. Advocate for your organization to use Zulip, become a sponsor, write a review in the mobile app stores, or help others find Zulip.
You may also be interested in reading our blog, and following us on Twitter and LinkedIn.
Zulip is distributed under the Apache 2.0 license.