Commit Graph

7341 Commits

Author SHA1 Message Date
Anders Kaseorg
ee2cb855f0 message_fetch: Add include_anchor parameter.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-16 21:13:44 -08:00
Anders Kaseorg
dae4633745 message_fetch: Extract fetch_messages helper to zerver.lib.narrow.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-16 21:13:44 -08:00
Anders Kaseorg
bd5d200722 narrow: Return a dataclass from post_process_limited_query.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-16 21:13:44 -08:00
Anders Kaseorg
0a0a70b33d message_fetch: Move limit_query_to_range to zerver.lib.narrow.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-16 21:13:44 -08:00
Anders Kaseorg
1095efeb52 message_fetch: Move parse_anchor_value to zerver.lib.narrow.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-16 21:13:44 -08:00
Anders Kaseorg
4eea5e102e message_fetch: Move ok_to_include_history to zerver.lib.narrow.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-16 21:13:44 -08:00
Anders Kaseorg
1b016bef73 ruff: Enable naming errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-16 09:29:11 -08:00
Anders Kaseorg
924d530292 ruff: Fix N813 camelcase imported as lowercase.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-16 09:29:11 -08:00
Anders Kaseorg
8f17093be2 ruff: Fix N812 lowercase imported as non-lowercase.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-16 09:29:11 -08:00
Anders Kaseorg
2876ae8e48 ruff: Fix N803 argument name should be lowercase.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-16 09:29:11 -08:00
Anders Kaseorg
46955da3a0 ruff: Fix ANN204 missing return type annotation for __init__.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-16 09:29:11 -08:00
Lauryn Menard
9aacb15530 tabbed-sections: Update "via-user-profile" to be "via-user-card". 2022-11-14 12:16:22 -08:00
Alex Vandiver
1f18fa6580 register: Not all clients send a User-Agent header.
In 1fce1c3c73, we added logic to parse
the User-Agent in /register requests; this logic crashed if an HTTP request
was missing that header.

Includes a test for `/register` with no user agent passed; this should catch
similar regressions in the future.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2022-11-10 16:21:04 -08:00
Zixuan James Li
4e46899494 webhooks: Pick a more reasonable length for short sha.
7 characters are not enough for large projects, so we change
it to reasonably longer. As an example, The Linux kernel needs
at least 11 characters of sha in its shortened form to identify
a revision. We pick 11 so it should work for most of the projects.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-11-08 08:20:51 -08:00
Lauryn Menard
19a4c3907f decorator: Put back check for dev env in ignore_unhashable_lru_cache.
Prior to 53231aa, the `ignore_unhashable_lru_cache` decorator had
a check for the development environment so that changes could be
seen on refresh.

Puts that check back in IgnoreUnhashableLruCacheWrapper class.
2022-11-07 12:17:59 -08:00
Alex Vandiver
d8ebbedbbb outgoing_webhook: Ignore the exception on failure if the stream is gone.
In the outgoing webhook handler, there is potentially several seconds
of trying between when a message triggering an outgoing webhook
arrives, and when it fails.  In the meantime, the stream the
triggering message was on may have been deleted, causing the
"Failure!" message to have no valid stream to be sent to.

Rather than raise an exception in the outgoing webhook worker, ignore
the exception and move on.
2022-11-04 14:46:17 -07:00
Alex Vandiver
eb7a2f2c38 queue: Do test retries in tests.
The lambda passed to `queue_json_publish` is used if
`settings.USING_RABBITMQ` is unset -- which is only true in tests.  As
such, this pattern causes failures to never actually retry within
tests.

This behaviour has existed ever since the outgoing webhook code was
introduced in 53a8b2ac87, with no explanation.  Not passing that
argument allows tests to verify the retry behaviour when webhooks
fail.
2022-11-04 14:46:17 -07:00
Zixuan James Li
7dbd98d25c rest: Restrict access to json views via basic auth.
Previously, test cases or clients accessing /json/ views using HTTP
Basic Auth would be accepted, while we intended to only allow clients
authenticated with a session cookie to access these views.

This adds a check on the accessed path to avoid this possibility.

It seems unlikely that any API clients clients were taking advantage
of this unintended quirk; so we're not going to bother documenting
this bug fix as an API change. In any case, it should be trivial for
anyone affected to consult the documentation and then switch their
/json/foo URL to a correct /api/v1/foo URL.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-11-04 14:44:07 -07:00
Sahil Batra
1fce1c3c73 register: Handle "Pronouns" type fields for older mobile clients.
Mobile clients older than v27.192 do not support PRONOUNS type
custom profile fields, so we instead change the type of it to
SHORT_TEXT in the data sent with register response and also in
the events sent to those clients.
2022-11-04 14:31:18 -07:00
M@
47ecd33d3a import: Narrow the id-window to just the current realm.
On multi-realm systems this results in traversal of all messages in
all realms and returns a massive payload of 1 row per stream on 
the server, not the intended one row per realm.
2022-11-03 12:13:43 -07:00
Anders Kaseorg
3d853caf16 ruff: Fix C417 Unnecessary map usage.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-03 12:10:15 -07:00
Anders Kaseorg
77c15547e6 ruff: Fix C414 Unnecessary list call within sorted().
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-03 12:10:15 -07:00
Anders Kaseorg
69e94b5991 ruff: Fix C413 Unnecessary list call around sorted().
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-03 12:10:15 -07:00
Mateusz Mandera
6e336ef6f6 delete_topic: Use the same timeout pattern as /mark_all_as_read.
We don't want to use the nginx 502 timeout as an API pattern. In
/mark_all_as_read we've already figured out how we want to handle this
instead.
2022-11-02 16:50:06 -04:00
Zixuan James Li
da9ad7c3a7 markdown: Pass realm down into render_stream_description.
This guarantees that the Realm is always non-None when we hit the
codepath is_static_or_current_realm_url via
do_change_stream_description, so that we can properly skip rewritting
some images.

Fixes #19405

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-11-01 08:20:47 -07:00
Lauryn Menard
0c947985bd tests: Update tests to use example profile picture.
So that we can stop using Tim's photo for tests, adds an open
license profile picture to use instead.

Updates tests that used `tim.png` to use the new example profile
picture, which is located in `static/images/test-images/avatars/`.
2022-10-31 14:36:54 -07:00
Mateusz Mandera
760587450e retention: Remove two redundant comments.
These two identical comments don't contribute anything useful and seem
just out of place at this point.
2022-10-31 10:23:57 -07:00
Mateusz Mandera
7b13204e8f retention: Use Message.realm to simplify private message query.
We no longer need to do the inner joins to figure out the message's
realm and split up the cross-realm and regular case - now we just look
at zerver_message.realm directly.
2022-10-31 10:23:57 -07:00
Mateusz Mandera
601f0677f3 update_first_visible_message_id: Use Message.realm column.
With the Message.realm column added, this query can now be improved.
2022-10-31 09:57:42 -07:00
Sahil Batra
2a8b7412ff url_encoding: Use user name instead of email in personal_narrow_url.
This commit updates the urls for personal narrow sent in email
notifications to be of form "{user_id}-{encoded_full_name}" to
make it consistent with the urls that we use for such narrows
in webapp which were recently updated in b4eddad for improving
performance. We encode the full name in the same way that we do in
webapp by replacing the url characters encoded by browser with "-".
2022-10-28 09:47:43 -07:00
Mateusz Mandera
a410f6b241 do_mark_all_as_read: Split up the work into batches.
Fixes #15403.
2022-10-27 16:59:54 -07:00
Alya Abbott
8f6cf65f28 help center: Reorganize and extend the "Logging in" help page.
Also move some instructions from related pages into shared /include files,
with minor tweaks.
2022-10-26 12:28:29 -07:00
Lauryn Menard
e66f125ee6 recent-topics: Update hash for view to be "#recent".
Updates the hash used for the recent conversations view to be
"#recent" instead of "#recent_topics".

We will need to keep the logic for handling "#recent_topics"
permanently because users potentially have messages from
Welcome Bot with links to that hash.

Including "recent_topics" as a web_public_allowed_hash in
hash_util.js can be changed once self-hosted servers cannot
upgrade directly to Zulip 5.x from the current version.

Fixes #23132.
2022-10-25 11:03:44 -07:00
Prakhar Pratyush
38aa1ee1ff test_classes: Improve assert_database_query_count output message.
Output message should talk about both the cases:
actual_count > expected_count and actual_count < expected_count.

The message now includes information for the case where
actual_query_count < expected_query_count.

Fixes: #23325
2022-10-24 10:18:58 -07:00
Lauryn Menard
c9044f7acd recent-topics: Rename to "Recent conversations" in web-app and docs.
Replaces instances of "recent topics" in the web-app and documentation
to be "recent conversations".

Renames both `recent-topics.md` files in the help center to be
`recent-conversations.md` and updates/redirects links to new URL.

Does not update instances of "recent topics" in frontend code comments
and does not update the main overview changelog, for now.

Does not change case study text where "recent topics" was referenced
in a quote, but does change generic text references to be "recent
conversations".
2022-10-21 10:28:19 -07:00
Anders Kaseorg
c5c180fda3 email_validation: Restore case-insensitive domain validation.
This was broken by commit b945aa3443
(#22604), because email_to_domain implicitly lowercased the result.

No adjustment is needed for is_disposable_domain, which already
lowercases its argument.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-20 08:19:07 -07:00
Anders Kaseorg
89e4233b68 timeout: Correct thread id type passed to PyThreadState_SetAsyncExc.
This type changed in Python 3.7:
https://github.com/python/cpython/pull/781

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-19 14:49:24 -07:00
David Rosa
64c31b7cb9 help: Rename "Add custom profile fields" -> "Custom profile fields".
Renames the help article on custom profile fields to reflect that
its content is not just about adding fields.

Adds a redirect from the old URL to the new URL and updates internal
links, linking to #add-a-custom-profile-field where appropriate.

Fixes #23170.
2022-10-19 11:58:00 -07:00
David Rosa
d9dd272a78 help: Update documentation on bots.
Adds new tab to `zerver/lib/markdown/tabbed_sections.py` to document
managing bots from both personal settings and organization settings.

Documents adding bots from the organization settings Bots panel.

Separates instructions for deactivating and reactivating a bot from
both personal settings and organization settings.

Fixes a few formatting issues such as missing bold formatting and
heading level.

Fixes: #23066.
2022-10-18 10:16:42 -07:00
Florian Pritz
f37ac80384 import_realm: Speed up first_message_id calculation.
On my data (about 10 million messages in 1600 streams) this used to take
about 40 hours, while the improved statement completes in roughly 30
seconds.

The old solution had postgres go through the entire table until the
first match for each stream. Thus, the time spent scanning the table
got longer and longer for each stream because postgres always started at
the beginning (and somehow it did not use any indices) and had to skip
over all rows until it found the first message from the stream that is
was looking for each time.

This new statement just performans a bulk operation, scanning the table
only once and then inserts the results directly into the destination
table.

Slightly more verbose inforation about this change can be found in:
https://chat.zulip.org/#narrow/stream/31-production-help/topic/Import.20Rocketchat.20data/near/1408867

Signed-off-by: Florian Pritz <bluewind@xinu.at>
2022-10-17 11:43:21 -07:00
Zixuan James Li
46329a2710 test_classes: Create a dedicate helper for query count check.
This adds a helper based on testing patterns of using the "queries_captured"
context manager with "assert_length" to check the number of queries
executed for preventing performance regression.

It explains the rationale of checking the query count through an
"AssertionError" and prints the queries captured as assert_length does,
but with a format optimized for displaying the queries in a more
readable manner.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-10-17 11:32:52 -07:00
Zixuan James Li
9117db8887 test_helpers: Tighten type annotation for queries_captured.
This adds CapturedQueryDict to provide a more accurate type annotation
for the return value of queries_captured. We also replace "Generator"
with "Iterator" because the latter two type parameters were unused.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-10-17 11:32:52 -07:00
Lauryn Menard
171da34546 events: Send empty list for custom_profile_fields in spectator view.
Send an empty list of `custom_profile_fields` in `page_params` for
spectators, rather than not sending the field at all.

Also, updates the user info popover to not show the manage user
three-dot menu when in a spectator view.
2022-10-14 13:05:35 -07:00
Alex Vandiver
c328de3372 cache: Log a warning when attempting to store a whole QuerySet.
As noted in the previous commit, this causes bloat in memcached, for
no purpose.  Log a warning when `cache_with_key` sees a QuerySet
returned from the function it is decorating.
2022-10-12 22:25:48 -07:00
Alex Vandiver
204f1b58e8 cache: Drop realm_id from realm_user_dict_fields.
Storing this key is superfluous, as it will be the same for all users,
and definitionally already known to fetch the cache for the realm.  It
is also not currently used by the callsites that read rows from the
cache.
2022-10-12 22:25:48 -07:00
Alex Vandiver
5d42a0cb00 linkifiers: Support %20 in URLs for topic links.
9381a3bd45 added support for linkifier pattern URLs containing
`%20`-style escapes, but only did so for the codepath which is used in
the message body -- topic links did not understand them.

Expand the support to include when they are substituted into topics.
2022-10-11 14:31:13 -07:00
Anders Kaseorg
6d19256877 notes: Separate __notes_map per-subclass.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:42:13 -07:00
Anders Kaseorg
1385a827c2 python: Clean up getattr, setattr, delattr calls with literal names.
These were useful as a transitional workaround to ignore type errors
that only show up with django-stubs, while avoiding errors about
unused type: ignore comments without django-stubs.  Now that the
django-stubs transition is complete, switch to type: ignore comments
so that mypy will tell us if they become unnecessary.  Many already
have.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:40:28 -07:00
Anders Kaseorg
b45484573e python: Use format string for logging str(obj).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:32:29 -07:00
Anders Kaseorg
fcd81a8473 python: Replace avoidable uses of __special__ attributes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:32:29 -07:00