171 Commits

Author SHA1 Message Date
Anders Kaseorg
40a022dcc3 zephyr: Remove Zephyr mirroring support.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-16 11:18:18 -07:00
Mateusz Mandera
51cef01c29 message: Use .is_channel_message column instead of is_stream_message().
This avoids a potential unnecessary message.recipient fetch required by
is_stream_message(). is_stream_message() methods precedes the addition
of the denormalized is_channel_message column and is now unnecessary.

In practice, we usually fetch Message objects with `.recipient` already,
so I don't expect any notable performance impact here - but it's still a
useful change to make.
2025-08-19 23:37:07 -07:00
Alex Vandiver
5319a2f33e narrow: Skip topic mutes for DM narrows. 2025-07-22 20:57:36 -07:00
akshatdalton
520c58bf6b narrow: Fix topic highlighting issue with apostrophes in search results.
This commit addresses the issue where the topic highlighting
in search results was offset by one character when an
apostrophe was present. The problem stemmed from the disparity
in HTML escaping generated by the function `func.escape_html` which
is used to obtain `topic_matches` differs from the escaping performed
by the function `django.utils.html.escape` for apostrophes (').

func.escape_html | django.utils.html.escape
-----------------+--------------------------
      &#39;      |           &#x27;

To fix this SQL query is changed to return the HTML-escaped
topic name generated by the function `func.escape_html`.

Fixes: #25633.

Co-authored-by: Pieter CK <pieterceka123@gmail.com>
2025-07-16 09:26:40 -07:00
PieterCK
29d42ab6ad url_encoding: Refactor a encode_user_ids.
This refactors out a function to encode the user ids into URL compatible
format. Previously we use the "-pm" decorator to encode user ids for
group direct messages. That decoration tag is not valid, so this also
updates some existing test cases.
2025-07-03 14:40:49 -07:00
Kislay Verma
6bd6f23549 url_encoding: Add support for with message links.
We abstract away "near" vs "with" from the function names and
allow callers to specify whether they want a conversation_link,
ie, use the "with" operator. The default choice is "near".
2025-06-30 11:19:00 -07:00
Mohammad Reza Kianifar
a0488715f3 direct_messages: Use DM groups for 1:1 or self DMs if present.
This is a preparatory refactor for migrating the internal structure of
Recipient objects for group DMs to use the DirectMessageGroup type,
not the legacy PERSONAL type. This step has the message-sending code
path check if a DirectMessageGroupe exists and prefer it if available.

It should have no effect in production other than doing a useless
database query for each outgoing DM, since we do not at present ever
create such DirectMessageGroup objects. (It will not add a marginal
database query once the migration is complete, just during this
transition).
2025-05-18 23:58:59 -07:00
Anders Kaseorg
ad31ef22f2 narrow: Fix get_base_query_for_search access restrictions.
The type_id is the id of a UserProfile, Stream, or DirectMessageGroup,
not the id of a type.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-04-10 15:26:57 -07:00
Alex Vandiver
d978363a75 topic: Enforce is_channel_message=True on topic queries. 2025-03-18 09:34:11 -07:00
Anders Kaseorg
cb0560d734 narrow: Add access restrictions to get_base_query_for_search.
This mirrors the restrictions in
zerver.lib.message.bulk_access_stream_messages_query, in order to
prevent leftover UserMessage rows from granting access to messages the
user was previously allowed to access but no longer is.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-03-14 16:24:34 -07:00
Anders Kaseorg
dc22b5d4bf narrow: Remove get_base_query_for_search need_message optimization.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-03-14 16:24:34 -07:00
Tim Abbott
887e3faa31 narrow: Fix -in:home logic when nothing is muted.
This didn't have test coverage, and was incorrectly a noop.
2025-03-11 23:18:07 -07:00
opmkumar
4f462970e4 search: Add is-muted search operator.
Add the `is:muted` search operator.
`-is:muted` is an alias for the `in:home` operator.

Co-authored-by: Kenneth <Kenneth012004@outlook.com>

Fixes #16943
2025-03-11 23:18:07 -07:00
opmkumar
1d33214b55 narrow: Allow spectators to use is:resolved.
Fixes #33568.
2025-03-04 11:37:21 -08:00
PieterCK
c861bdd494 narrow_term: Refactor a NeverNegatedNarrowTerm.
Previously `NarrowTerm` is only used in our event-handling paths and to
a lesser extent in the `detect_narrowed_window` in `view/home.py`. Both
of which haven't yet support or handle the `negated` term.

Since we're planning to parse a narrow URL into narrow terms (like in
`hash_util.ts`) in the web app, we're going to need a `NarrowTerm`
dataclass with all three flags.

This commit adds the `negated` term to `NarrowTerm` and adds a
`NeverNegatedNarrowTerm` which is a subclass of `NarrowTerm` that always
has the `negated` flag as `False`, so functionally it is the same as the
current `NarrowTerm`.
2025-02-24 15:37:40 -08:00
Abhay Upadhyay
f7ac7b885a narrow: Handle use of -is:dm with channels:public.
Made maybe_negated a required parameter in
check_not_both_channel_and_dm_narrow so that it doesn't raise error
when -is:dm is called with channels:public or
when -channels:public is called with is:dm
Fixes #33033
2025-01-29 16:26:36 -08:00
Sahil Batra
92376fc133 message_edit: Rename get_message_edit_request_object.
"build_message_edit_request" is a better name for the function.
2025-01-22 09:11:47 -08:00
Sahil Batra
37f2c5bc78 message_edit: Refactor message edit code.
We add two dataclasses which stored the info for what
needs to be changed the original values to make the
code easy to read.
2025-01-21 15:33:45 -08:00
Prakhar Pratyush
5d9beb3655 fetch_messages: Add support for empty topic names.
This commit is a part of the work to support empty string
as a topic name.

Previously, empty string was not a valid topic name.

Adds `allow_empty_topic_name` boolean parameter to `GET /messages`
and `GET /messages/{message_id}` endpoints to decide whether the
topic names in the fetched messages can be empty strings.

If False, the topic names in the fetched message will have the
value of `realm_empty_topic_display_name` field in `POST /register`
response replacing "" for channel messages.
2025-01-07 17:24:00 -08:00
Anders Kaseorg
e2d56db2a3 message_cache: Use the sender’s recipient_id for incoming 1:1 DMs.
For an incoming 1:1 DM, the recipient’s own recipient_id is useless to
the recipient themselves. Substitute the sender’s recipient_id, so the
recipient can use recipient_id as documented to uniquely represent the
set of 2 users in this conversation.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-12-11 12:05:03 -08:00
Aman Agrawal
4b82c3258f narrow: Fix unmuted and followed topics not visible in combined feed. 2024-12-08 18:34:28 -08:00
Anders Kaseorg
76df4b2574 message_cache: Require can_access_sender for finalize_payload.
An access control parameter should never be optional with a default of
open.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-12-05 12:07:22 -08:00
roanster007
8aad10b720 narrow: Update with operator behaviour when message not accessible.
Previously, when the message of the "with" operator can not be
accessed by the user, it used to return messages in combined feed
or stream feed.

This commit updates it to rather raise a BadNarrowOperatorError
if the message of "with" operand is not accessible to user, and the
narrow terms present can not define a conversation. However, if
the narrow terms can define a conversation, then the narrow falls
back to that of without the "with" operator.
2024-12-05 12:03:55 -08:00
whilstsomebody
8356a9d9e4 narrow_filter: Add negation support for in:home narrow filter.
The in:home narrow filter is used to filter messages that appear
in the home view, i.e., messages that are not muted. Conversely,
`-in:home` should filter messages that are not in the home view,
i.e., muted messages. However, `-in:home` did not work as expected
because this filter lacked negation support, unlike similar code
paths.

This commit adds negation support for the in:home filter.

For more information, see:
<https://chat.zulip.org/#narrow/channel/378-api-design/topic/mark.20muted-topic.20messages.20as.20read/near/1980534>.
2024-11-14 15:10:01 -08:00
Aman Agrawal
3f726e25e4 message_fetch: Add message_ids parameter to /messages request.
This allows us to fetch messages for a list of message ids in a
single request.
2024-10-07 11:00:40 -07:00
Tim Abbott
31623911d1 test_message_fetch: Avoid assumptions about fixture data set. 2024-08-23 17:15:35 -07:00
Alex Vandiver
2bfdccd065 narrow: Do not error when given an empty DM recipient list. 2024-08-14 10:39:40 -07:00
Anders Kaseorg
b96feb34f6 ruff: Fix SIM117 Use a single with statement with multiple contexts.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-14 13:48:32 -07:00
Anders Kaseorg
0fa5e7f629 ruff: Fix UP035 Import from collections.abc, typing instead.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg
531b34cb4c ruff: Fix UP007 Use X | Y for type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg
e08a24e47f ruff: Fix UP006 Use list instead of List for type annotation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Aman Agrawal
70be9e8c51 narrow: Implement rendering of with narrow operators.
Adds server and web app support for processing the new `with`
search operator.

Fixes part of #21505.

Co-authored-by: roanster007 <rohan.gudimetla07@gmail.com>
Co-authored-by: Tim Abbott <tabbott@zulip.com>
2024-07-13 07:18:24 -07:00
Alex Vandiver
79f858b4b8 upload: Pass bytes to create_attachment.
This will be used to analyze the bytes for image metadata.
2024-07-07 14:40:07 -07:00
Alex Vandiver
f97a30f240 upload: Reorder arguments to parallel upload_message_attachment. 2024-07-07 14:40:07 -07:00
Alex Vandiver
0a296b2a6e upload: Start storing content-type for new uploads. 2024-07-07 14:40:07 -07:00
roanster007
52692a6448 refactor: Rename huddle to direct_message_group in non API.
This commit performs a sweep on the first batch of non API
files to rename "huddle" to "direct_message_group`.

It also renames variables and methods of type -
"huddle_message" to "group_direct_message".

This is a part of #28640
2024-07-04 07:56:31 -07:00
Alex Vandiver
17fb23746f upload: Move methods into zerver.lib.upload from .base. 2024-06-26 16:43:11 -07:00
Kenneth Rodrigues
15accfc983 narrow: Use NarrowParameter instead of dictionary.
Earlier we were using the type `OptionalNarrowListT` for all functions
that required "narrow" as a parameter.
This commit changes all the functions accepting a "narrow"
to use a list of the new `NarrowParameter`
instead of `OptionalNarrowListT` which is a list of dicts.
2024-06-20 12:22:37 -07:00
Kenneth Rodrigues
4e05381897 message_fetch: Migrate to typed_endpoint.
Removed the old `narrow_parameter` as we have shifted to
the new `NarrowParameter` Pydantic Object.
This new object provides better error messages for data validation,
hence changed the error messages in `test_message_fetch`.
2024-06-20 12:22:37 -07:00
Kenneth Rodrigues
f791d1e1c0 test_message_fetch: Add support for list of error messages. 2024-06-20 12:22:37 -07:00
Kenneth Rodrigues
a865977bd5 get_messages_backend: Pass required parameters.
Earlier `num_before` and `num_after` wasn't being passed while
testing.
Moving to typed_endpoint requires all the "required" parameters
to be passed to the function explicitly in tests.
2024-06-20 12:22:37 -07:00
nimish
81ea09be19 search: Add is:followed filter.
Create the is:followed search operator.
Fetch all messages that are from followed topics
using exists.
Update API documentation and changelog.

Co-authored-by: Kenneth Rodrigues <kenneth.nrk123@gmail.com>

Fixes #27309.
2024-06-17 18:22:50 -07:00
roanster007
384a43c79c narrow: Fix server error of operand of "id" operator at large values.
Previously, when the operand of id operator was more than
2147483647, it was raising server error. This is because the
maximum permissible PostgreSQL integers value is 2147483647.

This is fixed by raising a BadNarrowOperatorError in case the
id operand is larger than 2147483647.
2024-06-01 22:38:18 -07:00
Kenneth Rodrigues
ba92cd8b23 search: Include history for is:resolved queries.
The previous logic assumed that all `is:` operators were those like `is:muted`
or `is:starred`, which depend on personal data.
2024-05-20 14:37:08 -07:00
Anders Kaseorg
38cae070ed typos: Fix typos caught by typos.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-05-20 13:55:00 -07:00
Prakhar Pratyush
fd335c06eb test_message_fetch: Fix a flaky test due to random message populated.
The messages populated in the test database are random in nature,
so there is a possibility of more than one resolved topic
messages returned by 'self.get_and_check_messages()' query
in 'test_get_messages_for_resolved_topics'.

That would result in a flaky behavior. This commit fixes the
flaky behavior.
2024-05-09 14:05:35 -07:00
Alex Vandiver
16a08a8b6b narrow: Do not create huddles when fetching messages.
9a682fb20a started performing message fetching in a read-only
transaction.  However, our use of `get_or_create_huddle` can violate
the read-only promise, and result in a user-facing 500.

In the cases where we're attempting to narrow to a huddle that does
not exist, this is equivalent to a false condition; catch those,
without making the huddle row, and insert a false.
2024-04-16 09:57:55 -07:00
Alex Vandiver
823f732132 narrow: Move python-only implemenation into its own file.
Tornado uses this, and none of the heavy-weight SQLAlchemy code; so it
just adds to the startup time of the Tornado server.
2024-04-16 09:48:11 -07:00
Lauryn Menard
608b30577a narrow: Add backend support for "channels" operator.
Adds backend support for "channels" operator.

This will deprecate/replace the "streams" operator eventually, but
we will keep support of the operator for backwards compatibility
for a while.

Part of renaming stream to channel project.
2024-04-12 13:44:25 -07:00
Lauryn Menard
0e972e26da narrow: Add backend support for "channel" operator.
Adds backend support for "channel" operator.

This will deprecate/replace the "stream" operator eventually, but
we will keep support of the operator for backwards compatibility
for a while.

Part of renaming stream to channel project.
2024-04-12 13:44:25 -07:00