Commit Graph

138 Commits

Author SHA1 Message Date
Alex Vandiver
721fd26442 send_email: Set the Date header according to local enqueue time.
Email clients tend to sort emails by the "Date" header, which is not
when the email was received -- emails can be arbitrarily delayed
during relaying.  Messages without a Date header (as all Zulip
messages previously) have one inserted by the first mailserver they
encounter.  As there are now multiple email-sending queues, we would
like the view of the database, as presented by the emails that are
sent out, to be consistent based on the Date header, which may not be
the same as when the client gets the email in their inbox.

Insert a Date header of when the Zulip system inserted the data into
the local queue, as that encodes when the full information was pulled
from the database.  This also opens the door to multiple workers
servicing the email_senders queue, to limit backlogging during large
notifications, without having to worry about inconsistent delivery
order between those two workers.

Messages which are sent synchronously via `send_email()` get a Date
header of when we attempt to send the message; this is, in practice,
no different from Django's default behaviour of doing so, but makes
the behaviour slightly more consistent.
2025-03-10 16:48:08 -07:00
Shubham Padia
3fb2695a32 message: Add is_modifying_message argument to bulk_access_messages.
Although, currently there are no scenarios where we are using
bulk_access_messages for edit. But we might do so in the future, and
it's better to have an explicit argument called is_modiying_message in
that case, so that the person making that change makes a conscious
decision of setting that property.
2025-02-28 10:10:33 -08:00
Prakhar Pratyush
9ab6729d41 missedmessage_emails: Show EMPTY_TOPIC_FALLBACK_NAME for topic="".
This commit adds support to display `Message.EMPTY_TOPIC_FALLBACK_NAME`
value (translated) in the missedmessage email body and subject for
topics having the actual value of empty string.

Fixes part of #32996.
2025-01-27 13:08:19 -08:00
Prakhar Pratyush
4bef1a510c handle_missedmessage_emails: Update codepath to queue event on commit.
Earlier, in 'handle_missedmessage_emails' codepath we were using
'queue_json_publish' which can lead to a situation where we enqueue
events but the transaction fails at a later stage.

Events should not be published until we know we're not rolling back.
2024-12-04 12:14:23 -08:00
Anders Kaseorg
42e1517255 email_notifications: Prevent html2text from mangling Unicode.
html2text mangles Unicode by default, with a --unicode-snob option to
disable it.  If I have to get called a “snob” for wanting to correctly
support non-English languages, then uh, I’ll take one for the team.

https://github.com/Alir3z4/html2text/blob/2024.2.26/html2text/config.py#L111-L150

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-10-30 09:49:23 -07:00
Lauryn Menard
f9de3f9a45 onboarding: Use "Moving to Zulip" guide in emails & Welcome bot message.
Replaces links to "Getting your organization started with Zulip"
in onboarding emails and Welcome bot direct message for owners of
new organizations.

Revises text in those emails and messages to reflect the new
"Moving to Zulip" help center guide that is now used.
2024-09-30 11:58:31 -07:00
Tim Abbott
9b67164270 email_notifications: Fix emoji being giant in Outlook.
Apparently, Outlook ignores height/width CSS rules, but does support
the attribute on the image element itself, so specify that instead.

I don't think there are likely to be image tag implementations that
don't support the attribute, given that's the only thing that works in
Outlook.
2024-09-26 11:58:12 -07:00
Anders Kaseorg
48202389b8 ruff: Bump target-version from py38 to py310.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg
531b34cb4c ruff: Fix UP007 Use X | Y for type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg
e08a24e47f ruff: Fix UP006 Use list instead of List for type annotation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
roanster007
52692a6448 refactor: Rename huddle to direct_message_group in non API.
This commit performs a sweep on the first batch of non API
files to rename "huddle" to "direct_message_group`.

It also renames variables and methods of type -
"huddle_message" to "group_direct_message".

This is a part of #28640
2024-07-04 07:56:31 -07:00
Vector73
8ab526a25a models: Replace realm.uri with realm.url.
In #23380, we are changing all occurrences of uri with url in order to
follow the latest URL standard. Previous PRs #25038 and #25045 has
replaced the occurences of uri that has no direct relation with realm.

This commit changes just the model property, which has no API
compatibility concerns.
2024-05-08 11:12:43 -07:00
Lauryn Menard
93dd1e9b25 help: Rename and redirect move-content-to-another-stream for channel. 2024-05-03 12:55:39 -07:00
Anders Kaseorg
96fbe060a6 python: Mark regexes as raw strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-26 12:30:31 -07:00
Prakhar Pratyush
c727b36e9c soft_reactivate: Soft reactivate if group mention has < 12 members.
Earlier, we didn't soft-reactivate users for group mentions
at all because it wasn't easy to calculate group size.

Now, we will soft reactivate if the user group mentions has
less than 12 members.

We don't reactivate all users because a user group can have a
very large size, which can lead to large backlogs in the
deferred-work queue.

Fixes part of #27586.
2024-04-24 17:40:37 -07:00
Prakhar Pratyush
43136abb70 push_notification: Use 'get_mentioned_user_group' function.
This commit updates the code in 'handle_push_notifications'
to use a common lib function 'get_mentioned_user_group'.

The code logically does the same thing in both the places,
hence the change.
2024-04-24 17:40:37 -07:00
Lauryn Menard
acb54a8bc5 emails: Update context/content for stream to channel rename.
In the missed message email, updates "stream_name" context variable
to be "channel_name". This varible is part of some translated
strings in the email's content.

In onboarding zulip topics email, updates content references to
streams to use channels instead for translated strings. Also,
updates "move_topic_to_different_stream_link" context variable to
be "move_channels_link".

Does not yet update the image used in the onboarding topics email
as that will be updated as part of the help center changes for
this rename.

Part of stream to channel rename project.
2024-04-24 14:35:05 -07:00
Anders Kaseorg
6e871e7731 ruff: Fix UP036 Version block is outdated for minimum Python version.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-01 18:32:52 -07:00
John Lu
a5cf0ec526 refactor: Replace HUDDLE with DIRECT_MESSAGE_GROUP.
Replaced HUDDLE attribute with DIRECT_MESSAGE_GROUP using VS Code search,
part of a general renaming of the object class.

Fixes part of #28640.

Co-authored-by: JohnLu2004 <JohnLu10212004@gmail.com>
2024-03-21 16:39:33 -07:00
Anders Kaseorg
59b0548433 timezone: Only look up canonical time zones from the system.
Legacy time zone aliases were removed from the Debian tzdata package
in tzdata 2023c-8.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1040997

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-01 17:38:08 -08:00
Anders Kaseorg
93198a19ed requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-29 10:41:54 -08:00
Prakhar Pratyush
b7e56ccbdc lib: Rename *topic local variables to *topic_name.
This is preparatory work towards adding a Topic model.
We plan to use the local variable name as 'topic' for
the Topic model objects.

Currently, we use *topic as the local variable name for
topic names.

We rename local variables of the form *topic to *topic_name
so that we don't need to think about type collisions in
individual code paths where we might want to talk about both
Topic objects and strings for the topic name.
2024-01-15 09:40:43 -08:00
Anders Kaseorg
c9c819e1d7 models: Extract zerver.models.scheduled_jobs.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg
b15999c799 models: Extract zerver.models.messages.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg
3c11fd9466 models: Move some functions to zerver.lib.display_recipient.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg
45bb8d2580 models: Extract zerver.models.users.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Prakhar Pratyush
d8cf12eaaa send_email: Improve configurability for outgoing email sender name.
Currently, the sender names for outgoing emails sent by Zulip
are hardcoded. It should be configurable for self-hosted systems.

This commit makes the 'Zulip' part a variable in the following
email sender names: 'Zulip Account Security', 'Zulip Digest',
and 'Zulip Notifications' by introducing a settings variable
'SERVICE_NAME' with the default value as f"{EXTERNAL_HOST} Zulip".

Fixes: #23857
2023-11-29 14:20:01 -08:00
Prakhar Pratyush
6f3b25d749 push_notification: Fix bad rendering of math formulas.
Earlier, for the push notifications having latex math
like "$$1 \oplus 0 = 1$$, the notification had the math
included multiple times.

This commit fixes the incorrect behavior by replacing
the KaTeX with the raw LaTeX source.

Fixes part of #25289.
2023-11-26 23:30:24 -08:00
Prakhar Pratyush
f7f5131aa8 email_notification: Fix bad rendering of math formulas.
Earlier, for the emails having latex math like
"$$d^* = +\infty$$", the bad rendering led to the math
being included multiple times in the email body.

This was due to displaying KaTeX HTML without the CSS.

This commit fixes the incorrect behavior by replacing
the KaTeX with the raw LaTex source.

Fixes part of #25289.
2023-11-20 10:41:02 -08:00
Prakhar Pratyush
94679d590f email_notification: Include prior message context only when mentioned.
Earlier, email message notifications included prior messages sent
to the same topic for context. This is more confusing than helpful
for messages that the user is likely to have received notifications
for all the prior messages in the conversation already (or read them
in the Zulip UI).

Now, we include prior context only when the user is mentioned via
personal, group, stream or topic wildcard mention.

Fixes #27479.
2023-11-07 11:44:38 -08:00
Lauryn Menard
02d6b3e16d onboarding-emails: Add new onboarding email for organization creator.
Adds a new onboarding email `onboarding_team_to_zulip` for the user
who created the new Zulip organization.

Co-authored by: Alya Abbott <alya@zulip.com>
2023-10-25 13:51:03 -07:00
Esther Anierobi
e957decd12 email: Drop support for SEND_MISSED_MESSAGE_EMAILS_AS_USER. (#27223)
Originally, this was how the notification emails worked, but that was changed
in 797a7ef97b, with this old behavior 
available as an option.

The footer and from address of emails that are sent when this
setting is set to True are confusing, especially when more people
are involved in a stream and since we have changed the way we send
emails, it should be removed. It’s also not widely used.

Fixes #26609.
2023-10-20 16:38:43 -07:00
Alex Vandiver
ffb6c95bba email_notifications: Make stream_id_map optional to build_message_list.
This feels cleaner than passing an empty dict.
2023-09-13 13:25:59 -07:00
Alex Vandiver
f8a9779b54 digest: Rename get_slim_stream_map slightly and explain its name more. 2023-09-13 13:25:59 -07:00
Alex Vandiver
b94402152d models: Always search Messages with a realm_id or id limit.
Unless there is a limit on `id`, always provide a `realm_id` limit as
well.  We also notate which index is expected to be used in each
query.
2023-09-11 15:00:37 -07:00
Lauryn Menard
11adc0f37d demo-organizations: Rename shortend versions of 'demo organization'.
Rename existing shortened references to demo organizations, like
`is_demo_org` or `demo-org-warning`, that have been used in the
codebase so far and replace them to be like the `models.py`
variable: `Realm.demo_organization_scheduled_deletion_date`.
2023-09-08 15:17:23 -07:00
Eeshan Garg
5e33ae8adf demo-orgs: Create dev environment demo organization without email.
To make creation of demo organizations feel lightweight for users,
we do not want to require an email address at sign-up. Instead an
empty string will used for the new realm owner's email. Currently
implements that for new demo organizations in the development
environment.

Because the user's email address does not exist, we don't enqueue
any of the welcome emails upon account/realm creation, and we
don't create/send new login emails.

This is a part of #19523.

Co-authored by: Tim Abbott <tabbott@zulip.com>
Co-authored by: Lauryn Menard <lauryn@zulip.com>
2023-08-31 15:02:16 -07:00
Lauryn Menard
5e29e025c5 email-templates: Add zulip_onboarding_topics email templates.
The "followup_day2" email template name is not clear or descriptive
about the purpose of the email. Creates a duplicate of those email
template files with the template name "zulip_onboarding_topics".

Because any existing scheduled emails that use the "followup_day2"
templates will need to be updated before the current templates can
be removed, we don't do a simple file rename here.
2023-08-18 16:25:48 -07:00
Lauryn Menard
c491bef07b email-templates: Add account_registered email templates.
The "followup_day1" email template name is not clear or descriptive
about the purpose of the email. Creates a duplicate of those email
template files with the template name "account_registered".

Because any existing scheduled emails that use the "followup_day1"
templates will need to be updated before the current templates can
be removed, we don't do a simple file rename here.
2023-08-18 16:25:48 -07:00
Steve Howell
f8ec00b895 mypy: Improve type checks for user display recipients. 2023-08-10 18:13:43 -07:00
Steve Howell
257b32a4a4 narrow urls: Avoid complicated optional types.
We no longer have to reason about the 12 possible
ways of invoking get_narrow_url. We also avoid
double computation in a couple places.

Finally, we get stricter type checks by just inlining
the calls.
2023-08-10 18:13:43 -07:00
Prakhar Pratyush
af648833f2 notifications: Remove the stray notification trigger strings.
This commit removes the stray strings used to refer to
various types of notification triggers.

We use the attributes of the 'NotificationTriggers' class instead.
2023-07-24 11:02:14 -07:00
Prakhar Pratyush
4c9d26ce17 mention: Send notifications for @topic wildcard mentions.
This commit completes the notifications part of the @topic
wildcard mention feature.

Notifications are sent to the topic participants for the
@topic wildcard mention.
2023-07-17 09:39:24 -07:00
Alex Vandiver
d87895a3ef missed-message: Merge before calling handle_missedmessage_emails.
The MissedMessage queue worker is the single callsite of
`handle_missedmessage_emails`, which immediately transforms the list
of events into a dict keyed by message-id.

Skip the intermediate list step, and use defaultdict and a dataclass
to simplify and make explicit the pieces.  This removes the unused
user_profile_id and message_id pieces of the data structure.
2023-07-13 11:50:42 -07:00
Lauryn Menard
3dfdbbc775 welcome-emails: Separate followup_day1 email from other welcome emails.
The initial followup_day1 email confirms that the new user account
has been successfully created and should be sent to the user
independently of an organization's setting for send_welcome_emails.

Here we separate out the followup_day1 email into a separate function
from enqueue_welcome_emails and create a helper function for setting
the shared welcome email sender information.

The followup_day1 email is still a scheduled email so that the initial
account creation and log-in process for the user remains unchanged.

Fixes #25268.
2023-07-11 14:15:52 -07:00
Prakhar Pratyush
179d5cb37d mention: Replace 'wildcards' with 'stream_wildcards'.
This prep commit replaces the 'wildcard' keyword in the codebase
with 'stream_wildcard' at some places for better readability, as
we plan to introduce 'topic_wildcards' as a part of the
'@topic mention' project.

Currently, 'wildcards = ["all", "everyone", "stream"]' which is an
alias to mention everyone in the stream, hence better renamed as
'stream_wildcards'.

Eventually, we will have:
'stream_wildcard' as an alias to mention everyone in the stream.
'topic_wildcard' as an alias to mention everyone in the topic.
'wildcard' refers to 'stream_wildcard' and 'topic_wildcard' as a whole.
2023-07-03 22:03:17 -07:00
Prakhar Pratyush
f7e41499fe notifications: Fix the 'senders' list.
This commit fixes the incorrect calculation of the
'senders' list.

The effect of 'followed_topic_wildcard_mention'
wasn't considered earlier.

The bug was introduced in b052c8980e.
2023-07-03 22:03:17 -07:00
Prakhar Pratyush
bf673546cb notifications: Eliminate the stale context variable 'mention_count'.
This commit removes the context variable 'mention_count',
which is not being used anywhere.
2023-07-03 22:03:17 -07:00
Prakhar Pratyush
e12cd9862a notification_triggers: Reduce loose string usage.
This commit uses 'NotificationTriggers' class attributes
instead of directly using loose strings.

This should have been ideally included in the commit
c3319a5231.
2023-07-03 22:03:17 -07:00
Lauryn Menard
d3f7cfccbc zerver: Update comments with "private message" or "PM".
Updates comments/doc-strings that use "private message" or "PM" in
files in the `/zerver` directory to instead use "direct message".
2023-06-23 11:24:13 -07:00