Commit Graph

7965 Commits

Author SHA1 Message Date
Mateusz Mandera
f0f02d05ab send_message_backend: Remove the realm_str API param.
This already became useless in 6e11754642,
as detailed in the API changelog entry here. At this point, we should
eliminate this param and the weird code around it.

This commit also deletes the associated tests added in
6e11754642, since with realm_str removed,
they make no sense anymore (and actually fail with an OpenAPI error due
to using params not used in the API). Hypothetically they could be
translated to use the subdomain= kwarg, but that also doesn't make
sense, since at that point they'd be just testing the case of a user
making an API request on a different subdomain than their current one
and that's just redundant and already tested generally in
test_decorators.
2023-02-28 12:12:15 -08:00
Lauryn Menard
514e5b990e templates: Update reset_done.html page text and title.
Updates the text and title used when the password reset done page
to work for situations where the user is resetting a forgotten
password and for situation where the user is setting a password
for the first time (e.g. SSO login, demo organizations).
2023-02-28 11:45:01 -08:00
Alex Vandiver
015a10637b docs: Document how to use SMTP without authentication.
This is the behaviour inherited from Django[^1].  While setting the
password to empty (`email_password = `) in
`/etc/zulip/zulip-secrets.conf` also would suffice, it's unclear what
the user would have been putting into `EMAIL_HOST_USER` in that
context.

Because we previously did not warn when `email_password` was not
present in `zulip-secrets.conf`, having the error message clarify the
correct configuration for disabling SMTP auth is important.

Fixes: #23938.

[^1]: https://docs.djangoproject.com/en/4.1/ref/settings/#std-setting-EMAIL_HOST_USER
2023-02-27 11:59:48 -08:00
Alex Vandiver
fe654b76b7 data_import: Stop tar'ing up converted data.
`./manage.py import` does not take a tarball; it takes a directory.
Making a separate tarball is a waste of CPU time and disk, as it is
never used.

This was included in the commit of the initial Slack conversion code
in 5b37c5562b and propagated from there into every conversion tool.

Remove the unnecessary tarball creation.
2023-02-26 17:42:01 -08:00
m-e-l-u-h-a-n
ab4e6a94c5 user groups: Make name and description optional in group update.
View that handled `PATCH user_groups/<int:user_group_id>` required
both name and description parameters to be passed. Due to this
clients had to pass values for both these parameters even if
one of them was changed.

To resolve this name description parameters to
`PATCH user_groups/<int:user_group_id>` are made optional.
2023-02-26 16:22:24 -08:00
Lauryn Menard
e39c689f36 email-notifications: Update email subjects with "PM" to "DM".
Updates email notification subjects with "PM" to use "DM" instead.

Also, updates `tools/lib/capitalization.py` to include both "DM"
and "DMs".
2023-02-24 11:47:26 -08:00
Lauryn Menard
182e6c0730 push-notifications: Update strings for private messages.
Updates strings with "private message" in push notifications to
use "direct message" instead.
2023-02-24 11:47:26 -08:00
Lauryn Menard
a0fd7b2afc private-messages: Update translated backend strings use "direct message".
Updates user-facing translated strings containing "private message" on
the backend to use "direct message" instead.
2023-02-24 11:47:26 -08:00
Sahil Batra
36584a3571 registration: Add code to set email_address_visibility during signup.
This commit adds backend code to set email_address_visibility when
registering a new user. The realm-level default and the value of
source profile gets overridden by the value user selected during
signup.
2023-02-24 09:23:34 -08:00
Anders Kaseorg
cea1119423 node_tests: Move to web/tests.
This lets us simplify the long-ish ‘../../static/js’ paths, and will
remove the need for the ‘zrequire’ wrapper.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-23 16:04:17 -08:00
Alex Vandiver
f78c2eab2c message_send: Use stream object for name in "no subscribers" message.
In the case where a stream existed but had no subscribers, the error
message used to send to the owner always used `stream_name`, which
may have been None.

Switch to using `stream.name` rather than `stream_name` for this case.
2023-02-23 11:02:55 -08:00
Anders Kaseorg
ed069ebe0e docs: Remove spaces before commas.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-22 17:17:25 -08:00
Sahil Batra
0cf99cf5c3 streams: Refactor code to set group-based stream settings.
We add stream_permission_group_settings object which is
similar to property_types framework used for realm settings.

This commit also adds GroupPermissionSetting dataclass for
defining settings inside stream_permission_group_settings.

We add "do_change_stream_group_based_setting" function which
is called in loop to update all the group-based stream settings
and it is now used to update 'can_remove_subscribers_group'
setting instead of "do_change_can_remove_subscribers_group".

We also change the variable name for event_type field of
RealmAuditLog objects to STREAM_GROUP_BASED_SETTING_CHANGED
since this will be used for all group-based stream settings.

'property' field is also added to extra_data field to identify
the setting for which RealmAuditLog object was created.

We will add a migration in further commits which will add the
property field to existing RealmAuditLog objects created for
changing can_remove_subscribers_group setting.
2023-02-22 12:17:46 -08:00
Mateusz Mandera
a593089770 presence: Return some presence params in the /register response. 2023-02-21 12:10:43 -08:00
Abhijeet Prasad Bodas
6e001d0672 user topic: Remove unnecessary check for double addition of muted topic.
This makes use of the new case insensitive UNIQUE index added in the
earlier commit. With that index present, we can now rely solely on the
database to correctly identify duplicates and throw integrity errors as
required.
2023-02-20 21:04:13 -08:00
Abhijeet Prasad Bodas
9fde88796a mute user: Remove unnecessary check for double muting.
In 141b0c4, we added code to handle races caused by duplicate muting
requests. That code can also handle the non-race condition, so we don't
require the first check.
2023-02-20 21:04:13 -08:00
Lauryn Menard
06dd6f8254 message-send: Deduplicate check of settings.MAX_MESSAGE_LENGTH.
Removes the initial check in `_internal_prep_message` of the length
of the message content because the `check_message` in the try block
will call `normalize_body` on the message content string, which
does a more robust check of the message content (empty string, null
bytes, length). If the message content length exceeds the value of
`settings.MAX_MESSAGE_LENGTH`, then it is truncated based on that
value. Updates associated backend test for these changes.

The removed length check would truncate the message content with a
hard coded value instead of using the value for
`settings.MAX_MESSAGE_LENGTH`.

Also, removes an extraneous comment about removing null bytes. If
there are null bytes in the message content, then `normalize_body`
will raise an error.

Note that the previous check had intentionally reduced any message over
the 10000 character limit to 3900 characters, with the code in
question dating to 2012's 100df7e349.

The 3900 character truncating rule was implemented for incoming emails
with the email gateway, and predated other features to help with
overly long messages (better stripping of email footers via Talon,
introduced in f1f48f305e, and
condensing, introduced in c92d664b44).
While we could preserve that logic if desired, it likely is no longer
a necessary or useful variation from our usual truncation rules.
2023-02-20 12:55:15 -08:00
Lauryn Menard
b25cba16a7 tests: Remove ignored parameter from authenticated_json_view test.
The password parameter being passed in the `_do_test` helper
function for `TestAuthenticatedJsonPostViewDecorator` tests was
being ignored, as the user needs to be logged in. Removes the
parameter from the helper function and updates the success test
to use `assert_json_success` instead of just checking the status
code.

Also adds a test case for when a user is not logged in to confirm
that it returns an UnauthorizedError.
2023-02-20 11:56:00 -08:00
Alex Vandiver
8998aa00cd Revert "create_user: Use transaction.atomic decorator for do_create_user."
This reverts commit 851d68e0fc.

That commit widened how long the transaction is open, which made it
much more likely that after the user was created in the transaction,
and the memcached caches were flushed, some other request will fill
the `get_realm_user_dicts` cache with data which did not include the
new user (because it had not been committed yet).

If a user creation request lost this race, the user would, upon first
request to `/`, get a blank page and a Javascript error:

    Unknown user_id in get_by_user_id: 12345

...where 12345 was their own user-id.  This error would persist until
the cache expired (in 7 days) or something else expunged it.

Reverting this does not prevent the race, as the post_save hook's call
to flush_user_profile is still in a transaction (and has been since
168f241ff0), and thus leaves the potential race window open.
However, it much shortens the potential window of opportunity, and is
a reasonable short-term stopgap.
2023-02-17 21:20:52 -05:00
Alex Vandiver
f8a16fb9af model: Handle parsed user-agents longer than 30 characters.
The Client.name field is only 30 characters long, but there is no
limit to the length of parsed User-Agent value which we may attempt to
store in it.  This can cause requests with long user-agents to 500
when the creation of the Client row fails.

Truncate the name at 30 characters for the cache key, and passing
`name` to `get_or_create`.
2023-02-14 14:58:15 -08:00
Alex Vandiver
dd21e472e7 subdomains: Extend tests for for same-host when STATIC_URL has domain. 2023-02-14 17:17:06 -05:00
Alex Vandiver
6806bec2e9 email_notifications: Use stable, unhashed, URLs for emoji in emails. 2023-02-14 17:17:06 -05:00
Alex Vandiver
e31767dda4 settings: Make DEFAULT_LOGO_URI/DEFAULT_AVATAR_URI use staticfiles. 2023-02-14 17:17:06 -05:00
Sahil Batra
9d1dc20e6e settings: Remove realm-level email_address_visibility setting.
This was replaced by the new user-level version in recent commits.

Fixes #20035.
Fixes #18149.
2023-02-10 17:40:33 -08:00
Sahil Batra
0ed5f76063 settings: Add backend code for using user email_address_visibility setting.
This commits update the code to use user-level email_address_visibility
setting instead of realm-level to set or update the value of UserProfile.email
field and to send the emails to clients.

Major changes are -

- UserProfile.email field is set while creating the user according to
RealmUserDefault.email_address_visbility.

- UserProfile.email field is updated according to change in the setting.

- 'email_address_visibility' is added to person objects in user add event
and in avatar change event.

- client_gravatar can be different for different users when computing
avatar_url for messages and user objects since email available to clients
is dependent on user-level setting.

- For bots, email_address_visibility is set to EVERYONE while creating
them irrespective of realm-default value.

- Test changes are basically setting user-level setting instead of realm
setting and modifying the checks accordingly.
2023-02-10 17:35:49 -08:00
Sahil Batra
ea0b2286e0 settings: Add email_address_visbility user setting. 2023-02-10 17:35:49 -08:00
Sahil Batra
2396e6858f users: Always pass "delivery_email" field in user objects.
Previously, user objects contained delivery_email field
only when user had access to real email. Also, delivery_email
was not present if visibility setting is set to "everyone"
as email field was itself set to real email.

This commit changes the code to pass "delivery_email" field
always in the user objects with its value being "None" if
user does not have access to real email and real email otherwise.
The "delivery_email" field value is None for logged-out users.
For bots, the "delivery_email" is always set to real email
irrespective of email_address_visibility setting.

Also, since user has access to real email if visibility is set
to "everyone", "delivery_email" field is passed in that case
too.

There is no change in email field and it is same as before.

This commit also adds code to send event to update delivery_email
field when email_address_visibility setting changes to all the
users whose access to emails changes and also changes the code to
send event on changing delivery_email to users who have access
to email.
2023-02-10 17:35:49 -08:00
Anders Kaseorg
41aadc8c63 message_send: Convert RecipientInfoResult from TypedDict to dataclass.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-10 15:51:22 -08:00
Abhijeet Prasad Bodas
7d02b20bdb user topics: Rename test file.
This makes the file name in line with the model name and other files,
for better discoverability.

This probably should have been a part of
48883f1580.
2023-02-10 15:39:57 -08:00
Abhijeet Prasad Bodas
9d1c131dc6 muted users: Make file naming consistent.
This makes the names of the relevant files consistant with the database
model name as well as the frontend JS files.
2023-02-10 15:39:57 -08:00
Anders Kaseorg
0a1904a6a7 markdown: Rewrite YouTube URL parser without regex spaghetti.
This also adds support for the new YouTube Shorts URLs.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-09 22:34:51 -08:00
Sahil Batra
891f83601d message_edit: Use move_messages_between_streams_limit_seconds setting.
This commit adds time restriction on moving messages between streams
using the move_messages_between_streams_limit_seconds setting in the
backend. There is no time limit for admins and moderators.
2023-02-08 12:46:05 -08:00
Sahil Batra
2c4e076fef message_edit: Use move_messages_within_stream_limit_seconds setting.
We now use the newly added move_messages_within_stream_limit_seconds
setting to check for how long the user can edit the topic replacing
the previously used 3-day limit. As it was previously, there is no
time limit for admins and moderators.
2023-02-08 12:46:05 -08:00
Sahil Batra
b919dfd489 realm: Add time limit setting for moving messages between streams.
This commit adds "move_messages_between_streams_limit_seconds"
setting which would be used to set a time limit to move messages
between streams.
2023-02-08 12:46:05 -08:00
Sahil Batra
73f0eae394 realm: Add time limit setting for moving message within stream.
This commit adds "move_messages_within_streams_limit_seconds"
setting which would be used to set a time limit to move messages
within stream.
2023-02-08 12:46:05 -08:00
Alex Vandiver
e3c976fed9 missed_messages: Include messages from topic which differ by case.
In Zulip, message topics are case-insensitive but case-preserving.
The `get_context_for_message` function erroneously did a
case-sensitive search, and thus only messages whose topic matched
exactly were pulled in as context.

Make the missed-message pipeline aware that message topics are not
case-sensitive.  This means that, when collapsing adjacent messages,
we merge messages with topic headers which are "different"; create a
separate explicit "grouping" to know which to collapse.
2023-02-07 11:04:21 -08:00
Prakhar Pratyush
906ff9243a user_mutes: Rename 'muting.py' to 'user_mutes.py'.
Rename 'muting.py' to 'user_mutes.py' because it, now
, contains only user-mute related functions.

Includes minor refactoring needed after renaming the file.
2023-02-07 00:23:47 +05:30
Prakhar Pratyush
49577bbdcd user_topics: Move topic muting functions to user_topics.py.
This commit moves topic related stuff i.e. topic muting functions
to a separate file 'views/user_topics.py'.

'views/muting.py' contains functions related to user-mutes only.
2023-02-07 00:23:47 +05:30
Kartik Srivastava
48883f1580 tests: Rename test_muting_topics to test_topic_visibility_policies.
As we'll be adding support for more visibility policies for topics,
this renamed file will contain the necessary tests.
2023-02-07 00:23:47 +05:30
Sahil Batra
73f11853ec streams: Allow setting can_remove_subscribers_group_id while creating streams.
This commit adds API support to set can_remove_subscribers_group setting
when creating streams.
2023-02-05 14:46:36 -08:00
Sahil Batra
c3759814be streams: Allow changing can_remove_subscribers_group through API.
This commit adds API support to change can_remove_subscribers_group
setting for a stream.
2023-02-05 14:46:36 -08:00
Anders Kaseorg
da3cf5ea7a ruff: Fix RSE102 Unnecessary parentheses on raised exception.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-04 16:34:55 -08:00
Prakhar Pratyush
9997131df3 message_edit: Fix resolve topic notifications.
When 'resolve|unresolve' and 'move stream' actions occurs in
the same api call, 'This topic was marked as resolved|unresolved'
notification is not sent.

Both 'topic moved' and 'topic resolved' notification should be generated.

This commit updates the logic of when and where to send
'topic resolve|unresolve' notification. Unlike previous logic, notification
may be sent even in the case 'new_stream' is not None.

In general, 'topic resolved|unresolved' notification is sent to
'stream_being_edited'. In this particular case ('new_stream' is not None),
notification is sent to the 'new_stream' after check.

Test case is included.

Fixes: #22973
2023-02-03 18:20:39 -08:00
Alessandro Toppi
ff89590558 auth: Add JWT-based user API key fetch.
This adds a new endpoint /jwt/fetch_api_key that accepts a JWT and can
be used to fetch API keys for a certain user. The target realm is
inferred from the request and the user email is part of the JWT.

A JSON containing an user API key, delivery email and (optionally)
raw user profile data is returned in response.
The profile data in the response is optional and can be retrieved by
setting the POST param "include_profile" to "true" (default=false).

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2023-02-03 15:23:35 -08:00
Mateusz Mandera
100f4a7152 auth: Add @require_post to remote_user_jwt. 2023-02-03 15:23:35 -08:00
Mateusz Mandera
d466da1064 auth: Use REQ for getting token value and rename param in jwt paths.
This makes us use REQ properly instead of fetching from request.POST
manually - also renaming the param to "token" which is more standard.
2023-02-03 15:23:35 -08:00
Mateusz Mandera
c9c255b3a8 auth: Improve JsonableError in get_..._jwt_authentication_request. 2023-02-03 15:23:35 -08:00
Mateusz Mandera
6c638a1057 auth: Extract token-check logic of remote_user_jwt.
This will be useful for re-use for implementation of an endpoint for
obtaining the API by submitting a JWT in the next commits.

It's not a pure refactor, as it requires some tweaks to remote_user_jwt
behavior:
1. The expected format of the request is changed a bit. It used to
   expect "user" and "realm" keys, from which the intended email was
   just generated by joining with @. Now it just expects "email"
   straight-up. The prior design was a bt strange to begin with, so this
   might be an improvement actually.
2. In the case of the codepath of new user signup, this will no longer
   pre-populate the Full Name in the registration form with the value
   from the "user" key. This should be a very minor lost of
   functionality, because the "user" value was not going to be a proper
   Full Name anyway. This functionality can be restored in a future
   commit if desired.

This is an API change, but this endpoint is nearly unused as far as
we're aware.
2023-02-03 15:23:35 -08:00
Aman Agrawal
c3f0d5898a index: Remove Loading... text from the app loading overlay.
The new loading spinner animation conveys the message well
that the app is loading and this text is no longer required.
2023-02-03 10:51:39 -08:00
Lauryn Menard
fc54ffd778 documentation: Move check for api-doc-template.md.
Moves the check for calling the `api-doc-template.md` directly,
so that we don't return a 500 error from the server, to happen
earlier with other checks for returning a 404 / missing page.

Also adds a specific test to `zerver/tests/test_urls` for this
template.

Prep commit for moving API documentation directory to be a top
level directory.
2023-02-02 17:23:31 -08:00