Commit Graph

8157 Commits

Author SHA1 Message Date
Alex Vandiver
ee04f42897 send_email: Only warn if EMAIL_HOST_PASSWORD is unset, not "".
Some email hosts actually do want an empty password; since the default
is `None`, we should key on that, and not just being false-y.
2022-04-12 16:41:27 -07:00
Lauryn Menard
9f8022de5e help_docs: Add 'About Zulip' to relative help links.
Adds the 'About Zulip' gear menu option to the available
relative link patterns used in help center documentation.
2022-04-12 15:22:58 -07:00
Alex Vandiver
a26d109e7a auth: Reset failed authentication attempts on password reset.
It's natural that someone might try a wrong password 5 times, and then
go through a successful password reset; forcing such users to wait
half an hour before typing in the password they just changed the
account to seems unnecessarily punitive.

Clear the rate-limit upon successful password change.
2022-04-11 16:25:32 -07:00
Alex Vandiver
04159a674c timeout: Minor comment cleanups.
We remove the StackOverflow link because it is now so dated as to be
irrelevant -- it does not use `self.ident`, and cargo-cults the return
value of PyThreadState_SetAsyncExc.
2022-04-07 17:26:01 -07:00
Alex Vandiver
3af2c8d9a3 timeout: Warn if the thread did not exit.
As noted in the docstring for this function, the timeout is
best-effort only -- if the thread is blocked in a syscall, it will not
service the exception until it returns.  It can also choose to catch
and ignore the TimeoutExpired; in either case it will still be running
even after the `timeout()` function returns.

Raising a vare TimeoutExpired it still somewhat accurate, but obscures
that the backend thread may still be running along merrily.  Notice
such cases, and log a warning about them.
2022-04-07 17:26:01 -07:00
Alex Vandiver
e714264756 timeout: Re-raise from where the TimeoutExpired hit the thread.
Having just thrown an exception into the thread, it is often useful to
know _what_ was the slow code that we interrupted.  Raising a bare
TimeoutExpired here obscures that information, as any `exc_info` will
end there.

Examine the thread for any exception information, and use that to
re-raise.  This exception information is not guaranteed to exist -- if
the thread didn't respond to the exception in time, or caught it, for
instance.
2022-04-07 17:26:01 -07:00
Alex Vandiver
85eeaf5f18 timeout: Remove cargo-culted and impossible-to-reach code block.
The quote in question originates in python/cpython@b8b6d0c2c6, when
the code was added.  However, the code stopped having that comment,
and was no longer able to return anything but 1 or 0, starting in
python/cpython@4643c2fda1 -- Python 2.5.

Remove the block.
2022-04-07 17:26:01 -07:00
jai2201
20e6315705 streams: Fix empty description in stream change notifications.
Send 'No description' text when either the old or new stream
description is empty, to avoid odd looking notifications.
2022-03-29 01:39:55 -07:00
Alya Abbott
f04fb51ecc help: Rewrite pages on inviting users and related permissions.
Fixes #21520.

Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2022-03-28 17:00:10 -07:00
Lauryn Menard
cbfe2707f4 help_docs: Clarify relative link text for Subscribed streams tab. 2022-03-28 15:13:39 -07:00
Alex Vandiver
ca506f71dc push_notifications: Increase severity of APNs ConnectionError.
This has only happened when our APNs certificate expired; logging at
the error level ensures that this shows up in Sentry.
2022-03-25 18:12:14 -07:00
Anders Kaseorg
55882fb343 python: Use modern set comprehension syntax.
Generated by pyupgrade.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-25 10:45:12 -07:00
Alex Vandiver
eae4643cb4 message_edit: Ignore duplicates when re-muting new topic name.
This avoids an error when a user has already muted the new topic name.
We do this by ignoring duplicates, rather than catching the
IntegrityError, because this edit happens in a transaction, and that
would abort the transaction.
2022-03-24 21:27:11 -07:00
Alex Vandiver
781107308d muting: Add a flag to allow user duplicate mutes to silently succeed. 2022-03-24 21:27:11 -07:00
Anders Kaseorg
83c90c53df db: Fix types.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-24 19:16:52 -07:00
Tim Abbott
f9f111f950 message_edit: Only move muted topic records when moving whole topics.
Our original implementation of moving muted topic records when a topic
is moved took a shortcut of treating all change_later usage as
something with intent to move the whole topic.

This works OK when moving the whole topic via this interface, but not
when moving a last off-topic message in the topic.

Address this by changing the rule to match the existing
moved_all_visible_messages variable.
2022-03-24 17:48:52 -07:00
Lauryn Menard
182c00248d help_docs: Update various docs related to permissions.
Adds tab for web-public streams in documentation for setting
who can create new streams, as well as some text about why
this is limited to certain roles.

Removes list of actions that can be restricted to full members
due to maintainability concerns for that type of list in the
documentation and replaces it with a short descriptive text
explaining that many settings in Zulip support this restriction.
2022-03-24 11:37:35 -07:00
Aman Agrawal
b799ec32b0 upload: Allow rate limited access to spectators for uploaded files.
We allow spectators access to uploaded files in web public streams
but rate limit the daily requests to 1000 per file by default.
2022-03-24 10:50:00 -07:00
Tim Abbott
d149af936d models: Rework Attachment.is_*_public to be a cache.
Previously, Attachment.is_realm_public and its cousin,
Attachment.is_web_public, were properties that began as False and
transitioned to True only when a message containing a link to the
attachment was sent to the appropriate class of stream, or such a link
was added as part of editing a message.

This pattern meant that neither field was updated in situations where
the access permissions for a message changed:

* Moving the message to a different stream.
* Changing the permissions for a stream containing links to the message.

This correctness issue has limited security impact, because uploaded
files are secured both by a random URL and by these access checks.

To fix this, we reformulate these fields as a cache, with code paths
that change the permissions affecting an attachment responsible for
setting these values to the `None` (uncached) state. We prefer setting
this `None` state over computing the correct permissions, because the
correct post-edit permissions are a function of all messages
containing the attachment, and we don't want to be responsible for
fetching all of those messages in the edit code paths.
2022-03-24 10:49:59 -07:00
Alex Vandiver
4f93b4b6e4 uploads: Skip the outgoing proxy if S3_KEY is unset.
When the credentials are provided by dint of being run on an EC2
instance with an assigned Role, we must be able to fetch the instance
metadata from IMDS -- which is precisely the type of internal-IP
request that Smokescreen denies.

While botocore supports a `proxies` argument to the `Config` object,
this is not actually respected when making the IMDS queries; only the
environment variables are read from.  See
https://github.com/boto/botocore/issues/2644

As such, implement S3_SKIP_PROXY by monkey-patching the
`botocore.utils.should_bypass_proxies` function, to allow requests to
IMDS to be made without Smokescreen impeding them.

Fixes #20715.
2022-03-24 10:21:35 -07:00
Alex Vandiver
1ac0035f8c markdown: Allow whitespace overlaps in topic linkifiers.
`prepare_linkifier_pattern`, as of db934be064, adds a match to the
end of the regex, of either the end of string, or a non-word character
-- this is in place of a negative look-ahead, which is no longer
possible in re2.  This causes the regex to consume trailing
whitespace, and thus not be able to match twice in succession with
`pattern.finditer` -- "#1234 #5678" fails to match because the space
is consumed by the first match of the regex.

Rather than use `pattern.finditer`, write own own version, which
rewinds over the non-word character consumed after the match, if any.
This allows the same "after" non-word character to also satisfy the
"before" of the next match.

Fixes #21502.
2022-03-22 15:40:03 -07:00
Alex Vandiver
abed174b12 uploads: Add an endpoint which forces a download.
This is most useful for images hosted in S3, which are otherwise
always displayed in the browser.
2022-03-22 15:05:02 -07:00
Lauryn Menard
b67288db67 help_docs: Extend options for getting links to Zulip content.
Extends the linking to Zulip documentation to cover:

- Getting URLs to messages via the message timestamp.
- Getting links to topics via the three-dots menu.
- Getting links to streams via right-click context menu.

Creates a new tabbed section for using the browser
address bar to copy URLs.
2022-03-22 12:38:09 -07:00
Aman Agrawal
7a25a80242 message_edit: Move muted status of the topic when moved between streams.
Add support for moving MutedTopic entries to another stream where
the user has access to shared history in both streams and
`propagate_mode != "change_one"`.

Also, we delete them the current user does not have access to the
target stream.
2022-03-21 12:42:39 -07:00
Tim Abbott
aebd81c440 management: Remove unnecessary default=SUPPRESS logic. 2022-03-21 12:05:59 -07:00
Tim Abbott
2be2393d3e create_user: Extract get_create_user_params.
We set nocoverage for the new function. Ideally it'd eventually get an
automated test, but we don't want to block this helpful refactoring on
doing so.
2022-03-21 12:05:59 -07:00
Tim Abbott
57fa62ae4b initial_password: Add explicit development environment assertion.
The construction of INITIAL_PASSWORD_SALT is such that it should only
be set in development environments, but we should enforce this rule.
2022-03-21 12:05:59 -07:00
Tim Abbott
6b00c748fd onboarding: Deduplicate realm creation initial user logic.
We now call this function inside do_create_user(...,
realm_creation=True), which generally improves readability and
robustness of the codebase.

This fixes a bug where this onboarding content was not correctly done
when creating a realm via LDAP, and also will be important as we add
new code paths that might let you create a realm.
2022-03-21 12:05:59 -07:00
Tim Abbott
a117b224a7 onboarding: Refactor setup_realm_internal_bots call.
This improves robustness of any code paths calling do_create_realm,
which previously needed to call this correctly to achieve the same
results as creating a user via the UI.

This also fixes a bug where this code was not called if a realm were
created using the LDAP code path.
2022-03-21 12:05:59 -07:00
Tim Abbott
b4507df8fa create_user: Remove tos parameter from management command.
This parameter was introduced in
ea11ce4ae6, and no longer serves a
purpose. Zulip will already correctly record that the user has not
agreed to ToS, and either prompt them on first login or not depending
whether the server is configured to require ToS.
2022-03-21 12:05:59 -07:00
Tim Abbott
4f3894f9f1 management: Improve help text for create_user. 2022-03-21 12:05:59 -07:00
Tim Abbott
9761711351 management: Extract add_create_user_args.
This will avoid code duplication when adding a create_realm management command.
2022-03-21 12:05:59 -07:00
Mateusz Mandera
af5d0d6f5e bots: Don't allow admins to change owner of bot with can_create_users.
Ordinary organization administrators shouldn't be allowed to change
ownership of a bot with the can_create_users permission.

This is a special permission that is granted manually by server
administrators to an organization (to a UserProfile of the org owners'
choice) after approval by a server administator. The code comments
provide more detail about why this is sensitive.
2022-03-20 17:18:21 -07:00
Tim Abbott
f8146cfaa1 help: Fix label for Subscribed tab in stream settings. 2022-03-18 14:28:20 -07:00
Tim Abbott
e45cebd636 message_edit: Fix unmute of topic when topic name is edited.
Previously, when a topic was edited (including being resolved), it
would become unmuted for any users who had muted it, which was
annoying.

While it's not possible to determine the user's intent completely,
this is clearly incorrect behavior in the `change_all` case, such as
resolving a topic.

The comments discuss some scenarios where we might want to enhance
this further, but this is the best we can do without large increases
in complexity.

Fixes #15210.

Co-authored-by: akshatdalton <akshat.dak@students.iiit.ac.in>
2022-03-18 12:32:46 -07:00
Suyash Vardhan Mathur
20a97bdb05 events: Add functionality to mark messages as unread.
Co-authored-by: Steve Howell <showell@zulip.com>
Co-authored-by: Tim Abbott <tabbott@zulip.com>

This commit adds the backend functionality to
mark messages as unread through update_message_flags
with `unread` flag and `remove` operation.

We also manage incoming events in the webapp.

Tweaked by tabbott to simplify the implementation and add an API
feature level update to the documentation.

This commit was originally drafted by showell, and showell
also finalized the changes.  Many thanks to Suyash here for
the main work here, which was to get all the tests and
documentation work moving forward.
2022-03-15 18:00:35 -07:00
Sahil Batra
6f93f07844 types: Create a new TypedDict RealmPlaygroundDict for playground objects.
This commit creates a new TypedDict RealmPlaygroundDict for realm
playground objects. Now the list of playgrounds in the events sent
to clients and the "added_playground" field of RealmAuditLog entry
use RealmPlaygroundDict instead of Dict.
2022-03-15 15:58:36 -07:00
Sahil Batra
4bbb5f18e4 actions: Use transaction.atomic for do_remove_realm_playground. 2022-03-15 15:58:36 -07:00
Sahil Batra
578b752766 actions: Use transaction.atomic for do_add_realm_playground. 2022-03-15 15:58:36 -07:00
Sahil Batra
b86b9bdc02 realm: Create RealmAuditLog entry when removing realm playgrounds.
This commit also adds 'acting_user' parameter to
do_remove_realm_playground function.

Fixes a part of #21268.
2022-03-15 15:58:36 -07:00
Sahil Batra
dea3389045 realm: Create RealmAuditLog entry when adding realm playgrounds.
This commit also adds 'acting_user' parameter to
do_add_realm_playground function.

Fixes a part of #21268.
2022-03-15 15:58:36 -07:00
Sahil Batra
9fd9c5f1d2 actions: Pass realm_playgrounds as argument to notify_realm_playgrounds.
This commit modifies the notify_realm_playgrounds function to accept
realm_playgrounds as argument from the caller instead of computing it
in the function to avoid duplicate queries since the realm playgrounds
list will be required in its caller functions as well in further commits.
2022-03-15 15:58:36 -07:00
Alex Vandiver
62ba8e455d CVE-2022-24751: Clear sessions outside of the transaction.
Clearing the sessions inside the transaction makes Zulip vulnerable to
a narrow window where the deleted session has not yet been committed,
but has been removed from the memcached cache.  During this window, a
request with the session-id which has just been deleted can
successfully re-fill the memcached cache, as the in-database delete is
not yet committed, and thus not yet visible.  After the delete
transaction commits, the cache will be left with a cached session,
which allows further site access until it expires (after
SESSION_COOKIE_AGE seconds), is ejected from the cache due to memory
pressure, or the server is upgraded.

Move the session deletion outside of the transaction.

Because the testsuite runs inside of a transaction, it is impossible
to test this is CI; the testsuite uses the non-caching
`django.contrib.sessions.backends.db` backend, regardless.  The test
added in this commit thus does not fail before this commit; it is
merely a base expression that the session should be deleted somehow,
and does not exercise the assert added in the previous commit.
2022-03-15 13:52:15 -07:00
Alex Vandiver
7650b5a972 session: Enforce that changes cannot happen in a transaction. 2022-03-15 13:52:15 -07:00
Anders Kaseorg
29ecf415fc validator: Add WildValue class for enforcing JSON type checking.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-15 13:02:02 -07:00
Anders Kaseorg
04d772b582 request: Support converter or json_validator with argument_type="body".
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-15 13:02:02 -07:00
Anders Kaseorg
970f22380a request: Replace default_assigned flag with continue.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-15 13:02:02 -07:00
Anders Kaseorg
5f92078d07 request: Add a var_name parameter to converter.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-15 13:02:02 -07:00
Alex Vandiver
95892a5ed3 emoji: Support animated PNGs. 2022-03-15 12:47:21 -07:00
Lauryn Menard
6cc818b672 help_docs: Document user management via user profile modal.
Adds documentation for admins to manage users via the user profile
modal for these actions:
- Deactivating a user
- Changing a user's role
- Changing a user's name

Creates two new tab sections because we still want to document
the ability to do these actions through the users section in
the organizational settings modal.

Also cleans up some text in the help center article for changing
a user's role.

Fixes #21318.
Fixes #21415.
2022-03-15 12:35:03 -07:00