Commit Graph

13378 Commits

Author SHA1 Message Date
Tim Abbott
18595fac58 models: Fix incorrect on_delete=CASCADE values for deleted streams.
In theory, we never delete streams, but if we did delete the
notifications stream for a realm, we certainly shouldn't delete the
realm.
2021-05-06 20:41:49 -07:00
Anders Kaseorg
405bc8dabf requirements: Remove Thumbor.
Thumbor and tc-aws have been dragging their feet on Python 3 support
for years, and even the alphas and unofficial forks we’ve been running
don’t seem to be maintained anymore.  Depending on these projects is
no longer viable for us.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-06 20:07:32 -07:00
Anders Kaseorg
779353b44e apps: Link to macOS Apple silicon native build.
Leave the Intel build as the prominent default, since it will run on
both platforms.  (I would have liked to detect the appropriate
platform, but Apple seems to have put significant effort into making
that impossible for anti-fingerprinting reasons, which is probably an
overall good.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-06 17:52:00 -07:00
Adam Birds
a3ba8b9efd integrations: Add Freshping webhook integration.
I have added a webhook integration for Freshping.
2021-05-06 11:18:35 -07:00
Adam Birds
c72ef7be12 integrations: Move get_setup_webhook_message to common.py.
Move `get_setup_webhook_message` to
`zerver/lib/webhooks/common.py` so multiple integrations can use this
rather than just those which import `zerver/lib/webhooks/git.py`. Also
added the documentation for this.
2021-05-06 11:18:35 -07:00
Tim Abbott
c598a84dd6 ldap: Check for deactivating owner role precisely.
Since the invariant we're trying to protect is that every realm has an
active owner, we should check precisely that.

The root bug here, which the parent commit failed to fix properly, is
that we were doing a "greater than" check when we clearly originally
meant a "less than" check -- lower role numbers have more permissions.
2021-05-06 10:13:29 -07:00
Tim Abbott
5db8e43d5c ldap: Fix incorrect check for deactivating all owners.
We apparently failed to update this when we did the migration from the
administrator role to the owner role as the top role in Zulip.
2021-05-06 09:45:52 -07:00
pletinckxc
cc4cc9d14f test_send_email: Shorten tests about build_email.
Commit b7fa41601d introduced some tests
to verify the implemented changes. These tests were longer than necessary.
See https://github.com/zulip/zulip/pull/17799 for the whole discussion.
2021-05-05 20:16:11 -07:00
Cyril Pletinckx
e4ff372fc3 emails: Transform SMTPException into EmailNotDeliveredException.
Django's default SMTP implementation can raise various exceptions
when trying to send an email. In order to allow Zulip calling code
to catch fewer exceptions to handle any cause of "email not
sent", we translate most of them into EmailNotDeliveredException.
The non-translated exceptions concern the connection with the
SMTP server. They were not merged with the rest to keep some
details about the nature of these.

Tests are implemented in the test_send_email.py module.
2021-05-05 20:16:11 -07:00
Alya Abbott
279f7b1c8d user docs: Create separate "Code blocks" user documentation page.
* Move the extended documentation of code blocks to a separate page.
* Merge "code playgrounds" documentation to be a section of that page.
* Document copy widget on code blocks.
* This commit changes how we refer to "```python" type syntax for code
  blocks. Instead of being called a syntax highlighting label, this is
  now referred to as a "language tag", since it serves both syntax
  highlighting and playgrounds.
* Remap all the links.
* Advertise this new page in various places that previously did not have a link.
2021-05-05 16:11:21 -07:00
Mateusz Mandera
cc96f02947 presence: Disable live presence updates in larger realms.
As discussed in the comment, this is a critical scalability
optimization for organizations with thousands of users.

With substantial comment updates by tabbott.
2021-05-05 08:50:42 -07:00
Sumanth V Rao
a40fe26586 help: Document playgrounds on /help, /features and /for/open-source.
Linked the Help Center document in places like
    - zulip.yaml (/events, /register/, realm/playgrounds,
                  /realm/playgrounds/{playground_id})
    - /help/format-your-message-using-markdown (Linked to make
      users reading the markdown code block style, aware of this
      feature)
    - /templates/settings/playground_settings_admin.hbs (Linked
      as a reference to read more about playgrounds before
      configuring one)

Also showcase the feature on /features and /for/open-source.
2021-05-04 12:19:01 -07:00
Dinesh
e481f0c869 set-typing-status_api: Document to as required argument.
This was accidentally removed in 27e4f5da92
as the initial idea was to have another parameter `stream_id`
for stream type messages instead of sending it in this `to` parameter.

Also fixed a description of `to` for `stream` type.
2021-05-04 11:35:36 -07:00
shanukun
5cee9dee12 test_email_change: User create_confirmation_link for url.
This is a prep refactor, instead of creating Confirmation
object and using `confirmation_url` for generating confirmation
link/url, using `create_confirmation_link` would be a cleaner
approach, also this can help us avoid failing test in case
Confirmation model is changed.

Part of #16359.
2021-05-04 10:48:51 -07:00
Alex Vandiver
a9688ceb75 worker: Allow long MissedMessageWorker consumes.
This will stop dropping events in the case that the background
`maybe_send_batched_email` thread takes longer than 30s.  However, see
also #15280 and the TODO comment about how we lose events upon
restart; this worker is still lossy.
2021-05-04 08:45:48 -07:00
sahil839
6e672ebbee settings: Add moderators option for wildcard_mention_policy. 2021-05-03 12:12:01 -07:00
sahil839
141232749b test_message_send: Modify test_wildcard_mention_restrictions.
This commit modifies the test_wildcard_mention_restrictions test
for checking that moderators are not allowed to send messages
with wildcard mention if wildcard_mention_policy is set to
WILDCARD_MENTION_POLICY_ADMINS. Previously, we were checking
for members, but it is better to check for moderators.
2021-05-03 12:12:01 -07:00
Tim Abbott
1489c24860 test_decorators: Don't try to mutate is_staff.
Apparently, after upgrading to Django 3.2, mutating is_staff and then
saving can result in a user's session being destroyed.

In any case, this test is probably better written using two different
users with the different roles, which we have in our initial database
anyway.
2021-05-03 09:26:00 -07:00
Mateusz Mandera
6a8586e989 upload: Mention new difference between sanitize_name and slugify.
In Django 3.2 slugify strips trailing dashes and underscores:
0382ecfe02

sanitize_name doesn't so this difference should be documented like the
others.
2021-05-03 08:36:22 -07:00
Mateusz Mandera
389c7bdb5a upload: Fix docstring and regex in sanitize_name regarding underscore.
Underscore character is already covered by \w, so _ in the regex is
redundant. Also the docstring is mildly incorrect - underscore already
is an allowed character by django's slugify (and always was) for the
aforementioned reason.
2021-05-03 08:36:22 -07:00
Mateusz Mandera
1d9fb4f988 django: Upgrade Zulip to Django 3.2 LTS.
This is a straightforward upgrade in terms of changes needed.

Necessary changes were:
- Set `DEFAULT_AUTO_FIELD`
  https://docs.djangoproject.com/en/3.2/releases/3.2/#customizing-type-of-auto-created-primary-keys

- `The default_app_config application configuration variable is deprecated, due
  to the now automatic AppConfig discovery.`
  https://docs.djangoproject.com/en/3.2/releases/3.2/#automatic-appconfig-discovery

To handle this one, we can remove default_app_config from
zerver/__init__.py because it satisfies what release notes describe in
https://docs.djangoproject.com/en/3.2/releases/3.2/#automatic-appconfig-discovery:
"Most pluggable applications define an AppConfig subclass in an apps.py
submodule. Many define a default_app_config variable pointing to this
class in their __init__.py.  When the apps.py submodule exists and
defines a single AppConfig subclass, Django now uses that configuration
automatically, so you can remove default_app_config."

An important note is that rebuild-test-database needs to be run after
this upgrade in dev environment - if tests are run with test db that was
built on the previous version, they will fail due to a mysterious bug
(?), where changing attributes of a user and .save()ing after logging in
in the test via self.login_user, causes getting logged out - the next
requests via self.client_get etc. are unauthed for some reason,
unless self.login_user is called again. This behavior is no longer
exhibited upon rebuilding the test db - and I can't reproduce it in
production or dev db. So this can likely be reasonably dismissed as some
quirk of the test client system that won't be relevant in the future and
doesn't impact production.
2021-05-03 08:36:22 -07:00
Tim Abbott
935a7b18cb test_signup: For test overly sensitive to whitespace.
This test started failing after a rebase from a branch 2 weeks ago
where it passed.
2021-05-02 11:35:32 -07:00
Tim Abbott
97b08b77df registration: Improve error handling for source_realm_id.
This restores the previous behavior that we store None rather than
throwing an exception in the presence of invalid input.
2021-05-02 11:12:49 -07:00
Siddharth Asthana
55da3329ef registration: Rename source_realm field to source_realm_id.
Now that we are passing source realm's id instead of string_id in
source realm selector, it makes sense to rename the "source_realm" field
to "source_realm_id".
2021-05-02 11:12:49 -07:00
Siddharth Asthana
4262c04db1 registration: Encode source realm as an integer.
In the source realm selector, when we select a realm from which we want
to import the data, we pass the source realm's string_id. The problem
with this approach is that the string_id can be an empty string. This
commit makes the source_realm pass the realm's id instead of string_id.
Now, the source_realm's value will either be an integer or "" (empty
string) when we don't want to import settings from any realm.
2021-05-02 11:12:49 -07:00
Anders Kaseorg
995389b4c1 markdown: Don’t apply further Markdown processing to KaTeX output.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-01 15:43:35 -07:00
Anders Kaseorg
802c8de0e8 fenced_code: Wrap math blocks in the frontend like the backend.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-01 15:36:15 -07:00
m-e-l-u-h-a-n
201dd7507f left-sidebar: Show starred message counts by default.
It is changed just to make this feature more discoverable.

Fixes: #18305.
2021-05-01 07:27:08 -07:00
pilgrim2308
a74b52db22 UI: Add Zulip version in gear menu.
Currently only enabled in development, since the exact details don't
seem right..

Co-Author-By: Signior-X <b19188@students.iitmandi.ac.in>
Co-Author-By: Aman Agrawal <amanagr@zulip.com>

Implements UI for #8005.
2021-04-30 17:58:08 -07:00
sahil839
3c66131fae models: Fix comment above INVITE_AS dict.
The comment mentioned the values in this dict to be consistent
wth values in settings_config.invited_as_values.
But settings_config.invited_as_values was replaced by
settings_config.user_role_values in 136c005f3f, so
updated the comment accordingly.
2021-04-30 15:57:09 -07:00
sahil839
e669818544 invites: Allow user to be invited as a moderator.
This commit adds both frontend and backend code to invite a user as
moderator. We allow only existing owners and admins to invite a user
as a moderator.
2021-04-30 15:57:09 -07:00
sahil839
a4c3224328 users: Remove redundant get_role_for_new_user in lib/create_user.py.
The function get_role_for_new_user was added to get role from the
invited_as value, as invited_as values were one of (1,2,3,4)
previously, but it was then changed to be the actual role value,
i.e. one of (100, 200, 400, 600), in 1f8f227444.

So, we can safely remove this function now and use invited_as value
directly and handle realm_creation case by using an if condition.
2021-04-30 15:57:09 -07:00
Alex Vandiver
fd1774dcba push_notifications: Give full stack information on an exception.
This error has been seen in production instances, but we need more
context to be able to determine what might be causing it.
2021-04-30 14:03:52 -07:00
Wesley Aptekar-Cassels
6b7a3fb74a markdown: Rewrite all external images to use Camo.
Requesting external images is a privacy risk, so route all external
images through Camo.

Tweaked by tabbott for better test coverage, more comments, and to fix
bugs.
2021-04-30 10:36:16 -07:00
Ganesh Pawar
ddf2127035 widgets: Prevent edits to widgets.
As of now, editing a widget doesn't update the rendered content.
It's important to ensure that existing votes or options added later on
don't get deleted when rendered.
This seems more complex than it's worth.

For now, we just prevent edits to widgets.
This commit makes the UI clearer that editing widgets isn't allowed.

See also:
https://github.com/zulip/zulip/issues/14229
https://github.com/zulip/zulip/issues/14799

Fixes #17156
2021-04-30 09:55:25 -07:00
Vishnu KS
7f3fc3423b audit log: Create audit log when a realm is created.
This is mainly useful in recording the user who created the realm,
when possible.
2021-04-30 09:25:11 -07:00
Ganesh Pawar
830f1fa8c5 upload: Refactor and add tests for ensure_avatar_image in upload.py.
`ensure_basic_avatar_image` and `ensure_medium_avatar_image` are
essentially the same thing, except a size parameter.
So, refactor them into a single function.

This doesn't introduce any functional changes.
2021-04-29 21:18:13 -07:00
Tim Abbott
615ad2d5d8 middleware: Simplify logic for parsing user-agent.
This avoids calling parse_user_agent twice when dealing with official
Zulip clients, and also makes the logical flow hopefully easier to read.

We move get_client_name out of decorator.py, since it no longer
belongs there, and give it a nicer name.
2021-04-29 17:47:41 -07:00
Tim Abbott
3cf0156997 context_processors: Remove unnecessary get_client_name call.
This detail is now being managed by our middleware.
2021-04-29 17:47:32 -07:00
orientor
fe260fb892 middleware: Show client version in logging if available.
Fixes #14067.
2021-04-29 17:07:37 -07:00
orientor
ac203cd9f1 middleware: Add client_version attribute to request. 2021-04-29 17:03:40 -07:00
orientor
6224d83dea middleware: Get client name in LogRequests instead of process_client.
This ensures it is present for all requests; while that was already
essentially true via process_client being called from every standard
decorator, this allows middleware and other code to rely on this
having been set.
2021-04-29 17:03:05 -07:00
Anders Kaseorg
871e73ab8f mypy: Don’t use Iterable for values iterated multiple times.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-29 16:06:17 -07:00
Anders Kaseorg
393ffcb318 actions: Replace SizedTextIterable with standard Collection[str].
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-29 16:06:17 -07:00
Anders Kaseorg
48a52ac56e events: Convert recursion to a loop in do_events_register.
Fixes #18310.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-29 16:03:05 -07:00
sahil839
bd78b1ff90 events: Add 'is_moderator' field to the page_params object sent to clients. 2021-04-29 15:18:49 -07:00
sahil839
dc771f3a14 users: Add support for changing user role to moderator in api.
This commit adds backend support for changing a user role to
moderator and also to change role from moderator to any other
role.
2021-04-29 15:17:45 -07:00
sahil839
34f134d58d users: Add role field to user objects returned by format_user_row.
This commit modifies the user objects returned by 'GET /users',
'GET /users/me', 'GET /users/{user_id}' and 'GET /users/{email}'
endpoints to include role field.

We also include role field in the page_params['realm_users'] dict
and in the person object sent in (type="realm_user", op="add")
event.
2021-04-29 15:13:50 -07:00
Alex Vandiver
8711ab7676 outgoing_webhook: Add a logging statement for each outgoing webhook.
This will help determine potentail timeout lengths, as well as serve
as a generally-useful log for locations which do not have Smokescreen
enabled.

In service of #17742.
2021-04-29 12:37:14 -07:00
Adam Birds
db16dd9d26 integrations: Change display name for Json integration to use a capital F.
Change display name for Json integration to use a cpital F instead of
lowercase which is the standard across the rest of our integrations.
2021-04-29 12:35:12 -07:00