Commit Graph

7343 Commits

Author SHA1 Message Date
Anders Kaseorg
e10b7f2f3c email_notifications: Run html2text as a subprocess.
html2text is GPL licensed.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:32:59 -07:00
Anders Kaseorg
5033beb99c request: Replace tornado_handler weak reference with tornado_handler_id.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-25 08:42:23 -07:00
Anders Kaseorg
952422d373 test_helpers: Remove default tornado_handler from HostRequestMock.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-25 08:42:23 -07:00
Anders Kaseorg
c1a54f4567 decorator: Only URL-decode application/x-www-form-urlencoded requests.
We previously parsed any request with method other than {GET, POST} and
Content-Type other than multipart/form-data as if it were
application/x-www-form-urlencoded.

Check that Content-Type is application/x-www-form-urlencoded before
parsing the body that way.  Restrict this logic to {DELETE, PATCH,
PUT} (having a body at all doesn’t make sense for {CONNECT, HEAD,
OPTIONS, TRACE}).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-25 08:36:16 -07:00
Zixuan James Li
fa51d8718c home: Prevent mypy from inferring the type of page_params.
Without an explicit type annotation, mypy infers the type of
values in `page_params` upon its initialization as a `Union`, while
other computed values haven't been assigned yet. We break this
over-conservative inferred type by annotating `page_params` as a
`Dict[str, object]`.

We could have created a `TypedDict` to have accurate type annotation
for all of the fields, but it does not seem worth it at this point since
the data structure is not widely used in the backend.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 22:09:05 -07:00
Zixuan James Li
fd9a0f4274 typing: Apply trivial none-checks with assertions as necessary.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00
Zixuan James Li
146b0c482c typing: Add none-check for WebhookIntegration.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00
Zixuan James Li
058dab5818 typing: Add none-checks for stream.recipient_id.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00
Zixuan James Li
c567e44408 typing: Avoid redefinition of variables with different types.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00
Zixuan James Li
75df60bd46 send_email: Define variable typed as a Union.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00
Zixuan James Li
40b4da8f58 emoji: Add none checks for uploaded file name.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00
Zixuan James Li
4cf3ba5744 typing: Fix typical typing typos.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00
Anders Kaseorg
345ed1d09d middleware: Pass unhandled API exceptions through to the test suite.
This results in more useful stack traces in failing tests.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-23 19:23:08 -07:00
Anders Kaseorg
74e94ae78c integrations: Fix config_options type.
The “validator” component of the tuple does not follow the Validator
contract as of 7e9db327b3 (#15498).
Define a separate type for it.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-23 19:22:12 -07:00
Anders Kaseorg
c944adfcc6 mypy: Enable redundant-expr errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-23 19:22:12 -07:00
Mateusz Mandera
acfa55138e retention: Add docstring info on how archive cleaning works.
In particular, it's important to record the special treatment around
ArchivedAttachment rows not being deleted in this step.
2022-06-08 15:12:36 -07:00
David Rosa
52ef574d3e help-docs: Fix minor errors in "Link to a message or conversation".
Removes the ":" which have accidentally ended up in the "Get a link
to a specific topic" and "Get a link to a specific stream" headings.

Renames the "Via browser's address bar" tab to "Web" so that it
stays consistent with other help center articles.

Fixes part of #22147.
2022-06-08 12:20:32 -07:00
Zixuan James Li
426f8ce385 tests: Replace HttpResponse with TestHttpResponse.
Since `HttpResponse` is an inaccurate representation of the
monkey-patched response object returned by the Django test client, we
replace it with `_MonkeyPatchedWSGIResponse` as `TestHttpResponse`.

This replaces `HttpResponse` in zerver/tests, analytics/tests, coporate/tests,
zerver/lib/test_classes.py, and zerver/lib/test_helpers.py with
`TestHttpResponse`. Several files in zerver/tests are excluded
from this substitution.

This commit is auto-generated by a script, with manual adjustments on certain
files squashed into it.

This is a part of the django-stubs refactorings.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-08 11:25:03 -07:00
Anders Kaseorg
d5fea08b8a cache: Remove needless monkey patching.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-07 09:37:43 -07:00
Sahil Batra
dfacd62d93 streams: Update error message returned for administrative access.
We have now decided to not continue with the stream administrator
concept as we are changing the permissions model to be based on
user groups as per #19525. So, this commit updates the error message
to "Must be an organization administrator".
2022-06-07 09:36:24 -07:00
Aman Agrawal
085443c722 home: Get default_language info from cookie for spectator.
This allows us to set default_language as cookie in frontend which
can be used on server load to send the relevant
translation data to the client.
2022-06-01 17:08:00 -07:00
Zixuan James Li
c572d9be5a typing: Add none-checks for db queries.
Signed-off-by: Zixuan James Li <359101898@qq.com>
2022-05-31 09:43:55 -07:00
Zixuan James Li
4a5043dd6e typing: Add none-checks for miscellaneous cases.
Signed-off-by: Zixuan James Li <359101898@qq.com>
2022-05-31 09:43:55 -07:00
Zixuan James Li
e338ada66c typing: Add none-checks for Recipient objects.
Signed-off-by: Zixuan James Li <359101898@qq.com>
2022-05-31 09:43:55 -07:00
Zixuan James Li
bb6a934c8d typing: Add appropriate none-checks for LOCAL_UPLOADS_DIR.
This is a part of django-stubs refactorings.

Signed-off-by: Zixuan James Li <359101898@qq.com>
2022-05-31 09:43:55 -07:00
Zixuan James Li
63e9ae8389 typing: Apply trivial fixes to adjust edge cases in typing.
Add none-checks, rename variables (to avoid redefinition of
the same variable with different types error), add necessary
type annotations.

This is a part of #18777.

Signed-off-by: Zixuan James Li <359101898@qq.com>
2022-05-30 12:03:51 -07:00
Sahil Batra
708204290e streams: Capitalize "ID" in invalid stream errors in API.
This commit changes the error message from "Invalid stream id"
to "Invalid stream ID" for cases where invalid stream IDs are
passed to API endpoints to make it consistent with other similar
error messages.
2022-05-27 17:06:03 -07:00
Zixuan James Li
44ecd66eae types: Better types for API fields.
Signed-off-by: Zixuan James Li <359101898@qq.com>
2022-05-27 14:43:00 -07:00
Zixuan James Li
3e95b59f2e soft_deactivation: Add a helper for queuing soft_reactivation.
Signed-off-by: Zixuan James Li <359101898@qq.com>
2022-05-27 14:28:52 -07:00
Anders Kaseorg
bb6bd900cd response: Replace response.asynchronous attribute with new class.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-27 14:27:34 -07:00
Alex Vandiver
134977b590 email: Attempt to suppress email auto-replies.
This applies a commonly-used, though non-RFC, header which suppresses
auto-replies to the message.  There is a small chance that this will
result in bad filters thinking the messages *from Zulip* are
themselves auto-replies, but this seems a small risk.

Fixes: #13193.
2022-05-26 18:04:12 -07:00
Mateusz Mandera
4e1befa209 sessions: Rename get_session_user to get_session_user_id.
The function returns a user id, not a user - we generally try to reflect
that in function names accurately.
2022-05-23 15:57:10 -07:00
Mateusz Mandera
66d8c2ae15 sessions: Don't filter by expire_date in delete_realm_user_sessions.
The other delete_... functions just do .all() - we should be consistent
one way or another.
2022-05-23 15:57:10 -07:00
Mateusz Mandera
003d086f2e sessions: Only fetch user ids in delete_realm_user_sessions.
No need to fetch entire UserProfiles, especially since we're dealing
with all UserProfiles in a Realm here, which would be quite large.
2022-05-23 15:57:09 -07:00
Alex Vandiver
21ec1f07d4 name_restrictions: Add your-org.zulipchat.com as a reserved name.
This is used as a placeholder in the mobile app.
2022-05-17 14:58:31 -07:00
Sahil Batra
dfd7902c77 user_groups: Rename subgroups fields to direct_subgroup_ids.
This commit renames subgroups and subgroup_ids field sent in user
group objects to direct_subgroup_ids for better readability.
2022-05-17 14:51:45 -07:00
Alex Vandiver
513fcb7bd5 Revert "settings: Add web-public streams beta subdomain list."
This reverts commit 20368a936c.  It is
no longer in beta, and this configuration is no longer needed.
2022-05-16 14:49:14 -07:00
Alex Vandiver
a6d1c41827 email_mirror: Handle invalid charset="..." values. 2022-05-16 12:04:38 -07:00
Anders Kaseorg
0043c0b6b2 django: Use HttpRequest.headers.
Fixes #14769.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-13 20:42:20 -07:00
Tim Abbott
24f527cb59 home: Don't send /register response for spectators.
In very large communities, computing page_params can be quite
expensive. Because we've moved the homepage for communities with web
public streams enabled to be the Zulip app, and it's common for
automation to frequently poll the homepage of a Zulip organization,
we'd like to keep those homepages cheap (as the login pages are).

We address this by prototyping something we may end up wanting to do
anyway -- having the web application do a `POST /register` API call in
order to fetch most page_params, and merging those with the mostly
webapp configuration page_params that we leave in the / response for
convenience.

This exact implementation is messy in a few ways:

* We rely on the assumption that ui_init.initialize_everything happens
  before all code that needs to inspect the page_params properties we
  are fetching via /register. This is likely mostly true, but nothing
  in the implementation enforces it.
* The bundle of ~25 keys that are in page_params ideally would be
  considered individually, with some moved to the /register API
  response and perhaps others eliminated or namespaced inside a
  webapp_settings object.
* It's weird to have the spectators network sequence different that
  from logged-in users, and potentially a maintainability risk.
* We might be able to arrange that the initial `/` response be
  cacheable, now that we're no longer embedding our metadata inside
  it. We've made no effort to do that as of yet.

Despite those issues, this commit solves an immediate problem and will
give us helpful experience with a model closer to the one we'll want
in order to happily support a web client that can be run locally
against a production Zulip server's data.

Co-authored-by: Anders Kaseorg <anders@zulip.com>
2022-05-05 15:20:46 -07:00
Tim Abbott
2e86ea6540 events: Add support for spectator access to /register.
This is necessary for the mobile/terminal clients to build spectator
support down the line. We'll also be using it for the web application,
in an upcoming commit.
2022-05-05 15:20:44 -07:00
Tim Abbott
53518e8a24 events: Pass an explicit realm to do_events_register. 2022-05-05 15:17:07 -07:00
Tim Abbott
d180f7ccfc right sidebar: Clean up rendered description logic.
Previously, we were masking the realm_description raw Markdown with
rendered Markdown, which was a type error.

When we switch to calling /register explicitly in a few commits, this
results in a bug, since the raw Markdown ends up taking priority.

Fix this by just using a different name for this different concept.
2022-05-05 15:17:07 -07:00
yogesh sirsat
08e8a21da0 custom_profile_fields: Clarify an external account error message.
This error message is for a very precise situation -- the pattern not
having the desired format. We should say that, rather than a generic
"Malformed".
2022-05-04 17:57:44 -07:00
NerdyLucifer
6a5d646739 settings (admin/org): Show error for same choices in "list of options".
Currently an user can create multiple options with same text/label in
the select/"list of options" custom profile field type.

Fix this issue by extending the validator to throw an error if there
are duplicate choices in the "list of options" in custom profile
field.

Tweaked by tabbott to use a simpler check.

Fixes: #21880
2022-05-04 17:55:28 -07:00
Sahil Batra
9a6886f630 users: Always pass delivery_email in user's own object.
This commit changes the code to always pass delivery_email
field in the user's own object in 'realm_users'.

This commit also fixes the events sent by notify_created_user.
In the "realm_user/add" event sent when creating the user,
the delivery_email field was set according to the access
for the created user itself as the created user was passed as
acting_user to format_user_row. But now since we have changed
the code to always allow the user themselves to have access
to the email, this bug was caught in tests and we fix the person
object in the event to have delivery_email field based on whether
the user receiving the event has access to email or not.
2022-05-04 12:52:43 -07:00
Sahil Batra
aa98b39429 bulk_create: Copy realm-level defaults in bulk_create_users.
This commit adds code to copy the realm-level default of
settings while creating users through bulk_create_users.

We do not directly call 'copy_default_settings' as it
calls ".save()" but here we want to bulk_create the objects
for efficiency.

We also add the code to set realm-default of enter_sends as
True for the Zulip dev server as done in 754b547e8 and thus
we remove enter_sends argument from create_user_profile as
it is of no use now.
2022-05-04 12:52:43 -07:00
Sahil Batra
8ff0cec2ed users: Pass email_address_visibility as parameter to can_access_delivery_email.
This is a prep commit for adding user-level email visibility setting.
2022-05-04 12:52:43 -07:00
Anders Kaseorg
f29553d809 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-03 10:10:06 -07:00
Anders Kaseorg
7acb642fa5 requirements: Upgrade to Tornado 6.
Fixes #8913.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00