Commit Graph

15352 Commits

Author SHA1 Message Date
Anders Kaseorg
4fc3845dc2 tornado: Ignore StreamClosedError.
This was also hidden until 81f7192ca3
(#22301).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-28 16:35:49 -07:00
Anders Kaseorg
0d93ec6214 tornado: Remove dead check for message format.
This was for the old /messages/latest API that was removed in commit
e06722657a.

If we wanted a new check like this, it shouldn’t go in zulip_finish,
because that only runs when the client gets an asynchronous response
from polling an initially-empty queue, and not when the client gets a
synchronous response from polling a nonempty queue.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-28 16:35:49 -07:00
Zixuan James Li
b65401ed47 tests: Tighten signature of the wrapped test client helpers.
We wrap methods of the django test client for the test suite, and
type keyword variadic arguments as `ClientArg` as it might called
with a mix of `bool` and `str`.

This is problematic when we call the original methods on the test
client as we attempt to unpack the dictionary of keyword arguments,
which has no type guarantee that certain keys that the test client
requires to be bool will certainly be bool.

For example, you can call
`self.client_post(url, info, follow="invalid")` without getting a
mypy error while the django test client requires `follow: bool`.

The unsafely typed keyword variadic arguments leads to error within
the body the wrapped test client functions as we call
`django_client.post` with `**kwargs` when django-stubs gets added,
making it necessary to refactor these wrappers for type safety.

The approach here minimizes the need to refactor callers, as we
keep `kwargs` being variadic while change its type from `ClientArg`
to `str` after defining all the possible `bool` arguments that might
previously appear in `kwargs`. We also copy the defaults from the
django test client as they are unlikely to change.

The tornado test cases are also refactored due to the change of
the signature of `set_http_headers` with the `skip_user_agent` being
added as a keyword argument. We want to unconditionally set this flag to
`True` because the `HTTP_USER_AGENT` is not supported. It also removes a
unnecessary duplication of an argument.

This is a part of the django-stubs refactorings.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:12:00 -07:00
Zixuan James Li
4e518a3852 tests: Call client methods with explicit keyword arguments.
This is a prep commit for tightening the types for our wrapped test
client.

The callers of the test client methods are refactored to either call
them without unpacking at all or create a TypedDict for the keyword
arguments to be unpacked. This allows the type checker to know exactly what
keys are present and their corresponding type.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:12:00 -07:00
Zixuan James Li
a3a0545aac typing: Populate POST and _files with cast and setattr.
`POST` is an immutable attribute and `_files` is an internal attribute
of `HttpRequest`. With type annotations provided by `django-stubs`, mypy
stops us from modifying these attributes. This uses `cast` and `setattr`
to avoid typing issues.

This is a part of django-stubs refactorings.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:07:18 -07:00
Zixuan James Li
27be27560b typing: Access LANGUAGE_CODE via django.utils.translation.get_language.
We no longer need to access the internal `LANGUAGE_CODE` attribute by
using `django.utils.translation.get_language`.

A test case overriding the translation is added to ensure the password
reset form sending to users requested from a wrong domain is properly
translated.

This is a part of django-stubs refactorings.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:07:18 -07:00
Zixuan James Li
417e1b5e81 logging_util: Save an assertion with __getitem__.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:05:24 -07:00
Zixuan James Li
fc3de0ba41 typing: Tighten type annotation for serialized user groups.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:05:24 -07:00
Zixuan James Li
5285fbb4d0 typing: Fix type annotation for missing messages in soft reactivation.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:05:24 -07:00
Zixuan James Li
9bfeebf064 user_profile: Fallback to "" for timezone upon creation.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:05:24 -07:00
Zixuan James Li
a9029c68ea tornado: Make _request an attribute on AsyncDjangoHandler.
For the same reason as `handler_id` has, we define `_request`
as an attribute. Note that the name `request` is already taken.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:03:09 -07:00
Zixuan James Li
56e1f3b725 tornado: Make handler_id an attribute on AsyncDjangoHandler.
This prevents us from relying on a side-effect of `allocate_handler_id`
that monkey-patches `handler_id` on the `AsyncDjangoHandler` object,
allowing mypy to acknowledge the existence of `handler_id` as an `int`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:03:09 -07:00
Zixuan James Li
f0e505d557 retention: Guarantee type-safeness when calling move_rows.
This ensures that all the keyword arguments in `move_rows`
have the correct types. Note that `returning_id` is supposed to be a
flag instead of a `Composable` `Literal`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-28 16:01:35 -07:00
Anders Kaseorg
bae4182e47 test_classes: Check responses in ZulipTestCase.register.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-28 13:15:21 -07:00
Anders Kaseorg
869fe60689 markdown: Parse included blocks in a new parser state.
This fixes inclusion of a multi-paragraph file into a list item.

Followup to commit dc33a0ae67 (#22315).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-27 17:37:00 -07:00
Anders Kaseorg
a2e1d61172 integrations: Satisfy Python-Markdown’s archaic 4-space requirement.
Followup to commit dc33a0ae67 (#22315).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-27 17:36:17 -07:00
Anders Kaseorg
53231aa9d9 decorator: Type cache_info, cache_clear for ignore_unhashable_lru_cache.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-27 10:20:05 -07:00
Lauryn Menard
9db8a59a56 integrations-docs: Add line break in GitLab integration doc. 2022-06-27 10:14:51 -07:00
Anders Kaseorg
a7e10ee47e tornado: Send request_started signal in Django thread.
Django’s ASGIHandler does this too and it seems like a good idea.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-27 10:14:02 -07:00
Anders Kaseorg
0dfde9db94 python: Simplify unnecessary getattr calls.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:38:22 -07:00
Anders Kaseorg
3bf8ee2156 python: Unquote some unnecessarily quoted type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:37:41 -07:00
Anders Kaseorg
6fe5cb65a1 templates: Fix Python-Markdown extension type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:37:14 -07:00
Anders Kaseorg
dc33a0ae67 markdown: Rewrite include plugin without markdown-include.
markdown-include is GPL licensed.

Also, rewrite it as a block processor, so that it works correctly
inside indented blocks.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:36:31 -07:00
Anders Kaseorg
7f0e11bd06 markdown: Rename preprocessor_priorities module to priorities.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:36:31 -07:00
Anders Kaseorg
f3254bb558 mattermost: Run html2text as a subprocess.
html2text is GPL licensed.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:32:59 -07:00
Anders Kaseorg
e10b7f2f3c email_notifications: Run html2text as a subprocess.
html2text is GPL licensed.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:32:59 -07:00
Alex Vandiver
5529639b84 change_realm_subdomain: Support skipping the redirect.
Sometimes (e.g. when moving an old realm out of the way of an import
into that name) we do *not* wish to add a redirect realm.  Add a flag
to support that.
2022-06-25 08:44:24 -07:00
Alex Vandiver
3e4d07952c test_signup: Test redirects more explicitly. 2022-06-25 08:44:24 -07:00
Anders Kaseorg
236ef8a077 test_push_notifications: Simplify with Python 3.8 AsyncMock.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-25 08:43:20 -07:00
Anders Kaseorg
5033beb99c request: Replace tornado_handler weak reference with tornado_handler_id.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-25 08:42:23 -07:00
Anders Kaseorg
d03892391a event_queue: Take (typed) keyword parameters for fetch_events.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-25 08:42:23 -07:00
Anders Kaseorg
4d4d1e50cc tornado: Simplify in_tornado_thread usage.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-25 08:42:23 -07:00
Anders Kaseorg
952422d373 test_helpers: Remove default tornado_handler from HostRequestMock.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-25 08:42:23 -07:00
Anders Kaseorg
c1a54f4567 decorator: Only URL-decode application/x-www-form-urlencoded requests.
We previously parsed any request with method other than {GET, POST} and
Content-Type other than multipart/form-data as if it were
application/x-www-form-urlencoded.

Check that Content-Type is application/x-www-form-urlencoded before
parsing the body that way.  Restrict this logic to {DELETE, PATCH,
PUT} (having a body at all doesn’t make sense for {CONNECT, HEAD,
OPTIONS, TRACE}).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-25 08:36:16 -07:00
Zixuan James Li
6400cd93ea test_middleware: Narrow the type of open_graph_description.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 22:09:05 -07:00
Zixuan James Li
cf455131ef message_edit: Make propagate_mode optional.
When there is no topic/stream being change, `propagate_mode` becomes
unnecessary. We add an assertion to ensure that the previous assumption
that `propagate_mode` is not `None` still holds when either `topic_name`
or `new_stream` is not `None`.

We can possibly improve this by overloading `do_update_message` and
`check_update_message`, but that's beyond the scope of the PR and
feasibility of doing that should also be further discussed.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 22:09:05 -07:00
Zixuan James Li
fa51d8718c home: Prevent mypy from inferring the type of page_params.
Without an explicit type annotation, mypy infers the type of
values in `page_params` upon its initialization as a `Union`, while
other computed values haven't been assigned yet. We break this
over-conservative inferred type by annotating `page_params` as a
`Dict[str, object]`.

We could have created a `TypedDict` to have accurate type annotation
for all of the fields, but it does not seem worth it at this point since
the data structure is not widely used in the backend.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 22:09:05 -07:00
Zixuan James Li
67fda5516f import_utils: Fix wrong usage of model_to_dict.
The argument `exclude` expects a `list` or `set` of field names,
not a `str`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 22:09:05 -07:00
Zixuan James Li
0ec561ab57 upload: Add assertions before accessing uploaded files.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 22:09:05 -07:00
Zixuan James Li
428dfd2693 typing: Add assertions for report returned from notify_server_error.
We don't have a specific type for the reports returned from the error
logging handlers. The check is necessary as they are currently typed
as `Dict[str, object]` in `run_handler`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 22:09:05 -07:00
Zixuan James Li
8ba8177dc5 typing: Correct the signautre of prepare_activation_url.
`request.session` is `django.contrib.sessions.backends.base.SessionBase`
`instead of django.contrib.sessions.models.Session`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 22:09:05 -07:00
Zixuan James Li
2ddea50745 typing: Correct type annotation of supported_backends.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 22:09:05 -07:00
Zixuan James Li
1860b0b7ac typing: Fix missing attribtute of RedisUtilsTest.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 22:05:12 -07:00
Zixuan James Li
fd9a0f4274 typing: Apply trivial none-checks with assertions as necessary.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00
Zixuan James Li
58e95cbfae typing: Add none-check for test_users.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00
Zixuan James Li
146b0c482c typing: Add none-check for WebhookIntegration.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00
Zixuan James Li
058dab5818 typing: Add none-checks for stream.recipient_id.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00
Zixuan James Li
944e602788 streams: Refactor bulk_add_subscription handling recipient_ids set.
While it is possible to have `stream.recipient_id` being `None`,
the code works under the assumption that it is not. Potentially
we will get a runtime error, but it is not quite explicit without
the assertion.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00
Zixuan James Li
c567e44408 typing: Avoid redefinition of variables with different types.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00
Zixuan James Li
5fd278bf09 user_settings: Add assertions before attribute access.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00