Commit Graph

52024 Commits

Author SHA1 Message Date
Satyam Bansal
d3048b8791 integrations: Rename fixture in Sentry integration.
This is done to clarify from where this fixture is coming from; as there
are two documented ways to test the integration.

(cherry picked from commit fdc14ee3f0)
2023-08-23 03:06:07 +00:00
Zixuan James Li
ec8a284ad5 webhooks: Use 200 status code for unknown events.
Because the third party might not be expecting a 400 from our
webhooks, we now instead use 200 status code for unknown events,
while sending back the error to Sentry. Because it is no longer an error
response, the response type should now be "success".

Fixes #24721.

(cherry picked from commit 84723654c8)
2023-08-23 03:06:07 +00:00
Alex Vandiver
61b5577cf4 sentry: Reduce http timeout.
This helps reduce the impact on busy uwsgi processes in case there are
slow timeout failures of Sentry servers.  The p99 is less than 300ms,
and p99.9 per day peaks at around 1s, so this will not affect more
than .1% of requests in normal operation.

This is not a complete solution (see #26229); it is merely stop-gap
mitigation.

(cherry picked from commit a076d49be7)
2023-08-23 03:06:07 +00:00
Lauryn Menard
21c5ea1444 sentry-webhook: Revise documentation page to be clearer.
(cherry picked from commit 3d8090a116)
2023-08-23 02:48:43 +00:00
Satyam Bansal
421ce4ffe0 integrations: Add support for sample events in Sentry Integration.
Fixes #25778.

(cherry picked from commit 16563a3217)
2023-08-23 02:29:07 +00:00
Satyam Bansal
792748ae63 integrations: Add exception fixture for Vue in Sentry Integration.
(cherry picked from commit 580d8c4dfe)
2023-08-23 02:29:07 +00:00
Satyam Bansal
1252fbe434 integrations: Check for Raven SDK only on python in Sentry Integration.
Fixes part of #25778.

(cherry picked from commit 3bdb806fba)
2023-08-23 02:29:07 +00:00
Satyam Bansal
d026f35c5b integrations: Add support for Rails backend in Sentry Integration.
(cherry picked from commit 142e455d81)
2023-08-23 02:29:07 +00:00
Satyam Bansal
c7839ff084 integrations: Update documentation for Sentry Integration.
(cherry picked from commit 9e793c37e6)
2023-08-23 02:29:07 +00:00
Greg Price
5b4f5a89fb docs: Consistently say "18 months" on compatibility, no specific version numbers.
Previously I've wanted to have this page spell out the concrete
version number that our clients support, rather than the policy we
use for determining that version number, because that's the sort of
question that I feel like as a user I'd want a straight answer to
and would be annoyed if I couldn't get one.

But as the text stands, it's come to look more like it's the policy
(something that's heavyweight to change) than like the value that
the policy currently happens to work out to.  Also, because this page
is kind of chaotically organized (and fixing that is a bigger yak
than I want to shave right now), it repeats the 18-month rule in
three separate places and the current value (version 4.0) is in
a fourth separate place, so it looks internally inconsistent.

Let's therefore take a different tack: like in those other three
spots on this page, state just the policy instead of the value it
currently works out to; but also add a link to help the reader
pin down for themselves what value that does work out to.

This also means we no longer need to update this page as old releases
age and that value advances.

Also fix a typo, and cut the reference to working degraded on
older releases.  Starting earlier this year we finally started
hard-refusing such connections:
  https://github.com/zulip/zulip-mobile/issues/5102
  https://github.com/zulip/zulip-mobile/pull/5633
(which was because there were some swathes of compatibility code
that we could only cut if we completely broke the handling of
ancient servers, and so we preferred to have the app communicate
that break clearly up front.)

(cherry picked from commit bb6fe0385e)
2023-08-23 02:29:07 +00:00
Anders Kaseorg
f54f07f6cb requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 113ac6eb98)
2023-08-23 02:29:07 +00:00
Anders Kaseorg
f1595cf7db requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit e32366638a)
2023-08-23 02:29:07 +00:00
Anders Kaseorg
e953ff71e4 tools: Use subprocess.check_call where appropriate.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 041dcdfbad)
2023-08-23 02:29:07 +00:00
Anders Kaseorg
47e2b07316 ruff: Fix PLW1510 subprocess.run without explicit check argument.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit c43629a222)
2023-08-23 02:29:07 +00:00
Anders Kaseorg
4ac849d006 ruff: Appease SIM118 "class" not in uncle.keys().
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 36dde99308)
2023-08-23 02:29:07 +00:00
Anders Kaseorg
43b09fd89d ruff: Fix PYI032 Prefer object for the second parameter to __eq__.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit ec00c2970f)
2023-08-23 02:29:07 +00:00
Anders Kaseorg
8a1704f765 ruff: Fix E721 Do not compare types, use isinstance().
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 53e8c0c497)
2023-08-23 02:29:07 +00:00
Anders Kaseorg
96be7b72c2 ruff: Collapse short multi-line import statements.
isort did this by default, though it’s unclear whether that was
intended; see https://github.com/astral-sh/ruff/issues/4153.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 733083c65d)
2023-08-23 02:29:07 +00:00
Anders Kaseorg
99b37de89f ruff: Fix UP032 Use f-string instead of format call.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit e932e2ce52)
2023-08-23 02:29:07 +00:00
Anders Kaseorg
323223db4c ruff: Fix SIM118 Use k not in d instead of k not in d.keys().
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 74d6d76046)
2023-08-23 02:29:07 +00:00
Anders Kaseorg
91e5ef39eb ruff: Fix PLR1714 Consider merging multiple comparisons.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 2ae285af7c)
2023-08-23 02:29:05 +00:00
Anders Kaseorg
b9aa772885 ruff: Fix RUF015 Prefer next(...) over single element slice.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 3b09197fdf)
2023-08-23 02:28:06 +00:00
Anders Kaseorg
a0ce536fa4 tests: Remove compatibility code for Python < 3.5.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 4e645c8ff9)
2023-08-23 02:28:06 +00:00
Anders Kaseorg
3325f2ef06 ruff: Fix UP032 Use f-string instead of format call.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit bca5564c1e)
2023-08-23 02:28:06 +00:00
Lauryn Menard
920c73a64e markdown-processor: Update insertion_index check for multiple classes.
Updates find_proper_insertion_index to check for the inline image
classes as matching at least one of the classes in the element's
attrib["class"] so that cases where an inline preview image has
multiple classes, like YouTube video previews, will have the
correct insertion index.

Fixes #26186.

(cherry picked from commit d84fd73db4)
2023-08-23 01:47:45 +00:00
Aman Agrawal
9c272d6ca7 portico_signin: Fix back to login button alignment.
(cherry picked from commit eda91378ec)
2023-08-23 01:41:09 +00:00
Alex Vandiver
7482a7e61f middleware: Allow HTTP from localhost, not through a reverse proxy.
In servers with `application_server.http_only = true` and
`loadbalancer.ips` set, the DetectProxyMisconfiguration middleware
prevents access over HTTP from IP addresses other than the
loadbalancer.

However, this misses the case of access from localhost over HTTP,
which is safe and expected -- for instance, the `email-mirror-postfix`
script used in the email gateway[^1] will post to `http://localhost/`
by default in such configurations.  With the
DetectProxyMisconfiguration installed, this will result in a 403
response.

Make an exception for requests from `127.0.0.1` and `::1` from
proxy-misconfiguration rejections.

[^1]: https://zulip.readthedocs.io/en/latest/production/email-gateway.html

(cherry picked from commit 5368d1bd4c)
2023-08-17 15:11:43 -04:00
Daniil Fadeev
4c4ec78a3a emails: Fix the issues with image width overflow.
Images in the follow-up day 2 email were overflowing the main email
layout. To prevent that, we created a separate class to handle this.

(cherry picked from commit 673e98e982)
2023-08-15 17:09:35 +00:00
Daniil Fadeev
62b1652b3a emails: Fix the image source for the follow-up day 2 email.
Copied the image from the `/help` directory to the `/emails` directory
 to use it with the `email_images_base_url` variable.

(cherry picked from commit 20dc70d395)
2023-08-15 17:09:35 +00:00
Alex Vandiver
931d989f53 i18n: Update translation data from Transifex. 2023-08-15 01:14:44 +00:00
Alex Vandiver
dbd0457f6e docs: Fix link to "Uprading PostgreSQL" section.
(cherry picked from commit ae3458a294)
2023-08-14 22:58:31 +00:00
David Rosa
1ae5194619 docs: Update .html links pointing to "Upgrade Zulip" or "Modify Zulip".
Follow-up to #24089.

(cherry picked from commit d205850d54)
2023-08-14 22:06:10 +00:00
Alex Vandiver
4331114329 topic: Set a max batch_size on bulk_upate call.
The number of affected objects may be quite high, and they are
selected by `id IN (...)` query, and updated with a giant `CASE`.
This turns out to be quadratic, and can cause large queries to take
hours, in a state where they cannot be terminated, when PostgreSQL >11
tries to JIT the query.

Set a batch_size as a stopgap performance fix before moving to
`.update()` as a real fix.

(cherry picked from commit 570ff08fde)
2023-08-14 22:02:42 +00:00
Lauryn Menard
f907102842 docs: Add multiple-organizations.md to production section of TOC.
Adds the existing article about hosting multiple Zulip organizations
to the production section of the table of contents.
2023-08-11 11:12:20 -07:00
Anders Kaseorg
1338cc32b4 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 29bdaaf5b5)
2023-08-10 17:01:52 -05:00
Anders Kaseorg
3f265b257a profile_request: Support only synchronous responses for now.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 32a8151ce8)
2023-08-10 17:01:52 -05:00
Anders Kaseorg
a715e6aa33 pyproject: Remove mypy exemption for coverage.
coverage added type annotations in 7.2.0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit ff6cf54716)
2023-08-10 17:01:52 -05:00
Anders Kaseorg
291cf12c45 name_restrictions: Update disposable_email_domains usage.
‘blocklist’ was added in 0.0.35 (with backwards compatibility for the
old name), and type annotations were added in 0.0.91 (with only the
new name).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 195efb3802)
2023-08-10 17:01:52 -05:00
Anders Kaseorg
948171a839 ruff: Fix B034 re.split, re.sub should pass keyword arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit d87eea1a67)
2023-08-10 17:01:52 -05:00
Anders Kaseorg
8fdcadb08a ruff: Fix UP032 Use f-string instead of format call.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 50e6cba1af)
2023-08-10 17:01:52 -05:00
Anders Kaseorg
2e9544a9a5 ruff: Fix RUF015 Prefer next(iter(…)) over list(…)[0].
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 0efc662eab)
2023-08-10 17:01:52 -05:00
Anders Kaseorg
a8d6c44db4 openapi: Switch to new openapi_core validation API.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 9bb3d15a79)
2023-08-10 17:01:52 -05:00
Anders Kaseorg
406f9f7852 logging_util: Remove dependence on get_current_request.
Pass the HttpRequest explicitly through the two webhooks that log to
the webhook loggers.

get_current_request is now unused, so remove it (in the same commit
for test coverage reasons).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 63be67af80)
2023-08-10 17:01:52 -05:00
Anders Kaseorg
cc21dc8ec7 sentry: Remove dependence on get_current_request.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit f66e2c3112)
2023-08-10 17:01:52 -05:00
Anders Kaseorg
775d9a889c requirements: Upgrade Python requirements.
(cherry picked from commit 784622ee5d)
2023-08-10 17:01:52 -05:00
Anders Kaseorg
89a3a627be codespell: Correct “requestor” to “requester”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit c09e7d6407)
2023-08-10 17:01:52 -05:00
Anders Kaseorg
a03ea9dc08 django_api: Extract send_event_on_commit helper.
django-stubs 4.2.1 gives transaction.on_commit a more accurate type
annotation, but this exposed that mypy can’t handle the lambda default
parameters that we use to recapture loop variables such as

    for stream_id in public_stream_ids:
        peer_user_ids = …
        event = …

        transaction.on_commit(
            lambda event=event, peer_user_ids=peer_user_ids: send_event(
                realm, event, peer_user_ids
            )
        )

https://github.com/python/mypy/issues/15459

A workaround that mypy accepts is

        transaction.on_commit(
            (
                lambda event, peer_user_ids: lambda: send_event(
                    realm, event, peer_user_ids
                )
            )(event, peer_user_ids)
        )

But that’s kind of ugly and potentially error-prone, so let’s make a
helper function for this very common pattern.

        send_event_on_commit(realm, event, peer_user_ids)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 7657cb4a0f)
2023-08-10 17:01:52 -05:00
Anders Kaseorg
252254db69 middleware: Do not consume StreamingHttpResponse.streaming_content.
streaming_content is an iterator. Consuming it within middleware
prevents it from being sent to the browser.

https://docs.djangoproject.com/en/4.2/ref/request-response/#streaminghttpresponse-objects

“The StreamingHttpResponse … has no content attribute. Instead, it has
a streaming_content attribute. This can be used in middleware to wrap
the response iterable, but should not be consumed.”

Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 98310f269b)
2023-08-10 17:01:52 -05:00
Anders Kaseorg
8a6ebeda87 tests: Remove assert_streaming_content helper in favor of getvalue.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 92c83c1df4)
2023-08-10 17:01:52 -05:00
Anders Kaseorg
7bb554d58b openapi: Convert deprecated Spec.create to Spec.from_dict.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 85681546ce)
2023-08-10 17:01:52 -05:00