1. Fetching from the `/users.list` endpoint is supposed to use
pagination. Slack will return at most 1000 results in a single
request. This means that our Slack import system hasn't worked
properly for workspaces with more than 1000 users. Users after the
first 1000 would be considered by our tool as mirror dummies and thus
created with is_active=False,is_mirror_dummy=True.
Ref https://api.slack.com/methods/users.list
2. Workspaces with a lot of users, and therefore requiring the use of
paginated requests to fetch them all, might also get us to run into
Slack's rate limits, since we'll be doing repeating requests to the
endpoint.
Therefore, the API fetch needs to also handle rate limiting errors
correctly.
Per, https://api.slack.com/apis/rate-limits#headers, we can just read
the retry-after header from the rsponse and wait the indicated number
of seconds before repeating the requests. This is an easy approach to
implement, so that's what we go with here.
In Slack webhook integration setup, the only scenario where a message
payload contains a channel mention without the channels name is when the
user mentions a private channel.
e.g <#C07AVLQ3AUQ|>
This commit updates the string we use for such mention to "private Slack
channel" for better clarity.
Previously, the notification message only informed users that the
integration URL was registered with Slack's Event API. However, it might
be misleading to send an "integration is successful" message when this
happens because we didn't verify that the token has the required scopes
or if the Slack token is even added to the URL at all.
Now that the integration also verifies the `slack_app_token`'s scope,
it's now more appropriate to send a notification message like
`get_setup_webhook_message` to let the user know that the setup is
indeed successful.
This commit makes the Slack incoming webhook use the original
`convert_slack_formatting` and `convert_slack_workspace_mentions` from
`slack_message_conversion.py`. Previously those were not refactored out
into two smaller functions yet.
Fixes part of #30827.
Previously the `check_token_access` is called for every request we get
from Slack webhook, this may introduce significant latency.
This commit moves `check_token_access` to the same condition for when we
need to handle Slack challenge handshake so that we only do API token
check once per URL registered.
Additionally, we now check for the specific scopes that we need to run
the Slack webhook integration (SLACK_INTEGRATION_TOKEN_SCOPES).
Fixes part of #30827.
Previously, the `check_token_access` function had a hardcoded
`required_parameters` variable because it was only used in the Slack
data importer. This commit refactors `required_parameters` into a
function parameter, enabling the function to check a Slack token’s scope
for other purposes, such as Slack webhook integration.
Additionally, this commit changes the Slack API call in
`check_token_access` from `teams.info` to `api.test`. The endpoint is
better suited for this purpose since we're only checking a token’s scope
using the response header here.
A Slack fail condition occurs when we don't respond with HTTP 200 within
3 seconds after Slack calls our endpoint. If this happens, Slack will
retry sending the same payload. This is often triggered because we need
to perform callbacks when converting messages. To avoid sending the same
message multiple times, we block subsequent retry calls from Slack.
This commit returns early HTTP 200 response as soon as we get any retry
calls from Slack.
Part of #30465.
Payloads from Slack's Events API don't include human-readbale
information for a Slack username and for Slack channels.
This commit makes the Slack integration callback to the relevant
Slack API endpoints for that information.
Fixes#30074.
This updates the Slack webhook integration to handle the Slack Events
API[1], while maintaining backwards compatibility with Slack's legacy
Outgoing Webhook service.
The Events API introduces the "challenge" handshake[2] to verify and
add a new webhook URL for them to call. This commit adds a handler for
the challenge handshake.
Additionally, this commit reformats incoming payloads using the Slack
text reformatter from `slack_message_conversion.py`. There is some
duplicative code here because of the difference in Slack export data
and Slack's webhook payload.
Part of #30465
[1]: https://api.slack.com/apis/events-api#using-events-api
[2]: https://api.slack.com/apis/events-api#handshake
Due to the channel_map_to_topics URL parameter in the Slack webhook,
it was not migrated to use the check_send_webhook_message.
By using check_send_webhook_message, any topic parameter in the
webhook URL will be prioritized over mapping Slack channels to
topics, e.g. when channel_map_to_topics is true. This is because
the default behaviour for incoming webhooks is to send a default
topic as a parameter to check_send_webhook_message in case there
is no topic specified in the URL.
In contrast, we can override the stream passed in the URL when
channel_map_to_topics is false by passing the Slack channel name
to check_send_webhook_message. The default behaviour for incoming
webhooks is to send a direct message if there is no specified
stream in the URL, so a default stream is not generally passed
to check_send_webhook_message.
Fixes#27601.
This is preparatory work towards adding a Topic model.
We plan to use the local variable name as 'topic' for
the Topic model objects.
Currently, we use *topic as the local variable name for
topic names.
We rename local variables of the form *topic to *topic_name
so that we don't need to think about type collisions in
individual code paths where we might want to talk about both
Topic objects and strings for the topic name.
These webhooks have some URL query params that do not need additional
validation or parsing from JSON. So WebhookPaylaod is not applicable to
these webhooks.
Black 23 enforces some slightly more specific rules about empty line
counts and redundant parenthesis removal, but the result is still
compatible with Black 22.
(This does not actually upgrade our Python environment to Black 23
yet.)
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Adds request as a parameter to json_success as a refactor towards
making `ignored_parameters_unsupported` functionality available
for all API endpoints.
Also, removes any data parameters that are an empty dict or
a dict with the generic success response values.
Prior to this commit, we wrapped all incoming messages from Slack
in backticks. This led to weird formatting errors when an incom-
ing message from Slack contains backticks, to refer to a function
name, for instance.
This utilizes the generic `BaseNotes` we added for multipurpose
patching. With this migration as an example, we can further support
more types of notes to replace the monkey-patching approach we have used
throughout the codebase for type safety.
This concludes the HttpRequest migration to eliminate arbitrary
attributes (except private ones that are belong to django) attached
to the request object during runtime and migrated them to a
separate data structure dedicated for the purpose of adding
information (so called notes) to a HttpRequest.
JsonableError has two major benefits over json_error:
* It can be raised from anywhere in the codebase, rather than
being a return value, which is much more convenient for refactoring,
as one doesn't potentially need to change error handling style when
extracting a bit of view code to a function.
* It is guaranteed to contain the `code` property, which is helpful
for API consistency.
Various stragglers are not updated because JsonableError requires
subclassing in order to specify custom data or HTTP status codes.
django.utils.translation.ugettext is a deprecated alias of
django.utils.translation.gettext as of Django 3.0, and will be removed
in Django 4.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Generated by pyupgrade --py36-plus --keep-percent-format, but with the
NamedTuple changes reverted (see commit
ba7906a3c6, #15132).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Generated by `pyupgrade --py3-plus --keep-percent-format` on all our
Python code except `zthumbor` and `zulip-ec2-configure-interfaces`,
followed by manual indentation fixes.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
There are only a handful of non-JSON webhooks that wouldn't
benefit from the notify_bot_owner_on_invalid_json feature.
Specifically, these are the webhooks where the third-party product
uses another format, whether it be HTML form-encoded, XML, or
something else.
Tweaked by tabbott to correc the list of excluded webhooks.
This commit migrates all webhooks to use check_send_stream_message
instead of check_send_message. The only two webhooks that still
use check_send_message are our yo and teamcity webhooks. They
both use check_send_message for private messages.
If an incoming payload contained a unicode character, it raised
a UnicodeEncodeError, because the message template was an str. Now,
the message template is unicode, so it can be formatted to include
unicode characters, should the incoming payloads contain any.
Previously, api_key_only_webhook_view passed 3 positional arguments
(request, user_profile, and client) into a function. However, most
of our other auth decorators only pass 2 positional arguments. For
the sake of consistency, we now make api_key_only_webhook_view set
request.client and pass only request and user_profile as positional
arguments.
Adds a new webhook integration for Slack to receive messages
from one's Slack team's public channels.
Contains negative tests for broken, missing or invalid data.
Allows two different option for integration:
1. Receive notification on a single stream with different topics
for each of Slack's public channels.
2. Receive notification on different streams for each of Slack's
public channels.
Steps to choose between the two options is described in the documentation.
Fixes#3569.
