This commit updates the Slack incoming webhook endpoint to use the same
Slack reformatting functions (`convert_to_zulip_markdown` and
`replace_links`) that are used for Slack data import and cleaning up any
duplicative functions.
This was previously not possible because the Slack reformatting regex in
`slack_message_conversion.py` could not handle these cases:
- Formatting applied to non-ASCII characters (e.g., emoji).
- Formatted strings separated by exactly one character.
- Formatted strings appearing immediately after a new line.
Fixes part of #31162.
The Slack text regexes match specific characters before and after the
formatted string to ensure that they only match at word boundaries.
However, because the capture groups consume each matching character,
including the characters used to determine word boundaries, two
formatted strings separated by a single matching character result in one
string not being matched, as the trailing whitespace is already
consumed and cannot also match as the leading pre-match whitespace for
the next character.
Switch to a look-ahead regex for the trailing word boundary
characters. This is zero-width, and as such the next match can still
also consume the same characters.
This also fixes Slack webhook integrations'
`test_message_with_complex_formatted_mentions` which was previously
expecting false output.
Fixes part of #30827.
This prep commit modifies the Slack text regex in
`slack_message_conversion.py` to use the `re.MULTILINE` flag capture
formatted strings that are adjacent to newline or end of line.
These kinds of strings are likely not uncommon to be generated by Slack
exporter itself and our Slack message block and attachment formatters
(`render_blocks` and `render_attachments`) also produces them.
This also fixes Slack webhook integration's
`test_message_with_complex_formatted_texts` which was previously
expecting false output.
Fixes part of #30827.
The Jira plugin integration doc will be moved in the next commit, to
"static/generated/integrations/jira/doc.md", as the directory name of
the integration is "jira" in zulip/python-zulip-api.
- Jira integration doc (previously) - "zerver/webhooks/jira/doc.md"
- Jira plugin integration doc (next commit) -
"static/generated/integrations/jira/doc.md"
Both of these will use the same path "jira/doc.md" as their
integration.doc value, and the actual file is loaded based on the order
of template directories listed in computed_settings.py.
Hence, use a custom path for the Jira integration doc to avoid this
collision.
This commit renames "(no topic)" to "" when used as
topic name, while sending or scheduling messages.
This is particularly helpful for older clients where
sending messages with empty topic input box resulted
in messages being sent to "(no topic)" topic.
Now, it will be sent in empty string topic and will
appear in the "general chat" topic.
This helps older clients to somewhat align with the UX
of the clients supporting empty string topic.
- Made the branch-filtering checks uniform across all the integrations,
by adding a helper function to git.py, and re-using it.
- Instead of checking if the name of the branch that generated the
event is a substring of the "branches" parameter, we now check if
there's an exact match.
For example, if there are two branches named "main" and
"release/v1.0-main", and the user wants to track pushes to only the
"release/v1.0-main" branch, they wouldn't have been able to
previously, it will always track pushes to both branches. There was no
way to filter out the smaller named branch when there were overlaps.
Edited the git-append-branches.md macro to match the previous
git-branches-additional-feature.md macro, and moved it inside the
webhook-url-with-bot-email.md macro, as it's only ever used
immediately after that macro.
Updated the Git-related integrations that support branch filtering:
- Removed the branch-filtering configuration option, and its macro.
- Added a new macro that integrates the branch-filtering instruction
with the Generate-URL step.
The new macro `generate-webhook-url-with-branch-filtering` builds off of
the `generate-webhook-url-basic` macro, adding only the branch filtering
instruction. But, it does not re-use the macro, instead duplicates it,
to avoid the newline break that would be added otherwise.
This commit updates the GitHub webhook integration to include the
old issue number in the transfer message sent for the new repository,
alongside the old repository's full name.
On transferring an issue to another repository, GitHub sends
- a "transferred" event to the old repository's webhook URL,
- and an "opened event", containing links to the old issue,
to the new repository's webhook URL.
This commit adds support for both events independently.
Fixes: #33450.
The integrations - GoSquared, Greenhouse, OpsGenie and Zendesk - have
been using `000.png` as their example screenshots, renamed them to
match the default name `001.png`, to avoid passing them as arguments.
The context property was previously being used in the macro
`git-webhook-url-with-branches` which has been removed as it is not in
use any longer.
The context property was now being used only in 2 integration docs, both
of which used it only in a single location, and directly used their own
names as the channel name. Making the context property unnecessary.
- Add a step to update the bot's API key if using a generic bot
for bidirectional bridging.
- Clarify setup options for how Slack channels are mapped.
- Edit for clarity and succinctness.
1. Fetching from the `/users.list` endpoint is supposed to use
pagination. Slack will return at most 1000 results in a single
request. This means that our Slack import system hasn't worked
properly for workspaces with more than 1000 users. Users after the
first 1000 would be considered by our tool as mirror dummies and thus
created with is_active=False,is_mirror_dummy=True.
Ref https://api.slack.com/methods/users.list
2. Workspaces with a lot of users, and therefore requiring the use of
paginated requests to fetch them all, might also get us to run into
Slack's rate limits, since we'll be doing repeating requests to the
endpoint.
Therefore, the API fetch needs to also handle rate limiting errors
correctly.
Per, https://api.slack.com/apis/rate-limits#headers, we can just read
the retry-after header from the rsponse and wait the indicated number
of seconds before repeating the requests. This is an easy approach to
implement, so that's what we go with here.
In Slack webhook integration setup, the only scenario where a message
payload contains a channel mention without the channels name is when the
user mentions a private channel.
e.g <#C07AVLQ3AUQ|>
This commit updates the string we use for such mention to "private Slack
channel" for better clarity.
Previously, the notification message only informed users that the
integration URL was registered with Slack's Event API. However, it might
be misleading to send an "integration is successful" message when this
happens because we didn't verify that the token has the required scopes
or if the Slack token is even added to the URL at all.
Now that the integration also verifies the `slack_app_token`'s scope,
it's now more appropriate to send a notification message like
`get_setup_webhook_message` to let the user know that the setup is
indeed successful.
GitHub generates spurious edited events on hiding/unhiding a comment,
and on clicking "edit"/"update" without any changes, when the
comment "body" contains certain specific characters.
Instead of sending a notification message, we suppress those edge cases
by comparing the "body" content of the comment and changes in the event
payload.
Fixes#31042, #26136.
This commit makes the Slack incoming webhook use the original
`convert_slack_formatting` and `convert_slack_workspace_mentions` from
`slack_message_conversion.py`. Previously those were not refactored out
into two smaller functions yet.
Fixes part of #30827.
Previously the `check_token_access` is called for every request we get
from Slack webhook, this may introduce significant latency.
This commit moves `check_token_access` to the same condition for when we
need to handle Slack challenge handshake so that we only do API token
check once per URL registered.
Additionally, we now check for the specific scopes that we need to run
the Slack webhook integration (SLACK_INTEGRATION_TOKEN_SCOPES).
Fixes part of #30827.
Previously, the `check_token_access` function had a hardcoded
`required_parameters` variable because it was only used in the Slack
data importer. This commit refactors `required_parameters` into a
function parameter, enabling the function to check a Slack token’s scope
for other purposes, such as Slack webhook integration.
Additionally, this commit changes the Slack API call in
`check_token_access` from `teams.info` to `api.test`. The endpoint is
better suited for this purpose since we're only checking a token’s scope
using the response header here.
The doc now shows instructions to setup the integration using
Slack's Events API instead of the legacy Outgoing webhook
service.
Co-authored-by: Alya Abbott <alya@zulip.com>
Co-authored-by: Lauryn Menard <lauryn@zulip.com>
A Slack fail condition occurs when we don't respond with HTTP 200 within
3 seconds after Slack calls our endpoint. If this happens, Slack will
retry sending the same payload. This is often triggered because we need
to perform callbacks when converting messages. To avoid sending the same
message multiple times, we block subsequent retry calls from Slack.
This commit returns early HTTP 200 response as soon as we get any retry
calls from Slack.
Part of #30465.
Payloads from Slack's Events API don't include human-readbale
information for a Slack username and for Slack channels.
This commit makes the Slack integration callback to the relevant
Slack API endpoints for that information.
Fixes#30074.
This commit adds 4 new payload fixtures to the Slack
integration for messages we don't process properly yet:
- Messages with code blocks.
- Messages with overlapping text formatting (e.g, bold
and italic).
- Messages with formatted mentions.
- Messages with quote blocks.
The follow-up plan to add support for these kind of
Slack messages is tracked in #30827.
Part of #30465.
This updates the Slack webhook integration to handle the Slack Events
API[1], while maintaining backwards compatibility with Slack's legacy
Outgoing Webhook service.
The Events API introduces the "challenge" handshake[2] to verify and
add a new webhook URL for them to call. This commit adds a handler for
the challenge handshake.
Additionally, this commit reformats incoming payloads using the Slack
text reformatter from `slack_message_conversion.py`. There is some
duplicative code here because of the difference in Slack export data
and Slack's webhook payload.
Part of #30465
[1]: https://api.slack.com/apis/events-api#using-events-api
[2]: https://api.slack.com/apis/events-api#handshake
