Commit Graph

1071 Commits

Author SHA1 Message Date
Anders Kaseorg
57a80856a5 python: Convert more "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format.

Now including %d, %i, %u, and multi-line strings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:39:00 -07:00
Anders Kaseorg
0d6c771baf python: Guard against default value mutation with read-only types.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg
91a86c24f5 python: Replace None defaults with empty collections where appropriate.
Use read-only types (List ↦ Sequence, Dict ↦ Mapping, Set ↦
AbstractSet) to guard against accidental mutation of the default
value.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg
69c0959f34 python: Fix misuse of Optional types for optional parameters.
There seems to have been a confusion between two different uses of the
word “optional”:

• An optional parameter may be omitted and replaced with a default
  value.
• An Optional type has None as a possible value.

Sometimes an optional parameter has a default value of None, or None
is otherwise a meaningful value to provide, in which case it makes
sense for the optional parameter to have an Optional type.  But in
other cases, optional parameters should not have Optional type.  Fix
them.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg
365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg
69730a78cc python: Use trailing commas consistently.
Automatically generated by the following script, based on the output
of lint with flake8-comma:

import re
import sys

last_filename = None
last_row = None
lines = []

for msg in sys.stdin:
    m = re.match(
        r"\x1b\[35mflake8    \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
    )
    if m:
        filename, row_str, col_str, err = m.groups()
        row, col = int(row_str), int(col_str)

        if filename == last_filename:
            assert last_row != row
        else:
            if last_filename is not None:
                with open(last_filename, "w") as f:
                    f.writelines(lines)

            with open(filename) as f:
                lines = f.readlines()
            last_filename = filename
        last_row = row

        line = lines[row - 1]
        if err in ["C812", "C815"]:
            lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
        elif err in ["C819"]:
            assert line[col - 2] == ","
            lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")

if last_filename is not None:
    with open(last_filename, "w") as f:
        f.writelines(lines)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 16:04:12 -07:00
Anders Kaseorg
67e7a3631d python: Convert percent formatting to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 15:02:09 -07:00
Anders Kaseorg
6480deaf27 python: Convert more "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, with more
restrictions patched out.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 14:48:09 -07:00
David Wood
f0f42f7a94 integrations: Add Thinkst Canary integration.
This commit adds an integration for Thinkst Canaries - physical, VM and
cloud-based canaries for detecting attackers to a network. Thinkst
Canaries can send webhook alerts when canaries have been tripped, and
this integration will post Zulip messages when these webhooks are
received.

Signed-off-by: David Wood <david@davidtw.co>
2020-06-09 16:28:28 -07:00
Anders Kaseorg
8dd83228e7 python: Convert "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, but with the
NamedTuple changes reverted (see commit
ba7906a3c6, #15132).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:31:20 -07:00
Anders Kaseorg
8c15081069 python: Further pyupgrade changes.
Generated by pyupgrade --py3-plus --keep-percent-format.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:23:10 -07:00
Johannes Hochreiter
d78d9099d1 gitlab: added pipeline number to event notification
- link to pipeline now also features the pipeline number
- updated tests accordingly
2020-06-07 14:01:04 -07:00
Johannes Hochreiter
60077f3097 gitlab: Added artifact link to pipeline event body
- if artifact was built file name, download link and browse link will be added to pipeline event body
- added test for pipeline with built artifacts
2020-06-07 14:01:04 -07:00
Mateusz Mandera
5b746b4128 docs: Add missing step to dropbox webhook instructions. 2020-06-05 10:21:25 -07:00
Anders Kaseorg
1f565a9f41 timezone: Use standard library datetime.timezone.utc consistently.
datetime.timezone is available in Python ≥ 3.2.  This also lets us
remove a pytz dependency from the PostgreSQL scripts.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-05 09:34:17 -07:00
Anders Kaseorg
95f29b5409 tests: Avoid deprecated TestCase method aliases.
Fixes these warnings with python -Wd:

/home/circleci/zulip/zerver/tests/test_middleware.py:43: DeprecationWarning: Please use assertRegex instead.
  self.assertRegexpMatches(
/home/circleci/zulip/zerver/webhooks/travis/tests.py:40: DeprecationWarning: Please use assertNotEqual instead.
  self.assertNotEquals(msg.topic_name(), self.TOPIC)

https://docs.python.org/3/library/unittest.html#deprecated-aliases

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-04 14:15:52 -07:00
Hemanth V. Alluri
9a9c1e0794 webhooks/stripe: Fix the invoice_created event.
The previous code for this event was using a key that's not actually
a part of the payload. So here we simple remove the usage of that key
and add a (previously missing) test for this event.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-06-03 17:18:43 -07:00
Puneeth Chaganti
bb37e4ef00 integrations: Remove corrupted png image in bitbucket docs. 2020-06-01 18:03:06 -07:00
Imran Iqbal
465947d1bb webhooks: Fix travis display emoji for Pending status.
The 🔄 emoji is spelled  :counterclockwise: in Zulip.
2020-05-28 15:02:30 -07:00
Anders Kaseorg
840cf4b885 requirements: Drop direct dependency on mock.
mock is just a backport of the standard library’s unittest.mock now.

The SAMLAuthBackendTest change is needed because
MagicMock.call_args.args wasn’t introduced until Python
3.8 (https://bugs.python.org/issue21269).

The PROVISION_VERSION bump is skipped because mock is still an
indirect dev requirement via moto.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-26 11:40:42 -07:00
Hemanth V. Alluri
ab03659095 webhooks/github: Provide more detailed info for unexpected events.
Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-05-21 12:31:43 -07:00
Imran Iqbal
da084528db feat(travis): display emoji for Pending status
* Encountered `No emoji specified for status 'Pending'` when using `on_start: always`:

```yaml
notifications:
  webhooks:
    ...
    on_start: always    # default: never
```
2020-05-20 13:09:30 -07:00
Hemanth V. Alluri
c8f4177872 webhooks/github: Add a support for team/edited events.
Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-05-16 14:59:08 -07:00
Hemanth V. Alluri
7e7847b55a webhooks/stripe: Add support for pyr_ events.
"pyr_" events are like refund events some source called it a pseudo
refund event (https://stackoverflow.com/questions/46296374/how-can-
i-get-the-original-charge-and-refund-ids-of-an-automatic-payout).
Though due to the lack of any documentation on this event I'm not
confident if this is the right fix.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-05-14 12:02:30 -07:00
Hemanth V. Alluri
20b39bd54e webhooks/stripe: Update refund updated event message.
Also add missing test coverage for this event.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-05-14 12:02:30 -07:00
cestrell
297185cc12 integrations: Use new function in Github releases.
Use get_release_event_message from webhooks/git.py to format release
events using the newly implemented release message template.

Tweaked by tabbott to handle name=None.

Builds on #14746. Proposed in #14934.
2020-05-12 17:42:49 -07:00
cestrell
e3f0b2f20f integrations: Add Gogs webhook for release event.
Extends Gogs integrations in order to support a published release.
Tested on my local Ubuntu development server running on WSL2.

Fixes #14746.
2020-05-12 17:39:38 -07:00
Hemanth V. Alluri
cd69da26dc webhooks/github: Ignore the label event. 2020-05-12 09:32:17 -07:00
Hemanth V. Alluri
4dd957faa2 webhooks/github: Add support for pull request ready for review.
GitHub supports opening a draft/WIP pull request and then marking it
as ready for review later on. This PR supports the ready_for_review
action for pull_request events.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-05-12 09:32:17 -07:00
Tim Abbott
d3c7579730 gitlab: Fix missing coverage/bug for job hooks corner case.
Apparently, the change and test I added before didn't quite cover the
corner case that was broken.  This does, and exposes a second bug as
well, which we fix.
2020-05-11 16:13:23 -07:00
Tim Abbott
d4f875f7c5 webhooks: Add missing gitlab test fixture. 2020-05-11 15:04:34 -07:00
Tim Abbott
11027161c3 webhooks: Fix handling of GitLab Job Hook events.
This fixes an exception for these events.

Fixture from https://docs.gitlab.com/ee/user/project/integrations/webhooks.html#job-events.
2020-05-11 14:37:01 -07:00
arpit551
f3ab8e66dc ci: add configuration for posting to czo whenever CI fails in master.
Since notify is mostly depreciated  in the latest versions of CircleCI.
Although we can use use notify in CircleCI 2.0 but currently
there is no documentation regarding it.

We could have use notify here rather than this hacky solution but
if we use notify it was not possible to trigger CircleCI webhook only
for the main Zulip repository.

Also corrected the circle ci webhook for the case where we don't receive
previous in post request
2020-05-11 13:45:22 -07:00
Jenny Ghose
180c16c80e integrations: Add an incoming webhook for Grafana.
Tweaked by tabbott to use formatted suggested in one of the various
duplicate PRs for this issue, showing the rule name clearly.

Fixes #12951.
2020-05-11 00:27:38 -07:00
Tim Abbott
a0c2121958 docs: Advertise Slack-compatible webhook a bit.
This should make it discoverable enough that users will try it out and
send us feedback.
2020-05-11 00:07:28 -07:00
Chris Heald
c80e913c7a webhooks: Add a webhook capable of parsing Slack payloads.
This adds a webhook that can be used to interpret standard Slack
payloads. Since there are a ton of existing Slack integrations out
there, having a webhook which can accept standard Slack payloads can
significantly ease transition pains. Obviously this can't do everything
that Slack payloads can (particularly WRT their widgets/interactions),
but we can ingest text and parse out multi-block payloads into a message
relatively reasonably.
2020-05-11 00:07:28 -07:00
Anders Kaseorg
8cdf2801f7 python: Convert more variable type annotations to Python 3.6 style.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-08 16:42:43 -07:00
Hemanth V. Alluri
fb757e91c1 webhooks/sentry: Add back support for the legacy integrations.
When the integration was originally rewritten, support for the
deprecated webhook payloads was removed. We later noticed that some
people using Zulip were still using versions of Sentry that required
the older integration code.

Thus this commit adds back the older integration code and whenever the
Sentry webhook payload does not have a "data" field (which must be
present in all modern payloads as per the documentation at
https://docs.sentry.io/workflow/integrations/integration-platform/webhooks)
we will use the older Sentry integration code.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-05-07 11:26:19 -07:00
Johannes Hochreiter
c20fdb0d8b gitlab: Add pipeline and job URLs to pipeline event body. 2020-05-06 13:36:39 -07:00
pemontto
1688863be0 gitlab: Add support for unapproved merge request. 2020-05-06 13:34:47 -07:00
pemontto
740e01e456 gitlab: Add support for Gitlab system hooks.
Fixes #13960.
2020-05-06 13:34:47 -07:00
pemontto
46a6eb7cf8 gitlab: Add project name and url to custom topic messages. 2020-05-06 13:34:43 -07:00
Hemanth V. Alluri
04811e724d webhooks/sentry: Rewrite the sentry webhook for the latest SDKs.
Sentry has client SDKs for many programming languages and frameworks.
Sentry has deprecated their old "Raven" series of client SDKs in favor
of a new series of client SDKs following their unified API format.

As it stood, our Sentry integration was already outdated being written
for the version 5 payloads (the Raven SDKs stopped at version 6 which
is already vastly different from version 5) when the current and
prominently used version is version 7.

This commit completely rewrites the existing Sentry integration.

Tested and supported events:
- Issue created, resolved, assigned, and ignored events.
- "Sentry events" for "capture exception" and "capture message" with
the Golang, Node.js, and Python SDKs (other SDKs should also work but
only these were used for testing).

For reference:
- Old (Raven) SDK for python:
    https://github.com/getsentry/raven-python
- New (Unified API format) SDK for python:
    https://github.com/getsentry/sentry-python

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-05-02 13:39:57 -07:00
Anders Kaseorg
bdc365d0fe logging: Pass format arguments to logging.
https://docs.python.org/3/howto/logging.html#optimization

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-02 10:18:02 -07:00
pemontto
01a57ddd9d webhooks: Use Gitlab project web_url over repo homepage.
This is preferred because the `repository` field isn't available for
project-level events.
2020-04-24 17:50:47 -07:00
Anders Kaseorg
fead14951c python: Convert assignment type annotations to Python 3.6 style.
This commit was split by tabbott; this piece covers the vast majority
of files in Zulip, but excludes scripts/, tools/, and puppet/ to help
ensure we at least show the right error messages for Xenial systems.

We can likely further refine the remaining pieces with some testing.

Generated by com2ann, with whitespace fixes and various manual fixes
for runtime issues:

-    invoiced_through: Optional[LicenseLedger] = models.ForeignKey(
+    invoiced_through: Optional["LicenseLedger"] = models.ForeignKey(

-_apns_client: Optional[APNsClient] = None
+_apns_client: Optional["APNsClient"] = None

-    notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
-    signup_notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    signup_notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)

-    author: Optional[UserProfile] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)
+    author: Optional["UserProfile"] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)

-    bot_owner: Optional[UserProfile] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)
+    bot_owner: Optional["UserProfile"] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)

-    default_sending_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
-    default_events_register_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_sending_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_events_register_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)

-descriptors_by_handler_id: Dict[int, ClientDescriptor] = {}
+descriptors_by_handler_id: Dict[int, "ClientDescriptor"] = {}

-worker_classes: Dict[str, Type[QueueProcessingWorker]] = {}
-queues: Dict[str, Dict[str, Type[QueueProcessingWorker]]] = {}
+worker_classes: Dict[str, Type["QueueProcessingWorker"]] = {}
+queues: Dict[str, Dict[str, Type["QueueProcessingWorker"]]] = {}

-AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional[LDAPSearch] = None
+AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional["LDAPSearch"] = None

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 11:02:32 -07:00
Anders Kaseorg
f8c95cda51 mypy: Add specific codes to type: ignore annotations.
https://mypy.readthedocs.io/en/stable/error_codes.html

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 10:46:33 -07:00
Anders Kaseorg
1cf63eb5bf python: Whitespace fixes from autopep8.
Generated by autopep8, with the setup.cfg configuration from #14532.
I’m not sure why pycodestyle didn’t already flag these.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-21 17:58:09 -07:00
Puneeth Chaganti
6abb412538 webhooks/teamcity: Change teamcity build triggering user to iago.
To be able to get a screenshot of the personal message using the
`generate-integration-docs-screenshot` tool, this commit changes the
personal build to be triggered by iago, instead of cordelia.
2020-04-17 09:41:55 -07:00
Puneeth Chaganti
177a547587 webhooks/github: Delete a duplicate fixture.
The request_review_pull_request.json file is a duplicate of
pull_request__review_requested.json with data for a different pull-request.
2020-04-17 09:41:55 -07:00