paulmataruso/zulip
1
0
Fork 0
mirror of https://github.com/zulip/zulip.git synced 2025-10-23 16:14:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

api-docs: Move markdown files to top level directory.

Browse Source 
- Updates `.prettierignore` for the new directory.
- Updates any reference to the API documentation directory for
  markdown files to be `api_docs/` instead of `zerver/api/`.
- Removes a reference link from `docs/documentation/api.md` that
  hasn't referenced anything in the text since commit 0542c60.
- Update rendering of API documentation for new directory.
This commit is contained in:
Lauryn Menard
2023-01-31 12:11:45 +01:00
committed by Tim Abbott
parent fc54ffd778
commit dbacc00f0f
39 changed files with 69 additions and 71 deletions
Download Patch File Download Diff File Expand all files Collapse all files

31
api_docs/api-doc-template.md Normal file
View File

@@ -0,0 +1,31 @@
{generate_api_header(API_ENDPOINT_NAME)}
## Usage examples
{start_tabs}
{generate_code_example(python)|API_ENDPOINT_NAME|example}
{generate_code_example(javascript)|API_ENDPOINT_NAME|example}
{tab|curl}
{generate_code_example(curl)|API_ENDPOINT_NAME|example}
{end_tabs}
## Parameters
{generate_api_arguments_table|zulip.yaml|API_ENDPOINT_NAME}
{generate_parameter_description(API_ENDPOINT_NAME)}
## Response
{generate_return_values_table|zulip.yaml|API_ENDPOINT_NAME}
{generate_response_description(API_ENDPOINT_NAME)}
#### Example response(s)
{generate_code_example|API_ENDPOINT_NAME|fixture}

34
api_docs/api-keys.md Normal file
View File

@@ -0,0 +1,34 @@
# API keys
An **API key** is how a bot identifies itself to Zulip. Anyone with a
bot's API key can impersonate the bot, so be careful with it!
## Get a bot's API key
{settings_tab|your-bots}
1. Click **Active bots**.
1. Find your bot. The bot's API key is under **API KEY**.
## Get your API key
Anyone with your API key can impersonate you, so be doubly careful with it.
{settings_tab|account-and-privacy}
1. Under **API key**, click **Show/change your API key**.
1. Enter your password, and click **Get API key**. If you never had a
password, click **Never had one? Forgotten it?** and follow the
instructions from there.
1. Copy your API key.
## Invalidate an API key
To invalidate a key, follow the instructions above, and click
**Generate new API key** or click the **refresh**
(<i class="fa fa-refresh"></i>) icon as appropriate.
This will generate a new key for you or the bot, and invalidate the old one.

1326
api_docs/changelog.md Normal file
View File

File diff suppressed because it is too large Load Diff

60
api_docs/client-libraries.md Normal file
View File

@@ -0,0 +1,60 @@
# Client libraries
These API client libraries make it easy to work with Zulip's REST API
in your favorite language.
## Official libraries
These libraries are maintained by members of the Zulip core team. The
Python library is the most complete and best documented.
* [Python](https://github.com/zulip/python-zulip-api)
* [JavaScript](https://github.com/zulip/zulip-js)
## User maintained libraries
The Zulip core team doesn't have the resources to maintain
high-quality libraries for every programming language. We've
collected a list of user-maintained libraries for popular languages:
* [Clojure](https://github.com/thieman/clojure-zulip)
* [C#](https://github.com/zulip/zulip-csharp)
* [Go](https://github.com/ifo/gozulipbot)
* [Java](https://github.com/taliox/zulip-java-rest)
* [Kotlin](https://gitlab.com/ppiag/kzulip)
* [PHP](https://github.com/mrferos/zulip-php-client)
* [Ruby](https://github.com/raws/wonder-llama)
* [Swift](https://github.com/zulip/swift-zulip-api)
### Contributing
Contributing to improve language libraries is appreciated, as is
writing new ones. If you actively maintain a Zulip language binding
and would like it to be listed here (or would like to collaborate with
us in making it an official library), post in [this
topic][integrations-thread] in
[the Zulip development community](https://zulip.com/development-community/)
or submit a pull request [updating this
page](https://zulip.readthedocs.io/en/latest/documentation/api.html).
[integrations-thread]: https://chat.zulip.org/#narrow/stream/127-integrations/topic/API.20client.20libraries/
### Outdated
!!! tip ""
The following projects are not actively maintained. Since
Zulip's core APIs have been stable for 5 years, even very
old libraries can be useful.
* [Lua](https://github.com/deckycoss/zulua)
* [Erlang](https://github.com/femnad/tuplre)
* [PHP](https://github.com/federicoq/zulip-php)
* [Go](https://github.com/decached/go-zulip)
* [Haskell](https://github.com/yamadapc/hzulip)
* [Chicken Scheme](https://github.com/yamadapc/zulip-scheme)
* [Scala](https://github.com/cqfd/zulip-scala)
* [EventMachine](https://github.com/cqfd/zulip_machine)
* [Ruby](https://github.com/verg/zulip-rb)
* [Perl](https://github.com/Stantheman/WebService-Zulip)
* [.Net](https://github.com/Shayan-To/ZulipClientApi)

104
api_docs/configuring-python-bindings.md Normal file
View File

@@ -0,0 +1,104 @@
# Configuring the Python bindings
Zulip provides a set of tools that allows interacting with its API more
easily, called the [Python bindings](https://pypi.python.org/pypi/zulip/).
One of the most notable use cases for these bindings are bots developed
using Zulip's [bot framework](/api/writing-bots).
In order to use them, you need to configure them with your API key and other
settings. There are two ways to achieve that:
- With a file called `.zuliprc`, located in your home directory.
- With
[environment variables](https://en.wikipedia.org/wiki/Environment_variable)
set up in your host machine.
A `.zuliprc` file is a plain text document that looks like this:
```
[api]
key=<API key from the web interface>
email=<your email address>
site=<your Zulip server's URI>
...
```
The keys you can use in this file (and their equivalent environment variables)
can be found in the following table:
<table class="table">
<thead>
<tr>
<th><code>.zuliprc</code> key</th>
<th>Environment variable</th>
<th>Required</th>
<th>Description</th>
</tr>
</thead>
<tr>
<td><code>key</code></td>
<td><code>ZULIP_API_KEY</code></td>
<td>Yes</td>
<td>
<a href="/api/api-keys">API key</a>, which you can get through
Zulip's web interface.
</td>
</tr>
<tr>
<td><code>email</code></td>
<td><code>ZULIP_EMAIL</code></td>
<td>Yes</td>
<td>
The email address of the user who owns the API key mentioned
above.
</td>
</tr>
<tr>
<td><code>site</code></td>
<td><code>ZULIP_SITE</code></td>
<td>No</td>
<td>
URL where your Zulip server is located.
</td>
</tr>
<tr>
<td><code>client_cert_key</code></td>
<td><code>ZULIP_CERT_KEY</code></td>
<td>No</td>
<td>
Path to the SSL/TLS private key that the binding should use to
connect to the server.
</td>
</tr>
<tr>
<td><code>client_cert</code></td>
<td><code>ZULIP_CERT</code></td>
<td>No*</td>
<td>
The public counterpart of <code>client_cert_key</code>/
<code>ZULIP_CERT_KEY</code>. <i>This setting is required if a cert
key has been set.</i>
</td>
</tr>
<tr>
<td><code>client_bundle</code></td>
<td><code>ZULIP_CERT_BUNDLE</code></td>
<td>No</td>
<td>
Path where the server's PEM-encoded certificate is located. CA
certificates are also accepted, in case those CA's have issued the
server's certificate. Defaults to the built-in CA bundle trusted
by Python.
</td>
</tr>
<tr>
<td><code>insecure</code></td>
<td><code>ZULIP_ALLOW_INSECURE</code></td>
<td>No</td>
<td>
Allows connecting to Zulip servers with an invalid SSL/TLS
certificate. Please note that enabling this will make the HTTPS
connection insecure. Defaults to <code>false</code>.
</td>
</tr>
</table>

85
api_docs/construct-narrow.md Normal file
View File

@@ -0,0 +1,85 @@
# Construct a narrow
A **narrow** is a set of filters for Zulip messages, that can be based
on many different factors (like sender, stream, topic, search
keywords, etc.). Narrows are used in various places in the the Zulip
API (most importantly, in the API for fetching messages).
It is simplest to explain the algorithm for encoding a search as a
narrow using a single example. Consider the following search query
(written as it would be entered in the Zulip web app's search box).
It filters for messages sent to stream `announce`, not sent by
`iago@zulip.com`, and containing the words `cool` and `sunglasses`:
```
stream:announce -sender:iago@zulip.com cool sunglasses
```
This query would be JSON-encoded for use in the Zulip API using JSON
as a list of simple objects, as follows:
```json
[
{
"operator": "stream",
"operand": "announce"
},
{
"operator": "sender",
"operand": "iago@zulip.com",
"negated": true
},
{
"operator": "search",
"operand": "cool sunglasses"
}
]
```
The Zulip help center article on [searching for messages](/help/search-for-messages)
documents the majority of the search/narrow options supported by the
Zulip API.
Note that many narrows, including all that lack a `stream` or `streams`
operator, search the current user's personal message history. See
[searching shared history](/help/search-for-messages#searching-shared-history)
for details.
## Narrows that use IDs
The `near` and `id` operators, documented in the help center, use message
IDs for their operands.
* `near:12345`: Search messages around the message with ID `12345`.
* `id:12345`: Search for only message with ID `12345`.
There are a few additional narrow/search options (new in Zulip 2.1)
that use either stream IDs or user IDs that are not documented in the
help center because they are primarily useful to API clients:
* `stream:1234`: Search messages sent to the stream with ID `1234`.
* `sender:1234`: Search messages sent by user ID `1234`.
* `pm-with:1234`: Search the private message conversation between
you and user ID `1234`.
* `pm-with:1234,5678`: Search the private message conversation between
you, user ID `1234`, and user ID `5678`.
* `group-pm-with:1234`: Search all group private messages that
include you and user ID `1234`.
The operands for these search options must be encoded either as an
integer ID or a JSON list of integer IDs. For example, to query
messages sent by a user 1234 to a PM thread with yourself, user 1234,
and user 5678, the correct JSON-encoded query is:
```json
[
{
"operator": "pm-with",
"operand": [1234, 5678]
},
{
"operator": "sender",
"operand": 1234
}
]
```

6
api_docs/create-stream.md Normal file
View File

@@ -0,0 +1,6 @@
# Create a stream
You can create a stream using Zulip's REST API by submitting a
[subscribe](/api/subscribe) request with a stream name that
doesn't yet exist and passing appropriate parameters to define
the initial configuration of the new stream.

245
api_docs/deploying-bots.md Normal file
View File

@@ -0,0 +1,245 @@
# Deploying bots in production
Usually, work on a bot starts on a laptop. At some point, you'll want
to deploy your bot in a production environment, so that it'll stay up
regardless of what's happening with your laptop. There are several
options for doing so:
* The simplest is running `zulip-run-bot` inside a `screen` session on
a server. This works, but if your server reboots, you'll need to
manually restart it, so we don't recommend it.
* Using `supervisord` or a similar tool for managing a production
process with `zulip-run-bot`. This consumes a bit of resources
(since you need a persistent process running), but otherwise works
great.
* Using the Zulip Botserver, which is a simple Flask server for
running a bot in production, and connecting that to Zulip's outgoing
webhooks feature. This can be deployed in environments like
Heroku's free tier without running a persistent process.
## Zulip Botserver
The Zulip Botserver is for people who want to
* run bots in production.
* run multiple bots at once.
The Zulip Botserver is a Python (Flask) server that implements Zulip's
outgoing webhooks API. You can of course write your own servers using
the outgoing webhooks API, but the Botserver is designed to make it
easy for a novice Python programmer to write a new bot and deploy it
in production.
### How Botserver works
Zulip Botserver starts a web server that listens to incoming messages
from your main Zulip server. The sequence of events in a successful
Botserver interaction are:
1. Your bot user is mentioned or receives a private message:
```
@**My Bot User** hello world
```
1. The Zulip server sends a POST request to the Botserver on `https://bot-server.example.com/`:
```json
{
"message":{
"content":"@**My Bot User** hello world",
},
"bot_email":"myuserbot-bot@example.com",
"trigger":"mention",
"token":"XXXX"
}
```
This URL is configured in the Zulip web-app in your Bot User's settings.
1. The Botserver searches for a bot to handle the message.
1. The Botserver executes your bot's `handle_message` code.
Your bot's code should work just like it does with `zulip-run-bot`;
for example, you reply using
[bot_handler.send_reply](writing-bots#bot_handlersend_reply)).
### Installing the Zulip Botserver
Install the `zulip_botserver` package:
```
pip3 install zulip_botserver
```
### Running a bot using the Zulip Botserver
1. Construct the URL for your bot, which will be of the form:
```
http://<hostname>:<port>
```
where the `hostname` is the hostname you'll be running the bot
server on, and `port` is the port for it (the recommended default
is `5002`).
1. Register new bot users on the Zulip server's web interface.
* Log in to the Zulip server.
* Navigate to *Personal settings (<i class="fa fa-cog"></i>)* -> *Bots* -> *Add a new bot*.
Select *Outgoing webhook* for bot type, fill out the form (using
the URL from above) and click on *Create bot*.
* A new bot user should appear in the *Active bots* panel.
1. Download the `zuliprc` file for your bot from the *Active Bots*
panel, using the download button.
1. Run the Botserver, where `helloworld` is the name of the bot you
want to run:
`zulip-botserver --config-file <path_to_zuliprc> --bot-name=helloworld`
You can specify the port number and various other options; run
`zulip-botserver --help` to see how to do this.
1. Congrats, everything is set up! Test your Botserver like you would
test a normal bot.
### Running multiple bots using the Zulip Botserver
The Zulip Botserver also supports running multiple bots from a single
Botserver process. You can do this with the following procedure.
1. Download the `botserverrc` from the `your-bots` settings page, using
the "Download config of all active outgoing webhook bots in Zulip
Botserver format." option at the top.
1. Open the `botserverrc`. It should contain one or more sections that look like this:
```
[]
email=foo-bot@hostname
key=dOHHlyqgpt5g0tVuVl6NHxDLlc9eFRX4
site=http://hostname
token=aQVQmSd6j6IHphJ9m1jhgHdbnhl5ZcsY
```
Each section contains the configuration for an outgoing webhook bot. For each
bot, enter the name of the bot you want to run in the square brackets `[]`.
For example, if we want `foo-bot@hostname` to run the `helloworld` bot, our
new section would look like this:
```
[helloworld]
email=foo-bot@hostname
key=dOHHlyqgpt5g0tVuVl6NHxDLlc9eFRX4
site=http://hostname
token=aQVQmSd6j6IHphJ9m1jhgHdbnhl5ZcsY
```
To run an external bot, enter the path to the bot's python file in the square
brackets `[]`. For example, if we want to run `~/Documents/my_new_bot.py`, our
new section could look like this:
```
[~/Documents/my_new_bot.py]
email=foo-bot@hostname
key=dOHHlyqgpt5g0tVuVl6NHxDLlc9eFRX4
site=http://hostname
```
1. Run the Zulip Botserver by passing the `botserverrc` to it. The
command format is:
```
zulip-botserver --config-file <path_to_botserverrc>
```
If omitted, `hostname` defaults to `127.0.0.1` and `port` to `5002`.
### Running Zulip Botserver with supervisord
[supervisord](http://supervisord.org/) is a popular tool for running
services in production. It helps ensure the service starts on boot,
manages log files, restarts the service if it crashes, etc. This
section documents how to run the Zulip Botserver using *supervisord*.
Running the Zulip Botserver with *supervisord* works almost like
running it manually.
1. Install *supervisord* via your package manager; e.g. on Debian/Ubuntu:
```
sudo apt-get install supervisor
```
1. Configure *supervisord*. *supervisord* stores its configuration in
`/etc/supervisor/conf.d`.
* Do **one** of the following:
* Download the [sample config file][supervisord-config-file]
and store it in `/etc/supervisor/conf.d/zulip-botserver.conf`.
* Copy the following section into your existing supervisord config file.
[program:zulip-botserver]
command=zulip-botserver --config-file=<path/to/your/botserverrc>
--hostname <address> --port <port>
startsecs=3
stdout_logfile=/var/log/zulip-botserver.log ; all output of your Botserver will be logged here
redirect_stderr=true
* Edit the `<>` sections according to your preferences.
[supervisord-config-file]: https://raw.githubusercontent.com/zulip/python-zulip-api/main/zulip_botserver/zulip-botserver-supervisord.conf
1. Update *supervisord* to read the configuration file:
```
supervisorctl reread
supervisorctl update
```
(or you can use `/etc/init.d/supervisord restart`, but this is less
disruptive if you're using *supervisord* for other services as well).
1. Test if your setup is successful:
```
supervisorctl status
```
The output should include a line similar to this:
> zulip-botserver RUNNING pid 28154, uptime 0:00:27
The standard output of the Botserver will be logged to the path in
your *supervisord* configuration.
If you are hosting the Botserver yourself (as opposed to using a
hosting service that provides SSL), we recommend securing your
Botserver with SSL using an `nginx` or `Apache` reverse proxy and
[Certbot](https://certbot.eff.org/).
### Troubleshooting
1. Make sure the API key you're using is for an [outgoing webhook
bot](/api/outgoing-webhooks) and you've
correctly configured the URL for your Botserver.
1. Your Botserver needs to be accessible from your Zulip server over
HTTP(S). Make sure any firewall allows the connection. We
recommend using [zulip-run-bot](running-bots) instead for
development/testing on a laptop or other non-server system.
If your Zulip server is self-hosted, you can test by running `curl
http://zulipbotserver.example.com:5002` from your Zulip server;
the output should be:
```
$ curl http://zulipbotserver.example.com:5002/
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>405 Method Not Allowed</title>
<h1>Method Not Allowed</h1>
<p>The method is not allowed for the requested URL.</p>
```

3
api_docs/include/api-admin-only.md Normal file
View File

@@ -0,0 +1,3 @@
!!! warn ""
This endpoint is only available to organization administrators.

0
api_docs/include/empty.md Normal file
View File

106
api_docs/include/rest-endpoints.md Normal file
View File

@@ -0,0 +1,106 @@
#### Messages
* [Send a message](/api/send-message)
* [Upload a file](/api/upload-file)
* [Edit a message](/api/update-message)
* [Delete a message](/api/delete-message)
* [Get messages](/api/get-messages)
* [Construct a narrow](/api/construct-narrow)
* [Add an emoji reaction](/api/add-reaction)
* [Remove an emoji reaction](/api/remove-reaction)
* [Render a message](/api/render-message)
* [Fetch a single message](/api/get-message)
* [Check if messages match narrow](/api/check-messages-match-narrow)
* [Get a message's edit history](/api/get-message-history)
* [Update personal message flags](/api/update-message-flags)
* [Update personal message flags for narrow](/api/update-message-flags-for-narrow)
* [Mark messages as read in bulk](/api/mark-all-as-read)
* [Get a message's read receipts](/api/get-read-receipts)
#### Drafts
* [Get drafts](/api/get-drafts)
* [Create drafts](/api/create-drafts)
* [Edit a draft](/api/edit-draft)
* [Delete a draft](/api/delete-draft)
#### Streams
* [Get subscribed streams](/api/get-subscriptions)
* [Subscribe to a stream](/api/subscribe)
* [Unsubscribe from a stream](/api/unsubscribe)
* [Get subscription status](/api/get-subscription-status)
* [Get all subscribers](/api/get-subscribers)
* [Update subscription settings](/api/update-subscription-settings)
* [Get all streams](/api/get-streams)
* [Get a stream by ID](/api/get-stream-by-id)
* [Get stream ID](/api/get-stream-id)
* [Create a stream](/api/create-stream)
* [Update a stream](/api/update-stream)
* [Archive a stream](/api/archive-stream)
* [Get topics in a stream](/api/get-stream-topics)
* [Topic muting](/api/mute-topic)
* [Delete a topic](/api/delete-topic)
* [Add a default stream](/api/add-default-stream)
* [Remove a default stream](/api/remove-default-stream)
#### Users
* [Get all users](/api/get-users)
* [Get own user](/api/get-own-user)
* [Get a user](/api/get-user)
* [Get a user by email](/api/get-user-by-email)
* [Update a user](/api/update-user)
* [Update your status](/api/update-status)
* [Create a user](/api/create-user)
* [Deactivate a user](/api/deactivate-user)
* [Reactivate a user](/api/reactivate-user)
* [Deactivate own user](/api/deactivate-own-user)
* [Set "typing" status](/api/set-typing-status)
* [Get user presence](/api/get-user-presence)
* [Get presence of all users](/api/get-presence)
* [Get attachments](/api/get-attachments)
* [Delete an attachment](/api/remove-attachment)
* [Update settings](/api/update-settings)
* [Get user groups](/api/get-user-groups)
* [Create a user group](/api/create-user-group)
* [Update a user group](/api/update-user-group)
* [Delete a user group](/api/remove-user-group)
* [Update user group members](/api/update-user-group-members)
* [Update user group subgroups](/api/update-user-group-subgroups)
* [Get user group membership status](/api/get-is-user-group-member)
* [Get user group members](/api/get-user-group-members)
* [Get subgroups of user group](/api/get-user-group-subgroups)
* [Mute a user](/api/mute-user)
* [Unmute a user](/api/unmute-user)
* [Get all alert words](/api/get-alert-words)
* [Add alert words](/api/add-alert-words)
* [Remove alert words](/api/remove-alert-words)
#### Server & organizations
* [Get server settings](/api/get-server-settings)
* [Get linkifiers](/api/get-linkifiers)
* [Add a linkifier](/api/add-linkifier)
* [Update a linkifier](/api/update-linkifier)
* [Remove a linkifier](/api/remove-linkifier)
* [Add a code playground](/api/add-code-playground)
* [Remove a code playground](/api/remove-code-playground)
* [Get all custom emoji](/api/get-custom-emoji)
* [Upload custom emoji](/api/upload-custom-emoji)
* [Get all custom profile fields](/api/get-custom-profile-fields)
* [Reorder custom profile fields](/api/reorder-custom-profile-fields)
* [Create a custom profile field](/api/create-custom-profile-field)
* [Update realm-level defaults of user settings](/api/update-realm-user-settings-defaults)
#### Real-time events
* [Real time events API](/api/real-time-events)
* [Register an event queue](/api/register-queue)
* [Get events from an event queue](/api/get-events)
* [Delete an event queue](/api/delete-queue)
#### Specialty endpoints
* [Fetch an API key (production)](/api/fetch-api-key)
* [Fetch an API key (development only)](/api/dev-fetch-api-key)

157
api_docs/incoming-webhooks-overview.md Normal file
View File

@@ -0,0 +1,157 @@
# Incoming webhook integrations
An incoming webhook allows a third-party service to push data to Zulip when
something happens. There's several ways to do an incoming webhook in
Zulip:
* Use our [REST API](/api/rest) endpoint for [sending
messages](/api/send-message). This works great for internal tools
or cases where the third-party tool wants to control the formatting
of the messages in Zulip.
* Use one of our supported [integration
frameworks](/integrations/meta-integration), such as the
[Slack-compatible incoming webhook](/integrations/doc/slack_incoming),
[Zapier integration](/integrations/docs/zapier), or
[IFTTT integration](/integrations/doc/ifttt).
* Adding an incoming webhook integration (detailed on this page),
where all the logic for formatting the Zulip messages lives in the
Zulip server. This is how most of [Zulip's official
integrations](/integrations) work, because they enable Zulip to
support third-party services that just have an "outgoing webhook"
feature (without the third party needing to do any work specific to
Zulip).
In an incoming webhook integration, the third-party service's
"outgoing webhook" feature sends an `HTTP POST`s to a special URL when
it has something for you, and then the Zulip "incoming webhook"
integration handles that incoming data to format and send a message in
Zulip.
New official Zulip webhook integrations can take just a few hours to
write, including tests and documentation, if you use the right
process.
## Quick guide
* Set up the
[Zulip development environment](https://zulip.readthedocs.io/en/latest/development/overview.html).
* Use [Zulip's JSON integration](/integrations/doc/json),
<https://webhook.site/>, or a similar site to capture an example
webhook payload from the third-party service. Create a
`zerver/webhooks/<mywebhook>/fixtures/` directory, and add the
captured payload as a test fixture.
* Create an `Integration` object, and add it to `WEBHOOK_INTEGRATIONS` in
`zerver/lib/integrations.py`. Search for `webhook` in that file to find an
existing one to copy.
* Write a draft webhook handler under `zerver/webhooks/`. There are a lot of
examples in that directory that you can copy. We recommend templating off
a short one, like `zendesk`.
* Add a test for your fixture at `zerver/webhooks/<mywebhook>/tests.py`.
Run the tests for your integration like this:
```
tools/test-backend zerver/webhooks/<mywebhook>/
```
Iterate on debugging the test and webhooks handler until it all
works.
* Capture payloads for the other common types of `POST`s the third-party
service will make, and add tests for them; usually this part of the
process is pretty fast.
* Document the integration (required for getting it merged into Zulip). You
can template off an existing guide, like
[this one](https://raw.githubusercontent.com/zulip/zulip/main/zerver/webhooks/github/doc.md).
This should not take more than 15 minutes, even if you don't speak English
as a first language (we'll clean up the text before merging).
## Hello world walkthrough
Check out the [detailed walkthrough](incoming-webhooks-walkthrough) for step-by-step
instructions.
## Checklist
### Files that need to be created
Select a name for your incoming webhook and use it consistently. The examples
below are for a webhook named `MyWebHook`.
* `zerver/webhooks/mywebhook/__init__.py`: Empty file that is an obligatory
part of every python package. Remember to `git add` it.
* `zerver/webhooks/mywebhook/view.py`: The main webhook integration function
as well as any needed helper functions.
* `zerver/webhooks/mywebhook/fixtures/messagetype.json`: Sample json payload data
used by tests. Add one fixture file per type of message supported by your
integration.
* `zerver/webhooks/mywebhook/tests.py`: Tests for your webhook.
* `zerver/webhooks/mywebhook/doc.md`: End-user documentation explaining
how to add the integration.
* `static/images/integrations/logos/mywebhook.svg`: A square logo for the
platform/server/product you are integrating. Used on the documentation
pages as well as the sender's avatar for messages sent by the integration.
* `static/images/integrations/mywebhook/001.svg`: A screenshot of a message
sent by the integration, used on the documentation page. This can be
generated by running `tools/generate-integration-docs-screenshot --integration mywebhook`.
* `static/images/integrations/bot_avatars/mywebhook.png`: A square logo for the
platform/server/product you are integrating which is used to create the avatar
for generating screenshots with. This can be generated automatically from
`static/images/integrations/logos/mywebhook.svg` by running
`tools/setup/generate_integration_bots_avatars.py`.
### Files that need to be updated
* `zerver/lib/integrations.py`: Add your integration to
`WEBHOOK_INTEGRATIONS`. This will automatically register a
URL for the incoming webhook of the form `api/v1/external/mywebhook` and
associate it with the function called `api_mywebhook_webhook` in
`zerver/webhooks/mywebhook/view.py`. Also add your integration to
`DOC_SCREENSHOT_CONFIG`. This will allow you to automatically generate
a screenshot for the documentation by running
`tools/generate-integration-docs-screenshot --integration mywebhook`.
## Common Helpers
* If your integration will receive a test webhook payload, you can use
`get_setup_webhook_message` to create our standard message for test payloads.
You can import this from `zerver/lib/webhooks/common.py`, and it will generate
a message like this: "GitHub webhook is successfully configured! 🎉"
## General advice
* Consider using our Zulip markup to make the output from your
integration especially attractive or useful (e.g. emoji, Markdown
emphasis or @-mentions).
* Use topics effectively to ensure sequential messages about the same
thing are threaded together; this makes for much better consumption
by users. E.g. for a bug tracker integration, put the bug number in
the topic for all messages; for an integration like Nagios, put the
service in the topic.
* Integrations that don't match a team's workflow can often be
uselessly spammy. Give careful thought to providing options for
triggering Zulip messages only for certain message types, certain
projects, or sending different messages to different streams/topics,
to make it easy for teams to configure the integration to support
their workflow.
* Consistently capitalize the name of the integration in the
documentation and the Client name the way the vendor does. It's OK
to use all-lower-case in the implementation.
* Sometimes it can be helpful to contact the vendor if it appears they
don't have an API or webhook we can use; sometimes the right API
is just not properly documented.
* A helpful tool for testing your integration is
[UltraHook](http://www.ultrahook.com/), which allows you to receive webhook
calls via your local Zulip development environment. This enables you to do end-to-end
testing with live data from the service you're integrating and can help you
spot why something isn't working or if the service is using custom HTTP
headers.

644
api_docs/incoming-webhooks-walkthrough.md Normal file
View File

@@ -0,0 +1,644 @@
# Incoming webhook walkthrough
Below, we explain each part of a simple incoming webhook integration,
called **Hello World**. This integration sends a "hello" message to the `test`
stream and includes a link to the Wikipedia article of the day, which
it formats from json data it receives in the http request.
Use this walkthrough to learn how to write your first webhook
integration.
## Step 0: Create fixtures
The first step in creating an incoming webhook is to examine the data that the
service you want to integrate will be sending to Zulip.
* Use [Zulip's JSON integration](/integrations/doc/json),
<https://webhook.site/>, or a similar tool to capture webhook
payload(s) from the service you are integrating. Examining this data
allows you to do two things:
1. Determine how you will need to structure your webhook code, including what
message types your integration should support and how.
2. Create fixtures for your webhook tests.
A test fixture is a small file containing test data, one for each test.
Fixtures enable the testing of webhook integration code without the need to
actually contact the service being integrated.
Because `Hello World` is a very simple integration that does one
thing, it requires only one fixture,
`zerver/webhooks/helloworld/fixtures/hello.json`:
```json
{
"featured_title":"Marilyn Monroe",
"featured_url":"https://en.wikipedia.org/wiki/Marilyn_Monroe",
}
```
When writing your own incoming webhook integration, you'll want to write a test function
for each distinct message condition your integration supports. You'll also need a
corresponding fixture for each of these tests. Depending on the type of data
the 3rd party service sends, your fixture may contain JSON, URL encoded text, or
some other kind of data. See [Step 5: Create automated tests](#step-5-create-automated-tests) or
[Testing](https://zulip.readthedocs.io/en/latest/testing/testing.html) for further details.
### HTTP Headers
Some third-party webhook APIs, such as GitHub's, don't encode all the
information about an event in the JSON request body. Instead, they
put key details like the event type in a separate HTTP header
(generally this is clear in their API documentation). In order to
test Zulip's handling of that integration, you will need to record
which HTTP headers are used with each fixture you capture.
Since this is integration-dependent, Zulip offers a simple API for
doing this, which is probably best explained by looking at the example
for GitHub: `zerver/webhooks/github/view.py`; basically, as part of
writing your integration, you'll write a special function in your
view.py file that maps the filename of the fixture to the set of HTTP
headers to use. This function must be named "fixture_to_headers". Most
integrations will use the same strategy as the GitHub integration:
encoding the third party variable header data (usually just an event
type) in the fixture filename, in such a case, you won't need to
explicitly write the logic for such a special function again,
instead you can just use the same helper method that the GitHub
integration uses.
## Step 1: Initialize your webhook python package
In the `zerver/webhooks/` directory, create new subdirectory that will
contain all of corresponding code. In our example it will be
`helloworld`. The new directory will be a python package, so you have
to create an empty `__init__.py` file in that directory via e.g.
`touch zerver/webhooks/helloworld/__init__.py`.
## Step 2: Create main webhook code
The majority of the code for your new integration will be in a single
python file, `zerver/webhooks/mywebhook/view.py`.
The Hello World integration is in `zerver/webhooks/helloworld/view.py`:
```python
from typing import Any, Dict, Sequence
from django.http import HttpRequest, HttpResponse
from zerver.decorator import webhook_view
from zerver.lib.request import REQ, has_request_variables
from zerver.lib.response import json_success
from zerver.lib.webhooks.common import check_send_webhook_message
from zerver.models import UserProfile
@webhook_view("HelloWorld")
@has_request_variables
def api_helloworld_webhook(
request: HttpRequest,
user_profile: UserProfile,
payload: Dict[str, Sequence[Dict[str, Any]]] = REQ(argument_type="body"),
) -> HttpResponse:
# construct the body of the message
body = "Hello! I am happy to be here! :smile:"
# try to add the Wikipedia article of the day
body_template = (
"\nThe Wikipedia featured article for today is **[{featured_title}]({featured_url})**"
)
body += body_template.format(**payload)
topic = "Hello World"
# send the message
check_send_webhook_message(request, user_profile, topic, body)
return json_success(request)
```
The above code imports the required functions and defines the main webhook
function `api_helloworld_webhook`, decorating it with `webhook_view` and
`has_request_variables`. The `has_request_variables` decorator allows you to
access request variables with `REQ()`. You can find more about `REQ` and request
variables in [Writing views](
https://zulip.readthedocs.io/en/latest/tutorials/writing-views.html#request-variables).
You must pass the name of your integration to the
`webhook_view` decorator; that name will be used to
describe your integration in Zulip's analytics (e.g. the `/stats`
page). Here we have used `HelloWorld`. To be consistent with other
integrations, use the name of the product you are integrating in camel
case, spelled as the product spells its own name (except always first
letter upper-case).
The `webhook_view` decorator indicates that the 3rd party service will
send the authorization as an API key in the query parameters. If your service uses
HTTP basic authentication, you would instead use the `authenticated_rest_api_view`
decorator.
You should name your webhook function as such
`api_webhookname_webhook` where `webhookname` is the name of your
integration and is always lower-case.
At minimum, the webhook function must accept `request` (Django
[HttpRequest](https://docs.djangoproject.com/en/3.2/ref/request-response/#django.http.HttpRequest)
object), and `user_profile` (Zulip's user object). You may also want to
define additional parameters using the `REQ` object.
In the example above, we have defined `payload` which is populated
from the body of the http request, `stream` with a default of `test`
(available by default in the Zulip development environment), and
`topic` with a default of `Hello World`. If your webhook uses a custom stream,
it must exist before a message can be created in it. (See
[Step 4: Create automated tests](#step-5-create-automated-tests) for how to handle this in tests.)
The line that begins `# type` is a mypy type annotation. See [this
page](https://zulip.readthedocs.io/en/latest/testing/mypy.html) for details about
how to properly annotate your webhook functions.
In the body of the function we define the body of the message as `Hello! I am
happy to be here! :smile:`. The `:smile:` indicates an emoji. Then we append a
link to the Wikipedia article of the day as provided by the json payload.
* Sometimes, it might occur that a json payload does not contain all required keys your
integration checks for. In such a case, any `KeyError` thrown is handled by the server
backend and will create an appropriate response.
Then we send a message with `check_send_webhook_message`, which will
validate the message and do the following:
* Send a public (stream) message if the `stream` query parameter is
specified in the webhook URL.
* If the `stream` query parameter isn't specified, it will send a private
message to the owner of the webhook bot.
Finally, we return a 200 http status with a JSON format success message via
`json_success(request)`.
## Step 3: Create an API endpoint for the webhook
In order for an incoming webhook to be externally available, it must be mapped
to a URL. This is done in `zerver/lib/integrations.py`.
Look for the lines beginning with:
```python
WEBHOOK_INTEGRATIONS = [
```
And you'll find the entry for Hello World:
```python
WebhookIntegration('helloworld', ['misc'], display_name='Hello World'),
```
This tells the Zulip API to call the `api_helloworld_webhook` function in
`zerver/webhooks/helloworld/view.py` when it receives a request at
`/api/v1/external/helloworld`.
This line also tells Zulip to generate an entry for Hello World on the Zulip
integrations page using `static/images/integrations/logos/helloworld.png` as its
icon. The second positional argument defines a list of categories for the
integration.
At this point, if you're following along and/or writing your own Hello World
webhook, you have written enough code to test your integration. There are three
tools which you can use to test your webhook - 2 command line tools and a GUI.
### Webhooks requiring custom configuration
In rare cases, it's necessary for an incoming webhook to require
additional user configuration beyond what is specified in the post
URL. The typical use case for this is APIs like the Stripe API that
require clients to do a callback to get details beyond an opaque
object ID that one would want to include in a Zulip notification.
These configuration options are declared as follows:
```python
WebhookIntegration('helloworld', ['misc'], display_name='Hello World',
config_options=[('HelloWorld API key', 'hw_api_key', check_string)])
```
`config_options` is a list describing the parameters the user should
configure:
1. A user-facing string describing the field to display to users.
2. The field name you'll use to access this from your `view.py` function.
3. A Validator, used to verify the input is valid.
Common validators are available in `zerver/lib/validators.py`.
## Step 4: Manually testing the webhook
For either one of the command line tools, first, you'll need to get an
API key from the **Bots** section of your Zulip user's **Personal
settings**. To test the webhook, you'll need to [create a
bot](https://zulip.com/help/add-a-bot-or-integration) with the
**Incoming webhook** type. Replace `<api_key>` with your bot's API key
in the examples presented below! This is how Zulip knows that the
request was made by an authorized user.
### Curl
Using curl:
```bash
curl -X POST -H "Content-Type: application/json" -d '{ "featured_title":"Marilyn Monroe", "featured_url":"https://en.wikipedia.org/wiki/Marilyn_Monroe" }' http://localhost:9991/api/v1/external/helloworld\?api_key\=<api_key>
```
After running the above command, you should see something similar to:
```json
{"msg":"","result":"success"}
```
### Management command: send_webhook_fixture_message
Using `manage.py` from within the Zulip development environment:
```console
(zulip-py3-venv) vagrant@vagrant:/srv/zulip$
./manage.py send_webhook_fixture_message \
--fixture=zerver/webhooks/helloworld/fixtures/hello.json \
'--url=http://localhost:9991/api/v1/external/helloworld?api_key=<api_key>'
```
After running the above command, you should see something similar to:
```
2016-07-07 15:06:59,187 INFO 127.0.0.1 POST 200 143ms (mem: 6ms/13) (md: 43ms/1) (db: 20ms/9q) (+start: 147ms) /api/v1/external/helloworld (helloworld-bot@zulip.com via ZulipHelloWorldWebhook)
```
Some webhooks require custom HTTP headers, which can be passed using
`./manage.py send_webhook_fixture_message --custom-headers`. For
example:
--custom-headers='{"X-Custom-Header": "value"}'
The format is a JSON dictionary, so make sure that the header names do
not contain any spaces in them and that you use the precise quoting
approach shown above.
For more information about `manage.py` command-line tools in Zulip, see
the [management commands][management-commands] documentation.
[management-commands]: https://zulip.readthedocs.io/en/latest/production/management-commands.html
### Integrations Dev Panel
This is the GUI tool.
1. Run `./tools/run-dev.py` then go to http://localhost:9991/devtools/integrations/.
2. Set the following mandatory fields:
**Bot** - Any incoming webhook bot.
**Integration** - One of the integrations.
**Fixture** - Though not mandatory, it's recommended that you select one and then tweak it if necessary.
The remaining fields are optional, and the URL will automatically be generated.
3. Click **Send**!
By opening Zulip in one tab and then this tool in another, you can quickly tweak
your code and send sample messages for many different test fixtures.
Note: Custom HTTP Headers must be entered as a JSON dictionary, if you want to use any in the first place that is.
Feel free to use 4-spaces as tabs for indentation if you'd like!
Your sample notification may look like:
<img class="screenshot" src="/static/images/api/helloworld-webhook.png" alt="screenshot" />
## Step 5: Create automated tests
Every webhook integration should have a corresponding test file:
`zerver/webhooks/mywebhook/tests.py`.
The Hello World integration's tests are in `zerver/webhooks/helloworld/tests.py`
You should name the class `<WebhookName>HookTests` and have it inherit from
the base class `WebhookTestCase`. For our HelloWorld webhook, we name the test
class `HelloWorldHookTests`:
```python
class HelloWorldHookTests(WebhookTestCase):
STREAM_NAME = 'test'
URL_TEMPLATE = "/api/v1/external/helloworld?&api_key={api_key}"
WEBHOOK_DIR_NAME = 'helloworld'
# Note: Include a test function per each distinct message condition your integration supports
def test_hello_message(self) -> None:
expected_topic = "Hello World";
expected_message = "Hello! I am happy to be here! :smile: \nThe Wikipedia featured article for today is **[Marilyn Monroe](https://en.wikipedia.org/wiki/Marilyn_Monroe)**";
# use fixture named helloworld_hello
self.check_webhook('hello', expected_topic, expected_message,
content_type="application/x-www-form-urlencoded")
```
In the above example, `STREAM_NAME`, `URL_TEMPLATE`, and `WEBHOOK_DIR_NAME` refer
to class attributes from the base class, `WebhookTestCase`. These are needed by
the helper function `check_webhook` to determine how to execute
your test. `STREAM_NAME` should be set to your default stream. If it doesn't exist,
`check_webhook` will create it while executing your test.
If your test expects a stream name from a test fixture, the value in the fixture
and the value you set for `STREAM_NAME` must match. The test helpers use `STREAM_NAME`
to create the destination stream, and then create the message to send using the
value from the fixture. If these don't match, the test will fail.
`URL_TEMPLATE` defines how the test runner will call your incoming webhook, in the same way
you would provide a webhook URL to the 3rd party service. `api_key={api_key}` says
that an API key is expected.
When writing tests for your webhook, you'll want to include one test function
(and corresponding fixture) per each distinct message condition that your
integration supports.
If, for example, we added support for sending a goodbye message to our `Hello
World` webhook, we would add another test function to `HelloWorldHookTests`
class called something like `test_goodbye_message`:
```python
def test_goodbye_message(self) -> None:
expected_topic = "Hello World";
expected_message = "Hello! I am happy to be here! :smile:\nThe Wikipedia featured article for today is **[Goodbye](https://en.wikipedia.org/wiki/Goodbye)**";
# use fixture named helloworld_goodbye
self.check_webhook('goodbye', expected_topic, expected_message,
content_type="application/x-www-form-urlencoded")
```
As well as a new fixture `goodbye.json` in
`zerver/webhooks/helloworld/fixtures/`:
```json
{
"featured_title":"Goodbye",
"featured_url":"https://en.wikipedia.org/wiki/Goodbye",
}
```
Also consider if your integration should have negative tests, a test where the
data from the test fixture should result in an error. For details see
[Negative tests](#negative-tests), below.
Once you have written some tests, you can run just these new tests from within
the Zulip development environment with this command:
```console
(zulip-py3-venv) vagrant@vagrant:/srv/zulip$
./tools/test-backend zerver/webhooks/helloworld
```
(Note: You must run the tests from the top level of your development directory.
The standard location in a Vagrant environment is `/srv/zulip`. If you are not
using Vagrant, use the directory where you have your development environment.)
You will see some script output and if all the tests have passed, you will see:
```console
Running zerver.webhooks.helloworld.tests.HelloWorldHookTests.test_goodbye_message
Running zerver.webhooks.helloworld.tests.HelloWorldHookTests.test_hello_message
DONE!
```
## Step 6: Create documentation
Next, we add end-user documentation for our integration. You
can see the existing examples at <https://zulip.com/integrations>
or by accessing `/integrations` in your Zulip development environment.
There are two parts to the end-user documentation on this page.
The first is the lozenge in the grid of integrations, showing your
integration logo and name, which links to the full documentation.
This is generated automatically once you've registered the integration
in `WEBHOOK_INTEGRATIONS` in `zerver/lib/integrations.py`, and supports
some customization via options to the `WebhookIntegration` class.
Second, you need to write the actual documentation content in
`zerver/webhooks/mywebhook/doc.md`.
```md
Learn how Zulip integrations work with this simple Hello World example!
1. The Hello World webhook will use the `test` stream, which is created
by default in the Zulip development environment. If you are running
Zulip in production, you should make sure that this stream exists.
1. {!create-bot-construct-url.md!}
1. To trigger a notification using this example webhook, you can use
`send_webhook_fixture_message` from a [Zulip development
environment](https://zulip.readthedocs.io/en/latest/development/overview.html):
```
(zulip-py3-venv) vagrant@vagrant:/srv/zulip$
./manage.py send_webhook_fixture_message \
> --fixture=zerver/tests/fixtures/helloworld/hello.json \
> '--url=http://localhost:9991/api/v1/external/helloworld?api_key=abcdefgh&stream=stream%20name;'
```
Or, use curl:
```
curl -X POST -H "Content-Type: application/json" -d '{ "featured_title":"Marilyn Monroe", "featured_url":"https://en.wikipedia.org/wiki/Marilyn_Monroe" }' http://localhost:9991/api/v1/external/helloworld\?api_key=abcdefgh&stream=stream%20name;
```
{!congrats.md!}
![Hello World integration](/static/images/integrations/helloworld/001.png)
```
`{!create-bot-construct-url.md!}` and `{!congrats.md!}` are examples of
a Markdown macro. Zulip has a macro-based Markdown/Jinja2 framework that
includes macros for common instructions in Zulip's webhooks/integrations
documentation.
See
[our guide on documenting an integration][integration-docs-guide]
for further details, including how to easily create the message
screenshot. Mostly you should plan on templating off an existing guide, like
[this one](https://raw.githubusercontent.com/zulip/zulip/main/zerver/webhooks/github/doc.md).
[integration-docs-guide]: https://zulip.readthedocs.io/en/latest/documentation/integrations.html
## Step 7: Preparing a pull request to zulip/zulip
When you have finished your webhook integration and are ready for it to be
available in the Zulip product, follow these steps to prepare your pull
request:
1. Run tests including linters and ensure you have addressed any issues they
report. See [Testing](https://zulip.readthedocs.io/en/latest/testing/testing.html)
and [Linters](https://zulip.readthedocs.io/en/latest/testing/linters.html) for details.
2. Read through [Code styles and conventions](
https://zulip.readthedocs.io/en/latest/contributing/code-style.html) and take a look
through your code to double-check that you've followed Zulip's guidelines.
3. Take a look at your Git history to ensure your commits have been clear and
logical (see [Commit discipline](
https://zulip.readthedocs.io/en/latest/contributing/commit-discipline.html) for tips). If not,
consider revising them with `git rebase --interactive`. For most incoming webhooks,
you'll want to squash your changes into a single commit and include a good,
clear commit message.
4. Push code to your fork.
5. Submit a pull request to zulip/zulip.
If you would like feedback on your integration as you go, feel free to post a
message on the [public Zulip instance](https://chat.zulip.org/#narrow/stream/bots).
You can also create a [draft pull request](
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests#draft-pull-requests) while you
are still working on your integration. See the
[Git guide](https://zulip.readthedocs.io/en/latest/git/pull-requests.html#create-a-pull-request)
for more on Zulip's pull request process.
## Advanced topics
More complex implementation or testing needs may require additional code, beyond
what the standard helper functions provide. This section discusses some of
these situations.
### Negative tests
A negative test is one that should result in an error, such as incorrect data.
The helper functions may interpret this as a test failure, when it should instead
be a successful test of an error condition. To correctly test these cases, you
must explicitly code your test's execution (using other helpers, as needed)
rather than call the usual helper function.
Here is an example from the WordPress integration:
```python
def test_unknown_action_no_data(self) -> None:
# Mimic check_webhook() to manually execute a negative test.
# Otherwise its call to send_webhook_payload() would assert on the non-success
# we are testing. The value of result is the error message the webhook should
# return if no params are sent. The fixture for this test is an empty file.
# subscribe to the target stream
self.subscribe(self.test_user, self.STREAM_NAME)
# post to the webhook url
post_params = {'stream_name': self.STREAM_NAME,
'content_type': 'application/x-www-form-urlencoded'}
result = self.client_post(self.url, 'unknown_action', **post_params)
# check that we got the expected error message
self.assert_json_error(result, "Unknown WordPress webhook action: WordPress action")
```
In a normal test, `check_webhook` would handle all the setup
and then check that the incoming webhook's response matches the expected result. If
the webhook returns an error, the test fails. Instead, explicitly do the
setup it would have done, and check the result yourself.
Here, `subscribe_to_stream` is a test helper that uses `TEST_USER_EMAIL` and
`STREAM_NAME` (attributes from the base class) to register the user to receive
messages in the given stream. If the stream doesn't exist, it creates it.
`client_post`, another helper, performs the HTTP POST that calls the incoming
webhook. As long as `self.url` is correct, you don't need to construct the webhook
URL yourself. (In most cases, it is.)
`assert_json_error` then checks if the result matches the expected error.
If you had used `check_webhook`, it would have called
`send_webhook_payload`, which checks the result with `assert_json_success`.
### Custom query parameters
Custom arguments passed in URL query parameters work as expected in the webhook
code, but require special handling in tests.
For example, here is the definition of a webhook function that gets both `stream`
and `topic` from the query parameters:
```python
def api_querytest_webhook(request: HttpRequest, user_profile: UserProfile,
payload: str=REQ(argument_type='body'),
stream: str=REQ(default='test'),
topic: str=REQ(default='Default Alert')):
```
In actual use, you might configure the 3rd party service to call your Zulip
integration with a URL like this:
```
http://myhost/api/v1/external/querytest?api_key=abcdefgh&stream=alerts&topic=queries
```
It provides values for `stream` and `topic`, and the webhook can get those
using `REQ` without any special handling. How does this work in a test?
The new attribute `TOPIC` exists only in our class so far. In order to
construct a URL with a query parameter for `topic`, you can pass the
attribute `TOPIC` as a keyword argument to `build_webhook_url`, like so:
```python
class QuerytestHookTests(WebhookTestCase):
STREAM_NAME = 'querytest'
TOPIC = "Default topic"
URL_TEMPLATE = "/api/v1/external/querytest?api_key={api_key}&stream={stream}"
FIXTURE_DIR_NAME = 'querytest'
def test_querytest_test_one(self) -> None:
# construct the URL used for this test
self.TOPIC = "Query test"
self.url = self.build_webhook_url(topic=self.TOPIC)
# define the expected message contents
expected_topic = "Query test"
expected_message = "This is a test of custom query parameters."
self.check_webhook('test_one', expected_topic, expected_message,
content_type="application/x-www-form-urlencoded")
```
You can also override `get_body` or `get_payload` if your test data
needs to be constructed in an unusual way.
For more, see the definition for the base class, `WebhookTestCase`
in `zerver/lib/test_classes.py`, or just grep for examples.
### Custom HTTP event-type headers
Some third-party services set a custom HTTP header to indicate the event type that
generates a particular payload. To extract such headers, we recommend using the
`validate_extract_webhook_http_header` function in `zerver/lib/webhooks/common.py`,
like so:
```python
event = validate_extract_webhook_http_header(request, header, integration_name)
```
`request` is the `HttpRequest` object passed to your main webhook function. `header`
is the name of the custom header you'd like to extract, such as `X-Event-Key`, and
`integration_name` is the name of the third-party service in question, such as
`GitHub`.
Because such headers are how some integrations indicate the event types of their
payloads, the absence of such a header usually indicates a configuration
issue, where one either entered the URL for a different integration, or happens to
be running an older version of the integration that doesn't set that header.
If the requisite header is missing, this function sends a PM to the owner of the
webhook bot, notifying them of the missing header.
### Handling unexpected webhook event types
Many third-party services have dozens of different event types. In
some cases, we may choose to explicitly ignore specific events. In
other cases, there may be events that are new or events that we don't
know about. In such cases, we recommend raising
`UnsupportedWebhookEventTypeError` (found in `zerver/lib/exceptions.py`),
with a string describing the unsupported event type, like so:
```
raise UnsupportedWebhookEventTypeError(event_type)
```

26
api_docs/index.md Normal file
View File

@@ -0,0 +1,26 @@
# The Zulip API
Zulip's APIs allow you to integrate other services with Zulip. This
guide should help you find the API you need:
* First, check if the tool you'd like to integrate with Zulip
[already has a native integration](/integrations).
* Next, check if [Zapier](https://zapier.com/apps) or
[IFTTT](https://ifttt.com/search) has an integration.
[Zulip's Zapier integration](/integrations/doc/zapier) and
[Zulip's IFTTT integration](/integrations/doc/ifttt) often allow
integrating a new service with Zulip without writing any code.
* If you'd like to send content into Zulip, you can
[write a native incoming webhook integration](/api/incoming-webhooks-overview)
or use [Zulip's API for sending messages](/api/send-message).
* If you're building an interactive bot that reacts to activity inside
Zulip, you'll want to look at Zulip's
[Python framework for interactive bots](/api/running-bots) or
[Zulip's real-time events API](/api/get-events).
And if you still need to build your own integration with Zulip, check out
the full [REST API](/api/rest), generally starting with
[installing the API client bindings](/api/installation-instructions).
In case you already know how you want to build your integration and you're
just looking for an API key, we've got you covered [here](/api/api-keys).

42
api_docs/installation-instructions.md Normal file
View File

@@ -0,0 +1,42 @@
# Installation instructions
Zulip's REST API is easy to work with directly, but there are API
libraries available for a few popular languages.
The Python library is the most advanced (and has tools for easily
writing interactive bots that react to messages), so we recommend it
if you're trying to decide.
{start_tabs}
{tab|python}
Install the Python API with [pip](https://pypi.python.org/pypi/zulip/):
```
pip install zulip
```
{tab|zulip-send}
Included with the Python bindings:
```
pip install zulip
```
{tab|js}
Install the JavaScript API with [npm](https://www.npmjs.com/package/zulip-js):
```
npm install zulip-js
```
{tab|curl}
No download required!
{end_tabs}
See also [user-contributed client libraries](/api/client-libraries)
for many other languages.

69
api_docs/integrations-overview.md Normal file
View File

@@ -0,0 +1,69 @@
# Integrations overview
Integrations allow you to send data from other products into or out of
Zulip. Zulip natively integrates with dozens of products, and with hundreds
more through Zapier and IFTTT.
Zulip also makes it very easy to write your own integration, and (if you'd
like) to get it merged into the main Zulip repository.
Integrations are one of the most important parts of a group chat tool like
Zulip, and we are committed to making integrating with Zulip as easy as
possible.
## Set up an existing integration
Most existing integrations send content from a third-party product into
Zulip.
* Search Zulip's [list of native integrations](/integrations) for the
third-party product. Each integration has a page describing how to set it
up.
* Check if [Zapier](https://zapier.com/apps) has an integration with the
product. If it does, follow [these instructions](/integrations/doc/zapier)
to set it up.
* Check if [IFTTT](https://ifttt.com/search) has an integration with the
product. If it does, follow [these instructions](/integrations/doc/ifttt)
to set it up.
* Use a third-party webhook integration designed to work with
[Slack's webhook API](https://api.slack.com/messaging/webhooks)
pointed at Zulip's
[Slack-compatible webhook API](/integrations/slack/slack_incoming).
* If the product can send email notifications, you can
[send those emails to a stream](/help/message-a-stream-by-email).
## Write your own integration
We've put a lot of effort into making this as easy as possible, but
all of the options below do require some comfort writing code. If you
need an integration and don't have an engineer on staff, [contact
us](/help/contact-support) and we'll see what we can do.
### Sending content into Zulip
* If the third-party service supports outgoing webhooks, you likely want to
build an [incoming webhook integration](/api/incoming-webhooks-overview).
* If it doesn't, you may want to write a
[script or plugin integration](/api/non-webhook-integrations).
* Finally, you can
[send messages using Zulip's API](/api/send-message).
### Sending and receiving content
* To react to activity inside Zulip, look at Zulip's
[Python framework for interactive bots](/api/running-bots) or
[Zulip's real-time events API](/api/get-events).
* If what you want isn't covered by the above, check out the full
[REST API](/api/rest). The web, mobile, desktop, and terminal apps are
built on top of this API, so it can do anything a human user can do. Most
but not all of the endpoints are documented on this site; if you need
something that isn't there check out Zulip's
[REST endpoints](https://github.com/zulip/zulip/blob/main/zproject/urls.py)
or [contact us](/help/contact-support) and we'll help you out.

89
api_docs/mark-all-as-read.md Normal file
View File

@@ -0,0 +1,89 @@
{generate_api_header(/mark_all_as_read:post)}
## Usage examples
{start_tabs}
{generate_code_example(python)|/mark_all_as_read:post|example}
{generate_code_example(javascript)|/mark_all_as_read:post|example}
{tab|curl}
{generate_code_example(curl)|/mark_all_as_read:post|example}
{end_tabs}
## Parameters
{generate_api_arguments_table|zulip.yaml|/mark_all_as_read:post}
{generate_parameter_description(/mark_all_as_read:post)}
## Response
{generate_response_description(/mark_all_as_read:post)}
#### Example response(s)
{generate_code_example|/mark_all_as_read:post|fixture}
{generate_api_header(/mark_stream_as_read:post)}
## Usage examples
{start_tabs}
{generate_code_example(python)|/mark_stream_as_read:post|example}
{generate_code_example(javascript)|/mark_all_as_read:post|example}
{tab|curl}
{generate_code_example(curl)|/mark_stream_as_read:post|example}
{end_tabs}
## Parameters
{generate_api_arguments_table|zulip.yaml|/mark_stream_as_read:post}
{generate_parameter_description(/mark_all_as_read:post)}
## Response
{generate_response_description(/mark_all_as_read:post)}
#### Example response(s)
{generate_code_example|/mark_stream_as_read:post|fixture}
{generate_api_header(/mark_topic_as_read:post)}
## Usage examples
{start_tabs}
{generate_code_example(python)|/mark_topic_as_read:post|example}
{generate_code_example(javascript)|/mark_all_as_read:post|example}
{tab|curl}
{generate_code_example(curl)|/mark_topic_as_read:post|example}
{end_tabs}
## Parameters
{generate_api_arguments_table|zulip.yaml|/mark_topic_as_read:post}
{generate_parameter_description(/mark_all_as_read:post)}
## Response
{generate_response_description(/mark_all_as_read:post)}
#### Example response(s)
{generate_code_example|/mark_topic_as_read:post|fixture}

1
api_docs/missing.md Normal file
View File

@@ -0,0 +1 @@
No such article.

53
api_docs/non-webhook-integrations.md Normal file
View File

@@ -0,0 +1,53 @@
# Non-webhook integrations
[Incoming webhook integrations](/api/incoming-webhooks-overview) are the
fastest to write, but sometimes a third-party product just doesn't support
them. Zulip supports several other types of integrations.
1. **Python script integrations**
(examples: SVN, Git), where we can get the service to call our integration
(by shelling out or otherwise), passing in the required data. Our preferred
model for these is to ship these integrations in the
[Zulip Python API distribution](https://github.com/zulip/python-zulip-api/tree/main/zulip),
within the `integrations` directory there.
1. **Plugin integrations** (examples:
Jenkins, Hubot, Trac) where the user needs to install a plugin into their
existing software. These are often more work, but for some products are the
only way to integrate with the product at all.
For plugin integrations, usually you will need to consult the
documentation for the third party software in order to learn how to
write the integration.
1. **Interactive bots**. See [Writing bots](/api/writing-bots).
A few notes on how to do these:
* You should always send messages by POSTing to URLs of the form
`https://zulip.example.com/v1/messages/`.
* We usually build Python script integrations with (at least) 2 files:
`zulip_foo_config.py` containing the configuration for the
integration including the bots' API keys, plus a script that reads
from this configuration to actually do the work (that way, it's
possible to update the script without breaking users' configurations).
* Be sure to test your integration carefully and
[document](https://zulip.readthedocs.io/en/latest/documentation/integrations.html)
how to install it.
* You should specify a clear HTTP User-Agent for your integration. The
user agent should at a minimum identify the integration and version
number, separated by a slash. If possible, you should collect platform
information and include that in `()`s after the version number. Some
examples of ideal UAs are:
```
ZulipDesktop/0.7.0 (Ubuntu; 14.04)
ZulipJenkins/0.1.0 (Windows; 7.2)
ZulipMobile/0.5.4 (Android; 4.2; maguro)
```
* The [general advice](/api/incoming-webhooks-overview#general-advice) for
webhook integrations applies here as well.

184
api_docs/outgoing-webhooks.md Normal file
View File

@@ -0,0 +1,184 @@
# Outgoing webhooks
Outgoing webhooks allow you to build or set up Zulip integrations
which are notified when certain types of messages are sent in
Zulip. When one of those events is triggered, we'll send a HTTP POST
payload to the webhook's configured URL. Webhooks can be used to
power a wide range of Zulip integrations. For example, the
[Zulip Botserver][zulip-botserver] is built on top of this API.
Zulip supports outgoing webhooks both in a clean native Zulip format,
as well as a format that's compatible with
[Slack's outgoing webhook API][slack-outgoing-webhook], which can help
with porting an existing Slack integration to work with Zulip.
[zulip-botserver]: /api/deploying-bots#zulip-botserver
[slack-outgoing-webhook]: https://api.slack.com/custom-integrations/outgoing-webhooks
To register an outgoing webhook:
* Log in to the Zulip server.
* Navigate to *Personal settings (<i class="fa fa-cog"></i>)* -> *Bots* ->
*Add a new bot*. Select *Outgoing webhook* for bot type, the URL
you'd like Zulip to post to as the **Endpoint URL**, the format you
want, and click on *Create bot*. to submit the form/
* Your new bot user will appear in the *Active bots* panel, which you
can use to edit the bot's settings.
## Triggering
There are currently two ways to trigger an outgoing webhook:
1. **@-mention** the bot user in a stream. If the bot replies, its
reply will be sent to that stream and topic.
2. **Send a private message** with the bot as one of the recipients.
If the bot replies, its reply will be sent to that thread.
## Timeouts
The remote server must respond to a `POST` request in a timely manner.
The default timeout for outgoing webhooks is 10 seconds, though this
can be configured by the administrator of the Zulip server by setting
`OUTGOING_WEBHOOKS_TIMEOUT_SECONDS` in the [server's
settings][settings].
[settings]: https://zulip.readthedocs.io/en/latest/subsystems/settings.html#server-settings
## Outgoing webhook format
{generate_code_example|/zulip-outgoing-webhook:post|fixture}
### Fields documentation
{generate_return_values_table|zulip.yaml|/zulip-outgoing-webhook:post}
## Replying with a message
Many bots implemented using this outgoing webhook API will want to
send a reply message into Zulip. Zulip's outgoing webhook API
provides a convenient way to do that by simply returning an
appropriate HTTP response to the Zulip server.
A correctly implemented bot will return a JSON object containing one
of two possible formats, described below.
### Example response payloads
If the bot code wants to opt out of responding, it can explicitly
encode a JSON dictionary that contains `response_not_required` set
to `True`, so that no response message is sent to the user. (This
is helpful to distinguish deliberate non-responses from bugs.)
Here's an example of the JSON your server should respond with if
you would not like to send a response message:
```json
{
"response_not_required": true
}
```
Here's an example of the JSON your server should respond with if
you would like to send a response message:
```json
{
"content": "Hey, we just received **something** from Zulip!"
}
```
The `content` field should contain Zulip-format Markdown.
Note that an outgoing webhook bot can use the [Zulip REST
API](/api/rest) with its API key in case your bot needs to do
something else, like add an emoji reaction or upload a file.
## Slack-format webhook format
This interface translates Zulip's outgoing webhook's request into the
format that Slack's outgoing webhook interface sends. As a result,
one should be able to use this to interact with third-party
integrations designed to work with Slack's outgoing webhook interface.
Here's how we fill in the fields that a Slack-format webhook expects:
<table class="table">
<thead>
<tr>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>token</code></td>
<td>A string of alphanumeric characters you can use to
authenticate the webhook request (each bot user uses a fixed token)</td>
</tr>
<tr>
<td><code>team_id</code></td>
<td>ID of the Zulip organization prefixed by "T".</td>
</tr>
<tr>
<td><code>team_domain</code></td>
<td>Hostname of the Zulip organization</td>
</tr>
<tr>
<td><code>channel_id</code></td>
<td>Stream ID prefixed by "C"</td>
</tr>
<tr>
<td><code>channel_name</code></td>
<td>Stream name</td>
</tr>
<tr>
<td><code>thread_ts</code></td>
<td>Timestamp for when message was sent</td>
</tr>
<tr>
<td><code>timestamp</code></td>
<td>Timestamp for when message was sent</td>
</tr>
<tr>
<td><code>user_id</code></td>
<td>ID of the user who sent the message prefixed by "U"</td>
</tr>
<tr>
<td><code>user_name</code></td>
<td>Full name of sender</td>
</tr>
<tr>
<td><code>text</code></td>
<td>The content of the message (in Markdown)</td>
</tr>
<tr>
<td><code>trigger_word</code></td>
<td>Trigger method</td>
</tr>
<tr>
<td><code>service_id</code></td>
<td>ID of the bot user</td>
</tr>
</tbody>
</table>
The above data is posted as list of tuples (not JSON), here's an example:
```
[('token', 'v9fpCdldZIej2bco3uoUvGp06PowKFOf'),
('team_id', 'T1512'),
('team_domain', 'zulip.example.com'),
('channel_id', 'C123'),
('channel_name', 'integrations'),
('thread_ts', 1532078950),
('timestamp', 1532078950),
('user_id', 'U21'),
('user_name', 'Full Name'),
('text', '@**test**'),
('trigger_word', 'mention'),
('service_id', 27)]
```
* For successful request, if data is returned, it returns that data,
else it returns a blank response.
* For failed request, it returns the reason of failure, as returned by
the server, or the exception message.

60
api_docs/real-time-events.md Normal file
View File

@@ -0,0 +1,60 @@
# Real-time events API
Zulip's real-time events API lets you write software that reacts
immediately to events happening in Zulip. This API is what powers the
real-time updates in the Zulip web and mobile apps. As a result, the
events available via this API cover all changes to data displayed in
the Zulip product, from new messages to stream descriptions to
emoji reactions to changes in user or organization-level settings.
## Using the events API
The simplest way to use Zulip's real-time events API is by using
`call_on_each_event` from our Python bindings. You just need to write
a Python function (in the examples below, the `lambda`s) and pass it
into `call_on_each_event`; your function will be called whenever a new
event matching the specified parameters (`event_types`, `narrow`,
etc.) occurs in Zulip.
`call_on_each_event` takes care of all the potentially tricky details
of long-polling, error handling, exponential backoff in retries, etc.
It's cousin, `call_on_each_message`, provides an even simpler
interface for processing Zulip messages.
More complex applications (like a Zulip terminal client) may need to
instead use the raw [register](/api/register-queue) and
[events](/api/get-events) endpoints.
## Usage examples
{start_tabs}
{tab|python}
```
#!/usr/bin/env python
import sys
import zulip
# Pass the path to your zuliprc file here.
client = zulip.Client(config_file="~/zuliprc")
# Print every message the current user would receive
# This is a blocking call that will run forever
client.call_on_each_message(lambda msg: sys.stdout.write(str(msg) + "\n"))
# Print every event relevant to the user
# This is a blocking call that will run forever
client.call_on_each_event(lambda event: sys.stdout.write(str(event) + "\n"))
```
{end_tabs}
## Parameters
You may also pass in the following keyword arguments to `call_on_each_event`:
{generate_api_arguments_table|zulip.yaml|/real-time:post}
See the [GET /events](/api/get-events) documentation for
more details on the format of individual events.

46
api_docs/rest-error-handling.md Normal file
View File

@@ -0,0 +1,46 @@
# Error handling
Zulip's API will always return a JSON format response.
The HTTP status code indicates whether the request was successful
(200 = success, 40x = user error, 50x = server error). Every response
will contain at least two keys: `msg` (a human-readable error message)
and `result`, which will be either `error` or `success` (this is
redundant with the HTTP status code, but is convenient when printing
responses while debugging).
For some common errors, Zulip provides a `code` attribute. Where
present, clients should check `code`, rather than `msg`, when looking
for specific error conditions, since the `msg` strings are
internationalized (e.g. the server will send the error message
translated into French if the user has a French locale).
Each endpoint documents its own unique errors; below, we document
errors common to many endpoints:
{generate_code_example|/rest-error-handling:post|fixture}
To help clients avoid exceeding rate limits, Zulip sets the following
HTTP headers in all API responses:
* `X-RateLimit-Remaining`: The number of additional requests of this
type that the client can send before exceeding its limit.
* `X-RateLimit-Limit`: The limit that would be applicable to a client
that had not made any recent requests of this type. This is useful
for designing a client's burst behavior so as to avoid ever reaching
a rate limit.
* `X-RateLimit-Reset`: The time at which the client will no longer
have any rate limits applied to it (and thus could do a burst of
`X-RateLimit-Limit` requests).
[Zulip's rate limiting rules are configurable][rate-limiting-rules],
and can vary by server and over time. The default configuration
currently limits:
* Every user is limited to 200 total API requests per minute.
* Separate, much lower limits for authentication/login attempts.
When the Zulip server has configured multiple rate limits that apply
to a given request, the values returned will be for the strictest
limit.
[rate-limiting-rules]: https://zulip.readthedocs.io/en/latest/production/security-model.html#rate-limiting

33
api_docs/rest.md Normal file
View File

@@ -0,0 +1,33 @@
# The Zulip REST API
The Zulip REST API powers the Zulip web and mobile apps, so anything
you can do in Zulip, you can do with Zulip's REST API. To use this API:
* You'll need to [get an API key](/api/api-keys). You will likely
want to [create a bot](/help/add-a-bot-or-integration), unless you're
using the API to interact with
your own account (e.g. exporting your personal message history).
* Choose what language you'd like to use. You can download the
[Python or JavaScript bindings](/api/installation-instructions), projects in
[other languages](/api/client-libraries), or
just make HTTP requests with your favorite programming language. If
you're making your own HTTP requests, you'll want to send the
appropriate HTTP basic authentication headers; see each endpoint's
`curl` option for details on the request format.
* The Zulip API has a standard
[system for reporting errors](/api/rest-error-handling).
Most other details are covered in the documentation for the individual
endpoints:
!!! tip ""
You may use the `client.call_endpoint` method of our Python API
bindings to call an endpoint that isn't documented here. For an
example, see [Upload a custom emoji](/api/upload-custom-emoji).
{!rest-endpoints.md!}
Since Zulip is open source, you can also consult the
[Zulip server source code](https://github.com/zulip/zulip/) as a
workaround for how to do anything not documented here.

120
api_docs/roles-and-permissions.md Normal file
View File

@@ -0,0 +1,120 @@
# Roles and permissions
Zulip offers several levels of permissions based on a
[user's role](/help/roles-and-permissions) in a Zulip organization.
Here are some important details to note when working with these
roles and permissions in Zulip's API:
## A user's role
A user's account data include a `role` property, which contains the
user's role in the Zulip organization. These roles are encoded as:
* Organization owner: 100
* Organization administrator: 200
* Organization moderator: 300
* Member: 400
* Guest: 600
User account data also include these boolean properties that duplicate
the related roles above:
* `is_owner` specifying whether the user is an organization owner.
* `is_admin` specifying whether the user is an organization administrator.
* `is_guest` specifying whether the user is a guest user.
These are intended as conveniences for simple clients, and clients
should prefer using the `role` field, since only that one is updated
by the [events API](/api/get-events).
Note that [`POST /register`](/api/register-queue) also returns an
`is_moderator` boolean property specifying whether the current user is
an organization moderator.
Additionally, user account data include an `is_billing_admin` property
specifying whether the user is a billing administrator for the Zulip
organization, which is not related to one of the roles listed above,
but rather allows for specific permissions related to billing
administration in [paid Zulip Cloud plans](https://zulip.com/plans/).
### User account data in the API
Endpoints that return the user account data / properties mentioned
above are:
* [`GET /users`](/api/get-users)
* [`GET /users/{user_id}`](/api/get-user)
* [`GET /users/{email}`](/api/get-user-by-email)
* [`GET /users/me`](/api/get-own-user)
* [`GET /events`](/api/get-events)
* [`POST /register`](/api/register-queue)
Note that the [`POST /register` endpoint](/api/register-queue) returns
the above boolean properties to describe the role of the current user,
when `realm_user` is present in `fetch_event_types`.
Additionally, the specific events returned by the
[`GET /events` endpoint](/api/get-events) containing data related
to user accounts and roles are the [`realm_user` add
event](/api/get-events#realm_user-add), and the
[`realm_user` update event](/api/get-events#realm_user-update).
## Permission levels
Many areas of Zulip are customizable by the roles
above, such as (but not limited to) [configuring message editing and
deletion](/help/configure-message-editing-and-deletion) and
[streams permissions](/help/stream-permissions). The potential
permission levels are:
* Everyone / Any user including Guests (least restrictive)
* Members
* Full members
* Moderators
* Administrators
* Owners
* Nobody (most restrictive)
These permission levels and policies in the API are designed to be
cutoffs in that users with the specified role and above have the
specified ability or access. For example, a permission level documented
as 'moderators only' includes organization moderators, administrators,
and owners.
Note that specific settings and policies in the Zulip API that use these
permission levels will likely support a subset of those listed above.
## Determining if a user is a full member
When a Zulip organization has set up a [waiting period before new members
turn into full members](/help/restrict-permissions-of-new-members),
clients will need to determine if a user's account has aged past the
organization's waiting period threshold.
The `realm_waiting_period_threshold`, which is the number of days until
a user's account is treated as a full member, is returned by the
[`POST /register` endpoint](/api/register-queue) when `realm` is present
in `fetch_event_types`.
Clients can compare the `realm_waiting_period_threshold` to a user
accounts's `date_joined` property, which is the time the user account
was created, to determine if a user has the permissions of a full
member or a new member.

65
api_docs/running-bots.md Normal file
View File

@@ -0,0 +1,65 @@
# Interactive bots
Zulip's API has a powerful framework for interactive bots that react
to messages in Zulip.
## Running a bot
This guide will show you how to run an existing Zulip bot
found in [zulip_bots/bots](
https://github.com/zulip/python-zulip-api/tree/main/zulip_bots/zulip_bots/bots).
You'll need:
* An account in a Zulip organization
(e.g. [the Zulip development community](https://zulip.com/development-community/),
`<yourSubdomain>.zulipchat.com`, or a Zulip organization on your own
[development](https://zulip.readthedocs.io/en/latest/development/overview.html) or
[production](https://zulip.readthedocs.io/en/latest/production/install.html) server).
* A computer where you're running the bot from.
**Note: Please be considerate when testing experimental bots on public servers such as chat.zulip.org.**
1. Go to your Zulip account and
[add a bot](/help/add-a-bot-or-integration). Use **Generic bot** as the bot type.
1. Download the bot's `zuliprc` configuration file to your computer.
1. Download the `zulip_bots` Python package to your computer using `pip3 install zulip_bots`.
*Note: Click
[here](
writing-bots#installing-a-development-version-of-the-zulip-bots-package)
to install the latest development version of the package.*
1. Start the bot process on your computer.
* Run
```
zulip-run-bot <bot-name> --config-file ~/path/to/zuliprc
```
(replacing `~/path/to/zuliprc` with the path to the `zuliprc` file you downloaded above).
* Check the output of the command. It should include the following line:
INFO:root:starting message handling...
Congrats! Your bot is running.
1. To talk with the bot, at-mention its name, like `@**bot-name**`.
You can now play around with the bot and get it configured the way you
like. Eventually, you'll probably want to run it in a production
environment where it'll stay up, by [deploying](/api/deploying-bots) it on a server using the
Zulip Botserver.
## Common problems
* My bot won't start
* Ensure that your API config file is correct (download the config file from the server).
* Ensure that your bot script is located in `zulip_bots/bots/<my-bot>/`
* Are you using your own Zulip development server? Ensure that you run your bot outside
the Vagrant environment.
* Some bots require Python 3. Try switching to a Python 3 environment before running
your bot.

77
api_docs/send-message.md Normal file
View File

@@ -0,0 +1,77 @@
{generate_api_header(/messages:post)}
## Usage examples
{start_tabs}
{generate_code_example(python)|/messages:post|example}
{generate_code_example(javascript)|/messages:post|example}
{tab|curl}
``` curl
# For stream messages
curl -X POST {{ api_url }}/v1/messages \
-u BOT_EMAIL_ADDRESS:BOT_API_KEY \
--data-urlencode type=stream \
--data-urlencode 'to="Denmark"' \
--data-urlencode topic=Castle \
--data-urlencode 'content=I come not, friends, to steal away your hearts.'
# For private messages
curl -X POST {{ api_url }}/v1/messages \
-u BOT_EMAIL_ADDRESS:BOT_API_KEY \
--data-urlencode type=private \
--data-urlencode 'to=[9]' \
--data-urlencode 'content=With mirth and laughter let old wrinkles come.'
```
{tab|zulip-send}
You can use `zulip-send`
(available after you `pip install zulip`) to easily send Zulips from
the command-line, providing the message content via STDIN.
```bash
# For stream messages
zulip-send --stream Denmark --subject Castle \
--user othello-bot@example.com --api-key a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5
# For private messages
zulip-send hamlet@example.com \
--user othello-bot@example.com --api-key a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5
```
#### Passing in the message on the command-line
If you'd like, you can also provide the message on the command-line with the
`-m` or `--message` flag, as follows:
```bash
zulip-send --stream Denmark --subject Castle \
--message 'I come not, friends, to steal away your hearts.' \
--user othello-bot@example.com --api-key a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5
```
You can omit the `user` and `api-key` parameters if you have a `~/.zuliprc`
file.
{end_tabs}
## Parameters
{generate_api_arguments_table|zulip.yaml|/messages:post}
{generate_parameter_description(/messages:post)}
## Response
{generate_return_values_table|zulip.yaml|/messages:post}
{generate_response_description(/messages:post)}
#### Example response(s)
{generate_code_example|/messages:post|fixture}

26
api_docs/sidebar_index.md Normal file
View File

@@ -0,0 +1,26 @@
## Integrations
* [Overview](/api/integrations-overview)
* [Incoming webhook integrations](/api/incoming-webhooks-overview)
* [Hello world walkthrough](/api/incoming-webhooks-walkthrough)
* [Non-webhook integrations](/api/non-webhook-integrations)
## Interactive bots (beta)
* [Running bots](/api/running-bots)
* [Deploying bots](/api/deploying-bots)
* [Writing bots](/api/writing-bots)
* [Outgoing webhooks](/api/outgoing-webhooks)
## REST API
* [Overview](/api/rest)
* [Installation instructions](/api/installation-instructions)
* [API keys](/api/api-keys)
* [Configuring the Python bindings](/api/configuring-python-bindings)
* [Error handling](/api/rest-error-handling)
* [Roles and permissions](/api/roles-and-permissions)
* [Client libraries](/api/client-libraries)
* [API changelog](/api/changelog)
{!rest-endpoints.md!}

514
api_docs/writing-bots.md Normal file
View File

@@ -0,0 +1,514 @@
# Writing interactive bots
This guide is about writing and testing interactive bots. We assume
familiarity with our [guide for running bots](running-bots).
On this page you'll find:
* A step-by-step
[guide](#installing-a-development-version-of-the-zulip-bots-package)
on how to set up a development environment for writing bots with all
of our nice tooling to make it easy to write and test your work.
* A [guide](#writing-a-bot) on writing a bot.
* A [guide](#adding-a-bot-to-zulip) on adding a bot to Zulip.
* A [guide](#testing-a-bots-output) on testing a bot's output.
* [Documentation](#bot-api) of the bot API.
* Common [problems](#common-problems) when developing/running bots and their solutions.
## Installing a development version of the Zulip bots package
1. `git clone https://github.com/zulip/python-zulip-api.git` - clone the [python-zulip-api](
https://github.com/zulip/python-zulip-api) repository.
2. `cd python-zulip-api` - navigate into your cloned repository.
3. `python3 ./tools/provision` - install all requirements in a Python virtualenv.
4. The output of `provision` will end with a command of the form `source .../activate`;
run that command to enter the new virtualenv.
5. *Finished*. You should now see the name of your venv preceding your prompt,
e.g. `(zulip-api-py3-venv)`.
*Hint: `provision` installs the `zulip`, `zulip_bots`, and
`zulip_botserver` packages in developer mode. This enables you to
modify these packages and then run your modified code without
having to first re-install the packages or re-provision.*
## Writing a bot
The tutorial below explains the structure of a bot `<my-bot>.py`,
which is the only file you need to create for a new bot. You
can use this as boilerplate code for developing your own bot.
Every bot is built upon this structure:
```python
class MyBotHandler(object):
'''
A docstring documenting this bot.
'''
def usage(self):
return '''Your description of the bot'''
def handle_message(self, message, bot_handler):
# add your code here
handler_class = MyBotHandler
```
* The class name (in this case *MyBotHandler*) can be defined by you
and should match the name of your bot. To register your bot's class,
adjust the last line `handler_class = MyBotHandler` to match your
class name.
* Every bot needs to implement the functions
* `usage(self)`
* `handle_message(self, message, bot_handler)`
* These functions are documented in the [next section](#bot-api).
## Adding a bot to Zulip
Zulip's bot system resides in the [python-zulip-api](
https://github.com/zulip/python-zulip-api) repository.
The structure of the bots ecosystem looks like the following:
```
zulip_bots
└───zulip_bots
├───bots
│ ├───bot1
│ └───bot2
│ │
│ ├───bot2.py
│ ├───bot2.conf
│ ├───doc.md
│ ├───requirements.txt
│ ├───test_bot2.py
│ ├───assets
│ │ │
│ │ └───pic.png
│ ├───fixtures
│ │ │
│ │ └───test1.json
│ └───libraries
│ │
│ └───lib1.py
   ├─── lib.py
   ├─── test_lib.py
   ├─── run.py
   └─── provision.py
```
Each subdirectory in `bots` contains a bot. When writing bots, try to use the structure outlined
above as an orientation.
## Testing a bot's output
If you just want to see how a bot reacts to a message, but don't want to set it up on a server,
we have a little tool to help you out: `zulip-bot-shell`
* [Install all requirements](#installing-a-development-version-of-the-zulip-bots-package).
* Run `zulip-bot-shell` to test one of the bots in
[`zulip_bots/bots`](https://github.com/zulip/python-zulip-api/tree/main/zulip_bots/zulip_bots/bots).
Example invocations are below:
```
> zulip-bot-shell converter
Enter your message: "12 meter yard"
Response: 12.0 meter = 13.12336 yard
> zulip-bot-shell -b ~/followup.conf followup
Enter your message: "Task completed"
Response: stream: followup topic: foo_sender@zulip.com
from foo_sender@zulip.com: Task completed
```
Note that the `-b` (aka `--bot-config-file`) argument is for an optional third party
config file (e.g. ~/giphy.conf), which only applies to certain types of bots.
## Bot API
This section documents functions available to the bot and the structure of the bot's config file.
With this API, you *can*
* intercept, view, and process messages sent by users on Zulip.
* send out new messages as replies to the processed messages.
With this API, you *cannot*
* modify an intercepted message (you have to send a new message).
* send messages on behalf of or impersonate other users.
* intercept private messages (except for PMs with the bot as an
explicit recipient).
### usage
*usage(self)*
is called to retrieve information about the bot.
#### Arguments
* self - the instance the method is called on.
#### Return values
* A string describing the bot's functionality
#### Example implementation
```python
def usage(self):
return '''
This plugin will allow users to flag messages
as being follow-up items. Users should preface
messages with "@followup".
Before running this, make sure to create a stream
called "followup" that your API user can send to.
'''
```
### handle_message
*handle_message(self, message, bot_handler)*
handles user message.
#### Arguments
* self - the instance the method is called on.
* message - a dictionary describing a Zulip message
* bot_handler - used to interact with the server, e.g. to send a message
#### Return values
None.
#### Example implementation
```python
def handle_message(self, message, bot_handler):
original_content = message['content']
original_sender = message['sender_email']
new_content = original_content.replace('@followup',
'from %s:' % (original_sender,))
bot_handler.send_message(dict(
type='stream',
to='followup',
subject=message['sender_email'],
content=new_content,
))
```
### bot_handler.send_message
*bot_handler.send_message(message)*
will send a message as the bot user. Generally, this is less
convenient than *send_reply*, but it offers additional flexibility
about where the message is sent to.
#### Arguments
* message - a dictionary describing the message to be sent by the bot
#### Example implementation
```python
bot_handler.send_message(dict(
type='stream', # can be 'stream' or 'private'
to=stream_name, # either the stream name or user's email
subject=subject, # message subject
content=message, # content of the sent message
))
```
### bot_handler.send_reply
*bot_handler.send_reply(message, response)*
will reply to the triggering message to the same place the original
message was sent to, with the content of the reply being *response*.
#### Arguments
* message - Dictionary containing information on message to respond to
(provided by `handle_message`).
* response - Response message from the bot (string).
### bot_handler.update_message
*bot_handler.update_message(message)*
will edit the content of a previously sent message.
#### Arguments
* message - dictionary defining what message to edit and the new content
#### Example
From `zulip_bots/bots/incrementor/incrementor.py`:
```python
bot_handler.update_message(dict(
message_id=self.message_id, # id of message to be updated
content=str(self.number), # string with which to update message with
))
```
### bot_handler.storage
A common problem when writing an interactive bot is that you want to
be able to store a bit of persistent state for the bot (e.g. for an
RSVP bot, the RSVPs). For a sufficiently complex bot, you want need
your own database, but for simpler bots, we offer a convenient way for
bot code to persistently store data.
The interface for doing this is `bot_handler.storage`.
The data is stored in the Zulip Server's database. Each bot user has
an independent storage quota available to it.
#### Performance considerations
Since each access to `bot_handler.storage` will involve a round-trip
to the server, we recommend writing bots so that they do a single
`bot_handler.storage.get` at the start of `handle_message`, and a
single `bot_handler.storage.put` at the end to submit the state to the
server. We plan to offer a context manager that takes care of this
automatically.
#### bot_handler.storage.put
*bot_handler.storage.put(key, value)*
will store the value `value` in the entry `key`.
##### Arguments
* key - a UTF-8 string
* value - a UTF-8 string
##### Example
```python
bot_handler.storage.put("foo", "bar") # set entry "foo" to "bar"
```
#### bot_handler.storage.get
*bot_handler.storage.get(key)*
will retrieve the value for the entry `key`.
###### Arguments
* key - a UTF-8 string
##### Example
```python
bot_handler.storage.put("foo", "bar")
print(bot_handler.storage.get("foo")) # print "bar"
```
#### bot_handler.storage.contains
*bot_handler.storage.contains(key)*
will check if the entry `key` exists.
##### Arguments
* key - a UTF-8 string
##### Example
```python
bot_handler.storage.contains("foo") # False
bot_handler.storage.put("foo", "bar")
bot_handler.storage.contains("foo") # True
```
#### bot_handler.storage marshaling
By default, `bot_handler.storage` accepts any object for keys and
values, as long as it is JSON-able. Internally, the object then gets
converted to an UTF-8 string. You can specify custom data marshaling
by setting the functions `bot_handler.storage.marshal` and
`bot_handler.storage.demarshal`. These functions parse your data on
every call to `put` and `get`, respectively.
### Configuration file
```
[api]
key=<api-key>
email=<email>
site=<dev-url>
```
* key - the API key you created for the bot; this is how Zulip knows
the request is from an authorized user.
* email - the email address of the bot, e.g. `some-bot@zulip.com`
* site - your development environment URL; if you are working on a
development environment hosted on your computer, use
`localhost:9991`
## Writing tests for bots
Bots, like most software that you want to work, should have unit tests. In this section,
we detail our framework for writing unit tests for bots. We require that bots in the main
[`python-zulip-api`](https://github.com/zulip/python-zulip-api/tree/main/zulip_bots/zulip_bots/bots)
repository include a reasonable set of unit tests, so that future developers can easily
refactor them.
*Unit tests for bots make heavy use of mocking. If you want to get comfortable with mocking,
mocking strategies, etc. you should check out our [mocking guide](
https://zulip.readthedocs.io/en/latest/testing/testing-with-django.html#testing-with-mocks).*
### A simple example
Let's have a look at a simple test suite for the [`helloworld`](
https://github.com/zulip/python-zulip-api/tree/main/zulip_bots/zulip_bots/bots/helloworld)
bot.
```python
from zulip_bots.test_lib import StubBotTestCase
class TestHelpBot(StubBotTestCase):
bot_name: str = "helloworld"
def test_bot(self) -> None:
dialog = [
('', 'beep boop'),
('help', 'beep boop'),
('foo', 'beep boop'),
]
self.verify_dialog(dialog)
```
The `helloworld` bot replies with "beep boop" to every message @-mentioning it. We
want our test to verify that the bot **actually** does that.
Note that our helper method `verify_dialog` simulates the conversation for us, and
we just need to set up a list of tuples with expected results.
The best way to learn about bot tests is to read all the existing tests in the
`bots` subdirectories.
### Testing your test
Once you have written a test suite, you want to verify that everything works as expected.
* To test a bot in [Zulip's bot directory](
https://github.com/zulip/python-zulip-api/tree/main/zulip_bots/zulip_bots/bots):
`tools/test-bots <botname>`
* To run all bot tests: `tools/test-bots`
### Advanced testing
This section shows advanced testing techniques for more complicated bots that have
configuration files or interact with third-party APIs.
*The code for the bot testing library can be found [here](
https://github.com/zulip/python-zulip-api/blob/main/zulip_bots/zulip_bots/test_lib.py).*
#### Testing bots with config files
Some bots, such as [Giphy](
https://github.com/zulip/python-zulip-api/tree/main/zulip_bots/zulip_bots/bots/giphy),
support or require user configuration options to control how the bot works.
To test such a bot, you can use the following pattern:
with self.mock_config_info(dict(api_key=12345)):
# self.verify_reply(...)
`mock_config_info()` replaces the actual step of reading configuration from the file
system and gives your test "dummy data" instead.
#### Testing bots with internet access
Some bots, such as [Giphy](
https://github.com/zulip/python-zulip-api/tree/main/zulip_bots/zulip_bots/bots/giphy),
depend on a third-party service, such as the Giphy web app, in order to work. Because
we want our test suite to be reliable and not add load to these third-party APIs, tests
for these services need to have "test fixtures": sample HTTP request/response pairs to
be used by the tests. You can specify which one to use in your test code using the
following helper method:
with self.mock_http_conversation('test_fixture_name'):
# self.assert_bot_response(...)
`mock_http_conversation(fixture_name)` patches `requests.get` and returns the data specified
in the file `fixtures/{fixture_name}.json`. Use the following JSON code as a skeleton for new
fixtures:
```json
{
"request": {
"api_url": "http://api.example.com/",
"params": {
}
},
"response": {
},
"response-headers": {
}
}
```
For an example, check out the [giphy bot](
https://github.com/zulip/python-zulip-api/tree/main/zulip_bots/zulip_bots/bots/giphy).
*Tip: You can use [requestbin](https://requestbin.com/) or a similar
tool to capture payloads from the service your bot is interacting
with.*
#### Examples
Check out our [bots](https://github.com/zulip/python-zulip-api/tree/main/zulip_bots/zulip_bots/bots)
to see examples of bot tests.
## Common problems
* I modified my bot's code, yet the changes don't seem to have an effect.
* Ensure that you restarted the `zulip-run-bot` script.
* My bot won't start
* Ensure that your API config file is correct (download the config file from the server).
* Ensure that you bot script is located in `zulip_bots/bots/<my-bot>/`
* Are you using your own Zulip development server? Ensure that you run your bot outside
the Vagrant environment.
* Some bots require Python 3. Try switching to a Python 3 environment before running
your bot.
## Future direction
The long-term plan for this bot system is to allow the same
`ExternalBotHandler` code to eventually be usable in several contexts:
* Run directly using the Zulip `call_on_each_message` API, which is
how the implementation above works. This is great for quick
development with minimal setup.
* Run in a simple Python webserver server, processing messages
received from Zulip's outgoing webhooks integration.
* For bots merged into the mainline Zulip codebase, enabled via a
button in the Zulip web UI, with no code deployment effort required.