From feb25b0e6bb47cf196931a15bf6486745a39b42b Mon Sep 17 00:00:00 2001 From: PieterCK Date: Wed, 4 Jun 2025 18:42:43 +0700 Subject: [PATCH] integrations: Move configs in `config_options` to `url_options`. Currently we have 2 implementations of `config_options`: - It's used for generating optional webhook URL parameters. These settings also come with custom UI in the "Generate integration URL" modal. - In `/bots` API, it's used as schema for the bots `BotConfigData`. Each type of bots have different ways of defining their `BotConfigData` fields. Currently, only embedded bots use `BotConfigData`, and only the incoming webhooks use `config_options` to configure a bot's `BotConfigData`; thus, the `config_options` remain unused. To avoid confusion as to which implementation of `config_options` is used by an integration, this separates the first use case -- to generate optional webhook URL -- to a new field called `url_options`. Thus, the `config_options` field is reserved only for the second use case. --- api_docs/incoming-webhooks-walkthrough.md | 56 +++++++++++++++----- api_docs/unmerged.d/ZF-f9d19d.md | 7 +++ web/src/integration_url_modal.ts | 20 +++---- web/src/state_data.ts | 9 ++++ zerver/lib/events.py | 10 ++++ zerver/lib/integrations.py | 48 +++++++---------- zerver/lib/webhooks/common.py | 7 +++ zerver/openapi/zulip.yaml | 63 +++++++++++++++++------ 8 files changed, 151 insertions(+), 69 deletions(-) create mode 100644 api_docs/unmerged.d/ZF-f9d19d.md diff --git a/api_docs/incoming-webhooks-walkthrough.md b/api_docs/incoming-webhooks-walkthrough.md index a6d0f17e29..db8bf4c3a3 100644 --- a/api_docs/incoming-webhooks-walkthrough.md +++ b/api_docs/incoming-webhooks-walkthrough.md @@ -210,26 +210,54 @@ tools which you can use to test your webhook - 2 command line tools and a GUI. ### Webhooks requiring custom configuration -In rare cases, it's necessary for an incoming webhook to require -additional user configuration beyond what is specified in the post -URL. The typical use case for this is APIs like the Stripe API that -require clients to do a callback to get details beyond an opaque -object ID that one would want to include in a Zulip notification. +In cases where an incoming webhook integration supports optional URL parameters, +one can use the `url_options` feature. It's a field in the `WebhookIntegration` +class that is used when [generating a URL for an integration](/help/generate-integration-url) +in the web app, which encodes the user input for each URL parameter in the +incoming webhook's URL. -These configuration options are declared as follows: +These URL options are declared as follows: ```python - WebhookIntegration('helloworld', ['misc'], display_name='Hello World', - config_options=[('HelloWorld API key', 'hw_api_key', check_string)]) + WebhookIntegration( + 'helloworld', + ... + url_options=[ + WebhookUrlOption( + name='ignore_private_repositories', + label='Exclude notifications from private repositories', + validator=check_string + ), + ], + ) ``` -`config_options` is a list describing the parameters the user should -configure: - 1. A user-facing string describing the field to display to users. - 2. The field name you'll use to access this from your `view.py` function. - 3. A Validator, used to verify the input is valid. +`url_options` is a list describing the parameters the web app UI should offer when +generating the incoming webhook URL: -Common validators are available in `zerver/lib/validators.py`. + - `name`: The parameter name that is used to encode the user input in the + integration's webhook URL. + - `label`: A short descriptive label for this URL parameter in the web app UI. + - `validator`: A validator function, which is used to determine the input type + for this option in the UI, and to indicate how to validate the input. + Currently, the web app UI only supports these validators: + - `check_bool` for checkbox/select input. + - `check_string` for text input. + +!!! warn "" + + **Note**: To add support for other validators, you can update + `web/src/integration_url_modal.ts`. Common validators are available in + `zerver/lib/validator.py`. + +In rare cases, it may be necessary for an incoming webhook to require +additional user configuration beyond what is specified in the POST +URL. A typical use case for this would be APIs that require clients +to do a callback to get details beyond an opaque object ID that one +would want to include in a Zulip notification message. + +The `config_options` field in the `WebhookIntegration` class is reserved +for this use case. ## Step 4: Manually testing the webhook diff --git a/api_docs/unmerged.d/ZF-f9d19d.md b/api_docs/unmerged.d/ZF-f9d19d.md new file mode 100644 index 0000000000..44f7aa300c --- /dev/null +++ b/api_docs/unmerged.d/ZF-f9d19d.md @@ -0,0 +1,7 @@ +* [`POST /register`](/api/register-queue): Added a `url_options` object + to the `realm_incoming_webhook_bots` object for incoming webhook + integration URL parameter options. Previously, these optional URL + parameters were included in the `config_options` field (see feature + level 318 entry). The `config_options` object is now reserved for + configuration data that can be set when creating an bot user for a + specific incoming webhook integration. diff --git a/web/src/integration_url_modal.ts b/web/src/integration_url_modal.ts index 206c505140..a88c0f7b58 100644 --- a/web/src/integration_url_modal.ts +++ b/web/src/integration_url_modal.ts @@ -19,19 +19,19 @@ import {realm} from "./state_data.ts"; import * as stream_data from "./stream_data.ts"; import * as util from "./util.ts"; -type ConfigOption = { +type UrlOption = { key: string; label: string; validator: string; }; -const config_option_schema = z.object({ +const url_option_schema = z.object({ key: z.string(), label: z.string(), validator: z.string(), }); -const config_options_schema = z.array(config_option_schema); +const url_options_schema = z.array(url_option_schema); export function show_generate_integration_url_modal(api_key: string): void { const default_url_message = $t_html({defaultMessage: "Integration URL will appear here."}); @@ -107,8 +107,8 @@ export function show_generate_integration_url_modal(api_key: string): void { update_url(); } - function render_config(config: ConfigOption[]): void { - const validated_config = config_options_schema.parse(config); + function render_url_options(config: UrlOption[]): void { + const validated_config = url_options_schema.parse(config); $config_container.empty(); for (const option of validated_config) { @@ -203,7 +203,7 @@ export function show_generate_integration_url_modal(api_key: string): void { (bot) => bot.name === selected_integration, ); const all_event_types = selected_integration_data?.all_event_types; - const config = selected_integration_data?.config_options; + const url_options = selected_integration_data?.url_options; if (all_event_types !== null) { $("#integration-events-parameter").removeClass("hide"); @@ -234,8 +234,8 @@ export function show_generate_integration_url_modal(api_key: string): void { const selected_events = set_events_param(params); - if (config) { - for (const option of config) { + if (url_options) { + for (const option of url_options) { let $input_element; if ( option.key === "branches" && @@ -316,8 +316,8 @@ export function show_generate_integration_url_modal(api_key: string): void { (bot) => bot.name === selected_integration, ); - if (selected_integration_data?.config_options) { - render_config(selected_integration_data.config_options); + if (selected_integration_data?.url_options) { + render_url_options(selected_integration_data.url_options); } dropdown.hide(); diff --git a/web/src/state_data.ts b/web/src/state_data.ts index db799dc37e..6821ec5bce 100644 --- a/web/src/state_data.ts +++ b/web/src/state_data.ts @@ -386,6 +386,15 @@ export const realm_schema = z.object({ }), ) .optional(), + url_options: z + .array( + z.object({ + key: z.string(), + label: z.string(), + validator: z.string(), + }), + ) + .optional(), }), ), realm_inline_image_preview: z.boolean(), diff --git a/zerver/lib/events.py b/zerver/lib/events.py index db8a24b753..c9dd3bd391 100644 --- a/zerver/lib/events.py +++ b/zerver/lib/events.py @@ -750,6 +750,16 @@ def fetch_initial_state_data( ] if integration.config_options else [], + "url_options": [ + { + "key": c.name, + "label": c.label, + "validator": c.validator.__name__, + } + for c in integration.url_options + ] + if integration.url_options + else [], } for integration in WEBHOOK_INTEGRATIONS if integration.legacy is False diff --git a/zerver/lib/integrations.py b/zerver/lib/integrations.py index 2a1d489100..5e2a6bb3c8 100644 --- a/zerver/lib/integrations.py +++ b/zerver/lib/integrations.py @@ -13,7 +13,7 @@ from django_stubs_ext import StrPromise from zerver.lib.storage import static_path from zerver.lib.validator import check_bool, check_string -from zerver.lib.webhooks.common import WebhookConfigOption +from zerver.lib.webhooks.common import WebhookConfigOption, WebhookUrlOption """This module declares all of the (documented) integrations available in the Zulip server. The Integration class is used as part of @@ -79,12 +79,14 @@ class Integration: stream_name: str | None = None, legacy: bool = False, config_options: Sequence[WebhookConfigOption] = [], + url_options: Sequence[WebhookUrlOption] = [], ) -> None: self.name = name self.client_name = client_name if client_name is not None else name self.secondary_line_text = secondary_line_text self.legacy = legacy self.doc = doc + self.url_options = url_options # Note: Currently only incoming webhook type bots use this list for # defining how the bot's BotConfigData should be. Embedded bots follow @@ -247,6 +249,7 @@ class WebhookIntegration(Integration): stream_name: str | None = None, legacy: bool = False, config_options: Sequence[WebhookConfigOption] = [], + url_options: Sequence[WebhookUrlOption] = [], dir_name: str | None = None, ) -> None: if client_name is None: @@ -261,6 +264,7 @@ class WebhookIntegration(Integration): stream_name=stream_name, legacy=legacy, config_options=config_options, + url_options=url_options, ) if function is None: @@ -410,9 +414,7 @@ WEBHOOK_INTEGRATIONS: list[WebhookIntegration] = [ "azuredevops", ["version-control"], display_name="AzureDevOps", - config_options=[ - WebhookConfigOption(name="branches", description="", validator=check_string) - ], + url_options=[WebhookUrlOption(name="branches", label="", validator=check_string)], ), WebhookIntegration("beanstalk", ["version-control"], stream_name="commits"), WebhookIntegration("basecamp", ["project-management"]), @@ -423,9 +425,7 @@ WEBHOOK_INTEGRATIONS: list[WebhookIntegration] = [ logo="images/integrations/logos/bitbucket.svg", display_name="Bitbucket Server", stream_name="bitbucket", - config_options=[ - WebhookConfigOption(name="branches", description="", validator=check_string) - ], + url_options=[WebhookUrlOption(name="branches", label="", validator=check_string)], ), WebhookIntegration( "bitbucket2", @@ -433,9 +433,7 @@ WEBHOOK_INTEGRATIONS: list[WebhookIntegration] = [ logo="images/integrations/logos/bitbucket.svg", display_name="Bitbucket", stream_name="bitbucket", - config_options=[ - WebhookConfigOption(name="branches", description="", validator=check_string) - ], + url_options=[WebhookUrlOption(name="branches", label="", validator=check_string)], ), WebhookIntegration( "bitbucket", @@ -464,9 +462,7 @@ WEBHOOK_INTEGRATIONS: list[WebhookIntegration] = [ "gitea", ["version-control"], stream_name="commits", - config_options=[ - WebhookConfigOption(name="branches", description="", validator=check_string) - ], + url_options=[WebhookUrlOption(name="branches", label="", validator=check_string)], ), WebhookIntegration( "github", @@ -474,11 +470,11 @@ WEBHOOK_INTEGRATIONS: list[WebhookIntegration] = [ display_name="GitHub", function="zerver.webhooks.github.view.api_github_webhook", stream_name="github", - config_options=[ - WebhookConfigOption(name="branches", description="", validator=check_string), - WebhookConfigOption( + url_options=[ + WebhookUrlOption(name="branches", label="", validator=check_string), + WebhookUrlOption( name="ignore_private_repositories", - description="Exclude notifications from private repositories", + label="Exclude notifications from private repositories", validator=check_bool, ), ], @@ -497,18 +493,14 @@ WEBHOOK_INTEGRATIONS: list[WebhookIntegration] = [ "gitlab", ["version-control"], display_name="GitLab", - config_options=[ - WebhookConfigOption(name="branches", description="", validator=check_string) - ], + url_options=[WebhookUrlOption(name="branches", label="", validator=check_string)], ), WebhookIntegration("gocd", ["continuous-integration"], display_name="GoCD"), WebhookIntegration( "gogs", ["version-control"], stream_name="commits", - config_options=[ - WebhookConfigOption(name="branches", description="", validator=check_string) - ], + url_options=[WebhookUrlOption(name="branches", label="", validator=check_string)], ), WebhookIntegration("gosquared", ["marketing"], display_name="GoSquared"), WebhookIntegration("grafana", ["monitoring"]), @@ -543,10 +535,10 @@ WEBHOOK_INTEGRATIONS: list[WebhookIntegration] = [ WebhookIntegration( "opsgenie", ["meta-integration", "monitoring"], - config_options=[ - WebhookConfigOption( + url_options=[ + WebhookUrlOption( name="eu_region", - description="Use Opsgenie's European service region", + label="Use Opsgenie's European service region", validator=check_bool, ) ], @@ -563,9 +555,7 @@ WEBHOOK_INTEGRATIONS: list[WebhookIntegration] = [ "rhodecode", ["version-control"], display_name="RhodeCode", - config_options=[ - WebhookConfigOption(name="branches", description="", validator=check_string) - ], + url_options=[WebhookUrlOption(name="branches", label="", validator=check_string)], ), WebhookIntegration("rundeck", ["deployment"]), WebhookIntegration("semaphore", ["continuous-integration", "deployment"]), diff --git a/zerver/lib/webhooks/common.py b/zerver/lib/webhooks/common.py index 5282d13129..5b3a9d99d2 100644 --- a/zerver/lib/webhooks/common.py +++ b/zerver/lib/webhooks/common.py @@ -61,6 +61,13 @@ class WebhookConfigOption: validator: Callable[[str, str], str | bool | None] +@dataclass +class WebhookUrlOption: + name: str + label: str + validator: Callable[[str, str], str | bool | None] + + def get_setup_webhook_message(integration: str, user_name: str | None = None) -> str: content = SETUP_MESSAGE_TEMPLATE.format(integration=integration) if user_name: diff --git a/zerver/openapi/zulip.yaml b/zerver/openapi/zulip.yaml index c1ffbf1550..f709e5f477 100644 --- a/zerver/openapi/zulip.yaml +++ b/zerver/openapi/zulip.yaml @@ -16351,6 +16351,8 @@ paths: **Changes**: New in Zulip 8.0 (feature level 207). config_options: $ref: "#/components/schemas/WebhookConfigOption" + url_options: + $ref: "#/components/schemas/WebhookUrlOption" recent_private_conversations: description: | Present if `recent_private_conversations` is present in `fetch_event_types`. @@ -24709,21 +24711,20 @@ components: WebhookConfigOption: type: array description: | - An array of configuration options where each option is an - object containing a unique identifier, a human-readable name, - and a validation function name hinting how to verify the - correct input format. + An array of configuration options that can be set when creating + a bot user for this incoming webhook integration. - This is an unstable API expected to be used only by the Zulip web - apps. Please discuss in chat.zulip.org before using it. + This is an unstable API. Please discuss in chat.zulip.org before + using it. - **Changes**: In Zulip 10.0 (feature level 318), changed `config` - to `config_options`. The `config_options` field defines which options - should be offered when creating URLs for this integration. Previously, - the `config` field was a key-value pair describing integration-specific - configuration data that needs to be included when creating an incoming - webhook bot. The feature was never operational because there is currently - no way to associate an incoming webhook bot with an integration. + **Changes**: As of Zulip 11.0 (feature level ZF-f9d19d), this + object is reserved for integration-specific configuration options + that can be set when creating a bot user. Previously, this object + also included optional webhook URL parameters, which are now + specified in the `url_options` object. + + Before Zulip 10.0 (feature level 318), this field was named `config`, + and was reserved for configuration data key-value pairs. items: type: object additionalProperties: false @@ -24731,7 +24732,7 @@ components: key: type: string description: | - A key for the configuration option to use in generated URLs. + A key for the configuration option. label: type: string description: | @@ -24740,8 +24741,38 @@ components: type: string description: | The name of the validator function for the configuration - option. Currently generated values are `check_bool` and - `check_string`. + option. + WebhookUrlOption: + type: array + description: | + An array of optional URL parameter options for the incoming webhook + integration. In the web app, these are used when + [generating a URL for an integration](/help/generate-integration-url). + + This is an unstable API expected to be used only by the Zulip web + app. Please discuss in chat.zulip.org before using it. + + **Changes**: New in Zulip 11.0 (feature level ZF-f9d19d). Previously, + these optional URL parameter options were included in the + `config_options` object. + items: + type: object + additionalProperties: false + properties: + key: + type: string + description: | + The parameter variable to encode the users input for this + option in the integrations webhook URL. + label: + type: string + description: | + A human-readable label of the url option. + validator: + type: string + description: | + The name of the validator function for the configuration + option. CustomProfileField: type: object additionalProperties: false