Commit Graph

259 Commits

Author SHA1 Message Date
orientor
c7a2c48bb7 openapi: Add tests for checking presence of various attributes.
The swagger validator is a basic tool to check whether our
openapi specification file follows the basic syntax. But to ensure
that our zulip.yaml file is not only syntactically compatible but
also describes our API well, we need to add custom tests. This
commit currently checks whether each endpoint has an `operationId`
and a valid tag. It also makes it easier to check for custom rules
in the future.
2020-05-20 13:10:58 -07:00
Rohitt Vashishtha
1e4ad0bdfc js-api: Migrate and test send-message example. 2020-05-20 10:18:29 -07:00
Rohitt Vashishtha
203b722624 js-api: Migrate and test create-user example. 2020-05-20 10:18:29 -07:00
Kartik Srivastava
5f3849b517 openapi/markdown_extension: Avoid shelling out to node for JS snippets.
This refactors render_javascript_code_example to avoid shelling out to
node and parse the javascript file with python instead, to get example
code snippets.
2020-05-20 10:18:29 -07:00
Rohitt Vashishtha
6dd4030e67 js-api: Add hooks to run tests and render docs of JS API examples.
This commit adds python code to call javascript_examples.js in its
two supported modes. tools/test-api asserts that the example output
is as expected, whereas the API markdown extension is used to render
these examples in the docs.
2020-05-20 10:18:29 -07:00
Kartik Srivastava
642d1a20d0 openapi/markdown_extension: Refactor extract_code_example.
This refactors `extract_code_example` to return a nested list
of code snippets between '{code_example|start/end}' instead of
returing a list of all the lines between '{code_example|start/end}'
markers in the code examples.

Appropriate changes have been made to render_python_code_example.
2020-05-20 10:18:29 -07:00
Kartik Srivastava
302906211d js-api: Refactor ExamplesHandler to avoid running examples in a loop.
This refactors `ExamplesHandler` to avoid running examples in a loop
and add result objects to `response_data` array one by one with
`generate_validation_data`.
2020-05-20 10:18:29 -07:00
Rohitt Vashishtha
31d04fb370 js-api: Add module to create and run JS API examples.
This file will act as the container for all JS API examples to use
in our documentation, similar to our python and curl API testing
and examples generation code.

This module has two modes of operation:

- node javascript_examples.js generate-responses

   This mode runs all the examples against a server and prints the JSON
   output of all the examples we ran.

- node javascript_examples.js generate-example <endpoint>

   This mode prints example code for endpoints like: /users:post. We then
   want to render this full example code in our docs.
2020-05-20 10:18:29 -07:00
orientor
915d801327 openapi: Add 'tags' attribute to endpoints.
'tags' attribute is helpful in differentiating and grouping the
endpoints on basis of their usage. For example tags like 'messages'
help in grouping all endpoints related to messages and thus make the
api specification more user-friendly. So give tags to the endpoints
on the basis of what heading they are under in the API docs.
2020-05-19 23:17:31 -07:00
orientor
3467d2fe68 openapi: Add 'operationId' attribute to endpoints.
'operationId' helps code generators in naming functions and other purposes.
So name operationId of endpoints as their function names in python-zulip-api
if it exists else use most appropriate function name.

Part of #14100 .
2020-05-19 23:13:39 -07:00
Kartik Srivastava
59ac3ca95f openapi/python_examples: Add reactivate_user example. 2020-05-19 23:10:50 -07:00
Kartik Srivastava
4af7ad70e7 openapi: Add OpenAPI data for /users/{user_id}/reactivate. 2020-05-19 23:10:50 -07:00
sahil839
9b78a73e36 populate_db: Add new admin user as 'Desdemona'.
This commit adds a second admin user named 'Desdemona' to dev and
test database.
2020-05-19 11:42:27 -07:00
Kartik Srivastava
112f539034 openapi/python_examples: Update update_user example.
This updates the `update_user` example to use `update_user_by_id`.
2020-05-16 14:45:47 -07:00
Kartik Srivastava
d2ab5673fd openapi/python_examples: Update deactivate_user example.
This updates the `deactivate_user` example to use
`deactivate_user_by_id`.
2020-05-16 14:45:47 -07:00
orientor
9170931da3 openapi: Add test for validating examples.
Zulip's openapi specification in zulip.yaml has various examples
for various schemas. Validate the example with their respective
schemas to ensure that all the examples are schematically correct.

Part of #14100.
2020-05-12 23:03:06 -07:00
orientor
fab2ec9e63 openapi: Rectify wrong examples and schemas.
Some examples mentioned in zulip.yaml did not match their schema.
Change either the schema or the example so that all examples are
valid with respect to their schemas.
2020-05-12 23:03:06 -07:00
orientor
3ffc9466c9 openapi: Combine two similar openapi markdown extensions.
Previously api_description and api_code_examples were two independent
markdown extensions for displaying OpenAPI content used in the same
places.  We combine them into a single markdown extension (with two
processors) and move them to the openapi folder to make the codebase
more readable and better group the openapi code in the same place.
2020-05-05 21:42:31 -07:00
Tim Abbott
5ea942caff api docs: Improve a few common OpenAPI descriptions.
This fixes a few regressions of documentation introduced earlier in
this branch.
2020-05-05 17:11:14 -07:00
shubhamgupta2956
57c38f1c79 api_docs: Add "EmojiCode" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "emoji_code" in components.
2020-05-05 17:02:56 -07:00
shubhamgupta2956
08314e3bf4 api_docs: Add "ReactionType" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "reaction_type" in components.
2020-05-05 17:02:56 -07:00
shubhamgupta2956
db3d95fd16 api_docs: Add "Principals" component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "principals" in components.
2020-05-05 17:02:56 -07:00
shubhamgupta2956
c4ebe98bba api_docs: Add IncludeCustomProfileFields component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "include_custom_profile_fields" in
components.
2020-05-05 17:02:53 -07:00
shubhamgupta2956
c28ccacb73 api_docs: Add "EmojiName" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "emoji_name" in components.
2020-05-05 17:02:53 -07:00
shubhamgupta2956
64ef886eb9 api_docs: Add "IncludeSubscribers" component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "IncludeSubscribers" in components.
2020-05-05 17:02:53 -07:00
shubhamgupta2956
597269c43e api_docs: Add HistoryPublicToSubscribers component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "history_public_to_subscribers" in
components.
2020-05-05 17:02:51 -07:00
shubhamgupta2956
c5bef400bb api_docs: Add "StreamPostPolicy" component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "stream_post_policy" in components.
2020-05-05 16:48:33 -07:00
shubhamgupta2956
b0c7daf7e5 api_docs: Add "UserId" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "user_id" in components.
2020-05-05 16:48:33 -07:00
Hashir Sarwar
55a8e7dff2 settings: Offer hiding presence info from other users.
For privacy-minded folks who don't want to leak the
information of whether they're online, this adds an
option to disable sending presence updates to other
users.

The new settings lies in the "Other notification
settings" section of the "Notification settings"
page, under a "Presence" subheading.

Closes #14798.
2020-05-02 15:09:05 -07:00
Steve Howell
2c63130195 api: Always return zulip_version/zulip_feature_level.
We no longer make these conditional to simplify writing clients;
there's no cost to including them, and a real cost to not doing so.
2020-04-29 11:58:22 -07:00
Tim Abbott
e8c57f2466 api docs: Document API level 2 change in custom emoji. 2020-04-28 22:23:25 -07:00
Tim Abbott
cbe0723199 api docs: Add an API changelog page.
I imagine this can be improved in various ways, but I've initialized
this with all the **Changes** entries recorded in either zulip.yaml or
the rest of the API documentation, and I expect we'll be able to
iterate on this effectively.

It'll also be useful as a record of changes that we should remember to
document the API documentation as we document more endpoints that
currently don't discuss these issues.

While working on this, I fixed various issues where feature levels
could be mentioned or endpoints didn't properly document changes.
2020-04-28 22:23:25 -07:00
Tim Abbott
1be4cc5f17 openapi: Fix servers section to work in online editor. 2020-04-28 14:27:43 -07:00
orientor
4d8c988ef2 openapi: Use description markdown for rendering endpoint descriptions.
Firstly, change endpoint descriptions in zulip.yaml so that they
match their counterpart in the api docs. Then edit the api docs
so that they use api description markdown extension for displaying
endpoint description.
2020-04-28 12:57:19 -07:00
orientor
64c6bab276 openapi: Create markdown extension for rendering endpoint descriptions.
Add function in openapi.py to access endpoint descriptions written
in zulip.yaml. Use this function for creating a markdown extension
for rendering endpoint descriptions written in zulip.yaml.

We use this extension for a single endpoint to get test coverage.
2020-04-28 12:57:19 -07:00
Eeshan Garg
75b2264a3f pypi: Upgrade zulip/zulip-bots dependencies to version 0.7.0.
Includes this change:
* openapi/python_examples: Update get_single_user.

This updates get_single_user to pass keyword arguments to
get_user_by_id instead of passing a dictionary.

Which is required for CI to pass, as we indeed fixed the API of that
function (which had only been present with the wrong API for one release).
2020-04-23 17:41:47 -07:00
Anders Kaseorg
fead14951c python: Convert assignment type annotations to Python 3.6 style.
This commit was split by tabbott; this piece covers the vast majority
of files in Zulip, but excludes scripts/, tools/, and puppet/ to help
ensure we at least show the right error messages for Xenial systems.

We can likely further refine the remaining pieces with some testing.

Generated by com2ann, with whitespace fixes and various manual fixes
for runtime issues:

-    invoiced_through: Optional[LicenseLedger] = models.ForeignKey(
+    invoiced_through: Optional["LicenseLedger"] = models.ForeignKey(

-_apns_client: Optional[APNsClient] = None
+_apns_client: Optional["APNsClient"] = None

-    notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
-    signup_notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    signup_notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)

-    author: Optional[UserProfile] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)
+    author: Optional["UserProfile"] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)

-    bot_owner: Optional[UserProfile] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)
+    bot_owner: Optional["UserProfile"] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)

-    default_sending_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
-    default_events_register_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_sending_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_events_register_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)

-descriptors_by_handler_id: Dict[int, ClientDescriptor] = {}
+descriptors_by_handler_id: Dict[int, "ClientDescriptor"] = {}

-worker_classes: Dict[str, Type[QueueProcessingWorker]] = {}
-queues: Dict[str, Dict[str, Type[QueueProcessingWorker]]] = {}
+worker_classes: Dict[str, Type["QueueProcessingWorker"]] = {}
+queues: Dict[str, Dict[str, Type["QueueProcessingWorker"]]] = {}

-AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional[LDAPSearch] = None
+AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional["LDAPSearch"] = None

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 11:02:32 -07:00
Hashir Sarwar
e3b90a5ec8 api: Add a monotonic integer "feature level" for non-webapp clients.
The purpose is to provide a way for (non-webapp) clients,
like the mobile and terminal apps, to tell whether the
server it's talking to is new enough to support a given
API feature -- in particular a way that

* is finer-grained than release numbers, so that for
features developed after e.g. 2.1.0 we can use them
immediately on servers deployed from master (like
chat.zulip.org and zulipchat.com) without waiting the
months until a 2.2 release;

* is reliable, unlike e.g. looking at the number of
commits since a release;

* doesn't lead to a growing bag of named feature flags
which the server has to go on sending forever.

Tweaked by tabbott to extend the documentation.

Closes #14618.
2020-04-21 13:37:57 -07:00
Steve Howell
105c168b19 test-api: Avoid flakes with curl tests. 2020-04-21 11:30:29 -04:00
shubhamgupta2956
7c3169a1d9 api_docs: Add "MessageId" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "message_id" in components.
2020-04-20 17:52:49 -07:00
shubhamgupta2956
05bb1e7075 api_docs: Add "Content" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "content" in components.
2020-04-20 17:52:49 -07:00
shubhamgupta2956
0425f53e88 api_docs: Add "ClientGravatar" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "client_gravatar" in components.
2020-04-20 17:52:49 -07:00
shubhamgupta2956
c814afd4a7 api_docs: Add "StreamIdInPath" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "stream_id" (in path) in components.
2020-04-20 17:52:48 -07:00
shubhamgupta2956
942fd39065 api_docs: Add "StreamIdInQuery" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "stream_id" (in query) in components.
2020-04-20 17:52:30 -07:00
shubhamgupta2956
d5f96c3c12 api_docs: Add "Stream" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "stream" in components.
2020-04-20 17:49:53 -07:00
shubhamgupta2956
22c273eacd api_docs: Add "QueueId" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "queue_id" in components.
2020-04-20 17:49:53 -07:00
shubhamgupta2956
86eed5d9bb api_docs: Add "Topic" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "topic" in components.
2020-04-20 17:49:52 -07:00
shubhamgupta2956
05b1ddeb84 api_docs: Add "GroupId" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "group_id" in components.
2020-04-20 17:47:13 -07:00
shubhamgupta2956
8e617d7f26 api_docs: Add "Narrow" common component.
To facilitate re-use of the same parameters in other paths, this commit
store the content of the parameter "narrow" in components.
2020-04-20 17:47:13 -07:00
Anders Kaseorg
5901e7ba7e python: Convert function type annotations to Python 3 style.
Generated by com2ann (slightly patched to avoid also converting
assignment type annotations, which require Python 3.6), followed by
some manual whitespace adjustment, and six fixes for runtime issues:

-    def __init__(self, token: Token, parent: Optional[Node]) -> None:
+    def __init__(self, token: Token, parent: "Optional[Node]") -> None:

-def main(options: argparse.Namespace) -> NoReturn:
+def main(options: argparse.Namespace) -> "NoReturn":

-def fetch_request(url: str, callback: Any, **kwargs: Any) -> Generator[Callable[..., Any], Any, None]:
+def fetch_request(url: str, callback: Any, **kwargs: Any) -> "Generator[Callable[..., Any], Any, None]":

-def assert_server_running(server: subprocess.Popen[bytes], log_file: Optional[str]) -> None:
+def assert_server_running(server: "subprocess.Popen[bytes]", log_file: Optional[str]) -> None:

-def server_is_up(server: subprocess.Popen[bytes], log_file: Optional[str]) -> bool:
+def server_is_up(server: "subprocess.Popen[bytes]", log_file: Optional[str]) -> bool:

-    method_kwarg_pairs: List[FuncKwargPair],
+    method_kwarg_pairs: "List[FuncKwargPair]",

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-18 20:42:48 -07:00