paulmataruso/zulip
1
0
Fork 0
mirror of https://github.com/zulip/zulip.git synced 2025-10-24 08:33:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

api: Add support for passing partial to include_subscribers parameter.

Browse Source 
Fixes #35318.
This commit is contained in:
Vector73
2025-08-04 07:06:43 +00:00
committed by Tim Abbott
parent 40b1f6eb4e
commit 0ac24bd437
8 changed files with 209 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
api_docs/changelog.md
View File

@@ -20,6 +20,16 @@ format used by the Zulip server that they are interacting with.
## Changes in Zulip 11.0
**Feature level 412**
* [`POST /register`](/api/register-queue),
[`GET /users/me/subscriptions`](/api/get-subscriptions):
Added support for passing `partial` as argument to `include_subscribers`
parameter to get only partial subscribers data of the channel.
* [`POST /register`](/api/register-queue),
[`GET /users/me/subscriptions`](/api/get-subscriptions):
Added `partial_subscribers` field in `subscription` objects.
**Feature level 411**
* [`POST /register`](/api/register-queue), [`PATCH /settings`](/api/update-settings),

2
version.py
View File

@@ -34,7 +34,7 @@ DESKTOP_WARNING_VERSION = "5.9.3"
# new level means in api_docs/changelog.md, as well as "**Changes**"
# entries in the endpoint's documentation in `zulip.yaml`.
API_FEATURE_LEVEL = 411
API_FEATURE_LEVEL = 412
# Bump the minor PROVISION_VERSION to indicate that folks should provision
# only when going from an old version of the code to a newer version. Bump

2
zerver/lib/subscription_info.py
View File

@@ -974,7 +974,7 @@ def gather_subscriptions_helper(
def gather_subscriptions(
user_profile: UserProfile,
include_subscribers: bool = False,
include_subscribers: bool | Literal["partial"] = False,
) -> tuple[list[SubscriptionStreamDict], list[SubscriptionStreamDict]]:
helper_result = gather_subscriptions_helper(
user_profile,

115
zerver/openapi/zulip.yaml
View File

@@ -15902,20 +15902,43 @@ paths:
Whether each returned channel object should include a `subscribers`
field containing a list of the user IDs of its subscribers.
(This may be significantly slower in organizations with
thousands of users subscribed to many channels.)
Client apps supporting organizations with many thousands of users
should not pass `true`, because the full subscriber matrix may be
several megabytes of data. The `partial` value, combined with the
`subscriber_count` and fetching subscribers for individual channels as
needed, is recommended to support client app features where channel
subscriber data is useful.
If a client passes `partial` for this parameter, the server may,
for some channels, return a subset of the channel's subscribers
in the `partial_subscribers` field instead of the `subscribers` field,
which always contains the complete set of subscribers.
The server guarantees that it will always return a `subscribers`
field for channels with fewer than 250 total subscribers. When
returning a `partial_subscribers` field, the server guarantees
that all bot users and users active within the last 14 days will
be included. For other cases, the server may use its discretion
to determine which channels and users to include, balancing between
payload size and usefulness of the data provided to the client.
Passing `true` in an [unauthenticated
request](/help/public-access-option) is an error.
**Changes**: Before Zulip 6.0 (feature level 149), this
parameter was silently ignored and processed as though it
were `false` in unauthenticated requests.
**Changes**: The `partial` value is new in Zulip 11.0 (feature level 412).
Before Zulip 6.0 (feature level 149), this parameter was silently
ignored and processed as though it were `false` in unauthenticated
requests.
New in Zulip 2.1.0.
type: boolean
default: false
example: true
type: string
enum:
- "true"
- "false"
- "partial"
default: "false"
example: "true"
slim_presence:
description: |
If `true`, the `presences` object returned in the response will be keyed
@@ -16069,8 +16092,6 @@ paths:
contentType: application/json
client_gravatar:
contentType: application/json
include_subscribers:
contentType: application/json
slim_presence:
contentType: application/json
event_types:
@@ -16897,6 +16918,28 @@ paths:
a channel, we will send an empty array. API authors
should use other data to determine whether users like
guest users are forbidden to know the subscribers.
partial_subscribers:
type: array
items:
type: integer
description: |
If [`include_subscribers="partial"`](/api/get-subscriptions#parameter-include_subscribers)
was requested, the server may, at its discretion, send a
`partial_subscribers` list rather than a `subscribers` list
for channels with a large number of subscribers.
The `partial_subscribers` list contains an arbitrary
subset of the channel's subscribers that is guaranteed
to include all bot user subscribers as well as all
users who have been active in the last 14 days, but
otherwise can be chosen arbitrarily by the server.
If a user is not allowed to know the subscribers for
a channel, we will send an empty array. API authors
should use other data to determine whether users like
guest users are forbidden to know the subscribers.
**Changes**: New in Zulip 11.0 (feature level 412).
description: |
Present if `subscription` is present in `fetch_event_types`.
@@ -26064,6 +26107,23 @@ components:
description: |
A list of user IDs of users who are also subscribed
to a given channel. Included only if `include_subscribers` is `true`.
partial_subscribers:
type: array
items:
type: integer
description: |
If [`include_subscribers="partial"`](/api/get-subscriptions#parameter-include_subscribers)
was requested, the server may, at its discretion, send a
`partial_subscribers` list rather than a `subscribers` list
for channels with a large number of subscribers.
The `partial_subscribers` list contains an arbitrary
subset of the channel's subscribers that is guaranteed
to include all bot user subscribers as well as all
users who have been active in the last 14 days, but
otherwise can be chosen arbitrarily by the server.
**Changes**: New in Zulip 11.0 (feature level 412).
desktop_notifications:
type: boolean
nullable: true
@@ -28561,14 +28621,37 @@ components:
Whether each returned channel object should include a `subscribers`
field containing a list of the user IDs of its subscribers.
(This may be significantly slower in organizations with
thousands of users subscribed to many channels.)
Client apps supporting organizations with many thousands of users
should not pass `true`, because the full subscriber matrix may be
several megabytes of data. The `partial` value, combined with the
`subscriber_count` and fetching subscribers for individual channels as
needed, is recommended to support client app features where
channel subscriber data is useful.
**Changes**: New in Zulip 2.1.0.
If a client passes `partial` for this parameter, the server may,
for some channels, return a subset of the channel's subscribers
in the `partial_subscribers` field instead of the `subscribers` field,
which always contains the complete set of subscribers.
The server guarantees that it will always return a `subscribers`
field for channels with fewer than 250 total subscribers. When
returning a `partial_subscribers` field, the server guarantees
that all bot users and users active within the last 14 days will
be included. For other cases, the server may use its discretion
to determine which channels and users to include, balancing between
payload size and usefulness of the data provided to the client.
**Changes**: The `partial` value is new in Zulip 11.0 (feature level 412).
New in Zulip 2.1.0.
schema:
type: boolean
default: false
example: true
type: string
enum:
- "true"
- "false"
- "partial"
default: "false"
example: "true"
IncludeCustomProfileFields:
name: include_custom_profile_fields
in: query

56
zerver/tests/test_channel_fetch.py
View File

@@ -763,6 +763,62 @@ class GetSubscribersTest(ZulipTestCase):
self.assertIsNone(sub.get("partial_subscribers"))
break
for sub in subscribed_streams:
# fewer than MIN_PARTIAL_SUBSCRIBERS_CHANNEL_SIZE subscribers,
# so we get all of them
if sub["name"] == "subscribed_more_than_bots_including_idle":
self.assertNotIn("partial_subscribers", sub)
self.assert_length(sub["subscribers"], 4)
if sub["name"] == "subscribed_many_more_than_bots":
# the bot, Othello (who is not long_term_idle), and current user
self.assert_length(sub["partial_subscribers"], 3)
self.assertNotIn("subscribers", sub)
@override_settings(MIN_PARTIAL_SUBSCRIBERS_CHANNEL_SIZE=5)
def test_gather_partial_subscriptions_api(self) -> None:
othello = self.example_user("othello")
idle_users = [
create_user(
email=f"original_user{i}@zulip.com",
password=None,
realm=othello.realm,
full_name=f"Full Name {i}",
)
for i in range(5)
]
for user in idle_users:
user.long_term_idle = True
user.save()
bot = self.create_test_bot("bot", othello, "Foo Bot")
stream_names = [
"subscribed_more_than_bots_including_idle",
"subscribed_many_more_than_bots",
]
for stream_name in stream_names:
self.make_stream(stream_name)
for user in [bot, othello, self.user_profile, idle_users[0]]:
self.subscribe(user, stream_names[0])
for user in [bot, othello, self.user_profile, *idle_users]:
self.subscribe(user, stream_names[1])
with self.assert_database_query_count(11):
result = self.api_get(
self.user_profile,
"/api/v1/users/me/subscriptions",
{"include_subscribers": "partial"},
)
sub_data = self.assert_json_success(result)
subscribed_streams = sub_data["subscriptions"]
self.assertGreaterEqual(len(subscribed_streams), 2)
# Streams with only bots have sent all of their subscribers,
# since we always send bots. We tell the client it doesn't
# need to fetch more, by filling "subscribers" instead
# of "partial_subscribers". If there are non-bot subscribers,
# a partial fetch will return only partial subscribers.
for sub in subscribed_streams:
# fewer than MIN_PARTIAL_SUBSCRIBERS_CHANNEL_SIZE subscribers,
# so we get all of them

22
zerver/tests/test_subs.py
View File

@@ -3261,6 +3261,27 @@ class SubscriptionAPITest(ZulipTestCase):
self.assertIsInstance(stream["name"], str)
self.assertIsInstance(stream["color"], str)
self.assertIsInstance(stream["invite_only"], bool)
self.assertNotIn("partial_subscribers", stream)
self.assertNotIn("subscribers", stream)
# check that the stream name corresponds to an actual
# stream; will throw Stream.DoesNotExist if it doesn't
get_stream(stream["name"], self.test_realm)
list_streams = [stream["name"] for stream in json["subscriptions"]]
# also check that this matches the list of your subscriptions
self.assertEqual(sorted(list_streams), sorted(self.streams))
# Text explicitly passing `include_subscribers` as "false"
result = self.api_get(
self.test_user, "/api/v1/users/me/subscriptions", {"include_subscribers": "false"}
)
json = self.assert_json_success(result)
self.assertIn("subscriptions", json)
for stream in json["subscriptions"]:
self.assertIsInstance(stream["name"], str)
self.assertIsInstance(stream["color"], str)
self.assertIsInstance(stream["invite_only"], bool)
self.assertNotIn("partial_subscribers", stream)
self.assertNotIn("subscribers", stream)
# check that the stream name corresponds to an actual
# stream; will throw Stream.DoesNotExist if it doesn't
get_stream(stream["name"], self.test_realm)
@@ -3283,6 +3304,7 @@ class SubscriptionAPITest(ZulipTestCase):
self.assertIsInstance(stream["name"], str)
self.assertIsInstance(stream["color"], str)
self.assertIsInstance(stream["invite_only"], bool)
self.assertIn("subscribers", stream)
# check that the stream name corresponds to an actual
# stream; will throw Stream.DoesNotExist if it doesn't
get_stream(stream["name"], self.test_realm)

11
zerver/views/events_register.py
View File

@@ -1,4 +1,4 @@
from typing import Annotated, TypeAlias
from typing import Annotated, Literal, TypeAlias
from annotated_types import Len
from django.conf import settings
@@ -16,6 +16,7 @@ from zerver.lib.request import RequestNotes
from zerver.lib.response import json_success
from zerver.lib.typed_endpoint import ApiParamConfig, DocumentationStatus, typed_endpoint
from zerver.models import Stream, UserProfile
from zerver.views.streams import parse_include_subscribers
def _default_all_public_streams(user_profile: UserProfile, all_public_streams: bool | None) -> bool:
@@ -46,7 +47,7 @@ def events_register_backend(
client_gravatar_raw: Annotated[Json[bool | None], ApiParamConfig("client_gravatar")] = None,
event_types: Json[list[str]] | None = None,
fetch_event_types: Json[list[str]] | None = None,
include_subscribers: Json[bool] = False,
include_subscribers: Literal["true", "false", "partial"] = "false",
narrow: Json[NarrowT] | None = None,
presence_history_limit_days: Json[int] | None = None,
queue_lifespan_secs: Annotated[
@@ -61,6 +62,8 @@ def events_register_backend(
else:
client_gravatar = client_gravatar_raw
parsed_include_subscribers = parse_include_subscribers(include_subscribers)
if maybe_user_profile.is_authenticated:
user_profile = maybe_user_profile
spectator_requested_language = None
@@ -84,7 +87,7 @@ def events_register_backend(
raise JsonableError(
_("Invalid '{key}' parameter for anonymous request").format(key="client_gravatar")
)
if include_subscribers:
if parsed_include_subscribers:
raise JsonableError(
_("Invalid '{key}' parameter for anonymous request").format(
key="include_subscribers"
@@ -123,7 +126,7 @@ def events_register_backend(
queue_lifespan_secs,
all_public_streams,
narrow=modern_narrow,
include_subscribers=include_subscribers,
include_subscribers=parsed_include_subscribers,
include_streams=include_streams,
client_capabilities=client_capabilities,
fetch_event_types=fetch_event_types,

16
zerver/views/streams.py
View File

@@ -1,7 +1,7 @@
import time
from collections import defaultdict
from collections.abc import Callable
from typing import Annotated, Any
from typing import Annotated, Any, Literal
import orjson
from django.conf import settings
@@ -518,16 +518,26 @@ def update_stream_backend(
return json_success(request)
def parse_include_subscribers(
include_subscribers: Literal["true", "false", "partial"],
) -> bool | Literal["partial"]:
if include_subscribers == "true":
return True
if include_subscribers == "false":
return False
return include_subscribers
@typed_endpoint
def list_subscriptions_backend(
request: HttpRequest,
user_profile: UserProfile,
*,
include_subscribers: Json[bool] = False,
include_subscribers: Literal["true", "false", "partial"] = "false",
) -> HttpResponse:
subscribed, _ = gather_subscriptions(
user_profile,
include_subscribers=include_subscribers,
include_subscribers=parse_include_subscribers(include_subscribers),
)
return json_success(request, data={"subscriptions": subscribed})