subscriber events: Change schema for peer_add/peer_remove.

We now can send an implied matrix of user/stream tuples
for peer_add and peer_remove events.

The client code basically does this:

    for stream_id in event['stream_ids']:
        for user_id in event['user_ids']:
            update_sub(stream_id, user_id)

We used to send individual events, which gets real
expensive when you are creating new streams. For
the case of copy-to-stream case, we should see
events go from U to 1, where U is the number of users
added.

Note that we don't yet fully optimize the potential
of this schema.  For adding a new user with lots
of default streams, we still send S peer_add events.

And if you subscribe a bunch of users to a bunch of
private streams, we only go from U * S to S; we can't
optimize it down to one event easily.
This commit is contained in:
Steve Howell
2020-10-22 12:14:02 +00:00
committed by Tim Abbott
parent efa8dd3a47
commit 7ff3859136
8 changed files with 144 additions and 137 deletions

View File

@@ -2966,14 +2966,13 @@ def send_peer_subscriber_events(
peer_user_ids = private_peer_dict[stream_id] - altered_user_ids
if peer_user_ids:
for new_user_id in altered_user_ids:
event = dict(
type="subscription",
op=op,
stream_id=stream_id,
user_id=new_user_id,
)
send_event(realm, event, peer_user_ids)
event = dict(
type="subscription",
op=op,
stream_ids=[stream_id],
user_ids=sorted(list(altered_user_ids))
)
send_event(realm, event, peer_user_ids)
public_stream_ids = [
stream_id for stream_id in altered_user_dict
@@ -2984,19 +2983,25 @@ def send_peer_subscriber_events(
if public_stream_ids:
public_peer_ids = set(active_non_guest_user_ids(realm.id))
# TODO:
#
# We eventually want a special optimization for a single user that
# subscribes to many streams. Right now we optimize for the other
# scenario, which is also kind of common--if we add/remove multiple
# users all for the same stream, we just send one event per stream.
for stream_id in public_stream_ids:
altered_user_ids = altered_user_dict[stream_id]
peer_user_ids = public_peer_ids - altered_user_ids
if peer_user_ids:
for new_user_id in altered_user_ids:
event = dict(
type="subscription",
op=op,
stream_id=stream_id,
user_id=new_user_id,
)
send_event(realm, event, peer_user_ids)
event = dict(
type="subscription",
op=op,
stream_ids=[stream_id],
user_ids=sorted(list(altered_user_ids)),
)
send_event(realm, event, peer_user_ids)
def send_peer_remove_events(
realm: Realm,