messages: Return shallow copy of message object.

When more than one outgoing webhook is configured,
the message which is send to the webhook bot passes
through finalize_payload function multiple times,
which mutated the message dict in a way that many keys
were lost from the dict obj.

This commit fixes that problem by having
`finalize_payload` return a shallow copy of the
incoming dict, instead of mutating it.  We still
mutate dicts inside of `post_process_dicts`, though,
for performance reasons.

This was slightly modified by @showell to fix the
`test_both_codepaths` test that was added concurrently
to this work.  (I used a slightly verbose style in the
tests to emphasize the transformation from `wide_dict`
to `narrow_dict`.)

I also removed a deepcopy call inside
`get_client_payload`, since we now no longer mutate
in `finalize_payload`.

Finally, I added some comments here and there.

For testing, I mostly protect against the root
cause of the bug happening again, by adding a line
to make sure that `sender_realm_id` does not get
wiped out from the "wide" dictionary.

A better test would exercise the actual code that
exposed the bug here by sending a message to a bot
with two or more services attached to it.  I will
do that in a future commit.

Fixes #14384
This commit is contained in:
Udit107710
2020-03-26 22:16:23 +00:00
committed by Tim Abbott
parent 4c51a94bcd
commit ef741bf317
6 changed files with 51 additions and 19 deletions

View File

@@ -840,10 +840,11 @@ def process_message_event(event_template: Mapping[str, Any], users: Iterable[Map
@cachify
def get_client_payload(apply_markdown: bool, client_gravatar: bool) -> Dict[str, Any]:
dct = copy.deepcopy(wide_dict)
MessageDict.finalize_payload(dct, apply_markdown, client_gravatar)
return dct
return MessageDict.finalize_payload(
wide_dict,
apply_markdown=apply_markdown,
client_gravatar=client_gravatar
)
# Extra user-specific data to include
extra_user_data = {} # type: Dict[int, Any]