message_edit: Carry the QuerySet through as much as possible.

Rather than pass around a list of message objects in-memory, we
instead keep the same constructed QuerySet which includes the later
propagated messages (if any), and use that same query to pick out
affected Attachment objects, rather than limiting to the set of ids.
This is not necessarily a win -- the list of message-ids *may* be very
long, and thus the query may be more concise, easier to send to
PostgreSQL, and faster for PostgreSQL to parse.  However, the list of
ids is almost certainly better-indexed.

After processing the move, the QuerySet must be re-defined as a search
of ids (and possibly a very long list of such), since there is no
other way which is guaranteed to correctly single out the moved
messages.  At this point, it is mostly equivalent to the list of
Message objects, and certainly takes no less memory.
This commit is contained in:
Alex Vandiver
2023-09-26 18:44:28 +00:00
committed by Tim Abbott
parent 5c96f94206
commit 22837fc1b4
4 changed files with 50 additions and 37 deletions

View File

@@ -1,5 +1,5 @@
from datetime import datetime
from typing import Any, Dict, List, Optional, Set, Tuple
from typing import Any, Callable, Dict, List, Optional, Set, Tuple
import orjson
from django.db import connection
@@ -154,7 +154,7 @@ def update_messages_for_topic_edit(
old_stream: Stream,
edit_history_event: EditHistoryEvent,
last_edit_time: datetime,
) -> List[Message]:
) -> Tuple[QuerySet[Message], Callable[[], QuerySet[Message]]]:
# Uses index: zerver_message_realm_recipient_upper_subject
messages = Message.objects.filter(
realm_id=old_stream.realm_id,
@@ -222,12 +222,15 @@ def update_messages_for_topic_edit(
# update, and then return a fresh collection -- so we know their
# metadata has been updated for the UPDATE command, and the caller
# can update the remote cache with that.
message_ids = list(messages.values_list("id", flat=True))
messages.update(**update_fields)
message_ids = [edited_message.id, *messages.values_list("id", flat=True)]
return list(
Message.objects.filter(id__in=message_ids).select_related(*Message.DEFAULT_SELECT_RELATED)
)
def propagate() -> QuerySet[Message]:
messages.update(**update_fields)
return Message.objects.filter(id__in=message_ids).select_related(
*Message.DEFAULT_SELECT_RELATED
)
return messages, propagate
def generate_topic_history_from_db_rows(rows: List[Tuple[str, int]]) -> List[Dict[str, Any]]: