Alex Vandiver 656ca17e14 thumbnail: Resolve a race condition when rendering messages. ...

Messages are rendered outside of a transaction, for performance
reasons, and then sent inside of one.  This opens thumbnailing up to a
race where the thumbnails have not yet been written when the message
is rendered, but the message has not been sent when thumbnailing
completes, causing `rewrite_thumbnailed_images` to be a no-op and the
message being left with a spinner which never resolves.

Explicitly lock and use he ImageAttachment data inside the
message-sending transaction, to rewrite the message content with the
latest information about the existing thumbnails.

Despite the thumbnailing worker taking a lock on Message rows to
update them, this does not lead to deadlocks -- the INSERT of the
Message rows happens in a transaction, ensuring that either the
message rending blocks the thumbnailing until the Message row is
created, or that the `rewrite_thumbnailed_images` and Message INSERT
waits until thumbnailing is complete (and updated no Message rows).

(cherry picked from commit 6f20c15ae9)

2024-08-02 14:40:13 -07:00

..

markdown

thumbnail: Resolve a race condition when rendering messages.

2024-08-02 14:40:13 -07:00

upload

thumbnail: Move get_image_thumbnail_path and split_thumbnail_path.

2024-07-18 13:50:28 -07:00

url_preview

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

webhooks

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

__init__.py

…

addressee.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

alert_words.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

async_utils.py

mypy: Enable new error explicit-override.

2023-10-12 12:28:41 -07:00

attachments.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

avatar_hash.py

avatars: Encode version into the filename.

2024-07-07 14:40:07 -07:00

avatar.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

bot_config.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

bot_lib.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

bot_storage.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

bulk_create.py

ruff: Fix FURB142 Use of set.add() in a for loop.

2024-07-14 13:52:59 -07:00

cache_helpers.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

cache.py

ruff: Fix FURB154 Use of repeated consecutive global, nonlocal.

2024-07-14 13:53:18 -07:00

camo.py

…

ccache.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

compatibility.py

ruff: Fix PLR6104 Use += to perform an augmented assignment directly.

2024-07-14 13:49:51 -07:00

context_managers.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

create_user.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

data_types.py

ruff: Fix UP038 Use X | Y in isinstance call instead of (X, Y).

2024-07-13 22:28:22 -07:00

db_connections.py

db: Split reset_queries into a new module zerver.lib.db_connections.

2024-04-17 16:49:03 -07:00

db.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

debug.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

default_streams.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

dev_ldap_directory.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

digest.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

display_recipient.py

ruff: Fix FURB142 Use of set.add() in a for loop.

2024-07-14 13:52:59 -07:00

domains.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

drafts.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

email_mirror_helpers.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

email_mirror.py

email_mirror: Change default topic name if subject ends up empty.

2024-07-22 22:31:45 -07:00

email_notifications.py

ruff: Bump target-version from py38 to py310.

2024-07-13 22:28:22 -07:00

email_validation.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

emoji_utils.py

emoji: Match emoji sequences in markdown.

2023-08-23 16:18:15 -07:00

emoji.py

emoji: Use a non-predictable filename.

2024-07-12 13:26:47 -07:00

event_schema.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

events.py

fetch_initial_state: Avoid doing one db query per announcement stream.

2024-07-22 10:21:22 -07:00

exceptions.py

exceptions: Add new error class to be used for invalid parameter values.

2024-07-19 09:24:54 -07:00

export.py

upload: Generate thumbnails when images are uploaded.

2024-07-16 13:22:15 -07:00

external_accounts.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

fix_unreads.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

generate_test_data.py

ruff: Fix PLR6104 Use += to perform an augmented assignment directly.

2024-07-14 13:49:51 -07:00

github.py

…

home.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

html_diff.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

html_to_text.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

i18n.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

import_realm.py

import_realm: Update incorrect comment about emoji dir in exported data.

2024-07-21 13:16:49 -07:00

initial_password.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

integrations.py

integrations: Remove Solano integration.

2024-07-20 15:44:17 -07:00

invites.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

logging_util.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

management.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

mdiff.py

…

mention.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

message_cache.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

message.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

migrate.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

mime_types.py

emoji: Derive the file extension from a limited set of content-types.

2024-07-12 13:26:47 -07:00

mobile_auth_otp.py

ruff: Fix B905 zip() without an explicit strict= parameter.

2024-07-13 22:28:22 -07:00

muted_users.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

name_restrictions.py

integrations: Remove Google Code-in integration.

2024-07-02 10:04:34 -07:00

narrow_helpers.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

narrow_predicate.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

narrow.py

narrow: Migrate legacy SQLAlchemy select syntax.

2024-07-16 14:50:30 -07:00

notes.py

ruff: Fix FURB180 Use of metaclass=abc.ABCMeta.

2024-07-14 13:53:40 -07:00

notification_data.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

onboarding_steps.py

compose: Show banner to explain interleaved view messages fading.

2024-07-16 13:52:30 -07:00

onboarding.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

outgoing_http.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

outgoing_webhook.py

ruff: Fix FURB180 Use of metaclass=abc.ABCMeta.

2024-07-14 13:53:40 -07:00

partial.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

per_request_cache.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

presence.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

profile.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

push_notifications.py

settings: Rework how push notifications service is configured.

2024-07-17 17:14:06 -07:00

pysa.py

…

query_helpers.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

queue.py

ruff: Fix FURB180 Use of metaclass=abc.ABCMeta.

2024-07-14 13:53:40 -07:00

rate_limiter.py

ruff: Fix B905 zip() without an explicit strict= parameter.

2024-07-13 22:28:22 -07:00

realm_description.py

…

realm_icon.py

…

realm_logo.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

recipient_parsing.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

recipient_users.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

redis_utils.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

remote_server.py

settings: Rework how push notifications service is configured.

2024-07-17 17:14:06 -07:00

request.py

ruff: Fix B905 zip() without an explicit strict= parameter.

2024-07-13 22:28:22 -07:00

response.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

rest.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

retention.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

safe_session_cached_db.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

scheduled_messages.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

scim_filter.py

zerver: Replace uri with url in local variables and comments.

2024-07-14 22:30:28 -07:00

scim.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

send_email.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

server_initialization.py

ruff: Fix FURB142 Use of set.add() in a for loop.

2024-07-14 13:52:59 -07:00

sessions.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

singleton_bmemcached.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

soft_deactivation.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

sounds.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

sqlalchemy_utils.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

storage.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

stream_color.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

stream_subscription.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

stream_topic.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

stream_traffic.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

streams.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

string_validation.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

subdomains.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

subscription_info.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

templates.py

ruff: Fix FURB154 Use of repeated consecutive global, nonlocal.

2024-07-14 13:53:18 -07:00

test_classes.py

test_classes: Extract a thumbnailing output format helper.

2024-07-21 18:41:59 -07:00

test_console_output.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

test_data.source.txt

…

test_fixtures.py

ruff: Fix FURB142 Use of set.add() in a for loop.

2024-07-14 13:52:59 -07:00

test_helpers.py

tests: Consume streaming responses.

2024-07-19 09:20:56 -07:00

test_runner.py

tests: Ensure we start with a local storage backend.

2024-07-16 13:22:15 -07:00

tex.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

thumbnail.py

thumbnail: Resolve a race condition when rendering messages.

2024-08-02 14:40:13 -07:00

timeout.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

timestamp.py

python: Consistently use from…import for datetime.

2023-12-05 12:01:18 -08:00

timezone.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

topic_sqlalchemy.py

narrow: Migrate legacy SQLAlchemy select syntax.

2024-07-16 14:50:30 -07:00

topic.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

transfer.py

migrations: Re-thumbnail and rename existing emoji.

2024-07-12 13:26:47 -07:00

typed_endpoint_validators.py

streams: Convert to typed_enpoint.

2024-07-21 12:57:50 -07:00

typed_endpoint.py

streams: Convert to typed_enpoint.

2024-07-21 12:57:50 -07:00

types.py

settings: Rework how push notifications service is configured.

2024-07-17 17:14:06 -07:00

url_decoding.py

url_decoding: Add 'is_same_server_message_link' function.

2024-06-17 10:42:32 -07:00

url_encoding.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

url_redirects.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

user_agent.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

user_counts.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

user_groups.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

user_message.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

user_status.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

user_topics.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

users.py

users: Reduce date_joined precision to minutes.

2024-08-01 11:24:58 -07:00

utils.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

validator.py

zerver: Migrate some files to typed_endpoint.

2024-07-20 15:46:48 -07:00

widget.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

zcommand.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

zephyr.py

…

zulip_update_announcements.py

updates: Add update message (level=8).

2024-07-25 13:19:25 -07:00