Alex Vandiver 656ca17e14 thumbnail: Resolve a race condition when rendering messages. ...

Messages are rendered outside of a transaction, for performance
reasons, and then sent inside of one.  This opens thumbnailing up to a
race where the thumbnails have not yet been written when the message
is rendered, but the message has not been sent when thumbnailing
completes, causing `rewrite_thumbnailed_images` to be a no-op and the
message being left with a spinner which never resolves.

Explicitly lock and use he ImageAttachment data inside the
message-sending transaction, to rewrite the message content with the
latest information about the existing thumbnails.

Despite the thumbnailing worker taking a lock on Message rows to
update them, this does not lead to deadlocks -- the INSERT of the
Message rows happens in a transaction, ensuring that either the
message rending blocks the thumbnailing until the Message row is
created, or that the `rewrite_thumbnailed_images` and Message INSERT
waits until thumbnailing is complete (and updated no Message rows).

(cherry picked from commit 6f20c15ae9)

2024-08-02 14:40:13 -07:00

..

fixtures

markdown: Show thumbnails for uploaded images.

2024-07-21 18:41:59 -07:00

images

thumbnail: Store the post-orientation-transformation dimensions.

2024-07-24 09:56:42 -07:00

__init__.py

…

test_alert_words.py

…

test_attachments.py

…

test_audit_log.py

settings: Add setting to control how animated images are played.

2024-07-22 14:53:31 -07:00

test_auth_backends.py

tests: Remove deprecated SHA1PasswordHasher.

2024-07-16 13:06:31 -07:00

test_bots.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

test_cache.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

test_compatibility.py

python: Reformat with Ruff formatter.

2024-02-29 17:07:16 -08:00

test_create_video_call.py

ruff: Bump target-version from py38 to py310.

2024-07-13 22:28:22 -07:00

test_custom_profile_data.py

ruff: Fix FURB148 enumerate index or value is unused.

2024-07-14 13:54:32 -07:00

test_data_types.py

…

test_decorators.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

test_delete_unclaimed_attachments.py

test_delete_unclaimed_attachments: Close file in make_attachment.

2024-07-19 09:20:56 -07:00

test_digest.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

test_docs.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

test_drafts.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

test_email_change.py

realm-deactivation: Send email to owners as part of deactivation.

2024-06-26 16:48:18 -07:00

test_email_log.py

ruff: Bump target-version from py38 to py310.

2024-07-13 22:28:22 -07:00

test_email_mirror.py

email_mirror: Change default topic name if subject ends up empty.

2024-07-22 22:31:45 -07:00

test_email_notifications.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

test_embedded_bot_system.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

test_event_queue.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

test_event_system.py

fetch_initial_state: Avoid doing one db query per announcement stream.

2024-07-22 10:21:22 -07:00

test_events.py

settings: Add setting to control how animated images are played.

2024-07-22 14:53:31 -07:00

test_example.py

ruff: Bump target-version from py38 to py310.

2024-07-13 22:28:22 -07:00

test_external.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

test_github.py

…

test_has_request_variables.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

test_health.py

ruff: Bump target-version from py38 to py310.

2024-07-13 22:28:22 -07:00

test_home.py

fetch_initial_state: Avoid doing one db query per announcement stream.

2024-07-22 10:21:22 -07:00

test_i18n.py

invites: Lock the realm when determining invitation counts.

2024-05-02 14:23:04 -07:00

test_import_export.py

test_import_export: Add test coverage for OnboardingUserMessage.

2024-07-22 10:26:33 -07:00

test_integrations_dev_panel.py

ruff: Bump target-version from py38 to py310.

2024-07-13 22:28:22 -07:00

test_integrations.py

refactor: Relocate screenshots scripts to new tools/screenshots dir.

2024-05-14 16:17:44 -07:00

test_internet.py

…

test_invite.py

zerver: Migrate some files to typed_endpoint.

2024-07-20 15:46:48 -07:00

test_legacy_subject.py

…

test_link_embed.py

events: Send delete_message event to user who deleted the message.

2024-07-16 23:29:02 -07:00

test_management_commands.py

ruff: Fix PLR6104 Use += to perform an augmented assignment directly.

2024-07-14 13:49:51 -07:00

test_markdown_thumbnail.py

thumbnail: Resolve a race condition when rendering messages.

2024-08-02 14:40:13 -07:00

test_markdown.py

tests: Remove THUMBNAIL_IMAGES overrides, since the setting was dropped.

2024-07-24 09:53:14 -07:00

test_mattermost_importer.py

mattermost: Remove unnecessary MD5 hashing of direct message groups.

2024-07-17 15:56:00 -07:00

test_message_delete.py

events: Send delete_message event to user who deleted the message.

2024-07-16 23:29:02 -07:00

test_message_dict.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

test_message_edit_notifications.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

test_message_edit.py

ruff: Fix PLR6104 Use += to perform an augmented assignment directly.

2024-07-14 13:49:51 -07:00

test_message_fetch.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

test_message_flags.py

ruff: Bump target-version from py38 to py310.

2024-07-13 22:28:22 -07:00

test_message_move_stream.py

ruff: Fix PLR6104 Use += to perform an augmented assignment directly.

2024-07-14 13:49:51 -07:00

test_message_move_topic.py

events: Send delete_message event to user who deleted the message.

2024-07-16 23:29:02 -07:00

test_message_notification_emails.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

test_message_send.py

ruff: Fix PLR6104 Use += to perform an augmented assignment directly.

2024-07-14 13:49:51 -07:00

test_message_topics.py

streams: Update translated errors for stream to channel rename.

2024-04-24 14:35:05 -07:00

test_messages.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

test_middleware.py

ruff: Bump target-version from py38 to py310.

2024-07-13 22:28:22 -07:00

test_migrations.py

CVE-2024-27286: Delete dangling UserMessage rows.

2024-03-19 18:22:24 -04:00

test_mirror_users.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

test_muted_users.py

refactor: Rename huddle to direct_message_group in non API.

2024-07-04 07:56:31 -07:00

test_new_users.py

ruff: Bump target-version from py38 to py310.

2024-07-13 22:28:22 -07:00

test_notification_data.py

models: Extract zerver.models.scheduled_jobs.

2023-12-16 22:08:44 -08:00

test_onboarding_steps.py

compose: Show banner to explain interleaved view messages fading.

2024-07-16 13:52:30 -07:00

test_onboarding.py

…

test_openapi.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

test_outgoing_http.py

…

test_outgoing_webhook_interfaces.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

test_outgoing_webhook_system.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

test_populate_db.py

…

test_presence.py

ruff: Bump target-version from py38 to py310.

2024-07-13 22:28:22 -07:00

test_push_notifications.py

settings: Rework how push notifications service is configured.

2024-07-17 17:14:06 -07:00

test_queue_worker.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

test_queue.py

ruff: Bump target-version from py38 to py310.

2024-07-13 22:28:22 -07:00

test_rate_limiter.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

test_reactions.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

test_read_receipts.py

refactor: Rename huddle to direct_message_group in non API.

2024-07-04 07:56:31 -07:00

test_realm_domains.py

models: Extract zerver.models.realms.

2023-12-16 22:08:44 -08:00

test_realm_emoji.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

test_realm_export.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

test_realm_linkifiers.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

test_realm_playgrounds.py

models: Extract zerver.models.realms.

2023-12-16 22:08:44 -08:00

test_realm.py

settings: Add setting to control how animated images are played.

2024-07-22 14:53:31 -07:00

test_recipient_parsing.py

i18n: Update translated errors for stream to channel rename.

2024-04-24 14:35:05 -07:00

test_redis_utils.py

…

test_report.py

…

test_retention.py

events: Send delete_message event to user who deleted the message.

2024-07-16 23:29:02 -07:00

test_rocketchat_importer.py

rocketchat: Remove unnecessary SHA-1 hashing of direct message groups.

2024-07-17 15:56:00 -07:00

test_scheduled_messages.py

api: Deprecate uri and add url parameter in "/user_uploads" endpoint.

2024-07-14 22:32:36 -07:00

test_scim.py

ruff: Bump target-version from py38 to py310.

2024-07-13 22:28:22 -07:00

test_send_email.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

test_server_settings.py

…

test_service_bot_system.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

test_sessions.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

test_settings.py

settings: Add setting to control how animated images are played.

2024-07-22 14:53:31 -07:00

test_signup.py

tests: Remove deprecated SHA1PasswordHasher.

2024-07-16 13:06:31 -07:00

test_slack_importer.py

ruff: Fix B905 zip() without an explicit strict= parameter.

2024-07-13 22:28:22 -07:00

test_slack_message_conversion.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

test_soft_deactivation.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

test_subdomains.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

test_submessage.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

test_subs.py

streams: Convert to typed_enpoint.

2024-07-21 12:57:50 -07:00

test_templates.py

…

test_thumbnail.py

thumbnail: Do not Camo old thumbor URLs; serve images directly.

2024-07-24 16:04:34 -07:00

test_timeout.py

timeout: Rename to unsafe_timeout.

2024-04-18 11:50:38 -07:00

test_timestamp.py

…

test_timezone.py

ruff: Bump target-version from py38 to py310.

2024-07-13 22:28:22 -07:00

test_tornado.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

test_transfer.py

emoji: Pass down content-type, rather than guessing from extension.

2024-07-12 13:26:47 -07:00

test_tutorial.py

onboarding: Update Welcome Bot's responses to commands.

2024-07-09 13:08:02 -07:00

test_typed_endpoint_validators.py

zerver: Migrate some files to typed_endpoint.

2024-07-20 15:46:48 -07:00

test_typed_endpoint.py

ruff: Fix UP035 Import from collections.abc, typing instead.

2024-07-13 22:28:22 -07:00

test_typing.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

test_upload_local.py

upload: Generate thumbnails when images are uploaded.

2024-07-16 13:22:15 -07:00

test_upload_s3.py

upload: Generate thumbnails when images are uploaded.

2024-07-16 13:22:15 -07:00

test_upload.py

tests: Consume streaming responses.

2024-07-19 09:20:56 -07:00

test_url_decoding.py

url_decoding: Add 'is_same_server_message_link' function.

2024-06-17 10:42:32 -07:00

test_urls.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

test_user_groups.py

ruff: Fix SIM117 Use a single with statement with multiple contexts.

2024-07-14 13:48:32 -07:00

test_user_status.py

ruff: Fix UP007 Use X | Y for type annotations.

2024-07-13 22:28:22 -07:00

test_user_topics.py

validators: Use cleaner syntax for AfterValidator.

2024-07-15 16:49:32 -07:00

test_users.py

users: Reduce date_joined precision to minutes.

2024-08-01 11:24:58 -07:00

test_validators.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

test_webhooks_common.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

test_widgets.py

ruff: Fix UP006 Use list instead of List for type annotation.

2024-07-13 22:28:22 -07:00

test_zcommand.py

zcommand: Rename night->dark and day->light in the color scheme backend.

2024-06-30 22:45:26 -07:00

test_zephyr.py

ruff: Bump target-version from py38 to py310.

2024-07-13 22:28:22 -07:00

test_zulip_update_announcements.py

refactor: Rename huddle to direct_message_group in non API.

2024-07-04 07:56:31 -07:00