missed-message: Add a try-catch to prevent killing background thread.

An exception which escapes from this loop can kill the background
worker thread; this results in consuming the queue (leading to the
illusion of progress) but more and more rows silently piling up in the
ScheduledMessageNotificationEmail table.

Wrap the inside of the `while True` loop in a try/catch to make sure
that no exceptions escape and kill the background thread.  To prevent
even more indentation, the inner loop is extracted into its own
function.  It returns true/false to signal if the `self.stopping` was
set to tell the loop to stop; we cannot check it ourselves in the
outer loop because it needs to hold the lock to be examined.
This commit is contained in:
Alex Vandiver
2023-07-25 14:42:05 +00:00
committed by Tim Abbott
parent 213387249e
commit c77c78f147

View File

@@ -657,9 +657,20 @@ class MissedMessageWorker(QueueProcessingWorker):
def work(self) -> None:
while True:
try:
finished = self.background_loop()
if finished:
break
except Exception:
logging.exception(
"Exception in MissedMessage background worker; restarting the loop",
stack_info=True,
)
def background_loop(self) -> bool:
with self.cv:
if self.stopping:
return
return True
# There are three conditions which we wait for:
#
# 1. We are being explicitly asked to stop; see the
@@ -708,6 +719,8 @@ class MissedMessageWorker(QueueProcessingWorker):
if not was_notified:
self.maybe_send_batched_emails()
return False
def maybe_send_batched_emails(self) -> None:
current_time = timezone_now()