Commit Graph

409 Commits

Author SHA1 Message Date
neiljp (Neil Pilgrim)
cb8d574648 mypy: Replace Any by Type[QueueProcessingWorker] in queue_processors.py. 2018-03-11 15:34:11 -07:00
neiljp (Neil Pilgrim)
fed51666d6 mypy: Migrate queue_processors.py to python3 syntax.
Forward-declarations of QueueProcessingWorker allow code order to remain
unchanged.

Note added re use of Dict vs Mapping.
2018-03-11 15:34:11 -07:00
neiljp (Neil Pilgrim)
6fc4d5bf40 mypy: Correct annotations in queue_processors.py. 2018-03-10 10:04:14 -08:00
Robert Hönig
81ba7a1e40 Mark DigestWorker and PushNotificationsWorker as nocoverage.
These two classes are tricky to test, and nocoverage-ing them
allows us to mark queue_processors.py as fully covered. We
still want to cover these two workers at some point, but for
now, it's nice to enforce full coverage for any future changes
to queue_processors.py.

Fixes (sort of) #6542.
2018-03-04 13:31:33 -08:00
Robert Hönig
93b2f0ea35 Mark MissedMessageSendingWorker as nocoverage.
The MissedMessageSendingWorker queue will be removed
after the 1.8 release, so covering it in tests is
useless.
2018-03-04 13:31:33 -08:00
Robert Hönig
4381ce74aa EmbeddedBotWorker: Remove @-mention check.
We already check in get_service_bot_events() if a bot is mentioned,
and then only pass on the call to the bot handler if it is. The
commit removes the additional check in the embedded bot queue
processor simply because it is impossible to obtain test coverage
for it (there is no meaningful way to trigger the content of the
if-clause, because there will never be a message reaching the bot
without @-mentioning it.

To alleviate the danger of a potential regression, the check is not
removed completely, but rather replaced by an assert statement.
2018-03-04 13:31:33 -08:00
Robert Hönig
9bee64de93 Exclude QueueProcessingWorker.stop() from coverage test.
This function is just a one-line wrapper, so adding
coverage isn't important.
2018-02-28 12:31:38 -08:00
Greg Price
4475950ddf queue: Restore prematurely-cut upgrade path.
Revert c8f034e9a "queue: Remove missedmessage_email_senders code."
As the comment in the code says, it ensures a smooth upgrade path
from 1.7.x; we can delete it in master after 1.8.0 is released.
The removal commit was merged early due to a communication failure.
2018-02-28 11:15:53 -08:00
Umair Khan
c8f034e9a0 queue: Remove missedmessage_email_senders code.
After 68513952fb, all emails are sent through email_senders queue.
This commit removes code related to the legacy queue.
2018-02-21 16:43:56 -08:00
Robert Hönig
a19a69bfe3 embedded bots: Log warning when bot quit()s.
External bots may call bot_handler.quit() when
they wish to terminate, e.g. due to a misconfiguration.
Currently, embedded bots ignore calls to quit(), even
though they signal a problem. This commit does the first
step in handling quit() calls by logging a warning.
2018-02-13 14:56:37 -08:00
rht
32f1523de2 zerver/worker: Remove u prefix from strings. 2018-02-05 12:12:58 -08:00
Tim Abbott
c2ceb3c13b EmailSendingWorker: Fix retry for sending emails.
If an exception was thrown inside `send_email` resulting in a retry,
we would include the `failed_tries` data in the event, which turned
out to thrown an exception itself.

This fixes that flow, including deepening the test so that it would
fail if we didn't have the new logic.
2018-01-30 11:28:09 -08:00
XavierCooney
35dc203d58 queue_processors.py: Remove unecessary imports. 2018-01-16 08:16:43 -05:00
Robert Hönig
83d0c7be31 embedded bots: Run bot.initialize() if bot has this function. 2018-01-07 20:05:52 +01:00
derAnfaenger
94c8e8b8e7 embedded bots: Strip @-mention from message.
This is in order to comply with the most recent
code in the `zulip_bots` package.
2017-12-26 08:50:00 -05:00
Umair Khan
68513952fb email-worker: Create EmailSendingWorker.
This commit just copies all the code from MissedMessageSendingWorker
class to a new EmailSendingWorker class. All the logic to send an email
through a queue was already there. This commit only makes the logic
generic. It does so by creating a special purpose queue called
'email_senders' to send any type of email. To make
MissedMessageSendingWorker still work we derive it from
EmailSendingWorker. All the tests that were testing
MissedMessageSendingWorker now run against EmailSendingWorker.
2017-12-20 19:36:27 -08:00
Vishnu Ks
16d8244c0a tests: Eliminate 'Sending invitation' output spam in test_signup.
Fixes #7563
2017-12-20 18:50:01 -08:00
Rishi Gupta
869b4d41ef models: Add ScheduledEmail.realm.
The two extra queries in the test are due to the assert in
send_future_email.
2017-12-19 17:46:36 -08:00
Shreyansh Dwivedi
47fcb27e39 invitations: Remove custom_body.
Fixes #7672
2017-12-11 19:23:54 -08:00
Rishi Gupta
968aae167b invitations: Remove get_prereg_user_by_email.
The original logic is buggy now that emails can belong to (and be
invited to) multiple realms.

The new logic in the `invites` queue worker also avoids the bug where
when the PreregistrationUser was gone by the time the queue worker got
to the invite (e.g., because it'd been revoked), we threw an exception.

[greg: fix upgrade-compatibility logic; add test; explain
revoked-invite race above]
2017-12-06 20:35:50 -08:00
Tim Abbott
b2cb443d24 subs: Fix clearing unread counts when leaving private streams.
Because we use access_stream_by_id here, and that checks for an active
subscription to interact with a private stream, this didn't work.

The correct fix to add an option to active_stream_by_id to accept an
argument indicating whether we need an active subscription; for this
use case, we definitely do not.
2017-11-29 14:40:08 -08:00
Eeshan Garg
c45517f544 python-zulip-api: Upgrade to PyPI package release 0.3.8.
There's one migration required by this release:

* queue_processors: Stop passing state_handler to handle_message.

  state_handler is now a property of bot_handler and thus, does
  not need to be passed to bot_handler.handle_message().

  The commit responsible is:
  2a74ad11c5
2017-11-27 20:31:37 -08:00
Vishnu Ks
766511e519 actions: Mark all messages as read when user unsubscribes from stream.
This fixes a bug where, when a user is unsubscribed from a stream,
they might have unread messages on that stream leak.  While it might
seem to be a minor problem, it can cause significant problems for
computing the `unread_msgs` data structures, since it means we need to
add an extra filter for whether the user is still subscribed, either
in the backend or in the UI.

Fixes #7095.
2017-11-21 20:09:17 -08:00
Tim Abbott
638eb7a8e4 docs: Update links to ReadTheDocs to always use https.
This is better security practice.

We also add a lint rule to enforce this for the future.
2017-11-16 10:59:24 -08:00
Tim Abbott
054952a44a docs: Update links from codebase to point to ReadTheDocs. 2017-11-16 10:53:49 -08:00
Steve Howell
dbcc76b996 Fix MissedMessageWorker.
This fixes a regression in 25c669df52.

We were draining the queue in both the superclass and the subclass,
so by the time the subclass started processing events, there were
no events to process.  Now the subclass properly uses the events
passed in from the superclass.
2017-11-15 10:26:28 -08:00
derAnfaenger
d3bee4fcc5 queue_processors: Exclude various lines from coverage.
The os.mkdir call is straightforward and doesn't testing.
Workers relying on LoopQueueProcessingWorker are tested
with its consume method that exists solely for this purpose.
2017-11-10 13:23:53 -08:00
Tim Abbott
25c669df52 queue_processors: Extract LoopQueueProcessingWorker class.
This should help make it easier to add tests coverage for these queue
processors, since they now use a system more similar to the other
queue processors.
2017-11-09 15:20:40 -08:00
rht
de319b4558 refactor: Remove six.moves.StringIO import. 2017-11-07 10:51:44 -08:00
derAnfaenger
c8e87a322d queue processors: Exclude TestWorker from coverage. 2017-11-07 09:38:58 -08:00
rht
8990b1046d zerver: Remove inheritance from object. 2017-11-06 08:53:48 -08:00
Tim Abbott
eade4d0052 tornado: Use call_consume_in_tests for message_sender queue.
This increases test coverage of queue_processors.py significantly,
with no real cost, achieving valuable progress torwards #6542.
2017-11-03 14:09:48 -07:00
rht
c4fcff7178 refactor: Replace super(.*self) with Python 3-specific super().
We change all the instances except for the `test_helpers.py`
TimeTrackingCursor monkey-patching, which actually needs to specify
the base class.
2017-10-30 14:30:25 -07:00
Tim Abbott
b936e8c24b lint: Fix lines in Python codebase longer than 125 characters. 2017-10-26 17:36:54 -07:00
derAnfaenger
1792dcbd09 tests: Call real consume method of queue processors.
This switches to more real tests for a first batch of
queue_json_publish() calls that don't cause trouble when
used with call_consume_tests=True.
2017-10-26 14:58:03 -07:00
Tim Abbott
ad165a6f8f settings: Remove remaining DEPLOYMENT_ROLE_* code remnants.
These should have been removed when we removed Zilencer.
2017-10-23 21:15:03 -07:00
derAnfaenger
c8a2702b9a embedded bots: Add functional state handler.
This replaces the former non-functional StateHandler
stub with a dictionary-like state object. Accessing it will
will read and store strings in the BotUserStateData model.

Each bot has a limited state size. To enforce this limit while
keeping data updates efficient, StateHandler caches the expensive
query for getting a bot's total state size. Assignments to a key
then only need to fetch that entry's previous size, if any, and
compare it to the new entry's size.
2017-10-19 13:09:23 -07:00
Tim Abbott
6176d0fbca json: Replace most use of simplejson with json.
This is progress towards removing simplejson as a dependency.
2017-10-11 22:55:35 -07:00
Umair Khan
435fe40199 worker: Retry signups queue event on 400. 2017-10-05 23:14:19 -07:00
Umair Khan
19e2551e82 mypy: Change type to Dict for SignupWorker. 2017-10-05 23:14:19 -07:00
Umair Khan
d95d34a66a Retry email failures in missed-message emails queue.
Fixes #6518.
2017-10-03 10:35:07 -07:00
Umair Khan
7d6ddaad26 mypy: Use Dict instead of Mapping in queues. 2017-10-03 10:35:07 -07:00
Greg Price
2a832386dc queue: Move more received-event logs to DEBUG level.
These can spew a lot of noise when starting the dev server, and aren't
in a super helpful form for normal server administration anyway.
2017-09-28 18:32:42 -07:00
Tim Abbott
630ecf9669 queue: Move user presence log events to DEBUG log level.
These are pretty noisy, and generally of little interest, since
presence requests come with a server log entry anyway.
2017-09-28 14:35:55 -07:00
rht
2ae47525d3 zerver/worker: Remove import six. 2017-09-27 17:06:16 -07:00
rht
41d8db373a zerver/worker: Remove absolute_import. 2017-09-27 10:00:39 -07:00
Tim Abbott
84926ff321 slow_queries: Log slow queries even if not reported via Zulip.
This should help make these logs more useful for debugging on an
arbitary Zulip system.
2017-09-22 14:11:07 -07:00
Tim Abbott
ec61a070b4 signups: Add basic logging for new signup processing. 2017-09-22 14:11:07 -07:00
Tim Abbott
f78dd33037 invites: Add logging for newly sent invitation emails. 2017-09-22 14:11:07 -07:00
Rishi Gupta
a7c8770f97 emails: Move enqueue_welcome_emails outside of signups queue.
The only thing this queue should do is sign you up for the newsletter, since
it is only populated if newsletter_data is not None.
2017-09-22 06:20:33 -07:00