Centralize dispatch logic for which queue processor to use.

Previous we had around 4 copies of the logic for deciding whether we
should publish data via a SimpleQueueClient queue, a
TornadoQueueClient queue, or to directly handle the operation, which
resulted in their getting out of sync and buggy (see e.g. the previous
commit).

We need to add a lock around adding things to the queue to work around
a bug with pika's BlockingConnection.

I should note that the previous logic in some places had a bunch of
tests of the form "elif settings.TEST_SUITE" for doing the work that
would have been done by the queue processor directly; these should
have just been "else" clauses -- since we generally want that code to
run on development environments whether or not the test suite is
currently running.

(imported from commit 16bdbed4fff04b1bda6fde3b16bee7359917720b)

zephyr/lib/queue.py

@@ -4,6 +4,7 @@ import logging
 import simplejson
 import random
 import time
+import threading
 from collections import defaultdict
 
 # This simple queuing library doesn't expose much of the power of
@@ -168,3 +169,23 @@ class TornadoQueueClient(SimpleQueueClient):
         self.ensure_queue(queue_name,
             lambda: self.channel.basic_consume(wrapped_consumer, queue=queue_name,
                 consumer_tag=self._generate_ctag(queue_name)))
+
+if settings.RUNNING_INSIDE_TORNADO and settings.USING_RABBITMQ:
+    queue_client = TornadoQueueClient()
+else:
+    queue_client = SimpleQueueClient()
+
+# We using a simple lock to prevent multiple RabbitMQ messages being
+# sent to the SimpleQueueClient at the same time; this is a workaround
+# for an issue with the pika BlockingConnection where using
+# BlockingConnection for multiple queues causes the channel to
+# randomly close.
+queue_lock = threading.RLock()
+
+def queue_json_publish(queue_name, event, processor):
+    with queue_lock:
+        if settings.USING_RABBITMQ:
+            queue_client.json_publish(queue_name, event)
+        else:
+            processor(event)
+