Massive digester rework.

This moves actual digest creation and sending into a new queue runner.  That
way, digest creation does not block the main incoming processor.  Lots of code
clean up and reorganization too, and many fixes.

digest_last_sent_at: Make this a DateTime
digest_size_threshold: Make this a Float
digest_volume_frequency: Make this an Enum

DigestFrequency: new enum

Fix up IMailingList interface to reflect reality.

In Runner base class, allow for msg.get_sender() to return a False value.  If
this happens (as it can with the empty marker messages used to signal the
digester that there's work to do), just use the list's preferred language.

Give the retry.py and incoming.py queue runner modules some proper epilogues.

The EmptyingRunner (for testing) logs itself as the runner its wrapping.

The digest file name now ends in .mmdf since that's the format we're using.

Add 'create_list' to doctest globals.

Add a Mailbox class to add support for the context manager protocol.

1 files changed, 9 insertions, 3 deletions

diff --git a/src/mailman/testing/helpers.py b/src/mailman/testing/helpers.py
index f92c5f012..10f5229d7 100644
--- a/src/mailman/testing/helpers.py
+++ b/src/mailman/testing/helpers.py
@@ -35,7 +35,6 @@ import errno
 import signal
 import socket
 import logging
-import mailbox
 import smtplib
 import threading
 
@@ -44,6 +43,7 @@ from Queue import Empty, Queue
 from mailman.bin.master import Loop as Master
 from mailman.config import config
 from mailman.testing.smtplistener import Server
+from mailman.utilities.mailbox import Mailbox
 
 
 log = logging.getLogger('mailman.debug')
@@ -69,6 +69,12 @@ def make_testable_runner(runner_class, name=None):
     class EmptyingRunner(runner_class):
         """Stop processing when the queue is empty."""
 
+        def __init__(self, *args, **kws):
+            super(EmptyingRunner, self).__init__(*args, **kws)
+            # We know it's an EmptyingRunner, so really we want to see the
+            # super class in the log files.
+            self.__class__.__name__ = runner_class.__name__
+
         def _do_periodic(self):
             """Stop when the queue is empty."""
             self._stop = (len(self.switchboard.files) == 0)
@@ -106,8 +112,8 @@ def digest_mbox(mlist):
     :param mlist: The mailing list.
     :return: The mailing list's pending digest as a mailbox.
     """
-    path = os.path.join(mlist.data_path, 'digest.mbox')
-    return mailbox.mbox(path)
+    path = os.path.join(mlist.data_path, 'digest.mmdf')
+    return Mailbox(path)