Merge bounce cleanup branch. This is a major cleansing and refactoring of the

bounce processor.  Note that one thing this does not do is implement any of
the policy around bounce scores.  It "merely" cleans up all the crud in the
BounceRunner, and all the logic around registering bounce events.  This means
the bounce runner can actually be enabled again.  More code needs to be
written to process bounces events occasionally.

Details:

* maybe_forward() moved to app/bounces.py, and it can now discard the
  message, forward it to the list administrators (moderators + owners), or
  site owners.  See UnrecognizedBounceDisposition.
* scan_message() always returns a set of addresses.
* The DSN bounce detector is cleaned up.
* An IBounceProcessor utility is added.
* IBounceEvents are added, with database support.
* bounce_unrecognized_goes_to_list_owner -> forward_unrecognized_bounces_to
* bounce_processing -> process_bounces
* ReopenableFileHandler.filename is exposed as a public attribute.  This aids
  in testing.
* Fix the signature of UserNotification.__init__() to be more PEP 8 compliant.
* Change the OwnerNotification.__init__() signature to take a roster object
  instead of `tomoderators`.  When the roster is None, the site owner is used
  instead.
* Fix the default style setting; s/personalization/personalize/
* BounceMixin is gone, baby gone.
* Add tests for the OutgoingRunner.
* make_testable_runner() can now take a predicate object, which can change how
  often the runner goes through its main loop.  Use this e.g. to go through
  only once when a message does not get dequeued.
* A new LogFileMark helper for testing messages to a log file.

2 files changed, 23 insertions, 13 deletions

diff --git a/src/mailman/pipeline/docs/acknowledge.txt b/src/mailman/pipeline/docs/acknowledge.txt
index b2ce11b7f..8c8552190 100644
--- a/src/mailman/pipeline/docs/acknowledge.txt
+++ b/src/mailman/pipeline/docs/acknowledge.txt
@@ -7,7 +7,7 @@ receive acknowledgments of their postings, Mailman will sent them such an
 acknowledgment.
 ::
 
-    >>> mlist = create_list('_xtest@example.com')
+    >>> mlist = create_list('test@example.com')
     >>> mlist.real_name = 'XTest'
     >>> mlist.preferred_language = 'en'
     >>> # XXX This will almost certainly change once we've worked out the web
@@ -30,7 +30,7 @@ Subscribe a user to the mailing list.
     >>> user_1 = user_manager.create_user('aperson@example.com')
     >>> address_1 = list(user_1.addresses)[0]
     >>> mlist.subscribe(address_1, MemberRole.member)
-    <Member: aperson@example.com on _xtest@example.com as MemberRole.member>
+    <Member: aperson@example.com on test@example.com as MemberRole.member>
 
 
 Non-member posts
@@ -83,7 +83,7 @@ will be sent.
     >>> user_2 = user_manager.create_user('dperson@example.com')
     >>> address_2 = list(user_2.addresses)[0]
     >>> mlist.subscribe(address_2, MemberRole.member)
-    <Member: dperson@example.com on _xtest@example.com as MemberRole.member>
+    <Member: dperson@example.com on test@example.com as MemberRole.member>
 
     >>> handler.process(mlist, msg,
     ...     dict(original_sender='dperson@example.com'))
@@ -112,14 +112,19 @@ The receipt will include the original message's subject in the response body,
     >>> qmsg, qdata = virginq.dequeue(virginq.files[0])
     >>> virginq.files
     []
-    >>> sorted(qdata.items())
-    [..., ('recipients', [u'aperson@example.com']), ...]
+    >>> dump_msgdata(qdata)
+    _parsemsg           : False
+    listname            : test@example.com
+    nodecorate          : True
+    recipients          : set([u'aperson@example.com'])
+    reduced_list_headers: True
+    ...
     >>> print qmsg.as_string()
     ...
     MIME-Version: 1.0
     ...
     Subject: XTest post acknowledgment
-    From: _xtest-bounces@example.com
+    From: test-bounces@example.com
     To: aperson@example.com
     ...
     Precedence: bulk
@@ -130,7 +135,7 @@ The receipt will include the original message's subject in the response body,
     <BLANKLINE>
     was successfully received by the XTest mailing list.
     <BLANKLINE>
-    List info page: http://lists.example.com/listinfo/_xtest@example.com
+    List info page: http://lists.example.com/listinfo/test@example.com
     Your preferences: http://example.com/aperson@example.com
     <BLANKLINE>
 
@@ -146,13 +151,18 @@ If there is no subject, then the receipt will use a generic message.
     >>> qmsg, qdata = virginq.dequeue(virginq.files[0])
     >>> virginq.files
     []
-    >>> sorted(qdata.items())
-    [..., ('recipients', [u'aperson@example.com']), ...]
+    >>> dump_msgdata(qdata)
+    _parsemsg           : False
+    listname            : test@example.com
+    nodecorate          : True
+    recipients          : set([u'aperson@example.com'])
+    reduced_list_headers: True
+    ...
     >>> print qmsg.as_string()
     MIME-Version: 1.0
     ...
     Subject: XTest post acknowledgment
-    From: _xtest-bounces@example.com
+    From: test-bounces@example.com
     To: aperson@example.com
     ...
     Precedence: bulk
@@ -163,6 +173,6 @@ If there is no subject, then the receipt will use a generic message.
     <BLANKLINE>
     was successfully received by the XTest mailing list.
     <BLANKLINE>
-    List info page: http://lists.example.com/listinfo/_xtest@example.com
+    List info page: http://lists.example.com/listinfo/test@example.com
     Your preferences: http://example.com/aperson@example.com
     <BLANKLINE>
diff --git a/src/mailman/pipeline/docs/replybot.txt b/src/mailman/pipeline/docs/replybot.txt
index 3a6d75499..208f6aae9 100644
--- a/src/mailman/pipeline/docs/replybot.txt
+++ b/src/mailman/pipeline/docs/replybot.txt
@@ -51,7 +51,7 @@ response.
     _parsemsg           : False
     listname            : _xtest@example.com
     nodecorate          : True
-    recipients          : [u'aperson@example.com']
+    recipients          : set([u'aperson@example.com'])
     reduced_list_headers: True
     version             : 3
 
@@ -143,7 +143,7 @@ Unless the ``X-Ack:`` header has a value of ``yes``, in which case, the
     _parsemsg           : False
     listname            : _xtest@example.com
     nodecorate          : True
-    recipients          : [u'asystem@example.com']
+    recipients          : set([u'asystem@example.com'])
     reduced_list_headers: True
     version             : 3