Convert the CookHeaders tests in test_handlers to using doctests, split up

into several sub-documents.

Defaults.py.in: Removed OLD_STYLE_PREFIXING.  So-called 'new style' prefixing
is the default and only option now.

CookHeaders.py is updated to the new API and some (but not all) of the code
has been updated to more modern Python idioms.

reply_goes_to_list attribute has been changed from a strict integer to a
munepy enum called ReplyToMunging.

RFC 2369 headers List-Subscribe and List-Unsubscribe now use the preferred
-join and -leave addresses instead of the -request address with a subject
value.

4 files changed, 779 insertions, 0 deletions

diff --git a/Mailman/docs/ack-headers.txt b/Mailman/docs/ack-headers.txt
new file mode 100644
index 000000000..b6264b465
--- /dev/null
+++ b/Mailman/docs/ack-headers.txt
@@ -0,0 +1,65 @@
+Acknowledgment headers
+======================
+
+Messages that flow through the global pipeline get their headers 'cooked',
+which basically means that their headers go through several mostly unrelated
+transformations.  Some headers get added, others get changed.  Some of these
+changes depend on mailing list settings and others depend on how the message
+is getting sent through the system.  We'll take things one-by-one.
+
+    >>> from email import message_from_string
+    >>> from Mailman.Message import Message
+    >>> from Mailman.Handlers.CookHeaders import process
+    >>> from Mailman.configuration import config
+    >>> from Mailman.database import flush
+    >>> mlist = config.list_manager.create('_xtest@example.com')
+    >>> mlist.subject_prefix = u''
+    >>> flush()
+
+When the message's metadata has a 'noack' key set, an 'X-Ack: no' header is
+added.
+
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ...
+    ... A message of great import.
+    ... """, Message)
+    >>> process(mlist, msg, dict(noack=True))
+    >>> print msg.as_string()
+    From: aperson@example.com
+    X-Ack: no
+    X-BeenThere: _xtest@example.com
+    X-Mailman-Version: ...
+    Precedence: list
+    <BLANKLINE>
+    A message of great import.
+    <BLANKLINE>
+
+Any existing X-Ack header in the original message is removed.
+
+    >>> msg = message_from_string("""\
+    ... X-Ack: yes
+    ... From: aperson@example.com
+    ...
+    ... A message of great import.
+    ... """, Message)
+    >>> process(mlist, msg, dict(noack=True))
+    >>> print msg.as_string()
+    From: aperson@example.com
+    X-Ack: no
+    X-BeenThere: _xtest@example.com
+    X-Mailman-Version: ...
+    Precedence: list
+    <BLANKLINE>
+    A message of great import.
+    <BLANKLINE>
+
+
+Clean up
+--------
+
+    >>> for mlist in config.list_manager.mailing_lists:
+    ...     config.list_manager.delete(mlist)
+    >>> flush()
+    >>> list(config.list_manager.mailing_lists)
+    []
diff --git a/Mailman/docs/cook-headers.txt b/Mailman/docs/cook-headers.txt
new file mode 100644
index 000000000..cb1b07de0
--- /dev/null
+++ b/Mailman/docs/cook-headers.txt
@@ -0,0 +1,304 @@
+Cooking headers
+===============
+
+Messages that flow through the global pipeline get their headers 'cooked',
+which basically means that their headers go through several mostly unrelated
+transformations.  Some headers get added, others get changed.  Some of these
+changes depend on mailing list settings and others depend on how the message
+is getting sent through the system.  We'll take things one-by-one.
+
+    >>> from email import message_from_string
+    >>> from Mailman.Message import Message
+    >>> from Mailman.Handlers.CookHeaders import process
+    >>> from Mailman.configuration import config
+    >>> from Mailman.database import flush
+    >>> mlist = config.list_manager.create('_xtest@example.com')
+    >>> mlist.subject_prefix = u''
+    >>> mlist.include_list_post_header = False
+    >>> mlist.archive = True
+    >>> # XXX This will almost certainly change once we've worked out the web
+    >>> # space layout for mailing lists now.
+    >>> mlist._data.web_page_url = 'http://lists.example.com/'
+    >>> flush()
+
+
+Saving the original sender
+--------------------------
+
+Because the original sender headers may get deleted or changed, CookHeaders
+will place the sender in the message metadata for safe keeping.
+
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ...
+    ... A message of great import.
+    ... """, Message)
+    >>> msgdata = {}
+    >>> process(mlist, msg, msgdata)
+    >>> msgdata['original_sender']
+    'aperson@example.com'
+
+But if there was no original sender, then the empty string will be saved.
+
+    >>> msg = message_from_string("""\
+    ... Subject: No original sender
+    ...
+    ... A message of great import.
+    ... """, Message)
+    >>> msgdata = {}
+    >>> process(mlist, msg, msgdata)
+    >>> msgdata['original_sender']
+    ''
+
+
+X-BeenThere header
+------------------
+
+The X-BeenThere header is what Mailman uses to recognize messages that have
+already been processed by this mailing list.  It's one small measure against
+mail loops.
+
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ...
+    ... A message of great import.
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> msg['x-beenthere']
+    '_xtest@example.com'
+
+Mailman appends X-BeenThere headers, so if there already is one in the
+original message, the posted message will contain two such headers.
+
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ... X-BeenThere: another@example.com
+    ...
+    ... A message of great import.
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> sorted(msg.get_all('x-beenthere'))
+    ['_xtest@example.com', 'another@example.com']
+
+
+Mailman version header
+----------------------
+
+Mailman will also insert an X-Mailman-Version header...
+
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ...
+    ... A message of great import.
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> from Mailman.Version import VERSION
+    >>> msg['x-mailman-version'] == VERSION
+    True
+
+...but only if one doesn't already exist.
+
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ... X-Mailman-Version: 3000
+    ...
+    ... A message of great import.
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> from Mailman.Version import VERSION
+    >>> msg['x-mailman-version']
+    '3000'
+
+
+Precedence header
+-----------------
+
+Mailman will insert a Precedence header, which is a de-facto standard for
+telling automatic reply software (e.g. vacation(1)) not to respond to this
+message.
+
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ...
+    ... A message of great import.
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> from Mailman.Version import VERSION
+    >>> msg['precedence']
+    'list'
+
+But Mailman will only add that header if the original message doesn't already
+have one of them.
+
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ... Precedence: junk
+    ...
+    ... A message of great import.
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> from Mailman.Version import VERSION
+    >>> msg['precedence']
+    'junk'
+
+
+RFC 2919 and 2369 headers
+-------------------------
+
+This is a helper function for the following section.
+
+    >>> def list_headers(msg):
+    ...     print '---start---'
+    ...     # Sort the List-* headers found in the message.  We need to do
+    ...     # this because CookHeaders puts them in a dictionary which does
+    ...     # not have a guaranteed sort order.
+    ...     for header in sorted(msg.keys()):
+    ...         parts = header.lower().split('-')
+    ...         if 'list' not in parts:
+    ...             continue
+    ...         for value in msg.get_all(header):
+    ...             print '%s: %s' % (header, value)
+    ...     print '---end---'
+
+These RFCs define headers for mailing list actions.  A mailing list should
+generally add these headers, but not for messages that aren't crafted for a
+specific list (e.g. password reminders in Mailman 2.x).
+
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, dict(_nolist=True))
+    >>> list_headers(msg)
+    ---start---
+    ---end---
+
+Some people don't like these headers because their mail readers aren't good
+about hiding them.  A list owner can turn these headers off.
+
+    >>> mlist.include_rfc2369_headers = False
+    >>> flush()
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> list_headers(msg)
+    ---start---
+    ---end---
+
+But normally, a list will include these headers.
+
+    >>> mlist.include_rfc2369_headers = True
+    >>> mlist.include_list_post_header = True
+    >>> mlist.preferred_language = 'en'
+    >>> flush()
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> list_headers(msg)
+    ---start---
+    List-Archive: <http://www.example.com/pipermail/_xtest@example.com>
+    List-Help: <mailto:_xtest-request@example.com?subject=help>
+    List-Id: <_xtest.example.com>
+    List-Post: <mailto:_xtest@example.com>
+    List-Subscribe: <http://lists.example.com/listinfo/_xtest@example.com>,
+    	<mailto:_xtest-join@example.com>
+    List-Unsubscribe: <http://lists.example.com/listinfo/_xtest@example.com>,
+    	<mailto:_xtest-leave@example.com>
+    ---end---
+
+If the mailing list has a description, then it is included in the List-Id
+header.
+
+    >>> mlist.description = 'My test mailing list'
+    >>> flush()
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> list_headers(msg)
+    ---start---
+    List-Archive: <http://www.example.com/pipermail/_xtest@example.com>
+    List-Help: <mailto:_xtest-request@example.com?subject=help>
+    List-Id: My test mailing list <_xtest.example.com>
+    List-Post: <mailto:_xtest@example.com>
+    List-Subscribe: <http://lists.example.com/listinfo/_xtest@example.com>,
+    	<mailto:_xtest-join@example.com>
+    List-Unsubscribe: <http://lists.example.com/listinfo/_xtest@example.com>,
+    	<mailto:_xtest-leave@example.com>
+    ---end---
+
+Administrative messages crafted by Mailman will have a reduced set of headers.
+
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, dict(reduced_list_headers=True))
+    >>> list_headers(msg)
+    ---start---
+    List-Help: <mailto:_xtest-request@example.com?subject=help>
+    List-Id: My test mailing list <_xtest.example.com>
+    List-Subscribe: <http://lists.example.com/listinfo/_xtest@example.com>,
+    	<mailto:_xtest-join@example.com>
+    List-Unsubscribe: <http://lists.example.com/listinfo/_xtest@example.com>,
+    	<mailto:_xtest-leave@example.com>
+    X-List-Administrivia: yes
+    ---end---
+
+With the normal set of List-* headers, it's still possible to suppress the
+List-Post header, which is reasonable for an announce only mailing list.
+
+    >>> mlist.include_list_post_header = False
+    >>> flush()
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> list_headers(msg)
+    ---start---
+    List-Archive: <http://www.example.com/pipermail/_xtest@example.com>
+    List-Help: <mailto:_xtest-request@example.com?subject=help>
+    List-Id: My test mailing list <_xtest.example.com>
+    List-Subscribe: <http://lists.example.com/listinfo/_xtest@example.com>,
+    	<mailto:_xtest-join@example.com>
+    List-Unsubscribe: <http://lists.example.com/listinfo/_xtest@example.com>,
+    	<mailto:_xtest-leave@example.com>
+    ---end---
+
+And if the list isn't being archived, it makes no sense to add the
+List-Archive header either.
+
+    >>> mlist.include_list_post_header = True
+    >>> mlist.archive = False
+    >>> flush()
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> list_headers(msg)
+    ---start---
+    List-Help: <mailto:_xtest-request@example.com?subject=help>
+    List-Id: My test mailing list <_xtest.example.com>
+    List-Post: <mailto:_xtest@example.com>
+    List-Subscribe: <http://lists.example.com/listinfo/_xtest@example.com>,
+    	<mailto:_xtest-join@example.com>
+    List-Unsubscribe: <http://lists.example.com/listinfo/_xtest@example.com>,
+    	<mailto:_xtest-leave@example.com>
+    ---end---
+
+
+Clean up
+--------
+
+    >>> for mlist in config.list_manager.mailing_lists:
+    ...     config.list_manager.delete(mlist)
+    >>> flush()
+    >>> list(config.list_manager.mailing_lists)
+    []
diff --git a/Mailman/docs/reply-to.txt b/Mailman/docs/reply-to.txt
new file mode 100644
index 000000000..43f2fcc06
--- /dev/null
+++ b/Mailman/docs/reply-to.txt
@@ -0,0 +1,148 @@
+Reply-to munging
+================
+
+Messages that flow through the global pipeline get their headers 'cooked',
+which basically means that their headers go through several mostly unrelated
+transformations.  Some headers get added, others get changed.  Some of these
+changes depend on mailing list settings and others depend on how the message
+is getting sent through the system.  We'll take things one-by-one.
+
+    >>> from email import message_from_string
+    >>> from Mailman.Message import Message
+    >>> from Mailman.Handlers.CookHeaders import process
+    >>> from Mailman.configuration import config
+    >>> from Mailman.database import flush
+    >>> mlist = config.list_manager.create('_xtest@example.com')
+    >>> mlist.subject_prefix = u''
+    >>> flush()
+
+Reply-to munging refers to the behavior where a mailing list can be configured
+to change or augment an existing Reply-To header in a message posted to the
+list.  Reply-to munging is fairly controversial, with arguments made either
+for or against munging.
+
+The Mailman developers, and I believe the majority consensus is to do no
+Reply-to munging, under several principles.  Primarily, most reply-to munging
+is requested by people who do not have both a Reply and Reply All button on
+their mail reader.  If you do not munge Reply-To, then these buttons will work
+properly, but if you munge the header, it is impossible for these buttons to
+work right, because both will reply to the list.  This leads to unfortunate
+accidents where a private message is accidentally posted to the entire list.
+
+However, Mailman gives list owners the option to do Reply-To munging anyway,
+mostly as a way to shut up the really vocal minority who seem to insist on
+this mis-feature.
+
+
+Reply to list
+-------------
+
+A list can be configured to add a Reply-To header pointing back to the mailing
+list's posting address.  If there's no Reply-To header in the original
+message, the list's posting address simply gets inserted.
+
+    >>> from Mailman.constants import ReplyToMunging
+    >>> mlist.reply_goes_to_list = ReplyToMunging.point_to_list
+    >>> mlist.preferred_language = 'en'
+    >>> mlist.description = ''
+    >>> flush()
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> len(msg.get_all('reply-to'))
+    1
+    >>> msg['reply-to']
+    '_xtest@example.com'
+
+It's also possible to strip any existing Reply-To header first, before adding
+the list's posting address.
+
+    >>> mlist.first_strip_reply_to = True
+    >>> flush()
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ... Reply-To: bperson@example.com
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> len(msg.get_all('reply-to'))
+    1
+    >>> msg['reply-to']
+    '_xtest@example.com'
+
+If you don't first strip the header, then the list's posting address will just
+get appended to whatever the original version was.
+
+    >>> mlist.first_strip_reply_to = False
+    >>> flush()
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ... Reply-To: bperson@example.com
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> len(msg.get_all('reply-to'))
+    1
+    >>> msg['reply-to']
+    'bperson@example.com, _xtest@example.com'
+
+
+Explicit Reply-To
+-----------------
+
+The list can also be configured to have an explicit Reply-To header.
+
+    >>> mlist.reply_goes_to_list = ReplyToMunging.explicit_header
+    >>> mlist.reply_to_address = 'my-list@example.com'
+    >>> flush()
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> len(msg.get_all('reply-to'))
+    1
+    >>> msg['reply-to']
+    'my-list@example.com'
+
+And as before, it's possible to either strip any existing Reply-To header...
+
+    >>> mlist.first_strip_reply_to = True
+    >>> flush()
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ... Reply-To: bperson@example.com
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> len(msg.get_all('reply-to'))
+    1
+    >>> msg['reply-to']
+    'my-list@example.com'
+
+...or not.
+
+    >>> mlist.first_strip_reply_to = False
+    >>> flush()
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ... Reply-To: bperson@example.com
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> len(msg.get_all('reply-to'))
+    1
+    >>> msg['reply-to']
+    'my-list@example.com, bperson@example.com'
+
+
+Clean up
+--------
+
+    >>> for mlist in config.list_manager.mailing_lists:
+    ...     config.list_manager.delete(mlist)
+    >>> flush()
+    >>> list(config.list_manager.mailing_lists)
+    []
diff --git a/Mailman/docs/subject-munging.txt b/Mailman/docs/subject-munging.txt
new file mode 100644
index 000000000..921efc889
--- /dev/null
+++ b/Mailman/docs/subject-munging.txt
@@ -0,0 +1,262 @@
+Subject munging
+===============
+
+Messages that flow through the global pipeline get their headers 'cooked',
+which basically means that their headers go through several mostly unrelated
+transformations.  Some headers get added, others get changed.  Some of these
+changes depend on mailing list settings and others depend on how the message
+is getting sent through the system.  We'll take things one-by-one.
+
+    >>> from email import message_from_string
+    >>> from Mailman.Message import Message
+    >>> from Mailman.Handlers.CookHeaders import process
+    >>> from Mailman.configuration import config
+    >>> from Mailman.database import flush
+    >>> mlist = config.list_manager.create('_xtest@example.com')
+    >>> mlist.subject_prefix = u''
+    >>> flush()
+
+
+Inserting a prefix
+------------------
+
+Another thing CookHeaders does is 'munge' the Subject header by inserting the
+subject prefix for the list at the front.  If there's no subject header in the
+original message, Mailman uses a canned default.  In order to do subject
+munging, a mailing list must have a preferred language.
+
+    >>> mlist.subject_prefix = u'[XTest] '
+    >>> mlist.preferred_language = u'en'
+    >>> flush()
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ...
+    ... A message of great import.
+    ... """, Message)
+    >>> msgdata = {}
+    >>> process(mlist, msg, msgdata)
+
+The original subject header is stored in the message metadata.  We must print
+the new Subject header because it gets converted from a string to an
+email.header.Header instance which has an unhelpful repr.
+
+    >>> msgdata['origsubj']
+    ''
+    >>> print msg['subject']
+    [XTest] (no subject)
+
+If the original message had a Subject header, then the prefix is inserted at
+the beginning of the header's value.
+
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ... Subject: Something important
+    ...
+    ... A message of great import.
+    ... """, Message)
+    >>> msgdata = {}
+    >>> process(mlist, msg, msgdata)
+    >>> msgdata['origsubj']
+    'Something important'
+    >>> print msg['subject']
+    [XTest] Something important
+
+Subject headers are not munged for digest messages.
+    
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ... Subject: Something important
+    ...
+    ... A message of great import.
+    ... """, Message)
+    >>> process(mlist, msg, dict(isdigest=True))
+    >>> msg['subject']
+    'Something important'
+
+Nor are they munged for 'fast tracked' messages, which are generally defined
+as messages that Mailman crafts internally.
+
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ... Subject: Something important
+    ...
+    ... A message of great import.
+    ... """, Message)
+    >>> process(mlist, msg, dict(_fasttrack=True))
+    >>> msg['subject']
+    'Something important'
+
+If a Subject header already has a prefix, usually following a Re: marker,
+another one will not be added but the prefix will be moved to the front of the
+header text.
+
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ... Subject: Re: [XTest] Something important
+    ...
+    ... A message of great import.
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> print msg['subject']
+    [XTest] Re: Something important
+
+If the Subjec header has a prefix at the front of the header text, that's
+where it will stay.  This is called 'new style' prefixing and is the only
+option available in Mailman 3.
+
+    >>> msg = message_from_string("""\
+    ... From: aperson@example.com
+    ... Subject: [XTest] Re: Something important
+    ...
+    ... A message of great import.
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> print msg['subject']
+    [XTest] Re: Something important
+
+
+Internationalized headers
+-------------------------
+
+Internationalization adds some interesting twists to the handling of subject
+prefixes.  Part of what makes this interesting is the encoding of i18n headers
+using RFC 2047, and lists whose preferred language is in a different character
+set than the encoded header.
+
+    >>> msg = message_from_string("""\
+    ... Subject: =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> print msg['subject']
+    [XTest] =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
+    >>> unicode(msg['subject'])
+    u'[XTest] \u30e1\u30fc\u30eb\u30de\u30f3'
+
+
+Prefix numbers
+--------------
+
+Subject prefixes support a placeholder for the numeric post id.  Every time a
+message is posted to the mailing list, a 'post id' gets incremented.  This is
+a purely sequential integer that increases monotonically.  By added a '%d'
+placeholder to the subject prefix, this post id can be included in the prefix.
+
+    >>> mlist.subject_prefix = '[XTest %d] '
+    >>> mlist.post_id = 456
+    >>> flush()
+    >>> msg = message_from_string("""\
+    ... Subject: Something important
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> print msg['subject']
+    [XTest 456] Something important
+
+This works even when the message is a reply, except that in this case, the
+numeric post id in the generated subject prefix is updated with the new post
+id.
+
+    >>> msg = message_from_string("""\
+    ... Subject: [XTest 123] Re: Something important
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> print msg['subject']
+    [XTest 456] Re: Something important
+
+If the Subject header had old style prefixing, the prefix is moved to the
+front of the header text.
+
+    >>> msg = message_from_string("""\
+    ... Subject: Re: [XTest 123] Something important
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> print msg['subject']
+    [XTest 456] Re: Something important
+
+
+And of course, the proper thing is done when posting id numbers are included
+in the subject prefix, and the subject is encoded non-ascii.
+
+    >>> msg = message_from_string("""\
+    ... Subject: =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> print msg['subject']
+    [XTest 456] =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
+    >>> unicode(msg['subject'])
+    u'[XTest 456] \u30e1\u30fc\u30eb\u30de\u30f3'
+
+Even more fun is when the i18n Subject header already has a prefix, possibly
+with a different posting number.
+
+    >>> msg = message_from_string("""\
+    ... Subject: [XTest 123] Re: =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> print msg['subject']
+    [XTest 456] Re: =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
+
+# XXX This requires Python email patch #1681333 to succeed.
+#    >>> unicode(msg['subject'])
+#    u'[XTest 456] Re: \u30e1\u30fc\u30eb\u30de\u30f3'
+
+As before, old style subject prefixes are re-ordered.
+
+    >>> msg = message_from_string("""\
+    ... Subject: Re: [XTest 123] =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> print msg['subject']
+    [XTest 456] Re:
+      =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
+
+# XXX This requires Python email patch #1681333 to succeed.
+#    >>> unicode(msg['subject'])
+#    u'[XTest 456] Re: \u30e1\u30fc\u30eb\u30de\u30f3'
+
+
+In this test case, we get an extra space between the prefix and the original
+subject.  It's because the original is 'crooked'.  Note that a Subject
+starting with '\n ' is generated by some version of Eudora Japanese edition.
+
+    >>> mlist.subject_prefix = '[XTest] '
+    >>> flush()
+    >>> msg = message_from_string("""\
+    ... Subject:
+    ...  Important message
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+    >>> print msg['subject']
+    [XTest]  Important message
+
+And again, with an RFC 2047 encoded header.
+
+    >>> msg = message_from_string("""\
+    ... Subject:
+    ...  =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
+    ...
+    ... """, Message)
+    >>> process(mlist, msg, {})
+
+# XXX This one does not appear to work the same way as
+# test_subject_munging_prefix_crooked() in the old Python-based tests.  I need
+# to get Tokio to look at this.
+#    >>> print msg['subject']
+#    [XTest] =?iso-2022-jp?b?IBskQiVhITwlayVeJXMbKEI=?=
+
+
+Clean up
+--------
+
+    >>> for mlist in config.list_manager.mailing_lists:
+    ...     config.list_manager.delete(mlist)
+    >>> flush()
+    >>> list(config.list_manager.mailing_lists)
+    []