diff options
| author | Barry Warsaw | 2011-05-29 12:45:19 -0400 |
|---|---|---|
| committer | Barry Warsaw | 2011-05-29 12:45:19 -0400 |
| commit | 521a179d309fac857fdbbe162d5db136c3ec3b1e (patch) | |
| tree | ec6e635e9c0f8a5bd655a254f9c346f1acb6dd8e /src/mailman/queue/docs | |
| parent | 0f760798fb2490a03041c42018afbd59749e6cbd (diff) | |
| download | mailman-521a179d309fac857fdbbe162d5db136c3ec3b1e.tar.gz mailman-521a179d309fac857fdbbe162d5db136c3ec3b1e.tar.zst mailman-521a179d309fac857fdbbe162d5db136c3ec3b1e.zip | |
Rename 'queue' directory to 'runners' since not all subprocesses run a queue.
Diffstat (limited to 'src/mailman/queue/docs')
| -rw-r--r-- | src/mailman/queue/docs/OVERVIEW.txt | 80 | ||||
| -rw-r--r-- | src/mailman/queue/docs/archiver.txt | 35 | ||||
| -rw-r--r-- | src/mailman/queue/docs/command.txt | 289 | ||||
| -rw-r--r-- | src/mailman/queue/docs/digester.txt | 604 | ||||
| -rw-r--r-- | src/mailman/queue/docs/incoming.txt | 263 | ||||
| -rw-r--r-- | src/mailman/queue/docs/lmtp.txt | 313 | ||||
| -rw-r--r-- | src/mailman/queue/docs/news.txt | 161 | ||||
| -rw-r--r-- | src/mailman/queue/docs/outgoing.txt | 413 | ||||
| -rw-r--r-- | src/mailman/queue/docs/rest.txt | 25 | ||||
| -rw-r--r-- | src/mailman/queue/docs/runner.txt | 73 | ||||
| -rw-r--r-- | src/mailman/queue/docs/switchboard.txt | 187 |
11 files changed, 0 insertions, 2443 deletions
diff --git a/src/mailman/queue/docs/OVERVIEW.txt b/src/mailman/queue/docs/OVERVIEW.txt deleted file mode 100644 index 41ccc18c9..000000000 --- a/src/mailman/queue/docs/OVERVIEW.txt +++ /dev/null @@ -1,80 +0,0 @@ -============== -Alias Overview -============== - -A typical Mailman list exposes nine aliases which point to seven different -wrapped scripts. E.g. for a list named ``mylist``, you'd have:: - - mylist-bounces -> bounces - mylist-confirm -> confirm - mylist-join -> join (-subscribe is an alias) - mylist-leave -> leave (-unsubscribe is an alias) - mylist-owner -> owner - mylist -> post - mylist-request -> request - -``-request``, ``-join``, and ``-leave`` are a robot addresses; their sole -purpose is to process emailed commands, although the latter two are hardcoded -to subscription and unsubscription requests. ``-bounces`` is the automated -bounce processor, and all messages to list members have their return address -set to ``-bounces``. If the bounce processor fails to extract a bouncing -member address, it can optionally forward the message on to the list owners. - -``-owner`` is for reaching a human operator with minimal list interaction -(i.e. no bounce processing). ``-confirm`` is another robot address which -processes replies to VERP-like confirmation notices. - -So delivery flow of messages look like this:: - - joerandom ---> mylist ---> list members - | | - | |[bounces] - | mylist-bounces <---+ <-------------------------------+ - | | | - | +--->[internal bounce processing] | - | ^ | | - | | | [bounce found] | - | [bounces *] +--->[register and discard] | - | | | | | - | | | |[*] | - | [list owners] |[no bounce found] | | - | ^ | | | - | | | | | - +-------> mylist-owner <--------+ | | - | | | - | data/owner-bounces.mbox <--[site list] <---+ | - | | - +-------> mylist-join--+ | - | | | - +------> mylist-leave--+ | - | | | - | v | - +-------> mylist-request | - | | | - | +---> [command processor] | - | | | - +-----> mylist-confirm ----> +---> joerandom | - | | - |[bounces] | - +----------------------+ - -A person can send an email to the list address (for posting), the ``-owner`` -address (to reach the human operator), or the ``-confirm``, ``-join``, -``-leave``, and ``-request`` mailbots. Message to the list address are then -forwarded on to the list membership, with bounces directed to the -bounces -address. - -[*] Messages sent to the ``-owner`` address are forwarded on to the list -owner/moderators. All ``-owner`` destined messages have their bounces -directed to the site list ``-bounces`` address, regardless of whether a human -sent the message or the message was crafted internally. The intention here is -that the site owners want to be notified when one of their list owners' -addresses starts bouncing (yes, the will be automated in a future release). - -Any messages to site owners has their bounces directed to a special *loop -killer* address, which just dumps the message into -``data/owners-bounces.mbox``. - -Finally, message to any of the mailbots causes the requested action to be -performed. Results notifications are sent to the author of the message, which -all bounces pointing back to the -bounces address. diff --git a/src/mailman/queue/docs/archiver.txt b/src/mailman/queue/docs/archiver.txt deleted file mode 100644 index cdee449e1..000000000 --- a/src/mailman/queue/docs/archiver.txt +++ /dev/null @@ -1,35 +0,0 @@ -========= -Archiving -========= - -Mailman can archive to any number of archivers that adhere to the -``IArchiver`` interface. By default, there's a Pipermail archiver. -:: - - >>> mlist = create_list('test@example.com') - >>> transaction.commit() - - >>> msg = message_from_string("""\ - ... From: aperson@example.com - ... To: test@example.com - ... Subject: My first post - ... Message-ID: <first> - ... - ... First post! - ... """) - - >>> archiver_queue = config.switchboards['archive'] - >>> ignore = archiver_queue.enqueue(msg, {}, listname=mlist.fqdn_listname) - - >>> from mailman.queue.archive import ArchiveRunner - >>> from mailman.testing.helpers import make_testable_runner - >>> runner = make_testable_runner(ArchiveRunner) - >>> runner.run() - - # The best we can do is verify some landmark exists. Let's use the - # Pipermail pickle file exists. - >>> listname = mlist.fqdn_listname - >>> import os - >>> os.path.exists(os.path.join( - ... config.PUBLIC_ARCHIVE_FILE_DIR, listname, 'pipermail.pck')) - True diff --git a/src/mailman/queue/docs/command.txt b/src/mailman/queue/docs/command.txt deleted file mode 100644 index c767e6f5f..000000000 --- a/src/mailman/queue/docs/command.txt +++ /dev/null @@ -1,289 +0,0 @@ -======================== -The command queue runner -======================== - -This queue runner's purpose is to process and respond to email commands. -Commands are extensible using the Mailman plug-in system, but Mailman comes -with a number of email commands out of the box. These are processed when a -message is sent to the list's ``-request`` address. - - >>> mlist = create_list('test@example.com') - - -A command in the Subject -======================== - -For example, the ``echo`` command simply echoes the original command back to -the sender. The command can be in the ``Subject`` header. -:: - - >>> msg = message_from_string("""\ - ... From: aperson@example.com - ... To: test-request@example.com - ... Subject: echo hello - ... Message-ID: <aardvark> - ... - ... """) - - >>> from mailman.inject import inject_message - >>> inject_message(mlist, msg, switchboard='command') - >>> from mailman.queue.command import CommandRunner - >>> from mailman.testing.helpers import make_testable_runner - >>> command = make_testable_runner(CommandRunner) - >>> command.run() - -And now the response is in the ``virgin`` queue. - - >>> from mailman.queue import Switchboard - >>> virgin_queue = config.switchboards['virgin'] - >>> len(virgin_queue.files) - 1 - >>> from mailman.testing.helpers import get_queue_messages - >>> item = get_queue_messages('virgin')[0] - >>> print item.msg.as_string() - Subject: The results of your email commands - From: test-bounces@example.com - To: aperson@example.com - ... - <BLANKLINE> - The results of your email command are provided below. - <BLANKLINE> - - Original message details: - From: aperson@example.com - Subject: echo hello - Date: ... - Message-ID: <aardvark> - <BLANKLINE> - - Results: - echo hello - <BLANKLINE> - - Done. - <BLANKLINE> - >>> dump_msgdata(item.msgdata) - _parsemsg : False - listname : test@example.com - nodecorate : True - recipients : set([u'aperson@example.com']) - reduced_list_headers: True - version : ... - - -A command in the body -===================== - -The command can also be found in the body of the message, as long as the -message is plain text. -:: - - >>> msg = message_from_string("""\ - ... From: bperson@example.com - ... To: test-request@example.com - ... Message-ID: <bobcat> - ... - ... echo foo bar - ... """) - - >>> inject_message(mlist, msg, switchboard='command') - >>> command.run() - >>> len(virgin_queue.files) - 1 - >>> item = get_queue_messages('virgin')[0] - >>> print item.msg.as_string() - Subject: The results of your email commands - From: test-bounces@example.com - To: bperson@example.com - ... - Precedence: bulk - <BLANKLINE> - The results of your email command are provided below. - <BLANKLINE> - - Original message details: - From: bperson@example.com - Subject: n/a - Date: ... - Message-ID: <bobcat> - <BLANKLINE> - - Results: - echo foo bar - <BLANKLINE> - - Done. - <BLANKLINE> - - -Implicit commands -================= - -For some commands, specifically for joining and leaving a mailing list, there -are email aliases that act like commands, even when there's nothing else in -the ``Subject`` or body. For example, to join a mailing list, a user need -only email the ``-join`` address or ``-subscribe`` address (the latter is -deprecated). - -Because Dirk has never registered with Mailman before, he gets two responses. -The first is a confirmation message so that Dirk can validate his email -address, and the other is the results of his email command. -:: - - >>> msg = message_from_string("""\ - ... From: Dirk Person <dperson@example.com> - ... To: test-join@example.com - ... - ... """) - - >>> inject_message(mlist, msg, switchboard='command', subaddress='join') - >>> command.run() - >>> len(virgin_queue.files) - 2 - - >>> def sortkey(item): - ... return str(item.msg['subject']) - >>> messages = sorted(get_queue_messages('virgin'), key=sortkey) - - >>> from mailman.interfaces.registrar import IRegistrar - >>> from zope.component import getUtility - >>> registrar = getUtility(IRegistrar) - >>> for item in messages: - ... subject = item.msg['subject'] - ... print 'Subject:', subject - ... if 'confirm' in str(subject): - ... token = str(subject).split()[1].strip() - ... status = registrar.confirm(token) - ... assert status, 'Confirmation failed' - Subject: The results of your email commands - Subject: confirm ... - -Similarly, to leave a mailing list, the user need only email the ``-leave`` or -``-unsubscribe`` address (the latter is deprecated). -:: - - >>> msg = message_from_string("""\ - ... From: dperson@example.com - ... To: test-leave@example.com - ... - ... """) - - >>> inject_message(mlist, msg, switchboard='command', subaddress='leave') - >>> command.run() - >>> len(virgin_queue.files) - 1 - >>> item = get_queue_messages('virgin')[0] - >>> print item.msg.as_string() - Subject: The results of your email commands - From: test-bounces@example.com - To: dperson@example.com - ... - <BLANKLINE> - The results of your email command are provided below. - <BLANKLINE> - - Original message details: - From: dperson@example.com - Subject: n/a - Date: ... - Message-ID: ... - <BLANKLINE> - - Results: - Dirk Person <dperson@example.com> left test@example.com - <BLANKLINE> - - Done. - <BLANKLINE> - -The ``-confirm`` address is also available as an implicit command. -:: - - >>> msg = message_from_string("""\ - ... From: dperson@example.com - ... To: test-confirm+123@example.com - ... - ... """) - - >>> inject_message(mlist, msg, switchboard='command', subaddress='confirm') - >>> command.run() - >>> len(virgin_queue.files) - 1 - >>> item = get_queue_messages('virgin')[0] - >>> print item.msg.as_string() - Subject: The results of your email commands - From: test-bounces@example.com - To: dperson@example.com - ... - <BLANKLINE> - The results of your email command are provided below. - <BLANKLINE> - - Original message details: - From: dperson@example.com - Subject: n/a - Date: ... - Message-ID: ... - <BLANKLINE> - - Results: - Confirmation token did not match - <BLANKLINE> - - Done. - <BLANKLINE> - - -Stopping command processing -=========================== - -The ``end`` command stops email processing, so that nothing following is -looked at by the command queue. -:: - - >>> msg = message_from_string("""\ - ... From: cperson@example.com - ... To: test-request@example.com - ... Message-ID: <caribou> - ... - ... echo foo bar - ... end ignored - ... echo baz qux - ... """) - - >>> inject_message(mlist, msg, switchboard='command') - >>> command.run() - >>> len(virgin_queue.files) - 1 - >>> item = get_queue_messages('virgin')[0] - >>> print item.msg.as_string() - Subject: The results of your email commands - ... - <BLANKLINE> - - Results: - echo foo bar - <BLANKLINE> - - Unprocessed: - echo baz qux - <BLANKLINE> - - Done. - <BLANKLINE> - -The ``stop`` command is an alias for ``end``. -:: - - >>> msg = message_from_string("""\ - ... From: cperson@example.com - ... To: test-request@example.com - ... Message-ID: <caribou> - ... - ... echo foo bar - ... stop ignored - ... echo baz qux - ... """) - - >>> inject_message(mlist, msg, switchboard='command') - >>> command.run() - >>> len(virgin_queue.files) - 1 - >>> item = get_queue_messages('virgin')[0] - >>> print item.msg.as_string() - Subject: The results of your email commands - ... - <BLANKLINE> - - Results: - echo foo bar - <BLANKLINE> - - Unprocessed: - echo baz qux - <BLANKLINE> - - Done. - <BLANKLINE> diff --git a/src/mailman/queue/docs/digester.txt b/src/mailman/queue/docs/digester.txt deleted file mode 100644 index 285b2072a..000000000 --- a/src/mailman/queue/docs/digester.txt +++ /dev/null @@ -1,604 +0,0 @@ -========= -Digesting -========= - -Mailman crafts and sends digests by a separate digest queue runner process. -This starts by a number of messages being posted to the mailing list. -:: - - >>> mlist = create_list('test@example.com') - >>> mlist.digest_size_threshold = 0.6 - >>> mlist.volume = 1 - >>> mlist.next_digest_number = 1 - - >>> from string import Template - >>> process = config.handlers['to-digest'].process - - >>> def fill_digest(): - ... size = 0 - ... for i in range(1, 5): - ... text = Template("""\ - ... From: aperson@example.com - ... To: xtest@example.com - ... Subject: Test message $i - ... List-Post: <test@example.com> - ... - ... Here is message $i - ... """).substitute(i=i) - ... msg = message_from_string(text) - ... process(mlist, msg, {}) - ... size += len(text) - ... if size >= mlist.digest_size_threshold * 1024: - ... break - - >>> fill_digest() - -The queue runner gets kicked off when a marker message gets dropped into the -digest queue. The message metadata points to the mailbox file containing the -messages to put in the digest. -:: - - >>> digestq = config.switchboards['digest'] - >>> len(digestq.files) - 1 - - >>> from mailman.testing.helpers import get_queue_messages - >>> entry = get_queue_messages('digest')[0] - -The marker message is empty. - - >>> print entry.msg.as_string() - -But the message metadata has a reference to the digest file. -:: - - >>> dump_msgdata(entry.msgdata) - _parsemsg : False - digest_number: 1 - digest_path : .../lists/test@example.com/digest.1.1.mmdf - listname : test@example.com - version : 3 - volume : 1 - - # Put the messages back in the queue for the runner to handle. - >>> filebase = digestq.enqueue(entry.msg, entry.msgdata) - -There are 4 messages in the digest. - - >>> from mailman.utilities.mailbox import Mailbox - >>> sum(1 for item in Mailbox(entry.msgdata['digest_path'])) - 4 - -When the queue runner runs, it processes the digest mailbox, crafting both the -plain text (RFC 1153) digest and the MIME digest. - - >>> from mailman.queue.digest import DigestRunner - >>> from mailman.testing.helpers import make_testable_runner - >>> runner = make_testable_runner(DigestRunner) - >>> runner.run() - -The digest runner places both digests into the virgin queue for final -delivery. - - >>> messages = get_queue_messages('virgin') - >>> len(messages) - 2 - -The MIME digest is a multipart, and the RFC 1153 digest is the other one. -:: - - >>> def mime_rfc1153(messages): - ... if messages[0].msg.is_multipart(): - ... return messages[0], messages[1] - ... return messages[1], messages[0] - - >>> mime, rfc1153 = mime_rfc1153(messages) - -The MIME digest has lots of good stuff, all contained in the multipart. - - >>> print mime.msg.as_string() - Content-Type: multipart/mixed; boundary="===============...==" - MIME-Version: 1.0 - From: test-request@example.com - Subject: Test Digest, Vol 1, Issue 1 - To: test@example.com - Reply-To: test@example.com - Date: ... - Message-ID: ... - <BLANKLINE> - --===============...== - Content-Type: text/plain; charset="us-ascii" - MIME-Version: 1.0 - Content-Transfer-Encoding: 7bit - Content-Description: Test Digest, Vol 1, Issue 1 - <BLANKLINE> - Send Test mailing list submissions to - test@example.com - <BLANKLINE> - To subscribe or unsubscribe via the World Wide Web, visit - http://lists.example.com/listinfo/test@example.com - or, via email, send a message with subject or body 'help' to - test-request@example.com - <BLANKLINE> - You can reach the person managing the list at - test-owner@example.com - <BLANKLINE> - When replying, please edit your Subject line so it is more specific - than "Re: Contents of Test digest..." - <BLANKLINE> - --===============...== - Content-Type: text/plain; charset="us-ascii" - MIME-Version: 1.0 - Content-Transfer-Encoding: 7bit - Content-Description: Today's Topics (4 messages) - <BLANKLINE> - Today's Topics: - <BLANKLINE> - 1. Test message 1 (aperson@example.com) - 2. Test message 2 (aperson@example.com) - 3. Test message 3 (aperson@example.com) - 4. Test message 4 (aperson@example.com) - <BLANKLINE> - --===============...== - Content-Type: message/rfc822 - MIME-Version: 1.0 - <BLANKLINE> - From: aperson@example.com - To: xtest@example.com - Subject: Test message 1 - List-Post: <test@example.com> - <BLANKLINE> - Here is message 1 - <BLANKLINE> - --===============...== - Content-Type: message/rfc822 - MIME-Version: 1.0 - <BLANKLINE> - From: aperson@example.com - To: xtest@example.com - Subject: Test message 2 - List-Post: <test@example.com> - <BLANKLINE> - Here is message 2 - <BLANKLINE> - --===============...== - Content-Type: message/rfc822 - MIME-Version: 1.0 - <BLANKLINE> - From: aperson@example.com - To: xtest@example.com - Subject: Test message 3 - List-Post: <test@example.com> - <BLANKLINE> - Here is message 3 - <BLANKLINE> - --===============...== - Content-Type: message/rfc822 - MIME-Version: 1.0 - <BLANKLINE> - From: aperson@example.com - To: xtest@example.com - Subject: Test message 4 - List-Post: <test@example.com> - <BLANKLINE> - Here is message 4 - <BLANKLINE> - --===============...== - Content-Type: text/plain; charset="us-ascii" - MIME-Version: 1.0 - Content-Transfer-Encoding: 7bit - Content-Description: Digest Footer - <BLANKLINE> - _______________________________________________ - Test mailing list - test@example.com - http://lists.example.com/listinfo/test@example.com - <BLANKLINE> - --===============...==-- - -The RFC 1153 contains the digest in a single plain text message. - - >>> print rfc1153.msg.as_string() - From: test-request@example.com - Subject: Test Digest, Vol 1, Issue 1 - To: test@example.com - Reply-To: test@example.com - Date: ... - Message-ID: ... - MIME-Version: 1.0 - Content-Type: text/plain; charset="us-ascii" - Content-Transfer-Encoding: 7bit - <BLANKLINE> - Send Test mailing list submissions to - test@example.com - <BLANKLINE> - To subscribe or unsubscribe via the World Wide Web, visit - http://lists.example.com/listinfo/test@example.com - or, via email, send a message with subject or body 'help' to - test-request@example.com - <BLANKLINE> - You can reach the person managing the list at - test-owner@example.com - <BLANKLINE> - When replying, please edit your Subject line so it is more specific - than "Re: Contents of Test digest..." - <BLANKLINE> - <BLANKLINE> - Today's Topics: - <BLANKLINE> - 1. Test message 1 (aperson@example.com) - 2. Test message 2 (aperson@example.com) - 3. Test message 3 (aperson@example.com) - 4. Test message 4 (aperson@example.com) - <BLANKLINE> - <BLANKLINE> - ---------------------------------------------------------------------- - <BLANKLINE> - From: aperson@example.com - Subject: Test message 1 - To: xtest@example.com - Message-ID: ... - <BLANKLINE> - Here is message 1 - <BLANKLINE> - ------------------------------ - <BLANKLINE> - From: aperson@example.com - Subject: Test message 2 - To: xtest@example.com - Message-ID: ... - <BLANKLINE> - Here is message 2 - <BLANKLINE> - ------------------------------ - <BLANKLINE> - From: aperson@example.com - Subject: Test message 3 - To: xtest@example.com - Message-ID: ... - <BLANKLINE> - Here is message 3 - <BLANKLINE> - ------------------------------ - <BLANKLINE> - From: aperson@example.com - Subject: Test message 4 - To: xtest@example.com - Message-ID: ... - <BLANKLINE> - Here is message 4 - <BLANKLINE> - ------------------------------ - <BLANKLINE> - _______________________________________________ - Test mailing list - test@example.com - http://lists.example.com/listinfo/test@example.com - <BLANKLINE> - <BLANKLINE> - End of Test Digest, Vol 1, Issue 1 - ********************************** - <BLANKLINE> - - -Internationalized digests -========================= - -When messages come in with a content-type character set different than that of -the list's preferred language, recipients will get an internationalized -digest. French is not enabled by default site-wide, so enable that now. -:: - - # Simulate the site administrator setting the default server language to - # French in the configuration file. Without this, the English template - # will be found and the masthead won't be translated. - >>> config.push('french', """ - ... [mailman] - ... default_language: fr - ... """) - - >>> mlist.preferred_language = 'fr' - >>> msg = message_from_string("""\ - ... From: aperson@example.org - ... To: test@example.com - ... Subject: =?iso-2022-jp?b?GyRCMGxIVhsoQg==?= - ... MIME-Version: 1.0 - ... Content-Type: text/plain; charset=iso-2022-jp - ... Content-Transfer-Encoding: 7bit - ... - ... \x1b$B0lHV\x1b(B - ... """) - -Set the digest threshold to zero so that the digests will be sent immediately. - - >>> mlist.digest_size_threshold = 0 - >>> process(mlist, msg, {}) - -The marker message is sitting in the digest queue. - - >>> len(digestq.files) - 1 - >>> entry = get_queue_messages('digest')[0] - >>> dump_msgdata(entry.msgdata) - _parsemsg : False - digest_number: 2 - digest_path : .../lists/test@example.com/digest.1.2.mmdf - listname : test@example.com - version : 3 - volume : 1 - -The digest queue runner runs a loop, placing the two digests into the virgin -queue. -:: - - # Put the messages back in the queue for the runner to handle. - >>> filebase = digestq.enqueue(entry.msg, entry.msgdata) - >>> runner.run() - >>> messages = get_queue_messages('virgin') - >>> len(messages) - 2 - -One of which is the MIME digest and the other of which is the RFC 1153 digest. - - >>> mime, rfc1153 = mime_rfc1153(messages) - -You can see that the digests contain a mix of French and Japanese. - - >>> print mime.msg.as_string() - Content-Type: multipart/mixed; boundary="===============...==" - MIME-Version: 1.0 - From: test-request@example.com - Subject: Groupe Test, Vol. 1, Parution 2 - To: test@example.com - Reply-To: test@example.com - Date: ... - Message-ID: ... - <BLANKLINE> - --===============...== - Content-Type: text/plain; charset="iso-8859-1" - MIME-Version: 1.0 - Content-Transfer-Encoding: quoted-printable - Content-Description: Groupe Test, Vol. 1, Parution 2 - <BLANKLINE> - Envoyez vos messages pour la liste Test =E0 - test@example.com - <BLANKLINE> - Pour vous (d=E9s)abonner par le web, consultez - http://lists.example.com/listinfo/test@example.com - <BLANKLINE> - ou, par courriel, envoyez un message avec =AB=A0help=A0=BB dans le corps ou - dans le sujet =E0 - test-request@example.com - <BLANKLINE> - Vous pouvez contacter l'administrateur de la liste =E0 l'adresse - test-owner@example.com - <BLANKLINE> - Si vous r=E9pondez, n'oubliez pas de changer l'objet du message afin - qu'il soit plus sp=E9cifique que =AB=A0Re: Contenu du groupe de Test...=A0= - =BB - --===============...== - Content-Type: text/plain; charset="utf-8" - MIME-Version: 1.0 - Content-Transfer-Encoding: base64 - Content-Description: Today's Topics (1 messages) - <BLANKLINE> - VGjDqG1lcyBkdSBqb3VyIDoKCiAgIDEuIOS4gOeVqiAoYXBlcnNvbkBleGFtcGxlLm9yZykK - <BLANKLINE> - --===============...== - Content-Type: message/rfc822 - MIME-Version: 1.0 - <BLANKLINE> - From: aperson@example.org - To: test@example.com - Subject: =?iso-2022-jp?b?GyRCMGxIVhsoQg==?= - MIME-Version: 1.0 - Content-Type: text/plain; charset=iso-2022-jp - Content-Transfer-Encoding: 7bit - <BLANKLINE> - 一番 - <BLANKLINE> - --===============...== - Content-Type: text/plain; charset="iso-8859-1" - MIME-Version: 1.0 - Content-Transfer-Encoding: quoted-printable - Content-Description: =?utf-8?q?Pied_de_page_des_remises_group=C3=A9es?= - <BLANKLINE> - _______________________________________________ - Test mailing list - test@example.com - http://lists.example.com/listinfo/test@example.com - <BLANKLINE> - --===============...==-- - -The RFC 1153 digest will be encoded in UTF-8 since it contains a mixture of -French and Japanese characters. - - >>> print rfc1153.msg.as_string() - From: test-request@example.com - Subject: Groupe Test, Vol. 1, Parution 2 - To: test@example.com - Reply-To: test@example.com - Date: ... - Message-ID: ... - MIME-Version: 1.0 - Content-Type: text/plain; charset="utf-8" - Content-Transfer-Encoding: base64 - <BLANKLINE> - RW52b... - <BLANKLINE> - -The content can be decoded to see the actual digest text. -:: - - # We must display the repr of the decoded value because doctests cannot - # handle the non-ascii characters. - >>> [repr(line) - ... for line in rfc1153.msg.get_payload(decode=True).splitlines()] - ["'Envoyez vos messages pour la liste Test \\xc3\\xa0'", - "'\\ttest@example.com'", - "''", - "'Pour vous (d\\xc3\\xa9s)abonner par le web, consultez'", - "'\\thttp://lists.example.com/listinfo/test@example.com'", - "''", - "'ou, par courriel, envoyez un message avec \\xc2\\xab\\xc2\\xa0... - "'dans le sujet \\xc3\\xa0'", - "'\\ttest-request@example.com'", - "''", - '"Vous pouvez contacter l\'administrateur de la liste \\xc3\\xa0 ... - "'\\ttest-owner@example.com'", - "''", - '"Si vous r\\xc3\\xa9pondez, n\'oubliez pas de changer l\'objet du ... - '"qu\'il soit plus sp\\xc3\\xa9cifique que \\xc2\\xab\\xc2\\xa0Re: ... - "''", - "'Th\\xc3\\xa8mes du jour :'", - "''", - "' 1. \\xe4\\xb8\\x80\\xe7\\x95\\xaa (aperson@example.org)'", - "''", - "''", - "'---------------------------------------------------------------------... - "''", - "'From: aperson@example.org'", - "'Subject: \\xe4\\xb8\\x80\\xe7\\x95\\xaa'", - "'To: test@example.com'", - "'Message-ID: ... - "'Content-Type: text/plain; charset=iso-2022-jp'", - "''", - "'\\xe4\\xb8\\x80\\xe7\\x95\\xaa'", - "''", - "'------------------------------'", - "''", - "'_______________________________________________'", - "'Test mailing list'", - "'test@example.com'", - "'http://lists.example.com/listinfo/test@example.com'", - "''", - "''", - "'Fin de Groupe Test, Vol. 1, Parution 2'", - "'**************************************'"] - - >>> config.pop('french') - - -Digest delivery -=============== - -A mailing list's members can choose to receive normal delivery, plain text -digests, or MIME digests. -:: - - >>> len(get_queue_messages('virgin')) - 0 - - >>> from mailman.interfaces.usermanager import IUserManager - >>> from zope.component import getUtility - >>> user_manager = getUtility(IUserManager) - - >>> from mailman.interfaces.member import DeliveryMode, MemberRole - >>> def subscribe(email, mode): - ... address = user_manager.create_address(email) - ... member = mlist.subscribe(address, MemberRole.member) - ... member.preferences.delivery_mode = mode - ... return member - -Two regular delivery members subscribe to the mailing list. - - >>> member_1 = subscribe('uperson@example.com', DeliveryMode.regular) - >>> member_2 = subscribe('vperson@example.com', DeliveryMode.regular) - -Two MIME digest members subscribe to the mailing list. - - >>> member_3 = subscribe('wperson@example.com', DeliveryMode.mime_digests) - >>> member_4 = subscribe('xperson@example.com', DeliveryMode.mime_digests) - -One RFC 1153 digest member subscribes to the mailing list. - - >>> member_5 = subscribe( - ... 'yperson@example.com', DeliveryMode.plaintext_digests) - >>> member_6 = subscribe( - ... 'zperson@example.com', DeliveryMode.plaintext_digests) - -When a digest gets sent, the appropriate recipient list is chosen. - - >>> mlist.preferred_language = 'en' - >>> mlist.digest_size_threshold = 0.5 - >>> fill_digest() - >>> runner.run() - -The digests are sitting in the virgin queue. One of them is the MIME digest -and the other is the RFC 1153 digest. -:: - - >>> messages = get_queue_messages('virgin') - >>> len(messages) - 2 - - >>> mime, rfc1153 = mime_rfc1153(messages) - -Only wperson and xperson get the MIME digests. - - >>> sorted(mime.msgdata['recipients']) - [u'wperson@example.com', u'xperson@example.com'] - -Only yperson and zperson get the RFC 1153 digests. - - >>> sorted(rfc1153.msgdata['recipients']) - [u'yperson@example.com', u'zperson@example.com'] - -Now uperson decides that they would like to start receiving digests too. -:: - - >>> member_1.preferences.delivery_mode = DeliveryMode.mime_digests - >>> fill_digest() - >>> runner.run() - - >>> messages = get_queue_messages('virgin') - >>> len(messages) - 2 - - >>> mime, rfc1153 = mime_rfc1153(messages) - >>> sorted(mime.msgdata['recipients']) - [u'uperson@example.com', u'wperson@example.com', u'xperson@example.com'] - - >>> sorted(rfc1153.msgdata['recipients']) - [u'yperson@example.com', u'zperson@example.com'] - -At this point, both uperson and wperson decide that they'd rather receive -regular deliveries instead of digests. uperson would like to get any last -digest that may be sent so that she doesn't miss anything. wperson does care -as much and does not want to receive one last digest. -:: - - >>> mlist.send_one_last_digest_to( - ... member_1.address, member_1.preferences.delivery_mode) - - >>> member_1.preferences.delivery_mode = DeliveryMode.regular - >>> member_3.preferences.delivery_mode = DeliveryMode.regular - - >>> fill_digest() - >>> runner.run() - - >>> messages = get_queue_messages('virgin') - >>> mime, rfc1153 = mime_rfc1153(messages) - >>> sorted(mime.msgdata['recipients']) - [u'uperson@example.com', u'xperson@example.com'] - - >>> sorted(rfc1153.msgdata['recipients']) - [u'yperson@example.com', u'zperson@example.com'] - -Since uperson has received their last digest, they will not get any more of -them. -:: - - >>> fill_digest() - >>> runner.run() - - >>> messages = get_queue_messages('virgin') - >>> len(messages) - 2 - - >>> mime, rfc1153 = mime_rfc1153(messages) - >>> sorted(mime.msgdata['recipients']) - [u'xperson@example.com'] - - >>> sorted(rfc1153.msgdata['recipients']) - [u'yperson@example.com', u'zperson@example.com'] diff --git a/src/mailman/queue/docs/incoming.txt b/src/mailman/queue/docs/incoming.txt deleted file mode 100644 index 6455db20b..000000000 --- a/src/mailman/queue/docs/incoming.txt +++ /dev/null @@ -1,263 +0,0 @@ -========================= -The incoming queue runner -========================= - -This runner's sole purpose in life is to decide the disposition of the -message. It can either be accepted for delivery, rejected (i.e. bounced), -held for moderator approval, or discarded. - -The runner operates by processing chains on a message/metadata pair in the -context of a mailing list. Each mailing list may have a 'start chain' where -processing begins, with a global default. This chain is processed with the -message eventually ending up in one of the four disposition states described -above. - - >>> mlist = create_list('test@example.com') - >>> print mlist.start_chain - built-in - - -Sender addresses -================ - -The incoming queue runner ensures that the sender addresses on the message are -registered with the system. This is used for determining nonmember posting -privileges. The addresses will not be linked to a user and will be -unverified, so if the real user comes along later and claims the address, it -will be linked to their user account (and must be verified). - -While configurable, the *sender addresses* by default are those named in the -`From:`, `Sender:`, and `Reply-To:` headers, as well as the envelope sender -(though we won't worry about the latter). -:: - - >>> msg = message_from_string("""\ - ... From: zperson@example.com - ... Reply-To: yperson@example.com - ... Sender: xperson@example.com - ... To: test@example.com - ... Subject: This is spiced ham - ... Message-ID: <bogus> - ... - ... """) - - >>> from zope.component import getUtility - >>> from mailman.interfaces.usermanager import IUserManager - >>> user_manager = getUtility(IUserManager) - >>> print user_manager.get_address('xperson@example.com') - None - >>> print user_manager.get_address('yperson@example.com') - None - >>> print user_manager.get_address('zperson@example.com') - None - -Inject the message into the incoming queue, similar to the way the upstream -mail server normally would. - - >>> from mailman.inject import inject_message - >>> inject_message(mlist, msg) - -The incoming queue runner runs until it is empty. - - >>> from mailman.queue.incoming import IncomingRunner - >>> from mailman.testing.helpers import make_testable_runner - >>> incoming = make_testable_runner(IncomingRunner, 'in') - >>> incoming.run() - -And now the addresses are known to the system. As mentioned above, they are -not linked to a user and are unverified. - - >>> for localpart in ('xperson', 'yperson', 'zperson'): - ... email = '{0}@example.com'.format(localpart) - ... address = user_manager.get_address(email) - ... print '{0}; verified? {1}; user? {2}'.format( - ... address.email, - ... ('No' if address.verified_on is None else 'Yes'), - ... user_manager.get_user(email)) - xperson@example.com; verified? No; user? None - yperson@example.com; verified? No; user? None - zperson@example.com; verified? No; user? None - -.. - Clear the pipeline queue of artifacts that affect the following tests. - >>> from mailman.testing.helpers import get_queue_messages - >>> ignore = get_queue_messages('pipeline') - - -Accepted messages -================= - -We have a message that is going to be sent to the mailing list. Once Anne is -a member of the mailing list, this message is so perfectly fine for posting -that it will be accepted and forward to the pipeline queue. - - >>> from mailman.testing.helpers import subscribe - >>> subscribe(mlist, 'Anne') - >>> msg = message_from_string("""\ - ... From: aperson@example.com - ... To: test@example.com - ... Subject: My first post - ... Message-ID: <first> - ... - ... First post! - ... """) - -Inject the message into the incoming queue and run until the queue is empty. - - >>> inject_message(mlist, msg) - >>> incoming.run() - -Now the message is in the pipeline queue. - - >>> pipeline_queue = config.switchboards['pipeline'] - >>> len(pipeline_queue.files) - 1 - >>> incoming_queue = config.switchboards['in'] - >>> len(incoming_queue.files) - 0 - >>> item = get_queue_messages('pipeline')[0] - >>> print item.msg.as_string() - From: aperson@example.com - To: test@example.com - Subject: My first post - Message-ID: <first> - Date: ... - X-Mailman-Rule-Misses: approved; emergency; loop; member-moderation; - administrivia; implicit-dest; max-recipients; max-size; - news-moderation; no-subject; suspicious-header; nonmember-moderation - <BLANKLINE> - First post! - <BLANKLINE> - >>> dump_msgdata(item.msgdata) - _parsemsg : False - envsender : noreply@example.com - ... - - -Held messages -============= - -The list moderator sets the emergency flag on the mailing list. The built-in -chain will now hold all posted messages, so nothing will show up in the -pipeline queue. -:: - - >>> from mailman.chains.base import ChainNotification - >>> def on_chain(event): - ... if isinstance(event, ChainNotification): - ... print event - ... print event.chain - ... print 'From: {0}\nTo: {1}\nMessage-ID: {2}'.format( - ... event.msg['from'], event.msg['to'], - ... event.msg['message-id']) - - >>> mlist.emergency = True - - >>> from mailman.testing.helpers import event_subscribers - >>> with event_subscribers(on_chain): - ... inject_message(mlist, msg) - ... incoming.run() - <mailman.chains.hold.HoldNotification ...> - <mailman.chains.hold.HoldChain ...> - From: aperson@example.com - To: test@example.com - Message-ID: <first> - - >>> mlist.emergency = False - - -Discarded messages -================== - -Another possibility is that the message would get immediately discarded. The -built-in chain does not have such a disposition by default, so let's craft a -new chain and set it as the mailing list's start chain. -:: - - >>> from mailman.chains.base import Chain, Link - >>> from mailman.interfaces.chain import LinkAction - >>> def make_chain(name, target_chain): - ... truth_rule = config.rules['truth'] - ... target_chain = config.chains[target_chain] - ... test_chain = Chain(name, 'Testing {0}'.format(target_chain)) - ... config.chains[test_chain.name] = test_chain - ... link = Link(truth_rule, LinkAction.jump, target_chain) - ... test_chain.append_link(link) - ... return test_chain - - >>> test_chain = make_chain('always-discard', 'discard') - >>> mlist.start_chain = test_chain.name - - >>> msg.replace_header('message-id', '<second>') - >>> with event_subscribers(on_chain): - ... inject_message(mlist, msg) - ... incoming.run() - <mailman.chains.discard.DiscardNotification ...> - <mailman.chains.discard.DiscardChain ...> - From: aperson@example.com - To: test@example.com - Message-ID: <second> - - >>> del config.chains[test_chain.name] - -.. - The virgin queue needs to be cleared out due to artifacts from the - previous tests above. - - >>> virgin_queue = config.switchboards['virgin'] - >>> ignore = get_queue_messages('virgin') - - -Rejected messages -================= - -Similar to discarded messages, a message can be rejected, or bounced back to -the original sender. Again, the built-in chain doesn't support this so we'll -just create a new chain that does. - - >>> test_chain = make_chain('always-reject', 'reject') - >>> mlist.start_chain = test_chain.name - - >>> msg.replace_header('message-id', '<third>') - >>> with event_subscribers(on_chain): - ... inject_message(mlist, msg) - ... incoming.run() - <mailman.chains.reject.RejectNotification ...> - <mailman.chains.reject.RejectChain ...> - From: aperson@example.com - To: test@example.com - Message-ID: <third> - -The rejection message is sitting in the virgin queue waiting to be delivered -to the original sender. - - >>> len(virgin_queue.files) - 1 - >>> item = get_queue_messages('virgin')[0] - >>> print item.msg.as_string() - Subject: My first post - From: test-owner@example.com - To: aperson@example.com - ... - <BLANKLINE> - --===============... - Content-Type: text/plain; charset="us-ascii" - MIME-Version: 1.0 - Content-Transfer-Encoding: 7bit - <BLANKLINE> - [No bounce details are available] - --===============... - Content-Type: message/rfc822 - MIME-Version: 1.0 - <BLANKLINE> - From: aperson@example.com - To: test@example.com - Subject: My first post - Message-ID: <third> - Date: ... - <BLANKLINE> - First post! - <BLANKLINE> - --===============... - - >>> del config.chains['always-reject'] diff --git a/src/mailman/queue/docs/lmtp.txt b/src/mailman/queue/docs/lmtp.txt deleted file mode 100644 index c95c6aa2b..000000000 --- a/src/mailman/queue/docs/lmtp.txt +++ /dev/null @@ -1,313 +0,0 @@ -=========== -LTMP server -=========== - -Mailman can accept messages via LMTP (RFC 2033). Most modern mail servers -support LMTP local delivery, so this is a very portable way to connect Mailman -with your mail server. - -Our LMTP server is fairly simple though; all it does is make sure that the -message is destined for a valid endpoint, e.g. ``mylist-join@example.com``. - -Let's start a testable LMTP queue runner. - - >>> from mailman.testing import helpers - >>> master = helpers.TestableMaster() - >>> master.start('lmtp') - -It also helps to have a nice LMTP client. - - >>> lmtp = helpers.get_lmtp_client() - (220, '... Python LMTP queue runner 1.0') - >>> lmtp.lhlo('remote.example.org') - (250, ...) - - -Posting address -=============== - -If the mail server tries to send a message to a nonexistent mailing list, it -will get a 550 error. - - >>> lmtp.sendmail( - ... 'anne.person@example.com', - ... ['mylist@example.com'], """\ - ... From: anne.person@example.com - ... To: mylist@example.com - ... Subject: An interesting message - ... Message-ID: <aardvark> - ... - ... This is an interesting message. - ... """) - Traceback (most recent call last): - ... - SMTPDataError: (550, 'Requested action not taken: mailbox unavailable') - -Once the mailing list is created, the posting address is valid. -:: - - >>> create_list('mylist@example.com') - <mailing list "mylist@example.com" at ...> - - >>> transaction.commit() - >>> lmtp.sendmail( - ... 'anne.person@example.com', - ... ['mylist@example.com'], """\ - ... From: anne.person@example.com - ... To: mylist@example.com - ... Subject: An interesting message - ... Message-ID: <badger> - ... - ... This is an interesting message. - ... """) - {} - - >>> from mailman.testing.helpers import get_queue_messages - >>> messages = get_queue_messages('in') - >>> len(messages) - 1 - >>> print messages[0].msg.as_string() - From: anne.person@example.com - To: mylist@example.com - Subject: An interesting message - Message-ID: <badger> - X-MailFrom: anne.person@example.com - <BLANKLINE> - This is an interesting message. - >>> dump_msgdata(messages[0].msgdata) - _parsemsg : False - listname : mylist@example.com - original_size: ... - to_list : True - version : ... - - -Sub-addresses -============= - -The LMTP server understands each of the list's sub-addreses, such as -join, --leave, -request and so on. If the message is posted to an invalid -sub-address though, it is rejected. - - >>> lmtp.sendmail( - ... 'anne.person@example.com', - ... ['mylist-bogus@example.com'], """\ - ... From: anne.person@example.com - ... To: mylist-bogus@example.com - ... Subject: Help - ... Message-ID: <cow> - ... - ... Please help me. - ... """) - Traceback (most recent call last): - ... - SMTPDataError: (550, 'Requested action not taken: mailbox unavailable') - -But the message is accepted if posted to a valid sub-address. - - >>> lmtp.sendmail( - ... 'anne.person@example.com', - ... ['mylist-request@example.com'], """\ - ... From: anne.person@example.com - ... To: mylist-request@example.com - ... Subject: Help - ... Message-ID: <dog> - ... - ... Please help me. - ... """) - {} - - -Request subaddress ------------------- - -Depending on the subaddress, there is a message in the appropriate queue for -later processing. For example, all -request messages are put into the command -queue for processing. - - >>> messages = get_queue_messages('command') - >>> len(messages) - 1 - >>> print messages[0].msg.as_string() - From: anne.person@example.com - To: mylist-request@example.com - Subject: Help - Message-ID: <dog> - X-MailFrom: anne.person@example.com - <BLANKLINE> - Please help me. - >>> dump_msgdata(messages[0].msgdata) - _parsemsg : False - listname : mylist@example.com - original_size: ... - subaddress : request - version : ... - - -Bounce processor ----------------- - -A message to the -bounces address goes to the bounce processor. - - >>> lmtp.sendmail( - ... 'mail-daemon@example.com', - ... ['mylist-bounces@example.com'], """\ - ... From: mail-daemon@example.com - ... To: mylist-bounces@example.com - ... Subject: A bounce - ... Message-ID: <elephant> - ... - ... Bouncy bouncy. - ... """) - {} - >>> messages = get_queue_messages('bounces') - >>> len(messages) - 1 - >>> dump_msgdata(messages[0].msgdata) - _parsemsg : False - listname : mylist@example.com - original_size: ... - subaddress : bounces - version : ... - - -Command processor ------------------ - -Confirmation messages go to the command processor... - - >>> lmtp.sendmail( - ... 'anne.person@example.com', - ... ['mylist-confirm@example.com'], """\ - ... From: anne.person@example.com - ... To: mylist-confirm@example.com - ... Subject: A bounce - ... Message-ID: <falcon> - ... - ... confirm 123 - ... """) - {} - >>> messages = get_queue_messages('command') - >>> len(messages) - 1 - >>> dump_msgdata(messages[0].msgdata) - _parsemsg : False - listname : mylist@example.com - original_size: ... - subaddress : confirm - version : ... - -...as do join messages... -:: - - >>> lmtp.sendmail( - ... 'anne.person@example.com', - ... ['mylist-join@example.com'], """\ - ... From: anne.person@example.com - ... To: mylist-join@example.com - ... Message-ID: <giraffe> - ... - ... """) - {} - >>> messages = get_queue_messages('command') - >>> len(messages) - 1 - >>> dump_msgdata(messages[0].msgdata) - _parsemsg : False - listname : mylist@example.com - original_size: ... - subaddress : join - version : ... - - >>> lmtp.sendmail( - ... 'anne.person@example.com', - ... ['mylist-subscribe@example.com'], """\ - ... From: anne.person@example.com - ... To: mylist-subscribe@example.com - ... Message-ID: <hippopotamus> - ... - ... """) - {} - >>> messages = get_queue_messages('command') - >>> len(messages) - 1 - >>> dump_msgdata(messages[0].msgdata) - _parsemsg : False - listname : mylist@example.com - original_size: ... - subaddress : join - version : ... - -...and leave messages. -:: - - >>> lmtp.sendmail( - ... 'anne.person@example.com', - ... ['mylist-leave@example.com'], """\ - ... From: anne.person@example.com - ... To: mylist-leave@example.com - ... Message-ID: <iguana> - ... - ... """) - {} - >>> messages = get_queue_messages('command') - >>> len(messages) - 1 - >>> dump_msgdata(messages[0].msgdata) - _parsemsg : False - listname : mylist@example.com - original_size: ... - subaddress : leave - version : ... - - >>> lmtp.sendmail( - ... 'anne.person@example.com', - ... ['mylist-unsubscribe@example.com'], """\ - ... From: anne.person@example.com - ... To: mylist-unsubscribe@example.com - ... Message-ID: <jackal> - ... - ... """) - {} - >>> messages = get_queue_messages('command') - >>> len(messages) - 1 - >>> dump_msgdata(messages[0].msgdata) - _parsemsg : False - listname : mylist@example.com - original_size: ... - subaddress : leave - version : ... - - -Incoming processor ------------------- - -Messages to the -owner address go to the incoming processor. - - >>> lmtp.sendmail( - ... 'anne.person@example.com', - ... ['mylist-owner@example.com'], """\ - ... From: anne.person@example.com - ... To: mylist-owner@example.com - ... Message-ID: <kangaroo> - ... - ... """) - {} - >>> messages = get_queue_messages('in') - >>> len(messages) - 1 - >>> dump_msgdata(messages[0].msgdata) - _parsemsg : False - envsender : changeme@example.com - listname : mylist@example.com - original_size: ... - subaddress : owner - to_owner : True - version : ... - - -Clean up -======== - - >>> master.stop() diff --git a/src/mailman/queue/docs/news.txt b/src/mailman/queue/docs/news.txt deleted file mode 100644 index 7261aa333..000000000 --- a/src/mailman/queue/docs/news.txt +++ /dev/null @@ -1,161 +0,0 @@ -=============== -The news runner -=============== - -The news runner is the queue runner that gateways mailing list messages to an -NNTP newsgroup. One of the most important things this runner does is prepare -the message for Usenet (yes, I know that NNTP is not Usenet, but this runner -was originally written to gate to Usenet, which has its own rules). - - >>> mlist = create_list('_xtest@example.com') - >>> mlist.linked_newsgroup = 'comp.lang.python' - -Some NNTP servers such as INN reject messages containing a set of prohibited -headers, so one of the things that the news runner does is remove these -prohibited headers. -:: - - >>> msg = message_from_string("""\ - ... From: aperson@example.com - ... To: _xtest@example.com - ... NNTP-Posting-Host: news.example.com - ... NNTP-Posting-Date: today - ... X-Trace: blah blah - ... X-Complaints-To: abuse@dom.ain - ... Xref: blah blah - ... Xref: blah blah - ... Date-Received: yesterday - ... Posted: tomorrow - ... Posting-Version: 99.99 - ... Relay-Version: 88.88 - ... Received: blah blah - ... - ... A message - ... """) - >>> msgdata = {} - - >>> from mailman.queue.news import prepare_message - >>> prepare_message(mlist, msg, msgdata) - >>> msgdata['prepped'] - True - >>> print msg.as_string() - From: aperson@example.com - To: _xtest@example.com - Newsgroups: comp.lang.python - Message-ID: ... - Lines: 1 - <BLANKLINE> - A message - <BLANKLINE> - -Some NNTP servers will reject messages where certain headers are duplicated, -so the news runner must collapse or move these duplicate headers to an -``X-Original-*`` header that the news server doesn't care about. - - >>> msg = message_from_string("""\ - ... From: aperson@example.com - ... To: _xtest@example.com - ... To: two@example.com - ... Cc: three@example.com - ... Cc: four@example.com - ... Cc: five@example.com - ... Content-Transfer-Encoding: yes - ... Content-Transfer-Encoding: no - ... Content-Transfer-Encoding: maybe - ... - ... A message - ... """) - >>> msgdata = {} - >>> prepare_message(mlist, msg, msgdata) - >>> msgdata['prepped'] - True - >>> print msg.as_string() - From: aperson@example.com - Newsgroups: comp.lang.python - Message-ID: ... - Lines: 1 - To: _xtest@example.com - X-Original-To: two@example.com - CC: three@example.com - X-Original-CC: four@example.com - X-Original-CC: five@example.com - Content-Transfer-Encoding: yes - X-Original-Content-Transfer-Encoding: no - X-Original-Content-Transfer-Encoding: maybe - <BLANKLINE> - A message - <BLANKLINE> - -But if no headers are duplicated, then the news runner doesn't need to modify -the message. - - >>> msg = message_from_string("""\ - ... From: aperson@example.com - ... To: _xtest@example.com - ... Cc: someother@example.com - ... Content-Transfer-Encoding: yes - ... - ... A message - ... """) - >>> msgdata = {} - >>> prepare_message(mlist, msg, msgdata) - >>> msgdata['prepped'] - True - >>> print msg.as_string() - From: aperson@example.com - To: _xtest@example.com - Cc: someother@example.com - Content-Transfer-Encoding: yes - Newsgroups: comp.lang.python - Message-ID: ... - Lines: 1 - <BLANKLINE> - A message - <BLANKLINE> - - -Newsgroup moderation -==================== - -When the newsgroup is moderated, an ``Approved:`` header with the list's -posting address is added for the benefit of the Usenet system. -:: - - >>> from mailman.interfaces.nntp import NewsModeration - >>> mlist.news_moderation = NewsModeration.open_moderated - >>> msg = message_from_string("""\ - ... From: aperson@example.com - ... To: _xtest@example.com - ... Approved: this gets deleted - ... - ... """) - >>> prepare_message(mlist, msg, {}) - >>> print msg['approved'] - _xtest@example.com - - >>> mlist.news_moderation = NewsModeration.moderated - >>> msg = message_from_string("""\ - ... From: aperson@example.com - ... To: _xtest@example.com - ... Approved: this gets deleted - ... - ... """) - >>> prepare_message(mlist, msg, {}) - >>> print msg['approved'] - _xtest@example.com - -But if the newsgroup is not moderated, the ``Approved:`` header is not changed. - - >>> mlist.news_moderation = NewsModeration.none - >>> msg = message_from_string("""\ - ... From: aperson@example.com - ... To: _xtest@example.com - ... Approved: this doesn't get deleted - ... - ... """) - >>> prepare_message(mlist, msg, {}) - >>> msg['approved'] - u"this doesn't get deleted" - - -XXX More of the NewsRunner should be tested. diff --git a/src/mailman/queue/docs/outgoing.txt b/src/mailman/queue/docs/outgoing.txt deleted file mode 100644 index 0af22b808..000000000 --- a/src/mailman/queue/docs/outgoing.txt +++ /dev/null @@ -1,413 +0,0 @@ -===================== -Outgoing queue runner -===================== - -The outgoing queue runner is the process that delivers messages to the -directly upstream SMTP server. It is this upstream SMTP server that performs -final delivery to the intended recipients. - -Messages that appear in the outgoing queue are processed individually through -a *delivery module*, essentially a pluggable interface for determining how the -recipient set will be batched, whether messages will be personalized and -VERP'd, etc. The outgoing runner doesn't itself support retrying but it can -move messages to the 'retry queue' for handling delivery failures. -:: - - >>> mlist = create_list('test@example.com') - - >>> from mailman.app.membership import add_member - >>> from mailman.interfaces.member import DeliveryMode - >>> add_member(mlist, 'aperson@example.com', 'Anne Person', - ... 'password', DeliveryMode.regular, 'en') - <Member: Anne Person <aperson@example.com> - on test@example.com as MemberRole.member> - >>> add_member(mlist, 'bperson@example.com', 'Bart Person', - ... 'password', DeliveryMode.regular, 'en') - <Member: Bart Person <bperson@example.com> - on test@example.com as MemberRole.member> - >>> add_member(mlist, 'cperson@example.com', 'Cris Person', - ... 'password', DeliveryMode.regular, 'en') - <Member: Cris Person <cperson@example.com> - on test@example.com as MemberRole.member> - -Normally, messages would show up in the outgoing queue after the message has -been processed by the rule set and pipeline. But we can simulate that here by -injecting a message directly into the outgoing queue. First though, we must -call the ``calculate-recipients`` handler so that the message metadata will be -populated with the list of addresses to deliver the message to. -:: - - >>> msg = message_from_string("""\ - ... From: aperson@example.com - ... To: test@example.com - ... Subject: My first post - ... Message-ID: <first> - ... - ... First post! - ... """) - - >>> msgdata = {} - >>> handler = config.handlers['calculate-recipients'] - >>> handler.process(mlist, msg, msgdata) - >>> outgoing_queue = config.switchboards['out'] - -The ``to-outgoing`` handler populates the message metadata with the -destination mailing list name. Simulate that here too. - - >>> ignore = outgoing_queue.enqueue( - ... msg, msgdata, - ... tolist=True, - ... listname=mlist.fqdn_listname) - -Running the outgoing queue runner processes the message, delivering it to the -upstream SMTP. - - >>> from mailman.queue.outgoing import OutgoingRunner - >>> from mailman.testing.helpers import make_testable_runner - >>> outgoing = make_testable_runner(OutgoingRunner, 'out') - >>> outgoing.run() - -Every recipient got the same copy of the message. -:: - - >>> messages = list(smtpd.messages) - >>> len(messages) - 1 - - >>> print messages[0].as_string() - From: aperson@example.com - To: test@example.com - Subject: My first post - Message-ID: <first> - X-Peer: ... - X-MailFrom: test-bounces@example.com - X-RcptTo: cperson@example.com, bperson@example.com, aperson@example.com - <BLANKLINE> - First post! - - -Personalization -=============== - -Mailman supports sending individual messages to each recipient by -personalizing delivery. This increases the bandwidth between Mailman and the -upstream mail server, and between the upstream mail server and the remote -recipient mail servers. The benefit is that personalization provides for a -much better user experience, because the messages can be tailored for each -recipient. - - >>> from mailman.interfaces.mailinglist import Personalization - >>> mlist.personalize = Personalization.individual - >>> transaction.commit() - -Now when we send the message, our mail server will get three copies instead of -just one. - - >>> ignore = outgoing_queue.enqueue( - ... msg, msgdata, - ... listname=mlist.fqdn_listname) - >>> outgoing.run() - >>> messages = list(smtpd.messages) - >>> len(messages) - 3 - -Since we've done no other configuration, the only difference in the messages -is the recipient address. Specifically, the Sender header is the same for all -recipients. -:: - - >>> from operator import itemgetter - >>> def show_headers(messages): - ... for message in sorted(messages, key=itemgetter('x-rcptto')): - ... print message['X-RcptTo'], message['X-MailFrom'] - - >>> show_headers(messages) - aperson@example.com test-bounces@example.com - bperson@example.com test-bounces@example.com - cperson@example.com test-bounces@example.com - - -VERP -==== - -An even more interesting personalization opportunity arises if VERP_ is -enabled. Here, Mailman takes advantage of the fact that it's sending -individualized messages anyway, so it also encodes the recipients address in -the Sender header. - -.. _VERP: ../../mta/docs/verp.html - - -Forcing VERP ------------- - -A handler can force VERP by setting the ``verp`` key in the message metadata. -:: - - >>> ignore = outgoing_queue.enqueue( - ... msg, msgdata, - ... verp=True, - ... listname=mlist.fqdn_listname) - >>> outgoing.run() - >>> messages = list(smtpd.messages) - >>> len(messages) - 3 - - >>> show_headers(messages) - aperson@example.com test-bounces+aperson=example.com@example.com - bperson@example.com test-bounces+bperson=example.com@example.com - cperson@example.com test-bounces+cperson=example.com@example.com - - -VERP personalized deliveries ----------------------------- - -The site administrator can enable VERP whenever messages are personalized. - - >>> config.push('verp', """ - ... [mta] - ... verp_personalized_deliveries: yes - ... """) - -Again, we get three individual messages, with VERP'd ``Sender`` headers. -:: - - >>> ignore = outgoing_queue.enqueue( - ... msg, msgdata, - ... listname=mlist.fqdn_listname) - >>> outgoing.run() - >>> messages = list(smtpd.messages) - >>> len(messages) - 3 - - >>> show_headers(messages) - aperson@example.com test-bounces+aperson=example.com@example.com - bperson@example.com test-bounces+bperson=example.com@example.com - cperson@example.com test-bounces+cperson=example.com@example.com - - >>> config.pop('verp') - >>> mlist.personalize = Personalization.none - >>> transaction.commit() - - -VERP every once in a while --------------------------- - -Perhaps personalization is too much of an overhead, but the list owners would -still like to occasionally get the benefits of VERP. The site administrator -can enable occasional VERPing of messages every so often, by setting a -delivery interval. Every N non-personalized deliveries turns on VERP for just -the next one. -:: - - >>> config.push('verp occasionally', """ - ... [mta] - ... verp_delivery_interval: 3 - ... """) - - # Reset the list's post_id, which is used to calculate the intervals. - >>> mlist.post_id = 1 - >>> transaction.commit() - -The first message is sent to the list, and it is neither personalized nor -VERP'd. -:: - - >>> ignore = outgoing_queue.enqueue( - ... msg, msgdata, - ... listname=mlist.fqdn_listname) - >>> outgoing.run() - >>> messages = list(smtpd.messages) - >>> len(messages) - 1 - - >>> show_headers(messages) - cperson@example.com, bperson@example.com, aperson@example.com - test-bounces@example.com - - # Perform post-delivery bookkeeping. - >>> after = config.handlers['after-delivery'] - >>> after.process(mlist, msg, msgdata) - >>> transaction.commit() - -The second message sent to the list is also not VERP'd. -:: - - >>> ignore = outgoing_queue.enqueue( - ... msg, msgdata, - ... listname=mlist.fqdn_listname) - >>> outgoing.run() - >>> messages = list(smtpd.messages) - >>> len(messages) - 1 - - >>> show_headers(messages) - cperson@example.com, bperson@example.com, aperson@example.com - test-bounces@example.com - - # Perform post-delivery bookkeeping. - >>> after.process(mlist, msg, msgdata) - >>> transaction.commit() - -The third message though is VERP'd. -:: - - >>> ignore = outgoing_queue.enqueue( - ... msg, msgdata, - ... listname=mlist.fqdn_listname) - >>> outgoing.run() - >>> messages = list(smtpd.messages) - >>> len(messages) - 3 - - >>> show_headers(messages) - aperson@example.com test-bounces+aperson=example.com@example.com - bperson@example.com test-bounces+bperson=example.com@example.com - cperson@example.com test-bounces+cperson=example.com@example.com - - # Perform post-delivery bookkeeping. - >>> after.process(mlist, msg, msgdata) - >>> transaction.commit() - -The next one is back to bulk delivery. -:: - - >>> ignore = outgoing_queue.enqueue( - ... msg, msgdata, - ... listname=mlist.fqdn_listname) - >>> outgoing.run() - >>> messages = list(smtpd.messages) - >>> len(messages) - 1 - - >>> show_headers(messages) - cperson@example.com, bperson@example.com, aperson@example.com - test-bounces@example.com - - >>> config.pop('verp occasionally') - - -VERP every time ---------------- - -If the site administrator wants to enable VERP for every delivery, even if no -personalization is going on, they can set the interval to 1. -:: - - >>> config.push('always verp', """ - ... [mta] - ... verp_delivery_interval: 1 - ... """) - - # Reset the list's post_id, which is used to calculate the intervals. - >>> mlist.post_id = 1 - >>> transaction.commit() - -The first message is VERP'd. -:: - - >>> ignore = outgoing_queue.enqueue( - ... msg, msgdata, - ... listname=mlist.fqdn_listname) - >>> outgoing.run() - >>> messages = list(smtpd.messages) - >>> len(messages) - 3 - - >>> show_headers(messages) - aperson@example.com test-bounces+aperson=example.com@example.com - bperson@example.com test-bounces+bperson=example.com@example.com - cperson@example.com test-bounces+cperson=example.com@example.com - - # Perform post-delivery bookkeeping. - >>> after.process(mlist, msg, msgdata) - >>> transaction.commit() - -As is the second message. -:: - - >>> ignore = outgoing_queue.enqueue( - ... msg, msgdata, - ... listname=mlist.fqdn_listname) - >>> outgoing.run() - >>> messages = list(smtpd.messages) - >>> len(messages) - 3 - - >>> show_headers(messages) - aperson@example.com test-bounces+aperson=example.com@example.com - bperson@example.com test-bounces+bperson=example.com@example.com - cperson@example.com test-bounces+cperson=example.com@example.com - - # Perform post-delivery bookkeeping. - >>> after.process(mlist, msg, msgdata) - >>> transaction.commit() - -And the third message. -:: - - >>> ignore = outgoing_queue.enqueue( - ... msg, msgdata, - ... listname=mlist.fqdn_listname) - >>> outgoing.run() - >>> messages = list(smtpd.messages) - >>> len(messages) - 3 - - >>> show_headers(messages) - aperson@example.com test-bounces+aperson=example.com@example.com - bperson@example.com test-bounces+bperson=example.com@example.com - cperson@example.com test-bounces+cperson=example.com@example.com - - # Perform post-delivery bookkeeping. - >>> after.process(mlist, msg, msgdata) - >>> transaction.commit() - - >>> config.pop('always verp') - - -Never VERP ----------- - -Similarly, the site administrator can disable occasional VERP'ing of -non-personalized messages by setting the interval to zero. -:: - - >>> config.push('never verp', """ - ... [mta] - ... verp_delivery_interval: 0 - ... """) - - # Reset the list's post_id, which is used to calculate the intervals. - >>> mlist.post_id = 1 - >>> transaction.commit() - -Neither the first message... -:: - - >>> ignore = outgoing_queue.enqueue( - ... msg, msgdata, - ... listname=mlist.fqdn_listname) - >>> outgoing.run() - >>> messages = list(smtpd.messages) - >>> len(messages) - 1 - - >>> show_headers(messages) - cperson@example.com, bperson@example.com, aperson@example.com - test-bounces@example.com - -...nor the second message is VERP'd. -:: - - >>> ignore = outgoing_queue.enqueue( - ... msg, msgdata, - ... listname=mlist.fqdn_listname) - >>> outgoing.run() - >>> messages = list(smtpd.messages) - >>> len(messages) - 1 - - >>> show_headers(messages) - cperson@example.com, bperson@example.com, aperson@example.com - test-bounces@example.com diff --git a/src/mailman/queue/docs/rest.txt b/src/mailman/queue/docs/rest.txt deleted file mode 100644 index 9e8851eca..000000000 --- a/src/mailman/queue/docs/rest.txt +++ /dev/null @@ -1,25 +0,0 @@ -=========== -REST server -=========== - -Mailman is controllable through an administrative `REST`_ HTTP server. - - >>> from mailman.testing import helpers - >>> master = helpers.TestableMaster(helpers.wait_for_webservice) - >>> master.start('rest') - -The RESTful server can be used to access basic version information. - - >>> dump_json('http://localhost:9001/3.0/system') - http_etag: "..." - mailman_version: GNU Mailman 3.0... (...) - python_version: ... - self_link: http://localhost:9001/3.0/system - - -Clean up -======== - - >>> master.stop() - -.. _REST: http://en.wikipedia.org/wiki/REST diff --git a/src/mailman/queue/docs/runner.txt b/src/mailman/queue/docs/runner.txt deleted file mode 100644 index 39e8fede2..000000000 --- a/src/mailman/queue/docs/runner.txt +++ /dev/null @@ -1,73 +0,0 @@ -============= -Queue runners -============= - -The queue runners (*qrunner*) are the processes that move messages around the -Mailman system. Each qrunner is responsible for a slice of the hash space in -a queue directory. It processes all the files in its slice, sleeps a little -while, then wakes up and runs through its queue files again. - - -Basic architecture -================== - -The basic architecture of qrunner is implemented in the base class that all -runners inherit from. This base class implements a ``.run()`` method that -runs continuously in a loop until the ``.stop()`` method is called. - - >>> mlist = create_list('_xtest@example.com') - -Here is a very simple derived qrunner class. Queue runners use a -configuration section in the configuration files to determine run -characteristics, such as the queue directory to use. Here we push a -configuration section for the test runner. -:: - - >>> config.push('test-runner', """ - ... [qrunner.test] - ... max_restarts: 1 - ... """) - - >>> from mailman.queue import Runner - >>> class TestableRunner(Runner): - ... def _dispose(self, mlist, msg, msgdata): - ... self.msg = msg - ... self.msgdata = msgdata - ... return False - ... - ... def _do_periodic(self): - ... self.stop() - ... - ... def _snooze(self, filecnt): - ... return - - >>> runner = TestableRunner('test') - -This qrunner doesn't do much except run once, storing the message and metadata -on instance variables. - - >>> msg = message_from_string("""\ - ... From: aperson@example.com - ... To: _xtest@example.com - ... - ... A test message. - ... """) - >>> switchboard = config.switchboards['test'] - >>> filebase = switchboard.enqueue(msg, listname=mlist.fqdn_listname, - ... foo='yes', bar='no') - >>> runner.run() - >>> print runner.msg.as_string() - From: aperson@example.com - To: _xtest@example.com - <BLANKLINE> - A test message. - <BLANKLINE> - >>> dump_msgdata(runner.msgdata) - _parsemsg: False - bar : no - foo : yes - lang : en - listname : _xtest@example.com - version : 3 - -XXX More of the Runner API should be tested. diff --git a/src/mailman/queue/docs/switchboard.txt b/src/mailman/queue/docs/switchboard.txt deleted file mode 100644 index d89aa3693..000000000 --- a/src/mailman/queue/docs/switchboard.txt +++ /dev/null @@ -1,187 +0,0 @@ -The switchboard -=============== - -The switchboard is subsystem that moves messages between queues. Each -instance of a switchboard is responsible for one queue directory. - - >>> msg = message_from_string("""\ - ... From: aperson@example.com - ... To: _xtest@example.com - ... - ... A test message. - ... """) - -Create a switchboard by giving its queue name and directory. - - >>> import os - >>> queue_directory = os.path.join(config.QUEUE_DIR, 'test') - >>> from mailman.queue import Switchboard - >>> switchboard = Switchboard('test', queue_directory) - >>> print switchboard.name - test - >>> switchboard.queue_directory == queue_directory - True - -Here's a helper function for ensuring things work correctly. - - >>> def check_qfiles(directory=None): - ... if directory is None: - ... directory = queue_directory - ... files = {} - ... for qfile in os.listdir(directory): - ... root, ext = os.path.splitext(qfile) - ... files[ext] = files.get(ext, 0) + 1 - ... if len(files) == 0: - ... print 'empty' - ... for ext in sorted(files): - ... print '{0}: {1}'.format(ext, files[ext]) - - -Enqueing and dequeing ---------------------- - -The message can be enqueued with metadata specified in the passed in -dictionary. - - >>> filebase = switchboard.enqueue(msg) - >>> check_qfiles() - .pck: 1 - -To read the contents of a queue file, dequeue it. - - >>> msg, msgdata = switchboard.dequeue(filebase) - >>> print msg.as_string() - From: aperson@example.com - To: _xtest@example.com - <BLANKLINE> - A test message. - <BLANKLINE> - >>> dump_msgdata(msgdata) - _parsemsg: False - version : 3 - >>> check_qfiles() - .bak: 1 - -To complete the dequeing process, removing all traces of the message file, -finish it (without preservation). - - >>> switchboard.finish(filebase) - >>> check_qfiles() - empty - -When enqueing a file, you can provide additional metadata keys by using -keyword arguments. - - >>> filebase = switchboard.enqueue(msg, {'foo': 1}, bar=2) - >>> msg, msgdata = switchboard.dequeue(filebase) - >>> switchboard.finish(filebase) - >>> dump_msgdata(msgdata) - _parsemsg: False - bar : 2 - foo : 1 - version : 3 - -Keyword arguments override keys from the metadata dictionary. - - >>> filebase = switchboard.enqueue(msg, {'foo': 1}, foo=2) - >>> msg, msgdata = switchboard.dequeue(filebase) - >>> switchboard.finish(filebase) - >>> dump_msgdata(msgdata) - _parsemsg: False - foo : 2 - version : 3 - - -Iterating over files --------------------- - -There are two ways to iterate over all the files in a switchboard's queue. -Normally, queue files end in .pck (for 'pickle') and the easiest way to -iterate over just these files is to use the .files attribute. - - >>> filebase_1 = switchboard.enqueue(msg, foo=1) - >>> filebase_2 = switchboard.enqueue(msg, foo=2) - >>> filebase_3 = switchboard.enqueue(msg, foo=3) - >>> filebases = sorted((filebase_1, filebase_2, filebase_3)) - >>> sorted(switchboard.files) == filebases - True - >>> check_qfiles() - .pck: 3 - -You can also use the .get_files() method if you want to iterate over all the -file bases for some other extension. - - >>> for filebase in switchboard.get_files(): - ... msg, msgdata = switchboard.dequeue(filebase) - >>> bakfiles = sorted(switchboard.get_files('.bak')) - >>> bakfiles == filebases - True - >>> check_qfiles() - .bak: 3 - >>> for filebase in switchboard.get_files('.bak'): - ... switchboard.finish(filebase) - >>> check_qfiles() - empty - - -Recovering files ----------------- - -Calling .dequeue() without calling .finish() leaves .bak backup files in -place. These can be recovered when the switchboard is instantiated. - - >>> filebase_1 = switchboard.enqueue(msg, foo=1) - >>> filebase_2 = switchboard.enqueue(msg, foo=2) - >>> filebase_3 = switchboard.enqueue(msg, foo=3) - >>> for filebase in switchboard.files: - ... msg, msgdata = switchboard.dequeue(filebase) - ... # Don't call .finish() - >>> check_qfiles() - .bak: 3 - >>> switchboard_2 = Switchboard('test', queue_directory, recover=True) - >>> check_qfiles() - .pck: 3 - -The files can be recovered explicitly. - - >>> for filebase in switchboard.files: - ... msg, msgdata = switchboard.dequeue(filebase) - ... # Don't call .finish() - >>> check_qfiles() - .bak: 3 - >>> switchboard.recover_backup_files() - >>> check_qfiles() - .pck: 3 - -But the files will only be recovered at most three times before they are -considered defective. In order to prevent mail bombs and loops, once this -maximum is reached, the files will be preserved in the 'bad' queue. -:: - - >>> for filebase in switchboard.files: - ... msg, msgdata = switchboard.dequeue(filebase) - ... # Don't call .finish() - >>> check_qfiles() - .bak: 3 - >>> switchboard.recover_backup_files() - >>> check_qfiles() - empty - - >>> bad = config.switchboards['bad'] - >>> check_qfiles(bad.queue_directory) - .psv: 3 - - -Clean up --------- - - >>> for file in os.listdir(bad.queue_directory): - ... os.remove(os.path.join(bad.queue_directory, file)) - >>> check_qfiles(bad.queue_directory) - empty - - -Queue slices ------------- - -XXX Add tests for queue slices. |
