======
Chains
======

When a new message comes into the system, Mailman uses a set of rule chains to
decide whether the message gets posted to the list, rejected, discarded, or
held for moderator approval.

There are a number of built-in chains available that act as end-points in the
processing of messages.


The Discard chain
=================

The Discard chain simply throws the message away.

    >>> from zope.interface.verify import verifyObject
    >>> from mailman.interfaces.chain import IChain
    >>> chain = config.chains['discard']
    >>> verifyObject(IChain, chain)
    True
    >>> print chain.name
    discard
    >>> print chain.description
    Discard a message and stop processing.

    >>> mlist = create_list('_xtest@example.com')
    >>> msg = message_from_string("""\
    ... From: aperson@example.com
    ... To: _xtest@example.com
    ... Subject: My first post
    ... Message-ID: <first>
    ...
    ... An important message.
    ... """)

    >>> from mailman.core.chains import process

    # XXX This checks the vette log file because there is no other evidence
    # that this chain has done anything.
    >>> import os
    >>> fp = open(os.path.join(config.LOG_DIR, 'vette'))
    >>> file_pos = fp.tell()
    >>> process(mlist, msg, {}, 'discard')
    >>> fp.seek(file_pos)
    >>> print 'LOG:', fp.read()
    LOG: ... DISCARD: <first>
    <BLANKLINE>


The Reject chain
================

The Reject chain bounces the message back to the original sender, and logs
this action.

    >>> chain = config.chains['reject']
    >>> verifyObject(IChain, chain)
    True
    >>> print chain.name
    reject
    >>> print chain.description
    Reject/bounce a message and stop processing.
    >>> file_pos = fp.tell()
    >>> process(mlist, msg, {}, 'reject')
    >>> fp.seek(file_pos)
    >>> print 'LOG:', fp.read()
    LOG: ... REJECT: <first>

The bounce message is now sitting in the Virgin queue.

    >>> virginq = config.switchboards['virgin']
    >>> len(virginq.files)
    1
    >>> qmsg, qdata = virginq.dequeue(virginq.files[0])
    >>> print qmsg.as_string()
    Subject: My first post
    From: _xtest-owner@example.com
    To: aperson@example.com
    ...
    [No bounce details are available]
    ...
    Content-Type: message/rfc822
    MIME-Version: 1.0
    <BLANKLINE>
    From: aperson@example.com
    To: _xtest@example.com
    Subject: My first post
    Message-ID: <first>
    <BLANKLINE>
    An important message.
    <BLANKLINE>
    ...


The Hold Chain
==============

The Hold chain places the message into the admin request database and
depending on the list's settings, sends a notification to both the original
sender and the list moderators.

    >>> chain = config.chains['hold']
    >>> verifyObject(IChain, chain)
    True
    >>> print chain.name
    hold
    >>> print chain.description
    Hold a message and stop processing.

    >>> file_pos = fp.tell()
    >>> process(mlist, msg, {}, 'hold')
    >>> fp.seek(file_pos)
    >>> print 'LOG:', fp.read()
    LOG: ... HOLD: _xtest@example.com post from aperson@example.com held,
        message-id=<first>: n/a
    <BLANKLINE>

There are now two messages in the Virgin queue, one to the list moderators and
one to the original author.

    >>> len(virginq.files)
    2
    >>> qfiles = []
    >>> for filebase in virginq.files:
    ...     qmsg, qdata = virginq.dequeue(filebase)
    ...     virginq.finish(filebase)
    ...     qfiles.append(qmsg)
    >>> from operator import itemgetter
    >>> qfiles.sort(key=itemgetter('to'))

This message is addressed to the mailing list moderators.

    >>> print qfiles[0].as_string()
    Subject: _xtest@example.com post from aperson@example.com requires approval
    From: _xtest-owner@example.com
    To: _xtest-owner@example.com
    MIME-Version: 1.0
    ...
    As list administrator, your authorization is requested for the
    following mailing list posting:
    <BLANKLINE>
        List:    _xtest@example.com
        From:    aperson@example.com
        Subject: My first post
        Reason:  XXX
    <BLANKLINE>
    At your convenience, visit:
    <BLANKLINE>
        http://lists.example.com/admindb/_xtest@example.com
    <BLANKLINE>
    to approve or deny the request.
    <BLANKLINE>
    ...
    Content-Type: message/rfc822
    MIME-Version: 1.0
    <BLANKLINE>
    From: aperson@example.com
    To: _xtest@example.com
    Subject: My first post
    Message-ID: <first>
    X-Message-ID-Hash: RXJU4JL6N2OUN3OYMXXPPSCR7P7JE2BW
    <BLANKLINE>
    An important message.
    <BLANKLINE>
    ...
    Content-Type: message/rfc822
    MIME-Version: 1.0
    <BLANKLINE>
    Content-Type: text/plain; charset="us-ascii"
    MIME-Version: 1.0
    Content-Transfer-Encoding: 7bit
    Subject: confirm ...
    Sender: _xtest-request@example.com
    From: _xtest-request@example.com
    ...
    <BLANKLINE>
    If you reply to this message, keeping the Subject: header intact,
    Mailman will discard the held message.  Do this if the message is
    spam.  If you reply to this message and include an Approved: header
    with the list password in it, the message will be approved for posting
    to the list.  The Approved: header can also appear in the first line
    of the body of the reply.
    ...

This message is addressed to the sender of the message.

    >>> print qfiles[1].as_string()
    MIME-Version: 1.0
    Content-Type: text/plain; charset="us-ascii"
    Content-Transfer-Encoding: 7bit
    Subject: Your message to _xtest@example.com awaits moderator approval
    From: _xtest-bounces@example.com
    To: aperson@example.com
    ...
    Your mail to '_xtest@example.com' with the subject
    <BLANKLINE>
        My first post
    <BLANKLINE>
    Is being held until the list moderator can review it for approval.
    <BLANKLINE>
    The reason it is being held:
    <BLANKLINE>
        XXX
    <BLANKLINE>
    Either the message will get posted to the list, or you will receive
    notification of the moderator's decision.  If you would like to cancel
    this posting, please visit the following URL:
    <BLANKLINE>
        http://lists.example.com/confirm/_xtest@example.com/...
    <BLANKLINE>
    <BLANKLINE>

In addition, the pending database is holding the original messages, waiting
for them to be disposed of by the original author or the list moderators.  The
database is essentially a dictionary, with the keys being the randomly
selected tokens included in the urls and the values being a 2-tuple where the
first item is a type code and the second item is a message id.

    >>> import re
    >>> cookie = None
    >>> for line in qfiles[1].get_payload().splitlines():
    ...     mo = re.search('confirm/[^/]+/(?P<cookie>.*)$', line)
    ...     if mo:
    ...         cookie = mo.group('cookie')
    ...         break
    >>> assert cookie is not None, 'No confirmation token found'

    >>> from mailman.interfaces.pending import IPendings
    >>> from zope.component import getUtility

    >>> data = getUtility(IPendings).confirm(cookie)
    >>> sorted(data.items())
    [(u'id', ...), (u'type', u'held message')]

The message itself is held in the message store.

    >>> from mailman.interfaces.requests import IRequests
    >>> list_requests = getUtility(IRequests).get_list_requests(mlist)
    >>> rkey, rdata = list_requests.get_request(data['id'])

    >>> from mailman.interfaces.messages import IMessageStore
    >>> from zope.component import getUtility
    >>> msg = getUtility(IMessageStore).get_message_by_id(
    ...     rdata['_mod_message_id'])

    >>> print msg.as_string()
    From: aperson@example.com
    To: _xtest@example.com
    Subject: My first post
    Message-ID: <first>
    X-Message-ID-Hash: RXJU4JL6N2OUN3OYMXXPPSCR7P7JE2BW
    <BLANKLINE>
    An important message.
    <BLANKLINE>


The Accept chain
================

The Accept chain sends the message on the 'prep' queue, where it will be
processed and sent on to the list membership.

    >>> chain = config.chains['accept']
    >>> verifyObject(IChain, chain)
    True
    >>> print chain.name
    accept
    >>> print chain.description
    Accept a message.
    >>> file_pos = fp.tell()
    >>> process(mlist, msg, {}, 'accept')
    >>> fp.seek(file_pos)
    >>> print 'LOG:', fp.read()
    LOG: ... ACCEPT: <first>

    >>> pipelineq = config.switchboards['pipeline']
    >>> len(pipelineq.files)
    1
    >>> qmsg, qdata = pipelineq.dequeue(pipelineq.files[0])
    >>> print qmsg.as_string()
    From: aperson@example.com
    To: _xtest@example.com
    Subject: My first post
    Message-ID: <first>
    X-Message-ID-Hash: RXJU4JL6N2OUN3OYMXXPPSCR7P7JE2BW
    <BLANKLINE>
    An important message.
    <BLANKLINE>


Run-time chains
===============

We can also define chains at run time, and these chains can be mutated.
Run-time chains are made up of links where each link associates both a rule
and a 'jump'.  The rule is really a rule name, which is looked up when
needed.  The jump names a chain which is jumped to if the rule matches.

There is one built-in run-time chain, called appropriately 'built-in'.  This
is the default chain to use when no other input chain is defined for a mailing
list.  It runs through the default rules, providing functionality similar to
the Hold handler from previous versions of Mailman.

    >>> chain = config.chains['built-in']
    >>> verifyObject(IChain, chain)
    True
    >>> print chain.name
    built-in
    >>> print chain.description
    The built-in moderation chain.

The previously created message is innocuous enough that it should pass through
all default rules.  This message will end up in the pipeline queue.

    >>> file_pos = fp.tell()
    >>> process(mlist, msg, {})
    >>> fp.seek(file_pos)
    >>> print 'LOG:', fp.read()
    LOG: ... ACCEPT: <first>

    >>> qmsg, qdata = pipelineq.dequeue(pipelineq.files[0])
    >>> print qmsg.as_string()
    From: aperson@example.com
    To: _xtest@example.com
    Subject: My first post
    Message-ID: <first>
    X-Message-ID-Hash: RXJU4JL6N2OUN3OYMXXPPSCR7P7JE2BW
    X-Mailman-Rule-Misses: approved; emergency; loop; administrivia;
        implicit-dest;
        max-recipients; max-size; news-moderation; no-subject;
        suspicious-header
    <BLANKLINE>
    An important message.
    <BLANKLINE>

In addition, the message metadata now contains lists of all rules that have
hit and all rules that have missed.

    >>> sorted(qdata['rule_hits'])
    []
    >>> for rule_name in sorted(qdata['rule_misses']):
    ...     print rule_name
    administrivia
    approved
    emergency
    implicit-dest
    loop
    max-recipients
    max-size
    news-moderation
    no-subject
    suspicious-header