Convert the Approve.py handler to an approved.py rule. Update the doctest to

use the rule check instead of handler processing.  Add a test for stripping
the header from the text/html part to the doctest.

Add Mailman.app.rules.find_rule() to return a named rule.

Fix a few typos.

1 files changed, 188 insertions, 128 deletions

diff --git a/Mailman/docs/approve.txt b/Mailman/docs/approve.txt
index 56afc1dd4..ea07058f8 100644
--- a/Mailman/docs/approve.txt
+++ b/Mailman/docs/approve.txt
@@ -13,98 +13,72 @@ approval queue.  This has several use cases:
 In order to support this, a mailing list can be given a 'moderator password'
 which is shared among all the administrators.
 
-    >>> from Mailman.Handlers.Approve import process
     >>> from Mailman.configuration import config
     >>> mlist = config.db.list_manager.create(u'_xtest@example.com')
+    >>> mlist.moderator_password = u'abcxyz'
+
+The 'approved' rule determines whether the message contains the proper
+approval or not.
 
+    >>> from Mailman.app.rules import find_rule
+    >>> rule = find_rule('approved')
+    >>> rule.name
+    'approved'
 
-Short circuiting
-----------------
 
-The message may have been approved by some other means, as evident in the
-message metadata.  In this case, the handler returns immediately.
+No approval
+-----------
+
+If the message has no Approve or Approved header, then the rule does not
+match.
 
     >>> msg = message_from_string(u"""\
     ... From: aperson@example.com
     ...
     ... An important message.
     ... """)
-    >>> msgdata = {'approved': True}
-    >>> process(mlist, msg, msgdata)
-    >>> print msg.as_string()
-    From: aperson@example.com
-    <BLANKLINE>
-    An important message.
-    <BLANKLINE>
-    >>> msgdata
-    {'approved': True}
+    >>> rule.check(mlist, msg, {})
+    False
 
+If the message has an Approve or Approved header with a value that does not
+match the moderator password, then the rule does not match.  However, the
+header is still removed.
 
-The Approved header
--------------------
+    >>> msg['Approve'] = u'12345'
+    >>> rule.check(mlist, msg, {})
+    False
+    >>> print msg['approve']
+    None
 
-If the moderator password is given in an Approved header, then the message
-gets sent through with no further posting moderation.  The Approved header is
-not stripped in this handler module, but instead in the Cleanse module.  This
-ensures that no moderator approval password in the headers will leak out.
+    >>> del msg['approve']
+    >>> msg['Approved'] = u'12345'
+    >>> rule.check(mlist, msg, {})
+    False
+    >>> print msg['approved']
+    None
 
-    >>> mlist.moderator_password = u'abcxyz'
-    >>> msg['Approved'] = u'abcxyz'
-    >>> msgdata = {}
-    >>> process(mlist, msg, msgdata)
-    >>> print msg.as_string()
-    From: aperson@example.com
-    Approved: abcxyz
-    <BLANKLINE>
-    An important message.
-    <BLANKLINE>
-    >>> sorted(msgdata.items())
-    [('adminapproved', True), ('approved', True)]
+    >>> del msg['approved']
 
-But if the wrong password is given, then the message is not marked as being
-approved.  The header is still removed though.
 
-    >>> del msg['Approved']
-    >>> msg['Approved'] = u'123456'
-    >>> msgdata = {}
-    >>> process(mlist, msg, msgdata)
-    >>> print msg.as_string()
-    From: aperson@example.com
-    Approved: 123456
-    <BLANKLINE>
-    An important message.
-    <BLANKLINE>
-    >>> msgdata
-    {}
+Using an approval header
+------------------------
 
-In the spirit of being liberal in what you accept, using an Approve header is
-completely synonymous.
+If the moderator password is given in an Approve header, then the rule
+matches, and the Approve header is stripped.
 
-    >>> del msg['Approved']
     >>> msg['Approve'] = u'abcxyz'
-    >>> msgdata = {}
-    >>> process(mlist, msg, msgdata)
-    >>> print msg.as_string()
-    From: aperson@example.com
-    Approve: abcxyz
-    <BLANKLINE>
-    An important message.
-    <BLANKLINE>
-    >>> sorted(msgdata.items())
-    [('adminapproved', True), ('approved', True)]
+    >>> rule.check(mlist, msg, {})
+    True
+    >>> print msg['approve']
+    None
 
-    >>> del msg['Approve']
-    >>> msg['Approve'] = u'123456'
-    >>> msgdata = {}
-    >>> process(mlist, msg, msgdata)
-    >>> print msg.as_string()
-    From: aperson@example.com
-    Approve: 123456
-    <BLANKLINE>
-    An important message.
-    <BLANKLINE>
-    >>> msgdata
-    {}
+Similarly, for the Approved header.
+
+    >>> msg['Approved'] = u'abcxyz'
+    >>> rule.check(mlist, msg, {})
+    True
+    >>> print msg['approved']
+    None
 
 
 Using a pseudo-header
@@ -113,17 +87,20 @@ Using a pseudo-header
 Different mail user agents have varying degrees to which they support custom
 headers like Approve and Approved.  For this reason, Mailman also supports
 using a 'pseudo-header', which is really just the first non-whitespace line in
-the payload of the message of the message.  If this pseudo-header looks like a
-matching Approve or Approved header, the message is similarly allowed to pass.
+the payload of the message.  If this pseudo-header looks like a matching
+Approve or Approved header, the message is similarly allowed to pass.
 
     >>> msg = message_from_string(u"""\
     ... From: aperson@example.com
     ...
-    ... Approved: abcxyz
+    ... Approve: abcxyz
     ... An important message.
     ... """)
-    >>> msgdata = {}
-    >>> process(mlist, msg, msgdata)
+    >>> rule.check(mlist, msg, {})
+    True
+
+The pseudo-header is removed.
+
     >>> print msg.as_string()
     From: aperson@example.com
     Content-Transfer-Encoding: 7bit
@@ -132,17 +109,18 @@ matching Approve or Approved header, the message is similarly allowed to pass.
     <BLANKLINE>
     An important message.
     <BLANKLINE>
-    >>> sorted(msgdata.items())
-    [('adminapproved', True), ('approved', True)]
+
+Similarly for the Approved header.
 
     >>> msg = message_from_string(u"""\
     ... From: aperson@example.com
     ...
-    ... Approve: abcxyz
+    ... Approved: abcxyz
     ... An important message.
     ... """)
-    >>> msgdata = {}
-    >>> process(mlist, msg, msgdata)
+    >>> rule.check(mlist, msg, {})
+    True
+
     >>> print msg.as_string()
     From: aperson@example.com
     Content-Transfer-Encoding: 7bit
@@ -151,8 +129,6 @@ matching Approve or Approved header, the message is similarly allowed to pass.
     <BLANKLINE>
     An important message.
     <BLANKLINE>
-    >>> sorted(msgdata.items())
-    [('adminapproved', True), ('approved', True)]
 
 As before, a mismatch in the pseudo-header does not approve the message, but
 the pseudo-header line is still removed.
@@ -160,11 +136,12 @@ the pseudo-header line is still removed.
     >>> msg = message_from_string(u"""\
     ... From: aperson@example.com
     ...
-    ... Approved: 123456
+    ... Approve: 123456
     ... An important message.
     ... """)
-    >>> msgdata = {}
-    >>> process(mlist, msg, msgdata)
+    >>> rule.check(mlist, msg, {})
+    False
+
     >>> print msg.as_string()
     From: aperson@example.com
     Content-Transfer-Encoding: 7bit
@@ -173,17 +150,18 @@ the pseudo-header line is still removed.
     <BLANKLINE>
     An important message.
     <BLANKLINE>
-    >>> msgdata
-    {}
+
+Similarly for the Approved header.
 
     >>> msg = message_from_string(u"""\
     ... From: aperson@example.com
     ...
-    ... Approve: 123456
+    ... Approved: 123456
     ... An important message.
     ... """)
-    >>> msgdata = {}
-    >>> process(mlist, msg, msgdata)
+    >>> rule.check(mlist, msg, {})
+    False
+
     >>> print msg.as_string()
     From: aperson@example.com
     Content-Transfer-Encoding: 7bit
@@ -192,8 +170,6 @@ the pseudo-header line is still removed.
     <BLANKLINE>
     An important message.
     <BLANKLINE>
-    >>> msgdata
-    {}
 
 
 MIME multipart support
@@ -211,18 +187,21 @@ used with MIME documents.
     ... --AAA
     ... Content-Type: application/x-ignore
     ...
-    ... Approved: 123456
+    ... Approve: 123456
     ... The above line will be ignored.
     ...
     ... --AAA
     ... Content-Type: text/plain
     ...
-    ... Approved: abcxyz
+    ... Approve: abcxyz
     ... An important message.
     ... --AAA--
     ... """)
-    >>> msgdata = {}
-    >>> process(mlist, msg, msgdata)
+    >>> rule.check(mlist, msg, {})
+    True
+
+Like before, the pseudo-header is removed, but only from the text parts.
+
     >>> print msg.as_string()
     From: aperson@example.com
     MIME-Version: 1.0
@@ -231,7 +210,7 @@ used with MIME documents.
     --AAA
     Content-Type: application/x-ignore
     <BLANKLINE>
-    Approved: 123456
+    Approve: 123456
     The above line will be ignored.
     <BLANKLINE>
     --AAA
@@ -242,8 +221,8 @@ used with MIME documents.
     An important message.
     --AAA--
     <BLANKLINE>
-    >>> sorted(msgdata.items())
-    [('adminapproved', True), ('approved', True)]
+
+The same goes for the Approved message.
 
     >>> msg = message_from_string(u"""\
     ... From: aperson@example.com
@@ -253,18 +232,21 @@ used with MIME documents.
     ... --AAA
     ... Content-Type: application/x-ignore
     ...
-    ... Approve: 123456
+    ... Approved: 123456
     ... The above line will be ignored.
     ...
     ... --AAA
     ... Content-Type: text/plain
     ...
-    ... Approve: abcxyz
+    ... Approved: abcxyz
     ... An important message.
     ... --AAA--
     ... """)
-    >>> msgdata = {}
-    >>> process(mlist, msg, msgdata)
+    >>> rule.check(mlist, msg, {})
+    True
+
+And the header is removed.
+
     >>> print msg.as_string()
     From: aperson@example.com
     MIME-Version: 1.0
@@ -273,7 +255,7 @@ used with MIME documents.
     --AAA
     Content-Type: application/x-ignore
     <BLANKLINE>
-    Approve: 123456
+    Approved: 123456
     The above line will be ignored.
     <BLANKLINE>
     --AAA
@@ -284,8 +266,6 @@ used with MIME documents.
     An important message.
     --AAA--
     <BLANKLINE>
-    >>> sorted(msgdata.items())
-    [('adminapproved', True), ('approved', True)]
 
 Here, the correct password is in the non-text/plain part, so it is ignored.
 
@@ -307,8 +287,11 @@ Here, the correct password is in the non-text/plain part, so it is ignored.
     ... An important message.
     ... --AAA--
     ... """)
-    >>> msgdata = {}
-    >>> process(mlist, msg, msgdata)
+    >>> rule.check(mlist, msg, {})
+    False
+
+And yet the pseudo-header is still stripped.
+
     >>> print msg.as_string()
     From: aperson@example.com
     MIME-Version: 1.0
@@ -327,8 +310,8 @@ Here, the correct password is in the non-text/plain part, so it is ignored.
     <BLANKLINE>
     An important message.
     --AAA--
-    >>> msgdata
-    {}
+
+As before, the same goes for the Approved header.
 
     >>> msg = message_from_string(u"""\
     ... From: aperson@example.com
@@ -338,18 +321,21 @@ Here, the correct password is in the non-text/plain part, so it is ignored.
     ... --AAA
     ... Content-Type: application/x-ignore
     ...
-    ... Approve: abcxyz
+    ... Approved: abcxyz
     ... The above line will be ignored.
     ...
     ... --AAA
     ... Content-Type: text/plain
     ...
-    ... Approve: 123456
+    ... Approved: 123456
     ... An important message.
     ... --AAA--
     ... """)
-    >>> msgdata = {}
-    >>> process(mlist, msg, msgdata)
+    >>> rule.check(mlist, msg, {})
+    False
+
+And the pseudo-header is removed.
+
     >>> print msg.as_string()
     From: aperson@example.com
     MIME-Version: 1.0
@@ -358,7 +344,7 @@ Here, the correct password is in the non-text/plain part, so it is ignored.
     --AAA
     Content-Type: application/x-ignore
     <BLANKLINE>
-    Approve: abcxyz
+    Approved: abcxyz
     The above line will be ignored.
     <BLANKLINE>
     --AAA
@@ -368,8 +354,14 @@ Here, the correct password is in the non-text/plain part, so it is ignored.
     <BLANKLINE>
     An important message.
     --AAA--
-    >>> msgdata
-    {}
+
+
+Stripping text/html parts
+-------------------------
+
+Because some mail readers will include both a text/plain part and a text/html
+alternative, the 'approved' rule has to search the alternatives and strip
+anything that looks like an Approve or Approved headers.
 
     >>> msg = message_from_string(u"""\
     ... From: aperson@example.com
@@ -377,30 +369,100 @@ Here, the correct password is in the non-text/plain part, so it is ignored.
     ... Content-Type: multipart/mixed; boundary="AAA"
     ...
     ... --AAA
-    ... Content-Type: application/x-ignore
+    ... Content-Type: text/html
+    ...
+    ... <html>
+    ... <head></head>
+    ... <body>
+    ... <b>Approved: abcxyz</b>
+    ... <p>The above line will be ignored.
+    ... </body>
+    ... </html>
+    ...
+    ... --AAA
+    ... Content-Type: text/plain
     ...
     ... Approved: abcxyz
-    ... The above line will be ignored.
+    ... An important message.
+    ... --AAA--
+    ... """)
+    >>> rule.check(mlist, msg, {})
+    True
+
+And the header-like text in the text/html part was stripped.
+
+    >>> print msg.as_string()
+    From: aperson@example.com
+    MIME-Version: 1.0
+    Content-Type: multipart/mixed; boundary="AAA"
+    <BLANKLINE>
+    --AAA
+    Content-Transfer-Encoding: 7bit
+    MIME-Version: 1.0
+    Content-Type: text/html; charset="us-ascii"
+    <BLANKLINE>
+    <html>
+    <head></head>
+    <body>
+    <b></b>
+    <p>The above line will be ignored.
+    </body>
+    </html>
+    <BLANKLINE>
+    --AAA
+    Content-Transfer-Encoding: 7bit
+    MIME-Version: 1.0
+    Content-Type: text/plain; charset="us-ascii"
+    <BLANKLINE>
+    An important message.
+    --AAA--
+    <BLANKLINE>
+
+This is true even if the rule does not match.
+
+    >>> msg = message_from_string(u"""\
+    ... From: aperson@example.com
+    ... MIME-Version: 1.0
+    ... Content-Type: multipart/mixed; boundary="AAA"
+    ...
+    ... --AAA
+    ... Content-Type: text/html
+    ...
+    ... <html>
+    ... <head></head>
+    ... <body>
+    ... <b>Approve: 123456</b>
+    ... <p>The above line will be ignored.
+    ... </body>
+    ... </html>
     ...
     ... --AAA
     ... Content-Type: text/plain
     ...
-    ... Approved: 123456
+    ... Approve: 123456
     ... An important message.
     ... --AAA--
     ... """)
-    >>> msgdata = {}
-    >>> process(mlist, msg, msgdata)
+    >>> rule.check(mlist, msg, {})
+    False
+
     >>> print msg.as_string()
     From: aperson@example.com
     MIME-Version: 1.0
     Content-Type: multipart/mixed; boundary="AAA"
     <BLANKLINE>
     --AAA
-    Content-Type: application/x-ignore
+    Content-Transfer-Encoding: 7bit
+    MIME-Version: 1.0
+    Content-Type: text/html; charset="us-ascii"
     <BLANKLINE>
-    Approved: abcxyz
-    The above line will be ignored.
+    <html>
+    <head></head>
+    <body>
+    <b></b>
+    <p>The above line will be ignored.
+    </body>
+    </html>
     <BLANKLINE>
     --AAA
     Content-Transfer-Encoding: 7bit
@@ -410,5 +472,3 @@ Here, the correct password is in the non-text/plain part, so it is ignored.
     An important message.
     --AAA--
     <BLANKLINE>
-    >>> msgdata
-    {}