Passwords done right.

First off, there are several password hashing schemes added including SHA,
salted-SHA, and RFC 2989 PBKDF2 (contributed by Bob Fleck).  Then we encode
the password using RFC 2307 style syntax.  At least I think: specifically
things like the PRF and iteration count for PBKDF2 are encoded the way I
/think/ is intended for RFC 2307 but I could be wrong.  Seems darn hard to
find definitive information about that.

In any event, even though CLEARTEXT passwords are supported, they are mostly
deprecated, even for user passwords.  It also allows us to easily update all
passwords to a new hashing scheme when the existing schemes get cracked.  The
default scheme (specified in Defaults.py.in) is salted-SHA with a 20 byte salt
(the salt length and PBKDF2 iteration counts can only be specified in the
passwords.py file).

These hashed passwords are used for user passwords, list owner and moderator
passwords, and site and list creator passwords.

Of course this means that user password reminders are impossible now.  They've
been ripped out of the code for a while, but now we'll need to implement
password resets since user passwords cannot be recovered.

bin/export has had several changes:

- export no longer converts to dollar strings.  Were assuming dollar strings
  are used by default for all new lists and any imported lists will already be
  converted to dollar strings.
- Likewise, rip out the password scheme stuff, since cleartext passwords can
  never be exported, so we might as well always include the member's hashed
  password.
- Fix exporting to stdout when that stream can only handle ascii by wrapping
  stdout in a utf-8 codec writer.

Other changes:

- add a missing import to HTTPRunner.py

- Convert GUIBase.py to use Defaults.* for constants instead of mm_cfg.*

- Remove pre-Python 2.4 compatibility from Utils.py.  We've already said
  Python 2.4 will be a minimum requirement.

- Change the permissions on the global password file.  The default 007 umask
  is used and should be good enough.

- bin/newlist adds the ability to specify the password scheme (or list the
  available schemes) for the list owner password.  It is not possible to set
  the scheme on a per-list basis.  bin/mmsitepass does the same, but for the
  site and list creator passwords.

- Fix a nasty problem with bin/import.  The comment in the code says it best:

                # XXX Here's what sucks.  Some properties need to have
                # _setValue() called on the gui component, because those
                # methods do some pre-processing on the values before they're
                # applied to the MailList instance.  But we don't have a good
                # way to find a category and sub-category that a particular
                # property belongs to.  Plus this will probably change.  So
                # for now, we'll just hard code the extra post-processing
                # here.  The good news is that not all _setValue() munging
                # needs to be done -- for example, we've already converted
                # everything to dollar strings.

- Set the 'debug' logger to logging.DEBUG level.  It doesn't seem to make much
  sense for the debugging log to ignore debug messages.

1 files changed, 182 insertions, 0 deletions

diff --git a/Mailman/passwords.py b/Mailman/passwords.py
new file mode 100644
index 000000000..30c436a01
--- /dev/null
+++ b/Mailman/passwords.py
@@ -0,0 +1,182 @@
+# Copyright (C) 2007 by the Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
+# USA.
+
+"""Password hashing and verification schemes.
+
+Represents passwords using RFC 2307 syntax.
+"""
+
+import os
+import re
+import sha
+import hmac
+
+from array import array
+from base64 import urlsafe_b64decode as decode
+from base64 import urlsafe_b64encode as encode
+
+SALT_LENGTH = 20 # bytes
+ITERATIONS  = 2000
+
+
+
+class PasswordScheme(object):
+    @staticmethod
+    def make_secret(password):
+        """Return the hashed password""" 
+        raise NotImplementedError
+
+    @staticmethod
+    def check_response(challenge, response):
+        """Return True if response matches challenge.
+
+        It is expected that the scheme specifier prefix is already stripped
+        from the response string.
+        """
+        raise NotImplementedError
+
+
+
+class NoPasswordScheme(PasswordScheme):
+    @staticmethod
+    def make_secret(password):
+        return '{NONE}'
+
+    @staticmethod
+    def check_response(challenge, response):
+        return False
+
+
+
+class ClearTextPasswordScheme(PasswordScheme):
+    @staticmethod
+    def make_secret(password):
+        return '{CLEARTEXT}' + password
+
+    @staticmethod
+    def check_response(challenge, response):
+        return challenge == response
+
+
+
+class SHAPasswordScheme(PasswordScheme):
+    @staticmethod
+    def make_secret(password):
+        h = sha.new(password)
+        return '{SHA}' + encode(h.digest())
+
+    @staticmethod
+    def check_response(challenge, response):
+        h = sha.new(response)
+        return challenge == encode(h.digest())
+
+
+
+class SSHAPasswordScheme(PasswordScheme):
+    @staticmethod
+    def make_secret(password):
+        salt = os.urandom(SALT_LENGTH)
+        h = sha.new(password)
+        h.update(salt)
+        return '{SSHA}' + encode(h.digest() + salt)
+
+    @staticmethod
+    def check_response(challenge, response):
+        # Get the salt from the challenge
+        challenge_bytes = decode(challenge)
+        digest = challenge_bytes[:20]
+        salt = challenge_bytes[20:]
+        h = sha.new(response)
+        h.update(salt)
+        return digest == h.digest()
+
+
+
+# Given by Bob Fleck
+class PBKDF2PasswordScheme(PasswordScheme):
+    @staticmethod
+    def _pbkdf2(password, salt, iterations):
+        """From RFC2898 sec. 5.2.  Simplified to handle only 20 byte output
+        case.  Output of 20 bytes means always exactly one block to handle,
+        and a constant block counter appended to the salt in the initial hmac
+        update.
+        """
+        h = hmac.new(password, None, sha)
+        prf = h.copy()
+        prf.update(salt + '\x00\x00\x00\x01')
+        T = U = array('l', prf.digest())
+        while iterations:
+            prf = h.copy()
+            prf.update(U.tostring())
+            U = array('l', prf.digest())
+            T = array('l', (t ^ u for t, u in zip(T, U)))
+            iterations -= 1
+        return T.tostring()
+
+    @staticmethod
+    def make_secret(password):
+        """From RFC2898 sec. 5.2.  Simplified to handle only 20 byte output
+        case.  Output of 20 bytes means always exactly one block to handle,
+        and a constant block counter appended to the salt in the initial hmac
+        update.
+        """
+        salt = os.urandom(SALT_LENGTH)
+        digest = PBKDF2PasswordScheme._pbkdf2(password, salt, ITERATIONS)
+        derived_key = encode(digest + salt)
+        return '{PBKDF2 SHA %d}' % ITERATIONS + derived_key
+
+    @staticmethod
+    def check_response(challenge, response, prf, iterations):
+        # Decode the challenge to get the number of iterations and salt
+        # XXX we don't support anything but sha prf
+        if prf.lower() <> 'sha':
+            return False
+        try:
+            iterations = int(iterations)
+        except (ValueError, TypeError):
+            return False
+        challenge_bytes = decode(challenge)
+        digest = challenge_bytes[:20]
+        salt = challenge_bytes[20:]
+        key = PBKDF2PasswordScheme._pbkdf2(response, salt, iterations)
+        return digest == key
+
+
+
+SCHEMES = {
+    'none'      : NoPasswordScheme,
+    'cleartext' : ClearTextPasswordScheme,
+    'sha'       : SHAPasswordScheme,
+    'ssha'      : SSHAPasswordScheme,
+    'pbkdf2'    : PBKDF2PasswordScheme,
+    }
+
+
+def make_secret(password, scheme):
+    scheme_class = SCHEMES.get(scheme.lower(), NoPasswordScheme)
+    return scheme_class.make_secret(password)
+
+
+def check_response(challenge, response):
+    mo = re.match(r'{(?P<scheme>[^}]+?)}(?P<rest>.*)',
+                  challenge, re.IGNORECASE)
+    if not mo:
+        return False
+    scheme, rest = mo.group('scheme', 'rest')
+    scheme_parts = scheme.split()
+    scheme_class = SCHEMES.get(scheme_parts[0].lower(), NoPasswordScheme)
+    return scheme_class.check_response(rest, response, *scheme_parts[1:])