Much progress, though not perfect, on migrating to SQLAlchemy 0.4 and Elixir

0.4.  Lots of things changes, which broke lots of our code.  There are still a
couple of failures in the test suite that I don't understand.  It seems that
for pending.txt and requests.txt, sometimes strings come back from the
database as 8-bit strings and other times as unicodes.  It's impossible to
make these tests work both separately and together.

users.txt is also failing intermittently.  Lots of different behavior between
running the full test suite all together and running individual tests.  Sigh.

Note also that actually, Elixir 0.4.0 doesn't work for us.  There's a bug in
that version that prevented zope.interfaces and Elixir working together.  Get
the latest 0.4.0 from source to fix this.

Other changes include:

- Remove Mailman/lockfile.py.  While I haven't totally eliminated locking, I
  have released the lockfile as a separate Python package called locknix,
  which Mailman 3.0 now depends on.

- Renamed Mailman/interfaces/messagestore.py and added an IMessage interface.

- bin/testall raises turns on SQLALCHEMY_ECHO when the verbosity is above 3
  (that's three -v's because the default verbosity is 1).

- add_domain() in config files now allows url_host to be optional.  If not
  given, it defaults to email_host.

- Added a non-public interface IDatabase._reset() used by the test suite to
  zap the database between doctests.  Added an implementation in the model
  which just runs through all rows in all entities, deleting them.

- [I]Pending renamed to [I]Pended

- Don't allow Pendings.add() to infloop.

- In the model's User impelementations, we don't need to append or remove the
  address when linking and unlinking.  By setting the address.user attribute,
  SQLAlchemy appears to do the right thing, though I'm not 100% sure of that
  (see the above mentioned failures).

2 files changed, 12 insertions, 90 deletions

diff --git a/Mailman/tests/test_documentation.py b/Mailman/tests/test_documentation.py
index 825a95a3c..d8578bd05 100644
--- a/Mailman/tests/test_documentation.py
+++ b/Mailman/tests/test_documentation.py
@@ -18,6 +18,7 @@
 """Harness for testing Mailman's documentation."""
 
 import os
+import pdb
 import doctest
 import unittest
 
@@ -33,48 +34,22 @@ COMMASPACE = ', '
 
 
 def cleaning_teardown(testobj):
-    usermgr = config.db.user_manager
-    listmgr = config.db.list_manager
-    # Remove all users, addresses and members, then delete all mailing lists.
-    for user in usermgr.users:
-        usermgr.delete_user(user)
-    for address in usermgr.addresses:
-        usermgr.delete_address(address)
-    for mlist in listmgr.mailing_lists:
-        for member in mlist.members.members:
-            member.unsubscribe()
-        for admin in mlist.administrators.members:
-            admin.unsubscribe()
-        requestdb = config.db.requests.get_list_requests(mlist)
-        for request in requestdb.held_requests:
-            requestdb.delete_request(request.id)
-        listmgr.delete(mlist)
+    """Clear all persistent data at the end of a doctest."""
+    # Clear the database of all rows.
+    config.db._reset()
     flush()
-    assert not list(listmgr.mailing_lists), (
-        'There should be no mailing lists left: %s' %
-        COMMASPACE.join(sorted(listmgr.names)))
-    assert not list(usermgr.users), (
-        'There should be no users left!')
-    assert not list(usermgr.addresses), (
-        'There should be no addresses left!')
-    # Remove all queue files.
-    for dirpath, dirnames, filenames in os.walk(config.QUEUE_DIR):
-        for filename in filenames:
-            os.remove(os.path.join(dirpath, filename))
     # Remove all but the default style.
     for style in style_manager.styles:
         if style.name <> 'default':
             style_manager.unregister(style)
-    # Clear the message store.
-    global_ids = []
-    for msg in config.db.message_store.messages:
-        global_ids.append('%s/%s' % (
-            msg['X-List-ID-Hash'], msg['X-List-Sequence-Number']))
-    for global_id in global_ids:
-        config.db.message_store.delete_message(global_id)
-    flush()
-    assert not list(config.db.message_store.messages), (
-        'There should be no messages left in the message store.')
+    # Remove all queue files.
+    for dirpath, dirnames, filenames in os.walk(config.QUEUE_DIR):
+        for filename in filenames:
+            os.remove(os.path.join(dirpath, filename))
+    # Clear out messages in the message store directory.
+    for dirpath, dirnames, filenames in os.walk(config.MESSAGES_DIR):
+        for filename in filenames:
+            os.remove(os.path.join(dirpath, filename))
 
 
 
diff --git a/Mailman/tests/test_lockfile.py b/Mailman/tests/test_lockfile.py
deleted file mode 100644
index 9d6420e74..000000000
--- a/Mailman/tests/test_lockfile.py
+++ /dev/null
@@ -1,53 +0,0 @@
-# Copyright (C) 2002-2007 by the Free Software Foundation, Inc.
-#
-# This program is free software; you can redistribute it and/or
-# modify it under the terms of the GNU General Public License
-# as published by the Free Software Foundation; either version 2
-# of the License, or (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program; if not, write to the Free Software
-# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
-# USA.
-
-"""Unit tests for the LockFile class."""
-
-import os
-import shutil
-import tempfile
-import unittest
-
-from Mailman.lockfile import LockFile
-
-LOCKFILE_NAME = '.mm-test-lock'
-
-
-
-class TestLockFile(unittest.TestCase):
-    def setUp(self):
-        self._tmpdir = tempfile.mkdtemp(prefix='mmtest')
-        self._lockf  = os.path.join(self._tmpdir, LOCKFILE_NAME)
-
-    def tearDown(self):
-        shutil.rmtree(self._tmpdir)
-
-    # XXX There really should be additional multi-thread or -proc tests, a la
-    # the __main__ of LockFile.py
-
-    def test_two_lockfiles_same_proc(self):
-        lf1 = LockFile(LOCKFILE_NAME)
-        lf2 = LockFile(LOCKFILE_NAME)
-        lf1.lock()
-        self.failIf(lf2.locked())
-
-
-
-def test_suite():
-    suite = unittest.TestSuite()
-    suite.addTest(unittest.makeSuite(TestLockFile))
-    return suite