Rework MailList.available_languages so that we don't need to use a PickleType

column in the database for this list of strings.  We use SQLAlchemy's
many-to-many relationship, however because of this, you cannot simply append
new unicodes to .available_languages.  You need to wrap the language code in a
Language instance and append that instance to the list.

In order to handle this, I added a property MailList.language_codes which
returns a list of the code strings (not Language instances).  Also new are
MailList.set_languages() for setting (i.e. overriding) the set of available
languages for the list; and add_language() which takes a single language code,
wraps it, and appends it.  The code does not and should not use
.available_languages directory any more.

MailList.GetAvailableLanguages() is removed.  The 'available_languages' column
is removed from the Listdata table.

Add a getValue() to Mailman.Gui.Language in order to unwrap the language codes
stored in the database's association table.  Modify _setValue() to do the
wrapping.

In dbcontext.py, don't import * from the sqlalchemy package.  It contains a
'logging' name which is not the standard Python logging package.  I also added
essentially a bag of attributes class called Tables which will hold references
to all the SA tables that are created.  Update the make_table() API to take an
instance of Tables.

Added a close() method to DBContext.  This is needed for the updated unit test
suite.

Changed bin/import.py so that when available_languages is being set, it calls
MailList.set_languages() instead of trying to set that attribute directly.

Updated some language idioms while I was at it.

More eradication of mm_cfg in favor of the config object and the Defaults
module.

In testall.py, call initialize() instead of loginit.initialize().

Promote MAX_RESTARTS into a Defaults.py.in variable.  This is because the unit
tests will knock that value down to something not so annoying should one of
the qrunner-required tests traceback.

Several other important changes to the unit test suite (which now completely
succeeds again!):

- Set the uid and gid of the temporary mailman.cfg and tmp*.db files to the
  Mailman user and group as specified in the config object.

- Make sure that all of the tests point to a SQLite database file that was
  created with the tempfile module.  This way we don't pollute our main
  database with data that is getting created during the unit tests.

- In the TestBase.setUp() method, be sure to close the existing dbcontext,
  clear out the mappers, and then reconnect the dbcontext with the new
  SQLALCHEMY_ENGINE_URL pointing to the tempfile.  However, we don't need to
  reload the MailList instance any more.

- Make all tests work, except for the tests that require crypt.  That upgrade
  path will not be available in this version of Mailman.

5 files changed, 91 insertions, 24 deletions

diff --git a/Mailman/database/address.py b/Mailman/database/address.py
index cea1ba072..672a366b1 100644
--- a/Mailman/database/address.py
+++ b/Mailman/database/address.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2006 by the Free Software Foundation, Inc.
+# Copyright (C) 2006-2007 by the Free Software Foundation, Inc.
 #
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
@@ -21,10 +21,10 @@ from sqlalchemy import *
 
 
 
-def make_table(metadata):
+def make_table(metadata, tables):
     table = Table(
         'Address', metadata,
         Column('address_id',    Integer, primary_key=True),
         Column('address',       Unicode(4096)),
         )
-    return table
+    tables.bind(table)
diff --git a/Mailman/database/dbcontext.py b/Mailman/database/dbcontext.py
index b2b1fb826..66c021447 100644
--- a/Mailman/database/dbcontext.py
+++ b/Mailman/database/dbcontext.py
@@ -16,15 +16,17 @@
 # USA.
 
 import os
+import logging
 import weakref
 
-from sqlalchemy import *
+from sqlalchemy import BoundMetaData, create_session
 from string import Template
 from urlparse import urlparse
 
 from Mailman import Version
 from Mailman.configuration import config
 from Mailman.database import address
+from Mailman.database import languages
 from Mailman.database import listdata
 from Mailman.database import version
 from Mailman.database.txnsupport import txn
@@ -37,10 +39,17 @@ class MlistRef(weakref.ref):
         self.fqdn_listname = mlist.fqdn_listname
 
 
+class Tables(object):
+    def bind(self, table, attrname=None):
+        if attrname is None:
+            attrname = table.name.lower()
+        setattr(self, attrname, table)
+
+
 
 class DBContext(object):
     def __init__(self):
-        self.tables = {}
+        self.tables = Tables()
         self.metadata = None
         self.session = None
         # Special transaction used only for MailList.Lock() .Save() and
@@ -69,21 +78,17 @@ class DBContext(object):
         self.metadata = BoundMetaData(url)
         self.metadata.engine.echo = config.SQLALCHEMY_ECHO
         # Create all the table objects, and then let SA conditionally create
-        # them if they don't yet exist.
-        version_table = None
-        for module in (address, listdata, version):
-            table = module.make_table(self.metadata)
-            self.tables[table.name] = table
-            if module is version:
-                version_table = table
+        # them if they don't yet exist.  NOTE: this order matters!
+        for module in (languages, address, listdata, version):
+            module.make_table(self.metadata, self.tables)
         self.metadata.create_all()
         # Validate schema version, updating if necessary (XXX)
-        from Mailman.interact import interact
-        r = version_table.select(version_table.c.component=='schema').execute()
+        r = self.tables.version.select(
+            self.tables.version.c.component=='schema').execute()
         row = r.fetchone()
         if row is None:
             # Database has not yet been initialized
-            version_table.insert().execute(
+            self.tables.version.insert().execute(
                 component='schema',
                 version=Version.DATABASE_SCHEMA_VERSION)
         elif row.version <> Version.DATABASE_SCHEMA_VERSION:
@@ -91,6 +96,9 @@ class DBContext(object):
             raise SchemaVersionMismatchError(row.version)
         self.session = create_session()
 
+    def close(self):
+        self.session.close()
+
     def _touch(self, url):
         parts = urlparse(url)
         # XXX Python 2.5; use parts.scheme and parts.path
@@ -176,7 +184,7 @@ class DBContext(object):
 
     @txn
     def api_get_list_names(self):
-        table = self.tables['Listdata']
+        table = self.tables.listdata
         results = table.select().execute()
         return [(row[table.c.list_name], row[table.c.host_name])
                 for row in results.fetchall()]
diff --git a/Mailman/database/languages.py b/Mailman/database/languages.py
new file mode 100644
index 000000000..6032a67df
--- /dev/null
+++ b/Mailman/database/languages.py
@@ -0,0 +1,53 @@
+# Copyright (C) 2007 by the Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
+# USA.
+
+"""Available languages table."""
+
+from sqlalchemy import *
+
+
+
+class Language(object):
+    def __init__(self, code):
+        self.code = code
+
+    def __repr__(self):
+        return u'<Language "%s">' % self.code
+
+    def __unicode__(self):
+        return self.code
+
+    __str__ = __unicode__
+
+
+
+def make_table(metadata, tables):
+    language_table = Table(
+        'Language', metadata,
+        # Two letter language code
+        Column('language_id',   Integer, primary_key=True),
+        Column('code',          Unicode),
+        )
+    # Associate List
+    available_languages_table = Table(
+        'AvailableLanguage', metadata,
+        Column('list_id',       Integer, ForeignKey('Listdata.list_id')),
+        Column('language_id',   Integer, ForeignKey('Language.language_id')),
+        )
+    mapper(Language, language_table)
+    tables.bind(language_table)
+    tables.bind(available_languages_table, 'available_languages')
diff --git a/Mailman/database/listdata.py b/Mailman/database/listdata.py
index 3c51f694b..be361d586 100644
--- a/Mailman/database/listdata.py
+++ b/Mailman/database/listdata.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2006 by the Free Software Foundation, Inc.
+# Copyright (C) 2006-2007 by the Free Software Foundation, Inc.
 #
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
@@ -21,7 +21,7 @@ from sqlalchemy import *
 
 
 
-def make_table(metadata):
+def make_table(metadata, tables):
     table = Table(
         'Listdata', metadata,
         # Attributes not directly modifiable via the web u/i
@@ -59,7 +59,6 @@ def make_table(metadata):
         Column('autoresponse_graceperiod',                      Integer),
         Column('autoresponse_postings_text',                    Unicode),
         Column('autoresponse_request_text',                     Unicode),
-        Column('available_languages',                           PickleType),
         Column('ban_list',                                      PickleType),
         Column('bounce_info_stale_after',                       Integer),
         Column('bounce_matching_headers',                       Unicode),
@@ -148,10 +147,17 @@ def make_table(metadata):
         )
     # Avoid circular imports
     from Mailman.MailList import MailList
+    from Mailman.database.languages import Language
     # We need to ensure MailList.InitTempVars() is called whenever a MailList
     # instance is created from a row.  Use a mapper extension for this.
-    mapper(MailList, table, extension=MailListMapperExtension())
-    return table
+    props = dict(available_languages=
+                 relation(Language,
+                          secondary=tables.available_languages,
+                          lazy=False))
+    mapper(MailList, table,
+           extension=MailListMapperExtension(),
+           properties=props)
+    tables.bind(table)
 
 
 
diff --git a/Mailman/database/version.py b/Mailman/database/version.py
index 93b97e470..57c50b0ef 100644
--- a/Mailman/database/version.py
+++ b/Mailman/database/version.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2006 by the Free Software Foundation, Inc.
+# Copyright (C) 2006-2007 by the Free Software Foundation, Inc.
 #
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
@@ -21,11 +21,11 @@ from sqlalchemy import *
 
 
 
-def make_table(metadata):
+def make_table(metadata, tables):
     table = Table(
         'Version', metadata,
         Column('version_id',    Integer, primary_key=True),
         Column('component',     String(20)),
         Column('version',       Integer),
         )
-    return table
+    tables.bind(table)