summaryrefslogtreecommitdiff
path: root/Mailman/MailList.py
Commit message (Collapse)AuthorAgeFilesLines
* Bite the bullet: rename the Mailman package to mailman.Barry Warsaw2008-02-271-731/+0
|
* Tweak copyright years.Barry Warsaw2008-02-071-1/+1
|
* General cleanups some of which is even tested <wink>. Mailman.LockFile moduleBarry Warsaw2007-10-101-104/+0
| | | | | | | | | | | | | is moved to Mailman.lockfile. Remove a few more MailList methods that aren't used any more, e.g. the lock related stuff, the Save() and CheckValues() methods, as well as ChangeMemberName(). Add a missing import to lifecycle.py. We no longer need withlist to unlock the mailing list. Also, expose config.db.flush() in the namespace of withlist directly, under 'flush'.
* Remove the action.py module, move this to Mailman/interfaces/__init__.py.Barry Warsaw2007-10-101-3/+0
| | | | | | Convert IMailingList.personalize to a enum. Change all non-obsolete occurances of GetListEmail() to posting_address.
* Removed a bunch of files that are obsolete. The interfaces are all foldedBarry Warsaw2007-09-271-20/+0
| | | | | | | | | | | | | into the IMailingList interface. OTOH, MemberAdaptor.py is completely useless now (though not entirely eradicated), as is OldStyleMemberships.py. versions.py isn't necessary any longer either because we'll have to do database migrations (and conversions from MM2.1) completely differently. New command line script 'set_members' which is used to take a CSV file and syncing that to a list's membership. Added back the DeliveryStatus.unknown item because we'll need it when we migrate MM 2.1 databases.
* OMGW00T: After over a decade, the MailList mixin class is gone! Well,Barry Warsaw2007-09-211-231/+8
| | | | | | | | | | | | mostly. It's no longer needed by anything in the test suite, and therefore the list manager returns database MailingList objects directly. The wrapper cruft has been removed. To accomplish this, a couple of hacks were added to the Mailman.app package, which will get cleaned up over time. The MailList module itself (and its few remaining mixins) aren't yet removed from the tree because some of the code is still not tested, and I want to leave this code around until I've finished converting it.
* InitTempVars() is completely eradicated. The only bit I think weBarry Warsaw2007-09-191-59/+9
| | | | | | | | | | | | | | | | | still need temporarily is the _gui component initialization, so this has been moved into MailList.__init__(). Fixed the __getattr__() super call to properly dispatch up. Removed the _memberadaptor instance variable initialization. The whole MemberAdaptor stuff is next on the chopping block. Essentially MailList locking is gone too now, although it's not yet completely eradicated. However, the __repr__() no longer states the lock status. The full_path property is now just an attribute on the underlying MailingList database object.
* Implement a context manager for Python 2.5's with statement, which isBarry Warsaw2007-09-191-7/+5
| | | | | | | | used where we used to do a try/except to temporarily change the global translation language. This makes the code shorter and cleaner. E.g. with i18n.using_language(another_language): # do something
* Finish up the request hold conversion. ListAdmin can go away thoughBarry Warsaw2007-09-161-31/+0
| | | | | | it hasn't yet. SendSubscribeAck(), SendUnsubscribeAck(), and ApprovedDeleteMember() are all removed, though the latter is not yet completely eradicated.
* Work out subscription holding and handling. ApprovedAddMember() isBarry Warsaw2007-09-161-80/+1
| | | | | | gone. Fix up OwnerNotification class. Rewrite subauth.txt and subscribeack.txt for new implementation, especially removing the password notification from the latter. Add lots of tests.
* ListAdmin mostly gone, but not quite.Barry Warsaw2007-09-091-3/+1
| | | | | | | | | | | | | | | | Mailman/app/moderator.py: Most of the application level interface provided by ListAdmin is moved here now, including the ability to hold messages, subscriptions, and unsubscriptions, and to handle message (defer, discard, reject, accept). More work needed. Some untested conversion of API in Mailman/Cgi/admindb.py, confirm.py, bin/checkdbs.py. messagestore.py: Don't use or require the Date: header in the global message ID calculation. As described on the mailing list, we're only going to use the Message-ID header. IListRequests: added count_of() and of_type() methods.
* It all started by trying to remove MailList.Create() and use the IListManagerBarry Warsaw2007-08-051-188/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | interface to create and delete lists. Mostly that's working now, but I need unit tests for most of the new work contained in this revision. Implemented a rudimentary 'list styles' subsystem, along with interfaces, but no tests yet. Moved all of MailList.InitVars() into a DefaultStyle, which is always available at priority zero. It's used by default if there are no matching styles for a mailing list. Because of the list styles, we can now get rid of (almost) all InitVars() methods. And because of /that/ we can get rid of the mixin clases whose sole purpose was to provide an InitVars() method. Yay for code removal! Mixin modules/classes removed: Autoresponder, GatewayManager, TopicManager. Removed the Mailman/ext crufty extension mechanism. Extensions will now be done using setuptools plugins. Hopefully this will take us everywhere we need to go, but I'll add Mailman.ext back if necessary later. Mailiman.app.create module added to implement a common, higher-level list creation feature. This is used by bin/newlist now, though some of that functionality (namely, ensuring the owners exist in the database, and notifying the owners) should be moved here. The MTA plugins aren't yet integrated into this, but need to be. Mailman.app.plugins module added to generalize setuptools plugin management. Defaults.DEFAULT_REPLY_GOES_TO_LIST now gets initialized with a proper enum. Also, the duplicate DeliveryMode and DeliveryStatus enums are removed from Defaults because they're in Mailman.constants. Added Errors.DuplicateStyleError. Updated Utils.list_exists() to use the new IListManager.get() interface, which has been changed to return None if the list doesn't exist (for consistency) instead of raising an exception. Utils.list_names() also needed to be fixed to use config.db.list_manager. bin/make_instance, bin/newlist, bin/rmlist changed to use parser.error() istead of printing to sys.stderr and sys.exit(1). bin/newlist and bin/rmlist now works with the IListManager interface, so you can create and delete lists from the command line again. The CLI for newlist has been much simplified; it no longer prompts for missing positional arguments. It now uses a more traditional CLI. newlist also accepts zero to many owners, and it ensures that the owners are all in the database. It no longer asks for a list password, because this doesn't make sense any more. bin/withlist has also been fixed to work with the IListManager interface. There are lots of XXXs and FIXMEs that need to be resolved before this can land. Also, we need to test all this stuff before it can land. Configuration.load() is now taught to search in sys.argv[0] for var/etc/mailman.cfg since this is where it is for egg development layouts. Also, VAR_DIR must be abspath'd. Added an __all__ to Mailman.constants, and added an Action enum. The listmanager implementation has to set the mlist.created_at time. There's also a bit of crufty refactoring going on to instantiate the roster objects whenever the list is created or retrieved from the database. Several MailingList column types are now set to our custom TimeDeltaType, which knows how to store a datetime.timedelta. A SQLAlchemny converter type is added to Mailman.database.types. I also fixed a bug in the EnumType implementation.
* Move the pending database into the SQLAlchemy/Elixir layer. The oldBarry Warsaw2007-08-011-6/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Pending.py module is removed. Added an interface to this functionality such that any IPendable (essentially a key/value mapping) can be associated with a token, and that token can be confirmed and has a lifetime. Any keys and values can be stored, as long as both are unicodes. Added a doctest. Modified initialization of the database layer to support pluggability via setuptools. No longer is this layer initialized from a module, but now it's instantiated from a class that implements IDatabase. The StockDatabase class implements the SQLAchemy/Elixir layer, but this can be overridden in a setup.py. Bye bye MANAGERS_INIT_FUNCTION, we hardly knew ye. Added a package Mailman.app which will contain certain application specific functionality. Right now, the only there there is an IRegistar implementation, which didn't seem to fit anywhere else. Speaking of which, the IRegistrar interface implements all the logic related to registration and verification of email addresses. Think the equivalent of MailList.AddMember() except generalized out of a mailing list context. This latter will eventually go away. The IRegistrar sends the confirmation email. Added an IDomain interface, though the only implementation of this so far lives in the registration.txt doctest. This defines the context necessary for domain-level things, like address confirmation. A bunch of other cleanups in modules that are necessary due to the refactoring of Pending, but don't affect anything that's actually tested yet, so I won't vouch for them (except that they don't throw errors on import!). Clean up Defaults.py; also turn the functions seconds(), minutes(), hours() and days() into their datetime.timedelta equivalents. Consolidated the bogus email address exceptions. In some places where appropriate, use email 4.0 module names instead of the older brand. Switch from Mailman.Utils.unique_message_id() to email.utils.make_msgid() everywhere. This is because we need to allow sending not in the context of a mailing list (i.e. domain-wide address confirmation message). So we can't use a Message-ID generator that requires a mailing list. OTOH, this breaks Message-ID collision detection in the mail->news gateway. I'll fix that eventually. Remove the 'verified' row on the Address table. Now verification is checked by Address.verified_on not being None.
* Major surgery to get the setuptools based installation passing all theBarry Warsaw2007-07-161-6/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | existing unit tests. Here's a summary of the changes. - Removed all dependent third party packages, since the setup.py file now claims all package dependencies such that they can be automatically installed from the cheeseshop. - Moved the misc directory into the Mailman package as Mailman/data. Moved templates and messages to Mailman subpackages. - Added an ILanguageManager interface, plus an implementation, so that we don't use Defaults.LC_DESCRIPTIONS directly anymore. Added a doctest for this interface and implementation. Defaults.LANGUAGES is moved into mailman.cfg. Defaults.LANGUAGE_DICT is moved to _DEFAULT_LANGUAGE_DATA, and LC_DESCRIPTIONS is removed. The calculation of the available and enabled languages is moved to the Configuration class, but this will probably still need work. Utils.GetLanguageDescr() and Utils.IsLanguage() are removed. I'd like to remove GetCharSet() eventually too, but there are too many uses of this currently, so I'm deferring it. - Utils.findtext(): Hacks added so that templates can be retrieved from the language catalog. The hack is that the template contents are used to find the translation, but in the one test case where this is actually flexed, the trailing newline in the file contents has to be trimmed. This is probably not right. - No more Defaults.py.in or mm_cfg.py! Defaults.py.in is moved to Defaults.py and is no longer created from a template file. The script called make_instance is added which creates an etc/mailman.cfg file from mailman.cfg.in (previously, mailman.cfg.sample) and /that/ file now has the small number of calculated values. In general, make_instance will not touch mailman.cfg if it exists, unless the --force option is given. CGIEXT is made the empty string by default (i.e. not generated). make_instance grows a --var-dir option. Fleshed out the --languages opton. - Defaults.py grows a DEFAULT_VAR_DIRECTORY variable, which is the default location of the 'var' directory. The Configuration class uses this as one of the directories it searches for its landmark, i.e. etc/mailman.cfg. RUNTIME_DIR is gone, as is VAR_PREFIX. - testall needs to write MAILMAN_USER, MAILMAN_UID, MAILMAN_GROUP, MAILMAN_GID, and LANGUAGES run time variables. - bin/withlist no longer needs to add config.BIN_DIR to sys.path, because in fact that variable doesn't exist any more. - Tweak the French catalog to make a test work. This is needed because of the conversion from %-strings to $-strings. - The setup.py now generates the .mo files before it does its thing. This will have to be fixed, but for now we must generate these files on setup build time instead of installation time. - Removed an unused interface.
* Convert TestHold to a doctest and update code as necessary. No generalBarry Warsaw2007-07-031-7/+2
| | | | modernization of Hold.py was performed.
* Convert TestFileRecips to a doctest, and update the handler to more modernBarry Warsaw2007-07-021-2/+3
| | | | | | | | Python idioms. The recipients are now returned as a set instead of a list, so duplicates are quashed. In MailList.InitTempVars() we need to create the list's data directory when the list is initialized. If the directory already exists, this does nothing.
* Implement the new, simplified membership model. Rosters and RosterSets asBarry Warsaw2007-06-091-8/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | they were previously known are now gone. Rosters, rather than being a database entity that collects users, is now just a filter on the member database. This way, we can use generic rosters to search for regular members, digest members, owners, or moderators. More advanced rosters can do all kinds of other membership queries. But rosters no longer need to be a database entity. Users have a name, password, optional preferences, and a set of addresses, but users are not subscribed to mailing lists. Addresses have the email address, some verification information, and optional preferences. Members tie an address to a mailing list, through a role, with optional preferences. Other changes here include: MailList.fqdn_listname() moved to the MailingList model entity. Added MemberRole enum and SystemDefaultPreferences to Mailman.constants. Profiles are renamed to Preferences (same with the interface), but the files are not yet moved. This happens later. We mostly don't need has_*() relationships on the entity classes, because we generally don't need the reverse relationship. Use belongs_to() because that creates the foreign key, even though the wording seems counter intuitive. IAddress.subscribe() added. Tell Elixir to use shortnames for all tables. Remove the OldStyleMembership fields from MailingList. Remove all the interface elements and database fields that talk about rosters and rostersets. Convert Version entity to has_field().
* Remove leftover reference to removed Mailman.database.tables.languagesbwarsaw2007-05-301-1/+2
| | | | | | | | | module. Add in a temporary reference (with XXX comment) to Mailman.database.model.Language where necessary, although I will need to remove this eventually. Added clean up code to the mlist-rosters.txt doctest, otherwise it leaks the test mailing list.
* Merge exp-elixir-branch to trunk. There is enough working to make me feelbwarsaw2007-05-281-128/+87
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | confident the Elixir branch is ready to become mainline. Also, fewer branches makes for an easier migration to a dvcs. Don't expect much of the old test suite to work, or even for much of the old functionality to work. The changes here are disruptive enough to break higher level parts of Mailman. But that's okay because I am slowly building up a new and improved test suite, which will lead to a functional system again. For now, only the doctests in Mailman/docs (and their related test harnesses) will pass, but they all do pass. Note that Mailman/docs serve as system documentation first and unit tests second. You should be able to read the doctest files to understand the underlying data model. Other changes included in this merge: - Added the Mailman.ext extension package. - zope.interfaces uses to describe major components - SQLAlchemy/Elixir used as the database model - Top level doinstall target renamed to justinstall - 3rd-party packages are now installed in pythonlib/lib/python to be more compliant with distutils standards. This allows us to use just --home instead of all the --install-* options. - No longer need to include the email package or pysqlite, as Python 2.5 is required (and comes with both packages). - munepy package is included, for Python enums - IRosterSets are added as a way to manage a collection of IRosters. Roster sets are named so that we can maintain the indirection between mailing lists and rosters, where the two are maintained in different storages. - IMailingListRosters: remove_*_roster() -> delete_*_roster() - Remove IMember interface. - Utils.list_names() -> config.list_manager.names - fqdn_listname() takes an optional hostname argument. - Added a bunch of new exceptions used throughout the new interfaces. - Make LockFile a context manager for use with the 'with' statement.
* Improve the way we handle avoiding InitTempVars() multiple times on anbwarsaw2007-04-091-11/+27
| | | | | | | | | | | | | | | | | | | instantiated MailList object via the mapper extension's populate_instance() method. This based on information from the SQLAlchemy folks. Add more useful output for LockFile debugging. Add checks in loginit.py's emit() method (and .flush()) so that if the stream has been closed, log messages will go to stderr. This happens under the test suite with SQLAlchemy, because SA keeps references to the MailList objects which it doesn't seem like we can clear before Python exits. So if say the lock logger is at debug level, when the lock object gets cleared at Python shutdown, the stream will have been closed by the time LockFile.__del__() gets called. This change avoids the traceback at the expense of a little extra stderr output. MailList.__lock -> MailList._lock MailList.__timestamp -> MailList._timestamp
* Moved the session.expire() to the MailList.Load() method, viabwarsaw2007-04-031-0/+1
| | | | | | | | | | | | | | | | | DBContext.api_load(). This is where it really ought to be based on the internal semantics of .Load()/.Lock(). MailListMapperExtension.populate_instance(): Checking the state of the isnew flag is not sufficient to know whether the MailList object we're getting is brand spankin' new or not. It turns out that when we session.expire() the MailList object, the next time SA loads this from the db, the populate_instance() will get called with isnew=True, even though the object really isn't new. Instead, check to make sure InitTempVars() isn't incorrectly called twice. Note that I might move this test, but I wanted to check in something that works, and then see if this is what we expect from the SA guys (this flag appears underdocumented). LockFile.py: Add some additional debugging.
* Test suite repair. All tests are now passing again.bwarsaw2007-03-211-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | - In i18n.py, change this method so that everything it returns will be guaranteed to be a unicode. Mailman 2.2 will be unicode-safe, meaning all strings internally will be unicodes. The translation service is one boundary point were strings come from the outside, so ensure that they are unicodes and convert if necessary. This may break some things, but it's better to fix those situations than to continue to return 8-bit strings from _(). - In Mailman/testing/base.py, craft a fake module called Mailman.MTA.stub and stick no-op functions on stub.create() and stub.remove(). We really don't need the MTA modules for testing purposes (yet at least), and if you're using the default configuration, you'll get tons of cruft on stdout when the Manual MTA tries to add and remove mailing lists. Set up the test configuration environment to use this stub MTA module. - In test_handlers.py, remove an extraneous str(). - Convert ToDigest.py, Hold.py and Acknowledge.py to __i18n_templates__. (I'm pretty darn close to just making everything use $-strings by default.) - In CookHeaders.py, there's no need to unicode()-ify the subject since that should already be a unicode when passed from _(). - In MailList.py, we can use the str.capitalize() method.
* Rework MailList.available_languages so that we don't need to use a PickleTypebwarsaw2007-01-181-12/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | column in the database for this list of strings. We use SQLAlchemy's many-to-many relationship, however because of this, you cannot simply append new unicodes to .available_languages. You need to wrap the language code in a Language instance and append that instance to the list. In order to handle this, I added a property MailList.language_codes which returns a list of the code strings (not Language instances). Also new are MailList.set_languages() for setting (i.e. overriding) the set of available languages for the list; and add_language() which takes a single language code, wraps it, and appends it. The code does not and should not use .available_languages directory any more. MailList.GetAvailableLanguages() is removed. The 'available_languages' column is removed from the Listdata table. Add a getValue() to Mailman.Gui.Language in order to unwrap the language codes stored in the database's association table. Modify _setValue() to do the wrapping. In dbcontext.py, don't import * from the sqlalchemy package. It contains a 'logging' name which is not the standard Python logging package. I also added essentially a bag of attributes class called Tables which will hold references to all the SA tables that are created. Update the make_table() API to take an instance of Tables. Added a close() method to DBContext. This is needed for the updated unit test suite. Changed bin/import.py so that when available_languages is being set, it calls MailList.set_languages() instead of trying to set that attribute directly. Updated some language idioms while I was at it. More eradication of mm_cfg in favor of the config object and the Defaults module. In testall.py, call initialize() instead of loginit.initialize(). Promote MAX_RESTARTS into a Defaults.py.in variable. This is because the unit tests will knock that value down to something not so annoying should one of the qrunner-required tests traceback. Several other important changes to the unit test suite (which now completely succeeds again!): - Set the uid and gid of the temporary mailman.cfg and tmp*.db files to the Mailman user and group as specified in the config object. - Make sure that all of the tests point to a SQLite database file that was created with the tempfile module. This way we don't pollute our main database with data that is getting created during the unit tests. - In the TestBase.setUp() method, be sure to close the existing dbcontext, clear out the mappers, and then reconnect the dbcontext with the new SQLALCHEMY_ENGINE_URL pointing to the tempfile. However, we don't need to reload the MailList instance any more. - Make all tests work, except for the tests that require crypt. That upgrade path will not be available in this version of Mailman.
* A rudimentary import script. Several things are not yet imported, includingbwarsaw2006-12-301-1/+1
| | | | | | | | | | | | | | | | | | | header_filters and topics (both list topics and user topic selections). Everything else seems to work pretty well. dbcontext.py: Don't key the mlist transactions off of mlist.fqdn_listname because this can change. For example, if you "bin/withlist -l mylist" and then "m.host_name = 'new.example.com'" the fqdn_listname property will change and the commit machinery won't be able to find the correct transaction. Instead, store the fqdn_listname as it's seen during the api_lock() call back on the mailing list under the _txnkey attribute. Use that attribute in api_save() and api_unlock(). Upgrade to SQLAlchemy 0.3.3 Port from MM2.1 the support for multiple password schemes. Change the MailList's repr to use the fqdn_listname.
* Merged revisions 8113-8121 via svnmerge from bwarsaw2006-12-291-189/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | https://mailman.svn.sourceforge.net/svnroot/mailman/branches/tmp-sqlalchemy-branch ................ r8114 | bwarsaw | 2006-12-06 00:16:54 -0500 (Wed, 06 Dec 2006) | 44 lines Initial take on using SQLAlchemy to store list data in lieu of Python pickles. While all the list data (including OldStyleMemberships attributes) are stored in the database, many attributes are stored as PickleTypes binary data. This isn't idea but it gets things working until a more sophisticated schema can be developed. MailList class is now a new-style class, as is required by SQLAlchemy. This makes several things, er, interesting. Rip out all the low-level pickle reading and writing stuff. Hook SA transaction events into Lock() and Unlock(). Move the hooking of the _memberadaptor into InitTempVars(), which gets called by the SQLAlchemy hooks (MailList.__init__() never is). Add an initialize.py module which centralizes all the initialization bits that command line scripts have to do, including configuration, logging, and atabase initialization. This change also converts bin/withlist to mmshell wrapper. Update to SQLAlchemy 0.3.1. Revamp paths.py.in considerably. There were several problems with the old way. We no longer disable default loading of site-packages so we don't need to add Python's site-packages back to sys.path. Also, because site.addsitedir() causes things like .pth paths to be /appended/ to sys.path, they actually won't override any site-installed packages. E.g. if SQLAlchemy is installed in the system Python, our version will not override. IIUC, setuptools-based packages can be configured to work properly in the face of package versions, however not all packages we currently depend on are setuptools-based. So instead, we steal a bit of stuff from site.py but change things so the prepend .pth stuff to sys.path. Update several modules to use True/False and whitespace normalization. Convert from mm_cfg to config object. Modernize a few coding constructs. Add a couple of exceptions to handle database problems. In the export script, include the widget type in the elements. This helped in my stupid little throw away conversion script, but I think it will be more generally useful. Add an interact.py module which refactors interactive interpreter access. Mostly this is used by withlist -i, but it lets us import Mailman.interact and drop into a prompt just about anywhere (e.g. debugging). ................ r8115 | bwarsaw | 2006-12-07 09:13:56 -0500 (Thu, 07 Dec 2006) | 22 lines Start to flesh out more of the SQLAlchemy mechanisms. Added a MailList.__new__() which hooks instantiation to use a query on dbcontext to get an existing mailing list. A 'no-args' call means we're doing a Create(), though eventually that will change too. For now, disable the CheckVersion() call. Eventually this will be folded into schema migration. list_exists(): Rewrite to use the dbcontext query to determine if the named mailing list exists or not. Requires the fqdn_listname. Eradicate two failed member adaptors: BDBMemberAdaptor and SAMemberships. Change the way the DBContext holds onto tables. It now keeps a dictionary mapping the table's name to the SA Table instance. This makes it easier to look up and use the individual tables. Add 'web_page_url' as an attribute managed by SA, and remove a debugging print. ................ r8116 | bwarsaw | 2006-12-11 07:27:47 -0500 (Mon, 11 Dec 2006) | 29 lines Rework the whole dbcontext and transaction framework. SA already handles nested transactions so we don't have to worry about them. However, we do have the weird situation where some transactions are tied to MailList .Lock()/.Unlock()/.Save() and some are tied to non-mlist actions. So now we use an @txn decorator to put methods in a session transaction, but then we also hook into the above MailList methods as possibly sub-transactions. We use a weakref subclass to manage the MailList interface, with a dictionary mapping MailList fqdn_listnames against transactions. The weakrefs come in by giving us a callback when a MailList gets derefed such that we're guaranteed to rollback any outstanding transaction. Also, we have one global DBContext instance but rather than force the rest of Mailman to deal with context objects, instead we expose API methods on that object into the Mailman.database module, which the rest of the code will use. Such methods must be prepended with 'api_' to get exposed this way. bin/rmlist now works with the SA-backend. I refactored the code here so that other code (namely, the test suite) can more easily and consistently remove a mailing list. This isn't the best place for it ultimately, but it's good enough for now. New convenience functions Utils.split_listname(), .fqdn_listname(). Convert testall to use Mailman.initialize.initialize(). Not all tests work, but I'm down to only 8 failures and 7 errors. Also, do a better job of recovering from failures in setUp(). MailList.__new__() now takes keyword arguments. ................ r8117 | bwarsaw | 2006-12-11 22:58:06 -0500 (Mon, 11 Dec 2006) | 7 lines Unit test repairs; even though the unit tests are still pretty fragile, everything now passes with the SQLAlchemy storage of list data. Added missing 'personalize' column. Converted mailmanctl and qrunner to initialize() interface. Fixed _cookie_path() to not fail if SCRIPT_NAME is not in the environment. ................ r8118 | bwarsaw | 2006-12-27 18:45:41 -0500 (Wed, 27 Dec 2006) | 21 lines Utils.list_names(): Use a database query to get all the list names. dbcontext.py: Added api_get_list_names() to support Utils.list_names(). listdata.py: Added two additional MailList attributes which need to be stored in the database. The first is 'admin_member_chunksize' which isn't modifiable from the web. The second is 'password' which holds the list's password. HTMLFormatObject: item strings can now be unicodes. bin/list_lists.py: Must call initialize() to get the database properly initialized, not just config.load(). This will be a common theme. SecurityManager.py: - Remove md5 and crypt support - Added mailman.debug logger, though it will be only used during debugging. - The 'secret' can be a unicode now. - A few coding style updates; repr() instead of backticks, 'key in dict' instead of 'dict.has_key(key)' ................ r8119 | bwarsaw | 2006-12-27 19:13:09 -0500 (Wed, 27 Dec 2006) | 2 lines genaliases.py: config.load() -> initialize() ................ r8120 | bwarsaw | 2006-12-27 19:17:26 -0500 (Wed, 27 Dec 2006) | 9 lines Blocked revisions 8113 via svnmerge ........ r8113 | bwarsaw | 2006-12-05 23:54:30 -0500 (Tue, 05 Dec 2006) | 3 lines Initialized merge tracking via "svnmerge" with revisions "1-8112" from https://mailman.svn.sourceforge.net/svnroot/mailman/branches/tmp-sqlalchemy-branch ........ ................ r8121 | bwarsaw | 2006-12-28 23:34:52 -0500 (Thu, 28 Dec 2006) | 20 lines Remove SIGTERM handling from all the CGI scripts. This messes with HTTPRunner because when you issue "mailmanctl stop" after the signal handler has been installed, the process will get a SIGTERM, the signal handler will run, and the process will exit with a normal zero code. This will cause mailmanctl to try to restart the HTTPRunner. I don't think we need that stuff at all when running under wsgi with a SQLAlchemy backend. If mailmanctl kills the HTTPRunner in the middle of the process, I believe (but have not tested) that the transaction should get properly rolled back at process exit. We need to make sure about this, and also we need to test the signal handling functionality under traditional CGI environment (if we even still want to support that). Also, make sure that we don't try to initialize the loggers twice in qrunner. This was the cause of all the double entries in logs/qrunner. Fix a coding style nit in mailmanctl.py. De-DOS-ify line endings in loginit.py. ................
* MailList.py ... GetScriptURL() absolute again because we need it for emailtkikuchi2006-11-091-2/+5
| | | | | | | | notifications. wsgi_app.py ... URI normalization by stripping trailing slash. We need Special care for 'private'. Strip dot only components in the PATH_INFO for sanitization.
* Convert dumpdb to mmshell and configuration object. Convert htmlformat.pybwarsaw2006-11-041-2/+3
| | | | | | | | | from mm_cfg to the configuration object. In the config object, add an add_runner() function for extending the QRUNNERS variable. De-DOS-line-ending-ify configuration.py
* More work on the WSGI support. So far, I've tested most of the admin.py linksbwarsaw2006-10-151-7/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | and some of the admindb.py links. There may still be breakage in other parts of the interface and I haven't gone back to verify that traditional CGI still works. Changes: - Add wsgiref-0.1.2-py2.4.egg so that we can still do WSGI in Python 2.4, which doesn't come with wsgiref. Of course this means we /also/ have to add setuptools-0.5c3 because eggs require setuptools. - Style cleanups in HTTPRunner.py and wsgi_app.py. Also, use cStringIO instead of StringIO. - All internal links within the listinfo and admin pages are (or at least should be ;) relative now. This should make other things better, such as running Mailman over https or alternative ports. It does kind of mean that web_page_url is obsolete, but I haven't looked at whether we can completely eradicate it. - ValidateEmail(): Use ' ' in s instead of s.count(' ') > 0. - GetPathPieces(): When path is false, return the empty list instead of None, so we can still len() it. - ScriptURL(): Much simpler. To support relative urls as the default, we change the API so that it only takes a 'target' argument (i.e. the script we want to link to). It no longer takes 'absolute' or 'web_page_url', and it constructs its link from GetPathPieces(), the target, and the cgi extension. - GetRequestURI(): code style updates. - Mailman/bin/show_config.py: De-DOS-line-ending-ification. - export.py: A few modifications, although this is likely still not final (I'm still working on the import script). First, for <option> elements, don't put the value in an attribute, put it in the text body of the element. Second, put the list <option> tags in a <configuration> element. Third, put the preferred language on an <option> tag with a 'preferred_language' name attribute value. - SecurityManager: Make sure that MakeCookie() and ZapCookie() use the same 'path' cookie value by refactoring that into a separate method. That method now returns just the SCRIPT_NAME and the full listname. web_page_url doesn't enter into it. - loginit.py: Add a 'debug' logger since it's just too useful to have :) - admin.py: Remove the extra / right before the query string in ?VARHELP urls. That extra / turns out to be problematic with the relative url scheme we're using now. - Auth.py: whitespace normalization and copyright years update. Also, remove a couple of unnecessary imports. Also, make sure that the actionurl is relative. - create.py: Typo. - private.py: mm_cfg -> config object - In MailList.py: GetScriptURL() can be written in terms of Utils.ScriptURL() now.
* - bin/show_mm_cfg.py renamed to bin/show_config.py.msapiro2006-10-121-4/+9
| | | | | | - added a -C/--config option to bin/show_config.py and made a few other changes. - MailList.py - r8040 broke Mailman/bin/update.py situate_list. Fixed it with this change.
* listname@email_host order.tkikuchi2006-10-011-1/+1
|
* Here are the patches needed in order to create new lists on my testtkikuchi2006-09-281-7/+10
| | | | | | | installation. I've tested four cases of combination of POSTFIX_STYLE_VIRTUAL_DOMAINS (Any/None) and USE_MAIL_DIR (Yes/No). Also, I've added POSTFIX_VIRTUAL_SEPARATOR = '_at_' for unique mapping of local part in the virtual-mailman/aliases files.
* Another milestone: you can now post to lists. Converted the following to usebwarsaw2006-09-251-4/+5
| | | | | | | | the new configuration object: admin, admindb, bounces, confirm, inject, join, leave, owner, post, request, unshunt, version. Also change MailList.GetScriptURL() to return the list's fully qualified name in links.
* Convert genaliases to mmshell, optparse, and configuration.configbwarsaw2006-09-251-5/+4
| | | | | | | | | | | | | | | | bin/withlist: If there's no '@' in the listname, append the DEFAULT_EMAIL_HOST so we always get a fully qualified list name. bin/mmsitepass: plumb through -C/--config switch and be sure to call config.load(). Convert Mailman/MTA/Postfix.py to configuration.config, and update MTA/Manual. In mailman/Cgi/create, we can't convert straight from a string to a bool, because bool('0') is True. We need to go through int first. MailList.InitTempVars(): The logic here looked weird because we could get 'name' = None and that would break. Assume name is never None.
* As Mark discovered, we need to load the configuration in the driver script forbwarsaw2006-09-241-1/+1
| | | | | | | | the web u/i to work. This also fixes the use of Utils.list_names() in the list and admin overviews. This API now returns a set, but the CGIs want to sort them, so we need to turn them back into lists. This change also elaborates an exception so that the list name is reported.
* Fix several problems reported by Mark. First, Python 2.3 doesn't havebwarsaw2006-08-291-2/+2
| | | | | | | | | | | | | | | | | | | built-in sets and sadly we still have to support it. Utils.list_names() therefore now uses a sets.Set if it finds the built-in set type missing. main() in list_lists.py also has to list()-ify the set in order to sort it. There's a bootstrapping problem when bin/update tries to re-situate a non-qualified list name to a qualified list name. The problem is that the MailList ctor will call Load() with the nonqual name, which in turn will call CheckVersion() by default. The latter will then turn around and try to call Load() with the fqdn list name, but that list name won't yet exist since it hasn't been situated. The solution (but maybe not the best one) then is to pass the check_version flag to the MailList ctor, which gets passed down to the Load() call. Only situate_list() will call this to avoid the version checking before the list has been situated.
* Add a framework for easier use of alternative MemberAdaptor implementations.bwarsaw2006-07-161-6/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | Also add an experimental (and currently non-functioning) SQLAlchemy implementation. The MemberAdaptor.py interface has been updated in a couple of ways. First, a "transaction interface" has been added so that Mailman can properly sync with the member adaptor. Newly supported methods are load(), lock(), save(), and unlock() and these correspond to methods in the MailList object. Second, __init__() is officially documented to take a MailList instance and nothing else. Third, some of the existing docstrings were incorrect w.r.t. the OldStyleMemberships implementation (such as rasing BadPasswordError in some cases). Most of these should not be the responsibility of the MemberAdaptor, so the docstrings have been updated. Test cases have been added and a new Defaults.py.in variable called MEMBER_ADAPTOR_CLASS has been added which names the class to use. Of course OldStyleMemberships are named by default. There's also a SQLALCHEMY_ENGINE_URL variable for use with the experimental member adaptor. Fix a bug in Configuration where if the etc/mailman.cfg file wasn't found and the mm_cfg.py file was used as a fallback, it would blow away the original namespace copied from Defaults.py.in. This wasn't a problem until we started adding additional names to that namespace, such as 'add_domain'.
* Fix a buglet that can cause bogus archives/private directories to be createdbwarsaw2006-07-081-5/+13
| | | | | | | | when a mailing list is created. The problem is that MailList.Create() will call MailList.InitVars() which in turn calls Archiver.InitVars(). The latter creates private archive directories based on the list's host_name attribute. But Create() couldn't set this to what it knows it should be because MailList.InitVars() always blew it away.
* Fix some buglets with virtual domain support and repair unit tests broken bybwarsaw2006-07-081-9/+16
| | | | | | | | | | | | | | | | | | | | | | | this change. More unit tests should be added. misc/sitelist.cfg is removed -- this is an ex-site list. MailList.GetNoReplyEmail() -> MailList.no_reply_address (property) UserNotification._enqueue(), OwnerNotification._enqueue(): when queing the message to the virgin queue, be sure to use the fully qualified (i.e. posting) address for the list. In the MTA modules, be sure to set up the target of the mail commands as the fqdn listname because otherwise we can't find the correct list. This needs some tweaking/testing for Postfix's virtual domain support. MailList.Load() has to grow an optional argument specifying the fqdn listname. The problem is that in some situations, we can't calculate that because we don't know _internal_name, so it has to be passed in. This is mostly the case in the MailList ctor where a Load hasn't happened yet. For backward compatibility though, if it's not passed in, just use mlist.fqdn_listname.
* First crack at real virtual domain support, i.e. mailing lists with the samebwarsaw2006-07-081-24/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | name in more than one domain. - Totally eradicate MAILMAN_SITE_LIST, and in fact the entire need for a site list. The functions that the site list previously performed are either removed or supported in other ways. For example, instead of forwarding owner bounces to the site list, we now have a SITE_OWNER_ADDRESS which should point to a human, and such bounces are sent there instead. There's also a "no reply" email address that should be set up to go to devnull. For any message that never expects a reply, the sender is set to this address. - Remove the Site.py module. It was an experimental approach to trying to support virtual domains, and we're going to do it so much better now that this module is no longer necessary. Site._makedirs() -> Utils.makedir(). - VIRTUAL_HOST_OVERVIEW is completely removed, since now virtual hosts are always enabled. Virtual domains should be added to mailman.cfg by using the new add_domain() function. add_virtualhost() is gone. If no virtual domains are added explicitly, we add the default one that configure guessed (but we never add that if domains are added explicitly). - Utils.get_domain() -> Utils.get_request_domain() - withlist code cleanup and make sure that we load etc/mailman.cfg - A new base exception called MailmanException is added, from which all exceptions defined in Errors.py ultimately derive. MailmanError is retained and derives from MailmanException. - BadDomainSpecificationError is added. - Remove the -V/--virtual-host-overview option from list_lists and add instead -d/--domain and -f/--full. - bin/update probably works but needs more testing. - bin/newlist and bin/rmlist take fqdn list names, but default to the default domain if @whatever isn't given. newlist's -u/--urlhost and -e/--emailhost options are removed. The domain that the list is being added to must already exist. - Minor code cleanup in Message.py - Bump version to 2.2.0a1 - The Configuration object grows a .domain dictionary which maps email hosts to url hosts. The reverse mapping is supported, but not directly; use Configuration.get_email_host() instead. - Mailman/Cgi/create is converted from mm_cfg to config, and some minor code cleanup is performed. Also, convert to __i18n_templates__ = True. - New MailList APIs: + property .fqdn_listname + GetNoReplyEmail() + Create() API changes and refactoring.
* Massive conversion process so that Mailman can be run from a user specifiedbwarsaw2006-07-081-75/+75
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | configuration file. While the full conversion is not yet complete, everything that seems to be required to run mailmanctl, qrunner, rmlist, and newlist have been updated. Basically, modules should no longer import mm_cfg, but instead they should import Mailman.configuration.config. The latter is an object that's guaranteed to exist, but not guaranteed to be initialized until some top-level script calls config.load(). The latter should be called with the argument to -C/--config which is a new convention the above scripts have been given. In most cases, where mm_cfg.<variable> is used config.<variable> can be used, but the exceptions are where the default value must be available before config.load() is called. Sometimes you can import Mailman.Default and get the variable from there, but other times the code has to be changed to work around this limitation. Take each on a case-by-case basis. Note that the various directories calculated from VAR_PREFIX, EXEC_PREFIX, and PREFIX are now calculated in config.py, not in Defaults.py. This way a configuration file can override the base directories and everything should work correctly. Other changes here include: - mailmanctl, qrunner, and update are switched to optparse and $-strings, and changed to the mmshell architecture - An etc directory has been added to /usr/local/mailman and a mailman.cfg.sample file is installed there. Sites should now edit an etc/mailman.cfg file to do their configurations, although the mm_cfg file is still honored. The formats of the two files are identical. - list_lists is given the -C/--config option - Some coding style fixes in bin/update, but not extensive - Get rid of nested scope hacks in qrunner.py - A start on getting EmailBase tests working (specifically test_message), although not yet complete.
* - Utils.py Fixed a security hole which allowed a crafted URI to injectmsapiro2006-07-071-2/+4
| | | | | | | | | | | | | | | | | | | | bogus apparent messages into the error log, possibly inducing an admin to visit a phishing site. - options.py Topics.py Tagger.py MailList.py Utils.py Version.py versions.py The processing of Topics regular expressions has changed. Previously the Topics regexp was compiled in verbose mode but not documented as such which caused some confusion. Also, the documentation indicated that topic keywords could be entered one per line, but these entries were not properly. Topics regexps are now compiled in non-verbose mode and multi- line entries are 'ored'. Existing Topics regexps will be converted when the list is updated so they will continue to work.
* More logging updates:bwarsaw2006-04-281-3/+1
| | | | | | | | | | | - Remove the LogStdErr() calls from all the scripts/* scripts. - Convert the LockFile.py code to use the logging logs. Also do general code cleanup in this module. Note that the 'withlogging' argument to __init__() has been removed, as we can use logging's mechanisms to control how much of LockFile will get logged (by default, not much). - Add the 'mailman.locks' logger to loginit.py - Remove Mailman/Handlers/Sendmail.py once and for all - Remove LIST_LOCK_DEBUGGING from Defaults.py.in
* MailList.__load(): Fix the sense of a test broken by the previous commit.bwarsaw2006-04-171-1/+1
| | | | Mailman/Makefile.in: Remove Logging subdirectory.
* Remove most uses of the types module, in favor of isinstance checks againstbwarsaw2006-04-171-7/+7
| | | | | | the builtin types. Two still remain: a check against ClassType and a check against MethodType. Also, fix some hinky type comparisons to use isinstance() consistently.
* - Convert all logging to Python's standard logging module. Get rid of allbwarsaw2006-04-171-38/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | traces of our crufty old Syslog. Most of this work was purely mechanical, except for: 1) Initializing the loggers. For this, there's a new module Mailman/loginit.py (yes all modules from now on will use PEP 8 names). We can't call this 'logging.py' because that will interfere with importing the stdlib module of the same name (can you say Python 2.5 and absolute imports?). If you want to write log messages both to the log file and to stderr, pass True to loginit.initialize(). This will turn on propagation of log messages to the parent 'mailman' logger, which is set up to print to stderr. This is how bin/qrunner works when not running as a subprocess of mailmanctl. 2) The driver script. I had to untwist the StampedLogger stuff and implement differently printing exceptions and such to log/error because standard logging objects don't have a write() method. So we write to a cStringIO and then pass that to the logger. 3) SMTPDirect.py because of the configurability of the log messages. This required changing SafeDict into a dict subclass (which is better than using UserDicts anyway -- yay Python 2.3!). It's probably still possible to flummox things up if you change the name of the loggers in the SMTP_LOG_* variables in mm_cfg.py. However, the worst you can do is cause output to go to stderr and not go to a log file. Note too that all entry points into the Mailman system must call Mailman.loginit.initialize() or the log output will go to stderr (which may occasionally be what you want). Currently all CGIs and qrunners should be working properly. I wish I could have tested all code paths that touch the logger, but that's infeasible. I have tested this, but it's possible that there were some mistakes in the translation. - Mailman.Bouncers.BounceAPI.Stop is a singleton, but not a class instance any more. - True/False code cleanup, PEP 8 import restructuring, whitespace normalization, and copyright year updates, as appropriate.
* Now that Python 2.3 is the minimum requirement for Mailman 2.2:bwarsaw2006-04-151-21/+14
| | | | | | | | | | | - Remove True/False binding cruft - Remove __future__ statements for nested scopes - Remove ascii_letters import hack from Utils.py - Remove mimetypes.guess_all_extensions import hack from Scrubber.py - In Pending.py, set _missing to object() (better than using []) Also, update copyright years where appropriate, and re-order imports more to my PEP 8 tastes. Whitespace normalize.
* Added a new list attribute 'subscribe_auto_approval' which is a list of ↵msapiro2006-03-181-9/+28
| | | | | | | email addresses and regular expressions matching email addresses whose subscriptions are exempt from admin approval. RFE 403066.
* Fixed long standing error - subscribe log 'new' entry reversed name and email.msapiro2006-03-121-1/+1
|
* Corrected typo in previous change.msapiro2006-03-031-1/+1
|
* Confirmed member change of address is logged in the 'subscribe' log,msapiro2006-03-031-0/+30
| | | | | and if admin_notify_mchanges is true, a notice is sent to the list owner using a new adminaddrchgack.txt template.
generated by cgit v1.2.3-70-g09d2 (git 2.51.2) at 2025-11-09 13:05:36 +0000