| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
| |
artifacts are removed, as is the C files which we will no longer need.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
classes live in Mailman/databae/model now.
Remove the TestDecorate test class from test_handlers.py and move them into a
doctest called decorate.txt (with harness in test_decorate.py).
Remove the dependence on SafeDict from the Decorate handler because I can now
use string.Template object to safely fill in header and footer templates.
Eventually I want to completely remove SafeDict from Mailman, but it's still
used in a few other places.
This also means that only $-strings will be supported in headers and footers,
and the import script will have to convert %-strings to $-strings. Also,
'_internal_name' is no longer a supported header/footer substitution
variable. Use $real_name or $list_name now. Added $fqdn_listname as a
substitution variable. Update the DEFAULT_MSG_FOOTER accordingly.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
confident the Elixir branch is ready to become mainline. Also, fewer branches
makes for an easier migration to a dvcs.
Don't expect much of the old test suite to work, or even for much of the old
functionality to work. The changes here are disruptive enough to break higher
level parts of Mailman. But that's okay because I am slowly building up a new
and improved test suite, which will lead to a functional system again.
For now, only the doctests in Mailman/docs (and their related test harnesses)
will pass, but they all do pass. Note that Mailman/docs serve as system
documentation first and unit tests second. You should be able to read the
doctest files to understand the underlying data model.
Other changes included in this merge:
- Added the Mailman.ext extension package.
- zope.interfaces uses to describe major components
- SQLAlchemy/Elixir used as the database model
- Top level doinstall target renamed to justinstall
- 3rd-party packages are now installed in pythonlib/lib/python to be more
compliant with distutils standards. This allows us to use just --home
instead of all the --install-* options.
- No longer need to include the email package or pysqlite, as Python 2.5 is
required (and comes with both packages).
- munepy package is included, for Python enums
- IRosterSets are added as a way to manage a collection of IRosters. Roster
sets are named so that we can maintain the indirection between mailing lists
and rosters, where the two are maintained in different storages.
- IMailingListRosters: remove_*_roster() -> delete_*_roster()
- Remove IMember interface.
- Utils.list_names() -> config.list_manager.names
- fqdn_listname() takes an optional hostname argument.
- Added a bunch of new exceptions used throughout the new interfaces.
- Make LockFile a context manager for use with the 'with' statement.
|
| |
|
|
| |
Python 2.5 is required.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
https://mailman.svn.sourceforge.net/svnroot/mailman/branches/tmp-sqlalchemy-branch
................
r8114 | bwarsaw | 2006-12-06 00:16:54 -0500 (Wed, 06 Dec 2006) | 44 lines
Initial take on using SQLAlchemy to store list data in lieu of Python pickles.
While all the list data (including OldStyleMemberships attributes) are stored
in the database, many attributes are stored as PickleTypes binary data. This
isn't idea but it gets things working until a more sophisticated schema can be
developed.
MailList class is now a new-style class, as is required by SQLAlchemy. This
makes several things, er, interesting. Rip out all the low-level pickle
reading and writing stuff. Hook SA transaction events into Lock() and
Unlock(). Move the hooking of the _memberadaptor into InitTempVars(), which
gets called by the SQLAlchemy hooks (MailList.__init__() never is).
Add an initialize.py module which centralizes all the initialization bits that
command line scripts have to do, including configuration, logging, and atabase
initialization.
This change also converts bin/withlist to mmshell wrapper.
Update to SQLAlchemy 0.3.1.
Revamp paths.py.in considerably. There were several problems with the old
way. We no longer disable default loading of site-packages so we don't need
to add Python's site-packages back to sys.path. Also, because
site.addsitedir() causes things like .pth paths to be /appended/ to sys.path,
they actually won't override any site-installed packages. E.g. if SQLAlchemy
is installed in the system Python, our version will not override. IIUC,
setuptools-based packages can be configured to work properly in the face of
package versions, however not all packages we currently depend on are
setuptools-based. So instead, we steal a bit of stuff from site.py but change
things so the prepend .pth stuff to sys.path.
Update several modules to use True/False and whitespace normalization.
Convert from mm_cfg to config object. Modernize a few coding constructs.
Add a couple of exceptions to handle database problems.
In the export script, include the widget type in the elements. This helped in
my stupid little throw away conversion script, but I think it will be more
generally useful.
Add an interact.py module which refactors interactive interpreter access.
Mostly this is used by withlist -i, but it lets us import Mailman.interact and
drop into a prompt just about anywhere (e.g. debugging).
................
r8115 | bwarsaw | 2006-12-07 09:13:56 -0500 (Thu, 07 Dec 2006) | 22 lines
Start to flesh out more of the SQLAlchemy mechanisms.
Added a MailList.__new__() which hooks instantiation to use a query on
dbcontext to get an existing mailing list. A 'no-args' call means we're doing
a Create(), though eventually that will change too.
For now, disable the CheckVersion() call. Eventually this will be folded into
schema migration.
list_exists(): Rewrite to use the dbcontext query to determine if the named
mailing list exists or not. Requires the fqdn_listname.
Eradicate two failed member adaptors: BDBMemberAdaptor and SAMemberships.
Change the way the DBContext holds onto tables. It now keeps a dictionary
mapping the table's name to the SA Table instance. This makes it easier to
look up and use the individual tables.
Add 'web_page_url' as an attribute managed by SA, and remove a debugging
print.
................
r8116 | bwarsaw | 2006-12-11 07:27:47 -0500 (Mon, 11 Dec 2006) | 29 lines
Rework the whole dbcontext and transaction framework. SA already handles
nested transactions so we don't have to worry about them. However, we do have
the weird situation where some transactions are tied to MailList
.Lock()/.Unlock()/.Save() and some are tied to non-mlist actions. So now we
use an @txn decorator to put methods in a session transaction, but then we
also hook into the above MailList methods as possibly sub-transactions. We
use a weakref subclass to manage the MailList interface, with a dictionary
mapping MailList fqdn_listnames against transactions. The weakrefs come in by
giving us a callback when a MailList gets derefed such that we're guaranteed
to rollback any outstanding transaction.
Also, we have one global DBContext instance but rather than force the rest of
Mailman to deal with context objects, instead we expose API methods on that
object into the Mailman.database module, which the rest of the code will use.
Such methods must be prepended with 'api_' to get exposed this way.
bin/rmlist now works with the SA-backend. I refactored the code here so that
other code (namely, the test suite) can more easily and consistently remove a
mailing list. This isn't the best place for it ultimately, but it's good
enough for now.
New convenience functions Utils.split_listname(), .fqdn_listname().
Convert testall to use Mailman.initialize.initialize(). Not all tests work,
but I'm down to only 8 failures and 7 errors. Also, do a better job of
recovering from failures in setUp().
MailList.__new__() now takes keyword arguments.
................
r8117 | bwarsaw | 2006-12-11 22:58:06 -0500 (Mon, 11 Dec 2006) | 7 lines
Unit test repairs; even though the unit tests are still pretty fragile,
everything now passes with the SQLAlchemy storage of list data.
Added missing 'personalize' column. Converted mailmanctl and qrunner to
initialize() interface. Fixed _cookie_path() to not fail if SCRIPT_NAME is
not in the environment.
................
r8118 | bwarsaw | 2006-12-27 18:45:41 -0500 (Wed, 27 Dec 2006) | 21 lines
Utils.list_names(): Use a database query to get all the list names.
dbcontext.py: Added api_get_list_names() to support Utils.list_names().
listdata.py: Added two additional MailList attributes which need to be stored
in the database. The first is 'admin_member_chunksize' which isn't modifiable
from the web. The second is 'password' which holds the list's password.
HTMLFormatObject: item strings can now be unicodes.
bin/list_lists.py: Must call initialize() to get the database properly
initialized, not just config.load(). This will be a common theme.
SecurityManager.py:
- Remove md5 and crypt support
- Added mailman.debug logger, though it will be only used during
debugging.
- The 'secret' can be a unicode now.
- A few coding style updates; repr() instead of backticks, 'key in dict'
instead of 'dict.has_key(key)'
................
r8119 | bwarsaw | 2006-12-27 19:13:09 -0500 (Wed, 27 Dec 2006) | 2 lines
genaliases.py: config.load() -> initialize()
................
r8120 | bwarsaw | 2006-12-27 19:17:26 -0500 (Wed, 27 Dec 2006) | 9 lines
Blocked revisions 8113 via svnmerge
........
r8113 | bwarsaw | 2006-12-05 23:54:30 -0500 (Tue, 05 Dec 2006) | 3 lines
Initialized merge tracking via "svnmerge" with revisions "1-8112" from
https://mailman.svn.sourceforge.net/svnroot/mailman/branches/tmp-sqlalchemy-branch
........
................
r8121 | bwarsaw | 2006-12-28 23:34:52 -0500 (Thu, 28 Dec 2006) | 20 lines
Remove SIGTERM handling from all the CGI scripts. This messes with HTTPRunner
because when you issue "mailmanctl stop" after the signal handler has been
installed, the process will get a SIGTERM, the signal handler will run, and
the process will exit with a normal zero code. This will cause mailmanctl to
try to restart the HTTPRunner.
I don't think we need that stuff at all when running under wsgi with a
SQLAlchemy backend. If mailmanctl kills the HTTPRunner in the middle of the
process, I believe (but have not tested) that the transaction should get
properly rolled back at process exit. We need to make sure about this, and
also we need to test the signal handling functionality under traditional CGI
environment (if we even still want to support that).
Also, make sure that we don't try to initialize the loggers twice in qrunner.
This was the cause of all the double entries in logs/qrunner.
Fix a coding style nit in mailmanctl.py.
De-DOS-ify line endings in loginit.py.
................
|
| |
|
|
|
|
|
|
|
| |
from mm_cfg to the configuration object.
In the config object, add an add_runner() function for extending the QRUNNERS
variable.
De-DOS-line-ending-ify configuration.py
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
membership. The next step is to write an XML importer that reads this file.
I'm not 100% sure that all the import data is included yet, but the intent is
that this will be the official way to move mailing lists.
A few notes: member passwords are not included by default, the idea being that
if we enable XML dumping from the web, we don't want the clear text user
passwords to be leaked. A command line option includes the member passwords.
Also, the various substitutable texts (i.e. those that include %-strings) will
be autoconverted to $-strings. In Mailman 2.2, we'll only have $-strings,
although this is not yet enforced in other parts of the code yet.
Convert config_list.py to mmshell, $-strings, and optparse.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
bin/withlist: If there's no '@' in the listname, append the DEFAULT_EMAIL_HOST
so we always get a fully qualified list name.
bin/mmsitepass: plumb through -C/--config switch and be sure to call
config.load().
Convert Mailman/MTA/Postfix.py to configuration.config, and update MTA/Manual.
In mailman/Cgi/create, we can't convert straight from a string to a bool,
because bool('0') is True. We need to go through int first.
MailList.InitTempVars(): The logic here looked weird because we could get
'name' = None and that would break. Assume name is never None.
|
| |
|
|
| |
usage.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
uncleanly (e.g. segfaults the Python interpreter), messages being processed
will not be lost.
The vulnerability, ideas, and patches are credited to Richard Barrett and Mark
Sapiro. Their original work was modified by Barry for this commit and any
bugs are his fault.
The basic idea is that instead of unlinking a .pck file in dequeue(), the file
is renamed to a .bak file. The Switchboard grows a finish() method which then
unlinks the .bak file. That class's constructor also grows a 'restore'
argument (defaulting to false), which when true moves all .bak files it finds
in its hash space to .pck, thereby restoring a file lost while "in flight".
This relies on the fact that even with multiple qrunners, exactly one process
will be responsible for one hash space slice, so it's never possible (under
normal operation) for a .bak file to be renamed to .pck by some other process.
Test cases for both the new Switchboard behavior and the use of that by Runner
subclasses has been added.
There are two things to watch out for, either of which may require some
additional changes. There is some small potential to duplicate messages in
various queues, if say 'mailmanctl' were improperly started more than once by
a site admin. This usually won't happen unless an admin is overly eager with
the mailmanctl -s switch, so we can chalk this one up to operator error. I'm
not sure what more we can do about that.
There's also a possibility that if we're processing a message that continually
causes the Python interpreter to crash, we could end up duplicating messages
endlessly. This is especially troublesome for the Outgoing runner which could
conceivably cause a mail flood. I consider this the more critical issue to
defend against, probably by adding a numbering scheme to the .bak file names
and refusing to restore a .bak file more than say 3 times without human
intervention.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
configuration file. While the full conversion is not yet complete, everything
that seems to be required to run mailmanctl, qrunner, rmlist, and newlist have
been updated.
Basically, modules should no longer import mm_cfg, but instead they should
import Mailman.configuration.config. The latter is an object that's
guaranteed to exist, but not guaranteed to be initialized until some top-level
script calls config.load(). The latter should be called with the argument to
-C/--config which is a new convention the above scripts have been given.
In most cases, where mm_cfg.<variable> is used config.<variable> can be used,
but the exceptions are where the default value must be available before
config.load() is called. Sometimes you can import Mailman.Default and get the
variable from there, but other times the code has to be changed to work around
this limitation. Take each on a case-by-case basis.
Note that the various directories calculated from VAR_PREFIX, EXEC_PREFIX, and
PREFIX are now calculated in config.py, not in Defaults.py. This way a
configuration file can override the base directories and everything should
work correctly.
Other changes here include:
- mailmanctl, qrunner, and update are switched to optparse and $-strings, and
changed to the mmshell architecture
- An etc directory has been added to /usr/local/mailman and a
mailman.cfg.sample file is installed there. Sites should now edit an
etc/mailman.cfg file to do their configurations, although the mm_cfg file is
still honored. The formats of the two files are identical.
- list_lists is given the -C/--config option
- Some coding style fixes in bin/update, but not extensive
- Get rid of nested scope hacks in qrunner.py
- A start on getting EmailBase tests working (specifically test_message),
although not yet complete.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Mailman
top level package. Rewrote the test runner and stuck it in bin (as an
mmshell
symlink). bin/testall now autodetects tests and allows running a
subset of
tests via regular expression filtering.
I also fixed all the tests so they all pass now, with the exception of
test_message.py tests. These still doesn't work because of coordination
issues between its smtpd-based reader and SMTPDirect, but... I have a
plan
(mwah, ha, ha! :)
I also fixed a code update bug in Decorate.py
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
require Python to be at least 2.3.0
Convert all scripts/* scripts (except driver) to use the
symlink-to-bin/mmshell trick.
For now (since we're debugging more often than releasing), set STEALTH_MODE to
False in scripts/driver. We need to remember to turn this back to True when
we start getting closer to releases!
In the various scripts, er, Mailman/bin scripts, convert them to $-strings for
i18n substitution, and clean up imports. Also, get rid of the crufty Emacs
-*- line, which is no longer needed now that the files end in .py.
Get rid of lots of unnecessary svn:executable properties.
Remove an unnecessary import from Mailman/bin/disabled.py
|
| |
|
|
|
| |
conversion to optparse style option parsing. Remove mailpasswds as password
reminders will go away for MM2.2.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
after this commit, I have to fix the filename of
Mailman/bin/show_qfiles because of silly svn restrictions.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
use $-strings and optparse.
A side effect of the rewrite is that list_members -d and -n now require their
argument, whereas before they were optional (but that is counter to command
line parsing standards and not supported by optparse). Now, instead of bare
-n use '-n any' and instead of bare -d use '-d any'.
Fix ReopenableFileHandler.emit() to ensure that all lines end in a newline.
In i18n.py, create and use a subclass of string.Template so that dotted
attribute paths are legal in $-strings. So for example, if you have an object
'foo' in one of the namespaces and that has an attribute 'bar', you can say
"$foo.bar" to print the value of foo.bar. Should reduce the number of
extraneous local variables just to do the attribute path traversal.
|
| |
|
|
|
|
|
|
|
|
| |
use optparse. Update newlist and rmlist to get their version string entirely
from mm_cfg.MAILMAN_VERSION instead of having to build that string up each
time.
Fix the symlink source in the Makefile.in.
Remove b4b5-archfix entirely.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
an uber-shell (called mmshell for now for lack of a better name). The reason
I'm doing this is because while I understand and still accept the reason for
it, I really hate having to re-run configure (or config.status) every time I
make a change to a bin script. mmshell looks at argv[0] to figure out which
Mailman.bin module to run.
Move newlist and rmlist to the new framework, but also, rewrite newlist to use
optparse instead of getopt. Much nicer. Also convert it to use $variables
for i18n even though the i18n._() function hasn't yet been updated to handle
these (it will soon). rmlist hasn't yet been optparse-ified, but that's soon
too. :)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
traces of our crufty old Syslog. Most of this work was purely mechanical,
except for:
1) Initializing the loggers. For this, there's a new module
Mailman/loginit.py (yes all modules from now on will use PEP 8
names). We can't call this 'logging.py' because that will
interfere with importing the stdlib module of the same name (can
you say Python 2.5 and absolute imports?).
If you want to write log messages both to the log file and to
stderr, pass True to loginit.initialize(). This will turn on
propagation of log messages to the parent 'mailman' logger, which
is set up to print to stderr. This is how bin/qrunner works when
not running as a subprocess of mailmanctl.
2) The driver script. I had to untwist the StampedLogger stuff and
implement differently printing exceptions and such to log/error
because standard logging objects don't have a write() method. So
we write to a cStringIO and then pass that to the logger.
3) SMTPDirect.py because of the configurability of the log messages.
This required changing SafeDict into a dict subclass (which is
better than using UserDicts anyway -- yay Python 2.3!). It's
probably still possible to flummox things up if you change the
name of the loggers in the SMTP_LOG_* variables in mm_cfg.py.
However, the worst you can do is cause output to go to stderr and
not go to a log file.
Note too that all entry points into the Mailman system must call
Mailman.loginit.initialize() or the log output will go to stderr
(which may occasionally be what you want). Currently all CGIs and
qrunners should be working properly.
I wish I could have tested all code paths that touch the logger, but
that's infeasible. I have tested this, but it's possible that there
were some mistakes in the translation.
- Mailman.Bouncers.BounceAPI.Stop is a singleton, but not a class
instance any more.
- True/False code cleanup, PEP 8 import restructuring, whitespace
normalization, and copyright year updates, as appropriate.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
including them in cvs.
|
| |
|
|
|
| |
the system, one of the build tests would fail with an exception, but
configure would carry on. This patch avoids the traceback.
|
| | |
|
| |
|
|
| |
bomb early for RH installations without the python2-devel package.
|
| | |
|
| |
|
|
|
| |
configure. This is helpful for when I have a shell and *compilation*
buffer open and can't remember if I've done a "make install" yet. ;)
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
named paths" is caused (somehow) by the distutils package not being
available. Some Linux distros package distutils in the python-devel
package instead of the main Python package (erroneously IMO --
distutils is critical).
These configure changes exit early if distutils isn't importable.
More detail is given in the README.LINUX file. Hopefully this will
eliminate this FAQ.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
of numeric ids. Initial idea and patch by Todd Vierling, fleshed out
by Barry.
Specific changes here:
Move the --with-permcheck checking to earlier in the file, because
this switch is now also going to control whether user/group name
verification will be skipped.
MM_FIND_GROUP_ID -> MM_FIND_GROUP_NAME. Also remove the AC_MSG_RESULT
from the macro, and make the code body return the group name instead
of the id.
MM_FIND_USER_ID -> MM_FIND_USER_NAME, and similar changes.
MAILMAN_UID -> MAILMAN_USER
MAILMAN_GID -> MAILMAN_GROUP
Also, rewrote a bunch of the error messages for clarity.
Removed the long commented out ALIAS_UID/ALIAS_GID crud.
Note that --with-mail-gid and --with-cgi-gid are retained, even though
they now control the group name being used.
|
| |
|
|
| |
edit transcheck for minor style nits.
|
| | |
|
| |
|
|
|
| |
permissions on the target (i.e. --prefix) directory. --with-permcheck
is the default.
|
| |
|
|
|
|
|
|
| |
components, and add --with switches for easier overrides.
Now, both default host names are chosen with socket.getfqdn(), and
--with-mailhost overrides the email hostname part, while
--with-urlhost overrides the url hostname part.
|
| |
|
|
| |
dirs.
|
| |
|
|
|
| |
intended to be run under bin/withlist, so when run standalone, they
print their module documentation (usage information).
|
| |
|
|
| |
Try again.
|
| |
|
|
| |
that broke it! Use src/common.h now.
|
| |
|
|
| |
Closes bug #527712 and patch #527715
|
| |
|
|
| |
qfiles/shunt back to their original queue.
|
| |
|
|
| |
has agreed to let us distribute them.
|
| |
|
|
|
| |
Also, add Ben Gertzfield's patch to produce warnings if the Japanese
and Chinese codecs aren't found.
|
