| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
this considerably, since it now just checks the permissions, handles command
parsing, and then just (in the child) execs master.py. master.py handles the
log files, etc.
Added 'BIN_DIR' to the configuration object. Remove references to
instance.cfg in the comments; that was never implemented.
I still need tests for both mailmanctl and master.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
functionality of mailmanctl into a separate master watcher script. mailmanctl
has not yet been updated but that'll happen next.
Fix DELIVERY_MODULE to name a handler instead of a module.
Change make_instance to use pkg_resources instead of module.__file__.
Change the qrunner and master processes coordination so that the qrunners are
not restarted on SIGINT, because otherwise C-c just doesn't work. Now SIGUSR1
is how we'll implement 'mailman restart'.
Add a database commit so that initializing the schema doesn't lock the sqlite
database. Also, don't try to initialize the schema if the tables already
exist. Use some sqlite magic to do this test.
Move mailman.cfg.in into a new package Mailman/extras inside the tree. Also,
MAILMAN_UID and MAILMAN_GID should be integers not strings.
Convert the command runner to use an IHandler instance instead of handler
module. Similarly for the outgoing runner, DELIVERY_MODULE now names an
IHandler instance instead of a handler module.
|
| |
|
|
| |
No more %-strings. Kill off all __i18n_templates__ hacks.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
0.4. Lots of things changes, which broke lots of our code. There are still a
couple of failures in the test suite that I don't understand. It seems that
for pending.txt and requests.txt, sometimes strings come back from the
database as 8-bit strings and other times as unicodes. It's impossible to
make these tests work both separately and together.
users.txt is also failing intermittently. Lots of different behavior between
running the full test suite all together and running individual tests. Sigh.
Note also that actually, Elixir 0.4.0 doesn't work for us. There's a bug in
that version that prevented zope.interfaces and Elixir working together. Get
the latest 0.4.0 from source to fix this.
Other changes include:
- Remove Mailman/lockfile.py. While I haven't totally eliminated locking, I
have released the lockfile as a separate Python package called locknix,
which Mailman 3.0 now depends on.
- Renamed Mailman/interfaces/messagestore.py and added an IMessage interface.
- bin/testall raises turns on SQLALCHEMY_ECHO when the verbosity is above 3
(that's three -v's because the default verbosity is 1).
- add_domain() in config files now allows url_host to be optional. If not
given, it defaults to email_host.
- Added a non-public interface IDatabase._reset() used by the test suite to
zap the database between doctests. Added an implementation in the model
which just runs through all rows in all entities, deleting them.
- [I]Pending renamed to [I]Pended
- Don't allow Pendings.add() to infloop.
- In the model's User impelementations, we don't need to append or remove the
address when linking and unlinking. By setting the address.user attribute,
SQLAlchemy appears to do the right thing, though I'm not 100% sure of that
(see the above mentioned failures).
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
is moved to Mailman.lockfile.
Remove a few more MailList methods that aren't used any more, e.g. the lock
related stuff, the Save() and CheckValues() methods, as well as
ChangeMemberName().
Add a missing import to lifecycle.py.
We no longer need withlist to unlock the mailing list. Also, expose
config.db.flush() in the namespace of withlist directly, under 'flush'.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
The convenience methods in Defaults for getting seconds, minutes, hours, and
days now returns a subtype of timedelta, which provides conversion to float
and int for compatibility with interfaces that require those values
(e.g. signal.alarm() and time.sleep().
In bin/make_instance, the var_dir really needs to be an absolute path,
otherwise it's possible to get a var dir nested inside the var dir.
More MailList object eradication.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Mailman/testing -> Mailman/test
* Removed Mailman/testing/base.py
* Fix mailmanctl by using a different way of calculating where the qrunner
script is. The configuration file no longer knows what BIN_DIR is, but the
mailmanctl script knows where it lives via sys.argv[0]. Also, PREFIX_DIR ->
VAR_DIR.
Also,
* Since the overwhelmingly predominant use of ILanguageManager is to get the
description, and since .get_language_data(code)[0] is not very readable,
split the interface into .get_description() and .get_charset().
* In the setup, automatically add all Mailman.bin modules as command line
scripts.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
column in the database for this list of strings. We use SQLAlchemy's
many-to-many relationship, however because of this, you cannot simply append
new unicodes to .available_languages. You need to wrap the language code in a
Language instance and append that instance to the list.
In order to handle this, I added a property MailList.language_codes which
returns a list of the code strings (not Language instances). Also new are
MailList.set_languages() for setting (i.e. overriding) the set of available
languages for the list; and add_language() which takes a single language code,
wraps it, and appends it. The code does not and should not use
.available_languages directory any more.
MailList.GetAvailableLanguages() is removed. The 'available_languages' column
is removed from the Listdata table.
Add a getValue() to Mailman.Gui.Language in order to unwrap the language codes
stored in the database's association table. Modify _setValue() to do the
wrapping.
In dbcontext.py, don't import * from the sqlalchemy package. It contains a
'logging' name which is not the standard Python logging package. I also added
essentially a bag of attributes class called Tables which will hold references
to all the SA tables that are created. Update the make_table() API to take an
instance of Tables.
Added a close() method to DBContext. This is needed for the updated unit test
suite.
Changed bin/import.py so that when available_languages is being set, it calls
MailList.set_languages() instead of trying to set that attribute directly.
Updated some language idioms while I was at it.
More eradication of mm_cfg in favor of the config object and the Defaults
module.
In testall.py, call initialize() instead of loginit.initialize().
Promote MAX_RESTARTS into a Defaults.py.in variable. This is because the unit
tests will knock that value down to something not so annoying should one of
the qrunner-required tests traceback.
Several other important changes to the unit test suite (which now completely
succeeds again!):
- Set the uid and gid of the temporary mailman.cfg and tmp*.db files to the
Mailman user and group as specified in the config object.
- Make sure that all of the tests point to a SQLite database file that was
created with the tempfile module. This way we don't pollute our main
database with data that is getting created during the unit tests.
- In the TestBase.setUp() method, be sure to close the existing dbcontext,
clear out the mappers, and then reconnect the dbcontext with the new
SQLALCHEMY_ENGINE_URL pointing to the tempfile. However, we don't need to
reload the MailList instance any more.
- Make all tests work, except for the tests that require crypt. That upgrade
path will not be available in this version of Mailman.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
don't re-raise the exception since that will show up in the log files.
Instead just exit with a code equal to SIGTERM.
Rework the way qrunners are specified in the mailman.cfg file. Always start
the default number of the default set of qrunners, but allow mailman.cfg to
delete some with the del_qrunner() function. Rename add_runner() to
add_qrunner() and make this actually work <wink>. Both take the shortened
qrunner name as the first argument (e.g. 'Arch' instead of 'ArchRunner').
Automatically start the MaildirRunner if USE_MAILDIR = Yes; same goes for
LMTPRunner and USE_LMTP = Yes. In both cases, you do not need to also use
add_qrunner() in your mailman.cfg file to enable them. You still do need to
put "add_qrunner('HTTP')" in your mailman.cfg if you want to enable the wsgi
server. This may end up being added to the default set.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
during early initialization so that we're guaranteed to get the right value
regardless of the shell umask used to invoke the command line script. While
we're at it, we can remove almost all individual umask settings previously in
the code, and make file permissions consistently -rw-rw---- (IOW, files are no
longer other readable).
The only subsystem that wasn't changed was the archiver, because it uses its
own umask settings to ensure that private archives have the proper
permissions. Eventually we'll mess with this, but if it ain't broken...
Note that check_perms complains about directory permissions, but I think
check_perms can be fixed (or perhaps, even removed?!). If we decide to use
LMTPRunner and HTTPRunner exclusively then no outside process will be touching
our files potentially with the incorrect permissions, umask, owner, or group.
If we control all of our own touch points then I think we can lock out
'other'.
Another open question is whether Utils.set_global_password() can have its
umask setting removed. It locks permissions down so even the group can't
write to the site password file, but the default umask of 007 might be good
enough even for this file.
Utils.makedirs() now takes an optional mode argument, which defaults to 02775
for backward compatibility. First, the default mode can probably be changed
to 02770 (see above). Second, all code that was tweaking the umask in order
to do a platform compatible os.mkdir() has now been refactored to use
Utils.makedirs().
Another tricky thing was getting SQLite via SQLAlchemy to create its
data/mailman.db file with the proper permissions. From the comment in
dbcontext.py:
# XXX By design of SQLite, database file creation does not honor
# umask. See their ticket #1193:
# http://www.sqlite.org/cvstrac/tktview?tn=1193,31
More details in that file, but the work around is to essentially 'touch' the
database file if 'sqlite' is the scheme of the SQLAlchemy URL. This little
pre-touch sets the right umask honoring permission and won't hurt if the file
already exists. SQLite will happily keep the existing permissions, and in
fact that ticket referenced above recommends doing things this way.
In the Mailman.database.initialize(), create a global lock that prevents more
than one process from entering this init function at the same time. It's
probably not strictly necessary given that I believe all the operations in
dbcontext.connect() are multi-processing safe, but it also doesn't seem to
hurt and prevents race conditions regardless of the database's own
safeguards (or lack thereof).
Make sure nightly_gzip.py calls initialize().
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
https://mailman.svn.sourceforge.net/svnroot/mailman/branches/tmp-sqlalchemy-branch
................
r8114 | bwarsaw | 2006-12-06 00:16:54 -0500 (Wed, 06 Dec 2006) | 44 lines
Initial take on using SQLAlchemy to store list data in lieu of Python pickles.
While all the list data (including OldStyleMemberships attributes) are stored
in the database, many attributes are stored as PickleTypes binary data. This
isn't idea but it gets things working until a more sophisticated schema can be
developed.
MailList class is now a new-style class, as is required by SQLAlchemy. This
makes several things, er, interesting. Rip out all the low-level pickle
reading and writing stuff. Hook SA transaction events into Lock() and
Unlock(). Move the hooking of the _memberadaptor into InitTempVars(), which
gets called by the SQLAlchemy hooks (MailList.__init__() never is).
Add an initialize.py module which centralizes all the initialization bits that
command line scripts have to do, including configuration, logging, and atabase
initialization.
This change also converts bin/withlist to mmshell wrapper.
Update to SQLAlchemy 0.3.1.
Revamp paths.py.in considerably. There were several problems with the old
way. We no longer disable default loading of site-packages so we don't need
to add Python's site-packages back to sys.path. Also, because
site.addsitedir() causes things like .pth paths to be /appended/ to sys.path,
they actually won't override any site-installed packages. E.g. if SQLAlchemy
is installed in the system Python, our version will not override. IIUC,
setuptools-based packages can be configured to work properly in the face of
package versions, however not all packages we currently depend on are
setuptools-based. So instead, we steal a bit of stuff from site.py but change
things so the prepend .pth stuff to sys.path.
Update several modules to use True/False and whitespace normalization.
Convert from mm_cfg to config object. Modernize a few coding constructs.
Add a couple of exceptions to handle database problems.
In the export script, include the widget type in the elements. This helped in
my stupid little throw away conversion script, but I think it will be more
generally useful.
Add an interact.py module which refactors interactive interpreter access.
Mostly this is used by withlist -i, but it lets us import Mailman.interact and
drop into a prompt just about anywhere (e.g. debugging).
................
r8115 | bwarsaw | 2006-12-07 09:13:56 -0500 (Thu, 07 Dec 2006) | 22 lines
Start to flesh out more of the SQLAlchemy mechanisms.
Added a MailList.__new__() which hooks instantiation to use a query on
dbcontext to get an existing mailing list. A 'no-args' call means we're doing
a Create(), though eventually that will change too.
For now, disable the CheckVersion() call. Eventually this will be folded into
schema migration.
list_exists(): Rewrite to use the dbcontext query to determine if the named
mailing list exists or not. Requires the fqdn_listname.
Eradicate two failed member adaptors: BDBMemberAdaptor and SAMemberships.
Change the way the DBContext holds onto tables. It now keeps a dictionary
mapping the table's name to the SA Table instance. This makes it easier to
look up and use the individual tables.
Add 'web_page_url' as an attribute managed by SA, and remove a debugging
print.
................
r8116 | bwarsaw | 2006-12-11 07:27:47 -0500 (Mon, 11 Dec 2006) | 29 lines
Rework the whole dbcontext and transaction framework. SA already handles
nested transactions so we don't have to worry about them. However, we do have
the weird situation where some transactions are tied to MailList
.Lock()/.Unlock()/.Save() and some are tied to non-mlist actions. So now we
use an @txn decorator to put methods in a session transaction, but then we
also hook into the above MailList methods as possibly sub-transactions. We
use a weakref subclass to manage the MailList interface, with a dictionary
mapping MailList fqdn_listnames against transactions. The weakrefs come in by
giving us a callback when a MailList gets derefed such that we're guaranteed
to rollback any outstanding transaction.
Also, we have one global DBContext instance but rather than force the rest of
Mailman to deal with context objects, instead we expose API methods on that
object into the Mailman.database module, which the rest of the code will use.
Such methods must be prepended with 'api_' to get exposed this way.
bin/rmlist now works with the SA-backend. I refactored the code here so that
other code (namely, the test suite) can more easily and consistently remove a
mailing list. This isn't the best place for it ultimately, but it's good
enough for now.
New convenience functions Utils.split_listname(), .fqdn_listname().
Convert testall to use Mailman.initialize.initialize(). Not all tests work,
but I'm down to only 8 failures and 7 errors. Also, do a better job of
recovering from failures in setUp().
MailList.__new__() now takes keyword arguments.
................
r8117 | bwarsaw | 2006-12-11 22:58:06 -0500 (Mon, 11 Dec 2006) | 7 lines
Unit test repairs; even though the unit tests are still pretty fragile,
everything now passes with the SQLAlchemy storage of list data.
Added missing 'personalize' column. Converted mailmanctl and qrunner to
initialize() interface. Fixed _cookie_path() to not fail if SCRIPT_NAME is
not in the environment.
................
r8118 | bwarsaw | 2006-12-27 18:45:41 -0500 (Wed, 27 Dec 2006) | 21 lines
Utils.list_names(): Use a database query to get all the list names.
dbcontext.py: Added api_get_list_names() to support Utils.list_names().
listdata.py: Added two additional MailList attributes which need to be stored
in the database. The first is 'admin_member_chunksize' which isn't modifiable
from the web. The second is 'password' which holds the list's password.
HTMLFormatObject: item strings can now be unicodes.
bin/list_lists.py: Must call initialize() to get the database properly
initialized, not just config.load(). This will be a common theme.
SecurityManager.py:
- Remove md5 and crypt support
- Added mailman.debug logger, though it will be only used during
debugging.
- The 'secret' can be a unicode now.
- A few coding style updates; repr() instead of backticks, 'key in dict'
instead of 'dict.has_key(key)'
................
r8119 | bwarsaw | 2006-12-27 19:13:09 -0500 (Wed, 27 Dec 2006) | 2 lines
genaliases.py: config.load() -> initialize()
................
r8120 | bwarsaw | 2006-12-27 19:17:26 -0500 (Wed, 27 Dec 2006) | 9 lines
Blocked revisions 8113 via svnmerge
........
r8113 | bwarsaw | 2006-12-05 23:54:30 -0500 (Tue, 05 Dec 2006) | 3 lines
Initialized merge tracking via "svnmerge" with revisions "1-8112" from
https://mailman.svn.sourceforge.net/svnroot/mailman/branches/tmp-sqlalchemy-branch
........
................
r8121 | bwarsaw | 2006-12-28 23:34:52 -0500 (Thu, 28 Dec 2006) | 20 lines
Remove SIGTERM handling from all the CGI scripts. This messes with HTTPRunner
because when you issue "mailmanctl stop" after the signal handler has been
installed, the process will get a SIGTERM, the signal handler will run, and
the process will exit with a normal zero code. This will cause mailmanctl to
try to restart the HTTPRunner.
I don't think we need that stuff at all when running under wsgi with a
SQLAlchemy backend. If mailmanctl kills the HTTPRunner in the middle of the
process, I believe (but have not tested) that the transaction should get
properly rolled back at process exit. We need to make sure about this, and
also we need to test the signal handling functionality under traditional CGI
environment (if we even still want to support that).
Also, make sure that we don't try to initialize the loggers twice in qrunner.
This was the cause of all the double entries in logs/qrunner.
Fix a coding style nit in mailmanctl.py.
De-DOS-ify line endings in loginit.py.
................
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
name in more than one domain.
- Totally eradicate MAILMAN_SITE_LIST, and in fact the entire need for a site
list. The functions that the site list previously performed are either
removed or supported in other ways. For example, instead of forwarding
owner bounces to the site list, we now have a SITE_OWNER_ADDRESS which
should point to a human, and such bounces are sent there instead. There's
also a "no reply" email address that should be set up to go to devnull. For
any message that never expects a reply, the sender is set to this address.
- Remove the Site.py module. It was an experimental approach to trying to
support virtual domains, and we're going to do it so much better now that
this module is no longer necessary. Site._makedirs() -> Utils.makedir().
- VIRTUAL_HOST_OVERVIEW is completely removed, since now virtual hosts are
always enabled. Virtual domains should be added to mailman.cfg by using the
new add_domain() function. add_virtualhost() is gone. If no virtual
domains are added explicitly, we add the default one that configure guessed
(but we never add that if domains are added explicitly).
- Utils.get_domain() -> Utils.get_request_domain()
- withlist code cleanup and make sure that we load etc/mailman.cfg
- A new base exception called MailmanException is added, from which all
exceptions defined in Errors.py ultimately derive. MailmanError is retained
and derives from MailmanException.
- BadDomainSpecificationError is added.
- Remove the -V/--virtual-host-overview option from list_lists and add instead
-d/--domain and -f/--full.
- bin/update probably works but needs more testing.
- bin/newlist and bin/rmlist take fqdn list names, but default to the default
domain if @whatever isn't given. newlist's -u/--urlhost and -e/--emailhost
options are removed. The domain that the list is being added to must
already exist.
- Minor code cleanup in Message.py
- Bump version to 2.2.0a1
- The Configuration object grows a .domain dictionary which maps email hosts
to url hosts. The reverse mapping is supported, but not directly; use
Configuration.get_email_host() instead.
- Mailman/Cgi/create is converted from mm_cfg to config, and some minor code
cleanup is performed. Also, convert to __i18n_templates__ = True.
- New MailList APIs:
+ property .fqdn_listname
+ GetNoReplyEmail()
+ Create() API changes and refactoring.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
and fixing the invocation and shutdown of mailmanctl. While the tests in this
module work individually, they do not yet work as a group.
-C added to testall.py, and mailmanctl now passes that flag on to qrunner.
UserNotification sets reduced_list_header in the msgdata, but the behavior of
this flag has changed. It used to suppress List-Help, List-Subscribe, and
List-Unsubscribe as well as List-Post and List-Archive. However, List-Help,
List-Subscribe and List-Unsubscribe should definitely be included in
UserNotifications, and List-Post has a different variable controlling it now.
Therefore, always add List-Help, List-Subscribe, and List-Unsubscribe.
Some style updates to Message.py
|
|
|
configuration file. While the full conversion is not yet complete, everything
that seems to be required to run mailmanctl, qrunner, rmlist, and newlist have
been updated.
Basically, modules should no longer import mm_cfg, but instead they should
import Mailman.configuration.config. The latter is an object that's
guaranteed to exist, but not guaranteed to be initialized until some top-level
script calls config.load(). The latter should be called with the argument to
-C/--config which is a new convention the above scripts have been given.
In most cases, where mm_cfg.<variable> is used config.<variable> can be used,
but the exceptions are where the default value must be available before
config.load() is called. Sometimes you can import Mailman.Default and get the
variable from there, but other times the code has to be changed to work around
this limitation. Take each on a case-by-case basis.
Note that the various directories calculated from VAR_PREFIX, EXEC_PREFIX, and
PREFIX are now calculated in config.py, not in Defaults.py. This way a
configuration file can override the base directories and everything should
work correctly.
Other changes here include:
- mailmanctl, qrunner, and update are switched to optparse and $-strings, and
changed to the mmshell architecture
- An etc directory has been added to /usr/local/mailman and a
mailman.cfg.sample file is installed there. Sites should now edit an
etc/mailman.cfg file to do their configurations, although the mm_cfg file is
still honored. The formats of the two files are identical.
- list_lists is given the -C/--config option
- Some coding style fixes in bin/update, but not extensive
- Get rid of nested scope hacks in qrunner.py
- A start on getting EmailBase tests working (specifically test_message),
although not yet complete.
|
