| Commit message (Collapse) | Author | Age | Files | Lines |
| ... | |
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Configurable no-list error.
- Ultimate loop stop address in transport -> aliases.
- LMTP_ONLY_DOMAIN needs no individual transport entry.
- Use of alias/lmtp is exclusive.
WSGI brushups.
- _cookie_path() was made simple and retain common cookie for
admin/admindb/... etc.
- Removed absolute=1 from admindb/confirm/create/options.
configuration.py
- Use of add_runner() in etc/mailman.cfg needs change. config
is not loaded yet?
|
| |
|
|
|
|
|
| |
- if config.USE_LMTP immediately after ALIASFILE add/remove; we need
to keep alias settings because postfix rejects as unknown if not present.
- STANZA listname@hostname.
- convert tabs to spaces.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
| |
Also, the now scripts can be accessed by arbitrary script base.
E.g. /listinfo can be accessed by /mailman/listinfo or /mailman/blah/listinfo
etc. etc. This is useful in testing wsgi directly without apache frontend
but we may have to limit the length or depth of prefixed script base.
|
| |
|
|
|
|
|
| |
Do this and switch PUBLIC_ARCHIVE_URL to use $-substitution strings instead of
%-substitution strings (no backward compatibility is provided).
Minor style nits.
|
| |
|
|
|
|
|
|
| |
notifications.
wsgi_app.py ... URI normalization by stripping trailing slash. We need
Special care for 'private'.
Strip dot only components in the PATH_INFO for sanitization.
|
| |
|
|
|
|
|
|
|
| |
from mm_cfg to the configuration object.
In the config object, add an add_runner() function for extending the QRUNNERS
variable.
De-DOS-line-ending-ify configuration.py
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
but a reverse proxy maps our wsgiref server into an upstream server's web
space.
Here's the basic problem: the Set-Cookie header we return has a Path attribute
which must match subsequent request uri's in order for the client to send the
cookie data back to us later in a Cookie header of that subsequent request.
The problem though is that we cannot guarantee that we know how our wsgi
server is mapped into the upstream proxy. It's probably '/mailman/' for
backward compatibility, but there's no way for us to tell, because there's
nothing specifically included in the request that tells us what the originally
requested uri is.
If we get the cookie path wrong, the effect is to require a login every time
an admin page is hit, because the client will not see a matching path prefix
and will not send us the cookie data.
We solve this (albeit, by hack) by looking at the HTTP_REFERER environment
variable we see. This will point to the admin login page on which the admin
password was entered. We'll pick this uri apart to attempt to find the prefix
at which our wsgi server was mapped. If we find this, we'll use it to craft
an appropriate cookie path. Hopefully. If that cgi environment variable is
not available, we just return the path as we've seen it.
This approach allows for accessing the admin pages either through a reverse
proxy or directly, with no additional configuration necessary. In fact, both
access mechanisms can work at the same time; try hitting these two uri's with
different browsers:
http://example.com/mailman/admin/mylist@example.com/general
http://example.com:2580/admin/mylist@example.com/general
The first will hit our reverse proxy, and the second will hit our wsgi server
directly. The responses will included two different cookie paths, but both
will work! This is an important principle to uphold, especially for debugging
purposes. Note that I could have added a configuration variable to handle the
cookie path remapping, but then we'd have to support either the reverse proxy
or the wsgi server, but not (easily) both. Plus, it's another configuration
variable. Yuck.
Note that Apache 2.2 has something called the ProxyPassReverseCookiePath
directive, which should probably be used if available. It's not in Apache
2.0, which is probably the most prevalent server in use with Mailman right
now, so we can't count on it. Plus, if ProxyPassReverseCookiePath is
available, our algorithm should still work.
What about other proxy servers? Dunno. We'll have to wait for feedback from
users.
This change also fixes a buglet with MTA/Postfix.py when
POSTFIX_STYLE_VIRTUAL_DOMAINS is used. You can end up trying to call
_update_maps() before data/aliases exist. This just defers that call until
it's guaranteed both the transport and the alias files have been created.
Also, finish converting SecurityManager.py to use the configuration object
(and the Defaults module where appropriate) instead of mm_cfg.py.
|
| |
|
|
|
|
|
|
|
|
|
| |
instead of mm_cfg.py. This involved mostly mechanical replacements, but
there were a few gotchas to make sure that various calls and assignments
that ultimately referenced the config were delayed until after the config
was loaded.
Updated configuration.py to throw an exception if config.load() is called
with a non-existent filename argument.
Updated loginit.py to add the fromusenet log used by gate_news.py.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
and some of the admindb.py links. There may still be breakage in other parts
of the interface and I haven't gone back to verify that traditional CGI still
works.
Changes:
- Add wsgiref-0.1.2-py2.4.egg so that we can still do WSGI in Python 2.4,
which doesn't come with wsgiref. Of course this means we /also/ have to add
setuptools-0.5c3 because eggs require setuptools.
- Style cleanups in HTTPRunner.py and wsgi_app.py. Also, use cStringIO
instead of StringIO.
- All internal links within the listinfo and admin pages are (or at least
should be ;) relative now. This should make other things better, such as
running Mailman over https or alternative ports. It does kind of mean that
web_page_url is obsolete, but I haven't looked at whether we can completely
eradicate it.
- ValidateEmail(): Use ' ' in s instead of s.count(' ') > 0.
- GetPathPieces(): When path is false, return the empty list instead of None,
so we can still len() it.
- ScriptURL(): Much simpler. To support relative urls as the default, we
change the API so that it only takes a 'target' argument (i.e. the script we
want to link to). It no longer takes 'absolute' or 'web_page_url', and it
constructs its link from GetPathPieces(), the target, and the cgi extension.
- GetRequestURI(): code style updates.
- Mailman/bin/show_config.py: De-DOS-line-ending-ification.
- export.py: A few modifications, although this is likely still not final (I'm
still working on the import script). First, for <option> elements, don't
put the value in an attribute, put it in the text body of the element.
Second, put the list <option> tags in a <configuration> element. Third, put
the preferred language on an <option> tag with a 'preferred_language'
name attribute value.
- SecurityManager: Make sure that MakeCookie() and ZapCookie() use the same
'path' cookie value by refactoring that into a separate method. That method
now returns just the SCRIPT_NAME and the full listname. web_page_url
doesn't enter into it.
- loginit.py: Add a 'debug' logger since it's just too useful to have :)
- admin.py: Remove the extra / right before the query string in ?VARHELP
urls. That extra / turns out to be problematic with the relative url scheme
we're using now.
- Auth.py: whitespace normalization and copyright years update. Also, remove
a couple of unnecessary imports. Also, make sure that the actionurl is
relative.
- create.py: Typo.
- private.py: mm_cfg -> config object
- In MailList.py: GetScriptURL() can be written in terms of Utils.ScriptURL()
now.
|
| |
|
|
|
|
| |
- added a -C/--config option to bin/show_config.py and made a few
other changes.
- MailList.py - r8040 broke Mailman/bin/update.py situate_list.
Fixed it with this change.
|
| | |
|
| |
|
|
|
|
|
|
| |
Add: HTTPRunner.py ... Start/Restart/Stop HTTP Server under Runner framework.
wsgi_app.py ... WSGI to CGI wrapper. Mostly taken from scripts/driver.
loginit.py ... Add http log. Time stamp is duplicated :-(
Defaults.py ... HTTP_HOST and HTTP_PORT.
Note that WSGI server should be used under reverse proxy environment.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
membership. The next step is to write an XML importer that reads this file.
I'm not 100% sure that all the import data is included yet, but the intent is
that this will be the official way to move mailing lists.
A few notes: member passwords are not included by default, the idea being that
if we enable XML dumping from the web, we don't want the clear text user
passwords to be leaked. A command line option includes the member passwords.
Also, the various substitutable texts (i.e. those that include %-strings) will
be autoconverted to $-strings. In Mailman 2.2, we'll only have $-strings,
although this is not yet enforced in other parts of the code yet.
Convert config_list.py to mmshell, $-strings, and optparse.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
now more conformant to RFC 2033. Changes include:
- Make sure LMTPRunner returns a status code for each recipient accepted
during the RCPT TO part of the protocol. This is required by RFC 2033.
- Use email.message_from_string() directly instead of creating a StringIO.
The email package will already do this for us.
- Use email 4.0 package names
- Create both an error and a queue logger, and write the appropriate messages
to each. Note that we'll never cleanly exit out of asyncore.loop() so the
trailing log message is kind of pointless.
- Remove the -admin address.
- Simplification, refactoring, and whitespace normalization
- Updated docstring
|
| | |
|
| |
|
|
| |
be convenient to put here because mailmanctl can restart it.
|
| | |
|
| |
|
|
|
|
|
| |
installation. I've tested four cases of combination of
POSTFIX_STYLE_VIRTUAL_DOMAINS (Any/None) and USE_MAIL_DIR (Yes/No).
Also, I've added POSTFIX_VIRTUAL_SEPARATOR = '_at_' for unique mapping of
local part in the virtual-mailman/aliases files.
|
| | |
|
| |
|
|
|
|
|
|
| |
the new configuration object: admin, admindb, bounces, confirm, inject, join,
leave, owner, post, request, unshunt, version.
Also change MailList.GetScriptURL() to return the list's fully qualified name
in links.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
bin/withlist: If there's no '@' in the listname, append the DEFAULT_EMAIL_HOST
so we always get a fully qualified list name.
bin/mmsitepass: plumb through -C/--config switch and be sure to call
config.load().
Convert Mailman/MTA/Postfix.py to configuration.config, and update MTA/Manual.
In mailman/Cgi/create, we can't convert straight from a string to a bool,
because bool('0') is True. We need to go through int first.
MailList.InitTempVars(): The logic here looked weird because we could get
'name' = None and that would break. Assume name is never None.
|
| |
|
|
|
|
|
|
| |
the web u/i to work. This also fixes the use of Utils.list_names() in the
list and admin overviews. This API now returns a set, but the CGIs want to
sort them, so we need to turn them back into lists.
This change also elaborates an exception so that the list name is reported.
|
| |
|
|
| |
usage.
|
| | |
|
| | |
|
| |
|
| |
details=all page.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
built-in sets and sadly we still have to support it. Utils.list_names()
therefore now uses a sets.Set if it finds the built-in set type missing.
main() in list_lists.py also has to list()-ify the set in order to sort it.
There's a bootstrapping problem when bin/update tries to re-situate a
non-qualified list name to a qualified list name. The problem is that the
MailList ctor will call Load() with the nonqual name, which in turn will call
CheckVersion() by default. The latter will then turn around and try to call
Load() with the fqdn list name, but that list name won't yet exist since it
hasn't been situated.
The solution (but maybe not the best one) then is to pass the check_version
flag to the MailList ctor, which gets passed down to the Load() call. Only
situate_list() will call this to avoid the version checking before the list
has been situated.
|
| |
|
|
| |
accurately represent their current meaning.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
compatible back to Python 2.3, this change should not get back ported to
Mailman 2.1.
Port to Python 2.5. The non-test suite changes should get back ported to
Mailman 2.1 (which I will do next), but don't worry about the test suite ones
because MM2.1's test suite is hopeless. Specifically:
- In SecurityManager.py, fix the parsecookie() code to work with Python 2.5
generated cookie text. The latter was changed to be more RFC compliant so
it does not output training semicolons for each line of cookie text. This
broke the splitting rules, so now first split on newlines, then on ';\s*'.
This should work across all Python versions.
- In Python 2.5, exceptions are new-style, and thus are no longer of
ClassType. The instantiation type test in hold_for_approval() was too
naive.
- Raising strings generates deprecation warnings in Python 2.5. Switch the
one weird use of this in Utils.py to use a class exception. Don't call it
"quick exit" though because it's probably not.
- In the tests, use True/False instead of 1/0
- Use failUnless/failIf instead of assertEqual against True/False.
- In the tests, use Message.get_content_type() instead of Message.get_type()
since the latter is gone in email 4.0.1. Same with get_content_maintype()
and get_main_type().
|
| |
|
| |
The caller may want to override the list setting.
|
| | |
|
| |
|
| |
tosb.finish() call is the wrong switchboard. Changed to sb.finish().
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Also add an experimental (and currently non-functioning) SQLAlchemy
implementation.
The MemberAdaptor.py interface has been updated in a couple of ways. First, a
"transaction interface" has been added so that Mailman can properly sync with
the member adaptor. Newly supported methods are load(), lock(), save(), and
unlock() and these correspond to methods in the MailList object. Second,
__init__() is officially documented to take a MailList instance and nothing
else. Third, some of the existing docstrings were incorrect w.r.t. the
OldStyleMemberships implementation (such as rasing BadPasswordError in some
cases). Most of these should not be the responsibility of the MemberAdaptor,
so the docstrings have been updated.
Test cases have been added and a new Defaults.py.in variable called
MEMBER_ADAPTOR_CLASS has been added which names the class to use. Of course
OldStyleMemberships are named by default. There's also a
SQLALCHEMY_ENGINE_URL variable for use with the experimental member adaptor.
Fix a bug in Configuration where if the etc/mailman.cfg file wasn't found and
the mm_cfg.py file was used as a fallback, it would blow away the original
namespace copied from Defaults.py.in. This wasn't a problem until we started
adding additional names to that namespace, such as 'add_domain'.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
uncleanly (e.g. segfaults the Python interpreter), messages being processed
will not be lost.
The vulnerability, ideas, and patches are credited to Richard Barrett and Mark
Sapiro. Their original work was modified by Barry for this commit and any
bugs are his fault.
The basic idea is that instead of unlinking a .pck file in dequeue(), the file
is renamed to a .bak file. The Switchboard grows a finish() method which then
unlinks the .bak file. That class's constructor also grows a 'restore'
argument (defaulting to false), which when true moves all .bak files it finds
in its hash space to .pck, thereby restoring a file lost while "in flight".
This relies on the fact that even with multiple qrunners, exactly one process
will be responsible for one hash space slice, so it's never possible (under
normal operation) for a .bak file to be renamed to .pck by some other process.
Test cases for both the new Switchboard behavior and the use of that by Runner
subclasses has been added.
There are two things to watch out for, either of which may require some
additional changes. There is some small potential to duplicate messages in
various queues, if say 'mailmanctl' were improperly started more than once by
a site admin. This usually won't happen unless an admin is overly eager with
the mailmanctl -s switch, so we can chalk this one up to operator error. I'm
not sure what more we can do about that.
There's also a possibility that if we're processing a message that continually
causes the Python interpreter to crash, we could end up duplicating messages
endlessly. This is especially troublesome for the Outgoing runner which could
conceivably cause a mail flood. I consider this the more critical issue to
defend against, probably by adding a numbering scheme to the .bak file names
and refusing to restore a .bak file more than say 3 times without human
intervention.
|
| |
|
|
|
| |
slices that could result in unprocessable queue entries. Improved FIFO
processing when two queue entries have the same timestamp.
|
| |
|
|
|
|
|
|
| |
when a mailing list is created. The problem is that MailList.Create() will
call MailList.InitVars() which in turn calls Archiver.InitVars(). The latter
creates private archive directories based on the list's host_name attribute.
But Create() couldn't set this to what it knows it should be because
MailList.InitVars() always blew it away.
|
| |
|
|
|
|
|
|
|
|
| |
see the traceback.
Make sure that in EmailBase.tearDown() we actually wait for the SinkServer to
exit before continuing.
We still leave test turds in archives/private, but I haven't yet figured it
out.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
this change. More unit tests should be added.
misc/sitelist.cfg is removed -- this is an ex-site list.
MailList.GetNoReplyEmail() -> MailList.no_reply_address (property)
UserNotification._enqueue(), OwnerNotification._enqueue(): when queing the
message to the virgin queue, be sure to use the fully qualified (i.e. posting)
address for the list.
In the MTA modules, be sure to set up the target of the mail commands as the
fqdn listname because otherwise we can't find the correct list. This needs
some tweaking/testing for Postfix's virtual domain support.
MailList.Load() has to grow an optional argument specifying the fqdn
listname. The problem is that in some situations, we can't calculate that
because we don't know _internal_name, so it has to be passed in. This is
mostly the case in the MailList ctor where a Load hasn't happened yet. For
backward compatibility though, if it's not passed in, just use
mlist.fqdn_listname.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
name in more than one domain.
- Totally eradicate MAILMAN_SITE_LIST, and in fact the entire need for a site
list. The functions that the site list previously performed are either
removed or supported in other ways. For example, instead of forwarding
owner bounces to the site list, we now have a SITE_OWNER_ADDRESS which
should point to a human, and such bounces are sent there instead. There's
also a "no reply" email address that should be set up to go to devnull. For
any message that never expects a reply, the sender is set to this address.
- Remove the Site.py module. It was an experimental approach to trying to
support virtual domains, and we're going to do it so much better now that
this module is no longer necessary. Site._makedirs() -> Utils.makedir().
- VIRTUAL_HOST_OVERVIEW is completely removed, since now virtual hosts are
always enabled. Virtual domains should be added to mailman.cfg by using the
new add_domain() function. add_virtualhost() is gone. If no virtual
domains are added explicitly, we add the default one that configure guessed
(but we never add that if domains are added explicitly).
- Utils.get_domain() -> Utils.get_request_domain()
- withlist code cleanup and make sure that we load etc/mailman.cfg
- A new base exception called MailmanException is added, from which all
exceptions defined in Errors.py ultimately derive. MailmanError is retained
and derives from MailmanException.
- BadDomainSpecificationError is added.
- Remove the -V/--virtual-host-overview option from list_lists and add instead
-d/--domain and -f/--full.
- bin/update probably works but needs more testing.
- bin/newlist and bin/rmlist take fqdn list names, but default to the default
domain if @whatever isn't given. newlist's -u/--urlhost and -e/--emailhost
options are removed. The domain that the list is being added to must
already exist.
- Minor code cleanup in Message.py
- Bump version to 2.2.0a1
- The Configuration object grows a .domain dictionary which maps email hosts
to url hosts. The reverse mapping is supported, but not directly; use
Configuration.get_email_host() instead.
- Mailman/Cgi/create is converted from mm_cfg to config, and some minor code
cleanup is performed. Also, convert to __i18n_templates__ = True.
- New MailList APIs:
+ property .fqdn_listname
+ GetNoReplyEmail()
+ Create() API changes and refactoring.
|
| |
|
|
| |
so that failures there won't cause stale _xtest lists to continue to exist.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
and fixing the invocation and shutdown of mailmanctl. While the tests in this
module work individually, they do not yet work as a group.
-C added to testall.py, and mailmanctl now passes that flag on to qrunner.
UserNotification sets reduced_list_header in the msgdata, but the behavior of
this flag has changed. It used to suppress List-Help, List-Subscribe, and
List-Unsubscribe as well as List-Post and List-Archive. However, List-Help,
List-Subscribe and List-Unsubscribe should definitely be included in
UserNotifications, and List-Post has a different variable controlling it now.
Therefore, always add List-Help, List-Subscribe, and List-Unsubscribe.
Some style updates to Message.py
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
configuration file. While the full conversion is not yet complete, everything
that seems to be required to run mailmanctl, qrunner, rmlist, and newlist have
been updated.
Basically, modules should no longer import mm_cfg, but instead they should
import Mailman.configuration.config. The latter is an object that's
guaranteed to exist, but not guaranteed to be initialized until some top-level
script calls config.load(). The latter should be called with the argument to
-C/--config which is a new convention the above scripts have been given.
In most cases, where mm_cfg.<variable> is used config.<variable> can be used,
but the exceptions are where the default value must be available before
config.load() is called. Sometimes you can import Mailman.Default and get the
variable from there, but other times the code has to be changed to work around
this limitation. Take each on a case-by-case basis.
Note that the various directories calculated from VAR_PREFIX, EXEC_PREFIX, and
PREFIX are now calculated in config.py, not in Defaults.py. This way a
configuration file can override the base directories and everything should
work correctly.
Other changes here include:
- mailmanctl, qrunner, and update are switched to optparse and $-strings, and
changed to the mmshell architecture
- An etc directory has been added to /usr/local/mailman and a
mailman.cfg.sample file is installed there. Sites should now edit an
etc/mailman.cfg file to do their configurations, although the mm_cfg file is
still honored. The formats of the two files are identical.
- list_lists is given the -C/--config option
- Some coding style fixes in bin/update, but not extensive
- Get rid of nested scope hacks in qrunner.py
- A start on getting EmailBase tests working (specifically test_message),
although not yet complete.
|