| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
from Mailman.Queue to Mailman.queue (note the case change to be more PEP 8
compliant).
The Switchboard and Runner classes have been moved into the package
__init__.py and the previous class modules have been removed.
The switchboard cache is removed; I don't think it was ultimately buying us
much. Now, just import the Switchboard class and instantiate it directly.
Added an IRunner interface.
Renamed the ArchRunner to ArchiveRunner.
bin/qrunner and bin/mailmanctl are updated accordingly. For the former, it no
long accepts -r=All to run all qrunners. You can still use the short name
(e.g. --runner=incoming) to run the built-in queue runners, but this design
will eventually allow for plugin qrunners by allowing them to be run
specifying the full package path to the class. It also now accepts a leading
dot to indicate a qrunner class relative to the Mailman.queue package.
|
| |
|
|
|
|
|
|
| |
a null byte or other garbage in the extension. Cleaned this.
- Improved handling of None payloads.
- Cleaned up a few charset coercions.
OutgoingRunner.py - Made probe bounce processing and queuing of bounces
conditional on having some permanent failure(s).
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
configuration file. While the full conversion is not yet complete, everything
that seems to be required to run mailmanctl, qrunner, rmlist, and newlist have
been updated.
Basically, modules should no longer import mm_cfg, but instead they should
import Mailman.configuration.config. The latter is an object that's
guaranteed to exist, but not guaranteed to be initialized until some top-level
script calls config.load(). The latter should be called with the argument to
-C/--config which is a new convention the above scripts have been given.
In most cases, where mm_cfg.<variable> is used config.<variable> can be used,
but the exceptions are where the default value must be available before
config.load() is called. Sometimes you can import Mailman.Default and get the
variable from there, but other times the code has to be changed to work around
this limitation. Take each on a case-by-case basis.
Note that the various directories calculated from VAR_PREFIX, EXEC_PREFIX, and
PREFIX are now calculated in config.py, not in Defaults.py. This way a
configuration file can override the base directories and everything should
work correctly.
Other changes here include:
- mailmanctl, qrunner, and update are switched to optparse and $-strings, and
changed to the mmshell architecture
- An etc directory has been added to /usr/local/mailman and a
mailman.cfg.sample file is installed there. Sites should now edit an
etc/mailman.cfg file to do their configurations, although the mm_cfg file is
still honored. The formats of the two files are identical.
- list_lists is given the -C/--config option
- Some coding style fixes in bin/update, but not extensive
- Get rid of nested scope hacks in qrunner.py
- A start on getting EmailBase tests working (specifically test_message),
although not yet complete.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
traces of our crufty old Syslog. Most of this work was purely mechanical,
except for:
1) Initializing the loggers. For this, there's a new module
Mailman/loginit.py (yes all modules from now on will use PEP 8
names). We can't call this 'logging.py' because that will
interfere with importing the stdlib module of the same name (can
you say Python 2.5 and absolute imports?).
If you want to write log messages both to the log file and to
stderr, pass True to loginit.initialize(). This will turn on
propagation of log messages to the parent 'mailman' logger, which
is set up to print to stderr. This is how bin/qrunner works when
not running as a subprocess of mailmanctl.
2) The driver script. I had to untwist the StampedLogger stuff and
implement differently printing exceptions and such to log/error
because standard logging objects don't have a write() method. So
we write to a cStringIO and then pass that to the logger.
3) SMTPDirect.py because of the configurability of the log messages.
This required changing SafeDict into a dict subclass (which is
better than using UserDicts anyway -- yay Python 2.3!). It's
probably still possible to flummox things up if you change the
name of the loggers in the SMTP_LOG_* variables in mm_cfg.py.
However, the worst you can do is cause output to go to stderr and
not go to a log file.
Note too that all entry points into the Mailman system must call
Mailman.loginit.initialize() or the log output will go to stderr
(which may occasionally be what you want). Currently all CGIs and
qrunners should be working properly.
I wish I could have tested all code paths that touch the logger, but
that's infeasible. I have tested this, but it's possible that there
were some mistakes in the translation.
- Mailman.Bouncers.BounceAPI.Stop is a singleton, but not a class
instance any more.
- True/False code cleanup, PEP 8 import restructuring, whitespace
normalization, and copyright year updates, as appropriate.
|
| |
|
|
|
|
|
|
|
|
|
| |
- Remove True/False binding cruft
- Remove __future__ statements for nested scopes
- Remove ascii_letters import hack from Utils.py
- Remove mimetypes.guess_all_extensions import hack from Scrubber.py
- In Pending.py, set _missing to object() (better than using [])
Also, update copyright years where appropriate, and re-order imports more to
my PEP 8 tastes. Whitespace normalize.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
loop.
_dispose(): Remove the logic for retrying temporary failures from here
to the RetryRunner. This eliminates total cpu consumption on the
re-queuing of messages with tempfailures (as can happen with delivery
via Postfix to non-existent local addresses in its default
configuration).
Restructure the SomeRecipientsFailed clause and removed the
deliver_after calculation -- we don't need it. DELIVERY_RETRY_WAIT is
also removed.
_cleanup(): When shutting down the qrunner, be sure to register all
deferred permanent failures.
Backport candidate.
|
| |
|
|
|
|
|
|
|
|
|
| |
_dispose(): Add another key to the msg metadata, deliver_after. If a
temporary failure occurs, this key will hold the time in epoch-seconds
after which we can retry delivery. This stops us from retrying every
second as reported in SF bug # 699900. Default is to retry once per
hour (see Defaults.py.in).
Also, use copy.deepcopy() instead of converting the message to a
string and re-parsing it.
|
| |
|
|
|
| |
msgdata's "recips" key to the list of recipients that were rejected,
otherwise lots of folks on the list will get duplicates!
|
| |
|
|
|
|
|
|
|
| |
delivery module (i.e. SMTPDirect) when it couldn't connect to the smtp
server. When this happens, log an error message, but only do it once
so we don't fill up logs/error with a message-per-second. We keep a
self.__logged flag which is set when we log the error and reset in the
class constructor, and when the process() function returned
successfully.
|
| |
|
|
| |
call mlist.Load() so we can be sure we've got the most current state.
|
| |
|
|
|
|
|
|
|
| |
_open_list() caches the MailList instances in a WeakValueDictionary
which don't contribute to the reference count, but ensure that each
runner will get one and only one copy of the MailList data.
This should help in the memory footprint of the runners, but I suspect
we may still be leaking something.
|
| |
|
|
|
|
| |
more conveniently use a class attribute to specify this. On the
BounceRunner, turn off MailList object caching so that it'll be more
friendly to long-term memory use.
|
| |
|
|
|
|
|
|
| |
extraction of the process() method into the constructor. I don't
think it's too much of a burden to have to restart the OutgoingRunner
if you want to change the DELIVERY_MODULE (and I suspect it won't
change that much anyway). This also seems to help control memory
growth in the OutgoingRunner.
|
| |
|
|
|
|
|
| |
specializing the queue directory, make QDIR a class attribute and have
Runner's ctor use that instead of an __init__() argument. This lets
us get rid of most of the __init__()'s in the derived classes.
(OutgoingRunner still needs one though.)
|
| |
|
|
|
|
|
|
| |
class knows to reload the state when pulling the object from the
cache. Hmm, why do we even have a cache now?
_dispose(): Don't do the `verp' key calculation here; it's done
earlier and better in the ToOutgoing.py handler.
|
| | |
|
| |
|
|
|
| |
Consults VERP_DELIVERY_INTERVAL to decide whether to set the `verp'
metadata key.
|
| |
|
|
|
| |
Also, use cStringIO directly instead of our own hack-around StringIO
module.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
try to deal with deferred permanent failures.
_doperiodic(): It can be a bit expensive to actually deal with
permanent failures every time through this loop, and we don't need
that level of accuracy. So now we keep a counter and only process
permanent failures every DEAL_WITH_PERMFAILURES_EVERY times. (Okay,
so this variable defaults to 1, but you get the idea. :)
Also, move the Save() inside the try clause. I've decided that
finally clauses should only unlock the list, not try to save it.
|
| |
|
|
|
|
|
| |
_dispose(): Remove the "except Exception" class which allows any
exceptions to percolate up to Runner.__oneloop()'s catch-all exception
handler. Runner properly implements the log-and-shunt procedure for
any such exceptions.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
RegisterBounce on known permanent failures. These are handed to us
via the exception object when SomeRecipientsFailed is raised. Append
any permanent failures to an instance variable to be handled in the
_doperiodic() method.
Temporary failures are still handled the same way, except we only
requeue the message if we've actually got temp failures to retry.
_doperiodic(): Periodically, we attempt to acquire the list lock and
register all permanent failures we've got pending for that list.
|
| | |
|
|
|
Runner is the base class providing the framework for other qrunner
specializations.
IncomingRunner runs through qfiles/in taking messages from the smtpd,
sending them through the standard message handler pipeline and
potentially writing the message to qfiles/out or qfiles/news.
OutgoingRunner takes message from qfiles/out and hands them over to
the MTA.
NewsRunner takes message from qfiles/news and hands them over to the
news server.
|
