| Commit message (Collapse) | Author | Files | Lines |
|---|
|
disgusting hack).
|
|
(based on sys.hexversion).
|
|
here because that reloads the database as a side-effect - but the
config.db file doesn't exist at this point! This fixes the newlist
problem. Also, there's no need to create the LOCK_DIR file; and use a
safer way to create the next-digest and next-digest-topics files
(which I'm not sure are strictly necessary anymore).
CreateFiles(): Removed.
|
|
subsequent changes won't be reflected in the request database. This
fixes Ricardo Kustner's perplexing problem.
|
|
will end up calling SaveRequestsDb() which closes the database. The
hack is to re-open the database so that the del in HandleRequest()
doesn't fail.
|
|
locked until absolutely necessary.
|
|
Lock(): Always re-load the config.db file once the lock has been
acquired. This fixes a very serious data consistency bug, explained
briefly as:
- Process A opens a list unlocked, loading the marshaled
dictionary. Process A then tries to acquire the list lock.
- Meanwhile, Process B does the same, but it actually acquires the
lock first. Process B makes some changes, saves the list's
marshal and relinquishes the lock.
- Process A now acquires the lock. However, its database is now
out-of-date because of the Process B write. It does its work
and blithely rewrites the marshal, blowing away all of Process
B's changes.
The solution is to reload the marshal when the list lock is acquired.
__init__(): Use the new Lock() semantics more efficiently.
|
|
|
|
|
|
happens in the parent process.
poll_newsgroup(): Take a `lock' argument to keep the gate_news lock
fresh while gating the newsgroup. Some changes to the log messages.
gate_news(): Removed -- folded into process_lists().
process_lists(): Take a `lock' argument to pass to poll_newsgroup().
Perform all work in the parent process. Better logging.
|
|
Ron Jarrell.
|
|
|
|
|
|
|
|
messages being delivered by qrunner it is quite easy to simply run out
of process resources. Now each message to be archived is done in the
parent process, with a bit of extra paranoia in case of archiver
errors.
This also gets rid of the individual archiver locks since it is
required that the list itself be locked in order to get here.
|
|
breaking out of the main loop when either are exceeded.
|
|
management variables for limiting the work a single qrunner process
can do.
|
|
self.database.getArticle() calls in a try/except KeyError. These are
stopgap measures to avoid exceptions percolating upwards when an
archive is regenerated with bin/arch. But it doesn't really fix
anything substantial (can you say "pipermail is broken"? ;)
|
|
likely", initialize the date variable to a float time seconds, not a
time tuple. This is because the following time.ctime() call expects a
float.
|
|
|
|
with an underscore).
|
|
hard-coded list (see change in qrunner).
|
|
LIKELY_BOUNCE_SENDERS, then this message is probably bounced from a
bogus admin address. Catch and stop the loop by not attempting
delivery again.
Fixes PR#285, BUG#106592
|
|
# List of addresses (lhs of the @) that likely come only from MTAs
# bouncing messages. This is used in qrunner and
# MailCommandHandler.py to stop processing or forwarding such
# messages.
|
|
|
|
|
|
understandable. Also, added the volatile _mass_catchup attribute for
explicitly catching up a linked newsgroup.
|
|
I.e. these are selections that can be presented in an admin page, but
don't directly affect the value of a saved list attribute. This is
used for a "Mass Catchup" button in the gateway category. Volatile
property convention is that the name starts with an underscore.
Specifically,
GetItemGuiValue(): For Radios and Toggles, only do a getattr() of
varname if the name does not start with an underscore. Otherwise
default the checked value to 0. This is bogus because it would be
better to take the default value from the property description, but
that's too much of an API change for now.
ChangeOptions(): If the property name starts with an underscore, do
something different. This has to check for specific volatile
parameters, of which the only one currently is `_mass_catchup'. If
that's the property and the value is true, then set usenet_watermark
to None, which is the signal to gate_news to do a mass catchup
(i.e. set the watermark to the last article currently on the server).
Note that the GatewayManager category used to always do a mass catchup
when gateway_to_mail was toggled. This isn't always the right thing,
so this change allows the admin to explicitly do a catch up.
|
|
|
|
function to call with no arguments, every time through the while
loop. This lets callers, e.g. keep a lock fresh while waiting for all
the children to finish.
|
|
dictionary, which gets filled in with the child pid. This allows it
to be reaped by functions higher up in the call chain.
Second, and more importantly, be quite paranoid about the fork and
child processes. Make absolutely sure that the child process exits.
Wrap all the child code in try blocks so that no matter what, the lock
will get released (by calling unlock() with the unconditionally flag
set). If no exception occurs, the child will exit with status 0,
otherwise the traceback will be printed to stderr, a message
indicating which list has a corrupt archive is logged to logs/error,
and the child process will exit with status 1.
Extend the archive.lock to 1 hour.
|
|
to be more robust so that if either fail, we end up with an empty
self.dict and self.sorted.
Note that the archiver subprocess will still fail with an exception.
Fixing this will require much more work on the archiver as a whole,
and isn't worth it right now. But this fix averts the problem when
regenerating the archive from scratch using bin/arch, so at least
corrupt archives can be rebuilt.
|
|
call. If the module does a fork, and the child process returns here,
we exit immediately.
|
|
from the gate_news cronjob is that the msgdata dictionary will contain
a `fromusenet' key. This was erroneously checking the msg object for
the `fromusenet' attribute (as was the case with the old interface),
which caused notifications to go to Usenet posters whose messages were
held. Bogus!
|
|
will get filled with the pids of any children that get forked.
|
|
Make absolutely sure that the child process exits without returning
control to the caller. If we complete successfully, we exit with
status 0. If any exception occurs we print a traceback and exit with
status 1.
|
|
errors go to the logs/gate_news file, while any exceptions that
percolate to the top would go to logs/error.
poll_newsgroup(): Always update the watermark, even if the message has
been seen before (i.e. we found an X-BeenThere: header) or if an
nntp.error_temp occurred (which can happen if the message has
expired). Any other exceptions will not be caught and thus will not
update the watermark.
process_lists(): Be extra paranoid about the forks and child
processes. First of all, the fork could fail and if so, we stop
cruising through the mailing lists and return to the parent the
dictionary of child pids gathered so far. Second, make absolutely
sure that the child process exits without returning control to the
caller. If we complete successfully or if a TimeOutError occurred, we
exit with status 0, otherwise we print a traceback and exit with
status 1.
main(): First off, we added a keepalive function to geep the gate_news
lock fresh while we're waiting for our child processes to exit.
Second, we added a failsafe around process_list() -- which does a fork
-- so that in the unlikely situation where the child process could
have gotten returned here, we exit immediately.
|
|
errors go to the logs/qrunner file, while any exceptions that
percolate to the top would go to logs/error.
main(): Refresh the qrunner lock each time through the main loop.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
development platform.
|
|
DeliverToList(): Handle `_enqueue_immediate' by bypassing the call to
do_pipeline and enquing the message immediately to disk. Still
maintains the same interface to the outside world.
Also, copy the pipeline list regardless of whether it comes from
LIST_PIPELINE or from msgdata.
|
|
marked as volatile and not saved to the .db file.
Use Utils.reap().
|
|
|
|
parameter to RadioButtonArray() constructor, and use the new symbolic
constants instead of the hardcoded values.
|
|
slightly different bounce reports froma NMS 3.6 servers. We now
search just for multipart/* parts, and look only for message/*
sub-parts.
|
