| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
0.4. Lots of things changes, which broke lots of our code. There are still a
couple of failures in the test suite that I don't understand. It seems that
for pending.txt and requests.txt, sometimes strings come back from the
database as 8-bit strings and other times as unicodes. It's impossible to
make these tests work both separately and together.
users.txt is also failing intermittently. Lots of different behavior between
running the full test suite all together and running individual tests. Sigh.
Note also that actually, Elixir 0.4.0 doesn't work for us. There's a bug in
that version that prevented zope.interfaces and Elixir working together. Get
the latest 0.4.0 from source to fix this.
Other changes include:
- Remove Mailman/lockfile.py. While I haven't totally eliminated locking, I
have released the lockfile as a separate Python package called locknix,
which Mailman 3.0 now depends on.
- Renamed Mailman/interfaces/messagestore.py and added an IMessage interface.
- bin/testall raises turns on SQLALCHEMY_ECHO when the verbosity is above 3
(that's three -v's because the default verbosity is 1).
- add_domain() in config files now allows url_host to be optional. If not
given, it defaults to email_host.
- Added a non-public interface IDatabase._reset() used by the test suite to
zap the database between doctests. Added an implementation in the model
which just runs through all rows in all entities, deleting them.
- [I]Pending renamed to [I]Pended
- Don't allow Pendings.add() to infloop.
- In the model's User impelementations, we don't need to append or remove the
address when linking and unlinking. By setting the address.user attribute,
SQLAlchemy appears to do the right thing, though I'm not 100% sure of that
(see the above mentioned failures).
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
is moved to Mailman.lockfile.
Remove a few more MailList methods that aren't used any more, e.g. the lock
related stuff, the Save() and CheckValues() methods, as well as
ChangeMemberName().
Add a missing import to lifecycle.py.
We no longer need withlist to unlock the mailing list. Also, expose
config.db.flush() in the namespace of withlist directly, under 'flush'.
|
| | |
|
| | |
|
| |
|
|
|
|
| |
unpickled Articles.
getArticle(): Do just that.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
replace open calls with explicit open_ex
add load method with code from __init__
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
appearing in index. pipermail generated several indexes by assuming
that date was unique. If two messages arrived with, e.g., the same
author and date, then the author index treated them as identical.
As a result, both messages were archived, but only the last one was
included in the index. Solution is to always include the msgid, which
is unique, in the index key.
Change database keys to combine elements using tuples instead of
string concatenation with \000 as separator.
Fix was accomplished by refactoring on pipermail.Database and its
subclasses. Push index-key generation into common concrete base class
Database; rename abstract base class to DatabaseInterface. Break up
addArticle method into several pieces.
TBD There is still more refactoring to do on Database class.
Because date key has changed, HyperDatabase method to return first and
last date changed to reflect format of date key.
Refactor pipermail.T.add_article into several pieces.
|
| |
|
|
|
|
|
|
|
|
|
| |
to be more robust so that if either fail, we end up with an empty
self.dict and self.sorted.
Note that the archiver subprocess will still fail with an exception.
Fixing this will require much more work on the archiver as a whole,
and isn't worth it right now. But this fix averts the problem when
regenerating the archive from scratch using bin/arch, so at least
corrupt archives can be rebuilt.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
should considerably help the performance of the archiver.
Specifically:
class DumbBTree: Don't sort the self.sorted list unless some client is
actually traversing the data structure. This saves a lot of work when
items are added. See also Jeremy's XXX comment for further
optimization ideas.
class HyperDatabase: Jeremy also has questions about the usefulness of
the cache used here. Because the items are traversed in linear order,
there isn't much locality of reference, so cache eviction doesn't buy
you much (it's actually more expensive than just keeping everything in
the cache, so that's what we do). That's a space for time trade-off
that might need a re-evaluation.
Clearly, more work could be done to improve the performance of the
archiver, but this should improve matters significantly. Caveat: this
has been only minimally tested in a production environment.
I call this the Hylton Band-aid.
|
| |
|
|
|
|
|
|
|
|
| |
This isn't part of the bsddb.btree interface assumed by Pipermail, but
it's only used in one place and /dramatically/ improves Mailman's
performance.
HyperDatabase.clearIndex(): Use DumbBTree.clear(). These changes may
not fix all the performance problems with Mailman, but certainly nails
the most serious problem I've been experiencing.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Import open_ex() from Mailman.Utils and assign it to open in the
module's globals, so this gets picked up before builtin open. Saves
rewriting lots of occurances of open(), but could be confusing when
reading a method. Hmmm...
__openIndices(): If the `database' directory does not exist, create it
specifically with mode = 02770. We want o-rx so this directory is not
accessible when the archive is public. But we must have g+wxs or the
competing processes (mail and web) that try to update these files
(when a message is posted, or web approved after being held), can get
to and write the files.
|
| |
|
|
|
|
|
|
|
| |
import Mailman.<module>
<module> = Mailman.<module>
to
from Mailman import <module>
I didn't realize you could do this, thanx for letting me know, Barry
scott
|
|
|
package.
Details:
changed Makefile to add the sub package as a directory to make
recursively in
changed configure to make the replacements to
Mailman/Archiver/Makefile.in
changed Mailman/Makefile.in to add Archiver as a sub package
moved Mailman/Archiver.py to Mailman/Archiver/Archiver.py and
Mailman/Hyper* to Mailman/Archiver
Mailman/pipermail.py to Mailman/Archiver/
created Mailman/Archiver/__init__.py to do a "from Archiver import *"
in order to make it's interface identical to previously.
import change: changed imports to import Mailman.<module> from import
<module> in order to accomodate the package import semantics, also
localized <module> by calling <module> = Mailman.<module>. see diffs
for details if this sounds confusing, it's not. this change was
applied to all of the moved files except pipermail, which didn't need
it.
did a basic new installation test to make sure all the
Makefile/configure changes took place atleast basically correctly.
scott
|
