| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| ... | |
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
confident the Elixir branch is ready to become mainline. Also, fewer branches
makes for an easier migration to a dvcs.
Don't expect much of the old test suite to work, or even for much of the old
functionality to work. The changes here are disruptive enough to break higher
level parts of Mailman. But that's okay because I am slowly building up a new
and improved test suite, which will lead to a functional system again.
For now, only the doctests in Mailman/docs (and their related test harnesses)
will pass, but they all do pass. Note that Mailman/docs serve as system
documentation first and unit tests second. You should be able to read the
doctest files to understand the underlying data model.
Other changes included in this merge:
- Added the Mailman.ext extension package.
- zope.interfaces uses to describe major components
- SQLAlchemy/Elixir used as the database model
- Top level doinstall target renamed to justinstall
- 3rd-party packages are now installed in pythonlib/lib/python to be more
compliant with distutils standards. This allows us to use just --home
instead of all the --install-* options.
- No longer need to include the email package or pysqlite, as Python 2.5 is
required (and comes with both packages).
- munepy package is included, for Python enums
- IRosterSets are added as a way to manage a collection of IRosters. Roster
sets are named so that we can maintain the indirection between mailing lists
and rosters, where the two are maintained in different storages.
- IMailingListRosters: remove_*_roster() -> delete_*_roster()
- Remove IMember interface.
- Utils.list_names() -> config.list_manager.names
- fqdn_listname() takes an optional hostname argument.
- Added a bunch of new exceptions used throughout the new interfaces.
- Make LockFile a context manager for use with the 'with' statement.
|
| |
|
|
|
|
|
| |
Mark Sapiro's patch for 'format' parameter. (Decorate.py, Scrubber.py)
Scrubber.py: More brush up of code ... 'Content-Transfer-Encoding' is not
updated by msg.set_payload(). 'Url:' to 'URL:' normalization.
test_handlers.py: Test codes for Decorate.py and Scrubber.py.
|
| |
|
|
|
|
|
|
|
| |
Subject munging code made unified for both i18n and ascii encodings.
test_handlers.py
Adding test code for i18n and numbering subject munging.
(also tab normalization for the last commit)
|
| |
|
|
|
|
|
|
| |
nor StringIO have 'encoding'. What we need was unicode string StringIO and
python StringIO has it already. Also, use utf-8 for the table of contents
in MIME digest if the subject has a different charset.
Adding a test code for the multi-language digest.
|
| |
|
|
|
|
| |
to encode by output_charset (language == 'ja' and cset == 'euc-jp').
Add a test code for list language is 'ja' and message is 'iso-2022-jp';
('Japanese' in header/footer, 'French' in message).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Switchboard.py
- Use listname.encode('utf-8') to produce the necessary 8-bit string,
instead of str(listname). Also update the preceding comment.
senddigests.py
- Remove an unnecessary import.
Decorate.py
- Remove a commented out section of code.
- Remove some redundant local variables
- Reorganize the section that's trying to find a usable encoding for the
payload of the modified message. I don't think it really hurts much to
try duplicate charsets when lcset == mcset, or when either == utf-8.
Just go ahead and try them and let them fail. This simplifies the code.
Also, try to get just the minimum necessary code under the
UnicodeError. I think it's enough to catch the payload.encode() call.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- In i18n.py, change this method so that everything it returns will be
guaranteed to be a unicode. Mailman 2.2 will be unicode-safe, meaning all
strings internally will be unicodes. The translation service is one
boundary point were strings come from the outside, so ensure that they are
unicodes and convert if necessary. This may break some things, but it's
better to fix those situations than to continue to return 8-bit strings from
_().
- In Mailman/testing/base.py, craft a fake module called Mailman.MTA.stub and
stick no-op functions on stub.create() and stub.remove(). We really don't
need the MTA modules for testing purposes (yet at least), and if you're
using the default configuration, you'll get tons of cruft on stdout when the
Manual MTA tries to add and remove mailing lists.
Set up the test configuration environment to use this stub MTA module.
- In test_handlers.py, remove an extraneous str().
- Convert ToDigest.py, Hold.py and Acknowledge.py to __i18n_templates__. (I'm
pretty darn close to just making everything use $-strings by default.)
- In CookHeaders.py, there's no need to unicode()-ify the subject since that
should already be a unicode when passed from _().
- In MailList.py, we can use the str.capitalize() method.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Mailman/Queue/Switchboard.py:
listname is returned in unicode.
( '\x80' + 'a' is OK, '\x80' + u'a' is NG)
Mailman/Utils.py:
Utils.oneline() is extended for returning unicode string.
Mailman/Digester.py:
next_post_number is not used anywhere.
Mailman/database/listdata.py:
Attributes added (esp. for non web u/i)
Mailman/bin/senddigests.py:
Initialization
Mailman/Handlers/ToDigest.py:
Internal string calculation is done in unicode. So, several fixes.
StringIO is used because cStringIO doesn't have encoding attribute.
|
| |
|
|
|
|
|
|
| |
- As the default type of string in mailman-2.2 was set to 'unicode',
i18n codes became need to be fixed.
- Fixed: admin web interface. Other web interfaces needs more verification.
- Fixed: non-digest delivery. Stil to go: digest and archive.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
during early initialization so that we're guaranteed to get the right value
regardless of the shell umask used to invoke the command line script. While
we're at it, we can remove almost all individual umask settings previously in
the code, and make file permissions consistently -rw-rw---- (IOW, files are no
longer other readable).
The only subsystem that wasn't changed was the archiver, because it uses its
own umask settings to ensure that private archives have the proper
permissions. Eventually we'll mess with this, but if it ain't broken...
Note that check_perms complains about directory permissions, but I think
check_perms can be fixed (or perhaps, even removed?!). If we decide to use
LMTPRunner and HTTPRunner exclusively then no outside process will be touching
our files potentially with the incorrect permissions, umask, owner, or group.
If we control all of our own touch points then I think we can lock out
'other'.
Another open question is whether Utils.set_global_password() can have its
umask setting removed. It locks permissions down so even the group can't
write to the site password file, but the default umask of 007 might be good
enough even for this file.
Utils.makedirs() now takes an optional mode argument, which defaults to 02775
for backward compatibility. First, the default mode can probably be changed
to 02770 (see above). Second, all code that was tweaking the umask in order
to do a platform compatible os.mkdir() has now been refactored to use
Utils.makedirs().
Another tricky thing was getting SQLite via SQLAlchemy to create its
data/mailman.db file with the proper permissions. From the comment in
dbcontext.py:
# XXX By design of SQLite, database file creation does not honor
# umask. See their ticket #1193:
# http://www.sqlite.org/cvstrac/tktview?tn=1193,31
More details in that file, but the work around is to essentially 'touch' the
database file if 'sqlite' is the scheme of the SQLAlchemy URL. This little
pre-touch sets the right umask honoring permission and won't hurt if the file
already exists. SQLite will happily keep the existing permissions, and in
fact that ticket referenced above recommends doing things this way.
In the Mailman.database.initialize(), create a global lock that prevents more
than one process from entering this init function at the same time. It's
probably not strictly necessary given that I believe all the operations in
dbcontext.connect() are multi-processing safe, but it also doesn't seem to
hurt and prevents race conditions regardless of the database's own
safeguards (or lack thereof).
Make sure nightly_gzip.py calls initialize().
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
compatible back to Python 2.3, this change should not get back ported to
Mailman 2.1.
Port to Python 2.5. The non-test suite changes should get back ported to
Mailman 2.1 (which I will do next), but don't worry about the test suite ones
because MM2.1's test suite is hopeless. Specifically:
- In SecurityManager.py, fix the parsecookie() code to work with Python 2.5
generated cookie text. The latter was changed to be more RFC compliant so
it does not output training semicolons for each line of cookie text. This
broke the splitting rules, so now first split on newlines, then on ';\s*'.
This should work across all Python versions.
- In Python 2.5, exceptions are new-style, and thus are no longer of
ClassType. The instantiation type test in hold_for_approval() was too
naive.
- Raising strings generates deprecation warnings in Python 2.5. Switch the
one weird use of this in Utils.py to use a class exception. Don't call it
"quick exit" though because it's probably not.
- In the tests, use True/False instead of 1/0
- Use failUnless/failIf instead of assertEqual against True/False.
- In the tests, use Message.get_content_type() instead of Message.get_type()
since the latter is gone in email 4.0.1. Same with get_content_maintype()
and get_main_type().
|
| |
|
|
|
|
|
|
|
|
| |
see the traceback.
Make sure that in EmailBase.tearDown() we actually wait for the SinkServer to
exit before continuing.
We still leave test turds in archives/private, but I haven't yet figured it
out.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
this change. More unit tests should be added.
misc/sitelist.cfg is removed -- this is an ex-site list.
MailList.GetNoReplyEmail() -> MailList.no_reply_address (property)
UserNotification._enqueue(), OwnerNotification._enqueue(): when queing the
message to the virgin queue, be sure to use the fully qualified (i.e. posting)
address for the list.
In the MTA modules, be sure to set up the target of the mail commands as the
fqdn listname because otherwise we can't find the correct list. This needs
some tweaking/testing for Postfix's virtual domain support.
MailList.Load() has to grow an optional argument specifying the fqdn
listname. The problem is that in some situations, we can't calculate that
because we don't know _internal_name, so it has to be passed in. This is
mostly the case in the MailList ctor where a Load hasn't happened yet. For
backward compatibility though, if it's not passed in, just use
mlist.fqdn_listname.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
name in more than one domain.
- Totally eradicate MAILMAN_SITE_LIST, and in fact the entire need for a site
list. The functions that the site list previously performed are either
removed or supported in other ways. For example, instead of forwarding
owner bounces to the site list, we now have a SITE_OWNER_ADDRESS which
should point to a human, and such bounces are sent there instead. There's
also a "no reply" email address that should be set up to go to devnull. For
any message that never expects a reply, the sender is set to this address.
- Remove the Site.py module. It was an experimental approach to trying to
support virtual domains, and we're going to do it so much better now that
this module is no longer necessary. Site._makedirs() -> Utils.makedir().
- VIRTUAL_HOST_OVERVIEW is completely removed, since now virtual hosts are
always enabled. Virtual domains should be added to mailman.cfg by using the
new add_domain() function. add_virtualhost() is gone. If no virtual
domains are added explicitly, we add the default one that configure guessed
(but we never add that if domains are added explicitly).
- Utils.get_domain() -> Utils.get_request_domain()
- withlist code cleanup and make sure that we load etc/mailman.cfg
- A new base exception called MailmanException is added, from which all
exceptions defined in Errors.py ultimately derive. MailmanError is retained
and derives from MailmanException.
- BadDomainSpecificationError is added.
- Remove the -V/--virtual-host-overview option from list_lists and add instead
-d/--domain and -f/--full.
- bin/update probably works but needs more testing.
- bin/newlist and bin/rmlist take fqdn list names, but default to the default
domain if @whatever isn't given. newlist's -u/--urlhost and -e/--emailhost
options are removed. The domain that the list is being added to must
already exist.
- Minor code cleanup in Message.py
- Bump version to 2.2.0a1
- The Configuration object grows a .domain dictionary which maps email hosts
to url hosts. The reverse mapping is supported, but not directly; use
Configuration.get_email_host() instead.
- Mailman/Cgi/create is converted from mm_cfg to config, and some minor code
cleanup is performed. Also, convert to __i18n_templates__ = True.
- New MailList APIs:
+ property .fqdn_listname
+ GetNoReplyEmail()
+ Create() API changes and refactoring.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
and fixing the invocation and shutdown of mailmanctl. While the tests in this
module work individually, they do not yet work as a group.
-C added to testall.py, and mailmanctl now passes that flag on to qrunner.
UserNotification sets reduced_list_header in the msgdata, but the behavior of
this flag has changed. It used to suppress List-Help, List-Subscribe, and
List-Unsubscribe as well as List-Post and List-Archive. However, List-Help,
List-Subscribe and List-Unsubscribe should definitely be included in
UserNotifications, and List-Post has a different variable controlling it now.
Therefore, always add List-Help, List-Subscribe, and List-Unsubscribe.
Some style updates to Message.py
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
configuration file. While the full conversion is not yet complete, everything
that seems to be required to run mailmanctl, qrunner, rmlist, and newlist have
been updated.
Basically, modules should no longer import mm_cfg, but instead they should
import Mailman.configuration.config. The latter is an object that's
guaranteed to exist, but not guaranteed to be initialized until some top-level
script calls config.load(). The latter should be called with the argument to
-C/--config which is a new convention the above scripts have been given.
In most cases, where mm_cfg.<variable> is used config.<variable> can be used,
but the exceptions are where the default value must be available before
config.load() is called. Sometimes you can import Mailman.Default and get the
variable from there, but other times the code has to be changed to work around
this limitation. Take each on a case-by-case basis.
Note that the various directories calculated from VAR_PREFIX, EXEC_PREFIX, and
PREFIX are now calculated in config.py, not in Defaults.py. This way a
configuration file can override the base directories and everything should
work correctly.
Other changes here include:
- mailmanctl, qrunner, and update are switched to optparse and $-strings, and
changed to the mmshell architecture
- An etc directory has been added to /usr/local/mailman and a
mailman.cfg.sample file is installed there. Sites should now edit an
etc/mailman.cfg file to do their configurations, although the mm_cfg file is
still honored. The formats of the two files are identical.
- list_lists is given the -C/--config option
- Some coding style fixes in bin/update, but not extensive
- Get rid of nested scope hacks in qrunner.py
- A start on getting EmailBase tests working (specifically test_message),
although not yet complete.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
bogus apparent messages into the error log, possibly inducing an admin to
visit a phishing site.
- options.py
Topics.py
Tagger.py
MailList.py
Utils.py
Version.py
versions.py
The processing of Topics regular expressions has changed. Previously the
Topics regexp was compiled in verbose mode but not documented as such
which caused some confusion. Also, the documentation indicated that topic
keywords could be entered one per line, but these entries were not
properly. Topics regexps are now compiled in non-verbose mode and multi-
line entries are 'ored'. Existing Topics regexps will be converted when
the list is updated so they will continue to work.
|
| |
|
|
| |
outside the character set of the list's language.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Mailman
top level package. Rewrote the test runner and stuck it in bin (as an
mmshell
symlink). bin/testall now autodetects tests and allows running a
subset of
tests via regular expression filtering.
I also fixed all the tests so they all pass now, with the exception of
test_message.py tests. These still doesn't work because of coordination
issues between its smtpd-based reader and SMTPDirect, but... I have a
plan
(mwah, ha, ha! :)
I also fixed a code update bug in Decorate.py
|
| |
|
|
| |
- Handlers/CalcRecips.py - Changed to not process topics if topics
are disabled for the list.
|
| |
|
|
|
|
|
|
|
|
|
| |
- Remove the LogStdErr() calls from all the scripts/* scripts.
- Convert the LockFile.py code to use the logging logs. Also do general code
cleanup in this module. Note that the 'withlogging' argument to __init__()
has been removed, as we can use logging's mechanisms to control how much of
LockFile will get logged (by default, not much).
- Add the 'mailman.locks' logger to loginit.py
- Remove Mailman/Handlers/Sendmail.py once and for all
- Remove LIST_LOCK_DEBUGGING from Defaults.py.in
|
| |
|
| |
message that Mailman adds to the admin notification.
|
| |
|
|
|
|
| |
the builtin types. Two still remain: a check against ClassType and a check
against MethodType. Also, fix some hinky type comparisons to use isinstance()
consistently.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
traces of our crufty old Syslog. Most of this work was purely mechanical,
except for:
1) Initializing the loggers. For this, there's a new module
Mailman/loginit.py (yes all modules from now on will use PEP 8
names). We can't call this 'logging.py' because that will
interfere with importing the stdlib module of the same name (can
you say Python 2.5 and absolute imports?).
If you want to write log messages both to the log file and to
stderr, pass True to loginit.initialize(). This will turn on
propagation of log messages to the parent 'mailman' logger, which
is set up to print to stderr. This is how bin/qrunner works when
not running as a subprocess of mailmanctl.
2) The driver script. I had to untwist the StampedLogger stuff and
implement differently printing exceptions and such to log/error
because standard logging objects don't have a write() method. So
we write to a cStringIO and then pass that to the logger.
3) SMTPDirect.py because of the configurability of the log messages.
This required changing SafeDict into a dict subclass (which is
better than using UserDicts anyway -- yay Python 2.3!). It's
probably still possible to flummox things up if you change the
name of the loggers in the SMTP_LOG_* variables in mm_cfg.py.
However, the worst you can do is cause output to go to stderr and
not go to a log file.
Note too that all entry points into the Mailman system must call
Mailman.loginit.initialize() or the log output will go to stderr
(which may occasionally be what you want). Currently all CGIs and
qrunners should be working properly.
I wish I could have tested all code paths that touch the logger, but
that's infeasible. I have tested this, but it's possible that there
were some mistakes in the translation.
- Mailman.Bouncers.BounceAPI.Stop is a singleton, but not a class
instance any more.
- True/False code cleanup, PEP 8 import restructuring, whitespace
normalization, and copyright year updates, as appropriate.
|
| |
|
|
|
|
|
|
|
|
|
| |
- Remove True/False binding cruft
- Remove __future__ statements for nested scopes
- Remove ascii_letters import hack from Utils.py
- Remove mimetypes.guess_all_extensions import hack from Scrubber.py
- In Pending.py, set _missing to object() (better than using [])
Also, update copyright years where appropriate, and re-order imports more to
my PEP 8 tastes. Whitespace normalize.
|
| |
|
|
|
| |
the trunk. This serves also to test write permission to the svn
repository, and tests email notifications.
|
| | |
|
| |
|
|
| |
when payload is None.
|
| |
|
|
|
| |
for multiple charset header for programming convenience and readablity.
(We may better use unicode for header munging eventually.)
|
| | |
|
| |
|
|
| |
because it is overwrapped in Mailman.Message.
|
| | |
|
| |
|
|
| |
new handler to be included in both GLOBAL_PIPELINE and OWNER_PIPELINE.
|
| | |
|
| | |
|
| |
|
|
| |
The keyword 'reduced_list_headers' is taken from CookHeaders.py.
|
| | |
|
| | |
|
| |
|
|
|
|
| |
Subject manipulation should be done in unicode string mode.
(temporary change in MAIN because higher version of python should allow
simpler algorithms.)
|
| | |
|
| | |
|
| |
|
|
| |
users of MS Outlook and Apple Mail.
|
| | |
|
| |
|
|
| |
to mm_cfg.HTML_TO_PLAIN_TEXT_COMMAND
|
| | |
|
| | |
|
| | |
|
