summaryrefslogtreecommitdiff
path: root/Mailman/Handlers/ToDigest.py
Commit message (Collapse)AuthorAgeFilesLines
* Reorganize doctests, moving handler documentation into Mailman/handlers/docs.Barry Warsaw2008-01-241-420/+0
| | | | Rename all handlers to be more PEP 8 friendly, i.e. lowercased.
* Reorganize the enums so that they live in the most appropriate interface. TheBarry Warsaw2007-10-091-1/+1
| | | | only constant left in Mailman.constants is now actually a constant.
* Reorganize the qrunner infrastructure. First, the package has been renamedBarry Warsaw2007-09-291-2/+2
| | | | | | | | | | | | | | | | | | | | | | from Mailman.Queue to Mailman.queue (note the case change to be more PEP 8 compliant). The Switchboard and Runner classes have been moved into the package __init__.py and the previous class modules have been removed. The switchboard cache is removed; I don't think it was ultimately buying us much. Now, just import the Switchboard class and instantiate it directly. Added an IRunner interface. Renamed the ArchRunner to ArchiveRunner. bin/qrunner and bin/mailmanctl are updated accordingly. For the former, it no long accepts -r=All to run all qrunners. You can still use the short name (e.g. --runner=incoming) to run the built-in queue runners, but this design will eventually allow for plugin qrunners by allowing them to be run specifying the full package path to the class. It also now accepts a leading dot to indicate a qrunner class relative to the Mailman.queue package.
* Removed a bunch of files that are obsolete. The interfaces are all foldedBarry Warsaw2007-09-271-1/+0
| | | | | | | | | | | | | into the IMailingList interface. OTOH, MemberAdaptor.py is completely useless now (though not entirely eradicated), as is OldStyleMemberships.py. versions.py isn't necessary any longer either because we'll have to do database migrations (and conversions from MM2.1) completely differently. New command line script 'set_members' which is used to take a CSV file and syncing that to a list's membership. Added back the DeliveryStatus.unknown item because we'll need it when we migrate MM 2.1 databases.
* OMGW00T: After over a decade, the MailList mixin class is gone! Well,Barry Warsaw2007-09-211-1/+1
| | | | | | | | | | | | mostly. It's no longer needed by anything in the test suite, and therefore the list manager returns database MailingList objects directly. The wrapper cruft has been removed. To accomplish this, a couple of hacks were added to the Mailman.app package, which will get cleaned up over time. The MailList module itself (and its few remaining mixins) aren't yet removed from the tree because some of the code is still not tested, and I want to leave this code around until I've finished converting it.
* Implement a context manager for Python 2.5's with statement, which isBarry Warsaw2007-09-191-5/+3
| | | | | | | | used where we used to do a try/except to temporarily change the global translation language. This makes the code shorter and cleaner. E.g. with i18n.using_language(another_language): # do something
* Move the pending database into the SQLAlchemy/Elixir layer. The oldBarry Warsaw2007-08-011-10/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Pending.py module is removed. Added an interface to this functionality such that any IPendable (essentially a key/value mapping) can be associated with a token, and that token can be confirmed and has a lifetime. Any keys and values can be stored, as long as both are unicodes. Added a doctest. Modified initialization of the database layer to support pluggability via setuptools. No longer is this layer initialized from a module, but now it's instantiated from a class that implements IDatabase. The StockDatabase class implements the SQLAchemy/Elixir layer, but this can be overridden in a setup.py. Bye bye MANAGERS_INIT_FUNCTION, we hardly knew ye. Added a package Mailman.app which will contain certain application specific functionality. Right now, the only there there is an IRegistar implementation, which didn't seem to fit anywhere else. Speaking of which, the IRegistrar interface implements all the logic related to registration and verification of email addresses. Think the equivalent of MailList.AddMember() except generalized out of a mailing list context. This latter will eventually go away. The IRegistrar sends the confirmation email. Added an IDomain interface, though the only implementation of this so far lives in the registration.txt doctest. This defines the context necessary for domain-level things, like address confirmation. A bunch of other cleanups in modules that are necessary due to the refactoring of Pending, but don't affect anything that's actually tested yet, so I won't vouch for them (except that they don't throw errors on import!). Clean up Defaults.py; also turn the functions seconds(), minutes(), hours() and days() into their datetime.timedelta equivalents. Consolidated the bogus email address exceptions. In some places where appropriate, use email 4.0 module names instead of the older brand. Switch from Mailman.Utils.unique_message_id() to email.utils.make_msgid() everywhere. This is because we need to allow sending not in the context of a mailing list (i.e. domain-wide address confirmation message). So we can't use a Message-ID generator that requires a mailing list. OTOH, this breaks Message-ID collision detection in the mail->news gateway. I'll fix that eventually. Remove the 'verified' row on the Address table. Now verification is checked by Address.verified_on not being None.
* Conversion of the ToDigest tests to doctests, along with just enough handlerBarry Warsaw2007-07-101-24/+36
| | | | | | | | | module update to make the thing pass. Much more coverage is still necessary, but this at least recreates the existing tests. Changes to the test infrastructure to make REPORT_ONLY_FIRST_FAILURE conditional on lower (default) verbosity. Increase the verbosity via 'bin/testall -v' and you get all the failure output.
* Brush up the fall-back-to-utf8 mechanizm in ToDigest.py. Neither cStringIOtkikuchi2007-03-241-16/+17
| | | | | | | | nor StringIO have 'encoding'. What we need was unicode string StringIO and python StringIO has it already. Also, use utf-8 for the table of contents in MIME digest if the subject has a different charset. Adding a test code for the multi-language digest.
* Test suite repair. All tests are now passing again.bwarsaw2007-03-211-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | - In i18n.py, change this method so that everything it returns will be guaranteed to be a unicode. Mailman 2.2 will be unicode-safe, meaning all strings internally will be unicodes. The translation service is one boundary point were strings come from the outside, so ensure that they are unicodes and convert if necessary. This may break some things, but it's better to fix those situations than to continue to return 8-bit strings from _(). - In Mailman/testing/base.py, craft a fake module called Mailman.MTA.stub and stick no-op functions on stub.create() and stub.remove(). We really don't need the MTA modules for testing purposes (yet at least), and if you're using the default configuration, you'll get tons of cruft on stdout when the Manual MTA tries to add and remove mailing lists. Set up the test configuration environment to use this stub MTA module. - In test_handlers.py, remove an extraneous str(). - Convert ToDigest.py, Hold.py and Acknowledge.py to __i18n_templates__. (I'm pretty darn close to just making everything use $-strings by default.) - In CookHeaders.py, there's no need to unicode()-ify the subject since that should already be a unicode when passed from _(). - In MailList.py, we can use the str.capitalize() method.
* Fixes for i18n digest to work.tkikuchi2007-03-021-9/+14
| | | | | | | | | | | | | | | | | | Mailman/Queue/Switchboard.py: listname is returned in unicode. ( '\x80' + 'a' is OK, '\x80' + u'a' is NG) Mailman/Utils.py: Utils.oneline() is extended for returning unicode string. Mailman/Digester.py: next_post_number is not used anywhere. Mailman/database/listdata.py: Attributes added (esp. for non web u/i) Mailman/bin/senddigests.py: Initialization Mailman/Handlers/ToDigest.py: Internal string calculation is done in unicode. So, several fixes. StringIO is used because cStringIO doesn't have encoding attribute.
* Clean up file permissions and umask settings. Now we set the umask to 007bwarsaw2007-01-051-6/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | during early initialization so that we're guaranteed to get the right value regardless of the shell umask used to invoke the command line script. While we're at it, we can remove almost all individual umask settings previously in the code, and make file permissions consistently -rw-rw---- (IOW, files are no longer other readable). The only subsystem that wasn't changed was the archiver, because it uses its own umask settings to ensure that private archives have the proper permissions. Eventually we'll mess with this, but if it ain't broken... Note that check_perms complains about directory permissions, but I think check_perms can be fixed (or perhaps, even removed?!). If we decide to use LMTPRunner and HTTPRunner exclusively then no outside process will be touching our files potentially with the incorrect permissions, umask, owner, or group. If we control all of our own touch points then I think we can lock out 'other'. Another open question is whether Utils.set_global_password() can have its umask setting removed. It locks permissions down so even the group can't write to the site password file, but the default umask of 007 might be good enough even for this file. Utils.makedirs() now takes an optional mode argument, which defaults to 02775 for backward compatibility. First, the default mode can probably be changed to 02770 (see above). Second, all code that was tweaking the umask in order to do a platform compatible os.mkdir() has now been refactored to use Utils.makedirs(). Another tricky thing was getting SQLite via SQLAlchemy to create its data/mailman.db file with the proper permissions. From the comment in dbcontext.py: # XXX By design of SQLite, database file creation does not honor # umask. See their ticket #1193: # http://www.sqlite.org/cvstrac/tktview?tn=1193,31 More details in that file, but the work around is to essentially 'touch' the database file if 'sqlite' is the scheme of the SQLAlchemy URL. This little pre-touch sets the right umask honoring permission and won't hurt if the file already exists. SQLite will happily keep the existing permissions, and in fact that ticket referenced above recommends doing things this way. In the Mailman.database.initialize(), create a global lock that prevents more than one process from entering this init function at the same time. It's probably not strictly necessary given that I believe all the operations in dbcontext.connect() are multi-processing safe, but it also doesn't seem to hurt and prevents race conditions regardless of the database's own safeguards (or lack thereof). Make sure nightly_gzip.py calls initialize().
* Fix some buglets with virtual domain support and repair unit tests broken bybwarsaw2006-07-081-2/+2
| | | | | | | | | | | | | | | | | | | | | | | this change. More unit tests should be added. misc/sitelist.cfg is removed -- this is an ex-site list. MailList.GetNoReplyEmail() -> MailList.no_reply_address (property) UserNotification._enqueue(), OwnerNotification._enqueue(): when queing the message to the virgin queue, be sure to use the fully qualified (i.e. posting) address for the list. In the MTA modules, be sure to set up the target of the mail commands as the fqdn listname because otherwise we can't find the correct list. This needs some tweaking/testing for Postfix's virtual domain support. MailList.Load() has to grow an optional argument specifying the fqdn listname. The problem is that in some situations, we can't calculate that because we don't know _internal_name, so it has to be passed in. This is mostly the case in the MailList ctor where a Load hasn't happened yet. For backward compatibility though, if it's not passed in, just use mlist.fqdn_listname.
* Fix test_message.py by finishing the wind-through of the configuration objectbwarsaw2006-07-081-7/+7
| | | | | | | | | | | | | | | | and fixing the invocation and shutdown of mailmanctl. While the tests in this module work individually, they do not yet work as a group. -C added to testall.py, and mailmanctl now passes that flag on to qrunner. UserNotification sets reduced_list_header in the msgdata, but the behavior of this flag has changed. It used to suppress List-Help, List-Subscribe, and List-Unsubscribe as well as List-Post and List-Archive. However, List-Help, List-Subscribe and List-Unsubscribe should definitely be included in UserNotifications, and List-Post has a different variable controlling it now. Therefore, always add List-Help, List-Subscribe, and List-Unsubscribe. Some style updates to Message.py
* Remove most uses of the types module, in favor of isinstance checks againstbwarsaw2006-04-171-2/+1
| | | | | | the builtin types. Two still remain: a check against ClassType and a check against MethodType. Also, fix some hinky type comparisons to use isinstance() consistently.
* - Convert all logging to Python's standard logging module. Get rid of allbwarsaw2006-04-171-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | traces of our crufty old Syslog. Most of this work was purely mechanical, except for: 1) Initializing the loggers. For this, there's a new module Mailman/loginit.py (yes all modules from now on will use PEP 8 names). We can't call this 'logging.py' because that will interfere with importing the stdlib module of the same name (can you say Python 2.5 and absolute imports?). If you want to write log messages both to the log file and to stderr, pass True to loginit.initialize(). This will turn on propagation of log messages to the parent 'mailman' logger, which is set up to print to stderr. This is how bin/qrunner works when not running as a subprocess of mailmanctl. 2) The driver script. I had to untwist the StampedLogger stuff and implement differently printing exceptions and such to log/error because standard logging objects don't have a write() method. So we write to a cStringIO and then pass that to the logger. 3) SMTPDirect.py because of the configurability of the log messages. This required changing SafeDict into a dict subclass (which is better than using UserDicts anyway -- yay Python 2.3!). It's probably still possible to flummox things up if you change the name of the loggers in the SMTP_LOG_* variables in mm_cfg.py. However, the worst you can do is cause output to go to stderr and not go to a log file. Note too that all entry points into the Mailman system must call Mailman.loginit.initialize() or the log output will go to stderr (which may occasionally be what you want). Currently all CGIs and qrunners should be working properly. I wish I could have tested all code paths that touch the logger, but that's infeasible. I have tested this, but it's possible that there were some mistakes in the translation. - Mailman.Bouncers.BounceAPI.Stop is a singleton, but not a class instance any more. - True/False code cleanup, PEP 8 import restructuring, whitespace normalization, and copyright year updates, as appropriate.
* Now that Python 2.3 is the minimum requirement for Mailman 2.2:bwarsaw2006-04-151-19/+13
| | | | | | | | | | | - Remove True/False binding cruft - Remove __future__ statements for nested scopes - Remove ascii_letters import hack from Utils.py - Remove mimetypes.guess_all_extensions import hack from Scrubber.py - In Pending.py, set _missing to object() (better than using []) Also, update copyright years where appropriate, and re-order imports more to my PEP 8 tastes. Whitespace normalize.
* Back out Revision 2.30 patch for email.Message.set_payload() bugtkikuchi2006-01-291-12/+5
| | | | because it is overwrapped in Mailman.Message.
* Port cleaning changes forward from 2.1-maint branch.bwarsaw2005-12-301-12/+10
|
* Fixes for email.set_payload() not distinguish parsed or virgin payload.tkikuchi2005-12-241-2/+11
|
* SF patch # 1246003tkikuchi2005-11-191-1/+1
|
* back porting from 2.1.6tkikuchi2005-08-281-24/+47
|
* FSF office has moved. chdcking in for MAIN branch.tkikuchi2005-08-271-1/+1
|
* process(): Use Mailman's Mailbox object to append new messages to thebwarsaw2003-08-151-2/+3
| | | | | | digest.mbox file. This is the same technique that the archiver uses, and should fix Chuq's problem with missing trailing newline separators in digest messages.
* send_i18n_digests(): Patch # 694912 by Tokio Kikuchi to fix thebwarsaw2003-04-111-2/+4
| | | | scrubbing of attachments in MIME digest messages.
* True/False where appropriate.bwarsaw2003-04-061-15/+24
|
* oneline(): Whoops! Typo fix.bwarsaw2003-02-071-1/+1
|
* Tokio Kikuchi's SF patch #674401 to fix some issues with i18n in thebwarsaw2003-02-071-15/+20
| | | | | | | | | | | | digests. Specifically, lheader() -> oneline(): Also, get rid of **kws since we're not going to use these. Catch any LookupError or UnicodeError that may occur, but do the .encode() with 'replace' for a better chance to succeed. send_i18n_digests(): Use Utils.wrap() to wrap the subject headers, and also in the kept headers in plain text digests. Eliminate the blank line between subjects in the ToC.
* I18n fixes for the digester, based on Tokio Kikuchi's patch #668819.bwarsaw2003-01-241-43/+65
| | | | | | | | | | | | | | | | | | | | | | | | | I've modified it though, so any brokenness is my fault. Specifically, KEEP: Removed in favor of Default.py.in's MIME_DIGEST_KEEP_HEADERS. process(): Use Generator.flatten() instead of the deprecated __call__ syntax. send_i18n_digest(): Make sure things like the Subject headers in the toc are coerced to the language the digest is being sent in (i.e. the list's preferred language). Make sure the masthead, header and footer attachments also get the right matching charset attribute. Make sure that the username extracted from the From header is also in the right character set. Use the Header object for proper wrapping of the Subject lines in the toc, not Utils.wrap() which was broken and not i18n aware. Add a blank line between entries in the toc (might be controversial). Send messages in the plain text digest through the scrubber so attachments (and their MIME goo) don't mess up the plain text digests. lheader(): New convenience function.
* send_digests(): Tokio Kikuchi patch of a bug reported by Chuq thatbwarsaw2002-10-181-5/+5
| | | | | caused a daily ratchet of the volume number. Also, one minor refactoring in the bump digest number conditional.
* send_i18n_digests(): Chuq politely reminds me that in MM2.0, digestsbwarsaw2002-10-071-0/+2
| | | | | had their Reply-To header pointing at the posting address, and that there was no good reason not to continue this tradition <wink>.
* KEEP: In digests, keep Reply-To header also.bwarsaw2002-09-051-1/+1
|
* Some fixes for problems identified by Peer Heinlein. It is possiblebwarsaw2002-08-291-4/+7
| | | | | | | | | | | | | that the digest may contain unparseable messages, e.g. with missing start boundaries. Instead of crashing, such messages should simply be ignored. If lax parsing can't grok it, it has a high likelihood of being spam. Specific changes here: send_i18n_digests(): The end case is now that "msg is None", not false, and if msg is the empty string, it was unparseable and we should skip it.
* send_i18n_digests(): Let's generate MIME digests the correct way, bybwarsaw2002-07-111-5/+9
| | | | | | | | | wrapping each message inside the multipart/digest in a MIMEMessage instance. We also no longer need to initialize the mimedigest instance's payload to the empty list. Also, when sending the digests, be sure to include isdigest=1 in the metadata, otherwise SMTPDirect will try to tack on another footer.
* send_i18n_digests(): Get rid of the add_payload() calls; these arebwarsaw2002-04-101-7/+7
| | | | | | deprecated in email 2.0. For calls on the mimemsg, change them into attach() calls. For calls on the rfc1153msg, change them into set_payload() calls.
* send_i18n_digests(): Be robust in the face of messages with no From:bwarsaw2002-03-151-2/+2
| | | | | | header. It seems like only broken configurations (or bugs?) could cause this since the RFC requires From: to exist, but this is better than throwing an exception.
* send_i18n_digests(): Use getDeliveryStatus().bwarsaw2001-12-191-1/+2
|
* msgfactory(): Get rid of this, now that our own Mailbox class alreadybwarsaw2001-10-151-12/+2
| | | | | | uses a factory that produces our own Message objects. send_i18n_digests(): Use our Mailbox instead of mailbox.UnixMailbox.
* send_i18n_digests(): Dan Mick noticed that the DisableDelivery flagbwarsaw2001-10-121-4/+5
| | | | was being ignored for digest delivery!
* send_i18n_digests(): When calling msg.get_all() for a KEEP header,bwarsaw2001-10-111-1/+1
| | | | | supply an empty list as the failobj so we can still iterate over the return value.
* process(): SF bug # 465477 indicates that using mboxfp.tell() won'tbwarsaw2001-10-011-2/+3
| | | | | | | | | always get the correct file size if the Generator didn't leave the file pointer at the EOF. Let's use os.path.getsize() -- after a flush -- instead. Closes this bug report. send_i18n_digests(): Update to email package API; use msg.get_all() intead of msg.getall().
* Convert from mimelib to email.bwarsaw2001-10-011-12/+12
| | | | | Also, use cStringIO directly instead of our own hack-around StringIO module.
* send_i18n_digests(): Cut-n-paste errors: self->mlist.bwarsaw2001-07-271-2/+2
|
* All membership related attribute access should use the MemberAdaptorbwarsaw2001-07-191-2/+10
| | | | | | | | | | | | API instead, e.g. GetUserOption() -> getMemberOption() GetPreferredLanguage() -> getMemberLanguage() IsMember() -> isMember() GetDigestDeliveryMembers() -> getDigestMemberKeys() ToDigest.py: clear the one_last_digest() dictionary after sending the digest.
* send_i18n_digests(): Because messages are now enqueued as pickles bybwarsaw2001-07-101-7/+3
| | | | | | | | | default, we can't use a StringableMIME class here, otherwise it won't have a get_sender() method when unpickled. So instead, use a Message.Message class and just set Content-Type: and MIME-Version: headers manually. StringableMIME: Removed.
* send_i18n_digests(): In addition to passing in the user's preferredbwarsaw2001-05-181-1/+1
| | | | | language, also pass in the MailList object; maketext() digs the preferred language out of the list object.
* mimelib's ReprMixin class and module was renamed to StringableMixin inbwarsaw2001-05-011-3/+34
| | | | | | | | | | | | | | | the 0.3 release. This means you must download and install the 0.3 version of mimelib to work with the current CVS snapshot! ReprMIME => StringableMIME send_digests(): Added support for auto-bumping of digest volume numbers based on the digest_volume_frequency configuration attribute. send_i18n_digests(): After creating the mimedigest container, set its payload to the empty list. This ensures that subsequent add_payload()s will succeed. mimelib 0.3 will raise an exception otherwise when appending to a scalar payload.
* Big i18n changes. Specifically,bwarsaw2001-03-021-22/+40
| | | | | | | | | | | | | send_digests() -> send_i18n_digests(): this does the bulk of the digest crafting, but it's wrapped in send_digests() which first sets up the language context for the digest (the list's preferred language). send_i18n_digests(): Also make sure that for text/plain parts, the charset is set correctly to the list's preferred language character set. Mark more strings as translatable.
* Conversion to mimelib.bwarsaw2001-02-151-340/+254
| | | | | | | | De-string-module-ify Other Python 2.0 constructs used where appropriate. Get rid of HandlerAPI references -- this module is obsolete.
* Latest set of I18N patches from jcrey. Specifically,bwarsaw2000-12-261-7/+10
| | | | | | process(): set $LANG to list's preferred language and use wherever necessary. Mark more strings as translatable. Don't hardcode Content-Type: charset to "us-ascii".
generated by cgit v1.2.3-70-g09d2 (git 2.51.2) at 2025-11-08 22:20:59 +0000