summaryrefslogtreecommitdiff
path: root/Mailman/Handlers/SMTPDirect.py
Commit message (Collapse)AuthorAgeFilesLines
* Reorganize doctests, moving handler documentation into Mailman/handlers/docs.Barry Warsaw2008-01-241-389/+0
| | | | Rename all handlers to be more PEP 8 friendly, i.e. lowercased.
* Remove the action.py module, move this to Mailman/interfaces/__init__.py.Barry Warsaw2007-10-101-2/+3
| | | | | | Convert IMailingList.personalize to a enum. Change all non-obsolete occurances of GetListEmail() to posting_address.
* Update copyright years.bwarsaw2007-01-191-1/+1
|
* First crack at real virtual domain support, i.e. mailing lists with the samebwarsaw2006-07-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | name in more than one domain. - Totally eradicate MAILMAN_SITE_LIST, and in fact the entire need for a site list. The functions that the site list previously performed are either removed or supported in other ways. For example, instead of forwarding owner bounces to the site list, we now have a SITE_OWNER_ADDRESS which should point to a human, and such bounces are sent there instead. There's also a "no reply" email address that should be set up to go to devnull. For any message that never expects a reply, the sender is set to this address. - Remove the Site.py module. It was an experimental approach to trying to support virtual domains, and we're going to do it so much better now that this module is no longer necessary. Site._makedirs() -> Utils.makedir(). - VIRTUAL_HOST_OVERVIEW is completely removed, since now virtual hosts are always enabled. Virtual domains should be added to mailman.cfg by using the new add_domain() function. add_virtualhost() is gone. If no virtual domains are added explicitly, we add the default one that configure guessed (but we never add that if domains are added explicitly). - Utils.get_domain() -> Utils.get_request_domain() - withlist code cleanup and make sure that we load etc/mailman.cfg - A new base exception called MailmanException is added, from which all exceptions defined in Errors.py ultimately derive. MailmanError is retained and derives from MailmanException. - BadDomainSpecificationError is added. - Remove the -V/--virtual-host-overview option from list_lists and add instead -d/--domain and -f/--full. - bin/update probably works but needs more testing. - bin/newlist and bin/rmlist take fqdn list names, but default to the default domain if @whatever isn't given. newlist's -u/--urlhost and -e/--emailhost options are removed. The domain that the list is being added to must already exist. - Minor code cleanup in Message.py - Bump version to 2.2.0a1 - The Configuration object grows a .domain dictionary which maps email hosts to url hosts. The reverse mapping is supported, but not directly; use Configuration.get_email_host() instead. - Mailman/Cgi/create is converted from mm_cfg to config, and some minor code cleanup is performed. Also, convert to __i18n_templates__ = True. - New MailList APIs: + property .fqdn_listname + GetNoReplyEmail() + Create() API changes and refactoring.
* Massive conversion process so that Mailman can be run from a user specifiedbwarsaw2006-07-081-18/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | configuration file. While the full conversion is not yet complete, everything that seems to be required to run mailmanctl, qrunner, rmlist, and newlist have been updated. Basically, modules should no longer import mm_cfg, but instead they should import Mailman.configuration.config. The latter is an object that's guaranteed to exist, but not guaranteed to be initialized until some top-level script calls config.load(). The latter should be called with the argument to -C/--config which is a new convention the above scripts have been given. In most cases, where mm_cfg.<variable> is used config.<variable> can be used, but the exceptions are where the default value must be available before config.load() is called. Sometimes you can import Mailman.Default and get the variable from there, but other times the code has to be changed to work around this limitation. Take each on a case-by-case basis. Note that the various directories calculated from VAR_PREFIX, EXEC_PREFIX, and PREFIX are now calculated in config.py, not in Defaults.py. This way a configuration file can override the base directories and everything should work correctly. Other changes here include: - mailmanctl, qrunner, and update are switched to optparse and $-strings, and changed to the mmshell architecture - An etc directory has been added to /usr/local/mailman and a mailman.cfg.sample file is installed there. Sites should now edit an etc/mailman.cfg file to do their configurations, although the mm_cfg file is still honored. The formats of the two files are identical. - list_lists is given the -C/--config option - Some coding style fixes in bin/update, but not extensive - Get rid of nested scope hacks in qrunner.py - A start on getting EmailBase tests working (specifically test_message), although not yet complete.
* Remove most uses of the types module, in favor of isinstance checks againstbwarsaw2006-04-171-2/+1
| | | | | | the builtin types. Two still remain: a check against ClassType and a check against MethodType. Also, fix some hinky type comparisons to use isinstance() consistently.
* - Convert all logging to Python's standard logging module. Get rid of allbwarsaw2006-04-171-21/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | traces of our crufty old Syslog. Most of this work was purely mechanical, except for: 1) Initializing the loggers. For this, there's a new module Mailman/loginit.py (yes all modules from now on will use PEP 8 names). We can't call this 'logging.py' because that will interfere with importing the stdlib module of the same name (can you say Python 2.5 and absolute imports?). If you want to write log messages both to the log file and to stderr, pass True to loginit.initialize(). This will turn on propagation of log messages to the parent 'mailman' logger, which is set up to print to stderr. This is how bin/qrunner works when not running as a subprocess of mailmanctl. 2) The driver script. I had to untwist the StampedLogger stuff and implement differently printing exceptions and such to log/error because standard logging objects don't have a write() method. So we write to a cStringIO and then pass that to the logger. 3) SMTPDirect.py because of the configurability of the log messages. This required changing SafeDict into a dict subclass (which is better than using UserDicts anyway -- yay Python 2.3!). It's probably still possible to flummox things up if you change the name of the loggers in the SMTP_LOG_* variables in mm_cfg.py. However, the worst you can do is cause output to go to stderr and not go to a log file. Note too that all entry points into the Mailman system must call Mailman.loginit.initialize() or the log output will go to stderr (which may occasionally be what you want). Currently all CGIs and qrunners should be working properly. I wish I could have tested all code paths that touch the logger, but that's infeasible. I have tested this, but it's possible that there were some mistakes in the translation. - Mailman.Bouncers.BounceAPI.Stop is a singleton, but not a class instance any more. - True/False code cleanup, PEP 8 import restructuring, whitespace normalization, and copyright year updates, as appropriate.
* Now that Python 2.3 is the minimum requirement for Mailman 2.2:bwarsaw2006-04-151-13/+7
| | | | | | | | | | | - Remove True/False binding cruft - Remove __future__ statements for nested scopes - Remove ascii_letters import hack from Utils.py - Remove mimetypes.guess_all_extensions import hack from Scrubber.py - In Pending.py, set _missing to object() (better than using []) Also, update copyright years where appropriate, and re-order imports more to my PEP 8 tastes. Whitespace normalize.
* Port cleaning changes forward from 2.1-maint branch.bwarsaw2005-12-301-6/+7
|
* Use original_sender put in msgdata by CookHeaders.py if it's availablemsapiro2005-11-131-1/+1
|
* Log the sender and not list-bounces when non-VERPmsapiro2005-11-131-2/+8
| | | | and mm_cfg.USE_ENVELOPE_SENDER = Yes
* back porting from 2.1.6tkikuchi2005-08-281-12/+36
|
* FSF office has moved. chdcking in for MAIN branch.tkikuchi2005-08-271-1/+1
|
* quit(): It's possible that self.__conn hasn't been opened yet.bwarsaw2003-04-191-0/+2
|
* Rewrite the Connection class to (hopefully) fix known problems withbwarsaw2003-04-191-9/+13
| | | | | | | | SMTP_MAX_SESSIONS_PER_CONNECTION not being honored and connection problems in the middle of a session not being properly recovered. Closing SF bug #707624, although I implemented this in a different way.
* verpdeliver(): Use the copy module's deepcopy() function instead doingbwarsaw2003-02-051-1/+2
| | | | a round trip through flattening.
* verpdeliver(): If we find a non-fully qualified recipient address whenbwarsaw2003-02-041-0/+8
| | | | | calculating the verp header, we log a message and skip this recipient, because we can't possibly deliver to this person via verp.
* bulkdeliver(): We've had reports of an IOError (code == 4, interruptedbwarsaw2003-01-071-5/+5
| | | | | system call) during the conn.sendmail() call. This just sets the message up to retry later.
* verpdeliver(): Re-enable To header hacking, but only if doing "Fullbwarsaw2002-11-061-27/+27
| | | | Personalization".
* verpdeliver(): Comment out the rewriting of the To header forbwarsaw2002-11-041-27/+27
| | | | | | personalized messages. It was too controversial to do this by default, but we might want to re-enabled this for a future version, probably controlled by a configuration variable.
* process(): If the metadata has an `envsender' key, use that as thebwarsaw2002-10-231-4/+6
| | | | envelope sender, otherwise calculate it as before.
* verpdeliver(): Skip all the realname -> unicode + encode conversionsbwarsaw2002-10-151-2/+4
| | | | when the member has no real name associated with their subscription.
* verpdeliver(): I think I've (finally!) nailed the full name encodingbwarsaw2002-10-041-16/+18
| | | | | | | | problem reported a few weeks ago. We do here largely what we do in CookHeaders.py for encoding non-ascii Subject prefixes. First, we convert the string to Unicode (possibly throwing out characters if necessary), then we use the Header class to properly RFC 2047 encode the name.
* verpdeliver(): When including the member's full name in a verp'dbwarsaw2002-09-171-2/+19
| | | | | | | | | | | | outgoing message, we must be sure to properly MIME encode the header, otherwise if the name has non-ascii characters in it, we'll be sending illegally formatted messages. So if the name has non-ascii characters in it, we'll encode it using the character set of the language of the outgoing message. Kludge: if the charset is us-ascii, we'll use iso-8859-1 for a slightly wider utility (more non-ascii names can be used in English lists). Closes SF bug # 601082 by Tokio Kikuchi.
* bulkdeliver(): I forgot to check this in. Fixes a problem whenbwarsaw2002-08-041-1/+2
| | | | | connection to the smtpd fails with a socket error during the sendmail() call.
* verpdeliver(): dump_address_pair() -> formataddr().bwarsaw2002-04-111-1/+1
|
* An adaptation of Marc MERLIN's fixes to actually honorbwarsaw2002-04-101-3/+6
| | | | | | | | | | | | | | | | | | SMTP_MAX_SESSIONS_PER_CONNECTION. Specifically, sendmail(): We can't return directly from the try-clause because then the decrement of __numsessions won't be executed. Instead, stash the results in a local variable, twiddle numsessions, and then return the results. process(): When doing bulkdeliver, be sure to only decorate the message once. This prevents multiple headers and footers if the message is requeued because of some temporary delivery problem. Note: we don't need to do this for verpdeliver because each message is crafted uniquely anyway. Also, get rid of an unused local variable.
* GetAdminEmail() eradication campaign.bwarsaw2002-03-261-1/+1
| | | | | | | process(), hold_for_approval(), do_discard(): Make the notification email appear to come from the -bounces address. When the sender is for human consumption, make it the -owner address (or in the case of Cleanse.py for anonymous lists, the list posting address).
* process(): We don't want messages sent to the list's -owner[s] to bebwarsaw2002-03-141-1/+2
| | | | | personalized, so look for an existing 'personalize' key and don't VERP if this is false.
* process(): Catch any exception coming out of the deliveryfunc() andbwarsaw2002-03-111-1/+9
| | | | | | | | push the last chunk back on the undelivered list, then re-raise the exception. This ensures that delivery will be re-attempted for the last chunk. They'll get dupes but that's better than them missing the message, and we don't know how many in that chunk may or may not have gotten the message.
* process(): A slight improvement for robustness. Now we assign thebwarsaw2002-03-111-1/+14
| | | | | | | `undelivered' key in the message metadata to the list of chunks the message will be sent to. If something breaks during delivery of this message, the list of undelivered recipients will be saved with the metadata, and should be restored when the message is unshunted.
* A substantial rewrite to support deferring the final stitching of thebwarsaw2002-03-091-107/+120
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | message until SMTP delivery time. This makes VERPish delivery much more efficient. One casualty is that threaded delivery (which was always marked as experimental) is now gone. It might be resurrected as a separate delivery module -- possibly called SMTPThreaded.py -- if there is enough pressure. Specific changes: Connection: New class which abstracts away the handling of disconnect and re-establishment of a connection to the SMTP server in response to the new SMTP_MAX_SESSIONS_PER_CONNECTION setting. process(): Streamline this function now that threaded delivery is gone, but add in the calculation of two types of single-threaded delivery: verpdeliver() for any VERP or personalized delivery, and bulkdeliver() for delivery of an identical copy to a set of recipients. Note that verpdeliver() calls bulkdeliver() after setting up the per-recipient specializations. pre_deliver(), threaded_deliver(): Gone; a victim of threaded delivery removal. verpdeliver(): New function which takes a list of recipients and crafts an in-memory copy of the message to be specialized and delivered to exactly one recipient. bulkdeliver(): Deliver an identical copy to a list of recipients. That list may be of length 1 <wink>.
* Mailman re-uses connections to the SMTP server when deliveringbwarsaw2001-12-161-8/+4
| | | | | | | | | | | | | | | | | multiple chunks. Marc Merlin points out that multi-transaction sessions aren't handled correctly; this fixes the problem. Specifically, process(): In the non-threaded delivery stanza, be sure to quit the connection when finished delivering instead of just closing it. This ensures an SMTP QUIT command is sent. threaded_deliver(): Same for thread-shared connections; quit instead of just close. deliver(): Get rid of the try/finally wrapper around the conn.sendmail() call. We definitely don't want to QUIT the session here.
* process(): When VERPing, blow away the Sender: and Errors-To: headersbwarsaw2001-11-211-2/+14
| | | | | so bouncing MTAs won't be tempted to delivery bounces there (and those headers won't contain the VERPed response address).
* Support for VERP deliveries.bwarsaw2001-11-201-22/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | process(): Don't hardcode the envelope sender to the -admin address. First off, we're directing bounces to the -bounces address. Second, if we're VERPing, we'll be calculating the envelope sender individually for each recipient in the deliver() function. When VERPIng, set the envsender local variable to a tuple containing some pre-calculated information needed by deliver(). Specifically it will contain the split mailbox and domain of the list's -bounces address. Also, when VERPing, each chunk will contain exactly one recipient, so use a list comprehension to calculate this (it's actually a list of lists where the inner list has one item). We'll also pass the envsender local variable to both threaded_deliver() and deliver(). To make VERP delivery slightly more efficient, every thread gets its own connection to the MTA. So we create an smtplib.SMTP instance once but do not connect it until we actually get to deliver() -- and there we have to use a kludge to determine whether we're connected or not. deliver() will re-use the connection to the MTA to do the conn.sendmail() calls. pre_deliver(): Add the conn argument. threaded_deliver(): Create one smtplib.SMTP instance for each thread. Be sure to close the connection when the thread is finished. deliver(): Add the conn argument. Craft the VERP envelope sender as described above. Make sure the MTA connection is connect()'ed.
* process(): Remove the last vestige of the string module. Also, makebwarsaw2001-10-041-3/+4
| | | | this work when mlist is None.
* process(): Get the message's text, sans Unix-From by using the emailbwarsaw2001-10-011-1/+1
| | | | | package's Message.as_string() method. (This is one module which doesn't yet have a unittest. );
* process(): Instead of using MAILMAN_OWNER, use Utils.get_site_email().bwarsaw2001-08-041-1/+1
| | | | The former config variable is going away.
* process(): Call syslog.write_ex() directly (instead of syslog.write()bwarsaw2001-07-061-8/+8
| | | | | | implied by syslog.__call__()), passing in the MsgSafeDict instance to the kws argument. This works around the Python limitation that **kws in extended calls must be concrete dictionary objects.
* process(): Add support for configurable log files. Four new variablesbwarsaw2001-06-271-14/+34
| | | | | | | | | | | | have been added: SMTP_LOG_EVERY_MESSAGE SMTP_LOG_SUCCESS SMTP_LOG_REFUSED SMTP_LOG_EACH_FAILURE The usage and format of these variables are described in Defaults.py.in.
* process(): It is possible that the mlist passed in is None, whichbwarsaw2001-05-221-1/+4
| | | | | means that the message does not originate from a mailing list. In that case, admin should be the email address of MAILMAN_OWNER.
* process(): We can't call mlist.RegisterBounce() here because we do notbwarsaw2001-02-161-8/+7
| | | | | | | have the list lock. Instead, just append permanently failing recipients to another list. Raise SomeRecipientsFailed if there were either temp or perm failures, and return both lists in the exception instance.
* Conversion to mimelib.bwarsaw2001-02-151-21/+13
| | | | | | | | De-string-module-ify Other Python 2.0 constructs used where appropriate. Get rid of HandlerAPI references -- this module is obsolete.
* process(): Only log successes to logs/post if the message actually wasbwarsaw2000-09-081-1/+5
| | | | | posted to the list (as opposed to the -admin, -owner, or -request addrs, or internally generated). Closes SF bug #111574.
* process(): Remove the saveifdirty flag from the RegisterBounce() call.bwarsaw2000-09-081-1/+1
|
* process(): Addressing both SF bug #108167 and patch #100861, thisbwarsaw2000-07-221-1/+11
| | | | | patch writes success and failure messages to logs/post. It retains the writing of smtp delivery timing messages to logs/smtp.
* process(): Wrap the actual message delivery loops in a mlistbwarsaw2000-06-031-12/+15
| | | | | | Save/Unlock followed by a Lock call. Because deliveries don't touch the list and can take a long time, this improves responsiveness of the system during message delivery.
* Support for delivering recipient chunks via multiple threads. Pythonbwarsaw2000-06-031-19/+86
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | must have been built with threading support, and MAX_DELIVERY_THREADS must be > 0. It is not enabled by default because it's not clear this is a performance win. This feature should be considered experimental for now. Specifically, process(), deliver(): No longer needs the MailList object in it's argument list. This allows us to multi-thread actual message delivery. We also calculate the message's text and the admin address only once. deliver() also takes a dictionary for failures, which it updates with chunk failures. Convert logging stuff to use new syslog() interface instead of MailList.LogMsg(). Log total delivery time instead of chunk delivery time. chunkify(): Instead of a separate bucket for each TLD, we now have only 4 buckets for recipients: .com .net/.org .edu/.ca/.us everything else buckets are not backfilled. pre_deliver(), threaded_deliver(): New functions.
* Changes to honor SMTP_MAX_RCPTS again. Specifically,bwarsaw2000-05-301-29/+79
| | | | | | | | | | | | | | | | | | | | | | deliver(): Does the actual job of delivering to a list of recipients. Takes the mailing list object, the message, and the list of recipients and returns a dictionary mapping refused/failed addresses to a 2-tuple containing the error code and the error message. This function also logs chunk delivery time -- the delivery time of the entire message isn't recorded any longer. chunkify(): Turn a list of recipient addresses into a list of lists, where each sublist is no larger than SMTP_MAX_RCPTS in length. A minimal amount of sorting is performed: addresses are collated on the top-level domain (i.e. `.com', `.net', etc.), but each chunk is filled completely. This should balance the ability to give the MTA a chance to do more detailed delivery sorting with the amount of work being done in this module. process(): Changed logic to use chunkifying algorithm. If SMTP_MAX_RCPTS is <= 0, no chunking is done. While we do still log the recipients that fail, we don't know whether they all failed or not. If any temporary failures occurred, raise SomeRecipientsFailed.
* Many changes to make message delivery more robust in the face ofbwarsaw2000-05-081-52/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | failures, bugs, and lock acquisition timeouts. Instead of storing information about the progress of the delivery on the Message object, we pass around a parallel data structure called `msgdata' (current just a dictionary). All calculated information is passed through this object, but this changes the API to handler modules. They now take three arguments: the mailing list, the message object, and the msgdata. WARNING: This may change before 2.0 final. Specific changes include: HandlerAPI DiscardMessage(), HandlerAPI.SomeRecipientsFailed(): New shared exceptions. pipeline_deliver(): removed LIST_PIPELINE: global containing the primary list delivery pipeline DelivertoList: Revamped main entry point into message delivery to list membership. Takes three arguments: the mailing list, the message object, and the msgdata dictionary. This digs the pipeline to use out of the msgdata (allowing resumption of prematurely interrupted pipeline deliveries). Then each module is called in turn, and the shared exceptions are caught. As each module is completed successfully, it is removed from the head of the pipeline. This function returns the number of pipeline modules remaining to be executed (i.e. a return of 0 means DeliverToList() is done with this message and it can be dequeued). A catch-all is included in case some unexpected exception occurs (say a bug or typo in one of the delivery modules). Such an error will queue the message, so at least it doesn't just get lost. We try to never just lose a message. RedeliverMessage(), DeliverToUser(): reimplemented in terms of DeliverToList(). Acknowledge, AfterDelivery, CalcRecips, Cleanse, CookHeaders, Decorate, Replybot, ToArchive, ToUsenet Fix the function signature to match the new API (three arguments), and changed the implementations to extract delivery information from msgdata instead of as attributes of the message object. Approved Same as above, but also removed NotApproved exception. LoopError is now multiply derived from HandlerAPI.DiscardMessage and Errors.MMLoopingPost. Hold Same as above, but also changed slightly the way an exception is raised when a message is held. hold_for_approval() now takes four arguments (the msgdata parameter has been added), and the exc object can be a class or instance. If it's a class, it is simply zero-arg'd instantiated. We also use the str() of the exception to get us the reason for the hold. This allows us to override HandlerAPI.MessageHeld.__str__() for MessageToBig so that we can include the size of the message being held. SMTPDirect Same as above, but instead of explicitly enqueuing the messages when some or all of the recipient deliveries failed, just raise a HandlerAPI.SomeRecipientsFailed exception and let DeliverToList() manage the enqueuing. Thus queue_message() is removed. Sendmail Same as above, but if any chunks fail delivery, those recipients are queued by raising SomeRecipientsFailed. SpamDetect Same as above, except that if a regexp matches, a SpamDetect exception is raised directly. The DeliverToList() framework discards these spam messages instead of holding them for approval. ToDigest Same as above, except that if a digest is prepared for delivery, it is not sent directly via mlist.Post(). Instead, the message is queued for delivery, thereby relinquishing the lock soon. This means that digests will only be sent the next time qrunner runs.
generated by cgit v1.2.3-70-g09d2 (git 2.51.2) at 2025-11-09 00:09:44 +0000