| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
https://mailman.svn.sourceforge.net/svnroot/mailman/branches/tmp-sqlalchemy-branch
................
r8114 | bwarsaw | 2006-12-06 00:16:54 -0500 (Wed, 06 Dec 2006) | 44 lines
Initial take on using SQLAlchemy to store list data in lieu of Python pickles.
While all the list data (including OldStyleMemberships attributes) are stored
in the database, many attributes are stored as PickleTypes binary data. This
isn't idea but it gets things working until a more sophisticated schema can be
developed.
MailList class is now a new-style class, as is required by SQLAlchemy. This
makes several things, er, interesting. Rip out all the low-level pickle
reading and writing stuff. Hook SA transaction events into Lock() and
Unlock(). Move the hooking of the _memberadaptor into InitTempVars(), which
gets called by the SQLAlchemy hooks (MailList.__init__() never is).
Add an initialize.py module which centralizes all the initialization bits that
command line scripts have to do, including configuration, logging, and atabase
initialization.
This change also converts bin/withlist to mmshell wrapper.
Update to SQLAlchemy 0.3.1.
Revamp paths.py.in considerably. There were several problems with the old
way. We no longer disable default loading of site-packages so we don't need
to add Python's site-packages back to sys.path. Also, because
site.addsitedir() causes things like .pth paths to be /appended/ to sys.path,
they actually won't override any site-installed packages. E.g. if SQLAlchemy
is installed in the system Python, our version will not override. IIUC,
setuptools-based packages can be configured to work properly in the face of
package versions, however not all packages we currently depend on are
setuptools-based. So instead, we steal a bit of stuff from site.py but change
things so the prepend .pth stuff to sys.path.
Update several modules to use True/False and whitespace normalization.
Convert from mm_cfg to config object. Modernize a few coding constructs.
Add a couple of exceptions to handle database problems.
In the export script, include the widget type in the elements. This helped in
my stupid little throw away conversion script, but I think it will be more
generally useful.
Add an interact.py module which refactors interactive interpreter access.
Mostly this is used by withlist -i, but it lets us import Mailman.interact and
drop into a prompt just about anywhere (e.g. debugging).
................
r8115 | bwarsaw | 2006-12-07 09:13:56 -0500 (Thu, 07 Dec 2006) | 22 lines
Start to flesh out more of the SQLAlchemy mechanisms.
Added a MailList.__new__() which hooks instantiation to use a query on
dbcontext to get an existing mailing list. A 'no-args' call means we're doing
a Create(), though eventually that will change too.
For now, disable the CheckVersion() call. Eventually this will be folded into
schema migration.
list_exists(): Rewrite to use the dbcontext query to determine if the named
mailing list exists or not. Requires the fqdn_listname.
Eradicate two failed member adaptors: BDBMemberAdaptor and SAMemberships.
Change the way the DBContext holds onto tables. It now keeps a dictionary
mapping the table's name to the SA Table instance. This makes it easier to
look up and use the individual tables.
Add 'web_page_url' as an attribute managed by SA, and remove a debugging
print.
................
r8116 | bwarsaw | 2006-12-11 07:27:47 -0500 (Mon, 11 Dec 2006) | 29 lines
Rework the whole dbcontext and transaction framework. SA already handles
nested transactions so we don't have to worry about them. However, we do have
the weird situation where some transactions are tied to MailList
.Lock()/.Unlock()/.Save() and some are tied to non-mlist actions. So now we
use an @txn decorator to put methods in a session transaction, but then we
also hook into the above MailList methods as possibly sub-transactions. We
use a weakref subclass to manage the MailList interface, with a dictionary
mapping MailList fqdn_listnames against transactions. The weakrefs come in by
giving us a callback when a MailList gets derefed such that we're guaranteed
to rollback any outstanding transaction.
Also, we have one global DBContext instance but rather than force the rest of
Mailman to deal with context objects, instead we expose API methods on that
object into the Mailman.database module, which the rest of the code will use.
Such methods must be prepended with 'api_' to get exposed this way.
bin/rmlist now works with the SA-backend. I refactored the code here so that
other code (namely, the test suite) can more easily and consistently remove a
mailing list. This isn't the best place for it ultimately, but it's good
enough for now.
New convenience functions Utils.split_listname(), .fqdn_listname().
Convert testall to use Mailman.initialize.initialize(). Not all tests work,
but I'm down to only 8 failures and 7 errors. Also, do a better job of
recovering from failures in setUp().
MailList.__new__() now takes keyword arguments.
................
r8117 | bwarsaw | 2006-12-11 22:58:06 -0500 (Mon, 11 Dec 2006) | 7 lines
Unit test repairs; even though the unit tests are still pretty fragile,
everything now passes with the SQLAlchemy storage of list data.
Added missing 'personalize' column. Converted mailmanctl and qrunner to
initialize() interface. Fixed _cookie_path() to not fail if SCRIPT_NAME is
not in the environment.
................
r8118 | bwarsaw | 2006-12-27 18:45:41 -0500 (Wed, 27 Dec 2006) | 21 lines
Utils.list_names(): Use a database query to get all the list names.
dbcontext.py: Added api_get_list_names() to support Utils.list_names().
listdata.py: Added two additional MailList attributes which need to be stored
in the database. The first is 'admin_member_chunksize' which isn't modifiable
from the web. The second is 'password' which holds the list's password.
HTMLFormatObject: item strings can now be unicodes.
bin/list_lists.py: Must call initialize() to get the database properly
initialized, not just config.load(). This will be a common theme.
SecurityManager.py:
- Remove md5 and crypt support
- Added mailman.debug logger, though it will be only used during
debugging.
- The 'secret' can be a unicode now.
- A few coding style updates; repr() instead of backticks, 'key in dict'
instead of 'dict.has_key(key)'
................
r8119 | bwarsaw | 2006-12-27 19:13:09 -0500 (Wed, 27 Dec 2006) | 2 lines
genaliases.py: config.load() -> initialize()
................
r8120 | bwarsaw | 2006-12-27 19:17:26 -0500 (Wed, 27 Dec 2006) | 9 lines
Blocked revisions 8113 via svnmerge
........
r8113 | bwarsaw | 2006-12-05 23:54:30 -0500 (Tue, 05 Dec 2006) | 3 lines
Initialized merge tracking via "svnmerge" with revisions "1-8112" from
https://mailman.svn.sourceforge.net/svnroot/mailman/branches/tmp-sqlalchemy-branch
........
................
r8121 | bwarsaw | 2006-12-28 23:34:52 -0500 (Thu, 28 Dec 2006) | 20 lines
Remove SIGTERM handling from all the CGI scripts. This messes with HTTPRunner
because when you issue "mailmanctl stop" after the signal handler has been
installed, the process will get a SIGTERM, the signal handler will run, and
the process will exit with a normal zero code. This will cause mailmanctl to
try to restart the HTTPRunner.
I don't think we need that stuff at all when running under wsgi with a
SQLAlchemy backend. If mailmanctl kills the HTTPRunner in the middle of the
process, I believe (but have not tested) that the transaction should get
properly rolled back at process exit. We need to make sure about this, and
also we need to test the signal handling functionality under traditional CGI
environment (if we even still want to support that).
Also, make sure that we don't try to initialize the loggers twice in qrunner.
This was the cause of all the double entries in logs/qrunner.
Fix a coding style nit in mailmanctl.py.
De-DOS-ify line endings in loginit.py.
................
|
| |
|
|
|
|
|
|
| |
the web u/i to work. This also fixes the use of Utils.list_names() in the
list and admin overviews. This API now returns a set, but the CGIs want to
sort them, so we need to turn them back into lists.
This change also elaborates an exception so that the list name is reported.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
require Python to be at least 2.3.0
Convert all scripts/* scripts (except driver) to use the
symlink-to-bin/mmshell trick.
For now (since we're debugging more often than releasing), set STEALTH_MODE to
False in scripts/driver. We need to remember to turn this back to True when
we start getting closer to releases!
In the various scripts, er, Mailman/bin scripts, convert them to $-strings for
i18n substitution, and clean up imports. Also, get rid of the crufty Emacs
-*- line, which is no longer needed now that the files end in .py.
Get rid of lots of unnecessary svn:executable properties.
Remove an unnecessary import from Mailman/bin/disabled.py
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
traces of our crufty old Syslog. Most of this work was purely mechanical,
except for:
1) Initializing the loggers. For this, there's a new module
Mailman/loginit.py (yes all modules from now on will use PEP 8
names). We can't call this 'logging.py' because that will
interfere with importing the stdlib module of the same name (can
you say Python 2.5 and absolute imports?).
If you want to write log messages both to the log file and to
stderr, pass True to loginit.initialize(). This will turn on
propagation of log messages to the parent 'mailman' logger, which
is set up to print to stderr. This is how bin/qrunner works when
not running as a subprocess of mailmanctl.
2) The driver script. I had to untwist the StampedLogger stuff and
implement differently printing exceptions and such to log/error
because standard logging objects don't have a write() method. So
we write to a cStringIO and then pass that to the logger.
3) SMTPDirect.py because of the configurability of the log messages.
This required changing SafeDict into a dict subclass (which is
better than using UserDicts anyway -- yay Python 2.3!). It's
probably still possible to flummox things up if you change the
name of the loggers in the SMTP_LOG_* variables in mm_cfg.py.
However, the worst you can do is cause output to go to stderr and
not go to a log file.
Note too that all entry points into the Mailman system must call
Mailman.loginit.initialize() or the log output will go to stderr
(which may occasionally be what you want). Currently all CGIs and
qrunners should be working properly.
I wish I could have tested all code paths that touch the logger, but
that's infeasible. I have tested this, but it's possible that there
were some mistakes in the translation.
- Mailman.Bouncers.BounceAPI.Stop is a singleton, but not a class
instance any more.
- True/False code cleanup, PEP 8 import restructuring, whitespace
normalization, and copyright year updates, as appropriate.
|
| | |
|
| | |
|
| |
|
|
| |
it to On.
|
| |
|
|
|
| |
closes bug # 716754. Must output any accumulated output to stdout
when a SystemExit is raised.
|
| |
|
|
|
|
|
| |
only if the cgi script runs to completion successfully do we write the
entire output to the real stdout (connected to the web server). This
way, if any exception occurs during the rendering of the page (e.g. in
"print doc.Format()") we won't mess up the diagnostics page.
|
| |
|
|
|
|
|
|
| |
in Defaults.py, we need to arrange for this script to import paths.py
to pick up the correct site-packages prefixed sys.path.
Because we do this, we can delete the sys.path.insert() call, which
was redundant.
|
| |
|
|
|
|
|
|
|
| |
Mailman/pythonlib/cgi.py because I'd like for Mailman to start using
the default Python module now that we require at least Python 2.0 (or
should that be Python 2.1?).
I'm sure any disparity between the cgi.py modules will show up in beta
testing. But if it looks okay, I'll remove our copy of cgi.py.
|
| | |
|
| |
|
|
|
| |
too magical so eventually explicit imports will be added to the
appropriate modules.
|
| |
|
|
| |
Modifications to use extended print syntax.
|
| | |
|
| |
|
|
|
| |
the try block. Should have no practical effect, but makes debugging
easier.
|
| |
|
|
|
| |
headers. These caused problems with the back button and didn't really
solve an existing problem.
|
| |
|
|
| |
no-cache" and "Expires: 0" to inhibit caching.
|
| | |
|
| |
|
|
|
| |
traceback, system, and environment information to the web pages. The
information is always still printed to the log files.
|
| |
|
|
|
|
|
|
| |
operating environment, namely some key sys module attributes (version,
prefix, etc.). We're going to need a way to scale back this reporting
for the Web pages in case folks do not want to reveal this much
information. And hey, since 2.0 will have no bugs, this stuff is all
moot anyway, right? :)
|
| |
|
|
|
|
|
|
| |
the logger object so they can simply sys.stderr.write() to get error
messages out to logs/error.
No need to set sys.__stderr__ and sys.__stdout__; we're requiring at
least Python 1.5.2 now.
|
| |
|
|
| |
is going to be required now.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
content-type header, or include a bogus one.
cgi.py
FieldStorage.__init__(): if there is no content-type header, use
text/plain for inner parts, but application/x-www-form-urlencoded
for outer parts. Honor any existing content-type header.
Lower down, if the content-type header is something we don't
understand, default to text/plain for inner parts, but
application/x-www-form-urlencoded for outer parts.
This patch will be proposed for Python 1.5.2
driver
run_main(): Play some games to get Mailman's special cgi.py
module. We import it from Mailman.pythonlib.cgi, but then we jam
this module into sys.modules['cgi'] so all those imports in all
those Mailman.Cgi scripts get our special one (without having to
hack all those scripts!)
|
| |
|
|
| |
into both the error log and the HTML page.
|
| |
|
|
|
|
| |
definition -- the only place it's used. Also, comment out the whole
def and __builtin__ hack. This debugging stuff probably shouldn't go
into the release (although its use should be documented).
|
| |
|
|
| |
portably re-raise the exception.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
Python 1.5.2 will do it (for debugging)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
uncaught exceptions through to the top level.
Uncommented out the open() definition. I think this is very helpful
during the beta debugging phase. We'll comment this out for 1.0
release (it also won't be necessary in Python 1.5.2, but nobody has
that yet!)
Exception printing has been changed so that the MultiLogger is no
longer necessary. Removes one more potential crash, but also
rationalizes print_exception() and print_environment(). There's
always exactly two places information goes:
1. To a plain text log file. This could be sys.__stderr__ if no
explicit log file is given -- e.g. the Web server's log file.
2. To an HTML sink. This will always be sys.stdout (since we use
the `print' statement), and will generally always be connected
to the input of the Web browser.
Log file printing happens first, just in case (it's better that the
server knows what's happening), but we always try to print both.
`Main' has been rewritten also. We now do not catch any problems with
import sys. If sys can't be imported, then the Python installation is
royally screwed -- even `print' won't work! Better to just let the
catastrophy percolate up to the Web browser. Even so this should
almost never happen.
Poke __stderr__ and __stdout__ into the sys module if they aren't
already there. GvR says this is a 1.5.1 feature, but we can make it
work well enough for older versions of Python, and it's convenient.
Give one final shot at printing the traceback and environment if the
exception percolates all the way up out of run_main(). This time,
just print them to stderr (usually meaning the Web server's error
log). Should this fail too, do whatever we can not to generate an
end-user visible server error.
|
| |
|
|
|
|
|
| |
case anything breaks between the start of this function and the
creation of the real objects, otherwise the calls in the except branch
will fail, kicking us back to a low-level exception. using sys.stderr
isn't great, but it's better than nothing.
|
| | |
|
| |
|
|
|
|
|
|
| |
errors. Now all tracebacks get sent to the user as HTML *and* logged
to logs/error, although the format of the output is slightly different
(not HTML in the latter case).
Some rearranging of code, simplification, etc.
|
|
|
Mailman.Cgi package. Be as paranoid as possible about catching
exceptions and reporting them as HTML (which has saved my butt
numerous times already :-).
See the comments in the file for things that can still cause Server
errors.
|
