1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
|
The switchboard
===============
The switchboard is subsystem that moves messages between queues. Each
instance of a switchboard is responsible for one queue directory.
>>> msg = message_from_string("""\
... From: aperson@example.com
... To: _xtest@example.com
...
... A test message.
... """)
Create a switchboard by giving its queue directory.
>>> import os
>>> queue_directory = os.path.join(config.QUEUE_DIR, 'test')
>>> from mailman.queue import Switchboard
>>> switchboard = Switchboard(queue_directory)
>>> switchboard.queue_directory == queue_directory
True
Here's a helper function for ensuring things work correctly.
>>> def check_qfiles(directory=None):
... if directory is None:
... directory = queue_directory
... files = {}
... for qfile in os.listdir(directory):
... root, ext = os.path.splitext(qfile)
... files[ext] = files.get(ext, 0) + 1
... if len(files) == 0:
... print 'empty'
... for ext in sorted(files):
... print '{0}: {1}'.format(ext, files[ext])
Enqueing and dequeing
---------------------
The message can be enqueued with metadata specified in the passed in
dictionary.
>>> filebase = switchboard.enqueue(msg)
>>> check_qfiles()
.pck: 1
To read the contents of a queue file, dequeue it.
>>> msg, msgdata = switchboard.dequeue(filebase)
>>> print msg.as_string()
From: aperson@example.com
To: _xtest@example.com
<BLANKLINE>
A test message.
<BLANKLINE>
>>> dump_msgdata(msgdata)
_parsemsg: False
version : 3
>>> check_qfiles()
.bak: 1
To complete the dequeing process, removing all traces of the message file,
finish it (without preservation).
>>> switchboard.finish(filebase)
>>> check_qfiles()
empty
When enqueing a file, you can provide additional metadata keys by using
keyword arguments.
>>> filebase = switchboard.enqueue(msg, {'foo': 1}, bar=2)
>>> msg, msgdata = switchboard.dequeue(filebase)
>>> switchboard.finish(filebase)
>>> dump_msgdata(msgdata)
_parsemsg: False
bar : 2
foo : 1
version : 3
Keyword arguments override keys from the metadata dictionary.
>>> filebase = switchboard.enqueue(msg, {'foo': 1}, foo=2)
>>> msg, msgdata = switchboard.dequeue(filebase)
>>> switchboard.finish(filebase)
>>> dump_msgdata(msgdata)
_parsemsg: False
foo : 2
version : 3
Iterating over files
--------------------
There are two ways to iterate over all the files in a switchboard's queue.
Normally, queue files end in .pck (for 'pickle') and the easiest way to
iterate over just these files is to use the .files attribute.
>>> filebase_1 = switchboard.enqueue(msg, foo=1)
>>> filebase_2 = switchboard.enqueue(msg, foo=2)
>>> filebase_3 = switchboard.enqueue(msg, foo=3)
>>> filebases = sorted((filebase_1, filebase_2, filebase_3))
>>> sorted(switchboard.files) == filebases
True
>>> check_qfiles()
.pck: 3
You can also use the .get_files() method if you want to iterate over all the
file bases for some other extension.
>>> for filebase in switchboard.get_files():
... msg, msgdata = switchboard.dequeue(filebase)
>>> bakfiles = sorted(switchboard.get_files('.bak'))
>>> bakfiles == filebases
True
>>> check_qfiles()
.bak: 3
>>> for filebase in switchboard.get_files('.bak'):
... switchboard.finish(filebase)
>>> check_qfiles()
empty
Recovering files
----------------
Calling .dequeue() without calling .finish() leaves .bak backup files in
place. These can be recovered when the switchboard is instantiated.
>>> filebase_1 = switchboard.enqueue(msg, foo=1)
>>> filebase_2 = switchboard.enqueue(msg, foo=2)
>>> filebase_3 = switchboard.enqueue(msg, foo=3)
>>> for filebase in switchboard.files:
... msg, msgdata = switchboard.dequeue(filebase)
... # Don't call .finish()
>>> check_qfiles()
.bak: 3
>>> switchboard_2 = Switchboard(queue_directory, recover=True)
>>> check_qfiles()
.pck: 3
The files can be recovered explicitly.
>>> for filebase in switchboard.files:
... msg, msgdata = switchboard.dequeue(filebase)
... # Don't call .finish()
>>> check_qfiles()
.bak: 3
>>> switchboard.recover_backup_files()
>>> check_qfiles()
.pck: 3
But the files will only be recovered at most three times before they are
considered defective. In order to prevent mail bombs and loops, once this
maximum is reached, the files will be preserved in the 'bad' queue.
>>> for filebase in switchboard.files:
... msg, msgdata = switchboard.dequeue(filebase)
... # Don't call .finish()
>>> check_qfiles()
.bak: 3
>>> switchboard.recover_backup_files()
>>> check_qfiles()
empty
>>> bad = config.switchboards['bad']
>>> check_qfiles(bad.queue_directory)
.psv: 3
Clean up
>>> for file in os.listdir(bad.queue_directory):
... os.remove(os.path.join(bad.queue_directory, file))
>>> check_qfiles(bad.queue_directory)
empty
Queue slices
------------
XXX Add tests for queue slices.
|
