[Twisted-Python] supressess warnings

Andrew Dalke dalke at dalkescientific.com
Tue Apr 1 07:00:53 EST 2003

I'm just starting to learn how to use Twisted.  My first project
is to write a simple XML-RPC server.  I started with the demo,
and that worked fine.

I then added an import statement for the module I need.

from openeye.oechem import *

Now, this uses a shared library extension which was built
for Python 2.2, and I'm running a CVS version of Python 2.3,
so I usually get

RuntimeWarning: Python C API version mismatch for module _oechem: This  
Python has API version 1012, module _oechem has version 1011.
   import _oechem

However, that didn't happen when I started my my simple
server.  I got no warning message.  I started to worry that my
file system wasn't properly saving files!  I played around with
the imports, and moved that import to be above the twisted
ones, and it started reporting the warning.

A bit of searching later and I found
which redefines (!) warnings.showwarning to dump the data to a
logfile, which is initialized to a NullFile(), which does nothing.

So the twisted code is throwing some (potentially useful) diagnostics.

Doing a search I found the commit statement
 > 2) implemented a custom warning handler; now warnings look sexy.
 >   (the hackish overriding of warnings.showwarning is the recommended  
way to
 >   do so, according to the library reference.)

I'll grant that's what the docs say.  However, why is the logfile for
showwarnings initialized to NullFile, and not to sys.stderr?

For that matter, I think it's better to delay replacement until  
is called.

Hmm, and why is os.linesep used in the logging?  The log
file should have been opened in text mode, in which case "\n"
will get translated to the proper os-dependent form.

I started to change the code, so I could submit a patch.  I got the
code out of CVS and am reading the documentation at
It says that the main test suite builder is in 'twisted.test.test_all'

That module doesn't exist (can't import it) nor is the appropriate
file present anywhere.  And the hyperlink from that page yields
   "No Such Resource
    File not found."

It then says
   Acceptance tests are all automated by the bin/accepttests
but that program is in admin/accepttests

I ran that for my Mac OS X machine, with 'open' as the
WEBBROWSER setting (where does the '-b' option go, since
open doesn't block .. do I pass it to 'accepttests'?  Yes.) and  
as my IRCCLIENT, since I wouldn't know what to do with one
even assuming I had one.

Anyway, after a while it gave this error

|#### Starting mail test.
|#### Output should be one email (postmaster at foo.bar)
|#### and one bounce (postmaster at no.such.domain).
Running Command: './admin/../bin/mktap mail --domain foo.bar=dump  
--user postmaster=postmaster  --pop 18110'
option --pop not a unique prefix
Usage: mktap mail [options]
   -p, --pop3=                 Port to start the POP3 server on (0 to  
                               [default: 8110]
   -S, --pop3s=                Port to start the POP3-over-SSL server on  
(0 to
                               disable). [default: 0]
   -s, --smtp=                 Port to start the SMTP server on (0 to  
                               [default: 8025]
   -r, --relay=                relay mail we do not know how to handle  
to this
                               IP, using the given path as a queue  
   -c, --certificate=          Certificate file to use for SSL  
   -d, --domain=               generate an SMTP/POP3 virtual domain  
which saves
                               to "path"
       --help                  Display this help and exit.
   -u, --user=                 add a user/password to the last specified  
       --bounce-to-postmaster  undelivered mails are sent to the  

This creates a mail.tap file that can be used by twistd.
Running Command: './admin/../bin/twistd -f mail.tap'

Failed to load application: [Errno 2] No such file or directory:  
Traceback (most recent call last):
   File "./admin/accepttests", line 271, in ?
   File "./admin/accepttests", line 247, in runAllTests
   File "./admin/accepttests", line 204, in runMailTest
     s = smtplib.SMTP('', 18026)
   File "/usr/local/lib/python2.3/smtplib.py", line 240, in __init__
     (code, msg) = self.connect(host, port)
   File "/usr/local/lib/python2.3/smtplib.py", line 302, in connect
     raise socket.error, msg
socket.error: (61, 'Connection refused')

Hmmm.... Is 'admin/runtests' the correct script to run?  I ran that,
and got the following output

FAILURE: testReadLimit (twisted.test.test_policies.ThrottlingTestCase)
Traceback (most recent call last):
    File "/Users/dalke/cvses/Twisted/twisted/test/test_policies.py",  
line 193, in testReadLimit
     self.assertEquals(tServer.readThisSecond, 0)
  FailTest: 20 != 0
FAILURE: testWriteLimit (twisted.test.test_policies.ThrottlingTestCase)
Traceback (most recent call last):
    File "/Users/dalke/cvses/Twisted/twisted/test/test_policies.py",  
line 152, in testWriteLimit
     self.assert_(abs(p.wrappedProtocol.paused - now - 1.0) < 0.1)
FAILURE: testPackages (twisted.test.test_setup.CheckingPackagesTestCase)
Traceback (most recent call last):
    File "/Users/dalke/cvses/Twisted/twisted/test/test_setup.py", line  
44, in testPackages
     self.failUnlessEqual(l, [])
  FailTest: ['twisted.bugs', 'twisted.coils', 'twisted.eco',  
'twisted.enterprise.userRequests', 'twisted.forum', 'twisted.inetd',  
'twisted.issues', 'twisted.issues.ui', 'twisted.issues.ui.templates',  
'twisted.lumberjack', 'twisted.metrics', 'twisted.pretzel',  
'twisted.protocols.ldap', 'twisted.reality', 'twisted.reality.ui',  
'twisted.secsh', 'twisted.test.interactive', 'twisted.web.blog',  
'twisted.words.ui', 'twisted.words.ui.gateways'] != []
FAILURE: testFailing (twisted.test.test_tcp.LoopbackTestCase)
Traceback (most recent call last):
    File "/Users/dalke/cvses/Twisted/twisted/test/test_tcp.py", line  
171, in testFailing
     self.assert_(time.time() - start < 0.1)
ERROR: testUnicodeTolerance (twisted.test.test_xml.MicroDOMTest)
Traceback (most recent call last):
    File "/Users/dalke/cvses/Twisted/twisted/trial/unittest.py", line  
165, in runOneTest
    File "/Users/dalke/cvses/Twisted/twisted/test/test_xml.py", line  
326, in testUnicodeTolerance
     ud = microdom.parseString(s.encode('UTF-16'))
    File "./twisted/web/microdom.py", line 644, in parseString
    File "./twisted/web/microdom.py", line 609, in parse
    File "./twisted/protocols/sux.py", line 155, in dataReceived
    File "./twisted/protocols/sux.py", line 176, in do_begin
    File "./twisted/protocols/sux.py", line 108, in _parseError
  ParseError: <xmlfile />:1:1: First char of document ['\xfe'] wasn't <
SKIPPED: test_modules (twisted.test.test_doc.DocCoverage)
Remove this line when you feel like writing docstrings.
Ran 494 tests in 227.667s

FAILED (failures=4, errors=1, skips=1)

Could not import twisted.test.test_conch: No module named Crypto.Cipher

So I tried to rewrite the code given that it looks like the self-tests
run reasonably well, but then ran into complications with this bit of  

# Make sure we have some basic logging setup.  This only works in  
except NameError:
     logOwner = LogOwner()


# Prevent logfile from being erased on reload.  This only works in  
except NameError:
     logfile = NullFile()
     logerr = sys.stderr

Okay, so we've got some code duplication here, and comment
skew.  I've fixed that up, and tidied up a few other bits of the code.
And for the life of me I can't figure out why the log file should
default (when no logging is enabled) to the NullFile rather than
sys.stderr -- if you really want the NullFile, just pass it in.  So
I switched it to use sys.stderr.

I then reran the 'admin/runtests' script and got ... some warnings
and comments which were being supressed.

DeprecationWarning: twisted.python.delay is deprecated. Please use  
reactor methods.
   from twisted.python import delay
/Users/dalke/cvses/Twisted/twisted/protocols/dns.py:46: RuntimeWarning:  
PyCrypto not available - proceeding with non-cryptographically secure  
random source
Enabling Multithreading.
Resolver added ('', 53) to server list
Resolver added ('', 53) to server list

Also, I removed those os.linesep references, and the os module,
and the needless string module.  And turned 'file_protocol' into
'_file_protocol' since it's for internal use only.

Why's the Log pickleable? What happens if a cStringIO is passed
in instead of a file?  Or an outgoing network socket?  Or a

It looks like Log.synchronized is old code that can be removed.  Yes?

Why does doc/examples/cursesclient.py set log.logfile directly?
Shouldn't it call log.startLogging(log.NullFile()) ?  It's the only  
piece of code which references log.NullFile.

Is 'discardLogs' the true inverse of startLogging?  If so, it
should add something to restore sys.stdout/sys.stderr if
startLogging asked them to be changed.

If not, where's the 'stopLogging' function?  And 'discard'
means to ignore logging?  Or should it throw away the
log files?  The terminology is confusing.

Anyway, the diff -u is attached for you all to review and
-------------- next part --------------
A non-text attachment was scrubbed...
Name: log.py.diff
Type: application/octet-stream
Size: 4148 bytes
Desc: not available
Url : http://twistedmatrix.com/pipermail/twisted-python/attachments/20030401/a162bb55/attachment.obj 
-------------- next part --------------

Oh, and with my changes, I now see the expected warning
when I start up, and I see two twisted messages ...

[Andrew-Dalkes-Computer:~/cvses/Twisted] dalke% python cansmi_rpc.py
RuntimeWarning: Python C API version mismatch for module _oechem: This 
Python has API version 1012, module _oechem has version 1011.
   import _oechem
twisted.web.server.Site starting on 8080
Starting factory <twisted.web.server.Site instance at 0x640850>

which tells me that those two messages ("Site starting" and "Starting
factory") were being sent to never-never-land the whole time.  Perhaps
the latter shouldn't be present?

					dalke at dalkescientific.com

More information about the Twisted-Python mailing list