[Twisted-Python] intermittent traceback using adbapi with pyPgSQL
Stephen Waterbury
golux at comcast.net
Fri Aug 6 14:58:18 EDT 2004
Twisted gurus,
I didn't want to create a bug report yet
because I'm not sure that it's a Twisted bug. I
have only observed it while using pyPgSQL as the adaptor;
haven't seen it yet (and have run the exact same code
that produces it) with psycopg.
I see this traceback intermittently occur at the end of
chains of deferreds that include 2 or more adbapi
calls. I can provide some example code, but I should
really develop a concise example (as opposed to my app code,
which is spread over 2 or 3 modules) that might be useful
as the basis for a unit test in case this is a Twisted
problem -- it's not clear to me where the problem is:
my code, Twisted, or pyPgSQL. My natural assumption
would be the former, but the intermittency and the
adaptor-dependence makes me wonder.
I can run the exact same code several times without
incident and then suddenly get this traceback, which makes
me think it might be some allocated limit being exceeded
(threads in the thread pool? sorry, clueless here).
Some observations:
* all database operations in the deferred chain complete
successfully before the traceback occurs.
* it only occurs with pyPgSQL -- not with psycopg
(at least I haven't yet seen it with psycopg, and
believe me I've tried to make it happen! ... perhaps
psycopg's thread management prevents it?)
* this always occurs at the end of a "test" in which there
is a succeed(result) callback that prints the result and
calls reactor.stop() -- I wonder if stopping the reactor
(which wouldn't normally happen in an app, of course)
breaks the pipe?
* I'm using
- PostgreSQL 7.4.2
- pyPgSQL 2.4
- psycopg 1.1.15
- Twisted 1.3.0
- Python Python 2.3.3 (#1, Jul 22 2004, 15:58:51)
[GCC 3.3.2 20031022 (Red Hat Linux 3.3.2-1)] on linux2
- Fedora Core 1, 2.4.22-1.2197.nptl
Anyway, here's the traceback:
---------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.3/threading.py", line 436, in __bootstrap
self.run()
File "/usr/local/lib/python2.3/threading.py", line 416, in run
self.__target(*self.__args, **self.__kwargs)
--- <exception caught here> ---
File "/usr/local/lib/python2.3/site-packages/twisted/python/threadpool.py",
line 157, in _worker
context.call(ctx, function, *args, **kwargs)
File "/usr/local/lib/python2.3/site-packages/twisted/python/context.py", line
64, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/usr/local/lib/python2.3/site-packages/twisted/python/context.py", line
43, in callWithContext
return func(*args,**kw)
File "/usr/local/lib/python2.3/site-packages/twisted/internet/threads.py",
line 42, in _putResultInDeferred
reactor.callFromThread(deferred.callback, result)
File "/usr/local/lib/python2.3/site-packages/twisted/internet/base.py", line
201, in callFromThread
self.wakeUp()
File "/usr/local/lib/python2.3/site-packages/twisted/internet/base.py", line
207, in wakeUp
self.waker.wakeUp()
File "/usr/local/lib/python2.3/site-packages/twisted/internet/default.py",
line 399, in wakeUp
self.o.flush()
exceptions.IOError: [Errno 32] Broken pipe
---------------------------------------------------------------------
Has anyone else observed this?
Cheers,
Steve
More information about the Twisted-Python
mailing list