[Twisted-Python] adbapi/postgresql and xml-rpc

James R. Saker Jr. jsaker at americanrelay.com
Wed May 12 15:52:34 EDT 2004

Have a design question to pass to the list. I've gotten my distributed
app working - pulling snort data from a postgresql database via
twisted's adbapi and handling via xml-rpc - though xml-rpc is screaming
about not being able to serialize postgresql's timestamp format. 

For instance, this query works (using a query test client and a "take
any SQL string in and process it" xmlrpc method on the server local to

# snipped python code setting things up. here's the xmlrpc call:
(answer,) = connection.remote("fetch", ('SELECT sid, cid, signature from
event LIMIT 5',))
print answer


[[1, 1, 1], [1, 2, 2], [1, 3, 1], [1, 4, 3], [1, 5, 1]]

(nice and easy to work with on client side)

but by adding the timestamp to the select query (SELECT sid, cid,
signature, timestamp...), I get the dreaded can't serialize output

Traceback (most recent call last):
  File "x8.py", line 59, in ?
    (answer,) = connection.remote("fetch", ('SELECT sid, cid, signature,
timestamp from event LIMIT 5',))
## some deleted
  File "/usr/lib/python2.3/xmlrpclib.py", line 742, in close
    raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 8002: "can't serialize output">

QUESTION: While I think I could build more precise queries on the server
side, e.g. xmlrpc_getAllSIDbyDate and massage timestamp and other
formats before dumping into xmlrpc, this might not be the most elegant
method. It makes every query something both ends have to be recoded for
- not just the client - simply because I can't get timestamp to pass
through XMLRPC. I've tried converting the tuples from the adbapi output
to strings before handing back but that has other issues.

Is this a limitation of XMLRPC that would merit using an approach like
PB instead? Eventually my text client will go the way of a Qt GUI and I
thought XMLRPC would make a nice lightweight distributed interface, but
the serialization issues are making me wonder if the approach is right.


More information about the Twisted-Python mailing list