[Twisted-web] Confused about XML-RPC Unicode behaviour (examples attached).

[Jean-Paul Calderone]

On Wed, 7 Feb 2007 20:41:41 +0200, kgi <iacovou at gmail.com> wrote:
>
>Hi all.
>
>I've got a bunch of Twisted XML-RPC servers that serve up a wide range of data
>types include dates. Up until now I've had some hacky code to protect against
>bad data types being returned to the serialization layer (Nones, unicodes,
>etc). However, this code turns out to be difficult to maintain, as well as
>being a modest performance hit.
>
>I thought I see if I couldn't clean up some of the code to make it simpler and
>less hacky, and I found something rather odd: a Twisted XML-RPC server
>seemingly can't return a xmlrpc.DateTime object it has only just received
>from the deserialization layer.
>
>I've attached a test client, using Twisted, and two test servers, one using
>Twisted, one using the standard library's SimpleXMLRPCServer.
>
>The standard library version succeeds in returning the received
>xmlrpclib.DateTime unmolested; the Twisted version complains about the
>message containing unicode:
>
>     File "/usr/lib/python2.4/site-packages/twisted/internet/abstract.py",line
>173, in write
>            raise TypeError("Data must not be unicode")
>        exceptions.TypeError: Data must not be unicode
>
>This seems wrong: surely, at the very least, the serialization and
>deserialization should allow a round-trip unmodified? Put a slightly
>different way: I don't have a fundamental problem with the restriction that
>return types not include unicode, but surely in that case, the same system
>that enforces this should see to it that I don't receive such a type in the
>first place (given that the original data type I created in the client was a
>xmlrpclib.DateTime, which is explicitly designed for serialising over
>XML-RPC!).
>
>Are there technical/historical/political reasons for this of which I know
>nothing?

Nope, it's probably just a bug.  It looks like it may be a bug in the
xmlrpclib module, though.  Consider this interaction:

exarkun at charm:~$ python
Python 2.4.3 (#2, Oct  6 2006, 07:52:30)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import xmlrpclib
>>> d1 = xmlrpclib.DateTime()
>>> d1
<DateTime '20070207T13:51:36' at -482e1234>
>>> s = xmlrpclib.dumps((d1,))
>>> s
'<params>\n<param>\n<value><dateTime.iso8601>20070207T13:51:36</dateTime.iso8601></value>\n</param>\n</params>\n'
>>> d2 = xmlrpclib.loads(s)[0][0]
>>> d2
<DateTime u'20070207T13:51:36' at -482ddab4>
>>> xmlrpclib.dumps((d2,))
u'<params>\n<param>\n<value><dateTime.iso8601>20070207T13:51:36</dateTime.iso8601></value>\n</param>\n</params>\n'
>>>                                                                                          

Twisted can probably work around the bug here, though.  It would be useful
if you could file a ticket in the tracker for this (one in Twisted's and
one in Python's).

Jean-Paul