Creating XML-RPC Servers and Clients with Twisted

  1. Introduction
  2. Creating a XML-RPC server
  3. SOAP Support
  4. Creating an XML-RPC Client
  5. XML-RPC and Authentication
  6. Migrating twisted.web XML-RPC to twisted.web2

Introduction

XML-RPC is a simple request/reply protocol that runs over HTTP. It is simple, easy to implement and supported by most programming languages. Twisted's XML-RPC support is implemented using the xmlrpclib library that is included with Python 2.2 and later.

This document assumes some familiarity with cred and suggests you read the twisted.web2 howtos as well as Using the Twisted Application Framework

Creating a XML-RPC server

Making a server is very easy - all you need to do is inherit from twisted.web2.xmlrpc.XMLRPC. You then create methods beginning with xmlrpc_. The methods' arguments determine what arguments it will accept from XML-RPC clients. The result is what will be returned to the clients.

Methods published via XML-RPC can return all the basic XML-RPC types, such as strings, lists and so on (just return a regular python integer, etc). They can also return Failure instances to indicate an error has occurred, or Binary, Boolean or DateTime instances (all of these are the same as the respective classes in xmlrpclib. In addition, XML-RPC published methods can return Deferred instances whose results are one of the above. This allows you to return results that can't be calculated immediately, such as database queries. See the Deferred documentation for more details.

XMLRPC instances are Resource objects, and they can thus be published using a Site. The following example has two methods published via XML-RPC, add(a, b) and echo(x).

from twisted.web2 import xmlrpc
from twisted.web2 import server
from twisted.web2 import channel
from twisted.application import service, strports

class Example(xmlrpc.XMLRPC):
    """An example object to be published."""

    addSlash = True

    def xmlrpc_echo(self, request, x):
        """Return all passed args."""
        return x

    def xmlrpc_add(self, request, a, b):
        """Return sum of arguments."""
        return a + b

if __name__ == '__main__':
    from twisted.internet import reactor
    site = server.Site(Example())
    reactor.listenTCP(7080, channel.HTTPFactory(site))
    reactor.run()

We could run example1.py in the background like so, python example1.py & and then connect with client code in the following manner:

>>> import xmlrpclib
>>> s = xmlrpclib.Server('http://localhost:7080/')
>>> s.echo('A self-perpetuating autocracy in which the working class...')
'A self-perpetuating autocracy in which the working class...'
>>> s.add(1, 2)
3

If you are coming from twisted.web.xmlrpc, then you probably noticed two differences: the use of addSlash and that request appears in the signature for methods.

To run this example as a twisted application, save the following as example1.tac:

from twisted.web2 import xmlrpc
from twisted.web2 import server
from twisted.web2 import channel
from twisted.application import service, strports

class Example(xmlrpc.XMLRPC):
    """An example object to be published."""

    addSlash = True

    def xmlrpc_echo(self, request, x):
        """Return all passed args."""
        return x

    def xmlrpc_add(self, request, a, b):
        """Return sum of arguments."""
        return a + b

site = server.Site(Example())
application = service.Application("Example XML-RPC Server")
s = strports.service('tcp:7080', channel.HTTPFactory(site))
s.setServiceParent(application)

Then execute with the command twistd -noy example.tac.

XML-RPC resources can be seamlessly integrated with other twisted.web2 resources. Here is an example of a web2 server running a simple HTTP resource as well as an XML-RPC resource:

from random import choice

from twisted.web2 import http
from twisted.web2 import server
from twisted.web2 import channel
from twisted.web2 import resource
from twisted.web2 import xmlrpc
from twisted.application import service, strports

def getQuote():
    quotes = [
        'The concept of progress acts as a protective mechanism ' +
            'to shield us from the terrors of the future.',
        'Truth suffers from too much analysis',
        'A process cannot be understood by stopping it.',
    ]
    return choice(quotes)

class Quoter(xmlrpc.XMLRPC):

    def xmlrpc_quote(self, request):
        return getQuote()

class Toplevel(resource.Resource):
  addSlash = True
  child_RPC2 = Quoter()
  def render(self, ctx):
        return http.Response(stream="Hello monkey!")

site = server.Site(Toplevel())
application = service.Application("Quote Server+")
s = strports.service('tcp:8080', channel.HTTPFactory(site))
s.setServiceParent(application)

Not only can one browse to http://localhost:8080, but one can also access the XML-RPC service available at http://localhost:8080/RPC2.

Using XML-RPC sub-handlers

XML-RPC resource can be nested so that one handler calls another if a method with a given prefix is called. This can be a very convenient mechanism for organizing RPC APIs. For example, we can add support for an XML-RPC method date.time() to the Example class, and move the add() method to math.add():

import time

from twisted.web2 import server
from twisted.web2 import channel
from twisted.web2 import xmlrpc
from twisted.application import service, strports

class Example(xmlrpc.XMLRPC):

    addSlash = True

    """An example object to be published."""
    def xmlrpc_echo(self, request, x):
        """Return all passed args."""
        return x

class Math(xmlrpc.XMLRPC):
    """Put our math methods here."""

    def xmlrpc_add(self, request, a, b):
        """Return sum of arguments."""
        return a + b

class Date(xmlrpc.XMLRPC):
    """Put our date methods here."""

    def xmlrpc_time(self, request):
        """Return UNIX time."""
        return time.time()

root = Example()
root.putSubHandler('math', Math())
root.putSubHandler('date', Date())

site = server.Site(root)
application = service.Application("XML-RPC Server With Sub-handlers")
s = strports.service('tcp:8080', channel.HTTPFactory(site))
s.setServiceParent(application)

By default, a period ('.') separates the prefix from the method name, but you can use a different character by overriding the XMLRPC.separator data member in your base XML-RPC server. XML-RPC servers may be nested to arbitrary depths using this method.

Running client code from the python interpreter, we can see the subhandlers in action:

>>> import xmlrpclib
>>> s = xmlrpclib.ServerProxy('http://127.0.0.1:8080/')
>>> s.echo('Oh, there you go bringing class into it again.')
'Oh, there you go bringing class into it again.'
>>> s.math.add(1,2)
3
>>> s.date.time()
1151921139.9897649

Adding XML-RPC Introspection support

XML-RPC has an informal Introspection API that specifies three methods in a system sub-handler which allow a client to query a server about the server's API. Adding Introspection support to the Example class is easy using the XMLRPCIntrospection class:

from twisted.web2 import server
from twisted.web2 import channel
from twisted.web2 import xmlrpc
from twisted.application import service, strports

class Example(xmlrpc.XMLRPC):
    """An example object to be published."""

    addSlash = True

    def xmlrpc_echo(self, request, x):
        """Return all passed args."""
        return x

    xmlrpc_echo.signature = [['string', 'string'],
                             ['int', 'int'],
                             ['double', 'double'],
                             ['array', 'array'],
                             ['struct', 'struct']]

    def xmlrpc_add(self, request, a, b):
        """Return sum of arguments."""
        return a + b

    xmlrpc_add.signature = [['int', 'int', 'int'],
                            ['double', 'double', 'double']]
    xmlrpc_add.help = "Add the arguments and return the sum."

root = Example()
xmlrpc.addIntrospection(root)

site = server.Site(root)
application = service.Application("XML-RPC Server With Sub-handlers")
s = strports.service('tcp:8080', channel.HTTPFactory(site))
s.setServiceParent(application)

Note the method attributes help and signature which are used by the Introspection API methods system.methodHelp and system.methodSignature respectively. If no help attribute is specified, the method's documentation string is used instead.

We can see how introspection can be used via this interactive python session:

>>> import xmlrpclib
>>> s = xmlrpclib.ServerProxy('http://127.0.0.1:8080/')
>>> s.system.listMethods()
['add', 'echo', 'system.methodHelp', 'system.listMethods', 'system.methodSignature']
>>> s.system.methodHelp('echo')
'Return all passed args.'
>>> s.system.methodSignature('echo')
[['string', 'string'], ['int', 'int'], ['double', 'double'], ['array', 'array'], ['struct', 'struct']]
>>> s.system.methodHelp('add')
'Add the arguments and return the sum.'
>>> s.system.methodSignature('add')
[['int', 'int', 'int'], ['double', 'double', 'double']]

SOAP Support

We are not currently supporting SOAP. However, there have been Twisted SOAP efforts made in the Zolera SOAP Infrastructure (ZSI) project.

Creating an XML-RPC Client

XML-RPC clients in Twisted are meant to look like something which will be familiar either to xmlrpclib or to Perspective Broker users, taking features from both, as appropriate. There are two major deviations from the xmlrpclib way which should be noted:

  1. No implicit /RPC2. If the services uses this path for the XML-RPC calls, then it will have to be given explicitly.
  2. No magic __getattr__: calls must be made by an explicit callRemote.

The interface Twisted presents to XML-RPC client is that of a proxy object: twisted.web.xmlrpc.Proxy. The constructor for the object receives a URL: it must be an HTTP or HTTPS URL. When an XML-RPC service is described, the URL to that service will be given there.

Having a proxy object, one can just call the callRemote method, which accepts a method name and a variable argument list (but no named arguments, as these are not supported by XML-RPC). It returns a deferred, which will be called back with the result. If there is any error, at any level, the errback will be called. The exception will be the relevant Twisted error in the case of a problem with the underlying connection (for example, a timeout), IOError containing the status and message in the case of a non-200 status or a xmlrpclib.Fault in the case of an XML-RPC level problem.

from twisted.web.xmlrpc import Proxy
from twisted.internet import reactor

def printValue(value):
    print repr(value)
    reactor.stop()

def printError(error):
    print 'error', error
    reactor.stop()

proxy = Proxy('http://advogato.org/XMLRPC')
proxy.callRemote('test.sumprod', 3, 5).addCallbacks(printValue, printError)
reactor.run()

prints:

[8, 15]

XML-RPC and Authentication

This section is based on the twisted.web2 Authentication Demo. Please keep in mind the caveats at the end of that howto, as the same conditions apply here.

Listed below is a working example from the above-mentioned demo. The important change made here is the substitution of a resource.Resouce instance with an xmlrpc.XMLRPC instance:

from zope.interface import Interface, implements

from twisted.cred import portal
from twisted.cred import checkers

from twisted.web2 import channel
from twisted.web2 import http
from twisted.web2 import responsecode
from twisted.web2 import server
from twisted.web2 import xmlrpc

from twisted.web2.auth import digest
from twisted.web2.auth import basic
from twisted.web2.auth import wrapper

from twisted.application import service, strports

class IHTTPUser(Interface):
    pass

class HTTPUser(object):
    implements(IHTTPUser)

class HTTPAuthRealm(object):
    implements(portal.IRealm)

    def requestAvatar(self, avatarId, mind, *interfaces):
        if IHTTPUser in interfaces:
            return IHTTPUser, HTTPUser()

        raise NotImplementedError("Only IHTTPUser interface is supported")

class Example(xmlrpc.XMLRPC):
    """An example object to be published."""

    addSlash = True

    def xmlrpc_echo(self, request, x):
        """Return all passed args."""
        return x

    def xmlrpc_add(self, request, a, b):
        """Return sum of arguments."""
        return a + b

portal = portal.Portal(HTTPAuthRealm())
checker = checkers.InMemoryUsernamePasswordDatabaseDontUse(guest='guest123')
portal.registerChecker(checker)
rsrc = Example()
credFactories = (basic.BasicCredentialFactory('My Realm'),
    digest.DigestCredentialFactory('md5', 'My Realm'))
ifaces = (IHTTPUser,)
root = wrapper.HTTPAuthResource(rsrc, credFactories, portal, ifaces)

site = server.Site(root)
application = service.Application("XML-RPC Auth Demo")
s = strports.service('tcp:8080', channel.HTTPFactory(site))
s.setServiceParent(application)

Accessing this from an interactive python session, we can see the inability to call the RPC methods unless we have supplied the correct name and password:

>>> from xmlrpclib import ServerProxy
>>> s = ServerProxy('http://localhost:8080/')
>>> s.echo('There are some who call me... Tim?')
Traceback (most recent call last):
  [snipped]
>>> s = ServerProxy('http://guest:guest123@localhost:8080/')
>>> s.echo('There are some who call me... Tim?')
'There are some who call me... Tim?'

Migrating twisted.web XML-RPC to twisted.web2

Migrating your XML-RPC servers written with twisted.web.xmlrpc to ones written with twisted.web2.xmlrpc should be fairly trivial. There are three things you you need to do in order to migrate:

These are detailed below.

Add addSlash

Each of your xmlrpc.XMLRPC classes needs to set the class attribute addSlash = True. All of the examples in this howto demonstrate this.

Add request to the Method Signatures

When defining xmlrpc_* methods in twisted.web.xmlrpc, you only needed to pass the self parameter. In twisted.web2.xmlrpc, you need to pass two parameters, at the very minimum: self and request. This is also demonstrated clearly in the examples of this howto.

Wrap the Site Object

In twisted.web.xmlrpc, a Site instance is passed to strports.service or TCPServer (in twisted.application.internet). In twisted.web2.xmlrpc, the Site instance is passed to HTTPFactory and then the factory is passed in strports.service or internet.TCPServer. As with the previous two, the examples in this howto show this clearly.

Index

Version: