[Twisted-Python] XML-RPC for paralell clients

[glyph at divmod.com]

On 07:51 am, petshmidt at googlemail.com wrote:
>after playing with reactor, factory and protocol, I'm trying to
>implement XML-RPC server, but it seems that it is not able to handle
>several clients in parallel. If I call the slow function with first
>client  and then with second client a fast function, second client
>will receiver the results after slow function returned. There is no
>factory for clients in server.Site() like in Protokoll? What is the
>way to do that for XML-RPC?

Whenever you perform some blocking work in a Twisted program, the 
reactor will block and no processing will occur in parallel.  This is 
not specific to XML-RPC.  server.Site() *is* a Factory for clients, 
exactly the same as any other protocol.  You can see its place in the 
inheritance hierarchy:

http://twistedmatrix.com/documents/8.2.0/api/classIndex.html#twisted.web.server.Site
>    def xmlrpc_addslow(self, a, b):
>        """
>        Return sum of arguments. takes time
>        """
>        sleep(20)
>        return a + b

The Twisted equivalent way to spell this is to return a Deferred which 
fires later from your xmlrpc_addslow method:

    from twisted.internet.task import deferLater
    from twisted.internet import reactor
    # ...
    def xmlrpc_addslow(self, a, b):
        return deferLater(reactor, 20, lambda : a + b)

However, if your "sleep(20)" is intended to simulate 20 seconds of work 
in a blocking API you can't control, you might want to return a Deferred 
that does the work in a thread instead:

    from time import sleep
    from twisted.internet.threads import deferToThread
    # ...
    def xmlrpc_addslow(self, a, b):
        def hardWork():
            sleep(20)
            return a + b
        return deferToThread(hardWork)

As with all Python programs, if you truly want CPU parallelism, then you 
will need to put the work into a subprocess instead.  You might want to 
look at the "ampoule" process-pool project: 
https://launchpad.net/ampoule

For many applications though, you shouldn't need to worry about 
parallelism this much.  Since Twisted's XMLRPC server support allows you 
to replace any method's return value with a Deferred without changing 
the externally observable behavior of XML-RPC server, you can add 
parallelism incrementally as you actually require it.  In other words, 
you probably don't need to worry about it as much as you think :).

Hope this helped!