[Twisted-Python] XML-RPC for paralell clients

Pet petshmidt at googlemail.com
Tue May 19 10:02:58 EDT 2009


On Tue, May 19, 2009 at 11:48 AM, Pet <petshmidt at googlemail.com> wrote:
> On Tue, May 19, 2009 at 10:59 AM,  <glyph at divmod.com> wrote:
>> On 07:51 am, petshmidt at googlemail.com wrote:
>>>after playing with reactor, factory and protocol, I'm trying to
>>>implement XML-RPC server, but it seems that it is not able to handle
>>>several clients in parallel. If I call the slow function with first
>>>client  and then with second client a fast function, second client
>>>will receiver the results after slow function returned. There is no
>>>factory for clients in server.Site() like in Protokoll? What is the
>>>way to do that for XML-RPC?
>>
>> Whenever you perform some blocking work in a Twisted program, the
>> reactor will block and no processing will occur in parallel.  This is
>> not specific to XML-RPC.  server.Site() *is* a Factory for clients,
>> exactly the same as any other protocol.  You can see its place in the
>> inheritance hierarchy:
>
> Thank you for explanations!
>
> I still have some questions :(
> if server.Site() is Factory clients, what object is created for each
> connection?
> How can I ensure, that each client gets answer belonging to him and
> not for other "slow" client? How can I close connection to a client?

deferToThread or db.runInteraction seems to be the solution for me.
The "only" thing I don't know how to do is, logging requests *after*
response was send to the client. Obviously, it should happens after a
xmlrpc_method returns, but how can accomplish this?

Thanks for any help!

>
>
>>
>> http://twistedmatrix.com/documents/8.2.0/api/classIndex.html#twisted.web.server.Site
>>>    def xmlrpc_addslow(self, a, b):
>>>        """
>>>        Return sum of arguments. takes time
>>>        """
>>>        sleep(20)
>>>        return a + b
>>
>> The Twisted equivalent way to spell this is to return a Deferred which
>> fires later from your xmlrpc_addslow method:
>>
>>    from twisted.internet.task import deferLater
>>    from twisted.internet import reactor
>>    # ...
>>    def xmlrpc_addslow(self, a, b):
>>        return deferLater(reactor, 20, lambda : a + b)
>>
>> However, if your "sleep(20)" is intended to simulate 20 seconds of work
>> in a blocking API you can't control, you might want to return a Deferred
>> that does the work in a thread instead:
>>
>>    from time import sleep
>>    from twisted.internet.threads import deferToThread
>>    # ...
>>    def xmlrpc_addslow(self, a, b):
>>        def hardWork():
>>            sleep(20)
>>            return a + b
>>        return deferToThread(hardWork)
>
> this should be ok for me.
>
>>
>> As with all Python programs, if you truly want CPU parallelism, then you
>> will need to put the work into a subprocess instead.  You might want to
>> look at the "ampoule" process-pool project:
>> https://launchpad.net/ampoule
>>
>> For many applications though, you shouldn't need to worry about
>> parallelism this much.  Since Twisted's XMLRPC server support allows you
>> to replace any method's return value with a Deferred without changing
>> the externally observable behavior of XML-RPC server, you can add
>> parallelism incrementally as you actually require it.  In other words,
>> you probably don't need to worry about it as much as you think :).
>>
>> Hope this helped!
>>
>> _______________________________________________
>> Twisted-Python mailing list
>> Twisted-Python at twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>
>




More information about the Twisted-Python mailing list