[Twisted-Python] xmlrpc deferred

Jp Calderone exarkun at divmod.com
Fri May 27 16:11:47 EDT 2005


On Fri, 27 May 2005 22:56:49 +0300, Catalin Constantin <catalin at dazoot.ro> wrote:
>Friday, May 27, 2005, 10:46:21 PM, Jp Calderone wrote:
>> On Fri, 27 May 2005 22:28:06 +0300, Catalin Constantin <catalin at dazoot.ro> wrote:
>>>Hi there,
>>>
>>>I have the following xmlrpc method:
>>>
>>>class FeederResource(xmlrpc.XMLRPC):
>>>        def __init__(self):
>>>                xmlrpc.XMLRPC.__init__(self)
>>>                self.feeder=Feeder()
>>>
>>>        def xmlrpc_getList(self, id):
>>>                return self.feeder.get_urls(id)
>>>
>>>The thing is that the self.feeder.get_urls takes too long to execute
>>>and while the request is running all the others are blocked.
>>>I want that while it computes the result the other XML RPC methods to
>>>be available.
>
>> The only answer here is to make get_urls() take less time.
>
>I can't :)
>
>> What is it doing?  Is it blocking on network I/O?  Querying a
>> database?  Prompting for user input?   _It_ should be creating and
>> returned a Deferred (and later calling it back with a result), most
>> likely, since it is the long-running operation.
>
>Mainly, database queries.

If the database doesn't provide an async API, threads may be the only reasonable solution.  If the database adapter is a DB-API 2.0 module, this is already available for you: twisted.enterprise.adbapi.  Otherwise, you may need to use the "low level" Twisted threading API, twisted.internet.threads.deferToThread(), or even reactor.callInThread() and reactor.callFromThread(), depending on your particular needs.

> [snip]
>
>What bothers me is that while the call to getList is not done all the
>other functionality of the app is on hold.
>

Yep.  Any time you have a function that takes a long time to run, this is going to happen.  The reactor needs to get execution time to process events, and it can't get that if application code (ie, your code) doesn't return quickly.

Jp




More information about the Twisted-Python mailing list