[Twisted-Python] Best way to trigger a future connection with data

[Nick Johnson]

Here's a cut-down version of the code which might be more illustrative:


class MyProtocol(Protocol):
    def __init__(self, s, d):
	<some init>

    def dataReceived(self, data):
	<do stuff with data>

    def connectionLost(self, reason):
        reactor.callLater(10, ...)

    def connectionMade(self):
        self.transport.write(...)
        self.transport.loseWriteConnection()

class MyFactory(ClientFactory):
    def __init__(self, src, dst, interval, type_req):
        self.s = src
        self.d = dst

    def buildProtocol(self, addr):
        p = MyProtocol(self.s, self.d)
        p.factory = self
        return p

if __name__ == '__main__':
    f = MyFactory("10", "20", 1, 1)
    l = task.LoopingCall(reactor.connectTCP ... f)
    l.start(.1)
    task.callLater(20, reactor.stop())
    reactor.run()

So, each call from task.LoopingCall sets up a new connection which then
starts a job. When that connection has finished, the protocol instance
disappears. I can store the data it received in a structure in the
Factory, no problems there.

I have to call transport.loseWriteConnection() in order to get data from
the server (I've no control over this).

The problem comes when the delayed connection is started. This will (in
my mind) create a new instance of MyProtocol by calling the
buildProtocol method of the Factory. Without any additional input, it
wont know what to do, start a new job or retrieve one from the server.
I've tried thinking about callbacks and deferreds but still get stuck
with the same problem of how to instruct a particular instance of
MyProtocol to either launch or retrieve a job.


Cheers,
-Nick.


On 19/07/13 09:25, Laurens Van Houtven wrote:
> On Fri, Jul 19, 2013 at 10:19 AM, Nick Johnson
> <Nick.Johnson at ed.ac.uk <mailto:Nick.Johnson at ed.ac.uk>> wrote:
> 
> Thanks lvh,
> 
> I did have to override the buildProtocol method in the Factory but I 
> then set Protocol.factory to be equal to the Factory (ie, 
> myprotocol.factory=self).
> 
> 
> You could (perhaps should) do this by calling 
> ClientFactory.buildProtocol(self, addr).
> 
> 
> I'm still stuck however with what to do when I get more complex than 
> this simple case. For example, I use a callingLoop to call multiple 
> connections with a 0.1 second interval to launch jobs and each of
> those connections does as mentioned by setting up a future connection
> to retrieve the output, say 10 seconds later. There is going to be
> some overlap, ie I might have launched 100 new jobs before the first
> one fires it's task.callLater.
> 
> Storing state in the factory class doesn't work in this case because 
> each new connection wont know whether to initiate a job or retrieve 
> output as I cannot pass it this extra information.
> 
> 
> I don't understand why not. Could you elaborate?
> 
> Either way, it seems to me that the API should be:
> 
> d = scheduleJob() d.addCallback(getJob)
> 
> That is: only get the job once it has been scheduled. getJob would 
> probably have to be split up into something that delays (consider 
> delayLater), and something that actually gets the job.
> 
> cheers lvh

-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.