[Twisted-Python] Re: Using deferreds when writing across unreliable network

[Martin Geisler]

"John Aherne" <johnaherne at rocs.co.uk> writes:

> Martin
>
> Thanks for the reply.
>
> That does clarify that I need to fire off my own deferreds.

Excellent! :-)

> The problem it leaves me at the moment is:
>
> How do I tell when my transport.write has completed. The ACKs for my
> messages happen at the TCP level. Does this mean that for writes and
> reads I have to work at the application level to register when the
> write has been successful and fire my callback then.

I think so -- you wont be told when the actual bytes have been pushed
onto the network, nor when they have arrived succesfully at their
destination. You have to add such logic yourself, maybe by having the
recepient send back a confirmation.

So one way to structure this is to let the code that calls
transport.write return a Deferred, call it d. Before returning it, the
code stores d in a well-known place, something like

  transport.write(...)
  d = Deferred()
  outstanding requests[some_id] = d
  return d

The code that handles incoming data will then have to recognize the
confirmation and will invoke the callback method on d:

  def dataReceived(self, data):
      id, payload = unpack(data)
      d = outstanding_requests[id]
      d.callback(payload)

This is sort of how my code works in the VIFF project.

> By the way I did find your example, but I could not reconcile why
> some deferred examples had 'automatic' callbacks and other fired
> their own.

That is because of the different ways they are used: Defered is the
basic building block, but DeferredList combines several Deferreds into
one.

-- 
Martin Geisler

VIFF (Virtual Ideal Functionality Framework) brings easy and efficient
SMPC (Secure Multiparty Computation) to Python. See: http://viff.dk/.