[Twisted-Python] FTP without the protocol

[David Bolen]

Lloyd Carothers <lloyd at passcal.nmt.edu> writes:

>  For such a use case I'm surprised not to find an example. I think I
> just need a push in the right direction. Is producers/consumers the
> right approach?

It's a bit dated at this point, but maybe this might spark some ideas:
   http://twistedmatrix.com/pipermail/twisted-python/2007-July/015738.html

It's producer/consumer, which yes, is very efficient for streaming
transfers.  The code the post was based on is actually still in active
use, but against an older twisted 2.5.0 installation, so I'm not sure
how much tweaking it may need to adjust to the latest Twisted version.

In practice this is paired with a separate set of code that implements a
PB-based control channel over which the files to upload are negotiated.
The binary transfer itself just has a small header in front of the data
containing some authentication and size information, so it's derived
from LineReceiver and switches to raw mode for the transfer.  While I
tend to prefer a separate control channel (though it certainly doesn't
need to be PB based), you could also in-line the control information (so
it becomes more like an HTTP transfer) if you wished.

BTW, passive FTP should be quite firewall friendly, unless you're
talking about really locked down locations where the only thing allowed
out is HTTP or something.  But if you're not using passive mode yet,
that might also be a quicker fix to your existing code base.

-- David