[Twisted-Python] custom FTP server is blocking

[Paul Wiseman]

On 19 March 2015 at 20:38, Louis D. Burr <ldanielburr at me.com> wrote:
> Hi Paul,
>
>> On Mar 19, 2015, at 1:18 PM, Paul Wiseman <poalman at gmail.com> wrote:
>>
>> I have some blocking going on in a custom twisted ftp server and I'm
>> trying to pin point what exactly is doing it.
>>
>> I think this line in openForReading on my FTPShell object could be the culprit:
>>
>> _FileReader(urllib2.urlopen(result.link,timeout=settings.GLOBAL_TIMEOUT))
>>
>
> Yeah, urllib2.urlopen() blocks, so this is at least one place in your code that will cause you a problem.
>
>> (_FileReader is twisted.protocols.ftp._FileReader and result.link a url string)
>>
>> I'm guessing urllib2.urlopen blocks resolving the domain and
>> connecting? What would be the twisted equivalent way of providing a
>> file like object of the download stream to the FileReader that
>> wouldn't block?
>>
>
> You probably want to use twisted.web.client.Agent for this, as it allows you to perform non-blocking http requests.  If you can live with buffering the http response body into, say, a tempfile or an io.BytesIO instance, then you can just pass that to _FileReader as the fObj argument and call it a day.

I want to stream the http request back to the client connected over
FTP. I guess I could do that with io.BytesIO right? I can't figure out
how to wire it up though. I've added agent.request which is firing
back a twisted.web.client.Response but I'm not sure how to stream the
data from that into the BytesIO object so that _FileReader has a file
like interface to read from.


>
> Hope this helps,
>
> --
> L. Daniel Burr
>
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python