[Twisted-Python] custom FTP server is blocking

[Louis D. Burr]

Hi Paul,

> On Mar 19, 2015, at 1:18 PM, Paul Wiseman <poalman at gmail.com> wrote:
> 
> I have some blocking going on in a custom twisted ftp server and I'm
> trying to pin point what exactly is doing it.
> 
> I think this line in openForReading on my FTPShell object could be the culprit:
> 
> _FileReader(urllib2.urlopen(result.link,timeout=settings.GLOBAL_TIMEOUT))
> 

Yeah, urllib2.urlopen() blocks, so this is at least one place in your code that will cause you a problem.

> (_FileReader is twisted.protocols.ftp._FileReader and result.link a url string)
> 
> I'm guessing urllib2.urlopen blocks resolving the domain and
> connecting? What would be the twisted equivalent way of providing a
> file like object of the download stream to the FileReader that
> wouldn't block?
> 

You probably want to use twisted.web.client.Agent for this, as it allows you to perform non-blocking http requests.  If you can live with buffering the http response body into, say, a tempfile or an io.BytesIO instance, then you can just pass that to _FileReader as the fObj argument and call it a day.


Hope this helps,

--
L. Daniel Burr