[Twisted-Python] Recursive filescanning in Twisted??

Jeff Bowden jlb at houseofdistraction.com
Thu Sep 9 13:23:27 EDT 2004

On Linux, file descriptors for disk files are always "ready".  Since 
twisted is based on select() or poll() I'm guessing you will end up 
actually blocking on file reads.  I think you can get non-blocking file 
reads using the new AIO stuff but it's not really a good match for a 
single process server like twisted. 

I don't know about BSD/OSX or Windows.

I would love to be wrong about this, BTW, so if anyone has better info 
I'd like to here it.

Thomas Weholt wrote:

>I want to run a web-server which displays information about files on
>my filesystems. The info will be stored in either a mysql or sqlite
>database. The number of files to be scanned can be many and huge in
>size. I want to click on a link which goes to a specified
>page/resource on the server which starts the filescanning, generates
>checksums for the files processed ( sha/md5 ) and stores this info in
>the database. The process cannot block the server, at least not the
>filescanning itself. The database interactions is not that critical.
>It would be nice to have some kind of indication available for users
>to see that a filescan was in progress, perhaps even some kind of
>progress indication.
>Can anybody show me how to do a non-blocking filescan like I've
>described in a Twisted-based web-application which generates
>sha/md5-checkums in the process?

More information about the Twisted-Python mailing list