[Twisted-Python] Bloody Twisted Tree (VFS)

Andy Gayton andy at thecablelounge.com
Mon Jul 7 21:02:06 MDT 2008


On Sun, Jul 6, 2008 at 11:12 AM,  <glyph at divmod.com> wrote:
> One related thing that we spoke about in person was pushing this negotiation
> of file-system features backwards to the initialization step, so that
> applications which needed unusual filesystem attributes could fail quickly
> with a clear error message if they weren't supported by the underlying
> platform. ("WebDAV requires extended filesystem attributes, and your
> backend, SFTP, does not provide that feature.", "txGnuStow requires symbolic
> links, and your backend, the Microsoft Windows filesystem, does not provide
> that feature.")

...

> One thing we didn't talk about in person: handling extremely large
> directories.  We had spoken about children() returning a Deferred of a list;
> I think it would be nice if it actually had a producer/consumer API of its
> own.  Maybe this is too much of a corner case to worry about in average
> applications (i.e. we could provide a give-me-a-deferred convenience API)
> but it would be nice if it were *possible* to implement things that were
> efficient against really big networked directories.

I really like these two ideas.  I've noted them down work on.  I think
we agree though that we could get a release out first, and then look
to add these?

> However, this reminds me about another concern which I did not remember to
> raise while Andy was here.  Should this really be twisted.<anything> at all?
>  I'd like twisted <x> "dot products" to generally be an application which
> does something <x>-ish.  I'm aware that not every package follows this rule,
> but the ones that don't are either (A) unmaintained and slated for removal,
> or (B) part of the core, not independent subprojects, as "vfs" seems slated
> to be.
>
> Put a different way: what should 'twistd tree' do?  My suggestion would be a
> simple multi-protocol file server: HTTP, FTP (although probably disable that
> by default), SFTP, maybe a "native" protocol for providing a generalized
> backend for any Twisted application that uses the 'tree' API, so that we can
> write a proxy that exposes every arbitrary combination of features from the
> protocols it's talking to.

I remember a chat I was having with you back in PyCon 2007, about
expanding twistd's command line interface to be able to multiplex
together several different services.

Having a collection of backend implementations available to use with
this would be cool.  But yeah, this could still be be provided without
using a twisted.<x> package.

Here's the vfs plugin so far:
http://twistedmatrix.com/trac/browser/branches/vfs-twistd-plugin-2821-2/twisted/vfs/plugin.py#L48
-- Options.longdesc talks about the above idea.  I'm ok either way on
this.

cheers,
Andy.




More information about the Twisted-Python mailing list