[Twisted-Python] Bloody Twisted VFS

[Jonathan Lange]

On Sat, Jul 5, 2008 at 3:27 AM, Andy Gayton <andy at thecablelounge.com> wrote:
> On Mon, Jun 30, 2008 at 9:06 PM, Jonathan Lange <jml at mumak.net> wrote:
>> Yeah, that would be a good time for me, as long at it's after 2200 UTC.
>
> Hey Jono, sorry again that we didn't get the conversation online into irc.
>

That's ok. Thanks for bringing it to the mailing list :)

> Here's a quick recap of what came out of the discussions:
>
>  * The immediate goal is to settle on just enough of the core
> interface to be certain it can be expanded in the directions which
> will be needed in backwards compatible ways, so we can do a release.
>

Good idea.

>  * open should explicit state the flags and permission it takes.  It
> shouldn't use posix constants.  At first we'll just support a small
> subset that makes sense for all platforms.
>

+1

>  * createFile should be subsumed by open - the exclusive flag should
> just be apart of open's flags.
>

Having a convenience method for creating a file seems like a good idea.

>  * The primitive interface for IO should be producer/consumers,
> replacing readChunk, writeChunk.  This interface is primitive enough
> to express all other interfaces, while still providing the opportunity
> to optimize streaming performance.  The producer/consumer interface
> will need to take an offset to allow readChunk and writeChunk to be
> implemented.
>

It would be nice to have things so that readChunks and writeChunks
(plural) could be implemented, to avoid potato programming.

>  * we're still postponing handling of symlinks
>
>  * we're still using getMetadata and setMetadata - its likely we want
> a layer on top of using arbitrary key/value dicts for metadata, but
> this can be introduced in a backwards compatible way.
>

This reminds me, it would be good for VFS to have an exception for
"this operation isn't supported" (say with symlinks on fat32) and
another exception for "supportable, but not actually implemented yet".

>  * we still need to decide whether path resolution should be moved to
> a separate interface, instead of being part of the node's interface.
>

I'm not 100% sure what this means? Does this relate to possibly
combining with FilePath?

>  * there's concern over the package name.  twisted.tree has
> considerable support :)
>

I kind of like that. I'm not sure what the concern is with 'vfs' though.


> I'll try and make these changes in the next week or so.  If you are
> interested in shaping how this goes, you can track what's going on in
> http://twistedmatrix.com/trac/ticket/2815 - just weigh in once the
> ticket goes back to review.
>

Here's some random stuff that I wanted to at least mention:

- Error translation. This should translate the exception types, but it
should also translate values, so the error contains the virtual path.
- Deferreds. You don't mention them at all, but the lack of
asynchronous interfaces was one of the biggest problems we had with
twisted.vfs.
- URL Escaping. I got bitten by this recently. It's obviously not a
general VFS problem, but it's an issue with enough of them that it
should be considered when defining interfaces.
- "Decorators" like "read-only" and "chroot" could prove useful. Is
there room in the design for such things?

jml