[Twisted-Python] RE: How do I upload files using FTPClient?

[Andrew Bennetts]

On Mon, Jan 13, 2003 at 05:14:26PM -0800,  Daniel Burr wrote:
> On Wed, Jan 08, 2003 at 23:53:17 +1100 Andrew Bennetts wrote:
> 
> >> Yes, that's my fault.  I haven't gotten around
> >> to that yet -- I might have a stab at it
> >> tonight, it shouldn't actually be that hard.
> >> I'll try to make sure  I get this done in time
> >> for the 1.0.2 release.
> 
> That would be awesome; I'd be interested in contributing some code/tests/docs
> for the revised module.

Now *that* would be awesome! :)

> > Just letting you know I haven't forgotten about
> > this, but solving this properly really involves
> > refactoring the mess in FTPClient.retrieve...
> > and if I do that properly, it'll make my planned
> > rewrite of the FTP server alot easier, as it
> > ought to be able to re-use the code.
> > Unfortunately this means it might take a few
> > more days before this happens.
> >
> > -Andrew.
> 
> While you are still thinking about the solution, might I suggest something
> with regard to FTPClient ?

Of course!  And you'd be welcome to even if I wasn't thinking about it.

> I've been going over the existing python ftplib.FTP code, as well as
> twisted.protocols.ftp.FTPClient, and I'm left wondering one thing:  Why don't
> either of these classes implement the client commands in the manner that your
> typical command-line FTP client does?  I think FTPClient should have simple
> methods, like "binary", "get", "mget", "mkdir", et cetera.  From the
> developer's viewpoint, you'd just use FTPClient in the same manner as you
> would an interactive ftp client session; the actual queueing of commands,
> creation of Deferreds, and callbacks could be hidden away a little more easily
> then, and make life easier for developers just looking to roll a simple FTP
> client application using Twisted.

Indeed.  FTPClient is already almost this simple for simple FTP commands,
ones that don't require opening a data socket.  This is stuff like CWD, MKD,
etc.  The only unfriendly thing about it is it currently doesn't offer much
help with parsing the response, beyond stating that it is a success or a
failure (by calling your Deferred's callback or errback) -- but that's
probably all you care about.

There are a fair few of these I haven't wrapped up yet, for instance
"binary" (which is the "TYPE I" command, and I issue that automatically on
log in anyway -- is there ever any time when you *don't* want this?).

The problem is "get", which is not that simple, let alone stuff like mget
that has no one-to-one analogy with the raw protocol.  I don't think
FTPClient should be restricted to thinking everything is a file on disk --
you may want to process the data incrementally in another way [I certainly
did :) ].  Of course, most of the time, you probably want to just write it
to a file.

Thus I think I may need two layers; the current FTPClient is the low-level
plumbing that you don't interact with unless you need it's generality, and
then there's a higher-level interface that provides things like mget.  It's
entirely possible that I can fit this all into one class after all, but I'm
not sure yet.

So in summary, I want to allow for nifty/insane things like directly
funnelling data from e.g. an HTTP download into an FTP upload -- but I also
want a class that's straightforward to use in the common case as well.

What I'd really like from you, if you don't mind, is a sketch of what *you*
think the API should look like for the high-level interface.  Hopefully
that'll help me design this in a way that satisfies everyone :)

-Andrew.