[Twisted-Python] RE: How do I upload files using FTPClient?

[Daniel Burr]

On Tue, 14 Jan 2003 at 13:21:54 +1100,  Andrew Bennetts wrote:
>On Mon, Jan 13, 2003 at 05:14:26PM -0800,  Daniel Burr wrote:
>> That would be awesome; I'd be interested in contributing some
code/tests/docs
>> for the revised module.
>
>Now *that* would be awesome! :)

Well, I think I grok the whole Deferred thing now,
and I'm poring over the Twisted developer docs for
information about contributing.  The only hurdle I
have to clear is getting up to speed with CVS.

It looks like the best place to start is writing
some tests for the proposed "high-level" ftp
interface.  That should help me to define it.

>> I've been going over the existing python ftplib.FTP code, as well as
>> twisted.protocols.ftp.FTPClient, and I'm left wondering one thing:  Why
don't
>> either of these classes implement the client commands in the manner that
your
>> typical command-line FTP client does?  I think FTPClient should have simple
>> methods, like "binary", "get", "mget", "mkdir", et cetera.  From the
>> developer's viewpoint, you'd just use FTPClient in the same manner as you
>> would an interactive ftp client session; the actual queueing of commands,
>> creation of Deferreds, and callbacks could be hidden away a little more
easily
>> then, and make life easier for developers just looking to roll a simple FTP
>> client application using Twisted.
>
>Indeed.  FTPClient is already almost this simple
>for simple FTP commands, ones that don't require
>opening a data socket.  This is stuff like CWD,
>MKD,etc.  The only unfriendly thing about it is
>it currently doesn't offer much help with parsing
>the response, beyond stating that it is a success
>or a failure (by calling your Deferred's callback
>or errback) -- but that's probably all you care
>about.

Yep, for those basic commands I think we are well
covered.

As for parsing the response, I noticed that the
effbot has some ftp-parsing code, I'll take a look
at that, and see if it contains any ideas.

>There are a fair few of these I haven't wrapped
>up yet, for instance "binary" (which is the "TYPE
>I" command, and I issue that automatically on log
>in anyway -- is there ever any time when you
>*don't* want this?).

Not that I can think of; the first thing I always
do in an interactive FTP session is to issue the
"binary" command.

>The problem is "get", which is not that simple,
>let alone stuff like mget that has no one-to-one
>analogy with the raw protocol.  I don't think
>FTPClient should be restricted to thinking
>everything is a file on disk -- you may want to
>process the data incrementally in another way [I
>certainly did :) ].  Of course, most of the time,
>you probably want to just write it to a file.

To be honest, I can't think of a single scenario
in which I wouldn't be using FTP to put a file
to disk or get a file from disk.  Granted, I am
not blessed with much in the way of imagination...
;)

>Thus I think I may need two layers; the current
>FTPClient is the low-level plumbing that you
>don't interact with unless you need it's
>generality, and then there's a higher-level
>interface that provides things like mget.  It's
>entirely possible that I can fit this all into
>one class after all, but I'm not sure yet.

I think putting it all into twisted.protocols.ftp
would suffice.  All in one class sounds harder.

>So in summary, I want to allow for nifty/insane
>things like directly funnelling data from e.g. an
>HTTP download into an FTP upload -- but I also
>want a class that's straightforward to use in the
>common case as well.

This sounds fine to me; obviously I'm really only
interested in the common case, but nifty/insane
stuff is cool too. ;)

>What I'd really like from you, if you don't mind,
>is a sketch of what *you* think the API should
>look like for the high-level interface.
>Hopefully that'll help me design this in a way
>that satisfies everyone :)
>
>-Andrew.

I think the API for the high-level client should
basically be identical to what you would use for
an interactive FTP session.  I'll do some thinking
on this and post a rough draft by the weekend.

L. Daniel Burr