[Twisted-Python] Large file transfers

[Glyph Lefkowitz]

On Thu, 25 Jul 2002 16:43:33 -0400, Steve Waterbury <steve.waterbury at gsfc.nasa.gov> wrote:
> Glyph Lefkowitz wrote:
> 
> > There will be a little bit of overhead in the initial implementation,
> > considering that it will be using PB calls.

As several people have pointed out to me, sometimes this additional overhead is
worth paying in order to get rid of the TCP connection startup cost.  Of
course, this will vary with your application, but it's probably worthwhile to
have the ability to reuse the connection.

In addition, sometimes you want to communicate large *objects*, and not just
large *files*; in those cases, it's really handy to have objects which can be
"paged" between systems, interleaved with other messages.

This is also a decent basis for "file transfer" between two objects, since
"very large string, stored on disk" is a degenerate case of "very large object,
stored somewhere".  I've checked in an implementation of this to
twisted.spread.util.Pager.  Not much in the way of docs yet, but the test cases
(twisted.test.test_pb, look for "pager") should at least explain some of it.

> > However, many of us are fans of Lisp-like languages and I'm sure that more than
> > one person would be thrilled to see a multi-vendor-supporting CL
> > implementation of PB.

> Yeah, the non-blocking sockets might be a problem ... I'll ask about 
> that.  If we do an implementation, we'll at least make an effort to 
> make it multi-vendor-supporting.  Our Lisp specialists have used both 
> Allegro and Harlequin at various times.  Let us know if other vendor 
> or open-source Lisps are of interest, etc.  

I can speak only to my own interests.  Other members of the Twisted community
are working in radically different environments and on radically different
problems.  As long as you asked, though ...

While I appreciate that they're technically superior, the proprietary lisps
don't interest me; the bang for your buck of using a proprietary language
solution is seldom worth the hefty license fees (especially in a situations
where "interoperability" and "ubiquity" are the most interesting parts, like
most of the applications I've come up with for Twisted).

Conversely, support for *any* open-source CL would be really cool.  Even GCL
would be fun to play around with.  I'd like to develop Lisp components for some
of the applications I'm working on, mostly to tinker with it for fun, but I
don't have enough time or inclincation to dive into the implementation myself.

-- 
 |    <`'>    |  Glyph Lefkowitz: Traveling Sorcerer   |
 |   < _/ >   |  Lead Developer,  the Twisted project  |
 |  < ___/ >  |      http://www.twistedmatrix.com      |
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://twistedmatrix.com/pipermail/twisted-python/attachments/20020726/4b83e880/attachment.pgp