[Twisted-Python] Server/Client bi-directional data-transfer - need a solution

[Sajit Rao]

Thanks for your reply, Maarten.

I'm still chewing over it. In answer to your questions:

I will be sending short strings between client and server (bi-directionally)
most of the time (> 100 times a day) and ocassionaly files (each of size 5
MB or less and > 10 but <100 times a day).

I wouldn't mind sending strings or files along the same channel - but I
don't know how to do that

While I like the brevity of python and realize that Twisted deffereds are
the heart of the async model,  I'm hampered by the scant documentation and
code fragments.

Could someone point me to how to send raw strings or binary files over the
same channel and have the other side interpret the data correctly?

Thanks,

Sajit

On Dec 12, 2007 5:20 AM, Maarten ter Huurne <maarten at treewalker.org> wrote:

> On Wednesday 12 December 2007, Sajit Rao wrote:
>
> > *The Goal:*
> >
> >    - I'm trying to write a simple server/client app where the client can
> >    send the server either a small string like "hello" OR a small file
> > (txt, pdf, ppt,.. let's say < 5MB ).
> >    - The server needs to figure out if the data it received is a simple
> >    string or a file and symmetrically may decide to send back to the
> > client a simple string or a small file.
> >    - As a bonus, It's highly desirable that the transfer be secure - but
> >    it's ok for the first version if it isn't
>
> Would an HTTP POST suffice? You could easily test it using a web browser
> or
> a program like wget and adding a secure channel is done by switching from
> HTTP to HTTPS later.
>
> If your application works like an instant messenger, consider using
> Twisted
> Words, for example with the XMPP (Jabber) protocol.
>
> Does the client/server connection have to be able to pass a firewall or
> proxy? In that case HTTP or something that can be run over HTTP would be a
> big advantage.
>
> > *Questions:*
> >
> > What's the cleanest way to accomplish this? I've been looking at the
> > single Howto/Example that exists for *twisted.conch* but still can't
> > understand several things. In particular
> >
> >    1. Should I create two separate channels between server and client -
> >    one for simple strings and one for files so that the client or server
> > knows the type of data on that channel?
>
> That depends on what protocol you want to use over the channel(s). If the
> protocol can send series of text strings and binaries over the same
> channel
> then you need only one. But if having separate channels helps in
> simplifying the protocol, then that might be better.
>
> One design decision to make is whether to keep the channel(s) open the
> whole
> time or open them when there is new data and then close them when the
> transfer is done. Or maybe keep the text string channel open all the time
> and open a file transfer channel on demand.
>
> What kind of performance characteristics do you need? Does latency on the
> text strings matter? Are you going to transfer 10 files a day or 10000?
>
> >    2. Should the files be mime-encoded at one end and mime-decoded at
> the
> >    other so that file-type is automatically handled?
>
> MIME encoding gives you a way to send the file name and type and it can be
> used over a text stream. However, the encoding of the data itself is not
> really needed if your channel is 8-bit clear and everything except e-mail
> is 8-bit clear nowadays (I think). For small files the overhead is
> neglible
> though.
>
> >    3. I've read that there is a python package called paramiko that
> >    implements ssh2 transfer - how does this compare with twisted.conch?
>
> I don't know. One thing you could look at is which main loop you want to
> use. If you want to use Twisted's reactor, conch would better integrate
> with that of course. But if you want to use a GUI toolkit not supported by
> Twisted, it would take some effort to merge the main loops, so then
> running
> paramiko in a thread might be easier.
>
> Another thing to look at is the license: paramiko is LGPL, which places
> more
> restrictions on its use than Twisted's MIT license. Depending on your
> situation that may or may not be important.
>
> Bye,
>                Maarten
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20071218/49733973/attachment.htm