[Twisted-Python] Server/Client bi-directional data-transfer - need a solution

[Maarten ter Huurne]

On Wednesday 12 December 2007, Sajit Rao wrote:

> *The Goal:*
>
>    - I'm trying to write a simple server/client app where the client can
>    send the server either a small string like "hello" OR a small file
> (txt, pdf, ppt,.. let's say < 5MB ).
>    - The server needs to figure out if the data it received is a simple
>    string or a file and symmetrically may decide to send back to the
> client a simple string or a small file.
>    - As a bonus, It's highly desirable that the transfer be secure - but
>    it's ok for the first version if it isn't

Would an HTTP POST suffice? You could easily test it using a web browser or 
a program like wget and adding a secure channel is done by switching from 
HTTP to HTTPS later.

If your application works like an instant messenger, consider using Twisted 
Words, for example with the XMPP (Jabber) protocol.

Does the client/server connection have to be able to pass a firewall or 
proxy? In that case HTTP or something that can be run over HTTP would be a 
big advantage.

> *Questions:*
>
> What's the cleanest way to accomplish this? I've been looking at the
> single Howto/Example that exists for *twisted.conch* but still can't
> understand several things. In particular
>
>    1. Should I create two separate channels between server and client -
>    one for simple strings and one for files so that the client or server
> knows the type of data on that channel?

That depends on what protocol you want to use over the channel(s). If the 
protocol can send series of text strings and binaries over the same channel 
then you need only one. But if having separate channels helps in 
simplifying the protocol, then that might be better.

One design decision to make is whether to keep the channel(s) open the whole 
time or open them when there is new data and then close them when the 
transfer is done. Or maybe keep the text string channel open all the time 
and open a file transfer channel on demand.

What kind of performance characteristics do you need? Does latency on the 
text strings matter? Are you going to transfer 10 files a day or 10000?

>    2. Should the files be mime-encoded at one end and mime-decoded at the
>    other so that file-type is automatically handled?

MIME encoding gives you a way to send the file name and type and it can be 
used over a text stream. However, the encoding of the data itself is not 
really needed if your channel is 8-bit clear and everything except e-mail 
is 8-bit clear nowadays (I think). For small files the overhead is neglible 
though.

>    3. I've read that there is a python package called paramiko that
>    implements ssh2 transfer - how does this compare with twisted.conch?

I don't know. One thing you could look at is which main loop you want to 
use. If you want to use Twisted's reactor, conch would better integrate 
with that of course. But if you want to use a GUI toolkit not supported by 
Twisted, it would take some effort to merge the main loops, so then running 
paramiko in a thread might be easier.

Another thing to look at is the license: paramiko is LGPL, which places more 
restrictions on its use than Twisted's MIT license. Depending on your 
situation that may or may not be important.

Bye,
		Maarten
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
Url : http://twistedmatrix.com/pipermail/twisted-python/attachments/20071212/2b28e8df/attachment.pgp