[Twisted-Python] Re: How can ftpclient.py work?
tv at twistedmatrix.com
Sun Jul 27 04:34:00 EDT 2003
On Sat, Jul 26, 2003 at 10:34:54PM -0500, John Goerzen wrote:
> But, wouldn't the above code have blocking problems, especially over
> slow links or for large values of data? (Say data is a 2MB string and
> you're on a modem... does your UI freeze for 15 minutes? Or is
> write() another layer above the system call that does some sort of magic?)
Well, the UI won't really freeze; self.transport.write()
will just buffer that 2MB inside the transport, and send
from the buffer whenever the socket is ready.
Of course, 2MB is quite a lot to buffer, and 100MB would
be insane in a "normal" application. That's why, when you have
lots of data, you use the producer/consumer pattern. See
twisted.internet.interfaces classes IConsumer, IProducer and
friends; though they are quite rarely used in normal apps,
and you won't likely ever need them.
> I guess what I'm thinking here is that switching from multi-threaded
> to poll() or select() isn't a "get out of jail free" card. And while
There's no such thing as "easy complex". That's why protocol and
application designers should avoid design decision that require
complex code to support.
However, for the easy cases, it goes something like this:
1. simplest: single process, single protocol chat
2. quite easy: asynchronous single process, many protocol chats
3. starts as easy and explodes later: threads, many protocol chats
Asynchronous programming removes most the time-related
variance in program behaviour; that is, it helps you write
more predictable code (very few race conditions compared to
threads). And when you get used to it, the state machine
thinking is just nice.
> I've written various C programs using select() and have a good idea
> what the potential pitfalls are there, Twisted seems to live up to its
> name just enough that I'm having a bit of trouble isolating them here :-)
Twisted actually removes most of the pitfalls you fight with
when doing your own event loop from scratch; that's the main
reason to use Twisted for me. Of course, the bundled protocols
etc. are nice, too.
> >> Now, I'm assuming that these problems do not actually exist. Yet I
> >> cannot work out why not. Can anybody shed some light?
> > I hope I've shed some light for you -- let us know if you need more :)
> Thanks -- all I need now is a small flashlight :-)
We could always get some rags and lighter fluid, and make torches
from our cluesticks ;)
/me notes he writes too much drivel. Blame it one the sunday morning.
More information about the Twisted-Python