[Twisted-Python] Question on push/pull producers inter-workings, was : "Is there a simple Producer/Consumer example or tutorial?"
exarkun at divmod.com
Sat Apr 19 09:33:55 EDT 2008
On Fri, 18 Apr 2008 09:57:35 +0200, Gabriel Rossetti <mailing_lists at evotex.ch> wrote:
>I am re-posting these questions with a different title, since they have
>ventured away from the original question
>I have a question though, I was looking at the way all this works by using a
>debugger, and I noticed that in twisted.internet.abstract.py in
>registerProducer() there is the following :
>if not streaming:
>Why is this done only for the pull producer? Shouldn't it also be called for
>the push producer since to have the data sent one has to call either
>self.transport.write() or resumeProducing() anyways? If you look at :
>it does :
So, this is a streaming producer. It starts producing right away without
having resumeProducing called on it.
>thus starting the writing process and in the pull producer :
>it doesn't need to start the writing process explicitly since it's started
>when the producer is registered.
This is a non-streaming producer. It doesn't do anything until something
calls resumeProducing on it.
>Oh, and also, since from what I see in the code the only difference with a
>push and pull producer is that the push producer is paused if the data being
>written/sent is very large (to let the reactor breath and process other
>things), if it isn't larger than the buffer it behaves like a pull producer,
>correct? If so, then why have both? Am I mixed up again?
As you say, they behave differently when there is a large amount of data.
However, this is more about the source of the data than where it ends up.
For example, if you have a large string and you want to producer it to a
transport, you probably want a pull producer, because there's no events
which will signal that you can send some more of the string *except* for
the reactor deciding that it is ready for some more. So that's how you
should decide which of these you want to write - if the consumer is the
only event source involved, as in the large string case, then you want a
pull producer (streaming = False); if the producer itself is event-driven
in its ability to provide data, then you want a push producer.
More information about the Twisted-Python