[Twisted-Python] Question on push/pull producers inter-workings, was : "Is there a simple Producer/Consumer example or tutorial?"

Jean-Paul Calderone exarkun at divmod.com
Sat Apr 19 09:33:55 EDT 2008

On Fri, 18 Apr 2008 09:57:35 +0200, Gabriel Rossetti <mailing_lists at evotex.ch> wrote:
>Hello everyone,
>I am re-posting these questions with a different title, since they have 
>ventured away from the original question
>I have a question though, I was looking at the way all this works by using a 
>debugger, and I noticed that in twisted.internet.abstract.py in 
>registerProducer() there is the following :
>if not streaming:
>   producer.resumeProducing()
>Why is this done only for the pull producer? Shouldn't it also be called for 
>the push producer since to have the data sent one has to call either 
>self.transport.write() or resumeProducing() anyways? If you look at :
>it does :
>transport.registerProducer(self, 1)
>       self.produce()

So, this is a streaming producer.  It starts producing right away without
having resumeProducing called on it.

>thus starting the writing process and in the pull producer :
>it doesn't need to start the writing process explicitly since it's started 
>when the producer is registered.

This is a non-streaming producer.  It doesn't do anything until something
calls resumeProducing on it.

>Oh, and also, since from what I see in the code the only difference with a 
>push and pull producer is that the push producer is paused if the data being 
>written/sent is very large (to let the reactor breath and process other 
>things), if it isn't larger than the buffer it behaves like a pull producer, 
>correct? If so, then why have both? Am I mixed up again?

As you say, they behave differently when there is a large amount of data.
However, this is more about the source of the data than where it ends up.
For example, if you have a large string and you want to producer it to a
transport, you probably want a pull producer, because there's no events
which will signal that you can send some more of the string *except* for
the reactor deciding that it is ready for some more.  So that's how you
should decide which of these you want to write - if the consumer is the
only event source involved, as in the large string case, then you want a
pull producer (streaming = False); if the producer itself is event-driven
in its ability to provide data, then you want a push producer.


More information about the Twisted-Python mailing list