[Twisted-Python] Clarification or pausing a consumer

[Glyph]

> On Oct 13, 2017, at 5:47 PM, Jean-Paul Calderone <exarkun at twistedmatrix.com> wrote:
> 
> On Fri, Oct 13, 2017 at 8:29 AM, Barry Scott <barry.scott at forcepoint.com <mailto:barry.scott at forcepoint.com>> wrote:
> I have a app that needs to send data as an http response. But the data is
> produced slowly. More slowly then the time taken to transmit it.
> 
> The consumer keeps calling resumeProducing, but there is no data to send
> often.
> 
> This sounds like a bug in the consumer.  There are two cases.
> 
> If the producer is "streaming" or "push" (two words, one meaning) then the consumer should call resumeProducing once and let data come.  If it wants to slow down the data, it can call pauseProducing.  There's nothing else it should be doing.

This is the good kind of producer.

> If the producer is "not streaming" or "pull" (again, one meaning) then the consumer should call resumeProducing and wait for a write call.  Then it should call resumeProducing again (if it wants more data).  It should not call resumeProducing again while waiting for a write call.

This is the bad kind.  It is basically pointless.  It was a design error to include it and it should be removed from Twisted eventually.

> What is the correct way to pause the consumer so that it does not keep
> calling resumeProducing? unregisterProducer?
> 
> Probably just fix the consumer implementation to not be broken.  Or switch to tubes which has a simpler model and (in principle) fewer places to make mistakes.

My guess is not that the consumer is broken, but rather, you registered your producer as the bad kind of producer ("pull") and it's just doing as it was told, which is to keep calling resumeProducing over and over again every time it wants data.

Let us know how it works out!

>  
> Then when the next block of data is available what is the correct way to
> resume the consumer? registerProducer?
> 
> Related to that why does HTTPChannel.registerProducer convert a IPullProducer
> into a IPushProducer using _PullToPush?
> 
> Probably as an attempt to simplify the implementation - to make it so the consumer can pretend there's just one kind of producer instead of two (which would be nice for everyone).
> 
> Jean-Paul
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20171013/8d3972c6/attachment-0002.html>