[Twisted-Python] COMMERCIAL:Re: COMMERCIAL:Re: Chunked decoding missing fromHTTPClient?

[Barry Scott]

On Monday, 21 August 2017 11:27:53 BST Jean-Paul Calderone wrote:
> On Mon, Aug 21, 2017 at 5:52 AM, Barry Scott <barry.scott at forcepoint.com>
> 
> wrote:
> > On Tuesday, 15 August 2017 14:12:21 BST Jean-Paul Calderone wrote:
> > > On Tue, Aug 15, 2017 at 8:43 AM, Barry Scott <barry.scott at forcepoint.com
> > > 
> > > wrote:
> > > > I see that HTTPChannel knows about chunking and indeed works well.
> > > > 
> > > > However shouldn't HTTPClient also support chunking?
> > > > Or Am I missing something to get chunking working for the response
> > 
> > from a
> > 
> > > > server?
> > > 
> > > Take a look at
> > > http://twistedmatrix.com/documents/current/web/howto/client.html which
> > > documents the newer HTTP client API in Twisted.
> > 
> > I'm interested in using a subclass of HTTPClientParser with Agent.
> > It does not seem that I can do this simply and have Agent use my class.
> > 
> > I see that its hardwired in HTTP11ClientProtocol.request().
> 
> Yep, seems to be.
> 
> > Is this possible?
> 
> If you can convince folks there's a good reason to want to do this, you can
> probably get a patch accepted that allows it.
> 
> Why do you want to do this?

There are places in twisted where we hardened it against malformed headers 
etc. Thoses changes might be something you want to take as a patch or we might 
need to keep subclassing pieces of the twisted stack.

I've got a long way with moving to twisted 17.5 from 2.0, the last piece,
I hope, is figuring out how to allow us to add code to the processing
without maintaining out of tree patches to twisted itself.

Barry

> 
> Jean-Paul
> 
> > > You may also want to have
> > > a look at https://treq.readthedocs.io/en/latest/ which provides another
> > > layer on top of that which may be even easier to use for your purposes.
> > 
> > Looks nice, but too high level for our use.
> > 
> > > HTTPClient is old, implements HTTP/1.0, and will probably be deprecated
> > 
> > and
> > 
> > > removed before too much longer.
> > 
> > Got it.
> > 
> > > Jean-Paul
> > 
> > Barry