Opened 15 years ago

Closed 15 years ago

#2302 defect closed fixed (fixed)

t.web2.channel.http.HTTPParser deals with HEAD requests incorrectly

Reported by: ghazel Owned by:
Priority: normal Milestone:
Component: web2 Keywords:
Cc: Branch:
Author:

Description

HTTPParser.setConnectionParams has a comment:

                # If no Content-Length, indeterminate length data
                # (unless the responsecode was one of the special ones, or
                #  the request method was HEAD.
                # If the request was HEAD, self.length has been set to 0 by
                # HTTPClientRequest.submit)
                if self.code in http.NO_BODY_CODES:
                    self.length = 0
                else:
                    self.length = connHeaders.getHeader('content-length', self.length)

However, if the HEAD response has a content-length, it then sets self.length anyway. This leads to a pipelining error, as HTTPParser.lineReceived expects there to be more data.

Change History (7)

comment:1 Changed 15 years ago by ghazel

Also, simply setting stream.length to 0 if the method is HEAD does cause pipelining to work, but then as a user there's no way to tell what the content-length was. Very similar to this problem: #2284

comment:2 Changed 15 years ago by jknight

The patch on #2284 fix for the bug, but I'm not quite sure where to put the content-length of the HEAD request. To be consistent, stream.length has to be 0.

comment:3 Changed 15 years ago by ghazel

Here's the hack I threw together when I encountered these problems. It just stores content-length on a member variable which I access later

from twisted.web2.channel.http import HTTPParser

# http://twistedmatrix.com/trac/ticket/2302
old_setConnectionParams = HTTPParser.setConnectionParams
def setConnectionParams(self, connHeaders):
    old_setConnectionParams(self, connHeaders)
    self.content_length = self.length
    if hasattr(self, 'request'):
        if self.request.method == "HEAD":
            self.length = 0
HTTPParser.setConnectionParams = setConnectionParams


from twisted.web2.client.http import HTTPClientChannelRequest

# http://twistedmatrix.com/trac/ticket/2284
old_createRequest = HTTPClientChannelRequest.createRequest
def createRequest(self):
    old_createRequest(self)
    self.stream.length = self.length
    self.response.content_length = self.content_length
HTTPClientChannelRequest.createRequest = createRequest

# http://twistedmatrix.com/trac/ticket/2304
old_connectionLost = HTTPClientChannelRequest.connectionLost
def connectionLost(self, reason):
    self._error(reason)
HTTPClientChannelRequest.connectionLost = connectionLost

comment:4 Changed 15 years ago by Jean-Paul Calderone

(In [19162]) Revert r19155 - test suite regression

Refs #2302 Refs #1870

twisted.trial.test.test_runner.DeferredSharedSuiteTest.test_waitForDeferred fails with gtk reactor.

twisted.trial.test.test_runner.TestSharedClassSuite.test_reporting fails with Python 2.5.

Some interaction seems to be taking place with the TODO feature. testTLS and testBackwardsTLS from twisted.test.test_ssl.SpammyTLSTestCase are reported as [FAIL] even though they are marked TODO (only in some configurations).

comment:5 Changed 15 years ago by Jean-Paul Calderone

no, that Refs #2303. :/

comment:6 Changed 15 years ago by foom

Resolution: fixed
Status: newclosed

(In [19180]) Fix content-length issues in HTTP client: Propagate content-length into stream.length. (Fixes #2284) Fix HEAD requests with content-length. (Fixes #2302) Also, refactor and fix the test_client tests and add some docstrings.

Reporter: ghazel Author: jknight Reviewer: jerub, exarkun

comment:7 Changed 11 years ago by <automation>

Owner: jknight deleted
Note: See TracTickets for help on using tickets.