[Twisted-Python] HTTP client should be more tolerant?
James Y Knight
foom at fuhm.net
Thu Feb 3 11:08:57 EST 2005
On Feb 3, 2005, at 10:36 AM, Markus Stenberg wrote:
> According to the HTTP RFCs, CRLF is the valid line termination
> sequence. However, the appendix (section 19.3 in old HTTP/1.1, too
> lazy to
> dig up more recent one) states as follows:
> The line terminator for message-header fields is the sequence CRLF.
> However, we recommend that applications, when parsing such headers,
> recognize a single LF as a line terminator and ignore the leading
> Some UNIXish servers, while breaking RFC, seem to work on browsers.
> I'd say
> following this advice would be good, and therefore tuned my twisted
> client to be 'tolerant' (all major and minor browsers that I am aware
OMG. *What* evil server does that?
> I'm also considering implementing HTTP/1.1 client. Has anyone done
> about it? Thoughts? 'Give up, bad idea'?
Thought about it, but still working on the new HTTP server. (see
twisted.web2 in svn trunk). Hopefully a new client could reuse much of
the work for the new server (esp. header parsing/generation).
> I also rewrote the proxy for my local use a bit, mostly to make it more
> component-oriented AND correct (currently HTTP/1.1 clients do not get
> persistent service, which is inefficient). Unfortunately changes are
> among some other code, but has there been any work on proxy recently?
> I might have interest in submitting patch or two to make it more
> correct in
> some things.
A proxy should only be a small glue layer between the server and
client, so once those two are working well, a correctly operating
(non-caching) proxy should be pretty easy. A caching proxy, of course,
is a whole other project.
More information about the Twisted-Python