[Twisted-Python] HTTP client should be more tolerant?
fingon at iki.fi
Thu Feb 3 10:36:19 EST 2005
As initial disclaimer, PLEASE change your FAQ about bug reporting - nobody
really wants to correctly report bug by first joining on some random
mailing list (OR allow moderated non-member postings, but we all know how
much spam THAT will drag in).
According to the HTTP RFCs, CRLF is the valid line termination
sequence. However, the appendix (section 19.3 in old HTTP/1.1, too lazy to
dig up more recent one) states as follows:
The line terminator for message-header fields is the sequence CRLF.
However, we recommend that applications, when parsing such headers,
recognize a single LF as a line terminator and ignore the leading CR.
Some UNIXish servers, while breaking RFC, seem to work on browsers. I'd say
following this advice would be good, and therefore tuned my twisted http
client to be 'tolerant' (all major and minor browsers that I am aware of
Changes required? web/http.py:
@@ -314,6 +314,7 @@
length = None
firstLine = 1
__buffer = ''
+ delimiter = '\n'
def sendCommand(self, command, path):
self.transport.write('%s %s HTTP/1.0\r\n' % (command, path))
@@ -325,6 +326,7 @@
def lineReceived(self, line):
+ if line and line[-1] == '\r': line = line[:-1]
self.firstLine = 0
l = line.split(None, 2)
I'm also considering implementing HTTP/1.1 client. Has anyone done anything
about it? Thoughts? 'Give up, bad idea'?
I also rewrote the proxy for my local use a bit, mostly to make it more
component-oriented AND correct (currently HTTP/1.1 clients do not get
persistent service, which is inefficient). Unfortunately changes are mixed
among some other code, but has there been any work on proxy recently?
I might have interest in submitting patch or two to make it more correct in
A: "You can do wonderful things in software if you get rid of the
assumption that Windows == Shit."
B: "But likewise you could do wonderful things in hardware if you
got rid of the assumption that Gravity == 9.81 metres per second
C: "But that's an invalid comparison. You can get rid of the gravity
problem quite easily by a change of location -- but do you
think Windows would get any better if you shot it to the moon?"
-- rec.humor.funny post by ermel at gmx.de (Erik Meltzer)
More information about the Twisted-Python