[Twisted-Python] Using lineReceived with HTTP requests

[Andrea Soong]

This seems like it has something to do with my problem, although it doesn't offer any clues as to a solution:

"Unfortunately, handling arbitrary data chunks is a hard way to code a server. This is why twisted has many classes sitting in submodules of the twisted.protocols package which give higher level interface to the data. For line oriented protocols, LineReceiver translates the low-level dataReceived events into lineReceived events. However, the first naive implementation of LineReceiver proved to be too simple. Protocols like HTTP/1.1 or Freenet have packets which begin with header lines that include length information, and then byte streams. LineReceiver was rewritten to have a simple interface for switching at the protocol layer between line-oriented parts and byte-stream parts. "
(from http://www.python10.org/p10-papers/09/index.htm)

Andrea Soong <girloncrack at yahoo.com> wrote:Hi all,
 
First off, I'd like to say thanks to Ken Whitesell for giving me some much needed help.
 
Here's my problem:  I am using an HTTPClient instance to connect to a webpage, and am trying to retrieve its contents and display to the screen line by line, using the lineReceived method.  This is all being done in an .rpy file.  For some reason, I am only receiving the header information:
---------------------------------------------
[start line received]
       HTTP/1.0 200 OK
[end line received]
[start line received]
       Cache-Control: private
[end line received]
[start line received]
       Content-Type: text/html
[end line received]
[start line received]
       Set-Cookie: PREF=ID=aacd1eed2e435ca5:TM=1118474192:LM=1118474192:S=QyMnwhj46cXbhy_1; expires=Sun, 17-Jan-2038 19:14:07 GMT; path=/; domain=.google.com
[end line received]
[start line received]
       Server: GWS/2.1
[end line received]
[start line received]
       Date: Sat, 11 Jun 2005 07:16:32 GMT
[end line received]
[start line received]
       Connection: Close
[end line received]
[start line received]
       
[end line received]
---------------------------------------------
 
Does anyone know if there is some incompatibility in using lineReceived with the format of a webpage (i.e. maybe they are missing the standard EOL delimiters)?  dataReceived works fine and outputs the whole webpage at once.  
 
Also, I am using handleRequestEnd to detect the end of the transmission and call request.finish(),  but it does not seem to be getting called at all - the browser just keeps waiting for more data.  I've put a simple request.write("handleRequestEnd called") call on the first line of the method and it does not get written to the screen.  I can't find any explanation for how to use most of the methods in the HTTPClient class in the API - is there a better way to do this (i.e. one that works!) ?
 
 
Thanks,
Andrea



---------------------------------
Discover Yahoo!
Have fun online with music videos, cool games, IM & more. Check it out!_______________________________________________
Twisted-Python mailing list
Twisted-Python at twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python


		
---------------------------------
Do you Yahoo!?
 Yahoo! Mail - Find what you need with new enhanced search. Learn more.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20050611/34235645/attachment.html>