[Twisted-web] [PATCH] twisted.web2 support for gzip stream in/out

[Valentino Volonghi]

Following there's a patch for twisted.web2 to support gzip
Content-Encoding of chunks.

Thanks to Bob Ippolito for answering immediately to my mail about the
permission to include his gzstream work in twisted.web2.

Here is his mail:
===
From: Bob Ippolito <bob at redivi.com>
To: Valentino Volonghi <dialtone at gmail.com>
Date: Fri, 1 Oct 2004 20:50:35 -0400
Subject: Re: gzstream integration with twisted

Twisted.web2 may use my gzstream.

On Oct 1, 2004, at 8:33 PM, Valentino Volonghi wrote:

> Hi,
> I'm helping out Twisted developers to get twisted.web2 out before
> October 31.
>
> We are going to have a very complete implementation of HTTP 1.1 RFC.
> Doing so requires also having a gzip/gunzip stream to handle the
> Transfer-Coding.
>
> I've already talked to foom (the main developer of t.web2) and he
> asked for your permission to include gzstream in twisted.web2.
>
> That would be very helpful and your copyright will be kept entirely.
>
> Let me now your decision please. Thanks for attention.
>
> --
> Valentino Volonghi aka Dialtone
> Now running FreeBSD 5.3-beta6
> Blog: http://vvolonghi.blogspot.com
> Home Page: http://xoomer.virgilio.it/dialtone/
===

files needed are attached. and patch following.
I'll write also a bug for web2 and attach everything, but it's 3.30 AM
now, and I'll do that tomorrow.

Comments are welcome.

Index: twisted/web2/http.py
===================================================================
--- twisted/web2/http.py        (revision 11886)
+++ twisted/web2/http.py        (working copy)
@@ -33,6 +33,7 @@
 from twisted.web2 import http_headers
 from twisted.web2 import iweb
 from twisted.web2 import error
+from twisted.web2 import gzstream

 PERSIST_NO_PIPELINE = 2

@@ -301,6 +302,12 @@
         else:
             return ifrange == self.out_headers.getHeader("last-modified")

+    def requestContentEncoding(self, name):
+        if name.lower() != 'gzip':
+            warnings.warn("WARNING! Currently only gzip content is supported")
+            return
+        self.out_headers.setHeader('Content-Encoding', name)
+

 # FIXME: these last 3 methods don't belong here.

@@ -347,6 +354,8 @@
     partialHeader = ''
     queued = 0
     finishedReading = False
+    inChunkHandler = False
+    outChunkHandler = False

     channel = None
     request = None
@@ -449,13 +458,14 @@

     def rawDataReceived(self, data):
         datalen = len(data)
+
         if datalen < self.length:
             if not self.finished:
-                self.request.handleContentChunk(data)
+                self.inChunkHandler(data)
             self.length = self.length - datalen
         else:
             if not self.finished:
-                self.request.handleContentChunk(data[:self.length])
+                self.inChunkHandler(data[:self.length])
             extraneous = data[self.length:]
             channel = self.channel # could go away from allContentReceived.
             if not self.chunkedIn:
@@ -493,6 +503,18 @@

         self.channel.queueRequest(self)
         request = self.channel.requestFactory(self, self.command,
self.path, self.version, self.reqHeaders)
+
+        def decode_gzip(self, chunkHandler):
+            gunzipper = gzstream.GunzipStream()
+            gunzipper.write = chunkHandler
+            return gunzipper
+
+        # Set the client chunkhandler if the content is gzipped.
+        for encoding in self.reqHeaders.get('Content-Encoding', ()):
+            if encoding == 'gzip':
+                self.inChunkHandler = decode_gzip(request.handleContentChunk)
+            else:
+                self.inChunkHandler = request.handleContentChunk

         # Reset header state variables
         del self.reqHeaders
@@ -633,11 +655,22 @@

         l.append("\r\n")
         self.transport.writeSequence(l)
+
+        def encode_gzip(chunkHandler):
+            gzipper = gzstream.GzipStream()
+            gzipper.write = chunkHandler
+            return gzipper
+
+        for encoding in headers.getHeader('Content-Encoding', ()):
+            if encoding == 'gzip':
+                self.outChunkHandler = encode_gzip(toChunk)
+            else:
+                self.outChunkHandler = toChunk


     def writeData(self, data):
         if self.chunkedOut:
-            self.transport.writeSequence(toChunk(data))
+            self.transport.writeSequence(self.outChunkHandler(data))
         else:
             self.transport.write(data)

=======

HTH

-- 
Valentino Volonghi aka Dialtone
Now running FreeBSD 5.3-beta6
Blog: http://vvolonghi.blogspot.com
Home Page: http://xoomer.virgilio.it/dialtone/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gzstream.tbz2
Type: application/octet-stream
Size: 2911 bytes
Desc: not available
Url : http://twistedmatrix.com/pipermail/twisted-web/attachments/20041002/1579a28b/gzstream.obj