[Twisted-web] Web client through https proxy - is it even possible?

[Bruno Harbulot]

Hi,

I'm not sure how this would work with the twisted web client, but I've 
managed to get it to work in this library 
http://code.google.com/p/python-httpclient/ (which can also verify 
certificates, btw).

You don't need to send the Host header with CONNECT as far as I'm aware. 
You might want to wait until you read the 200 OK status code (or some 
other code if it fails) in response to CONNECT before turning the socket 
into an SSL socket.

Best wishes,

Bruno.


Matt Pruden wrote:
> I'm trying to post some data to a web server, but need to go through
> an https proxy.  I can't seem to find anyone that's pulled this off
> with a twisted web client.   I've been sniffing traffic and have it
> close, but can't quite get past creating the SSL socket.  Anything I
> attempt to write to the transport after swapping out the standard
> socket for SSL.Connection fails the SSL handshake on the remote end.
> 
> Here's the code so far:
> class MyProtocol(client.HTTPPageGetter):
> 
>     got_connect = False
> 
>     def connectionMade(self):
>         proxy = os.environ.get('https_proxy')
>         if proxy:
>             self.transport.write('CONNECT %s:%s HTTP/1.0\r\n' %
> (self.factory.host, self.factory.port))
>             self.sendHeader('Host', self.factory.headers.get("host",
> self.factory.host))
>             self.sendHeader('User-Agent', self.factory.agent)
>             self.transport.write('\r\n')
>         else:
>             self.got_connect = True
> 
>         if self.got_connect:
>             client.HTTPPageGetter.connectionMade(self)
> 
>     def lineReceived(self, line):
>         if not self.got_connect:
>             self.got_connect == True
>             ctx = ssl.ClientContextFactory()
>             self.transport.stopReading()
>             self.transport.stopWriting()
>             self.transport.socket = SSL.Connection(ctx.getContext(),
>                                                 self.transport.socket)
>             self.transport.fileno = self.transport.socket.fileno
>             self.transport.startReading()
>             client.HTTPPageGetter.connectionMade(self)
>         else:
>             client.HTTPPageGetter.lineReceived(self, line)