[Twisted-web] Proxy Authentication with a Password Protected Site

Jean-Paul Calderone exarkun at divmod.com
Mon Sep 22 10:25:09 EDT 2008

On Mon, 22 Sep 2008 10:08:56 -0400, Robert Hancock <hancock.robert at gmail.com> wrote:
>On Mon, Sep 22, 2008 at 9:44 AM, Jean-Paul Calderone <exarkun at divmod.com> wrote:
>> On Mon, 22 Sep 2008 09:38:57 -0400, Robert Hancock
>> <hancock.robert at gmail.com> wrote:
>>> On Mon, Sep 22, 2008 at 4:57 AM, Phil Mayers <p.mayers at imperial.ac.uk>
>>> wrote:
>>>>> If I run this against a password protected HTTP(S) site from a host
>>>>> that has direct access to the Internet it works fine.  I now have to
>>>>> move it behind a proxy that requires authentication.  The Twisted
>>>>> documentation did not make it clear (to me) how to add proxy
>>>>> authentication and I cannot find an example on the Internet.
>>>>> I've tried adding an additional Proxy-Authentication header to the
>>>>> call, but that doesn't help  Any ideas would be greatly appreciated.
>>>> "Didn't help" how? What didn't work?
>>>> What kind of proxy is it?
>>> Didn't help in that I receive an HTTP 407 Proxy Authentication required.
>>> It is a proxy that requires Basic HTTP authentication.
>> Maybe you meant to add the "Proxy-Authenticate" header instead of the
>> "Proxy-Authentication" header?
>> Twisted Web provides no particular support for authenticating with proxies,
>> but it doesn't do anything that would prevent you from handling this in your
>> code, either.
>> Jean-Paul
>proxyUsername = 'bob_hancock'
>proxyPassword = 'myproxypass'
>proxyAuth = base64.encodestring('%s:%s' % (proxyUsername, proxyPassword))
>proxy_authHeader = "Basic " + proxyAuth.strip()
>url = "https:/www.myexampl.com" #password protected site
>username = "bob_hancock"
>password = "myremotepass"
>basicAuth = base64.encodestring('%s:%s' % (username, password))
>authHeader = "Basic " + basicAuth.strip()
>return client.getPage(url, headers={"Authorization": authHeader,
>"Proxy-Authenticate": proxy_authHeader})
>Is this the approach?  I'm behind a corporate proxy/firewall so that
>if I make a direct HTTPS request I receive an HTTP 407 response.
>Thanks for the rapid reply. This is my first attempt at Twisted.

This looks largely correct to me.  I don't have a proxy (and I definitely
don't have /your/ proxy) to actually test it against, so I may be missing
something.  One thing that does come to mind is that a proxy might require
that the request URL is absolute rather than relative (<http://foo/bar>
instead of </bar>).  I'm not sure when exactly this is required, nor how
to convince `getPage´ to make a request of that form.  This might be related
to <http://twistedmatrix.com/trac/ticket/2714>.


More information about the Twisted-web mailing list