[Twisted-web] Proxy Authentication with a Password Protected Site

[Jean-Paul Calderone]

On Mon, 22 Sep 2008 10:08:56 -0400, Robert Hancock <hancock.robert at gmail.com> wrote:
>On Mon, Sep 22, 2008 at 9:44 AM, Jean-Paul Calderone <exarkun at divmod.com> wrote:
>> On Mon, 22 Sep 2008 09:38:57 -0400, Robert Hancock
>> <hancock.robert at gmail.com> wrote:
>>>
>>> On Mon, Sep 22, 2008 at 4:57 AM, Phil Mayers <p.mayers at imperial.ac.uk>
>>> wrote:
>>>>>
>>>>> If I run this against a password protected HTTP(S) site from a host
>>>>> that has direct access to the Internet it works fine.  I now have to
>>>>> move it behind a proxy that requires authentication.  The Twisted
>>>>> documentation did not make it clear (to me) how to add proxy
>>>>> authentication and I cannot find an example on the Internet.
>>>>>
>>>>> I've tried adding an additional Proxy-Authentication header to the
>>>>> call, but that doesn't help  Any ideas would be greatly appreciated.
>>>>
>>>> "Didn't help" how? What didn't work?
>>>>
>>>> What kind of proxy is it?
>>>>
>>>
>>> Didn't help in that I receive an HTTP 407 Proxy Authentication required.
>>>
>>> It is a proxy that requires Basic HTTP authentication.
>>>
>>
>> Maybe you meant to add the "Proxy-Authenticate" header instead of the
>> "Proxy-Authentication" header?
>>
>> Twisted Web provides no particular support for authenticating with proxies,
>> but it doesn't do anything that would prevent you from handling this in your
>> code, either.
>>
>> Jean-Paul
>>
>
>proxyUsername = 'bob_hancock'
>proxyPassword = 'myproxypass'
>proxyAuth = base64.encodestring('%s:%s' % (proxyUsername, proxyPassword))
>proxy_authHeader = "Basic " + proxyAuth.strip()
>
>url = "https:/www.myexampl.com" #password protected site
>username = "bob_hancock"
>password = "myremotepass"
>basicAuth = base64.encodestring('%s:%s' % (username, password))
>authHeader = "Basic " + basicAuth.strip()
>
>return client.getPage(url, headers={"Authorization": authHeader,
>"Proxy-Authenticate": proxy_authHeader})
>
>Is this the approach?  I'm behind a corporate proxy/firewall so that
>if I make a direct HTTPS request I receive an HTTP 407 response.
>
>Thanks for the rapid reply. This is my first attempt at Twisted.
>

This looks largely correct to me.  I don't have a proxy (and I definitely
don't have /your/ proxy) to actually test it against, so I may be missing
something.  One thing that does come to mind is that a proxy might require
that the request URL is absolute rather than relative (<http://foo/bar>
instead of </bar>).  I'm not sure when exactly this is required, nor how
to convince `getPage´ to make a request of that form.  This might be related
to <http://twistedmatrix.com/trac/ticket/2714>.

Jean-Paul