[Twisted-web] Twisted and reverse proxies

[Maarten ter Huurne]

On Wednesday 31 March 2010, Rob Newman wrote:

> We wish to use an Apache reverse proxy to allow our web users to
> transparently view the Twisted application 'within' our existing Apache
> served site. So that, something like:
> 
> http://www.companyname.com/tools/twisted_app
> 
> proxies to:
> 
> http://hostname:8080

This is what the Apache configuration would look like:

===

# Serve static content in Apache:
# (the ProxyPass exception must be located before the ProxyPass rule)
ProxyPass /tools/twisted_app/static !
Alias /tools/twisted_app/static /some/path/on/disk/static

# Act as a proxy for Twisted:
<Location /tools/twisted_app>
  ProxyPass http://hostname:8080 retry=1
  ProxyPassReverse http://hostname:8080
  ProxyPassReverseCookiePath / /tools/twisted_app/
</Location>

===

"ProxyPass" sets up the reverse proxy. The "retry=1" ensures that if your 
Twisted instance returns from being unavailable (for example after an 
upgrade), Apache will retry contacting it 1 second later. The default is a 
lot longer than 1 second.

"ProxyPassReverse" rewrites headers so redirections continue to work. It 
does not rewrite content.

"ProxyPassReverseCookiePath" rewrites cookies to make their path correspond 
to the URL of your web app. Without this, if you run multiple Twisted 
applications reverse proxied on the same host, their (session) cookies will 
have identical paths and start overwriting each other.

> As someone relatively new to Twisted, how will our static (and other)
> resources react to being reverse proxy served? Will they barf? Is there
> any online documentation I can look at for case studies?

As Ray wrote, in most cases it is best to let Apache serve the static 
content and use Twisted for the dynamic content.

If your app puts absolute URLs in the content it produces, such as 
registration conformation e-mails or RSS/Atom feeds, it must use the 
"/tools/twisted_app" outer URL for this, since Apache does not rewrite 
content. So the application must be aware that it is reverse proxied and 
under which URL.

One thing to remember is that Apache buffers the data it gets from Twisted 
before sending it back to the requester. So if you stream events instead of 
writing a document, the requester will not receive the events until much 
later. I worked around this by reading streaming data directly from the 
Twisted HTTP server.

Bye,
		Maarten