[Twisted-web] Need help getting started...

[Matt Goodall]

Andrea Soong wrote:
> Hello, 
>  
> I'm taking a class in which we're supposed to be implementing a
> pseudo-proxy server that does some text parsing on web pages before
> passing them back to the client.  The proxy is supposed to be accessed
> via the URL and port that the server is running on, using an .rpy file.
>  
> I've looked through all of the available Twisted documentation, but just
> can't seem to get a handle on how to get started.  It's mostly setup
> questions - what code should go in which files?

I understand you are restricted to deploying a single .rpy script, so
all your code will have to be contained in that file. That's not a good
way to deploy a Twisted web app but you have no choice here.

>  How do I run it?  

Create and run a web.tap that serves files from the directory containing
your .rpy script.

> How do I test it?

Request the .rpy using your browser.

> There are a lot of code examples available, but very
> little information on how they interact and call each other.  It doesn't
> help that I am not too familiar with the OO-style API documentation.
>  
> I know that I need to be creating a subclass of HTTPClient, and creating
> a resource in the .rpy file.  I'm guessing that the .rpy file should
> create an instance of the HTTPClient subclass and use it to retrieve the
> requested URL (we have to get it line by line), but I don't understand
> how to create the instance.

Typically, you add a client protocol factory to the reactor, giving it a
host and port. However, the twisted.web.client module has some
convenient functions that will make things easier for you. See getPage
in that module for more detail.

> A single good example that creates a web proxy server which simply
> retrieves the data line by line and displays it to the screen, plus the
> shell commands needed to run/test would be extremely helpful.  I am
> completely confused as to how the web server and .rpy file would interact.

Everything you need is in the "Using twisted.web" HOWTO which can be
found at http://twistedmatrix.com/projects/web/documentation/howto/.
Points of interest are:

  * .rpy (resource) scripts
  * creating and running a web.tap server
  * the twisted.web.client module
  * deferred resource rendering

>  
> I hope someone has the time and know-how to help me out.
> THANKS

I realise that this reply is a little terse but I suspect part of the
class is working out how to put all this together ;-).

Good luck!

Cheers, Matt

-- 
     __
    /  \__     Matt Goodall, Pollenation Internet Ltd
    \__/  \    w: http://www.pollenation.net
  __/  \__/    e: matt at pollenation.net
 /  \__/  \    t: +44 (0)113 2252500
 \__/  \__/
 /  \	       Any views expressed are my own and do not necessarily
 \__/          reflect the views of my employer.