[Twisted-web] Re: Serving files from many web-servers thru one
central web-server
Thomas Weholt
thomas.weholt at gmail.com
Wed Sep 29 01:10:38 MDT 2004
Sweet!!! I'm going to try it when I get off work. In my original
design and implemented prototype the slave nodes on the subnet answers
a UDP broadcast from the main server, which in turns keeps a list of
slave-nodes on the local subnet, connecting to them and communicating
with them using a mix of UDP and XMLRPC. Making a connection to the
local slavenodes or adding them to the main server will be a bit more
tricky using the code you supplied, but hey !!! If I can re-implement
my idea using something like this it would be so much better.
Thanks again!! :-)
Best regards,
Thomas
On 29 Sep 2004 02:40:47 -0400, David Bolen <db3l at fitlinxx.com> wrote:
> Thomas Weholt <thomas.weholt at gmail.com> writes:
>
> > Oh, darn!! Forgot all about this when I started my project. I've read
> > thru most of the docs I've found so far, but I'm still somewhat
> > clueless. Can anybody provide a simple example of how to do this,
> > preferrably without using the examples in
> > http://twistedmatrix.com/documents/current/howto/using-twistedweb#auto19.
> > I'm hooking a xmlrpc-handler and a UDP listener into it as well and
> > find the good ol'
> (...)
>
> Well, I experimented tonight and here's a quick example of some
> situations. In re-reading your original post, I realized it wasn't
> clear if your "reads data from several subnet servers" comment
> referred to wanting to access web resources on the internal servers,
> or just some other service you needed data from. If the latter, then
> you can use your own PB session that can be richer in interface than a
> web resource if you wanted, so I gave a simple example of that too.
>
> This example just runs both the simulated main server and subset
> server objects within the same process over a loopback connection, but
> should work identically over any other link.
>
> There is a main and child resource on the server side, and the same on
> the internal/subnet side, tied into a URL on the server side by using
> the ResourcePublisher/ResourceSubscription. An additional server side
> resource turns into its own pure PB call to a remote server object.
> Once running you can access the following URLs:
>
> http://localhost:8000 ExternalRoot
> http://localhost:8000/child ExternalChild
> http://localhost:8000/data InternalData.remote_getData()
> http://localhost:8000/internal InternalRoot
> http://localhost:8000/internal/child InternalChild
>
> In the InternalData case, I'm making a new PB session for each
> rendering request. Presumably you'd want to structure things to
> maintain a persistent connection only reconnecting when necessary
> (which, BTW, is basically what ResourceSubscription does).
>
> It all seems to work as expected... it's simplistic but hopefully
> it'll point you in the right direction. Shouldn't be any problem to
> tie in additional protocols (such as XMLRPC/UDP) into either the main
> server or subnet server processes.
>
> -- David
>
> - - - - - - - - - - - - - - - - - - - - - - - - -
>
> import sys
>
> from twisted.python import log
> from twisted.internet import reactor
> from twisted.spread import pb
> from twisted.web import server, resource, distrib
>
> #
> # An internal PB server object on the internal subnet server, with simple
> # direct access (no authentication) via a root object.
>
> class InternalData(pb.Root):
>
> def remote_getData(self):
> # This could be its own deferrable operation
> return 'Internal data'
>
> #
> # Resources on the internal subnet servers
> #
>
> class InternalChild(resource.Resource):
> """A child resource rendered on the internal server"""
>
> def render(self, request):
> return '<html><body>Internal Child</body></html>'
>
> class InternalRoot(resource.Resource):
> """The root of the tree rendered on the internal server"""
>
> def getChild(self, path, request):
> # Support direct rendering (no trailing "/" on request)
> if path == '':
> return self
> else:
> return resource.Resource.getChild(self, path, request)
>
> def render(self, request):
> return '<html><body>Internal Root</body></html>'
>
> #
> # Resources on the primary web server
> #
>
> class MainData(resource.Resource):
> """A child resource that renders a data call to the internal server"""
>
> def __init__(self, host, port):
> resource.Resource.__init__(self)
> self.host = host
> self.port = port
>
> def render(self, request):
> """Make a request to the remote root object, and use that result
> as the result of our rendering"""
>
> def failure(value):
> request.write('<html><body>'
> 'Unable to access data:<br>%s'
> '</body></html>' % value)
>
> def success(value):
> request.write('<html><body>%s</body></html>' % value)
>
> # Right now we make a new connection to the internal host for
> # each rendering request (pretty darn inefficient!)
> factory = pb.PBClientFactory()
> reactor.connectTCP(self.host, self.port, factory)
>
> # Obtain a reference to the remote object, call the getData method
> # and then disconnect.
> root = factory.getRootObject()
> root.addCallback(lambda root:
> root.callRemote('getData').addCallback(success))
> root.addErrback(failure)
> root.addCallback(lambda _: request.finish())
> root.addCallback(lambda _: factory.disconnect())
> return server.NOT_DONE_YET
>
> class MainChild(resource.Resource):
> """A child resource rendered directly on the main server"""
>
> def render(self, request):
> return '<html><body>Main Server Child</body></html>'
>
> class MainRoot(resource.Resource):
> """The primary root resource on the main server"""
>
> def getChild(self, path, request):
> # Support direct rendering (no trailing "/" on request)
> if path == '':
> return self
> else:
> return resource.Resource.getChild(self, path, request)
>
> def render(self, request):
> return '<html><body>Main Server Root</body></html>'
>
> #
> # Simulate main and subnet servers. The main server will listen on port
> # 8000 and the subnet server will listen (for PB connections) on port 8001.
> # Additionally the subnet server will provide the InternalData object
> # on port 8002.
> #
>
> if __name__ == "__main__":
>
> #
> # Build up a subnet server "site":
> # / InternalRoot
> # /child InternalChild
> #
> iroot = InternalRoot()
> iroot.putChild('child', InternalChild())
> isite = server.Site(iroot)
>
> #
> # Build up the main server "site":
> # / MainRoot
> # /child MainChild
> # /data Render result of call to InternalData's retrieveData
> # /internal Request to / on subnet server
> #
> root = MainRoot()
> root.putChild('child', MainChild())
> root.putChild('data', MainData('localhost', 8002))
> root.putChild('internal', distrib.ResourceSubscription('localhost',8001))
> site = server.Site(root)
>
> #
> # Now start both servers listening. Note that if these were really
> # running on separate machines, the internal server could do a listenTCP
> # for isite on 8000 to support normal web lookups, while also supporting
> # port 8001 for the PB proxied lookups.
> #
> reactor.listenTCP(8000, site)
> reactor.listenTCP(8001,
> pb.PBServerFactory(distrib.ResourcePublisher(isite)))
> reactor.listenTCP(8002,pb.PBServerFactory(InternalData()))
>
> log.startLogging(sys.stdout)
> reactor.run()
>
>
>
>
> _______________________________________________
> Twisted-web mailing list
> Twisted-web at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
>
--
Mvh/Best regards,
Thomas Weholt
http://www.weholt.org
More information about the Twisted-web
mailing list