<div dir="ltr"><div>Thanks for both those suggestions.<br>I'll be taking a closer look at txLoadbabancer when I get time as it looks like it'll take care of a lot of my desired functionality out the box.<br>To get started tho I'll move my async routing decision call into the protocol as suggested.<br>
<br></div><div>Is there any reason why the internal calls to buildProtocol shouldn't be wrapped in a maybeDeferred?<br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Nov 16, 2013 at 7:05 PM, Lucas Taylor <span dir="ltr"><<a href="mailto:ltaylor.volks@gmail.com" target="_blank">ltaylor.volks@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><br><div><div class="im"><div>On Nov 16, 2013, at 7:09 AM, Tom van Neerijnen wrote:</div>
<br><blockquote type="cite"><div dir="ltr"><div><div><div>Hi all<br><br></div>I'm building a simple TCP load balancer based on a code snippet from Glyph on SO: <a href="http://stackoverflow.com/questions/4096061/general-question-regarding-wether-or-not-use-twisted-in-tcp-proxy-project" target="_blank">http://stackoverflow.com/questions/4096061/general-question-regarding-wether-or-not-use-twisted-in-tcp-proxy-project</a><br>
<br></div>It's served me well but I can't work out how to convert Glyphs round robin retrieval of the server endpoint into an async balancing decision in the buildProtocol method of the Factory. If I return a deferred here it fails with an AttributeError: Deferred instance has no attribute 'makeConnection'.<br>
<br>Currently I'm working around this by running a separate management loop that periodically updates a dictionary with all the data necessary to make my routing decision so that I can do it without a deferred. This worries me because I may be making my decision on slightly stale data and I'd really like this to be a real time decision as the connection comes in. Does anyone have a clever way of doing this?<br>
<br></div></div></blockquote><div><br></div><div><br></div></div><div>Hi Tom,</div><div><br></div><div>One possibly unexpected aspect of using @inlineCallbacks is that the decorated function itself returns a Deferred. This is why you see the AttributeError...the machinery calling buildProtocol expects an IProtocol instance (or None), and the function is returning a Deferred. `defer.returnValue()` is provided to the callback on that Deferred, not as a direct return value from the decorated function.</div>
<div><br></div><div>If you want to make the routing decision when the client connects, then you could push the decision-making process down into the Protocol itself.</div><div><br></div><div>Here's a quick mockup overriding connectionMade in a ProxyServer protocol subclass. It calls the factory routing function (which may or may not return a deferred), and connects the proxy once the decision has been made.</div>
<div><div><br></div><div><br></div><div>from twisted.internet.protocol import Factory</div><div>from twisted.protocols.portforward import ProxyServer</div><div><br></div><div><br></div><div><div>class Balancer(Factory):</div>
<div> protocol = RoutingProxyServer</div><div> routing_func = port_routing_decision_async</div><div><br></div><div><br></div><div>class RoutingProxyServer(ProxyServer):</div><div><br></div><div> def connectionMade(self):</div>
<div><div> # Don't read anything from the connecting client until we have</div><div> # somewhere to send it to.</div><div> self.transport.pauseProducing()</div></div><div><br></div><div> client = self.clientProtocolFactory()</div>
<div> client.setServer(self)</div><div> </div><div> if self.reactor is None:</div><div> from twisted.internet import reactor</div><div> self.reactor = reactor</div><div> </div>
<div> def connectProxy(host, port):</div><div> self.reactor.connectTCP(host, port, client)</div><div><br></div><div> d = maybeDeferred(self.factory.routing_func)</div><div> d.addCallback(connectProxy)</div>
<div> d.addErrback(log.err)</div></div><span class="HOEnZb"><font color="#888888"><div><br></div></font></span></div><span class="HOEnZb"><font color="#888888"><div><div><br></div><div>Lucas</div><div><br></div><div>
<br></div><div><br></div><div><br></div><div><br></div></div></font></span><div><div class="h5"><blockquote type="cite"><div dir="ltr">An example is below. The hashed out buildProtocol is a synchronous decision which works. Thanks in advance!<br>
<br>from twisted.internet.protocol import Factory<br>from twisted.protocols.portforward import ProxyFactory<br>
from twisted.internet import reactor, defer<br>import random<br><br>from twisted.python import log<br>import sys<br>log.startLogging(sys.stderr)<br><br>local_ports = set([1024, 1025])<br><br>def port_routing_decision_sync():<br>
return random.choice(list(local_ports))<br><br>def port_routing_decision_async():<br> d = defer.Deferred()<br> reactor.callLater(1, d.callback, port_routing_decision_sync())<br> return d<br><br>class Balancer(Factory):<br>
# def buildProtocol(self, addr):<br> # port = port_routing_decision_sync()<br> # print "connecting to local port {}".format(port)<br> # return ProxyFactory("127.0.0.1", port).buildProtocol(addr)<br>
<br> @defer.inlineCallbacks<br> def buildProtocol(self, addr):<br> port = yield port_routing_decision_async()<br> print "connecting to local port {}".format(port)<br> defer.returnValue(ProxyFactory("127.0.0.1", port).buildProtocol(addr))<br>
<br>def main():<br> factory = Balancer()<br> reactor.listenTCP(5678, factory)<br> reactor.run()<br><br>if __name__ == "__main__":<br> main()<br></div></blockquote></div></div></div><br></div><br>_______________________________________________<br>
Twisted-Python mailing list<br>
<a href="mailto:Twisted-Python@twistedmatrix.com">Twisted-Python@twistedmatrix.com</a><br>
<a href="http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python" target="_blank">http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python</a><br>
<br></blockquote></div><br></div>