[Twisted-Python] How to dispatch message to different servers

Benjamin BERTRAND beenje at gmail.com
Sat Mar 2 08:16:11 MST 2013


Le 2 mars 2013 à 15:33, Laurens Van Houtven <_ at lvh.cc> a écrit :

> I'm guessing that this is another question that will be solved as soon as I see the code (perhaps you should put all your code up somewhere); but all I do know is that all Deferreds buy you is an abstraction for organizing callbacks; it's not a dispatch mechanism (and if you're using it as one now, you do have dispatch logic, it just lives somewhere else).

The dispatch is done in the pcapDataReceived method (in the ServerFactory) thanks to the dictionary of deferred (using the line id as key).
And each line is associated to a server on a specific port.
It's maybe a bit strange, but does what I want.

Here is the code if you are interested: http://pastebin.com/VeQgQ85z
Unfortunately, I can't give any data to test it (I only have live data that I don't think I can share)...


> 
> 
> On Sat, Mar 2, 2013 at 3:25 PM, Benjamin BERTRAND <beenje at gmail.com> wrote:
> 
> Le 2 mars 2013 à 14:53, Laurens Van Houtven <_ at lvh.cc> a écrit :
> 
>> Ah, but that too appears to be missing in the original code ;-)
>> 
>> The stuff you're doing with deferreds there seems a bit strange. In your example, why not just call sendMessage when you get the packet?
> 
> If I had only one server yes.
> But the thing is I have to send the message to a different server depending on the line id.
> That was my initial problem.
> I don't see how I could call the sendMessage method corresponding to a specific server directly.
> Or?
> 
>> 
>> 
>> On Sat, Mar 2, 2013 at 2:28 PM, Benjamin BERTRAND <beenje at gmail.com> wrote:
>> 
>> Le 2 mars 2013 à 10:08, Laurens Van Houtven <_ at lvh.cc> a écrit :
>> 
>>> Yes, that looks okay, but that wasn't in your original sample ;-)
>>> 
>>> 
>> 
>> Yep, sorry about that.
>> I was more focused on the ServerFactory and Protocol.
>> The pcap in a thread comes from the link I mentioned in my first post: http://dound.com/2009/09/integrating-twisted-with-a-pcap-based-python-packet-sniffer/
>> But I know, it's better to put everything in one post. People shouldn't have to click links.
>> 
>>> I'm on my phone at the moment which isn't great for code review, but it looks like you only fire one deferred per line?
>>> 
>>> 
>> 
>> There is a specific deferred by line.
>> I re-arm it in the messageToSend method (that wasn't in the sample either).
>> In the Oldimon class, I have:
>> 
>>     def messageToSend(self, message):
>>         self.sendMessage(message)
>>         # Re-arm the deferred
>>         self.factory.deferred[self.line] = defer.Deferred()
>>         self.factory.deferred[self.line].addCallback(self.messageToSend)
>> 
>> 
>>> On Mar 2, 2013 9:50 AM, "Benjamin BERTRAND" <beenje at gmail.com> wrote:
>>> 
>>> Le 2 mars 2013 à 04:34, exarkun at twistedmatrix.com a écrit :
>>> 
>>> > On 1 Mar, 09:52 pm, beenje at gmail.com wrote:
>>> >> Thanks for the answer!
>>> >>
>>> >> I was hoping to avoid having to put something like AMP in place,
>>> >> because it looked a bit overkill for my case.
>>> >> I think I actually found a way :-)
>>> >
>>> > Unfortunately, it looks like the code that you shared will only work
>>> > accidentally (if at all).  You cannot use Twisted APIs except in the
>>> > reactor thread.  You will at least need to add in some code to send data
>>> > back to the reactor thread before you use Twisted APIs (such as
>>> > `Deferred.callback`).
>>> 
>>> 
>>> In run_pcap, I call reactor.callFromThread(callback, x25_data, line_id). See below.
>>> That seems to work with the tests I did.
>>> Am I missing something?
>>> 
>>> /Benjamin
>>> 
>>> def run_pcap(device, pcap_filter, callback):
>>> 
>>>     def analyse_packet(hdr, data):
>>>         # check the data
>>>         reactor.callFromThread(callback, x25_data, line_id)
>>> 
>>>     p = pcapy.open_live(device, 65535, 1, 100)
>>>     p.setfilter(pcap_filter)
>>>     p.loop(-1, analyse_packet)
>>> 
>>> 
>>> class Oldimon(Protocol):
>>> 
>>>     def __init__(self, factory):
>>>         self.factory = factory
>>>         self.line = None
>>> 
>>>     def connectionMade(self):
>>>         # Check the server port to get the line
>>>         # associated to this protocol
>>>         port = self.transport.getHost().port
>>>         self.line = LINES_PORT[port]
>>>         # Add the callback for this line
>>>         self.factory.deferred[self.line] = defer.Deferred()
>>>         self.factory.deferred[self.line].addCallback(self.messageToSend)
>>> 
>>> 
>>> class OldimonFactory(ServerFactory):
>>> 
>>>     def __init__(self, device, pcap_filter):
>>>         # pcapDataReceived callback is called everytime a message
>>>         # is received
>>>         reactor.callInThread(run_pcap, device, pcap_filter, self.pcapDataReceived)
>>>         # Dict with a deferred for each line
>>>         self.deferred = dict(zip(LINES_PORT.values(), [None] * len(LINES_PORT)))
>>> 
>>>     def buildProtocol(self, addr):
>>>         return Oldimon(self)
>>> 
>>>     def pcapDataReceived(self, data, line):
>>>         if self.deferred[line] is not None:
>>>             # Fire the callback for line
>>>             d, self.deferred[line] = self.deferred[line], None
>>>             d.callback(data)
>>> 
>>> oldimon_factory = OldimonFactory(device, pcap_filter)
>>> for port in LINES_PORT.keys():
>>>     reactor.listenTCP(port, oldimon_factory)
>>> reactor.run()
>>> 
>>> 
>>> >
>>> > Jean-Paul
>>> >
>>> > _______________________________________________
>>> > Twisted-Python mailing list
>>> > Twisted-Python at twistedmatrix.com
>>> > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>> 
>>> 
>>> _______________________________________________
>>> Twisted-Python mailing list
>>> Twisted-Python at twistedmatrix.com
>>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>> _______________________________________________
>>> Twisted-Python mailing list
>>> Twisted-Python at twistedmatrix.com
>>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>> 
>> 
>> _______________________________________________
>> Twisted-Python mailing list
>> Twisted-Python at twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>> 
>> 
>> 
>> 
>> -- 
>> cheers
>> lvh
>> _______________________________________________
>> Twisted-Python mailing list
>> Twisted-Python at twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
> 
> 
> 
> -- 
> cheers
> lvh
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20130302/013c4f74/attachment.html>


More information about the Twisted-Python mailing list