[Twisted-web] integrating inotify into a protocol

Jason Pepas cell at phunware.com
Wed Mar 30 17:06:32 EDT 2011


Phil, that's a huge help, thanks so much for pointing me in the right
direction.  The notion of the factory being persistent and "owning" a
protocol for each connection was what I needed to wrap my head around
the problem.

In fact, when I thought about it some more, I realized that the
factory should "own" the inotify watcher.  This then makes it
straightforward for the factory to pass in its instance method as
inotify's callback.

I've implemented your suggestions and retooled my "Echo" example as
"TmpTell" (attached).  When you run it, it creates several persistent
clients (one for each port number listed on the command line).  Then,
each time a file is created in /tmp, each client tells its server
about the new file.

here's an example usage.

first, start up a "server"


 $ socat - tcp4-listen:1234


and another:


 $ socat - tcp4-listen:1235


now start up tmptell:


 $ python tmptell.py 1234 1235
 calling TmpTellClientFactory.__init__
 calling TmpTellClientFactory.startFactory
 calling TmpTellClientFactory.startedConnecting
 calling TmpTellClientFactory.startedConnecting
 calling TmpTellClientFactory.buildProtocol
 calling TmpTell.__init__
 calling TmpTellClientFactory.buildProtocol
 calling TmpTell.__init__


finally, in a third terminal, run 'mktemp':


 $ mktemp
 /tmp/tmp.1xtuY16NKr


you will then see this additional output in the tmptell terminal:


 calling TmpTellClientFactory.inotifyEventHappened
 calling TmpTell.announceNewFile
 calling TmpTell.announceNewFile


and your "servers" will look like this:


 $ socat - tcp4-listen:1234
 new file created at FilePath('/tmp/tmp.1xtuY16NKr')

 $ socat - tcp4-listen:1235
 new file created at FilePath('/tmp/tmp.1xtuY16NKr')


if you kill one of the servers with CTRL+c, you see this in the
tmptell terminal:


 calling TmpTell.connectionLost
 calling TmpTellClientFactory.removeProtocolObject
 calling TmpTellClientFactory.clientConnectionLost


and now we run mktemp a second time:


 $ mktemp
 /tmp/tmp.L5EeFSdXRJ


and now tmptell correctly calls announceNewFile just once, instead of twice:


 calling TmpTellClientFactory.inotifyEventHappened
 calling TmpTell.announceNewFile


and the server we didn't kill looks like this now:


 $ socat - tcp4-listen:1235
 new file created at FilePath('/tmp/tmp.1xtuY16NKr')
 new file created at FilePath('/tmp/tmp.L5EeFSdXRJ')


Thanks again Phil, I think I've got it now!

-jason
-------------- next part --------------
from twisted.internet.protocol import Protocol, ClientFactory
from sys import stdout


class TmpTell(Protocol):
    def __init__(self, factory):
        print "calling %s.%s" % (self.__class__.__name__, sys._getframe().f_code.co_name)
        self.factory = factory

    def announceNewFile(self, watch_obj, path, mask):
        print "calling %s.%s" % (self.__class__.__name__, sys._getframe().f_code.co_name)
        self.transport.write("new file created at %s\n" % path)

    def connectionLost(self, reason):
        print "calling %s.%s" % (self.__class__.__name__, sys._getframe().f_code.co_name)
        self.factory.removeProtocolObject(self)


class TmpTellClientFactory(ClientFactory):
    def __init__(self, path, mask):
        print "calling %s.%s" % (self.__class__.__name__, sys._getframe().f_code.co_name)
        self.protocol_objects = []
        self.watch_path = filepath.FilePath(path)
        self.watch_mask = mask
        self.notifier = inotify.INotify()

    def startFactory(self):
        print "calling %s.%s" % (self.__class__.__name__, sys._getframe().f_code.co_name)
        self.notifier.startReading()
        self.notifier.watch(self.watch_path, mask=self.watch_mask, callbacks=[self.inotifyEventHappened])

    def startedConnecting(self, connector):
        print "calling %s.%s" % (self.__class__.__name__, sys._getframe().f_code.co_name)

    def buildProtocol(self, addr):
        print "calling %s.%s" % (self.__class__.__name__, sys._getframe().f_code.co_name)
        new_protocol_object = TmpTell(factory=self)
        self.protocol_objects.append(new_protocol_object)
        return new_protocol_object

    def inotifyEventHappened(self, watch_obj, path, mask):
        print "calling %s.%s" % (self.__class__.__name__, sys._getframe().f_code.co_name)
        for p in self.protocol_objects:
            p.announceNewFile(watch_obj, path, mask)

    def clientConnectionLost(self, connector, reason):
        print "calling %s.%s" % (self.__class__.__name__, sys._getframe().f_code.co_name)

    def clientConnectionFailed(self, connector, reason):
        print "calling %s.%s" % (self.__class__.__name__, sys._getframe().f_code.co_name)

    def removeProtocolObject(self, dead_protocol_object):
        print "calling %s.%s" % (self.__class__.__name__, sys._getframe().f_code.co_name)
        self.protocol_objects.remove(dead_protocol_object)

if __name__ == "__main__":
    from twisted.internet import reactor, inotify
    from twisted.python import filepath
    import sys
    factory = TmpTellClientFactory('/tmp', inotify.IN_CREATE)
    for port in sys.argv[1:]:
        reactor.connectTCP("localhost", int(port), factory)
    reactor.run()


More information about the Twisted-web mailing list