[Twisted-Python] Modifying the logstring

Einar S. Idsø einar.twisted at norsk-esport.no
Tue Dec 7 08:14:26 EST 2010

To continue my monologue: For future reference, it is worth noting
that the code previously posted will not work for reconnecting clients
since only the first connection receives the modified logPrefix
method. A better approach is to bind logPrefix() in
onClientConnected() to ensure it is bound for each new connection.

def startClient(self):
    self.f = self.getFactory()
    reactor.connectTCP(self.ip, self.queryPort, self.f)

def f_onClientConnected(self, protocolInstance):
    def logPrefix():
        return "whateveryouwant"
    self.f.connection = protocolInstance
    self.f.connection.transport.logPrefix = logPrefix

def getFactory(self):
    f = ReconnectingClientFactory()
    f.protocol = whatever
    f.onClientConnected = self.f_onClientConnected
    return f

I am still in the dark when it comes to modifying the logstring in
other cases, though. I am sure I could transfer the results from my
UDP logger to my service using strings instead of bound methods, but
then the logstring would end up being simply [-], which is of no use
to me. Is there really no way in which I can tell Twisted that "All
log messages generated by this object shall be on the following
format"? It seems like such a simple task to accomplish, yet it
completely avoids me.

Also let me add that I am completely taken aback with the genius that
is Twisted Python. The more I work with it, the more I realise this.
Writing clients or servers for *anything* is so simple, and the
efficiency of the code, especially with respect to CPU-usage, is
wonderful. My application started by polling each server via an
external executable written in C, but now fetches info from the same
number of servers using protocols written in Python, and even gathers
a whole lot more info than before via event logs.CPU-usage has dropped
from 20% to 1%, and the memory footprint is only around 40 MB :)


On Mon, Dec 6, 2010 at 11:51 AM, Einar S. Idsø
<einar.twisted at norsk-esport.no> wrote:
> I was able to modify the logstring for the TCP query by following a
> similar approach to what I did to the UDP logger:
>    def logPrefix(self):
>        return self.connection.transport.protocol.__class__.__name__ +
> ", " + str(self.id)
>    def startClient(self):
>        self.f = self.factory(self, password=self.queryPassword)
>        self.connection = reactor.connectTCP(self.ip, self.queryPort, self.f)
>        self.connection.transport.logPrefix = self.logPrefix
> Perhaps not very elegant, but it does the job and allows me to get a
> log prefix which not only tells me the protocol but also the exact
> serverinstance.
> But is it even possibly to do something similar for the UDP scenario
> where I have one UDP-logger which calls bound functions from the
> individual server-representing instances to do the actual parsing? The
> UDP-listener class is defined like this:
> class UDPListener(DatagramProtocol):
>    _serverList = {}
>    def addParser(self, ip, port, f):
>        self._serverList[(str(ip), int(port))] = f
>    def removeParser(self, ip, port):
>        if (str(host), int(port)) in self._serverList:
>            del self._serverList[(str(ip), int(port))]
>    def datagramReceived(self, data, (host, port)):
>        #print "received %r from %s:%d" % (data, host, port)
>        if (host, port) in self._serverList:
>            self._serverList[(host, port)](data)
> Cheers,
> Einar
> On Mon, Dec 6, 2010 at 10:21 AM, Einar S. Idsø
> <einar.twisted at norsk-esport.no> wrote:
>> Hi,
>> I have an application which logs activity on a number of servers by
>> two methods:
>> 1. TCP-based queries (app is client)
>> 2. UDP-based logging (app is server)
>> Each server that is being logged is represented by an instance of a
>> server class. TCP-queries are performed by each instance at set
>> intervals, while the UDP-logs are gathered by a global logging
>> instance which listens to a specific port. Events for a given server
>> can be triggered by both protocols, but are handled by the server
>> instance. In other words: An instance of a server class gathers events
>> by polling the server via TCP /and/ by receiving strings from a UDP
>> logging instance which passes received datagrams as strings to the
>> correct instance.
>> Whenever a loggable event results from the TCP queries, the name of
>> the protocol class, which is a subclass of
>> twisted.internet.protocol.Protocol), is used as the logstring. That is
>> fine, as it allows me to distinguish between different server classes,
>> although I cannot distinguish between individual servers.
>> However, when a loggable event occurs on the UDP logging instance, it
>> writes out a longish protocol name which is the same for all the
>> different server classes that are subscribing to events on that UDP
>> port. I have managed to modify the string by doing the following:
>>        self.udpLogger = reactor.listenUDP(myPort, UDPListener())
>>        self.udpLogger.logstr = "UDPLogger"
>> Yet when the logged string is passed, as a string, to a server
>> instance for processing, the logstring for any result remains
>> "UDPLogger" no matter how I pass it or what I do to it. I suppose it
>> is expected behaviour that the protocol should "follow" a result, no
>> matter how it is processed down the line, but is there any way in
>> which I can alter the logstring so that instead of "UDPLogger" or the
>> name of the protocol class, it instead uses the server id or any other
>> value that I specify?
>> What I am getting in my logs is:
>> 2010-11-30 10:23:09+0100 [CSS,client] Something happened on TCP
>> 2010-11-30 10:23:09+0100 [UDPLogger] Something happened on UDP
>> What I would like is:
>> 2010-11-30 10:23:09+0100 [CSS (TCP),12345] Something happened on TCP
>> 2010-11-30 10:23:09+0100 [CSS (UDP),12345] Something happened on UDP
>> where 12345 is a unique identifier of the specific server. Or at least:
>> 2010-11-30 10:23:09+0100 [CSS,client] Something happened on TCP
>> 2010-11-30 10:23:09+0100 [CSS] Something happened on UDP
>> so that I know the result from UDP concerns a CSS-server.
>> I hope this was possible to understand. Please let me know if anything
>> needs to be clarified. :)
>> Cheers,
>> Einar

More information about the Twisted-Python mailing list