[Twisted-Python] UDP Logging Server

Tim Allen screwtape at froup.com
Fri Mar 11 18:32:27 MST 2011


On Fri, Mar 11, 2011 at 01:15:47PM -0600, SIC FS LIST wrote:
> So far I have a "working" implementation ... but I'm noticing that if I do
> the following:
> -- log when a message is received
> -- that for that message it "might" show up in the file a pretty lengthy
> period of time later

Assuming the objects stored in DISKINFO[1] etc. are file objects, you
seem to be writing to the files but never calling flush(). If you don't
call flush(), Python (well, the C standard library) won't send the data
on to the OS until its buffer is full, or the file handle is closed. If
you're not getting that many log lines, it can take a while for that to
happen.

Of course, if you flush after every disk read, your program will run
a bit more slowly and with more I/O... for an application where
reliability is more important than performance (like logging) that's
probably acceptable.

> The actual UDP protocol:
> 
> class VocsLogger(DatagramProtocol):
>     def datagramReceived(self, data, (host, port)):
>         _proc_msg(self.transport, data, (host,
> port))._new_msg().addCallback(handler)

_proc_msg doesn't seem to be complicated enough to need its own class,
why not just do what _proc_msg does in VocsLogger?

> The _proc_msg class:
> 
> class _proc_msg:
>     def __init__(self, sck, data, (host, port)):
>         self._sck = sck
>         self._data = data
>         self._host = host
>         self._port = port
> 
>     def _new_msg(self):
>         d, _ = LogMsg().ParseSocketMsg(self._data)
>         if d.type.upper() == DISKINFO[0]:
>             DISKINFO[1].write(d.ToString() + "\n%s\n" % (LOG_DELIM))
>         elif d.type.upper() == LOADAVG[0]:
>             LOADAVG[1].write(d.ToString() + "\n%s\n" % (LOG_DELIM))
>         elif d.type.upper() == MEMINFO[0]:
>             MEMINFO[1].write(d.ToString() + "\n%s\n" % (LOG_DELIM))
>         elif d.type.upper() == NETDEV[0]:
>             NETDEV[1].write(d.ToString() + "\n%s\n" % (LOG_DELIM))
>         elif d.type.upper() == PSAUX[0]:
>             PSAUX[1].write(d.ToString() + "\n%s\n" % (LOG_DELIM))
>         elif d.type.upper() == WHOINFO[0]:
>             WHOINFO[1].write(d.ToString() + "\n%s\n" % (LOG_DELIM))
>         else:
>             DEFAULT[1].write(d.ToString() + "\n%s\n" % (LOG_DELIM))

It depends on what DISKINFO[0] and DISKINFO[1] actually are, but
assuming they're a string and a file-handle, this code would look more
Pythonic as something like this:

LOGSINKS = {
	"DISKINFO": open("/var/log/diskinfo", "w"),
	"LOADAVG": open("/var/log/loadavg", "w"),
	"MEMINFO": open("/var/log/meminfo", "w"),
	"NETDEV": open("/var/log/netdev", "w"),
	"PSAUX": open("/var/log/psaux", "w"),
	"WHOINFO": open("/var/log/whoinfo", "w"),
	"DEFAULT": open("/var/log/default", "w"),
    }

def _new_msg(self, data):
    d, _ = LogMsg().ParseSocketMsg(data)
    type = d.type.upper()
    sink = LOGSINKS.get(type, LOGSINKS['DEFAULT'])
    sink.write("%s\n%s\n" % (d.ToString(), LOG_DELIM))

Hope that helps!




More information about the Twisted-Python mailing list