[Twisted-Python] thoughts on a reactor feature

Bruce Mitchener bruce at cubik.org
Tue Sep 24 02:25:31 EDT 2002


Glyph Lefkowitz wrote:
> On Mon, 23 Sep 2002 12:54:22 -0600, Bruce Mitchener <bruce at cubik.org> wrote:
>>One would be have an interface to POSIX clocks and track the amount of 
>>time spent between calls out of reactor.run() and report anything that 
>>takes over a specified threshold of time to execute.
> If I understand your proposal correctly, I think this could be implemented
> quite simply by writing a Python wrapper around the reactor, that would wrap
> each protocol and protocol factory being passed to it in something that would
> catch events and time their execution.  This should work for just about any
> reactor.  With a few extra features, it might even be good for measuring their
> differences.  Would you need a wrapper around POSIX clocks more robust than
> time.clock()?

time.clock() looks like it'd work well enough so long as the accuracy is 
good enough.  (clock_gettime() deals in nanoseconds.)

Wrapping the reactor in Python seems harder.  This could also be a usage 
of the metrics gathering interfaces that I'd mentioned yesterday. 
That'd be far simpler than wrapping the reactor, intercepting 
everything, and providing a new iteration function to catch all of the 
calls out of the reactor while it processed the queued calls.

And .. this should be something that is easy to use, easy to turn on, 
and readily available.  We have primitive facilities for this sort of 
thing in the server tech that I use at work and the only reason that 
they're not more advanced is that I didn't want to spend the time (and 
add the risk) on refactoring our interpreter.  They greatly simplified 
diagnosing server performance issues that popped up.

That would still leave printing out an identifier that indicates what it 
was that just ran for more than the allowed amount of time.  I'll have 
to experiment a bit inside the reactor in the time day or so if I have 
time for Twisted and see what types of things we can reasonably print 
from there.

> (I think this could also compare time.clock() to time.time() to see if
> potentially blocking calls were being made.)

That might be interesting, but not sure that time.time() is of similar 
resolution as the values that I want.  But if I do end up wrapping 
clock_gettime() in a C module, then I could also wrap the function for 
getting wall time in addition to CPU time and per-thread CPU time.  Not 
a problem at all. :)

>>Another is to implement some sort of stack trace sampling where you
>>periodically take a stack trace when a utility program is running and you've
>>noted periods of long latency.
> This I don't know how to do; possibly it's doable with Python's
> debugger/profiler hooks?  I'm not quite clear on how you do the "noting" -- is
> this a message that the user sends to Twisted or is it something that the
> profiler performs some heuristic to get?

Well, the bit above with the time.clock() is probably enough for now.

It'd be nice to be able to do more introspection on thread pools, but 
that can probably wait a bit.

  - Bruce

More information about the Twisted-Python mailing list