[Twisted-Python] thoughts on a reactor feature
bruce at cubik.org
Tue Sep 24 02:25:31 EDT 2002
Glyph Lefkowitz wrote:
> On Mon, 23 Sep 2002 12:54:22 -0600, Bruce Mitchener <bruce at cubik.org> wrote:
>>One would be have an interface to POSIX clocks and track the amount of
>>time spent between calls out of reactor.run() and report anything that
>>takes over a specified threshold of time to execute.
> If I understand your proposal correctly, I think this could be implemented
> quite simply by writing a Python wrapper around the reactor, that would wrap
> each protocol and protocol factory being passed to it in something that would
> catch events and time their execution. This should work for just about any
> reactor. With a few extra features, it might even be good for measuring their
> differences. Would you need a wrapper around POSIX clocks more robust than
time.clock() looks like it'd work well enough so long as the accuracy is
good enough. (clock_gettime() deals in nanoseconds.)
Wrapping the reactor in Python seems harder. This could also be a usage
of the metrics gathering interfaces that I'd mentioned yesterday.
That'd be far simpler than wrapping the reactor, intercepting
everything, and providing a new iteration function to catch all of the
calls out of the reactor while it processed the queued calls.
And .. this should be something that is easy to use, easy to turn on,
and readily available. We have primitive facilities for this sort of
thing in the server tech that I use at work and the only reason that
they're not more advanced is that I didn't want to spend the time (and
add the risk) on refactoring our interpreter. They greatly simplified
diagnosing server performance issues that popped up.
That would still leave printing out an identifier that indicates what it
was that just ran for more than the allowed amount of time. I'll have
to experiment a bit inside the reactor in the time day or so if I have
time for Twisted and see what types of things we can reasonably print
> (I think this could also compare time.clock() to time.time() to see if
> potentially blocking calls were being made.)
That might be interesting, but not sure that time.time() is of similar
resolution as the values that I want. But if I do end up wrapping
clock_gettime() in a C module, then I could also wrap the function for
getting wall time in addition to CPU time and per-thread CPU time. Not
a problem at all. :)
>>Another is to implement some sort of stack trace sampling where you
>>periodically take a stack trace when a utility program is running and you've
>>noted periods of long latency.
> This I don't know how to do; possibly it's doable with Python's
> debugger/profiler hooks? I'm not quite clear on how you do the "noting" -- is
> this a message that the user sends to Twisted or is it something that the
> profiler performs some heuristic to get?
Well, the bit above with the time.clock() is probably enough for now.
It'd be nice to be able to do more introspection on thread pools, but
that can probably wait a bit.
More information about the Twisted-Python