[Twisted-Python] Logging observers and threads

Justin Venus justin.venus at gmail.com
Thu Jul 12 07:56:20 MDT 2012


For logging could we look at implementing posix aio on platforms that
support it?  I would be willing to help code that.

Justin
On Jul 12, 2012 8:51 AM, <exarkun at twistedmatrix.com> wrote:

> On 11:32 am, itamar at itamarst.org wrote:
> >On 07/12/2012 05:17 AM, Tristan Seligmann wrote:
> >>3) log observers should expect to be inovked in non-reactor threads,
> >>making this a Mantissa bug.
> >In theory this is the case - the logging howto says "The observer needs
> >to be thread safe if you anticipate using threads in your program." The
> >API docs don't mention threads, though..
> >
> >In practice however, probably every custom observer gets this wrong, so
> >we really need some infrastructure to make this easier or to fix it.
> >Possibly we should have some way to indicate an observer is thread-
> >safe,
> >assume by default they are not, and if they are not wrap them in a
> >reactor.callFromThread wrapper.
>
> The logging system is already a bottleneck in some applications.  Doing
> even more work to try to make it properly thread-safe will probably slow
> it down even more.
>
> I think it was a mistake to try to make it possible to use
> twisted.python.log free-threaded.  I think we should think about getting
> rid of this claimed feature.  Perhaps it could be replaced with a more
> explicit, probably more restricted, logging feature for non-reactor
> threads.
>
> Unless someone can magically fix the threading issues without hurting
> performance.  Then, great, go for it.
> >I would also like some infrastructure to make the file log observer run
> >in its own thread, but that's almost the opposite use case :)
>
> Sure, but that's probably based on top of an asynchronous file I/O API
> that's not explicitly thread-based (but probably is thread-based in
> practice, because what other kind of asynchronous file I/O even exists).
>
> Also, to explicitly answer Tristan's question:
>
>     It's a bug in the Mantissa log observer.
>
> Jean-Paul
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20120712/5b6be496/attachment.html>


More information about the Twisted-Python mailing list