Opened 5 years ago

Last modified 5 years ago

#5567 defect new

failure.Failure() isn't 100% thread-safe

Reported by: teratorn Owned by:
Priority: lowest Milestone:
Component: core Keywords: threads
Cc: Branch:


It's often necessary to instantiate Failure objects from non-reactor threads to e.g. fail a Deferred which was passed in to the thread-using code to report its success or failure:

    reactor.callFromThread(d.errback, failure.Failure())

Thus, instantiating Failure really ought to be fully thread-safe. The current implementation increments a global 'count' variable non-atomically, for one thing. I've not audited the rest of the code to determine its apparent thread-safety, but someone should before closing this ticket.

The 'count' variable isn't super-critical as it's just used for logging statements. Other problems may exist which are critical.

Change History (4)

comment:1 Changed 5 years ago by Jean-Paul Calderone

Priority: normallowest

I don't think it's *often* necessary. More frequently, you just raise an exception and the deferToThread implementation takes care of the rest (except for the niggling issue of #4219).

Perhaps it's once in a while attractive. However, you can always do this instead:

        info = sys.exc_info()
        reactor.callFromThread(lambda: d.errback(Failure(*info)))

Perhaps it would still be nice if Failure creation were thread-safe (it would resolve #4219, after all). However, I don't think this is particularly urgent for 99.99% of our users.

comment:2 Changed 5 years ago by teratorn

I agree it's not very important for the vast majority of users, but I have a pretty solid use-case in a project I'm working on right now. I do database interactions from a dedicated thread, each interaction should be serialized, and so the thread does its work out of a Queue.

The interactions have a Deferred which needs to be fired on the main thread, and it would be nice to pass real Failures of the actual Exceptions to them.

Thanks for the work-around. I think I'll start using it until this issue can be fixed.

comment:3 Changed 5 years ago by gelin yan

Hi all

As regards #4219, I want to know when Failure creation will become thread unsafe? I use deferToThread daily.

comment:4 Changed 5 years ago by Jean-Paul Calderone

Please use the mailing list or IRC for that sort of question, not the issue tracker. Thanks!

Note: See TracTickets for help on using tickets.