[Twisted-Python] doWrite called on a twisted.internet.unix.Port

exarkun at twistedmatrix.com exarkun at twistedmatrix.com
Fri May 23 08:56:27 MDT 2014


On 01:49 pm, killiands at gmail.com wrote:
>Hi Exarkun,
>
>I tried hacking the addWriter method as follows:
>        add_writer_orig = reactor.__class__.addWriter 
>#@UndefinedVariable
>        def my_add_writer(self, writer):
>            logging.warn(writer.__class__.__mro__)
>            logging.warn(''.join(traceback.format_stack()))
>            if isinstance(writer,Port):
>                raise Exception("Shouldn't add a port as a writer")
>            return add_writer_orig(self, writer)
>        reactor.addWriter = types.MethodType(my_add_writer,reactor)
>But I actually got nothing out of it.

This seems alright to me.  I'm not sure why it hasn't revealed any extra 
information.  Did you test it in the trivial case?  For example, set it 
up and then do:

    port = reactor.listenTCP(0, Factory.forProtocol(Protocol))
    port.startWriting()

If you don't see anything logged then there's something wrong with the 
instrumentation.

Here's a slightly different version.  I don't see any reason why it 
would work if your version doesn't, it's basically doing the same thing. 
Maybe there's some obscure detail that prevents your version from 
working, though.

    from __future__ import print_function

    from traceback import print_stack

    from twisted.internet import reactor
    from twisted.internet.interfaces import IListeningPort
    from twisted.internet.protocol import Protocol, Factory

    def addWriter(writer):
        if IListeningPort.providedBy(writer):
            print("Adding a listening port as a writer: ", writer)
            print_stack()
        return reactor.__class__.addWriter(reactor, writer)

    reactor.addWriter = addWriter

    # Demonstrates that it works in the trivial case.
    port = reactor.listenTCP(0, Factory.forProtocol(Protocol))
    port.startWriting()

Another scenario that occurs to me is that the port itself is never 
added as a writer.  Instead, some other object that is actually supposed 
to write sometimes is added.  Then, the file descriptor for that object 
is closed.  Then, a new port is created and is accidentally assigned the 
same integer as its file descriptor as the one that just got closed. 
Plus, you're using a reactor that doesn't notice when file descriptors 
are closed (this is a complex and subtle corner case and the handling 
varies between reactors because the underlying platform behavior varies 
and it's really difficult, if not impossible, to paper over differences 
in this area due to missing platform services).

Another question about your environment, does the process that is 
affected by this error ever launch any child processes or fork for any 
other reason (having duplicates of the file descriptor, which often 
happens when you fork, is one way this issue can affect the epoll 
reactor)?
Another debug strategy might be to strace all file-descriptor related 
syscalls and see if you can catch an integer being reused and then being 
subject to this error.  Logging the port's file descriptor in the port's 
doWrite might help with this too.

Jean-Paul
>I also noticed this backtrace (or
>similar) is sometimes with a Udp.Port, not only a Unix.Port:
>Unhandled Error
>Traceback (most recent call last):
>  File "/path/to/twisted.zip/twisted/python/log.py", line 88, in
>callWithLogger
>  File "/path/to/twisted.zip/twisted/python/log.py", line 73, in
>callWithContext
>  File "/path/to/twisted.zip/twisted/python/context.py", line 118, in
>callWithContext
>  File "/path/to/twisted.zip/twisted/python/context.py", line 81, in
>callWithContext
>--- <exception caught here> ---
>  File "/path/to/twisted.zip/twisted/internet/posixbase.py", line 619, 
>in
>_doReadOrWrite
>  File "/path/to/twisted.zip/twisted/internet/base.py", line 1117, in
>doWrite
>exceptions.RuntimeError: doWrite called on a twisted.internet.udp.Port
>
>Any other ideas how I could find out the culprit?
>
>Thank you,
>
>Killian
>
>
>On 7 May 2014 17:09, Killian De Smedt <killiands at gmail.com> wrote:
>>Hi Exarkun,
>>
>>Thanks for the quick response. I should have specified those things
>>immediately.
>>I manually merged the UDP ipv6 branch in the trunk somewhere in august 
>>and
>>used that one, the version number is reported as [twisted, version 
>>13.1.0].
>>The platform is always centos though the centos version might range 
>>from
>>5.x to 6.x, 32 bit, but most of the time it runs on a centos 5.2
>>installation (kernel on my working machine is 2.6.18). Python is 2.7.1 
>>.
>>  I use the default reactor which should come down to the epoll one.
>>
>>I'll try to hack up the addwriter, it shouldn't be that hard to for 
>>just
>>that application.
>>
>>Thank you,
>>
>>Killian
>>
>>
>>On 7 May 2014 16:11, <exarkun at twistedmatrix.com> wrote:
>>>On 01:47 pm, killiands at gmail.com wrote:
>>>>Hello everybody,
>>>>
>>>>I sometimes see the following error logged by a twisted application, 
>>>>it
>>>>only happens sporadically and I cannot even reproduce when trying to
>>>>re-execute the exact sequence of those failures. So giving an SSCCE 
>>>>is
>>>>quite impossible for now (sorry). Given this trace it's also hard to 
>>>>find
>>>>what was actually called/executed.
>>>
>>>I've seen something like this with a somewhat old version of Twisted 
>>>and
>>>a custom reactor.  I never tracked down the cause.
>>>
>>>What version of Twisted are you using, what platform are you on, and 
>>>what
>>>reactor are you using?
>>>
>>>Another useful bit of debug information would be to hack up the 
>>>reactor's
>>>`addWriter` method to do a check of the argument.  The call stack at 
>>>*that*
>>>point (when the argument is a Port) is more interesting than the call 
>>>stack
>>>at the point where `doWrite` is called.
>>>
>>>Jean-Paul
>>>
>>>_______________________________________________
>>>Twisted-Python mailing list
>>>Twisted-Python at twistedmatrix.com
>>>http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>
>>
>>
>>--
>>Killian De Smedt
>>mobile: +32 486/825 951
>>mail: killiands at gmail.com
>
>
>
>--
>Killian De Smedt
>mobile: +32 486/825 951
>mail: killiands at gmail.com




More information about the Twisted-Python mailing list