[Twisted-Python] kqueue reactor / ticket #1918

Tobias Oberstein tobias.oberstein at tavendo.de
Sat Nov 5 07:48:52 EDT 2011


> > if there is anything I can do to clarify (like i.e. running on non-virtualized
> hardware FreeBSD) let me know.
> 
> Have you run the tests on OS X as well?

I'll do that.

> > Caveat: as attached to the ticket, there are some errs when running
> > trial which don't happen with select reactor.
> 
> Do you think you could start fixing some of those errors?  Do they make

I've looked at the logs in more detail, the error _diff_ between select/kqueue
yields:

twisted.test.test_tcp.HalfCloseBuggyApplicationTests.test_readNotificationRaises
twisted.test.test_tcp.HalfCloseTestCase.testCloseWriteCloser
twisted.test.test_tcp.HalfCloseTestCase.testWriteCloseNotification

which all seem to be testing TCP Half-Close.

I've looked at the case code .. guess I can grasp that after reading some background.

But I have no clue where to start "fixing" / looking for why this works with
the select reactor vs kqueue.

> sense to you?  If you're interested in moving this ticket forward, I'm sure that
> folks on this mailing list (me included) would love to provide you some
> guidance.
> 

Ok, will see what I can do.

Need some advice/hints though.

Generally, there are a couple of skipped tests missing modules (i.e. pycrypto)
which I could install. Should I do that, or are those not necessary?
Note, that there are packages I could, but would not like to install: GTK etc.

Then, whats the difference between FAIL and ERROR?

Then, there are a couple of tests which are skipped because

xxx_KQueueReactor
No module named kqsyscall

The kqsyscall should now be select.kqueue/kevent .. so I guess those
test cases needed to be adapted as well, right?

Then, baseline. I've run the trial using select reactor in the expectation of
finding all tests passed.

This is not the case, there are a couple of failed ones. Those are also failed
with the kqueue reactor, so I need to know whether I need to inspect those
or those are problems unrelated to the reactor, but of general (platform?) nature.

[1]
twisted.internet.test.test_posixprocess
  FileDescriptorTests
    test_expectedFDs ...                                                 [FAIL]

[2]
twisted.internet.test.test_process
  PTYProcessTestsBuilder_PollReactor
    test_openFileDescriptors ...                                        [ERROR]
  PTYProcessTestsBuilder_SelectReactor
    test_openFileDescriptors ...                                        [ERROR]
  ProcessTestsBuilder_PollReactor
    test_openFileDescriptors ...                                        [ERROR]
  ProcessTestsBuilder_SelectReactor
    test_openFileDescriptors ...                                        [ERROR]

[3]
twisted.python.test.test_components
  RegistrationTestCase
    test_duplicateAdapterForInterfaceAllowed ...                        [ERROR]
    test_registerAdapterForInterface ...                                [ERROR]
    test_subclassAdapterRegistrationForInterface ...                    [ERROR]

Now, [2] could be a consequence of [3], since they all fail because of

exceptions.ValueError: an adapter (<function <lambda> at 0x2c0dc41c>) was already registered.

See:

twisted.internet.test.test_process.PTYProcessTestsBuilder_PollReactor.test_openFileDescriptors
twisted.internet.test.test_process.PTYProcessTestsBuilder_SelectReactor.test_openFileDescriptors
twisted.internet.test.test_process.ProcessTestsBuilder_PollReactor.test_openFileDescriptors
twisted.internet.test.test_process.ProcessTestsBuilder_SelectReactor.test_openFileDescriptors
===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/test/test_components.py", line 351, in test_duplicateAdapterForInterfaceAllowed
    return self._duplicateAdapterForClassOrInterfaceAllowed(TheOriginal)
  File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/test/test_components.py", line 312, in _duplicateAdapterForClassOrInterfaceAllowed
    components.registerAdapter(firstAdapter, original, TheInterface)
  File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/components.py", line 93, in registerAdapter
    raise ValueError("an adapter (%s) was already registered." % (factory, ))
exceptions.ValueError: an adapter (<function <lambda> at 0x2c0dc41c>) was already registered.

twisted.python.test.test_components.RegistrationTestCase.test_duplicateAdapterForInterfaceAllowed
===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/test/test_components.py", line 268, in test_registerAdapterForInterface
    return self._registerAdapterForClassOrInterface(TheOriginal)
  File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/test/test_components.py", line 245, in _registerAdapterForClassOrInterface
    components.registerAdapter(adapter, original, TheInterface)
  File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/components.py", line 93, in registerAdapter
    raise ValueError("an adapter (%s) was already registered." % (factory, ))
exceptions.ValueError: an adapter (<function <lambda> at 0x2c0dc41c>) was already registered.

twisted.python.test.test_components.RegistrationTestCase.test_registerAdapterForInterface
===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/test/test_components.py", line 423, in test_subclassAdapterRegistrationForInterface
    return self._subclassAdapterRegistrationForClassOrInterface(TheOriginal)
  File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/test/test_components.py", line 396, in _subclassAdapterRegistrationForClassOrInterface
    components.registerAdapter(firstAdapter, original, TheInterface)
  File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/components.py", line 93, in registerAdapter
    raise ValueError("an adapter (%s) was already registered." % (factory, ))
exceptions.ValueError: an adapter (<function <lambda> at 0x2c0dc41c>) was already registered.

twisted.python.test.test_components.RegistrationTestCase.test_subclassAdapterRegistrationForInterface

=======

And [1]:

twisted.internet.test.test_posixprocess.FileDescriptorTests.test_expectedFDs
===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/_baseprocess.py", line 60, in maybeCallProcessEnded
    proto.processEnded(Failure(reason))
  File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/test/test_process.py", line 289, in processEnded
    checkOutput("".join(self.output))
  File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/test/test_process.py", line 276, in checkOutput
    self.assertEquals('[0, 1, 2, 3]', output)
twisted.trial.unittest.FailTest: not equal:
a = '[0, 1, 2, 3]'
b = 'xrange(256)'


**********************************************

Now, for the kqeue reactor, the following _additional_ errors appear:

twisted.test.test_tcp
  HalfCloseBuggyApplicationTests
    test_readNotificationRaises ...                                      [FAIL]
  HalfCloseTestCase
    testCloseWriteCloser ...                                               [ERROR]
    testWriteCloseNotification ...                                      [ERROR]


twisted.test.test_tcp.HalfCloseBuggyApplicationTests.test_readNotificationRaises
===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/_baseprocess.py", line 60, in maybeCallProcessEnded
    proto.processEnded(Failure(reason))
  File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/test/test_process.py", line 289, in processEnded
    checkOutput("".join(self.output))
  File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/test/test_process.py", line 276, in checkOutput
    self.assertEquals('[0, 1, 2, 3]', output)
twisted.trial.unittest.FailTest: not equal:
a = '[0, 1, 2, 3]'
b = 'xrange(256)'

===============================================================================
[ERROR]
Traceback (most recent call last):
Failure: twisted.internet.defer.TimeoutError: <twisted.test.test_tcp.HalfCloseTestCase testMethod=testCloseWriteCloser> (testCloseWriteCloser) still running at 120.0 secs

twisted.test.test_tcp.HalfCloseTestCase.testCloseWriteCloser
===============================================================================
[ERROR]
Traceback (most recent call last):
Failure: twisted.trial.util.DirtyReactorAggregateError: Reactor was unclean.
DelayedCalls: (set twisted.internet.base.DelayedCall.debug = True to debug)
<DelayedCall 0x29c0462c [-6.91413879395e-05s] called=0 cancelled=0 LoopingCall<0>(check, *(), **{})()>

twisted.test.test_tcp.HalfCloseTestCase.testCloseWriteCloser
===============================================================================
[ERROR]
Traceback (most recent call last):
Failure: twisted.trial.util.DirtyReactorAggregateError: Reactor was unclean.
Selectables:
<<class 'twisted.internet.tcp.Port'> of twisted.test.test_tcp.MyHCFactory on 53746>

twisted.test.test_tcp.HalfCloseTestCase.testCloseWriteCloser
===============================================================================
[ERROR]
Traceback (most recent call last):
Failure: twisted.internet.defer.TimeoutError: <twisted.test.test_tcp.HalfCloseTestCase testMethod=testWriteCloseNotification> (testWriteCloseNotification) still running at 120.0 secs

twisted.test.test_tcp.HalfCloseTestCase.testWriteCloseNotification
===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/test/test_tcp.py", line 1602, in tearDown
    self.assertEquals(self.client.closed, 0)
twisted.trial.unittest.FailTest: not equal:
a = 1
b = 0


twisted.test.test_tcp.HalfCloseTestCase.testWriteCloseNotification
===============================================================================
[ERROR]
Traceback (most recent call last):
Failure: twisted.trial.util.DirtyReactorAggregateError: Reactor was unclean.
DelayedCalls: (set twisted.internet.base.DelayedCall.debug = True to debug)
<DelayedCall 0x2bf7f4ac [-6.29425048828e-05s] called=0 cancelled=0 LoopingCall<0>(check, *(), **{})()>

twisted.test.test_tcp.HalfCloseTestCase.testWriteCloseNotification






More information about the Twisted-Python mailing list