[Twisted-Python] Failure of twisted.test.test_plugin.DeveloperSetupTests.test_freshPyReplacesStalePyc under pypy2

[Glyph]

On Thu, Jul 13, 2017, at 04:22 PM, Daniel Sutcliffe wrote:
> > On Thu, Jul 13, 2017 at 5:57 AM, Daniel Sutcliffe <dansut at gmail.com> wrote:
> >> I shall do my best as my intention is to use Twisted in a live
> >> environment with PyPy so it would be great if the project viewed these
> >> builds as required to be clean sometime in the future. This was just
> >> the first ERROR I attacked, it was always my intention to look into
> >> the others too, which I am also seeing.
> 
> On Thu, Jul 13, 2017 at 11:31 AM, Craig Rodrigues
> <rodrigc at crodrigues.org> wrote:
> > Thanks for your motivation.
> 
> No problem, this is one of the few ways I can give back to the Twisted
> project that is ultimately helping me achieve my goals;
> besides getting to the bottom of these type of things is a fun
> learning exercise for me :)
> 
> > Please keep going, and share your findings on the mailing list.
> >
> > Last year, Mark Williams worked on fixing many Twisted unit tests to either
> > pass or skip on Pypy, and even found one issue with Pypy itself (
> > https://bitbucket.org/pypy/pypy/issues/2335/maximum-recursion-depth-exceeded-with
> > ).
> 
> This one is likely to fall on deaf ears in the PyPy project...
> The reason for the ERROR and then FAIL is as I originally thought,
> that __import__() under pypy is not loading .pyc files where the .py
> is missing and:
>   http://doc.pypy.org/en/latest/config/objspace.lonepycfiles.html
> tl;dr - they view it as dangerous due to pyc binary variations and
> only allow by build time configuration option.
> 
> So the next question I guess is how to deal with this:
> 
> - presumably my first step is to log a bug in Trac as any change will
> require one, I had only held off from doing this until I was more sure
> of the cause of the issue.
> 
> - the FAIL goes away if the .py is not removed from the plugins dir
> and it seems to me that the .py removal is not actually required to
> strictly test what
> twisted.test.test_plugin.DeveloperSetupTests.test_freshPyReplacesStalePyc
> purports to be testing - so that could fix this test
> 
> - the fact that the test ERRORs and then carries on and FAILs later in
> a strange fashion is due to the fact twisted.plugin.getCache() simply
> does a log.err() on encountering the odd situation - so maybe this
> could be dealt with better knowing that PyPy does different things
> than CPython here... ?
> 
> Could the more experienced please advise :)

It is totally fine to skip this test on PyPy.  It's testing the ability
to interact with a particular feature of the CPython interpreter, which
is intentionally absent on PyPy.  Ideally, the implementation would not
enumerate that module on PyPy since it isn't going to be importable as a
module.

In fact, many of the failing tests are for obscure corner cases which
can and should be skipped so that we can declare PyPy a supported
platform, then work on fixing them after the fact :-).

> > If you feel there are improvements or additions to the buildbot jobs using
> > Pypy, please share your thoughts.  The buildbot master.cfg file is at
> > https://github.com/twisted-infra/braid/tree/master/services/buildbot/master
> 
> I have not looked into why buildbot comes up with a different error to
> the one I did on various platforms - fix for my situation and see if
> it fixes the pypy buildbot before investigating?
> 
> The other error that I and the buildbot see on pypy is 12x (always) of
>   Failure: twisted.internet.defer.TimeoutError
> for
>   twisted.protocols.test.test_tls.TLSMemoryBIOTests.test_hugeWrite_TLSv1_1
> which is odd as if I run this test on its own it succeeds every time...
> Thoughts on this appreciated but probably should start a new thread for
> that...
> 
> Cheers
> /dan
> -- 
> Daniel Sutcliffe <dansut at gmail.com>
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python