Opened 4 years ago

Closed 4 years ago

#9226 defect closed fixed (fixed)

twisted.python.rebuild broken under PyPy2

Reported by: Daniel Sutcliffe Owned by: mark williams
Priority: normal Milestone: PyPy-support
Component: core Keywords:
Cc: Branch:
Author:

Description (last modified by Daniel Sutcliffe)

The twisted.test.test_rebuild.RebuildTests.testFileRebuild has a FAIL status on PyPy2 but that is not the half of it... other rebuild stuff also on top of it (updateInstance) can also potentially, seemingly randomly, fail under current PyPy.

I've traced the issue to problems with gc.get_referrers(Cls) not always bringing back everything it should... PyPy bug here logged here: https://bitbucket.org/pypy/pypy/issues/2612/

This patch provides a workaround until PyPy fixed

diff --git a/src/twisted/python/rebuild.py b/src/twisted/python/rebuild.py
index 22cdecf7b..df7ca3603 100644
--- a/src/twisted/python/rebuild.py
+++ b/src/twisted/python/rebuild.py
@@ -210,12 +210,13 @@ def rebuild(module, doLog=1):
             clazz.__module__ = module.__name__
     if newclasses:
         import gc
+        gc.collect()
     for nclass in newclasses:
         ga = getattr(module, nclass.__name__)
         if ga is nclass:
             log.msg("WARNING: new-class %s not replaced by reload!" % reflect.qual(nclass))
         else:
-            for r in gc.get_referrers(nclass):
+            for r in gc.get_objects():
                 if getattr(r, '__class__', None) is nclass:
                     r.__class__ = ga
     if doLog:

twisted.python.rebuild doesn't even really work under PY3 yet (ticket:8887 ticket:8213) so PyPy3 is def a no go ...

There's also another related rebuild PyPy ticket:8575 that may or may not be fixed ... probably needs a test ...

Change History (6)

comment:1 Changed 4 years ago by Daniel Sutcliffe

Description: modified (diff)

comment:2 Changed 4 years ago by Daniel Sutcliffe

Owner: set to Daniel Sutcliffe

Armin Rigo of PyPy worked out the exact problem here (see [PyPY ticket 2612](https://bitbucket.org/pypy/pypy/issues/2612/)) and has just landed [a commit which should fix](https://bitbucket.org/pypy/pypy/commits/3c4fb99e0c59) this.

However it's not going to be massively efficient so the suggestion is actually to implement in a similar (but more efficient) way to my original patch - have a PR to land once 8213's PR is merged.

comment:3 Changed 4 years ago by Daniel Sutcliffe

As of the release of PyPy 5.9.0 this problem does not show for both the 2.7 and 3.5 builds of PyPy.

twisted.test.test_rebuild.RebuildTests.testFileRebuild PASSES

comment:4 Changed 4 years ago by Craig Rodrigues

Owner: changed from Daniel Sutcliffe to mark williams

comment:6 Changed 4 years ago by Craig Rodrigues

Resolution: fixed
Status: newclosed
Note: See TracTickets for help on using tickets.