[Twisted-Python] Unjellying and circular references?

[Glyph]

Out official bug tracker is at https://twistedmatrix.com/trac/newticket — if you could file it there it would be great.  (Github login required.)

Thanks!

> On Dec 16, 2020, at 2:21 AM, Jasper Phillips <jasperisjaded at gmail.com> wrote:
> 
> 
> 
> 
>> On Fri, Dec 11, 2020 at 6:08 AM Jean-Paul Calderone <exarkun at twistedmatrix.com> wrote:
>>> On Fri, Dec 11, 2020 at 8:19 AM Jasper Phillips <jasperisjaded at gmail.com> wrote:
>> 
>>> I'm using perspective broker to transfer objects in a networked game, which I'm having trouble unjellying -- the remote versions wind up with dangling twisted.persisted.crefutil._Dereference instances, so don't match the original objects.
>>> 
>>> I'm seeing this for objects that have circular references to each other. I've refactored things to mostly avoid circular references and sidestep this, but have one remaining case where I find circular references mean clearer code that I'm reluctant to refactor.
>>> 
>>> Is there some trick I'm missing to avoid _Dereferences?
>> 
>> No, it's supposed to Just Work™ so you've found a bug in some part of the implementation.  If you can produce a minimal reproducing example then it may be worth a bug report.  But after that, I'd suggest investigating HTTP or AMP as a replacement for PB.
> 
> Here's a test case demonstrating the bug:
> 
> import sys
> from twisted.persisted.crefutil import _Dereference
> from twisted.spread             import pb, jelly
> 
> class RemoteCopyable( pb.Copyable, pb.RemoteCopy ):  pass
> jelly.globalSecurity.allowInstancesOf( RemoteCopyable )
> pb.setCopierForClassTree( sys.modules[__name__], pb.Copyable )
> 
> if __name__ == '__main__':
>     # circular object ref
>     cell           = RemoteCopyable()
>     cell.link      = RemoteCopyable()
>     cell.link.cell = cell
> 
>     # Mimic sending across network
>     broker         = pb.Broker()
>     serializedCell = broker.serialize( cell )
>     remoteCell     = broker.unserialize( serializedCell )
> 
>     print( _Dereference is type(remoteCell.link.cell) )
> 
> This bug stems from twisted.spread.flavors.RemoteCopy.setCopyableState(), which works fine for Python 2! But the UTF fix for Python 3 broke circular references. It looks like this:
> 
> def setCopyableState(self, state):
>     if _PY3:
>         state = {x.decode('utf8') if isinstance(x, bytes)
>                  else x:y for x,y in state.items()}
>     self.__dict__ = state
> 
> Here is a simple tweak that fixes the problem:
> 
> def setCopyableState(self, state):
>     if _PY3:
>         for x, y in list(state.items()):
>             if isinstance(x, bytes):
>                 del state[x]
>                 state[x.decode('utf8')] = y
>     self.__dict__ = state
> 
> 
> Basically the reference unwinding that takes place in twisted.spread.jelly._Unjellier._unjelly_reference()'s call to ref.resolveDependants() relies upon setCopyableStates()'s passed in state being used directly, such that all matching references' __dict__ point to the same object.
> 
> Hope this helps clarify. Is there some more formal location than this list that you'd like a bug report filed?
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20201226/fef651fc/attachment.htm>