[Twisted-Python] Problems with PB and Jelly...
Jasper Phillips
jasper at peak.org
Mon Mar 24 16:02:17 MST 2003
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Sun, 23 Mar 2003, Brian Warner wrote:
Thanks for the response!
[snip: pb.Copyable copies all references found]
> > However, I'm sometimes getting an exception when the actual dictionary
> > copying is done, as something other than a dict is being copied into
> > __dict__. At this point the "jelType" is "dereference"...
[snip]
> If it's something else, let us know (and provide a small test case??) so we
> can fix it at the sprint.
Here's the relavent end of the error:
File "..\twisted\spread\flavors.py", line 386, in setCopyableState
self.__dict__ = state
exceptions.TypeError: __dict__ must be set to a dictionary
I've included a test case at the end of the message.
> > Damn, it looks like this might be the cuplrit. "reference" jelyTypes are
> > recursively descended into before they are stored, and if a dereference is
> > found before it's stored... some sort of _Dereference object is created?
> > An attempt is then made to copy this into __dict__, and boom.
>
> Yes, the current Jelly code looks for objects that are referenced multiple
> times in the same jellying call and marks them with "reference" tags. When
> another reference to the same object is detected, it is jellied with a
> "dereference" tag that points to the earlier "reference" marker. The "cook",
> "prepare", and "preserve" methods are used to implement these multiple
> phases. Circular or recursive references are handled because the reference
> number is allocated when we start to jelly the object, even though the state
> is not yet known.
The number is allocated earlier on, but it isn't stored for use by
"derference" jelTypes until after recursion, as I will attempt to describe:
Here are the relavent parts of jelly._Unjellier. Note that the reference is
stored after it's value is recursively computed -+
|
def _unjelly_reference(self, lst): |
refid = lst[0] |
exp = lst[1] |
o = self.unjelly(exp) <--------+
ref = self.references.get(refid) <--------+
if (ref is None):
self.references[refid] = o
elif isinstance(ref, NotKnown):
ref.resolveDependants(o)
self.references[refid] = o
else:
assert 0, "Multiple references with same ID!"
return o
This ends up causing the following method to return a _Dereference, rather
than a state dictionary that unjelly() would normally return. This then
results in the error I list above.
def _unjelly_dereference(self, lst):
refid = lst[0]
print refid #!!! added
x = self.references.get(refid)
if x is not None:
return x
der = _Dereference(refid)
self.references[refid] = der
return der
> This scheme will change on Tuesday. The "reference" tags will go away and be
> replaced by an implicit marker that is notionally inserted every time we
> start jellying a new mutable object. The "dereference" tags will then point
> to these implicit markers. This should improve performance quite a bit, and
> will pave the way to a combined jelly+banana extension module that should
> give an enormous speedup (doing everything in C).
Sounds good! No point in my suggesting a fix then. ;-) Will a new version
of twisted be released along with these changes, or would I have to go
through CVS?
[snip]
Here are 3 files for reproducing the "Unjellying a Circular Reference Bug".
Just stick them all in the same place, and start the server then the
client.
- -Jasper
Listing for objectBug.py
=-=-=-=-=-=-=-=-=-=-=-=-
from twisted.spread import pb
class ClassA( pb.Copyable, pb.RemoteCopy ):
def __init__( self ):
self.ref = ClassB( self )
class ClassB( pb.Copyable, pb.RemoteCopy ):
def __init__( self, ref ):
self.ref = ref
import sys
pb.setCopierForClassTree( sys.modules[__name__], pb.Copyable )
Listing for serverBug.py
=-=-=-=-=-=-=-=-=-=-=-=-
#/usr/bin/env python
from twisted.internet import app
from twisted.spread import pb
from twisted.cred import authorizer
import clientBug, objectBug
class MyPer( pb.Perspective ):
def attached( self, client, identity ):
self.client = client
return pb.Perspective.attached( self, client, identity )
def perspective_receive( self, obj ):
self.client.callRemote( "receive", obj )
class MyService( pb.Service ):
perspectiveClass = MyPer
def __init__( self, serviceParent, auth ):
pb.Service.__init__( self, "MyService", serviceParent, auth )
def startServer():
myApp = app.Application("pbServer")
auth = authorizer.DefaultAuthorizer( myApp )
s = MyService( myApp, auth )
s.createPerspective( "player1" ).makeIdentity( "password1" )
myApp.listenTCP( 9000, pb.BrokerFactory( pb.AuthRoot( auth )))
myApp.run( save=0 )
if __name__ == '__main__':
startServer()
Listing for clientBug.py
=-=-=-=-=-=-=-=-=-=-=-=-
#/usr/bin/env python
from twisted.internet import reactor
from twisted.spread import pb
import objectBug
class MyClient( pb.Referenceable ):
def connect( self, ipAddress, port, user, password ):
defer = pb.connect( ipAddress, port, user, password,
"MyService", client=self, timeout=3 )
defer.addCallback( self.connected )
def connected( self, perspective ):
perspective.callRemote( "receive", objectBug.ClassA() )
def remote_receive( self, obj ):
print "Object received back"
def startClient():
client = MyClient()
client.connect( "localhost", 9000, "player1", "password1" )
reactor.run()
if __name__ == '__main__':
startClient()
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE+f45+8EpjZ7/X9bIRAr0yAJ9P2dwozcY478sVL6GKemwYoHAZmwCfYbhT
ppjMznAw88wjXItJDczxTVU=
=LvPx
-----END PGP SIGNATURE-----
More information about the Twisted-Python
mailing list