[Twisted-Python] Unjelly - recursion limit reached
David K. Hess
dhess at verscend.com
Fri Nov 11 10:11:42 EST 2005
On Nov 11, 2005, at 8:46 AM, Jean-Paul Calderone wrote:
> On Fri, 11 Nov 2005 08:26:03 -0600, "David K. Hess"
> <dhess at verscend.com> wrote:
>> On Nov 7, 2005, at 8:42 AM, David K. Hess wrote:
>>> Bottom line, if all of these things are true, then I'm not sure
>>> what is going wrong. Is it possible that jelly is at times
>>> walking my object graph (which is interconnected to a certain
>>> degree and changes dynamically) in such a way that it has to
>>> perform a significant amount of recursion? In other words, maybe
>>> this isn't a bug but I just need to increase the runtime
>>> recursion limit?
>>> FYI, here's a snippet of the exception:
>> I hate replying to my own post but I'm beginning to believe that
>> recursion is the root of the problem and want to focus on it.
>> From the exception trace, it looks like it takes on average 3.5
>> stack frames to unjelly a nested object. With a default recursion
>> limit of 1000 stack frames, this means that Twisted is limited to
>> transferring an object graph that has a depth first traversal
>> through it of length 285 objects (roughly speaking). You could of
>> course easily create a linked list of 285 objects to force the
> Linked list? Hee hee. Crazy.
>> This is a pretty scary limitation. Is anybody working on a non-
>> recursion based version of jelly? Am I going to have to move to
>> Stackless Python?
> You can raise the recursion limit. See sys.setrecursionlimit().
Yes, but with the standard Python build on Windows, you can only go
up to 2000. So now, I can do 570 objects. I'm working on trying to
track down editbin.exe so I can raise the stack size on python.exe
and see how much farther I can go.
> Stackless has nothing to do with this. It won't help fix this
Are we talking about the same thing? The key architectural feature of
Stackless Python is "practically unlimited recursion". Forget about
all the tasklet and channel stuff.
> The real way to "fix" this is to re-implement the jellying process
> iteratively. Or, don't use such deeply nested objects.
My object graph is a natural and clean data structure. I don't want
to lose that just because of limitations in my tools.
Is this currently being addressed or in a road map? Am I unknowingly
volunteering for it? :-o
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
More information about the Twisted-Python