[Twisted-Python] Strange recursion error with twisted.web
Glyph
glyph at twistedmatrix.com
Wed Mar 10 16:11:06 MST 2021
> On Mar 10, 2021, at 3:09 AM, Peter Westlake <peter.westlake at pobox.com> wrote:
>
> On Tue, 9 Mar 2021, at 19:28, Glyph wrote:
>>
>>
>>> On Mar 9, 2021, at 4:54 AM, Peter Westlake <peter.westlake at pobox.com> wrote:
>>>
>>> I'm getting a "maximum recursion depth exceeded" error that appears to be coming from flatten(). The odd thing is that it only happens sometimes. The HTML that's being flattened does have a few Deferreds in it. Those come from function calls, which cache the results, which might explain why I only see the error on the first visit to the page (as far as I can tell).
>>>
>>> The system recursion limit is the standard 1000. My HTML is only nested a few tags deep, two orders of magnitude short of that. Is there anything about the way flatten() works that might cause this behaviour?
>>
>> flatten() can definitely result in some deep recursive stacks,
>> particularly in combination with synchronous Deferreds which have their
>> own accumulating stack costs. I'd be interested to see a minimal
>> reproducer for this though, I'm sure we could do a lot better.
>
> Here it is:
>
> import sys
> from twisted.internet import reactor, defer, task
> from twisted.web.template import flatten
>
> def output(stuff):
> sys.stdout.write(stuff.decode())
>
> def sync(reactor):
> return flatten(None, [defer.succeed(str(i)+'\n') for i in range(1000)], output)
>
> task.react(sync)
>
>
> It fails after printing 197 lines. The same sort of thing using deferLater instead of defer.succeed printed 1000 without error.
>
> Peter.
Would you mind filing a ticket in trac, or digging one up if you can find it? This problem rings a bell, and I think I might actually have some code for this lying around somewhere already.
-g
More information about the Twisted-Python
mailing list