[Twisted-Python] Strange recursion error with twisted.web

Glyph glyph at twistedmatrix.com
Wed Mar 10 16:11:06 MST 2021



> On Mar 10, 2021, at 3:09 AM, Peter Westlake <peter.westlake at pobox.com> wrote:
> 
> On Tue, 9 Mar 2021, at 19:28, Glyph wrote:
>> 
>> 
>>> On Mar 9, 2021, at 4:54 AM, Peter Westlake <peter.westlake at pobox.com> wrote:
>>> 
>>> I'm getting a "maximum recursion depth exceeded" error that appears to be coming from flatten(). The odd thing is that it only happens sometimes. The HTML that's being flattened does have a few Deferreds in it. Those come from function calls, which cache the results, which might explain why I only see the error on the first visit to the page (as far as I can tell). 
>>> 
>>> The system recursion limit is the standard 1000. My HTML is only nested a few tags deep, two orders of magnitude short of that. Is there anything about the way flatten() works that might cause this behaviour?
>> 
>> flatten() can definitely result in some deep recursive stacks, 
>> particularly in combination with synchronous Deferreds which have their 
>> own accumulating stack costs. I'd be interested to see a minimal 
>> reproducer for this though, I'm sure we could do a lot better.
> 
> Here it is:
> 
> import sys
> from twisted.internet import reactor, defer, task
> from twisted.web.template import flatten
> 
> def output(stuff):
>    sys.stdout.write(stuff.decode())
> 
> def sync(reactor):
>    return flatten(None, [defer.succeed(str(i)+'\n') for i in range(1000)], output)
> 
> task.react(sync)
> 
> 
> It fails after printing 197 lines. The same sort of thing using deferLater instead of defer.succeed printed 1000 without error.
> 
> Peter.

Would you mind filing a ticket in trac, or digging one up if you can find it? This problem rings a bell, and I think I might actually have some code for this lying around somewhere already.

-g


More information about the Twisted-Python mailing list