Opened 11 years ago

Closed 11 years ago

#2580 defect closed duplicate (duplicate)

twisted.web2.stream infinite loop

Reported by: gthomas Owned by:
Priority: low Milestone:
Component: web2 Keywords: web2 ProducerStream
Cc: Branch:
Author:

Description

I'm trying to write a "streaming XMLRPC server" - thus I modified xmlrpclib.dumps to not dump everything in memory, but take a function (write) and use that to publish chunks.

Either I use ProducerStream or StreamProducer, I see a "maximum recursion depth exceeded" exception as seen in the attached log.

Can anyone suggest anything?

Thanks.

Attachments (2)

ProducerStream.log (4.9 KB) - added by gthomas 11 years ago.
error log
stream_err.py (14.6 KB) - added by gthomas 11 years ago.
test case

Download all attachments as: .zip

Change History (7)

Changed 11 years ago by gthomas

Attachment: ProducerStream.log added

error log

comment:1 Changed 11 years ago by Jean-Paul Calderone

Please attach a minimal example which reproduces this traceback.

comment:2 Changed 11 years ago by David Reid

Probably related to, if not a duplicate of #1993.

comment:3 in reply to:  2 Changed 11 years ago by gthomas

Replying to dreid:

Probably related to, if not a duplicate of #1993.

Exactly.

Also tried another way: (reinvite ProducerStream) - implement IByteStream, with write and finish methods for the streaming xmlrpclib.dumps, but for long (100kb) data the last part of the data is left there: no more read attempts, so I can't answer None to finish it ?!?

I'll attach a simplified skeleton soon.

Changed 11 years ago by gthomas

Attachment: stream_err.py added

test case

comment:4 Changed 11 years ago by gthomas

Priority: normallow
Resolution: duplicate
Status: newclosed

OK, found an error/a solution for my special case:

As I implemented IByteStream, read always gave back Deferred, and data went as callback. Now I read thoroughly http.py, iweb.py and stream.py, and saw that

a) StreamProducer gets the result stream, b) StreamProducer stops producing after the first Deferred as answer for read.

I modified my IByteStream so now it gives back data if possible, and Deferred only if there's no data yet; the deferreds are fed with all remainig data after a write call.

Sorry to trouble you, it seems that I could circumvent #1993. Anyway if it could helped anyone (and possible/suggested) I may put this "solution" (with the traps) in a cleare form on a wiki or such thing.

Thanks again.

comment:5 Changed 7 years ago by <automation>

Owner: jknight deleted
Note: See TracTickets for help on using tickets.