exarkun at divmod.com
Fri Sep 23 15:37:43 MDT 2005
On Fri, 23 Sep 2005 14:27:41 -0700, Scott Lamb <slamb at slamb.org> wrote:
>Hi David and all,
>I stumbled across your twisted web2-client branch from google. I need to
>make a specialized HTTP load generator that uses Keep-Alives.
>twisted.web2.client seems the best way, so I'm trying to make this branch
>work for me.
>My load generator needs to make a bunch of sequences of authenticated
>POSTs, picking up a cookie along the way.
>I've stumbled onto a bug: the first request will return a 401Unauthorized.
>HTTPClientFactory.handleStatus_401 deals with this...but the postdata is
>This code generates the new request:
> req = Request(request.method, request.uri, request.args,
> request.headers, request.stream)
>It looks like web2.stream.IStream is single-use - it returns a None when
>there's no more data. I think this - or maybe a IResettableStream subclass
>- needs a reset() method.
>I've got a patch that allowed me to proceed. I wouldn't recommend applying
>as-is - I only messed with the MemoryStream that I'm using, and there's no
>unit test. I want to send this in before I forget about it, though.
Many stream sources cannot be rewound. Implementing reset for IStream would require many implementations to hold their entire contents in memory. This is completely unreasonable. Creating a separate interface for streams which could be rewound would avoid this problem, but at the same time limit the actual implementations your code could work with to an extreme subset.
Instead of extending IStream or creating a new interface, you probably want to wrap request.stream before passing it to the request. The wrapper can take care of providing restartability. It can also prevent the concurrency bugs that handing a single stream to multiple Requests will present.
More information about the Twisted-web