[Twisted-web] web2.client pipelining (was: Web2: What is to be Done)

[Scott Lamb]

On Sep 14, 2006, at 11:36 AM, James Y Knight wrote:
> The client that's in there right now is a low-level http client.  
> The higher level web client (the connection manager, queuing,  
> pooling, etc.) is not completed. That is where the retry logic and  
> the determination of which requests can be pipelined and which  
> cannot would go.

Okay, cool. So is twisted.web.client (and its dependencies) not going  
to be included in this deprecation, or is there going to be a  
temporary regression in functionality? I have no strong opinion; an  
HTTP/1.0 client wasn't that useful to me to begin with.

> Please note that even *keepalive* can only be (reliably) used for  
> idempotent actions. For example, client sends request A, gets  
> response A, sends request B, but at the same time, server closes  
> the connection from a timeout condition.

Ugh, you're right. Looking over RFC 2616 section 8.1.4, they say the  
same thing you do:

    "Non-idempotent methods or sequences
    MUST NOT be automatically retried, although user agents MAY offer a
    human operator the choice of retrying the request(s)."

It does seem like this problem could have been avoided by the server  
(after gracefully closing) simply reading and discarding that request  
- the client would get the FIN and no RST. There's no circumstance in  
which a server handles a request and then cleanly closes the  
connection without a response, so it would know the server did not  
handle request B. It would have to retry, but it could do so safely  
even for non-idempotent requests. I wonder why they didn't do  
that...bandwidth?

This could be a practical problem for me. I'm stuck using a protocol  
created by a wannabe standards body that has mandated (1) a non- 
idempotent sequence and (2) the client never closing the connection.  
(And no, this is not the first time they've contradicted an  
underlying standard...)

-- 
Scott Lamb <http://www.slamb.org/>