Allow the Deferred returned by Agent.request to be cancelled

[exarkun]

Cancelling the Deferred returned by Agent.request should fail the Deferred with a CancelledError and either:

1. Cancel the connection setup attempt, if it is still in progress.
2. Disconnect abort the in-progress request, if the connection has been set up and a request has been initiated.

This depends on #4329 to handle the former case.

[glyph]

#4329 may be a perfectly reasonable way to handle this, but would depending on #1442 instead be a reasonable option, or would that involve rewriting too much code?

[exarkun]

Well, I basically finished #4329 already. And I like small changes.

[exarkun]

(In [28848]) Branching to 'webclient-request-cancel-4330'

[jonathanj]

A discussion on IRC brought up the point that Agent now uses endpoints instead of ClientCreator, which are not yet cancelable.

[glyph]

Replying to jonathanj:

A discussion on IRC brought up the point that Agent now uses endpoints instead of ClientCreator, which are not yet cancelable.

What? Yes they are:  http://twistedmatrix.com/documents/current/core/howto/endpoints.html#auto4

[exarkun]

Unfortunately #5485

[glyph]

Replying to exarkun:

Unfortunately #5485

Ouch. Well, thanks for the reference.

[habnabit]

(In [34323]) Branching to 'webclient-request-cancel-4330-2'

[habnabit]

(In [34324]) Initial implementation (with tests!) of Agent request cancellation.

refs #4330

[itamar]

Thanks for implementing this! We should have more cancellable Deferreds. This is looking pretty good. However, some improvements are needed:

1. Please write tests for cancelling after the response download has started, but hasn't finished. E.g. your download is too slow so you decide to stop it. To make sure you cover all cases maybe mention the covered state machine states in the cancellation tests, and make sure they match the full set of tests.
2. Needs a news file.
3. You should add a mention and/or example to the web client howto about the fact that you can cancel requests.
4. If request was cancelled, and any bytes at all have been written (or for completeness probably always is easier), rather than using loseConnection, abortConnection should be used. Given that you may be dealing with uncooperative server, and/or you're giving up and don't care about writing any more, having the connection close immediately is what you want, as opposed to waiting until all write buffers are cleared, which may take an arbitrarily long amount of time.

[itamar]

habnabit pointed out, re point 1, that there is no Deferred at that point in the code. So I'm not sure what I was referring to.

[habnabit]

(In [36263]) Branching to 'webclient-request-cancel-4330-3'

[habnabit]

(In [36264]) Merging forward (refs #4330)

[therve]

I've handle the review comments, build results here:  http://buildbot.twistedmatrix.com/boxes-supported?branch=/branches/webclient-request-cancel-4330-3