Opened 5 years ago

Closed 4 years ago

#4364 defect closed fixed (fixed)

Support for afterFoundGet in HTTPDownloader

Reported by: zimmer Owned by:
Priority: normal Milestone:
Component: web Keywords: HTTPClientFactory, HTTPDownloader, afterFoundGet, 302 HTTP Redirect
Cc: Branch:
Author: Launchpad Bug:

Description

In twisted's current version HTTPDownloader http://twistedmatrix.com/trac/browser/trunk/twisted/web/client.py#L353
doesn't support afterFoundGet like HTTPClientFactory http://twistedmatrix.com/trac/browser/trunk/twisted/web/client.py#L220 does. Here is a way to do (patch?) that.

Replace HTTPDownloader's _ _init_ _ with this

class HTTPDownloader(HTTPClientFactory):
    """Download to a file."""

    protocol = HTTPPageDownloader
    value = None

    def __init__(self, url, fileOrName,
                 method='GET', postdata=None, headers=None,
                 agent="Twisted client", supportPartial=0,
                 timeout=0, cookies=None, followRedirect=1,
                 redirectLimit=20, afterFoundGet = False):
        self.requestedPartial = 0
        if isinstance(fileOrName, types.StringTypes):
            self.fileName = fileOrName
            self.file = None
            if supportPartial and os.path.exists(self.fileName):
                fileLength = os.path.getsize(self.fileName)
                if fileLength:
                    self.requestedPartial = fileLength
                    if headers == None:
                        headers = {}
                    headers["range"] = "bytes=%d-" % fileLength
        else:
            self.file = fileOrName
        HTTPClientFactory.__init__(
            self, url, method=method, postdata=postdata, headers=headers,
            agent=agent, timeout=timeout, cookies=cookies,
            followRedirect=followRedirect, redirectLimit=redirectLimit,
            afterFoundGet = afterFoundGet)

This code (above) adds a new argument to the init of HTTPDownloader (afterFoundGet) and then also passes it to the unbound init of HTTPClientFactory.

Attachments (1)

after-found-get.diff (2.0 KB) - added by magcius 4 years ago.
Patch.

Download all attachments as: .zip

Change History (5)

Changed 4 years ago by magcius

Patch.

comment:1 Changed 4 years ago by magcius

  • Keywords review added; removed

comment:2 Changed 4 years ago by exarkun

  • Keywords review removed
  • Owner changed from jknight to exarkun
  • Status changed from new to assigned

Thanks! Patch looks good to me. I'll apply it with just a couple minor tweaks:

  1. add a .feature news fragment describing the new parameter
  2. describe the behavior being tested more specifically in the test method docstring
  3. add an extra blank line after the new test method, so there are two between it and the next test method

comment:3 Changed 4 years ago by exarkun

  • Resolution set to fixed
  • Status changed from assigned to closed

(In [30712]) Add afterFoundGet parameter to twisted.web.client.downloadPage

Author: magcius
Reviewer: exarkun
Fixes: #4364

Add a parameter to downloadPage to allow controlling what type of request is issued after a FOUND
response is received in response to a request. This afterFoundGet parameter works exactly the same
way the getPage parameter of the same name works.

comment:4 Changed 4 years ago by <automation>

  • Owner exarkun deleted
Note: See TracTickets for help on using tickets.