Opened 7 years ago

Closed 6 years ago

#4364 defect closed fixed (fixed)

Support for afterFoundGet in HTTPDownloader

Reported by: zimmer Owned by:
Priority: normal Milestone:
Component: web Keywords: HTTPClientFactory, HTTPDownloader, afterFoundGet, 302 HTTP Redirect
Cc: Branch:


In twisted's current version HTTPDownloader doesn't support afterFoundGet like HTTPClientFactory does. Here is a way to do (patch?) that.

Replace HTTPDownloader's _ _init_ _ with this

class HTTPDownloader(HTTPClientFactory):
    """Download to a file."""

    protocol = HTTPPageDownloader
    value = None

    def __init__(self, url, fileOrName,
                 method='GET', postdata=None, headers=None,
                 agent="Twisted client", supportPartial=0,
                 timeout=0, cookies=None, followRedirect=1,
                 redirectLimit=20, afterFoundGet = False):
        self.requestedPartial = 0
        if isinstance(fileOrName, types.StringTypes):
            self.fileName = fileOrName
            self.file = None
            if supportPartial and os.path.exists(self.fileName):
                fileLength = os.path.getsize(self.fileName)
                if fileLength:
                    self.requestedPartial = fileLength
                    if headers == None:
                        headers = {}
                    headers["range"] = "bytes=%d-" % fileLength
            self.file = fileOrName
            self, url, method=method, postdata=postdata, headers=headers,
            agent=agent, timeout=timeout, cookies=cookies,
            followRedirect=followRedirect, redirectLimit=redirectLimit,
            afterFoundGet = afterFoundGet)

This code (above) adds a new argument to the init of HTTPDownloader (afterFoundGet) and then also passes it to the unbound init of HTTPClientFactory.

Attachments (1)

after-found-get.diff (2.0 KB) - added by magcius 6 years ago.

Download all attachments as: .zip

Change History (5)

Changed 6 years ago by magcius


comment:1 Changed 6 years ago by magcius

  • Keywords review added; removed

comment:2 Changed 6 years ago by exarkun

  • Keywords review removed
  • Owner changed from jknight to exarkun
  • Status changed from new to assigned

Thanks! Patch looks good to me. I'll apply it with just a couple minor tweaks:

  1. add a .feature news fragment describing the new parameter
  2. describe the behavior being tested more specifically in the test method docstring
  3. add an extra blank line after the new test method, so there are two between it and the next test method

comment:3 Changed 6 years ago by exarkun

  • Resolution set to fixed
  • Status changed from assigned to closed

(In [30712]) Add afterFoundGet parameter to twisted.web.client.downloadPage

Author: magcius Reviewer: exarkun Fixes: #4364

Add a parameter to downloadPage to allow controlling what type of request is issued after a FOUND response is received in response to a request. This afterFoundGet parameter works exactly the same way the getPage parameter of the same name works.

comment:4 Changed 6 years ago by <automation>

  • Owner exarkun deleted
Note: See TracTickets for help on using tickets.