[Twisted-Python] understanding twisted, better error handling

[steven meiers]

hi,



i use twisted to send a web request over a proxy, this works over
treq.get(url, agent=myagent) without problems.

but in the process (started out with agent.request) i could not figure
out from what line of twisted code this error:

[Failure instance: Traceback (failure with no frames): <class
'twisted.web._newclient.RequestGenerationFailed'>:
[<twisted.python.failure.Failure builtins.TypeError: sequence item 0:
expected a bytes-like object, str found>]

came from.

btw, is there a way to have a debugger attached to twisted code that
shows what code is currently executed and has a step forward option?


the error does come up when you give agent.request a "GET" instead of a
b"GET".


heres the test code:
from twisted.python.log import err
from twisted.web.client import ProxyAgent
from twisted.internet import reactor, defer
from twisted.internet.endpoints import TCP4ClientEndpoint
import treq

def display(response):
    print("Received response")
    print(response.content)


def err(failure):
    print(failure)

def main():
    endpoint = TCP4ClientEndpoint(reactor, "223.25.102.186", 8080)
    agent = ProxyAgent(endpoint)
    d = agent.request("GET", bytes("http://somedomain.de", 'utf-8'))
    d.addCallbacks(display)
    d.addErrback(err)

if __name__ == "__main__":
    main()
    reactor.run()


this code will produce this error:
[Failure instance: Traceback (failure with no frames): <class
'twisted.web._newclient.RequestGenerationFailed'>:
[<twisted.python.failure.Failure builtins.TypeError: sequence item 0:
expected a bytes-like object, str found>]

and some .py files with line numbers. none of which really are the
problem.



as it turns out, the "GET" as a string gets problematic here:
https://github.com/twisted/twisted/blob/twisted-16.6.0/src/twisted/web/
_newclient.py#L651

requestLines.append(b' '.join([self.method, self.uri,
                b'HTTP/1.1\r\n']))


wouldnt it be better to have something like this:


try:
            requestLines.append(b' '.join([self.method, self.uri,
                b'HTTP/1.1\r\n']))
        except TypeError as e:
            raise TypeError("could not join: " + str(self.method) + " "
+ str(self.uri) + ' ' + 'HTTP/1.1\r\n' )




which will produce:
<class 'twisted.python.failure.Failure'> [Failure instance: Traceback:
<class 'TypeError'>: could not join: GET b'http://somedomain.de'
HTTP/1.1

to give the user a chance to find out what he did wrong?
...when he uses agent instead of treq for whatever reason.