[Twisted-Python] Writing a low-level network debugging tool

Phil Mayers p.mayers at imperial.ac.uk
Fri Nov 27 07:39:22 MST 2015


On 27/11/15 14:05, Jonathan Ballet wrote:

> * how long does it take to resolv the domain name to (at least) one of
> its IP address
>    - against a specified name server or using the system configured servers

That is relatively straightforward.

>    - how many tries did it require
>      * if there were several tries, the timing of each ones

Typically, application-layer code doesn't retry a DNS lookup; rather the 
c or other runtime will handle this, for example getaddrinfo() in glibc, 
according to settings read from /etc/resolv.conf or compiled-in defaults.

So it depends on whether you want to emulate "typical" application code, 
a specific application stack that may or may not do it's own resolution 
(e.g. modern browsers) or something else.

>
> * how long does it take to get the first bytes of the endpoint
>    - how long does it take to complete the TCP connection handshake
>    - the status of the packets exchanged (how many retries, how many
> packets lost, etc.)

Some of this is available in platform-specific APIs e.g. SIOCGSTAMP and 
TCP_INFO socket options available on Linux.

In general, any timings you make based on return of control from kernel 
will include error relating to system/scheduling issues. If you're 
concerned about getting raw, on-the-wire timings, this is extremely 
difficult without being in-kernel, and even then various issues - TCP 
offload for example - can end up hiding data from you.

> How far can I do this kind of things with Twisted? I know I can somewhat
> easily get the timings of the name resolution, the TCP connection
> handshake also and the time to first byte(s), but what about the
> packets? I haven't look at the code of Twisted Names yet, but if it's
> doing the DNS request by itself, I may be able to plug-in somewhere and
> have my request counter and the timers associated, but I'm not sure if
> the underlying details of the TCP protocol are exposed to the upper
> layer such as Twisted?

Only via platform-specific options.

To do this kind of thing "reliably", you'd need to reimplement TCP in 
user-space.

But the info above may be a helpful start.




More information about the Twisted-Python mailing list