Version 13 (modified by Richard Wall, 7 years ago) (diff)

add dnskey and rrsig ticket links


An attempt to understand the pieces of EDNS0 and DNSSEC and the changes necessary to have Twisted Names support it:


Twisted.names includes widely used public APIs that have existed for over twelve years (r440).

However many of these existing APIs are inelegant and too broad. eg

So we will introduce new EDNS related classes with carefully designed, minimal public APIs. New APIs will be designed for maximum usability but as narrow as possible, to minimise future backwards compatibility complications.

Twisted has a strict wiki:CompatibilityPolicy. So we will aim for zero changes to existing public APIs.

Existing classes will be wrapped where appropriate and in other cases, shared functionality will be re-factored for reuse in new classes.

With the introduction of the new EDNS APIs, old sub-optimal APIs can then be deprecated and eventually removed in future releases.

Before designing and implementing new APIs, we will examine existing open source DNS projects for APIs and code that we can reuse / integrate.

Use of an existing Python DNS library (or a C library with Python bindings) will be considered if it is well written, actively maintained, well tested and has a compatible licence. However the potential time savings from reusing an existing library will be weighed against the overhead of introducing an additional Twisted install dependency. eg

Below are some specific examples of new APIs that we plan to introduce.


OPT record

  1. Explicit OPT record class/type
    1. wrap the existing RRHeader class - to minimise code duplication and maintenance overhead by only requiring a single implementation of RRHeader byte string parser.
    2. extracts OPT specific header fields from the equivalent RRHeader fields.
    3. Maximum UDP datagram size field
    4. Arbitrary extension support (attribute/value pairs in RDATA section)
    5. OPTHeader will encode and decode OPT variable options in an underlying UnknownRecord payload class.
  2. Discussion
    1. Teach cache about it - MUST NOT BE CACHED
    2. Teach recursive resolver about it - MUST NOT BE FORWARDED
    3. EDNSMessage will remove OPT records during decoding so they may never be seen by cache or clients if EDNSMessage becomes the standard message factory for DNSDatagramProtocol.
    4. Teach loaders about - MUST NOT BE LOADED FROM MASTER FILE
    5. Extended label ("0" "1" type) parsing (REJECTED BY 2671bis)

Extended Message object with additional EDNS0 items

  1. Write a new EDNSMessage class.
    1. It will wrap the existing dns.Message class.
    2. Its decode method will extract an OPT record from the additional section and merge its attributes with those of the underlying non-EDNS Message.
    3. Its encode method will check for EDNS specific attributes (eg RCODE > 15) and add an OPT record to the additional records of its underlying Message class before encoding.
  2. Discussion
    1. maxSize (Message has it already, can we re-use it?)


  1. A new EDNSAgent client API
    1. Automatically send EDNS0 queries
    2. Fall back to DNS queries when detecting server does not support EDNS0?
    3. It will be modelled on twisted.web.client.Agent which was introduced to replace the old twisted.web.client APIs.
    4. Agents can be chainable / composable.
    5. Eg An EDNSAgent will be introduced with a fixed maxUdpPayloadSize parameter.
    6. Another AutoMTUDiscoveryAgent can wrap EDNSAgent and will reissue queries with successively lower advertised UDP payload size until a response is received.
    7. A future DNSSECValidatingAgent can validate responses by issuing queries for the DS and DNSKEY records from the root down.
  2. Discussion
    1. How to choose max UDP size?
      1. Accept max UDP size as configuration?
      2. Determine it from system automatically?
      3. Server and client need this


  1. A new EDNS server API
    1. Send EDNS0 responses to EDNS0 queries
    2. (Do not send EDNS0 responses to DNS queries)
    3. Ensures the EDNS requests are responded to using the advertised UDP payload size.
    4. Limits additional RRSETs so that responses fit the client max UDP payload size or marks response Messages truncated.
    5. Sets EDNS RCODEs in response to malformed EDNS requests.

RRSET improvements

DNSSEC seems to rely on stable RRSETs. eg for signatures, for predictable caching of records and signatures, for predictable truncation of large DNSSEC responses.

There appear to be various problems with twisted.names current handling of RRSETs.

Canonical Form and Order of Resource Records

RRSETs must be arranged in canonical order before their signatures are calculated / verified.

This ticket will introduce an algorithm for sorting records according to the rules described in

Sorting appears to be the responsibility of the verifying client not the server.

The fact that Bind has config options for changing the order of RRSETS on the server side and the client side suggests that the canonical ordering of records should be done only for the purpose of DNSSEC validation. It should probably not change the order of records returned by various IResolver methods.

Serving RRSETs

twisted.names.dns.Message should follow the guidance in RFC2181 regarding the handling of TTLs for records in the same RRSET. Alternatively consider handling this higher up in t.n.authority.FileAuthority, t.n.cache, t.n.resolver etc.

Receiving RRSETs

twisted.names.client should somehow signal an error if it receives RRSETs whose RRs have different TTLs.

twisted.names.resolver,root should discard RRSETs whose RRs have diffent TTLs.

  • "Should a client receive a response containing RRs from an RRSet with differing TTLs, it should treat this as an error. If the RRSet concerned is from a non-authoritative source for this data, the client should simply ignore the RRSet,"

Caching RRSETs

twisted.names.cache should follow the RRSET ranking guidance when serving and replacing items in its cache.

  • "Servers must never merge RRs from a response with RRs in their cache to form an RRSet. If a response contains data that would form an RRSet with data in a server's cache the server must either ignore the RRs in the response, or discard the entire RRSet currently in the cache, as appropriate."
  • "When considering whether to accept an RRSet in a reply, or retain an RRSet already in its cache instead, a server should consider the relative likely trustworthiness of the various data"

Selective truncate

twisted.names.dns.Message currently truncates messages based on the combined length of the answer, auth, and additional sections. Instead it (or something higher up) should look at the length of the answers and only set the truncate flag if the entire answers RRSET is greater than the maxPayloadSize. If there is room, the auth and additional records can also be included, but entire RRSETs must be included or none.


Much of this work (including tests) has already been implemented by BobNovas in two large patches attached to original tickets: #5450, #5453, #5454. Look there before implementing anything from scratch.

New DNSSEC Records and Lookup Methods

  1. DNSSEC introduces six new resource record types. Each new record type will require a new dns.Record subclass and a new lookupMethod added to t.i.interfaces.IResolver, t.n.common.ResolverBase and a corresponding free function in t.n.client.
    1. #6664 DNSKey
    2. #6665 RRSIG
    3. NSEC
    4. DS
    5. NSEC3
    6. NSEC3Param
  2. For ease of review, this work can be split into six tickets.
  3. These new records can be implemented independently of EDNS and independently of DNSSEC validation and new DNSSEC related message headers.
  4. Initially, this will allow twisted.names clients to explicitly request these DNSSEC related records.
  5. lookupZone will return DNSSEC related records when transferring from DNSSEC authoritative servers.
  6. t.n.secondary.SecondaryAuthority will download DNSSEC records and serve them when queries specifically ask for them by type.
  7. t.n.authority.FileAuthority will load DNSSEC records and serve them when queries specifically ask for them by type.

Security-aware Non-validating Client

A twisted.names.client will be able to generate EDNS queries with one or both the DO bit and the AD bit set.

It will examine the state of the AD bit in the response to determine whether the upstream resolver claims to have validated the records in the response.

If the DO bit was set, it will expect to receive DNSSEC related records in the response.

Validating Client

A twisted.names.client which sends DO + CD flagged queries and performs its own validation of the returned DNSSEC signatures.

TODO: needs more thought.

Validating Recursive / Forwarding Server

This can use the Validating client API above, but may need to do some processing of answers based on the query flags.

TODO: needs more thought.

DNSSEC Aware Authoritative Server

A twisted.names.authority API which knows where and when to include RRSIG, DNSKEY, NSEC, NSEC3 records etc with responses.

The actual generation of the DNSSEC records can be performed using external tools such as dnssec-signzone

TODO: needs more thought.



  3. 2181: Clarifications to the DNS Specification
  4. 4697: Observed DNS Resolution Misbehavior
  5. 5625: DNS Proxy Implementation Guidelines


  1. 6891: Extension Mechanisms for DNS (EDNS(0))


  1. 4033: DNS Security Introduction and Requirements
  2. 4034: Resource Records for the DNS Security Extensions
  3. 4035: Protocol Modifications for the DNS Security Extensions
  4. 5155: DNS Security (DNSSEC) Hashed Authenticated Denial of Existence
  5. 6781: DNSSEC Operational Practices, Version 2
  6. 6840: Clarifications and Implementation Notes for DNS Security (DNSSEC)
  7. 3833: Threat Analysis of the Domain Name System (DNS)

MISC / TODO / Notes

  1. "If a query message with more than one OPT RR is received, a FORMERR (RCODE=1) MUST be returned." RFC6891 6.1.1
    1. Multiple OPT RR will be detected in EDNSMessage.decode
    2. If multiple OPT RR are detected, set EDNSMessage.decodingErrors.append(EFORMAT)
    3. DNSServerFactory.messageReceived checks message.decodingErrors
    4. If not empty, it sets message.rCode = message.decodingErrors.pop(0) before sending the message back to client without further processing.
    5. If DNSServerFactory.verbose logging is enabled, all message.decodingErrors will be logged.
    6. I tested Google DNS and Bind Authoritative DNS responses to multi OPT queries.
      1. Google seems to ignore all but the first OPT RR and responds without error.
      2. Bind responds with RCODE=1
        } else if (rdtype == dns_rdatatype_opt) {
        * The name of an OPT record must be ".", it
        * must be in the additional data section, and
        * it must be the first OPT we've seen.
                if (!dns_name_equal(dns_rootname, name) ||
                    msg->opt != NULL)
                skip_name_search = ISC_TRUE;
                skip_type_search = ISC_TRUE;
      4. PowerDNS appears to ignore all but the first OPT RR (untested)
        bool getEDNSOpts(const MOADNSParser& mdp, EDNSOpts* eo)
          if(mdp.d_header.arcount && !mdp.d_answers.empty()) {
            BOOST_FOREACH(const MOADNSParser::answers_t::value_type& val, mdp.d_answers) {
              if(val.first.d_place == DNSRecord::Additional && val.first.d_type == QType::OPT) {
                EDNS0Record stuff;
                uint32_t ttl=ntohl(val.first.d_ttl);
                memcpy(&stuff, &ttl, sizeof(stuff));
                eo->d_Z = ntohs(stuff.Z);
                OPTRecordContent* orc =
                  return false;
                return true;
          return false;
  1. "EXTENDED-RCODE Forms the upper 8 bits of extended 12-bit RCODE" RFC6891 6.1.3
    1. Decode: EDNSMessage.decode will merge the RCODE and OPTHeader.extendedRCODE setting the rCode attribute.
    2. Encode: If rCode is > 15, EDNSMessage.encode will set the lower 4 bits as RCODE and set the upper 8 bits on a new or existing OPTHeader which will be added to the additional section.
  1. "VERSION Indicates the implementation level of the setter" RFC6891 6.1.3
    1. "If a responder does not implement the VERSION level of the request, then it MUST respond with RCODE=BADVERS"
      1. Decode: in EDNSMessage.decode, if version > 0, EDNSMessage.decodingErrors.append(EBADVERS)
  1. "This document assigns EDNS Extended RCODE 16 to "BADVERS" in the DNS RCODES registry"
    1. Create dns.EBADVERS = 16
  1. "DO DNSSEC OK bit as defined by [RFC3225]."
    1. Set EDNSMessage.dnssecOK during decode using value from OPTHeader.dnssecOK.
    2. Add or update an existing OPTHeader instance during encode.