Designate/Blueprints/MiniDNS

Summary
The intent of this blueprint is to describe the reasoning for, advantages and disadvantages of implementing a "Mini DNS" server directly in Designate. Low-level Implementation details will not be discussed.

Reasoning
Designate's current "Backends" implementation is flawed, leaving many opportunities for for Designate and a Backend to become out of sync, and being excessively complex to implement certain kinds of Bakends (For example: writing BIND9 zonefiles to N DNS servers). The reasons for these failings can be broken down into two core ideas:


 * 1) Designate does not perform Backend operations transactionally. Implementing a two-phase commit protocol spanning Storage and Backends has been attempted in the past, the complexity of such a solution for non database based backends (BIND9, NSD etc) introduced a significant amount of complexity, and to be truly reliable, would require a fix to reason #2 below.
 * 2) Non-Database based Backends (for example BIND9) require that each change is applied to each and every DNS server, and that the changes are applied in the same same order across all servers. Designate currently lacks a reliable way to perform either of these tasks. Additionally, any implementation would be required to behave correctly under the two-phase commit protocol discussed above.

Most solutions to both of those problems are prohibitively complex. While the Mini-DNS proposal is certainly complex, it provides several additional advantages that no other solution provides. The additional features Mini-DNS can allow for make the complexly acceptable.

High Level Implementation
Designate will implement a Mini-DNS server capable of handling a very limited number of DNS operations, from a pre-defined list of "Supported DNS clients". End user QUERY's will not be processed by this service. Initially, this service will perform two core tasks:


 * 1) Provide an AXFR source for the public DNS servers. This involves accepting both SOA and AXFR QUERY's from a known sub-subset of supported DNS servers (BIND9, PowerDNS, NSD etc).
 * 2) Publish DNS NOTIFY's upon zone changes, triggering an AXFR by the public DNS servers.

This service will be implemented in two parts:


 * 1) A DNS protocol implementation (Proof of concept DIY code exists, as do off-the-shelf libraries like dnspython)
 * 2) A new designate-minidns service. This service will act as what is commonly called a "Hidden" or "Stealth" nameserver. Listening for AXFR QUERY's and sending NOTIFY's.

Advantages
Immediate Advantages:


 * 1) Transactional zones changes can be performed by simply performing a transactional update to Storage. No two-phase commit protocol is required.
 * 2) Ensuring all public DNS servers receive all zones changes is simplified to sending N DNS NOTIFY's
 * 3) Ensuring zone changes are applied on all public DNS servers in the appropriate order is no longer an issue, AXFR's resolve this issue with no additional work on our part.
 * 4) Zone changes are applied to the public DNS servers asynchronously, and DNS's in-built zone refresh interval ensures eventual consistency in the case of a missed updated (i.e. a nameserver was down during the change)
 * 5) Backend implementations are reduced to the minimal amount of functionality required to add and remove zones
 * 6) For BIND9, we can implement this using RNDC over TCP
 * 7) For PowerDNS, we can implement this using the same DB interactions as we currently perform.
 * 8) TODO: Understand how NSD / FreeIPA DNS / Akamai EDNS etc etc would tie-in

Future Advantages:

With this code in place, we have a solid foundation for additional future features;


 * 1) Ability to support RFC2136 (nsupdate style) Dynamic DNS updates
 * 2) Ability to support inbound AXFRs from customers
 * 3) Ability to DNSSEC sign zones on-demand
 * 4) Consider the possibility of having multiple views of a DNS zone (The most common example of this is for split horizon). Without on-demand signing, every possible view of a zone must be pre-signed.
 * 5) Simplifies the implementation of the Server Pools proposal

Disadvantges

 * 1) We have to maintain our own Mini-DNS server.
 * 2) The core DNS RFC's are simple, but have been extended so many times that being truly compliant will all DNS RFC's is hard. The scope of RFCs required to be implement for the feature set we require is limited, making this issue less pressing.
 * 3) Some DNS clients are buggy, send dodgy or corrupt data, or otherwise misbehave. As the number of use-cases for Mini-DNS grows from AXFR -> publicly accessible services (Dynamic DNS, inbound AXFR etc), the requirement to gracefully handle these bugs will increase the complexity of the implementation.

Update Storage layer to return objects rather than simple dictonaries
storage-objects This has advantages beyond MiniDNS, for example, validation rules can be segregated into isolated classes for each object.

This step will create the following objects, have Storage return them, and update the remaining code to handle these changes:


 * Quota
 * Server
 * TLD
 * TsigKey
 * Tenant[1]
 * Domain
 * RecordSet
 * Record
 * Blacklist

[1]: Tenant is very much an "odd one out", and is only used in a V1 admin API extension.

Update Network API return's to match Storage changes
networkapi-objects

This step will create the following objects, have NetworkAPI return them, and update the remaining code to handle these changes:


 * FloatingIP

This step is purely to keep the various APIs consistent.

Update the Record object to make available the structured, as well as textual representations of the records's RData
structured-rdata

This allows the RData to be serliazed by the DNS protocol implemementation chosen. Currently designate/objects has an object for record. This object (designate.objects.Record) would provide from_text, to_text methods. designate.objects.base would provide a from_dict, to_dict methods. The various types of records supported ("A", "AAAA", "CNAME", "MX", "SRV", "TXT", "SPF", "NS", "PTR", "SSHFP") would each have a separate type that inherit from designate.objects.Record. Storage's get_record, find_record and find_records would be able to return one of the specific record types rather than the generic designate.objects.Record.

''For storage's create_record, update_record to take in one of these record types (rather than a dict that as it is currently) and be able to create/update it, we would need to modify either the central and/or api layer to supply one of the record types rather than a dict. Should this be done as another blueprint?''

Build the core `designate-mdns` service
core-mdns-service

This step involves building the service class/daemon that listens on tcp/udp 53, and dispatches incoming requests to a stub request handler.

This does not yet involve parsing/responding to DNS queries.

Build out the base DNS protocol implementation
dns-implementation

Here, we write the base protocol implementation - capable of encoding and decoding the foundational DNS constructs:


 * Packet        - The DNS Packet itself, holds the Header, Questions, Answers, Authorities and Additionals
 * Header        - The DNS Packet Header
 * Name          - Representation of a DNS Name, with logic for handle DNS Name compression/pointers
 * Question      - Representation of a DNS Query, holds the name, type and class of the Query.
 * ResourceRecord - Represents a single Record in a Result[1].

[1]: This one may be skipped, with the Record object instead implementing this piece.

Extend the Record+Rdata objects from Steps 1 and 3 to include DNS protocol encoding and decoding methods
record-rdata-dns-implementation

These should be short, rtype specific encoders and decoders that make use of the structured RData objects created in Step 3.

Extend the Record+Rdata objects
Extend the Record+Rdata objects from Steps 1 and 3 to include translation methods to convert to/from DNSPython's representations

Extend the `designate-mdns` service from Step 4 to decode incoming Queries, gather results from Storage, and construct + return answers
designate-mdns-functional

This will be limited to standard, non AXFR/TSIG, queries for now.

Extend the `designate-mdns` service from Step 7 to support AXFR
designate-mdns-axfr

This will be limited to non-TSIG AXFR queries for now.

Extend Designate's TSIG Key support to allow scoped TSIG keys
For example, it should be possible to create a TSIG key with:


 * Scope = Global. This would be unlimited, and can AXFR all zones.


 * Scope = Zone, Resource = . This would be limited to a single zone.


 * Scope = Pool, Resource = . This would be limited to a zones in a pool.

Extend the `designate-mdns` with TSIG Key support
designate-mdns-tsig

Extend the `designate-mdns` service from Step 8 to support authentication and authorization via the TSIG Key extensions from Step 9

Extend the `designate-mdns` service from Step 10 with NOTIFY support
designate-mdns-notify

designate-mdns will be notified by Central when changes are made to a zone. The service is issue a NOTIFY to all slave nameservers.

Before Pools, this will be a pre-configured list of slaves to NOTIFY, after Pools, the list of slaves to NOTIFY will be loaded dynamically.

Update Backends to only handle Zone create/update/delete
Backends will no longer need to be called for RecordSet or Record changes, so this code can now safely be removed