Jump to: navigation, search

Difference between revisions of "Designate/Blueprints/Filtering API"

(API Changes)
m (API Changes)
Line 47: Line 47:
  
 
* Create a '''/records''' endpoint for admin use only, to search for addresses across all tenants.  Use the original '''/zones/{id}/recordsets''' endpoint for individual tenants to search for an address within a single zone.  Having tenants search for an address across all of their zones, however, is not clear.  Perhaps the '''/records''' endpoint could be used for this, with an all_tenants option to differentiate between searching across one tenant and all tenants?
 
* Create a '''/records''' endpoint for admin use only, to search for addresses across all tenants.  Use the original '''/zones/{id}/recordsets''' endpoint for individual tenants to search for an address within a single zone.  Having tenants search for an address across all of their zones, however, is not clear.  Perhaps the '''/records''' endpoint could be used for this, with an all_tenants option to differentiate between searching across one tenant and all tenants?
 
* Create a new sub-resource of zones '''ipaddresses''', and create a '''/zones/ipaddresses''' endpoint. The only valid filter parameter on this resource would be “address” and only the GET operation will be valid on this resource (other operations will be No-Ops on this resource). One of the justifications for this sub-resource is that a given IP address could be pointed to from zones owned by different tenants and therefore we cannot use the existing /zones/{zone-id}/recordsets endpoint to perform filtering across multiple zones.
 
  
 
* Create a '''/search/records''' endpoint.  Similar to point 1 otherwise.
 
* Create a '''/search/records''' endpoint.  Similar to point 1 otherwise.
  
 +
* Create a new sub-resource of zones '''ipaddresses''', and create a '''/zones/ipaddresses''' endpoint. The only valid filter parameter on this resource would be “address” and only the GET operation will be valid on this resource (other operations will be No-Ops on this resource). One of the justifications for this sub-resource is that a given IP address could be pointed to from zones owned by different tenants and therefore we cannot use the existing /zones/{zone-id}/recordsets endpoint to perform filtering across multiple zones.
  
 
For illustration, below is a table showing a filter for the IP address ADDR, constructed in the different ways posed above.  All of these requests are GET requests.
 
For illustration, below is a table showing a filter for the IP address ADDR, constructed in the different ways posed above.  All of these requests are GET requests.
Line 58: Line 57:
 
|-
 
|-
 
! One recordset !! Entire tenant !! All tenants
 
! One recordset !! Entire tenant !! All tenants
 +
|-
 +
| /zones/{z-id}/recordsets/{rs-id}/records?address=ADDR || /zones/ipaddresses?address=ADDR || /zones/ipaddresses?address=ADDR&all_tenants=true
 
|-
 
|-
 
| /zones/{z-id}/recordsets/{rs-id}/records?address=ADDR || /records?address=ADDR || /records?address=ADDR&all_tenants=true
 
| /zones/{z-id}/recordsets/{rs-id}/records?address=ADDR || /records?address=ADDR || /records?address=ADDR&all_tenants=true
|-
 
| /zones/{z-id}/recordsets/{rs-id}/records?address=ADDR || /zones/ipaddresses?address=ADDR || /zones/ipaddresses?address=ADDR&all_tenants=true
 
 
|-
 
|-
 
| /zones/{z-id}/recordsets/{rs-id}/records?address=ADDR || /search/records?address=ADDR || /search/records?address=ADDR&all_tenants=true
 
| /zones/{z-id}/recordsets/{rs-id}/records?address=ADDR || /search/records?address=ADDR || /search/records?address=ADDR&all_tenants=true

Revision as of 15:27, 25 June 2014

Overview

Gerrit Patch []
Launchpad Blueprint [1]

Filtering provides the ability to qualify the result set returned by a query to the designate api and thereby provides a search capability. Filtering will ultimately be available on all collections - zones, record sets, rdata of record sets, and pools. However, the first implementation of Filtering will restrict filtering to zones and record sets. Where applicable, privileged users (support and admin) will be able to perform filtering across all tenants by specifying an additional parameter, all_tenants=true, in their queries.

Filtering is controlled using query parameters which match the name of the attribute being filtered. It is *not* required that all attributes are available as filter targets, but the majority will be.

Filters are either an exact match or substring match. Wildcard and regular expression matching may be introduced in a later revision of the v2 API, so long as the implementation is backward compatible.

If the filtering request is successful, basic unique information is returned including links for retrieving more details. The basic information would include Id and Name.


Pagination of results will take advantage of proposed Designate pagination (https://blueprints.launchpad.net/designate/+spec/pagination)

Change History

This spec has been partially implemented. The following is a list of features that have been completed as well as a list of features that are pending.

Completed

  • Basic filtering for:
    • Blacklists
    • Records
    • Recordsets
    • TLDs
    • Zones
  • Substring search using SQL pattern matching

Pending

  • Filtering by address
  • Filtering across all tenants
  • Restrictions on substring search (currently substring searching is unrestricted)


API Changes

The large part of the Filtering API will be extensions of existing GET APIs on existing Resources (zones, record sets, rdata of record sets, and pool) to specify filters are query parameters.

Additionally, where applicable, privileged users (support and admin) will be able to perform filtering across all tenants by specifying an additional parameter, all_tenants=true, in their GET requests.

Filtering by address is a special case and has undergone much discussion. This is primarily because a tenant may want to search for an address over all of their records (which could be distributed among several recordsets, in several zones), or over an individual recordset. In addition, admins need to be able to search for an address over all tenants' records. Several solutions have been proposed, some of which are listed below:

  • Create a /records endpoint for admin use only, to search for addresses across all tenants. Use the original /zones/{id}/recordsets endpoint for individual tenants to search for an address within a single zone. Having tenants search for an address across all of their zones, however, is not clear. Perhaps the /records endpoint could be used for this, with an all_tenants option to differentiate between searching across one tenant and all tenants?
  • Create a /search/records endpoint. Similar to point 1 otherwise.
  • Create a new sub-resource of zones ipaddresses, and create a /zones/ipaddresses endpoint. The only valid filter parameter on this resource would be “address” and only the GET operation will be valid on this resource (other operations will be No-Ops on this resource). One of the justifications for this sub-resource is that a given IP address could be pointed to from zones owned by different tenants and therefore we cannot use the existing /zones/{zone-id}/recordsets endpoint to perform filtering across multiple zones.

For illustration, below is a table showing a filter for the IP address ADDR, constructed in the different ways posed above. All of these requests are GET requests.

One recordset Entire tenant All tenants
/zones/{z-id}/recordsets/{rs-id}/records?address=ADDR /zones/ipaddresses?address=ADDR /zones/ipaddresses?address=ADDR&all_tenants=true
/zones/{z-id}/recordsets/{rs-id}/records?address=ADDR /records?address=ADDR /records?address=ADDR&all_tenants=true
/zones/{z-id}/recordsets/{rs-id}/records?address=ADDR /search/records?address=ADDR /search/records?address=ADDR&all_tenants=true
Verb Resource Description
GET /zones?name=example.com&match-type=exact&all_tenants=true Return zones that have names that exactly match the specified filter (“name”) across all tenants. Note: Only admins and support can search across all tenants.
GET zones/{zone-id}/recordsets?name=example.com&match-type=substr From the specified zone, return recordsets that have name that the specified filter (“name”) is a substring of.
GET /zones/{zone-id}/recordsets?address=10.1.2.3&match-type=exact From the specified zone for this tenant, return Address recordsets that exactly match the specified IP address.
GET /zones/ipaddresses?address=172.192.112.82&match-type=exact&all_tenants=true From all the zones belonging to all tenants, return Address recordsets that exactly match the specified IP address.
GET /zones/ipaddresses?address=172.192.112.82&match-type=exact From all the zones belonging to this tenant, return Address recordsets that exactly match the specified IP address.
GET,

PUT, PATH, POST, DELETE

/zones/ipaddresses No Op.
PUT, PATH, POST, DELETE /zones/ipaddresses?address=172.192.112.82 No Op.

These should be done through /zone/{zone-id}/recordsets/

High level description of implementation logic

Following is a high level description of the flow of logic. More detailed request/response formats are specified in the examples and use case section below.

1. User issues a search request on a resource with appropriate filters. Optionally, two other parameters can be specified – match-type (exact/substr) and all_tenants (true/false).

For example: GET /v2/zones?name=example.com&match-type=exact HTTP/1.1

Host: dns.provider.com
Accept: application/json
X-Auth-Token: KeyStoneAuth_*****

2. In middleware.py:KeystoneMiddleware::process_request(), during the construction of DesignateContext

    • determined that all_tenants is set to True signaling intent to perform this operation on all tenants
      • perform policy check (in the API before we pass on to Central)
        • if confirmed that the user has this right to perform the operation across all tenants,
          • set context.all_tenants=True
          • let the request proceed to Central
        • if user cannot be confirmed to have the right to perform the operation across all tenants, reject the request with appropriate message

3. In Central:

  • issue the correct SQL query against the Storage database using the request query parameter as WHERE clause value.
  • return to designate-api

4. Api - return the result set.

API Details

Privileged user searching for a specified zone name across all tenants

Request:

 GET /v2/zones?name=example.com&match-type=exact&all_tenants=true HTTP/1.1
 Host: dns.provider.com
 Accept: application/json
 X-Auth-Token: KeyStoneAuth_*****
 X-Tenant-ID:

Response:

 {
  "zone": {
    "id": "a86dba58-0043-4cc6-a1bb-69d5e86f3ca3",
    "pool_id": "7d62d10d-3a16-4828-85dd-7b3fdc0ba989",
    "project_id": "4335d1f0-f793-11e2-b778-0800200c9a66",
    "name": "example.com.",
    "email": "joe@example.com.",
    "ttl": 7200,
    "serial": 1351800588,
    "status": "ACTIVE",
    "version": 1,
    "created_at": "...",
    "updated_at": null,
    "links": {
      "self": "https://dns.provider.com/v2/zones/a86dba58-0043-4cc6-a1bb-69d5e86f3ca3"
    }
  }
 }

Privileged user searching in one tenant

Request:

 GET /v2/zones?name=example.com&match-type=exact HTTP/1.1
 Host: dns.provider.com
 Accept: application/json
 X-Auth-Token: KeyStoneAuth_*****
 X-Tenant-ID: 12345

Response:

(Empty - this particular tenant does not have domain example.com):

 {
  "zone": {
  }
 }

A Customer searching across all tenants

Throw exception when policy check in API shows that tenant 1234 is not privileged to search across all tenants

A Customer searching in one tenant

Request:

 GET /v2/zones?name=exp.com&match-type=exact HTTP/1.1
 Host: dns.provider.com
 Accept: application/json
 X-Auth-Token: KeyStoneAuth_*****
 X-Tenant-ID: 12345

Response:

 {
  "zone": {
    "id": "a66dba58-0043-4cc6-a1bb-69d5e86f3ca3",
    "pool_id": "6d62d10d-3a16-4828-85dd-7b3fdc0ba989",
    "project_id": "5335d1f0-f793-11e2-b778-0800200c9a66",
    "name": "exp.com.",
    "email": "jane@exp.com.",
    "ttl": 7200,
    "serial": 1351800588,
    "status": "ACTIVE",
    "version": 1,
    "created_at": "...",
    "updated_at": null,
    "links": {
      "self": "https://dns.provider.com/v2/zones/c86dba58-0043-4cc6-a1bb-69d5e86f3cc1"
    }
  }
 }


A Privileged user searching for a specified recordset by name within a specified zone for a given tenant

This is the ability to search for a record by name in a given zone that might have lots of records. By definition, a specific zone belongs to one tenant so even a privileged user (support/admin) is able to search only within any specific account but not across all accounts at the same time.

Request:

 GET zones/{zone-id}/recordsets?name=example&match-type=substr
 Host: dns.provider.com
 Accept: application/json
 X-Auth-Token: KeyStoneAuth_*****
 X-Tenant-ID: 44441

Response:

 {
  "recordsets": [{
    "id": "9e27811d-0320-4179-abb7-0e00e371e25b",
    "zone_id": "a86dba58-0043-4cc6-a1bb-69d5e86f3ca3",
    "name": "www.example.org.",
    "type": "A",
    "ttl": 3600,
    "status": "ACTIVE",
    "version": 1,
    "created_at": "...",
    "updated_at": null,
    "links": {
      "self": "https://dns.provider.com/v2/zones/a86dba58-0043-4cc6-a1bb-69d5e86f3ca3/recordsets/9e27811d-0320-4179-abb7-0e00e371e25b"
    }
  }, {
    "id": "dedf6879-fd9a-41d6-a7c2-eeac316fa7b3",
    "zone_id": "a86dba58-0043-4cc6-a1bb-69d5e86f3ca3",
    "name": "_xmpp-server._tcp.example.org.",
    "type": "SRV",
    "ttl": 3600,
    "status": "ACTIVE",
    "version": 1,
    "created_at": "...",
    "updated_at": null,
    "links": {
      "self": "https://dns.provider.com/v2/zones/a86dba58-0043-4cc6-a1bb-69d5e86f3ca3/recordsets/dedf6879-fd9a-41d6-a7c2-eeac316fa7b3"
    }
  }],
  "links": {
    "self": "https://dns.provider.com/v2/zones/a86dba58-0043-4cc6-a1bb-69d5e86f3ca3/recordsets",
    "next": "https://dns.provider.com/v2/zones/a86dba58-0043-4cc6-a1bb-69d5e86f3ca3/recordsets?marker=dedf6879-fd9a-41d6-a7c2-eeac316fa7b3"
  }
 }


A Customer searching for a specified recordset by name within a specified zone

A customer is able to search on zone that belongs to customer By definition, a specific zone belongs to one tenant. Besides, ordinary tenants are not privileged to perform any operation on any other tenant id.

Request:

 GET zones/{zone-id}/recordsets?name=exam&match-type=substr
 Host: dns.provider.com
 Accept: application/json
 X-Auth-Token: KeyStoneAuth_*****
 X-Tenant-ID: 33377

Response:

 {
  "recordsets": [{
    "id": "dedf6879-fd9a-41d6-a7c2-eeac316fa7b3",
    "zone_id": "986dba58-0043-4cc6-a1bb-69d5e86f3ca9",
    "name": "_xmpp-server._tcp.example.org.",
    "type": "SRV",
    "ttl": 3600,
    "status": "ACTIVE",
    "version": 1,
    "created_at": "...",
    "updated_at": null,
    "links": {
      "self": "https://dns.provider.com/v2/zones/986dba58-0043-4cc6-a1bb-69d5e86f3ca9/recordsets/dedf6879-fd9a-41d6-a7c2-eeac316fa7b3"
    }
  }],
 }


A Privileged user searching for records pointing to a specific IP Address across all tenants

Request:

 GET /v2/zones/ipaddresses?address=172.192.112.82&match-type=exact&all_tenants=true HTTP/1.1
 Host: dns.provider.com
 Accept: application/json
 X-Auth-Token: KeyStoneAuth_*****
 X-Tenant-ID:

Response:

 {
    "records": [
        {
         "type": "A",
         "address": "172.192.112.82",
         “id”: “9e27811d-0320-4179-abb7-0e00e371e25b”,
         "zone_id": "a86dba58-0043-4cc6-a1bb-69d5e86f3ca3",
         “name”: "www.example.org.",
         "links": {
           "self": "https://dns.provider.com/v2/zones/986dba58-0043-4cc6-a1bb-69d5e86f3ca9/recordsets/dedf6879-fd9a-41d6-a7c2-eeac316fa7b3"
    }
    },
        {
        "type": "A",
        "address": "172.192.112.82",
        “id”: “4e27811d-0320-4179-abb7-0e00e371e25c”,
        "zone_id": "b46dba58-0043-4cc6-a1bb-69d5e86f3cc1",
        “name”: "exp.com.",
        "links": {
         "self": "https://dns.provider.com/v2/zones/436dba58-0043-4cc6-a1bb-69d5e86f3ca9/recordsets/dedf6879-fd9a-41d6-a7c2-eeac316fa7d2"
    }
    }
 ] }


A Privileged user searching for records pointing to a specific IP Address in a given tenant

Request:

 GET /v2/zones/ipaddresses?address=172.192.112.82&match-type=exact HTTP/1.1
 Host: dns.provider.com
 Accept: application/json
 X-Auth-Token: KeyStoneAuth_*****
 X-Tenant-ID: 31476

Response:

 {
    "records": [
           {
            "type": "A",
            "address": "172.192.112.82",
            “id”: “4e27811d-0320-4179-abb7-0e00e371e25c”,
            "zone_id": "b46dba58-0043-4cc6-a1bb-69d5e86f3cc1",
            “name”: "exp.com.",
            "links": {
              "self": https://dns.provider.com/v2/zones/436dba58-0043-4cc6-a1bb-69d5e86f3ca9/recordsets/dedf6879-fd9a-41d6-a7c2-eeac316fa7d2 
            }
    }
 ] }

A Customer searching for records pointing to a specific IP Address across all tenants

Exception - no customer can search across all tenants

A Customer searching for records pointing to a specific IP in the customer's own space

Request:

 GET /v2/zones/ipaddresses?address=172.192.112.82&match-type=exact HTTP/1.1
 Host: dns.provider.com
 Accept: application/json
 X-Auth-Token: KeyStoneAuth_*****
 X-Tenant-ID: 44441

Response:

 {
    "records": [
           {
            "type": "A",
            "address": "172.192.112.82",
            “id”: “9e27811d-0320-4179-abb7-0e00e371e25b”,
            "zone_id": "a86dba58-0043-4cc6-a1bb-69d5e86f3ca3",
            “name”: "www.example.org.",
            "links": {
                 "self": "https://dns.provider.com/v2/zones/986dba58-0043-4cc6-a1bb-69d5e86f3ca9/recordsets/dedf6879-fd9a-41d6-a7c2-eeac316fa7b3"
         }
     }
 ] }

Database Changes

N/A

Proposed Test Scenarios