Jump to: navigation, search

Difference between revisions of "Designate/Blueprints/Filtering API"

(API Changes)
Line 17: Line 17:
  
 
Pagination of results will take advantage of proposed Designate pagination  (https://blueprints.launchpad.net/designate/+spec/pagination)
 
Pagination of results will take advantage of proposed Designate pagination  (https://blueprints.launchpad.net/designate/+spec/pagination)
== Change History ==
 
So far, this feature has been partially implemented.  Here is a list of the tasks that have been completed and the tasks that are still pending.
 
 
=== Completed ===
 
* Basic filtering for:
 
** Blacklists
 
** Records
 
** Recordsets
 
** TLDs
 
** Zones
 
* Wildcards for the above using SQL pattern matching
 
 
=== Pending ===
 
* Filtering for IP Addresses
 
* Discuss wildcard restrictions (wildcards are currently unrestricted)
 
** match_type option related to this
 
* all_tenants option
 
 
 
== API Changes ==
 
== API Changes ==
  
 
The large part of the Filtering API will be extensions of existing GET APIs on existing Resources (zones, record sets, rdata of record sets, and pool) to specify filters are query parameters.
 
The large part of the Filtering API will be extensions of existing GET APIs on existing Resources (zones, record sets, rdata of record sets, and pool) to specify filters are query parameters.
 +
One new additional sub-resource of zones ('''ipaddresses'''), a collection sub-resource, is introduced to support the search for IP address records by specifying the IP address as a filter in the request. The only valid filter parameter on this resource is “address”.  Only the GET operation will be valid on this resource; other operations will be No-Ops on this resource. One of the justifications for this sub-resource is that a given IP address could be pointed to from zones owned by different tenants and therefore we cannot use the existing /zones/{zone-id}/recordsets endpoint to perform filtering across multiple zones.
  
The API implementation for IP address filtering is still under discussion.  Many different solutions have been proposed, including the following:
+
Additionally, where applicable, privileged users (support and admin) will be able to perform filtering across all tenants by specifying an additional parameter, ''all_tenants=true'', in their GET requests.
* Create a '''/records''' endpoint for admin use only, to search for records across all tenants.  Keep the '''/zones/{id}/recordsets/{id}/records''' available for individual tenant use.  For both, address would be specified as a URL parameter, such as: '''/records?address="..."''' and '''/zones/{id}/recordsets/{id}/records?address="..."'''.
 
 
 
* Create a collection sub-resource of zones called '''ipaddresses''' and create a '''/zones/ipaddresses''' endpoint.  This resource could be used by both admins and individual tenants, and would use the all_tenants option to ensure that the appropriate search range is used.  The only valid filter parameters on this resource would be “address”.  Only the GET operation will be valid on this resource; other operations will be No-Ops on this resource. One of the justifications for this sub-resource is that a given IP address could be pointed to from zones owned by different tenants and therefore we cannot use the existing /zones/{zone-id}/recordsets endpoint to perform filtering across multiple zones.
 
 
 
* Create a '''/search''' endpoint for admin use only, to search for any resource across all tenants.  Each endpoint would have its own endpoint within '''/search'''; for example, '''/search/records''' or '''/search/zones'''.  For addresses, use '''/search/records?address="..."''' to search across all tenants, and '''/zones/{id}/recordsets/{id}/records?address="..."''' to search within a single tenant.
 
 
 
 
 
Additionally, where applicable, privileged users (support and admin) will be able to perform filtering across all tenants by specifying an additional parameter, ''all_tenants=true'', in their GET requests (this parameter is tentative as well; if a multi-endpoint approach is used, then an all_tenants option may not be necessary).
 
 
 
The following table illustrates examples of an address filter in the different forms proposed above.  Both a search across one tenant and a search among all tenants are shown.
 
 
 
{| class="wikitable"
 
|-
 
! Verb !! Search All Tenants !! Search One Tenant
 
|-
 
| GET || /records?address=172.192.112.82 || /zones/{id}/recordsets/{id}/records?address=172.192.112.82
 
|-
 
| GET || /zones/ipaddresses?address=172.192.112.82&all_tenants=true || /zones/{id}/recordsets/{id}/records?address=172.192.112.82
 
|-
 
| GET || /search/records?address=172.192.112.82 || /zones/{id}/recordsets/{id}/records?address=172.192.112.82
 
|}
 
  
 
{| class="wikitable"
 
{| class="wikitable"

Revision as of 23:29, 24 June 2014

Overview

Gerrit Patch []
Launchpad Blueprint [1]

Filtering provides the ability to qualify the result set returned by a query to the designate api and thereby provides a search capability. Filtering will ultimately be available on all collections - zones, record sets, rdata of record sets, and pools. However, the first implementation of Filtering will restrict filtering to zones and record sets. Where applicable, privileged users (support and admin) will be able to perform filtering across all tenants by specifying an additional parameter, all_tenants=true, in their queries.

Filtering is controlled using query parameters which match the name of the attribute being filtered. It is *not* required that all attributes are available as filter targets, but the majority will be.

Filters are either an exact match or substring match. Wildcard and regular expression matching may be introduced in a later revision of the v2 API, so long as the implementation is backward compatible.

If the filtering request is successful, basic unique information is returned including links for retrieving more details. The basic information would include Id and Name.


Pagination of results will take advantage of proposed Designate pagination (https://blueprints.launchpad.net/designate/+spec/pagination)

API Changes

The large part of the Filtering API will be extensions of existing GET APIs on existing Resources (zones, record sets, rdata of record sets, and pool) to specify filters are query parameters. One new additional sub-resource of zones (ipaddresses), a collection sub-resource, is introduced to support the search for IP address records by specifying the IP address as a filter in the request. The only valid filter parameter on this resource is “address”. Only the GET operation will be valid on this resource; other operations will be No-Ops on this resource. One of the justifications for this sub-resource is that a given IP address could be pointed to from zones owned by different tenants and therefore we cannot use the existing /zones/{zone-id}/recordsets endpoint to perform filtering across multiple zones.

Additionally, where applicable, privileged users (support and admin) will be able to perform filtering across all tenants by specifying an additional parameter, all_tenants=true, in their GET requests.

Verb Resource Description
GET /zones?name=example.com&match-type=exact&all_tenants=true Return zones that have names that exactly match the specified filter (“name”) across all tenants. Note: Only admins and support can search across all tenants.
GET zones/{zone-id}/recordsets?name=example.com&match-type=substr From the specified zone, return recordsets that have name that the specified filter (“name”) is a substring of.
GET /zones/{zone-id}/recordsets?address=10.1.2.3&match-type=exact From the specified zone for this tenant, return Address recordsets that exactly match the specified IP address.
GET /zones/ipaddresses?address=172.192.112.82&match-type=exact&all_tenants=true From all the zones belonging to all tenants, return Address recordsets that exactly match the specified IP address.
GET /zones/ipaddresses?address=172.192.112.82&match-type=exact From all the zones belonging to this tenant, return Address recordsets that exactly match the specified IP address.
GET,

PUT, PATH, POST, DELETE

/zones/ipaddresses No Op.
PUT, PATH, POST, DELETE /zones/ipaddresses?address=172.192.112.82 No Op.

These should be done through /zone/{zone-id}/recordsets/

High level description of implementation logic

Following is a high level description of the flow of logic. More detailed request/response formats are specified in the examples and use case section below.

1. User issues a search request on a resource with appropriate filters. Optionally, two other parameters can be specified – match-type (exact/substr) and all_tenants (true/false).

For example: GET /v2/zones?name=example.com&match-type=exact HTTP/1.1

Host: dns.provider.com
Accept: application/json
X-Auth-Token: KeyStoneAuth_*****

2. In middleware.py:KeystoneMiddleware::process_request(), during the construction of DesignateContext

    • determined that all_tenants is set to True signaling intent to perform this operation on all tenants
      • perform policy check (in the API before we pass on to Central)
        • if confirmed that the user has this right to perform the operation across all tenants,
          • set context.all_tenants=True
          • let the request proceed to Central
        • if user cannot be confirmed to have the right to perform the operation across all tenants, reject the request with appropriate message

3. In Central:

  • issue the correct SQL query against the Storage database using the request query parameter as WHERE clause value.
  • return to designate-api

4. Api - return the result set.

API Details

Privileged user searching for a specified zone name across all tenants

Request:

 GET /v2/zones?name=example.com&match-type=exact&all_tenants=true HTTP/1.1
 Host: dns.provider.com
 Accept: application/json
 X-Auth-Token: KeyStoneAuth_*****
 X-Tenant-ID:

Response:

 {
  "zone": {
    "id": "a86dba58-0043-4cc6-a1bb-69d5e86f3ca3",
    "pool_id": "7d62d10d-3a16-4828-85dd-7b3fdc0ba989",
    "project_id": "4335d1f0-f793-11e2-b778-0800200c9a66",
    "name": "example.com.",
    "email": "joe@example.com.",
    "ttl": 7200,
    "serial": 1351800588,
    "status": "ACTIVE",
    "version": 1,
    "created_at": "...",
    "updated_at": null,
    "links": {
      "self": "https://dns.provider.com/v2/zones/a86dba58-0043-4cc6-a1bb-69d5e86f3ca3"
    }
  }
 }

Privileged user searching in one tenant

Request:

 GET /v2/zones?name=example.com&match-type=exact HTTP/1.1
 Host: dns.provider.com
 Accept: application/json
 X-Auth-Token: KeyStoneAuth_*****
 X-Tenant-ID: 12345

Response:

(Empty - this particular tenant does not have domain example.com):

 {
  "zone": {
  }
 }

A Customer searching across all tenants

Throw exception when policy check in API shows that tenant 1234 is not privileged to search across all tenants

A Customer searching in one tenant

Request:

 GET /v2/zones?name=exp.com&match-type=exact HTTP/1.1
 Host: dns.provider.com
 Accept: application/json
 X-Auth-Token: KeyStoneAuth_*****
 X-Tenant-ID: 12345

Response:

 {
  "zone": {
    "id": "a66dba58-0043-4cc6-a1bb-69d5e86f3ca3",
    "pool_id": "6d62d10d-3a16-4828-85dd-7b3fdc0ba989",
    "project_id": "5335d1f0-f793-11e2-b778-0800200c9a66",
    "name": "exp.com.",
    "email": "jane@exp.com.",
    "ttl": 7200,
    "serial": 1351800588,
    "status": "ACTIVE",
    "version": 1,
    "created_at": "...",
    "updated_at": null,
    "links": {
      "self": "https://dns.provider.com/v2/zones/c86dba58-0043-4cc6-a1bb-69d5e86f3cc1"
    }
  }
 }


A Privileged user searching for a specified recordset by name within a specified zone for a given tenant

This is the ability to search for a record by name in a given zone that might have lots of records. By definition, a specific zone belongs to one tenant so even a privileged user (support/admin) is able to search only within any specific account but not across all accounts at the same time.

Request:

 GET zones/{zone-id}/recordsets?name=example&match-type=substr
 Host: dns.provider.com
 Accept: application/json
 X-Auth-Token: KeyStoneAuth_*****
 X-Tenant-ID: 44441

Response:

 {
  "recordsets": [{
    "id": "9e27811d-0320-4179-abb7-0e00e371e25b",
    "zone_id": "a86dba58-0043-4cc6-a1bb-69d5e86f3ca3",
    "name": "www.example.org.",
    "type": "A",
    "ttl": 3600,
    "status": "ACTIVE",
    "version": 1,
    "created_at": "...",
    "updated_at": null,
    "links": {
      "self": "https://dns.provider.com/v2/zones/a86dba58-0043-4cc6-a1bb-69d5e86f3ca3/recordsets/9e27811d-0320-4179-abb7-0e00e371e25b"
    }
  }, {
    "id": "dedf6879-fd9a-41d6-a7c2-eeac316fa7b3",
    "zone_id": "a86dba58-0043-4cc6-a1bb-69d5e86f3ca3",
    "name": "_xmpp-server._tcp.example.org.",
    "type": "SRV",
    "ttl": 3600,
    "status": "ACTIVE",
    "version": 1,
    "created_at": "...",
    "updated_at": null,
    "links": {
      "self": "https://dns.provider.com/v2/zones/a86dba58-0043-4cc6-a1bb-69d5e86f3ca3/recordsets/dedf6879-fd9a-41d6-a7c2-eeac316fa7b3"
    }
  }],
  "links": {
    "self": "https://dns.provider.com/v2/zones/a86dba58-0043-4cc6-a1bb-69d5e86f3ca3/recordsets",
    "next": "https://dns.provider.com/v2/zones/a86dba58-0043-4cc6-a1bb-69d5e86f3ca3/recordsets?marker=dedf6879-fd9a-41d6-a7c2-eeac316fa7b3"
  }
 }


A Customer searching for a specified recordset by name within a specified zone

A customer is able to search on zone that belongs to customer By definition, a specific zone belongs to one tenant. Besides, ordinary tenants are not privileged to perform any operation on any other tenant id.

Request:

 GET zones/{zone-id}/recordsets?name=exam&match-type=substr
 Host: dns.provider.com
 Accept: application/json
 X-Auth-Token: KeyStoneAuth_*****
 X-Tenant-ID: 33377

Response:

 {
  "recordsets": [{
    "id": "dedf6879-fd9a-41d6-a7c2-eeac316fa7b3",
    "zone_id": "986dba58-0043-4cc6-a1bb-69d5e86f3ca9",
    "name": "_xmpp-server._tcp.example.org.",
    "type": "SRV",
    "ttl": 3600,
    "status": "ACTIVE",
    "version": 1,
    "created_at": "...",
    "updated_at": null,
    "links": {
      "self": "https://dns.provider.com/v2/zones/986dba58-0043-4cc6-a1bb-69d5e86f3ca9/recordsets/dedf6879-fd9a-41d6-a7c2-eeac316fa7b3"
    }
  }],
 }


A Privileged user searching for records pointing to a specific IP Address across all tenants

Request:

 GET /v2/zones/ipaddresses?address=172.192.112.82&match-type=exact&all_tenants=true HTTP/1.1
 Host: dns.provider.com
 Accept: application/json
 X-Auth-Token: KeyStoneAuth_*****
 X-Tenant-ID:

Response:

 {
    "records": [
        {
         "type": "A",
         "address": "172.192.112.82",
         “id”: “9e27811d-0320-4179-abb7-0e00e371e25b”,
         "zone_id": "a86dba58-0043-4cc6-a1bb-69d5e86f3ca3",
         “name”: "www.example.org.",
         "links": {
           "self": "https://dns.provider.com/v2/zones/986dba58-0043-4cc6-a1bb-69d5e86f3ca9/recordsets/dedf6879-fd9a-41d6-a7c2-eeac316fa7b3"
    }
    },
        {
        "type": "A",
        "address": "172.192.112.82",
        “id”: “4e27811d-0320-4179-abb7-0e00e371e25c”,
        "zone_id": "b46dba58-0043-4cc6-a1bb-69d5e86f3cc1",
        “name”: "exp.com.",
        "links": {
         "self": "https://dns.provider.com/v2/zones/436dba58-0043-4cc6-a1bb-69d5e86f3ca9/recordsets/dedf6879-fd9a-41d6-a7c2-eeac316fa7d2"
    }
    }
 ] }


A Privileged user searching for records pointing to a specific IP Address in a given tenant

Request:

 GET /v2/zones/ipaddresses?address=172.192.112.82&match-type=exact HTTP/1.1
 Host: dns.provider.com
 Accept: application/json
 X-Auth-Token: KeyStoneAuth_*****
 X-Tenant-ID: 31476

Response:

 {
    "records": [
           {
            "type": "A",
            "address": "172.192.112.82",
            “id”: “4e27811d-0320-4179-abb7-0e00e371e25c”,
            "zone_id": "b46dba58-0043-4cc6-a1bb-69d5e86f3cc1",
            “name”: "exp.com.",
            "links": {
              "self": https://dns.provider.com/v2/zones/436dba58-0043-4cc6-a1bb-69d5e86f3ca9/recordsets/dedf6879-fd9a-41d6-a7c2-eeac316fa7d2 
            }
    }
 ] }

A Customer searching for records pointing to a specific IP Address across all tenants

Exception - no customer can search across all tenants

A Customer searching for records pointing to a specific IP in the customer's own space

Request:

 GET /v2/zones/ipaddresses?address=172.192.112.82&match-type=exact HTTP/1.1
 Host: dns.provider.com
 Accept: application/json
 X-Auth-Token: KeyStoneAuth_*****
 X-Tenant-ID: 44441

Response:

 {
    "records": [
           {
            "type": "A",
            "address": "172.192.112.82",
            “id”: “9e27811d-0320-4179-abb7-0e00e371e25b”,
            "zone_id": "a86dba58-0043-4cc6-a1bb-69d5e86f3ca3",
            “name”: "www.example.org.",
            "links": {
                 "self": "https://dns.provider.com/v2/zones/986dba58-0043-4cc6-a1bb-69d5e86f3ca9/recordsets/dedf6879-fd9a-41d6-a7c2-eeac316fa7b3"
         }
     }
 ] }

Database Changes

N/A

Proposed Test Scenarios