Jump to: navigation, search

Zaqar/bp/actions-audit

< Zaqar
Revision as of 15:10, 14 June 2013 by Kgriffs (talk | contribs) (Get Actions)

Notes

  • Either post things to an audit queue, and/or archive and upload to a swift container.
  • Should NOT be on by default, configure using queue metadata.
  • Need to decide whether all operations are audited, or just "invisible" ones

Possible design for API if we create an audit queue:

Get Actions

NOTE: Implementation has been postponed until after our 1.0 release.

Template

GET {base_url}/queues/{queue_name}/actions/{?marker,limit}


Request

GET /v1/480924/queues/fizbit/actions&marker=1355-237242-783&limit=10 HTTP/1.1
Host: marconi.example.com

...


Response

HTTP 200 if the query matched 1 or more actions, HTTP 204 otherwise (with no body).

HTTP/1.1 200 OK
Content-Location: /v1/480924/queues/fizbit/actions?marker=1355-237242-783&limit=10

...

{
  "links": [
    {
      "rel": "next",
      "href": "/v1/480924/queues/fizbit/actions?marker=6244-244224-783&limit=10"
    }
  ],
  "messages": [
    {
      "href": "/v1/480924/queues/fizbit/actions/10b00a50d6f5b8c8a7c62ccb",
      "ttl": 300,
      "age": 120,
      "body": {
        "event": "ClaimExpired",
        "details": {
          "message_id": "50b68a50d6f5b8c8a7c62a87",
          "user_agent": "python/2.7 killer-rabbit/1.2",
          "client_id": "79ed56f8-2519-11e2-b835-acf6018e45a1"
        }
      }
    },
    {
      "href": "/v1/480924/queues/fizbit/actions/10b00a50d6f5b8c8a7c62ccc",
      "ttl": 300,
      "age": 121,
      "body": {
        "event": "MessageDeleted",
        "details": {
          "message_id": "50b68a50d6f5b8c8a7c62b01",
          "user_agent": "python/2.7 killer-rabbit/1.2",
          "client_id": "79ed56f8-2519-11e2-b835-acf6018e45a1"
        }
      }
    }
  ]
}


Discussion

The interactions of various agents/workers with a cloud queuing services can be difficult to audit and debug. Marconi emits action messages to a special queue (hence action messages use the same format as regular messages), from which auditors can retrieve a list of recent actions involving a specific queue's messages. These actions can then be archived by the auditor for future analysis and diagnostics.

Action messages expire after 5 minutes, so auditors should regularly poll for actions and persist them for posterity.

The following actions are recorded, since they cannot be otherwise observed:

  • Delete message
  • Claim messages
  • Update claim
  • Release claim
  • Claim expired


Message IDs and markers are opaque strings; clients should make no assumptions on their format or length. Furthermore, clients should assume there is no relationship between markers and message IDs (i.e., one cannot be derived from the other). This ensures maximum flexibility when implementing storage drivers. Additionally, clients SHOULD use the given hrefs rather than manually constructing URIs.

Results are always ordered by age, oldest message first.

limit specifies up to 50 messages to return. If not specified, limit defaults to 10. When more messages are available than can be returned in a single request, the client can pick up the next batch by simply using the URI template parameters returned from the previous call in the "next" field.

marker is an opaque string that the client can use to request the next batch of messages. It communicates to the server which messages the client has already received.

Note: If marker is not specified, the API will return all messages at the head of the queue (up to limit).