Zaqar/specs/api/next

Marconi API - Next
Brainstorming page for API improvements. The ideas below need to be discussed with the Marconi team and the OS community. They will be turned into blueprints as appropriate.

Generic
summary | log


 * Auto-generate client UUID if not given
 * Makes things more confusing, since
 * Clearly define whether client ID is required for every request, and enforce it in the implementation
 * Always require it, and log the id in every log line for tracing, analytics, support, etc.
 * Plan for v1.1
 * Consider allowing opaque string for client ID rather than UUID (will need to understand what else people want to use?)
 * it isn't *that* hard to generate one
 * saying it has to be uuid makes validation easier, and reduces confusion
 * Remove deprecated "partial results" semantics from message posting
 * This is no longer a possiblity with the mongo driver, and other backends probably don't need it either
 * Simplifies client implementations
 * Plan for v1.1
 * Include claim information when listing messages and retrieving individual messages
 * Include Client ID in claim data
 * Helps when auditing a queue, diagnostics?
 * Extra storage is trivial when storing UUID as binary (16 bytes)
 * Plan tentatively for v1.1 - need to think more about use cases
 * List claims for a given queue
 * Worker should be able to trust claim was created as expected
 * Other workers should not be looking at claims they did not create
 * Auditor client can list messages and see claim IDs on them if it wants to verify
 * Automatically create queues the first time a message is posted to it
 * Revisit for v2.0
 * Metadata?
 * Typos create queues? Could mitigate with an operation flag (lazy=True by default)
 * Is metadata useful for a queue?

HTTP
summary | log


 * Return an href for deleting all claimed messages with a single call
 * Probably an anti-pattern, since processing multiple messages before deleting only increases the chance for orphaned messages when a worker crashes.
 * If this is for perf reasons, let's solve that with alternate transports instead (persistent connections with small operation payloads, so N individual delete operations is really fast)
 * Allow PUTing metadata when creating a queue
 * We made it a separate resource since later we want the queue resource itself to just have options, such as default TTL (metadata != queue properties)
 * Set queue options for defaults, such as message and claim TTL.
 * Clients can just implement defaults instead?
 * Would require caching to make fast, but still takes a little time to check.
 * homedoc should return relative URIs or encode version in href-template
 * Fix this in v1.0 - well-behaved "follow-your-nose" clients are broken at the moment
 * https://bugs.launchpad.net/marconi/+bug/1245656
 * 204 vs. 200 + empty array (e.g., when listing messages and queues, and claiming messages)
 * Current behavior: https://wiki.openstack.org/wiki/Marconi/specs/api/v1#List_Messages
 * while 204 isn't necessarily incorrect, and saves a few bytes on the wire, it is inconsistent with other OpenStack APIs
 * may make client implementations a tad bit easier
 * Plan for v1.1
 * Consistency around response envelope for /messages vs /messages?ids
 * Make it a different resource so the distinction between the operations is more clear
 * Plan for v2
 * Response envelopes in general - don't return raw arrays (to allow for extensions)
 * [] ===> {"things":[]}
 * Plan for v1.1
 * Q. Should this also apply to request bodies, i.e., posting messages?
 * Return full URIs in location/content-location headers (rather than relative ones)
 * Repose doesn't (yet) support partial URIs in headers - it tries to parse them rather than just passing them through
 * Clients don't have any trouble, and other rproxies don't seem to care.
 * Is content-location header really necessary?
 * it is really only helpful if you have alternate representations of a resource that can be accessed using a different path vs. specifying the media type in Accept
 * Don't set this header any more - save a few bytes on the wire, and it isn't useful
 * Plan for v1.1
 * Polling model is based on the assumption of a "streaming" interaction model. Is there a place for a "point in time" listing model, where you don't always get a "next" href?
 * Revisit when planning v2
 * Query parameters are usually considered to be "optional", and yet DELETE queues/my-queue/messages requires an "ids" parameter. Some options:
 * Leave as is. See if anyone complains.
 * Make "ids" optional. In this case, DELETE my-queue/messages would be defined to delete all messages in the queue. See also: https://blueprints.launchpad.net/marconi/+spec/flush-queue
 * Allow lists of id’s in the path, such as my-queue/messages/1,2,3. We experimented with this is an early API draft, but a lot of people thought it looked strange/unusual, so we didn’t do it.
 * Create a separate resource - may be same as one used for GET messages?ids=1,2,3
 * Don’t support bulk DELETE operations on messages.
 * Revisit when planning v2
 * Should resources list in response from posting a message use relative path instead of absolute?
 * Yes, plan for v1.1
 * Consider returning 404 in some cases rather than 403 if it reduces leaking information that could be used in an attack