Zaqar/specs/api/v2.0

Marconi API v2.0
Brainstorming

== ==
 * Add a Swift storage backend
 * Remove SQLAlchemy for realz
 * Remove get-message-by-id endopoint
 * Rename Zaqar to Zarconi (jk)


 * Consider removing include_claimed and always listing all messages
 * Doesn't seem to be terribly useful, since when observing a task distribution queue, you are likely auditing or diagnosing, and will want to see all messages?
 * Remove getting one or more messages by ID
 * Isn't terribly useful to users, and may (TBD) provide for performance optimizations in the drivers
 * Add support for STOMP and/or MQTT?
 * Or is this scope creep? Would it be better to leave it to other messaging systems to deliver on the use cases that these protocols enable?
 * Migrate to topic-based semantics.
 * Can post a single message to multiple topics in one request
 * Can consumers read more than one topic in a single request?
 * Delete a claim to delete all it's messages?
 * Need to decide if we want to encourage deleting multiple messages at a time, or if that is an anti-pattern
 * Only provide a message ID for claimed messages?
 * Con: user may want to have an auditor that uses the feeds api, and the auditor will need to match up message IDs with the ones being claimed by the workers
 * Support msgpack in addition to JSON?
 * Main difficulty is in replying in JSON when a message was enqueued using msgpack, and includes one or more binary fields.
 * Check out Protobuf and Apache Arvo
 * Allow custom media types?
 * Message attributes move into X-headers
 * Allows for all kinds of binary types
 * Con: if an error occurs, there isn't an obvious media type the client accepts to use in reporting structured error information
 * Con: X-headers are unwieldy if we need structured message attributes (YAGNI?)
 * Things that may allow the use for a wider variety of backends [/flameon]
 * For example, make paging through messages without claiming them optional
 * But if people just want Kafka or Qpid or whatever, why not just use those technologies directly? Once you put a REST API on top, it starts to look a lot like v1.x. For extremely high-throughput use cases, apps won't ever get what they want over python + REST. But, on the other hand, there are important use cases where REST is preferable, and if we continue to hold at ~20ms response times, that is still pretty darn good performance for going over the web.