Difference between revisions of "Zaqar/specs/api/v2.0"
< Zaqar
(→Marconi API v2.0: Added note about custom media types) |
(→Marconi API v2.0 - Ideas for review) |
||
(6 intermediate revisions by 2 users not shown) | |||
Line 2: | Line 2: | ||
Brainstorming | Brainstorming | ||
+ | |||
+ | == | ||
+ | * Add a Swift storage backend | ||
+ | * Remove SQLAlchemy for realz | ||
+ | * Remove get-message-by-id endopoint | ||
+ | * Rename Zaqar to Zarconi (jk) | ||
+ | == | ||
+ | |||
+ | * Consider removing include_claimed and always listing all messages | ||
+ | ** Doesn't seem to be terribly useful, since when observing a task distribution queue, you are likely auditing or diagnosing, and will want to see all messages? | ||
+ | * Remove getting one or more messages by ID | ||
+ | ** Isn't terribly useful to users, and may (TBD) provide for performance optimizations in the drivers | ||
+ | * Add support for STOMP and/or MQTT? | ||
+ | ** Or is this scope creep? Would it be better to leave it to other messaging systems to deliver on the use cases that these protocols enable? | ||
* Migrate to topic-based semantics. | * Migrate to topic-based semantics. | ||
** Can post a single message to multiple topics in one request | ** Can post a single message to multiple topics in one request | ||
Line 9: | Line 23: | ||
* Only provide a message ID for claimed messages? | * Only provide a message ID for claimed messages? | ||
** Con: user may want to have an auditor that uses the feeds api, and the auditor will need to match up message IDs with the ones being claimed by the workers | ** Con: user may want to have an auditor that uses the feeds api, and the auditor will need to match up message IDs with the ones being claimed by the workers | ||
+ | * Support msgpack in addition to JSON? | ||
+ | ** Main difficulty is in replying in JSON when a message was enqueued using msgpack, and includes one or more binary fields. | ||
+ | ** Check out Protobuf and Apache Arvo | ||
* Allow custom media types? | * Allow custom media types? | ||
** Message attributes move into X-headers | ** Message attributes move into X-headers | ||
Line 14: | Line 31: | ||
** Con: if an error occurs, there isn't an obvious media type the client accepts to use in reporting structured error information | ** Con: if an error occurs, there isn't an obvious media type the client accepts to use in reporting structured error information | ||
** Con: X-headers are unwieldy if we need structured message attributes (YAGNI?) | ** Con: X-headers are unwieldy if we need structured message attributes (YAGNI?) | ||
+ | * Things that may allow the use for a wider variety of backends [/flameon] | ||
+ | ** For example, make paging through messages without claiming them optional | ||
+ | ** But if people just want Kafka or Qpid or whatever, why not just use those technologies directly? Once you put a REST API on top, it starts to look a lot like v1.x. For extremely high-throughput use cases, apps won't ever get what they want over python + REST. But, on the other hand, there are important use cases where REST is preferable, and if we continue to hold at ~20ms response times, that is still pretty darn good performance for going over the web. |
Latest revision as of 15:12, 7 November 2014
Marconi API v2.0
Brainstorming
==
- Add a Swift storage backend
- Remove SQLAlchemy for realz
- Remove get-message-by-id endopoint
- Rename Zaqar to Zarconi (jk)
==
- Consider removing include_claimed and always listing all messages
- Doesn't seem to be terribly useful, since when observing a task distribution queue, you are likely auditing or diagnosing, and will want to see all messages?
- Remove getting one or more messages by ID
- Isn't terribly useful to users, and may (TBD) provide for performance optimizations in the drivers
- Add support for STOMP and/or MQTT?
- Or is this scope creep? Would it be better to leave it to other messaging systems to deliver on the use cases that these protocols enable?
- Migrate to topic-based semantics.
- Can post a single message to multiple topics in one request
- Can consumers read more than one topic in a single request?
- Delete a claim to delete all it's messages?
- Need to decide if we want to encourage deleting multiple messages at a time, or if that is an anti-pattern
- Only provide a message ID for claimed messages?
- Con: user may want to have an auditor that uses the feeds api, and the auditor will need to match up message IDs with the ones being claimed by the workers
- Support msgpack in addition to JSON?
- Main difficulty is in replying in JSON when a message was enqueued using msgpack, and includes one or more binary fields.
- Check out Protobuf and Apache Arvo
- Allow custom media types?
- Message attributes move into X-headers
- Allows for all kinds of binary types
- Con: if an error occurs, there isn't an obvious media type the client accepts to use in reporting structured error information
- Con: X-headers are unwieldy if we need structured message attributes (YAGNI?)
- Things that may allow the use for a wider variety of backends [/flameon]
- For example, make paging through messages without claiming them optional
- But if people just want Kafka or Qpid or whatever, why not just use those technologies directly? Once you put a REST API on top, it starts to look a lot like v1.x. For extremely high-throughput use cases, apps won't ever get what they want over python + REST. But, on the other hand, there are important use cases where REST is preferable, and if we continue to hold at ~20ms response times, that is still pretty darn good performance for going over the web.