Jump to: navigation, search

Difference between revisions of "Python Client Library (Marconi)"

m (Flaper87 moved page Python Marconi Client to Marconi/PythonClient)
(Rename to zaqar)
 
(9 intermediate revisions by 3 users not shown)
Line 1: Line 1:
This document exists to establish the feel of working with the python Marconi client. It serves as both a vision and a direction for implementors. Check out the blueprints for more details.
+
This document exists to establish the feel of working with the python Zaqar client. It serves as both a vision and a direction for implementors. Check out the blueprints for more details.
  
If you have questions, reach us at freenode.irc.net #openstack-marconi!
+
If you have questions, reach us at freenode.irc.net #openstack-zaqar!
  
 
== Features ==
 
== Features ==
 +
 +
The client library is available on PyPI: https://pypi.python.org/pypi/python-marconiclient/
  
 
* Certificate verification
 
* Certificate verification
 
* Reauthentication on token expiration
 
* Reauthentication on token expiration
 
* Async operations
 
* Async operations
* Full coverage of Marconi API
+
* Full coverage of Zaqar API
 +
 
 +
== Etherpads ==
 +
 
 +
* [https://etherpad.openstack.org/zaqar-client Client Brainstorm]
 +
* [https://etherpad.openstack.org/zaqar-client-api API Brainstorm]
  
 
== Classes ==
 
== Classes ==
Line 21: Line 28:
 
=== Core ===
 
=== Core ===
  
* Connection: handles authentication with Keystone, acquires Marconi endpoint, handles requests and networking logic
+
* Connection: handles authentication with Keystone, acquires Zaqar endpoint, handles requests and networking logic
  
 
=== Utility ===
 
=== Utility ===
  
* ErrorBase: the foundation for Marconi client specific errors
+
* ErrorBase: the foundation for Zaqar client specific errors
  
 
== API Synopsis ==
 
== API Synopsis ==
  
See [[Marconi/specs/api/v1]]
+
See [[Zaqar/specs/api/v1]]
 +
 
 +
== Client API Synopsis ==
 +
 
 +
<pre><nowiki>
 +
# Given the following vars defined with the following types:
 +
# - Client:client
 +
# - Subscriber:sub
 +
# - Queue:queue
 +
# - Message:message
 +
# - Claim:claim
 +
>>> dir(client)
 +
['async', 'create_queue', 'home', 'queues']
 +
>>> dir(sub)
 +
['channels', 'listen', 'subscribe', 'unsubscribe']
 +
>>> dir(queue)
 +
['claim', 'delete', 'href', 'messages', 'metadata', 'name', 'post', 'stats']
 +
>>> dir(metadata)
 +
['content', 'reload', 'update']
 +
>>> dir(claim)
 +
['delete', 'grace', 'href', 'messages', 'patch', 'release', 'ttl']
 +
>>> dir(message)
 +
['age', 'body', 'delete', 'href', 'reload', 'status', 'ttl']
 +
</nowiki></pre>
  
 
== Usage ==
 
== Usage ==
Line 40: Line 70:
  
 
<pre><nowiki>
 
<pre><nowiki>
     >>> # given a class marconiclient.connection implemented using marconiclient.common.apiclient.client
+
     >>> # given a class zaqarclient.connection implemented using zaqarclient.common.apiclient.client
     >>> from marconiclient import connection
+
     >>> from zaqarclient import connection
 
     >>> conn = connection.Connection(auth_plugin=...,
 
     >>> conn = connection.Connection(auth_plugin=...,
 
                                     username='tacocat', password='queue_master')
 
                                     username='tacocat', password='queue_master')
Line 48: Line 78:
 
</nowiki></pre>
 
</nowiki></pre>
  
The hope is to be able to update the Http client to use flufl.enum, detailed below. It's good enough for now, though.
+
The hope is to be able to update the Http client to use enum34, detailed below. It's good enough for now, though.
  
See [https://pypi.python.org/pypi/flufl.enum flufl.enum] and [http://www.python.org/dev/peps/pep-0435/ PEP 435] for more information on Python enumerations.
+
See [https://pypi.python.org/pypi/enum34 enum34] and [http://www.python.org/dev/peps/pep-0435/ PEP 435] for more information on Python enumerations.
  
 
=== Client Operation ===
 
=== Client Operation ===
  
 
<pre><nowiki>
 
<pre><nowiki>
    # Implement auth using common lib. Client calls to that.
+
     >>> client = Connection(async=False)
     >>> client = Client(conn, async=False)
+
     >>> client.queues(marker=..., limit=10, detailed=False)
     >>> client.list(marker=..., limit=10, detailed=False) # lists queues
 
 
     <generator object <genexpr> ar 0x7fd3ef1ed730>
 
     <generator object <genexpr> ar 0x7fd3ef1ed730>
     >>> client.put(name='wot') # creates a queue, returns it
+
     >>> client.create_queue(name='wot')
     <MarconiQueue [wot]>
+
     <ZaqarQueue [wot]>
 
     >>> client.home
 
     >>> client.home
 
     <HomeDoc ...>
 
     <HomeDoc ...>
Line 99: Line 128:
 
     >>> queue.stats
 
     >>> queue.stats
 
     # a dictionary derived from a JSON response
 
     # a dictionary derived from a JSON response
     >>> queue.list(include_claimed=False)
+
     >>> queue.messages(include_claimed=False)
 
     <generator object <genexpr> ar 0x7fd3ef1ed742>
 
     <generator object <genexpr> ar 0x7fd3ef1ed742>
     >>> queue.list(ids=[50b68a50d6f5b8c8a7c62b01, 50b68a50d6f5b8c8a7c62b02],
+
     >>> queue.messages(ids=[50b68a50d6f5b8c8a7c62b01, 50b68a50d6f5b8c8a7c62b02],
                          claim=a28ee94e-6cb4-11e2-b4d5-7703267a7926, limit=1)
+
                              claim=a28ee94e-6cb4-11e2-b4d5-7703267a7926, limit=1)
 
     <generator object <genexpr> ar 0x7fd3ef1ed742>
 
     <generator object <genexpr> ar 0x7fd3ef1ed742>
     >>> queue.post(messages=...)
+
     >>> queue.post_messages(messages=...)
 
     >>> queue.metadata
 
     >>> queue.metadata
 
     <Metadata ...>  # fetches metadata from API, returns a Metadata controller
 
     <Metadata ...>  # fetches metadata from API, returns a Metadata controller
Line 116: Line 145:
 
<pre><nowiki>
 
<pre><nowiki>
 
     >>> meta = queue.metadata
 
     >>> meta = queue.metadata
     >>> meta.put({'max_size': 1000})  # communicate with API, replaces
+
     >>> meta.update({'max_size': 1000})  # communicate with API, replaces
 
     >>> meta.reload()  # gets most recent attributes from API
 
     >>> meta.reload()  # gets most recent attributes from API
 
</nowiki></pre>
 
</nowiki></pre>
Line 142: Line 171:
 
<pre><nowiki>
 
<pre><nowiki>
 
>>> claim = queue.claim(limit=10)
 
>>> claim = queue.claim(limit=10)
>>> claim.list(...)
+
>>> claim.messages(...)
 
<generator object <genexpr> ar 0x7fd3ef1ed742>
 
<generator object <genexpr> ar 0x7fd3ef1ed742>
>>> msg = next(claim.list(...))
+
>>> msg = next(claim.messages(...))
 
>>> msg.status
 
>>> msg.status
 
<EnumValue: Message.Claimed [value=2]>
 
<EnumValue: Message.Claimed [value=2]>
Line 159: Line 188:
 
== Error Management ==
 
== Error Management ==
  
The wiki gives a thorough explanation of [https://wiki.openstack.org/wiki/Marconi/specs/api/v1#Errors Marconi Errors]. Error handling at the client-level is a matter of transforming responses returned by Marconi into exceptions that are meaningful to users.
+
The wiki gives a thorough explanation of [https://wiki.openstack.org/wiki/Marconi/specs/api/v1#Errors Zaqar Errors]. Error handling at the client-level is a matter of transforming responses returned by Zaqar into exceptions that are meaningful to users.
  
 
Here's a quick mock up of error usage at the level of the client:
 
Here's a quick mock up of error usage at the level of the client:
  
 
<pre><nowiki>
 
<pre><nowiki>
     >> error = marconi.error.ErrorBase()
+
     >> error = zaqar.error.ErrorBase()
 
     >>> error.title
 
     >>> error.title
 
     u'...'
 
     u'...'
Line 178: Line 207:
 
== Workflow ==
 
== Workflow ==
  
This section is more top-down than the rest. Consider this what it feels like to spin up python-marconiclient in an [http://ipython.org/ ipython] environment:
+
This section is more top-down than the rest. Consider this what it feels like to spin up python-zaqarclient in an [http://ipython.org/ ipython] environment:
  
 
<pre><nowiki>
 
<pre><nowiki>
     >>> from marconiclient.connection import Connection
+
     >>> from zaqarclient.connection import Connection
     >>> from marconiclient.client import Client
+
     >>> from zaqarclient.client import Client
     >>> conn = Connection(auth_url='https://keystone.example.com/', username='me', password='win')
+
     >>> client = Connection(auth_url='https://keystone.example.com/', username='me', password='win')
    >>> client = Client(conn)
+
     >>> client.create_queue('wot')
     >>> client.put('wot')
+
     >>> queue = next(client.queues())
     >>> queue = next(client.list())
+
     >>> queue.post_messages(messages=[{'event': {'data': 'winning', 'score': 10}})
     >>> queue.post(messages=[{'event': {'data': 'winning', 'score': 10}})
+
     >>> message = next(queue.messages())
     >>> message = next(queue.list())
 
 
     >>> message.body
 
     >>> message.body
 
     {'event': {'data': 'winning', 'score': 10}}
 
     {'event': {'data': 'winning', 'score': 10}}
Line 196: Line 224:
 
     {...}
 
     {...}
 
     >>> claim = queue.claim(1)
 
     >>> claim = queue.claim(1)
     >>> message = next(claim.list())
+
     >>> message = next(claim.messages())
 
     >>> message.status
 
     >>> message.status
 
     <Claimed...>
 
     <Claimed...>

Latest revision as of 23:24, 4 August 2014

This document exists to establish the feel of working with the python Zaqar client. It serves as both a vision and a direction for implementors. Check out the blueprints for more details.

If you have questions, reach us at freenode.irc.net #openstack-zaqar!

Features

The client library is available on PyPI: https://pypi.python.org/pypi/python-marconiclient/

  • Certificate verification
  • Reauthentication on token expiration
  • Async operations
  • Full coverage of Zaqar API

Etherpads

Classes

Controllers

  • Client: handles account-wide operations - queue retrieval, etc.
  • Queue: gives access to some metadata ops, as well as message handling
  • Message: gives access to properties of message, as well as deletion
  • Claim: Collection of claimed messages - handle querying, updating, and deleting

Core

  • Connection: handles authentication with Keystone, acquires Zaqar endpoint, handles requests and networking logic

Utility

  • ErrorBase: the foundation for Zaqar client specific errors

API Synopsis

See Zaqar/specs/api/v1

Client API Synopsis

# Given the following vars defined with the following types:
# - Client:client
# - Subscriber:sub
# - Queue:queue
# - Message:message
# - Claim:claim
>>> dir(client)
['async', 'create_queue', 'home', 'queues']
>>> dir(sub)
['channels', 'listen', 'subscribe', 'unsubscribe']
>>> dir(queue)
['claim', 'delete', 'href', 'messages', 'metadata', 'name', 'post', 'stats']
>>> dir(metadata)
['content', 'reload', 'update']
>>> dir(claim)
['delete', 'grace', 'href', 'messages', 'patch', 'release', 'ttl']
>>> dir(message)
['age', 'body', 'delete', 'href', 'reload', 'status', 'ttl']

Usage

Working With a Connection

This section has been mostly simplified by the inclusion of the common lib BaseClient. What Openstack calls a "Client" we'll call a Connection. The reason for this is that the connection handles networking-level details: puts, patches, deletes, posts, etc. What we call a client is interested in higher level details, like queues and claims.

The interface is as follows, based on code reading:

    >>> # given a class zaqarclient.connection implemented using zaqarclient.common.apiclient.client
    >>> from zaqarclient import connection
    >>> conn = connection.Connection(auth_plugin=...,
                                     username='tacocat', password='queue_master')
    >>> dir(conn)
    ['client_request', 'head', 'get', 'post', 'put', 'delete', 'patch', 'get_class']

The hope is to be able to update the Http client to use enum34, detailed below. It's good enough for now, though.

See enum34 and PEP 435 for more information on Python enumerations.

Client Operation

    >>> client = Connection(async=False)
    >>> client.queues(marker=..., limit=10, detailed=False)
    <generator object <genexpr> ar 0x7fd3ef1ed730>
    >>> client.create_queue(name='wot')
    <ZaqarQueue [wot]>
    >>> client.home
    <HomeDoc ...>
    # affects all operations for objects acquired from the client
    >>> client.async
    False

Setting Up a Pub/Sub Connection

This borrows some ideas from redis-py:

    >>> sub = client.subscriber()
    >>> sub.<TAB>
    sub.channels  sub.listen    sub.subscribe    sub.unsubscribe
    >>> sub.channels
    set([])
    >>> sub.subscribe('darn_good_queue')
    >>> sub.channels
    set(['darn_good_queue'])
    >>> for msg in sub.listen():
            # blocks until a message arrives in any of the subscribed queues
            # polling implementation by default
    <Ctrl-C>
    >>> 

Queue Handling

    >>> queue = next(q for q in client.queues if q.name == 'tacocat')
    >>> queue.name
    u'tacocat'
    >>> queue.href
    u'/v1/queues/tacocat'
    >>> queue.stats
    # a dictionary derived from a JSON response
    >>> queue.messages(include_claimed=False)
    <generator object <genexpr> ar 0x7fd3ef1ed742>
    >>> queue.messages(ids=[50b68a50d6f5b8c8a7c62b01, 50b68a50d6f5b8c8a7c62b02],
                               claim=a28ee94e-6cb4-11e2-b4d5-7703267a7926, limit=1)
    <generator object <genexpr> ar 0x7fd3ef1ed742>
    >>> queue.post_messages(messages=...)
    >>> queue.metadata
    <Metadata ...>  # fetches metadata from API, returns a Metadata controller
    >>> queue.claim(limit=10)
    <Claim size:8 ...>  # size: actual number of messages received
    >>> queue.delete()

Queue Metadata Handling

    >>> meta = queue.metadata
    >>> meta.update({'max_size': 1000})  # communicate with API, replaces
    >>> meta.reload()  # gets most recent attributes from API

Message Handling

    >>> message = next(queue.messages(...))
    >>> message.age
    90
    >>> message.ttl
    120
    >>> message.href
    u'/v1/queues/darn_good_queue/messages/91wqe9bqwsbq98'
    >>> message.body
    {u'action': u'win'}
    >>> message.reload()
    >>> message.delete()
    >>> message.status
    <EnumValue: Message.Free [value=1]>

Claim Management

>>> claim = queue.claim(limit=10)
>>> claim.messages(...)
<generator object <genexpr> ar 0x7fd3ef1ed742>
>>> msg = next(claim.messages(...))
>>> msg.status
<EnumValue: Message.Claimed [value=2]>
>>> claim.href
u'/v1/queues/tacocat/claims/a28ee94e-6cb4-11e2-b4d5-7703267a7926'
>>> claim.ttl
90
>>> claim.grace
30
>>> claim.patch(ttl=..., grace=...)
>>> claim.delete()

Error Management

The wiki gives a thorough explanation of Zaqar Errors. Error handling at the client-level is a matter of transforming responses returned by Zaqar into exceptions that are meaningful to users.

Here's a quick mock up of error usage at the level of the client:

    >> error = zaqar.error.ErrorBase()
    >>> error.title
    u'...'
    >>> error.description
    u'...'
    >>> error.code
    1092
    >>> raise error
    ErrorBase (error.code): error.title
    error.description

Workflow

This section is more top-down than the rest. Consider this what it feels like to spin up python-zaqarclient in an ipython environment:

    >>> from zaqarclient.connection import Connection
    >>> from zaqarclient.client import Client
    >>> client = Connection(auth_url='https://keystone.example.com/', username='me', password='win')
    >>> client.create_queue('wot')
    >>> queue = next(client.queues())
    >>> queue.post_messages(messages=[{'event': {'data': 'winning', 'score': 10}})
    >>> message = next(queue.messages())
    >>> message.body
    {'event': {'data': 'winning', 'score': 10}}
    >>> message.status
    <Free...>
    >>> queue.stats
    {...}
    >>> claim = queue.claim(1)
    >>> message = next(claim.messages())
    >>> message.status
    <Claimed...>
    >>> message
    <Message ttl:120>
    >>> message.delete()
    >>> claim
    <Claim size:1>
    >>> claim.delete()
    >>> queue
    <Queue [wot]>
    >>> queue.delete()
    >>> client.async
    False