Jump to: navigation, search

Graffiti/Architecture

< Graffiti
Revision as of 00:18, 3 May 2014 by Travis Tripp (talk | contribs) (Concept Screencasts)

Graffiti Architecture

Graffiti provides cross service "metadata tagging" and search aggregation for cloud resources. In Graffiti, metadata definitions may be simple property types or may be structured into simple hierarchy of objects that we call capabilities.

Base Concepts

  • Various OpenStack services provide techniques to abstract low level resource selection to one level higher, such as flavors, volume types, or artifact types. These resource abstractions often allow "metadata" in terms of key-value pair properties to further specialize and describe instances of each resource type. However, collaborating on those properties is largely a disconnected and difficult process. This often involves searching outdated wikis and opening the source code. It becomes more difficult as a cloud's scale grows. In addition, many times the properties can apply to resources from several different services. Graffiti makes this easier by creating the following concepts:
    • Capabilities and Requirements: Graffiti embraces the idea that cloud resources may be described using capabilities and requirements, a concept influenced by some parts of OpenStack today as well as by industry specifications like OASIS TOSCA (Please note, Graffiti is NOT an orchestration engine, it only assists in describing and locating existing resources in the cloud.).
    • Dictionary: A common API for services, admins, and users to discover and share their metadata vocabulary. This is the basis for creating an agreement on how to describe the various capabilities the cloud provides. It supports both unstructured properties and structured / hierarchical metadata in the form of "capability types". All definitions in the Dictionary are "namespaced".
    • Directory: A common API to "tag" and search across existing and new services for cloud content based on the metadata vocabulary, ad-hoc properties, names, and descriptions.
    • Resource Capability Registry: A persistent shared repository for services to publish information about cloud resources. This can optionally be used by services instead of or in addition to having their own local native storage to describe resources.


In Summary: Graffiti provides cross service and cross environment:

  • metadata definition aggregation and administration
  • resource "tagging" aggregation
  • resource metadata search aggregation

Workflow / Component Concepts

  1. Load your metadata definitions (called property types or capability types)
    1. Into the Graffiti central dictionary
    2. Or configure Graffiti plugins to include existing definitions provided by the various services
  2. "Tag" the resources in the cloud with your properties and capabilities
  3. Let users find the resources with your desired properties and capabilities

Graffiti-Architecture-ConceptOverlay.png

Workflow Concept Screenshot

Graffit-Tag-Then-Use-Concept.png

Graffiti Service Benefits

When the Graffiti service in integrated into the ecosystem the following additional benefits will be available:

  • Command line and REST API for cross service searching
  • Ability to import / export definitions across deployments
  • Common persistence DB for definitions in multi-node / HA deployments
  • Private tag / metadata libraries. Users / projects will be able to have their own vocabulary for "tagging" resources
  • Resource search performance optimizations. We would like to introduce a high performance indexing mechanism that crosses service boundaries.
  • Authoring - We will provide an authoring and administration UI for creating and managing namespaces, capability types, etc

Proposed Horizon Widgets

We believe that the concepts can be fulfilled in Horizon with reusable widgets that we will plug into Horizon. The widgets will provide the ability to "tag" capabilities and requirements on various resources. They will also be able to generate filter resources based on filtering on resource capabilities and properties.

Related blueprints:

Concept Screencasts

To explore and explain the ideas of the project, HP and Intel have created a couple of screencasts showing the concepts running under POC code. The styling is only representative.

Style Mockups
Widget Mockups
Concept Flow Mockup

The basic proposed flow is that on any screen that wants to be able to "tag" capabilities onto the resource type, they will be able to add the widget. The only customization required will be for the code using the widget to be able to provide the resource type that is being tagged. The resource type is sent to the API which then returns back the capabilities applicable for that type of resource.

Widget Screenshots from screencasts

Proposed Horizon Component Architecture

We would like there to be a common way in Horizon to support "tagging" key-value pairs that also will support the overall Graffiti concepts. In the proposed architecture, we will support Horizon gaining the value of Graffiti concepts through a thin API plugin layer directly in Horizon without the full Graffiti service. This will provide benefits to Horizon now, without requiring Graffiti to either be incubated or be adopted into other projects (which we are actively seeking input and advice). The widgets will be built to work with the common resource and syntax that Graffiti will also provide.

The entire concept can be run in a lightweight way through a thin filesystem provider on the Horizon server that allows reading dictionary definition files directly from the filesystem. This would suffice for single node deployments or deployments that are managed through configuration management provider to ensure consistency of the definitions across Horizon nodes.

When the full Graffiti API service is available, the widgets don't change. They still go to the Graffiti Horizon API, which would swap the plugin to talk to the Graffiti service.

Graffiti-Widgets.png