Jump to: navigation, search

Difference between revisions of "Graffiti"

(→‎Screencasts)
(→‎Current Status)
 
(35 intermediate revisions by the same user not shown)
Line 1: Line 1:
  
−
=== What's in my cloud? ===
+
== What's in my cloud? ==
  
 
I've got a lot of resources in my cloud.
 
I've got a lot of resources in my cloud.
Line 7: Line 7:
 
* How do I describe what I have?
 
* How do I describe what I have?
  
 +
At its most basic concept, Graffiti's intent is to enable better metadata collaboration across services and projects so that OpenStack users can take advantage of an Enhanced Platform Awareness.
  
−
Graffiti has the initial intent of providing cross service metadata “tagging" and search aggregation for cloud resources.
+
== Current Status ==
  
−
== Introduction and Project Goals ==
+
The Graffiti project was proposed at the Atlanta (Juno) OpenStack summit. Since then the concepts have been adopted and implemented as part of multiple different OpenStack Projects.
  
−
The higher level goal of Graffiti is for OpenStack users to be able to declare the capabilities and service level objectives they require and then be guided in selecting the actual cloud resources that match their desired capabilities.
+
* Glance Metadata Definition Catalog
 +
** http://docs.openstack.org/developer/glance/metadefs-concepts.html
 +
** https://github.com/openstack/glance/tree/master/etc/metadefs
 +
** https://youtu.be/zJpHXdBOoeM
 +
* Searchlight
 +
** http://launchpad.net/searchlight
 +
** https://wiki.openstack.org/wiki/Searchlight
 +
* Nova features
 +
** Scheduling filters like Numa topology
 +
* Horizon features
 +
** An admin UI for managing the catalog
 +
*** (Admin —> Metadata Definitions) (Kilo)
 +
** A widget for associating metadata to different resources
 +
*** (Update Metadata action on each row item below)
 +
*** admin -> images (Juno)
 +
*** admin -> flavors (Kilo)
 +
*** admin —> Host Aggregates (Kilo)
 +
*** project —> images (Liberty)
 +
*** project —> instances (Mitaka)
 +
** The ability to add metadata at launch time
 +
*** project —> Launch Instance (ng launch instance enabled) (Mitaka)
  
−
Various OpenStack services provide techniques to abstract low level resource selection to one level higher, such as flavors, volume types, or artifact types. These resource abstractions often allow "metadata" in terms of key-value pair properties to further specialize and describe instances of each resource type. However, collaborating on those properties can be a disconnected and difficult process. This often involves searching outdated wikis and opening the source code. In addition, the metadata properties often need to be correlated across several different services.  It becomes more difficult as a cloud's scale grows and the number of resources being managed increases.
+
The following information provides much of the background information on where these concepts originated.
  
−
At its most basic concept Graffiti is a project to enable better metadata collaboration across services and projects.  Graffiti intends to provide a common methodology to describe resource capabilities in the cloud which we believe can then be leveraged by other projects such as Horizon, Nova, Heat, scheduling, reservation, and policy enforcement to enable better cross service collaboration and consistency.
+
== Overview ==
  
−
== How you use it ==
+
A challenge we've experienced with using OpenStack is discovering, sharing, and correlating metadata across services and different types of resources. We believe this affects both end users and administrators.
  
−
# Load your metadata definitions (called property types or capability types)
+
For end users, we feel like doing basic tasks like launching instances is too technical for end users and require too much pre-existing knowledge of OpenStack concepts. For example, you should be able to just specify categories like "Big Data" or an "OS Family" and then let the system find the boot source for you, whether that is an image, snapshot, or volume.  It should also allow finer grained filtering like filtering on specific versions of software that you want.
−
## Into the Graffiti central dictionary
 
−
## Or configure Graffiti plugins to include existing definitions provided by the various services
 
−
# "Tag" the resources in the cloud with your properties and capabilities
 
−
# Let users find the resources with your desired properties and capabilities
 
  
−
=== Terminology Note ===
+
For administrators, we’d like there to be an easier way to meaningfully collaborate on properties across host aggregates, flavors, images, volumes, or other cloud resources.
  
−
Within Graffiti, we use the term "tag" interchangeably with the traditional meaning for the word "tag". In Graffiti, we believe that a user can simply tag a capability onto a resource. Sometimes tagging the capability results in a simple keyword and sometimes it may map to multiple properties that they can specialize.  To the end user, it shouldn't matter how the data is handled.
+
Various OpenStack services provide techniques to abstract low level resource selection to one level higher, such as flavors, volume types, or artifact types. These resource abstractions often allow "metadata" in terms of key-value pair properties to further specialize and describe instances of each resource type. However, collaborating on those properties can be a disconnected and difficult process. This often involves searching wikis and opening the source code. In addition, the metadata properties often need to be correlated across several different services.  It becomes more difficult as a cloud's scale grows and the number of resources being managed increases.
  
−
=== Frequently Asked Questions ===
+
We, HP and Intel, believe that both of the above problems come back to needing a better way for users to collaborate on metadata across services and resource types.  We started a project called Graffiti to explore ideas and concepts for how to make this easier and more approachable for end users. Please join with us to help move forward together as a community!
  
−
Please see our  [[Graffiti/FAQ]] for answers to common questions.
+
We believe that we can make some immediate improvements in Horizon, but that they can't be achieved through Horizon alone and that the benefits should extend to the API and CLI interactions as well. Better cross service collaboration and consistency on metadata should provide benefits that can be leveraged by other projects such as scheduling, reservation, orchestration, and policy enforcement.
  
−
== How it works ==
+
=== Terminology Note ===
  
−
Graffiti provides cross service "metadata tagging" and search aggregation for cloud resources. In Graffiti, metadata definitions may be simple property types or may be structured into simple hierarchy of objects that we call capabilities.  Each of those capabilities may contain many properties.
+
We think the term "metadata" is a somewhat unapproachable term, so we have been exploring with the concept of a "capability".  A capability can simply be thought of as a named "tag" which may or may not have properties. The idea is that a user can simply "tag" a capability onto various cloud resources such as images, volumes, host aggregates, flavors, and so on. To the end user, the exact mechanism for how the data is stored is handled for them.
−
 
 
−
Graffiti provides cross service and cross environment:
 
−
* metadata definition aggregation and administration
 
−
* resource "tagging" aggregation
 
−
* resource metadata search aggregation (metadata properties, name, and description)
 
  
 
=== Screencasts ===
 
=== Screencasts ===
−
To explore and explain the ideas of the project, HP and Intel have created a couple of screencasts showing the concepts running under POC code.
 
  
−
* [https://www.youtube.com/watch?v=f0SZtPgcxk4| 15 Minute Concept Overview]
+
To help explain the ideas of the project, we have a quick screencast demonstrating the concepts running under POC code. Please take a look!
−
* [https://www.youtube.com/watch?v=m1cTnXwxgks| 5 Minute Brief (for those running tight on time)]
 
  
−
=== Architecture ===
+
* [https://www.youtube.com/watch?v=f0SZtPgcxk4| Concept Overview]
 +
* [https://youtu.be/zJpHXdBOoeM Availability as of the mitaka release in Horizon and Glance]
  
−
The full architecture may be found on the [[Graffiti/Architecture|Architecture]] page.
+
=== Usage Concepts ===
  
−
== Status ==
+
# Load your metadata definitions (sometimes called properties, tags, or capabilities)
 +
## Into the central metadata catalog
 +
# Update the resources in the cloud with your tags and capabilities
 +
# Let users find the resources with your desired tags and capabilities
  
−
We are working on a proof of concept with Horizon that crosses Nova, Glance, and Cinder. We are using the POC to help us better understand technical issues and will use that knowledge to help contribute to existing projects and to build out the Graffiti concepts appropriately.
+
== Design Concepts ==
  
−
The Graffiti project currently has the following aspects:
+
Additional architecture concepts on the [[Graffiti/Architecture|Architecture]] page.
−
* [[Graffiti/Architecture#Proposed_Horizon_Widgets|Immediate improvements in Horizon]] with no required changes to existing services, which will have some inherent limitations.
 
−
* [[Graffiti/Architecture#Graffiti_Service_Benefits|A central collaboration service]] without the limitations of a Horizon only solution
 
−
** With the intent of also enhancing existing services as needed through a variety of blueprints
 
  
 +
=== Juno Summit Design Sessioɲ ===
  
−
We will be at the summit with a demo of the POC (in whatever state it is) and have a scheduled design session where we'd love to discuss with the community where Graffiti can best fit in to the ecosystem and how it may relate to other projects and use cases.
+
POC Demo reviewː
 +
̽ * https://www.youtube.com/watch?v=Dhrthnq1bnw
  
−
== Get Involved ==
+
http://sched.co/1m7wghx
 +
* Etherpadː https://etherpad.openstack.org/p/juno-summit-graffiti
  
−
Graffiti is new project that HP and Intel have started working on and we invite your participation. Please join with us to help us refine our ideas and identify where Graffiti can best fit in to the ecosystem.  We believe that our best path to success is one where we can contribute as either a new project in an existing program or a series of enhancements to existing projects. Your insight and feedback will be critical to making this decision.  We look forward to growing this initiative with you and hope you’ll also contribute!
+
=== IRC ===
  
−
=== IRC ===
+
The various features are maintained by teams in the following IRC channels  on [http://freenode.net/ Freenode].
  
−
The developers use IRC in <code>#graffiti</code> on [http://freenode.net/ Freenode] for development discussion.
+
#openstack-searchlight
 +
#openstack-horizon
 +
#openstack-glance
  
 
=== Development ===
 
=== Development ===
 
* Open source under Apache 2.0
 
* Open source under Apache 2.0
−
* [https://github.com/stackforge/graffiti Graffiti API Service Source Repository]
+
* [https://github.com/stackforge/graffiti Graffiti POC API Service Source Repository] - No Longer Maintained (See Glance, Horizon, Searchlight)
−
* [https://github.com/ttripp/horizon Temporary Horizon POC Fork Repository]
 
−
* [https://bugs.launchpad.net/graffiti Bug tracker]
 
−
* [https://blueprints.launchpad.net/graffiti Feature tracker]
 

Latest revision as of 20:26, 8 January 2021

What's in my cloud?

I've got a lot of resources in my cloud.

  • How do I find what I need?
  • How do I describe what I have?

At its most basic concept, Graffiti's intent is to enable better metadata collaboration across services and projects so that OpenStack users can take advantage of an Enhanced Platform Awareness.

Current Status

The Graffiti project was proposed at the Atlanta (Juno) OpenStack summit. Since then the concepts have been adopted and implemented as part of multiple different OpenStack Projects.

The following information provides much of the background information on where these concepts originated.

Overview

A challenge we've experienced with using OpenStack is discovering, sharing, and correlating metadata across services and different types of resources. We believe this affects both end users and administrators.

For end users, we feel like doing basic tasks like launching instances is too technical for end users and require too much pre-existing knowledge of OpenStack concepts. For example, you should be able to just specify categories like "Big Data" or an "OS Family" and then let the system find the boot source for you, whether that is an image, snapshot, or volume. It should also allow finer grained filtering like filtering on specific versions of software that you want.

For administrators, we’d like there to be an easier way to meaningfully collaborate on properties across host aggregates, flavors, images, volumes, or other cloud resources.

Various OpenStack services provide techniques to abstract low level resource selection to one level higher, such as flavors, volume types, or artifact types. These resource abstractions often allow "metadata" in terms of key-value pair properties to further specialize and describe instances of each resource type. However, collaborating on those properties can be a disconnected and difficult process. This often involves searching wikis and opening the source code. In addition, the metadata properties often need to be correlated across several different services. It becomes more difficult as a cloud's scale grows and the number of resources being managed increases.

We, HP and Intel, believe that both of the above problems come back to needing a better way for users to collaborate on metadata across services and resource types. We started a project called Graffiti to explore ideas and concepts for how to make this easier and more approachable for end users. Please join with us to help move forward together as a community!

We believe that we can make some immediate improvements in Horizon, but that they can't be achieved through Horizon alone and that the benefits should extend to the API and CLI interactions as well. Better cross service collaboration and consistency on metadata should provide benefits that can be leveraged by other projects such as scheduling, reservation, orchestration, and policy enforcement.

Terminology Note

We think the term "metadata" is a somewhat unapproachable term, so we have been exploring with the concept of a "capability". A capability can simply be thought of as a named "tag" which may or may not have properties. The idea is that a user can simply "tag" a capability onto various cloud resources such as images, volumes, host aggregates, flavors, and so on. To the end user, the exact mechanism for how the data is stored is handled for them.

Screencasts

To help explain the ideas of the project, we have a quick screencast demonstrating the concepts running under POC code. Please take a look!

Usage Concepts

  1. Load your metadata definitions (sometimes called properties, tags, or capabilities)
    1. Into the central metadata catalog
  2. Update the resources in the cloud with your tags and capabilities
  3. Let users find the resources with your desired tags and capabilities

Design Concepts

Additional architecture concepts on the Architecture page.

Juno Summit Design Sessioɲ

POC Demo reviewː ̽ * https://www.youtube.com/watch?v=Dhrthnq1bnw

http://sched.co/1m7wghx

IRC

The various features are maintained by teams in the following IRC channels on Freenode. 

#openstack-searchlight
#openstack-horizon
#openstack-glance

Development

Retrieved from "https://wiki.openstack.org/w/index.php?title=Graffiti&oldid=177286"