Jump to: navigation, search

Difference between revisions of "Graffiti"

m (→‎Initial use case)
(→‎Current Status)
 
(87 intermediate revisions by the same user not shown)
Line 1: Line 1:
 +
 +
== What's in my cloud? ==
 +
 +
I've got a lot of resources in my cloud.
 +
 +
* How do I find what I need?
 +
* How do I describe what I have?
 +
 +
At its most basic concept, Graffiti's intent is to enable better metadata collaboration across services and projects so that OpenStack users can take advantage of an Enhanced Platform Awareness.
 +
 +
== Current Status ==
 +
 +
The Graffiti project was proposed at the Atlanta (Juno) OpenStack summit. Since then the concepts have been adopted and implemented as part of multiple different OpenStack Projects.
 +
 +
* Glance Metadata Definition Catalog
 +
** http://docs.openstack.org/developer/glance/metadefs-concepts.html
 +
** https://github.com/openstack/glance/tree/master/etc/metadefs
 +
** https://youtu.be/zJpHXdBOoeM
 +
* Searchlight
 +
** http://launchpad.net/searchlight
 +
** https://wiki.openstack.org/wiki/Searchlight
 +
* Nova features
 +
** Scheduling filters like Numa topology
 +
* Horizon features
 +
** An admin UI for managing the catalog
 +
*** (Admin —> Metadata Definitions) (Kilo)
 +
** A widget for associating metadata to different resources
 +
*** (Update Metadata action on each row item below)
 +
*** admin -> images (Juno)
 +
*** admin -> flavors (Kilo)
 +
*** admin —> Host Aggregates (Kilo)
 +
*** project —> images (Liberty)
 +
*** project —> instances (Mitaka)
 +
** The ability to add metadata at launch time
 +
*** project —> Launch Instance (ng launch instance enabled) (Mitaka)
 +
 +
The following information provides much of the background information on where these concepts originated.
  
 
== Overview ==
 
== Overview ==
−
We love OpenStack and we want to make it more approachable for end users.
 
  
−
We want users to be able to declare the capabilities they want from the system without being technical experts on the nuances of OpenStack, whether this is to launch simple instances or to launch full application stacks with their desired level of service assurance.
+
A challenge we've experienced with using OpenStack is discovering, sharing, and correlating metadata across services and different types of resources. We believe this affects both end users and administrators.
 +
 
 +
For end users, we feel like doing basic tasks like launching instances is too technical for end users and require too much pre-existing knowledge of OpenStack concepts. For example, you should be able to just specify categories like "Big Data" or an "OS Family" and then let the system find the boot source for you, whether that is an image, snapshot, or volume.  It should also allow finer grained filtering like filtering on specific versions of software that you want.
 +
 
 +
For administrators, we’d like there to be an easier way to meaningfully collaborate on properties across host aggregates, flavors, images, volumes, or other cloud resources.
 +
 
 +
Various OpenStack services provide techniques to abstract low level resource selection to one level higher, such as flavors, volume types, or artifact types. These resource abstractions often allow "metadata" in terms of key-value pair properties to further specialize and describe instances of each resource type. However, collaborating on those properties can be a disconnected and difficult process. This often involves searching wikis and opening the source code. In addition, the metadata properties often need to be correlated across several different services.  It becomes more difficult as a cloud's scale grows and the number of resources being managed increases.
 +
 
 +
We, HP and Intel, believe that both of the above problems come back to needing a better way for users to collaborate on metadata across services and resource types.  We started a project called Graffiti to explore ideas and concepts for how to make this easier and more approachable for end users. Please join with us to help move forward together as a community!
  
−
We are starting small with the idea that we can demonstrate how we can make even "simple" tasks like launching a VM became even simpler for non-technical users.  
+
We believe that we can make some immediate improvements in Horizon, but that they can't be achieved through Horizon alone and that the benefits should extend to the API and CLI interactions as well. Better cross service collaboration and consistency on metadata should provide benefits that can be leveraged by other projects such as scheduling, reservation, orchestration, and policy enforcement.
  
−
At its most basic concept Graffiti is a metadata collaboration tool. Many OpenStack services are great at providing various mechanisms to use metadata in terms of key-value pairs, but collaborating on the actual metadata between developers, admins, services, UI, and end users is largely a disconnected and difficult process that often involves searching outdated wikis, opening source code, and in the best cases making a lot of command line calls. This makes things hard on everybody. We want to make things easier.
+
=== Terminology Note ===
  
−
=== Initial use case ===
+
We think the term "metadata" is a somewhat unapproachable term, so we have been exploring with the concept of a "capability".  A capability can simply be thought of as a named "tag" which may or may not have properties.  The idea is that a user can simply "tag" a capability onto various cloud resources such as images, volumes, host aggregates, flavors, and so on. To the end user, the exact mechanism for how the data is stored is handled for them.
  
−
The initial use case we are targeting is to improve the launch instance workflow with the following:
+
=== Screencasts ===
−
# Make it easy for users to definitively find the images, snapshots, and volumes that have the software capabilities they need. This is particularly needed in clouds with large numbers of images, snapshots, and volumes.
 
−
# Once an actual boot source is selected the compatible flavors will be automatically filtered based on its compute requirements.
 
−
# In addition, we want user to be able to choose their desired service level objectives (SLO) and then automatically filter the flavors that can provide the SLO or completely separate the flavor selection from SLO selection by sending in the correct scheduler hints for the selected SLOs.
 
  
 +
To help explain the ideas of the project, we have a quick screencast demonstrating the concepts running under POC code. Please take a look!
  
−
An integral part of making the above possible includes improving the image, snapshot, volume, and flavor management UIs to be able to apply the correct metadata across these cloud resources.
+
* [https://www.youtube.com/watch?v=f0SZtPgcxk4| Concept Overview]
 +
* [https://youtu.be/zJpHXdBOoeM Availability as of the mitaka release in Horizon and Glance]
  
 +
=== Usage Concepts ===
  
−
==== Demo ====
+
# Load your metadata definitions (sometimes called properties, tags, or capabilities)
 +
## Into the central metadata catalog
 +
# Update the resources in the cloud with your tags and capabilities
 +
# Let users find the resources with your desired tags and capabilities
  
−
== Architecture ==
+
== Design Concepts ==
  
−
Coming soon...
+
Additional architecture concepts on the [[Graffiti/Architecture|Architecture]] page.
  
−
== API ==
+
=== Juno Summit Design Sessioɲ ===
  
−
=== Capability Type Management API ===
+
POC Demo reviewː
 +
̽ * https://www.youtube.com/watch?v=Dhrthnq1bnw
  
 +
http://sched.co/1m7wghx
 +
* Etherpadː https://etherpad.openstack.org/p/juno-summit-graffiti
  
−
=== Resource Search API ===
+
=== IRC ===
  
−
== Get Involved ==
+
The various features are maintained by teams in the following IRC channels  on [http://freenode.net/ Freenode].
  
−
We are still in the very early stages, but we welcome involvement!
+
#openstack-searchlight
 +
#openstack-horizon
 +
#openstack-glance
  
−
* [https://bugs.launchpad.net/graffiti Bug tracker]
+
=== Development ===
−
* [https://blueprints.launchpad.net/graffiti Feature Tracker]
+
* Open source under Apache 2.0
−
* [https://github.com/stackforge/graffiti Code Repository]
+
* [https://github.com/stackforge/graffiti Graffiti POC API Service Source Repository] - No Longer Maintained (See Glance, Horizon, Searchlight)

Latest revision as of 20:26, 8 January 2021

What's in my cloud?

I've got a lot of resources in my cloud.

  • How do I find what I need?
  • How do I describe what I have?

At its most basic concept, Graffiti's intent is to enable better metadata collaboration across services and projects so that OpenStack users can take advantage of an Enhanced Platform Awareness.

Current Status

The Graffiti project was proposed at the Atlanta (Juno) OpenStack summit. Since then the concepts have been adopted and implemented as part of multiple different OpenStack Projects.

The following information provides much of the background information on where these concepts originated.

Overview

A challenge we've experienced with using OpenStack is discovering, sharing, and correlating metadata across services and different types of resources. We believe this affects both end users and administrators.

For end users, we feel like doing basic tasks like launching instances is too technical for end users and require too much pre-existing knowledge of OpenStack concepts. For example, you should be able to just specify categories like "Big Data" or an "OS Family" and then let the system find the boot source for you, whether that is an image, snapshot, or volume. It should also allow finer grained filtering like filtering on specific versions of software that you want.

For administrators, we’d like there to be an easier way to meaningfully collaborate on properties across host aggregates, flavors, images, volumes, or other cloud resources.

Various OpenStack services provide techniques to abstract low level resource selection to one level higher, such as flavors, volume types, or artifact types. These resource abstractions often allow "metadata" in terms of key-value pair properties to further specialize and describe instances of each resource type. However, collaborating on those properties can be a disconnected and difficult process. This often involves searching wikis and opening the source code. In addition, the metadata properties often need to be correlated across several different services. It becomes more difficult as a cloud's scale grows and the number of resources being managed increases.

We, HP and Intel, believe that both of the above problems come back to needing a better way for users to collaborate on metadata across services and resource types. We started a project called Graffiti to explore ideas and concepts for how to make this easier and more approachable for end users. Please join with us to help move forward together as a community!

We believe that we can make some immediate improvements in Horizon, but that they can't be achieved through Horizon alone and that the benefits should extend to the API and CLI interactions as well. Better cross service collaboration and consistency on metadata should provide benefits that can be leveraged by other projects such as scheduling, reservation, orchestration, and policy enforcement.

Terminology Note

We think the term "metadata" is a somewhat unapproachable term, so we have been exploring with the concept of a "capability". A capability can simply be thought of as a named "tag" which may or may not have properties. The idea is that a user can simply "tag" a capability onto various cloud resources such as images, volumes, host aggregates, flavors, and so on. To the end user, the exact mechanism for how the data is stored is handled for them.

Screencasts

To help explain the ideas of the project, we have a quick screencast demonstrating the concepts running under POC code. Please take a look!

Usage Concepts

  1. Load your metadata definitions (sometimes called properties, tags, or capabilities)
    1. Into the central metadata catalog
  2. Update the resources in the cloud with your tags and capabilities
  3. Let users find the resources with your desired tags and capabilities

Design Concepts

Additional architecture concepts on the Architecture page.

Juno Summit Design Sessioɲ

POC Demo reviewː ̽ * https://www.youtube.com/watch?v=Dhrthnq1bnw

http://sched.co/1m7wghx

IRC

The various features are maintained by teams in the following IRC channels on Freenode. 

#openstack-searchlight
#openstack-horizon
#openstack-glance

Development

Retrieved from "https://wiki.openstack.org/w/index.php?title=Graffiti&oldid=177286"