Jump to: navigation, search


< Graffiti
Revision as of 01:11, 10 April 2014 by Travis Tripp (talk | contribs) (Can you give me some examples of how Graffiti may be used?)

Graffiti FAQ

What is Graffiti?

  • Graffiti allow users and services to better collaborate on metadata.
    • OpenStack services are great at providing somewhere to use metadata in terms of key-value pairs, but collaborating on metadata is largely a disconnected and difficult process that often involves searching outdated wikis and opening the source code. It becomes more difficult as a cloud's scale grows in terms of the number of resources being managed and services being integrated. Graffiti makes this easier by creating the following concepts:
      • The Dictionary: A common place for services, admins, and users to expose their metadata vocabulary. This is the basis for creating an agreement on how to describe the various capabilities the cloud provides. It supports both unstructured and structured / hierarchical metadata.
      • The Directory: A common API to tag and search across existing and new services for cloud content based on the metadata vocabulary, ad-hoc properties, or freeform text.
      • The Resource Metadata Repository: A persistent shared metadata repository for services to publish information about cloud resources. This can optionally be used by services instead of or in addition to having their own local native storage.
      • Capabilities and Requirements: Graffiti also embraces the idea that cloud resources may be described using capabilities and requirements, a concept influenced by some parts of OpenStack today as well as by industry specifications like OASIS TOSCA and CAMP.

Can you give me some examples of how Graffiti may be used?

  • The initial use case we are targeting is to improve the launch instance workflow with the following.
    • We will show how Graffiti slides right into Horizon to enhance the image, snapshot, volume, and flavor UIs to describe the resources using a common vocabulary.
    • Because of this, we'll show how the existing launch instance wizard can easily be improved for users to definitively find the images, snapshots, and volumes that have the software capabilities they need. This is particularly needed in clouds with large numbers of images, snapshots, and volumes.
    • Once an actual boot source is selected the compatible flavors will be automatically filtered based on its compute requirements.
    • In addition, we want user to be able to choose their desired service level objectives (SLO) and then automatically filter the flavors that can provide the SLO or completely separate the flavor selection from SLO selection by sending in the correct scheduler hints for the selected SLOs.
  • With the above, you can easily create a marketplace like experience for launching instances from images and bootable volumes using Graffiti.
    • For example, with Amazon you can categorize images, snapshots, and volumes into a hierarchy of application categories like Business Software or Developer Tools. Each of those can have sub categories. With Graffiti, an admin can create this kind of hierarchy in the dictionary. Users can then simply tag images and volumes with the appropriate categories from Graffiti (Graffiti flattens the tag to match the name / value pair format in Glance / Cinder). Later, when an instance is launched or images are browsed, the Graffiti API can be used to display / filter the available application category hierarchy. When a user selects a category, the Graffiti API automatically searches Glance for the correct images or Cinder for the appropriate volumes. This of course isn't limited to just categories and hierarchy, but can include capabilities like specific OS and software packages installed on the image or volume.
    • In addition, with Graffiti, you can differentiate whether the image / volume provides a capability or requires a capability. So, when you tag an image in Glance, you can specify that it requires a certain amount of CPU or a certain host capability (like a GPU) and then Graffiti can be used to automatically find flavors that provide the desired capabilities.

  • Similar to images, you could use Graffiti to apply metadata to application template catalogs
    • Since the metadata definitions (such as application categories) are stored in the Graffiti dictionary, they can be leveraged / reused in any other kind template repository. So, for example, if Glance is extended to be able to host other kinds of artifacts such as Heat templates or another project supports adding metadata to its catalog, the same metadata can easily be used there.

  • OpenStack provisioning tools like Fuel can be enhanced to store host information in the Graffiti resource reposotory so that other services can leverage the information.
    • Today, Fuel captures OpenStack host information that could be shared in a central repository, making it available to other services. The discovered information about the hosts could be stored as metadata about the hosts. Other services, such as Intel's Trust Execution Technology or Intel's Service Assurance Administrator could leverage this repository and publish additional static and runtime metadata about the hosts. For example, a compute host could be published into the repository. It then could be tagged with having TXT capabilities. The actual status of being trusted could be updated directly in the resource repository. Then downstream scheduling activities and filters could directly use the metadata in the shared repository to help make decisions.

  • Allow policy engines to define a vocabulary that they use to make intelligent decisions.
    • Perhaps a general policy engine could be built that supports creating rules based on metadata tagged on various cloud resources (anything from host aggregates to subnets). This could be its own vocabulary with built in rules or end user supplied rules. For example, perhaps certain applications can only be deployed on networks / subnets / storage that have been tagged with a certain level of guaranteed data privacy or renundancy. How would admins discover / track the available metadata so that they can tag this information on the various networks, subnets, storage, host aggregates and know that their policy engine will be able to find the right resources? We believe Graffiti could provide that central Dictionary of metadata. The rule metadata (expressed as capability types) could be loaded into the dictionary so that it is discoverable and usable from API, CLI, and UI.

  • Create redistributable application packages without relying on manual mappings.
    • The key is giving users, admins, and template designers a vocabulary to describe their resources so they can connect them using the concepts of requirements and capabilities.
    • The Graffiti concepts are based on a common pattern that we've observed in enterprise software products specializing in portable application deployment and reiterated in industry standard concepts such as the OASIS TOSCA and Oasis CAMP specifications. One of the base concepts is that you can take elements of applications and describe what capabilities it provides and what capabilities it requires. A central notion in Graffiti is that when you create a metadata definition in the Dictionary, you are describing a type of capability. You then "tag" that capability on different resources and specify either whether the resource provides the capability or if it requires the capability.
      • For example, an orchestration designer may provide a software configuration template that deploys and configures Wordpress. This template could be tagged with providing a certain release of Wordpress and also tagged as requiring a certain OS, certain amount of disk, a certain amount of CPU, and a certain version of MySQL. This template could be published, shared, and imported into various cloud deployments independent of any of the other pieces. When imported into a new cloud deployment, the required capabilities can be scanned to provide automatic filtering of Images with the required OS, flavors with required compute, and other orchestration templates or images that provide MySQL. In addition, the template itself could be found based on filtering for the target OS platform.

Does Graffiti store all meta data?

  • Not necessarily. There are different types of data.
    • Metadata Definitions (Dictionary): Stored in Graffiti and available through REST/CLI API. The dictionary also is intended to be able to proxy Metadata definitions from external sources. For example, if a Nova filter exposes its available extra specs through the Nova API, Graffiti will be able to also show these extra specs.
    • Resource Metadata : This can continue to be stored directly in existing services like Glance, Nova etc using the vocabulary from dictionary. Graffiti will provide a common API (Directory) which will allow users to search and modify metadata in Glance, Nova, etc using a single interface. Internally Graffiti will take care of searching its own metadata repository as well external services like Glance, Nova etc. In addition, Graffiti is designed to have a cache provider plugged in for enhanced search response time across data providers.

Do we all have to agree on the same metadata vocabulary?

  • No, since it is difficult, if not impossible, to achieve a single agreed upon vocabulary, Graffiti supports a separation of vocabularies using namespaces and RBAC.
    • Effectively, you can load a vocabulary into a namespace. Any terms / hierarchy / etc in that namespace is treated distinctly from the same terms / hierarchy / etc in other namespaces. We then allow specifying role permissions on visbility and modification of the namespace. We support user, project, and cloud level namespace visibility and editting privileges. So, a single user can have their own namespace that is distinct from a namespace visible to the world. We have created various suggested namespaces with capabilities for people to get started. We also think it makes sense to start formalizing some of them so that every OpenStack deployment will have a standard set of namespaces. And finally, we intend to provide a mapping of existing standards into Graffiti dictionary namespaces.

Are there other related blueprints that need to be submitted?

  • Yes, since we believe Graffiti will be useful across services and is especially useful for the user experience, there are a number of related areas that we believe we need to submit blueprints.
      • The specific blueprints will be determined based on feedback we get from the community. For example, if Graffiti remains a standalone, non-integrated OpenStack project, we would like more plugin points to the Dashboard (AKA Horizon) to allow inline integration for non-core OpenStack projects to be added to existing workflows and resource actions. For example, this would allows us to seamlessly plugin to the instance launcher, the images / snapshots, and volumes pages. If we get feedback that this should be part of an existing project (or two), then we will revise the concepts and contribute blueprints within the context of the existing project. We also think some existing services will need related blueprints, such as adding metadata tagging on network resources.

Shouldn't this be a part of Glance?

  • We believe the concepts are much broader than just images and artifact metadata, but perhaps the concepts can make sense as a starting point in Glance that we then contribute and incubate. We're open to feedback.
    • For example, we believe Cinder Volumes, Nova flavors, Nova compute hosts, and Neutron network segments also should be able to be tagged with capabilities (metadata) from the shared Graffiti dictionary. This will facilitate tagging capabilities (metadata) across cloud resources. Doing that will then enable users, policy engines, and orchestration templates to declare what they require using the common vocabulary and then allow automation to select or at least guide the user to the actual resources (flavors, networks, etc) that meet the requirements.

How does this relate to Murano?

  • We believe that Graffiti is a tool that can be used by multiple services and Dashboard UIs, including Murano.
      • For example, the concept of "Application Categories" is metadata that can be defined in Graffiti and then be used on both images and application definitions. This can be used by simple VM launching or by a broader application catalog. In addition, we believe Nova flavors, Nova compute hosts, and Neutron network segments also should be able to be tagged with capabilities (metadata) from the shared Graffiti dictionary. This will facilitate tagging capabilities (metadata) across cloud resources. Doing that will then enable users, policy engines, and orchestration templates to declare what they require using the common vocabulary and then allow automation to select or at least guide the user to the actual resources (flavors, networks, etc) that meet the requirements.