Jump to: navigation, search


< Graffiti
Revision as of 18:21, 25 March 2014 by Lakshmisampath (talk | contribs) (Graffiti FAQ)

Graffiti FAQ

What is Graffiti?

  • Graffiti allow users and services to better collaborate on metadata.
    • OpenStack services are great at providing somewhere to use metadata in terms of key-value pairs, but collaborating on metadata is largely a disconnected and difficult process that often involves searching outdated wikis and opening the source code. Graffiti makes this easier by creating three concepts:
      • The Dictionary: A common place for for services and users to expose their metadata vocabulary. Metadata definitions can be flat or hierarchical.
      • The Directory: A common API to tag and search across existing and new services for cloud content based on the metadata vocabulary.
      • The Resource Metadata Repository: A shared metadata repository for services to publish information about cloud resources. Can optionally be used by services instead of or in addition to their local native storage.

Does Graffiti store all meta data?

Not necessarily. There are different types of data.

  • Metadata about Metadata (Dictionary): Stored in graffiti and available through REST/CLI API
  • Metadata : Users can continue to store the metadata at the existing storage like glance, nova etc using the vocabulary from dictionary. Graffiti will provide a common API(Directory) which will allow users to search metadata in glance, nova etc using a single interface. Internally graffiti will take care of searching glance, nova etc. Optionally graffiti can act as cache for the retrieved metadata(Resource Metadata Repository) enhancing response time instead of always going to the actual storage.

Can you give me some examples of how Graffiti may be used?

  • You can easily create a marketplace like experience for images and launching instances using Graffiti.
    • For example, with Amazon you can categorize images into a hierarchy of application categories like Business Software or Developer Tools. Each of those can have sub categories. With Graffiti, an admin can create this kind of hierarchy. So, when the images are uploaded, users can simply tag the image with the appropriate categories from Graffiti (Graffiti flattens the tag to match the name / value pair format in Glance). Later, when an instance is launched or images are browsed, the Graffiti API can be used to display / filter the available application category hierarchy. When a user selects a category, the Graffiti API automatically searches Glance for the correct images.
    • In addition, with Graffiti, you can differentiate whether the image requires something or provides something. So, when you tag an image in Glance, you can specify that it requires a certain amount of CPU or a certain host capability (like a GPU) and then Graffiti can be used to automatically find flavors that provide the desired capabilities.

  • Similar to images, you could use the Graffiti dictionary to supplement application catalogs that do more than just images.
    • Since the definitions of the application categories is a set of common metadata stored in the dictionary, the process of defining application categories can be leveraged / reused to tag applications in a catalog of Heat templates or any other kind of advanced application orchestration catalog.

  • OpenStack provisioning tools like Fuel can be enhanced to store host information in the Graffiti resource reposotory so that other services can leverage the information.
    • Today, Fuel captures OpenStack host information that could be shared in a central repository, making it available to other services. The discovered information about the hosts could be stored as metadata about the hosts. Other services, such as Intel's Trust Execution Technology or Intel's Service Assurance Administrator could leverage this repository and publish additional static and runtime metadata about the hosts. For example, a compute host could be published into the repository. It then could be tagged with having TXT capabilities. The actual status of being trusted could be updated directly in the resource repository. Then downstream scheduling activities and filters could directly use the metadata in the shared repository to help make decisions.

  • Allow policy engines to define a vocabulary that they use to make intelligent decisions.
    • Perhaps a general policy engine could be built that supports creating rules based on metadata tagged on various cloud resources (anything from host aggregates to subnets). This could be its own vocabulary with built in rules or end user supplied rules. For example, perhaps certain applications can only be deployed on networks / subnets / storage that have been tagged with a certain level of guaranteed data privacy or renundancy. How would admins discover / track the available metadata so that they can tag this information on the various networks, subnets, storage, host aggregates and know that their policy engine will be able to find the right resources? We believe Graffiti could provide that central Dictionary of metadata. The rule metadata (expressed as capability types) could be loaded into the dictionary so that it is discoverable and usable from API, CLI, and UI.

  • Create redistributable application packages without relying on manual mappings.
    • The key is giving users, admins, and template designers a vocabulary to describe their resources so they can connect them using the concepts of requirements and capabilities.
    • The Graffiti concepts were originally conceived and used in several released enterprise software products that specialized in portable application deployment. The concepts also are based on concepts in the OASIS TOSCA and Oasis CAMP specifications. One of the base concepts is that you can take elements of applications and describe what capabilities it provides and what capabilities it requires. A central notion in Graffiti is that when you create a metadata definition in the Dictionary, you are describing a type of capability. You then "tag" that capability on different resources and specify either whether the resource provides the capability or if it requires the capability.
      • For example, an orchestration designer may provide a software configuration template that deploys and configures Wordpress. This template could be tagged with providing a certain release of Wordpress and also tagged as requiring a certain OS, certain amount of disk, a certain amount of CPU, and a certain version of MySQL. This template could be published, shared, and imported into various cloud deployments independent of any of the other pieces. When imported into a new cloud deployment, the required capabilities can be scanned to provide automatic filtering of Images with the required OS, flavors with required compute, and other orchestration templates or images that provide MySQL. In addition, the template itself could be found based on filtering for the target OS platform.
    • There are some more interesting uses of requirements and capabilities to form a contract on the available properties without requiring explicit input parameters, but that gets more involved than we'll cover here.

Do we all have to agree on the same metadata vocabulary?

  • No, since it is difficult, if not impossible, to achieve a single agreed upon vocabulary, Graffiti supports a separation of vocabularies using namespaces and RBAC.
    • Effectively, you can load a vocabulary into a namespace. Any terms / hierarchy / etc in that namespace is treated distinctly from the same terms / hierarchy / etc in other namespaces. We then allow specifying role permissions on visbility and modification of the namespace. We support user, project, and cloud level namespace visibility and editting privileges. So, a single user can have their own namespace that is distinct from a namespace visible to the world. We have created various suggested namespaces with capabilities for people to get started. We also think it makes sense to start formalizing some of them so that every OpenStack deployment will have a standard set of namespaces. And finally, we intend to provide a mapping of existing standards into Graffiti dictionary namespaces.

Shouldn't this be a part of Glance?

  • We believe the concepts are much broader than just images and artifact metadata, but perhaps the concepts can make sense as a starting point in Glance that we then contribute and incubate. In short, we're trying to get that question answered ourselves.
    • For example, we believe Nova flavors, Nova compute hosts, and Neutron network segments also should be able to be tagged with capabilities (metadata) from the shared Graffiti dictionary. This will facilitate tagging capabilities (metadata) across cloud resources. Doing that will then enable users, policy engines, and orchestration templates to declare what they require using the common vocabulary and then allow automation to select or at least guide the user to the actual resources (flavors, networks, etc) that meet the requirements.