Blueprint-os-api-docs


 * Launchpad Entry: Design for OpenStack API Docs
 * Created: Diane Fleming
 * Contributors: Anne Gentle, Sam Harwell
 * Related Blueprint: Blueprint-restructure-documentation
 * Related Bug: 1326515
 * Related documents: JSON/REST checklist
 * Planned implementation release: Icehouse (partial); Juno (complete)

Overview
This blueprint describes a plan to improve documentation for OpenStack application developers and contributors in the Icehouse and Juno releases.

In addition to providing documentation to experienced developers, this revised documentation will describe the next step for end users who know how to use the dashboard and client commands to interact with an OpenStack cloud and who want to progress to using the APIs directly or indirectly through cURL commands or open SDKs.

Because we lack API guidance information and because we publish the API specifications on the docs.openstack.org page, OpenStack users mistake these specs for official API references and guides (when they are not). This blueprint proposes changes that will address this issue.

History
Since mid-2012, the OpenStack API docs and dev teams have produced the following API docs:


 * A series of API Reference pages. Originally one long page, now split into one page per OpenStack project. OpenStack users access this reference information to develop applications that run on OpenStack. These pages reflect fully implemented APIs. Sourced from WADL files with some XML wrapper text in this repository.


 * A series of API Specifications. These specifications record the design decisions made by API developers as they develop the OpenStack APIs. These documents reflect work-in-progress APIs rather than fully implemented APIs. These documents are maintained by both dev and doc teams. Problem: Because we lack API guidance information and because we publish the API specifications on the docs.openstack.org page, OpenStack users mistake these specs for official API references and guides (when they are not). These documents are sourced from XML and WADL files and reside in the following repositories:


 * * http://git.openstack.org/cgit/openstack/compute-api/ - Compute API v2 and Extensions, Compute API v1.0
 * * http://git.openstack.org/cgit/openstack/database-api/ - Cloud Databases API v1.0
 * * http://git.openstack.org/cgit/openstack/identity-api/ - Identity Service API v3, Identity Service API v2.0
 * * http://git.openstack.org/cgit/openstack/image-api/ - Image Service API v2, Image Service API v1
 * * http://git.openstack.org/cgit/openstack/netconn-api/ - Networking API v2.0 and Extensions
 * * http://git.openstack.org/cgit/openstack/object-api/ - Object Storage API v1
 * * http://git.openstack.org/cgit/openstack/volume-api/ - Block Storage API v2.0

Issues
This blueprint seeks to address these two main issues:


 * 1) We lack an OpenStack developer guide. Because the APIs are always new to someone, we need a companion book for the API reference pages that walks new users through API concepts and usage. Because we lack API guidance information and we publish the API specifications on the docs.openstack.org page, end users mistake these specs for official API references and guides (when they are not).
 * 2) Each API project currently has two versions of their WADL files: one in the openstack/api-site repository, and one in the openstack/project-api repository. These versions are not in sync. Additionally, some API documentation is sourced purely from XML even if a WADL (or WADLs) exist. Consequently any given API method has two and possibly three versions of its documentation. Having multiple API versions and doc locations confuses end users and creates unnecessary work for writers and developers.

Anne Gentle documents these issues in an email.

Goals
In the Icehouse and Juno releases, we plan to address the previously-stated issues as follows:

Goal 1 - New API Guide
Add a new API guide that captures and adds to the existing guidance information in the specs. Benefits: Rather than having multiple places to look for guidance information, end users can go to a single document that guides them through API usage.

Goal 2 - Improved API Reference pages
Continue to provide one API reference page for each implemented OpenStack API. Incorporate all existing reference information in the specs (next bullet) into the WADLs for the API Reference pages. Benefits: By presenting the implemented API reference information in one place, end users are clear about where this documentation resides.

Goal 3 - PDF files generated for each API Reference page
Add PDF files for each API reference page. Each PDF file will be generated from the same WADL files from which the API reference pages are generated. Each API reference page will provide links to these PDFs. Benefits: Users have a printable version of the API reference page information.

Goal 4 - Move API Specs to project repositories and off docs landing page
After moving content from existing API reference/s (specs) into the WADLs for the API Reference pages, move repositories to the appropriate project repositories. Finally, remove the API specs from the http://docs.openstack.org page. The docs team will no longer maintain, build, or publish these docs because these documents describe the work-in-progress APIs. Benefits: Reduces API documentation redundancy. Eliminates duplication of effort by writers for API reference and guidance documentation. Keeps the documentation for work-in-progress APIs separate from the documentation for implemented APIs. Makes it clear to OpenStack developers where the documentation for work-in-progress APIs resides.

BLUEPRINT: API Guide (new)
Purpose


 * Goal 1 - New API Guide.
 * Introduce developers to API concepts, common tasks, and ways of interacting with the OpenStack APIs.
 * Provide guidance for developers who want to extend the OpenStack APIs or write applications that run on an OpenStack cloud.
 * From Anne Gentle: "We need what the Google Developer Support Handbook calls the Developers Guide: A conversational written guide to using the API. A developer’s guide is a walkthrough of how to use the API - it’s like how a teacher would explain the API to their students, but it’s penned down in digital ink (and there’s no talking back!). Some of those students will be new to the API, some will be new to web development entirely, and some will be old pros - the guide should work for all of them."
 * From Sam Harwell: A good rule of thumb is the OpenStack API Guide (item 4 of the blueprint) should not include any new information that cannot be derived from the API complete reference. In other words, you should be able to explain and justify every piece of information presented in the API Guide through a combination of specific information presented at various places throughout the API Complete Reference. Note that this does not detract from the importance of the API Guide; just because the information is available in the complete reference doesn’t mean users will be able to understand the detailed relations between the API calls at first read.

Source material


 * Python Developer Documentation
 * Language Bindings Documentation
 * Tempest Testing
 * Block Storage Service API v2 Reference
 * Compute API v2 and Extensions Reference
 * Identity Service API v2.0 Reference
 * Networking API v2.0 Reference
 * Image Service API v2 Reference
 * Image Service API v1 Reference
 * Object Storage API v1 Reference

CHAPTER: Getting started
Describe each of the APIs, and discuss the ways users can access the APIs.

Note: The guide will provide guidance information for the API Complete Reference for a project. The API Complete Reference for a project will include access information in the form of listing the service type that should be used to locate the service endpoint(s) in the service catalog returned by Authenticate. It will also include information about text encodings, content types, and so on.

CHAPTER: Limits
Discuss limits and billing.

Cull material from existing API Reference/s. Provide working examples.

Note: The API Complete Reference will include information about return values indicating that an API call failed due to an artificial limit (e.g. quota or rate limit). Based on this information, clients can be written to perform well regardless of the actual limits that are in place. Additional information in the API Guide might include details about the limits in place for a specific vendor’s implementation of the product.

CHAPTER: Concepts
Cull material from existing API Reference/s. For example, General API Information (each reference has a section like this - we need to consolidate these in an intelligent way.) Also talk about how to access the APIs.

SECTION: Authentication
Cull material from existing API Reference/s. Provide working examples.

Note: In the API Complete Reference, the Identity Service Authenticate method will describe the authentication process. The summary/overview section of the API reference for a specific project can then mention that all calls for that project require the inclusion of an X-Auth-Token header, with a value obtained from the Authenticate method (linking back to the API complete reference for the identity project). Additional information in the API Guide, which is not directly expressed in the API Complete Reference, would be recommended client practices regarding re-authentication of expired tokens.

SECTION: Request/Response Types
Cull material from existing API Reference/s. Provide working examples.

SECTION: Links and References
Cull material from existing API Reference/s. Provide working examples.

SECTION: Paginated Collections
Cull material from existing API Reference/s. Provide working examples.

SECTION: Efficient Polling with the Changes-Since Parameter
Cull material from existing API Reference/s. Provide working examples.

SECTION: Versions
Cull material from existing API Reference/s. Provide working examples.

SECTION: Extensions
Cull material from existing API Reference/s. Provide working examples.

SECTION: Faults
Cull material from existing API Reference/s. Provide working examples.

Note: All available information about faults for an API call should appear in the API Complete Reference for that call. This is covered in more detail under the Responses section of The JSON Checklist.

Step1: Compiling info from the following guides into one.

CHAPTER: Use the APIs
Tutorials, examples, interspersed with specific API concepts (such as types of storage, images, instances, instance rescue, and so on).

BLUEPRINT: API Complete Reference pages (revised)
Purpose


 * Goal 2 - Improved API Reference pages
 * Goal 3 - PDF files generated for each API Reference page
 * Provide developers with searchable and comprehensive Web pages that list API methods and parameters, with examples for each.
 * Provide one reference page for each project: Compute, Image Service, Identity Service, Object Storage, Networking, Orchestration, and Block Storage Service (already implemented). From Sam Harwell: My big concern is the size of the resulting complete reference page. If the entire API for a product is presented on a single page, it will not be friendly for mobile devices, or for hyper-linking users to information about specific calls as part of a community answer. In addition to the URI information and the examples, all of the information from the JSON/REST checklist is critical to the lasting value of the complete API reference.
 * Before deleting the API References, pull reference material from them into WADL files.
 * Source reference information for the API Reference pages from WADL files (already implemented, but WADL files will be updated with more information).
 * Review and improve the existing content so that it includes complete descriptions, request and response parameters, and tested code samples.
 * Use Sam Harwell's JSON/REST checklist to improve the documentation. In a separate effort, Sam Harwell and David Cramer plan to identify where each piece of information in the JSON/REST checklist fits in a WADL file. Then, David plans to enhance clouddocs-mvn-plugin so that it validates WADL files for comprehensiveness and pulls more information from WADL files for display.
 * Note: The API Guide will provide guidance information for the API Complete Reference for a project. The API Complete Reference for a project will include access information in the form of listing the service type that should be used to locate the service endpoint(s) in the service catalog returned by Authenticate. It will also include information about text encodings, content types, and so on.

Source materials
 * API Reference - sourced from WADL files
 * API Reference/s - Pull material from existing API Reference/s into WADL files so everything is single-sourced from WADL files.

BLUEPRINT: API References (specs) (move to project specs repos)
Strategy

Remove these books from the docs.openstack.org site and move source from the docs project-api repositories to the project specs repositories (see below):
 * Goal 4 - Move API specs to project specs repositories
 * Block Storage Service API v2 Reference
 * Block Storage Service API v1 Reference (in the repo, but not on docs.openstack.org)


 * Compute API v2 and Extensions Reference
 * Compute API v1.0 Reference (in the repo, but not on docs.openstack.org)


 * Cloud Databases Developer Guide v1.0 (in the repo, but not on docs.openstack.org)


 * Identity Service API v3 Reference (in the repo, but not on docs.openstack.org)
 * Identity Service API v2.0 Reference


 * Networking API v2.0 Reference
 * Networking API v1.0 Reference (in the repo, but not on docs.openstack.org)


 * Image Service API v2 Reference
 * Image Service API v1 Reference


 * Object Storage API v1 Reference


 * Programming Compute API with Shell and Python, 1st ed. (in the repo, but not on docs.openstack.org)

Move material from these books into the WADL files that source the API Complete Reference web pages.

Move the actual content to the project repositories:

Note
When the existing source files move to the project repositories, the docs team will no longer be responsible for maintenance, builds, and publication of these docs.

Docs would ensure that application developers have usable API information on the API Reference page and in the (single) API Guide.

Also, any existing bookmarks to these books will redirect to http://api.openstack.org/api-ref.html.