QA/AuthInterface

Auth Interface Consideration for Tempest

Motivation

Currently, QA team tries to address “tempest-lib migration”, which means separating auth functions and test cases from Tempest and moving only functions to other source tree. This effort makes it easy to create project-specific tests in each project source and makes Tempest slimmer.

In addition, writer (dmorita) regards this event that it is the best time to broaden use cases of Tempest. Current Tempest is very useful for automated regression testing but it is very difficult or impossible to run Tempest tests to deployed OpenStack clusters. One of the major reasons is that current Tempest is closely coupled with Keystone. It seems that completely isolating auth modules from functional tests makes it easy to extend those for other identity services and broaden use cases for more useful operations.

Following description contains analysis of current Tempest, suggested use cases of tempest’s and tempest-lib’s future and ideas of refactoring auth modules. Anyone is welcome to add or modify the following description.

Current Implementation (as of Oct. 2014)

A following conceptual diagram describes relations among auth-related classes and a flow of requests from test cases.

Now, Tempest auth modules consist of Credentials, CredentialsProvider, and AuthProvider. Credentials is a structured class which contains authentication information such as an user name, a tenant name and a password. It depends on the specification of a targeting Auth API. The instance of Credentials is provided by CredentialsProvider. AuthProvider plays two roles in this architecture. One role is to send credentials to an identity server and get auth token and service endpoints. Another is to add a token and an endpoint host to requests provided by RestClient in order to make a complete REST request.

Those three classes are a kind of abstract classes therefore those have child classes as shown below.

Both AuthProvider and Credentials classes have sub-classes supporting keystone v2 and v3 API. Each AuthProvider has TokenClient to send a token request in each supported method and extracts a token and service endpoint URLs returned from TokenClient. Each Credentials class has keystone v2 or v3 specific credentials info.

CredentialsProvider manages how to supply credentials. Tempest now supports dynamic and static ways. A dynamic way is implemented in IsolatedCreds class. IsoratedCreds has IdentityClient instance as an identity_admin_client attribute which sends requests to an identity server (keystone v2) to create tenants and users to be used for tests. However, current implementation of IsoratedCreds class has network-project-specific methods, so this class should be refactored in the tempest-lib migration process. A static way is implemented in Accounts and NotLockingAccounts classes. Accounts class produces Credentials instance by loading accounts.yaml file. NotLockingAccounts works in the same manner as Accounts but it restricts from running tests in parallel with multi users.

In addition, as shown in the above diagram, current codes are loosely or closely tied to Tempest’s configuration files. Auth classes must be unlinked from configuration files to migrate auth functions to tempest-lib.

Principles of Auth Design for tempest-lib Migration

1. Auth modules in tempest-lib must not be dependent on any outer file.
2. Auth modules in tempest-lib must not have project-specific features of functional tests. Tempest is also regarded as independent “project”.
3. Only auth modules for official OpenStack identity service can be included in tempest-lib source tree. Auth modules for other auth servers should be extended from tempest-lib’s modules but those must be placed on the outside of tempest-lib.

Just functions should be described in auth modules of tempest-lib. A part of auth-related classes of current Tempest have if-else statements affected by tempest.conf settings, therefore those statements must be removed in the refactoring process. A function to choose appropriate sub-classes of Credentials and AuthProvider for a particular auth server should be placed on the outside of tempest-lib because those depend on what auth server is used in the test cluster.

Auth interface must not include any project-specific feature. For example, current codes of AuthProvider class have (re)set_alt_auth_data methods, which are Tempest-specific features to require more than one test account for running some tests, so these functions should be refactored in future.

Keystone’s Credentials and AuthProvider sub-classes should be included tempest-lib. Keystone is official OpenStack project for an identity service and is used by the largest number of OpenStack clusters. On the other hand, auth modules for other auth services should not be placed on tempest-lib. For example, Credentials and AuthProvider class for TempAuth (Swift’s internal auth system) should be placed on Swift code tree.

Where to Place Auth Modules and Drivers

Use Case 1: Tempest tests for OpenStack cluster with Keystone V2/V3

This is the most major case in large public cloud driven by OpenStack.

In this use case, CredentialsSelector (assumed name) will choose appropriate auth modules for functional tests based on tempest.conf file.

Use Case 2: Project functional tests for limited project Openstack cluster with project-specific auth

One of distinctive cases is testing a small-to-medium sized public or private cloud driven by a part of OpenStack projects. The following example shows the case to run tempest tests for OpenStack Swift clusters where TempAuth is used as its auth service (TempAuth is a Swift-specific auth service).

In this situation, sub-classes of AuthProvider and Credentials for TempAuth should be contained in openstack/swift source tree. In addition, TempAuth does not have an API to create an authentication account, therefore using FileBasedCredsProvider (assumed name) which provides credentials from yaml file must be suitable.

Use Case 3: Tempest tests for an additional front-end server

In many cases, service front-end URL registered in Keystone is load balancer endpoint. To accept much more requests, adding servers behind load balancer is a common use case. Before adding a server to LB, you would like to run Tempest’s test cases toward the server but this is troublesome. This is because Keystone server normally has only load balancer URLs as service endpoints, so additional configuration to add an additional front-end server to service endpoints is mandatory just for temporal testing. This problem can be simply resolved by creating a sub-class of AuthProvider which gives RestClient a service host and a token defined in a conf file preliminarily (a token can be acquired from Keystone in advance). Following diagram shows Swift example, but Nova API or other projects API endpoints can be the same situation.

FixedTokenAndBaseURLProvider (assumed name) decorates a request URL provided by RestClient in the same way as other AuthProviders, but it passes a token, an account info and a baseURL predetermined in tempest.conf.

Future Auth Modules after Migration

From above consideration, auth modules after tempest-lib migration can be as following diagram.

Base auth module classes consist in AuthProvider, Credentials, and CredentialsProvider. This is well-separated interface in current Tempest, therefore future tempest-lib auth will be also the same architecture. Sub-classes of AuthProvider and Credentials are also well-separated by identity APIs in Tempest, but it seems that sub-classes of CredentialsProvider are disorganized and confusing in some degree. Tempest’s IsolatedCreds class has two problems. One is that it contains not only identity-related functions but also network-related functions. Therefore, network-related functions should be separated into a child class and it should be called only in network API tests. Another is the confusing class name. It can be misread with a child class of Credentials, so it should be renamed to IsolatedCredsProvider to be regarded as a child of CredentialsProvider correctly. In addition, “static” CredentialsProvider classes, Accounts and NotLockingAccounts, are too general names and it is nearly impossible to be regarded them as child classes of CredentialsProvider. Therefore, those names also should be changed to more particular names like FileBasedCredsProvider.

Classes or functions which are placed in tempest-lib must not refer conf files. Therefore, many of classes should be refactored to be independent of outer files before migrating those from Tempest to tempest-lib. Project-specific auth modules should be placed in the project source tree. Undoubtedly, modules for individual auth server should be maintained as proprietary but those are highly recommended to be extended by tempest-lib modules. However, testing an additional front-end server (use case #3) can be more common use case of Tempest in future and can be applied for almost all OpenStack services, therefore FixedTokenAndBaseURLProvider should be in tempest source tree.