KeystonePerformance

= Keystone Performance = This is to track the performance work related to Keystone.

Methodology

 * 1) Install Havana Stable   on a bare metal.
 * 2) Create one instance of m1.medium flavor and other of type m1.large, so that we can have different CPU and memory config
 * 3) Install Keystone Manually on both of the above created instances
 * 4) Using python multiprocessing module create users in parallel using keystoneclient.v2_0 module on each one of them.
 * 5) key.users.create(, "test", "test@test.com") where key = client.Client( .... )
 * 6) Collect the CPU, Disk, Memory and Database related stats while user creation is in progress.

Methodology
In this story we will be measuring keystone performance for those sequencial tests:
 * User creation
 * User token generation
 * User token validation

Test setup
Devstack on ubuntu server 13.10, with most services disabled (beside keystone, mysql and ldap): for service in g-api g-reg n-api n-crt n-obj n-cpu n-net n-cond \ cinder c-sch c-api c-vol n-sch n-novnc n-xvnc n-cauth \ horizon rabbit tempest; do   echo disable_service $service >> localrc done

Added at the end of localrc: ADMIN_PASSWORD=nomoresecrete SERVICE_TOKEN=ADMIN OFFLINE=True

KEYSTONE_TOKEN_FORMAT=UUID KEYSTONE_IDENTITY_BACKEND=ldap

enable_service ldap KEYSTONE_CLEAR_LDAP=yes LDAP_PASSWORD=ldappass

Test scripts
Used scripts can be found there: https://github.com/TristanCacqueray/keystone-perfs/tree/master/src Basically:
 * User creation (perf-create-users.sh):
 * Generate user's tokens list (perf-gen-tokens.sh):
 * Token validation (perf-validate-tokens.sh):

Performance data acquisition
And the main script to automate the benchmark is bench.sh. This will write real time (userland + kernelland) in /tmp/perf-test_name-TOKEN_FORMAT-BACKEND:

Performances results
Note that this is a sequencial benchmarks (no parallel requests). After running the data acquisition several time, here is the average observed time in seconds for 100 items (users or tokens):

First remarks:
 * UUID token generation is faster than PKI (obviously).
 * User creation does not depends on token type, and is faster on LDAP.
 * Overall, LDAP performs better, even for token validation.

Description
In this story, we will measure the max number of PKI_token_authenticate or PKI_token_revocation_list_get requests that Keystone V3 can handle per second. And then we will try to improve the value.

The test environment
Hardware info of host:8 cores, 24G memory OS:suse11.2 Keystone log level:WARN Keystone DB:postgres sql Token format:PKI Api version:V3

Methodology

 * 1) When server is ready, the client process will start to send sequencial requests constantly.
 * 2) Change the number of client process to make the number of reponse reached the maximum.
 * 3) Run the client processes 20 seconds
 * 4) Calculate the concurrency value per second.

Deployment
There are two hosts. The one used to run client processes, the other used to run keystone server and sql server.

Deployment
There are two hosts. The one used to run client processes, the other used to run keystone server running inside Apache and sql server.

Deployment
There are there hosts:client processes, keystone server running inside Apache and sql server, keystone server running inside Apache and haproxy.

Deployment
There are four hosts:client processes, keystone server running inside Apache, keystone server running inside Apache and sql server, keystone server running inside Apache and haproxy.

Deployment
There are five hosts:client processes, keystone server running inside Apache, keystone server running inside Apache, keystone server running inside Apache and sql server, keystone server running inside Apache and haproxy.

Next step
Through the data above we can find that the concurrency value per second does not increase when the number of hosts increased from three to four. We guess that the postgres sql is bottleneck. So, we will try to improve sql next step.