KeystonePerformance - Revision history

Hero: /* Performances results */

2013-12-20T03:10:07Z

‎Performances results

Tristan Cacqueray: Moved scripts to github

2013-12-19T15:52:41Z

Moved scripts to github

Neependra Khare: /* Methodology */

2013-12-19T06:56:59Z

‎Methodology

Tristan Cacqueray: /* Performances results */

2013-12-16T17:11:08Z

‎Performances results

Tristan Cacqueray: /* Performances results */

2013-12-16T15:57:22Z

‎Performances results

Tristan Cacqueray: Add KeystonePerformance tests for UUID vs PKI, SQL vs LDAP

2013-12-16T15:54:16Z

Add KeystonePerformance tests for UUID vs PKI, SQL vs LDAP

Neependra Khare at 07:57, 16 December 2013

2013-12-16T07:57:03Z

Neependra Khare at 07:14, 16 December 2013

2013-12-16T07:14:33Z

Ayoung: /* Methodology */

2013-12-13T20:05:10Z

‎Methodology

Neependra Khare: /* Methodology */

2013-12-13T13:13:46Z

‎Methodology

← Older revision		Revision as of 03:10, 20 December 2013
Line 88:		Line 88:
	* User creation does not depends on token type, and is faster on LDAP.		* User creation does not depends on token type, and is faster on LDAP.
	* Overall, LDAP performs better, even for token validation.		* Overall, LDAP performs better, even for token validation.
		+
		+	=== The concurrency value against PKI V3 ===
		+	==== Description ====
		+	In this story, we will measure the max number of PKI_token_authenticate or PKI_token_revocation_list_get requests that Keystone V3 can handle per second. And then we will try to improve the value.
		+	==== The test environment ====
		+	Hardware info of host:8 cores, 24G memory
		+	OS:suse11.2
		+	Keystone log level:WARN
		+	Keystone DB:postgres sql
		+	Token format:PKI
		+	Api version:V3
		+	==== Methodology ====
		+	# When server is ready, the client process will start to send sequencial requests constantly.
		+	# Change the number of client process to make the number of reponse reached the maximum.
		+	# Run the client processes 20 seconds
		+	# Calculate the concurrency value per second.
		+	==== A single-process Pthon-only keystone server ====
		+	===== Deployment =====
		+	There are two hosts. The one used to run client processes, the other used to run keystone server and sql server.
		+	===== Performances results =====
		+	{\| class="wikitable"
		+	\|-
		+	\| '''token_authenticate''' \|\| '''revocation_list_get'''
		+	\|-
		+	\| 10/s \|\| 58/s
		+	\|}
		+	==== A multi-processes Keystone server running inside an Apache container server ====
		+	===== Deployment =====
		+	There are two hosts. The one used to run client processes, the other used to run keystone server running inside Apache and sql server.
		+	===== Performances results =====
		+	{\| class="wikitable"
		+	\|-
		+	\| '''token_authenticate''' \|\| '''revocation_list_get'''
		+	\|-
		+	\| 57/s \|\| 330/s
		+	\|}
		+	==== Two multi-processes Keystone servers running inside an Apache container server ====
		+	===== Deployment =====
		+	There are there hosts:client processes, keystone server running inside Apache and sql server, keystone server running inside Apache and haproxy.
		+	===== Performances results =====
		+	{\| class="wikitable"
		+	\|-
		+	\| '''token_authenticate''' \|\| '''revocation_list_get'''
		+	\|-
		+	\| 105/s \|\| 560/s
		+	\|}
		+	==== There multi-processes Keystone servers running inside an Apache container server ====
		+	===== Deployment =====
		+	There are four hosts:client processes, keystone server running inside Apache, keystone server running inside Apache and sql server, keystone server running inside Apache and haproxy.
		+	===== Performances results =====
		+	{\| class="wikitable"
		+	\|-
		+	\| '''token_authenticate''' \|\| '''revocation_list_get'''
		+	\|-
		+	\| 140/s \|\| 800/s
		+	\|}
		+	==== Four multi-processes Keystone servers running inside an Apache container server ====
		+	===== Deployment =====
		+	There are five hosts:client processes, keystone server running inside Apache, keystone server running inside Apache, keystone server running inside Apache and sql server, keystone server running inside Apache and haproxy.
		+	===== Performances results =====
		+	{\| class="wikitable"
		+	\|-
		+	\| '''token_authenticate''' \|\| '''revocation_list_get'''
		+	\|-
		+	\| 140/s \|\| 1000/s
		+	\|}
		+	==== Next step ====
		+	Through the data above we can find that the concurrency value per second does not increase when the number of hosts increased from three to four. We guess that the postgres sql is bottleneck. So, we will try to improve sql next step.

@@ Line 54: / Line 54: @@
 ===== Test scripts =====
-User creation (perf-create-users.sh):
+Used scripts can be found there: https://github.com/TristanCacqueray/keystone-perfs/tree/master/src
-<pre>
+Basically:
-#!/bin/sh
+* User creation (perf-create-users.sh):
+* Generate user's tokens list (perf-gen-tokens.sh):
-export OS_SERVICE_TOKEN=ADMIN
+* Token validation (perf-validate-tokens.sh):
 ==== Performance data acquisition ====
+And the main script to automate the benchmark is bench.sh.
-This script will write real time (userland + kernelland) in /tmp/perf-test_name-TOKEN_FORMAT-BACKEND:
+This will write real time (userland + kernelland) in /tmp/perf-test_name-TOKEN_FORMAT-BACKEND:
 ==== Performances results ====

@@ Line 7: / Line 7: @@
 ===== Test #1, Create users in parallel and look for CPU, disk or memory bottleneck. =====
 ===== Methodology =====
-# Install RDO Havana Stable [http://openstack.redhat.com/]  on a bare metal.
+# Install Havana Stable [http://openstack.redhat.com/]  on a bare metal.
 # Create one instance of ''m1.medium'' flavor and other of type ''m1.large'', so that we can have different CPU and memory config
-# Install Keystone Manually (from RDO release) on both of the above created instances
+# Install Keystone Manually on both of the above created instances
 # Using python multiprocessing module create users in parallel using keystoneclient.v2_0 module on each one of them.
 ## key.users.create(<user>, "test", "test@test.com") where ''key = client.Client( .... )''

@@ Line 165: / Line 165: @@
 ==== Performances results ====
-After running the data acquisition several time, here is the average observed time:
+Note that this is a sequencial benchmarks (no parallel requests).
 {| class="wikitable"
 |-
 |  || '''LDAP''' || '''SQL'''
 |-
-| '''UUID''' || <pre>user creation   :  0.00
+| '''UUID''' || <pre>user creation   : 29.36
-token generation:  0.00
+token generation: 27.74
-token validation:  0.00</pre>  || <pre>user creation   :  0.00
+token validation:  2.29</pre>  || <pre>user creation   : 31.35
-token generation:  0.00
+token generation: 29.73
-token validation:  0.00</pre>
+token validation:  2.32 </pre>
 |-
-| '''PKI''' || <pre>user creation   :  0.00
+| '''PKI''' || <pre>user creation   : 29.39
-token generation:  0.00
+token generation: 30.40
-token validation:  0.00</pre> || <pre>user creation   :  0.00
+token validation: 2.84</pre> || <pre>user creation   : 31.34
-token generation:  0.00
+token generation: 32.27
-token validation:  0.00</pre>
+token validation:  2.86</pre>
 |}

@@ Line 168: / Line 168: @@
 {| class="wikitable"
 |-
-|  || LDAP || SQL
+|  || '''LDAP''' || '''SQL'''
 |-
-| UUID ||  ||
+| '''UUID''' || <pre>user creation   :  0.00
 |-
-| PKI ||  ||
+| '''PKI''' || <pre>user creation   :  0.00
 |}

@@ Line 22: / Line 22: @@
 === Improvement with multi-core keystone service ===
-=== Compare PKI vs UUID ===
+=== Compare PKI vs UUID, SQL vs LDAP ===
-=== Compare SQL vs LDAP ===
+===== Test setup =====

@@ Line 9: / Line 9: @@
 # Install RDO [http://openstack.redhat.com/]  on a bare metal
 # Create one instance of ''m1.medium'' flavor and other of type ''m1.large'', so that we can have different CPU and memory config
-# Install KeyStone on both of the above created instances
+# Install Keystone (from RDO release) on both of the above created instances
 # Using python multiprocessing module create users in parallel using keystoneclient.v2_0 module on each one of them.
 ## key.users.create(<user>, "test", "test@test.com")