Jump to: navigation, search

Difference between revisions of "KeystonePerformance"

m (Methodology)
(Moved scripts to github)
Line 54: Line 54:
  
 
===== Test scripts =====
 
===== Test scripts =====
User creation (perf-create-users.sh):
+
Used scripts can be found there: https://github.com/TristanCacqueray/keystone-perfs/tree/master/src
<pre>
+
Basically:
#!/bin/sh
+
* User creation (perf-create-users.sh):
 
+
* Generate user's tokens list (perf-gen-tokens.sh):
export OS_SERVICE_TOKEN=ADMIN
+
* Token validation (perf-validate-tokens.sh):
export OS_SERVICE_ENDPOINT=http://localhost:35357/v2.0
 
for i in $(seq 100); do
 
    keystone user-create --name perf_testuser_$(printf "%04d" $i) \
 
                        --tenant demo --pass demopass 2>&1 > /dev/null
 
    if [ "$?" != 0 ]; then
 
        echo "User creation failed"
 
        break
 
    fi
 
done
 
</pre>
 
 
 
Generate user's tokens list (perf-gen-tokens.sh):
 
<pre>
 
#!/bin/sh
 
 
 
get_id () {
 
    echo `"$@" | awk '/ id / { print $4 }'`
 
}
 
 
 
unset OS_SERVICE_TOKEN OS_SERVICE_ENDPOINT
 
export OS_AUTH_URL=http://localhost:5000/v2.0
 
export OS_TENANT_NAME=demo
 
export OS_PASSWORD=demopass
 
echo -n > /tmp/tokens_list.txt
 
for i in $(seq 100); do
 
    export OS_USERNAME=perf_testuser_$(printf "%04d" $i)·
 
    get_id keystone token-get >> /tmp/tokens_list.txt
 
    if [ "$?" != 0 ]; then
 
        echo "Token get failed"
 
        break
 
    fi
 
done
 
</pre>
 
 
 
Token validation (perf-validate-tokens.sh):
 
<pre>
 
#!/bin/sh
 
 
 
get_id () {
 
    echo `"$@" | awk '/ id / { print $4 }'`
 
}
 
unset OS_SERVICE_TOKEN OS_SERVICE_ENDPOINT
 
export OS_AUTH_URL=http://localhost:5000/v2.0
 
export OS_TENANT_NAME=demo
 
export OS_PASSWORD=nomoresecrete
 
export OS_USERNAME=admin
 
ADMIN_TOKEN=$(get_id keystone token-get)
 
while read USER_TOKEN; do
 
    curl -H "X-Auth-Token:${ADMIN_TOKEN}" http://0.0.0.0:5000/v2.0/tokens/${USER_TOKEN} \
 
        2>&1 | grep "issued_at" > /dev/null
 
    if [ "$?" != 0 ]; then
 
        echo "Token validation failed"
 
        break
 
    fi
 
done < /tmp/tokens_list.txt
 
</pre>
 
  
 
==== Performance data acquisition ====
 
==== Performance data acquisition ====
 
+
And the main script to automate the benchmark is bench.sh.
This script will write real time (userland + kernelland) in /tmp/perf-test_name-TOKEN_FORMAT-BACKEND:
+
This will write real time (userland + kernelland) in /tmp/perf-test_name-TOKEN_FORMAT-BACKEND:
<pre>
 
#!/bin/sh
 
 
 
cd ~/devstack
 
for TOKEN_FORMAT in UUID PKI; do
 
    for BACKEND in ldap sql; do
 
        echo "== ${TOKEN_FORMAT} - ${BACKEND} =="
 
        echo "[+] Unstack // stack..."
 
        ./unstack.sh
 
 
 
        sed -e "s/^KEYSTONE_TOKEN_FORMAT=.*/KEYSTONE_TOKEN_FORMAT=${TOKEN_FORMAT}/"    \
 
            -e "s/^KEYSTONE_IDENTITY_BACKEND=.*/KEYSTONE_IDENTITY_BACKEND=${BACKEND}/"  \
 
            -i localrc
 
        if [ "${BACKEND}" = "ldap" ]; then
 
            sed -e "s/^disable_service ldap$/enable_service ldap/" -i localrc
 
        else
 
            sed -e "s/^enable_service ldap$/disable_service ldap/" -i localrc
 
        fi
 
        ./stack.sh 2>&1 > /dev/null
 
        if [ "$?" != 0 ]; then
 
            echo "Stack.sh failed"
 
            exit 1
 
        fi
 
 
 
        echo "[+] Create users..."
 
        PERF_OUTPUT="/tmp/perf-create-users_${TOKEN_FORMAT}_${BACKEND}"
 
        /usr/bin/time -o ${PERF_OUTPUT} -f "%e" -a ~/bin/perf-create-users.sh
 
        echo -n "-> "; tail -n 1 ${PERF_OUTPUT}
 
        sleep 1
 
 
 
        echo "[+] Generate tokens..."
 
        PERF_OUTPUT="/tmp/perf-gen-tokens_${TOKEN_FORMAT}_${BACKEND}"
 
        /usr/bin/time -o ${PERF_OUTPUT} -f "%e" -a ~/bin/perf-gen-tokens.sh
 
        echo -n "-> "; tail -n 1 ${PERF_OUTPUT}
 
        sleep 1
 
 
 
        echo "[+] Validate tokens..."
 
        PERF_OUTPUT="/tmp/perf-validate-tokens_${TOKEN_FORMAT}_${BACKEND}"
 
        /usr/bin/time -o ${PERF_OUTPUT} -f "%e" -a ~/bin/perf-validate-tokens.sh
 
        echo -n "-> "; tail -n 1 ${PERF_OUTPUT}
 
        sleep 1
 
    done
 
done
 
</pre>
 
  
 
==== Performances results ====
 
==== Performances results ====

Revision as of 15:52, 19 December 2013

Keystone Performance

This is to track the performance work related to Keystone.

Work Items

Identify CPU, Disk, Memory, Database bottlenecks

Test #1, Create users in parallel and look for CPU, disk or memory bottleneck.
Methodology
  1. Install Havana Stable [1] on a bare metal.
  2. Create one instance of m1.medium flavor and other of type m1.large, so that we can have different CPU and memory config
  3. Install Keystone Manually on both of the above created instances
  4. Using python multiprocessing module create users in parallel using keystoneclient.v2_0 module on each one of them.
    1. key.users.create(<user>, "test", "test@test.com") where key = client.Client( .... )
  5. Collect the CPU, Disk, Memory and Database related stats while user creation is in progress.

Effect of caching - memcached

Effect of expired tokens

Overhead of syncing revocation list

Improvement with multi-core keystone service

Compare PKI vs UUID, SQL vs LDAP

Methodology

In this story we will be measuring keystone performance for those sequencial tests:

  • User creation
  • User token generation
  • User token validation
Test setup

Devstack on ubuntu server 13.10, with most services disabled (beside keystone, mysql and ldap):

for service in g-api g-reg n-api n-crt n-obj n-cpu n-net n-cond \
                     cinder c-sch c-api c-vol n-sch n-novnc n-xvnc n-cauth \
                     horizon rabbit tempest; do
    echo disable_service $service >> localrc
done

Added at the end of localrc:

ADMIN_PASSWORD=nomoresecrete
SERVICE_TOKEN=ADMIN
OFFLINE=True

KEYSTONE_TOKEN_FORMAT=UUID
KEYSTONE_IDENTITY_BACKEND=ldap

enable_service ldap
KEYSTONE_CLEAR_LDAP=yes
LDAP_PASSWORD=ldappass
Test scripts

Used scripts can be found there: https://github.com/TristanCacqueray/keystone-perfs/tree/master/src Basically:

  • User creation (perf-create-users.sh):
  • Generate user's tokens list (perf-gen-tokens.sh):
  • Token validation (perf-validate-tokens.sh):

Performance data acquisition

And the main script to automate the benchmark is bench.sh. This will write real time (userland + kernelland) in /tmp/perf-test_name-TOKEN_FORMAT-BACKEND:

Performances results

Note that this is a sequencial benchmarks (no parallel requests). After running the data acquisition several time, here is the average observed time in seconds for 100 items (users or tokens):

LDAP SQL
UUID
user creation   : 29.36
token generation: 27.74
token validation:  2.29
user creation   : 31.35
token generation: 29.73
token validation:  2.32 
PKI
user creation   : 29.39
token generation: 30.40
token validation: 2.84
user creation   : 31.34
token generation: 32.27
token validation:  2.86

First remarks:

  • UUID token generation is faster than PKI (obviously).
  • User creation does not depends on token type, and is faster on LDAP.
  • Overall, LDAP performs better, even for token validation.