Jump to: navigation, search


< Network/LBaaS
Revision as of 16:08, 17 March 2015 by Rm you (talk | contribs) (Barbican secrets and containers:)

How To Create A TLS Enabled Load Balancer

The following article will walk through the steps required to set up a load balancer to serve TLS terminated traffic. This article is geared towards the current state of the project and will evolve along with the project itself.

Some of the items to be discussed are:

  • Barbican devstack setup
  • Certificate and key generation
  • Barbican secret and container operations
  • Required patchsets for neutron-lbaas
  • Verifying the setup

Neutron-LBaaS utilizes Barbican as its keystore and requires some setting up, let's get started.

Firstly, this article assumes that the reader is familiar with Git, Openstack, Devstack and other related tools and technologies to get this going. This article will not go through setting up Devstack, Openstack, Git or any thing in this regard.

Barbican Devstack:

With that said, assuming you don't have Devstack set up already use this script to get started with Devstack and Barbican: https://gist.github.com/rm-you/6feacb91182f5c011018 Alternatively, the reader can copy the two needed files and add 'barbican' to the enabled services in your localrc/local.conf as described below or follow the official instructions found here: https://wiki.openstack.org/wiki/BarbicanDevStack

$ git clone https://github.com/openstack/barbican.git
$ cp barbican/contrib/devstack/lib/barbican <devstack_dir>/lib/
$ cp barbican/contrib/devstack/extras.d/70-barbican.sh <devstack_dir>/extras.d/

Add 'barbican' to the end of 'enabled_services' in your localrc, then run stack.sh

$ ./<devstack_dir>/stack.sh
Create certificate chain and key.
  • The following may be a little more verbose then whats actually needed, please feel free to create/retrieve cert/key/intermediates as you see fit.
openssl genrsa -des3 -out ca.key 1024 
openssl req -new -x509 -days 3650 -key ca.key -out ca.crt  
openssl x509  -in  ca.crt -out ca.pem 
openssl genrsa -des3 -out ca-int_encrypted.key 1024 
openssl rsa -in ca-int_encrypted.key -out ca-int.key 
openssl req -new -key ca-int.key -out ca-int.csr -subj "/CN=ca-int@acme.com" 
openssl x509 -req -days 3650 -in ca-int.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out ca-int.crt 
openssl genrsa -des3 -out server_encrypted.key 1024 
openssl rsa -in server_encrypted.key -out server.key 
openssl req -new -key server.key -out server.csr -subj "/CN=server@acme.com" 
openssl x509 -req -days 3650 -in server.csr -CA ca-int.crt -CAkey ca-int.key -set_serial 01 -out server.crt

If youd like to also test SNI create atleast one more cert chain with different CN:

openssl genrsa -des3 -out ca2.key 1024 
openssl req -new -x509 -days 3650 -key ca2.key -out ca2.crt  
openssl x509  -in  ca2.crt -out ca2.pem 
openssl genrsa -des3 -out ca-int_encrypted2.key 1024 
openssl rsa -in ca-int_encrypted2.key -out ca-int.2key 
openssl req -new -key ca-int2.key -out ca-int2.csr -subj "/CN=ca-int-test2@stacme.com" 
openssl x509 -req -days 3650 -in ca-int2.csr -CA ca2.crt -CAkey ca2.key -set_serial 01 -out ca-int2.crt 
openssl genrsa -des3 -out server_encrypted2.key 1024 
openssl rsa -in server_encrypted2.key -out server2.key 
openssl req -new -key server2.key -out server2.csr -subj "/CN=test2@stacme.com" 
openssl x509 -req -days 3650 -in server2.csr -CA ca-int2.crt -CAkey ca-int2.key -set_serial 01 -out server2.crt
  • Note the CN for both chains as well use them to verify SNI later
Barbican secrets and containers:
barbican secret store --payload-content-type='application/octet-stream' --payload-content-encoding='base64' --name='certificate' --payload="$(cat serverb64.crt)"
barbican secret store --payload-content-type='application/octet-stream' --payload-content-encoding='base64' --name='private_key' --payload="$(cat serverb64.key)"
barbican container create --name='tls_container' --type='certificate' --secret="certificate=$(barbican secret list | awk '/ certificate / {print $2}')" --secret="private_key=$(barbican secret list | awk '/ private_key / {print $2}')"
  • Note: above is written with a Barbican client bug work around and expects the cert, key, intermediates to be Base64 encoded files.

An example of using OpenSSL to this:

openssl base64 -in <infile> -out <outfile>

Otherwise, leaving the files how they are generated, removing the payload-content-encoding and specifying 'text/plain' for the content-type will work just fine:

barbican secret store --payload-content-type='text/plain' --name='certificate' --payload="$(cat server.crt)"
barbican secret store --payload-content-type='text/plain' --name='private_key' --payload="$(cat server.key)"
barbican container create --name='tls_container' --type='certificate' --secret="certificate=$(barbican secret list | awk '/ certificate / {print $2}')" --secret="private_key=$(barbican secret list | awk '/ private_key / {print $2}')"

Add the second certificate chain to test SNI:

barbican secret store --payload-content-type='text/plain' --name='certificate2' --payload="$(cat server2.crt)"
barbican secret store --payload-content-type='text/plain' --name='private_key2' --payload="$(cat server2.key)"
barbican container create --name='tls_container2' --type='certificate' --secret="certificate=$(barbican secret list | awk '/ certificate2 / {print $2}')" --secret="private_key=$(barbican secret list | awk '/ private_key2 / {print $2}')"
Patch the code:

At this point we should have our certificate container created in Barbican using the certificates we created and are now ready to create the TLS load balancer. Before we can dive into lbaas commands we will need to checkout a few patches and restart a few services to get everything up to date with the un-merged code.

  • python-neutronclient to allow the new TLS settings
  • neutron-lbaas TLS extension, plugin and other related code and Cert manager fixes

Checkout the latest patch of neutron-lbaas from:


Cherry-pick, also neutron-lbaas, from:


Cherry-pick into the python-neutronclient

Update neutron config.


admin_project_name = admin
auth_version = v2

to the 'keystone_authtoken' group.

Setup neutron-lbaas and neutronclient:
$ sudo python setup.py install

within both neutron-lbaas and python-neutronclient to get things set up.

Restart: Restart neutron-server from the Devstack screen session, or any other means of running the server.

Create member for testing (basic devstack cirros instance):
nova keypair-add default --pub-key ~/.ssh/id_rsa.pub 
nova secgroup-add-rule default tcp 22 22
nova secgroup-add-rule default icmp -1 -1
nova boot --image cirros-0.3.0-x86_64-disk --flavor 2 --nic net-id=$(neutron net-list | awk '/ private / {print $2}') member1 --security_groups default --key-name default
Create TLS enabled load balancer:
neutron lbaas-loadbalancer-create $(neutron subnet-list | awk '/ private-subnet / {print $2}') --name lb1

Create listener without SNI:

neutron lbaas-listener-create --loadbalancer lb1 --protocol-port 443 --protocol TERMINATED_HTTPS --name listener1 --default-tls-container=$(barbican container list | awk '/ tls_container / {print $2}')

Create listener with SNI:

neutron lbaas-listener-create --loadbalancer lb1 --protocol-port 443 --protocol TERMINATED_HTTPS --name listener1 --default-tls-container=$(barbican container list | awk '/ tls_container / {print $2}') --sni-container-ids $(barbican container list | awk '/ tls_container / {print $2}') $(barbican container2 list | awk '/ tls_container / {print $2}')

Create pool:

neutron lbaas-pool-create --name pool1 --protocol HTTP --listener listener1 --lb-algorithm ROUND_ROBIN

Create member:

neutron lbaas-member-create pool1 --address <member_address>  --protocol-port 80 --subnet $(neutron subnet-list | awk '/ private-subnet / {print $2}') 
Test it out:
  • For a simple example, our member can run a simple web server on the instance:
while true; do echo -e 'HTTP/1.0 200 OK\r\n\r\nIt Works!' | sudo nc -l -p 80 ; done 

Verify the response:

$ curl -k https://<loadbalancer_address>

Should result in:

It Works!

Verify SNI:

openssl s_client -servername test2@stacme.com -connect <load_balancer_ip>:443

The certificate information should print to screen, verify the CN matches the CN you passed to '-servername'