Jump to: navigation, search

Difference between revisions of "ReleaseNotes/Kilo"

(Identity Federation)
Line 221: Line 221:
* Pluggable Assignment Model - TBD - More Info
* Pluggable Assignment Model - TBD - More Info
* Assignment / Resource Split - TBD - More Info
* Assignment / Resource Split - TBD - More Info
* Extensions to Core functionality - The Keystone team is moving away from the concept of "extensions", all previous extensions are now enabled by default. These now core functions will be marked as Experimental or Stable (as assessed by the Keystone team). The previous extensions that are now enabled by default are: OS-FEDERATION, OS-OAUTH1, OS-ENDPOINT-POLICY and OS-EP-FILTER
* All previous extensions (OS-FEDERATION, OS-OAUTH1, OS-ENDPOINT-POLICY and OS-EP-FILTER) are now enabled by default, and are [http://docs.openstack.org/developer/keystone/extensions.html correspondingly marked] as either "experimental" or "stable".
* Added support for trust redelegation. If allowed when the trust is initially created, a trustee can redelegate the roles from the trust via another trust
* Added support for trust redelegation. If allowed when the trust is initially created, a trustee can redelegate the roles from the trust via another trust
* Added support for explicitly requesting an unscoped token from Keystone, even if the user has a <code>default_project_id</code> attribute set.
* Added support for explicitly requesting an unscoped token from Keystone, even if the user has a <code>default_project_id</code> attribute set.

Revision as of 23:28, 29 April 2015

OpenStack 2015.1.0 (Kilo) Release Notes

The Kilo release of OpenStack is dedicated to the loving memory of Chris Yeoh, who left his family and us way too soon.


General Upgrade Notes

  • TBD

OpenStack Object Storage (Swift)

Key New Features

Erasure Code (beta)

Swift now supports an erasure-code (EC) storage policy type. This allows deployers to achieve very high durability with less raw capacity as used in replicated storage. However, EC requires more CPU and network resources, so it is not good for every use case. EC is great for storing large, infrequently accessed data in a single region.

Swift's implementation of erasure codes is meant to be transparent to end users. There is no API difference between replicated storage and EC storage.

To support erasure codes, Swift now depends on PyECLib and liberasurecode. liberasurecode is a pluggable library that allows for the actual EC algorithm to be implemented in a library of your choosing.

Full docs are at http://swift.openstack.org/overview_erasure_code.html

Composite tokens

Composite tokens allow other OpenStack services to store data in Swift on behalf of a client so that neither the client nor the service can update the data without both party's consent.

An example of this is that a user requests that Nova save a snapshot of a VM. Nova passes the request to Glance, Glance writes the image to a Swift container as a set of objects. In this case, the user cannot modify the snapshot without also having a valid token from the service. Nor can the service update the data without a valid token from the user. But the data is still stored in the user's account in Swift, which makes accounting simpler.

Full docs are at http://swift.openstack.org/overview_backing_store.html

Data placement updates for smaller, unbalanceable clusters

Swift's data placement now accounts for device weight. This allows operators to gradually add new zones and regions without immediately causing a large amount of data to be moved. Also, if a cluster is inbalanced (eg a two-zone cluster where one zone has twice the capacity of the other), Swift will more efficiently use the available space and warn when replicas are placed without enough dispersion in the cluster.

Global cluster replication improvements

Replication between regions will now only move one replica per replication run. This gives the remote region a chance to replicate internally and thus avoid more data moving over the WAN

Known Issues

  • As a beta release, EC support is nearly fully feature complete, but it is lacking support for some features (like multi-range reads) and has not had a full performance characterization. This feature relies on ssync for durability. Deployers are urged to do extensive testing and not deploy production data using an erasure code storage policy.

Upgrade Notes

As always, you can upgrade to this version of Swift with no end-user downtime

  • In order to support erasure codes, Swift has a new dependency on PyECLib (and liberasurecode, transitively). Also, the minimum required version of eventlet has been raised.

OpenStack Compute (Nova)

Key New Features

API v2.1

  • For kilo, by default we are still using v2.0 API code to server v2.0 API requests. It is hoped that in liberty that v2.1 will be used to serve requests for both v2.0 and v2.1.
  • For liberty v2.0 is now frozen, and all new features will now be added into the v2.1 API using the microversions mechanism. Microversion increments released with kilo are:
    • Extending the keypair API to support for x509 certificates, to be used with Windows WinRM, is one of the first API features added as a microversion in the v2.1 API.
    • Exposing additional attributes in os-extended-server-attributes
  • python-novaclient does not yet have support for the v2.1 API
  • The policy enforcement of Nova v2.1 API get improvement.
    • Policy only enforce at the entry of API.
    • Without duplicated rules for single one API anymore.
    • All the v2.1 API policy rule use 'os_compute_api' as prefix which distinguish with v2 API.
    • Due to hard-code permission checks at db layer, part of Nova API isn't configurable by policy before. It's always required admin user. Part of Nova v2.1 API's hard-code permission checks is removed which make API policy configurable. The rest of hard-code permission checks will be removed at Liberty.

Upgrade Support

  • Change https://review.openstack.org/#/c/97946/ adds database migration 267 which scans for null instances.uuid records and will fail if any are found since the migrate ultimately needs to make instances.uuid non-nullable and adds a UniqueConstraint on that column. A helper script is provided to search for null instances.uuid records before running the database migrations. Before running 'nova-manage db sync', run the helper script 'nova-manage db null_instance_uuid_scan' which, by default, will just search and dump results, it does not change anything. Pass the --delete option to the null_instance_uuid_scan command to automatically remove any null records were instances.uuid is null.


  • A selection of performance optimisations
  • We are in the process of making structural changes to the scheduler that will help improve our ability to evolve and improve scheduling. This should not be visible from an end user perspective.

Cells v2

  • There are some initial parts of cell v2 supported added, but this feature is not yet ready to use.
  • new 'nova-manage api_db sync' and 'nova-manage api_db version' commands for working with the new api database for cells, but nothing is using this database yet so it is not necessary to set it up.

Compute Drivers

Libvirt (KVM)

Known Issues

  • Evacuate recovery code has the potential to destroy data. On nova-compute startup, instances reported by the hypervisor are examined to see if they have moved (i.e. been evacuated) from the current host during the outage. If the determination is made that they were, then they are destroyed locally. This has the potential to choose incorrectly and destroy instances unexpectedly. On libvirt-like nodes, this can be triggered by changing the system hostname. On vmware-like nodes, this can be triggered by attempting to manage a single vcenter deployment from two different hosts (with different hostnames). This will be fixed properly in Liberty, but for now deployments that wish to disable this behavior as a preventive measure can set workarounds.destroy_after_evacuate=False. NOTE: This is not a regression and has been a flaw in the design of the evacuate feature since its introduction. There is no easy fix for this, hence this workaround to limit the potential for damage. The proposed fix in liberty is here: https://review.openstack.org/#/c/161444/.
  • The generate config examples possibly missing some oslo related configuration

Upgrade Notes

Below are changes you should be aware of when upgrading. Where possible, The git commit hash is provided for you to find more information:

  • Neutron ports are no longer deleted after your server is deleted, if you created them outside of Nova: 1153a46738fc3ffff98a1df9d94b5a55fdd58777
  • EC2 API support has been deprecated, and is likely to be removed in kilo, see f098398a836e3671c49bb884b4a1a1988053f4b2
  • Websocket proxies need to be upgraded in a lockstep with the API nodes, as older API nodes will not be sending the access_url when authorizing console access, and newer proxy services (this commit and onward) would fail to authorize such requests 9621ccaf05900009d67cdadeb1aac27368114a61
  • After fully upgrading to kilo (i.e. all nodes are running kilo code), you should start a background migration of flavor information from its old home to its new home. Kilo conductor nodes will do this on the fly when necessary, but the rest of the idle data needs to be migrated in the the background. This is critical to complete before the Liberty release, where support for the old location will be dropped. Use "nova-manage migrate-flavor-data" to perform this transition.
  • Due to the improvement on Nova v2.1 API policy enforcement. There are a lot of change happened to v2.1 API policy. Because v2.1 API didn't released before, those change won't keep back-compatible. It is better to use policy sample configuration instead of old one.
  • VMware rescue VM behaviour no longer creates a new VM and instead happens in place: cd1765459a24e52e1b933c8e05517fed75ac9d41
  • force_config_drive = always has been deprecated, and force_config_drive = True should be used instead: c12a78b35dc910fa97df888960ef2b9a64557254
  • Running hyper-v, if you deployed code that was past this commit: b4d57ab65836460d0d9cb8889ec2e6c3986c0a9b but before this commit: c8e9f8e71de64273f10498c5ad959634bfe79975 you make have problems to manually resolve see: c8e9f8e71de64273f10498c5ad959634bfe79975
  • Changed the default value of: multi_instance_display_name_template see: 609b2df339785bff9e30a9d67d5c853562ae3344
  • Please use "nova-manage db null_instance_uuid_scan" to ensure the DB migrations will apply cleanly, see: c0ea53ce353684b48303fc59393930c3fa5ade58

OpenStack Image Service (Glance)

Key New Features

  • TBD

Known Issues

Upgrade Notes

OpenStack Dashboard (Horizon)

Key New Features

  • Support for Federated authentication via Web Single-Sign-On

When configured in keystone, the user will be able to choose the authentication mechanism to use from those support by the deployment. This feature must be enabled by changes to local_settings.py to be utilized. The related settings to enable and configure can be found here.

  • Support for Theming

A simpler mechanism to specify a custom theme for Horizon has been included. Allowing for use of CSS values for Bootstrap and Horizon variables, as well as the inclusion of custom CSS. See: http://docs.openstack.org/developer/horizon/topics/settings.html#custom-theme-path

  • Sahara UX Improvements

Dramatic improvements to the Sahara user experience have been made with the addition of guided cluster creation and guided job creation pages.

  • Launch Instance Wizard (beta)

A full replacement for the launch instance workflow has been implemented in AngularJS to address usability issues in the existing launch instance workflow. Due to the late inclusion date and limited testing, this feature is marked as beta for Kilo and not enabled by default. To use the new workflow, the following change to local_settings.py is required:


Additionally, you can disable the default launch instance wizard with the following:


This new work is a view into future development in Horizon.

Known Issues

Upgrade Notes

OpenStack Identity (Keystone)

Key New Features

Hierarchical multitenancy

Projects can be nested under other projects by setting the parent_id attribute to an existing project when creating a new project. You can also discovery the parent-child hierarchy through the existing /v3/projects API.

Role assignments can now be assigned to both users and groups on subtrees in the project hierarchy.

This feature will require corresponding support across other OpenStack services (such as hierarchical quotas) in order to become broadly useful.

Fernet tokens

Unlike UUID tokens which must be persisted to a database, Fernet tokens are entirely non-persistent. Deployers can enable the Fernet [token] provider = keystone.token.providers.fernet.Provider in keystone.conf.

Fernet tokens require symmetric encryption keys which can be established using keystone-manage fernet_setup and periodically rotated using keystone-manage fernet_rotate. These keys must be shared by all keystone nodes in a multi-node (or multi-region) deployment, such that tokens generated by one node can be immediately validated against another.

Identity Federation

  • Keystone can now act as a federated identity provider (IdP) for another instance of keystone by issuing SAML assertions for local users, which may be ECP-wrapped.
  • Added support for OpenID Connect as a federated identity authentication mechanism.
  • Added the ability to associate many "Remote IDs" to a single identity provider in keystone. This will help in a case where many identity providers use a common mapping.
  • Added the ability for a user to authenticate via a web browser with an existing IdP, through a Single Sign-On page.


  • Filter parameters specified by API users are now processed by LDAP itself, instead of by keystone.
  • Experimental support was added to store domain-specific identity backend configuration in SQL using the HTTP API. The primary use case for this is to create a new domain the the HTTP API, and then immediately configure a domain-specific LDAP driver for it without restarting keystone.


  • Pluggable Assignment Model - TBD - More Info
  • Assignment / Resource Split - TBD - More Info
  • All previous extensions (OS-FEDERATION, OS-OAUTH1, OS-ENDPOINT-POLICY and OS-EP-FILTER) are now enabled by default, and are correspondingly marked as either "experimental" or "stable".
  • Added support for trust redelegation. If allowed when the trust is initially created, a trustee can redelegate the roles from the trust via another trust
  • Added support for explicitly requesting an unscoped token from Keystone, even if the user has a default_project_id attribute set.
  • Deployers may now opt into disallowing the re-scoping of scoped tokens by setting [token] allow_rescope_scoped_token = false in keystone.conf.

Known Issues

  • TBD

Upgrade Notes

  • XML support in Keystone has been removed as of Kilo. When upgrading from Juno to Kilo, it is recommended that references to XML and XmlBodyMiddleware be removed from the Keystone Paste configuration. This includes removing the XML middleware filters and there references from the public_api, admin_api, api_v3, public_version_api, admin_version_api and any other pipelines that may contain the XML filters.
  • SQL Schema Downgrades are no longer supported. This change is the result of evaluation that downward SQL migrations are not well tested and become increasingly difficult to support with the volume of data-change that occurs in many of the migrations.
  • The following python libraries are now required: cryptography, msgpack-python, pysaml2 and oauthlib
  • keystone.middleware.RequestBodySizeLimiter is now deprecated in favor of oslo_middleware.sizelimit.RequestBodySizeLimiter and will be removed in Liberty.
  • Eventlet-specific configuration options such as public_bind_host, bind_host, admin_bind_host, admin_port, public_port, public_workers, admin_workers, tcp_keepalive, tcp_keepidle have been moved from the [DEFAULT] configuration section to a new configuration section called [eventlet_server]. Similarly, Eventlet-specific SSL configuration options such as enable</enable>, <code>certfile, keyfile, ca_certs, cert_required have been moved from the [ssl] configuration section to a new configuration section called [eventlet_server_ssl].

OpenStack Network Service (Neutron)

Key New Features

  • DVR now supports VLANs in addition to VXLAN/GRE
  • ML2 Hierarchical Port Binding
  • New LBaaS Version 2 API
  • Portsecurity support for the OVS ML2 Driver
  • New Plugins supported in Kilo include the following:
    • A10 Networks LBaaS V2 Driver
    • Brocade LBaaS V2 Driver
    • Brocade ML2 driver for MLX and ICX switches
    • Brocade L3 routing plugin for MLX switch
    • Brocade Vyatta vRouter L3 Plugin
    • Brocade Vyatta vRouter Firewall Driver
    • Brocade Vyatta vRouter VPN Driver
    • Cisco CSR VPNaaS Driver
    • Dragonflow SDN based Distributed Virtual Router L3 Plugin
    • Freescale FWaaS Driver
    • Intel Mcafee NGFW FWaaS Driver
    • IPSEC Strongswan VPNaaS Driver

Known Issues

  • The Firewall-as-a-Service project is still marked as experimental for the Kilo release.
  • Bug 1438819
    • When a new subnet is created on an external network, all existing routers with gateways on the network will get a new address allocated from it. For IPv4 networks, this could consume the entire subnet for router gateway ports.

Upgrade Notes

From Havana, Neutron no longer supported an explicit lease database (https://bugs.launchpad.net/bugs/1202392). This left dead code including unused environment variable. In order to remove the dead code (https://review.openstack.org/#/c/152398/), a change to the dhcp.filter is required, so that line:

dnsmasq: EnvFilter, dnsmasq, root, NEUTRON_NETWORK_ID=

Be replaced by:

dnsmasq: CommandFilter, dnsmasq, root

After advanced services were split into separate packages and received their own service configuration files (specifically, etc/neutron/neutron_lbaas.conf, etc/neutron/neutron_fwaas.conf and etc/neutron/neutron_vpnaas.conf), active service provider configuration can be different after upgrade (specifically, default load balancer (haproxy) and vpn (openswan) providers can be enabled for you even though you previously disabled them in neutron.conf). Please make sure you review configuration after upgrade so that it reflects the desired state of service providers.

Note: this will have no effect if the related service plugin is not loaded in neutron.conf.

Other Notes (Deprecation/EOL etc)

  • Deprecation
    • Brocade Classic plugin for Brocade's VDX/VCS series of hardware switches will be deprecated in the L-Release. The functionality provided by this plugin is now addressed by the ML2 Driver available for the VDX series of hardware. The plugin is slated for removal after this release cycle.

OpenStack Block Storage (Cinder)

Key New Features

  • From this point forward any new database schema upgrades will not require restarting Cinder services right away. The services are now independent of schema upgrades. This is part one to Cinder supporting rolling upgrades!
  • Ability to add/remove volumes from an existing consistency group. Read docs for more info.
  • Ability to create a consistency group from an existing consistency group snapshot. Read docs for more info.
  • Create more fine tuned filters/weighers to set how the scheduler will choose a volume backend. Read the docs for more info.
  • Encrypted volumes can now be backed up using the Cinder backup service. Read the docs for more info.
  • Ability to create private volume types. This is perfect when you want to make volume types available to only a specific tenant or to test it before making available to your cloud. To do so use the cinder type-create <name> --is-public.
  • Oversubscription with thin provision is configurable. Read docs for more info.
  • Ability to add descriptions to volume types. To do so use cinder type-create <name> <description>
  • Cinder now can return multiple iSCSI paths information so that the connector can attach volumes even when the primary path is down (when connector's multipath feature is enabled or not enabled).

Known Issues

None yet

Upgrade Notes

  • The 'host' config option for multiple-storage backends in cinder.conf is renamed to 'backend_host' in order to avoid a naming conflict with the 'host' to locate redis. If you use this option, please ensure your configuration files are updated.

OpenStack Telemetry (Ceilometer)

Key New Features

  • Support to add jitter to polling cycles to ensure pollsters are not querying service's api at the same time
  • Ceilometer API RBAC support
  • Improved Event support:
    • Multi-pipeline support to enable unique processing and publishing of events
    • Enabled ability to capture raw notification messages for auditing and postmortem analysis
    • Support for persisting events into ElasticSearch
    • Publishing support to database, http, file, kafa and oslo.messaging supported message queues
    • Option to split off the events persistence into a separate database
    • Telemetry now supports to collect and store all the event type meters as events. A new option, disable_non_metric_meters, was added to the configuration in order to provide the possibility to turn off storing these events as samples. For further information please see the Telemetry Configuration Reference
    • The Administrator Guide in OpenStack Manuals was updated with a new Events section, where you can find further information about this functionality.
  • Improved pipeline publishing support:
    • Support to publish events and samples to Kafka or HTTP targets
    • Publish data to multiple queues
  • Additional meters
    • memory and disk meters for Hyper-V
    • disk meters for LibVirt
    • power and thermal related IPMI meters, more meters from NodeManager
    • ability to meter Ceph
  • IPv6 support enabled in Ceilometer udp publisher and collector
  • Gnocchi dispatch support for ceilometer-collector
  • Self-disabled pollster mechanism

Known Issues

  • TBD

Upgrade Notes

  • Deprecated meters:
    • The instance:<flavor> meter is deprecated in the Kilo release. In order to retrieve samples or statistics based on flavor you can use the following queries:
  ceilometer statistics -m instance -g resource_metadata.instance_type
  ceilometer sample-list -m instance -q metadata.instance_type=<value>

OpenStack Orchestration (Heat)

Key New Features

  • Improved scaling using nested stacks
    • Heat will RPC actions on any resource that is based on a template. This should help to spread the load when dealing with large complex stacks.
  • oslo versioned objects
    • The database layer now uses oslo versioned objects to aid in future upgrades. This will allow a newly upgraded heat-engine to use a database with an older schema. Note that this will not help with upgrading to kilo.
  • New template functions
    • There is a new HOT template version "20150430" which includes two new functions "digest" and "repeat".
  • Multiregion stacks
  • Software config signalling using swift
  • Triggering new software deployments from heatclient
  • stack snapshots
  • Access to Heat services
    • The admin now has similar access to services as other projects. This is in the form of "heat-manage service-list" and via horizon. This feature reports the active heat-engines.
  • Improved validation for nova and neutron properties.
  • stack hooks
  • New contributed resources
    • Mistral resources
    • keystone resources

supported with Keystone v3 server for Project, Role, User and Group

Known Issues

None yet

Upgrade Notes

  • The default of the configuration option "num_engine_workers" has changed from 1 to a number based on the the number of CPUs. This is now the same as the way other projects set the number of workers.
  • The default for the configuration option "max_nested_stack_depth" has been increased to 5.
  • There is a new configuration option "convergence" it is by default off. This feature is not yet complete and this option should remain off.

Other Notes (Deprecation/EOL etc)


  • The follow resources are deprecated OS::Heat::HARestarter and OS::Heat::CWLiteAlarm
  • The CloudWatch API (heat-api-cw)

OpenStack Database service (Trove)

Key New Features

Known Issues

None yet

Upgrade Notes

None yet

OpenStack Data Processing service (Sahara)

Key New Features

  • New plugins, their features and versions:
    • MAPR
    • Apache Storm
    • Apache Hadoop 2.6.0 was added, Apache Hadoop 2.4.1 deprecated
    • New services for CDH plugin added up to HDFS, YARN, Spark, Oozie, HBase, ZooKeeper and other services
  • Added indirect VM access for better utilization of floating IPs
  • Added event log support to have detailed info about provisioning progress
  • Optional default node group and cluster templates per plugin
  • Horizon updates:
    • Guided cluster creation and job execution
    • Filtering on search for objects
  • Editing of Node Group templates and Cluster templates implemented
  • Added Shell Job Type for clusters running Oozie
  • New Job Types endpoint to query list of the supported Job Types

Known Issues


Upgrade Notes

Details: http://docs.openstack.org/developer/sahara/userdoc/upgrade.guide.html#juno-kilo

  • Sahara now requires policy.json configuration file.

OpenStack Bare Metal service (Ironic)

Key New Features

State Machine

Ironic now uses a formal model for the logical state of each node it manages.[1] This has enabled the addition of two new processes: cleaning and inspection.

  • Automatic disk erasure between tenants is now enabled by default. This may be extended to perform additional cleaning steps, such as re-applying firmware, resetting BIOS settings, etc.[2]
  • Both in-band and out-of-band methods are available to inspect hardware. These methods may be used to update Node properties automatically.[3]

Version Headers

The Ironic REST API expects a new X-OpenStack-Ironic-API-Version header be passed with each HTTP[S] request. This header allows client and server to negotiate a mutually supported interface.[4] In the absence of this header, the REST service will default to a compatibility mode and yield responses compatible with Juno clients. This mode, however, prevents access to most features introduced in Kilo.

Hardware Driver Changes

The following new drivers were added:

The following enhancements were made to existing drivers:

Support for third-party and out-of-tree drivers is enhanced by the following two changes:

  • Drivers may store their own "internal" information about Nodes.
  • Drivers may register their own periodic tasks to be run by the Conductor.
  • vendor_passthru methods now support additional HTTP methods (eg, PUT and POST).
  • vendor_passthru methods are now discoverable in the REST API. See node vendor passthru and driver vendor passthru

Other Changes

  • Logical names may be used to address Nodes, in addition to their canonical UUID.
  • For servers with varied local disks, hints may be supplied that affect which disk device the OS is provisioned to.
  • Support for fetching kernel, ramdisk, and instance images from HTTP[S] sources directly has been added to remove the dependency on Glance. Using ironic as a standalone service
  • Nodes may be placed into maintenance mode via REST API calls. An optional maintenance reason may be specified when doing so.

Known Issues

  • Running more than one nova-compute process is not officially supported.
    • While Ironic does include a ClusteredComputeManager, which allows running more than one nova-compute process with Ironic, it should be considered experimental and has many known problems.
  • Drivers using the "agent" deploy mechanism do not support "rebuild --preserve-ephemeral"

Upgrade Notes

  • IPMI Passwords are now obfuscated in REST API responses. This may be disabled by changing API policy settings.
  • The "agent" class of drivers now support both whole-disk and partition based images.
  • The driver_info parameters of "pxe_deploy_kernel" and "pxe_deploy_ramdisk" are deprecated in favour of "deploy_kernel" and "deploy_ramdisk".
  • Drivers implementing their own version of the vendor_passthru() method has been deprecated in favour of the new @passthru decorator.

Juno to Kilo

The recommended upgrade process is documented here:

Upgrading from Icehouse "nova-baremetal"

An upgrade from an Icehouse Nova installation using the "baremetal" driver directly to Kilo Ironic is untested and unsupported. Instead, please follow the following upgrade path:

  1. Icehouse Nova "baremetal" -> Juno Nova "baremetal"
  2. Juno Nova "baremetal" -> Juno Ironic
  3. Juno Ironic -> Kilo Ironic

Documentation for steps 1 and 2 is available at: https://wiki.openstack.org/wiki/Ironic/NovaBaremetalIronicMigration

OpenStack Documentation

  • [1]New Ironic State Machine
  • [2]Node Cleaning
  • [3]Hardware Inspection
  • [4]REST API "micro" versions