Jump to: navigation, search

ReleaseNotes/Juno

Revision as of 05:47, 19 October 2014 by Ian Y. Choi (talk | contribs) (Created page with "* 아직 없음")
Other languages:
English • ‎فارسی • ‎日本語 • ‎한국어 • ‎中文(简体)‎ • ‎中文(台灣)‎

OpenStack 2014.2 (Juno) 릴리즈 노트

Contents

종합적인 업그레이드 노트

  • simplejson 패키지는 대부분의 프로젝트에서 선택 가능한 의존 패키지입니다, 따라서 전체 프로젝트의 requirements.txt 파일에 포함되어 있지는 않습니다. 그러나이 패키지를 사용하는 경우, 예를 들어 RHEL 6에서 Python 2.6에서 보다 나은 성능 향상을 얻고자 하는 경우, simplejson 버전 2.2.0 이상이 필요합니다. 자세한 내용은 https://bugs.launchpad.net/oslo-incubator/+bug/1361230 을 참조하십시오.

OpenStack Object Storage (Swift)

새로운 주요 기능

Juno 통합 릴리즈에서는 3개의 OpenStack Swift 릴리즈 (2.0.0, 2.1.0, 2.2.0)를 포함하고 있습니다. 이 릴리즈의 변경 내역은 https://github.com/openstack/swift/blob/2.2.0.rc1/CHANGELOG#L1-L173 에서 확인할 수 있습니다. 자세한 릴리즈 사항은 해당 문서를 참고하십시오.

중요한 새로운 기능은 다음과 같습니다. CHANGELOG 및 관련 문서를 읽어보십시오.

  • 스토리지 정책
  • Keystone v3 지원
  • 서버 사이드에서의 계정간 복사
  • 새로운 서버, 존, 지역 추가 시 파티션 배치 개선
  • splice() 시스템 호출을 사용한 Zero-copy GET 응답
  • 병렬화된 개체 감사 기능


알려진 이슈

  • 아직 없음

업그레이드 노트

항상 그래왔듯이, Swift 클러스터는 최종 사용자의 다운 타임 없이 업그레이드 가능합니다. 매 릴리즈 전에 샘플 구성 파일 및 문서를 참고하십시오.

  • 언급할 필요가 있는 몇몇 로깅 변화 사항이 있습니다. 모든 경우에 대해, 이미 잘 동작하고 있는 로그 처리 과정에는 영향을 미치지 않을 것입니다.
    • 스토리지 노드 (account, container, object)의 로그 라인의 끝 부분에 이제 PID를 기록합니다.
    • Object 데몬은 이제 데몬 전체 이름과 함께 user-agent 문자열을 붙여서 보냅니다. (예 : "obj"⇒ "object")
  • 추가 스토리지 정책을 한 번 활성화한 이후에는, Swift 2.0.0 이전 버전으로 다운그레이드할 때 어떤 추가적인 스토리지 정책도 사용이 불가능합니다.
  • IANA의 할당되지 않은 범위에 해당하는 Swift 기본 포트에 대한 업데이트 작업의 일부분으로 bind_port가 이제 필수 설정이 되었습니다. 현재 명시적으로 해당 포트를 설정하는 쪽에는 영향을 주지 않습니다. 그러나, 현재 해당 포트를 설정하지 않은 경우 *_server.conf에 bind_port 설정이 여러분의 ring과 일치하는지 업그레이드의 한 부분으로 확인하십시오.
  • 참고로, 스토리지 정책에는 새로운 데몬인 container-reconciler를 포함합니다.
  • TempURL가 기본으로 허용하는 메소드 구성 설정에 이제 POST과 DELETE 역시 허용합니다. 이는 tempurl이 이러한 동사들을 통해 생성될 수 있음을 의미합니다. 이는 기존 tempurl에 영향을 미치지는 않습니다.
  • Swift에서 업데이트, deprecated, 제거된 옵션 목록은 여기서 확인 가능합니다. http://docs.openstack.org/trunk/config-reference/content/swift-conf-changes-master.html

OpenStack Compute (Nova)

인스턴스 기능

  • 복구에 사용하는 원본 베이스 이미지 대신 사용자가 이미지를 지정할 수 있도록 허용합니다. launchpad 세부 내용
  • config drive를 사용해야만 할 때 이미지를 지정할 수 있도록 허용합니다. launchpad 세부 내용
  • Flavor를 통해 게스트 사용자에게 제공되는 가상 CPU 토폴로지를 사용자와 관리자가 제어 가능하도록 제공합니다. launchpad 세부 내용
  • 복구시 모든 로컬 디스크들을 붙일 수 있도록 허용합니다. launchpad 세부 내용

Networking

  • 네트워크 별로 설정이 가능하도록 nova-network 코드를 개선하였습니다. launchpad 세부 내용
  • 배포자가 인스턴스에 대한 네트워크 정보에 변경 사항이 있을 때 즉시 알려줄 수 있도록 하는 hook을 추가하도록 허용합니다. launchpad 세부 내용
  • Nova 인스턴스가 SR-IOV의 Neutron 포트와 같이 구동 가능하게 되었습니다. launchpad 세부 내용
  • VM이 1개의 네트워크에 여러 인터페이스를 연결할 수 있도록 허용합니다. launchpad 세부 내용

Scheduling

  • 확장 가능한 자원 추적. Extensible Resource Tracking.

Nova에 의해 추적되는 자원 집합은 하드코딩 되어 있는데, 이를 스케줄링을 위한 새로운 유형의 자원에 대해서 추적하는 플러그인을 허용하도록 확장 가능하게 변경되었습니다. launchpad 세부 내용

  • 호스트에 evacuate 기능을 허용하지만, 인스턴스가 이동할 대상 호스트를 선택하는 스케줄러와 함께 허용합니다. launchpad 세부 내용
  • 스케줄러 필터에 호스트 집합을 지원하도록 추가되었습니다. launchpad: disk; instances; IO ops 세부 내용

다른것들

  • Nova i18n 지원, 제약이 가해졌던 Oslo i18n와 업데이트되는 Nova에 번역 가능한 문자열을 추가하여 지연된 번역 지원을 활성화하였습니다. launchpad 세부 내용
  • Slave SQL 서버가 구성되어 있는 경우 주기적으로 처리하는 SQL 쿼리 로드를 Slave SQL 서버로 오프로드시킬 수 있습니다. launchpad 세부 내용
  • 매 60초 마다 업데이트하는 대신 SQL 데이터베이스의 호스트 상태가 바뀌었을 때만 호스트 상태를 업데이트합니다. launchpad 세부 내용
  • 하이퍼바이저 호스트 API 목록에 상태 정보를 추가했습니다. launchpad 세부 내용
  • 서비스 목록을 보고자 할 때 API 호출자가 필터에 여러 상태를 지정할 수 있도록 허용합니다. launchpad 세부 내용
  • 사용자가 생성 가능한 서버 그룹의 수와 크기를 제한하는 quota를 추가했습니다. launchpad 세부 내용

하이퍼바이져 드라이버 별

Hyper-V

Ironic

libvirt

  • 최신 libvirt에서의 인스턴스 목록 나열에 대한 성능이 향상되었습니다. launchpad 세부 내용
  • 네트워크 뒤에 있는 디스크에 대한 스냅샷을 지원합니다. launchpad 세부 내용
  • Ceilometer 리포팅을 위한 QEMO 메모리 벌룬 통계를 지원합니다. launchpad 세부 내용
  • 사용하지 않는 디스크 블록을 스토리지 시스템에 반환하여 핸들링하도록 지원합니다. launchpad 세부 내용
  • 인스턴스에 대한 메타 데이터가 이제 libvirt 도메인 XML에 기록됩니다. 이는 관리자가 문제를 디버깅하는 동안 돕기 위한 용도입니다. launchpad 세부 내용
  • LXC 컨테이너에 대한 네임스페이스를 지원합니다. launchpad 세부 내용
  • RBD 백엔드 디스크에 대한 Copy-on-write 복제 지원 launchpad 세부 내용
  • 인터렉티브한 시리얼 콘솔 제공 launchpad 세부 내용
  • VM 전원 종료 중 게스트 운영체제에 대한 종료 제어를 허용합니다. launchpad 세부 내용
  • 게스트를 위한 지능적인 NUMA 노드 배치 지원 launchpad 세부 내용

vmware

  • VMware 드라이버가 oslo vmware 헬퍼 라이브러리를 사용하도록 이전됨 launchpad 세부 내용
  • VMware에 네트워크 인터페이스 핫 플러그인 지원 추가 launchpad 세부 내용
  • 유지 보수성 향상을 위해 VMware 드라이버의 생성(spawn) 기능이 리팩토링되었습니다. 내부적인 개선 작업이지만, VMware 드라이버의 지원성을 크게 향상시켰기에 여기에서 언급합니다. launchpad 세부 내용

알려진 이슈

  • 아직 없음

Upgrade Notes

  • A list of all updated, deprecated or removed options in Nova can be found at: http://docs.openstack.org/trunk/config-reference/content/nova-conf-changes-master.html
  • The nova-manage flavor subcommand is deprecated in Juno and will be removed in the 2015.1 (K) release: https://review.openstack.org/#/c/86122/
  • https://review.openstack.org/#/c/102212/
  • Minimum required libvirt version is now 0.9.11: https://review.openstack.org/#/c/58494/
  • Nova is now supporting the Cinder V2 API. The Cinder V1 API is deprecated in Juno and Nova will switch over to Cinder V2 by default in the "L" release.
  • Debug log output in python-novaclient has changed slightly to improve readability. The sha1 hash of the keystone token is now printed instead of the token itself - greatly shortening the amount of content being printed while still retaining the ability to determine token mismatch scenarios. In addition, some extra '\n' characters that were being added are removed. Double-check any log parsers!
  • libvirt.volume_drivers config param for nova.conf is deprecated, to be removed in the Lxxxx release. In general, this should affect only a small number of developers working on drivers. If this is you, the recommended approach is to continue your work inside a nova tree.

OpenStack Image Service (Glance)

Key New Features

  • Asynchronous Processing
  • Pull of glance.store into its own library
  • Metadata Definitions Catalog
  • Restricted policy for downloading images.
  • Enhanced Scrubber service allows single instance services multiple glance-api servers cross nodes.

Known Issues

Upgrade Notes

  • A list of all updated, deprecated or removed options in Glance can be found at: http://docs.openstack.org/trunk/config-reference/content/glance-conf-changes-master.html
  • The ability to upload a public image is now admin-only by default. To continue to use the previous behaviour, edit the publicize_image flag in etc/policy.json to remove the role restriction.
  • The requirement and check on UTF-8 charset for DB tables is enforced, operator need to migration tables and existing data to UTF-8 manually if glance-manage complains it during the sync.

OpenStack Dashboard (Horizon)

Key New Features

Sahara

The OpenStack Data Processing project (Sahara) was formally included into the integrated release in Juno and Horizon includes broad support for managing your data processing. You can specify and build clusters to utilize several data types with user specified jobs while tracking the progress of those jobs.

Neutron Features

Neutron added several new features in Juno, including:

  • DVR (Distributed Virtual Routing)
  • L3 HA support
  • IPv6 subnet modes

Horizon provides support for these new features with the Juno release. These features provide much greater flexibility in specifying software defined networks.

An existing feature in Neutron that Horizon now supports is the MAC learning extension.

Glance Features

In Juno, Glance introduced the ability to manage a catalog of metadata definitions where users can register the metadata definitions to be used on various resource types including images, volumes, aggregates, and flavors. Support for viewing and editing the assignment of these metadata tags is included in Horizon.

Cinder Features

In a continued effort to provide fuller API support, several features supported by Cinder are now supported in Horizon in the Juno release. Users can now utilize swift to store volume backups from Horizon as well as restore volumes from these backups.

Other features of the Cinder API not previously supported by Horizon added in Juno include:

  • Enabling resetting the state of a snapshot
  • Enabling resetting the state of a volume
  • Supporting upload-to-image
  • Volume retype
  • QoS (quality of service) support

Trove

Trove supports potentially using numerous different datastores, e.g., mysql, redis, mongodb. Users can now select from the list of datastores supported by the cloud operator when creating their database instances.

Another addition is support for utilizing and restoring from incremental database backups.

To improve support for Neutron based clouds, when creating a database instance, the user can now specify the NIC for the database instance on creation allowing direct access to the instance by the user.

Nova

The new nova instance actions panel provides a list of all actions taken on all instances in the current project allowing users to view resulting errors or actions taken by other users on those instances.

Administrators now have the ability to evacuate hosts off hypervisors which can aid in system maintenance by providing a mechanism to migrate all instances to other hosts.

Improved Plugin Support

The plugin system in Horizon continued to improve in the Juno release. Some of those improvements:

  • Support for adding plugin specific AngularJS modules
  • Support for adding static files, e.g., CSS, JS, images
  • Ability to add exceptions
  • Fixing ordering issues
  • Numerous other bug fixes

Enhanced RBAC support

In an ongoing effort to support richer role based access control (RBAC) in Horizon, the views for several more services were enhanced with RBAC checks to determine user access to actions. The newly supported services are compute, network and orchestration. These changes allow operators to implement finer grained access control than just "member" and "admin".

The identity panels (domains, projects, users, roles, groups) have also been converted to support RBAC at the view level. The identity panels have been moved from the admin dashboard into their own 'Identity' dashboard and accessibility is determined by policies alone. This is the first step toward consolidating the near duplicate content of the project and admin dashboards into single views supporting a wide range of roles.

UX Changes

In Juno, Horizon transitioned to utilizing Bootstrap v3. Horizon had been pinned to an older version of Bootstrap for several releases. This change now allows Horizon to pick up numerous bug fixes and overall improvements in the Bootstrap framework. The look and feel remains mainly consistent with the Havana release.

JavaScript Libraries Extracted

As part of the Horizon team's ongoing efforts to split the repository into more logical pieces, all the 3rd party JavaScript libraries that Horizon depends on have been removed from the Horizon code base and python xstatic packages have been utilized instead. The xstatic format allows for easy consumption by the Django framework Horizon is built on. Now JavaScript libraries are utilized like any other python dependency in Horizon.

Conversion from LESS to SCSS

The supported stylesheets in Horizon have been converted to utilize SCSS rather than LESS. The change was necessary due to a prevalent lack of support for LESS compilers in python. This change also allowed us to upgrade to Bootstrap 3, as parts of the Bootstrap 3 LESS stylesheets were not supported by existing python based LESS compilers.

Known Issues

Rendering issues in extensions

The conversion to utilizing Bootstrap v3 can cause content extensions written on top of Horizon to have rendering issues. Most of these are fixed by a simple CSS class name substitutions. These issues are primarily seen with buttons and panel content widths.

Online Compression

With the move to SCSS, there may be issues with utilizing online compression in non-DEBUG mode in Horizon. Offline compression continues to work as in previous releases.

Neutron L3 HA

The HA property is updateable in the UI, however, the migration itself fails on the agent.

Upgrade Notes

  • FLAVOR_EXTRA_KEYS setting deprecated. The use of this key has been replaced with direct calls to the nova and glance api as appropriate.

OpenStack Identity (Keystone)

Key New Features

  • Keystone now has experimental support for Keystone-to-Keystone federation, where one instance acts as an Identity Provider, and the other a Service Provider.
  • PKIZ is a new token provider available for users of PKI tokens, which simply adds zlib-based compression to traditional PKI tokens.
  • The hashing algorithm used for PKI tokens has been made configurable (the default is still MD5, but the Keystone team recommends that deployments migrate to SHA256).
  • Identity-driver-configuration-per-domain now supports Internet domain names of arbitrary hierarchical complexity (for example, customer.cloud.example.com).
  • The LDAP identity backend now supports description as an attribute of users.
  • Identity API v3 requests are now validated via JSON Schema.
  • In the case of multiple identity backends, Keystone can now map arbitrary resource IDs to arbitrary backends.
  • keystoneclient.middleware.auth_token has been moved into it's own repository, keystonemiddleware.auth_token.
  • Identity API v3 now supports a discrete call to retrieve a service catalog, GET /v3/auth/catalog.
  • Federated authentication events and local role assignment operations now result in CADF (audit) notifications.
  • Keystone can now associate a given policy blob with one or more endpoints.
  • Keystone now provides JSON Home documents on the root API endpoints in response to Accept: application/json-home headers.
  • Hiding endpoints from client's service catalogs is now more easily manageable via OS-EP-FILTER.
  • The credentials collection API is now filterable per associated user (GET /v3/credentials?user_id={user_id}).
  • New, generic API endpoints are available for retrieving authentication-related data, such as a service catalog, available project scopes, and available domain scopes.
  • Keystone now supports mapping the user enabled attribute to the lock attribute in LDAP (and inverting the corresponding boolean value accordingly).
  • A CA certificate file is now configurable for LDAPS connections.
  • The templated catalog backend now supports generating service catalogs for Identity API v3.
  • Service names were added to the v3 service catalog.
  • Services can now be filtered by name ( GET /v3/services?name={service_name}).

Known Issues

LDAP paged search results don't work with python-ldap 2.4

When using an LDAP backend with paged search results enabled, AttributeErrors will be encountered if python-ldap 2.4 is being used. This is due to a backwards incompatible API change in python-ldap. The issue can be worked around in a few ways:

  • Disabling paging of search results by setting page_size to 0 in the [ldap] section of keystone.conf.
  • Downgrade python-ldap to version 2.3.x.

A fix for this issue has been proposed, which is expected to be made available in a stable update for Juno. For more details see https://bugs.launchpad.net/keystone/+bug/1381768

Upgrade Notes

  • Due to the simpler out-of-the-box experience, the default token provider is now UUID instead of PKI.
  • Database migrations for releases prior to Havana have been dropped, meaning that you must upgrade to the Juno release from either a Havana or Icehouse deployment.
  • A comprehensive list of all updated, deprecated or removed options in Keystone can be found at: http://docs.openstack.org/trunk/config-reference/content/keystone-conf-changes-master.html
    • All token_api methods are now deprecated.
    • LDAP configuration options that previously contained the deprecated tenant terminology have been superseded by options using the term project.
    • Proxy methods from the identity backend to the assignment backend (created to provide backwards compatibility as a result of the split of the Assignment backend from the Identity backend), have been removed. This should only affect custom, out-of-tree API extensions.
    • Loading authentication plugins solely by class name in keystone.conf is now deprecated in favor of loading them by custom-method-name = custom_package.CustomClass pairs, and then defining the sequence of authentication methods as a list (methods = custom-method-name, password).
    • In-tree token drivers (keystone.token.backends) have been moved to keystone.token.persistence.backends. Proxy objects exist to maintain compatibility. If a non-default value is used, it is recommended the value of the driver option in the [token] section of keystone.conf is updated to use the new location.
  • All KVS backends besides the token driver have been formally deprecated.
  • LDAP/AD configuration: All configuration options containing the term "tenant" have been deprecated in favor of similarly named configuration options using the term "project" (for example, tenant_id_attribute has been replaced by project_id_attribute).

OpenStack Network Service (Neutron)

Key New Features

  • DB migration refactor and new timeline
  • Distributed Virtual Router Support (DVR)
  • Full IPV6 support for tenant networks
  • High Availability for the L3 Agent
  • ipset support for security groups in place of iptables (this option is configurable)
  • L3 agent performance improvements
  • Migration to oslo.messaging library for RPC communication.
  • Security group rules for devices RPC call refactoring (a huge performance improvement)
  • New Plugins supported in Juno include the following:
    • A10 Networks LBaaS driver for the LBaaS V1 API
    • Arista L3 routing plugin
    • Big Switch L3 routing plugin
    • Brocade L3 routing plugin
    • Cisco APIC ML2 Driver (including a L3 routing plugin).
    • Cisco CSR L3 routing plugin
    • Freescale SDN ML2 Mechanism Driver
    • Nuage Networks ML2 Mechanism Driver
    • SR-IOV capable NIC ML2 Mechanism Driver

Known Issues

  • This is the first release for DVR and HA L3. The Neutron team desires to designate these features as production ready in Kilo and requests that deployers test on non-critical workloads and report any issues.
  • FWaaS is still labeled as experimental, as it does not allow you to have more than one FW per tenant.

Upgrade Notes

  • A list of all updated, deprecated or removed options in neutron can be found at: http://docs.openstack.org/trunk/config-reference/content/neutron-conf-changes-master.html
  • Attribute level policies dependent on resources are not enforced anymore. Meaning that some older policies from Icehouse are not needed. (e.g. "get_port:binding:vnic_type": "rule:admin_or_owner").
  • The following plugins are deprecated in Juno:
    • Cisco Nexus Sub-Plugin (The Nexus 1000V Sub-Plugin is still retained and supported in Juno).
    • Mellanox Plugin
    • Ryu Plugin
  • XML support in the API is deprecated. Users and deployers should migrate to JSON for API interactions as soon as possible since the XML support will be removed in the Kilo (2015.1) release.

OpenStack Block Storage (Cinder)

Key New Features

  • Support for Volume Replication.
  • Support for Consistency Groups and Snapshots of Consistency Groups.
  • Support for Volume Pools.
  • Completion of i18n-enablement
  • Honor Glance protected properties in Image Upload
  • Enable ability to restrict bandwidth usage on volume-copy operations
  • Add Volume Num Weigher Scheduling

New Drivers/Plugins

  • Datera
  • Fujitsu ETERNUS
  • Fusion IO
  • Hitachi HBSD
  • Huawei
  • Nimble
  • Prophetstor
  • Pure
  • XtremIO
  • Oracle ZFS

Known Issues

None yet

Upgrade Notes

OpenStack Telemetry (Ceilometer)

Key New Features

  • Support for partitioning metric collection load across horizontally scaled-out central agents
  • New method of partitioning alarm evaluation load using tooz coordination, as opposed to a hand-crafted protocol
  • Much improved SQLAlchemy storage performance & scalability, so that MySQL or PostgreSQL can be used as the metering store for PoCs or small deployments
  • Support for hardware-oriented monitoring of IPMI sensors via notifications from either Ironic or a new standalone agent
  • More flexible & efficient SNMP monitoring:
    • batching queries for multiple SNMP metrics into a single call to each daemon
    • dynamic discovery of nodes deployed by TripleO for SNMP polling
    • the ability to more easily extend the range of SNMP metrics that ceilometer gathers
    • the ability to derive new metrics from arithmetic transformations applied to multiple primary metrics
  • Option to split off the alarms persistence into a separate database
  • Option to use notifications instead of RPC for metering messages
  • Metering of Neutron networking services: LBaaS, FWaaS & VPNaaS
  • New XenAPI compute inspector
  • Support for persisting events via the MongoDB & Hbase storage drivers (previously limited to SQLAlchemy)
  • Support for per-device metering of instance disks
  • Use of ceilometer as a collector for os-profiler data
  • New Telemetry section of the Cloud Administrator Guide

Known Issues

  • 1381600 The new ceilometer-agent-ipmi fails to emit any samples when it encounters unparseable data from ipmitool.

Upgrade Notes

OpenStack Orchestration (Heat)

Key New Features

  • Recovery from failures during stack updates
  • API to cancel and roll back an in-progress stack update
  • Implementation of new resource types:
    • OS::Glance::Image
    • OS::Heat::SwiftSignal
      • Provides the option to store Wait Condition (and Software Deployment) data in Swift
    • OS::Heat::StructuredDeployments
      • Groups code for multiple lifecycle events into a single deployment resource
    • OS::Heat::SoftwareDeployments
      • Provides a way of avoiding circular dependencies when deploying an interdependent cluster of servers
    • OS::Heat::SoftwareComponent
    • OS::Nova::ServerGroup
    • OS::Sahara::NodeGroupTemplate
    • OS::Sahara::ClusterTemplate
  • Remember the previously-supplied parameters when updating a stack
  • Improved scalability
  • Improved visibility into trees of nested stacks

Known Issues

None yet

Upgrade Notes

OpenStack Database service (Trove)

Key New Features

  • Support for Asynchronous Replication (master-slave replicas) between provisioned mysql instances.
  • Introduction of a new Clustering API with initial support for MongoDB clusters.
  • Support for deploying Trove on an OpenStack solution that is using Neutron for networking. Prior to this, only nova-network was supported.
  • Support for provisioning PostgreSQL datastore instances.
  • Backup and Restore support for Couchbase.
  • Support to optionally restrict the Cinder backend used for Trove volumes.
  • Support for defining custom datastore configuration parameters in the Trove database (using mgmt API).
  • The ability to list all datastore types and versions in a single call

Other Incremental Improvements

  • Logging audit to improve log levels throughout the trove components.
  • The extensions loading mechanism was improved by adding support for stevedore.
  • The ability to support volumes for data is now on a per datastore bases.
  • Created and updated timestamps and instance count were added to configuration groups list and details calls.

Known Issues

  • 1333852: Trove does not support flavor UUIDs -- the Trove flavors API requires flavors with a numerical ID in order to be consistent with the API response for icehouse Trove.

Upgrade Notes

None yet

OpenStack Data Processing (Sahara)

New Key Features

  • Data processing UI was fully merged into OpenStack Dashboard (horizon).
  • Support of CDH 5.x was added.
  • Support of Apache Spark was added. Supported versions are 0.9.1 and 1.0.0. Elastic data processing (EDP) engine was refactored a lot to support non-Oozie workflow engines.
  • Support of Apache Hadoop 2.4.1 was added in addition to existing 1.2.1 and 2.3.0. Version 2.3.0 is deprecated in Juno.
  • Support of multi region deployments.
  • Hadoop Swift authentication using keystone trust mechanism. Now Hadoop can access data in Swift without storing credentials in config files.
  • Ceilometer integration was added. Now Sahara notifies Ceilometer about all cluster state changes.
  • Cluster provisioning error handling was improved. If something goes wrong during scaling, cluster will rollback to original state.
  • Added ability to specify security groups for a node group. Also Sahara could automatically create security group with only required ports open.
  • Implemented distributed mode for Sahara: sahara-all process is decoupled into sahara-api and sahara-engine. You can run several instances of sahara-api and sahara-engine on different hosts. Note that the feature implementation is considered to be in alpha-state.

Known Issues

  • Bug 1271349: Sahara requires root privileges to access VMs via namespaces.

Upgrade Notes

Main binary renamed to sahara-all

Please, note that you should use `sahara-all` instead of `sahara-api` to start the All-In-One Sahara.

sahara.conf upgrade

We've migrated from custom auth_token middleware config options to the common config options. To update your config file you should replace the following old config opts with the new ones.

  • "os_auth_protocol", "os_auth_host", "os_auth_port" -> "[keystone_authtoken]/auth_uri" and "[keystone_authtoken]/identity_uri"
  • "os_admin_username" -> "[keystone_authtoken]/admin_user"
  • "os_admin_password" -> "[keystone_authtoken]/admin_password"
  • "os_admin_tenant_name" -> "[keystone_authtoken]/admin_tenant_name"

We've replaced oslo code from sahara.openstack.common.db by usage of oslo.db library.

Also sqlite database is not supported anymore. Please use MySQL or PostgreSQL db backends for Sahara. Sqlite support was dropped because it doesn't support (and not going to support, see http://www.sqlite.org/omitted.html) ALTER COLUMN and DROP COLUMN commands required for DB migrations between versions.

You can find more info about config file options in Sahara repository in file "etc/sahara/sahara.conf.sample".

Sahara Dashboard was merged into OpenStack Dashboard

The Sahara Dashboard is not available in Juno release. Instead it's functionality is provided by OpenStack Dashboard out of the box. The Sahara UI is available in OpenStack Dashboard in "Project" -> "Data Processing" tab.

Note that you have to properly register Sahara in Keystone in order for Sahara UI in the Dashboard to work.

VM user name changed for HEAT infrastructure engine

We've updated HEAT infrastructure engine ("infrastructure_engine=heat") to use the same rules for instance user name as in direct engine. Before the change user name for VMs created by Sahara using HEAT engine was always 'ec2-user'. Now user name is taken from the image registry as it is described in the documentation.

Note, this change breaks Sahara backward compatibility for clusters created using HEAT infrastructure engine before the change. Clusters will continue to operate, but it is not recommended to perform scale operation over them.

Anti affinity implementation changed

Starting with Juno release anti affinity feature is implemented using server groups. There should not be much difference in Sahara behavior from user perspective, but there are internal changes:

  • Server group object will be created if anti affinity feature is enabled.
  • New implementation doesn't allow several affected instances on the same host even if they don't have common processes. So, if anti affinity enabled for 'datanode' and 'tasktracker' processes, previous implementation allowed to have instance with 'datanode' process and other instance with 'tasktracker' process on one host. New implementation guarantees that instances will be on different hosts.

Note, new implementation will be applied for new clusters only. Old implementation will be applied if user scales cluster created in Icehouse.


OpenStack Documentation

  • This release, the OpenStack Foundation funded a five-day book sprint to write the new OpenStack Architecture Design Guide. It offers architectures for general purpose, compute-focused, storage-focused, network-focused, multi-site, hybrid, massively scalable, and specialized clouds.
  • The Install Guides have had a lot of clean up and standardization: uses common message queue (RabbitMQ), replaces openstack-config (crudini) commands with config file editing for improved learning opportunities and consistency, references a generic SQL database so that MariaDB or MySQL can be substituted, and replaces auth_port and auth_protocol with identity_uri, and auth_host with auth_uri throughout. The Install Guides are thoroughly tested on each distribution and continuously published until the official release packages are available to everyone.
  • The High Availability Guide now has a separate review team and has moved into a separate repository.
  • The Security Guide now has a specialized review team and has moved into a separate repository.
  • The long-form API reference documents have been re-purposed to focus on the API Complete Reference.
  • The User Guide now contains Database Service for OpenStack information.
  • The Command-Line Reference has been updated with new client releases and now contains additional chapters for the common OpenStack client, the trove-manage client, and the Data processing client (sahara).
  • The OpenStack Cloud Administrator Guide now contains information about Telemetry (ceilometer).