Difference between revisions of "Trove-Guest-Agent-Upgrades"

Revision as of 21:31, 3 March 2014

TODO: draft up the design here DN 2014-02-27

Introduction

This article describes the design for the Guest Agent upgrades in Trove. Currently Guest Agent upgrades are implemented through external deployment tools that push new code to each guest instance. Usually the same deployment tools for upgrading the control plane handle guest agent upgrades. This can create a bottle neck on the deployment infrastructure.

Goals

1. Version the RPC API and tie it to the API version (see nova for examples)
   • This is to help prevent non-backward compatibility between the Trove API and the guest
2. Implement a notification based upgrade path for guest agent
3. Allow for different upgrade strategies (swift, jenkins, local disk, rysnc, etc)
4. Avoid upgrading during times when guest agents are doing other work (i.e. backups, resize, restart)
5. Reduce overall downtime during upgrade cycles

Description

Configuration

New properties will be added to the trove configs to allow:

• Enabling/Disabling Guest Agent Upgrades

guest_automatic_updates: False

• Specifying an upgrade strategy

guest_upgrade_strategy: swift

Affected Trove Components

• python-troveclient (optional)
• trove admin API
• guest agent

Schema Changes

Workflow

• An external process will create a upgrade package or artifact

• An Admin user will notify a Guest Agent through the Trove Management API
• The Guest Agent will process the RPC message created by the API call and handle the upgrade accordingly

• The Guest Agent will download the package from the location specified in the RPC message

Trove Management REST API

Create a notification request to upgrade an trove guest agent

Relative URL:  /v1.0/{admin_tenant_id}/mgmt/upgrades

HTTP Method: Post

HTTP Headers:
  Accept: application/json
  Content-Type: application/json 
  User-Agent: python-troveclient 
  X-Auth-Project-Id: tenant_name 
  X-Auth-Token: HPAuth10_xxxx 

HTTP Post Body
{
    "db_instance": '<UUID>'
    "trove_guest_version": "v1.0.1",
    "strategy": "swift",
    "location": "http://swift/tenant/container/trove-guestagent-v1.0.1.tar.gz"
}

Trove RPC API

unpacked context

{
   ...
    "is_admin": True,
    "tenant":"<SANITIZED>",
    "method": "upgrade",
    "db_instance": '<UUID>'
    "trove_guest_version": "v1.0.1",
    "strategy": "swift",
    "location": "http://swift/tenant/container/trove-guestagent-v1.0.1.tar.gz"
}

Versioning and Package Validation

• The Guest Agent will be responsible for validating the package before upgrading

Scenarios

1. What happens when the Guest Agent is in a non-upgradeable state? (backup/restore, resize, restart, error)
   • The message should remain in the queue until the next time the Guest Agent checks and the state is in 'Running'
2. What happens when an upgrade fails, and how does that feedback to Trove?
   • Record it as a FAIL in the Trove Database, Admin will have to query.
3. Can we rollback or install a previous version