LiveMigrateNonActiveStates

Extending live-migration to instances in non-ACTIVE states

Why restrict live-migration feature to only active instances?

I believe that there are two parts to an answer to this question: technical and usability.

The rest of the text is based on a libvirt/kvm installation with block-migration. It is possible that Xen and other hypervisors may have different issues.

Technical issues

1. It's not possible to migrate when there's no running hypervisor process running. For example, in a libvirt/kvm setup, if an instance is shutoff, there's no kvm process and a migrate job cannot be sent.
2. (TODO:)

Usability issues

1. The term live migration suggests a running instance and users may expect an error returned as per the API. This also covers backward compatibilty issues.
2. (TODO:)

A possible solution

1. Bring all non-active instances to 'PAUSED' state. This will require shutoff instances to be started as PAUSED.
2. Migrate the instance and leave it at PAUSED state.
3. In post live-migration at destination, take the VM to the required state by using the vm_state as a hint.
4. (TODO:)

The usability issue may be handled in one of the following ways.

1. Add an extra option to differentiate from the current live-migration. (Eg: --all-states)
2. Add a different API entry point
3. Leave the API unchanged
4. (TODO:)

We propose that we don't change the API at all (option 3). The "--all-states" forces all states to be supported at once. Also since these are non-active instances, it might have less impact on user experience. The onus is on an operator to choose an appropriate instance to be migrated. If a non-active instance is chosen we might as well try our best to migrate it.

Proposed states to be supported

1. PAUSED
2. STOPPED
3. SUSPENDED
4. RESCUED