Jump to: navigation, search

Difference between revisions of "XenServer/XenAndXenServer"

(first dump from manuals)
Line 6: Line 6:
  
 
To get started, take a look at: [[XenServer/GettingStarted|Getting started with XenServer and [[OpenStack]]]]
 
To get started, take a look at: [[XenServer/GettingStarted|Getting started with XenServer and [[OpenStack]]]]
 +
 +
= Introduction to using Xen, XCP and [[XenServer]] with [[OpenStack]] =
 +
In this section we will describe Xen, XCP, and [[XenServer]], the
 +
differences between them, and how to use them with [[OpenStack]].  Xen's
 +
architecture is different from KVM's in important ways, and we discuss those
 +
differences and when each might make sense in your [[OpenStack]] cloud.
 +
 +
== Basic terminology ==
 +
 +
=== Xen ===
 +
Xen is a hypervisor.  It provides the
 +
fundamental isolation between virtual machines.  Xen is open source (GPLv2)
 +
and is managed by Xen.org, an cross-industry organization.
 +
Xen is a component of many different products and projects.  The
 +
hypervisor itself is very similar across all these projects, but the way that
 +
it is managed can be different, which can cause confusion if you're not clear
 +
which toolstack you are using.  Make sure you know what toolstack you want
 +
before you get started.</para>
 +
 +
=== XCP ===
 +
XCP is an open source (GPLv2) toolstack
 +
for Xen.  It is designed specifically as platform for enterprise and cloud
 +
computing, and is well integrated with [[OpenStack]].</para>
 +
 +
=== Citrix [[XenServer]] ===
 +
 +
Citrix [[XenServer]] is a commercial
 +
product.  It is based on XCP, and exposes the same toolstack and managment
 +
API.  As an analogy, think of [[XenServer]] being based on XCP in the way that Red
 +
Hat Enterprise Linux is based on Fedora.  [[XenServer]] has a free version (which
 +
is very similar to XCP) and paid-for versions with additional features
 +
enabled.
 +
 +
=== xapi / XAPI / XenAPI ===
 +
 +
Both [[XenServer]] and XCP include Xen, Linux, and the primary control
 +
daemon known as xapi.
 +
 +
The API shared between XCP and [[XenServer]] is called ""XenAPI"".  [[OpenStack]] usually refers to XenAPI, to
 +
indicate that the integration works equally well on XCP and [[XenServer]].
 +
Sometimes, a careless person will refer to [[XenServer]] specifically, but you can
 +
be reasonably confident that anything that works on [[XenServer]] will also work
 +
on the latest version of XCP.
 +
 +
== Privileged and unprivileged domains ==
 +
 +
A Xen host will run a number of virtual machines, VMs, or domains (the
 +
terms are synonymous on Xen).  One of these is in charge of running the rest
 +
of the system, and is known as "domain 0", or "dom0".  It is the first domain
 +
to boot after Xen, and owns the storage and networking hardware, the device
 +
drivers, and the primary control software.  Any other VM is unprivileged, and
 +
are known as a "domU" or "guest".  All customer VMs are unprivileged of
 +
course, but you should note that on Xen the [[OpenStack]] control software
 +
(nova-compute) also runs in a domU.  This gives a level of security isolation
 +
between the privileged system software and the [[OpenStack]] sofware (much of
 +
which is customer-facing).  This architecture is described in more detail
 +
later.
 +
 +
There is an ongoing project to split domain 0 into multiple privileged
 +
domains known as <emphasis role="bold">driver domains</emphasis> and <emphasis
 +
role="bold">stub domains</emphasis>.  This would give even better separation
 +
between critical components.  This is an ongoing research project and you
 +
shouldn't worry about it right now.  The current architecture just has three
 +
levels of separation: dom0, the [[OpenStack]] domU, and the completely
 +
unprivileged customer VMs.
 +
 +
== Paravirtualized versus hardware virtualized domains ==
 +
 +
A Xen virtual machine can be <emphasis role="bold">paravirtualized
 +
(PV)</emphasis> or <emphasis role="bold">hardware virtualized
 +
(HVM)</emphasis>.  This refers to the interaction between Xen, domain 0, and
 +
the guest VM's kernel.  PV guests are aware of the fact that they are
 +
virtualized and will co-operate with Xen and domain 0; this gives them better
 +
performance characteristics.  HVM guests are not aware of their environment,
 +
and the hardware has to pretend that they are running on an unvirtualized
 +
machine.  HVM guests have the advantage that there is no need to modify the
 +
guest operating system, which is essential when running Windows.
 +
 +
In [[OpenStack]], customer VMs may run in either PV or HVM mode.  However,
 +
the [[OpenStack]] domU (that's the one running nova-compute) <emphasis
 +
role="bold">must</emphasis> be running in PV mode.
 +
 +
= Deployment architecture =
 +
 +
When you deploy [[OpenStack]] on XCP or [[XenServer]] you will get something
 +
similar to this:
 +
 +
{{[[XenServer]]-dom0-domU.png}}
 +
 +
Key things to note:
 +
 +
<itemizedlist>
 +
    <listitem><para>The hypervisor: Xen</para></listitem>
 +
 +
    <listitem><para>Domain 0: runs xapi and some small pieces from [[OpenStack]]
 +
    (some xapi plugins and network isolation rules).  The majority of this is
 +
    provided by [[XenServer]] or XCP (or yourself using Kronos).</para></listitem>
 +
 +
    <listitem><para>[[OpenStack]] domU: The nova-compute code runs in a
 +
    paravirtualized virtual machine, running on the host under management.
 +
    Each host runs a local instance of nova-compute.  It will often also be
 +
    running nova-network (depending on your network mode).  In this case,
 +
    nova-network is managing the addresses given to the tenant VMs through
 +
    DHCP.</para></listitem>
 +
 +
    <listitem><para>Nova uses the XenAPI Python library to talk to xapi, and
 +
    it uses the Host Internal Management Network to reach from the domU to
 +
    dom0 without leaving the host.</para></listitem>
 +
</itemizedlist>
 +
</para>
 +
 +
<para>
 +
Some notes on the networking:
 +
 +
<itemizedlist>
 +
    <listitem><para>The above diagram assumes FlatDHCP networking (the
 +
    [[DevStack]] default).</para></listitem>
 +
 +
    <listitem><para>There are three main [[OpenStack]] networks: Management
 +
    traffic (RabbitMQ, MySQL, etc), Tenant network traffic (controlled by
 +
    nova-network) and Public traffic (floating IPs, public API end
 +
    points).</para></listitem>
 +
 +
    <listitem><para>Each network that leaves the host has been put through
 +
    a separate physical network interface.  This is the simplest model, but
 +
    it's not the only one possible.  You may choose to isolate this traffic
 +
    using VLANs instead, for example.</para></listitem>
 +
</itemizedlist>
 +
</para>
 +
 +
</section>
 +
 +
<section xml:id="pools">
 +
<title>[[XenServer]] pools</title>
 +
 +
<para>Before [[OpenStack]] 2012.1 ("Essex"), all [[XenServer]] machines used with
 +
[[OpenStack]] are standalone machines, usually only using local storage.</para>
 +
 +
<para>However in 2012.1 and later, the host-aggregates feature allows you to
 +
create pools of [[XenServer]] hosts (configuring shared storage is still an out of
 +
band activity). This move will enable live migration when using shared
 +
storage.</para>
 +
 +
</section>
 +
 +
<section xml:id="xen-and-libvirt">
 +
<title>Xen and libvirt</title>
 +
 +
<para>It may possible to manage Xen using libvirt.  This would be necessary
 +
for any Xen-based system that isn't using the XCP toolstack, such as SUSE
 +
Linux or Oracle Linux.  Unfortunately, this is not well tested or supported
 +
today, and using the XCP or [[XenServer]] toolstacks with the [[OpenStack]] XenAPI
 +
backend is the only recommended method.</para>
 +
 +
</section>
 +
 +
<section xml:id="further-reading">
 +
<title>Further reading</title>
 +
 +
<para>What is Xen? by Xen.org: <link
 +
xlink:href="http://xen.org/files/Marketing/WhatisXen.pdf">
 +
http://xen.org/files/Marketing/WhatisXen.pdf</link>.</para>
 +
 +
<para>Xen Hypervisor project: <link
 +
xlink:href="http://xen.org/products/xenhyp.html">
 +
http://xen.org/products/xenhyp.html</link>.</para>
 +
 +
<para>XCP project: <link xlink:href="http://xen.org/products/cloudxen.html">
 +
http://xen.org/products/cloudxen.html</link>.</para>
  
 
= [[XenServer]] Deployments =
 
= [[XenServer]] Deployments =

Revision as of 14:16, 21 March 2012

<<TableOfContents()>>

Getting Started

To get started, take a look at: [[XenServer/GettingStarted|Getting started with XenServer and OpenStack]]

Introduction to using Xen, XCP and XenServer with OpenStack

In this section we will describe Xen, XCP, and XenServer, the differences between them, and how to use them with OpenStack. Xen's architecture is different from KVM's in important ways, and we discuss those differences and when each might make sense in your OpenStack cloud.

Basic terminology

Xen

Xen is a hypervisor. It provides the fundamental isolation between virtual machines. Xen is open source (GPLv2) and is managed by Xen.org, an cross-industry organization. Xen is a component of many different products and projects. The hypervisor itself is very similar across all these projects, but the way that it is managed can be different, which can cause confusion if you're not clear which toolstack you are using. Make sure you know what toolstack you want before you get started.</para>

XCP

XCP is an open source (GPLv2) toolstack for Xen. It is designed specifically as platform for enterprise and cloud computing, and is well integrated with OpenStack.</para>

Citrix XenServer

Citrix XenServer is a commercial product. It is based on XCP, and exposes the same toolstack and managment API. As an analogy, think of XenServer being based on XCP in the way that Red Hat Enterprise Linux is based on Fedora. XenServer has a free version (which is very similar to XCP) and paid-for versions with additional features enabled.

xapi / XAPI / XenAPI

Both XenServer and XCP include Xen, Linux, and the primary control daemon known as xapi.

The API shared between XCP and XenServer is called ""XenAPI"". OpenStack usually refers to XenAPI, to indicate that the integration works equally well on XCP and XenServer. Sometimes, a careless person will refer to XenServer specifically, but you can be reasonably confident that anything that works on XenServer will also work on the latest version of XCP.

Privileged and unprivileged domains

A Xen host will run a number of virtual machines, VMs, or domains (the terms are synonymous on Xen). One of these is in charge of running the rest of the system, and is known as "domain 0", or "dom0". It is the first domain to boot after Xen, and owns the storage and networking hardware, the device drivers, and the primary control software. Any other VM is unprivileged, and are known as a "domU" or "guest". All customer VMs are unprivileged of course, but you should note that on Xen the OpenStack control software (nova-compute) also runs in a domU. This gives a level of security isolation between the privileged system software and the OpenStack sofware (much of which is customer-facing). This architecture is described in more detail later.

There is an ongoing project to split domain 0 into multiple privileged domains known as <emphasis role="bold">driver domains</emphasis> and <emphasis role="bold">stub domains</emphasis>. This would give even better separation between critical components. This is an ongoing research project and you shouldn't worry about it right now. The current architecture just has three levels of separation: dom0, the OpenStack domU, and the completely unprivileged customer VMs.

Paravirtualized versus hardware virtualized domains

A Xen virtual machine can be <emphasis role="bold">paravirtualized (PV)</emphasis> or <emphasis role="bold">hardware virtualized (HVM)</emphasis>. This refers to the interaction between Xen, domain 0, and the guest VM's kernel. PV guests are aware of the fact that they are virtualized and will co-operate with Xen and domain 0; this gives them better performance characteristics. HVM guests are not aware of their environment, and the hardware has to pretend that they are running on an unvirtualized machine. HVM guests have the advantage that there is no need to modify the guest operating system, which is essential when running Windows.

In OpenStack, customer VMs may run in either PV or HVM mode. However, the OpenStack domU (that's the one running nova-compute) <emphasis role="bold">must</emphasis> be running in PV mode.

Deployment architecture

When you deploy OpenStack on XCP or XenServer you will get something similar to this:

{{XenServer-dom0-domU.png}}

Key things to note:

<itemizedlist>

   <listitem><para>The hypervisor: Xen</para></listitem>
   <listitem><para>Domain 0: runs xapi and some small pieces from OpenStack
   (some xapi plugins and network isolation rules).  The majority of this is
   provided by XenServer or XCP (or yourself using Kronos).</para></listitem>
   <listitem><para>OpenStack domU: The nova-compute code runs in a
   paravirtualized virtual machine, running on the host under management.
   Each host runs a local instance of nova-compute.  It will often also be
   running nova-network (depending on your network mode).  In this case,
   nova-network is managing the addresses given to the tenant VMs through
   DHCP.</para></listitem>
   <listitem><para>Nova uses the XenAPI Python library to talk to xapi, and
   it uses the Host Internal Management Network to reach from the domU to
   dom0 without leaving the host.</para></listitem>

</itemizedlist> </para>

<para> Some notes on the networking:

<itemizedlist>

   <listitem><para>The above diagram assumes FlatDHCP networking (the
   DevStack default).</para></listitem>
   <listitem><para>There are three main OpenStack networks: Management
   traffic (RabbitMQ, MySQL, etc), Tenant network traffic (controlled by
   nova-network) and Public traffic (floating IPs, public API end
   points).</para></listitem>
   <listitem><para>Each network that leaves the host has been put through
   a separate physical network interface.  This is the simplest model, but
   it's not the only one possible.  You may choose to isolate this traffic
   using VLANs instead, for example.</para></listitem>

</itemizedlist> </para>

</section>

<section xml:id="pools"> <title>XenServer pools</title>

<para>Before OpenStack 2012.1 ("Essex"), all XenServer machines used with OpenStack are standalone machines, usually only using local storage.</para>

<para>However in 2012.1 and later, the host-aggregates feature allows you to create pools of XenServer hosts (configuring shared storage is still an out of band activity). This move will enable live migration when using shared storage.</para>

</section>

<section xml:id="xen-and-libvirt"> <title>Xen and libvirt</title>

<para>It may possible to manage Xen using libvirt. This would be necessary for any Xen-based system that isn't using the XCP toolstack, such as SUSE Linux or Oracle Linux. Unfortunately, this is not well tested or supported today, and using the XCP or XenServer toolstacks with the OpenStack XenAPI backend is the only recommended method.</para>

</section>

<section xml:id="further-reading"> <title>Further reading</title>

<para>What is Xen? by Xen.org: <link xlink:href="http://xen.org/files/Marketing/WhatisXen.pdf"> http://xen.org/files/Marketing/WhatisXen.pdf</link>.</para>

<para>Xen Hypervisor project: <link xlink:href="http://xen.org/products/xenhyp.html"> http://xen.org/products/xenhyp.html</link>.</para>

<para>XCP project: <link xlink:href="http://xen.org/products/cloudxen.html"> http://xen.org/products/cloudxen.html</link>.</para>

XenServer Deployments

When deploy OpenStack on XenServer you will get something similar to this:

File:XenServer$$XenXCPAndXenServer$DevStackDiagram.png

Key things to note:

  • The hypervisor: Xen
  • Domain0 control VM: runs xapi/XenAPI and is provided by XenServer or XCP (or yourself using Kronos)
  • The nova code runs in a Para-virtulized virtual machine, running on the hyper visor it is wanting to manage
  • Nova uses xapi python bindings to talk to the hypervisor, and it uses a host local network, called the Guest Installer Network, that has a DHCP server

Some notes on the networking:

  • The above diagram assume FlatDHCP networking (the DevStack default)
  • There are three main OpenStack networks: Management traffic (Rabbit, MySQL, etc), Guest network traffic (controlled by nova-network) and Public traffic (floating ips, public api end points)
  • Above we show how you can use VLANs to reduce the required number of network cards on the server, in this case running the instance traffic on a VLAN when exiting the XenServer machine.

XenServer pools

Before 2012.1, all XenServer machines used with OpenStack are deployments are standalone machines, usually only using local storage.

However in 2012.1 and later, the host-aggregates feature allows you to create pools of XenServer hosts (configuring shared storage is still an out of band activity). This move will enable live migration when using shared storage.

Choosing between Xen, XCP and XenServer with OpenStack

Xen or XenServer power some of the worlds largest clouds, including Amazon (Xen) and Rackspace (XenServer).

There are several ways to use the Xen hypervisor with OpenStack:

  • Use Xen directly through libvirt
  • Use XenServer or Xen Cloud Platform (XCP) to manage Xen using xapi/XenAPI
  • Convert your Debian/Ubuntu or RedHat/Fedora/CentOS box into an XCP Domain0 using Project Kronos, then use xapi as above

Please note:

Xen and libvirt

It may possible to talk directly to Xen using libvirt.

However, it is easiest to get started using XenServer or XCP, because this approach is better tested and documented.