Neutron/nsx-integrated-services

= Scope =

This blueprint is about making the Nicira plugin capable of handling dhcp and metadata proxy services provided directly by the NSX platform. Once the feature is complete, Neutron deployments using NSX no longer require the installation and management of one or more dhcp and metadata proxy agents. Dhcp and metadata proxy services are going to provided by Logical Services Nodes, new components introduced by NSX 4.x. Logical Services Node is introduced to represent a logical entity that provides one or more services (i.e. dhcp) in the logical space (i.e. a logical network).

= Use Cases =


 * Brown-field OpenStack deployments that use Neutron/NSX as the networking engine: such existing deployments use dhcp and metadata proxy agents to provide dhcp services to tenant VM's and to proxy requests to the Nova metadata servers in certain conditions. Once the feature is complete the following use cases will be implemented:


 * 1) A tenant creates a network/subnet with dhcp enabled: the IP address set for a port created on the network is going to be handed out by a dhcp service ran and managed by NSX
 * 2) A tenant uplinks a subnet to a router: metadata requests are going to proxied via a metadata proxy service that is ran and managed by NSX
 * 3) An admin can migrate existing networks whose dhcp and metadata proxy services are provided by the neutron-dhcp-agent and neutron-metadata-proxy components to their respective entities managed by NSX.
 * Green-field OpenStack deployments that use Neutron/NSX as the networking engine: such existing deployments will use NSX Logical Services Nodes to provide dhcp and metadata proxy services and both 1 and 2 as detailed above will be implemented.

= Implementation Overview =

The following entities have been introduced:


 * LsnManager: This class manages LSN (Logical Services Node) entities associated with neutron networks. Every Neutron network has a 1-1 mapping with a LSN node that exists in NSX. A LSN node can provide both dhcp and metadata proxy services. These services are provided by connecting the NSX logical switch representing the neutron network to the LSN node via a patch attachment between a NSX logical switch port and LSN port. Every LSN node can have multiple LSN ports but only one LSN port can be attached to a single NSX logical switch. A LSN port configuration holds the dhcp options to be used to provide dhcp services (allocation pool, IP and MAC mappings, etc). Currently only a single logical switch port can be patch-attached to a LSN port. This means that only single subnet can be created per a single neutron network. This limitation may be lifted in the future. Overall this class takes care of the following:
 * 1) CRUD operations of LSN entities in NSX
 * 2) CRUD operations of LSN port entities in NSX
 * 3) Puggling/Unplugging of Logical Services on/from the neutron network
 * 4)  DHCP and Metadata configuration for the LSN port


 * PersistentLsnManager: this class inherits from LsnManager and adds persistence capabilities to LsnManager, so to offload access to NSX. This class also adds the capability to recover from local datastore loss by pulling the LSN state from NSX on demand.
 * MigrationManager: this class manages the migration process for neutron networks created before this feature was avaialable. It validates that a network is eligible for migration and executes it in case it is.
 * DhcpMetadataBuilder: this class handles the deallocation of old dhcp and metadata proxy resources for a network being migrated and allocates the needed NSX LSN resources.
 * nvp.DhcpAgentNotifyAPI: this class handles lifecycle events for subnets and ports when DHCP and Metadata proxy services are provided by NSX.
 * combined.DhcpAgentNotifyAPI: this class handles lifecycle events for subnets and ports for both cases where DHCP and Metadata proxy services are provided by NSX (new networks only) or by the dhcp and metadata proxy agents (existing networks).

= Data Model Changes =

None. Two additions have been made:


 * A Lsn table that stores the mapping between the neutron network and the LSN
 * A LsnPort table that stores the mapping between the LSN and the LSN port, plus some attributes like the subnet cidr, etc;

= Configuration variables =

New configuration variables have been introduced:


 * agent_mode: this option adds to more values:
 * 1) agentless: If 'agentless' is chosen (experimental in Icehouse), the plugin will use NSX logical services for DHCP and metadata proxy. This simplifies the deployment model for Neutron, in that the plugin no longer requires the RPC agents to operate. When 'agentless' is chosen, the config option metadata_mode becomes ineffective. The mode 'agentless' is supported from NVP 4.1 or above.
 * 2) combined: this is used to support existing deployments that want to adopt the agentless mode going forward. With this mode, existing networks keep being served by the existing infrastructure, whereas new networks will be served by the new infrastructure. Migration tools are provided to 'move' one network from one model to another; however, were this not possible both networks served by the agent and networks served by NSX services can coexist in the deployment. With agent_mode set to 'combined', option 'network_auto_schedule' in neutron.conf is ignored, as new networks will no longer be scheduled to existing dhcp agents.
 * default_service_cluster_uuid: UUID for the default service cluster. A service cluster is introduced to represent a group of gateways and it is needed in order to use Logical Services like dhcp and metadata in the logical space. NOTE: If agent_mode is set to 'agentless' this config parameter *MUST BE* set to a valid pre-existent service cluster uuid.
 * sync_on_missing_data: set to True to pull LSN information from NSX in case it is missing from the local data store. This is useful to rebuild the local store in case of server recovery.
 * extra_domain_name_servers: Comma separated list of additional domain name servers.
 * domain_name: Domain to use for building the hostnames.
 * default_lease_time: Default DHCP lease time.
 * metadata_server_address: IP address used by Metadata server.
 * metadata_server_port: TCP Port used by Metadata server.
 * metadata_shared_secret: When proxying metadata requests, Neutron signs the Instance-ID header with a shared secret to prevent spoofing. You may select any string for a secret, but it *MUST* match with the configuration used by the Metadata server.

= API =

The following changes to the API extensions occur:


 * In combined mode, the nicira plugin supports the following extensions:
 * 1) agent: used to support dhcp agents
 * 2)  dhcp_agent_scheduler: used to support scheduling of dhcp requests across multiple dhcp agents
 * 3) lsn: used to support migrations as detailed above.
 * In agentless mode extensions 1 and 2 as detailed above are no longer supported.

= Plugin Interface =

The 'lsn' extension adds the following new operations:


 * create_lsn: to implement the migration plan for a network;
 * get_lsn: to retrieve the lsn resources used by a network;

= CLI Requirements =

A new tool is added called neutron-nsx-manage. This has the following two operations (beyond the neutron ones):


 * net-report : returns the resources being used by the network.
 * net-migrate : check that the network can migrate and performs the migration.

= Horizon Requirements =

None

= Test Cases =

The following test cases are implemented:


 * DhcpAgentNotifyAPITestCase: test coverage for the notifier class.
 * DhcpMetadataBuilderTestCase: test coverage for the builder class.
 * MigrationManagerTestCase: test coverage for the migration manager class.
 * DhcpTestCase: test coverage for operations pertaining dhcp configuration.
 * MetadataTestCase: test coverage for operations pertaining metadata configuration.
 * LsnManagerTestCase: test coverage for the LSNManager class.
 * PersistentLsnManagerTestCase: test coverage for the PersistentLSNManager class.
 * lib.LSNTestCase: test coverage for lsn operations made against NSX.
 * db.LSNTestCase: test coverage for lsn operations made against the db.