Jump to: navigation, search

Difference between revisions of "TricircleHowToReadCode"

(Step 8: Boot virtual machines)
(Experience the multi-region Tricircle networking first)
Line 4: Line 4:
 
Follow the installation guide: https://github.com/openstack/tricircle/blob/master/doc/source/multi-pod-installation-devstack.rst
 
Follow the installation guide: https://github.com/openstack/tricircle/blob/master/doc/source/multi-pod-installation-devstack.rst
  
Learn the source code will be used in the "How to play":
+
The source code which is involved in the installation guide is described as follows:
  
 
=== Step 1 ~ Step 4: no code of Tricircle will be involved ===
 
=== Step 1 ~ Step 4: no code of Tricircle will be involved ===

Revision as of 08:44, 20 February 2017

(Work in progress)Learn the source code of Tricircle step by step

Experience the multi-region Tricircle networking first

Follow the installation guide: https://github.com/openstack/tricircle/blob/master/doc/source/multi-pod-installation-devstack.rst

The source code which is involved in the installation guide is described as follows:

Step 1 ~ Step 4: no code of Tricircle will be involved

Step 5: Create pod instances

Command issued:

 curl -X POST http://127.0.0.1:19999/v1.0/pods -H "Content-Type: application/json" \
     -H "X-Auth-Token: $token" -d '{"pod": {"region_name":  "CentralRegion"}}'
 curl -X POST http://127.0.0.1:19999/v1.0/pods -H "Content-Type: application/json" \
     -H "X-Auth-Token: $token" -d '{"pod": {"region_name":  "RegionOne", "az_name": "az1"}}'


  • The restful API request will be routed to V1Controller from RootController for the version is 'v1.0':
 tricircle/tricircle/api/controllers/root.py
 ...
 class RootController(object):
      ...
     @expose()
     def _lookup(self, version, *remainder):
         if version == 'v1.0':
             return V1Controller(), remainder
  • In V1Controller, because the request is to create pod, so the request will be forwarded to the post function in PodsController. Post function will create the pod and az mapping record in pod table:
 tricircle/tricircle/api/controllers/pod.py
 ...
 class PodsController(rest.RestController):
   ...
   @expose(generic=True, template='json')
   def post(self, **kw):
       context = t_context.extract_context_from_environ()
       ...
       new_pod = core.create_resource(
                   context, models.Pod,
                   {'pod_id': _uuid,
                    'region_name': region_name,
                    'pod_az_name': pod_az_name,
                    'dc_name': dc_name,
                    'az_name': az_name})

Step 6: Create network/subnet in central Neutron server

The central_plugin.py will be called because the core plugin of central Neutron is configured like this:

   core_plugin = tricircle.network.central_plugin.TricirclePlugin

Command issued:

 neutron --os-region-name=CentralRegion net-create net1
 neutron --os-region-name=CentralRegion subnet-create net1 10.0.1.0/24
  • The request is handled by create_network when net1 is to be created:
 tricircle/tricircle/network/central_plugin.py
 ...
 class TricirclePlugin(db_base_plugin_v2.NeutronDbPluginV2,
                     security_groups.TricircleSecurityGroupMixin,
                     external_net_db.External_net_db_mixin,
                     portbindings_db.PortBindingMixin,
                     extradhcpopt_db.ExtraDhcpOptMixin,
                     l3_db.L3_NAT_dbonly_mixin,
                     l3_attrs_db.ExtraAttributesMixin):
  ...
  def create_network(self, context, network):
       net_data = network[attributes.NETWORK]
       ...
  • The request is handled by create_subnet when 10.0.1.0/24 subnet is created in net1:
 tricircle/tricircle/network/central_plugin.py
 ...
 class TricirclePlugin(db_base_plugin_v2.NeutronDbPluginV2,
                     security_groups.TricircleSecurityGroupMixin,
                     external_net_db.External_net_db_mixin,
                     portbindings_db.PortBindingMixin,
                     extradhcpopt_db.ExtraDhcpOptMixin,
                     l3_db.L3_NAT_dbonly_mixin,
                     l3_attrs_db.ExtraAttributesMixin):
 ...
 def create_subnet(self, context, subnet):
       subnet_data = subnet['subnet']
       ...

Step 7: Get image ID and flavor ID which will be used in VM booting

Command issued:

 glance --os-region-name=RegionOne image-list
 nova --os-region-name=RegionOne flavor-list

No source code of Tricircle will be involved in this step.

Step 8: Boot virtual machines

Command issued:

 nova --os-region-name=RegionOne boot --flavor 1 --image $image1_id --nic net-id=$net1_id vm1

Please note, Nova in RegionOne is configured to use local Neutron in RegionOne, and RegionTwo's Nova will use local Neutron in RegionTwo. So different local Neutron will be called when booting instances in different Region.

In RegionOne local Neutron's configuration, the core plugin is configured like this:

 core_plugin = tricircle.network.local_plugin.TricirclePlugin
  • When nova boot is executed in RegionOne Nova, Nova will send networks query with network id to local Neutron, and local plugin will send networks query request to central Neutron:
 tricircle/tricircle/network/local_plugin.py
 ...
 def get_networks(self, context, filters=None, fields=None,
                    sorts=None, limit=None, marker=None, page_reverse=False):
 ...
 t_networks = raw_client.list_networks(**params)['networks']
  • raw_client.list_networks is to query networks in central Neutron, if the network existed in central Neutron, local plugin will create network/subnet with same uuid as that in central Neutron:
 tricircle/tricircle/network/local_plugin.py
 ...
 def get_networks(self, context, filters=None, fields=None,
                    sorts=None, limit=None, marker=None, page_reverse=False):
       ...
               b_network = self.core_plugin.create_network(
                   context, {'network': network})
               subnet_ids = self._ensure_subnet(context, network)
  • later nova in RegionOne will issue a request to create a port in local Neutron in RegionOne, the creation process will create the port in central Neutron first, then create the port and security group in local Neutron with same uuid.
 tricircle/tricircle/network/local_plugin.py
 ...
 def create_port(self, context, port):
 ...
           self._adapt_port_body_for_client(port['port'])
           t_port = raw_client.create_port(port)['port']
  ....
   self._handle_security_group(t_ctx, context, t_port)
   b_port = self.core_plugin.create_port(context, {'port': t_port})
  • next step, nova in RegionOne will issue a request to update port binding in local Neutron in RegionOne, the binding process in local Neutron plugin will forward the port update to central Neutron, and then update local port too.
 tricircle/tricircle/network/local_plugin.py
 ...
 def update_port(self, context, _id, port):
 ...
           t_ctx = t_context.get_context_from_neutron_context(context)
           self.neutron_handle.handle_update(t_ctx, 'port', _id,
                                             {'port': update_dict})
       return self.core_plugin.update_port(context, _id, port)
  • Once central Neutron receives the port update request, need to create resource routing entries in routing table.
 tricircle/tricircle/network/central_plugin.py
 ...
 def update_port(self, context, port_id, port):
       t_ctx = t_context.get_context_from_neutron_context(context)
       top_port = super(TricirclePlugin, self).get_port(context, port_id)
       ...
          for resource_id, resource_type in entries:
               if db_api.get_bottom_id_by_top_id_region_name(
                       t_ctx, resource_id, pod['region_name'], resource_type):
                   continue
               db_api.create_resource_mapping(t_ctx, resource_id, resource_id,
                                              pod['pod_id'], res['tenant_id'],
                                              resource_type)
  • If the network has interface connected to a router, then start asynchronous job to (create if not exist in RegionOne) set up the router in Region.
 tricircle/tricircle/network/central_plugin.py
 ...
 def update_port(self, context, port_id, port):
      ...
      interfaces = super(TricirclePlugin, self).get_ports(
               context,
               {'network_id': [res['network_id']],
                'device_owner': [constants.DEVICE_OWNER_ROUTER_INTF]})
           interfaces = [inf for inf in interfaces if inf['device_id']]
           if interfaces:
               # request may be come from service, we use an admin context
               # to run the xjob
               admin_context = t_context.get_admin_context()
               self.xjob_handler.setup_bottom_router(
                   admin_context, res['network_id'],
                   interfaces[0]['device_id'], pod['pod_id'])
  • If the network has interface connected to a router, xjob_handler.setup_bottom_router will register the asynchronous job and send one RPC message to xmanager,py to trigger the asynchronous job.
  tricircle/tricircle/xjob/xmanager.py
  ...
   @_job_handle(constants.JT_ROUTER_SETUP)
   def setup_bottom_router(self, ctx, payload):
       (b_pod_id,
        t_router_id, t_net_id) = payload[constants.JT_ROUTER_SETUP].split('#')
        ...
  • Except the setup_bottom_router async. job, there are still other resources need to be check in update_port function in central_plugin.py, whether to create or update these resources in RegionOne, for example, security group.
 tricircle/tricircle/network/central_plugin.py
 ...
 def update_port(self, context, port_id, port):
      ...
      self.xjob_handler.configure_security_group_rules(t_ctx,
                                                            res['tenant_id'])

xjob_handler.configure_security_group_rules will register the security group rules asynchronous job, and then send RPC message to trigger the async.job in xmanager.py.

  • You can find the configure_security_group_rules job in xmanager.py.
    tricircle/tricircle/xjob/xmanager.py
  ...
  @_job_handle(constants.JT_SEG_RULE_SETUP)
   def configure_security_group_rules(self, ctx, payload):
       project_id = payload[constants.JT_SEG_RULE_SETUP]
       top_client = self._get_client()
       sg_filters = [{'key': 'tenant_id', 'comparator': 'eq',
                      'value': project_id}] 

The router setup and security group rules configuration (and other resources will be included later) are handled as async. jobs in XJOB process, the reason is to enhance the experience instances booting. If all this resources are created in local_plugin, then it'll take long time and may lead to the time out of booting, bad user experience and low reliability. At the same time, through the aync. job redo and reliability consideration design, the reliability of the networking automation process is ensured.

Please note: the central Neutron in the source code sometimes is called "top", local neutron as "bottom".

Step 9: Verify the VMs are connected to the networks

Tricircle is not involved in this step.