Jump to: navigation, search

Difference between revisions of "StarlingX/Networking/TSN"

(Requirements)
Line 32: Line 32:
 
The following diagram shows the setup for the demo:
 
The following diagram shows the setup for the demo:
  
=== Walk through to deploying TSN applications on STX ===
+
The demo includes below hardware and software components:
 +
 
 +
* Edge cloud platform: building based on STX All-In-One(AIO) which provides IaaS infrastructure for edge cloud environment
 +
 
 +
* Edge device: device in edge side to generate TSN data for processing
 +
 
 +
* Intel® Ethernet Controller I210: installed on both edge cloud platform node and edge device and connected by CAT-5E Ethernet cable. Igb.ko is its Linux kernel driver which had been included in Linux
 +
 
 +
* Edge Gateway: Virtual Machine created by STX and worked as gateway to collect TSN data, perform edge computing then send to data center. The I210 adapter is exposed to edge gateway from host node through OpenStack Nova’s PCI pass-through support
 +
 
 +
* p2p4l, phc2sys: utility from LinuxPTP project to support the time synchronization over the PTP
 +
 
 +
* tc: utility/command used to configure Traffic Control in the Linux kernel
 +
 
 +
* TSN Sender/Receiver: TSN reference application from [1] to send/receive TSN data for processing
 +
 
 +
 
 +
=== Walk through to deploy TSN applications on STX ===
 +
 
 +
* <big>Edge Cloud Platform</big>
 +
1. Install STX environment: following the guide [2] to install one STX environment (e.g. STX AIO)
 +
 
 +
2. Preparing a Virtual Machine image
 +
 
 +
Create image tsn_ubuntu_19_04.img based on Ubuntu 19.04 with below required binaries:
 +
# linuxPTP
 +
git clone https://git.code.sf.net/p/linuxptp/code linuxptp-code
 +
cd linuxptp-code
 +
make
 +
make install
 +
# iproute2
 +
apt install bison flex elfutils -y
 +
git clone http://git.kernel.org/pub/scm/network/iproute2/iproute2.git
 +
cd iproute2
 +
make
 +
make install
 +
 
 +
Upload image to OpenStack Glance and enable pci passthrough in flavor property
 +
# upload image
 +
openstack image create --container-format bare --disk-format qcow2 --file tsn_ubuntu_19_04.img --public tsn-ubuntu-19-04
 +
# add pci-passthrough property to flavor (e.g. m1.medium), “h210-1” is the alias name of the PCI device configured in nova.config
 +
openstack flavor set m1.medium --property pci_passthrough:alias=h210-1:1
 +
3. Configure OpenStack Nova to allow for PCI pass-through
 +
 
 +
Create nova-tsn-pt.yaml file to allow PCI pass-through for i210 adapter (e.g. device id: 8086:1533)
 +
conf:
 +
nova:
 +
  pci:
 +
    alias:
 +
        type: multistring
 +
        values:
 +
        - '{"vendor_id": "8086", "product_id": "1533","device_type":"type-PCI","name": "h210-1"}'
 +
    passthrough_whitelist:
 +
        type: multistring
 +
        values:
 +
        - '{"class_id": "8086", "product_id":"1533"}'
 +
overrides:
 +
  nova_compute:
 +
    hosts:
 +
    - conf:
 +
        nova:
 +
          DEFAULT:
 +
            my_ip: {host_ip}
 +
            shared_pcpu_map: '""'
 +
            vcpu_pin_set: '"2-5"'
 +
          libvirt:
 +
            images_type: default
 +
            live_migration_inbound_addr: {host_ip}
 +
          pci:
 +
            passthrough_whitelist:
 +
              type: multistring
 +
              values:
 +
              - '{"class_id": "8086", "product_id": "1533"}'
 +
          vnc:
 +
            vncserver_listen: 0.0.0.0
 +
            vncserver_proxyclient_address: {host_ip}
 +
      name: {controller_name}
 +
 
 +
Note: other configurations besides pci in nova_compute hosts such as libvirt, vnc (which are configuration generated based on STX compute node information) are also required due to the override mechanism of openstack-helm for list (e.g. hosts) which will replace all content of the list instead of replace single configuration item for each list element.
 +
 
 +
Enable nova pci-passthrough configuration in STX
 +
# set pci-passthrough config
 +
system helm-override-update  stx-openstack nova openstack --values nova-tsn-pt.yaml
 +
system application-apply stx-openstack
 +
 
 +
4. Create Virtual Machine instance and install TSN reference application
 +
 
 +
Create virtual machine instance:
 +
openstack server create --image tsn-ubuntu-19-04 --network ${network_uuid} --flavor m1.medium tsn-demo
 +
 
 +
Install TSN reference application and other test applications:
 +
 
 +
Following the instructions on [1] to compile and install below applications in the VM instances:
 +
 
 +
(1) iperf3: running in server mode to receive best effort traffic
 +
 
 +
(2) simple_listener: TSN test application which receives IEEE1722 class A traffic, and it is used to test IEEE 802.1Qav or Credit Based Shaper use case
 +
 
 +
(3) sample-app-taprio: TSN test application which receives traffic and measure Tx latency, and it is used to test IEEE 802.1Qbv or Time Aware Shaper use case
 +
 
 +
* <big>Edge Device</big>
 +
 
 +
1. Install OS and required libraries
 +
 
 +
Install Ubuntu 19.04
 +
 
 +
Install required libraries
 +
# linuxPTP
 +
git clone https://git.code.sf.net/p/linuxptp/code linuxptp-code
 +
cd linuxptp-code
 +
make
 +
make install
 +
 
 +
# iproute2
 +
apt install bison flex elfutils -y
 +
git clone http://git.kernel.org/pub/scm/network/iproute2/iproute2.git
 +
cd iproute2
 +
make
 +
make install
 +
 
 +
2. Install TSN reference applications and other test applications
 +
 
 +
Following the instructions on [1] to compile and install below applications in the device:
 +
 
 +
(1) iperf3: running in server mode to receive best effort traffic
 +
 
 +
(2) simple_talker-cmsg: TSN test application which sends IEEE1722 class A traffic, and it is used to test IEEE 802.1Qav or Credit Based Shaper use case
 +
 
 +
(3) sample-app-taprio: TSN test application which sends scheduled traffic, and it is used to test IEEE 802.1Qbv or Time Aware Shaper use case
  
 
=== Demos ===
 
=== Demos ===

Revision as of 04:55, 23 August 2019

Deploying and Running TSN application in StarlingX Virtual Machine Workload Mode

Introduction

Embedded sectors such as Automotive, Industrial, professional audio/video networking as well as blockchain and high frequency trading have emphasized the need for real-time networks. While Common LAN models are based on Internet Protocols and the IEEE 802 architecture and most of operation is best-effort which is not suitable for use cases (special for edge computing) that require high /known/deterministic availability.

Time Sensitive Networking (TSN) is a set of evolving standards developed by IEEE 802.1 Working Group to cover a group of vendor-neutral and IEEE standards with the aim of guaranteeing determinism in delivering time-sensitive traffic with low and bounded latency within an infinitesimal packet loss over the network, while allowing non time-sensitive traffic to be carried through the same network. It is also a key technology that targets for above edge computing segments.

StarlingX (STX) is a complete cloud infrastructure software stack for the edge and it provides running workloads on both virtual machine and container environment.

This Wiki introduces how to deploy and run TSN application in STX virtual machine workload, sample TSN reference applications are taken from [1] with focus on 2 key use cases:

1) IEEE 802.1Qav or Credit Based Shaper (CBS) Ensure bounded transmission latency for time sensitive, loss-sensitive real-time data stream in some critical user scenarios. For instance, when time sensitive traffic and best effort traffic are transmitted together, users require the bandwidth and latency of time-sensitive traffic is protected in the midst of overloaded traffic on the same network, i.e. ensure time-sensitive traffic to have constant transmission rate and latency.

2) IEEE 802.1Qbv or Time Aware Shaper (TAS) Create a protected transmission window for scheduled traffic, which requires low and bounded transmission latency. Scheduled traffic is the term used in IEEE 802.1Qbv to refer to periodic traffic such as industrial automation control frames. This type of traffic is short in frame length and requires immediate transmission when its schedule starts.

Requirements

The TSN reference application had been verified on the following environments:

Hardware
Software * Linux Kernel 4.19.04 +

Demo Environment Setup

The following diagram shows the setup for the demo:

The demo includes below hardware and software components:

  • Edge cloud platform: building based on STX All-In-One(AIO) which provides IaaS infrastructure for edge cloud environment
  • Edge device: device in edge side to generate TSN data for processing
  • Intel® Ethernet Controller I210: installed on both edge cloud platform node and edge device and connected by CAT-5E Ethernet cable. Igb.ko is its Linux kernel driver which had been included in Linux
  • Edge Gateway: Virtual Machine created by STX and worked as gateway to collect TSN data, perform edge computing then send to data center. The I210 adapter is exposed to edge gateway from host node through OpenStack Nova’s PCI pass-through support
  • p2p4l, phc2sys: utility from LinuxPTP project to support the time synchronization over the PTP
  • tc: utility/command used to configure Traffic Control in the Linux kernel
  • TSN Sender/Receiver: TSN reference application from [1] to send/receive TSN data for processing


Walk through to deploy TSN applications on STX

  • Edge Cloud Platform

1. Install STX environment: following the guide [2] to install one STX environment (e.g. STX AIO)

2. Preparing a Virtual Machine image

Create image tsn_ubuntu_19_04.img based on Ubuntu 19.04 with below required binaries:

# linuxPTP
git clone https://git.code.sf.net/p/linuxptp/code linuxptp-code
cd linuxptp-code
make
make install
# iproute2
apt install bison flex elfutils -y
git clone http://git.kernel.org/pub/scm/network/iproute2/iproute2.git 
cd iproute2
make
make install

Upload image to OpenStack Glance and enable pci passthrough in flavor property

# upload image
openstack image create --container-format bare --disk-format qcow2 --file tsn_ubuntu_19_04.img --public tsn-ubuntu-19-04
# add pci-passthrough property to flavor (e.g. m1.medium), “h210-1” is the alias name of the PCI device configured in nova.config
openstack flavor set m1.medium --property pci_passthrough:alias=h210-1:1

3. Configure OpenStack Nova to allow for PCI pass-through

Create nova-tsn-pt.yaml file to allow PCI pass-through for i210 adapter (e.g. device id: 8086:1533)

conf:
nova:
 pci:
   alias:
       type: multistring
       values:
       - '{"vendor_id": "8086", "product_id": "1533","device_type":"type-PCI","name": "h210-1"}'
   passthrough_whitelist:
       type: multistring
       values:
       - '{"class_id": "8086", "product_id":"1533"}'
overrides:
  nova_compute:
    hosts:
    - conf:
        nova:
         DEFAULT:
           my_ip: {host_ip}
           shared_pcpu_map: '""'
           vcpu_pin_set: '"2-5"'
         libvirt:
           images_type: default
           live_migration_inbound_addr: {host_ip}
         pci:
           passthrough_whitelist:
             type: multistring
             values:
             - '{"class_id": "8086", "product_id": "1533"}'
         vnc:
           vncserver_listen: 0.0.0.0
           vncserver_proxyclient_address: {host_ip}
      name: {controller_name}

Note: other configurations besides pci in nova_compute hosts such as libvirt, vnc (which are configuration generated based on STX compute node information) are also required due to the override mechanism of openstack-helm for list (e.g. hosts) which will replace all content of the list instead of replace single configuration item for each list element.

Enable nova pci-passthrough configuration in STX

# set pci-passthrough config
system helm-override-update  stx-openstack nova openstack --values nova-tsn-pt.yaml
system application-apply stx-openstack

4. Create Virtual Machine instance and install TSN reference application

Create virtual machine instance:

openstack server create --image tsn-ubuntu-19-04 --network ${network_uuid} --flavor m1.medium tsn-demo

Install TSN reference application and other test applications:

Following the instructions on [1] to compile and install below applications in the VM instances:

(1) iperf3: running in server mode to receive best effort traffic

(2) simple_listener: TSN test application which receives IEEE1722 class A traffic, and it is used to test IEEE 802.1Qav or Credit Based Shaper use case

(3) sample-app-taprio: TSN test application which receives traffic and measure Tx latency, and it is used to test IEEE 802.1Qbv or Time Aware Shaper use case

  • Edge Device

1. Install OS and required libraries

Install Ubuntu 19.04

Install required libraries

# linuxPTP
git clone https://git.code.sf.net/p/linuxptp/code linuxptp-code
cd linuxptp-code
make
make install
# iproute2
apt install bison flex elfutils -y
git clone http://git.kernel.org/pub/scm/network/iproute2/iproute2.git 
cd iproute2
make
make install

2. Install TSN reference applications and other test applications

Following the instructions on [1] to compile and install below applications in the device:

(1) iperf3: running in server mode to receive best effort traffic

(2) simple_talker-cmsg: TSN test application which sends IEEE1722 class A traffic, and it is used to test IEEE 802.1Qav or Credit Based Shaper use case

(3) sample-app-taprio: TSN test application which sends scheduled traffic, and it is used to test IEEE 802.1Qbv or Time Aware Shaper use case

Demos