VTN Manual Installation Guide

(1)

VTN Manual Installation Guide

Introduction

Installation Steps

Pre-requisites on the OpenStack compute nodes Installing ONOS and VTN

Installing OpenStack with DevStack

Installing and Running XOS and R-CORD profile References

Introduction

This page describes how to setup a working development environment that can be used in developing with VTN. It includes manual build, configuration and installation of ONOS, OpenStack and XOS, which might help understanding how VTN interacts with the other software components. For those who just wants to try VTN out, it is recommended to try CORD-in-a-Box. Note that this instructions assume you’re familiar with ONOS and OpenStack, and do not provide a guide to how to install or trouble shooting these services. However, if you aren’t, please find a guide from ONOS(http://wiki.onosproject.org) and OpenStack(http://docs.openstack.org), respectively.

You will need:

Ubuntu machines for ONOS cluster

Ubuntu machines for OpenStack controller(at least 4G RAM is recommended) and compute nodes(at least 8G RAM is recommended) Ubuntu machine for XOS

Installation Steps

Pre-requisites on the OpenStack compute nodes

1. Upgrade OVS version to 2.3.0 or later. This guide works very well for me (don't forget to change the version in the guide to 2.3.0 or later).

2. Set OVSDB passive mode.

Install OVS version 2.3.0 or later and then set OVSDB in passive mode in compute nodes by running the following command.

$ ovs-appctl -t ovsdb-server ovsdb-server/add-remote ptcp:6640:[host_ip]

Or you can set permanently by adding the following line to /usr/share/openvswitch/scripts/ovs-ctl, right after "set ovsdb-server "$DB_FILE" line. After modifying the script, restart openvswitch-switch service.

set "$@" --remote=ptcp:6640

Check if it listens to the TCP port 6640.

$ netstat -ntl

Active Internet connections (only servers)

Proto Recv-Q Send-Q Local Address Foreign Address State

tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:6640 0.0.0.0:* LISTEN

3. Setup passwordless SSH from ONOS instance to compute nodes with the "root" user. And then copy the private key with the name of "node_key". Note that this node key will be used again later.

sdn@onos:~$ cp ~/.ssh/id_rsa ~/.ssh/node_key

sdn@onos:~$ ssh -i ~/.ssh/node_key root@[compute-01] check if it's successful

(2)

Installing ONOS and VTN

1. Download, build, and install ONOS.

Please refer to https://wiki.onosproject.org/display/ONOS/Tutorials and https://wiki.onosproject.org/display/ONOS/Developer+Guide.

2. Activate the following ONOS applications (don't care about the version number in the examples).

onos> apps -a -s

* 10 org.onosproject.optical-model 1.10.0.SNAPSHOT Optical information model * 42 org.onosproject.drivers 1.10.0.SNAPSHOT Default device drivers * 51 org.onosproject.openflow-base 1.10.0.SNAPSHOT OpenFlow Provider

Any other ONOS applications, especially any kind of host provider, can cause conflicts with VTN. So please be careful when you activate other ONOS applications.

3. Download and build cord-config and vtn.

$ git clone https://gerrit.opencord.org/config $ cd config && mci

$ git clone https://gerrit.opencord.org/vtn $ cd vtn && mci

4. Run the command below from the ONOS build machine to install cord-config and VTN to running ONOS.

$ onos-app $OC1 install! config/target/cord-config-1.2-SNAPSHOT.oar $ onos-app $OC1 install! vtn/target/vtn-1.2-SNAPSHOT.oar

5. Check all the applications are activated successfully. (Don't mind the version numbers.)

onos> apps -a -s

* 10 org.onosproject.optical-model 1.10.0.SNAPSHOT Optical information model * 21 org.onosproject.ovsdb-base 1.10.0.SNAPSHOT OVSDB Provider

* 24 org.onosproject.drivers.ovsdb 1.10.0.SNAPSHOT OVSDB Device Drivers * 42 org.onosproject.drivers 1.10.0.SNAPSHOT Default device drivers * 51 org.onosproject.openflow-base 1.10.0.SNAPSHOT OpenFlow Provider

* 98 org.opencord.config 1.2.SNAPSHOT CORD configuration meta application * 99 org.opencord.vtn 1.2.SNAPSHOT VTN App

Installing OpenStack with DevStack

1. Download and install ONOS ML2 mechanism driver.

$ mkdir -p /opt/stack && cd /opt/stack

$ git clone https://github.com/openstack/networking-onos.git $ cd networking-onos

$ sudo pip install ./networking-onos

(3)

networking-onos/etc/neutron/plugins/ml2/ml2_conf_onos.ini

# Configuration options for ONOS ML2 Mechanism driver [onos]

# (StrOpt) ONOS ReST interface URL. This is a mandatory field. url_path = http://[onos_ip]:8181/onos/cordvtn

# (StrOpt) Username for authentication. This is a mandatory field. username = onos

# (StrOpt) Password for authentication. This is a mandatory field. password = rocks

3. Download DevStack.

$ git clone https://git.openstack.org/openstack-dev/devstack -b stable/mitaka

4. Create local.conf for each OpenStack node.

Here is the sample local.conf for controller node, which runs Keystone, Nova, Neutron, and Glance services.

Controller Node local.conf

[[local|localrc]] HOST_IP=10.90.0.58 SERVICE_HOST=10.90.0.58 RABBIT_HOST=10.90.0.58 DATABASE_HOST=10.90.0.58 ADMIN_PASSWORD=[admin_password] DATABASE_PASSWORD=$ADMIN_PASSWORD RABBIT_PASSWORD=$ADMIN_PASSWORD SERVICE_PASSWORD=$ADMIN_PASSWORD SERVICE_TOKEN=$ADMIN_PASSWORD DATABASE_TYPE=mysql

FORCE_CONFIG_DRIVE=True USE_SSL=True

# Networks

Q_ML2_TENANT_NETWORK_TYPE=vxlan

Q_ML2_PLUGIN_MECHANISM_DRIVERS=onos_ml2

Q_PLUGIN_EXTRA_CONF_PATH=/opt/stack/networking-onos/etc/neutron/plugins/ml2 Q_PLUGIN_EXTRA_CONF_FILES=(ml2_conf_onos.ini)

NEUTRON_CREATE_INITIAL_NETWORKS=False # Services

enable_service q-svc disable_service n-net disable_service n-cpu disable_service tempest disable_service c-sch disable_service c-api disable_service c-vol # Branches

GLANCE_BRANCH=stable/mitaka HORIZON_BRANCH=stable/mitaka KEYSTONE_BRANCH=stable/mitaka NEUTRON_BRANCH=stable/mitaka NOVA_BRANCH=stable/mitaka

(4)

Here is the sample local.conf for compute node, which runs Nova compute agent.

Compute Node local.conf

[[local|localrc]]

HOST_IP=10.90.0.64 <-- local IP

SERVICE_HOST=162.243.x.x <-- controller IP, must be reachable from your test browser for console access from Horizon

RABBIT_HOST=10.90.0.58 DATABASE_HOST=10.90.0.58 ADMIN_PASSWORD=nova

DATABASE_PASSWORD=$ADMIN_PASSWORD RABBIT_PASSWORD=$ADMIN_PASSWORD SERVICE_PASSWORD=$ADMIN_PASSWORD SERVICE_TOKEN=$ADMIN_PASSWORD DATABASE_TYPE=mysql

NOVA_VNC_ENABLED=True

VNCSERVER_PROXYCLIENT_ADDRESS=$HOST_IP VNCSERVER_LISTEN=$HOST_IP

LIBVIRT_TYPE=kvm # Services

ENABLED_SERVICES=n-cpu,neutron # Branches

NOVA_BRANCH=stable/mitaka KEYSTONE_BRANCH=stable/mitaka NEUTRON_BRANCH=stable/mitaka

For those who installs OpenStack with other deploy tools, here are the Nova and Neutron configurations to use VTN as a ML2 mechanism driver. Also, make sure to enable SSL for all services.

/etc/nova/nova.conf

[DEFAULT]

force_config_drive = True

network_api_class = nova.network.neutronv2.api.API

/etc/neutron/neutron.conf

core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin

/etc/neutron/plugins/ml2/ml2_conf.ini

[ml2]

tenant_network_types = vxlan type_drivers = vxlan

mechanism_drivers = onos_ml2

5. Run DevStack.

(5)

Installing and Running XOS and R-CORD profile

1. In the XOS machine, download and run cord-bootstrap.sh script under /opt.

$ curl -o ~/cord-bootstrap.sh https://raw.githubusercontent.com/opencord/platform-install/master/scripts/cord-bootstrap.sh

$ bash cord-bootstrap.sh

2. Edit rcord inventory to add compute nodes. Hosts file(/etc/hosts) should have IP addresses of the compute nodes as well.

cord/build/platform-install/inventory/rcord

; rcord configuration [all:vars]

cord_profile=rcord [config]

localhost ansible_connection=local [head]

localhost ansible_connection=local [build]

localhost ansible_connection=local [compute]

compute-01 compute-02

3. Edit vtn-service.yaml.j2 to fix ONOS rest_port to 8181, ovsdbPort to 6640, controllerPort to onos-cord:6653 and the path of node_key in ONOS machine.

(6)

/root/cord/build/platform-install/roles/cord-profile/templates/vtn-service.yaml.j2

topology_template: node_templates: service#ONOS_CORD:

type: tosca.nodes.ONOSService requirements:

properties: kind: onos

view_url: /admin/onos/onosservice/$id$/ no_container: true

rest_hostname: onos-cord

rest_port: 8181 // FIX THIS VALUE replaces: service_ONOS_CORD

service#vtn:

type: tosca.nodes.VTNService properties:

view_url: /admin/vtn/vtnservice/$id$/ privateGatewayMac: 00:00:00:00:00:01

localManagementIp: {{ management_network_ip }} ovsdbPort: 6640 // FIX THIS VALUE

sshUser: root

sshKeyFile: ~/.ssh/node_key // FIX THIS VALUE sshPort: 22

xosEndpoint: http://xos:{{ xos_ui_port }}/ xosUser: {{ xos_admin_user }}

xosPassword: {{ xos_admin_pass }} replaces: service_vtn

vtnAPIVersion: 2

controllerPort: onos-cord:6653 // FIX THIS VALUE

4. Now we need to manually create or edit several files that MAAS does for us in CiaB install. They are, /etc/hosts, extra_hosts

/root/.ssh/id_rsa, /root/.ssh/id_rsa.pub, /root/node_key /root/openstack-compute.yaml

/root/openstack-compute-vtn.yaml

/root/cord/build/platform-install/profile_manifests/local_vars.yml

etc/hosts, extra_hosts

First, edit hostfile and add the followings.

127.0.0.1 localhost xos.cord.lab xos xos-gui xos-ws xos-chameleon COMPUTE_01_IP compute-01

COMPUTE_01_IP compute-02

OPENSTACK_IP openstack keystone.cord.lab ONOS_CORD_IP onos-cord

(7)

xos_ui:

image: {{ deploy_docker_registry }}xosproject/xos-ui:{{ deploy_docker_tag }} networks:

{% for network in xos_docker_networks %} - {{ network }}

{% endfor %}

{% if extra_hosts %} // ADD THIS BLOCK - START extra_hosts:

{% for extra_host in extra_hosts %} - {{ extra_host }}

{% endfor %}

{% endif %} // ADD THIS BLOCK - END

{% if svc.synchronizer is not defined or svc.synchronizer %} {{ svc.name }}-synchronizer:

{% if extra_hosts %} // ADD THIS BLOCK - START extra_hosts:

{% for extra_host in extra_hosts %} - {{ extra_host }}

{% endfor %}

{% endif %} // ADD THIS BLOCK - END

image: {{ deploy_docker_registry }}xosproject/{{ svc.name }}-synchronizer:{{ deploy_docker_tag }} networks:

{% for network in xos_docker_networks %} - {{ network }}

{% endfor %}

/root/.ssh/id_rsa, /root/.ssh/id_rsa.pub, /root/node_key

Copy SSH keys created during the pre-requisites step 3 from ONOS node to ansible user home .ssh directory, /root/.ssh in this example. Copy id_rsa to /root/node_key as well.

/root/openstack-compute.yaml

Create /root/openstack-compute.yaml file. Here is the sample with two compute nodes, compute-01 and compute-02. You'll need to fix only the hostname according to your setup. Note that the hostname here must be the same with the hostname registered in OpenStack service. You can check the hostname on OpenStack with "nova host-list" command.

(8)

/root/openstack-compute.yaml

tosca_definitions_version: tosca_simple_yaml_1_0 imports:

- custom_types/xos.yaml

description: Adds OpenStack compute nodes topology_template:

node_templates:

# Site/Deployment, fully defined in deployment.yaml mysite:

type: tosca.nodes.Site properties:

no-delete: true no-create: true no-update: true MyDeployment:

type: tosca.nodes.Deployment properties:

no-delete: true no-create: true no-update: true # OpenStack compute nodes

compute-01: // FIX THE HOSTNAME IF NECESSARY type: tosca.nodes.Node

requirements: - site:

node: mysite

relationship: tosca.relationships.MemberOfSite - deployment:

node: MyDeployment

relationship: tosca.relationships.MemberOfDeployment compute-02: // FIX THE HOSTNAME IF NECESSARY type: tosca.nodes.Node

requirements: - site:

node: mysite

relationship: tosca.relationships.MemberOfSite - deployment:

node: MyDeployment

relationship: tosca.relationships.MemberOfDeployment

/root/openstack-compute-vtn.yaml

Create openstack-compute-vtn.yaml file and place it under /root/. Here is the sample with the two compute nodes. You might need to fix most of the fields including hostname, dataPlaneIntf and dataPlaneIp. Refer to VTN Configuration Guide to get an idea of these fields.

/root/openstack-compute-vtn.yaml

tosca_definitions_version: tosca_simple_yaml_1_0 imports:

- custom_types/xos.yaml

description: Configures VTN networking for OpenStack compute nodes topology_template:

node_templates:

(9)

service#ONOS_CORD: type: tosca.nodes.ONOSService properties: no-delete: true no-create: true no-update: true

# VTN networking for OpenStack Compute Nodes # Compute node, fully defined in compute-nodes.yaml compute-01: // FIX THE HOSTNAME IF NECESSARY type: tosca.nodes.Node

properties: no-delete: true no-create: true no-update: true

# VTN bridgeId field for node compute-01

compute-01_bridgeId_tag: // FIX THE HOSTNAME IF NECESSARY type: tosca.nodes.Tag

properties: name: bridgeId

value: of:000000000000001 // FIX THE BRIDGE ID IF NECESSARY requirements:

- target:

node: compute-01 // FIX THE HOSTNAME IF NECESSARY relationship: tosca.relationships.TagsObject - service:

node: service#ONOS_CORD

relationship: tosca.relationships.MemberOfService # VTN dataPlaneIntf field for node compute-01

compute-01_dataPlaneIntf_tag: // FIX THE HOSTNAME IF NECESSARY type: tosca.nodes.Tag

properties:

value: veth1 // FIX THE DATA PLANE INTERFACE requirements:

- target:

relationship: tosca.relationships.MemberOfService # VTN dataPlaneIp field for node compute-01

compute-01_dataPlaneIp_tag: // FIX THE HOSTNAME IF NECESSARY type: tosca.nodes.Tag

properties:

value: 10.2.2.28/24 // FIX THE DATA PLANE IP requirements:

- target:

relationship: tosca.relationships.MemberOfService # Compute node, fully defined in compute-nodes.yaml

compute-02: // FIX THE HOSTNAME IF NECESSARY type: tosca.nodes.Node

properties: no-delete: true no-create: true no-update: true

# VTN bridgeId field for node compute-02

compute-02_bridgeId_tag: // FIX THE HOSTNAME IF NECESSARY type: tosca.nodes.Tag

(10)

value: of:0000000000000002 // FIX THE BRIDGE ID IF NECESSARY requirements:

- target:

relationship: tosca.relationships.MemberOfService # VTN dataPlaneIntf field for node compute-02

compute-02_dataPlaneIntf_tag: // FIX THE HOSTNAME IF NECESSARY type: tosca.nodes.Tag

properties:

value: veth1 // FIX THE DATA PLANE INTERFACE requirements:

- target:

relationship: tosca.relationships.MemberOfService # VTN dataPlaneIp field for node compute-02

compute-02_dataPlaneIp_tag: // FIX THE HOSTNAME IF NECESSARY type: tosca.nodes.Tag

properties:

value: 10.2.2.29/24 // FIX THE DATA PLANE IP requirements:

- target:

relationship: tosca.relationships.MemberOfService

/root/cord/build/platform-install/profile_manifests/local_vars.yml Edit local_vars.yml for extra configurations.

cord/build/platform-install/profile_manifests/local_vars.yml

# local_custom_vars.yaml

# Put any local customizations to variables in this file.

extra_hosts: [ "onos-cord:ONOS_IP", "compute-01:COMPUTE_01_IP", "compute-02:COMPUTE_02_IP", "keystone.cord.lab: OPENSTACK_CTRL_IP", "xos-core.cord.lab:172.18.0.1", "cordloghost:172.18.0.1" ]

head_cord_profile_dir: "{{ ansible_user_dir + '/cord_profile' }}" head_cord_dir: "{{ ansible_user_dir + '/cord' }}"

keystone_admin_password: OPENSTACK_ADMIN_PASSWD

5. Lastly, you need to install ElasticSearch.

$ cd ~/cord/build/platform-install/

$ ansible-playbook -i inventory/head-localhost deploy-elasticstack-playbook.yml

6. All configurations are ready. Run XOS with R-CORD profile with deploy" command below. You can also teardown R-CORD profile with "xos-teardown" command.

(11)

alias xos-teardown="rm -rf /opt/credentials; pushd /root/cord/build/platform-install; ansible-playbook -i inventory/rcord teardown-playbook.yml;"

alias xos-deploy="mkdir /root/cord_profile; cp /root/openstack-*.yaml /root/cord_profile/; pushd /root/cord /build/platform-install; ansible-playbook -i inventory/rcord deploy-xos-playbook.yml; mkdir /opt/credentials; cp /root/cord/build/platform-install/credentials/* /opt/credentials/;"

7. You'll need to configure VTN by pushing network configurations.

$ docker exec rcord_xos_ui_1 python tosca/run.py [email protected] /opt/cord_profile/vtn-service.yaml $ docker exec rcord_xos_ui_1 python tosca/run.py [email protected] /opt/cord_profile/openstack-compute.yaml $ docker exec rcord_xos_ui_1 python tosca/run.py [email protected] /opt/cord_profile/openstack-compute-vtn. yaml

8. Check if all compute nodes are in COMPLETE state and all necessary networks are created.

$ ssh -p 8101 karaf@onos-cord onos> cordvtn-nodes

Hostname Management IP Data IP Data Iface Br-int State

compute-01 10.1.1.122/24 10.2.2.28/24 veth1 of:0000000000000001 COMPLETE

compute-02 10.1.1.126/24 10.2.2.29/24 veth1 of:0000000000000002 COMPLETE

Total 2 nodes

onos> cordvtn-networks

ID Name Type VNI Subnet Service IP

5302bef6-a070-4fb9-a6b9-bea721abdcba management MANAGEMENT_LOCAL 1073 172.27.0.0 /24 172.27.0.1

bfa7366e-2622-416a-bed0-9e310d76530e mysite_vsg-access VSG 1029 10.0.2.0 /24 10.0.2.1

d04edff5-9666-4613-915e-f6e42aa7cd94 public PUBLIC 1009 10.6.1.192 /26 10.6.1.193

9. Run pod-test-playbook.yaml to bring up test VSG tenant and on-boarding exampleservice. Note that you'll need to disable "maas-test-client-install" role for successful test.

/root/cord/build/platform-install/pod-test-playbook.yml

Comment out the following lines #- name: Create test client # hosts: head

# become: yes # roles:

# - maas-test-client-install

$ cd /root/cord/build/platform-install; ansible-playbook -i inventory/rcord pod-test-playbook.yml

10. Check if VSG instance is up and running. You should be able to log-in to the VSG VM with the management IP(172.27.0.X) and the private key /root /cord_profile/key_import/vsg_rsa from the compute node.

(12)

References

[1] CORD platform install: https://github.com/opencord/platform-install