3.4 The OpenStack Cloud Network Infrastructure DeepDive
3.4.6 Controller Node
The Controller Node contains the majority of the infrastructure for the Cloud Networking such as the Horizon Dashboard and the database. The Compute node connects through the VXLAN tunnel to the Controller. Similarly to the Compute node, the controller.sh script file was configured to produce the results reported in the subsections below. All the outputs were saved in the controller-output-mar17.txt.
3.4.6.1 Physical ports
The following Table 3.13 shows the interfaces visible at the Controller Node level: port number, Name, MAC, IP, which can be displayed by using the command: $ ip a.
# in the list Name MAC IP
1 Lo 00:00:00:00:00:00 127.0.0.1/8 2 eth0 08:00:27:62:08:29 192.168.137.10/24 3 eth1 08:00:27:5f:9d:7b 4 eth2 08:00:27:82:cc:17 10.0.4.15/24 5 ovs-system 8e:84:89:13:9a:02 6 br-int 6e:ee:e6:50:5f:41 7 br-ex 08:00:27:5f:9d:7b 8 br-tun 56:54:ec:de:f8:4
Table 3.13:Controller Node ports list.
Other than the main physical interfaces of the controller (eth0, eth1 and eth2), there are three additional interfaces: Br-int, br-ex and br-tun, which are local interfaces of the OVS bridges. This indicates that similarly to the Compute node, this node uses OVS from the port ovs-system and the three interfaces that represent their three ovs bridges. As discussed in §3.2.4, br-ex is a key requirement in the controller of the cloud setup.
3.4.6.2 OVS
Table 3.14 lists the OVS commands used in the Controller Node, which outnumber the commands used in the Compute node. As the Controller node creates an extra OVS bridge, as described in §3.2.4, an additional list of OVS commands is used to collect the details of that bridge. Table 3.15 displays the OVS output of the commands listed in Table 3.14.
Command Gathered information Description
sudo ovs-vsctl list-br:. OVS bridges lists all OVS bridges created in the Node sudo ovs-vsctl list-ifaces br-int: Interfaces lists all the interfaces listed in br-int bridge. sudo ovs-vsctl list-ifaces br-tun: Interfaces lists all the interfaces listed in br-tun bridge. sudo ovs-vsctl list-ifaces br-ex: Interfaces lists all the interfaces listed in br-ex bridge sudo ovs-vsctl show: OVS bridges, Interfaces, shows all the OVS bridges and their
VLAN Tag, patched inter- faces
associated interfaces, Tags, patches and extra information.
sudo ovs-ofctl show br-int: Interfaces #, Name, MAC lists br-int interfaces, listed #, MAC sudo ovs-ofctl show br-ex: Interfaces #, Name, MAC lists br-ex interfaces, listed #, MAC sudo ovs-ofctl show br-tun: Interfaces #, Name, MAC lists br-tun interfaces, listed #, MAC
Table 3.14:Controller Node OVS Command List.
Table 3.15 combines the result from the above commands to represent the following information about bridges and interfaces as follows.
Important points derived:
1. Picking up the connection tracking where we stopped in the Compute node, the traffic transferred from the tunneling connection from the Compute node is received by the tunneling bridge (br-tun) in the Controller node.
2. The name of thetunneling interface vxlan-c0a8890b in br-tun matches the VXLAN interface in the Compute Node br-tun bridge as shown in Table 3.12. This means that vxlan-c0a8890b is one tunnel connecting the two nodes using the same tunnel code for each tenant traffic. 3. As in the Compute Node, br-tun and br-int are connected through the veth link using the
option peer. If the option column listed peer=interface-name info, then it is a veth linked (peered) with another veth from another bridge
4. Similarly, the bridges br-int and br-ex are connected through veth link in the same way phy-br-ex in br-ex bridge is linked to int-br-ex in br-int.
5. Br-int consists of another type of interface; some start with "tap" and others start with "qr". The numbering system in the Controller node is different form that of the Compute node, however they can be grouped using their tag number.
Bridge Br-ex
Port Tag Type option Listed # MAC
eth1 1 08:00:27:5f:9d:7b
br-ex internal LOCAL 08:00:27:5f:9d:7b
phy-br-ex patch peer=int-br-ex 2 fe:31:88:a5:85:a4
qg-3e2b0e76-9a internal 5
qg-c7b01624-a8 internal 4
qg-e3372ecf-1c internal 3
Bridge br-int
Port Tag Type option Listed # MAC
int-br-ex patch peer=phy-br-ex 1 12:bd:ad:92:77:5d patch-tun patch peer=patch-int 2 a2:88:a4:e4:78:fd
br-int internal LOCAL 6e:ee:e6:50:5f:41
qr-350d30d0-3b 1 internal 4 qr-6db257fa-4b 3 internal 8 qr-986e808c-9e 2 internal 6 tap750abee1-f3 2 internal 5 tapc532ed28-1c 3 internal 7 tape593bcca-7b 1 internal 3 Bridge br-tun
Port Tag Type option Listed # MAC
br-tun internal LOCAL 56:54:ec:de:f8:4c
patch-int peer=patch-tun 1 ee:92:70:fe:ba:fb
vxlan-c0a8890b
vxlan local_ip="192.168.137.10",remote_ip="192.168.137.11" 2 f2:c3:da:b7:43:10
Table 3.15:Controller Node OVS Bridges and Interface Details.
6. br-int interfaces are tagged with numbers (VLAN Tag). Figure 3.7 lists all ports with the same tag next to each other as they share the same VLAN since they belong to the same network.
7. At this point we have listed the ports and their tags in each bridge as shown in Figure 3.7 lists ports and their tags in each bridge without any additional information, such as IPs or where they are connected to cloud network infrastructure.
3.4.6.3 Namespaces
This was one of the hardest to find points in the DeepDive experiment as this information was not visible either through the Testbed or the Dashboard. Namespaces exist only in the Controller Node and their details can only be retrieved using the commands in the controller.sh script. Table 3.16 & 3.17contains namespace details collected using the following commands, where $x is qdhcp_ns or qrouter_ns:
• ip netns: lists namespaces (qdhcp, qrouter).
• sudo ip netns exec $x ifconfig: port name, MAC, IP, Bcast • sudo ip netns exec $x ip link: port list, port name
• sudo ip netns exec $x route -n: Destination, gateway, use iface.
• sudo ip netns exec $x ip route: default IP, GW dev, network, dev, scope link src. NameSpace qdhcp-2ef08b44-ef89-40ff-a4bc-4e11a70acd6d
Port info
Port list Port name IP MAC Bcast Scope Global
15 tapc532ed28-1c 192.168.20.2/24 fa:16:3e:91:60:23 192.168.20.255 tapc532ed28-1c Routing info (GW) Routing info (Network)
Gateway/default IP Use Iface/dev Net range Use Iface/dev Scope link src 192.168.20.1 tapc532ed28-1c 192.168.20.0 tapc532ed28-1c 192.168.20.2
NameSpace qdhcp-839e7c38-44bf-4865-b162-7f9324be8a8c Port info
Port list Port name IP MAC Bcast Scope Global
12 tap750abee1-f3 192.168.10.2/24 fa:16:3e:ea:1b:e1 192.168.10.255 tap750abee1-f3 Routing info (GW) Routing info (Network)
Gateway/default IP Use Iface/dev Net range Use Iface/dev Scope link src
192.168.10.1 tap750abee1-f3 192.168.10.0/24 tap750abee1-f3 192.168.10.2
NameSpace qdhcp-61389eed-95de-4290-989b-006e44c19413 Port info
Port list Port name IP MAC Bcast Scope Global
9 tape593bcca-7b 10.190.0.5/24 fa:16:3e:f5:47:a5 10.190.0.255 tape593bcca-7b Routing info (GW) Routing info (Network)
Gateway/default IP Use Iface/dev Net range Use Iface/dev Scope link src 10.190.0.1 tape593bcca-7b 10.190.0.0 /24 tape593bcca-7b 10.190.0.5
Table 3.16:Controller Node DHCP Namespaces Details.
Important points derived from Table 3.16 & 3.17:
1. Using the information from Table 3.16 & 3.17, the name spaces are linked to an interface that belongs to the OVS bridges as shown in Table 3.15. Therefore the namespaces are located between br-int and br-ex (see Figure 3.7) where the namespaces are linked to interfaces from both bridges.
2. qdhcp namespace is a DHCP server associated with each VN created by a tenant, which is linked to a tap interface in br-int where the ip address of the tap is the ip address to represents the DHCP server.
3. qrouter is a virtual router that the tenant creates and is linked to two interfaces; (1) the gateway (qr-# from br-int bridge) of the private VN the tenant created earlier and (2) an
NameSpace qrouter-b9211dbd-0f99-4912-9565-18793cad5f3b Port info
Port list Port name IP MAC Bcast Scope Global
16 qr-6db257fa-4b 192.168.20.1/24 fa:16:3e:11:18:87 192.168.20.255 qr-6db257fa-4b
17 qg-3e2b0e76-9a 192.168.100.13/24fa:16:3e:61:6b:f7 192.168.100.255 qg-3e2b0e76-9a 192.168.100.15/32 192.168.100.15
Routing info (GW) Routing info (Network)
Gateway/default IP Use Iface/dev Net range Use Iface/dev Scope link src
192.168.100.1 qg-3e2b0e76-9a 192.168.20.0/24192.168.100.0/24 qg-3e2b0e76-9aqr-6db257fa-4b 192.168.20.1192.168.100.13
NameSpace qrouter-84d7db89-8cf1-45fb-9383-b92f47b85117 Port info
Port list Port name IP MAC Bcast Scope Global
13 qr-986e808c-9e 192.168.10.1/24 fa:16:3e:66:3a:58 192.168.10.255 qr-986e808c-9e
14 qg-c7b01624-a8 192.168.100.12/24fa:16:3e:5a:08:14 192.168.100.255 qg-c7b01624-a8 192.168.100.14/32 192.168.100.14 qg-c7b01624-a8 Routing info (GW) Routing info (Network)
Gateway/default IP Use Iface/dev Net range Use Iface/dev Scope link src
192.168.100.1 qg-c7b01624-a8 192.168.10.0/24192.168.100.0/24 qg-c7b01624-a8qr-986e808c-9e 192.168.10.1192.168.100.12
NameSpace qrouter-870b7e86-5674-40b5-9ec4-43e0f81c29eb Port info
Port list Port name IP MAC Bcast Scope Global
10 qr-350d30d0-3b 10.190.0.1/24 fa:16:3e:21:f6:3e 10.190.0.255 qr-350d30d0-3b 11 qg-e3372ecf-1c 192.168.100.10/24 fa:16:3e:d0:0e:a9 192.168.100.255 qg-e3372ecf-1c 192.168.100.11/32 192.168.100.11 192.168.100.16/32 192.168.100.16 Routing info (GW) Routing info (Network)
Gateway/default IP Use Iface/dev Net range Use Iface/dev Scope link src
192.168.100.1 qg-e3372ecf-1c 10.190.0.0/24 qr-350d30d0-3b 10.190.0.1 192.168.100.0/24 qg-e3372ecf-1c 192.168.100.10
Table 3.17:Controller Node vRouter Namespaces Details.
interface (qg-# from be-ex) with the external network IP and qrouter. To provide the instances external access, NAT rules are used between the private and public subnets.
4. qg-# ports are associated with a list of IPs from the external network. The first IP from the list is usually with /24 is the IP for the port. The remaining list, usually with /32, are the floating IPs associated with the tenant instances(VMs) with private IPs belongs to the private network that the qr-# belong to.
5. A question remains of how to identify which elements belong to which tenant? a) Networks and IP address:
i. Every network shown in Table 3.6 is associated with a subnet. Admin owns 2 networks, ext-net and Network. Tenant 1 owns T1user1net and t1user2net.
ii. IPs from ext-net are shown only on the side of the br-ex bridge.
iii. As Network1 is created by Admin as a shared network, all the tenants can see Network1 and the Test tenant uses this network as its private network IPs for its instances.
iv. As shown in Table 3.9, VMs owned by a specific tenant and usera are assigned private IP addresses from the same range as in tap-# and qr-#. Moreover, the instances associated with floating IP addresses are listed within qg-# in qrouter-#. b) Owners of namespaces and ports:
i. The DHCP namespace is defined as qdhcp-network ID. The network ID attached to qdhcp can be matched with network ID in Table 3.6 shows ownership.
ii. The router namespace is defined as qrouter-router ID, hence, the data analysis of Table 3.6 prove the ownership of the router. Although Router1 is owned by Admin, and it is used by Test tenant, the router and the router floating gateway is not visible by the Test tenant. However,data obtained from the scripts output files indicates that Router 1 is connected to Network1, which is used by Test.
iii. As mentioned in §3.4.4 ports are defined as qr-/tap-/qg- first 10 hex numbers of port ID as has been explained before in .
iv. Ports defined for floating IP addresses are not presented with interfaces.Instead, they are associated with instances shown in Table 3.9. Ports assigned local IPs in VMs and associated with floating IP are listed in qrouter namespaces under qg-10 hex of port ID in Table 3.17.