Our use case requires networking for three unique VM workloads: one for Web Servers, another for Application Servers, and a third for the Database Servers. Because each one of these workloads uses a different VLAN ID, we need three unique port groups. You might wonder, however, why the port groups must be unique?
Each port group will carry one single VLAN ID because our guest workloads, the servers themselves, are not configured to handle VLAN tags. We need to configure the vSwitch to handle this on the workload’s behalf, which goes back to the Virtual Switch Tagging (VST) methodology outlined in Chapter 7, “How Virtual Switching Differs from Physical Switching.” If our workloads could handle VLAN tags, another option would be to use a single port group that carried all three VLAN IDs as a trunk, which would be the Virtual Guest Tagging (VGT) configuration.
Let’s begin by creating the port group for the Web Servers. To start, navigate to the Host and Clusters view in the vSphere Web Client, select host esx2, click on the Manage tab, Networking sub-tab, and then select the virtual switch named vSwitch0. Click the “Add Host Networking” link on vSwitch0, which looks like a little globe with a plus sign on it.
The results are shown in Figure 12.4.
Figure 12.4 Adding a new network to vSwitch0
154 CHAPTER 12 Standard vSwitch Design
Because we want to make a new port group for VMs, we want to select the “Virtual Machine Port Group for a Standard Switch” radio button.
The next prompt asks what the target device is. You can leave the default selection of
“Select an existing standard switch” highlighted with vSwitch0, as shown in Figure 12.5.
If you chose to rename vSwitch0, or created a new vSwitch, you would want to select that vSwitch instead of vSwitch0.
Figure 12.5 Selecting vSwitch0 as the target device
You are now ready to enter the important details for your VM port group. It really boils down to nothing more than a network label (the name) and VLAN ID. Since we’re start-ing with the Web Server port group, the values would be:
Q Network label: Web_192.168.100.x_V100
Q VLAN ID: 100
I’ve entered the values shown in Figure 12.6. Note that when you click on the VLAN ID box, a few premade selections will appear for None (0) and All (4095). You can safely ignore these values and enter your own value—in this case, it is 100.
The last step is to review the requested configuration and click Finish. Congratulations, you have created a VM port group, as shown in Figure 12.7!
Virtual Machine Traffi c 155
Figure 12.6 Entering the network label and VLAN ID for the Web Server port group
Figure 12.7 The Web Server port group is now part of vSwitch0
156 CHAPTER 12 Standard vSwitch Design
Note the gold line leading from the Web Server port group to the two physical adapters.
This indicates that the port group is able to use either network adapter for passing along traffic.
You’re not done yet—there are still two more port groups to create. We’ve gone ahead and created them on vSwitch0 and show the final configuration in Figure 12.8. You’ll need to repeat the process in this section for the other two VM port groups.
Figure 12.8 All the VM port groups have been created on vSwitch0
NOTE
Delete the “VM Network” port group if you’re not going to rename and use it for some-thing else. No sense having a name that doesn’t match the naming convention.
Failover Order
By default, any new port group created will use the policies inherited by the vSwitch itself.
For vSwitch0, the policy is to actively use all of the network adapters. We’re going to leave
Virtual Machine Traffi c 157
this setting as-is for the VM port groups and let the VMs use either of the two available network adapters. In fact, Figure 12.9 provides a view of the default teaming and failover policies for vSwitch0. As you can see, both vmnic0 and vmnic1 are listed as Active adapters for vSwitch0, meaning they will both be used.
Figure 12.9 Default teaming and failover policies for vSwitch0
Why? With few exceptions, VMs are some of the least bandwidth hogging entities on a network. And, since there is no way to easily load balance them across the two network adapters, having two active uplinks with the default “route based on originating virtual port” gives them a solid chance at being spread out evenly across the uplinks.
REAL WORLD
There are some situations where you really should define specific network adapters as Active and others as Standby for VM traffic. If you have a specific use case, such as a workload that needs to use an uplink that goes to a specific network (such as a DMZ uplink), make sure to define a failover policy for that port group. It’s just that we rarely find this the case specifi-cally for VM throughput—they often consume a very small percentage (<10%) of the total bandwidth available to them.
We’ll come back to this in greater detail for the VMkernel ports, since they will be using specific failover orders to help alleviate specific traffic congestion scenarios.
158 CHAPTER 12 Standard vSwitch Design
VMkernel Ports
Now comes the slightly trickier part of creating a standard vSwitch for the Initech Corp environment: configuring VMkernel ports. With a standard vSwitch, creating a new VMkernel port automatically generates a new port group for the VMkernel port to live inside. VMs cannot use this port group. You can only have one VMkernel port residing inside of each port group.
We’ll tackle each of the VMkernel ports needed for this environment in the next few sec-tions. As a reminder from Chapter 11, “Lab Scenario,” we’re going to use the following VLAN IDs for the VMkernel networks:
Q Management traffic: 10.20.0.0 /16, routable, with a gateway of 10.20.0.1, on VLAN 20
Q vMotion traffic: 192.168.205.0 /24, non-routable, on VLAN 205
Q FT logging: 192.168.210.0 /24, non-routable, on VLAN 210
Q NFS traffic: 192.168.220.0/24, non-routable, on VLAN 220
Management
The Management VMkernel port is commonly used as the means to manage a host. If you were to ping a host by its Fully Qualified Domain Name (FQDN), the IP address you received in response should be the one mapped to the Management VMkernel port. It’s also what we commonly use when performing work via Secure Shell (SSH).
NOTE
Enabling “Management traffic” on a VMkernel port, such as what is shown in Figures 12.10 and 12.14, tell vSphere High Availability (HA) which VMkernel port to use for heart-beats. It’s not actually required for managing the host. Still, it’s often best to leave it on for whichever VMkernel port you wish to use to manage the host, which is vmk0 by default.
If you look carefully at vSwitch0, you’ll notice that the Management Network port group is in fact just a container that houses a single VMkernel port called vmk0 (see Figure 12.10). This is used for your host management and is responsible for being the default gateway for any unknown VMkernel traffic that doesn’t match a network mapped to any other VMkernel port. Don’t let that confuse you—the default gateway is not used for VM traffic in any way. VMs will have a default gateway configured within their operating sys-tem. The VMkernel default gateway is there only for other VMkernel ports.
VMkernel Ports 159
Figure 12.10 vmk0 is the default VMkernel port used for host management
You cannot put VMs inside of this port group because it is made specifically for a VMker-nel port. This helps avoid any confusion when an administrator, for example, you, is look-ing for a network to place a VM on.
NOTE
vmk0 is a special VMkernel port generated by the hypervisor at the time of installation.
Unlike other VMkernel ports, vmk0 uses the first network adapter’s hardware MAC
address—also known as the burned-in address (BIA)—as its own MAC address. We find that it’s often best to leave vmk0 as the Management VMkernel port and not try to fiddle with it to become more than that, as bad things can (and do) happen.
It seems like most of your work is done for the Management VMkernel port, right? Obvi-ously this port is operational, because otherwise the host would not be able to connect to the vCenter Server. However, it’s important to review the failover order for every VMker-nel port. If you edit the settings of the Management Network port group (which contains vmk0), you’ll see a screen like that shown in Figure 12.11.
160 CHAPTER 12 Standard vSwitch Design
Figure 12.11 The failover policy settings for the Management Network
Hopefully, you’ll see the same issue here that we see—vmnic0 is set to Active, and vmnic1 is set to Unused. This means that a failure of vmnic0 will bring down the vmk0 port, even if vmnic1 is still operational. We definitely don’t want that; so let’s fix it by modifying the failover order for the port group.
In order to balance out the VMkernel ports over a variety of different network adapters, we’re going to purposely mark specific network adapters as Active and others as Standby.
This is to help minimize traffic on any one specific network adapter. It might not make total sense now, but we will review the failover settings with each VMkernel port on a one-by-one basis here, and then go over the entire logical configuration in the final segment of this chapter. When you see all of their failover settings together, it should make more sense.
For the Management VMkernel port, vmk0, we’re going to set vmnic0 as Active and vmnic1 as Standby. Select vmnic1 and click the blue up arrow to change vmnic1 from Unused to Standby, as shown in Figure 12.12.
That’s much better. Now, if vmnic0 fails for some reason, vmk0 on the Management Net-work will be moved over to vmnic1 and the host will remain accessible on the netNet-work.
When vmnic0 is repaired, vmk0 will move back over to vmnic0 due to the fact that the Failback option is set to Yes. We have introduced redundancy to this network, which is always a good thing to help avoid frantic calls in the middle of the night.
VMkernel Ports 161
Figure 12.12 Marking vmnic1 as a Standby adapter for the Management Network
REAL WORLD
You might also choose to set failback to “No” for the Management VMkernel port to reduce the risk of a “Lights On” switching event during physical network maintenance. This occurs when the upstream switch is restarted, and then begins initiating the startup process and powers on the physical network port. vSphere might incorrectly believe that the port is up and available while the upstream switch is still going through the boot process and try to erroneously use the port for Management traffic.
Note that we’re not talking about a blocked port via Spanning Tree Protocol (STP). Per the Portfast recommended practice mentioned in Chapter 4, “Advanced Layer 2,” you should have already configured all your upstream switch ports connected to the vSphere hosts to immediately transition to a forwarding state.
vMotion
Whenever a VM is whisked away from one vSphere host to another, one or more VMkernel ports with the vMotion service tag will be used to transmit the VM’s running state between them. As you might imagine, the amount of traffic traversing the vMotion VMkernel port is somewhat bursty. When no vMotions are occurring, the traffic being
162 CHAPTER 12 Standard vSwitch Design
sent and received is zero. When vMotions are occurring, there is a whole bunch of traffic being sent. This makes vMotion a somewhat difficult traffic to plan for in new environ-ments where you’re not sure just how much workload migration will be taking place.
NOTE
Traditionally, one VMkernel port was created on each host for vMotion traffic, although the concept of multiple-NIC vMotion was introduced with vSphere 5.0. In this chapter, Initech Corp only has two network adapters, and as such, we don’t want to saturate both of them with vMotion traffic simultaneously. However, we do cover multiple-NIC vMotion in detail in Chapter 19, “Multi-NIC vMotion Architecture.”
One way to help mitigate the bursty nature of vMotion is to use a failover order to place it on a specific network adapter. We’re actually going to ensure that vMotion is Active on vmnic1 and Standby on vmnic0, which is the opposite order we used with the Man-agement traffic in the previous section. This keeps the two from fighting on a standard vSwitch unless we’re in a scenario where one of the two network adapters has failed.
To begin with, let’s create the VMkernel port for vMotion by selecting the Add Network-ing function on vSwitch0—the same function we used to add a VM port group earlier.
This time, however, choose the “VMkernel Network Adapter” as your connection type as shown in Figure 12.13.
Figure 12.13 Adding a new VMkernel port to vSwitch0 for vMotion traffic
VMkernel Ports 163
Ensure that the target device is vSwitch0 and continue forward. You’ll be greeted with a screen asking for a variety of port properties. Let’s review the properties:
Q Network label: If you look back at the “Naming Conventions” section, you’ll see we decided to call this port group “vMotion.”
Q VLAN ID: We’ve already selected VLAN 205 for vMotion, so enter 205 here.
Q IP settings: To keep things simple, we’re going to select IPv4, although if you are feeling adventurous in your environment, you can feel free to opt for IPv6.
Q TCP/IP stack: Use the “Default” stack, as we have no need to use any custom gate-ways for traffic. (Our vMotion isn’t even routable on our network.)
Q Enabled services: Select the “vMotion traffic” checkbox to allow the hypervisor to use this VMkernel port for vMotion. If you forget to check this box, the host will not be marked as a valid source to send or receive VMs within the vSphere Cluster.
You can also review the settings in Figure 12.14.
Figure 12.14 Port properties for the vMotion VMkernel port
In the next screen, we need to enter an IP address for this VMkernel port. That’s different from what you might remember in the VM port group creation process. This is because a VM port group is just a housing for VMs, each of which have their own IP address, while a VMkernel port is used by the host to communicate on the network for its services.
164 CHAPTER 12 Standard vSwitch Design
In the case of Initech Corp, we’ve already been assigned a network subnet of
192.168.205.0/24 to use for vMotion traffic. To make life easier, we’ll pick 192.168.205.2 since the host we’re configuring is called esx2. The CIDR notation mask of /24 won’t work for the vSphere Web Client—we’ll have to provide a full dotted decimal mask. A /24 translates to 255.255.255.0. Thus, your screen should look something like that shown in Figure 12.15.
Figure 12.15 IPv4 settings for the vMotion VMkernel port
Review your settings and click Finish to complete the vMotion VMkernel port creation. If all was successful, you should see that your vSwitch0 now has two VMkernel ports similar to that shown in Figure 12.16.
However, notice the failover policy for the new vMotion network uses both vmnic0 and vmnic1 as Active adapters? We need to fix that and mark vmnic0 as a Standby adapter.
Repeat the steps you went through with the Management Network, with two differences:
Enable the Override checkbox for Failover order, and make sure vmnic0 is Standby and vmnic1 is Active.
Figure 12.17 shows that vmnic0 is a standby NIC for our vMotion VMkernel port.
We’ve now made sure that vMotion will primarily use the vmnic1 adapter, and if it fails, will switch over to the vmnic0 adapter. Again, introducing redundancy is a great thing that should be done whenever possible.
VMkernel Ports 165
Figure 12.16 A list of VMkernel ports in use on vSwitch0
Figure 12.17 Marking vmnic0 as the Standby adapter for vMotion
166 CHAPTER 12 Standard vSwitch Design