VMware Enhanced vMotion Compatibility (EVC) facilitates vMotion between different CPU generations through use of Intel Flex Migration and AMD-V Extended Migration technologies.
When enabled, EVC ensures that all CPUs within the cluster are vMotion compatible.
Interaction of EVC and hardware virtualization support
VMware's hypervisor is unique in that it supports a variety of execution modes depending on the capabilities of the underlying hardware. The VMkernel automatically selects the hypervisor execution mode that will deliver the best virtual machine performance given the capabilities of the hardware and type of guest OS. Virtualization acceleration features such as VT-x/AMD-V and RVI/EPT are available for use by the hypervisor independent of EVC or the EVC baseline and the VMkernel switches on the fly to whichever mode is the best performing one for the Guest OS as the virtual machine is migrated around the cluster.
What is The Benefit Of EVC?
Because EVC allows you to migrate virtual machines between different generations of CPUs, older and newer server generations can be mixed in the same cluster and still allow vMotion migration between these hosts. This makes adding new hardware into your existing
infrastructure easier and helps extend the value of your existing hosts.
How Does EVC Work?
EVC leverages a defined baseline that allows all the hosts in the cluster to advertise the same CPU feature set. The EVC baseline does not disable the features within a CPU, but indicates to a virtual machine that specific features are not available.
It is crucial to understand that EVC only focuses on CPU features specific to CPU generations, such as SIMD (SSE) or AMD-now instructions. EVC hides these CPU features from software running inside virtual machines by not advertising these features. This means that the features are still available and active, but they are not “publically broadcasted.”
When enabling EVC, a CPU baseline must be selected. This baseline represents a feature set of the selected CPU generation and exposes specific CPU generation features. When a virtual machine powers-on within an EVC cluster, this cluster’s baseline will be attached to the virtual machine until it powers-off.
Note
The EVC baseline is attached to the virtual machine until it powers off even if it is migrated to another EVC cluster.
If an ESXi host with a newer generation CPU is joined to the cluster, the baseline will automatically “hide” the CPU features that are new and unique to that CPU generation.
For example: A cluster contains ESXi hosts configured with Intel® Xeon® Core™ i7 CPUs, commonly known as Intel Nehalem. The baseline selected – Intel® "Nehalem" Generation – presents the cumulative features of the Intel® "Merom" Generation, Intel® "Penryn"
Generation and the Intel® "Nehalem" Generation to the virtual machine. This has the net effect of providing the standard Intel® "Merom" Generation features plus SSE4.1, SSE4.2, Popcount and RDTSCP features available to all the virtual machines
Figure 49: Intel EVC baselines
When an ESXi host with a Westmere (32nm) CPU joins the cluster, the additional CPU instruction sets like AES/AESNI and PCLMULQDQ will be suppressed automatically.
Will EVC Impact Application Performance?
It is possible, but highly unlikely, that an application running in a virtual machine would benefit from the instruction set hidden by EVC. However, keep in mind that enabling EVC may impact the performance of applications specifically written to benefit from these special instructions.
In general, it should be expected that your applications will incur no noticeable performance loss from using EVC. Although EVC hides CPU features from the virtual machine Guest OS and applications, these features do not influence performance for most software commonly found inside a virtual infrastructure. For example, EVC does not affect the number of instructions per second, number of cores, hardware acceleration, caching or other CPU features that most software uses.
After new CPU generations are released, software vendors might update or release new versions of their software to take advantage of the new instructions introduced by the latest CPU generations. Software development cycles always trail hardware development, resulting in a delayed implementation and utilization of new instruction sets.
Furthermore, new instruction sets often have specialized use cases, meaning that most new instructions are typically irrelevant to the business applications active in the virtual
infrastructure.
One exception is encryption and decryption acceleration using the AES-NI instruction set. Many popular applications that perform encryption, including SSL libraries, now utilize this instruction set in their latest releases. Depending on workload, using these AES instructions can provide over 500% performance improvement.
Communication between virtual machines utilizing mutual SSL for authentication and transport level security leverages this AES instruction set.
Building Block Approach
A very popular design methodology is the use of a building block strategy. Building blocks extend the concept of a framework and outline a pre-defined set of items or modules that allow for scalability while maintaining standardization. Cluster configuration is usually treated as a building block. While some companies buy one cluster at a time, others use a modular approach and buy a fixed subset of physical hosts, planning to expand their clusters gradually.
EVC allows mixing different generations of CPUs, allowing for future expansion of clusters and aligning with building block architectures.
One potential caveat of mixing hardware from different major generations in the same cluster can be fluctuating performance. Newer generation CPUs can offer a substantial performance increase over older generation CPUs, resulting in different progress of the virtual machine and its application. For example, 500MHz on Intel Xeon Nehalem EX from 2010 can result in quicker computations than 500 MHz on an Intel Xeon Core 2 “Merom” from 2006. Configurations like this can increase the complexity of performance troubleshooting. This is an extreme example, as most customers mix different but adjacent generations of processor hardware rather than combining radically different hardware into a single cluster.
EVC’s impact on DRS Automation of Fault Tolerance Virtual Machines
As of vSphere 4.1, EVC allows DRS to generate initial placement for Fault Tolerance (FT) enabled virtual machines and allows DRS to move them around during load balancing operations.
Without EVC enabled, FT primary virtual machines are powered on by DRS on their registered host and pinned there. DRS will automatically place the secondary virtual machine, but will never move primary and secondary virtual machines during load balancing operations.
Basic design principle
By enabling EVC, DRS-FT integration will be supported
Allowing DRS to control initial placement of FT-enabled virtual machines can lead to better virtual machine performance as DRS is able to select the most suitable host. Permitting DRS to incorporate FT-enabled virtual machines in load-balancing calculations helps DRS to reach steady state sooner. Fixed virtual machines impact load-balancing calculations and can
constrain DRS’s ability to achieve a proper load-balanced state. By allowing DRS to move the FT-enabled virtual machines, DRS has fewer restrictions to deal with and more virtual machine candidates to move around to find a better distribution. Furthermore, FT-DRS integration allows for hosts to be placed into maintenance mode because DRS will be able to evacuate enabled virtual machines. DPM becomes more effective because DRS is able to migrate FT-enabled virtual machines off the hosts that are not required to provide resources to virtual machines.
FT-Enabled Virtual Machine Initial Placement
When calculating initial placement recommendations, DRS selects a host for the primary virtual machine and a host for the secondary virtual machine. DRS defines a compatible host set that contains ESXi hosts which are capable of vMotioning the FT-enabled machines. EVC assists with defining the host compatibility set. By applying an EVC baseline, each ESXi host in the cluster presents an identical instruction set, resulting in a general host compatibility set. For example, with EVC enabled, if virtual machine VM1 powers up on host ESXi-02, its host compatibility set contains ESXi-01 and ESXi-03 as it can vMotion to these hosts. It is also true that VM1 can be hosted by ESXi-03 and vMotion to ESXi-01 and ESXi-02. Likewise, if powered-on at ESXi-01, it is able to vMotion to hosts ESXi-02 and ESXi-03.
Figure 50: Host compatibility set
Without EVC, DRS needs to determine the host compatibility set by computing a separate host-to-host compatibility for each host. For example, with 32 hosts in the cluster, the
calculation is N*(N-1), where N= number of hosts * (number of hosts - the host itself) = 32 *(32-1) = 992 compatibility checks. Besides the increased overhead, the power-on operation of the FT-enabled virtual machine would take an enormously long time if DRS is required to execute this number of compatibility checks.
During initial placement, DRS also ensures that the secondary virtual machine will not run on the same host as the primary virtual machine.
FT-Enabled Virtual Machine Load Balancing
As mentioned previously, both primary and secondary virtual machines can be migrated by DRS if EVC is enabled. However, DRS is still restricted when generating particular migration
recommendations for FT-enabled virtual machines: due to an internal anti-affinity rule, DRS is prevented from swapping places of the primary and secondary FT virtual machines and FT-enabled virtual machines are never allowed to run on the same host at any time, so DRS will not move a primary virtual machine to the host where its secondary is running and vice versa within one recommendation. In addition, DRS is not allowed to place more than four FT-enabled virtual machines on a single host and will take this into account during load-balancing calculations.
FT-Enabled Virtual Machine DRS Automation Settings
The DRS automation settings can be configured for an FT-enabled virtual machine. The secondary virtual machine inherits the primary virtual machine’s DRS automation settings, resulting in an identical configuration of both virtual machines. It is not possible to configure the automation setting of the primary and secondary virtual machines independently. If the automation setting is set to Disabled, vCenter powers on the primary and secondary virtual machines on their currently registered hosts and will not move them.
Enabling And Disabling EVC
EVC can be enabled even when virtual machines are active. Powered-on virtual machines will not block configuration of EVC for a cluster as long as the virtual machine itself is compatible with the desired EVC mode.
If EVC is disabled, the virtual machines continue to operate at the same EVC mode and are not forced to restart. If complete removal of the EVC mode is required, the virtual machine must go through a complete power-cycle; a reboot is not sufficient.
If EVC is disabled on a cluster containing FT-enabled virtual machines, their DRS automation settings are changed to Disabled and DRS will be unable to migrate the primary and secondary during load-balancing and maintenance mode operations. If EVC is enabled again, these virtual
machines will once again receive the default cluster DRS automation level.
Power Off Virtual Machine Instead Of Reboot
It is important to remember that EVC baselines are applied only during power-on operations. If the EVC cluster mode is changed, active virtual machines are required to complete a full power-cycle to receive the new baseline. A virtual machine continues to run if the EVC mode is
increased, for example from Intel® “Merom” Generation to Intel® “Nehalem” Generation, but will operate with the knowledge of the instruction set of the original Intel® “Merom”
Generation baseline until it is power-cycled, at which point the new (Intel® “Nehalem”
Generation) baseline will be propagated to the virtual machine.
EVC Requirements
To enable EVC on a cluster, the cluster must meet the following requirements:
All hosts in the cluster must have CPUs from a single vendor, either AMD or Intel.
All hosts in the cluster must have advanced CPU features, such as hardware
virtualization support (AMD-V or Intel VT) and AMD No eXecute (NX) or Intel eXecute Disable (XD) and must be enabled in the BIOS.
All hosts in the cluster should be configured for vMotion. See the section, Host Configuration Requirements for vMotion.
All hosts in the cluster must be connected to the same vCenter Server system.
In addition, all hosts in the cluster must have CPUs that support the EVC mode you want to enable. To check EVC support for a specific processor or server model, see the VMware Compatibility Guide.