To achieve the maximum performance for the guest workloads, consideration must be given to closely aligning a VM’s vCPUs presentation with the physical CPU topology.
A physical server could contain anywhere from one to 32 CPU sockets. A physical CPU socket and its associated memory, which is closely aligned in proximity on the server mainboard, is called a Non-Uniform Memory Access, or NUMA, node. Note that some modern CPUs have multiple NUMA nodes on the same CPU package and are commonly referred to as cluster-on-die topologies. For the purposes of this document, we will refer to the CPU socket and the associated memory as a NUMA node.
If a VM’s CPU or memory spans multiple NUMA nodes, but this “awareness” of the underlying hardware topology is not available, the workload’s performance can degrade. Any time a CPU or memory request is initiated between physical NUMA nodes, it can slow down that command. The degradation rate varies based on the hardware architecture and could be negligible up to quite substantial in nature.
The goal is to closely align the VM with the physical hardware topology to maximize the VM’s speed. The presentation of this alignment is critical to help NUMA-aware applications make the most of the VM’s resources. It also allows for certain configurations to leverage a multi-NUMA configuration to boost speed.
The rule of thumb is relatively straightforward. If a VM’s CPU and RAM allocation “fits” inside one physical CPU socket’s core and memory configuration (Hyperthreading ignored), then configure the VM for one socket with that number of vCPU cores. For example, the physical server this VM resides on has two CPU sockets, each with eight physical cores, and 384GB of RAM. This configuration yields a NUMA node with eight cores and 192GB of RAM.
This VM was configured with eight vCPUs and 8GB of RAM, which comfortably “fits” inside one physical NUMA node as shown.
If the VM is scaled up and exceeds the boundaries of the physical NUMA node, the VM should be subdivided to “fit” across multiple physical sockets. For example, if this VM were reconfigured for 16 virtual processors while the host remained a two physical socket by twelve cores per socket architecture, a 16 CPU core footprint will not “fit” inside one physical socket, even though the memory footprint will. As a result, configure the VM for two sockets, each with eight vCPUs. The “Use Hardware Topology” button, as shown in Figure 17 can help you properly detect and align the VM’s vCPU allocation appropriately.
If Dynamic Memory is enabled, vNUMA will not be extended into the VM. A single vSocket will be presented, no matter if the VM “fits” within one physical NUMA node or not and could deliver a performance penalty on the workload.
The rule of thumb is to minimize the number of NUMA nodes the VM resides on to just what is needed to sufficiently power the VM’s application workload.
For more information on Hyper-V and vNUMA, reference the Learn article “Hyper-V Virtual NUMA Overview”. While older, the contents are still applicable.