diff --git a/doc/developer-guides/hld/hv-partitionmode.rst b/doc/developer-guides/hld/hv-partitionmode.rst index 26a6014e1..2d08d7668 100644 --- a/doc/developer-guides/hld/hv-partitionmode.rst +++ b/doc/developer-guides/hld/hv-partitionmode.rst @@ -3,13 +3,13 @@ Partition Mode ############## -ACRN is a type-1 hypervisor that supports running multiple guest operating +ACRN is a type 1 hypervisor that supports running multiple guest operating systems (OS). Typically, the platform BIOS/bootloader boots ACRN, and ACRN loads single or multiple guest OSes. Refer to :ref:`hv-startup` for details on the start-up flow of the ACRN hypervisor. -ACRN supports two modes of operation: Sharing mode and Partition mode. -This document describes ACRN's high-level design for Partition mode +ACRN supports two modes of operation: sharing mode and partition mode. +This document describes ACRN's high-level design for partition mode support. .. contents:: @@ -23,10 +23,10 @@ In partition mode, ACRN provides guests with exclusive access to cores, memory, cache, and peripheral devices. Partition mode enables developers to dedicate resources exclusively among the guests. However, there is no support today in x86 hardware or in ACRN to partition resources such as -peripheral buses (e.g. PCI). On x86 platforms that support Cache -Allocation Technology (CAT) and Memory Bandwidth Allocation(MBA), resources -such as Cache and memory bandwidth can be used by developers to partition -L2, Last Level Cache (LLC), and memory bandwidth among the guests. Refer to +peripheral buses (e.g., PCI). On x86 platforms that support Cache +Allocation Technology (CAT) and Memory Bandwidth Allocation (MBA), developers +can partition Level 2 (L2) cache, Last Level Cache (LLC), and memory bandwidth +among the guests. Refer to :ref:`hv_rdt` for more details on ACRN RDT high-level design and :ref:`rdt_configuration` for RDT configuration. @@ -34,15 +34,15 @@ L2, Last Level Cache (LLC), and memory bandwidth among the guests. Refer to ACRN expects static partitioning of resources either by code modification for guest configuration or through compile-time config options. All the devices exposed to the guests are either physical -resources or are emulated in the hypervisor. So, there is no need for a -device-model and Service OS. :numref:`pmode2vms` shows a partition mode +resources or are emulated in the hypervisor. There is no need for a +Device Model and Service VM. :numref:`pmode2vms` shows a partition mode example of two VMs with exclusive access to physical resources. .. figure:: images/partition-image3.png :align: center :name: pmode2vms - Partition Mode example with two VMs + Partition Mode Example with Two VMs Guest Info ********** @@ -51,12 +51,14 @@ ACRN uses multi-boot info passed from the platform bootloader to know the location of each guest kernel in memory. ACRN creates a copy of each guest kernel into each of the guests' memory. Current implementation of ACRN requires developers to specify kernel parameters for the guests as -part of guest configuration. ACRN picks up kernel parameters from guest +part of the guest configuration. ACRN picks up kernel parameters from the guest configuration and copies them to the corresponding guest memory. .. figure:: images/partition-image18.png :align: center + Guest Info + ACRN Setup for Guests ********************* @@ -65,9 +67,9 @@ Cores ACRN requires the developer to specify the number of guests and the cores dedicated for each guest. Also, the developer needs to specify -the physical core used as the Boot Strap Processor (BSP) for each guest. As +the physical core used as the bootstrap processor (BSP) for each guest. As the processors are brought to life in the hypervisor, it checks if they are -configured as BSP for any of the guests. If a processor is BSP of any of +configured as BSP for any of the guests. If a processor is the BSP of any of the guests, ACRN proceeds to build the memory mapping for the guest, mptable, E820 entries, and zero page for the guest. As described in `Guest info`_, ACRN creates copies of guest kernel and kernel @@ -78,7 +80,7 @@ events in chronological order. :align: center :name: partBSPsetup - Event Order for Processor Set Up + Event Order for Processor Setup Memory ====== @@ -103,7 +105,7 @@ E820 and Zero Page Info A default E820 is used for all the guests in partition mode. This table shows the reference E820 layout. Zero page is created with this -e820 info for all the guests. +E820 info for all the guests. +------------------------+ | RAM | @@ -146,9 +148,9 @@ host-bridge at BDF (Bus Device Function) 0.0:0 to each guest. Access to I/O - Passthrough Devices ========================= -ACRN, in partition mode, supports passing thru PCI devices on the +ACRN, in partition mode, supports passing through PCI devices on the platform. All the passthrough devices are exposed as child devices under -the virtual host bridge. ACRN does not support either passing thru +the virtual host bridge. ACRN does not support either passing through bridges or emulating virtual bridges. Passthrough devices should be statically allocated to each guest using the guest configuration. ACRN expects the developer to provide the virtual BDF to BDF of the @@ -158,11 +160,11 @@ configuration. Runtime ACRN Support for Guests ******************************* -ACRN, in partition mode, supports an option to passthrough LAPIC of the +ACRN, in partition mode, supports an option to pass through LAPIC of the physical CPUs to the guest. ACRN expects developers to specify if the -guest needs LAPIC passthrough using guest configuration. When guest +guest needs LAPIC passthrough using guest configuration. When the guest configures vLAPIC as x2APIC, and if the guest configuration has LAPIC -passthrough enabled, ACRN passes the LAPIC to the guest. Guest can access +passthrough enabled, ACRN passes the LAPIC to the guest. The guest can access the LAPIC hardware directly without hypervisor interception. During runtime of the guest, this option differentiates how ACRN supports inter-processor interrupt handling and device interrupt handling. This @@ -171,13 +173,14 @@ will be discussed in detail in the corresponding sections. .. figure:: images/partition-image16.png :align: center + LAPIC Passthrough Guest SMP Boot Flow =================== The core APIC IDs are reported to the guest using mptable info. SMP boot flow is similar to sharing mode. Refer to :ref:`vm-startup` -for guest SMP boot flow in ACRN. Partition mode guests startup is same as +for guest SMP boot flow in ACRN. Partition mode guests startup is the same as the Service VM startup in sharing mode. Inter-Processor Interrupt (IPI) Handling @@ -195,7 +198,7 @@ Guests With LAPIC Passthrough ACRN supports passthrough if and only if the guest is using x2APIC mode for the vLAPIC. In LAPIC passthrough mode, writes to the Interrupt Command -Register (ICR) x2APIC MSR is intercepted. Guest writes the IPI info, +Register (ICR) x2APIC MSR are intercepted. The guest writes the IPI info, including vector, and destination APIC IDs to the ICR. Upon an IPI request from the guest, ACRN does a sanity check on the destination processors programmed into the ICR. If the destination is a valid target for the guest, @@ -205,6 +208,7 @@ corresponding to the destination processor info in the ICR. .. figure:: images/partition-image14.png :align: center + IPI Handling for Guests With LAPIC Passthrough Passthrough Device Support ========================== @@ -224,6 +228,7 @@ passthrough devices. Refer to the `I/O`_ section below for more details. .. figure:: images/partition-image1.png :align: center + Configuration Space Access DMA --- @@ -247,12 +252,13 @@ ACRN supports I/O for passthrough devices with two restrictions. As the guest PCI sub-system scans the PCI bus and assigns a Guest Physical Address (GPA) to the MMIO BAR, ACRN maps the GPA to the address in the physical BAR of the passthrough device using EPT. The following timeline chart -explains how PCI devices are assigned to guest and BARs are mapped upon +explains how PCI devices are assigned to the guest and how BARs are mapped upon guest initialization. .. figure:: images/partition-image13.png :align: center + I/O for Passthrough Devices Interrupt Configuration ----------------------- @@ -265,21 +271,21 @@ INTx Support ACRN expects developers to identify the interrupt line info (0x3CH) from the physical BAR of the passthrough device and build an interrupt entry in -the mptable for the corresponding guest. As guest configures the vIOAPIC +the mptable for the corresponding guest. As the guest configures the vIOAPIC for the interrupt RTE, ACRN writes the info from the guest RTE into the physical IOAPIC RTE. Upon the guest kernel request to mask the interrupt, ACRN writes to the physical RTE to mask the interrupt at the physical -IOAPIC. When guest masks the RTE in vIOAPIC, ACRN masks the interrupt +IOAPIC. When the guest masks the RTE in vIOAPIC, ACRN masks the interrupt RTE in the physical IOAPIC. Level triggered interrupts are not supported. MSI Support ~~~~~~~~~~~ -Guest reads/writes to PCI configuration space for configuring MSI -interrupts using an address. Data and control registers are passthrough to +The guest reads/writes to the PCI configuration space to configure MSI +interrupts using an address. Data and control registers are passed through to the physical BAR of the passthrough device. Refer to `Configuration -space access`_ for details on how the PCI configuration space is emulated. +Space Access`_ for details on how the PCI configuration space is emulated. Virtual Device Support ====================== @@ -296,18 +302,19 @@ Interrupt Delivery Guests Without LAPIC Passthrough -------------------------------- -In partition mode of ACRN, interrupts stay disabled after a vmexit. The +In ACRN partition mode, interrupts stay disabled after a vmexit. The processor does not take interrupts when it is executing in VMX root mode. ACRN configures the processor to take vmexit upon external interrupt if the processor is executing in VMX non-root mode. Upon an external interrupt, after sending EOI to the physical LAPIC, ACRN -injects the vector into the vLAPIC of vCPU currently running on the -processor. Guests using Linux as kernel, uses vectors less than 0xECh +injects the vector into the vLAPIC of the vCPU currently running on the +processor. Guests using a Linux kernel use vectors less than 0xECh for device interrupts. .. figure:: images/partition-image20.png :align: center + Interrupt Delivery for Guests Without LAPIC Passthrough Guests With LAPIC Passthrough ----------------------------- @@ -320,7 +327,7 @@ Hypervisor IPI Service ====================== ACRN needs IPIs for events such as flushing TLBs across CPUs, sending virtual -device interrupts (e.g. vUART to vCPUs), and others. +device interrupts (e.g., vUART to vCPUs), and others. Guests Without LAPIC Passthrough -------------------------------- @@ -330,7 +337,7 @@ Hypervisor IPIs work the same way as in sharing mode. Guests With LAPIC Passthrough ----------------------------- -Since external interrupts are passthrough to the guest IDT, IPIs do not +Since external interrupts are passed through to the guest IDT, IPIs do not trigger vmexit. ACRN uses NMI delivery mode and the NMI exiting is chosen for vCPUs. At the time of NMI interrupt on the target processor, if the processor is in non-root mode, vmexit happens on the processor @@ -339,7 +346,7 @@ and the event mask is checked for servicing the events. Debug Console ============= -For details on how hypervisor console works, refer to +For details on how the hypervisor console works, refer to :ref:`hv-console`. For a guest console in partition mode, ACRN provides an option to pass @@ -356,16 +363,16 @@ Hypervisor Console ACRN uses the TSC deadline timer to provide a timer service. The hypervisor console uses a timer on CPU0 to poll characters on the serial device. To -support LAPIC passthrough, the TSC deadline MSR is passthrough and the local +support LAPIC passthrough, the TSC deadline MSR is passed through and the local timer interrupt is also delivered to the guest IDT. Instead of the TSC deadline timer, ACRN uses the VMX preemption timer to poll the serial device. Guest Console ============= -ACRN exposes vUART to partition mode guests. vUART uses vPIC to inject -interrupt to the guest BSP. In cases of the guest having more than one core, +ACRN exposes vUART to partition mode guests. vUART uses vPIC to inject an +interrupt to the guest BSP. If the guest has more than one core, during runtime, vUART might need to inject an interrupt to the guest BSP from -another core (other than BSP). As mentioned in section , ACRN uses NMI delivery mode for notifying the CPU running the BSP +another core (other than BSP). As mentioned in section `Hypervisor IPI +Service`_, ACRN uses NMI delivery mode for notifying the CPU running the BSP of the guest. diff --git a/doc/developer-guides/hld/images/partition-image18.png b/doc/developer-guides/hld/images/partition-image18.png index ec7f58f3c..ec19027a8 100644 Binary files a/doc/developer-guides/hld/images/partition-image18.png and b/doc/developer-guides/hld/images/partition-image18.png differ