diff --git a/doc/developer-guides/hld/hld-overview.rst b/doc/developer-guides/hld/hld-overview.rst index 17dff802f..94f6551fd 100644 --- a/doc/developer-guides/hld/hld-overview.rst +++ b/doc/developer-guides/hld/hld-overview.rst @@ -3,10 +3,9 @@ ACRN high-level design overview ############################### -ACRN is an open source reference hypervisor (HV) running on top of Intel -platforms(APL, KBL etc) for heterogeneous scenarios - like Software Defined -Cockpit (SDC) or In-Vehicle Experience (IVE) for automotive, HMI & Real-Time -OS for industry . ACRN provides embedded hypervisor vendors with a reference +ACRN is an open source reference hypervisor (HV) that runs on top of Intel +platforms (APL, KBL, etc) for heterogeneous scenarios such as the Software Defined +Cockpit (SDC), or the In-Vehicle Experience (IVE) for automotives, or HMI & Real-Time OS for industry. ACRN provides embedded hypervisor vendors with a reference I/O mediation solution with a permissive license and provides auto makers and industry users a reference software stack for corresponding use. @@ -21,21 +20,21 @@ system, the In-vehicle Infotainment (IVI) system, and one or more rear seat entertainment (RSE) systems. Each system runs as a VM for better isolation. -The Instrument Control (IC) system manages graphics display of +The Instrument Control (IC) system manages graphic displays of - driving speed, engine RPM, temperature, fuel level, odometer, trip mile, etc. - alerts of low fuel or tire pressure -- rear-view camera(RVC) and surround-camera view for driving assistance. +- rear-view camera (RVC) and surround-camera view for driving assistance In-Vehicle Infotainment ======================= -A typical In-Vehicle Infotainment (IVI) system would support: +A typical In-Vehicle Infotainment (IVI) system supports: -- Navigation systems; -- Radios, audio and video playback; -- Mobile devices connection for calls, music and applications via voice - recognition and/or gesture Recognition / Touch. +- Navigation systems +- Radios, audio and video playback +- Mobile devices connection for calls, music, and applications via voice + recognition and/or gesture Recognition / Touch - Rear-seat RSE services such as: - entertainment system @@ -44,7 +43,7 @@ A typical In-Vehicle Infotainment (IVI) system would support: connectivity) ACRN supports guest OSes of Clear Linux OS and Android. OEMs can use the ACRN -hypervisor and Linux or Android guest OS reference code to implement their own +hypervisor and the Linux or Android guest OS reference code to implement their own VMs for a customized IC/IVI/RSE. Industry Usage @@ -56,7 +55,7 @@ A typical industry usage would include one windows HMI + one RT VM: - RT VM which running specific RTOS on it to provide capability of handling real-time workloads like PLC control -ACRN supports guest OS of Windows, at the meantime, ACRN added/is adding a +ACRN supports guest OS of Windows; ACRN has also added/is adding a series features to enhance its real-time performance then meet hard-RT KPI for its RT VM: @@ -84,17 +83,17 @@ Recommended Memory: 4GB, 8GB preferred. ACRN Architecture ***************** -ACRN is a type-I hypervisor, running on top of bare metal. It supports +ACRN is a type-I hypervisor that runs on top of bare metal. It supports Intel APL & KBL platforms and can be easily extended to support future platforms. ACRN implements a hybrid VMM architecture, using a privileged -service VM running the service OS (SOS) to manage I/O devices and +service VM to manage I/O devices and provide I/O mediation. Multiple user VMs can be supported, running Clear -Linux OS or Android OS as the user OS (UOS). +Linux OS or Android OS as the User VM. ACRN 1.0 ======== -ACRN 1.0 is designed mainly for auto use case - like SDC & IVI. +ACRN 1.0 is designed mainly for auto use cases such as SDC & IVI. Instrument cluster applications are critical in the Software Defined Cockpit (SDC) use case, and may require functional safety certification @@ -104,13 +103,13 @@ and minimizing potential interference. However, running the IC system in a separate VM introduces additional latency for the IC applications. Some country regulations requires an IVE system to show a rear-view camera (RVC) within 2 seconds, which is difficult to achieve if a -separate instrument cluster VM is started after the SOS is booted. +separate instrument cluster VM is started after the User VM is booted. :numref:`overview-arch1.0` shows the architecture of ACRN 1.0 together with -IC VM and service VM. As shown, SOS owns most of platform devices and -provides I/O mediation to VMs. Some of the PCIe devices function as -pass-through mode to UOSs according to VM configuration. In addition, -the SOS could run the IC applications and HV helper applications such +the IC VM and Service VM. As shown, the Service VM owns most of platform devices and +provides I/O mediation to VMs. Some of the PCIe devices function as a +pass-through mode to User VMs according to VM configuration. In addition, +the Service VM could run the IC applications and HV helper applications such as the Device Model, VM manager, etc. where the VM manager is responsible for VM start/stop/pause, virtual CPU pause/resume,etc. @@ -124,21 +123,21 @@ ACRN 2.0 ======== ACRN 2.0 is extending ACRN to support pre-launched VM (mainly for safety VM) -and Real Time VM. +and Real-Time (RT) VM. -:numref:`overview-arch2.0` shows the architecture of ACRN 2.0, the main difference -compare to ACRN 1.0 is that: +:numref:`overview-arch2.0` shows the architecture of ACRN 2.0; the main difference +compared to ACRN 1.0 is that: -- a pre-launched VM is supported in ACRN 2.0, with isolated resource including - CPU, memory and HW devices etc +- a pre-launched VM is supported in ACRN 2.0, with isolated resources, including + CPU, memory, and HW devices etc -- add a few necessary device emulation in hypervisor like vPCI, vUART to avoid +- ACRN 2.0 adds a few necessary device emulations in hypervisor like vPCI and vUART to avoid interference between different VMs -- support RT VM for a post-launched User VM, with assistant features like LAPIC - pass-thru, PMD virtio driver +- ACRN 2.0 supports RT VM for a post-launched User VM, with assistant features like LAPIC + pass-thru and PMD virtio driver -ACRN 2.0 is still WIP, and some of its features already merged in the master. +ACRN 2.0 is still WIP, and some of its features are already merged in the master. .. figure:: images/over-image35.png :align: center @@ -151,37 +150,37 @@ ACRN 2.0 is still WIP, and some of its features already merged in the master. Device Emulation ================ -ACRN adopts various approaches for emulating devices for UOS: +ACRN adopts various approaches for emulating devices for the User VM: - **Emulated device**: A virtual device using this approach is emulated in - the SOS by trapping accesses to the device in UOS. Two sub-categories + the Service VM by trapping accesses to the device in the User VM. Two sub-categories exist for emulated device: - fully emulated, allowing native drivers to be used - unmodified in the UOS, and + unmodified in the User VM, and - para-virtualized, requiring front-end drivers in - the UOS to function. + the User VM to function. -- **Pass-through device**: A device passed through to UOS is fully - accessible to UOS without interception. However, interrupts +- **Pass-through device**: A device passed through to the User VM is fully + accessible to the User VM without interception. However, interrupts are first handled by the hypervisor before - being injected to the UOS. + being injected to the User VM. - **Mediated pass-through device**: A mediated pass-through device is a hybrid of the previous two approaches. Performance-critical - resources (mostly data-plane related) are passed-through to UOSes and + resources (mostly data-plane related) are passed-through to the User VMs and others (mostly control-plane related) are emulated. I/O Emulation ------------- -The device model (DM) is a place for managing UOS devices: it allocates -memory for UOSes, configures and initializes the devices shared by the +The device model (DM) is a place for managing User VM devices: it allocates +memory for the User VMs, configures and initializes the devices shared by the guest, loads the virtual BIOS and initializes the virtual CPU state, and -invokes hypervisor service to execute the guest instructions. +invokes the hypervisor service to execute the guest instructions. The following diagram illustrates the control flow of emulating a port -I/O read from UOS. +I/O read from the User VM. .. figure:: images/over-image29.png :align: center @@ -191,18 +190,18 @@ I/O read from UOS. :numref:`overview-io-emu-path` shows an example I/O emulation flow path. When a guest executes an I/O instruction (port I/O or MMIO), an VM exit -happens. HV takes control, and executes the request based on the VM exit +happens. The HV takes control and executes the request based on the VM exit reason ``VMX_EXIT_REASON_IO_INSTRUCTION`` for port I/O access, for -example. HV will then fetch the additional guest instructions, if any, +example. The HV will then fetch the additional guest instructions, if any, and processes the port I/O instructions at a pre-configured port address (in ``AL, 20h`` for example), and place the decoded information such as the port I/O address, size of access, read/write, and target register into the I/O request in the I/O request buffer (shown in -:numref:`overview-io-emu-path`) and notify/interrupt SOS to process. +:numref:`overview-io-emu-path`) and then notify/interrupt the Service VM to process. -The virtio and HV service module (VHM) in SOS intercepts HV interrupts, +The virtio and HV service module (VHM) in the Service VM intercepts HV interrupts, and accesses the I/O request buffer for the port I/O instructions. It will -then check if there is any kernel device claiming ownership of the +then check to see if any kernel device claims ownership of the I/O port. The owning device, if any, executes the requested APIs from a VM. Otherwise, the VHM module leaves the I/O request in the request buffer and wakes up the DM thread for processing. @@ -227,7 +226,7 @@ violation*. DMA Emulation ------------- -Currently the only fully virtualized devices to UOS are USB xHCI, UART, +Currently the only fully virtualized devices to the User VM are USB xHCI, UART, and Automotive I/O controller. None of these require emulating DMA transactions. ACRN does not currently support virtual DMA. @@ -236,14 +235,14 @@ Hypervisor ACRN takes advantage of Intel Virtualization Technology (Intel VT). The ACRN HV runs in Virtual Machine Extension (VMX) root operation, -host mode, or VMM mode, while the SOS and UOS guests run +host mode, or VMM mode, while the Serivce and User VM guests run in VMX non-root operation, or guest mode. (We'll use "root mode" and "non-root mode" for simplicity). The VMM mode has 4 rings. ACRN runs HV in ring 0 privilege only, and leaves ring 1-3 unused. A guest -running in non-root mode, has its own full rings (ring 0 to 3). The -guest kernel runs in ring 0 in guest mode, while guest user land +running in non-root mode has its own full rings (ring 0 to 3). The +guest kernel runs in ring 0 in guest mode, while the guest user land applications run in ring 3 of guest mode (ring 1 and 2 are usually not used by commercial OS). @@ -269,13 +268,13 @@ used by commercial OS). - A layer siting on top of hardware management enables virtual CPUs (or vCPUs), leveraging Intel VT. A vCPU loop runs a vCPU in non-root mode and handles VM exit events triggered by the vCPU. - This layer handles CPU and memory related VM + This layer handles CPU and memory-related VM exits and provides a way to inject exceptions or interrupts to a vCPU. - On top of vCPUs are three components for device emulation: one for emulation inside the hypervisor, another for communicating with - SOS for mediation, and the third for managing pass-through + the Service VM for mediation, and the third for managing pass-through devices. - The highest layer is a VM management module providing @@ -285,31 +284,31 @@ used by commercial OS). hypervisor, including encryption algorithms, mutual-exclusion primitives, etc. -There are three ways that the hypervisor interacts with SOS: -VM exits (including hypercalls), upcalls, and through the I/O request buffer. -Interaction between the hypervisor and UOS is more restricted, including +There are three ways that the hypervisor interacts with the Service VM: +the VM exits (including hypercalls), upcalls, and through the I/O request buffer. +Interaction between the hypervisor and the User VM is more restricted, including only VM exits and hypercalls related to trusty. -SOS -*** +Service VM +********** -SOS (Service OS) is an important guest OS in the ACRN architecture. It -runs in non-root mode, and contains many critical components including VM -manager, device model (DM), ACRN services, kernel mediation, and virtio -and hypercall module (VHM). DM manages UOS (User OS) and -provide device emulation for it. The SOS also provides services -for system power lifecycle management through ACRN service and VM manager, +The Service VM is an important guest OS in the ACRN architecture. It +runs in non-root mode, and contains many critical components, including the VM +manager, the device model (DM), ACRN services, kernel mediation, and virtio +and hypercall modules (VHM). The DM manages the User VM and +provides device emulation for it. The User VMS also provides services +for system power lifecycle management through the ACRN service and VM manager, and services for system debugging through ACRN log/trace tools. DM == -DM (Device Model) is an user level QEMU-like application in SOS -responsible for creating an UOS VM and then performing devices emulation +DM (Device Model) is a user-level QEMU-like application in the Service VM +responsible for creating the User VM and then performing devices emulation based on command line configurations. -Based on a VHM kernel module, DM interacts with VM manager to create UOS -VM. It then emulates devices through full virtualization in DM user +Based on a VHM kernel module, DM interacts with VM manager to create the User +VM. It then emulates devices through full virtualization on the DM user level, or para-virtualized based on kernel mediator (such as virtio, GVT), or pass-through based on kernel VHM APIs. @@ -318,11 +317,11 @@ Refer to :ref:`hld-devicemodel` for more details. VM Manager ========== -VM Manager is an user level service in SOS handling UOS VM creation and +VM Manager is a user-level service in the Service VM handling User VM creation and VM state management, according to the application requirements or system power operations. -VM Manager creates UOS VM based on DM application, and does UOS VM state +VM Manager creates the User VM based on DM application, and does User VM state management by interacting with lifecycle service in ACRN service. Please refer to VM management chapter for more details. @@ -331,37 +330,37 @@ ACRN Service ============ ACRN service provides -system lifecycle management based on IOC polling. It communicates with -VM manager to handle UOS VM state, such as S3 and power-off. +system lifecycle management based on IOC polling. It communicates with the +VM manager to handle the User VM state, such as S3 and power-off. VHM === -VHM (virtio & hypercall module) kernel module is an SOS kernel driver -supporting UOS VM management and device emulation. Device Model follows +The VHM (virtio & hypercall module) kernel module is the Service VM kernel driver +supporting User VM management and device emulation. Device Model follows the standard Linux char device API (ioctl) to access VHM functionalities. VHM communicates with the ACRN hypervisor through hypercall or upcall interrupts. -Please refer to VHM chapter for more details. +Refer to the VHM chapter for more details. Kernel Mediators ================ Kernel mediators are kernel modules providing a para-virtualization method -for the UOS VMs, for example, an i915 gvt driver. +for the User VMs, for example, an i915 gvt driver. Log/Trace Tools =============== -ACRN Log/Trace tools are user level applications used to +ACRN Log/Trace tools are user-level applications used to capture ACRN hypervisor log and trace data. The VHM kernel module provides a middle layer to support these tools. Refer to :ref:`hld-trace-log` for more details. -UOS -*** +User VM +******* Currently, ACRN can boot Linux and Android guest OSes. For Android guest OS, ACRN provides a VM environment with two worlds: normal world and trusty @@ -370,10 +369,10 @@ security sensitive applications run in the trusty world. The trusty world can see the memory of normal world, but normal world cannot see trusty world. -Guest Physical Memory Layout - UOS E820 -======================================= +Guest Physical Memory Layout - User VM E820 +=========================================== -DM will create E820 table for a User OS VM based on these simple rules: +DM will create E820 table for a User VM based on these simple rules: - If requested VM memory size < low memory limitation (currently 2 GB, defined in DM), then low memory range = [0, requested VM memory @@ -386,13 +385,13 @@ DM will create E820 table for a User OS VM based on these simple rules: .. figure:: images/over-image13.png :align: center - UOS Physical Memory Layout + User VM Physical Memory Layout -UOS Memory Allocation -===================== +User VM Memory Allocation +========================= -DM does UOS memory allocation based on hugetlb mechanism by default. -The real memory mapping may be scattered in SOS physical +The DM does User VM memory allocation based on the hugetlb mechanism by default. +The real memory mapping may be scattered in the Service VM physical memory space, as shown in :numref:`overview-mem-layout`: .. figure:: images/over-image15.png @@ -400,15 +399,15 @@ memory space, as shown in :numref:`overview-mem-layout`: :name: overview-mem-layout - UOS Physical Memory Layout Based on Hugetlb + User VM Physical Memory Layout Based on Hugetlb -User OS's memory is allocated by Service OS DM application, it may come +The User VM's memory is allocated by Service OS DM application; it may come from different huge pages in Service OS as shown in :numref:`overview-mem-layout`. -As Service OS has full knowledge of these huge pages size, +As the Service VM has full knowledge of these huge pages size, GPA\ :sup:`SOS` and GPA\ :sup:`UOS`, it works with the hypervisor -to complete UOS's host-to-guest mapping using this pseudo code: +to complete the User VM's host-to-guest mapping using this pseudo code: .. code-block: none @@ -420,8 +419,8 @@ to complete UOS's host-to-guest mapping using this pseudo code: Virtual Slim bootloader ======================= -Virtual Slim bootloader (vSBL) is the virtual bootloader that supports -booting the UOS on the ACRN hypervisor platform. The vSBL design is +The Virtual Slim bootloader (vSBL) is the virtual bootloader that supports +booting the User VM on the ACRN hypervisor platform. The vSBL design is derived from Slim Bootloader. It follows a staged design approach that provides hardware initialization and payload launching that provides the boot logic. As shown in :numref:`overview-sbl`, the virtual SBL has an @@ -434,13 +433,13 @@ to boot Linux or Android guest OS. vSBL System Context Diagram -The vSBL image is released as a part of the Service OS (SOS) root -filesystem (rootfs). The vSBL is copied to UOS memory by the VM manager -in the SOS while creating the UOS virtual BSP of UOS. The SOS passes the -start of vSBL and related information to HV. HV sets guest RIP of UOS +The vSBL image is released as a part of the Service OS root +filesystem (rootfs). The vSBL is copied to the User VM memory by the VM manager +in the Service VM while creating the the User VM virtual BSP of the User VM. The Service VM passes the +start of vSBL and related information to HV. HV sets the guest RIP of the User VM's virtual BSP as the start of vSBL and related guest registers, and -launches the UOS virtual BSP. The vSBL starts running in the virtual -real mode within the UOS. Conceptually, vSBL is part of the UOS runtime. +launches the the User VM virtual BSP. The vSBL starts running in the virtual +real mode within the User VM. Conceptually, vSBL is part of the User VM runtime. In the current design, the vSBL supports booting Android guest OS or Linux guest OS using the same vSBL image. @@ -453,14 +452,13 @@ OVMF bootloader ======================= Open Virtual Machine Firmware (OVMF) is the virtual bootloader that supports -EFI boot of UOS on the ACRN hypervisor platform. +the EFI boot of the User VM on the ACRN hypervisor platform. -The OVMF is copied to UOS memory by the VM manager in the SOS while creating -the UOS virtual BSP of UOS. The SOS passes the start of OVMF and related -information to HV. HV sets guest RIP of UOS virtual BSP as the start of OVMF -and related guest registers, and launches the UOS virtual BSP. The OVMF starts -running in the virtual real mode within the UOS. Conceptually, OVMF is part of -the UOS runtime. +The OVMF is copied to the User VM memory by the VM manager in the Service VM while creating +the User VM virtual BSP of the User VM. The Service VM passes the start of OVMF and related +information to HV. HV sets guest RIP of the User VM virtual BSP as the start of OVMF +and related guest registers, and launches the the User VM virtual BSP. The OVMF starts +running in the virtual real mode within the the User VM. Conceptually, OVMF is part of the User VM runtime. Freedom From Interference ************************* @@ -491,12 +489,12 @@ the following mechanisms: Memory corruption can be a common failure mode. ACRN hypervisor properly sets up the memory-related hardware mechanisms to ensure that: - 1. SOS cannot access the memory of the hypervisor, unless explicitly - allowed, + 1. The Service VM cannot access the memory of the hypervisor, unless explicitly + allowed - 2. UOS cannot access the memory of SOS and the hypervisor, and + 2. The User VM cannot access the memory of the Service VM and the hypervisor - 3. The hypervisor does not unintendedly access the memory of SOS or UOS. + 3. The hypervisor does not unintendedly access the memory of the Serivce or User VM. - Destination of external interrupts are set to be the physical core where the VM that handles them is running. @@ -511,9 +509,9 @@ the following mechanisms: that runs the vCPU where virtual interrupts will be injected. 2. The hypervisor maintains statistics on the total number of received - interrupts to SOS via a hypercall, and has a delay mechanism to + interrupts to the Service VM via a hypercall, and has a delay mechanism to temporarily block certain virtual interrupts from being injected. - This allows SOS to detect the occurrence of an interrupt storm and + This allows the Service VM to detect the occurrence of an interrupt storm and control the interrupt injection rate when necessary. - Mitigation of DMA storm. @@ -539,10 +537,10 @@ CPU P-state & C-state ===================== In ACRN, CPU P-state and C-state (Px/Cx) are controlled by the guest OS. -The corresponding governors are managed in SOS/UOS for best power +The corresponding governors are managed in the Service/User VM for best power efficiency and simplicity. -Guest should be able to process the ACPI P/C-state request from OSPM. +Guests should be able to process the ACPI P/C-state request from OSPM. The needed ACPI objects for P/C-state management should be ready in ACPI table. @@ -577,22 +575,22 @@ it traps the power state transition request from guest and emulates it. :numref:`overview-pm-block` shows the basic diagram block for ACRN PM. The OSPM in each guest manages the guest power state transition. The -Device Model running in SOS traps and emulates the power state -transition of UOS (Linux VM or Android VM in -:numref:`overview-pm-block`). VM Manager knows all UOS power states and -notifies OSPM of SOS (Service OS in :numref:`overview-pm-block`) once -active UOS is in the required power state. +Device Model running in the Service VM traps and emulates the power state +transition of the User VM (Linux VM or Android VM in +:numref:`overview-pm-block`). VM Manager knows all User VM power states and +notifies the OSPM of the Service VM (Service OS in :numref:`overview-pm-block`) once +active the User VM is in the required power state. -Then OSPM of the SOS starts the power state transition of SOS which is +Then the OSPM of the Service VM starts the power state transition of the Service VM which is trapped to "Sx Agency" in ACRN, and it will start the power state transition. -Some details about the ACPI table for UOS and SOS: +Some details about the ACPI table for the User and Service VMs: -- The ACPI table in UOS is emulated by Device Model. The Device Model - knows which register the UOS writes to trigger power state +- The ACPI table in the User VM is emulated by the Device Model. The Device Model + knows which register the User VM writes to trigger power state transitions. Device Model must register an I/O handler for it. -- The ACPI table in SOS is passthru. There is no ACPI parser +- The ACPI table in the Service VM is passthru. There is no ACPI parser in ACRN HV. The power management related ACPI table is generated offline and hardcoded in ACRN HV.