github/acrn-hypervisor
1
0
Fork 0
mirror of https://github.com/projectacrn/acrn-hypervisor.git synced 2025-05-31 03:15:42 +00:00
Code Issues Projects Releases Wiki Activity

Doc: Final edits to the HLD Overview doc.

Browse Source 
Signed-off-by: Deb Taylor <deb.taylor@intel.com>
This commit is contained in:
Deb Taylor 2019-11-08 16:33:24 -05:00 committed by deb-intel
parent c4ecc92177
commit a578d15ad1
1 changed files with 123 additions and 125 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

248
doc/developer-guides/hld/hld-overview.rst
View File

@ -3,10 +3,9 @@
ACRN high-level design overview
###############################
ACRN is an open source reference hypervisor (HV) running on top of Intel
platforms(APL, KBL etc) for heterogeneous scenarios - like Software Defined
Cockpit (SDC) or In-Vehicle Experience (IVE) for automotive, HMI & Real-Time
OS for industry . ACRN provides embedded hypervisor vendors with a reference
ACRN is an open source reference hypervisor (HV) that runs on top of Intel
platforms (APL, KBL, etc) for heterogeneous scenarios such as the Software Defined
Cockpit (SDC), or the In-Vehicle Experience (IVE) for automotives, or HMI & Real-Time OS for industry. ACRN provides embedded hypervisor vendors with a reference
I/O mediation solution with a permissive license and provides auto makers and
industry users a reference software stack for corresponding use.
@ -21,21 +20,21 @@ system, the In-vehicle Infotainment (IVI) system, and one or more rear
seat entertainment (RSE) systems. Each system runs as a VM for better
isolation.
The Instrument Control (IC) system manages graphics display of
The Instrument Control (IC) system manages graphic displays of
- driving speed, engine RPM, temperature, fuel level, odometer, trip mile, etc.
- alerts of low fuel or tire pressure
- rear-view camera(RVC) and surround-camera view for driving assistance.
- rear-view camera (RVC) and surround-camera view for driving assistance
In-Vehicle Infotainment
=======================
A typical In-Vehicle Infotainment (IVI) system would support:
A typical In-Vehicle Infotainment (IVI) system supports:
- Navigation systems;
- Radios, audio and video playback;
- Mobile devices connection for calls, music and applications via voice
recognition and/or gesture Recognition / Touch.
- Navigation systems
- Radios, audio and video playback
- Mobile devices connection for calls, music, and applications via voice
recognition and/or gesture Recognition / Touch
- Rear-seat RSE services such as:
- entertainment system
@ -44,7 +43,7 @@ A typical In-Vehicle Infotainment (IVI) system would support:
connectivity)
ACRN supports guest OSes of Clear Linux OS and Android. OEMs can use the ACRN
hypervisor and Linux or Android guest OS reference code to implement their own
hypervisor and the Linux or Android guest OS reference code to implement their own
VMs for a customized IC/IVI/RSE.
Industry Usage
@ -56,7 +55,7 @@ A typical industry usage would include one windows HMI + one RT VM:
- RT VM which running specific RTOS on it to provide capability of handling
real-time workloads like PLC control
ACRN supports guest OS of Windows, at the meantime, ACRN added/is adding a
ACRN supports guest OS of Windows; ACRN has also added/is adding a
series features to enhance its real-time performance then meet hard-RT KPI
for its RT VM:
@ -84,17 +83,17 @@ Recommended Memory: 4GB, 8GB preferred.
ACRN Architecture
*****************
ACRN is a type-I hypervisor, running on top of bare metal. It supports
ACRN is a type-I hypervisor that runs on top of bare metal. It supports
Intel APL & KBL platforms and can be easily extended to support future
platforms. ACRN implements a hybrid VMM architecture, using a privileged
service VM running the service OS (SOS) to manage I/O devices and
service VM to manage I/O devices and
provide I/O mediation. Multiple user VMs can be supported, running Clear
Linux OS or Android OS as the user OS (UOS).
Linux OS or Android OS as the User VM.
ACRN 1.0
========
ACRN 1.0 is designed mainly for auto use case - like SDC & IVI.
ACRN 1.0 is designed mainly for auto use cases such as SDC & IVI.
Instrument cluster applications are critical in the Software Defined
Cockpit (SDC) use case, and may require functional safety certification
@ -104,13 +103,13 @@ and minimizing potential interference. However, running the IC system in
a separate VM introduces additional latency for the IC applications.
Some country regulations requires an IVE system to show a rear-view
camera (RVC) within 2 seconds, which is difficult to achieve if a
separate instrument cluster VM is started after the SOS is booted.
separate instrument cluster VM is started after the User VM is booted.
:numref:`overview-arch1.0` shows the architecture of ACRN 1.0 together with
IC VM and service VM. As shown, SOS owns most of platform devices and
provides I/O mediation to VMs. Some of the PCIe devices function as
pass-through mode to UOSs according to VM configuration. In addition,
the SOS could run the IC applications and HV helper applications such
the IC VM and Service VM. As shown, the Service VM owns most of platform devices and
provides I/O mediation to VMs. Some of the PCIe devices function as a
pass-through mode to User VMs according to VM configuration. In addition,
the Service VM could run the IC applications and HV helper applications such
as the Device Model, VM manager, etc. where the VM manager is responsible
for VM start/stop/pause, virtual CPU pause/resume,etc.
@ -124,21 +123,21 @@ ACRN 2.0
========
ACRN 2.0 is extending ACRN to support pre-launched VM (mainly for safety VM)
and Real Time VM.
and Real-Time (RT) VM.
:numref:`overview-arch2.0` shows the architecture of ACRN 2.0, the main difference
compare to ACRN 1.0 is that:
:numref:`overview-arch2.0` shows the architecture of ACRN 2.0; the main difference
compared to ACRN 1.0 is that:
- a pre-launched VM is supported in ACRN 2.0, with isolated resource including
CPU, memory and HW devices etc
- a pre-launched VM is supported in ACRN 2.0, with isolated resources, including
CPU, memory, and HW devices etc
- add a few necessary device emulation in hypervisor like vPCI, vUART to avoid
- ACRN 2.0 adds a few necessary device emulations in hypervisor like vPCI and vUART to avoid
interference between different VMs
- support RT VM for a post-launched User VM, with assistant features like LAPIC
pass-thru, PMD virtio driver
- ACRN 2.0 supports RT VM for a post-launched User VM, with assistant features like LAPIC
pass-thru and PMD virtio driver
ACRN 2.0 is still WIP, and some of its features already merged in the master.
ACRN 2.0 is still WIP, and some of its features are already merged in the master.
.. figure:: images/over-image35.png
:align: center
@ -151,37 +150,37 @@ ACRN 2.0 is still WIP, and some of its features already merged in the master.
Device Emulation
================
ACRN adopts various approaches for emulating devices for UOS:
ACRN adopts various approaches for emulating devices for the User VM:
- **Emulated device**: A virtual device using this approach is emulated in
the SOS by trapping accesses to the device in UOS. Two sub-categories
the Service VM by trapping accesses to the device in the User VM. Two sub-categories
exist for emulated device:
- fully emulated, allowing native drivers to be used
unmodified in the UOS, and
unmodified in the User VM, and
- para-virtualized, requiring front-end drivers in
the UOS to function.
the User VM to function.
- **Pass-through device**: A device passed through to UOS is fully
accessible to UOS without interception. However, interrupts
- **Pass-through device**: A device passed through to the User VM is fully
accessible to the User VM without interception. However, interrupts
are first handled by the hypervisor before
being injected to the UOS.
being injected to the User VM.
- **Mediated pass-through device**: A mediated pass-through device is a
hybrid of the previous two approaches. Performance-critical
resources (mostly data-plane related) are passed-through to UOSes and
resources (mostly data-plane related) are passed-through to the User VMs and
others (mostly control-plane related) are emulated.
I/O Emulation
-------------
The device model (DM) is a place for managing UOS devices: it allocates
memory for UOSes, configures and initializes the devices shared by the
The device model (DM) is a place for managing User VM devices: it allocates
memory for the User VMs, configures and initializes the devices shared by the
guest, loads the virtual BIOS and initializes the virtual CPU state, and
invokes hypervisor service to execute the guest instructions.
invokes the hypervisor service to execute the guest instructions.
The following diagram illustrates the control flow of emulating a port
I/O read from UOS.
I/O read from the User VM.
.. figure:: images/over-image29.png
:align: center
@ -191,18 +190,18 @@ I/O read from UOS.
:numref:`overview-io-emu-path` shows an example I/O emulation flow path.
When a guest executes an I/O instruction (port I/O or MMIO), an VM exit
happens. HV takes control, and executes the request based on the VM exit
happens. The HV takes control and executes the request based on the VM exit
reason ``VMX_EXIT_REASON_IO_INSTRUCTION`` for port I/O access, for
example. HV will then fetch the additional guest instructions, if any,
example. The HV will then fetch the additional guest instructions, if any,
and processes the port I/O instructions at a pre-configured port address
(in ``AL, 20h`` for example), and place the decoded information such as
the port I/O address, size of access, read/write, and target register
into the I/O request in the I/O request buffer (shown in
:numref:`overview-io-emu-path`) and notify/interrupt SOS to process.
:numref:`overview-io-emu-path`) and then notify/interrupt the Service VM to process.
The virtio and HV service module (VHM) in SOS intercepts HV interrupts,
The virtio and HV service module (VHM) in the Service VM intercepts HV interrupts,
and accesses the I/O request buffer for the port I/O instructions. It will
then check if there is any kernel device claiming ownership of the
then check to see if any kernel device claims ownership of the
I/O port. The owning device, if any, executes the requested APIs from a
VM. Otherwise, the VHM module leaves the I/O request in the request buffer
and wakes up the DM thread for processing.
@ -227,7 +226,7 @@ violation*.
DMA Emulation
-------------
Currently the only fully virtualized devices to UOS are USB xHCI, UART,
Currently the only fully virtualized devices to the User VM are USB xHCI, UART,
and Automotive I/O controller. None of these require emulating
DMA transactions. ACRN does not currently support virtual DMA.
@ -236,14 +235,14 @@ Hypervisor
ACRN takes advantage of Intel Virtualization Technology (Intel VT).
The ACRN HV runs in Virtual Machine Extension (VMX) root operation,
host mode, or VMM mode, while the SOS and UOS guests run
host mode, or VMM mode, while the Serivce and User VM guests run
in VMX non-root operation, or guest mode. (We'll use "root mode"
and "non-root mode" for simplicity).
The VMM mode has 4 rings. ACRN
runs HV in ring 0 privilege only, and leaves ring 1-3 unused. A guest
running in non-root mode, has its own full rings (ring 0 to 3). The
guest kernel runs in ring 0 in guest mode, while guest user land
running in non-root mode has its own full rings (ring 0 to 3). The
guest kernel runs in ring 0 in guest mode, while the guest user land
applications run in ring 3 of guest mode (ring 1 and 2 are usually not
used by commercial OS).
@ -269,13 +268,13 @@ used by commercial OS).
- A layer siting on top of hardware management enables virtual
CPUs (or vCPUs), leveraging Intel VT. A vCPU loop runs a vCPU in
non-root mode and handles VM exit events triggered by the vCPU.
This layer handles CPU and memory related VM
This layer handles CPU and memory-related VM
exits and provides a way to inject exceptions or interrupts to a
vCPU.
- On top of vCPUs are three components for device emulation: one for
emulation inside the hypervisor, another for communicating with
SOS for mediation, and the third for managing pass-through
the Service VM for mediation, and the third for managing pass-through
devices.
- The highest layer is a VM management module providing
@ -285,31 +284,31 @@ used by commercial OS).
hypervisor, including encryption algorithms, mutual-exclusion
primitives, etc.
There are three ways that the hypervisor interacts with SOS:
VM exits (including hypercalls), upcalls, and through the I/O request buffer.
Interaction between the hypervisor and UOS is more restricted, including
There are three ways that the hypervisor interacts with the Service VM:
the VM exits (including hypercalls), upcalls, and through the I/O request buffer.
Interaction between the hypervisor and the User VM is more restricted, including
only VM exits and hypercalls related to trusty.
SOS
***
Service VM
**********
SOS (Service OS) is an important guest OS in the ACRN architecture. It
runs in non-root mode, and contains many critical components including VM
manager, device model (DM), ACRN services, kernel mediation, and virtio
and hypercall module (VHM). DM manages UOS (User OS) and
provide device emulation for it. The SOS also provides services
for system power lifecycle management through ACRN service and VM manager,
The Service VM is an important guest OS in the ACRN architecture. It
runs in non-root mode, and contains many critical components, including the VM
manager, the device model (DM), ACRN services, kernel mediation, and virtio
and hypercall modules (VHM). The DM manages the User VM and
provides device emulation for it. The User VMS also provides services
for system power lifecycle management through the ACRN service and VM manager,
and services for system debugging through ACRN log/trace tools.
DM
==
DM (Device Model) is an user level QEMU-like application in SOS
responsible for creating an UOS VM and then performing devices emulation
DM (Device Model) is a user-level QEMU-like application in the Service VM
responsible for creating the User VM and then performing devices emulation
based on command line configurations.
Based on a VHM kernel module, DM interacts with VM manager to create UOS
VM. It then emulates devices through full virtualization in DM user
Based on a VHM kernel module, DM interacts with VM manager to create the User
VM. It then emulates devices through full virtualization on the DM user
level, or para-virtualized based on kernel mediator (such as virtio,
GVT), or pass-through based on kernel VHM APIs.
@ -318,11 +317,11 @@ Refer to :ref:`hld-devicemodel` for more details.
VM Manager
==========
VM Manager is an user level service in SOS handling UOS VM creation and
VM Manager is a user-level service in the Service VM handling User VM creation and
VM state management, according to the application requirements or system
power operations.
VM Manager creates UOS VM based on DM application, and does UOS VM state
VM Manager creates the User VM based on DM application, and does User VM state
management by interacting with lifecycle service in ACRN service.
Please refer to VM management chapter for more details.
@ -331,37 +330,37 @@ ACRN Service
============
ACRN service provides
system lifecycle management based on IOC polling. It communicates with
VM manager to handle UOS VM state, such as S3 and power-off.
system lifecycle management based on IOC polling. It communicates with the
VM manager to handle the User VM state, such as S3 and power-off.
VHM
===
VHM (virtio & hypercall module) kernel module is an SOS kernel driver
supporting UOS VM management and device emulation. Device Model follows
The VHM (virtio & hypercall module) kernel module is the Service VM kernel driver
supporting User VM management and device emulation. Device Model follows
the standard Linux char device API (ioctl) to access VHM
functionalities. VHM communicates with the ACRN hypervisor through
hypercall or upcall interrupts.
Please refer to VHM chapter for more details.
Refer to the VHM chapter for more details.
Kernel Mediators
================
Kernel mediators are kernel modules providing a para-virtualization method
for the UOS VMs, for example, an i915 gvt driver.
for the User VMs, for example, an i915 gvt driver.
Log/Trace Tools
===============
ACRN Log/Trace tools are user level applications used to
ACRN Log/Trace tools are user-level applications used to
capture ACRN hypervisor log and trace data. The VHM kernel module provides a
middle layer to support these tools.
Refer to :ref:`hld-trace-log` for more details.
UOS
***
User VM
*******
Currently, ACRN can boot Linux and Android guest OSes. For Android guest OS, ACRN
provides a VM environment with two worlds: normal world and trusty
@ -370,10 +369,10 @@ security sensitive applications run in the trusty world. The trusty
world can see the memory of normal world, but normal world cannot see
trusty world.
Guest Physical Memory Layout - UOS E820
=======================================
Guest Physical Memory Layout - User VM E820
===========================================
DM will create E820 table for a User OS VM based on these simple rules:
DM will create E820 table for a User VM based on these simple rules:
- If requested VM memory size < low memory limitation (currently 2 GB,
defined in DM), then low memory range = [0, requested VM memory
@ -386,13 +385,13 @@ DM will create E820 table for a User OS VM based on these simple rules:
.. figure:: images/over-image13.png
:align: center
UOS Physical Memory Layout
User VM Physical Memory Layout
UOS Memory Allocation
=====================
User VM Memory Allocation
=========================
DM does UOS memory allocation based on hugetlb mechanism by default.
The real memory mapping may be scattered in SOS physical
The DM does User VM memory allocation based on the hugetlb mechanism by default.
The real memory mapping may be scattered in the Service VM physical
memory space, as shown in :numref:`overview-mem-layout`:
.. figure:: images/over-image15.png
@ -400,15 +399,15 @@ memory space, as shown in :numref:`overview-mem-layout`:
:name: overview-mem-layout
UOS Physical Memory Layout Based on Hugetlb
User VM Physical Memory Layout Based on Hugetlb
User OS's memory is allocated by Service OS DM application, it may come
The User VM's memory is allocated by Service OS DM application; it may come
from different huge pages in Service OS as shown in
:numref:`overview-mem-layout`.
As Service OS has full knowledge of these huge pages size,
As the Service VM has full knowledge of these huge pages size,
GPA\ :sup:`SOS` and GPA\ :sup:`UOS`, it works with the hypervisor
to complete UOS's host-to-guest mapping using this pseudo code:
to complete the User VM's host-to-guest mapping using this pseudo code:
.. code-block: none
@ -420,8 +419,8 @@ to complete UOS's host-to-guest mapping using this pseudo code:
Virtual Slim bootloader
=======================
Virtual Slim bootloader (vSBL) is the virtual bootloader that supports
booting the UOS on the ACRN hypervisor platform. The vSBL design is
The Virtual Slim bootloader (vSBL) is the virtual bootloader that supports
booting the User VM on the ACRN hypervisor platform. The vSBL design is
derived from Slim Bootloader. It follows a staged design approach that
provides hardware initialization and payload launching that provides the
boot logic. As shown in :numref:`overview-sbl`, the virtual SBL has an
@ -434,13 +433,13 @@ to boot Linux or Android guest OS.
vSBL System Context Diagram
The vSBL image is released as a part of the Service OS (SOS) root
filesystem (rootfs). The vSBL is copied to UOS memory by the VM manager
in the SOS while creating the UOS virtual BSP of UOS. The SOS passes the
start of vSBL and related information to HV. HV sets guest RIP of UOS
The vSBL image is released as a part of the Service OS root
filesystem (rootfs). The vSBL is copied to the User VM memory by the VM manager
in the Service VM while creating the the User VM virtual BSP of the User VM. The Service VM passes the
start of vSBL and related information to HV. HV sets the guest RIP of the User VM's
virtual BSP as the start of vSBL and related guest registers, and
launches the UOS virtual BSP. The vSBL starts running in the virtual
real mode within the UOS. Conceptually, vSBL is part of the UOS runtime.
launches the the User VM virtual BSP. The vSBL starts running in the virtual
real mode within the User VM. Conceptually, vSBL is part of the User VM runtime.
In the current design, the vSBL supports booting Android guest OS or
Linux guest OS using the same vSBL image.
@ -453,14 +452,13 @@ OVMF bootloader
=======================
Open Virtual Machine Firmware (OVMF) is the virtual bootloader that supports
EFI boot of UOS on the ACRN hypervisor platform.
the EFI boot of the User VM on the ACRN hypervisor platform.
The OVMF is copied to UOS memory by the VM manager in the SOS while creating
the UOS virtual BSP of UOS. The SOS passes the start of OVMF and related
information to HV. HV sets guest RIP of UOS virtual BSP as the start of OVMF
and related guest registers, and launches the UOS virtual BSP. The OVMF starts
running in the virtual real mode within the UOS. Conceptually, OVMF is part of
the UOS runtime.
The OVMF is copied to the User VM memory by the VM manager in the Service VM while creating
the User VM virtual BSP of the User VM. The Service VM passes the start of OVMF and related
information to HV. HV sets guest RIP of the User VM virtual BSP as the start of OVMF
and related guest registers, and launches the the User VM virtual BSP. The OVMF starts
running in the virtual real mode within the the User VM. Conceptually, OVMF is part of the User VM runtime.
Freedom From Interference
*************************
@ -491,12 +489,12 @@ the following mechanisms:
Memory corruption can be a common failure mode. ACRN hypervisor properly
sets up the memory-related hardware mechanisms to ensure that:
1. SOS cannot access the memory of the hypervisor, unless explicitly
allowed,
1. The Service VM cannot access the memory of the hypervisor, unless explicitly
allowed
2. UOS cannot access the memory of SOS and the hypervisor, and
2. The User VM cannot access the memory of the Service VM and the hypervisor
3. The hypervisor does not unintendedly access the memory of SOS or UOS.
3. The hypervisor does not unintendedly access the memory of the Serivce or User VM.
- Destination of external interrupts are set to be the physical core
where the VM that handles them is running.
@ -511,9 +509,9 @@ the following mechanisms:
that runs the vCPU where virtual interrupts will be injected.
2. The hypervisor maintains statistics on the total number of received
interrupts to SOS via a hypercall, and has a delay mechanism to
interrupts to the Service VM via a hypercall, and has a delay mechanism to
temporarily block certain virtual interrupts from being injected.
This allows SOS to detect the occurrence of an interrupt storm and
This allows the Service VM to detect the occurrence of an interrupt storm and
control the interrupt injection rate when necessary.
- Mitigation of DMA storm.
@ -539,10 +537,10 @@ CPU P-state & C-state
=====================
In ACRN, CPU P-state and C-state (Px/Cx) are controlled by the guest OS.
The corresponding governors are managed in SOS/UOS for best power
The corresponding governors are managed in the Service/User VM for best power
efficiency and simplicity.
Guest should be able to process the ACPI P/C-state request from OSPM.
Guests should be able to process the ACPI P/C-state request from OSPM.
The needed ACPI objects for P/C-state management should be ready in
ACPI table.
@ -577,22 +575,22 @@ it traps the power state transition request from guest and emulates it.
:numref:`overview-pm-block` shows the basic diagram block for ACRN PM.
The OSPM in each guest manages the guest power state transition. The
Device Model running in SOS traps and emulates the power state
transition of UOS (Linux VM or Android VM in
:numref:`overview-pm-block`). VM Manager knows all UOS power states and
notifies OSPM of SOS (Service OS in :numref:`overview-pm-block`) once
active UOS is in the required power state.
Device Model running in the Service VM traps and emulates the power state
transition of the User VM (Linux VM or Android VM in
:numref:`overview-pm-block`). VM Manager knows all User VM power states and
notifies the OSPM of the Service VM (Service OS in :numref:`overview-pm-block`) once
active the User VM is in the required power state.
Then OSPM of the SOS starts the power state transition of SOS which is
Then the OSPM of the Service VM starts the power state transition of the Service VM which is
trapped to "Sx Agency" in ACRN, and it will start the power state
transition.
Some details about the ACPI table for UOS and SOS:
Some details about the ACPI table for the User and Service VMs:
- The ACPI table in UOS is emulated by Device Model. The Device Model
knows which register the UOS writes to trigger power state
- The ACPI table in the User VM is emulated by the Device Model. The Device Model
knows which register the User VM writes to trigger power state
transitions. Device Model must register an I/O handler for it.
- The ACPI table in SOS is passthru. There is no ACPI parser
- The ACPI table in the Service VM is passthru. There is no ACPI parser
in ACRN HV. The power management related ACPI table is
generated offline and hardcoded in ACRN HV.