Minor edits to VT-d page.

Signed-off-by: Deb Taylor <deb.taylor@intel.com>
This commit is contained in:
Deb Taylor 2020-03-31 17:12:08 -04:00 committed by wenlingz
parent 844dba1806
commit d7426e5a87

View File

@ -3,8 +3,8 @@
VT-d VT-d
#### ####
VT-d stands for Intel Virtual Technology for Directed IO, and provides Intel Virtual Technology for Directed I/O, or VT-d, provides
hardware capabilities to assign I/O devices to VMs and extending the hardware support for I/O device virtualization. It extends the
protection and isolation properties of VMs for I/O operations. protection and isolation properties of VMs for I/O operations.
VT-d provides the following main functions: VT-d provides the following main functions:
@ -13,17 +13,17 @@ VT-d provides the following main functions:
devices. devices.
- **Interrupt remapping**: for supporting isolation and routing of - **Interrupt remapping**: for supporting isolation and routing of
interrupts from devices and external interrupt controllers to interrupts from devices and external interrupt controllers to the
appropriate VMs. appropriate VMs.
- **Interrupt posting**: for supporting direct delivery of virtual - **Interrupt posting**: for supporting direct delivery of virtual
interrupts from devices and external controllers to virtual interrupts from devices and external controllers to virtual
processors. processors.
ACRN hypervisor supports DMA remapping that provides address translation The ACRN hypervisor supports DMA remapping that provides address translation
capability for PCI pass-through devices, and second-level translation, capability for PCI passthrough devices, and second-level translation,
which applies to requests-without-PASID. ACRN does not support which applies to requests-without-PASID. ACRN does not support
First-level / nested translation. First-level/nested translation.
DMAR Engines Discovery DMAR Engines Discovery
********************** **********************
@ -31,8 +31,8 @@ DMAR Engines Discovery
DMA Remapping Report ACPI table DMA Remapping Report ACPI table
=============================== ===============================
For generic platforms, ACRN hypervisor retrieves DMAR information from For generic platforms, the ACRN hypervisor retrieves DMAR information from
the ACPI table, and parses the DMAR reporting structure to discover the the ACPI table and then parses the DMAR reporting structure to discover the
number of DMA-remapping hardware units present in the platform as well as number of DMA-remapping hardware units present in the platform as well as
the devices under the scope of a remapping hardware unit, as shown in the devices under the scope of a remapping hardware unit, as shown in
:numref:`dma-remap-report`: :numref:`dma-remap-report`:
@ -46,17 +46,17 @@ the devices under the scope of a remapping hardware unit, as shown in
Pre-parsed DMAR information Pre-parsed DMAR information
=========================== ===========================
For specific platforms, ACRN hypervisor uses pre-parsed DMA remapping For specific platforms, the ACRN hypervisor uses pre-parsed DMA remapping
reporting information directly to save time for hypervisor boot-up. reporting information directly to save hypervisor bootup time.
DMA remapping unit for integrated graphics device DMA remapping unit for integrated graphics device
================================================= =================================================
Generally, there is a dedicated remapping hardware unit for the Intel Generally, there is a dedicated remapping hardware unit for the Intel
integrated graphics device. ACRN implements GVT-g for graphics, but integrated graphics device. ACRN implements GVT-g for graphics, but
GVT-g is not compatible with VT-d. The remapping hardware unit for GVT-g is not compatible with VT-d. The remapping hardware unit for the
graphics device is disabled on ACRN if GVT-g is enabled. If the graphics graphics device is disabled on ACRN if GVT-g is enabled. If the graphics
device needs to pass-through to a VM, then the remapping hardware unit device needs to passthrough to a VM, then the remapping hardware unit
must be enabled. must be enabled.
DMA Remapping DMA Remapping
@ -101,16 +101,16 @@ memory to be used for remapping requests-without-PASID processed through
the context-entry. the context-entry.
For a given Bus, Device, and Function combination as shown in For a given Bus, Device, and Function combination as shown in
:numref:`bdf-passthru`, a pass-through device can be associated with :numref:`bdf-passthru`, a passthrough device can be associated with the
address translation structures for a domain. address translation structures for a domain.
.. figure:: images/vt-d-image19.png .. figure:: images/vt-d-image19.png
:align: center :align: center
:name: bdf-passthru :name: bdf-passthru
BDF Format of Pass-through Device BDF Format of Passthrough Device
Refer to the `VT-d spec`_ for the more details of Device to domain Refer to the `VT-d spec`_ for more details on device-to-domain
mapping structures. mapping structures.
.. _VT-d spec: .. _VT-d spec:
@ -119,8 +119,8 @@ mapping structures.
Address Translation Structures Address Translation Structures
============================== ==============================
On ACRN, EPT table of a domain is used as the address translation For ACRN, the EPT table of a domain is used as the address translation
structures for the devices assigned to the domain, as shown structures for the devices assigned to the domain, as shown in
:numref:`vt-d-DMA`. :numref:`vt-d-DMA`.
.. figure:: images/vt-d-image40.png .. figure:: images/vt-d-image40.png
@ -129,72 +129,74 @@ structures for the devices assigned to the domain, as shown
DMA Remapping Diagram DMA Remapping Diagram
When the device attempts to access system memory, the DMA When the device attempts to access system memory, the DMA remapping hardware
remapping hardware intercepts the access, utilizes the EPT table of the intercepts the access and utilizes the EPT table of the domain to determine
domain to determine whether the access is allowed, and translates the DMA whether the access is allowed. It then translates the DMA address according
address according to the EPT table from guest physical address (GPA) to to the EPT table from the guest physical address (GPA) to the host physical
host physical address (HPA). address (HPA).
Domains and Memory Isolation Domains and Memory Isolation
============================ ============================
There are no DMA operations inside the hypervisor, so ACRN doesn't DMA operations do not exist inside the hypervisor, so ACRN doesn't
create a domain for the hypervisor. No DMA operations from pass-through create a domain for the hypervisor. No DMA operations from passthrough
devices can access the hypervisor memory. devices can access the hypervisor memory.
ACRN treats each virtual machine (VM) as a separate domain. For a VM, ACRN treats each virtual machine (VM) as a separate domain. For a VM,
there is a EPT table for Normal world, and there may be a EPT table for an EPT table exists for Normal world; an EPT table for Secure world might
Secure World. Secure world can access Normal World's memory, but Normal also exist. Secure world can access Normal world's memory, but Normal
world cannot access Secure World's memory. world cannot access Secure world's memory.
SOS_VM domain SOS_VM domain
SOS_VM domain is created when the hypervisor creates VM for the The SOS_VM domain is created when the hypervisor creates the VM for the
Service OS. Service OS.
IOMMU uses the EPT table of Normal world of SOS_VM as the address IOMMU uses the EPT table of Normal world of SOS_VM as the address
translation structures for the devices in SOS_VM domain. The Normal world's translation structures for the devices in the SOS_VM domain. The Normal
EPT table of SOS_VM doesn't include the memory resource of the hypervisor world's EPT table of SOS_VM doesn't include the memory resource of the
and Secure worlds if any. So the devices in SOS_VM domain can't access the hypervisor and Secure worlds (if any exists). So the devices in SOS_VM
memory belong to hypervisor or secure worlds. domain can't access the memory belonging to the hypervisor or secure
worlds.
Other domains Other domains
Other VM domains will be created when hypervisor creates User OS. One Other VM domains will be created when the hypervisor creates the User OS.
domain for each User OS. One domain for each User OS.
IOMMU uses the EPT table of Normal world of a VM as the address IOMMU uses the EPT table of the Normal world of a VM as the address
translation structures for the devices in the domain. The Normal world's translation structures for the devices in the domain. The Normal world's
EPT table of the VM only allows devices to access the memory EPT table of the VM only allows devices to access the memory
allocated for Normal world of the VM. allocated for the Normal world of the VM.
Page-walk coherency Page-walk coherency
=================== ===================
For the VT-d hardware, which doesn't support page-walk coherency, For the VT-d hardware, which doesn't support page-walk coherency, the
hypervisor needs to make sure the updates of VT-d tables are synced in hypervisor needs to make sure the updates of VT-d tables are synced in
memory: memory:
- Device to Domain Mapping Structures, including Root-entries and - Device to Domain Mapping Structures, including Root-entries and
Context-entries Context-entries.
- EPT table of a VM. - The EPT table of a VM.
ACRN will flush the related cache line after updates of these structures ACRN flushes the related cache line after these structures are updated
if the VT-d hardware doesn't support page-walk coherency. if the VT-d hardware doesn't support page-walk coherency.
Super-page support Super-page support
================== ==================
ACRN VT-d reuses the EPT table as address a translation table. VT-d capability The ACRN VT-d reuses the EPT table as the address translation table. VT-d
for super-page support should be identical with the usage of EPT table. capability or super-page support should be identical with the usage of the
EPT table.
Snoop control Snoop control
============= =============
If VT-d hardware supports snoop control, it allows VT-d to control to If VT-d hardware supports snoop control, iVT-d can control the
ignore the "no-snoop attribute" in PCI-E transactions. ability to ignore the "no-snoop attribute" in PCI-E transactions.
The following table shows the snoop behavior of DMA operation controlled by the The following table shows the snoop behavior of a DMA operation controlled by
combination of: the following:
- Snoop Control capability of VT-d DMAR unit - Snoop Control capability of VT-d DMAR unit
- The setting of SNP filed in leaf PTE - The setting of SNP filed in leaf PTE
@ -234,24 +236,25 @@ combination of:
- snoop - snoop
- Snoop - Snoop
If VT-d DMAR units doesn't support Snoop Control, then SNP Bit (bit 11) If VT-d DMAR units do not support Snoop Control, then the SNP Bit (bit 11)
of leaf PETs of EPT is not set since the field is treated as reserved(0) of leaf PETs of the EPT is not set since the field is treated as reserved (0)
by VT-d hardware implementations not supporting Snoop Control. by the VT-d hardware implementations of not supporting Snoop Control.
VT-d DMAR unit of Intel integrated graphics device doesn't support Snoop The VT-d DMAR unit of the Intel integrated graphics device doesn't support
Control. ACRN hypervisor uses a same copy of EPT as the secondary address Snoop Control. The ACRN hypervisor uses the same copy of EPT as the
translation table for a VM. When enalbe DMAR unit for Intel integrated secondary address translation table for a VM. When the DMAR unit for the
graphics device, SNP Bit cannot be set in lead PTEs of EPT. Intel integrated graphics device is enabled, the SNP Bit cannot be set in
the lead PTEs of the EPT.
No matter ACRN enables or disables Snoop Control, the DMA operations of No matter if ACRN enables or disables Snoop Control, the DMA operations of
passthrough devices behave correctly from guests' point of view. ACRN passthrough devices behave correctly from the guest's point of view. ACRN
disables Snoop Control in VT-d DMAR engines for simplifing the implementation. disables Snoop Control in VT-d DMAR engines that simplify the implementation.
Also, since the snoop behavior of PCIE transactions can be controlled by Also, since the snoop behavior of PCIE transactions can be controlled by
guest drivers, some devices may take the advantage of the NO_SNOOP_ATTRIBUTE guest drivers, some devices may take advantage of the NO_SNOOP_ATTRIBUTE
of PCIE transactions for better performance when snoop is not needed. of PCIE transactions for better performance when snoop is not needed.
It's driver's responsibility to configure correct attribute in PCIE transactions. The driver is responsible for configuring correct attribute in PCIE
Otherwise, the corresponding device may not work properly. transactions. Otherwise, the corresponding device may not work properly.
Initialization Initialization
************** **************
@ -272,29 +275,29 @@ translation for DMAR unit(s) if they are not marked as ignored.
Device assignment Device assignment
***************** *****************
All devices are initially added to SOS_VM domain. All devices are initially added to the SOS_VM domain. To assign a device
To assign a device means to assign the device to an User OS. The device means to assign the device to a User VM. The device is removed from the
is remove from SOS_VM domain and added to the VM domain related to the User SOS_VM domain and is added to the VM domain related to the User VM, which
OS, which changes the address translation table from EPT of SOS_VM to EPT changes the address translation table from the EPT of SOS_VM to the EPT
of User OS for the device. of the User OS for the device.
To unassign a device means to unassign the device from an User OS. The To unassign a device means to unassign the device from a User OS. The
device is remove from the VM domain related to the User OS, then added device is removed from the VM domain related to the User OS and then added
back to SOS_VM domain, which changes the address translation table from EPT back to the SOS_VM domain; this changes the address translation table from
of User OS to EPT of SOS_VM for the device. the EPT of the User OS to the EPT of the SOS_VM for the device.
Power Management support for S3 Power Management support for S3
******************************* *******************************
During platform S3 suspend and resume, the VT-d register values will be During platform S3 suspend and resume, the VT-d register values are
lost. ACRN VT-d provide APIs to be called during S3 suspend and resume. lost. ACRN VT-d provides APIs tthat are called during S3 suspend and resume.
During S3 suspend, some register values are saved in the memory, and During S3 suspend, some register values are saved in the memory, and
DMAR translation is disabled. During S3 resume, the register values DMAR translation is disabled. During S3 resume, the register values
saved are restored. Root table address register is set. DMAR translation saved are restored. The Root table address register is set. The DMAR
is enabled. translation is enabled.
All the operations for S3 suspend and resume are performed on all DMAR All operations for S3 suspend and resume are performed on all DMAR
units on the platform, except for the DMAR units marked ignored. units on the platform, except for the DMAR units marked ignored.
Error Handling Error Handling
@ -304,7 +307,7 @@ ACRN VT-d supports DMA remapping error reporting. ACRN VT-d requests a
IRQ / vector for DMAR error reporting. A DMAR fault handler is IRQ / vector for DMAR error reporting. A DMAR fault handler is
registered for the IRQ. DMAR unit supports report fault event via MSI. registered for the IRQ. DMAR unit supports report fault event via MSI.
When a fault event occurs, a MSI is generated, so that the DMAR fault When a fault event occurs, a MSI is generated, so that the DMAR fault
handler will be called to report error event. handler will be called to report the error event.
Data structures and interfaces Data structures and interfaces
****************************** ******************************