This is a preparation patch for adding support for VT-d PI
related vCPU scheduling.
ACRN does not support vCPU migration, one vCPU always runs on
the same pCPU, so PI's ndst is never changed after startup.
VCPUs of a VM won’t share same pCPU. So the maximum possible number
of VCPUs that can run on a pCPU is CONFIG_MAX_VM_NUM.
Allocate unique Activation Notification Vectors (ANV) for each vCPU
that belongs to the same pCPU, the ANVs need only be unique within each
pCPU, not across all vCPUs. This reduces # of pre-allocated ANVs for
posted interrupts to CONFIG_MAX_VM_NUM, and enables ACRN to avoid
switching between active and wake-up vector values in the posted
interrupt descriptor on vCPU scheduling state changes.
A total of CONFIG_MAX_VM_NUM consecutive IRQs/vectors are reserved
for posted interrupts use.
The code first initializes vcpu->arch.pid.control.bits.nv dynamically
(will be added in subsequent patch), the other code shall use
vcpu->arch.pid.control.bits.nv instead of the hard-coded notification vectors.
Rename some functions:
apicv_post_intr --> apicv_trigger_pi_anv
posted_intr_notification --> handle_pi_notification
setup_posted_intr_notification --> setup_pi_notification
Tracked-On: #4506
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@Intel.com>
Fill in posted interrupt fields (vector, pda, etc) and set mode to 1 to
enable VT-d PI (posted mode) for this ptdev.
If intr_src->pi_vcpu is 0, fall back to use the remapped mode.
Tracked-On: #4506
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@Intel.com>
Given the vcpumask, check if the IRQ is single destination
and return the destination vCPU if so, the address of associated PI
descriptor for this vCPU can then be passed to dmar_assign_irte() to
set up the posted interrupt IRTE for this device.
For fixed mode interrupt delivery, all vCPUs listed in vcpumask should
service the interrupt requested. But VT-d PI cannot support multicast/broadcast
IRQs, it only supports single CPU destination. So the number of vCPUs
shall be 1 in order to handle IRQ in posted mode for this device.
Add pid_paddr to struct intr_source. If platform_caps.pi is true and
the IRQ is single-destination, pass the physical address of the destination
vCPU's PID to ptirq_build_physical_msi and dmar_assign_irte
Tracked-On: #4506
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@Intel.com>
Add platform_caps.c to maintain platform related information
Set platform_caps.pi to true if all iommus are posted interrupt capable, false
otherwise
If lapic passthru is not configured and platform_caps.pi is true, the vm
may be able to use posted interrupt for a ptdev, if the ptdev's IRQ is
single-destination
Tracked-On: #4506
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@Intel.com>
Exend union dmar_ir_entry to support VT-d posted interrupts.
Rename some fields of union dmar_ir_entry:
entry --> value
sw_bits --> avail
Tracked-On: #4506
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@Intel.com>
Pass intr_src and dmar_ir_entry irte as pointers to dmar_assign_irte(),
which fixes the "Attempt to change parameter passed by value" MISRA C violation.
A few coding style fixes
Tracked-On: #4506
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@Intel.com>
For CPU side posted interrupts, it only uses bit 0 (ON) of the PI's 64-bit control
, other bits are don't care. This is not the case for VT-d posted
interrupts, define more bit fields for the PI's 64-bit control.
Use bitmap functions to manipulate the bit fields atomically.
Some MISRA-C violation and coding style fixes
Tracked-On: #4506
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@Intel.com>
The posted interrupt descriptor is more of a vmx/vmcs concept than a vlapic
concept. struct acrn_vcpu_arch stores the vmx/vmcs info, so put struct pi_desc
in struct acrn_vcpu_arch.
Remove the function apicv_get_pir_desc_paddr()
A few coding style/typo fixes
Tracked-On: #4506
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@Intel.com>
Rename struct vlapic_pir_desc to pi_desc
Rename struct member and local variable pir_desc to pid
pir=posted interrupt request, pi=posted interrupt
pid=posted interrupt descriptor
pir is part of pi descriptor, so it is better to use pi instead of pir
struct pi_desc will be moved to vmx.h in subsequent commit.
Tracked-On: #4506
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Reviewed-by: Eddie Dong <eddie.dong@Intel.com>
config app supports hv config in scenario setting UI.
Tracked-On: #4634
Signed-off-by: Shuang Zheng <shuang.zheng@intel.com>
Reviewed-by: Victor Sun <victor.sun@intel.com>
Acked by: Terry Zou <terry.zou@intel.com>
Update the board defconfig to scenario config xmls, and user can configure
it from webUI.
Tracked-On: #4634
Signed-off-by: Wei Liu <weix.w.liu@intel.com>
Acked-by: Victor Sun <victor.sun@intel.com>
Acked-by: Terry Zou <terry.zou@intel.com>
Board defconfig depends on hypervisor configurations and vm configurations,
add this to support to parse board defconfig from vm configuration.
Tracked-On: #4634
Signed-off-by: Wei Liu <weix.w.liu@intel.com>
Acked-by: Victor Sun <victor.sun@intel.com>
Acked-by: Terry Zou <terry.zou@intel.com>
Modify launch config tool to support passthroug GVT for WaaG by default.
Tracked-On: #4625
Signed-off-by: Wei Liu <weix.w.liu@intel.com>
Acked-by: Victor Sun <victor.sun@intel.com>
Acked-by: Terry Zou <terry.zou@intel.com>
PCI devices with 64-bit MMIO BARs and requiring large MMIO space
can be assigned with physical address range at the very high end of
platform supported physical address space.
This patch uses the board info for 64-bit MMIO window as programmed
by BIOS and constructs 1G page tables for the same.
As ACRN uses identity mapping from Linear to Physical address space
physical addresses upto 48 bit or 256TB can be supported.
Tracked-On: #4586
Signed-off-by: Sainath Grandhi <sainath.grandhi@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
Add 64-bit MMIO window related MACROs to the supported board files
in the hypervisor source code.
Tracked-On: #4586
Signed-off-by: Sainath Grandhi <sainath.grandhi@intel.com>
Rounding HI_MMIO_START, HI_MMIO_END to the closest 1G. This avoids
round up logic in the hypervisor.
Tracked-On: #4586
Signed-off-by: Sainath Grandhi <sainath.grandhi@intel.com>
Waag will send NMIs to all its cores during reboot. But currently,
NMI cannot be injected to vcpu which is in HLT state.
To fix the problem, need to wakeup target vcpu, and inject NMI through
interrupt-window.
Tracked-On: #4620
Signed-off-by: Conghui Chen <conghui.chen@intel.com>
EPT table can be changed concurrently by more than one vcpus.
This patch add a lock to protect the add/modify/delete operations
from different vcpus concurrently.
Tracked-On: #4253
Signed-off-by: Jian Jun Chen <jian.jun.chen@intel.com>
Reviewed-by: Li, Fei1 <fei1.li@intel.com>
As ACRN prepares to support servers with large amounts of memory
current logic to allocate space for 4K pages of EPT at compile time
will increase the size of .bss section of ACRN binary.
Bootloaders could run into a situation where they cannot
find enough contiguous space to load ACRN binary under 4GB,
which is typically heavily fragmented with E820 types Reserved,
ACPI data, 32-bit PCI hole etc.
This patch does the following
1) Works only for "direct" mode of vboot
2) reserves space for 4K pages of EPT, after boot by parsing
platform E820 table, for all types of VMs.
Size comparison:
w/o patch
Size of DRAM Size of .bss
48 GB 0xe1bbc98 (~226 MB)
128 GB 0x222abc98 (~548 MB)
w/ patch
Size of DRAM Size of .bss
48 GB 0x1991c98 (~26 MB)
128 GB 0x1a81c98 (~28 MB)
Tracked-On: #4563
Signed-off-by: Sainath Grandhi <sainath.grandhi@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
To support server platforms with more than 8 IO-APICs
Tracked-On: #4151
Signed-off-by: Sainath Grandhi <sainath.grandhi@intel.com>
Acked-by: Eddie Dong <eddie.dong@Intel.com>
For SOS VM, when the target platform has multiple IO-APICs, there
should be equal number of virtual IO-APICs.
This patch adds support for emulating multiple vIOAPICs per VM.
Tracked-On: #4151
Signed-off-by: Sainath Grandhi <sainath.grandhi@intel.com>
Acked-by: Eddie Dong <eddie.dong@Intel.com>
MADT is used to specify the GSI base for each IO-APIC and the number of
interrupt pins per IO-APIC is programmed into Max. Redir. Entry register of
that IO-APIC.
On platforms with multiple IO-APICs, there can be holes in the GSI space.
For example, on a platform with 2 IO-APICs, the following configuration has
a hole (from 24 to 31) in the GSI space.
IO-APIC 1: GSI base - 0, number of pins - 24
IO-APIC 2: GSI base - 32, number of pins - 8
This patch also adjusts the size for variables used to represent the total
number of IO-APICs on the system from uint16_t to uint8_t as the ACPI MADT
uses only 8-bits to indicate the unique IO-APIC IDs.
Tracked-On: #4151
Signed-off-by: Sainath Grandhi <sainath.grandhi@intel.com>
Acked-by: Eddie Dong <eddie.dong@Intel.com>
As ACRN prepares to support platforms with multiple IO-APICs,
GSI is a better way to represent physical and virtual INTx interrupt
source.
1) This patch replaces usage of "pin" with "gsi" whereever applicable
across the modules.
2) PIC pin to gsi is trickier and needs to consider the usage of
"Interrupt Source Override" structure in ACPI for the corresponding VM.
Tracked-On: #4151
Signed-off-by: Sainath Grandhi <sainath.grandhi@intel.com>
Acked-by: Eddie Dong <eddie.dong@Intel.com>
Changes the mmio handler data from that of the acrn_vm struct to
the acrn_vioapic.
Add nr_pins and base_addr to the acrn_vioapic data structure.
Tracked-On: #4151
Signed-off-by: Sainath Grandhi <sainath.grandhi@intel.com>
Acked-by: Eddie Dong <eddie.dong@Intel.com>
Reverts 538ba08c: hv:Add vpin to ptdev entry mapping for vpic/vioapic
ACRN uses an array of size per VM to store ptirq entries against the vIOAPIC pin
and an array of size per VM to store ptirq entries against the vPIC pin.
This is done to speed up "ptirq entry" lookup at runtime for Level triggered
interrupts in API ptirq_intx_ack used on EOI.
This patch switches the lookup API for INTx interrupts to the API,
ptirq_lookup_entry_by_sid
This could add delay to processing EOI for Level triggered interrupts.
Trade-off here is space saved for array/s of size CONFIG_MAX_IOAPIC_LINES with 8 bytes
per data. On a server platform, ACRN needs to emulate multiple vIOAPICs for
SOS VM, same as the number of physical IO-APICs. Thereby ACRN would need around
10 such arrays per VM.
Removes the need of "pic_pin" except for the APIs facing the hypercalls
hcall_set_ptdev_intr_info, hcall_reset_ptdev_intr_info
Tracked-On: #4151
Signed-off-by: Sainath Grandhi <sainath.grandhi@intel.com>
Acked-by: Eddie Dong <eddie.dong@Intel.com>
Launch config should relay on library/common.py and launch_cfg_lib,
then remove the duplicate parameters and functions.
Tracked-On: #3854
Signed-off-by: Wei Liu <weix.w.liu@intel.com>
Acked-by: Victor Sun <victor.sun@intel.com>
Acked-by: Terry Zou <terry.zou@intel.com>
Scenario config should relay on library/common.py and scenario_cfg_lib,
then remove the duplicate parameters and functions.
Tracked-On: #3854
Signed-off-by: Wei Liu <weix.w.liu@intel.com>
Acked-by: Victor Sun <victor.sun@intel.com>
Acked-by: Terry Zou <terry.zou@intel.com>
Board config should relay on library/common.py and board_cfg_lib, then
remove the duplicate parameters and functions.
Tracked-On: #3854
Signed-off-by: Wei Liu <weix.w.liu@intel.com>
Acked-by: Victor Sun <victor.sun@intel.com>
Acked-by: Terry Zou <terry.zou@intel.com>
For purpose to refine the library/common and xx_cfg_lib, should have
to expands parameters and functions to common.py firstly.
Tracked-On: #3854
Signed-off-by: Wei Liu <weix.w.liu@intel.com>
Acked-by: Victor Sun <victor.sun@intel.com>
Acked-by: Terry Zou <terry.zou@intel.com>
This patch does the following
1) Removes the limitation on BAR size for 1 GB
2) Align the BAR address to the BAR size
Tracked-On: #4586
Signed-off-by: Sainath Grandhi <sainath.grandhi@intel.com>
Acked-by: Terry Zou <terry.zou@intel.com>
Acked-by: Victor Sun <victor.sun@intel.com>
As we did in commit: 697f3fc09f, need
to update IOMEM_INFO for tgl-rvp.
Tracked-On: #4582
Signed-off-by: Wei Liu <weix.w.liu@intel.com>
Acked-by: Victor Sun <victor.sun@intel.com>
Customer might have specific folder where stores their own configurations
for their customized scenario/board, so add TARGET_DIR parameter to support
option that build hyprvisor with specified configurations.
So valid usages are: (target = all | hypervisor)
1. make <target>
2. make <target> KCONFIG_FILE=xxx [TARGET_DIR=xxx]
3. make <target> BOARD=xxx SCENARIO=xxx [TARGET_DIR=xxx]
4. make <target> BOARD_FILE=xxx SCENARIO_FILE=xxx [TARGET_DIR=xxx]
5. make <target> KCONFIG_FILE=xxx BOARD_FILE=xxx SCENARIO_FILE=xxx [TARGET_DIR=xxx]
If TARGET_DIR parameter is not specified in make command, hypervisor will be
built with board configurations under hypervisor/arch/x86/configs/ and scenario
configurations under hypervisor/scenarios/. Moreover, the configurations would
be overwritten if BOARD/SCENARIO files are specified in make command.
If TARGET_DIR parameter is specified in make command, hypervisor will be built
with configuration under that folder if no BOARD/SCENARIO files are specified.
When BOARD/SCENARIO files are available in make command, the TARGET_DIR is used
to store configurations that BOARD/SCENARIO file provided, i.e. Configurations
in TARGET_DIR folder will be overwritten.
Tracked-On: #4517
Signed-off-by: Victor Sun <victor.sun@intel.com>
When user use make parameters to specify BOARD and SCENARIO, there might
be some conflict because parameter of KCONFIG_FILE/BOARD_FILE/SCENARIO_FILE
also includes BOARD/SCENARIO info. To simplify, we only alow below valid
usages:
1. make <target>
2. make <target> KCONFIG_FILE=xxx
3. make <target> BOARD=xxx SCENARIO=xxx
4. make <target> BOARD_FILE=xxx SCENARIO_FILE=xxx
5. make <target> KCONFIG_FILE=xxx BOARD_FILE=xxx SCENARIO_FILE=xxx
Especially for case 1 that no any parameters are specified:
a. If hypervisor/build/.config file which generated by "make menuconfig"
exist, the .config file will be loaded as KCONFIG_FILE:
i.e. equal: make <target> KCONFIG_FILE=hypervisor/build/.config
b. If hypervisor/build/.config file does not exist,
the default BOARD/SCENARIO will be loaded:
i.e. equal: make <target> BOARD=$(BOARD) SCENARIO=$(SCENARIO)
Tracked-On: #4517
Signed-off-by: Victor Sun <victor.sun@intel.com>
When user finish doing "make menuconfig", the .config file is saved in
acrn-config/hypervisor/build/ by default. So current Makefile will check
whether this customized .config file exists and then continue to do
following make process. But this might not true in the real user case,
user could put this customized .config file anywhere.
To resolve this problem, the patch add a KCONFIG_FILE parameter to handle
such case. When KCONFIG_FILE is specified, hypervisor could build hypervisor
with the specific configuration file.
Tracked-On: #4517
Signed-off-by: Victor Sun <victor.sun@intel.com>
Added "vcpu_clos" to configuration XML, here is an example of VM2 with 2 vCPUs:
<clos desc="Class of Service for Cache Allocation Technology. Please refer SDM 17.19.2 for details and use with caution.">
<vcpu_clos>1</vcpu_clos>
<vcpu_clos>0</vcpu_clos>
</clos>
A macro will be generated in vm_configuration.h:
#define VM2_VCPU_CLOS {1U, 0U}
And the macro will be used in vm_configuration.c:
struct acrn_vm_config vm_configs[CONFIG_MAX_VM_NUM] = {
...
{
...
.clos = VM2_VCPU_CLOS,
...
}
}
Tracked-On: #4566
Signed-off-by: Yan, Like <like.yan@intel.com>
Reviewed-by: Victor Sun <victor.sun@intel.com>
This commit allows hypervisor to allocate cache to vcpu by assigning different clos
to vcpus of a same VM.
For example, we could allocate different cache to housekeeping core and real-time core
of an RTVM in order to isolate the interference of housekeeping core via cache hierarchy.
Tracked-On: #4566
Signed-off-by: Yan, Like <like.yan@intel.com>
Reviewed-by: Chen, Zide <zide.chen@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
Find the 64-bit mmio window from /proc/iomem to generate HI_MMIO_START/HI_MMIO_END.
Tracked-On: #4569
Signed-off-by: Wei Liu <weix.w.liu@intel.com>
Acked-by: Victor Sun <victor.sun@intel.com>
Acked-by: Terry Zou <terry.zou@intel.com>
Replace SYSTEM_INFO with IOMEM_INFO for board xml files.
Tracked-On: #4569
Signed-off-by: Wei Liu <weix.w.liu@intel.com>
Acked-by: Victor Sun <victor.sun@intel.com>
Acked-by: Terry Zou <terry.zou@intel.com>
Need more information from iomem, so dump all the iomem information
from /proc/iomem to instead of system ram information.
Tracked-On: #4569
Signed-off-by: Wei Liu <weix.w.liu@intel.com>
Acked-by: Victor Sun <victor.sun@intel.com>
Acked-by: Terry Zou <terry.zou@intel.com>
Currently, config tool generate board files and scenario files in
acrn-hypervisor directory, the origin souce code would be corrupted by
the config tool. Config tool add '--out' option for user to give a path
to store the generated files, without this option, tool will generate
files in origin source code.
Tracked-On: #4517
Signed-off-by: Wei Liu <weix.w.liu@intel.com>
Acked-by: Victor Sun <victor.sun@intel.com>
Correct mismatch PCI audio device for passthrough in nuc6cayh.
Tracked-On: #4530
Signed-off-by: Wei Liu <weix.w.liu@intel.com>
Acked-by: Victor Sun <victor.sun@intel.com>
Modify the pattern to grap the correct PCI BDF from board information
file.
Tracked-On: #4530
Signed-off-by: Wei Liu <weix.w.liu@intel.com>
Acked-by: Victor Sun <victor.sun@intel.com>
1. add CONFIG_scenario=y config in $(BOARD).config file so that
SCENARIO parameter will not be needed when build with xmls.
2. add CONFIG_MAX_KATA_VM_NUM;
3. add CONFIG_ENFORCE_VALIDATED_ACPI_INFO=y because all ACPI info
generated by acrn-config has been validated;
Tracked-On: #3854
Signed-off-by: Wei Liu <weix.w.liu@intel.com>
Acked-by: Victor Sun <victor.sun@intel.com>
Remove '--enable_commit' argument since acrn-config will not use git tool any
more.
Tracked-On: #3854
Signed-off-by: Wei Liu <weix.w.liu@intel.com>
Acked-by: Victor Sun <victor.sun@intel.com>
