hv: Fix the interrupts lost issue with PI support

Currently, not all platforms support posted interrupt processing of both
VT-x and VT-d. On EHL, VT-d doesn't support posted interrupt processing.
So in such scenario, is_pi_capable() in vcpu_handle_pi_notification()
will bypass the PIR pending bits check which might cause a self-NV-IPI
lost.

With commit "bf1ff8c98 (hv: Offload syncing PIR to vIRR to processor
hardware)", the syncing PIR to vIRR is postponed and it is handled by a
self-NV-IPI in the following VMEnter. The process looks like,
a) vcpu A accepts a virtual interrupt ->
   1) ACRN_REQUEST_EVENT is set
   2) corresponding bit in PIR is set
   3) Posted Interrupt ON bit is set
b) vcpu A does virtual interrupt injection on resume path due to
   the pending ACRN_REQUEST_EVENT ->
   1) hypervisor disables host interrupt
   2) ACRN_REQUEST_EVENT is cleared
   3) a self-NV-IPI is sent via ICR of LAPIC.
   4) IRR bit of the self-NV-IPI is set
c) (VM-ENTRY) vcpu A returns into non-root mode
   1) host interrupt enable(by HW)
   2) posted interrupt processing clears the ON bit, sync PIR to vIRR
   3) deliver the virtual interrupt if guest rflags.IF=1
d) (VM-EXIT) vcpu A traps due to a instruction execution (e.g. HLT)
   1) host interrupt disable(by HW)
   2) hypervisor enable host interrupt

Above illustrates a normal process of the virtual interrupt injection
with cpu PI support. However, a failing case is observed. The failing
case is that the self-NV-IPI from b-3 is not accepted by the core until
a timing between d-1 and d-2. b-4 happening between d-1 and d-2 is
observed by debug trace. So the self-NV-IPI will be handled in root-mode
which cannot do the syncing PIR to vIRR processing. Due to the bug
described in the first paragraph, vcpu_handle_pi_notification() cannot
succeed the virtual interrupt injection request. This patch fix it by
removing the wrong check in vcpu_handle_pi_notification() because
vcpu_handle_pi_notification() only happens on platform with cpu PI
support.

Here are some cost data for sending IPI via LAPIC ICR regsiter.
Normally, the cycles between ICR write and IRR got set is 140~260,
which is not accurate due to the MSR read overhead.
And from b-3 to c is about 560 cycles. So b-4 happens during this
period. But in bad case, b-4 doesn't happen even c is triggered.
The worse case i captured is that ICR write and IRR got set costs more
than 1900 cycles. Now, the best GUESS of the huge cost of IPI via ICR is
the ACPI bus arbitration(refer to SDM 10.6.3, 10.7 and Figure 10-17).

Tracked-On: #4937
Signed-off-by: Shuo A Liu <shuo.a.liu@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
This commit is contained in:
Shuo A Liu 2020-06-14 10:50:58 +08:00 committed by wenlingz
parent 055fe2c615
commit 0397cb7174

View File

@ -890,7 +890,7 @@ void vcpu_handle_pi_notification(uint32_t vcpu_index)
ASSERT(vcpu_index < CONFIG_MAX_VM_NUM, "");
if ((vcpu != NULL) && is_pi_capable(vcpu->vm)) {
if (vcpu != NULL) {
struct pi_desc *pid = get_pi_desc(vcpu);
if (bitmap_test(POSTED_INTR_ON, &(pid->control.value))) {