mirror of
https://github.com/projectacrn/acrn-hypervisor.git
synced 2025-07-16 00:16:12 +00:00
Doc: HLD for Split-locked Access Handling
Signed-off-by: Tao Yuhong <yuhong.tao@intel.com>
This commit is contained in:
yuhong.tao@intel.com
David Kinder
parent
d797fefb59
commit
9d327bfdf8
@ -24,3 +24,4 @@ Hypervisor high-level design
|
||||
Hypercall / VHM upcall <hv-hypercall>
|
||||
Compile-time configuration <hv-config>
|
||||
RDT support <hv-rdt>
|
||||
Split-locked Access handling <hld-splitlock>
|
||||
|
||||
118
doc/developer-guides/hld/hld-splitlock.rst
Normal file
118
doc/developer-guides/hld/hld-splitlock.rst
Normal file
@ -0,0 +1,118 @@
|
||||
.. _hld_splitlock:
|
||||
|
||||
Handling Split-locked Access in Acrn
|
||||
####################################
|
||||
|
||||
This document explains what is Split-locked Access, how to detect it and what
|
||||
Acrn do for handling it.
|
||||
|
||||
Split-locked Access Introduction
|
||||
********************************
|
||||
The Intel-64 and IA32 multiple-processor systems support locked atomic
|
||||
operations on locations in system memory. For example, The LOCK instruction
|
||||
prefix be prepended to the following instructions: ADD, ADC, AND, BTC, BTR, BTS,
|
||||
CMPXCHG, CMPXCH8B, CMPXCHG16B, DEC, INC, NEG, NOT, OR, SBB, SUB, XOR, XADD,
|
||||
XCHG. and these instructions must with memory destination operand forms.
|
||||
Reading or writing a byte in system memory are always guaranteed to be handled
|
||||
atomically, otherwise, these locked atomic operations can impact system in two
|
||||
ways:
|
||||
|
||||
- The destination operand located in same cache line. Cache coherency protocols
|
||||
that ensure that atomic operations can be carried out on cached data
|
||||
structures with cache lock.
|
||||
|
||||
- The destination operand located in two cache line. This atomic operation is
|
||||
called a Split-locked Access. For this situation, LOCK# bus signal is asserted
|
||||
to lock the system bus, to ensure they are atomic. See `Intel 64 and IA-32 Architectures Software Developer's Manual(SDM), Volume 3, (Section 8.1.2 Bus Locking) <https://software.intel.com/en-us/download/intel-64-and-ia-32-architectures-sdm-combined-volumes-3a-3b-3c-and-3d-system-programming-guide>`_.
|
||||
|
||||
Split-locked Access may lead to long latency, to ordinary memory operations on
|
||||
other CPUs, which is hard to investigate.
|
||||
|
||||
Split-locked Access Detection
|
||||
*****************************
|
||||
From the Microarchitecture Tremont, a new CPU capability is introduced for
|
||||
detection of Split-locked Access. When this feature is enabled, an alignment
|
||||
check exception(#AC) with error code 0 is raised for instructions which are
|
||||
doing Split-locked Access. #AC is a fault, the instruction is not executed,
|
||||
there is a chance to decide what to do with this instruction in the #AC handler:
|
||||
|
||||
- Allow the instruction to run with LOCK# bus signal, may have impact to other
|
||||
CPUs.
|
||||
- Disable LOCK# assertion for split locked access. That is not right because
|
||||
instruction becomes non-atomic, and Intel plans to remove this CPU feature
|
||||
from select products starting from a specific year. See `SDM, Volume 1, (Section 2.4 PROPOSED REMOVAL FROM UPCOMING PRODUCTS) <https://software.intel.com/en-us/download/intel-64-and-ia-32-architectures-software-developers-manual-volume-1-basic-architecture>`_.
|
||||
- Terminate the software at this instruction.
|
||||
|
||||
Feature Enumeration and Control
|
||||
*******************************
|
||||
#AC for Split-locked Access feature is enumarated and controled via CPUID and
|
||||
MSR registers.
|
||||
|
||||
- CPUID.(EAX=0x7, ECX=0):EDX[30], the 30th bit of output value in EDX indicates
|
||||
if the platform has IA32_CORE_CAPABILITIES MSR.
|
||||
|
||||
- The 5th bit of IA32_CORE_CAPABILITIES MSR(0xcf), enumerates whether the CPU
|
||||
support #AC for Split-locked Access(And has TEST_CTRL MSR).
|
||||
|
||||
- The 29th bit of TEST_CTL MSR(0x33) control enable/disable #AC for Split-locked
|
||||
Access.
|
||||
|
||||
Acrn Handling Split-locked Access
|
||||
*********************************
|
||||
Split-locked Access is not expected in Acrn hypervisor, should never happen. But
|
||||
that is not guaranteed inside VMs. Acrn support use this CPU capability, that
|
||||
are design principles:
|
||||
|
||||
- Always enable #AC on Split-locked Access in physical side.
|
||||
|
||||
- Present virtual split lock capability to guest, and directly deliver #AC to
|
||||
guest (virtual split-lock capability helps the guest to isolate violations
|
||||
from user land).
|
||||
|
||||
- Guest write of MSR_TEST_CTL is ignored, guest read gets the written value.
|
||||
|
||||
- If Split-locked Access happens in ACRN, this is a software bug we must fix it.
|
||||
|
||||
- If split-locked Access happens in guest kernel, the guest may not be able to
|
||||
fix the issue gracefully (the guest may behavior differently with that of
|
||||
native OS). The RT guest should avoid the Split-locked Access, otherwise it is
|
||||
a software bug.
|
||||
|
||||
Fore Enable This Feature
|
||||
========================
|
||||
This feature is enumerated at PCPU pre-initialization stage, where Acrn detects
|
||||
CPU capabilities. If the PCPU supports this feature:
|
||||
|
||||
- Enabled at each PCPU post-initialization stage.
|
||||
|
||||
- Acrn hypervisor presents virtual emulated TEST_CTRL MSR to each VCPU.
|
||||
Setting/clearing TEST_CTRL[bit 29] from VCPU, has no any effect.
|
||||
|
||||
If PCPU does not have this capability, VCPU does not have the virtual TEST_CTRL
|
||||
neither.
|
||||
|
||||
Expected Behavior in Acrn
|
||||
=========================
|
||||
Acrn HV should never trigger Split-locked Access, it is not allowed to run with
|
||||
Split-locked Access. If it does trigger split-lock, Acrn report #AC at the
|
||||
instruction and stop running. This HV instruction is considered as a bug, must
|
||||
fix it.
|
||||
|
||||
Expected Behavior in VM
|
||||
=======================
|
||||
If VM process have a Split-locked Access at user space, it will be terminated by
|
||||
SIGBUS. When debug inside VM for a reason, you may find it triggers an #AC, no
|
||||
matter alignment checking is enabled or not.
|
||||
If VM kernel have a Split-locked Access, it would hang or oops for #AC. VM
|
||||
kernel may try disable #AC for Split-locked Access and continue, but it will not
|
||||
make it. For that case, you may find HV warning message that says VM tries
|
||||
writing to TEST_CTRL MSR, and help to identify the problem.
|
||||
|
||||
|
||||
Disable Split-locked Access Detection
|
||||
=====================================
|
||||
When the CPU support Split-locked Access detection, Acrn HV will use it to
|
||||
prevent any VM to run a burden software(which contains split-locked instructions). Sometimes customer may really want to run the software and does not care
|
||||
about performance of entire system, he can configure this feature out of Acrn
|
||||
code.
|
||||
The next work in the plan is to use Acrn Configuration Tools to disable split lock detection, that will be done later.
|
||||
Loading…
Reference in New Issue
Block a user