mirror of
https://github.com/projectacrn/acrn-hypervisor.git
synced 2025-06-24 06:29:19 +00:00
doc: merge static core with cpu virt
Update the converted HLD 0.7 section 3.2 with a previously converted static partitioning document (instead of a separate doc for this topic). Tracked-on: #1533 Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
This commit is contained in:
David B. Kinder
David Kinder
parent
e01f47778a
commit
acc51877f1
@ -9,6 +9,6 @@ Hypervisor high-level design
|
|||||||
|
|
||||||
hv-startup
|
hv-startup
|
||||||
hv-cpu-virt
|
hv-cpu-virt
|
||||||
static-core-hld
|
|
||||||
Memory management <memmgt-hld>
|
Memory management <memmgt-hld>
|
||||||
|
|
||||||
Interrupt management <interrupt-hld>
|
Interrupt management <interrupt-hld>
|
||||||
|
|||||||
@ -9,6 +9,9 @@ CPU Virtualization
|
|||||||
|
|
||||||
ACRN Hypervisor CPU Virtualization Components
|
ACRN Hypervisor CPU Virtualization Components
|
||||||
|
|
||||||
|
The following sections discuss the major modules (shown in blue) in the
|
||||||
|
CPU virtualization overview shown in :numref:`hv-cpu-virt-components`.
|
||||||
|
|
||||||
Based on Intel VT-x virtualization technology, ACRN emulates a virtual CPU
|
Based on Intel VT-x virtualization technology, ACRN emulates a virtual CPU
|
||||||
(vCPU) with the following methods:
|
(vCPU) with the following methods:
|
||||||
|
|
||||||
@ -16,7 +19,8 @@ Based on Intel VT-x virtualization technology, ACRN emulates a virtual CPU
|
|||||||
physical CPU (pCPU),
|
physical CPU (pCPU),
|
||||||
making much of hardware register emulation simply
|
making much of hardware register emulation simply
|
||||||
pass-through and provides good isolation for physical interrupt
|
pass-through and provides good isolation for physical interrupt
|
||||||
and guest execution.
|
and guest execution. (See `Static CPU Partitioning`_ for more
|
||||||
|
information.)
|
||||||
|
|
||||||
- **simple schedule**: only two thread loops are maintained for a CPU -
|
- **simple schedule**: only two thread loops are maintained for a CPU -
|
||||||
vCPU thread and default idle thread. A CPU runs most of the time in
|
vCPU thread and default idle thread. A CPU runs most of the time in
|
||||||
@ -25,8 +29,137 @@ Based on Intel VT-x virtualization technology, ACRN emulates a virtual CPU
|
|||||||
operation needs it to stay in VMX root mode, such as when waiting for
|
operation needs it to stay in VMX root mode, such as when waiting for
|
||||||
an I/O request from DM or ready to destroy.
|
an I/O request from DM or ready to destroy.
|
||||||
|
|
||||||
The following sections discuss the major modules (shown in blue) in the
|
Static CPU Partitioning
|
||||||
CPU virtualization overview shown in :numref:`hv-cpu-virt-components`.
|
***********************
|
||||||
|
|
||||||
|
CPU partitioning is a policy for mapping a virtual
|
||||||
|
CPU (VCPU) to a physical CPU. The current ACRN implementation forces a
|
||||||
|
static 1:1 mapping between VCPUs and physical CPUs and does
|
||||||
|
not support multiple VCPUs running on a physical CPU and does not
|
||||||
|
support VCPU migration from one physical CPU to another.
|
||||||
|
|
||||||
|
ACRN forces a fixed 1:1 mapping between a VCPU and a physical CPU when
|
||||||
|
creating a VCPU for the guest Operating System. This makes the VCPU
|
||||||
|
management code much simpler.
|
||||||
|
|
||||||
|
An array is used to track the physical CPU allocation information. When
|
||||||
|
a VCPU is created, we query, allocate, and update the array to map the
|
||||||
|
VCPU to an available physical CPU.
|
||||||
|
|
||||||
|
The physical CPU number assignment for each guest is pre-defined. For
|
||||||
|
example, on a platform with four CPU cores, one physical CPU is assigned
|
||||||
|
to run the Service Operating System (SOS) and other three physical CPUs
|
||||||
|
are assigned to run the User Operating System (UOS) instances.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
To improvement SOS boot time, all physical CPUs are assigned to the SOS
|
||||||
|
during the SOS boot. Afterward, the physical CPUs defined for the UOS
|
||||||
|
are allocated by the Device Model (DM) by running the launch_uos.sh
|
||||||
|
script.
|
||||||
|
|
||||||
|
CPU management in SOS
|
||||||
|
=====================
|
||||||
|
|
||||||
|
With ACRN, all ACPI table entries are pass-thru to the SOS, including
|
||||||
|
the Multiple Interrupt Controller Table (MADT). The SOS sees all
|
||||||
|
physical CPUs by parsing the MADT when the SOS kernel boots. All
|
||||||
|
physical CPUs are initially assigned to the SOS by creating the same
|
||||||
|
number of virtual CPUs.
|
||||||
|
|
||||||
|
When the SOS boot is finished, it releases the physical CPUs intended
|
||||||
|
for UOS use.
|
||||||
|
|
||||||
|
Here is an example flow of CPU allocation on a multi-core platform.
|
||||||
|
|
||||||
|
.. figure:: images/static-core-image2.png
|
||||||
|
:width: 600px
|
||||||
|
:align: center
|
||||||
|
:name: static-core-cpu-allocation
|
||||||
|
|
||||||
|
CPU allocation on a multi-core platform
|
||||||
|
|
||||||
|
CPU management in UOS
|
||||||
|
=====================
|
||||||
|
|
||||||
|
From the UOS point of view, CPU management is very simple, using a
|
||||||
|
hypercall to create the virtual CPUs. Here's an example from from the DM
|
||||||
|
code:
|
||||||
|
|
||||||
|
.. code-block:: c
|
||||||
|
|
||||||
|
int vm_create_vcpu(struct vmctx *ctx, uint16_t vcpu_id)
|
||||||
|
{
|
||||||
|
struct acrn_create_vcpu cv;
|
||||||
|
int error;
|
||||||
|
|
||||||
|
bzero(&cv, sizeof(struct acrn_create_vcpu));
|
||||||
|
cv.vcpu_id = vcpu_id;
|
||||||
|
error = ioctl(ctx->fd, IC_CREATE_VCPU, &cv);
|
||||||
|
return error;
|
||||||
|
}
|
||||||
|
|
||||||
|
The VHM will respond to the ioctl:
|
||||||
|
|
||||||
|
.. code-block:: c
|
||||||
|
|
||||||
|
case IC_CREATE_VCPU: {
|
||||||
|
struct acrn_create_vcpu cv;
|
||||||
|
|
||||||
|
if (copy_from_user(&cv, (void *)ioctl_param,
|
||||||
|
sizeof(struct acrn_create_vcpu)))
|
||||||
|
return -EFAULT;
|
||||||
|
|
||||||
|
ret = acrn_hypercall2(HC_CREATE_VCPU, vm->vmid,
|
||||||
|
virt_to_phys(&cv));
|
||||||
|
if (ret < 0) {
|
||||||
|
pr_err("vhm: failed to create vcpu %d!\\n",
|
||||||
|
cv.vcpu_id);
|
||||||
|
return -EFAULT;
|
||||||
|
}
|
||||||
|
|
||||||
|
atomic_inc(&vm->vcpu_num);
|
||||||
|
return ret;
|
||||||
|
}
|
||||||
|
|
||||||
|
The hypercall ``HC_CREATE_VCPU`` is handled in the hypervisor with
|
||||||
|
the parameter:
|
||||||
|
|
||||||
|
.. code-block:: c
|
||||||
|
|
||||||
|
struct acrn_create_vcpu {
|
||||||
|
/** the virtual CPU ID for the VCPU created */
|
||||||
|
uint16_t vcpu_id;
|
||||||
|
/** the physical CPU ID for the VCPU created */
|
||||||
|
uint16_t pcpu_id;
|
||||||
|
} __attribute__((aligned(8)));
|
||||||
|
|
||||||
|
CPU assignment management in HV
|
||||||
|
===============================
|
||||||
|
|
||||||
|
When we create a VCPU in the hypervisor, an available physical CPU is
|
||||||
|
picked and marked as used. When we destroy the VCPU, we mark the
|
||||||
|
physical CPU as available again.
|
||||||
|
|
||||||
|
.. figure:: images/static-core-image1.png
|
||||||
|
:width: 600px
|
||||||
|
:align: center
|
||||||
|
:name: static-core-cpu-assign
|
||||||
|
|
||||||
|
HV CPU Assignment Management
|
||||||
|
|
||||||
|
#. ``allocate_pcpu()`` queries the physical CPU allocation info to get an
|
||||||
|
available physical CPU and marks physical CPU as not available
|
||||||
|
#. Physical CPU info is passed to ``create_vcpu()`` and a mapping is built
|
||||||
|
between the physical CPU and virtual CPU
|
||||||
|
#. When the VCPU is destroyed VCPU, the physical CPU is passed to the
|
||||||
|
``free_pcpu()`` function
|
||||||
|
#. ``free_pcpu()`` marks the physical CPU available again.
|
||||||
|
|
||||||
|
Currently, the ACRN hypervisor does not support virtual CPU migration to
|
||||||
|
different physical CPUs. This means no changes to the virtual CPU to
|
||||||
|
physical CPU can happen without first calling destroy_vcpu.
|
||||||
|
|
||||||
|
|
||||||
.. _vCPU_lifecycle:
|
.. _vCPU_lifecycle:
|
||||||
|
|
||||||
|
|||||||
@ -1,138 +0,0 @@
|
|||||||
.. cpu-partitioning-hld:
|
|
||||||
|
|
||||||
Static CPU Partitioning
|
|
||||||
#######################
|
|
||||||
|
|
||||||
CPU partitioning is a policy for mapping a virtual
|
|
||||||
CPU (VCPU) to a physical CPU. The current ACRN implementation forces a
|
|
||||||
static 1:1 mapping between VCPUs and physical CPUs and does
|
|
||||||
not support multiple VCPUs running on a physical CPU and does not
|
|
||||||
support VCPU migration from one physical CPU to another.
|
|
||||||
|
|
||||||
This section describes the static CPU partitioning implementation in
|
|
||||||
ACRN.
|
|
||||||
|
|
||||||
Overview
|
|
||||||
********
|
|
||||||
|
|
||||||
ACRN forces a fixed 1:1 mapping between a VCPU and a physical CPU when
|
|
||||||
creating a VCPU for the guest Operating System. This makes the VCPU
|
|
||||||
management code much simpler.
|
|
||||||
|
|
||||||
An array is used to track the physical CPU allocation information. When
|
|
||||||
a VCPU is created, we query, allocate, and update the array to map the
|
|
||||||
VCPU to an available physical CPU.
|
|
||||||
|
|
||||||
The physical CPU number assignment for each guest is pre-defined. For
|
|
||||||
example, on a platform with four CPU cores, one physical CPU is assigned
|
|
||||||
to run the Service Operating System (SOS) and other three physical CPUs
|
|
||||||
are assigned to run the User Operating System (UOS) instances.
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
To improvement SOS boot time, all physical CPUs are assigned to the SOS
|
|
||||||
during the SOS boot. Afterward, the physical CPUs defined for the UOS
|
|
||||||
are allocated by the Device Model (DM) by running the launch_uos.sh
|
|
||||||
script.
|
|
||||||
|
|
||||||
CPU management in SOS
|
|
||||||
*********************
|
|
||||||
|
|
||||||
With ACRN, all ACPI table entries are pass-thru to the SOS, including
|
|
||||||
the Multiple Interrupt Controller Table (MADT). The SOS sees all
|
|
||||||
physical CPUs by parsing the MADT when the SOS kernel boots. All
|
|
||||||
physical CPUs are initially assigned to the SOS by creating the same
|
|
||||||
number of virtual CPUs.
|
|
||||||
|
|
||||||
When the SOS boot is finished, it releases the physical CPUs intended
|
|
||||||
for UOS use.
|
|
||||||
|
|
||||||
Here is an example flow of CPU allocation on a multi-core platform.
|
|
||||||
|
|
||||||
.. figure:: images/static-core-image2.png
|
|
||||||
:width: 600px
|
|
||||||
:align: center
|
|
||||||
:name: static-core-cpu-allocation
|
|
||||||
|
|
||||||
CPU allocation on a multi-core platform
|
|
||||||
|
|
||||||
CPU management in UOS
|
|
||||||
*********************
|
|
||||||
|
|
||||||
From the UOS point of view, CPU management is very simple, using a
|
|
||||||
hypercall to create the virtual CPUs. Here's an example from from the DM
|
|
||||||
code:
|
|
||||||
|
|
||||||
.. code-block:: c
|
|
||||||
|
|
||||||
int vm_create_vcpu(struct vmctx *ctx, uint16_t vcpu_id)
|
|
||||||
{
|
|
||||||
struct acrn_create_vcpu cv;
|
|
||||||
int error;
|
|
||||||
|
|
||||||
bzero(&cv, sizeof(struct acrn_create_vcpu));
|
|
||||||
cv.vcpu_id = vcpu_id;
|
|
||||||
error = ioctl(ctx->fd, IC_CREATE_VCPU, &cv);
|
|
||||||
return error;
|
|
||||||
}
|
|
||||||
|
|
||||||
The VHM will respond to the ioctl:
|
|
||||||
|
|
||||||
.. code-block:: c
|
|
||||||
|
|
||||||
case IC_CREATE_VCPU: {
|
|
||||||
struct acrn_create_vcpu cv;
|
|
||||||
|
|
||||||
if (copy_from_user(&cv, (void *)ioctl_param,
|
|
||||||
sizeof(struct acrn_create_vcpu)))
|
|
||||||
return -EFAULT;
|
|
||||||
|
|
||||||
ret = acrn_hypercall2(HC_CREATE_VCPU, vm->vmid,
|
|
||||||
virt_to_phys(&cv));
|
|
||||||
if (ret < 0) {
|
|
||||||
pr_err("vhm: failed to create vcpu %d!\\n",
|
|
||||||
cv.vcpu_id);
|
|
||||||
return -EFAULT;
|
|
||||||
}
|
|
||||||
|
|
||||||
atomic_inc(&vm->vcpu_num);
|
|
||||||
return ret;
|
|
||||||
}
|
|
||||||
|
|
||||||
The hypercall ``HC_CREATE_VCPU`` is handled in the hypervisor with
|
|
||||||
the parameter:
|
|
||||||
|
|
||||||
.. code-block:: c
|
|
||||||
|
|
||||||
struct acrn_create_vcpu {
|
|
||||||
/** the virtual CPU ID for the VCPU created */
|
|
||||||
uint16_t vcpu_id;
|
|
||||||
/** the physical CPU ID for the VCPU created */
|
|
||||||
uint16_t pcpu_id;
|
|
||||||
} __attribute__((aligned(8)));
|
|
||||||
|
|
||||||
CPU assignment management in HV
|
|
||||||
*******************************
|
|
||||||
|
|
||||||
When we create a VCPU in the hypervisor, an available physical CPU is
|
|
||||||
picked and marked as used. When we destroy the VCPU, we mark the
|
|
||||||
physical CPU as available again.
|
|
||||||
|
|
||||||
.. figure:: images/static-core-image1.png
|
|
||||||
:width: 600px
|
|
||||||
:align: center
|
|
||||||
:name: static-core-cpu-assign
|
|
||||||
|
|
||||||
HV CPU Assignment Management
|
|
||||||
|
|
||||||
#. ``allocate_pcpu()`` queries the physical CPU allocation info to get an
|
|
||||||
available physical CPU and marks physical CPU as not available
|
|
||||||
#. Physical CPU info is passed to ``create_vcpu()`` and a mapping is built
|
|
||||||
between the physical CPU and virtual CPU
|
|
||||||
#. When the VCPU is destroyed VCPU, the physical CPU is passed to the
|
|
||||||
``free_pcpu()`` function
|
|
||||||
#. ``free_pcpu()`` marks the physical CPU available again.
|
|
||||||
|
|
||||||
Currently, the ACRN hypervisor does not support virtual CPU migration to
|
|
||||||
different physical CPUs. This means no changes to the virtual CPU to
|
|
||||||
physical CPU can happen without first calling destroy_vcpu.
|
|
||||||
Loading…
Reference in New Issue
Block a user