mirror of
https://github.com/projectacrn/acrn-hypervisor.git
synced 2025-05-05 23:16:59 +00:00
e2d3653976
Changing the folder structure will cause too many broken links for external references (from other sites). So, let's put the content back where it was before the reorg, and instead use the new persona-based navigation to point to documents in the original locations. Also, introduce redirects for some documents that no longer exits. Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
148 lines
6.7 KiB
ReStructuredText
148 lines
6.7 KiB
ReStructuredText
.. _vbsk-overhead:
|
|
|
|
VBS-K Framework Virtualization Overhead Analysis
|
|
################################################
|
|
|
|
Introduction
|
|
************
|
|
|
|
The ACRN Hypervisor follows the Virtual I/O Device (virtio)
|
|
specification to realize I/O virtualization for many
|
|
performance-critical devices supported in the ACRN project. The
|
|
hypervisor provides the virtio backend service (VBS) APIs, that make it
|
|
very straightforward to implement a virtio device in the hypervisor. We
|
|
can evaluate the virtio backend service in kernel-land (VBS-K) framework
|
|
overhead through a test virtual device called virtio-echo. The total
|
|
overhead of a frontend-backend application based on VBS-K consists
|
|
of VBS-K framework overhead and application-specific overhead. The
|
|
application-specific overhead depends on the specific frontend-backend
|
|
design, from microseconds to seconds. In our HW case, the overall VBS-K
|
|
framework overhead is on the microsecond level, sufficient to meet the
|
|
needs of most applications.
|
|
|
|
Architecture of VIRTIO-ECHO
|
|
***************************
|
|
|
|
virtio-echo is a virtual device based on virtio, and designed for
|
|
testing ACRN virtio backend services in the kernel (VBS-K) framework. It
|
|
includes a virtio-echo frontend driver, a virtio-echo driver in ACRN
|
|
device model (DM) for initialization, and a virtio-echo driver based on
|
|
VBS-K for data reception and transmission. For more virtualization
|
|
background introduction, please refer to:
|
|
|
|
* :ref:`introduction`
|
|
* :ref:`virtio-hld`
|
|
|
|
virtio-echo is implemented as a virtio legacy device in the ACRN device
|
|
model (DM), and is registered as a PCI virtio device to the guest OS
|
|
(UOS). The virtio-echo software has three parts:
|
|
|
|
- **virtio-echo Frontend Driver**: This driver runs in the UOS. It prepares
|
|
the RXQ and notifies the backend for receiving incoming data when the
|
|
UOS starts. Second, it copies the received data from the RXQ to TXQ
|
|
and sends them to the backend. After receiving the message that the
|
|
transmission is completed, it starts again another round of reception
|
|
and transmission, and keeps running until a specified number of cycle
|
|
is reached.
|
|
- **virtio-echo Driver in DM**: This driver is used for initialization
|
|
configuration. It simulates a virtual PCI device for the frontend
|
|
driver use, and sets necessary information such as the device
|
|
configuration and virtqueue information to the VBS-K. After
|
|
initialization, all data exchange are taken over by the VBS-K
|
|
vbs-echo driver.
|
|
- **vbs-echo Backend Driver**: This driver sets all frontend RX buffers to
|
|
be a specific value and sends the data to the frontend driver. After
|
|
receiving the data in RXQ, the fronted driver copies the data to the
|
|
TXQ, and then sends them back to the backend. The backend driver then
|
|
notifies the frontend driver that the data in the TXQ has
|
|
been successfully received. In virtio-echo, the backend driver
|
|
doesn't process or use the received data.
|
|
|
|
:numref:`vbsk-virtio-echo-arch` shows the whole architecture of virtio-echo.
|
|
|
|
.. figure:: images/vbsk-image2.png
|
|
:width: 900px
|
|
:align: center
|
|
:name: vbsk-virtio-echo-arch
|
|
|
|
virtio-echo Architecture
|
|
|
|
Virtualization Overhead Analysis
|
|
********************************
|
|
|
|
Let's analyze the overhead of the VBS-K framework. As we know, the VBS-K
|
|
handles notifications in the SOS kernel instead of in the SOS user space
|
|
DM. This can avoid overhead from switching between kernel space and user
|
|
space. Virtqueues are allocated by UOS, and virtqueue information is
|
|
configured to VBS-K backend by the virtio-echo driver in DM, thus
|
|
virtqueues can be shared between UOS and SOS. There is no copy overhead
|
|
in this sense. The overhead of VBS-K framework mainly contains two
|
|
parts: kick overhead and notify overhead.
|
|
|
|
- **Kick Overhead**: The UOS gets trapped when it executes sensitive
|
|
instructions that notify the hypervisor first. The notification is
|
|
assembled into an IOREQ, saved in a shared IO page, and then
|
|
forwarded to the VHM module by the hypervisor. The VHM notifies its
|
|
client for this IOREQ, in this case, the client is the vbs-echo
|
|
backend driver. Kick overhead is defined as the interval from the
|
|
beginning of UOS trap to a specific VBS-K driver e.g. when
|
|
virtio-echo gets notified.
|
|
- **Notify Overhead**: After the data in virtqueue being processed by the
|
|
backend driver, vbs-echo calls the VHM module to inject an interrupt
|
|
into the frontend. The VHM then uses the hypercall provided by the
|
|
hypervisor, which causes a UOS VMEXIT. The hypervisor finally injects
|
|
an interrupt into the vLAPIC of the UOS and resumes it. The UOS
|
|
therefore receives the interrupt notification. Notify overhead is
|
|
defined as the interval from the beginning of the interrupt injection
|
|
to when the UOS starts interrupt processing.
|
|
|
|
The overhead of a specific application based on VBS-K includes two
|
|
parts: VBS-K framework overhead and application-specific overhead.
|
|
|
|
- **VBS-K Framework Overhead**: As defined above, VBS-K framework overhead
|
|
refers to kick overhead and notify overhead.
|
|
- **Application-Specific Overhead**: A specific virtual device has its own
|
|
frontend driver and backend driver. The application-specific overhead
|
|
depends on its own design.
|
|
|
|
:numref:`vbsk-virtio-echo-e2e` shows the overhead of one end-to-end
|
|
operation in virtio-echo. Overhead of steps marked as red are caused by
|
|
the virtualization scheme based on VBS-K framework. Costs of one "kick"
|
|
operation and one "notify" operation are both on a microsecond level.
|
|
Overhead of steps marked as blue depend on specific frontend and backend
|
|
virtual device drivers. For virtio-echo, the whole end-to-end process
|
|
(from step1 to step 9) costs about 4 dozens of microsecond. That's
|
|
because virtio-echo does little things in its frontend and backend
|
|
driver which is just for testing and there is very little process
|
|
overhead.
|
|
|
|
.. figure:: images/vbsk-image1.png
|
|
:width: 600px
|
|
:align: center
|
|
:name: vbsk-virtio-echo-e2e
|
|
|
|
End to End Overhead of virtio-echo
|
|
|
|
:numref:`vbsk-virtio-echo-path` details the path of kick and notify
|
|
operation shown in :numref:`vbsk-virtio-echo-e2e`. The VBS-K framework
|
|
overhead is caused by operations through these paths. As we can see, all
|
|
these operations are processed in kernel mode which avoids extra
|
|
overhead of passing IOREQ to userspace processing.
|
|
|
|
.. figure:: images/vbsk-image3.png
|
|
:width: 900px
|
|
:align: center
|
|
:name: vbsk-virtio-echo-path
|
|
|
|
Path of VBS-K Framework Overhead
|
|
|
|
Conclusion
|
|
**********
|
|
|
|
Unlike VBS-U processing in user mode, VBS-K moves things into the kernel
|
|
mode and can be used to accelerate processing. A virtual device
|
|
virtio-echo based on VBS-K framework is used to evaluate the VBS-K
|
|
framework overhead. In our test, the VBS-K framework overhead (one kick
|
|
operation and one notify operation) is on the microsecond level which
|
|
can meet the needs of most applications.
|