doc:add tutorial about enabling GVT-d

Previous tutorial "Enable GPU Passthrough on the Skylake NUC"
is out of date, so delete it here.

v2 -> v1:
add attached file

Signed-off-by: Junming Liu <junming.liu@intel.com>
This commit is contained in:
Junming Liu
2020-05-11 17:31:55 +08:00
committed by deb-intel
parent 53af096726
commit 3ccbb20087
8 changed files with 297 additions and 126 deletions

View File

@@ -61,7 +61,7 @@ Enable ACRN Features
tutorials/enable_s5 tutorials/enable_s5
tutorials/cpu_sharing tutorials/cpu_sharing
tutorials/sriov_virtualization tutorials/sriov_virtualization
tutorials/skl-nuc tutorials/gpu-passthru
tutorials/run_kata_containers tutorials/run_kata_containers
tutorials/trustyACRN tutorials/trustyACRN
tutorials/rtvm_workload_design_guideline tutorials/rtvm_workload_design_guideline

View File

@@ -61,7 +61,7 @@ for step-by-step instructions.
**Document updates**: Several new documents have been added in this release, including: **Document updates**: Several new documents have been added in this release, including:
* :ref:`skl-nuc-gpu-passthrough` * :ref:`gpu-passthrough`
* :ref:`acrn-dm_parameters` * :ref:`acrn-dm_parameters`
- :acrn-issue:`878` - Virtualization HLD - :acrn-issue:`878` - Virtualization HLD

View File

@@ -202,7 +202,7 @@ Document updates
We have many reference documents `available We have many reference documents `available
<https://projectacrn.github.io>`_, including: <https://projectacrn.github.io>`_, including:
* :ref:`GPU Passthrough on Skylake NUC <skl-nuc-gpu-passthrough>` * :ref:`Enable GVT-d in ACRN <gpu-passthrough>`
* :ref:`Device Model Parameters <acrn-dm_parameters>` * :ref:`Device Model Parameters <acrn-dm_parameters>`
* :ref:`Running Automotive Grade Linux as a VM <agl-vms>` * :ref:`Running Automotive Grade Linux as a VM <agl-vms>`
* Using PREEMPT_RT-Linux for real-time UOS * Using PREEMPT_RT-Linux for real-time UOS

View File

@@ -0,0 +1,62 @@
From 8bb9e3e2015bea8e45431b31752e56e2e3797640 Mon Sep 17 00:00:00 2001
From: Junming Liu <junming.liu@intel.com>
Date: Thu, 30 Apr 2020 00:44:11 +0800
Subject: [PATCH 2/2] Integrate IntelGopDriver into OVMF
Tracked-On: projectacrn/acrn-hypervisor#4365
Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: Junming Liu <junming.liu@intel.com>
---
OvmfPkg/IntelGop/IntelGopDriver.inf | 17 +++++++++++++++++
OvmfPkg/OvmfPkgX64.dsc | 1 +
OvmfPkg/OvmfPkgX64.fdf | 1 +
3 files changed, 19 insertions(+)
create mode 100644 OvmfPkg/IntelGop/IntelGopDriver.inf
Index: acrn-edk2/OvmfPkg/IntelGop/IntelGopDriver.inf
===================================================================
--- /dev/null
+++ acrn-edk2/OvmfPkg/IntelGop/IntelGopDriver.inf
@@ -0,0 +1,17 @@
+## @file
+# IntelGopDriver Binary
+#
+# Copyright (c) 2006 - 2011, Intel Corporation. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+##
+
+[Defines]
+ INF_VERSION = 0x00010005
+ BASE_NAME = IntelGopDriver
+ FILE_GUID = 1647B4F3-3E8A-4FDD-81C8-328ED647AB1A
+ MODULE_TYPE = UEFI_DRIVER
+ VERSION_STRING = 1.0
+
+[Binaries.X64]
+ PE32|IntelGopDriver.efi|*
Index: acrn-edk2/OvmfPkg/OvmfPkgX64.dsc
===================================================================
--- acrn-edk2.orig/OvmfPkg/OvmfPkgX64.dsc
+++ acrn-edk2/OvmfPkg/OvmfPkgX64.dsc
@@ -756,6 +756,7 @@
#OvmfPkg/XenIoPciDxe/XenIoPciDxe.inf
#OvmfPkg/XenBusDxe/XenBusDxe.inf
#OvmfPkg/XenPvBlkDxe/XenPvBlkDxe.inf
+ OvmfPkg/IntelGop/IntelGopDriver.inf
OvmfPkg/Vbt/Vbt.inf
OvmfPkg/PlatformGopPolicy/PlatformGopPolicy.inf
MdeModulePkg/Universal/WatchdogTimerDxe/WatchdogTimer.inf
Index: acrn-edk2/OvmfPkg/OvmfPkgX64.fdf
===================================================================
--- acrn-edk2.orig/OvmfPkg/OvmfPkgX64.fdf
+++ acrn-edk2/OvmfPkg/OvmfPkgX64.fdf
@@ -235,6 +235,7 @@ INF OvmfPkg/GvtGopDxe/GvtGopDxe.inf
#INF OvmfPkg/XenIoPciDxe/XenIoPciDxe.inf
#INF OvmfPkg/XenBusDxe/XenBusDxe.inf
#INF OvmfPkg/XenPvBlkDxe/XenPvBlkDxe.inf
+INF OvmfPkg/IntelGop/IntelGopDriver.inf
INF RuleOverride=CSM OvmfPkg/Vbt/Vbt.inf
INF OvmfPkg/PlatformGopPolicy/PlatformGopPolicy.inf

View File

@@ -0,0 +1,64 @@
From ad5c297dfebc2b7c8379c752ad7f2e6d27740893 Mon Sep 17 00:00:00 2001
From: Junming Liu <junming.liu@intel.com>
Date: Thu, 30 Apr 2020 00:42:35 +0800
Subject: [PATCH 1/2] Use the default vbt released with GOP driver
This patch use the default vbt which released along with GOP driver.
Tracked-On: projectacrn/acrn-hypervisor#4365
Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: Junming Liu <junming.liu@intel.com>
---
OvmfPkg/OvmfPkgX64.dsc | 1 +
OvmfPkg/OvmfPkgX64.fdf | 1 +
OvmfPkg/Vbt/Vbt.inf | 17 +++++++++++++++++
3 files changed, 19 insertions(+)
create mode 100644 OvmfPkg/Vbt/Vbt.inf
Index: acrn-edk2/OvmfPkg/OvmfPkgX64.dsc
===================================================================
--- acrn-edk2.orig/OvmfPkg/OvmfPkgX64.dsc
+++ acrn-edk2/OvmfPkg/OvmfPkgX64.dsc
@@ -756,6 +756,7 @@
#OvmfPkg/XenIoPciDxe/XenIoPciDxe.inf
#OvmfPkg/XenBusDxe/XenBusDxe.inf
#OvmfPkg/XenPvBlkDxe/XenPvBlkDxe.inf
+ OvmfPkg/Vbt/Vbt.inf
OvmfPkg/PlatformGopPolicy/PlatformGopPolicy.inf
MdeModulePkg/Universal/WatchdogTimerDxe/WatchdogTimer.inf
MdeModulePkg/Universal/MonotonicCounterRuntimeDxe/MonotonicCounterRuntimeDxe.inf
Index: acrn-edk2/OvmfPkg/OvmfPkgX64.fdf
===================================================================
--- acrn-edk2.orig/OvmfPkg/OvmfPkgX64.fdf
+++ acrn-edk2/OvmfPkg/OvmfPkgX64.fdf
@@ -235,6 +235,7 @@ INF OvmfPkg/GvtGopDxe/GvtGopDxe.inf
#INF OvmfPkg/XenIoPciDxe/XenIoPciDxe.inf
#INF OvmfPkg/XenBusDxe/XenBusDxe.inf
#INF OvmfPkg/XenPvBlkDxe/XenPvBlkDxe.inf
+INF RuleOverride=CSM OvmfPkg/Vbt/Vbt.inf
INF OvmfPkg/PlatformGopPolicy/PlatformGopPolicy.inf
!if $(SECURE_BOOT_ENABLE) == TRUE
Index: acrn-edk2/OvmfPkg/Vbt/Vbt.inf
===================================================================
--- /dev/null
+++ acrn-edk2/OvmfPkg/Vbt/Vbt.inf
@@ -0,0 +1,17 @@
+## @file
+# Vbt Binary
+#
+# Copyright (c) 2006 - 2011, Intel Corporation. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+##
+
+[Defines]
+ INF_VERSION = 0x00010005
+ BASE_NAME = Vbt
+ FILE_GUID = 1647B4F3-3E8A-4FEF-81C8-328ED647AB1A
+ MODULE_TYPE = USER_DEFINED
+ VERSION_STRING = 1.0
+
+[Binaries]
+ BIN|Vbt.bin|*

View File

@@ -0,0 +1,167 @@
.. _gpu-passthrough:
Enable GVT-d in ACRN
#########################################
This tutorial describes how to enable GVT-d in ACRN.
.. note:: It is recommended to have either a serial port
or SSH session open in the Service VM to be able to
continue interact with it after enabling GVT-d.
Introduction
*****************
Intel GVT-d is one flavor of graphics virtualization approaches also
names as Intel-Graphics-Device pass-through feature.
It based on Intel VT-d technology,
and some special graphics related configuration.
This flavor allows direct assignment of an entire GPUs prowess to a single user,
passing the native driver capabilities through the hypervisor without any limitations.
Verified version
*****************
- ACRN-hypervisor tag: **acrn-2020w17.4-140000p**
- ACRN-Kernel (Service VM kernel): **master** branch, commit id **095509221660daf82584ebdd8c50ea0078da3c2d**
- ACRN-EDK2 (OVMF): **ovmf-acrn** branch, commit id **0ff86f6b9a3500e4c7ea0c1064e77d98e9745947**
Prerequisites
*****************
Follow :ref:`these instructions <kbl-nuc-sdc>` to set up the ACRN Service VM.
Hardware Platform Supported
***************************
Currently, ACRN has enabled GVT-d on the following platforms:
* Kaby Lake
* Whiskey Lake
* Elkhart Lake
BIOS Setting
**********************
Kaby Lake Platform
==================
* set "IGD Minimum Memory" to "64MB" in "Devices → Video → IGD Minimum Memory".
Whiskey Lake Platform
=====================
* set "PM Support" to "Enabled" in "chipset → System Agent (SA) Configuration → Graphics Configuration → PM support".
* set "DVMT Pre-Allocated" to "64MB" in "chipset → System Agent (SA) Configuration → Graphics Configuration → DVMT Pre-Allocated"
Elkhart Lake Platform
=====================
* set "DMVT Pre-Allocated" to "64MB" in "Intel Advanced Menu → System Agent(SA) Configuration → Graphics Configuration → DMVT Pre-Allocated"
Pass-through GPU to Guest
**************************
1. Copy ``/usr/share/acrn/samples/nuc/launch_win.sh`` to ``install_win.sh``
::
cp /usr/share/acrn/samples/nuc/launch_win.sh ~/install_win.sh
2. Modify ``install_win.sh`` script to specify the Windows image you use.
3. Modify ``install_win.sh`` script to enable GVT-d.
Add the following commands before ``acrn-dm -A -m $mem_size -s 0:0,hostbridge \``
::
gpudevice=`cat /sys/bus/pci/devices/0000:00:02.0/device`
echo "8086 $gpudevice" > /sys/bus/pci/drivers/pci-stub/new_id
echo "0000:00:02.0" > /sys/bus/pci/devices/0000:00:02.0/driver/unbind
echo "0000:00:02.0" > /sys/bus/pci/drivers/pci-stub/bind
Replace ``-s 2,pci-gvt -G "$2" \`` with ``-s 2,passthru,0/2/0,gpu \``
4. Run the ``launch_win.sh``.
.. note:: If you want to pass-through GPU to Clear Linux User VM, the steps are
the same as above except your script.
Enable GVT-d GOP driver
***********************
Issue
======
When enabling GVT-d, guest OS couldnt light up physical screen
before OS driver load. So guest BIOS and
grub GUI couldnt be showed on the physical screen.
Reason
==========
Physical display is initialized by GOP driver or VBIOS
before OS driver load, but guest BIOS doesnt have them.
Solution
==========
Integrate GOP driver binary into OVMF as a Dxe driver,
then guest OVMF could see GOP driver and run it in
graphic pass-through environment.
So physical display could be initialized
by GOP and used by guest BIOS and guest grub.
Steps
======
1. fetch ACRN OVMF
::
git clone https://github.com/projectacrn/acrn-edk2.git
2. fetch vbt and gop driver
Fetch vbt and gop driver from the board manufacturer according to your CPU model name.
3. add vbt and gop driver into OVMF
::
cp IntelGopDriver.efi acrn-edk2/OvmfPkg/IntelGop/IntelGopDriver.efi
cp Vbt.bin acrn-edk2/OvmfPkg/Vbt/Vbt.bin
Notes:
- Please confirm these binaries names with board manufacturer.
4. git apply the following two patches
* `Use-the-default-vbt-released-with-GOP-driver.patch
<../_static/downloads/Use-the-default-vbt-released-with-GOP-driver.patch>`_
* `Integrate-IntelGopDriver-into-OVMF.patch
<../_static/downloads/Integrate-IntelGopDriver-into-OVMF.patch>`_
5. compile OVMF
::
cd acrn-edk2
git submodule update --init CryptoPkg/Library/OpensslLib/openssl
source edksetup.sh
make -C BaseTools
vim Conf/target.txt
ACTIVE_PLATFORM = OvmfPkg/OvmfPkgX64.dsc
TARGET_ARCH = X64
TOOL_CHAIN_TAG = GCC5
build -DFD_SIZE_2MB -DDEBUG_ON_SERIAL_PORT=TRUE
Notes:
- You need a build machine that has a GCC 5.X version installed.
- ``source edksetup.sh``, this step is needed for compilation every time
a shell is created.
- This will generate the binary at
``Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd``, transfer the binary to
your target machine.
- Modify the launch script to specify the OVMF you built just now.

View File

@@ -1,121 +0,0 @@
.. _skl-nuc-gpu-passthrough:
Enable GPU Passthrough on the Skylake NUC
#########################################
This community reference release for the Skylake NUC with GPU
passthrough is a one-time snapshot release and is not supported
or maintained.
Hardware platform
*****************
* `Skylake Skull Canyon NUC NUC6i7KYK
<https://www.intel.com/content/www/us/en/products/boards-kits/nuc/kits/nuc6i7kyk.html>`_
Software Configuration
**********************
* `acrn-hypervisor tag acrn-2018w39.6-140000p
<https://github.com/projectacrn/acrn-hypervisor/releases/tag/acrn-2018w39.6-140000p>`_
* `acrn-kernel tag acrn-2018w39.6-140000p
<https://github.com/projectacrn/acrn-kernel/releases/tag/acrn-2018w39.6-140000p>`_
* Clear Linux OS: version: 25130 (User VM and Service VM use this version)
Source code patches are provided in `skl-patches-for-acrn.tar file
<../_static/downloads/skl-patches-for-acrn.tar>`_ to work around or add support for
enabling GPU passthrough:
* 0001-hv-workaround-for-system-hang-on-non-apicv-devices.patch
* 0002-hv-More-changes-to-enable-GPU-passthru.patch
* 0003-dm-increase-interrupt-storm-threshold-for-gpu-passth.patch
* 0004-dm-passthrough-opregion-to-uos-gpu.patch
* 0005-dm-modify-launch-script-to-support-gpu-passthrough.patch
Software Setup
**************
Please follow the :ref:`kbl-nuc-sdc`, with the following changes:
1. Set up a Clear Linux Operating System
Clear Linux OS will update to the latest version during installation.
Run this command (as root) to roll back to version 25130, using the
``-x`` switch to ignore version mismatch::
# swupd verify -x --fix --picky -m 25130
# swupd autoupdate --disable
# reboot
#. Add the ACRN hypervisor to the EFI Partition
Refer to :ref:`getting-started-building`
to build the hypervisor, device model, and tools.
Download and untar this `skl-patches-for-acrn.tar file
<../_static/downloads/skl-patches-for-acrn.tar>`_, apply these patches to the
acrn-hypervisor, and build it::
$ git clone https://github.com/projectacrn/acrn-hypervisor
$ cd acrn-hypervisor
$ git checkout acrn-2018w39.6-140000p
$ curl https://projectacrn.github.io/latest/_static/downloads/skl-patches-for-acrn.tar | tar x
$ git am *.patch
$ make
This build process creates new ``acrn-dm``, ``acrn.efi`` and
``launch_uos.sh`` files.
#. Replace ``acrn-dm`` with this new version (as root)::
# cp build/devicemodel/acrn-dm /usr/bin/acrn-dm
#. Put the new ``acrn.efi`` hypervisor application (included in the
Clear Linux OS release) on the EFI partition (as root)::
# mount /dev/nvme0n1p1 /mnt
# mkdir /mnt/EFI/acrn
# cp build/hypervisor/acrn.efi /mnt/EFI/acrn/
#. Configure the EFI firmware to boot the ACRN hypervisor by default.
This assumes you are on an NVMe SSD as in the Skull Canyon::
# efibootmgr -c -l "\EFI\acrn\acrn.efi" -d /dev/nvme0n1 -p 1 -L "ACRN"
#. Create a boot entry for ACRN Service OS by making a few edits to the
``acrn.conf`` file (note the options line must be one long line, without
any line breaks)::
# vim /mnt/loader/entries/acrn.conf
title The ACRN Service OS
linux /EFI/org.clearlinux/kernel-org.clearlinux.pk414-sos.4.14.68-99
options pci_devices_ignore=(0:18:1) console=tty0 console=ttyS2 root=/dev/nvme0n1p3 rw rootwait ignore_loglevel no_timer_check consoleblank=0 i915.nuclear_pageflip=1 i915.avail_planes_per_pipe=0x01010F i915.domain_plane_owners=0x011111110000 i915.enable_gvt=1 i915.enable_guc=0 hvlog=2M@0x1FE00000
#. Don't Enable weston service (skip this step found in the NUC's getting
started guide).
#. Set up Reference User VM by running the modified ``launch_uos.sh`` in
``acrn-hypervisor/devicemodel/samples/nuc/launch_uos.sh``
#. After User VM is launched, do these steps to run GFX workloads:
a) install weston and glmark2::
#swupd bundle-add desktop glmark2
#) Add new user cl_uos::
# useradd cl_uos
# passwd cl_uos
# usermod -G wheel -a cl_uos
#) Enable weston service::
# systemctl enable weston@cl_uos
# systemctl start weston@cl_uos
#) Disable weston screen saver::
# vim .config/weston.ini
[core]
idle-time=0
#) run glmark2::
# glmark2-es2-wayland

View File

@@ -238,8 +238,7 @@ Boot Windows on ACRN with default configuration
#. Run the ``launch_win.sh``. The WaaG desktop displays on the HDMI monitor. #. Run the ``launch_win.sh``. The WaaG desktop displays on the HDMI monitor.
.. note:: We support GVT-g and GVT-d while launching Windows guest. .. note:: We support GVT-g and GVT-d while launching Windows guest.
If you use GVT-g, you can set up Weston in the Service VM, If you use GVT-g, you can set up Weston in the Service VM, and set up Weston as the
and follow the steps in :ref:`skl-nuc-gpu-passthrough` to set up Weston as the
desktop environment in the Service VM to experience Windows with the desktop environment in the Service VM to experience Windows with the
AcrnGT local display feature. If you use GVT-d, please set 64MB for DVMT Pre-Allocated and AcrnGT local display feature. If you use GVT-d, please set 64MB for DVMT Pre-Allocated and
Enabled for PM Support in BIOS at first, then only Windows has display. Enabled for PM Support in BIOS at first, then only Windows has display.