mirror of
https://github.com/projectacrn/acrn-hypervisor.git
synced 2025-09-11 05:39:45 +00:00
doc:add tutorial about enabling GVT-d
Previous tutorial "Enable GPU Passthrough on the Skylake NUC" is out of date, so delete it here. v2 -> v1: add attached file Signed-off-by: Junming Liu <junming.liu@intel.com>
This commit is contained in:
Junming Liu
deb-intel
167
doc/tutorials/gpu-passthru.rst
Normal file
167
doc/tutorials/gpu-passthru.rst
Normal file
@@ -0,0 +1,167 @@
|
||||
.. _gpu-passthrough:
|
||||
|
||||
Enable GVT-d in ACRN
|
||||
#########################################
|
||||
|
||||
This tutorial describes how to enable GVT-d in ACRN.
|
||||
|
||||
.. note:: It is recommended to have either a serial port
|
||||
or SSH session open in the Service VM to be able to
|
||||
continue interact with it after enabling GVT-d.
|
||||
|
||||
Introduction
|
||||
*****************
|
||||
Intel GVT-d is one flavor of graphics virtualization approaches also
|
||||
names as Intel-Graphics-Device pass-through feature.
|
||||
It based on Intel VT-d technology,
|
||||
and some special graphics related configuration.
|
||||
This flavor allows direct assignment of an entire GPU’s prowess to a single user,
|
||||
passing the native driver capabilities through the hypervisor without any limitations.
|
||||
|
||||
Verified version
|
||||
*****************
|
||||
|
||||
- ACRN-hypervisor tag: **acrn-2020w17.4-140000p**
|
||||
- ACRN-Kernel (Service VM kernel): **master** branch, commit id **095509221660daf82584ebdd8c50ea0078da3c2d**
|
||||
- ACRN-EDK2 (OVMF): **ovmf-acrn** branch, commit id **0ff86f6b9a3500e4c7ea0c1064e77d98e9745947**
|
||||
|
||||
Prerequisites
|
||||
*****************
|
||||
Follow :ref:`these instructions <kbl-nuc-sdc>` to set up the ACRN Service VM.
|
||||
|
||||
Hardware Platform Supported
|
||||
***************************
|
||||
Currently, ACRN has enabled GVT-d on the following platforms:
|
||||
|
||||
* Kaby Lake
|
||||
* Whiskey Lake
|
||||
* Elkhart Lake
|
||||
|
||||
BIOS Setting
|
||||
**********************
|
||||
|
||||
Kaby Lake Platform
|
||||
==================
|
||||
* set "IGD Minimum Memory" to "64MB" in "Devices → Video → IGD Minimum Memory".
|
||||
|
||||
Whiskey Lake Platform
|
||||
=====================
|
||||
* set "PM Support" to "Enabled" in "chipset → System Agent (SA) Configuration → Graphics Configuration → PM support".
|
||||
* set "DVMT Pre-Allocated" to "64MB" in "chipset → System Agent (SA) Configuration → Graphics Configuration → DVMT Pre-Allocated"
|
||||
|
||||
Elkhart Lake Platform
|
||||
=====================
|
||||
* set "DMVT Pre-Allocated" to "64MB" in "Intel Advanced Menu → System Agent(SA) Configuration → Graphics Configuration → DMVT Pre-Allocated"
|
||||
|
||||
Pass-through GPU to Guest
|
||||
**************************
|
||||
|
||||
1. Copy ``/usr/share/acrn/samples/nuc/launch_win.sh`` to ``install_win.sh``
|
||||
|
||||
::
|
||||
|
||||
cp /usr/share/acrn/samples/nuc/launch_win.sh ~/install_win.sh
|
||||
|
||||
2. Modify ``install_win.sh`` script to specify the Windows image you use.
|
||||
|
||||
3. Modify ``install_win.sh`` script to enable GVT-d.
|
||||
|
||||
Add the following commands before ``acrn-dm -A -m $mem_size -s 0:0,hostbridge \``
|
||||
|
||||
::
|
||||
|
||||
gpudevice=`cat /sys/bus/pci/devices/0000:00:02.0/device`
|
||||
|
||||
echo "8086 $gpudevice" > /sys/bus/pci/drivers/pci-stub/new_id
|
||||
echo "0000:00:02.0" > /sys/bus/pci/devices/0000:00:02.0/driver/unbind
|
||||
echo "0000:00:02.0" > /sys/bus/pci/drivers/pci-stub/bind
|
||||
|
||||
Replace ``-s 2,pci-gvt -G "$2" \`` with ``-s 2,passthru,0/2/0,gpu \``
|
||||
|
||||
4. Run the ``launch_win.sh``.
|
||||
|
||||
.. note:: If you want to pass-through GPU to Clear Linux User VM, the steps are
|
||||
the same as above except your script.
|
||||
|
||||
Enable GVT-d GOP driver
|
||||
***********************
|
||||
|
||||
Issue
|
||||
======
|
||||
When enabling GVT-d, guest OS couldn’t light up physical screen
|
||||
before OS driver load. So guest BIOS and
|
||||
grub GUI couldn’t be showed on the physical screen.
|
||||
|
||||
Reason
|
||||
==========
|
||||
Physical display is initialized by GOP driver or VBIOS
|
||||
before OS driver load, but guest BIOS doesn’t have them.
|
||||
|
||||
Solution
|
||||
==========
|
||||
Integrate GOP driver binary into OVMF as a Dxe driver,
|
||||
then guest OVMF could see GOP driver and run it in
|
||||
graphic pass-through environment.
|
||||
So physical display could be initialized
|
||||
by GOP and used by guest BIOS and guest grub.
|
||||
|
||||
Steps
|
||||
======
|
||||
1. fetch ACRN OVMF
|
||||
|
||||
::
|
||||
|
||||
git clone https://github.com/projectacrn/acrn-edk2.git
|
||||
|
||||
2. fetch vbt and gop driver
|
||||
|
||||
Fetch vbt and gop driver from the board manufacturer according to your CPU model name.
|
||||
|
||||
3. add vbt and gop driver into OVMF
|
||||
|
||||
::
|
||||
|
||||
cp IntelGopDriver.efi acrn-edk2/OvmfPkg/IntelGop/IntelGopDriver.efi
|
||||
cp Vbt.bin acrn-edk2/OvmfPkg/Vbt/Vbt.bin
|
||||
|
||||
Notes:
|
||||
|
||||
- Please confirm these binaries names with board manufacturer.
|
||||
|
||||
4. git apply the following two patches
|
||||
|
||||
* `Use-the-default-vbt-released-with-GOP-driver.patch
|
||||
<../_static/downloads/Use-the-default-vbt-released-with-GOP-driver.patch>`_
|
||||
* `Integrate-IntelGopDriver-into-OVMF.patch
|
||||
<../_static/downloads/Integrate-IntelGopDriver-into-OVMF.patch>`_
|
||||
|
||||
5. compile OVMF
|
||||
|
||||
::
|
||||
|
||||
cd acrn-edk2
|
||||
git submodule update --init CryptoPkg/Library/OpensslLib/openssl
|
||||
|
||||
source edksetup.sh
|
||||
make -C BaseTools
|
||||
|
||||
vim Conf/target.txt
|
||||
|
||||
ACTIVE_PLATFORM = OvmfPkg/OvmfPkgX64.dsc
|
||||
TARGET_ARCH = X64
|
||||
TOOL_CHAIN_TAG = GCC5
|
||||
|
||||
build -DFD_SIZE_2MB -DDEBUG_ON_SERIAL_PORT=TRUE
|
||||
|
||||
Notes:
|
||||
|
||||
- You need a build machine that has a GCC 5.X version installed.
|
||||
|
||||
- ``source edksetup.sh``, this step is needed for compilation every time
|
||||
a shell is created.
|
||||
|
||||
- This will generate the binary at
|
||||
``Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd``, transfer the binary to
|
||||
your target machine.
|
||||
|
||||
- Modify the launch script to specify the OVMF you built just now.
|
||||
@@ -1,121 +0,0 @@
|
||||
.. _skl-nuc-gpu-passthrough:
|
||||
|
||||
Enable GPU Passthrough on the Skylake NUC
|
||||
#########################################
|
||||
|
||||
This community reference release for the Skylake NUC with GPU
|
||||
passthrough is a one-time snapshot release and is not supported
|
||||
or maintained.
|
||||
|
||||
Hardware platform
|
||||
*****************
|
||||
|
||||
* `Skylake Skull Canyon NUC NUC6i7KYK
|
||||
<https://www.intel.com/content/www/us/en/products/boards-kits/nuc/kits/nuc6i7kyk.html>`_
|
||||
|
||||
Software Configuration
|
||||
**********************
|
||||
|
||||
* `acrn-hypervisor tag acrn-2018w39.6-140000p
|
||||
<https://github.com/projectacrn/acrn-hypervisor/releases/tag/acrn-2018w39.6-140000p>`_
|
||||
* `acrn-kernel tag acrn-2018w39.6-140000p
|
||||
<https://github.com/projectacrn/acrn-kernel/releases/tag/acrn-2018w39.6-140000p>`_
|
||||
* Clear Linux OS: version: 25130 (User VM and Service VM use this version)
|
||||
|
||||
Source code patches are provided in `skl-patches-for-acrn.tar file
|
||||
<../_static/downloads/skl-patches-for-acrn.tar>`_ to work around or add support for
|
||||
enabling GPU passthrough:
|
||||
|
||||
* 0001-hv-workaround-for-system-hang-on-non-apicv-devices.patch
|
||||
* 0002-hv-More-changes-to-enable-GPU-passthru.patch
|
||||
* 0003-dm-increase-interrupt-storm-threshold-for-gpu-passth.patch
|
||||
* 0004-dm-passthrough-opregion-to-uos-gpu.patch
|
||||
* 0005-dm-modify-launch-script-to-support-gpu-passthrough.patch
|
||||
|
||||
Software Setup
|
||||
**************
|
||||
|
||||
Please follow the :ref:`kbl-nuc-sdc`, with the following changes:
|
||||
|
||||
1. Set up a Clear Linux Operating System
|
||||
|
||||
Clear Linux OS will update to the latest version during installation.
|
||||
Run this command (as root) to roll back to version 25130, using the
|
||||
``-x`` switch to ignore version mismatch::
|
||||
|
||||
# swupd verify -x --fix --picky -m 25130
|
||||
# swupd autoupdate --disable
|
||||
# reboot
|
||||
|
||||
#. Add the ACRN hypervisor to the EFI Partition
|
||||
|
||||
Refer to :ref:`getting-started-building`
|
||||
to build the hypervisor, device model, and tools.
|
||||
|
||||
Download and untar this `skl-patches-for-acrn.tar file
|
||||
<../_static/downloads/skl-patches-for-acrn.tar>`_, apply these patches to the
|
||||
acrn-hypervisor, and build it::
|
||||
|
||||
$ git clone https://github.com/projectacrn/acrn-hypervisor
|
||||
$ cd acrn-hypervisor
|
||||
$ git checkout acrn-2018w39.6-140000p
|
||||
$ curl https://projectacrn.github.io/latest/_static/downloads/skl-patches-for-acrn.tar | tar x
|
||||
$ git am *.patch
|
||||
$ make
|
||||
|
||||
This build process creates new ``acrn-dm``, ``acrn.efi`` and
|
||||
``launch_uos.sh`` files.
|
||||
|
||||
#. Replace ``acrn-dm`` with this new version (as root)::
|
||||
|
||||
# cp build/devicemodel/acrn-dm /usr/bin/acrn-dm
|
||||
|
||||
#. Put the new ``acrn.efi`` hypervisor application (included in the
|
||||
Clear Linux OS release) on the EFI partition (as root)::
|
||||
|
||||
# mount /dev/nvme0n1p1 /mnt
|
||||
# mkdir /mnt/EFI/acrn
|
||||
# cp build/hypervisor/acrn.efi /mnt/EFI/acrn/
|
||||
|
||||
#. Configure the EFI firmware to boot the ACRN hypervisor by default.
|
||||
This assumes you are on an NVMe SSD as in the Skull Canyon::
|
||||
|
||||
# efibootmgr -c -l "\EFI\acrn\acrn.efi" -d /dev/nvme0n1 -p 1 -L "ACRN"
|
||||
|
||||
#. Create a boot entry for ACRN Service OS by making a few edits to the
|
||||
``acrn.conf`` file (note the options line must be one long line, without
|
||||
any line breaks)::
|
||||
|
||||
# vim /mnt/loader/entries/acrn.conf
|
||||
title The ACRN Service OS
|
||||
linux /EFI/org.clearlinux/kernel-org.clearlinux.pk414-sos.4.14.68-99
|
||||
options pci_devices_ignore=(0:18:1) console=tty0 console=ttyS2 root=/dev/nvme0n1p3 rw rootwait ignore_loglevel no_timer_check consoleblank=0 i915.nuclear_pageflip=1 i915.avail_planes_per_pipe=0x01010F i915.domain_plane_owners=0x011111110000 i915.enable_gvt=1 i915.enable_guc=0 hvlog=2M@0x1FE00000
|
||||
|
||||
#. Don't Enable weston service (skip this step found in the NUC's getting
|
||||
started guide).
|
||||
|
||||
#. Set up Reference User VM by running the modified ``launch_uos.sh`` in
|
||||
``acrn-hypervisor/devicemodel/samples/nuc/launch_uos.sh``
|
||||
|
||||
#. After User VM is launched, do these steps to run GFX workloads:
|
||||
|
||||
a) install weston and glmark2::
|
||||
|
||||
#swupd bundle-add desktop glmark2
|
||||
#) Add new user cl_uos::
|
||||
|
||||
# useradd cl_uos
|
||||
# passwd cl_uos
|
||||
# usermod -G wheel -a cl_uos
|
||||
#) Enable weston service::
|
||||
|
||||
# systemctl enable weston@cl_uos
|
||||
# systemctl start weston@cl_uos
|
||||
#) Disable weston screen saver::
|
||||
|
||||
# vim .config/weston.ini
|
||||
[core]
|
||||
idle-time=0
|
||||
#) run glmark2::
|
||||
|
||||
# glmark2-es2-wayland
|
||||
@@ -238,8 +238,7 @@ Boot Windows on ACRN with default configuration
|
||||
#. Run the ``launch_win.sh``. The WaaG desktop displays on the HDMI monitor.
|
||||
|
||||
.. note:: We support GVT-g and GVT-d while launching Windows guest.
|
||||
If you use GVT-g, you can set up Weston in the Service VM,
|
||||
and follow the steps in :ref:`skl-nuc-gpu-passthrough` to set up Weston as the
|
||||
If you use GVT-g, you can set up Weston in the Service VM, and set up Weston as the
|
||||
desktop environment in the Service VM to experience Windows with the
|
||||
AcrnGT local display feature. If you use GVT-d, please set 64MB for DVMT Pre-Allocated and
|
||||
Enabled for PM Support in BIOS at first, then only Windows has display.
|
||||
|
||||
Reference in New Issue
Block a user