github/kata-containers
1
0
Fork 2
mirror of https://github.com/kata-containers/kata-containers.git synced 2026-04-26 10:32:28 +00:00
Code Issues Projects Releases Wiki Activity

docs: Remove obsolete and update documentation index

Browse Source 
This commit prunes the documentation tree by removing file
that are either no longer relevant to the current architecture
or have been superseded by newer guides.

Specifically, the doc Intel-Discrete-GPU-passthrough-and-Kata.md
and update using-Intel-QAT-and-kata.md index in nav.yaml

Refining the documentation helps ensure that new contributors
find accurate and up-to-date information.

Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit is contained in:
Alex Lyn
2026-04-20 15:31:24 +08:00
parent 59609463e0
commit 978f40d631
4 changed files with 2 additions and 563 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/.nav.yml
View File

@@ -15,7 +15,7 @@ nav:
- Use Cases:
- NVIDIA GPU Passthrough: use-cases/NVIDIA-GPU-passthrough-and-Kata-QEMU.md
- NVIDIA vGPU: use-cases/NVIDIA-GPU-passthrough-and-Kata.md
- Intel Discrete GPU: use-cases/Intel-Discrete-GPU-passthrough-and-Kata.md
- Intel QAT: use-cases/using-Intel-QAT-and-kata.md
- Contributing:
- Documentation: doc-contributing.md
- Misc:

2
docs/use-cases/GPU-passthrough-and-Kata.md
View File

@@ -2,5 +2,5 @@
Kata Containers supports passing certain GPUs from the host into the container. Select the GPU vendor for detailed information:
- [Intel Discrete GPUs](Intel-Discrete-GPU-passthrough-and-Kata.md)/[Intel Integrated GPUs](Intel-GPU-passthrough-and-Kata.md)
- [NVIDIA GPUs](NVIDIA-GPU-passthrough-and-Kata.md) and [Enabling NVIDIA GPU workloads using GPU passthrough with Kata Containers](NVIDIA-GPU-passthrough-and-Kata-QEMU.md)
- PLACE HOLDER: for other GPU vendors (e.g., AMD, Intel)

274
docs/use-cases/Intel-Discrete-GPU-passthrough-and-Kata.md
View File

@@ -1,274 +0,0 @@
# Using Intel Discrete GPU device with Kata Containers
This guide covers the use case for passing Intel Discrete GPUs to Kata.
These include the Intel® Data Center GPU Max Series and Intel® Data Center GPU Flex Series.
For integrated GPUs please refer to [Integrate-Intel-GPUs-with-Kata](Intel-GPU-passthrough-and-Kata.md)
> **Note:** These instructions are for a system that has an x86_64 CPU.
An Intel Discrete GPU can be passed to a Kata Container using GPU passthrough,
or SR-IOV passthrough.
In Intel GPU pass-through mode, an entire physical GPU is directly assigned to one VM.
In this mode of operation, the GPU is accessed exclusively by the Intel driver running in
the VM to which it is assigned. The GPU is not shared among VMs.
With SR-IOV mode, it is possible to pass a Virtual GPU instance to a virtual machine.
With this, multiple Virtual GPU instances can be carved out of a single physical GPU
and be passed to different VMs, allowing the GPU to be shared.
| Technology | Description |
|-|-|
| GPU passthrough | Physical GPU assigned to a single VM |
| SR-IOV passthrough | Physical GPU shared by multiple VMs |
## Hardware Requirements
Intel GPUs Recommended for Virtualization:
- Intel® Data Center GPU Max Series (`Ponte Vecchio`)
- Intel® Data Center GPU Flex Series (`Arctic Sound-M`)
- Intel® Data Center GPU Arc Series
The following steps outline the workflow for using an Intel Graphics device with Kata Containers.
## Host BIOS requirements
Hardware such as Intel Max and Flex series require larger PCI BARs.
For large BAR devices, MMIO mapping above the 4GB address space should be enabled in the PCI configuration of the BIOS.
Some hardware vendors use a different name in the BIOS, such as:
- Above 4GB Decoding
- Memory Hole for PCI MMIO
- Memory Mapped I/O above 4GB
## Host Kernel Requirements
For device passthrough to work with the Max and Flex Series, an out of tree kernel driver is required.
For Ubuntu 22.04 server, follow these instructions to install the out of tree GPU driver:
```bash
$ sudo apt update
$ sudo apt install -y gpg-agent wget
$ wget -qO - https://repositories.intel.com/gpu/intel-graphics.key | \
sudo gpg --dearmor --output /usr/share/keyrings/intel-graphics.gpg
$ source /etc/os-release
$ echo "deb [arch=amd64 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/gpu/ubuntu ${VERSION_CODENAME}/lts/2350 unified" | \
sudo tee /etc/apt/sources.list.d/intel-gpu-${VERSION_CODENAME}.list
$ sudo apt update
$ sudo apt install -y linux-headers-"$(uname -r)" flex bison intel-fw-gpu intel-i915-dkms xpu-smi
$ sudo reboot
```
For support on other distributions, please refer to [DGPU-docs](https://dgpu-docs.intel.com/driver/installation.html)
You can also install the driver from source which is maintained at [intel-gpu-i915-backports](https://github.com/intel-gpu/intel-gpu-i915-backports)
Detailed instructions for reference can be found at: https://github.com/intel-gpu/intel-gpu-i915-backports/blob/backport/main/docs/README_ubuntu.md.
Below are the steps for installing the driver from source on an Ubuntu 22.04 LTS system:
```bash
$ export I915_BRANCH="backport/main"
$ git clone -b ${I915_BRANCH} --depth 1 https://github.com/intel-gpu/intel-gpu-i915-backports.git
$ cd intel-gpu-i915-backports/
$ sudo apt install -y dkms make debhelper devscripts build-essential flex bison mawk
$ sudo apt install -y linux-headers-"$(uname -r)" linux-image-unsigned-"$(uname -r)"
$ make i915dkmsdeb-pkg
```
The above make command will create Debian package in parent folder: `intel-i915-dkms_<release version>.<kernel-version>.deb`
Install the package as:
```bash
$ sudo dpkg -i intel-i915-dkms_<release version>.<kernel-version>.deb
$ sudo reboot
```
Additionally, verify that the following kernel configs are enabled for your host kernel:
```
CONFIG_VFIO
CONFIG_VFIO_IOMMU_TYPE1
CONFIG_VFIO_PCI
```
## Host kernel command line
Your host kernel needs to be booted with `intel_iommu=on` and `i915.enable_iaf=0` on the kernel command
line.
1. Run the following to change the kernel command line using grub:
```bash
$ sudo vim /etc/default/grub
```
2. At the end of the GRUB_CMDLINE_LINUX_DEFAULT append the below line:
`intel_iommu=on iommu=pt i915.max_vfs=63 i915.enable_iaf=0`
3. Update grub as per OS distribution:
For Ubuntu:
```bash
$ sudo update-grub
```
For CentOS/RHEL:
```bash
$ sudo grub2-mkconfig -o /boot/grub2/grub.cfg
```
4. Reboot the system
```bash
$ sudo reboot
```
## Install and configure Kata Containers
To use this feature, you need Kata version 1.3.0 or above.
Follow the [Kata Containers setup instructions](../install/README.md)
to install the latest version of Kata.
To use large BARs devices (for example, NVIDIA Tesla P100), you need Kata version 1.11.0 or above.
In order to pass a GPU to a Kata Container, you need to enable the `hotplug_vfio_on_root_bus`
configuration in the Kata `configuration.toml` file as shown below.
```bash
$ sudo sed -i -e 's/^# *\(hotplug_vfio_on_root_bus\).*=.*$/\1 = true/g' /usr/share/defaults/kata-containers/configuration.toml
```
Make sure you are using the `q35` machine type by verifying `machine_type = "q35"` is
set in the `configuration.toml`. Make sure `pcie_root_port` is set to a positive value.
After making the above changes, configuration in the `configuration.toml` should look like this:
```
machine_type = "q35"
hotplug_vfio_on_root_bus = true
pcie_root_port = 1
```
## GPU passthrough with Kata Containers
Use the following steps to pass an Intel discrete GPU with Kata:
1. Find the Bus-Device-Function (BDF) for GPU device:
```
$ sudo lspci -nn -D | grep Display
```
Run the previous command to determine the BDF for the GPU device on host.<br/>
From the previous output, PCI address `0000:29:00.0` is assigned to the hardware GPU device.<br/>
We choose this BDF to use it later to unbind the GPU device from the host for the purpose of demonstration.<br/>
2. Find the IOMMU group for the GPU device:
```bash
$ BDF="0000:29:00.0"
$ readlink -e /sys/bus/pci/devices/$BDF/iommu_group
/sys/kernel/iommu_groups/27
```
The previous output shows that the GPU belongs to IOMMU group 27.
3. Bind the GPU to the `vfio-pci` device driver:
```bash
$ BDF="0000:29:00.0"
$ DEV="/sys/bus/pci/devices/$BDF"
$ echo "vfio-pci" | sudo tee "$DEV"/driver_override
$ echo $BDF | sudo tee "$DEV"/driver/unbind
$ echo "$BDF" | sudo tee "/sys/bus/pci/drivers_probe"
```
After you run the previous commands, the GPU is bound to `vfio-pci` driver.<br/>
A new directory with the IOMMU group number is created under `/dev/vfio`:
```bash
$ ls -l /dev/vfio
total 0
crw------- 1 root root 241, 0 May 18 15:38 27
crw-rw-rw- 1 root root 10, 196 May 18 15:37 vfio
```
Later, to return the device to the standard driver, we simply clear the
`driver_override` and re-probe the device, ex:
```bash
$ echo | sudo tee "$DEV/preferred_driver"
$ echo $BDF | sudo tee $DEV/driver/unbind
$ echo $BDF | sudo tee /sys/bus/pci/drivers_probe
```
5. Start a Kata container with GPU device:
```bash
$ sudo ctr --debug run --runtime "io.containerd.kata.v2" --device "/dev/vfio/27" --rm -t "docker.io/library/archlinux:latest" arch uname -r
```
Run `lspci` within the container to verify the GPU device is seen in the list of
the PCI devices. Note the vendor-device id of the GPU ("8086:0bd5") in the `lspci` output.
## SR-IOV mode for Intel Discrete GPUs
Use the following steps to pass an Intel Graphics device in SR-IOV mode to a Kata Container:
1. Find the BDF for GPU device:
```sh
$ sudo lspci -nn -D | grep Display
0000:29:00.0 Display controller [0380]: Intel Corporation Ponte Vecchio 1T [8086:0bd5] (rev 2f)
0000:3a:00.0 Display controller [0380]: Intel Corporation Ponte Vecchio 1T [8086:0bd5] (rev 2f)
0000:9a:00.0 Display controller [0380]: Intel Corporation Ponte Vecchio 1T [8086:0bd5] (rev 2f)
0000:ca:00.0 Display controller [0380]: Intel Corporation Ponte Vecchio 1T [8086:0bd5] (rev 2f)
```
Run the previous command to find out the BDF for the GPU device on host.
We choose the GPU with PCI address "0000:3a:00.0" to assign a GPU SR-IOV interface.
2. Carve out SR-IOV slice for the GPU:
List our total possible SR-IOV virtual interfaces for the GPU:
```bash
$ BDF="0000:3a:00.0"
$ cat "/sys/bus/pci/devices/$BDF/sriov_totalvfs"
63
```
Create SR-IOV interfaces for the GPU:
```sh
$ echo 4 | sudo tee /sys/bus/pci/devices/$BDF/sriov_numvfs
4
$ sudo lspci | grep Display
29:00.0 Display controller: Intel Corporation Ponte Vecchio 1T (rev 2f)
3a:00.0 Display controller: Intel Corporation Ponte Vecchio 1T (rev 2f)
3a:00.1 Display controller: Intel Corporation Ponte Vecchio 1T (rev 2f)
3a:00.2 Display controller: Intel Corporation Ponte Vecchio 1T (rev 2f)
3a:00.3 Display controller: Intel Corporation Ponte Vecchio 1T (rev 2f)
3a:00.4 Display controller: Intel Corporation Ponte Vecchio 1T (rev 2f)
9a:00.0 Display controller: Intel Corporation Ponte Vecchio 1T (rev 2f)
ca:00.0 Display controller: Intel Corporation Ponte Vecchio 1T (rev 2f)
```
The above output shows the SR-IOV interfaces created for the GPU.
3. Find the IOMMU group for the GPU SR-IOV interface(VGPU):
```bash
$ BDF="0000:3a:00:1"
$ readlink -e "/sys/bus/pci/devices/$BDF/iommu_group"
/sys/kernel/iommu_groups/437
$ ls -l /dev/vfio
total 0
crw------- 1 root root 241, 0 May 18 11:30 437
crw-rw-rw- 1 root root 10, 196 May 18 11:29 vfio
```
Now you can use the device node `/dev/vfio/437` in docker command line to pass
the VGPU to a Kata Container.
4. Start a Kata Containers container with GPU device enabled:
```bash
$ sudo ctr --debug run --runtime "io.containerd.kata.v2" --device /dev/vfio/437 --rm -t "docker.io/library/archlinux:latest" arch uname -r
```

287
docs/use-cases/Intel-GPU-passthrough-and-Kata.md
View File

@@ -1,287 +0,0 @@
# Using Intel GPU device with Kata Containers
An Intel Graphics device can be passed to a Kata Containers container using GPU
passthrough (Intel GVT-d) as well as GPU mediated passthrough (Intel GVT-g).
Intel GVT-d (one VM to one physical GPU) also named as Intel-Graphics-Device
passthrough feature is one flavor of graphics virtualization approach.
This flavor allows direct assignment of an entire GPU to a single user,
passing the native driver capabilities through the hypervisor without any limitations.
Intel GVT-g (multiple VMs to one physical GPU) is a full GPU virtualization solution
with mediated pass-through.<br/>
A virtual GPU instance is maintained for each VM, with part of performance critical
resources, directly assigned. The ability to run a native graphics driver inside a
VM without hypervisor intervention in performance critical paths, achieves a good
balance among performance, feature, and sharing capability.
| Technology | Description | Behaviour | Detail |
|-|-|-|-|
| Intel GVT-d | GPU passthrough | Physical GPU assigned to a single VM | Direct GPU assignment to VM without limitation |
| Intel GVT-g | GPU sharing | Physical GPU shared by multiple VMs | Mediated passthrough |
## Hardware Requirements
- For client platforms, 5th generation Intel® Core Processor Graphics or higher are required.
- For server platforms, E3_v4 or higher Xeon Processor Graphics are required.
The following steps outline the workflow for using an Intel Graphics device with Kata.
## Host Kernel Requirements
The following configurations need to be enabled on your host kernel:
```
CONFIG_VFIO_IOMMU_TYPE1=m
CONFIG_VFIO=m
CONFIG_VFIO_PCI=m
CONFIG_VFIO_MDEV=m
CONFIG_VFIO_MDEV_DEVICE=m
CONFIG_DRM_I915_GVT=m
CONFIG_DRM_I915_GVT_KVMGT=m
```
Your host kernel needs to be booted with `intel_iommu=on` on the kernel command
line.
## Install and configure Kata Containers
To use this feature, you need Kata version 1.3.0 or above.
Follow the [Kata Containers setup instructions](../install/README.md)
to install the latest version of Kata.
In order to pass a GPU to a Kata Container, you need to enable the `hotplug_vfio_on_root_bus`
configuration in the Kata `configuration.toml` file as shown below.
```
$ sudo sed -i -e 's/^# *\(hotplug_vfio_on_root_bus\).*=.*$/\1 = true/g' /usr/share/defaults/kata-containers/configuration.toml
```
Make sure you are using the `q35` machine type by verifying `machine_type = "q35"` is
set in the `configuration.toml`. Make sure `pcie_root_port` is set to a positive value.
## Build Kata Containers kernel with GPU support
The default guest kernel installed with Kata Containers does not provide GPU support.
To use an Intel GPU with Kata Containers, you need to build a kernel with the necessary
GPU support.
The following i915 kernel config options need to be enabled:
```
CONFIG_DRM=y
CONFIG_DRM_I915=y
CONFIG_DRM_I915_USERPTR=y
```
Build the Kata Containers kernel with the previous config options, using the instructions
described in [Building Kata Containers kernel](../../tools/packaging/kernel).
For further details on building and installing guest kernels, see [the developer guide](../Developer-Guide.md#install-guest-kernel-images).
There is an easy way to build a guest kernel that supports Intel GPU:
```
## Build guest kernel with ../../tools/packaging/kernel
# Prepare (download guest kernel source, generate .config)
$ ./build-kernel.sh -g intel -f setup
# Build guest kernel
$ ./build-kernel.sh -g intel build
# Install guest kernel
$ sudo -E ./build-kernel.sh -g intel install
/usr/share/kata-containers/vmlinux-intel-gpu.container -> vmlinux-5.4.15-70-intel-gpu
/usr/share/kata-containers/vmlinuz-intel-gpu.container -> vmlinuz-5.4.15-70-intel-gpu
```
Before using the new guest kernel, please update the `kernel` parameters in `configuration.toml`.
```
kernel = "/usr/share/kata-containers/vmlinuz-intel-gpu.container"
```
## GVT-d with Kata Containers
Use the following steps to pass an Intel Graphics device in GVT-d mode with Kata:
1. Find the Bus-Device-Function (BDF) for GPU device:
```
$ sudo lspci -nn -D | grep Graphics
0000:00:02.0 VGA compatible controller [0300]: Intel Corporation Broadwell-U Integrated Graphics [8086:1616] (rev 09)
```
Run the previous command to determine the BDF for the GPU device on host.<br/>
From the previous output, PCI address `0000:00:02.0` is assigned to the hardware GPU device.<br/>
This BDF is used later to unbind the GPU device from the host.<br/>
"8086 1616" is the device ID of the hardware GPU device. It is used later to
rebind the GPU device to `vfio-pci` driver.
2. Find the IOMMU group for the GPU device:
```
$ BDF="0000:00:02.0"
$ readlink -e /sys/bus/pci/devices/$BDF/iommu_group
/sys/kernel/iommu_groups/1
```
The previous output shows that the GPU belongs to IOMMU group 1.
3. Unbind the GPU:
```
$ echo $BDF | sudo tee /sys/bus/pci/devices/$BDF/driver/unbind
```
4. Bind the GPU to the `vfio-pci` device driver:
```
$ sudo modprobe vfio-pci
$ echo 8086 1616 | sudo tee /sys/bus/pci/drivers/vfio-pci/new_id
$ echo $BDF | sudo tee --append /sys/bus/pci/drivers/vfio-pci/bind
```
After you run the previous commands, the GPU is bound to `vfio-pci` driver.<br/>
A new directory with the IOMMU group number is created under `/dev/vfio`:
```
$ ls -l /dev/vfio
total 0
crw------- 1 root root 241, 0 May 18 15:38 1
crw-rw-rw- 1 root root 10, 196 May 18 15:37 vfio
```
5. Start a Kata container with GPU device:
```
$ sudo docker run -it --runtime=kata-runtime --rm --device /dev/vfio/1 -v /dev:/dev debian /bin/bash
```
Run `lspci` within the container to verify the GPU device is seen in the list of
the PCI devices. Note the vendor-device id of the GPU ("8086:1616") in the `lspci` output.
```
$ lspci -nn -D
0000:00:00.0 Class [0600]: Device [8086:1237] (rev 02)
0000:00:01.0 Class [0601]: Device [8086:7000]
0000:00:01.1 Class [0101]: Device [8086:7010]
0000:00:01.3 Class [0680]: Device [8086:7113] (rev 03)
0000:00:02.0 Class [0604]: Device [1b36:0001]
0000:00:03.0 Class [0780]: Device [1af4:1003]
0000:00:04.0 Class [0100]: Device [1af4:1004]
0000:00:05.0 Class [0002]: Device [1af4:1009]
0000:00:06.0 Class [0200]: Device [1af4:1000]
0000:00:0f.0 Class [0300]: Device [8086:1616] (rev 09)
```
Additionally, you can access the device node for the graphics device:
```
$ ls /dev/dri
card0 renderD128
```
## GVT-g with Kata Containers
For GVT-g, you append `i915.enable_gvt=1` in addition to `intel_iommu=on`
on your host kernel command line and then reboot your host.
Use the following steps to pass an Intel Graphics device in GVT-g mode to a Kata Container:
1. Find the BDF for GPU device:
```
$ sudo lspci -nn -D | grep Graphics
0000:00:02.0 VGA compatible controller [0300]: Intel Corporation Broadwell-U Integrated Graphics [8086:1616] (rev 09)
```
Run the previous command to find out the BDF for the GPU device on host.
The previous output shows PCI address "0000:00:02.0" is assigned to the GPU device.
2. Choose the MDEV (Mediated Device) type for VGPU (Virtual GPU):
For background on `mdev` types, please follow this [kernel documentation](https://github.com/torvalds/linux/blob/master/Documentation/driver-api/vfio-mediated-device.rst).
* List out the `mdev` types for the VGPU:
```
$ BDF="0000:00:02.0"
$ ls /sys/devices/pci0000:00/$BDF/mdev_supported_types
i915-GVTg_V4_1 i915-GVTg_V4_2 i915-GVTg_V4_4 i915-GVTg_V4_8
```
* Inspect the `mdev` types and choose one that fits your requirement:
```
$ cd /sys/devices/pci0000:00/0000:00:02.0/mdev_supported_types/i915-GVTg_V4_8 && ls
available_instances create description device_api devices
$ cat description
low_gm_size: 64MB
high_gm_size: 384MB
fence: 4
resolution: 1024x768
weight: 2
$ cat available_instances
7
```
The output of file `description` represents the GPU resources that are
assigned to the VGPU with specified MDEV type.The output of file `available_instances`
represents the remaining amount of VGPUs you can create with specified MDEV type.
3. Create a VGPU:
* Generate a UUID:
```
$ gpu_uuid=$(uuid)
```
* Write the UUID to the `create` file under the chosen `mdev` type:
```
$ echo $(gpu_uuid) | sudo tee /sys/devices/pci0000:00/0000:00:02.0/mdev_supported_types/i915-GVTg_V4_8/create
```
4. Find the IOMMU group for the VGPU:
```
$ ls -la /sys/devices/pci0000:00/0000:00:02.0/mdev_supported_types/i915-GVTg_V4_8/devices/${gpu_uuid}/iommu_group
lrwxrwxrwx 1 root root 0 May 18 14:35 devices/bbc4aafe-5807-11e8-a43e-03533cceae7d/iommu_group -> ../../../../kernel/iommu_groups/0
$ ls -l /dev/vfio
total 0
crw------- 1 root root 241, 0 May 18 11:30 0
crw-rw-rw- 1 root root 10, 196 May 18 11:29 vfio
```
The IOMMU group "0" is created from the previous output.<br/>
Now you can use the device node `/dev/vfio/0` in docker command line to pass
the VGPU to a Kata Container.
5. Start Kata container with GPU device enabled:
```
$ sudo docker run -it --runtime=kata-runtime --rm --device /dev/vfio/0 -v /dev:/dev debian /bin/bash
$ lspci -nn -D
0000:00:00.0 Class [0600]: Device [8086:1237] (rev 02)
0000:00:01.0 Class [0601]: Device [8086:7000]
0000:00:01.1 Class [0101]: Device [8086:7010]
0000:00:01.3 Class [0680]: Device [8086:7113] (rev 03)
0000:00:02.0 Class [0604]: Device [1b36:0001]
0000:00:03.0 Class [0780]: Device [1af4:1003]
0000:00:04.0 Class [0100]: Device [1af4:1004]
0000:00:05.0 Class [0002]: Device [1af4:1009]
0000:00:06.0 Class [0200]: Device [1af4:1000]
0000:00:0f.0 Class [0300]: Device [8086:1616] (rev 09)
```
BDF "0000:00:0f.0" is assigned to the VGPU device.
Additionally, you can access the device node for the graphics device:
```
$ ls /dev/dri
card0 renderD128
```