From bc385fcd880e3e8573b21613dc7809c8188f4852 Mon Sep 17 00:00:00 2001 From: Zvonko Kaiser Date: Fri, 17 Jul 2026 14:41:49 +0000 Subject: [PATCH] docs: backtick technical terms added by the Phase 0-3 sweep Same treatment as the earlier spellcheck pass: wrap identifiers and hardware acronyms introduced by the sweep rewrite in backticks so cspell's inline-code ignore applies. Assisted-by: Claude Signed-off-by: Zvonko Kaiser --- .../hypervisor/src/qemu/ARCHITECTURE.md | 62 +++++++++---------- 1 file changed, 31 insertions(+), 31 deletions(-) diff --git a/src/runtime-rs/crates/hypervisor/src/qemu/ARCHITECTURE.md b/src/runtime-rs/crates/hypervisor/src/qemu/ARCHITECTURE.md index 385e23c62a..2b2037a0cc 100644 --- a/src/runtime-rs/crates/hypervisor/src/qemu/ARCHITECTURE.md +++ b/src/runtime-rs/crates/hypervisor/src/qemu/ARCHITECTURE.md @@ -46,7 +46,7 @@ This pattern is duplicated across `DeviceVhostUserFs`, `DeviceVirtioBlk`, `VhostVsock`, `DeviceVirtioNet`, `DeviceVirtioSerial`, `DeviceVirtconsole`, `DeviceRng`, `DeviceIntelIommu`, and `DeviceVirtioScsi`. -### 2. Stringly-typed flat `Machine` struct +### 2. `Stringly-typed` flat `Machine` struct ```rust struct Machine { @@ -187,7 +187,7 @@ re-scheduled Pods fail remote attestation (#12329). Pinning to `EPYC-v4` gives every SNP guest the same deterministic CPUID regardless of which silicon generation it lands on. The tradeoff is that -`EPYC-v4` strips AVX-512 and VAES extensions, cutting AES-GCM throughput +`EPYC-v4` strips AVX-512 and `VAES` extensions, cutting AES-GCM throughput roughly in half (~4 GB/s instead of ~8 GB/s), which is measurable when an H100 GPU is the encryption bottleneck (#12382). @@ -277,11 +277,11 @@ pub struct SmmuV3Config { } ``` -**SMMU grouping rule:** GPUs that share a physical SMMU on the host **must** be +**`SMMU` grouping rule:** GPUs that share a physical `SMMU` on the host **must** be placed on the same `PciRootComplex` in the guest (they share the same `arm-smmuv3` device). The IOMMU group boundaries in host sysfs determine the grouping. See [Config 3](#config-3--4-gpus-2-gpus-per-smmu-33-numa-nodes) for -the 2-GPUs-per-SMMU topology. +the `2-GPUs-per-SMMU` topology. #### `Objects` — shared QEMU `-object` backends @@ -414,7 +414,7 @@ impl Platform { ### NUMA Layout Rules -The guest Linux kernel processes ACPI SRAT entries in a fixed order: +The guest Linux kernel processes ACPI `SRAT` entries in a fixed order: 1. **CPU Affinity** — nodes with a `cpus=` range (CpuMem nodes, one per socket) 2. **Generic Affinity** — initiator nodes for PCIe devices (8 per GPU for MIG) @@ -466,8 +466,8 @@ caller checks `config.memory_info.enable_hugepages` and calls `Platform::with_hugepages("/dev/hugepages/")` once. No device code changes. EGM memory is natively physically contiguous, so vEGM implicitly satisfies the -vCMDQ contiguity requirement without hugepages. `cmdqv=on` is still required -in the `arm-smmuv3` device args when vCMDQ is enabled. +`vCMDQ` contiguity requirement without hugepages. `cmdqv=on` is still required +in the `arm-smmuv3` device args when `vCMDQ` is enabled. ### Emission order @@ -486,7 +486,7 @@ carries `memory-backend=` on the machine flag itself: 7. `PciTopology::roots` — `pxb-pcie`, `arm-smmuv3`, root ports, vfio devices 8. `Objects::acpi_links` — `acpi-generic-initiator` (8×GPU), `acpi-egm-memory` (1×GPU) -Steps 4–6 must be in that order to match Linux ACPI SRAT processing. +Steps 4–6 must be in that order to match Linux ACPI `SRAT` processing. **Q35 (x86_64)** — machine line carries no memory-backend reference; backends follow the machine line, interleaved with their NUMA node: @@ -495,7 +495,7 @@ follow the machine line, interleaved with their NUMA node: 2. `-machine q35,...` 3. Per socket: `-object memory-backend-{ram,file}` then `-numa node,memdev=,cpus=` 4. `-numa dist` entries -5. `PciTopology::roots` — `pxb-pcie`, root ports, per-device iommufd, `vfio-pci` +5. `PciTopology::roots` — `pxb-pcie`, root ports, per-device `iommufd`, `vfio-pci` 6. `PciTopology::pcie_root_port` — pre-provisioned empty ports on `pcie.0` ### VFIO Device Assignment Model @@ -585,7 +585,7 @@ All Grace configurations share these constants: - Host kernel driver: `nvgrace-gpu-vfio-pci` (replaces standard `vfio-pci`) - EGM kernel module: `nvgrace-egm` (creates `/dev/egm*` character devices) -### Config 1 — Single GPU, 1 SMMU (9 NUMA nodes) +### Config 1 — Single GPU, 1 `SMMU` (9 NUMA nodes) ```text -object iommufd,id=iommufd0 @@ -606,10 +606,10 @@ All Grace configurations share these constants: `HostTopology`: 1 socket, 1 `GpuSmmuGroup { pci_bus_addrs: ["0008:06:00.0"], socket: 0 }`. -### Config 2 — 4 GPUs, 1 GPU per SMMU (33 NUMA nodes) +### Config 2 — 4 GPUs, 1 GPU per `SMMU` (33 NUMA nodes) Each GPU gets its own `PciRootComplex` (one `pxb-pcie` + one `arm-smmuv3` + one -root port). Repeat the pxb-pcie/smmuv3/root-port/vfio block 4 times: +root port). Repeat the pxb-pcie/`smmuv3`/root-port/vfio block 4 times: ```text -object iommufd,id=iommufd0 @@ -629,9 +629,9 @@ root port). Repeat the pxb-pcie/smmuv3/root-port/vfio block 4 times: `HostTopology`: 1 socket, 4 `GpuSmmuGroup` each with 1 address. -### Config 3 — 4 GPUs, 2 GPUs per SMMU (33 NUMA nodes) +### Config 3 — 4 GPUs, 2 GPUs per `SMMU` (33 NUMA nodes) -GPUs sharing a physical SMMU share one `PciRootComplex` with **2 root ports**. +GPUs sharing a physical `SMMU` share one `PciRootComplex` with **2 root ports**. 2 complexes × 2 GPUs each: ```text @@ -667,13 +667,13 @@ with the NIC's PCI address). That root port does **not** emit `acpi-generic-initiator` links — the NIC has no GPU memory to online. `VfioDevice` carries a `kind: VfioDeviceKind` field (enum `Gpu` / `Nic` / …) -that gates initiator emission. The NIC shares the host SMMU with no GPU on its +that gates initiator emission. The NIC shares the host `SMMU` with no GPU on its bus, so it gets its own `PciRootComplex`. -### Config 5 — vCMDQ (hugepages + SMMU command-queue virtualisation) +### Config 5 — `vCMDQ` (hugepages + `SMMU` command-queue virtualisation) Same PCIe topology as Config 1 or 2, but `MemoryBackend::Ram` is replaced with -`MemoryBackend::File` for physically contiguous memory (required by the vCMDQ +`MemoryBackend::File` for physically contiguous memory (required by the `vCMDQ` hardware for the queue base address), and `cmdqv=on` is added to `arm-smmuv3`: ```text @@ -764,10 +764,10 @@ each supported machine type and that `HostTopology` round-trips through ### Phase 2 — Platform emission for virt / Grace - `Platform::to_qemu_args` implemented for `Machine::Virt`. -- Emission order: iommufd → backends → machine → NUMA nodes → pxb+smmuv3+ports+vfio → acpi_links. -- `Platform::with_hugepages` wires `memory-backend-file` + `cmdqv=on` on smmuv3. +- Emission order: `iommufd` → backends → machine → NUMA nodes → pxb+`smmuv3`+ports+vfio → acpi_links. +- `Platform::with_hugepages` wires `memory-backend-file` + `cmdqv=on` on `smmuv3`. - All 7 Grace fixture tests written; ignored pending Phase 4 (apply_host_defaults). -- Q35 and s390x/pseries emit `todo!()` — unblocked in Phase 3. +- Q35 and s390x/`pseries` emit `todo!()` — unblocked in Phase 3. ### Phase 3 — Q35 emission and CoCo support @@ -800,12 +800,12 @@ each supported machine type and that `HostTopology` round-trips through ### Phase 4 — Multi-RC PCIe and NUMA layout - Emit `pxb-pcie` (with `numa_node=`) + per-RC `arm-smmuv3` from `PciTopology`. -- Support N root ports per `PciRootComplex` (Config 3 shape: 2 GPUs per SMMU). +- Support N root ports per `PciRootComplex` (Config 3 shape: 2 GPUs per `SMMU`). - Add `VfioPciNoHotplug` with typed `IommufdRef` and `VfioDeviceKind`. - Emit NUMA nodes in the correct order (CpuMem → GPU initiators → memory-only). - `apply_host_defaults` wired end-to-end: Configs 1–4 golden fixtures pass. -### Phase 5 — vCMDQ and vEGM +### Phase 5 — `vCMDQ` and vEGM - `SmmuV3 { cmdqv: true }` + `MemoryBackend::File { path: "/dev/hugepages/", … }`. - `AcpiPciNodeLink::EgmMemory` + per-socket `MemoryBackend::File { path: "/dev/egmN", … }`. @@ -932,8 +932,8 @@ Key observations from the SEV-SNP + GPU invocation: CoCo uses RAM backend (not file-backed `/dev/shm`) — single NUMA node - GPU passthrough via `pxb-pcie + pcie-root-port + vfio-pci` (same shape as Grace but `vfio-pci` NOT `vfio-pci-nohotplug`, no `arm-smmuv3` — x86 uses global IOMMU) -- iommufd is **per-device** (`id=iommufdvfio-`), NOT the shared `iommufd0` - used on Grace; one iommufd object per GPU +- `iommufd` is **per-device** (`id=iommufdvfio-`), NOT the shared `iommufd0` + used on Grace; one `iommufd` object per GPU - `x-pci-vendor-id=0x10de,x-pci-device-id=0x2321` overrides required so the guest sees the correct device IDs for measured boot / attestation - `pxb-pcie bus_nr=32` (not the Grace 1-indexed cumulative formula) @@ -945,7 +945,7 @@ Platform fields added in Phase 3: - `Q35::kernel_irqchip: Option` (`"split"` for CoCo, absent for vanilla) - `Q35::confidential_guest_support: Option` referencing the protection object id - `MemoryBackend::Ram { host_nodes, policy }` for NUMA-pinned RAM backend -- `VfioDevice::iommufd_id: Option` — per-device iommufd for CoCo x86 +- `VfioDevice::iommufd_id: Option` — per-device `iommufd` for CoCo x86 - `VfioDevice::pci_vendor_id / pci_device_id` for CoCo attestation overrides **TDX production data captured** (Intel TDX host, 2026-07-15). @@ -1003,7 +1003,7 @@ distance 21) but without CoCo. Key differences from the TDX capture: `confidential-guest-support`. - Memory backend is `memory-backend-file` via `/dev/shm` (same pattern as `q35_vanilla_kata_x86`), not `memory-backend-ram` as in the CoCo captures. -- Per-device iommufd is still present: the modern VFIO iommufd interface is used +- Per-device `iommufd` is still present: the modern VFIO `iommufd` interface is used for GPU passthrough regardless of CoCo mode. - `x-pci-vendor-id`/`x-pci-device-id` overrides are still present: the kata runtime applies them for any GPU passthrough, not only for CoCo attestation. @@ -1023,7 +1023,7 @@ cleanly. ### 4+ Blackwell (B200/B300) GPU passthrough fails due to 40-bit GPA cap (#13270) `-cpu host` without `host-phys-bits=on` limits the guest physical address space -to ~40 bits (~1 TiB). Each B200/B300 GPU has a 128 GiB 64-bit prefetchable BAR; +to ~40 bits (~1 TiB). Each B200/B300 GPU has a 128 GiB 64-bit `prefetchable` BAR; three GPUs (384 GiB) fit within 1 TiB, but four GPUs (512 GiB) do not, causing the fourth GPU to fail PCIe BAR assignment at VM boot. @@ -1086,7 +1086,7 @@ legacy `Machine` struct can be deleted. 2. **Singletons are first-class.** `iommufd0` is emitted once and referenced via `IommufdRef`. No string - concatenation in device impls. + concatenation in device `impls`. 3. **One location for host-specific wiring.** `Platform::apply_host_defaults` is the only place that knows about DGX, @@ -1096,8 +1096,8 @@ legacy `Machine` struct can be deleted. Bus-attached IOMMUs (`arm-smmuv3`) live on `PciRootComplex`. Global IOMMUs (`intel-iommu`) live on the machine type (`Machine::Q35`). These are different device placement models and must not share a field. - SMMU grouping: devices sharing a physical SMMU are placed on the same - `PciRootComplex`; devices on separate physical SMMUs get separate entries. + `SMMU` grouping: devices sharing a physical `SMMU` are placed on the same + `PciRootComplex`; devices on separate physical `SMMUs` get separate entries. 5. **NUMA emission order is non-negotiable.** CpuMem → GenericInitiator → MemoryOnly. The `Platform` builder enforces @@ -1119,5 +1119,5 @@ legacy `Machine` struct can be deleted. - [Issue #12125](https://github.com/kata-containers/kata-containers/issues/12125) — NUMA and hugepages roadmap - [Issue #12210](https://github.com/kata-containers/kata-containers/issues/12210) — make CPU model configurable for kata-qemu-snp (origin of the cpu=host discussion) - [PR #12329](https://github.com/kata-containers/kata-containers/pull/12329) — switch SNP to `cpu=host`; held on do-not-merge due to attestation portability concerns raised in review -- [Issue #12382](https://github.com/kata-containers/kata-containers/issues/12382) — AVX-512/VAES stripped by EPYC-v4 halves AES-GCM throughput; motivates `SNP_CRYPTO_FEATURES` +- [Issue #12382](https://github.com/kata-containers/kata-containers/issues/12382) — AVX-512/`VAES` stripped by EPYC-v4 halves AES-GCM throughput; motivates `SNP_CRYPTO_FEATURES` - [Issue #13270](https://github.com/kata-containers/kata-containers/issues/13270) — 4+ Blackwell GPU BAR mapping fails without `host-phys-bits=on`; motivates `Host::extra_features`