Mark Phase 4 and Phase 5 complete (✅). Enumerate what each phase
delivered vs. the original plan:
Phase 4: NIC groups, prealloc fix, fixture corrections, GB300 dry-run,
all 12 tests passing without #[ignore].
Phase 5: prealloc=on for hugepages, vCMDQ/vEGM wired end-to-end.
Add "auto" mode note under Phase 4: the kata config fields cold_plug_vfio
and hot_plug_vfio currently require an explicit topology string. An "auto"
value is the natural next step once PlatformProbe is implemented to scan
/sys/bus/pci/devices/ and classify devices; apply_host_defaults already
contains the logic to derive the right topology.
Update Phase 6 checklist: add PlatformProbe/"auto" wiring and switch-port
topology (NVSwitch/DAN fan-out) as explicit deliverables.
Add nic_smmu_groups to the HostTopology struct block.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Add HostTopology::nic_smmu_groups (Vec<GpuSmmuGroup>): NIC devices get
their own pxb+smmuv3 complex allocated after all GPU complexes, but emit
no acpi-generic-initiator links and no NUMA initiator nodes. Reuses
GpuSmmuGroup to carry socket affinity.
apply_virt_defaults processes nic_smmu_groups after gpu_smmu_groups,
continuing bus_nr and port_idx counters; VfioDeviceKind::Nic maps to
vfio-pci in emit_vfio_grace.
grace_4_gpu_and_nic fixture updated: GPU on pcie.1 (bus_nr=1), NIC on
pcie.2 (bus_nr=2), NIC emits vfio-pci not vfio-pci-nohotplug.
Unignore grace_1 through grace_4, add the gb300_nvl_2gpu dry-run test
(fixture landed in the previous commit), and add nic_smmu_groups:
vec![] to all existing HostTopology constructions.
Result: 9 fixture tests pass (4 Grace + 4 Q35 + GB300), grace_5-7
stay ignored until Phase 5.
Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
GB300 (Grace + Blackwell B200) requires highmem-mmio-size=8T because B200
BARs exceed the 4T MMIO window used by GH200/GB200. All other topology
properties are identical to Config 3 (2 GPUs per SMMU on one Grace socket).
The test overrides highmem_mmio_size after from_config_defaults:
if let Machine::Virt(ref mut v) = platform.machine {
v.highmem_mmio_size = Some(8 << 40);
}
This demonstrates that apply_host_defaults is topology-agnostic; the only
GB300-specific change is the MMIO window size, which is a machine property,
not a HostTopology property. A future from_config path will derive 4T vs 8T
from the GPU device IDs detected by PlatformProbe.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Same treatment as the earlier spellcheck pass: wrap identifiers and
hardware acronyms introduced by the sweep rewrite in backticks so
cspell's inline-code ignore applies.
Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The emit_backend (RAM: id before size) and emit_pxb (bus before bus_nr)
functions had their field order updated in Phase 3, but the 7 Grace fixture
files still contained the old ordering from the original authoring pass.
Running the ignored Phase 4/5 tests surfaces the divergence as assertion
failures even though the code output is correct.
Sync fixture files to the current code output; no logic changes.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Three bugs made apply_q35_defaults produce incorrect output for any
GpuSmmuGroup with more than one device:
- slot: Some(0) on every port — QEMU requires unique slots within a pxb;
fixed to slot: Some(port_index_within_group).
- chassis: 10 on every pxb complex — must be unique per complex so QEMU
can distinguish PCIe chassis during bus enumeration;
fixed to 10 + group_idx.
- rp-numa{group}-{global_gpu_idx} naming — the global index made
pxb-numa1's ports read as rp-numa1-4…rp-numa1-7 instead of
rp-numa1-0…rp-numa1-3 as production logs show;
fixed to use the per-pxb port_index.
- bus_nr advancing by 1 per pxb — production captures show 32-bus
spacing (32, 64) to give room for subordinate bus assignment;
fixed to 32 + group_idx × 32.
ARCHITECTURE.md gains a "Scalability (1–8 GPUs)" note under the VFIO
Device Assignment Model section, including the B200/B300 PCIe layout table
and an explanation of why NVSwitch classification is deferred to Phase 4.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Same viking-prod-243 physical host as the TDX capture but without CoCo:
-machine q35,accel=kvm (no kernel_irqchip, no confidential-guest-support),
memory-backend-file via /dev/shm (not memory-backend-ram as in CoCo).
Key observation: per-device iommufd and x-pci-vendor-id/device-id overrides
are present even without CoCo — the runtime applies them for any GPU
passthrough, not only for attestation. NUMA node 1 is memory-only (no cpus=).
No new Platform types required; exercises the model already in place.
ARCHITECTURE.md gains a "Vanilla kata + GPU passthrough" planned-fixture entry.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Rename cold_plug_ports -> pcie_root_port (probe.rs, topology.rs, platform.rs,
tests.rs):
The field pre-provisions empty PCIe slots at boot for cold or hot-plug device
assignment. "cold_plug_ports" implied it was the cold-plug device path; the
correct name mirrors the `pcie_root_port =` kata config field that controls it.
Cold-plug with a device already attached is driven by gpu_smmu_groups instead.
pub -> pub(crate) on platform.rs internal API (apply_host_defaults,
with_hugepages, to_qemu_args):
These are intra-crate entry points, not public API. from_config (HypervisorConfig
entry point) remains pub. Promised in Phase 0 PR review, now applied.
q35.rs: update "-- Phase 3+" stub comment to "-- Phase 4+".
ARCHITECTURE.md sweeping update:
- Add "Emission order" split for Q35 vs virt/Grace (different ordering
required because virt carries memory-backend= on the machine line)
- Add "VFIO Device Assignment Model" section: all four port types
(no-port, root-port, switch-port, bridge-port) for both cold and hot plug;
phase coverage table; distinction between gpu_smmu_groups (cold-plug
device+port together) and pcie_root_port (pre-provisioned empty slots)
- Update stale struct snippets: PciTopology, PciRootPort, VfioDevice,
VfioDeviceKind, Objects, HostTopology, SocketInfo, ProtectionDevice
all reflect Phase 3 shapes
- Fix Migration Phases 2-3 descriptions to match what was actually built
(Phase 2: virt emission; Phase 3: Q35 emission + CoCo)
- Fix "q35_*_test (ignored, Phase 3)" -> "(passing, Phase 3)"
- Fix "Phase 3 will introduce" -> "Phase 3 introduced"
- Fix cold_plug_ports mention in vanilla Q35 fixture description
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
tests.rs:
- check() helper: from_config_defaults("virt", ...) now takes machine type
- single_socket() and all HostTopology literals: fill new fields
(numa_distances, cold_plug_ports, protection; SocketInfo host_node,
mem_path, mem_size) with None/vec![]/0 for existing Grace tests
- Grace 1-7: update HostTopology and SocketInfo field sets; keep existing
Phase 4/5 ignore annotations
- q35_coco_snp_single_gpu: build Platform directly with sev-snp-guest
protection, memory-backend-ram with host-nodes pinning, pxb-pcie with
a single GpuPci vfio device carrying per-device iommufd_id and
x-pci-vendor-id/device-id CoCo attestation overrides. Direct
construction is used because per-device iommufd UUIDs are assigned at
VM launch time and are not derivable from HostTopology alone.
- q35_vanilla_kata_x86: drive apply_host_defaults from a 2-socket
HostTopology with /dev/shm file-backed NUMA memory, NUMA distances, and
8 cold-plug root ports; verify against fixture.
Fixtures (q35_coco_snp_single_gpu.args, q35_vanilla_kata_x86.args):
- Correct NUMA node field order to match emit_numa_node output:
node,memdev=...,cpus=...,nodeid=N (nodeid last)
- Use format_memory canonical output: 36864M -> 36G, 57344M -> 56G
(format_memory rounds to G when divisible by 1<<30)
- q35_vanilla_kata_x86.args: fix header comment 65 vCPUs -> 66 vCPUs
(2 sockets x 33 vCPUs = 66 total)
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
platform.rs changes for Phase 3 Q35 support:
apply_q35_defaults:
Build per-socket memory backends (Ram or File based on mem_path), populate
NUMA nodes, propagate numa_distances from HostTopology, and generate
cold-plug pcie-root-ports on pcie.0 (slot=0..N-1, multifunction=off).
GPU SMMU groups on Q35 use vfio-pci + GpuPci kind (not nohotplug) and no
arm-smmuv3 (x86 uses the global IOMMU).
apply_host_defaults:
Wire ProtectionDevice into Q35::kernel_irqchip="split" and
confidential_guest_support, and copy it into Objects::protection. Dispatch
to apply_q35_defaults or apply_virt_defaults by machine type.
to_qemu_args / emit_q35_args:
Q35 and virt/Grace require different emission ordering: virt needs
memory-backend= on the machine line so backends must precede the machine
flag, while Q35 does not. Split into emit_q35_args and emit_virt_args.
Q35 order: protection object → -machine → (backend + NUMA node) per socket
→ -numa dist entries → pxb GPU roots (pxb + root ports + per-device iommufd
+ vfio-pci) → cold-plug root ports on pcie.0.
emit_vfio_q35 / emit_vfio_grace:
Two distinct VFIO device formats. Q35: host→id→[vendor/device-id]→bus→
[iommufd]. Grace: device→host→bus→[rombar]→id→[iommufd].
emit_root_port:
Handles optional slot, multifunction, and io_reserve fields, covering both
Q35 cold-plug (slot+multifunction) and Grace (io_reserve) formats.
emit_machine:
Q35 path now emits kernel_irqchip and confidential-guest-support when set.
from_config_defaults:
Takes machine_type as first arg (was 0-arg in tests). Q35 variant sets
memory_backend: None since Q35 memory is expressed per NUMA node.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Add the fields needed to express Q35-specific topologies:
probe.rs:
- HostTopology: +numa_distances (src/dst/val triples), +cold_plug_ports
(pre-provisioned pcie-root-port count), +protection (sev-snp-guest /
tdx-guest capability detected by the host probe)
- SocketInfo: +host_node, +mem_path, +mem_size for per-socket NUMA pinning
and /dev/shm or EGM file-backed memory
- ProtectionDevice enum: SevSnp and Tdx variants, with an id() accessor used
by platform.rs to set confidential_guest_support on the machine line
topology.rs:
- PciRootPort: +slot, +multifunction (Q35 cold-plug ports carry these;
Grace ports do not), +io_reserve (Grace only, absent on Q35)
- VfioDevice: rombar: bool -> Option<bool> (CoCo Q35 has no rombar field),
+iommufd_id for per-device CoCo iommufd (Grace uses a shared iommufd0),
+pci_vendor_id/pci_device_id for x86 CoCo measured-boot attestation
- VfioDeviceKind::GpuPci: vfio-pci (Q35 / CoCo) vs existing Gpu (vfio-pci-
nohotplug for Grace aarch64)
- PciTopology: +cold_plug_ports for pre-provisioned Q35 root ports on pcie.0
q35.rs:
- Q35: +kernel_irqchip (split for CoCo, absent for vanilla) and
+confidential_guest_support referencing the protection object id
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Emit the EGM memory-backend-file line as id, size, mem-path,
prealloc=on, share=on, matching the vCMDQ and Q35 SHM emission paths
and the normalized golden fixtures. QEMU treats the options as
unordered; one canonical order keeps the emitter free of per-backend
special cases.
Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Q35 NVIDIA GPU configs set hot_plug_vfio=no-port and cold_plug_vfio=root-port
with pcie_root_port=8. The 8 pcie-root-ports in the production invocation are
pre-provisioned for cold-plug: GPU VFIO devices are added to the static QEMU
command line before the VM boots, not via QMP after boot.
Update fixture comments, test comments, and ARCHITECTURE.md to use the correct
term and cite the config knobs.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds fixture and ignored test (Phase 3) for the x86_64 kata use-case
based on a production QEMU invocation from a DGX x86 host (2026-07-07):
65 vCPUs across two NUMA sockets, 73728M total, 36864M per socket pinned
to host NUMA nodes via /dev/shm, 8 pcie-root-ports pre-provisioned on
pcie.0 for GPU hotplug via QMP.
Key architectural differences from the Grace/virt topology documented
in ARCHITECTURE.md:
- No kernel-irqchip on vanilla Q35 (only required for CoCo)
- NUMA memory model: separate file-backed backends with host-nodes+policy
rather than a single backend referenced on the -machine line
- GPU passthrough via QMP hotplug onto pcie-root-ports, not static
vfio-pci-nohotplug on pxb-pcie buses
Phase 3 items needed before the test can pass:
MemoryBackend::File { host_nodes, policy }, Objects::numa_distances,
HostTopology NUMA SHM fields, Q35 branch in to_qemu_args.
Also corrects the Platform Parity baseline for Q35 (no kernel-irqchip
in vanilla; add note that it is only needed for CoCo).
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
build("virt", ...) is called from both from_config (production) and
from_config_defaults (test). Hardcoding gic_version=3, ras=on and
highmem_mmio_size=4T in build would set Grace-specific values on every
virt machine regardless of the actual hardware, breaking vanilla aarch64
VMs that need none of these options.
Move the Grace defaults into from_config_defaults, which is #[cfg(test)]
and explicitly models the Grace fixture topology. The production path via
from_config will read these values from HypervisorConfig when that wiring
lands in a later phase.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
seccompsandbox already maps to -sandbox in cmdline_generator.rs via
seccomp_sandbox: Option<String> in HypervisorConfig. When Platform
takes over argument emission in Phase 3, sandbox support needs a typed
Objects::seccomp_sandbox field so the legacy generator can be removed
cleanly.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
from_config and from_config_defaults now build real Platform values
instead of todo!-ing. apply_host_defaults populates PciTopology from
HostTopology (one PciRootComplex + arm-smmuv3 per GpuSmmuGroup, one
PciRootPort per GPU), assigns GenericInitiator NUMA nodes (8 per GPU
after all CpuMem nodes), and adds EGM File backends plus EgmMemory
links when egm_sockets is present. with_hugepages swaps the primary
RAM backend for a File backend backed by the hugepages mount.
Six structural unit tests cover: default Virt construction, single GPU
topology, multi-GPU/multi-SMMU grouping, EGM backend and link counts,
hugepages swap, and hugepages-preserves-EGM. Golden fixture tests
remain #[ignore]d until to_qemu_args is wired in Phase 2+.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
grace_4_gpu_and_nic: state explicitly that HostTopology cannot
represent NIC passthrough yet and that Phase 4 rewrites the body;
the fixture already defines the full expected output.
grace_5_vcmdq: expand the body to call with_hugepages() before
emission, documenting the step Config 5 requires. The method is a
stub until Phase 5, so the test stays ignored, but the body now
matches the shape Phase 5 lands.
Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The vEGM fixtures used id,mem-path,size,share,prealloc while the vCMDQ
and Q35 SHM fixtures use id,size,mem-path with prealloc before share.
QEMU treats the options as unordered, but the golden tests compare
strings exactly, so a single canonical order keeps the emitter free of
per-backend special cases. Canonical: id, size, mem-path, then
host-nodes/policy where pinned, then prealloc=on, share=on.
Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The highmem_mmio_size doc comment ended with a dangling "bytes." line;
fold the unit into the opening sentence, in virt.rs and the copy quoted
in ARCHITECTURE.md.
ARCHITECTURE.md claimed the Makefile already defaults CPUFEATURES and
TDXCPUFEATURES to include host-phys-bits=on; the repo defaults are
still pmu=off only. State the actual situation: the flag must be set
per configuration file until the Makefile defaults are extended.
Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
cspell flags domain terms and identifiers (SMMU, CMDQ, SRAT, VAES,
iommufd, pseries, prefetchable, and friends) as unknown words. Wrap
them in backticks: the spellcheck config ignores inline code spans,
and these tokens are identifiers or hardware acronyms that belong in
code style anyway.
Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
- q35_vanilla_kata_x86.args: header said 65 vCPUs but the NUMA ranges
(0-32, 33-65) cover 66; fix the count.
- q35_coco_snp_single_gpu.args: replace the production-captured
host-data blob with a clearly synthetic placeholder (base64 of
"KATA-SYNTHETIC-HOST-DATA-0000000"); the fixture validates arg shape
and ordering, not attestation material.
- ARCHITECTURE.md: kernel-irqchip -> kernel_irqchip in four places to
match the actual QEMU -machine option spelling used by the code and
fixtures.
Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
For CoCo SNP the CPU model is part of the attestation identity: a fleet
with mixed Milan/Genoa/Turin nodes produces different CPUID per node with
cpu=host, so attestation reference values diverge and rescheduled Pods
fail (#12329). Pinning to EPYC-v4 gives a stable, portable identity
across the fleet, but strips AVX-512 and VAES extensions, halving
AES-GCM throughput to ~4 GB/s (#12382).
CpuModel::EpycV4 { extra_features } resolves both: attestation identity
is fixed while SNP_CRYPTO_FEATURES re-enables the stripped extensions.
CpuModel::Host { extra_features } carries pmu=off and host-phys-bits=on
for vanilla/TDX x86_64 guests (#13270).
ARCHITECTURE.md gains a BaseMachine/CpuConfig section with the full
rationale table and links to the four issues as breadcrumbs.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Explains the root cause (40-bit GPA without host-phys-bits=on), the
short-term fix in the Makefile, and the post-refactor path (Q35 Platform
builder should emit host-phys-bits=on unconditionally for x86_64/KVM).
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Four items promised in replies to Copilot review:
virt.rs: add "bytes." to highmem_mmio_size doc comment so the unit is
unambiguous when emitting QEMU highmem-mmio-size=.
platform.rs: clarify BaseMachine::memory_backend doc — None is correct
for multi-socket vEGM where each socket supplies its own memory-backend
via -numa node,memdev= rather than a single machine-wide backend.
ARCHITECTURE.md: update module layout diagram to reflect what Phase 0
actually introduces (platform.rs, topology.rs, tests.rs all present;
probe.rs holds HostTopology only — PlatformProbe is Phase 1). Annotate
runtime: RuntimeFeatures fields in both Machine code samples as Phase 3+
so the doc is self-consistent with the current stubs.
tests.rs: run cargo fmt (long assignment line on check() helper).
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds fixture and ignored test (Phase 3) for Q35 CoCo (SEV-SNP) with one
GPU passed through, based on a production invocation (AMD EPYC host,
2026-07-13, H100 80GB at 0000:e1:00.0).
Key differences from Grace/virt and vanilla Q35 documented in fixture
and ARCHITECTURE.md Planned Fixtures section:
- sev-snp-guest object emitted before the -machine line (QEMU requires
the protection object first); kernel_irqchip=split required for SNP
- Memory model: memory-backend-ram with host-nodes=1,policy=bind (not
file-backed; CoCo uses RAM backend for NUMA pinning on x86)
- GPU passthrough via pxb-pcie + pcie-root-port + vfio-pci (not
vfio-pci-nohotplug); no arm-smmuv3 — x86 uses global AMD/Intel IOMMU
- iommufd is per-device (id=iommufdvfio-<uuid>), not the shared
iommufd0 used on Grace
- x-pci-vendor-id/x-pci-device-id overrides required so the guest sees
correct device IDs for measured boot attestation
- AMDSEV.fd firmware (AMD-specific OVMF, not generic OVMF.fd)
- pxb-pcie bus_nr=32 (not the Grace 1-indexed cumulative formula)
Phase 3 items enumerated: Objects::protection, Q35::kernel_irqchip and
confidential_guest_support typed fields, MemoryBackend::Ram host_nodes
and policy, per-device iommufd, VfioDevice vendor/device id overrides.
TDX capture still needed.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Q35 vanilla kata fixture (q35_vanilla_kata_x86.args) from a production
DGX x86 invocation (2026-07-07): 2-socket NUMA, 65 vCPUs, 36864M per
socket pinned to /dev/shm, 8 cold-plug pcie-root-ports on pcie.0.
Test added as #[ignore = "Phase 3"]; it drives the Phase 3 API design.
Key differences from the Grace/virt topology that inform Phase 3 scope:
- No kernel-irqchip on vanilla Q35 (only required for CoCo)
- NUMA SHM memory model: per-socket file-backed backends with host-nodes
and policy=bind rather than a single backend on the -machine line
- Q35 GPU passthrough uses cold-plug onto pre-provisioned pcie-root-ports
(cold_plug_vfio=root-port, pcie_root_port=8, hot_plug_vfio=no-port);
no pxb-pcie, no arm-smmuv3
Platform Parity section: documents legacy Machine fields without typed
homes (machine_accelerators, confidential_guest_support), baseline
-machine output for each supported machine type, and Platform fields
required before the legacy struct can be deleted.
Planned Fixture Configurations section: enumerates the four fixture sets
needed before Phase 6 closes (vanilla virt, Q35 vanilla, CoCo+GPU,
8-GPU+4-NVSwitch), notes what data is captured vs. still needed, and
lists the new types required for each.
Known Issues and Follow-up Items section: QMP startup timeout (#13343),
seccomp_sandbox not yet in Platform, machine_accelerators /
confidential_guest_support cross-references.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Seven fixture files encode the expected QEMU argument list for each
Grace passthrough topology derived from tested production deployments:
grace_1_single_gpu -- 1 GPU, 1 SMMU, 9 NUMA nodes
grace_2_four_gpus_1_per_smmu -- 4 GPUs x 1-per-SMMU, 33 NUMA nodes
grace_3_four_gpus_2_per_smmu -- 4 GPUs x 2-per-SMMU, 33 NUMA nodes
grace_4_gpu_and_nic -- GPU + NIC, NIC emits no generic-initiators
grace_5_vcmdq -- hugepages backing + cmdqv=on on arm-smmuv3
grace_6_vegm_1_per_socket -- vEGM, 4 sockets, 1 GPU each
grace_7_vegm_2_per_socket -- vEGM, 2 sockets, 2 GPUs each
Each test is #[ignore] until the corresponding phase lands. The harness
reads fixtures as one Vec<String> element per line; blank lines and
lines starting with '#' are skipped.
The fixtures are the contract: any Platform implementation must reproduce
them exactly before a phase PR can be merged.
Refs: #12187, #12125
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The existing cmdline_generator.rs (~3700 lines) encodes bus-assignment
logic inside each device and uses a single stringly-typed Machine struct
for all machine types. This makes adding Grace/vCMDQ/vEGM support
impractical without touching every device.
Introduce src/qemu/machine/ with the target types:
Platform -- top-level wiring point (machine + PCIe topology + objects)
Machine -- per-machine-type structs (Q35, Virt, Pseries, S390xCcwVirtio)
PciTopology -- PCIe expander buses with per-RC numa_node and BusIommu
BusIommu -- bus-attached IOMMU (SmmuV3); IntelIommu lives on Q35 directly
Objects -- typed -object registry (MemoryBackend, AcpiPciNodeLink, ...)
HostTopology -- probe result consumed by apply_host_defaults
All methods are todo!() stubs. No behaviour change; cmdline_generator.rs
is untouched. Implementations land phase by phase starting at Phase 2.
Refs: #12187, #12125
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The current cmdline_generator.rs (~3700 lines) has three structural
problems: bus-assignment logic duplicated across every device via
#[cfg(target_arch)], a single stringly-typed Machine struct shared by
all machine types, and no NUMA-aware memory backend support.
Add ARCHITECTURE.md describing the target design before any code
changes land. The document covers:
current pain points
target module layout and core types (Platform, Machine, PciTopology,
BusIommu, Objects, HostTopology)
NUMA layout rules (ACPI SRAT ordering, 8 nodes per GPU for MIG,
highmem-mmio-size sizing, hotplug placeholder)
7 Grace platform configurations as golden test fixtures
6-phase migration plan (Phase 0 test harness through Phase 6 cleanup)
design principles
The document is a living reference updated with each phase PR.
It becomes a clean stable reference once Phase 6 completes.
Refs: #12187, #12125
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Only enable discard for block-plain emptyDir volumes when the
hypervisor reports block device discard support.
The resource manager now queries hypervisor capabilities once and
passes block discard support through VolumeContext. Block emptyDir
volume setup uses that capability to decide whether to add the
discard mount option and related mount metadata, avoiding unsupported
discard mounts on hypervisors that cannot expose discard/unmap to the
guest.
It will correctly adjust the related codes with the new support.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Map BlockConfigModern.discard_unmap to Dragonball block device
sparse support when adding modern block devices.
This lets runtime-rs request guest-visible discard support through
the Dragonball block device configuration while leaving vhost-user
block devices disabled.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Add a sparse flag to Dragonball block device configuration and use
it to expose virtio-blk discard support to the guest. Handle discard
requests by validating the requested range and punching holes in
sparse local file backends, returning an I/O error instead of
panicking on discard failures.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Add the Cloud Hypervisor disk sparse option to the local DiskConfig
model and keep its default aligned with upstream. Map
BlockConfigModern.discard_unmap to DiskConfig.sparse so block devices
that request discard/unmap explicitly expose sparse discard support
through the CH API.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
To tells if the hypervisor exposes block discard/unmap support to
the guest, we introduce a new capability of discard/unmap named
`BlockDeviceDiscardSupport` in the capability set.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Remove the remaining legacy Block device implementation now that
BlockModern covers all block paths, finishing the migration started
by the QEMU and DeviceManager legacy-path drops.
Delete virtio_blk.rs (BlockConfig, BlockDevice and its Device impl)
and stop exporting it from driver/mod.rs. Drop the BlockCfg variant
from DeviceConfig and the Block variant from DeviceType. Remove the
DeviceConfig::BlockCfg handling and the create_block_device helper
from DeviceManager, and drop DeviceType::Block from the QEMU
hotunplug unsupported list.
No new behavior; pure removal of the dead legacy path.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Remove DeviceType::Block so BlockModern is the single code path,
completing the BlockConfig -> BlockConfigModern migration at the
Device Manager layer.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The preceding "Use BlockConfigModern for ..." series converted every
block-device producer in the Qemu path to emit BlockModern. The Qemu
inner module still held the matching legacy Block consumer code across
all three lifecycle ops (coldplug/hotplug/hotunplug), now unreachable.
Remove it so BlockModern is the single code path, completing the
BlockConfig -> BlockConfigModern migration at the Qemu layer.
The hotunplug arm for Block now returns an explicit "unsupported"
error instead of a stale QMP detach. The new added Block(_) is just a
place holder and it will be removed in a follow-up commit.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Migrate the block config from BlockConfig to BlockConfigModern,
routing it through DeviceConfig::BlockCfgModern for firecacker.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Migrate the block config from BlockConfig to BlockConfigModern,
routing it through DeviceConfig::BlockCfgModern for agent-ctl vm_ops.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Migrate the erofs rootfs path from BlockConfig to BlockConfigModern,
routing its block-device attachment through BlockCfgModern.
This covers the rw layer, single-device raw, multi-device VMDK, and
GPT-partitioned VMDK attachment points in erofs_rootfs.rs.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Migrate the block config from BlockConfig to BlockConfigModern,
routing it through DeviceConfig::BlockCfgModern for swap task.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Migrate the block config from BlockConfig to BlockConfigModern,
routing it through DeviceConfig::BlockCfgModern for block rootfs.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Migrate the host block-device (major/minor) passthrough path in
ResourceManagerInner from BlockConfig to BlockConfigModern, routing it
through DeviceConfig::BlockCfgModern. This continues the BlockModern
migration to cover block-mode devices carried in the OCI spec.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
