The --without-default-devices flag was parked in a comment while the
per-architecture device allowlist was incomplete: the runtimes emit
devices that were missing from it, so a build with the flag active
produced a QEMU that failed at VM start rather than at build time.
Complete the allowlist with every device the Go runtime and runtime-rs
can emit:
- VIRTIO_SCSI virtio-scsi + scsi-hd (block_device_driver=virtio-scsi)
- VHOST_USER_BLK / VHOST_USER_SCSI vhost-user block backends
- NVDIMM DAX rootfs image (x86_64, aarch64, ppc64le)
- PCIE_PORT pcie-root-port: hot-plug slots and root-port topology
- XIO3130 x3130-upstream/xio3130-downstream: switch-port topology
- PCI_BRIDGE pci-bridge: bridge-port topology
- PCIE_PCI_BRIDGE pcie-pci-bridge
- PXB pxb-pcie on x86_64 (NUMA-pinned GPU root complexes);
aarch64 already carried it as a CXL dependency
- PVPANIC_ISA pvpanic guest kernel panic reporting (x86_64)
The allowlist is split into transport-independent device models
(_COMMON_DEVS, all architectures) and PCI transport + PCIe slot
topology (_PCIE_DEVS, x86_64/aarch64; ppc64le takes the conventional
PCI subset since pSeries PHBs have no PCIe ports; s390x is pure CCW).
A post-build verify_devices check runs `qemu-system-* -device help`
and fails the build if any required device is missing, so allowlist
gaps surface at build time, never at VM start. The check is skipped
when cross-compiling since the binary cannot run on the build host.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude <noreply@anthropic.com>
An earlier commit in this PR accidentally removed the shellcheck
source directives, CROSS_BUILD initializer, repo_root_dir fallback,
and nounset-safe ${1:-} defaults, causing the "Shellcheck required"
CI check to fail.
Restore the file to match the main-branch state for these lines.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace rm -rf qemu + clone-into-relative-path with mktemp -d so the
build never touches a pre-existing qemu/ directory in the bind-mounted
host working directory. The trap ensures cleanup on any exit path.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Use 'CONFIG_VIRTIO_MEM=' prefix match instead of '\b' word boundary in
grep -v: \b is a GNU extension not guaranteed by POSIX, making the
ppc64le VIRTIO_MEM exclusion unreliable on non-GNU grep implementations.
Add a comment on the rm -rf call to make clear this script always runs
inside a dedicated build container, so deleting a relative path is safe.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
QEMU ships firmware for every emulated platform it was compiled for.
Ship only the firmware the target architecture actually boots:
- x86_64: edk2-x86_64-*.fd (OVMF)
- aarch64: edk2-aarch64-*.fd (AAVMF)
- s390x: s390-ccw.img (CCW BIOS)
- ppc64le: slof.bin / vof.bin (Open Firmware)
Firmware for unrelated architectures (ARM32, RISC-V, LoongArch, HPPA …)
is added to the blacklist at packaging time so it never lands in the
deployed tarball. This reduces installed size and removes executable
blobs that have no function on the target host.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
--without-default-devices uses allnoconfig semantics: every Kconfig
entry with 'default y' is suppressed, which silently removes all virtio
devices from the binary. Write the minimal set of CONFIG_* entries we
actually need into each arch's default.mak before configure runs.
Also fix two bugs in the existing code:
- default.mak was written after ./configure, so meson never picked up
the entries; move it before the configure call
- aarch64 path used arm-softmmu (32-bit) instead of aarch64-softmmu
- make had a typo: 'nproc +--ignore' instead of 'nproc --ignore'
Common set: VIRTIO_{BLK,NET,SERIAL,RNG,BALLOON,MEM}, VHOST_USER_FS,
VIRTIO_9P, VHOST_VSOCK, VFIO_PCI, IOMMUFD. Architecture-specific:
- x86_64: Q35, VTD/VTD_ACCEL (IOMMUFD-backed), AMD_IOMMU
- aarch64: ARM_VIRT, PXB, CXL/CXL_MEM_DEVICE (required for Rubin vCXL)
- s390x: S390_CCW_VIRTIO, VIRTIO_CCW, VFIO_CCW/VFIO_AP
(VIRTIO_PCI/VFIO_PCI/IOMMUFD excluded — CCW bus, not PCI)
- ppc64le: PSERIES (VIRTIO_MEM excluded — not supported on ppc64)
Build only the system emulator binary instead of the full tool set;
ppc64le uses the 'ppc64' target name.
All four arch configs validated with minikconf --allnoconfig (exit 0).
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The guest kernel build derived a CROSS_COMPILE=<arch>-linux-gnu- prefix
whenever the target arch differed from the host, the static-build wrapper
switched to a per-arch `-cross-build` image (with buildx/--platform for
s390x), and the builder image installed gcc-<arch>-linux-gnu /
binutils-<arch>-linux-gnu cross toolchains.
None of this is exercised on our side: the kernel is always built on a
native-arch runner. Remove the CROSS_COMPILE plumbing, the cross-build
image/buildx handling and the cross toolchain install.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Building QEMU for a foreign arch relied on configure-hypervisor.sh
emitting --cross-prefix, build-qemu.sh passing --cc/--cross-prefix for
s390x, and the builder image installing foreign-arch -dev packages
(via dpkg multiarch/DPKG_ARCH) plus a gcc-<arch>-linux-gnu cross toolchain.
None of this is exercised on our side: QEMU is always built on a
native-arch runner. Remove the cross-prefix logic, the s390x cross
configure branch, and the DPKG_ARCH/cross-gcc plumbing from the Dockerfile
and build-base-qemu.sh.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
The shared lib.sh exported CROSS_BUILD/TARGET_ARCH/BUILDX/PLATFORM so the
per-component static-build scripts could switch to `-cross-build` builder
images, run `docker buildx build --platform=...` under QEMU emulation and,
for shim-v2, swap in the messense/rust-musl-cross image with a matching
cross CC.
These cross paths are not exercised on our side (all builds run on
native-arch runners), so remove them: builder images are now built with a
plain `docker build`, and shim-v2 builds runtime-rs natively without the
musl-cross image or CC override.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Drop the unfinished ARM CCA stack (qemu-cca shim, experimental QEMU/OVMF
and kernel assets, packaging targets, and CI builds) since it is not
tested or shipped in any runtime bundle.
This will be supported by the moment we have development done for real
hardware available.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Teach the shim-v2 build to feed the "nvidia" base root hash into the Go
runtime's KERNELVERITYPARAMS_NV_BASE - its KERNELVERITYPARAMS_NV is the
monolithic nvidia-gpu hash - so qemu-nvidia-cpu boots the base image
verity-backed just like qemu-nvidia-cpu-runtime-rs already does.
List the new qemu-nvidia-cpu / qemu-nvidia-cpu-runtime-rs classes in
shim-components.json so the build ships them: both boot the nvidia base
image with the nvidia kernel and no GPU extension, dropping OVMF on x86_64
and keeping it on aarch64 (which boots via UEFI).
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Add the rootfs-image-nvidia-{base,gpu-extension} build targets to the
local-build Makefile and the nvgpu bundle.
Build the base/extension rootfs images in the amd64/arm64 static tarball
workflows, and point the runtime-rs NVIDIA shim components at the
composable base + gpu extension (+ coco extension for the confidential
classes). The Go shim components keep the monolithic nvidia-gpu /
nvidia-gpu-confidential images.
Ship cryptsetup in the CoCo guest-components extension (and bundle
libnvat's non-glibc dependency closure next to the nvidia attester): the
chiseled nvidia base image carries only veritysetup, so the coco-extension
manifest's usr/sbin PATH entry needs a cryptsetup to resolve for NVIDIA
confidential encrypted-storage.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Flip the runtime-rs NVIDIA GPU configs (plain/tdx/snp) from the monolithic
image to the driver-agnostic nvidia base boot image plus a cold-plugged,
driver-versioned gpu extension mounted at /run/kata-extensions/gpu.
Confidential runtime-rs configs additionally cold-plug the CoCo
guest-components extension and enforce the nvidia base dm-verity hash.
The Go runtime keeps booting the monolithic nvidia-gpu / nvidia-gpu-confidential
images, mirroring the CoCo split where runtime-rs is the first-class citizen and
Go stays on the monolithic layout.
Add the IMAGE{NAME,PATH}_NV_{BASE,EXTENSION} and NVIDIAGPUEXTENSIONVERITYPARAMS
make vars (runtime-rs only), and read the nvidia base + gpu-extension root
hashes into runtime-rs's shim-v2 opts while the Go opts keep reading the
monolithic nvidia-gpu / nvidia-gpu-confidential hashes.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Carve the monolithic NVIDIA GPU rootfs into a driver-agnostic
nvidia base image (NVRC init + agent + base libs + in-tree modules)
and a driver-versioned gpu extension (GPU userspace, configs, firmware,
NVIDIA modules) laid out for /run/kata-extensions/gpu. Both halves are
produced from the same chiseled tree via a partition-after step, so
the monolith build stays byte-identical and continues to ship.
The shared stage-one (driver install) is reused across the monolith,
base and extension variants; nvidia base is cached without the driver/ctk
versions so one base image can back multiple driver extensions.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
We've been carrying a full, 1200+ line busybox .config for the NVIDIA
images. That's a pain to review and to keep in sync with a busybox
bump, as basically every default that changes upstream ends up as noise
in our file.
Switch to shipping only the handful of options we actually care about in
a small fragment and let busybox's own Kconfig fill in the rest at build
time: start from allnoconfig, drop the baseline entries for whatever the
fragment overrides (Kconfig keeps the first value it reads), append the
fragment and run silentoldconfig. This reproduces the exact same
.config we had frozen, module loading included, so there's no functional
change here.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Add an opt-in STATIC_RUNTIME=yes knob to the shim-v2 static build. When
set, the Go host binaries are built with the runtime STATIC=yes profile
(CGO_ENABLED=0, no PIE), producing fully static, libc-agnostic binaries
in the kata-deploy payload.
This lets a payload image be produced that runs on musl-only hosts which
have no glibc dynamic loader, where the default glibc-linked binaries
fail to exec. The default build is unchanged.
Signed-off-by: Dimitris Karakasilis <dimitris@spectrocloud.com>
Generated-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
During the transition to composable (base + extension) CoCo images, run the
two runtimes on different image layouts so CI exercises both paths via
the existing qemu-{snp,tdx,coco-dev} (Go) and -runtime-rs (Rust) shims:
- runtime-rs (Rust) CoCo templates use the standard base image plus a
CoCo extension extra_image:
image = "@IMAGEPATH@" (was @IMAGECONFIDENTIALPATH@)
[[hypervisor.qemu.guest_extension_images]]
name = "coco"
path = "@COCOIMAGEPATH@"
verity_params = "@COCOVERITYPARAMS@"
COCOIMAGENAME/COCOIMAGEPATH/COCOVERITYPARAMS are added to the
runtime-rs Makefile only.
- runtime (Go) CoCo templates keep the monolithic confidential image
(@IMAGECONFIDENTIALPATH@) and pull rootfs-image-confidential.
shim-components.json reflects this: Go CoCo shims depend on
rootfs-image-confidential while the runtime-rs CoCo shims depend on
rootfs-image + rootfs-image-coco-extension.
shim-v2/build.sh feeds each runtime its own dm-verity params: Rust gets
the measured base image hash (@KERNELVERITYPARAMS@) plus the extension hash
(@COCOVERITYPARAMS@); Go gets the monolithic confidential image hash
(@KERNELVERITYPARAMS@). This is wired per make invocation so the same
@KERNELVERITYPARAMS@ placeholder resolves correctly for each.
Once the split path is validated we can flip the Go templates too and
drop the monolithic confidential image.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Install libdevmapper-dev and pkg-config in the agent build container
so devicemapper-sys can link against libdevmapper. Add the GNU libc
rustup target alongside musl since USE_DEVMAPPER forces LIBC=gnu.
Forward USE_DEVMAPPER through build.sh and build-static-agent.sh.
And you can compile the device mapper in kata-agent as below:
```
$ make LIBC=gnu USE_DEVMAPPER=yes
```
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
build-qemu.sh runs in the per-target builddir (e.g.
build/qemu-tarball/builddir/), which persists across runs. If a previous
build left the cloned `qemu` tree behind (e.g. after an interrupted
build), the next run errors out with:
fatal: destination path 'qemu' already exists and is not an empty
directory.
Wipe `qemu` before cloning so the build is repeatable from a dirty
builddir.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-By: Claude <noreply@anthropic.com>
Update CDH to a newer version and:
- adjust the NVIDIA root filesystem build to reflect the change from
using libcryptsetup to using the cryptsetup binary.
- adjust image-pull test cases to conduct parallel write operations
on the /dev/trusted_store backed guest image pull location since
issue #12721 has been solved on CDH side.
Fixes#12721
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Now shipped in the vendored code tarball.
Drop the git tree status check since it isn't needed anymore.
Also stop building with `-mod=vendor`. This requires to
expose GOMODCACHE as suggested by Fabiano Fidêncio.
Signed-off-by: Greg Kurz <groug@kaod.org>
Address shellcheck warnings including proper variable quoting,
use of [[ ]] over [ ], declaring and assigning variables separately,
and adding appropriate shellcheck disable directives where needed.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
Address shellcheck warnings including proper variable quoting,
use of [[ ]] over [ ], declaring and assigning variables separately,
and adding appropriate shellcheck disable directives where needed.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
Address shellcheck warnings including proper variable quoting,
use of [[ ]] over [ ], declaring and assigning variables separately,
and adding appropriate shellcheck disable directives where needed.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
Address shellcheck warnings including proper variable quoting,
use of [[ ]] over [ ], declaring and assigning variables separately,
and adding appropriate shellcheck disable directives where needed.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
Address shellcheck warnings including proper variable quoting,
use of [[ ]] over [ ], declaring and assigning variables separately,
and adding appropriate shellcheck disable directives where needed.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
Address shellcheck warnings including proper variable quoting,
use of [[ ]] over [ ], declaring and assigning variables separately,
and adding appropriate shellcheck disable directives where needed.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
Address shellcheck warnings including proper variable quoting,
use of [[ ]] over [ ], declaring and assigning variables separately,
and adding appropriate shellcheck disable directives where needed.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
Address shellcheck warnings including proper variable quoting,
use of [[ ]] over [ ], declaring and assigning variables separately,
and adding appropriate shellcheck disable directives where needed.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
Address shellcheck warnings including proper variable quoting,
use of [[ ]] over [ ], declaring and assigning variables separately,
and adding appropriate shellcheck disable directives where needed.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
Address shellcheck warnings including proper variable quoting,
use of [[ ]] over [ ], declaring and assigning variables separately,
and adding appropriate shellcheck disable directives where needed.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
The arm64 build workflow was missing the tools build entirely.
Add build-tools-asset and create-kata-tools-tarball jobs mirroring
the amd64 workflow so that genpolicy and the other tools are
available for coco-dev tests that need auto-generated policy.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
The attestation agent will soon rely on the NVAT rust bindings, which
have some built-time dependencies.
There is currently no nvattest-dev package, so we need to build from
source to get the headers and .so file.
Signed-off-by: Tobin Feldman-Fitzthum <tfeldmanfitz@nvidia.com>
Fixed a bug with the debug kernel build where common/ was repeated
after the common path variable, resulting in the debug
confs never being picked up.
This exposed a subsequent bug where the debug conf
was included in other builds, this is also fixed by creating a
separate directory for debug confs with one file at the moment,
debug.conf that contains debug configurations and bpf specific
configs.
To enable kernel builds (specifically for bpf) the dwarves package was added
to the kernel dockerfile for the pahole package.
Signed-off-by: Agam Dua <agam_dua@apple.com>
