An earlier commit in this PR accidentally removed the shellcheck
source directives, CROSS_BUILD initializer, repo_root_dir fallback,
and nounset-safe ${1:-} defaults, causing the "Shellcheck required"
CI check to fail.
Restore the file to match the main-branch state for these lines.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
avx512bw is an x86_64 host-CPU JIT option that does not exist in
QEMU's configure on s390x, aarch64, or ppc64le. Previously the
build succeeded because the Docker builder cache hid the failure;
our script change invalidated the cache and exposed the bug.
Add [[ "${arch}" == x86_64 ]] alongside the existing version guard
so the flag is only passed where configure accepts it.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace rm -rf qemu + clone-into-relative-path with mktemp -d so the
build never touches a pre-existing qemu/ directory in the bind-mounted
host working directory. The trap ensures cleanup on any exit path.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
--disable-avx512bw and --enable-avx512bw were removed in QEMU 10.1.0.
Passing them to qemu-snp-experimental (11.0.0) or any other post-10.1.0
build causes configure to abort with 'unknown option'.
Add the same version guard that upstream main carries.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Use 'CONFIG_VIRTIO_MEM=' prefix match instead of '\b' word boundary in
grep -v: \b is a GNU extension not guaranteed by POSIX, making the
ppc64le VIRTIO_MEM exclusion unreliable on non-GNU grep implementations.
Add a comment on the rm -rf call to make clear this script always runs
inside a dedicated build container, so deleting a relative path is safe.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
QEMU ships firmware for every emulated platform it was compiled for.
Ship only the firmware the target architecture actually boots:
- x86_64: edk2-x86_64-*.fd (OVMF)
- aarch64: edk2-aarch64-*.fd (AAVMF)
- s390x: s390-ccw.img (CCW BIOS)
- ppc64le: slof.bin / vof.bin (Open Firmware)
Firmware for unrelated architectures (ARM32, RISC-V, LoongArch, HPPA …)
is added to the blacklist at packaging time so it never lands in the
deployed tarball. This reduces installed size and removes executable
blobs that have no function on the target host.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
--without-default-devices uses allnoconfig semantics: every Kconfig
entry with 'default y' is suppressed, which silently removes all virtio
devices from the binary. Write the minimal set of CONFIG_* entries we
actually need into each arch's default.mak before configure runs.
Also fix two bugs in the existing code:
- default.mak was written after ./configure, so meson never picked up
the entries; move it before the configure call
- aarch64 path used arm-softmmu (32-bit) instead of aarch64-softmmu
- make had a typo: 'nproc +--ignore' instead of 'nproc --ignore'
Common set: VIRTIO_{BLK,NET,SERIAL,RNG,BALLOON,MEM}, VHOST_USER_FS,
VIRTIO_9P, VHOST_VSOCK, VFIO_PCI, IOMMUFD. Architecture-specific:
- x86_64: Q35, VTD/VTD_ACCEL (IOMMUFD-backed), AMD_IOMMU
- aarch64: ARM_VIRT, PXB, CXL/CXL_MEM_DEVICE (required for Rubin vCXL)
- s390x: S390_CCW_VIRTIO, VIRTIO_CCW, VFIO_CCW/VFIO_AP
(VIRTIO_PCI/VFIO_PCI/IOMMUFD excluded — CCW bus, not PCI)
- ppc64le: PSERIES (VIRTIO_MEM excluded — not supported on ppc64)
Build only the system emulator binary instead of the full tool set;
ppc64le uses the 'ppc64' target name.
All four arch configs validated with minikconf --allnoconfig (exit 0).
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Add --enable-vhost-kernel and --enable-vhost-user to restore vhost
acceleration silently broken by auto_features=disabled (both features
are 'auto' in meson and become disabled under --without-default-features).
Add --enable-strip and --enable-lto: strip removes debug symbols from
installed binaries; LTO enables cross-unit dead-code elimination which
with our minimal device set typically shrinks the binary by 15-25% at
the cost of ~2-3x longer build times.
Annotate the aarch64 TCG gap (all other arches already pass
--disable-tcg) with a TODO so it is visible during review without
changing behaviour before aarch64 maintainers have been consulted.
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Additionally to the qemu script dir we need to also include the
changes made in configure-hypervisor.sh
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
We have a very long list of disable calls for QEMU that we
can replace by using the configure script to disable all default
features and default devices.
We enable selectively the features that we need.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The SE VMs how-to documented building the s390x boot image on an x86_64
host via CROSS_BUILD=true TARGET_ARCH=s390x ARCH=s390x.
Cross builds are not exercised on our side and the supporting code has been
removed, so drop this instruction; the SE boot image is built on a native
s390x host.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
When the requested ARCH differed from the host, install_libseccomp.sh
selected a <arch>-linux-gnu-gcc cross compiler and passed --host=<arch> to
configure to cross-build the static libseccomp.
This is not exercised on our side: libseccomp is always built for the host
arch. Build it natively without the cross CC or --host.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
kata-ctl shipped a Cross.toml and a "Cross-builds" README section
describing how to cross compile the tool for other architectures with
cross-rs (installing foreign-arch libssl via dpkg multiarch pre-build
hooks).
This is not exercised on our side: kata-ctl is built on native-arch
runners. Remove Cross.toml and the cross-build documentation.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Building a foreign-arch rootfs/image relied on CROSS_BUILD to enable docker
buildx/--platform emulation, a <arch>-linux-gnu-strip stripping tool, an
Ubuntu builder installing gcc-<arch>-linux-gnu + libc6-dev-<arch>-cross
(via the @ARCH@ template) and config.sh exporting a cross CC.
None of this is exercised on our side: rootfs and image are always built
on a native-arch runner. Remove the CROSS_BUILD/TARGET_ARCH/BUILDX/PLATFORM
plumbing, the cross toolchain/@ARCH@ substitution and the cross CC export;
the guest agent is stripped with the native `strip`.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
The guest kernel build derived a CROSS_COMPILE=<arch>-linux-gnu- prefix
whenever the target arch differed from the host, the static-build wrapper
switched to a per-arch `-cross-build` image (with buildx/--platform for
s390x), and the builder image installed gcc-<arch>-linux-gnu /
binutils-<arch>-linux-gnu cross toolchains.
None of this is exercised on our side: the kernel is always built on a
native-arch runner. Remove the CROSS_COMPILE plumbing, the cross-build
image/buildx handling and the cross toolchain install.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Building QEMU for a foreign arch relied on configure-hypervisor.sh
emitting --cross-prefix, build-qemu.sh passing --cc/--cross-prefix for
s390x, and the builder image installing foreign-arch -dev packages
(via dpkg multiarch/DPKG_ARCH) plus a gcc-<arch>-linux-gnu cross toolchain.
None of this is exercised on our side: QEMU is always built on a
native-arch runner. Remove the cross-prefix logic, the s390x cross
configure branch, and the DPKG_ARCH/cross-gcc plumbing from the Dockerfile
and build-base-qemu.sh.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
The shared lib.sh exported CROSS_BUILD/TARGET_ARCH/BUILDX/PLATFORM so the
per-component static-build scripts could switch to `-cross-build` builder
images, run `docker buildx build --platform=...` under QEMU emulation and,
for shim-v2, swap in the messense/rust-musl-cross image with a matching
cross CC.
These cross paths are not exercised on our side (all builds run on
native-arch runners), so remove them: builder images are now built with a
plain `docker build`, and shim-v2 builds runtime-rs natively without the
musl-cross image or CC override.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
The kata-deploy local-build pipeline auto-enabled a cross build whenever
the host arch differed from TARGET_ARCH, wiring up docker buildx, QEMU
binfmt emulation and an s390x cross toolchain (incl. building s390-tools
genprotimg with CROSS_COMPILE) in the builder image.
None of this is exercised on our side: every release/CI build runs on a
native-arch runner, so the cross paths are dead code. Remove the
CROSS_BUILD/TARGET_ARCH/BUILDX/PLATFORM/binfmt machinery and the s390x
cross toolchain from the builder image; the tarballs are always built for
the host arch now.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
The `HOST_ARCH != ARCH` branch in utils.mk existed solely to support
cross-compiling the Rust components (overriding the linker via `CC`).
Cross-compilation is not exercised on our side: all release/CI builds run
on native-arch runners, so this path is dead code. Remove it.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Fail startup when KATA_CONF_FILE does not resolve to one of the shipped
default config files, preventing untrusted config path injection via
environment overrides.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Stop accepting io.katacontainers.config_path so config selection comes
only from shim options, KATA_CONF_FILE, and defaults.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
The sandbox_bind_mounts option accepts host paths that runtime-rs
exposes to the guest shared filesystem. Runtime annotations are not
gated by the hypervisor annotation allowlist, allowing an untrusted
workload annotation to add arbitrary host paths to the sandbox bind
mount configuration.
Ignore KATA_ANNO_CFG_SANDBOX_BIND_MOUNTS when processing workload
annotations. Sandbox bind mounts remain available through the
operator-controlled TOML configuration, preserving the sandbox-level
host/guest sharing feature while removing the untrusted annotation
path.
Really appreciate it that Anthropic and Ada Logics report this issue.
Reported-by: Anthropic and Ada Logics
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Move PCI enumeration, VFIO helpers, and InfiniBand diagnostics out of
kata-sys-util into the external pcilibs-rs crate so they can be reused
independently of kata-containers.
Depend on pcilibs-rs 0.1.0 from crates.io.
Add pcilibs to the cspell dictionary for the new crate reference in
src/libs/README.md.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
open() waits for the binary logger readiness handshake with an unbounded
async read(). If the logger never writes to fd 5 (or inherits a bad fd),
container start can hang indefinitely.
Let's add a timeout around the readiness read and fail the start path if
it expires (the child will be killed on drop due to kill_on_drop(true)).
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Suggested-by: CoPilot
Disable filesystem sharing for the non-confidential NVIDIA runtime-rs
handler and use the EROFS snapshotter as its Kubernetes image-layer
transport. This moves the runtime class toward a guest-owned storage
model instead of relying on virtio-fs for container image layers and
writable cache volumes.
Configure kata-deploy's NVIDIA GPU values to install EROFS and select
it for the qemu-nvidia-gpu-runtime-rs Kubernetes handler. Use
memory-backed writable layers and dm-verity for lower-layer integrity.
Adjust the NVIDIA GPU Kubernetes CI matrix so the non-confidential
runtime-rs job exercises EROFS, while the Go and TEE jobs keep their
existing snapshotter choices.
Keep the Docker smoke test path on virtio-fs. The EROFS setup is
targeted at the Kubernetes runtime-rs handler, where containerd can use
the EROFS snapshotter for image layers.
Adjust the runtime-rs NIM test selection and manifests closer to the
TEE case, where filesystem sharing is already disabled and cache
storage is guest-owned and ephemeral.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Assisted-by: OpenAI Codex <codex@openai.com>
Set the non-confidential NVIDIA runtime-rs emptyDir mode to
block-plain through a dedicated Makefile default.
This prepares the runtime class for configurations where filesystem
sharing is disabled and emptyDir volumes need to be backed by
guest-mounted block devices instead of shared-fs.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Assisted-by: OpenAI Codex <codex@openai.com>
Support containerd binary:// stdio URIs in runtime-rs by spawning the
logger with stdout, stderr, and readiness on file descriptors 3, 4, and 5.
Preserve URI arguments and container metadata, and stream output through
nonblocking pipes with backpressure.
Processes using a binary logger fall back from passfd to the existing agent
streams so the host shim can feed the logger. After EOF, allow the logger to
drain before using SIGTERM and SIGKILL as bounded fallbacks.
Fixes: #4391
Related: #4420
Signed-off-by: cayoub-oai <276123840+cayoub-oai@users.noreply.github.com>
Assisted-By: OpenAI Codex
Commit e8bb619b1b ("runtime: do not require OVMF for qemu-nvidia-cpu
on arm64") dropped the generic OVMF/AAVMF firmware inheritance for the
Go qemu-nvidia-cpu runtime, but assumed runtime-rs already did the right
thing by default. It didn't: the runtime-rs profile still pointed at the
generic @FIRMWAREPATH@ / @FIRMWAREVOLUMEPATH@ and still pulled in the
ovmf component on aarch64.
Mirror the fix on the runtime-rs side by introducing dedicated empty
FIRMWAREPATH_NV_CPU / FIRMWAREVOLUMEPATH_NV_CPU variables, wiring them
into the qemu-nvidia-cpu-runtime-rs configuration, and dropping ovmf
from its aarch64 shim components.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor with Claude Opus 4.8
The termination-log tests exercise termination-message propagation over
the agent GetDiagnosticData RPC, which only matters when the host cannot
read guest files directly, i.e. on shared_fs=none configurations. They
previously gated on is_confidential_runtime_class and therefore skipped
on the NVIDIA CPU runtime-rs handler, which now also runs with
shared_fs=none.
Add an is_shared_fs_none_runtime_class helper (confidential runtime
classes plus qemu-nvidia-cpu-runtime-rs) and use it for the setup and
teardown gating so the tests run on the NVIDIA CPU runtime-rs handler as
well. The plain Go qemu-nvidia-cpu class still uses virtio-fs and is
intentionally excluded. Keep the "blocked by default CoCo policy"
assertion confidential-only, since non-confidential shared_fs=none
classes do not ship a default policy that denies the RPC.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor with Claude Opus 4.8
