Add the nvrc static-build target and make the NVIDIA rootfs targets
depend on it (the rootfs build consumes kata-static-nvrc.tar.zst), and
add the rootfs-image-nvidia-gpu-{base,addon} build targets to the
local-build Makefile and the nvgpu bundle.
Build nvrc and the base/addon rootfs images in the amd64/arm64 static
tarball workflows (nvrc is baked into the rootfs and pruned from the
release like the other guest components), and point the NVIDIA shim
components at the composable base + gpu addon (+ coco addon for the
confidential classes). The monolithic images are still built/shipped.
Flip the NVIDIA GPU configs (Go + runtime-rs, plain/tdx/snp) from the
monolithic image to the driver-agnostic base-nvidia boot image plus a
cold-plugged, driver-versioned gpu addon mounted at /run/kata-addons/gpu.
Confidential configs additionally cold-plug the CoCo guest-components
addon and enforce the base-nvidia dm-verity hash.
Add the IMAGE{NAME,PATH}_NV_{BASE,ADDON} and NVIDIAGPUADDONVERITYPARAMS
make vars, and repoint the shim-v2 verity_variants at the base-nvidia and
gpu-addon root hashes. The monolithic images are still built and shipped
during the transition, but no generated config references their hashes.
Carve the monolithic NVIDIA GPU rootfs into a driver-agnostic
base-nvidia image (NVRC init + agent + base libs + in-tree modules)
and a driver-versioned gpu addon (GPU userspace, configs, firmware,
NVIDIA modules) laid out for /run/kata-addons/gpu. Both halves are
produced from the same chiseled tree via a partition-after step, so
the monolith build stays byte-identical and continues to ship.
The shared stage-one (driver install) is reused across the monolith,
base and addon variants; base-nvidia is cached without the driver/ctk
versions so one base image can back multiple driver addons.
Add a Kata static-build component for NVRC (tools/packaging/static-build/nvrc)
that clones the NVRC repository at a pinned git reference and builds a static
musl binary, mirroring the coco-guest-components build. This lets the
composable-image work track NVRC changes from source instead of depending on a
signed upstream release.
- versions.yaml: add repo/ref/toolchain to externals.nvrc (and bump the
version); the existing url is kept for the signed-release path.
- lib.sh: add get_nvrc_image_name for the builder image.
- kata-deploy-binaries.sh: add the nvrc build target, install_nvrc and its
tarball path; get_latest_nvidia_nvrc_version now tracks ref+toolchain so
the dependent NVIDIA images rebuild when the ref changes.
- nvidia_rootfs.sh: consume kata-static-nvrc.tar.zst instead of downloading
and cosign-verifying the release; the Sigstore .cert/.sig are now copied
only when present (i.e. the signed-release path), since a source build
emits just the binary.
Signed-off-by: Fabiano Fidêncio <fidencio@nvidia.com>
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Generate etc/kata-addons/components.toml inside the CoCo addon rootfs
before building the image, so the manifest becomes part of the
dm-verity-measured erofs image.
The manifest declares the addon's path-only components (ocicrypt config,
pause bundle) and the launchable processes (attestation-agent,
confidential-data-hub, api-server-rest) with paths relative to the addon
mount point (/run/kata-addons/coco) and `${var}` placeholders resolved
by kata-agent at runtime. This is the data the agent consumes to launch
the CoCo guest components without per-bundle code changes.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Make the agent's handling of CoCo guest components data-driven so that a
new addon bundle can be introduced without changing agent code.
Add a new `addon` module defining a generic component manifest contract.
When a CoCo addon image is mounted at /run/kata-addons/coco/, the agent
reads etc/kata-addons/components.toml from it to discover:
- path-only components (looked up by id under the `[paths]` table), used
for the ocicrypt config and the pause bundle, and
- launchable processes (`[[process]]` entries) describing the binary
path, args, optional args, config file, environment, socket to wait on
and timeout.
`${var}` tokens in process fields are substituted from a fixed,
documented context the agent builds at runtime (attestation socket/uri,
config paths, rest-api features, initdata toml path, launch timeout,
resolved ocicrypt config path). Referencing an unknown variable is a
hard error, and manifest paths are validated against traversal, so the
contract stays fail-closed.
Processes are gated by a numeric `level` mapped from
guest_components_procs (AttestationAgent < ConfidentialDataHub <
ApiServerRest), preserving the existing implication semantics.
When no addon is mounted, launch_plan() returns None and the agent uses
a built-in plan that reproduces the legacy behaviour byte-for-byte
(binaries under /usr/local/bin, /etc/ocicrypt_config.json, /pause_bundle).
Monolithic / non-split confidential images are therefore unaffected.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Add rootfs-image-coco-addon to the build-asset-rootfs matrix in the
amd64, arm64 and s390x static tarball workflows so the CoCo addon image
is built as part of CI alongside the other rootfs images.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Add install_image_coco_addon() to kata-deploy-binaries.sh which:
- Unpacks the CoCo guest components and pause image tarballs into a
temporary rootfs directory (under the repo root so Docker-in-Docker
volume mounts resolve correctly)
- Calls image_builder.sh with USE_DOCKER=1, FS_TYPE=erofs,
MEASURED_ROOTFS=yes, SKIP_DAX_HEADER=yes, and SKIP_ROOTFS_CHECK=yes
to produce kata-containers-coco-addon.img + root_hash_coco-addon.txt
Add the rootfs-image-coco-addon-tarball Makefile target with
dependencies on pause-image-tarball and coco-guest-components-tarball.
Remove pause-image-tarball and coco-guest-components-tarball from the
standard confidential image dependencies -- those components now live
exclusively in the CoCo addon image. NVIDIA confidential images
retain them until the NVIDIA addon split lands.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Update all six CoCo configuration templates (coco-dev, snp, tdx for
both Go and Rust runtimes) to use the standard base image instead of
the monolithic confidential image, and add an [[extra_images]] section
for the CoCo addon:
image = "@IMAGEPATH@" (was @IMAGECONFIDENTIALPATH@)
[[hypervisor.qemu.extra_images]]
name = "coco"
path = "@COCOIMAGEPATH@"
verity_params = "@COCOVERITYPARAMS@"
Add COCOIMAGENAME (kata-containers-coco-addon.img), COCOIMAGEPATH, and
COCOVERITYPARAMS to both runtime Makefiles so the placeholders are
substituted at install time.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Addon images (e.g. the CoCo guest components addon) are not full root
filesystems -- they contain only the binaries and configuration that
get bind-mounted into the real rootfs at boot. The existing
check_rootfs() validation requires /sbin/init and systemd, which are
not present in addon images.
Add a SKIP_ROOTFS_CHECK environment variable that, when set to "yes",
bypasses the check_rootfs() call. Forward the variable into the
container environment when using the Docker-based build path so it
works in both direct and containerised invocations.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Create a symlink to enable kata-addon-mount@coco.service in
kata-containers.target.wants during rootfs construction for
systemd-based (non-AGENT_INIT) guests.
The unit's ConditionPathExists guard ensures it only activates when
the virtio-addon-coco block device is actually present in the VM,
so enabling it unconditionally in the base image is safe.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Add a systemd template unit kata-addon-mount@.service and companion
helper scripts (kata-addon-mount.sh, kata-addon-umount.sh) that handle
the guest-side setup of addon block device images.
The mount script:
- Discovers the block device via /dev/disk/by-id/virtio-addon-<name>
- Reads verity parameters from kata.addon.<name>.verity_params on the
kernel command line
- Creates a dm-verity device and mounts the EROFS filesystem
- Bind-mounts addon directories (usr/local/bin, etc, pause_bundle)
into the rootfs
The unit uses ConditionPathExists to only activate when the
corresponding virtio-blk device is present, and is ordered before
kata-agent.service so addon binaries are available when the agent
starts.
The agent Makefile is updated to install the unit and scripts.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Mirror the Rust runtime's extra_images support in the Go runtime:
- hypervisor.go: add ExtraImage struct and ExtraImages field to
HypervisorConfig.
- config.go: parse [[hypervisor.qemu.extra_images]] TOML sections
via toExtraImages(), validating that every addon has a non-empty
name and valid dm-verity parameters.
- qemu.go: in buildDevices(), attach each extra image as a virtio-blk
drive with ID addon-<name>; in kernelParameters(), append
kata.addon.<name>.verity_params=... for each extra image.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Add a serial_override field to BlockConfig and DeviceVirtioBlk so that
addon images get a deterministic QEMU serial (addon-<name>) instead of
the auto-generated image-<id>. This lets the guest discover each addon
via /dev/disk/by-id/virtio-addon-<name>.
Wire up the full cold-plug path for extra_images:
- sandbox.rs: iterate boot_info.extra_images, create read-only
BlockConfig entries with the correct serial_override, and push them
as ResourceConfig::ExtraImage.
- resource lib.rs / manager_inner.rs: handle the new ExtraImage variant
by delegating to the existing block device path.
- cmdline_generator.rs: use serial_override when present for the QEMU
serial= parameter; append kata.addon.<name>.verity_params=... to the
kernel command line for each extra image.
- inner.rs: propagate serial_override from BlockConfig to the cmdline
generator's add_block_device().
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Introduce the ExtraImage type to describe addon block-device images
(e.g. CoCo guest components) that are cold-plugged into the VM
alongside the main rootfs.
Each ExtraImage carries a short name (used as the virtio-blk serial
and in kernel cmdline verity parameters), a host-side path, and
dm-verity parameters (required -- addon images must be integrity-
protected).
A new extra_images Vec is added to BootInfo so that both the Rust
and Go runtimes can iterate the configured addons when building the
VM device list and kernel command line.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Replace `type=local` with `type=tar` in kata-deploy build to reduce
export time and avoid build hangs during the export-to-client-directory
phase.
Update callers to extract binaries directly from the tar archive instead
of copying from an intermediate directory.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
When the agent-protocol-forwarder's inbound connection restarts (e.g.
during a Cloud API Adaptor restart in peer pod environments), the shim
re-sends a GetOOMEvent request through the new connection. Since the
forwarder→agent Unix socket survives the restart, the old handler from
the previous connection remains alive, holding the event_rx lock while
blocked in recv().await.
The new handler acquires the sandbox lock, then attempts to acquire the
event_rx lock — which is held by the old handler. Because the sandbox
lock is still held during this wait, every subsequent RPC
(ExecProcess, WaitProcess, StatsContainer, SignalProcess, etc.) blocks
on the sandbox lock, rendering the pod completely unresponsive.
The root cause is a lock ordering violation: get_oom_event held the
sandbox lock while acquiring the event_rx lock. Fix this by scoping the
sandbox lock acquisition so it is dropped before the event_rx lock is
acquired. The sandbox lock is only needed to clone the Arc<Mutex<Receiver>>
— once cloned, it can be released immediately.
Assisted-by: Claude Code <noreply@anthropic.com>
Signed-off-by: Thejas N <thn@redhat.com>
Clean the runtime configuration section by focusing first on the helm
configuration. Then, pivot into a further explanation on how the runtime
can be directly configured. Link to where these config parameters are
explained more in-depth.
Add open-in-new-tab (already downloaded in requirements.txt) in the
mkdocs plugin config so that links don't open in the same tab.
Signed-off-by: LandonTClipp <lclipp@coreweave.com>
prepare_protection_device_config() called available_guest_protection()
unconditionally and propagated any error before the "confidential_guest
is not set" case was handled.
On AMD hosts where the kvm_amd `sev` module parameter is "Y" but the CPU
does not expose the SEV-SNP CPUID bit (8000_001f EAX[4]) -- e.g. consumer
Ryzen -- available_guest_protection() returns Err("SEV not supported"),
which blocked every non-confidential VM from booting even though no
protection was requested.
When confidential_guest is not set there is no reason to probe the host,
so return Ok(None) before calling available_guest_protection(). Detection
(and any error it produces) now runs only when a confidential guest is
actually requested.
Signed-off-by: nikolasgkou <nikolasgkou@disroot.org>
containerd uses the proxy plugin root export when reporting CRI image
filesystem paths. Without this export, the CRI plugin falls back to
/var/lib/containerd/io.containerd.snapshotter.v1.<snapshotter>.
For nydus-for-kata-tee this fallback does not match the actual
snapshotter root under /var/lib/nydus-for-kata-tee.
Kubelet/cAdvisor then fails stats collection when it tries to inspect
the nonexistent fallback path.
Export the nydus proxy snapshotter root so containerd reports the real
filesystem path for resource accounting.
When using trusted ephemeral storage or a new ephemeral storage wip
feature for providing plain disks, resource accounting would not kick
in and pods which exhausted their emptyDir sizeLimits would not get
evicted.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
The kata-deploy main image pinned its gcr.io/distroless/static-debian13
base by sha256 digest. distroless does not publish versioned tags, so a
pinned digest just goes stale with no clear upgrade path. Track the
rolling tag instead (guarded with a hadolint DL3007 ignore plus a comment
explaining why), matching the kata-deploy-job-dispatcher image base.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Document the new opt-in deploymentMode: job alongside the default
DaemonSet model in the maintained docs (not just the chart README):
- helm-configuration.md: add a "Deployment Modes (DaemonSet vs Job)"
section covering the dispatcher-driven staged install/cleanup
pipelines, why a dispatcher is used instead of Helm-rendered per-node
Jobs (O(1) release, guaranteed coverage, paced rollout, explicit
privilege split), the "re-run helm upgrade to cover newly added
nodes" model (no always-on reconcile component), and the
node-selection precedence (job.nodes > job.nodeSelector +
job.nodeSelectorExpressions) that defaults to worker nodes.
- installation.md: note that the DaemonSet is the default but no longer
the only model, linking to the section above.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
The verification Job assumed the DaemonSet model: it waited for the
DaemonSet to exist, for its pods, and for `rollout status daemonset/...`,
then required every node in the cluster to be labeled. None of that holds
for deploymentMode: job, where install happens via the dispatcher and the
per-node Jobs it fans out, and only the targeted (worker) nodes get
labeled.
Make the hook mode-aware:
- Hook weight: in job mode the install dispatcher runs as a
post-install hook at weight 5, so verification now runs at weight 10
(after it); daemonset mode keeps weight 0 (the DaemonSet is a normal
resource).
- Readiness wait: in job mode, wait for the install dispatcher Job to
complete and then for the per-node install Jobs
(kata-deploy/stage=install) to finish (with the same CRI-restart
retry logic) instead of a DaemonSet rollout.
- Label check: in job mode, verify exactly the nodes the dispatcher
targeted are labeled, rather than comparing the labeled count against
all nodes in the cluster.
- Grant the verification ClusterRole read access to batch/jobs (used by
the job-mode waits; harmless in daemonset mode).
The daemonset code path is unchanged and the default render (no
verification.pod) is byte-for-byte identical.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
