Map BlockConfigModern.discard_unmap to Dragonball block device
sparse support when adding modern block devices.
This lets runtime-rs request guest-visible discard support through
the Dragonball block device configuration while leaving vhost-user
block devices disabled.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Add a sparse flag to Dragonball block device configuration and use
it to expose virtio-blk discard support to the guest. Handle discard
requests by validating the requested range and punching holes in
sparse local file backends, returning an I/O error instead of
panicking on discard failures.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Add the Cloud Hypervisor disk sparse option to the local DiskConfig
model and keep its default aligned with upstream. Map
BlockConfigModern.discard_unmap to DiskConfig.sparse so block devices
that request discard/unmap explicitly expose sparse discard support
through the CH API.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
To tells if the hypervisor exposes block discard/unmap support to
the guest, we introduce a new capability of discard/unmap named
`BlockDeviceDiscardSupport` in the capability set.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Remove the remaining legacy Block device implementation now that
BlockModern covers all block paths, finishing the migration started
by the QEMU and DeviceManager legacy-path drops.
Delete virtio_blk.rs (BlockConfig, BlockDevice and its Device impl)
and stop exporting it from driver/mod.rs. Drop the BlockCfg variant
from DeviceConfig and the Block variant from DeviceType. Remove the
DeviceConfig::BlockCfg handling and the create_block_device helper
from DeviceManager, and drop DeviceType::Block from the QEMU
hotunplug unsupported list.
No new behavior; pure removal of the dead legacy path.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Remove DeviceType::Block so BlockModern is the single code path,
completing the BlockConfig -> BlockConfigModern migration at the
Device Manager layer.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The preceding "Use BlockConfigModern for ..." series converted every
block-device producer in the Qemu path to emit BlockModern. The Qemu
inner module still held the matching legacy Block consumer code across
all three lifecycle ops (coldplug/hotplug/hotunplug), now unreachable.
Remove it so BlockModern is the single code path, completing the
BlockConfig -> BlockConfigModern migration at the Qemu layer.
The hotunplug arm for Block now returns an explicit "unsupported"
error instead of a stale QMP detach. The new added Block(_) is just a
place holder and it will be removed in a follow-up commit.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Migrate the block config from BlockConfig to BlockConfigModern,
routing it through DeviceConfig::BlockCfgModern for firecacker.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Migrate the block config from BlockConfig to BlockConfigModern,
routing it through DeviceConfig::BlockCfgModern for agent-ctl vm_ops.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Migrate the erofs rootfs path from BlockConfig to BlockConfigModern,
routing its block-device attachment through BlockCfgModern.
This covers the rw layer, single-device raw, multi-device VMDK, and
GPT-partitioned VMDK attachment points in erofs_rootfs.rs.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Migrate the block config from BlockConfig to BlockConfigModern,
routing it through DeviceConfig::BlockCfgModern for swap task.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Migrate the block config from BlockConfig to BlockConfigModern,
routing it through DeviceConfig::BlockCfgModern for block rootfs.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Migrate the host block-device (major/minor) passthrough path in
ResourceManagerInner from BlockConfig to BlockConfigModern, routing it
through DeviceConfig::BlockCfgModern. This continues the BlockModern
migration to cover block-mode devices carried in the OCI spec.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Switch the InitData resource path from BlockConfig to BlockConfigModern
so the InitData is handled through the modern block device pipeline,
consistent with the recent BlockModern migration for block volumes.
This changes ResourceConfig::InitData to wrap BlockConfigModern,
routes its handling through DeviceConfig::BlockCfgModern, and updates
its related helper function in virt_container to build a
BlockConfigModern.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Switch the VmRootfs resource path from BlockConfig to BlockConfigModern
so the VM rootfs is handled through the modern block device pipeline,
consistent with the recent BlockModern migration for block volumes.
This changes ResourceConfig::VmRootfs to wrap BlockConfigModern, routes
its handling through DeviceConfig::BlockCfgModern, and updates
prepare_rootfs_config() in virt_container to build a BlockConfigModern.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Introduce a new field of serial_override in the BlockModern config.
Add a DeviceType::BlockModern branch to the QEMU QMP hotplug path,
mirroring the existing Block semantics: skip block devices that
represent the initrd, and dispatch to nvdimm or blk/ccw/scsi
according to driver_option. Snapshot the config parameters under
lock and run the asynchronous hotplug without holding it to avoid
awaiting across the lock.
Pass the real block format and, when enabled, the first independent
iothread (indep_iothread_0) to hotplug_block_device instead of the
default format and None, and return DeviceType::BlockModern so the
hotplugged device is tracked.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Treat SPDK and VFIO direct volumes as a two-step mount. The agent
storage entry mounts the device inside the guest with block storage
options, while the OCI mount bind-mounts that guest path into the
container.
Filter bind flags from storage options, preserve direct-volume
mountInfo options when present, and add ro when the volume is
read-only.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Build regular block volumes with BlockConfigModern and submit them
through DeviceConfig::BlockCfgModern. Preserve the existing block
device attributes while also recording the host path required by the
modern block device model.
Update block volume helper call sites with the default option slice
so this change remains buildable with the modern helper signature.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Switch the Cloud Hypervisor device handling path from the legacy
Block device variant to BlockModern. Keep the block device behind
its shared mutex while snapshotting the configuration for the
hotplug request, and persist the PCI path assigned by Cloud
Hypervisor back into the shared device state.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Introduce build_bind_mount_options() to construct OCI mount options for
container-side bind mount, preferring volume_options over OCI options
and ensuring a bind/rbind flag is present.
And addfilter_block_storage_options() to strip bind/rbind flags from
Storage.options, preventing ENOTDIR when the agent mounts a block
device to a filesystem inside the guest.
And then Extend handle_block_volume() with a volume_options parameter
and replace hardcoded ro/empty options with the new filter+ro logic,
narrowing the function to a BlockModern-only path by removing the
DeviceType::Block branch.
Fix generate_shared_path to always create under rw/ since ro/
is a read-only bind mount of rw/ and direct creation fails with ReadOnly
FS. Switch the OCI Mount type from fs_type to KATA_MOUNT_BIND_TYPE because
the mount source is a directory (guest_path), not a block device.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Move the block device type definitions shared between BlockDevice and
BlockDeviceModern — VIRTIO_BLOCK_{PCI,MMIO,CCW}/VIRTIO_PMEM constants,
KATA_*_DEV_TYPE aliases, and the BlockDeviceAio / BlockDeviceFormat
enums — from virtio_blk.rs into virtio_blk_modern.rs, so they live
alongside the BlockModern handler that will replace the legacy
BlockDevice.
Add a `format: BlockDeviceFormat` field to BlockConfigModern to match
the legacy BlockConfig. Update mod.rs re-exports accordingly.
No behavior change; pure relocation plus the new field.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Extend add_block_device with num_queues and queue_size parameters
to support multi-queue virtio-blk configuration, and add BlockModern
hotplug/hotunplug handler that extracts config from
Arc<Mutex<BlockDeviceModern>>, passes the new parameters to
add_block_device, and updates pci_path on successful hotplug.
And we will use BlockDeviceModern to handle block device instead of
Legacy BlockDevice.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Add BlockModern handling in DeviceManager so that device index is
correctly released both in get_device_info and in the error path.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Use the hotplug-returned SCSI address as Storage.source when a
container rootfs is backed by a block device.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
src/dragonball/out is an 893 KB `cargo metadata` JSON dump that was swept
into the tree by accident in commit 1d2d2cc3d ("ci: Remove duplicated
rust-vmm dependencies"), whose only intended change was a one-line edit to
.github/dependabot.yml. Nothing references the file -- no script, CI job,
or Makefile -- so it is pure build cruft. Remove it.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
At src/dragonball/, the dragonball crate's own module tree (src/) sat as a
sibling of whole separate crates (dbs_acpi/, dbs_pci/, ...). That level
mixed one crate's guts with other crates -- neither cleanly one crate nor
a clean set of crates. Move the dbs_* under src/dragonball/crates/ so the
top level is the dragonball crate plus a crates/ folder of its libraries,
matching the layout src/runtime-rs already uses.
The dbs_* were imported ~2 years ago from the standalone dragonball-sandbox
repo and kept that repo's flat layout verbatim. They are now private to
kata (unpublished, versions frozen) -- plain internal workspace members --
so no external consumers are affected and the "independently published
building blocks" rationale for the flat layout no longer holds. Pure
layout change; no code or build behavior changes.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
In confidential mode, the host is not trusted and might try to create
malicious files in the guest VM. This commit changes the implementation
of the CopyFile method to use pathrs, which is designed to prevent
accidental writes to locations outside the shared directory
(/run/kata-containers/shared/containers). It also adds a detailed
description of what the method does, and test cases for intended and
malicious use cases.
This was originally considered as a mitigation candidate for
https://github.com/kata-containers/kata-containers/security/advisories/GHSA-q49m-57vm-c8cc,
but we decided to go with the simpler change to address the immediate
risk. It's still desirable to lock down CopyFile writes for defense in
depth, thus the resubmission outside the GHSA.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
Use dedicated empty firmware makefile variables so the NVIDIA CPU
runtime does not inherit generic OVMF/AAVMF firmware paths.
runtime-rs already sets the correct option by default, so, no changed
needed on that front.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
The network hotplug (handle_network_device) passed `queue_num`, a
queue pair count, straight as `num_queues`, while cloud-hypervisor's
virtio-net expects a raw virtqueue count.
With the default network_queues=1 this hit "Number of queues (1) to
virtio_net should be higher than 2" on the post-start netdev add,
failing sandbox start.
Convert the pair count to the raw queue count (queue_num.max(1) * 2),
matching the coldplug path which already does network_queues_pairs * 2.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
NetworkConfig.queue_num is a queue pair count, but Dragonball's
virtio/vhost-net backends expect an even raw virtqueue count. Passing
it through without doubling caused InvalidQueueNum(1) on the default
network_queues=1, failing sandbox start.
Convert pair count to raw count (queue_num.max(1) * 2) at both
Dragonball network entry points, mirroring the Cloud Hypervisor
backend.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
As part of the effort to thread network queues from the top-level
configuration all the way down to each endpoint instead of having
low-level helpers re-read the global configuration, propagate the
per-network queue_num into the QEMU cmdline generator and hotplug
path.
add_network_device() and get_network_device() now take an explicit
num_queues argument sourced from network.config.queue_num, replacing
the direct read of network_info.network_queues, so each network
endpoint honors its own configured queue count.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
As there's no field to map the configuration's network_queues item
in the `DanConfig`, this commit introduces a network_queues to do this.
And accordingly, we also make it passed down from sandbox layer to
Dan network configurations.
To make it work well, it make it more robust for queues and queue_size
settings with checking logics. And related UT is added.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Set queue_num (derived from net_pair.queues, one TX + one RX per
queue pair) and queue_size (256) when building the network pair for
veth, macvlan, ipvlan and vlan endpoints so the virtio-net device is
created with the configured multi-queue layout.
Meanwhile, update endpoint tests accordingly.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Number of virtio queue pairs (each pair = 1 RX + 1 TX).
Derived from `network_queues` in the hypervisor TOML config.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Move runtime-rs onto containerd-shim/containerd-shim-protos 0.11.0,
which drops the legacy cgroups-rs 0.3.x dependency. Adapt to ttrpc
0.9 API changes for async connect, server listener setup, and event
forwarding.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Composer <cursor@cursor.com>
Unify agent, rustjail, and runtime-rs on cgroups-rs 0.5.1 from
crates.io instead of the v0.3.5 git fork. Add a rustjail shim for
the 0.5.x API layout where the legacy cgroupfs types moved under
cgroups::fs.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Composer <cursor@cursor.com>
cdi-rs 1.1.1 is the first release built against oci-spec 0.10, so
the Spec type its inject_devices() takes now unifies with the
workspace oci-spec and the kata-agent builds again.
kata-agent and pod-resources-rs each carried their own pin (0.1.1
and 0.1.2). Hoist the dependency to the root workspace like
oci-spec in the previous commit, so both consumers stay on the
same cdi types and future bumps apply to the whole tree at once.
The kata code compiles unchanged against the 0.1.2 -> 1.1.1 jump;
the API surface we use (annotations, cache, spec_dirs,
inject_devices) is source-compatible.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Assisted-by: Claude Fable 5 <noreply@anthropic.com>
Bumping oci-spec to 0.10.0 in the root manifest broke the
kata-agent build: kata-types, protocols and kata-sys-util
declare their own oci-spec pin (0.8.1) instead of inheriting
the workspace dependency, so the bump left them behind on
0.8.4 while the agent and runtime-rs crates moved to 0.10.0.
Since 0.x lines are semver-incompatible, cargo compiles both
versions side by side, and every oci-spec type these libs
expose in their public API (e.g. oci_spec::runtime::Spec) no
longer matches the type the agent expects — a wall of
"expected Spec, found Spec" mismatches.
Switch the three crates to `workspace = true` like the other
members so the root manifest is the single source of truth
and future bumps apply to the whole tree atomically. The
`runtime` feature is inherited from the workspace declaration.
No resolution change yet: both sides currently resolve to
0.8.4; the actual bump follows in the next commit.
Assisted-by: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
On hosts with memory-only NUMA nodes (GPU/CXL memory, e.g. the empty
nodes 1-8 a Grace-Hopper GH200 exposes next to its single CPU node), the
auto NUMA path built a guest node for every host node and then failed in
DistributeVCPUsProportionally with "HostCPUs for NUMA node N must not be
empty". This broke sandbox creation for any qemu-nvidia-cpu pod once
enable_numa is on, not just the NUMA tests.
Filter CPU-less nodes out of the auto-derived topology in
maybeRightSizeAutoNUMA() so the guest CPU topology only spans nodes that
can host vCPUs. Keep the full set when a VFIO device is attached, since a
passed-through GPU may live on a CPU-less node that still needs a guest
NUMA node for pxb-pcie placement.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
qemu-nvidia-gpu boots the driver-agnostic NVIDIA base image and then
cold-plugs the GPU extension on top. Nothing, however, just boots that
base image on NVIDIA hosts that have no GPU (or where the GPU isn't handed
to the guest), even though the base image is perfectly usable on its own.
Add qemu-nvidia-cpu (Go) and qemu-nvidia-cpu-runtime-rs (Rust), cloned
from the qemu-nvidia-gpu configs with every GPU-passthrough bit dropped:
no guest_extension_images, cold_plug_vfio = "no-port", pcie_root_port = 0,
no VFIO pod-resource wiring and no GPU NUMA validation. Resource defaults
and timeouts are pulled back in line with qemu[-runtime-rs] (1 vCPU,
DEFMEMSZ, small overheads) rather than the GPU-tuned values, and firmware
is left to the arch default so x86_64 boots without OVMF.
Both classes boot the measured nvidia base image verity-backed. The Rust
runtime already maps KERNELVERITYPARAMS_NV to the base hash; the Go runtime
maps it to the monolith, so a dedicated KERNELVERITYPARAMS_NV_BASE var is
added for qemu-nvidia-cpu.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Under static_sandbox_resource_mgmt the VM is sized to exactly what OCI
requests (base default_vcpus plus workload CPU limits) and CPU hotplug is
off, so DefaultMaxVCPUs is set to the sized NumVCPUs().
When a pod requests default_vcpus=0 with no CPU limit, that sized count
rounds to 0. DefaultMaxVCPUs then becomes 0, which the hypervisor later
expands to the host maximum (govmm.MaxVCPUs()). With NUMA enabled the SMP
topology becomes sockets=maxcpus=host_max, e.g. maxcpus=240, which fails
to boot on hosts where KVM caps the recommended vCPU count (the sandbox
never comes up and k8s-sandbox-vcpus-allocation / k8s-cpu-ns time out on
qemu-nvidia-cpu).
A VM cannot boot with zero vCPUs (QEMU already clamps -smp to 1), so floor
DefaultMaxVCPUs at that same single boot vCPU rather than falling back to
the host maximum. This keeps the value equal to the OCI-requested count in
every non-degenerate case and never consults the host maximum under static
management.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Assisted-by: Cursor <cursoragent@cursor.com>
Use an explicit memory-backend-ram object for normal,
non-file-backed QEMU memory and bind it through the machine
memory-backend option.
This aligns the runtime-rs QEMU command line more closely with the
runtime-go QEMU path when filesystem sharing is disabled. Further,
when enable_mem_prealloc is enabled, runtime-rs now emits
prealloc=on for that backend, matching the runtime-go behavior.
This supports #13363, where the non-confidential NVIDIA runtime-rs
handler disables filesystem sharing and uses the EROFS snapshotter as
its Kubernetes image-layer transport. This is expected to reduce the
likelihood of the QMP startup delays seen in #13084 for NVIDIA GPU
passthrough with filesystem sharing disabled. A separate issue on the
behavior of the QMP timeout logic is additionally tracked in #13343.
In addition to these changes, #12187 tracks a machine-centric QEMU
refactor. In particular, the phase 3 work described there plans to
lift memory backends into QEMU Objects. After this work, command
line generation will be refactored, and, full NUMA support be ported
to runtime-rs.
This commit also adds the first command-line generator unit tests for
this file to cover the new RAM backend behavior.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Assisted-by: OpenAI Codex <codex@openai.com>
When containerd wants to terminate a container, it sends a KillRequest with exec_id="", all=false, signal=SIGTERM.
Kata-shim translates that to a SignalProcessRequest with exec_id="", signal=SIGTERM
Kata agent used to interpret exec_id="" as implying all=true.
That was correct for forceful deletion of containers, as it uses a KillRequest with all=true, signal=SIGKILL.
However, for graceful termination, we want to only signal the root process of the container (all=false).
This changes makes it so that Kata agent interprets exec_id="" && signal=SIGKILL as implying all=true, and uses all=false otherwise.
It also fixes an unrelated bug, where Kata agent would signal the root process twice when all=true.
Fixes#13152 - sending SIGTERM twice to the container's init process on graceful shutdown of a container.
Signed-off-by: Bozhidar Marinov <bozhidar.marinov1@digits.schwarz>
Co-authored-by: Markus Rudy <webmaster@burgerdev.de>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
