328383adc3
InitialSizeManager::setup_config() is responsible for applying the
sandbox workload sizing (computed from containerd/CRI-O sandbox
annotations) to the hypervisor configuration before VM creation.
Previously, the workload vCPU count was only logged but never actually
added to default_vcpus, so the VM was always created with only the base
vCPUs from the configuration/annotations. This caused the
k8s-sandbox-vcpus-allocation test to fail with qemu-snp-runtime-rs:
a pod with default_vcpus=0.75 and a container CPU limit of 1.2 should
see ceil(0.75 + 1.2) = 2 vCPUs, but only got 1.
Additionally, the workload memory was being added to default_memory
unconditionally, diverging from the Go runtime which only applies both
CPU and memory additions when static_sandbox_resource_mgmt is enabled.
In the non-static path, adding workload resources here would cause
double-counting: once from setup_config() at sandbox creation, and
again from update_cpu_resources()/update_mem_resources() when
individual containers are added.
Guard both additions behind static_sandbox_resource_mgmt, matching the
Go runtime's behavior in src/runtime/pkg/oci/utils.go:
if sandboxConfig.StaticResourceMgmt {
sandboxConfig.HypervisorConfig.NumVCPUsF += sandboxConfig.SandboxResources.WorkloadCPUs
sandboxConfig.HypervisorConfig.MemorySize += sandboxConfig.SandboxResources.WorkloadMemMB
}
Fixes: k8s-sandbox-vcpus-allocation test failure on qemu-snp-runtime-rs
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
runtime-rs
What is runtime-rs
runtime-rs is a core component of Kata Containers 4.0. It is a high-performance, Rust-based implementation of the containerd shim v2 runtime.
Key characteristics:
- Implementation Language: Rust, leveraging memory safety and zero-cost abstractions
- Project Maturity: Production-ready component of Kata Containers 4.0
- Architectural Design: Modular framework optimized for Kata Containers 4.0
For architecture details, see Architecture Overview.
Architecture Overview
Key features:
- Built-in VMM (Dragonball): Deeply integrated into shim lifecycle, eliminating IPC overhead for peak performance
- Asynchronous I/O: Tokio-based async runtime for high-concurrency with reduced thread footprint
- Extensible Framework: Pluggable hypervisors, network interfaces, and storage backends
- Resource Lifecycle Management: Comprehensive sandbox and container resource management
Crates
| Crate | Description |
|---|---|
shim |
Containerd shim v2 entry point (start, delete, run commands) |
service |
Services including TaskService for containerd shim protocol |
runtimes |
Runtime handlers: VirtContainer (default), LinuxContainer(experimental), WasmContainer(experimental) |
resource |
Resource management: network, share_fs, rootfs, volume, cgroups, cpu_mem |
hypervisor |
Hypervisor implementations |
agent |
Guest agent communication (KataAgent) |
persist |
State persistence to disk (JSON format) |
shim-ctl |
Development tool for testing shim without containerd |
shim
Entry point implementing containerd shim v2 binary protocol:
start: Start new shim processdelete: Delete existing shim processrun: Run ttRPC service
service
Extensible service framework. Currently implements TaskService conforming to containerd shim protocol.
runtimes
Runtime handlers manage sandbox and container operations:
| Handler | Feature Flag | Description |
|---|---|---|
VirtContainer |
virt (default) |
Virtual machine-based containers |
LinuxContainer |
linux |
Linux container runtime (experimental) |
WasmContainer |
wasm |
WebAssembly runtime (experimental) |
resource
All resources abstracted uniformly:
- Sandbox resources: network, share-fs
- Container resources: rootfs, volume, cgroup
Sub-modules: cpu_mem, cdi_devices, coco_data, network, share_fs, rootfs, volume
hypervisor
Supported hypervisors:
| Hypervisor | Mode | Description |
|---|---|---|
| Dragonball | Built-in | Integrated VMM for peak performance (default) |
| QEMU | External | Full-featured emulator |
| Cloud Hypervisor | External | Modern VMM (x86_64, aarch64) |
| Firecracker | External | Lightweight microVM |
| Remote | External | Remote hypervisor |
The built-in VMM mode (Dragonball) is recommended for production, offering superior performance by eliminating IPC overhead.
agent
Communication with guest OS agent via ttRPC. Supports KataAgent for full container lifecycle management.
persist
State serialization to disk for sandbox recovery after restart. Stores state.json under /run/kata/<sandbox-id>/.
Build from Source and Install
Prerequisites
Download Rustup and install Rust. For Rust version, see languages.rust.meta.newest-version in versions.yaml.
Example for x86_64:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env
rustup install ${RUST_VERSION}
rustup default ${RUST_VERSION}-x86_64-unknown-linux-gnu
Musl Support (Optional)
For fully static binary:
# Add musl target
rustup target add x86_64-unknown-linux-musl
# Install musl libc (example: musl 1.2.3)
curl -O https://git.musl-libc.org/cgit/musl/snapshot/musl-1.2.3.tar.gz
tar vxf musl-1.2.3.tar.gz
cd musl-1.2.3/
./configure --prefix=/usr/local/
make && sudo make install
Install Kata 4.0 Rust Runtime Shim
git clone https://github.com/kata-containers/kata-containers.git
cd kata-containers/src/runtime-rs
make && sudo make install
After installation:
- Config file:
/usr/share/defaults/kata-containers/configuration.toml - Binary:
/usr/local/bin/containerd-shim-kata-v2
Install Without Built-in Dragonball VMM
To build without the built-in Dragonball hypervisor:
make USE_BUILTIN_DB=false
Specify hypervisor during installation:
sudo make install HYPERVISOR=qemu
# or
sudo make install HYPERVISOR=cloud-hypervisor
Configuration
Configuration files in config/:
| Config File | Hypervisor | Notes |
|---|---|---|
configuration-dragonball.toml.in |
Dragonball | Built-in VMM |
configuration-qemu-runtime-rs.toml.in |
QEMU | Default external |
configuration-cloud-hypervisor.toml.in |
Cloud Hypervisor | Modern VMM |
configuration-rs-fc.toml.in |
Firecracker | Lightweight microVM |
configuration-remote.toml.in |
Remote | Remote hypervisor |
configuration-qemu-tdx-runtime-rs.toml.in |
QEMU + TDX | Intel TDX confidential computing |
configuration-qemu-snp-runtime-rs.toml.in |
QEMU + SEV-SNP | AMD SEV-SNP confidential computing |
configuration-qemu-se-runtime-rs.toml.in |
QEMU + SEV | AMD SEV confidential computing |
configuration-qemu-coco-dev-runtime-rs.toml.in |
QEMU + CoCo | CoCo development |
See runtime configuration for configuration options.
Logging
See Developer Guide - Troubleshooting.
Debugging
For development, use shim-ctl to test shim without containerd dependencies.
Limitations
See Limitations for details.