Compare commits

...

62 Commits

Author SHA1 Message Date
dependabot[bot]
46fb789778 build(deps): bump the tracing group across 3 directories with 1 update
Bumps the tracing group with 1 update in the /src/agent directory: [tracing](https://github.com/tokio-rs/tracing).
Bumps the tracing group with 1 update in the /src/tools/agent-ctl directory: [tracing](https://github.com/tokio-rs/tracing).
Bumps the tracing group with 1 update in the /src/tools/kata-ctl directory: [tracing](https://github.com/tokio-rs/tracing).


Updates `tracing` from 0.1.41 to 0.1.44
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.41...tracing-0.1.44)

Updates `tracing` from 0.1.41 to 0.1.44
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.41...tracing-0.1.44)

Updates `tracing` from 0.1.41 to 0.1.44
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.41...tracing-0.1.44)

Updates `tracing` from 0.1.41 to 0.1.44
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.41...tracing-0.1.44)

Updates `tracing` from 0.1.41 to 0.1.44
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.41...tracing-0.1.44)

Updates `tracing` from 0.1.41 to 0.1.44
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.41...tracing-0.1.44)

Updates `tracing` from 0.1.41 to 0.1.44
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.41...tracing-0.1.44)

Updates `tracing` from 0.1.41 to 0.1.44
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.41...tracing-0.1.44)

Updates `tracing` from 0.1.41 to 0.1.44
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.41...tracing-0.1.44)

---
updated-dependencies:
- dependency-name: tracing
  dependency-version: 0.1.44
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: tracing
- dependency-name: tracing
  dependency-version: 0.1.44
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: tracing
- dependency-name: tracing
  dependency-version: 0.1.44
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: tracing
- dependency-name: tracing
  dependency-version: 0.1.44
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: tracing
- dependency-name: tracing
  dependency-version: 0.1.44
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: tracing
- dependency-name: tracing
  dependency-version: 0.1.44
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: tracing
- dependency-name: tracing
  dependency-version: 0.1.44
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: tracing
- dependency-name: tracing
  dependency-version: 0.1.44
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: tracing
- dependency-name: tracing
  dependency-version: 0.1.44
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: tracing
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-09 15:09:20 +00:00
Hyounggyu Choi
f15f7f49f1 Merge pull request #12787 from fidencio/topic/runtime-rs-qemu-arm64-use-static-sandbox-resource-mgmt
runtime: qemu: Enable static sandbox resource management on ARM & s390x
2026-04-09 09:18:11 +02:00
Fabiano Fidêncio
80b0ed273f Merge pull request #12784 from hgowda-amd/sev-snp-tests-required
Add sev-snp, qemu-snp CIs as required
2026-04-09 00:22:49 +02:00
Harshitha Gowda
bb1165b23f tests: Set sev-snp, qemu-snp CIs as required
run-k8s-tests-on-tee (sev-snp, qemu-snp)

Signed-off-by: Harshitha Gowda <hgowda@amd.com>
2026-04-08 22:36:58 +02:00
Fabiano Fidêncio
2148afe243 Merge pull request #12796 from fidencio/topic/kata-deploy-run-cargo-fmt-and-cargo-check
kata-deploy: Run cargo clippy during build
2026-04-08 22:32:31 +02:00
Fabiano Fidêncio
8ff630059a Merge pull request #12778 from amd-aliem/enable-img-rootfs-snp
runtime: SNP img-based rootfs with dm-verity
2026-04-08 22:06:31 +02:00
Fabiano Fidêncio
4561ae3e29 Merge pull request #12799 from fitzthum/fixup-nv-doc-1
docs: update flow for setting nvidia devices to ready
2026-04-08 21:32:55 +02:00
Tobin Feldman-Fitzthum
9119b4982c docs: update flow for setting nvidia devices to ready
Now, we include the nvrc.smi.srs=1 flag in the default kernel cmdline.
Thus, we can remove the guidance for people to add it themselves when
not using attestation. In fact, users don't really need to know about
this flag at all.

Signed-off-by: Tobin Feldman-Fitzthum <tfeldmanfitz@nvidia.com>
2026-04-08 18:59:51 +00:00
Fabiano Fidêncio
21466eb4e5 kata-deploy: Fix clippy warnings across crate
Fix all clippy warnings triggered by -D warnings:

- install.rs: remove useless .into() conversions on PathBuf values
  and replace vec! with an array literal where a Vec is not needed
- utils/toml.rs: replace while-let-on-iterator with a for loop and
  drop the now-unnecessary mut on the iterator binding
- main.rs: replace match-with-single-pattern with if-let in two
  places dealing with experimental_setup_snapshotter
- utils/yaml.rs: extract repeated serde_yaml::Value::String key into
  a local variable, removing needless borrows on temporary values

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
2026-04-08 20:47:59 +02:00
Fabiano Fidêncio
1874d4617b kata-deploy: Run cargo clippy during build
Ensure code formatting and compilation are verified early in the
Docker build pipeline, before tests and the release build.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
2026-04-08 20:47:59 +02:00
Amanda Liem
79f844d057 runtime: SNP img-based rootfs with dm-verity
Follow-on to kata-containers/kata-containers#12396

Switch SNP config from initrd-based to image-based rootfs with
dm-verity. The runtime assembles the dm-mod.create kernel cmdline
from kernel_verity_params, and with kernel-hashes=on the root hash
is included in the SNP launch measurement.

Also add qemu-snp to the measured rootfs integration test.

Signed-off-by: Amanda Liem <aliem@amd.com>
2026-04-08 16:46:32 +00:00
Greg Kurz
817580e35d Merge pull request #12795 from fidencio/topic/kata-deploy-do-not-try-to-install-a-snapshotter-when-using-crio
kata-deploy: Skip snapshotter install/uninstall on CRI-O
2026-04-08 17:18:05 +02:00
Fabiano Fidêncio
e93bfbe01a tests: Remove qemu-coco-dev* skip from sandbox vCPU allocation test
With static_sandbox_resource_mgmt calculation fixed for runtime-rs, the
VM is correctly pre-sized at creation time. The vCPU allocation test no
longer depends on CPU hotplug, so the qemu-coco-dev* skip is no longer
needed.

Fixes: #10928

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
2026-04-08 16:36:00 +02:00
Fabiano Fidêncio
6bc2452664 tests: Remove aarch64 skip from sandbox vCPU allocation test
With static_sandbox_resource_mgmt now enabled for ARM on runtime-rs,
the VM is correctly pre-sized at creation time. The vCPU allocation
test no longer depends on CPU hotplug, so the aarch64 skip (issue
 #10928) is no longer needed.

Fixes: #10928

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
2026-04-08 16:36:00 +02:00
Fabiano Fidêncio
e0141991d3 runtime-rs: Enable static sandbox resource management on s390x
runtime-rs memory hotplug hard-codes the `pc-dimm` device driver, which
is an x86-only QEMU device model. On s390x, the `s390-ccw-virtio`
machine type does not support `pc-dimm` at all — the Go runtime handles
this by using `virtio-mem-ccw` instead (controlled by the
`enable_virtio_mem` config knob, defaulting to true on s390x).

runtime-rs has no virtio-mem support, so any attempt to dynamically
hotplug memory on s390x fails with:

  'pc-dimm' is not a valid device model name

This is a pre-existing limitation on main — it has never worked. It is
now visible because commit 45dfb6ff25 ("runtime-rs: Fix initial vCPU /
memory with static_sandbox_resource_mgmt") expanded runtime-rs test
coverage, causing k8s-memory.bats and k8s-oom.bats to actually exercise
this code path on s390x.

Let's enforce using static_sandbox_resources_mgmt also for s390x so the
VM is sized upfront at creation time, bypassing the broken dynamic
hotplug path entirely.

If someone decides to implement hotplug support for s390x, the work
would basically be an implemntation of virtio-mem-ccw support in the
runtime-rs QEMU backend (boot-time device creation, qom-set based
resize, and virtio-mem aware memory accounting), mirroring what the Go
runtime already does, but I'm not game for this (sorry).

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
2026-04-08 16:36:00 +02:00
Fabiano Fidêncio
ffab9b7eee runtime: qemu: Enable static sandbox resource management on ARM
runtime-rs lacks several features needed for CPU hotplug on ARM:
pflash/UEFI firmware passthrough, SMP topology in -smp, nr_cpus
kernel parameter, and QMP vCPU add handling for the virt machine
type (which requires core-id only placement with socket/thread/die
set to -1).

Without static sandbox resource management, these gaps cause
failures in tests like k8s-memory.bats where the VM is not correctly
sized for the workload.

Enable static_sandbox_resource_mgmt for aarch64 in the QEMU
runtime-rs configuration so the VM is pre-sized at creation time,
sidestepping the need for hotplug entirely.

Together with this we're aligning the go runtime to the very same
behaviour.

Fixes: #10928

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
2026-04-08 16:36:00 +02:00
Fabiano Fidêncio
0e5e4802d7 runtime-rs: Fix initial vCPU / memory with static_sandbox_resource_mgmt
InitialSizeManager::setup_config() is responsible for applying the
sandbox workload sizing (computed from containerd/CRI-O sandbox
annotations) to the hypervisor configuration before VM creation.

Previously, the workload vCPU count was only logged but never actually
added to default_vcpus, so the VM was always created with only the base
vCPUs from the configuration/annotations. This caused the
k8s-sandbox-vcpus-allocation test to fail with qemu-snp-runtime-rs:
a pod with default_vcpus=0.75 and a container CPU limit of 1.2 should
see ceil(0.75 + 1.2) = 2 vCPUs, but only got 1.

Additionally, the workload memory was being added to default_memory
unconditionally, diverging from the Go runtime which only applies both
CPU and memory additions when static_sandbox_resource_mgmt is enabled.
In the non-static path, adding workload resources here would cause
double-counting: once from setup_config() at sandbox creation, and
again from update_cpu_resources()/update_mem_resources() when
individual containers are added.

Guard both additions behind static_sandbox_resource_mgmt, matching the
Go runtime's behavior in src/runtime/pkg/oci/utils.go:

    if sandboxConfig.StaticResourceMgmt {
        sandboxConfig.HypervisorConfig.NumVCPUsF += sandboxConfig.SandboxResources.WorkloadCPUs
        sandboxConfig.HypervisorConfig.MemorySize += sandboxConfig.SandboxResources.WorkloadMemMB
    }

Fixes: k8s-sandbox-vcpus-allocation test failure on qemu-snp-runtime-rs

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Made-with: Cursor
2026-04-08 16:36:00 +02:00
Fabiano Fidêncio
bb051bb16a Merge pull request #12788 from fidencio/topic/kata-deploy-re-apply-GPU-specific-labels
kata-deploy: re-apply labels for the GPU runtime classes
2026-04-08 16:27:59 +02:00
Fabiano Fidêncio
bacc3f4ef1 Merge pull request #12785 from fidencio/topic/runtime-rs-deny-config
runtime-rs: Deny config of unknown fields & change dbg_monitor_socket name
2026-04-08 15:12:53 +02:00
Fabiano Fidêncio
f27def1a5b kata-deploy: Skip snapshotter install/uninstall on CRI-O
Snapshotters (nydus, erofs) are containerd-specific. The validation code
already warned that EXPERIMENTAL_SETUP_SNAPSHOTTER would be ignored on
CRI-O, but the actual install/configure and uninstall loops still ran
unconditionally, attempting containerd-specific operations on CRI-O
nodes.

Guard both the install and cleanup snapshotter loops with a `runtime !=
"crio"` check so the binary itself skips snapshotter work when it
detects CRI-O as the container runtime.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-04-08 14:41:49 +02:00
Fabiano Fidêncio
bc719a66eb kata-deploy: nvidia: Align force_guest_pull with default values.yaml
The defdault is already false, but let's keep those aligned on
explicitly setting the default.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-04-08 14:41:21 +02:00
Fabiano Fidêncio
78f02f2155 kata-deploy: nvidia: Align labels with default values.yaml
Joji's added the labels for the default values.yaml, but we missed
adding those to the nvidia specific values.yaml file.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-04-08 14:41:21 +02:00
Fabiano Fidêncio
f00b589ccd Revert "kata-deploy: Temporarily comment GPU specific labels"
This reverts commit 02c9a4b23c, as GPU
Operator v26.3.0 is out, and becomes a requirement.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-04-08 14:41:21 +02:00
Alex Lyn
c00f895338 kata-deploy: Fix noisy caused by unformatted code
When do cargo fmt --all, some files changes as unformatted with
`cargo fmt`. This commit is just to address it.

Just use this as an example:
```
         // Generate the common drop-in files (shared with standard
         // runtimes)
-        write_common_drop_ins(config, &runtime.base_config,
         &config_d_dir, container_runtime)?;
+        write_common_drop_ins(
+            config,
+            &runtime.base_config,
+            &config_d_dir,
+            container_runtime,
+        )?;
```

Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
2026-04-08 14:39:57 +02:00
Fabiano Fidêncio
6269b3ecde Merge pull request #12792 from fidencio/topic/nvidia-rootfs-take-nvrc-and-nvat-versions-in-consideration
build: cache: Take NVRC & NVAT version into consideration
2026-04-08 12:44:41 +02:00
Fabiano Fidêncio
a12e0f1204 build: cache: Take NVRC & NVAT version into consideration
Without those, we'd end up pulling the same / old rootfs that's cached
without re-building it in case of a bump in any of those components.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-04-08 10:14:11 +02:00
RuoqingHe
a4fb9aef54 Merge pull request #12789 from kata-containers/pin-actions-rs-toolchain
gha: Pin action for cargo-deny workflow
2026-04-08 08:36:13 +08:00
Fabiano Fidêncio
995767330d Merge pull request #12782 from pavithiran34/pavi-ras-version-update
fix: updated image-rs to v0.18.0
2026-04-07 23:32:05 +02:00
Aurélien Bombo
8916f5f301 gha: Pin action for cargo-deny workflow
The cargo-deny workflow should be the last workflow to not use a pinned version.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2026-04-07 15:41:09 -05:00
pavithiran34
528fa80953 fix: updated image-rs to v0.18.0
- Updated image-rs from rev 026694d4 to tag v0.18.0
- This update brings rsa 0.9.10 which fixes CVE-2026-21895
- Resolves vulnerability in indirect dependencies

Signed-off-by: pavithiran34 <pavithiran.p@ibm.com>
2026-04-07 21:40:01 +02:00
Fabiano Fidêncio
b3ae6ef99c Merge pull request #12760 from fitzthum/bump-nvat
Bump trustee and guest-components to add nvswitch / ppcie support
2026-04-07 19:07:50 +02:00
Aurélien Bombo
79fab93041 Merge pull request #12779 from rophy/fix/strip-cr-from-tty-exec
tests: strip \r from kubectl exec output for TTY containers
2026-04-07 10:19:21 -05:00
Tobin Feldman-Fitzthum
e40abcf72d nvidia: add nvrc.smi.srs=1 to default nvidia kernel params
The attestation-agent no longer sets nvidia devices to ready
automatically. Instead, we should use nvrc for this. Since this is
required for all nvidia workloads, add it to the default nv kernel
params.

With bounce buffers, the timing of attesting a device versus setting it
to ready is not so important.

Signed-off-by: Tobin Feldman-Fitzthum <tfeldmanfitz@nvidia.com>
2026-04-07 14:28:50 +00:00
Manuel Huber
0fd4559f7e docs: Update NVIDIA GPU passthrough QEMU scenario
Updates for the NVIDIA GPU passthrough scenario for the
kata-containers release 3.29.0.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2026-04-07 14:58:40 +02:00
Fabiano Fidêncio
9a5aaf7ecb runtime-rs: move create_container_timeout before [mem_agent] section
The create_container_timeout key was placed after the
[agent.@PROJECT_TYPE@.mem_agent] TOML section header, which meant
TOML parsed it as a field of mem_agent rather than of the parent
agent table. This was silently ignored before, but now that
MemAgent has #[serde(deny_unknown_fields)] it causes a parse error.

Move the key above the [mem_agent] section so it belongs to the
correct [agent.@PROJECT_TYPE@] table.

Also fix configuration-qemu-coco-dev which had a duplicate entry:
keep only the correctly placed one with the COCO timeout value.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-04-07 11:23:59 +02:00
Fabiano Fidêncio
a6e891e733 runtime-rs: s/dbg_monitor_socket/extra_monitor_socket/g
Let's align this with what's been already used for the go runtime.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-04-07 10:50:42 +02:00
Paul Meyer
b32c5234f4 runtime-rs: deny unknown fields in config
..where possible. Failing on unknown fields makes migration easier,
as we do not silently ignore configuration options that previously
worked in runtime-go. However, serde can't deny unknown fields
where flatten is used, so this can't be used everywhere sadly.

There were also errors in test fixtures that were unnoticed.
These are fixed here, too.

Signed-off-by: Paul Meyer <katexochen0@gmail.com>
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-04-07 10:50:25 +02:00
Tobin Feldman-Fitzthum
7385938c57 tests: fix default KBS Policy path
We recently moved the default policy in the Trustee repo. Now it's in
the same place as all the other policies. Update the test code to match.

Signed-off-by: Tobin Feldman-Fitzthum <tfeldmanfitz@nvidia.com>
2026-04-07 05:46:27 +00:00
Tobin Feldman-Fitzthum
38e04bb6d8 versions: bump guest-components for switch attestation
Pick up the new version of guest-components which uses NVAT bindings
instead of NVML bindings. This will allow us to attests guests with
nvswitches.

Signed-off-by: Tobin Feldman-Fitzthum <tfeldmanfitz@nvidia.com>
2026-04-07 05:46:27 +00:00
RuoqingHe
feaec78ad0 Merge pull request #12776 from fidencio/topic/kata-deploy-move-into-the-root-workspace
kata-deploy: Move into the root workspace
2026-04-07 12:45:26 +08:00
Fabiano Fidêncio
461907918d kata-deploy: pin nydus-snapshotter via versions.yaml
Resolve externals.nydus-snapshotter version and url in the Docker image build
with yq from the repo-root versions.yaml instead of Dockerfile ARG defaults.

Drop the redundant workflow that only enforced parity between those two sources.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-04-07 10:07:06 +08:00
Fabiano Fidêncio
9e1f595160 kata-deploy: add Rust binary to root workspace
Add tools/packaging/kata-deploy/binary as a workspace member, inherit shared
dependency versions from the root manifest, and refresh Cargo.lock.

Build the kata-deploy image from the repository root: copy the workspace
layout into the rust-builder stage, run cargo test/build with -p kata-deploy,
and adjust artifact and static asset COPY paths. Update the payload build
script to invoke docker buildx with -f .../Dockerfile from the repo root.

Add a repo-root .dockerignore to keep the Docker build context smaller.
Document running unit tests with cargo test -p kata-deploy from the root.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-04-07 10:07:06 +08:00
Rophy Tsai
f7d9024249 tests: strip \r from kubectl exec output for TTY containers
The busybox-pod.yaml test fixture sets tty: true on the second
container. When a container has a TTY, kubectl exec may return
\r\n line endings. The invisible \r causes string comparisons
to fail:

  container_name=$(kubectl exec ... -- env | grep CONTAINER_NAME)
  [ "$container_name" == "CONTAINER_NAME=second-test-container" ]

This comparison fails because $container_name contains a trailing
\r character.

Fix by piping through tr -d '\r' after grep. This is harmless
when \r is absent and fixes the mismatch when present.

Fixes: #9136

Signed-off-by: Rophy Tsai <rophy@users.noreply.github.com>
2026-04-07 01:35:10 +00:00
Alex Lyn
46a7b9e75d Merge pull request #12775 from RuoqingHe/put-libs-to-root-workspace
libs: Move libs into root workspace
2026-04-07 09:25:26 +08:00
Tobin Feldman-Fitzthum
3d60196735 versions: bump Trustee to pickup PPCIE support
Trustee is compatible with old guest components (using NVML bindings) or
new guest components (using NVAT). If we have the new version of gc, we
can attest PPCIE guests, which we need the new version of Trustee to
verify.

Signed-off-by: Tobin Feldman-Fitzthum <tfeldmanfitz@nvidia.com>
2026-04-06 17:51:12 +00:00
Tobin Feldman-Fitzthum
0444d70704 rootfs: add runtime support for NVAT
Update NVIDIA rootfs builder to include runtime dependencies for NVAT
Rust bindings.

The nvattest package does not include the .so file, so we need to build
from source.

Signed-off-by: Tobin Feldman-Fitzthum <tfeldmanfitz@nvidia.com>
2026-04-06 17:51:12 +00:00
Tobin Feldman-Fitzthum
78c61459f8 packaging: add built-time support for NVAT
The attestation agent will soon rely on the NVAT rust bindings, which
have some built-time dependencies.

There is currently no nvattest-dev package, so we need to build from
source to get the headers and .so file.

Signed-off-by: Tobin Feldman-Fitzthum <tfeldmanfitz@nvidia.com>
2026-04-06 17:51:12 +00:00
Dan Mihai
9b770793ba Merge pull request #12728 from manuelh-dev/mahuber/empty-dir-fsgrou-policy
genpolicy: adjust GID after passwd GID handling and set fs_group for encrypted emptyDir volumes
2026-04-06 10:22:34 -07:00
Fabiano Fidêncio
47770daa3b helm: Align values.yaml with try-kata-nvidia-gpu.values.yaml
We've switched to nydus there, but never did for the values.yaml.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-04-06 18:51:54 +02:00
Fabiano Fidêncio
1300145f7a tests: add k3s/rke2 to OCI 1.3.0 drop-in overlay condition
k3s and rke2 ship containerd 2.2.2, which requires the OCI 1.3.0
drop-in overlay. Move them from the separate OCI 1.2.1 branch into
the OCI 1.3.0 condition alongside nvidia-gpu, qemu-snp, qemu-tdx,
and custom container engine versions.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-04-06 18:50:20 +02:00
Fabiano Fidêncio
0a739b3b55 Merge pull request #12755 from katexochen/runtime-rs-config-cleanup
runtime-rs: cleanup config
2026-04-06 13:14:58 +02:00
Ruoqing He
cb7c790dc7 libs: Specify crates explicitly in Makefile
--all option would trigger building and testing for everything within
our root workspace, which is not desired here. Let's specify the crates
of libs explicitly in our Makefile.

Signed-off-by: Ruoqing He <ruoqing.he@lingcage.com>
2026-04-06 11:03:38 +02:00
Ruoqing He
2a024f55d0 libs: Move libs into root workspace
Remove libs from exclude list, and move them explicitly into root
workspace to make sure our core components are in a consistent state.

This is a follow up of #12413.

Signed-off-by: Ruoqing He <ruoqing.he@lingcage.com>
2026-04-06 11:03:38 +02:00
Fabiano Fidêncio
9a2825a429 runtime: config: Use OVMF for the qemu-nvidia-gpu
2ba0cb0d4a7 did the ground work for using OVMF even for the
qemu-nvidia-gpu, but missed actually setting the OVMF path to be used,
which we'e fixing now.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-04-06 03:54:56 +02:00
Fabiano Fidêncio
e1fae11509 Merge pull request #12392 from Apokleos/enhance-tdx
runtime-rs: Enhance TDX in qemu
2026-04-05 20:54:43 +02:00
Alex Lyn
35cafe8715 runtime-rs: configure TDX machine options with kernel_irqchip=split
When TDX confidential guest support is enabled, set `kernel_irqchip=split`
for TDX CVM:
...
-machine \
   q35,accel=kvm,kernel_irqchip=split,confidential-guest-support=tdx \
...

Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
2026-04-05 10:18:47 +02:00
Manuel Huber
57e42b10f1 tests: nvidia: Do not use elevated privileges
Do not run the NIM containers with elevated privileges. Note that,
using hostPath requires proper host folder permissions, and that
using emptyDir requires a proper fsGroup ID.
Once issue 11162 is resolved, we can further refine the securityContext
fields for the TEE manifests.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2026-04-01 10:23:26 -07:00
Manuel Huber
a762b136de tests: generate policy for pod-empty-dir-fsgroup
The logic in the k8s-empty-dirs.bats file missed to add a security
policy for the pod-empty-dir-fsgroup.yaml manifest. With this change,
we add the policy annotation.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2026-04-01 10:23:26 -07:00
Manuel Huber
43489f6d56 genpolicy: fs_group for encrypted emptyDir volumes
The shim uses Storage.fs_group on block/scsi encrypted emptyDir while
genpolicy used fsgid= in options and null fs_group, leading to
denying CreateContainerRequest when using block-encrypted emptyDir in
combination with fsGroup. Thus, emit fs_group in that scenario and keep
fsgid= for the existing shared-fs/local emptyDir behavior.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2026-04-01 10:23:26 -07:00
Manuel Huber
9923f251f5 genpolicy: adjust GID after passwd GID handling
After pod runAsUser triggers passwd-based GID resolution, genpolicy
clears AdditionalGids and inserts only the primary GID.
PodSecurityContext fsGroup and supplementalGroups get cleared, so
policy enforcement would deny CreateContainer when the runtime
includes those when specified.

This change applies fsGroup/supplementalGroups once in
get_container_process via apply_pod_fs_group_and_supplemental_groups.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2026-04-01 10:23:25 -07:00
Tobin Feldman-Fitzthum
8944058a5b versions: add nvat version
Keep track of which version of NVIDIA Attestation SDK to use when
building the attestation agent with NVIDIA support.

Signed-off-by: Tobin Feldman-Fitzthum <tfeldmanfitz@nvidia.com>
2026-03-31 21:15:03 +00:00
Paul Meyer
086c0ed18e runtime-rs: cleanup config
Some fields were misspelled, misplaced at an outdated path or copied
over from runtime-go but aren't supported in runtime-rs.
Cleaning them up to avoid confusion and ease migration.

Signed-off-by: Paul Meyer <katexochen0@gmail.com>
2026-03-31 10:51:25 +02:00
81 changed files with 1656 additions and 966 deletions

13
.dockerignore Normal file
View File

@@ -0,0 +1,13 @@
# Context for tools/packaging/kata-deploy/Dockerfile (build from repo root: -f tools/packaging/kata-deploy/Dockerfile .)
#
# The Dockerfile only needs: Cargo.toml, Cargo.lock, src/, tools/packaging/kata-deploy/,
# and versions.yaml. Exclude heavy or irrelevant trees to keep context small.
.git
.github
target
kata-artifacts
docs
tests
utils
tools/packaging/kata-deploy/local-build
tools/packaging/kata-deploy/binary/target

View File

@@ -14,7 +14,7 @@ runs:
using: "composite"
steps:
- name: Install Rust
uses: actions-rs/toolchain@v1
uses: actions-rs/toolchain@b2417cde72dcf67f306c0ae8e0828a81bf0b189f # v1.0.6
with:
profile: minimal
toolchain: nightly

View File

@@ -1,35 +0,0 @@
name: nydus-snapshotter-version-sync
on:
pull_request:
types:
- opened
- edited
- reopened
- synchronize
permissions: {}
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
nydus-snapshotter-version-check:
name: nydus-snapshotter-version-check
runs-on: ubuntu-22.04
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Ensure nydus-snapshotter-version is in sync inside our repo
run: |
dockerfile_version=$(grep "ARG NYDUS_SNAPSHOTTER_VERSION" tools/packaging/kata-deploy/Dockerfile | cut -f2 -d'=')
versions_version=$(yq ".externals.nydus-snapshotter.version | explode(.)" versions.yaml)
if [[ "${dockerfile_version}" != "${versions_version}" ]]; then
echo "nydus-snapshotter version must be the same in the following places: "
echo "- versions.yaml: ${versions_version}"
echo "- tools/packaging/kata-deploy/Dockerfile: ${dockerfile_version}"
exit 1
fi

478
Cargo.lock generated
View File

@@ -94,6 +94,12 @@ dependencies = [
"memchr",
]
[[package]]
name = "allocator-api2"
version = "0.2.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
[[package]]
name = "android_system_properties"
version = "0.1.5"
@@ -409,6 +415,28 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "async-stream"
version = "0.3.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b5a71a6f37880a80d1d7f19efd781e4b5de42c88f0722cc13bcb6cc2cfe8476"
dependencies = [
"async-stream-impl",
"futures-core",
"pin-project-lite",
]
[[package]]
name = "async-stream-impl"
version = "0.3.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c7c24de15d275a1ecfd47a380fb4d5ec9bfe0933f309ed5e705b775596a3574d"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "async-task"
version = "4.7.1"
@@ -511,6 +539,17 @@ dependencies = [
"tower-service",
]
[[package]]
name = "backon"
version = "1.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cffb0e931875b666fc4fcb20fee52e9bbd1ef836fd9e9e04ec21555f9f85f7ef"
dependencies = [
"fastrand 2.3.0",
"gloo-timers",
"tokio",
]
[[package]]
name = "backtrace"
version = "0.3.76"
@@ -1272,6 +1311,16 @@ dependencies = [
"darling_macro 0.20.11",
]
[[package]]
name = "darling"
version = "0.21.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9cdf337090841a411e2a7f3deb9187445851f91b309c0c0a29e05f74a00a48c0"
dependencies = [
"darling_core 0.21.3",
"darling_macro 0.21.3",
]
[[package]]
name = "darling_core"
version = "0.14.4"
@@ -1299,6 +1348,20 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "darling_core"
version = "0.21.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1247195ecd7e3c85f83c8d2a366e4210d588e802133e1e355180a9870b517ea4"
dependencies = [
"fnv",
"ident_case",
"proc-macro2",
"quote",
"strsim",
"syn 2.0.117",
]
[[package]]
name = "darling_macro"
version = "0.14.4"
@@ -1321,6 +1384,17 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "darling_macro"
version = "0.21.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d38308df82d1080de0afee5d069fa14b0326a88c14f15c5ccda35b4a6c414c81"
dependencies = [
"darling_core 0.21.3",
"quote",
"syn 2.0.117",
]
[[package]]
name = "dashmap"
version = "5.5.3"
@@ -1600,6 +1674,27 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "derive_more"
version = "2.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d751e9e49156b02b44f9c1815bcb94b984cdcc4396ecc32521c739452808b134"
dependencies = [
"derive_more-impl",
]
[[package]]
name = "derive_more-impl"
version = "2.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "799a97264921d8623a957f6c3b9011f3b5492f557bbb7a5a19b7fa6d06ba8dcb"
dependencies = [
"proc-macro2",
"quote",
"rustc_version",
"syn 2.0.117",
]
[[package]]
name = "device_tree"
version = "1.1.0"
@@ -1726,6 +1821,18 @@ version = "1.0.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d0881ea181b1df73ff77ffaaf9c7544ecc11e82fba9b5f27b262a3c73a332555"
[[package]]
name = "educe"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d7bc049e1bd8cdeb31b68bbd586a9464ecf9f3944af3958a7a9d0f8b9799417"
dependencies = [
"enum-ordinalize",
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "either"
version = "1.15.0"
@@ -1774,6 +1881,26 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "enum-ordinalize"
version = "4.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4a1091a7bb1f8f2c4b28f1fe2cef4980ca2d410a3d727d67ecc3178c9b0800f0"
dependencies = [
"enum-ordinalize-derive",
]
[[package]]
name = "enum-ordinalize-derive"
version = "4.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ca9601fb2d62598ee17836250842873a413586e5d7ed88b356e38ddbb0ec631"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "enumflags2"
version = "0.7.12"
@@ -2338,6 +2465,18 @@ version = "0.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0cc23270f6e1808e30a928bdc84dea0b9b4136a8bc82338574f23baf47bbd280"
[[package]]
name = "gloo-timers"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bbb143cf96099802033e0d4f4963b19fd2e0b728bcf076cd9cf7f6634f092994"
dependencies = [
"futures-channel",
"futures-core",
"js-sys",
"wasm-bindgen",
]
[[package]]
name = "go-flag"
version = "0.1.0"
@@ -2406,6 +2545,8 @@ version = "0.15.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1"
dependencies = [
"allocator-api2",
"equivalent",
"foldhash",
]
@@ -2506,6 +2647,17 @@ dependencies = [
"windows-sys 0.52.0",
]
[[package]]
name = "hostname"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "617aaa3557aef3810a6369d0a99fac8a080891b68bd9f9812a1eeda0c0730cbd"
dependencies = [
"cfg-if 1.0.4",
"libc",
"windows-link",
]
[[package]]
name = "http"
version = "0.2.12"
@@ -2643,7 +2795,9 @@ dependencies = [
"http 1.4.0",
"hyper 1.8.1",
"hyper-util",
"log",
"rustls",
"rustls-native-certs",
"rustls-pki-types",
"tokio",
"tokio-rustls",
@@ -2662,6 +2816,19 @@ dependencies = [
"tokio-io-timeout",
]
[[package]]
name = "hyper-timeout"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2b90d566bffbce6a75bd8b09a05aa8c2cb1fabb6cb348f8840c9e4c90a0d83b0"
dependencies = [
"hyper 1.8.1",
"hyper-util",
"pin-project-lite",
"tokio",
"tower-service",
]
[[package]]
name = "hyper-tls"
version = "0.6.0"
@@ -3127,6 +3294,19 @@ dependencies = [
"thiserror 1.0.69",
]
[[package]]
name = "jsonpath-rust"
version = "0.7.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c00ae348f9f8fd2d09f82a98ca381c60df9e0820d8d79fce43e649b4dc3128b"
dependencies = [
"pest",
"pest_derive",
"regex",
"serde_json",
"thiserror 2.0.18",
]
[[package]]
name = "jsonptr"
version = "0.4.7"
@@ -3201,6 +3381,18 @@ dependencies = [
"tonic-build 0.8.4",
]
[[package]]
name = "k8s-openapi"
version = "0.26.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "06d9e5e61dd037cdc51da0d7e2b2be10f497478ea7e120d85dad632adb99882b"
dependencies = [
"base64 0.22.1",
"chrono",
"serde",
"serde_json",
]
[[package]]
name = "kata-agent"
version = "0.1.0"
@@ -3285,6 +3477,28 @@ dependencies = [
"tokio",
]
[[package]]
name = "kata-deploy"
version = "0.1.0"
dependencies = [
"anyhow",
"clap",
"env_logger",
"k8s-openapi",
"kube",
"libc",
"log",
"regex",
"rstest",
"serde_json",
"serde_yaml 0.9.34+deprecated",
"serial_test 0.10.0",
"tempfile",
"tokio",
"toml_edit 0.22.27",
"walkdir",
]
[[package]]
name = "kata-sys-util"
version = "0.1.0"
@@ -3307,6 +3521,8 @@ dependencies = [
"slog",
"slog-scope",
"subprocess",
"tempfile",
"test-utils",
"thiserror 1.0.69",
]
@@ -3325,6 +3541,7 @@ dependencies = [
"num_cpus",
"oci-spec 0.8.4",
"regex",
"rstest",
"safe-path 0.1.0",
"serde",
"serde-enum-str",
@@ -3334,6 +3551,8 @@ dependencies = [
"slog-scope",
"sysctl",
"sysinfo",
"tempfile",
"test-utils",
"thiserror 1.0.69",
"toml",
]
@@ -3358,6 +3577,115 @@ dependencies = [
"libc",
]
[[package]]
name = "kube"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "48e7bb0b6a46502cc20e4575b6ff401af45cfea150b34ba272a3410b78aa014e"
dependencies = [
"k8s-openapi",
"kube-client",
"kube-core",
"kube-derive",
"kube-runtime",
]
[[package]]
name = "kube-client"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4987d57a184d2b5294fdad3d7fc7f278899469d21a4da39a8f6ca16426567a36"
dependencies = [
"base64 0.22.1",
"bytes 1.11.1",
"chrono",
"either",
"futures",
"home",
"http 1.4.0",
"http-body 1.0.1",
"http-body-util",
"hyper 1.8.1",
"hyper-rustls",
"hyper-timeout 0.5.2",
"hyper-util",
"jsonpath-rust",
"k8s-openapi",
"kube-core",
"pem",
"rustls",
"secrecy",
"serde",
"serde_json",
"serde_yaml 0.9.34+deprecated",
"thiserror 2.0.18",
"tokio",
"tokio-util",
"tower 0.5.3",
"tower-http",
"tracing",
]
[[package]]
name = "kube-core"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "914bbb770e7bb721a06e3538c0edd2babed46447d128f7c21caa68747060ee73"
dependencies = [
"chrono",
"derive_more",
"form_urlencoded",
"http 1.4.0",
"json-patch 4.1.0",
"k8s-openapi",
"schemars",
"serde",
"serde-value",
"serde_json",
"thiserror 2.0.18",
]
[[package]]
name = "kube-derive"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "03dee8252be137772a6ab3508b81cd797dee62ee771112a2453bc85cbbe150d2"
dependencies = [
"darling 0.21.3",
"proc-macro2",
"quote",
"serde",
"serde_json",
"syn 2.0.117",
]
[[package]]
name = "kube-runtime"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6aea4de4b562c5cc89ab10300bb63474ae1fa57ff5a19275f2e26401a323e3fd"
dependencies = [
"ahash 0.8.12",
"async-broadcast 0.7.2",
"async-stream",
"backon",
"educe",
"futures",
"hashbrown 0.15.5",
"hostname",
"json-patch 4.1.0",
"k8s-openapi",
"kube-client",
"parking_lot",
"pin-project",
"serde",
"serde_json",
"thiserror 2.0.18",
"tokio",
"tokio-util",
"tracing",
]
[[package]]
name = "kvm-bindings"
version = "0.14.0"
@@ -3539,6 +3867,7 @@ dependencies = [
"slog-json",
"slog-scope",
"slog-term",
"tempfile",
]
[[package]]
@@ -3584,11 +3913,16 @@ version = "0.2.0"
dependencies = [
"anyhow",
"chrono",
"lazy_static",
"maplit",
"nix 0.30.1",
"once_cell",
"page_size",
"slog",
"slog-async",
"slog-scope",
"slog-term",
"test-utils",
"tokio",
]
@@ -4493,6 +4827,15 @@ dependencies = [
"num-traits",
]
[[package]]
name = "ordered-float"
version = "2.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "68f19d67e5a2795c94e73e0bb1cc1a7edeb2e28efd39e2e1c9b7a40c1108b11c"
dependencies = [
"num-traits",
]
[[package]]
name = "ordered-multimap"
version = "0.4.3"
@@ -4602,6 +4945,16 @@ dependencies = [
"quote",
]
[[package]]
name = "pem"
version = "3.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d30c53c26bc5b31a98cd02d20f25a7c8567146caf63ed593a9d87b2775291be"
dependencies = [
"base64 0.22.1",
"serde_core",
]
[[package]]
name = "percent-encoding"
version = "2.3.2"
@@ -4621,6 +4974,49 @@ dependencies = [
"serde_json",
]
[[package]]
name = "pest"
version = "2.8.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e0848c601009d37dfa3430c4666e147e49cdcf1b92ecd3e63657d8a5f19da662"
dependencies = [
"memchr",
"ucd-trie",
]
[[package]]
name = "pest_derive"
version = "2.8.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "11f486f1ea21e6c10ed15d5a7c77165d0ee443402f0780849d1768e7d9d6fe77"
dependencies = [
"pest",
"pest_generator",
]
[[package]]
name = "pest_generator"
version = "2.8.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8040c4647b13b210a963c1ed407c1ff4fdfa01c31d6d2a098218702e6664f94f"
dependencies = [
"pest",
"pest_meta",
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "pest_meta"
version = "2.8.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "89815c69d36021a140146f26659a81d6c2afa33d216d736dd4be5381a7362220"
dependencies = [
"pest",
"sha2 0.10.9",
]
[[package]]
name = "petgraph"
version = "0.5.1"
@@ -5974,7 +6370,9 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "758025cb5fccfd3bc2fd74708fd4682be41d99e5dff73c377c0646c6012c73a4"
dependencies = [
"aws-lc-rs",
"log",
"once_cell",
"ring",
"rustls-pki-types",
"rustls-webpki",
"subtle",
@@ -6073,6 +6471,7 @@ name = "safe-path"
version = "0.1.0"
dependencies = [
"libc",
"tempfile",
]
[[package]]
@@ -6119,10 +6518,23 @@ checksum = "a2b42f36aa1cd011945615b92222f6bf73c599a102a300334cd7f8dbeec726cc"
dependencies = [
"dyn-clone",
"ref-cast",
"schemars_derive",
"serde",
"serde_json",
]
[[package]]
name = "schemars_derive"
version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7d115b50f4aaeea07e79c1912f645c7513d81715d0420f8bc77a18c6260b307f"
dependencies = [
"proc-macro2",
"quote",
"serde_derive_internals",
"syn 2.0.117",
]
[[package]]
name = "scientific"
version = "0.5.3"
@@ -6164,6 +6576,15 @@ dependencies = [
"libc",
]
[[package]]
name = "secrecy"
version = "0.10.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e891af845473308773346dc847b2c23ee78fe442e0472ac50e22a18a93d3ae5a"
dependencies = [
"zeroize",
]
[[package]]
name = "security-framework"
version = "3.7.0"
@@ -6243,6 +6664,16 @@ version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8a059d895f1a31dd928f40abbea4e7177e3d8ff3aa4152fdb7a396ae1ef63a3"
[[package]]
name = "serde-value"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f3a1a3341211875ef120e117ea7fd5228530ae7e7036a779fdc9117be6b3282c"
dependencies = [
"ordered-float 2.10.1",
"serde",
]
[[package]]
name = "serde_core"
version = "1.0.228"
@@ -6263,6 +6694,17 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "serde_derive_internals"
version = "0.29.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "18d26a20a969b9e3fdf2fc2d9f21eda6c40e2de84c9408bb5d3b05d499aae711"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "serde_ignored"
version = "0.1.14"
@@ -6496,6 +6938,8 @@ dependencies = [
"kata-sys-util",
"kata-types",
"nix 0.26.4",
"tempfile",
"test-utils",
"tokio",
]
@@ -7015,7 +7459,7 @@ dependencies = [
"byteorder",
"integer-encoding",
"log",
"ordered-float",
"ordered-float 1.1.1",
"threadpool",
]
@@ -7163,6 +7607,7 @@ dependencies = [
"futures-core",
"futures-sink",
"pin-project-lite",
"slab",
"tokio",
]
@@ -7227,6 +7672,18 @@ dependencies = [
"winnow 0.5.40",
]
[[package]]
name = "toml_edit"
version = "0.22.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41fe8c660ae4257887cf66394862d21dbca4a6ddd26f04a3560410406a2f819a"
dependencies = [
"indexmap 2.13.0",
"toml_datetime 0.6.11",
"toml_write",
"winnow 0.7.15",
]
[[package]]
name = "toml_edit"
version = "0.25.4+spec-1.1.0"
@@ -7248,6 +7705,12 @@ dependencies = [
"winnow 0.7.15",
]
[[package]]
name = "toml_write"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d99f8c9a7727884afe522e9bd5edbfc91a3312b36a77b5fb8926e4c31a41801"
[[package]]
name = "tonic"
version = "0.9.2"
@@ -7264,7 +7727,7 @@ dependencies = [
"http 0.2.12",
"http-body 0.4.6",
"hyper 0.14.32",
"hyper-timeout",
"hyper-timeout 0.4.1",
"percent-encoding",
"pin-project",
"prost 0.11.9",
@@ -7333,8 +7796,10 @@ dependencies = [
"pin-project-lite",
"sync_wrapper 1.0.2",
"tokio",
"tokio-util",
"tower-layer",
"tower-service",
"tracing",
]
[[package]]
@@ -7343,16 +7808,19 @@ version = "0.6.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d4e6559d53cc268e5031cd8429d05415bc4cb4aefc4aa5d6cc35fbf5b924a1f8"
dependencies = [
"base64 0.22.1",
"bitflags 2.11.0",
"bytes 1.11.1",
"futures-util",
"http 1.4.0",
"http-body 1.0.1",
"iri-string",
"mime",
"pin-project-lite",
"tower 0.5.3",
"tower-layer",
"tower-service",
"tracing",
]
[[package]]
@@ -7525,6 +7993,12 @@ version = "1.19.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "562d481066bde0658276a35467c4af00bdc6ee726305698a55b86e61d7ad82bb"
[[package]]
name = "ucd-trie"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2896d95c02a80c6d6a5d6e953d479f5ddf2dfdb6a244441010e373ac0fb88971"
[[package]]
name = "uds_windows"
version = "1.2.0"

View File

@@ -6,6 +6,17 @@ rust-version = "1.88"
[workspace]
members = [
# libs
"src/libs/kata-sys-util",
"src/libs/kata-types",
"src/libs/logging",
"src/libs/mem-agent",
"src/libs/protocols",
"src/libs/runtime-spec",
"src/libs/safe-path",
"src/libs/shim-interface",
"src/libs/test-utils",
# kata-agent
"src/agent",
"src/agent/rustjail",
@@ -31,6 +42,9 @@ members = [
# genpolicy
"src/tools/genpolicy",
# kata-deploy (Kubernetes installer binary)
"tools/packaging/kata-deploy/binary",
# runtime-rs
"src/runtime-rs",
"src/runtime-rs/crates/agent",
@@ -48,10 +62,6 @@ resolver = "2"
# TODO: Add all excluded crates to root workspace
exclude = [
"src/tools",
"src/libs",
# kata-deploy binary is standalone and has its own Cargo.toml for now
"tools/packaging/kata-deploy/binary",
# We are cloning and building rust packages under
# "tools/packaging/kata-deploy/local-build/build" folder, which may mislead
@@ -174,7 +184,7 @@ thiserror = "1.0.26"
tokio = "1.46.1"
tokio-vsock = "0.3.4"
toml = "0.5.8"
tracing = "0.1.41"
tracing = "0.1.44"
tracing-opentelemetry = "0.18.0"
tracing-subscriber = "0.3.20"
ttrpc = "0.8.4"

View File

@@ -213,12 +213,10 @@ API and kernel drivers, interacting with the pass-through GPU device.
An additional step is exercised in our CI samples: when using images from an
authenticated registry, the guest-pull mechanism triggers attestation using
Trustee's Key Broker Service (KBS) for secure release of the NGC API
authentication key used to access the NVCR container registry. As part of
this, the attestation agent exercises composite attestation and transitions
the GPU into `Ready` state (without this, the GPU has to explicitly be
transitioned into `Ready` state by passing the `nvrc.smi.srs=1` kernel
parameter via the shim config, causing NVRC to transition the GPU into the
`Ready` state).
authentication key used to access the NVCR container registry.
In this flow the CPU and all additional devices are attested.
GPUs will automatically be set to ready by NVRC per the NVRC configuration
flag in the default kernel command line.
## Deployment Guidance
@@ -227,7 +225,7 @@ Kata's non-TEE and TEE GPU workload deployment scenarios for your Kubernetes
nodes. We provide guidance based on the upstream Kata CI procedures for the
NVIDIA GPU CI validation jobs. Note that, this setup:
- uses the guest image pull method to pull container image layers
- uses the nydus snapshotter to pull container image layers in the guest
- uses the genpolicy tool to attach Kata agent security policies to the pod
manifest
- has dedicated (composite) attestation tests, a CUDA vectorAdd test, and a
@@ -250,6 +248,17 @@ Service NRAS
- container image signature verification and encrypted container images
- ephemeral container data and image layer storage
For the use of these features, we refer to separate documentation in the
kata-containers and confidential-containers documentation resources.
For example, see a
[list of features](https://confidentialcontainers.org/docs/features/) along
with their documentation in the confidential-containers documentation.
> **Note:**
>
> Image signature verification for signed multi-arch images is currently not
> supported.
### Requirements
The requirements for the TEE scenario are:
@@ -272,8 +281,8 @@ selecting proper hardware and on properly configuring its firmware and OS.
#### Containerd and Kubernetes
First, set up your Kubernetes cluster. For instance, in Kata CI, our NVIDIA
jobs use a single-node vanilla Kubernetes cluster with a 2.1 containerd
version and Kata's current supported Kubernetes version. This cluster is
jobs use a single-node vanilla Kubernetes cluster with containerd v2.2
and Kata's current supported Kubernetes version. This cluster is
being set up using the `deploy_k8s` function from the script file
`tests/integration/kubernetes/gha-run.sh`. If you intend to run this script,
follow these steps, and make sure you have `yq` and `helm` installed. Note
@@ -284,7 +293,7 @@ You can execute the function as follows:
$ export GH_TOKEN="<your-gh-pat>"
$ export KUBERNETES="vanilla"
$ export CONTAINER_ENGINE="containerd"
$ export CONTAINER_ENGINE_VERSION="v2.1"
$ export CONTAINER_ENGINE_VERSION="v2.2"
$ source tests/gha-run-k8s-common.sh
$ deploy_k8s
```
@@ -300,6 +309,13 @@ $ deploy_k8s
> `create_container_timeout` of 1200s, which is the equivalent value on shim
> side, controlling the time the shim allows for a container to remain in
> *container creating* state.
> If you need a timeout of more than 1200s, you will also need to adjust the
> agent's `image_pull_timeout`, which in turn sets the confidential data
> hub's image pull API timeout in seconds. For this, add the
> `agent.image_pull_timeout=<seconds>` kernel parameter to your shim
> configuration's `kernel_params` field, or pass the parameter explicitly
> via the `io.katacontainers.config.hypervisor.kernel_params: "..."` pod
> annotation. The default value for this timeout is 1200s.
> **Note:**
>
@@ -356,7 +372,7 @@ $ helm install --wait --generate-name \
Install the latest Kata Containers helm chart, similar to
[existing documentation](https://github.com/kata-containers/kata-containers/blob/main/tools/packaging/kata-deploy/helm-chart/README.md)
(minimum version: `3.24.0`).
(minimum version: `3.29.0`).
```bash
$ export VERSION=$(curl -sSL https://api.github.com/repos/kata-containers/kata-containers/releases/latest | jq .tag_name | tr -d '"')
@@ -371,6 +387,13 @@ $ helm install kata-deploy \
"${CHART}" --version "${VERSION}"
```
> **Note:**
>
> For node lifecycle management, see the
> [lifecycle-manager](https://github.com/kata-containers/lifecycle-manager)
> repository which enables Argo Workflows-based lifecycle management for your
> node's Kata deployments.
#### Trustee's KBS for remote attestation
For our Kata CI runners we use Trustee's KBS for composite attestation for
@@ -440,8 +463,6 @@ kind: Pod
metadata:
name: cuda-vectoradd-kata
namespace: default
annotations:
io.katacontainers.config.hypervisor.kernel_params: "nvrc.smi.srs=1"
spec:
runtimeClassName: ${GPU_RUNTIME_CLASS_NAME}
restartPolicy: Never
@@ -566,21 +587,21 @@ With GPU passthrough being supported by the
you can use the tool to create a Kata agent security policy. Our CI deploys
all sample pod manifests with a Kata agent security policy.
Note that, using containerd 2.1 in upstream's CI, we use the following
modification to the genpolicy default settings:
Note that, in Kata CI, we use snippets such as the following to modify the
genpolicy default settings:
```bash
[
{
"op": "replace",
"path": "/kata_config/oci_version",
"value": "1.2.1"
"value": "1.3.0"
}
]
```
This modification is applied via the genpolicy drop-in configuration file
`src\tools\genpolicy\drop-in-examples\20-oci-1.2.1-drop-in.json`.
When using a newer containerd version, such as containerd 2.2, the OCI
version field needs to be adjusted to "1.3.0", for instance.
`src/tools/genpolicy/drop-in-examples/20-oci-1.3.0-drop-in.json`.
When using a newer (or older) containerd version, the OCI version field
may need to be adjusted accordingly.
#### Deploy pods using your own containers and manifests
@@ -588,12 +609,7 @@ You can author pod manifests leveraging your own containers, for instance,
containers built using the CUDA container toolkit. We recommend to start
with a CUDA base container.
The GPU is transitioned into the `Ready` state via attestation, for instance,
when pulling authenticated images. If your deployment scenario does not use
attestation, please refer back to the CUDA vectorAdd pod manifest. In this
manifest, we ensure that NVRC sets the GPU to `Ready` state by adding the
following annotation in the manifest:
`io.katacontainers.config.hypervisor.kernel_params: "nvrc.smi.srs=1"`
When using the GPU runtime classes, the GPUs will automatically be set to ready.
> **Notes:**
>

View File

@@ -1,13 +0,0 @@
[workspace]
members = [
"kata-sys-util",
"kata-types",
"logging",
"mem-agent",
"protocols",
"runtime-spec",
"safe-path",
"shim-interface",
"test-utils",
]
resolver = "2"

View File

@@ -11,6 +11,17 @@ ifeq ($(USERID), 0)
override EXTRA_TEST_FLAGS = --ignored
endif
LIBS := \
-p kata-sys-util \
-p kata-types \
-p logging \
-p mem-agent \
-p protocols \
-p runtime-spec \
-p safe-path \
-p shim-interface \
-p test-utils
default: build
build:
@@ -23,13 +34,13 @@ check: clippy format
clippy:
@echo "INFO: cargo clippy..."
cargo clippy --all-targets --all-features --release \
cargo clippy $(LIBS) --all-features --release \
-- \
-D warnings
format:
@echo "INFO: cargo fmt..."
cargo fmt -- --check
cargo fmt $(LIBS) -- --check
clean:
cargo clean
@@ -38,8 +49,8 @@ clean:
# See the `test_logger_levels()` test for further information.
test:
@echo "INFO: testing libraries for development build"
cargo test --all $(EXTRA_RUSTFEATURES) -- --nocapture $(EXTRA_TEST_FLAGS)
cargo test $(LIBS) $(EXTRA_RUSTFEATURES) -- --nocapture $(EXTRA_TEST_FLAGS)
@echo "INFO: testing libraries for release build"
cargo test --release --all $(EXTRA_RUSTFEATURES) -- --nocapture $(EXTRA_TEST_FLAGS)
cargo test --release $(LIBS) $(EXTRA_RUSTFEATURES) -- --nocapture $(EXTRA_TEST_FLAGS)
.PHONY: install vendor

View File

@@ -19,6 +19,7 @@ use super::default::{
pub const AGENT_NAME_KATA: &str = "kata";
#[derive(Default, Debug, Deserialize, Serialize, Clone)]
#[serde(deny_unknown_fields)]
pub struct MemAgent {
#[serde(default, alias = "mem_agent_enable")]
pub enable: bool,
@@ -58,6 +59,7 @@ pub struct MemAgent {
/// Kata agent configuration information.
#[derive(Debug, Deserialize, Serialize, Clone)]
#[serde(deny_unknown_fields)]
pub struct Agent {
/// If enabled, the agent will log additional debug messages to the system log.
#[serde(default, rename = "enable_debug")]

View File

@@ -708,10 +708,10 @@ pub struct DebugInfo {
///
/// Example usage in configuration:
/// ```toml
/// dbg_monitor_socket = "hmp"
/// extra_monitor_socket = "hmp"
/// ```
#[serde(default)]
pub dbg_monitor_socket: String,
#[serde(default, alias = "dbg_monitor_socket")]
pub extra_monitor_socket: String,
}
impl DebugInfo {
@@ -1238,6 +1238,7 @@ impl NetworkInfo {
/// Configuration information for rootless user.
#[derive(Clone, Debug, Default, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct RootlessUser {
/// The UID of the rootless user.
#[serde(default)]
@@ -1643,6 +1644,7 @@ impl VmTemplateInfo {
/// Configuration information for VM factory (templating, caches, etc.).
#[derive(Clone, Debug, Default, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct Factory {
/// Enable VM templating support.
/// When enabled, new VMs may be created from a template to speed up creation.

View File

@@ -8,6 +8,7 @@ use std::io::Result;
use regex::Regex;
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
#[serde(deny_unknown_fields)]
pub struct SharedMount {
/// Name is used to identify a pair of shared mount points.
/// This field cannot be omitted.
@@ -143,7 +144,6 @@ mod tests {
shared_mount_annotation: r#"
{
"name": "test",
"src": "sidecar",
"src_path": "/mnt/storage",
"dst_ctr": "app",
"dst_path": "/mnt/storage"
@@ -156,7 +156,6 @@ mod tests {
{
"name": "test",
"src_ctr": "sidecar",
"src_dir": "/mnt/storage",
"dst_ctr": "app",
"dst_path": "/mnt/storage"
}"#,
@@ -169,7 +168,6 @@ mod tests {
"name": "test",
"src_ctr": "sidecar",
"src_path": "/mnt/storage",
"dst_container": "app",
"dst_path": "/mnt/storage"
}"#,
result: false,
@@ -181,8 +179,7 @@ mod tests {
"name": "test",
"src_ctr": "sidecar",
"src_path": "/mnt/storage",
"dst_ctr": "app",
"path": "/mnt/storage"
"dst_ctr": "app"
}"#,
result: false,
message: "shared_mount: field 'dst_path' couldn't be empty.",

View File

@@ -65,8 +65,8 @@ enable_guest_swap = true
[agent.agent0]
enable_tracing = true
debug_console_enabled = true
debug = true
dial_timeout = 1
enable_debug = true
dial_timeout_ms = 1000
kernel_modules = ["e1000e InterruptThrottleRate=3000,3000,3000 EEE=1","i915_enabled_ppgtt=0"]
container_pipe_size = 2
[runtime]

View File

@@ -64,8 +64,8 @@ enable_guest_swap = true
[agent.agent0]
enable_tracing = true
debug_console_enabled = true
debug = true
dial_timeout = 1
enable_debug = true
dial_timeout_ms = 1000
kernel_modules = ["e1000e InterruptThrottleRate=3000,3000,3000 EEE=1","i915_enabled_ppgtt=0"]
container_pipe_size = 2
[runtime]

View File

@@ -137,16 +137,12 @@ ifeq ($(ARCH), aarch64)
EDK2_NAME := aavmf
endif
# Set firmware paths from QEMUFW/QEMUFWVOL if defined
# Set firmware path from QEMUFW if defined
FIRMWAREPATH :=
FIRMWAREVOLUMEPATH :=
ifneq (,$(QEMUCMD))
ifneq (,$(QEMUFW))
FIRMWAREPATH := $(PREFIXDEPS)/share/$(EDK2_NAME)/$(QEMUFW)
endif
ifneq (,$(QEMUFWVOL))
FIRMWAREVOLUMEPATH := $(PREFIXDEPS)/share/$(EDK2_NAME)/$(QEMUFWVOL)
endif
endif
KERNELVERITYPARAMS ?= ""
@@ -157,7 +153,6 @@ FIRMWARETDVFPATH := $(PREFIXDEPS)/share/ovmf/OVMF.inteltdx.fd
# SEV-SNP
FIRMWARE_SNP_PATH := $(PREFIXDEPS)/share/ovmf/AMDSEV.fd
FIRMWARE_VOLUME_SNP_PATH :=
##VAR DEFVCPUS=<number> Default number of vCPUs
DEFVCPUS := 1
@@ -204,7 +199,6 @@ DEFVIRTIOFSQUEUESIZE ?= 1024
# Make sure you quote args.
DEFVIRTIOFSEXTRAARGS ?= [\"--thread-pool-size=1\", \"-o\", \"announce_submounts\"]
DEFENABLEIOTHREADS := false
DEFINDEPIOTHREADS := 0
DEFENABLEVHOSTUSERSTORE := false
DEFVHOSTUSERSTOREPATH := $(PKGRUNDIR)/vhost-user
DEFVALIDVHOSTUSERSTOREPATHS := [\"$(DEFVHOSTUSERSTOREPATH)\"]
@@ -222,7 +216,6 @@ DEFCREATECONTAINERTIMEOUT ?= 30
DEFCREATECONTAINERTIMEOUT_COCO ?= 60
DEFSTATICRESOURCEMGMT_COCO = true
DEFDISABLEIMAGENVDIMM ?= false
DEFPODRESOURCEAPISOCK := ""
SED = sed
CLI_DIR = cmd
@@ -391,11 +384,19 @@ ifneq (,$(QEMUCMD))
ifeq ($(ARCH), s390x)
VMROOTFSDRIVER_QEMU := virtio-blk-ccw
DEFBLOCKSTORAGEDRIVER_QEMU := virtio-blk-ccw
# runtime-rs memory hotplug hard-codes pc-dimm, which is x86-only.
# s390x (s390-ccw-virtio) requires virtio-mem-ccw instead. Use static
# sandbox resource management to size the VM upfront and bypass hotplug.
DEFSTATICRESOURCEMGMT_QEMU := true
else ifeq ($(ARCH), aarch64)
# NVDIMM/virtio-pmem has issues on arm64 (cache coherency problems with DAX),
# so we use virtio-blk-pci instead.
VMROOTFSDRIVER_QEMU := virtio-blk-pci
DEFBLOCKSTORAGEDRIVER_QEMU := virtio-scsi
# runtime-rs lacks ARM CPU hotplug support (pflash/UEFI, SMP topology,
# QMP vCPU add for virt machine type). Use static sandbox resource
# management to size the VM upfront instead.
DEFSTATICRESOURCEMGMT_QEMU := true
else
VMROOTFSDRIVER_QEMU := virtio-pmem
DEFBLOCKSTORAGEDRIVER_QEMU := virtio-scsi
@@ -412,9 +413,6 @@ endif
# Most users will want to set this to "on,obsolete=deny,spawn=deny,resourcecontrol=deny"
# for better security. Note: "elevateprivileges=deny" doesn't work with daemonize option.
DEFSECCOMPSANDBOXPARAM := ""
# Default is empty string "" to match Rust default None (when commented out in config).
# Most users will want to set this to "system_u:system_r:container_t" for SELinux support.
DEFGUESTSELINUXLABEL := ""
endif
ifneq (,$(FCCMD))
@@ -524,7 +522,6 @@ USER_VARS += KERNELPATH_COCO
USER_VARS += KERNELPATH
USER_VARS += KERNELVIRTIOFSPATH
USER_VARS += FIRMWAREPATH
USER_VARS += FIRMWAREVOLUMEPATH
USER_VARS += MACHINEACCELERATORS
USER_VARS += CPUFEATURES
USER_VARS += DEFMACHINETYPE_CLH
@@ -584,9 +581,7 @@ USER_VARS += DEFVIRTIOFSEXTRAARGS
USER_VARS += DEFENABLEANNOTATIONS
USER_VARS += DEFENABLEANNOTATIONS_COCO
USER_VARS += DEFENABLEIOTHREADS
USER_VARS += DEFINDEPIOTHREADS
USER_VARS += DEFSECCOMPSANDBOXPARAM
USER_VARS += DEFGUESTSELINUXLABEL
USER_VARS += DEFENABLEVHOSTUSERSTORE
USER_VARS += DEFVHOSTUSERSTOREPATH
USER_VARS += DEFVALIDVHOSTUSERSTOREPATHS
@@ -628,11 +623,9 @@ USER_VARS += DEFCREATECONTAINERTIMEOUT
USER_VARS += DEFCREATECONTAINERTIMEOUT_COCO
USER_VARS += QEMUTDXEXPERIMENTALCMD
USER_VARS += FIRMWARE_SNP_PATH
USER_VARS += FIRMWARE_VOLUME_SNP_PATH
USER_VARS += KERNELTDXPARAMS
USER_VARS += DEFSHAREDFS_QEMU_TDX_VIRTIOFS
USER_VARS += FIRMWARETDVFPATH
USER_VARS += DEFPODRESOURCEAPISOCK
SOURCES := \
$(shell find . 2>&1 | grep -E '.*\.rs$$') \

View File

@@ -13,7 +13,6 @@ CPUFEATURES := pmu=off
QEMUCMD := qemu-system-aarch64
QEMUFW := AAVMF_CODE.fd
QEMUFWVOL := AAVMF_VARS.fd
# dragonball binary name
DBCMD := dragonball

View File

@@ -311,6 +311,19 @@ dial_timeout_ms = 10
# (default: 3000)
reconnect_timeout_ms = 3000
# Create Container Request Timeout
# This timeout value is used to set the maximum duration for the agent to process a CreateContainerRequest.
# It's also used to ensure that workloads, especially those involving large image pulls within the guest,
# have sufficient time to complete.
#
# Effective Timeout Determination:
# The effective timeout for a CreateContainerRequest is determined by taking the minimum of the following two values:
# - create_container_timeout: The timeout value configured for creating containers (default: 30 seconds).
# - runtime-request-timeout: The timeout value specified in the Kubelet configuration described as the link below:
# (https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/#:~:text=runtime%2Drequest%2Dtimeout)
# Defaults to @DEFCREATECONTAINERTIMEOUT@ second(s)
create_container_timeout = @DEFCREATECONTAINERTIMEOUT@
[agent.@PROJECT_TYPE@.mem_agent]
# Control the mem-agent function enable or disable.
# Default to false
@@ -409,19 +422,6 @@ compact_threshold = 1024
# Using 9223372036854775807 (i64::MAX) which is effectively "never" for practical purposes
compact_force_times = 9223372036854775807
# Create Container Request Timeout
# This timeout value is used to set the maximum duration for the agent to process a CreateContainerRequest.
# It's also used to ensure that workloads, especially those involving large image pulls within the guest,
# have sufficient time to complete.
#
# Effective Timeout Determination:
# The effective timeout for a CreateContainerRequest is determined by taking the minimum of the following two values:
# - create_container_timeout: The timeout value configured for creating containers (default: 30 seconds).
# - runtime-request-timeout: The timeout value specified in the Kubelet configuration described as the link below:
# (https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/#:~:text=runtime%2Drequest%2Dtimeout)
# Defaults to @DEFCREATECONTAINERTIMEOUT@ second(s)
create_container_timeout = @DEFCREATECONTAINERTIMEOUT@
[runtime]
# If enabled, the runtime will log additional debug messages to the
# system log

View File

@@ -76,12 +76,6 @@ kernel_params = "@KERNELPARAMS@"
# If you want that qemu uses the default firmware leave this option empty
firmware = "@FIRMWAREPATH@"
# Path to the firmware volume.
# firmware TDVF or OVMF can be split into FIRMWARE_VARS.fd (UEFI variables
# as configuration) and FIRMWARE_CODE.fd (UEFI program image). UEFI variables
# can be customized per each user while UEFI code is kept same.
firmware_volume = "@FIRMWAREVOLUMEPATH@"
# Machine accelerators
# comma-separated list of machine accelerators to pass to the hypervisor.
# For example, `machine_accelerators = "nosmm,nosmbus,nosata,nopit,static-prt,nofw"`
@@ -89,12 +83,12 @@ machine_accelerators = "@MACHINEACCELERATORS@"
# Qemu seccomp sandbox feature
# comma-separated list of seccomp sandbox features to control the syscall access.
# For example, `seccompsandbox= "on,obsolete=deny,spawn=deny,resourcecontrol=deny"`
# For example, `seccomp_sandbox = "on,obsolete=deny,spawn=deny,resourcecontrol=deny"`
# Note: "elevateprivileges=deny" doesn't work with daemonize option, so it's removed from the seccomp sandbox
# Another note: enabling this feature may reduce performance, you may enable
# /proc/sys/net/core/bpf_jit_enable to reduce the impact. see https://man7.org/linux/man-pages/man8/bpfc.8.html
# Recommended value when enabling: "on,obsolete=deny,spawn=deny,resourcecontrol=deny"
seccompsandbox = "@DEFSECCOMPSANDBOXPARAM@"
seccomp_sandbox = "@DEFSECCOMPSANDBOXPARAM@"
# CPU features
# comma-separated list of cpu features to pass to the cpu
@@ -311,11 +305,6 @@ enable_iommu_platform = false
# Your distribution recommends: @DEFVALIDVHOSTUSERSTOREPATHS@
valid_vhost_user_store_paths = @DEFVALIDVHOSTUSERSTOREPATHS@
# The timeout for reconnecting on non-server spdk sockets when the remote end goes away.
# qemu will delay this many seconds and then attempt to reconnect.
# Zero disables reconnecting, and the default is zero.
vhost_user_reconnect_timeout_sec = 0
# Enable file based guest memory support. The default is an empty string which
# will disable this feature. In the case of virtio-fs, this is enabled
# automatically and '/dev/shm' is used as the backing folder.
@@ -378,18 +367,6 @@ disable_image_nvdimm = false
# Default false
hotplug_vfio_on_root_bus = false
# Enable hot-plugging of VFIO devices to a bridge-port,
# root-port or switch-port.
# The default setting is "no-port"
hot_plug_vfio = "no-port"
# In a confidential compute environment hot-plugging can compromise
# security.
# Enable cold-plugging of VFIO devices to a bridge-port,
# root-port or switch-port.
# The default setting is "no-port", which means disabled.
cold_plug_vfio = "no-port"
# Before hot plugging a PCIe device, you need to add a pcie_root_port device.
# Use this parameter when using some large PCI bar devices, such as Nvidia GPU
# The value means the number of pcie_root_port
@@ -483,9 +460,6 @@ guest_memory_dump_path = ""
# See: https://www.qemu.org/docs/master/qemu-qmp-ref.html#Dump-guest-memory for details
#guest_memory_dump_paging=false
# use legacy serial for guest console if available and implemented for architecture. Default false
use_legacy_serial = false
# disable applying SELinux on the VMM process (default false)
disable_selinux = @DEFDISABLESELINUX@
@@ -497,7 +471,7 @@ disable_selinux = @DEFDISABLESELINUX@
disable_guest_selinux = @DEFDISABLEGUESTSELINUX@
[factory]
[hypervisor.qemu.factory]
# VM templating support. Once enabled, new VMs are created from template
# using vm cloning. They will share the same initial kernel, initramfs and
# agent memory by mapping it readonly. It helps speeding up new container
@@ -578,8 +552,8 @@ reconnect_timeout_ms = 3000
# - create_container_timeout: The timeout value configured for creating containers (default: 30 seconds).
# - runtime-request-timeout: The timeout value specified in the Kubelet configuration described as the link below:
# (https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/#:~:text=runtime%2Drequest%2Dtimeout)
# Defaults to @DEFCREATECONTAINERTIMEOUT@ second(s)
create_container_timeout = @DEFCREATECONTAINERTIMEOUT@
# Defaults to @DEFCREATECONTAINERTIMEOUT_COCO@ second(s)
create_container_timeout = @DEFCREATECONTAINERTIMEOUT_COCO@
[agent.@PROJECT_TYPE@.mem_agent]
# Control the mem-agent function enable or disable.
@@ -677,19 +651,6 @@ compact_threshold = 1024
# Default to 9223372036854775807
compact_force_times = 9223372036854775807
# Create Container Request Timeout
# This timeout value is used to set the maximum duration for the agent to process a CreateContainerRequest.
# It's also used to ensure that workloads, especially those involving large image pulls within the guest,
# have sufficient time to complete.
#
# Effective Timeout Determination:
# The effective timeout for a CreateContainerRequest is determined by taking the minimum of the following two values:
# - create_container_timeout: The timeout value configured for creating containers (default: @DEFCREATECONTAINERTIMEOUT_COCO@ seconds).
# - runtime-request-timeout: The timeout value specified in the Kubelet configuration described as the link below:
# (https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/#:~:text=runtime%2Drequest%2Dtimeout)
# Defaults to @DEFCREATECONTAINERTIMEOUT_COCO@ second(s)
create_container_timeout = @DEFCREATECONTAINERTIMEOUT_COCO@
[runtime]
# If enabled, the runtime will log additional debug messages to the
# system log
@@ -726,20 +687,6 @@ agent_name = "@PROJECT_TYPE@"
# (default: true)
disable_guest_seccomp = @DEFDISABLEGUESTSECCOMP@
# vCPUs pinning settings
# if enabled, each vCPU thread will be scheduled to a fixed CPU
# qualified condition: num(vCPU threads) == num(CPUs in sandbox's CPUSet)
enable_vcpus_pinning = false
# Apply a custom SELinux security policy to the container process inside the VM.
# This is used when you want to apply a type other than the default `container_t`,
# so general users should not uncomment and apply it.
# (format: "user:role:type")
# Note: You cannot specify MCS policy with the label because the sensitivity levels and
# categories are determined automatically by high-level container runtimes such as containerd.
# Example value when enabling: "system_u:system_r:container_t"
guest_selinux_label = "@DEFGUESTSELINUXLABEL@"
# If enabled, the runtime will create opentracing.io traces and spans.
# (See https://www.jaegertracing.io/docs/getting-started).
# (default: disabled)

View File

@@ -60,12 +60,6 @@ kernel_params = "@KERNELPARAMS@"
# If you want that qemu uses the default firmware leave this option empty
firmware = "@FIRMWAREPATH@"
# Path to the firmware volume.
# firmware TDVF or OVMF can be split into FIRMWARE_VARS.fd (UEFI variables
# as configuration) and FIRMWARE_CODE.fd (UEFI program image). UEFI variables
# can be customized per each user while UEFI code is kept same.
firmware_volume = "@FIRMWAREVOLUMEPATH@"
# Machine accelerators
# comma-separated list of machine accelerators to pass to the hypervisor.
# For example, `machine_accelerators = "nosmm,nosmbus,nosata,nopit,static-prt,nofw"`
@@ -73,12 +67,12 @@ machine_accelerators = "@MACHINEACCELERATORS@"
# Qemu seccomp sandbox feature
# comma-separated list of seccomp sandbox features to control the syscall access.
# For example, `seccompsandbox= "on,obsolete=deny,spawn=deny,resourcecontrol=deny"`
# For example, `seccomp_sandbox = "on,obsolete=deny,spawn=deny,resourcecontrol=deny"`
# Note: "elevateprivileges=deny" doesn't work with daemonize option, so it's removed from the seccomp sandbox
# Another note: enabling this feature may reduce performance, you may enable
# /proc/sys/net/core/bpf_jit_enable to reduce the impact. see https://man7.org/linux/man-pages/man8/bpfc.8.html
# Recommended value when enabling: "on,obsolete=deny,spawn=deny,resourcecontrol=deny"
seccompsandbox = "@DEFSECCOMPSANDBOXPARAM@"
seccomp_sandbox = "@DEFSECCOMPSANDBOXPARAM@"
# CPU features
# comma-separated list of cpu features to pass to the cpu
@@ -307,11 +301,6 @@ enable_iommu_platform = false
# Your distribution recommends: @DEFVALIDVHOSTUSERSTOREPATHS@
valid_vhost_user_store_paths = @DEFVALIDVHOSTUSERSTOREPATHS@
# The timeout for reconnecting on non-server spdk sockets when the remote end goes away.
# qemu will delay this many seconds and then attempt to reconnect.
# Zero disables reconnecting, and the default is zero.
vhost_user_reconnect_timeout_sec = 0
# Enable file based guest memory support. The default is an empty string which
# will disable this feature. In the case of virtio-fs, this is enabled
# automatically and '/dev/shm' is used as the backing folder.
@@ -373,18 +362,6 @@ disable_image_nvdimm = false
# Default false
hotplug_vfio_on_root_bus = false
# Enable hot-plugging of VFIO devices to a bridge-port,
# root-port or switch-port.
# The default setting is "no-port"
hot_plug_vfio = "no-port"
# In a confidential compute environment hot-plugging can compromise
# security.
# Enable cold-plugging of VFIO devices to a bridge-port,
# root-port or switch-port.
# The default setting is "no-port", which means disabled.
cold_plug_vfio = "no-port"
# Before hot plugging a PCIe device, you need to add a pcie_root_port device.
# Use this parameter when using some large PCI bar devices, such as Nvidia GPU
# The value means the number of pcie_root_port
@@ -489,9 +466,6 @@ guest_memory_dump_path = ""
# See: https://www.qemu.org/docs/master/qemu-qmp-ref.html#Dump-guest-memory for details
guest_memory_dump_paging = false
# use legacy serial for guest console if available and implemented for architecture. Default false
use_legacy_serial = false
# disable applying SELinux on the VMM process (default false)
disable_selinux = @DEFDISABLESELINUX@
@@ -574,6 +548,19 @@ dial_timeout_ms = 10
# (default: 3000)
reconnect_timeout_ms = 3000
# Create Container Request Timeout
# This timeout value is used to set the maximum duration for the agent to process a CreateContainerRequest.
# It's also used to ensure that workloads, especially those involving large image pulls within the guest,
# have sufficient time to complete.
#
# Effective Timeout Determination:
# The effective timeout for a CreateContainerRequest is determined by taking the minimum of the following two values:
# - create_container_timeout: The timeout value configured for creating containers (default: 30 seconds).
# - runtime-request-timeout: The timeout value specified in the Kubelet configuration described as the link below:
# (https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/#:~:text=runtime%2Drequest%2Dtimeout)
# Defaults to @DEFCREATECONTAINERTIMEOUT@ second(s)
create_container_timeout = @DEFCREATECONTAINERTIMEOUT@
[agent.@PROJECT_TYPE@.mem_agent]
# Control the mem-agent function enable or disable.
# Default to false
@@ -672,19 +659,6 @@ compact_threshold = 1024
# Using 9223372036854775807 (i64::MAX) which is effectively "never" for practical purposes
compact_force_times = 9223372036854775807
# Create Container Request Timeout
# This timeout value is used to set the maximum duration for the agent to process a CreateContainerRequest.
# It's also used to ensure that workloads, especially those involving large image pulls within the guest,
# have sufficient time to complete.
#
# Effective Timeout Determination:
# The effective timeout for a CreateContainerRequest is determined by taking the minimum of the following two values:
# - create_container_timeout: The timeout value configured for creating containers (default: 30 seconds).
# - runtime-request-timeout: The timeout value specified in the Kubelet configuration described as the link below:
# (https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/#:~:text=runtime%2Drequest%2Dtimeout)
# Defaults to @DEFCREATECONTAINERTIMEOUT@ second(s)
create_container_timeout = @DEFCREATECONTAINERTIMEOUT@
[runtime]
# If enabled, the runtime will log additional debug messages to the
# system log
@@ -720,20 +694,6 @@ agent_name = "@PROJECT_TYPE@"
# (default: true)
disable_guest_seccomp = @DEFDISABLEGUESTSECCOMP@
# vCPUs pinning settings
# if enabled, each vCPU thread will be scheduled to a fixed CPU
# qualified condition: num(vCPU threads) == num(CPUs in sandbox's CPUSet)
enable_vcpus_pinning = false
# Apply a custom SELinux security policy to the container process inside the VM.
# This is used when you want to apply a type other than the default `container_t`,
# so general users should not uncomment and apply it.
# (format: "user:role:type")
# Note: You cannot specify MCS policy with the label because the sensitivity levels and
# categories are determined automatically by high-level container runtimes such as containerd.
# Example value when enabling: "system_u:system_r:container_t"
guest_selinux_label = "@DEFGUESTSELINUXLABEL@"
# If enabled, the runtime will create opentracing.io traces and spans.
# (See https://www.jaegertracing.io/docs/getting-started).
# (default: disabled)

View File

@@ -69,12 +69,6 @@ kernel_params = "@KERNELPARAMS@"
# If you want that qemu uses the default firmware leave this option empty
firmware = "@FIRMWAREPATH@"
# Path to the firmware volume.
# firmware TDVF or OVMF can be split into FIRMWARE_VARS.fd (UEFI variables
# as configuration) and FIRMWARE_CODE.fd (UEFI program image). UEFI variables
# can be customized per each user while UEFI code is kept same.
firmware_volume = "@FIRMWAREVOLUMEPATH@"
# Machine accelerators
# comma-separated list of machine accelerators to pass to the hypervisor.
# For example, `machine_accelerators = "nosmm,nosmbus,nosata,nopit,static-prt,nofw"`
@@ -82,12 +76,12 @@ machine_accelerators = "@MACHINEACCELERATORS@"
# Qemu seccomp sandbox feature
# comma-separated list of seccomp sandbox features to control the syscall access.
# For example, `seccompsandbox= "on,obsolete=deny,spawn=deny,resourcecontrol=deny"`
# For example, `seccomp_sandbox = "on,obsolete=deny,spawn=deny,resourcecontrol=deny"`
# Note: "elevateprivileges=deny" doesn't work with daemonize option, so it's removed from the seccomp sandbox
# Another note: enabling this feature may reduce performance, you may enable
# /proc/sys/net/core/bpf_jit_enable to reduce the impact. see https://man7.org/linux/man-pages/man8/bpfc.8.html
# Recommended value when enabling: "on,obsolete=deny,spawn=deny,resourcecontrol=deny"
seccompsandbox = "@DEFSECCOMPSANDBOXPARAM@"
seccomp_sandbox = "@DEFSECCOMPSANDBOXPARAM@"
# CPU features
# comma-separated list of cpu features to pass to the cpu
@@ -345,18 +339,6 @@ msize_9p = @DEFMSIZE9P@
# Default is false
disable_image_nvdimm = true
# Enable hot-plugging of VFIO devices to a bridge-port,
# root-port or switch-port.
# The default setting is "no-port"
hot_plug_vfio = "no-port"
# In a confidential compute environment hot-plugging can compromise
# security.
# Enable cold-plugging of VFIO devices to a bridge-port,
# root-port or switch-port.
# The default setting is "no-port", which means disabled.
cold_plug_vfio = "no-port"
# VFIO devices are hotplugged on a bridge by default.
# Enable hotplugging on root bus. This may be required for devices with
# a large PCI bar, as this is a current limitation with hotplugging on
@@ -460,9 +442,6 @@ guest_memory_dump_paging = false
# be default_memory.
enable_guest_swap = false
# use legacy serial for guest console if available and implemented for architecture. Default false
use_legacy_serial = false
# disable applying SELinux on the VMM process (default false)
disable_selinux = @DEFDISABLESELINUX@
@@ -474,7 +453,7 @@ disable_selinux = @DEFDISABLESELINUX@
disable_guest_selinux = @DEFDISABLEGUESTSELINUX@
[factory]
[hypervisor.qemu.factory]
# VM templating support. Once enabled, new VMs are created from template
# using vm cloning. They will share the same initial kernel, initramfs and
# agent memory by mapping it readonly. It helps speeding up new container
@@ -593,20 +572,6 @@ agent_name = "@PROJECT_TYPE@"
# (default: true)
disable_guest_seccomp = @DEFDISABLEGUESTSECCOMP@
# vCPUs pinning settings
# if enabled, each vCPU thread will be scheduled to a fixed CPU
# qualified condition: num(vCPU threads) == num(CPUs in sandbox's CPUSet)
enable_vcpus_pinning = false
# Apply a custom SELinux security policy to the container process inside the VM.
# This is used when you want to apply a type other than the default `container_t`,
# so general users should not uncomment and apply it.
# (format: "user:role:type")
# Note: You cannot specify MCS policy with the label because the sensitivity levels and
# categories are determined automatically by high-level container runtimes such as containerd.
# Example value when enabling: "system_u:system_r:container_t"
guest_selinux_label = "@DEFGUESTSELINUXLABEL@"
# If enabled, the runtime will create opentracing.io traces and spans.
# (See https://www.jaegertracing.io/docs/getting-started).
# (default: disabled)

View File

@@ -16,8 +16,7 @@
[hypervisor.qemu]
path = "@QEMUPATH@"
kernel = "@KERNELPATH_COCO@"
initrd = "@INITRDCONFIDENTIALPATH@"
# image = "@IMAGECONFIDENTIALPATH@"
image = "@IMAGECONFIDENTIALPATH@"
machine_type = "@MACHINETYPE@"
# Enable confidential guest support.
@@ -99,16 +98,15 @@ valid_hypervisor_paths = @QEMUVALIDHYPERVISORPATHS@
# container and look for 'default-kernel-parameters' log entries.
kernel_params = "@KERNELPARAMS@"
# Optional dm-verity parameters (comma-separated key=value list):
# root_hash=...,salt=...,data_blocks=...,data_block_size=...,hash_block_size=...
# These are used by the runtime to assemble dm-verity kernel params.
kernel_verity_params = "@KERNELVERITYPARAMS@"
# Path to the firmware.
# If you want that qemu uses the default firmware leave this option empty
firmware = "@FIRMWARE_SNP_PATH@"
# Path to the firmware volume.
# firmware TDVF or OVMF can be split into FIRMWARE_VARS.fd (UEFI variables
# as configuration) and FIRMWARE_CODE.fd (UEFI program image). UEFI variables
# can be customized per each user while UEFI code is kept same.
firmware_volume = "@FIRMWARE_VOLUME_SNP_PATH@"
# Machine accelerators
# comma-separated list of machine accelerators to pass to the hypervisor.
# For example, `machine_accelerators = "nosmm,nosmbus,nosata,nopit,static-prt,nofw"`
@@ -116,12 +114,12 @@ machine_accelerators = "@MACHINEACCELERATORS@"
# Qemu seccomp sandbox feature
# comma-separated list of seccomp sandbox features to control the syscall access.
# For example, `seccompsandbox= "on,obsolete=deny,spawn=deny,resourcecontrol=deny"`
# For example, `seccomp_sandbox = "on,obsolete=deny,spawn=deny,resourcecontrol=deny"`
# Note: "elevateprivileges=deny" doesn't work with daemonize option, so it's removed from the seccomp sandbox
# Another note: enabling this feature may reduce performance, you may enable
# /proc/sys/net/core/bpf_jit_enable to reduce the impact. see https://man7.org/linux/man-pages/man8/bpfc.8.html
# Recommended value when enabling: "on,obsolete=deny,spawn=deny,resourcecontrol=deny"
seccompsandbox = "@DEFSECCOMPSANDBOXPARAM@"
seccomp_sandbox = "@DEFSECCOMPSANDBOXPARAM@"
# CPU features
# comma-separated list of cpu features to pass to the cpu
@@ -289,10 +287,6 @@ block_device_cache_noflush = false
#
enable_iothreads = @DEFENABLEIOTHREADS@
# Independent IOThreads enables IO to be processed in a separate thread, it is
# for QEMU hotplug device attach to iothread, like virtio-blk.
indep_iothreads = @DEFINDEPIOTHREADS@
# Enable pre allocation of VM RAM, default false
# Enabling this will result in lower container density
# as all of the memory will be allocated and locked
@@ -346,11 +340,6 @@ enable_iommu_platform = false
# Your distribution recommends: @DEFVALIDVHOSTUSERSTOREPATHS@
valid_vhost_user_store_paths = @DEFVALIDVHOSTUSERSTOREPATHS@
# The timeout for reconnecting on non-server spdk sockets when the remote end goes away.
# qemu will delay this many seconds and then attempt to reconnect.
# Zero disables reconnecting, and the default is zero.
vhost_user_reconnect_timeout_sec = 0
# Enable file based guest memory support. The default is an empty string which
# will disable this feature. In the case of virtio-fs, this is enabled
# automatically and '/dev/shm' is used as the backing folder.
@@ -407,7 +396,7 @@ disable_vhost_net = false
#
# If set to the empty string "", no extra monitor socket is added. This is
# the default.
#extra_monitor_socket = "hmp"
extra_monitor_socket = ""
#
# Default entropy source.
@@ -495,9 +484,6 @@ guest_memory_dump_paging = false
# be default_memory.
enable_guest_swap = false
# use legacy serial for guest console if available and implemented for architecture. Default false
use_legacy_serial = false
# disable applying SELinux on the VMM process (default false)
disable_selinux = @DEFDISABLESELINUX@
@@ -509,7 +495,7 @@ disable_selinux = @DEFDISABLESELINUX@
disable_guest_selinux = @DEFDISABLEGUESTSELINUX@
[factory]
[hypervisor.qemu.factory]
# VM templating support. Once enabled, new VMs are created from template
# using vm cloning. They will share the same initial kernel, initramfs and
# agent memory by mapping it readonly. It helps speeding up new container
@@ -528,30 +514,6 @@ enable_template = false
# Default "/run/vc/vm/template"
template_path = "/run/vc/vm/template"
# The number of caches of VMCache:
# unspecified or == 0 --> VMCache is disabled
# > 0 --> will be set to the specified number
#
# VMCache is a function that creates VMs as caches before using it.
# It helps speed up new container creation.
# The function consists of a server and some clients communicating
# through Unix socket. The protocol is gRPC in protocols/cache/cache.proto.
# The VMCache server will create some VMs and cache them by factory cache.
# It will convert the VM to gRPC format and transport it when gets
# requestion from clients.
# Factory grpccache is the VMCache client. It will request gRPC format
# VM and convert it back to a VM. If VMCache function is enabled,
# kata-runtime will request VM from factory grpccache when it creates
# a new sandbox.
#
# Default 0
vm_cache_number = 0
# Specify the address of the Unix socket that is used by VMCache.
#
# Default /var/run/kata-containers/cache.sock
vm_cache_endpoint = "/var/run/kata-containers/cache.sock"
[agent.@PROJECT_TYPE@]
# If enabled, make the agent display debug-level messages.
# (default: disabled)
@@ -651,19 +613,6 @@ agent_name="@PROJECT_TYPE@"
# (default: true)
disable_guest_seccomp = @DEFDISABLEGUESTSECCOMP@
# vCPUs pinning settings
# if enabled, each vCPU thread will be scheduled to a fixed CPU
# qualified condition: num(vCPU threads) == num(CPUs in sandbox's CPUSet)
enable_vcpus_pinning = false
# Apply a custom SELinux security policy to the container process inside the VM.
# This is used when you want to apply a type other than the default `container_t`,
# so general users should not uncomment and apply it.
# (format: "user:role:type")
# Note: You cannot specify MCS policy with the label because the sensitivity levels and
# categories are determined automatically by high-level container runtimes such as containerd.
guest_selinux_label = "@DEFGUESTSELINUXLABEL@"
# If enabled, the runtime will create opentracing.io traces and spans.
# (See https://www.jaegertracing.io/docs/getting-started).
# (default: disabled)
@@ -757,22 +706,3 @@ enable_pprof = false
# to the hypervisor.
# (default: /run/kata-containers/dans)
dan_conf = "@DEFDANCONF@"
# pod_resource_api_sock specifies the unix socket for the Kubelet's
# PodResource API endpoint. If empty, kubernetes based cold plug
# will not be attempted. In order for this feature to work, the
# KubeletPodResourcesGet featureGate must be enabled in Kubelet,
# if using Kubelet older than 1.34.
#
# The pod resource API's socket is relative to the Kubelet's root-dir,
# which is defined by the cluster admin, and its location is:
# ${KubeletRootDir}/pod-resources/kubelet.sock
#
# cold_plug_vfio(see hypervisor config) acts as a feature gate:
# cold_plug_vfio = no_port (default) => no cold plug
# cold_plug_vfio != no_port AND pod_resource_api_sock = "" => need
# explicit CDI annotation for cold plug (applies mainly
# to non-k8s cases)
# cold_plug_vfio != no_port AND pod_resource_api_sock != "" => kubelet
# based cold plug.
pod_resource_api_sock = "@DEFPODRESOURCEAPISOCK@"

View File

@@ -83,12 +83,6 @@ kernel_verity_params = "@KERNELVERITYPARAMS@"
# If you want that qemu uses the default firmware leave this option empty
firmware = "@FIRMWARETDVFPATH@"
# Path to the firmware volume.
# firmware TDVF or OVMF can be split into FIRMWARE_VARS.fd (UEFI variables
# as configuration) and FIRMWARE_CODE.fd (UEFI program image). UEFI variables
# can be customized per each user while UEFI code is kept same.
firmware_volume = "@FIRMWAREVOLUMEPATH@"
# Machine accelerators
# comma-separated list of machine accelerators to pass to the hypervisor.
# For example, `machine_accelerators = "nosmm,nosmbus,nosata,nopit,static-prt,nofw"`
@@ -96,12 +90,12 @@ machine_accelerators = "@MACHINEACCELERATORS@"
# Qemu seccomp sandbox feature
# comma-separated list of seccomp sandbox features to control the syscall access.
# For example, `seccompsandbox= "on,obsolete=deny,spawn=deny,resourcecontrol=deny"`
# For example, `seccomp_sandbox = "on,obsolete=deny,spawn=deny,resourcecontrol=deny"`
# Note: "elevateprivileges=deny" doesn't work with daemonize option, so it's removed from the seccomp sandbox
# Another note: enabling this feature may reduce performance, you may enable
# /proc/sys/net/core/bpf_jit_enable to reduce the impact. see https://man7.org/linux/man-pages/man8/bpfc.8.html
# Recommended value when enabling: "on,obsolete=deny,spawn=deny,resourcecontrol=deny"
seccompsandbox = "@DEFSECCOMPSANDBOXPARAM@"
seccomp_sandbox = "@DEFSECCOMPSANDBOXPARAM@"
# CPU features
# comma-separated list of cpu features to pass to the cpu
@@ -268,10 +262,6 @@ block_device_cache_noflush = false
#
enable_iothreads = @DEFENABLEIOTHREADS@
# Independent IOThreads enables IO to be processed in a separate thread, it is
# for QEMU hotplug device attach to iothread, like virtio-blk.
indep_iothreads = @DEFINDEPIOTHREADS@
# Enable pre allocation of VM RAM, default false
# Enabling this will result in lower container density
# as all of the memory will be allocated and locked
@@ -325,11 +315,6 @@ enable_iommu_platform = false
# Your distribution recommends: @DEFVALIDVHOSTUSERSTOREPATHS@
valid_vhost_user_store_paths = @DEFVALIDVHOSTUSERSTOREPATHS@
# The timeout for reconnecting on non-server spdk sockets when the remote end goes away.
# qemu will delay this many seconds and then attempt to reconnect.
# Zero disables reconnecting, and the default is zero.
vhost_user_reconnect_timeout_sec = 0
# Enable file based guest memory support. The default is an empty string which
# will disable this feature. In the case of virtio-fs, this is enabled
# automatically and '/dev/shm' is used as the backing folder.
@@ -474,9 +459,6 @@ guest_memory_dump_paging = false
# be default_memory.
enable_guest_swap = false
# use legacy serial for guest console if available and implemented for architecture. Default false
use_legacy_serial = false
# disable applying SELinux on the VMM process (default false)
disable_selinux = @DEFDISABLESELINUX@
@@ -488,7 +470,7 @@ disable_selinux = @DEFDISABLESELINUX@
disable_guest_selinux = @DEFDISABLEGUESTSELINUX@
[factory]
[hypervisor.qemu.factory]
# VM templating support. Once enabled, new VMs are created from template
# using vm cloning. They will share the same initial kernel, initramfs and
# agent memory by mapping it readonly. It helps speeding up new container
@@ -507,30 +489,6 @@ enable_template = false
# Default "/run/vc/vm/template"
template_path = "/run/vc/vm/template"
# The number of caches of VMCache:
# unspecified or == 0 --> VMCache is disabled
# > 0 --> will be set to the specified number
#
# VMCache is a function that creates VMs as caches before using it.
# It helps speed up new container creation.
# The function consists of a server and some clients communicating
# through Unix socket. The protocol is gRPC in protocols/cache/cache.proto.
# The VMCache server will create some VMs and cache them by factory cache.
# It will convert the VM to gRPC format and transport it when gets
# requestion from clients.
# Factory grpccache is the VMCache client. It will request gRPC format
# VM and convert it back to a VM. If VMCache function is enabled,
# kata-runtime will request VM from factory grpccache when it creates
# a new sandbox.
#
# Default 0
vm_cache_number = 0
# Specify the address of the Unix socket that is used by VMCache.
#
# Default /var/run/kata-containers/cache.sock
vm_cache_endpoint = "/var/run/kata-containers/cache.sock"
[agent.@PROJECT_TYPE@]
# If enabled, make the agent display debug-level messages.
# (default: disabled)
@@ -631,20 +589,6 @@ agent_name="@PROJECT_TYPE@"
# (default: true)
disable_guest_seccomp = @DEFDISABLEGUESTSECCOMP@
# vCPUs pinning settings
# if enabled, each vCPU thread will be scheduled to a fixed CPU
# qualified condition: num(vCPU threads) == num(CPUs in sandbox's CPUSet)
enable_vcpus_pinning = false
# Apply a custom SELinux security policy to the container process inside the VM.
# This is used when you want to apply a type other than the default `container_t`,
# so general users should not uncomment and apply it.
# (format: "user:role:type")
# Note: You cannot specify MCS policy with the label because the sensitivity levels and
# categories are determined automatically by high-level container runtimes such as containerd.
# Example value when enabling: "system_u:system_r:container_t"
guest_selinux_label = "@DEFGUESTSELINUXLABEL@"
# If enabled, the runtime will create opentracing.io traces and spans.
# (See https://www.jaegertracing.io/docs/getting-started).
# (default: disabled)
@@ -739,21 +683,3 @@ enable_pprof = false
# (default: /run/kata-containers/dans)
dan_conf = "@DEFDANCONF@"
# pod_resource_api_sock specifies the unix socket for the Kubelet's
# PodResource API endpoint. If empty, kubernetes based cold plug
# will not be attempted. In order for this feature to work, the
# KubeletPodResourcesGet featureGate must be enabled in Kubelet,
# if using Kubelet older than 1.34.
#
# The pod resource API's socket is relative to the Kubelet's root-dir,
# which is defined by the cluster admin, and its location is:
# ${KubeletRootDir}/pod-resources/kubelet.sock
#
# cold_plug_vfio(see hypervisor config) acts as a feature gate:
# cold_plug_vfio = no_port (default) => no cold plug
# cold_plug_vfio != no_port AND pod_resource_api_sock = "" => need
# explicit CDI annotation for cold plug (applies mainly
# to non-k8s cases)
# cold_plug_vfio != no_port AND pod_resource_api_sock != "" => kubelet
# based cold plug.
pod_resource_api_sock = "@DEFPODRESOURCEAPISOCK@"

View File

@@ -150,9 +150,9 @@ enable_tracing = false
debug_console_enabled = false
# Agent connection dialing timeout value in seconds
# (default: 30)
dial_timeout = 30
# Agent connection dialing timeout value in milliseconds
# (default: 30000)
dial_timeout_ms = 30000
# Create Container Request Timeout
# This timeout value is used to set the maximum duration for the agent to process a CreateContainerRequest.
@@ -205,15 +205,6 @@ agent_name = "kata"
disable_guest_seccomp = true
# Apply a custom SELinux security policy to the container process inside the VM.
# This is used when you want to apply a type other than the default `container_t`,
# so general users should not uncomment and apply it.
# (format: "user:role:type")
# Note: You cannot specify MCS policy with the label because the sensitivity levels and
# categories are determined automatically by high-level container runtimes such as containerd.
# Example value when enabling: "system_u:system_r:container_t"
guest_selinux_label = "@DEFGUESTSELINUXLABEL@"
# If enabled, the runtime will create opentracing.io traces and spans.
# (See https://www.jaegertracing.io/docs/getting-started).
# (default: disabled)

View File

@@ -310,9 +310,9 @@ kernel_modules = []
debug_console_enabled = false
# Agent connection dialing timeout value in seconds
# (default: 45)
dial_timeout = 45
# Agent connection dialing timeout value in milliseconds
# (default: 45000)
dial_timeout_ms = 45000
# Confidential Data Hub API timeout value in seconds
# (default: 50)

View File

@@ -2248,8 +2248,8 @@ impl<'a> QemuCmdLine<'a> {
qemu_cmd_line.add_iommu();
}
if config.debug_info.enable_debug && !config.debug_info.dbg_monitor_socket.is_empty() {
qemu_cmd_line.add_monitor(&config.debug_info.dbg_monitor_socket)?;
if config.debug_info.enable_debug && !config.debug_info.extra_monitor_socket.is_empty() {
qemu_cmd_line.add_monitor(&config.debug_info.extra_monitor_socket)?;
}
qemu_cmd_line.add_rtc();
@@ -2610,6 +2610,7 @@ impl<'a> QemuCmdLine<'a> {
self.devices.push(Box::new(Bios::new(firmware.to_owned())));
self.machine
.set_kernel_irqchip("split")
.set_confidential_guest_support("tdx")
.set_nvdimm(false);
}

View File

@@ -142,15 +142,23 @@ impl InitialSizeManager {
if self.resource.vcpu > 0.0 {
info!(sl!(), "resource with vcpu {}", self.resource.vcpu);
if config.runtime.static_sandbox_resource_mgmt {
hv.cpu_info.default_vcpus += self.resource.vcpu;
let new_vcpus_ceil = hv.cpu_info.default_vcpus.ceil() as u32;
if hv.cpu_info.default_maxvcpus < new_vcpus_ceil {
hv.cpu_info.default_maxvcpus = new_vcpus_ceil;
}
}
}
self.resource.orig_toml_default_mem = hv.memory_info.default_memory;
if self.resource.mem_mb > 0 {
// since the memory overhead introduced by kata-agent and system components
// will really affect the amount of memory the user can use, so we choose to
// plus the default_memory here, instead of overriding it.
// (if we override the default_memory here, and user apllications still
// use memory as they orignally expected, it would be easy to OOM.)
hv.memory_info.default_memory += self.resource.mem_mb;
info!(sl!(), "resource with memory {}", self.resource.mem_mb);
if config.runtime.static_sandbox_resource_mgmt {
hv.memory_info.default_memory += self.resource.mem_mb;
if hv.memory_info.default_maxmemory < hv.memory_info.default_memory {
hv.memory_info.default_maxmemory = hv.memory_info.default_memory;
}
}
}
Ok(())
}
@@ -366,4 +374,114 @@ mod tests {
);
}
}
fn make_config(
default_vcpus: f32,
default_maxvcpus: u32,
default_memory: u32,
default_maxmemory: u32,
static_sandbox_resource_mgmt: bool,
) -> TomlConfig {
use kata_types::config::Hypervisor;
let mut config = TomlConfig::default();
config
.hypervisor
.insert("qemu".to_owned(), Hypervisor::default());
config
.hypervisor
.entry("qemu".to_owned())
.and_modify(|hv| {
hv.cpu_info.default_vcpus = default_vcpus;
hv.cpu_info.default_maxvcpus = default_maxvcpus;
hv.memory_info.default_memory = default_memory;
hv.memory_info.default_maxmemory = default_maxmemory;
});
config.runtime.hypervisor_name = "qemu".to_owned();
config.runtime.static_sandbox_resource_mgmt = static_sandbox_resource_mgmt;
config
}
#[test]
fn test_setup_config_static_applies_vcpu_and_memory() {
let mut config = make_config(1.0, 4, 256, 4096, true);
let mut mgr = InitialSizeManager {
resource: InitialSize {
vcpu: 1.2,
mem_mb: 512,
orig_toml_default_mem: 0,
},
};
mgr.setup_config(&mut config).unwrap();
let hv = config.hypervisor.get("qemu").unwrap();
assert_eq!(hv.cpu_info.default_vcpus, 2.2);
assert_eq!(hv.memory_info.default_memory, 768);
}
#[test]
fn test_setup_config_non_static_does_not_apply() {
let mut config = make_config(1.0, 4, 256, 4096, false);
let mut mgr = InitialSizeManager {
resource: InitialSize {
vcpu: 1.2,
mem_mb: 512,
orig_toml_default_mem: 0,
},
};
mgr.setup_config(&mut config).unwrap();
let hv = config.hypervisor.get("qemu").unwrap();
assert_eq!(hv.cpu_info.default_vcpus, 1.0);
assert_eq!(hv.memory_info.default_memory, 256);
}
#[test]
fn test_setup_config_clamps_maxvcpus() {
let mut config = make_config(1.0, 2, 256, 4096, true);
let mut mgr = InitialSizeManager {
resource: InitialSize {
vcpu: 2.5,
mem_mb: 0,
orig_toml_default_mem: 0,
},
};
mgr.setup_config(&mut config).unwrap();
let hv = config.hypervisor.get("qemu").unwrap();
assert_eq!(hv.cpu_info.default_vcpus, 3.5);
assert_eq!(hv.cpu_info.default_maxvcpus, 4);
}
#[test]
fn test_setup_config_clamps_maxmemory() {
let mut config = make_config(1.0, 4, 256, 300, true);
let mut mgr = InitialSizeManager {
resource: InitialSize {
vcpu: 0.0,
mem_mb: 512,
orig_toml_default_mem: 0,
},
};
mgr.setup_config(&mut config).unwrap();
let hv = config.hypervisor.get("qemu").unwrap();
assert_eq!(hv.memory_info.default_memory, 768);
assert_eq!(hv.memory_info.default_maxmemory, 768);
}
#[test]
fn test_setup_config_preserves_orig_toml_default_mem() {
let mut config = make_config(1.0, 4, 256, 4096, true);
let mut mgr = InitialSizeManager {
resource: InitialSize {
vcpu: 0.0,
mem_mb: 128,
orig_toml_default_mem: 0,
},
};
mgr.setup_config(&mut config).unwrap();
assert_eq!(mgr.get_orig_toml_default_mem(), 256);
}
}

View File

@@ -65,8 +65,8 @@ enable_guest_swap = true
[agent.agent0]
enable_tracing = true
debug_console_enabled = true
debug = true
dial_timeout = 1
enable_debug = true
dial_timeout_ms = 1000
kernel_modules = ["e1000e InterruptThrottleRate=3000,3000,3000 EEE=1","i915_enabled_ppgtt=0"]
container_pipe_size = 2
[runtime]

View File

@@ -143,7 +143,13 @@ DEFROOTFSTYPE := $(ROOTFSTYPE_EXT4)
FIRMWAREPATH :=
FIRMWAREVOLUMEPATH :=
FIRMWAREPATH_NV = $(FIRMWAREPATH)
FIRMWAREPATH_NV :=
ifeq ($(ARCH),amd64)
FIRMWAREPATH_NV := $(PREFIXDEPS)/share/$(EDK2_NAME)/OVMF.fd
endif
ifeq ($(ARCH),arm64)
FIRMWAREPATH_NV := $(PREFIXDEPS)/share/$(EDK2_NAME)/AAVMF_CODE.fd
endif
FIRMWARETDVFPATH := $(PREFIXDEPS)/share/ovmf/OVMF.inteltdx.fd
FIRMWARETDVFPATH_NV := $(FIRMWARETDVFPATH)
@@ -286,6 +292,10 @@ DEFVFIOMODE := guest-kernel
DEFSANDBOXCGROUPONLY ?= false
DEFSTATICRESOURCEMGMT ?= false
DEFSTATICRESOURCEMGMT_QEMU ?= false
ifeq ($(ARCH),arm64)
DEFSTATICRESOURCEMGMT_QEMU = true
endif
DEFSTATICRESOURCEMGMT_TEE = true
DEFSTATICRESOURCEMGMT_NV = true
@@ -489,6 +499,9 @@ ifneq (,$(QEMUCMD))
KERNELPARAMS_NV += "pci=nocrs"
KERNELPARAMS_NV += "pci=assign-busses"
KERNELPARAMS_CONFIDENTIAL_NV = $(KERNELPARAMS_NV)
KERNELPARAMS_CONFIDENTIAL_NV += "nvrc.smi.srs=1"
# Setting this to false can lead to cgroup leakages in the host
# Best practice for production is to set this to true
DEFSANDBOXCGROUPONLY_NV = true
@@ -661,6 +674,7 @@ USER_VARS += DEFAULTMEMORY_NV
USER_VARS += DEFAULTVFIOPORT_NV
USER_VARS += DEFAULTPCIEROOTPORT_NV
USER_VARS += KERNELPARAMS_NV
USER_VARS += KERNELPARAMS_CONFIDENTIAL_NV
USER_VARS += KERNELVERITYPARAMS_NV
USER_VARS += KERNELVERITYPARAMS_CONFIDENTIAL_NV
USER_VARS += DEFAULTTIMEOUT_NV
@@ -781,6 +795,7 @@ USER_VARS += DEFENTROPYSOURCE
USER_VARS += DEFVALIDENTROPYSOURCES
USER_VARS += DEFSANDBOXCGROUPONLY
USER_VARS += DEFSTATICRESOURCEMGMT
USER_VARS += DEFSTATICRESOURCEMGMT_QEMU
USER_VARS += DEFSTATICRESOURCEMGMT_CLH
USER_VARS += DEFSTATICRESOURCEMGMT_FC
USER_VARS += DEFSTATICRESOURCEMGMT_STRATOVIRT

View File

@@ -90,7 +90,7 @@ snp_guest_policy = 196608
# may stop the virtual machine from booting.
# To see the list of default parameters, enable hypervisor debug, create a
# container and look for 'default-kernel-parameters' log entries.
kernel_params = "@KERNELPARAMS_NV@"
kernel_params = "@KERNELPARAMS_CONFIDENTIAL_NV@"
# Optional dm-verity parameters (comma-separated key=value list):
# root_hash=...,salt=...,data_blocks=...,data_block_size=...,hash_block_size=...

View File

@@ -67,7 +67,7 @@ valid_hypervisor_paths = @QEMUTDXEXPERIMENTALVALIDHYPERVISORPATHS@
# may stop the virtual machine from booting.
# To see the list of default parameters, enable hypervisor debug, create a
# container and look for 'default-kernel-parameters' log entries.
kernel_params = "@KERNELPARAMS_NV@"
kernel_params = "@KERNELPARAMS_CONFIDENTIAL_NV@"
# Optional dm-verity parameters (comma-separated key=value list):
# root_hash=...,salt=...,data_blocks=...,data_block_size=...,hash_block_size=...

View File

@@ -15,7 +15,7 @@
[hypervisor.qemu]
path = "@QEMUPATH@"
kernel = "@KERNELCONFIDENTIALPATH@"
initrd = "@INITRDCONFIDENTIALPATH@"
image = "@IMAGECONFIDENTIALPATH@"
machine_type = "@MACHINETYPE@"
# rootfs filesystem type:
@@ -91,6 +91,11 @@ snp_guest_policy = 196608
# container and look for 'default-kernel-parameters' log entries.
kernel_params = "@KERNELPARAMS@"
# Optional dm-verity parameters (comma-separated key=value list):
# root_hash=...,salt=...,data_blocks=...,data_block_size=...,hash_block_size=...
# These are used by the runtime to assemble dm-verity kernel params.
kernel_verity_params = "@KERNELVERITYPARAMS@"
# Path to the firmware.
# If you want that qemu uses the default firmware leave this option empty
firmware = "@FIRMWARESNPPATH@"

View File

@@ -672,7 +672,7 @@ sandbox_cgroup_only = @DEFSANDBOXCGROUPONLY@
# - When running with pods, sandbox sizing information will only be available if using Kubernetes >= 1.23 and containerd >= 1.6. CRI-O
# does not yet support sandbox sizing annotations.
# - When running single containers using a tool like ctr, container sizing information will be available.
static_sandbox_resource_mgmt = @DEFSTATICRESOURCEMGMT@
static_sandbox_resource_mgmt = @DEFSTATICRESOURCEMGMT_QEMU@
# If specified, sandbox_bind_mounts identifieds host paths to be mounted (ro) into the sandboxes shared path.
# This is only valid if filesystem sharing is utilized. The provided path(s) will be bindmounted into the shared fs directory.

View File

@@ -23,15 +23,6 @@ dependencies = [
"tokio",
]
[[package]]
name = "addr2line"
version = "0.25.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1b5d307320b3181d6d7954e663bd7c774a838b8220fe0593c86d9fb09f498b4b"
dependencies = [
"gimli",
]
[[package]]
name = "adler2"
version = "2.0.0"
@@ -341,21 +332,6 @@ dependencies = [
"tower-service",
]
[[package]]
name = "backtrace"
version = "0.3.76"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb531853791a215d7c62a30daf0dde835f381ab5de4589cfe7c649d2cbe92bd6"
dependencies = [
"addr2line",
"cfg-if 1.0.4",
"libc",
"miniz_oxide",
"object",
"rustc-demangle",
"windows-link",
]
[[package]]
name = "base16ct"
version = "0.2.0"
@@ -1595,9 +1571,9 @@ checksum = "e6d5a32815ae3f33302d95fdcb2ce17862f8c65363dcfd29360480ba1001fc9c"
[[package]]
name = "futures"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "65bc07b1a8bc7c85c5f2e110c476c7389b4554ba72af57d8445ea63a576b0876"
checksum = "8b147ee9d1f6d097cef9ce628cd2ee62288d963e16fb287bd9286455b241382d"
dependencies = [
"futures-channel",
"futures-core",
@@ -1610,9 +1586,9 @@ dependencies = [
[[package]]
name = "futures-channel"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2dff15bf788c671c1934e366d07e30c1814a8ef514e1af724a602e8a2fbe1b10"
checksum = "07bbe89c50d7a535e539b8c17bc0b49bdb77747034daa8087407d655f3f7cc1d"
dependencies = [
"futures-core",
"futures-sink",
@@ -1620,15 +1596,15 @@ dependencies = [
[[package]]
name = "futures-core"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05f29059c0c2090612e8d742178b0580d2dc940c837851ad723096f87af6663e"
checksum = "7e3450815272ef58cec6d564423f6e755e25379b217b0bc688e295ba24df6b1d"
[[package]]
name = "futures-executor"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e28d1d997f585e54aebc3f97d39e72338912123a67330d723fdbb564d646c9f"
checksum = "baf29c38818342a3b26b5b923639e7b1f4a61fc5e76102d4b1981c6dc7a7579d"
dependencies = [
"futures-core",
"futures-task",
@@ -1637,15 +1613,15 @@ dependencies = [
[[package]]
name = "futures-io"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9e5c1b78ca4aae1ac06c48a526a655760685149f0d465d21f37abfe57ce075c6"
checksum = "cecba35d7ad927e23624b22ad55235f2239cfa44fd10428eecbeba6d6a717718"
[[package]]
name = "futures-macro"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "162ee34ebcb7c64a8abebc059ce0fee27c2262618d7b60ed8faf72fef13c3650"
checksum = "e835b70203e41293343137df5c0664546da5745f82ec9b84d40be8336958447b"
dependencies = [
"proc-macro2",
"quote",
@@ -1654,21 +1630,21 @@ dependencies = [
[[package]]
name = "futures-sink"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e575fab7d1e0dcb8d0c7bcf9a63ee213816ab51902e6d244a95819acacf1d4f7"
checksum = "c39754e157331b013978ec91992bde1ac089843443c49cbc7f46150b0fad0893"
[[package]]
name = "futures-task"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f90f7dce0722e95104fcb095585910c0977252f286e354b5e3bd38902cd99988"
checksum = "037711b3d59c33004d3856fbdc83b99d4ff37a24768fa1be9ce3538a1cde4393"
[[package]]
name = "futures-util"
version = "0.3.31"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9fa08315bb612088cc391249efdc3bc77536f16c91f6cf495e6fbe85b20a4a81"
checksum = "389ca41296e6190b48053de0321d02a77f32f8a5d2461dd38762c0593805c6d6"
dependencies = [
"futures-channel",
"futures-core",
@@ -1678,7 +1654,6 @@ dependencies = [
"futures-task",
"memchr",
"pin-project-lite",
"pin-utils",
"slab",
]
@@ -1742,12 +1717,6 @@ dependencies = [
"polyval",
]
[[package]]
name = "gimli"
version = "0.32.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e629b9b98ef3dd8afe6ca2bd0f89306cec16d43d907889945bc5d6687f2f13c7"
[[package]]
name = "glob"
version = "0.3.1"
@@ -2052,6 +2021,7 @@ version = "0.1.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc2fdfdbff08affe55bb779f33b053aa1fe5dd5b54c257343c17edfa55711bdb"
dependencies = [
"base64 0.22.1",
"bytes",
"futures-channel",
"futures-core",
@@ -2059,7 +2029,9 @@ dependencies = [
"http 1.1.0",
"http-body 1.0.1",
"hyper 1.6.0",
"ipnet",
"libc",
"percent-encoding",
"pin-project-lite",
"socket2 0.5.10",
"tokio",
@@ -2271,7 +2243,7 @@ dependencies = [
[[package]]
name = "image-rs"
version = "0.1.0"
source = "git+https://github.com/confidential-containers/guest-components?rev=026694d44d4ec483465d2fa5f80a0376166b174d#026694d44d4ec483465d2fa5f80a0376166b174d"
source = "git+https://github.com/confidential-containers/guest-components?tag=v0.18.0#ab95914ac84c32a43102463cc0ae330710af47be"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -2284,27 +2256,27 @@ dependencies = [
"futures",
"futures-util",
"hex",
"log",
"loopdev",
"nix 0.30.1",
"nix 0.31.2",
"oci-client",
"oci-spec",
"ocicrypt-rs",
"protos",
"reqwest",
"reqwest 0.13.2",
"sequoia-openpgp",
"serde",
"serde_json",
"serde_yaml",
"sha2 0.10.9",
"sigstore",
"strum",
"strum_macros",
"strum 0.28.0",
"strum_macros 0.28.0",
"thiserror 2.0.18",
"tokio",
"tokio-util",
"toml 0.9.11+spec-1.1.0",
"toml 1.1.2+spec-1.1.0",
"tonic",
"tracing",
"url",
"walkdir",
"xattr",
@@ -2364,23 +2336,22 @@ dependencies = [
"windows-sys 0.48.0",
]
[[package]]
name = "io-uring"
version = "0.7.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b86e202f00093dcba4275d4636b93ef9dd75d025ae560d2521b45ea28ab49013"
dependencies = [
"bitflags 2.6.0",
"cfg-if 1.0.4",
"libc",
]
[[package]]
name = "ipnet"
version = "2.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ddc24109865250148c2e0f3d25d4f0f479571723792d3802153c60922a4fb708"
[[package]]
name = "iri-string"
version = "0.7.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "25e659a4bb38e810ebc252e53b5814ff908a8c58c2a9ce2fae1bbec24cbf4e20"
dependencies = [
"memchr",
"serde",
]
[[package]]
name = "is-terminal"
version = "0.4.16"
@@ -2595,9 +2566,9 @@ checksum = "830d08ce1d1d941e6b30645f1a0eb5643013d835ce3779a5fc208261dbe10f55"
[[package]]
name = "libc"
version = "0.2.172"
version = "0.2.184"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d750af042f7ef4f724306de029d18836c26c1765a54a6a3f094cbd23a7267ffa"
checksum = "48f5d2a454e16a5ea0f4ced81bd44e4cfc7bd3a507b61887c99fd3538b28e4af"
[[package]]
name = "libloading"
@@ -2788,14 +2759,13 @@ dependencies = [
[[package]]
name = "mio"
version = "1.0.2"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "80e04d1dcff3aae0704555fe5fee3bcfaf3d1fdf8a7e521d5b9d2b42acb52cec"
checksum = "50b7e5b27aa02a74bac8c3f23f448f8d87ff11f92d3aac1a6ed369ee08cc56c1"
dependencies = [
"hermit-abi 0.3.9",
"libc",
"wasi",
"windows-sys 0.52.0",
"windows-sys 0.61.2",
]
[[package]]
@@ -2876,9 +2846,9 @@ dependencies = [
[[package]]
name = "nix"
version = "0.30.1"
version = "0.31.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "74523f3a35e05aba87a1d978330aef40f67b0304ac79c1c00b294c9830543db6"
checksum = "5d6d0705320c1e6ba1d912b5e37cf18071b6c2e9b7fa8215a1e8a7651966f5d3"
dependencies = [
"bitflags 2.6.0",
"cfg-if 1.0.4",
@@ -2907,11 +2877,10 @@ dependencies = [
[[package]]
name = "num-bigint-dig"
version = "0.8.4"
version = "0.8.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc84195820f291c7697304f3cbdadd1cb7199c0efc917ff5eafd71225c136151"
checksum = "e661dda6640fad38e827a6d4a310ff4763082116fe217f279885c97f511bb0b7"
dependencies = [
"byteorder",
"lazy_static",
"libm",
"num-integer",
@@ -2979,7 +2948,7 @@ dependencies = [
"getrandom 0.2.15",
"http 1.1.0",
"rand 0.8.5",
"reqwest",
"reqwest 0.12.8",
"serde",
"serde_json",
"serde_path_to_error",
@@ -2997,15 +2966,6 @@ dependencies = [
"bitflags 2.6.0",
]
[[package]]
name = "object"
version = "0.37.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ff76201f031d8863c38aa7f905eca4f53abbfa15f609db4277d44cd8938f33fe"
dependencies = [
"memchr",
]
[[package]]
name = "ocb3"
version = "0.1.0"
@@ -3034,7 +2994,7 @@ dependencies = [
"oci-spec",
"olpc-cjson",
"regex",
"reqwest",
"reqwest 0.12.8",
"serde",
"serde_json",
"sha2 0.10.9",
@@ -3056,15 +3016,15 @@ dependencies = [
"regex",
"serde",
"serde_json",
"strum",
"strum_macros",
"strum 0.27.1",
"strum_macros 0.27.1",
"thiserror 2.0.18",
]
[[package]]
name = "ocicrypt-rs"
version = "0.1.0"
source = "git+https://github.com/confidential-containers/guest-components?rev=026694d44d4ec483465d2fa5f80a0376166b174d#026694d44d4ec483465d2fa5f80a0376166b174d"
source = "git+https://github.com/confidential-containers/guest-components?tag=v0.18.0#ab95914ac84c32a43102463cc0ae330710af47be"
dependencies = [
"anyhow",
"base64 0.22.1",
@@ -3722,7 +3682,7 @@ dependencies = [
[[package]]
name = "protos"
version = "0.1.0"
source = "git+https://github.com/confidential-containers/guest-components?rev=026694d44d4ec483465d2fa5f80a0376166b174d#026694d44d4ec483465d2fa5f80a0376166b174d"
source = "git+https://github.com/confidential-containers/guest-components?tag=v0.18.0#ab95914ac84c32a43102463cc0ae330710af47be"
dependencies = [
"prost 0.14.3",
"tonic",
@@ -4083,6 +4043,38 @@ dependencies = [
"windows-registry",
]
[[package]]
name = "reqwest"
version = "0.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ab3f43e3283ab1488b624b44b0e988d0acea0b3214e694730a055cb6b2efa801"
dependencies = [
"base64 0.22.1",
"bytes",
"futures-core",
"http 1.1.0",
"http-body 1.0.1",
"http-body-util",
"hyper 1.6.0",
"hyper-util",
"js-sys",
"log",
"percent-encoding",
"pin-project-lite",
"serde",
"serde_json",
"serde_urlencoded",
"sync_wrapper",
"tokio",
"tower",
"tower-http",
"tower-service",
"url",
"wasm-bindgen",
"wasm-bindgen-futures",
"web-sys",
]
[[package]]
name = "rfc6979"
version = "0.4.0"
@@ -4147,9 +4139,9 @@ dependencies = [
[[package]]
name = "rsa"
version = "0.9.9"
version = "0.9.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "40a0376c50d0358279d9d643e4bf7b7be212f1f4ff1da9070a7b54d22ef75c88"
checksum = "b8573f03f5883dcaebdfcf4725caa1ecb9c15b2ef50c43a07b816e06799bb12d"
dependencies = [
"const-oid",
"digest 0.10.7",
@@ -4200,12 +4192,6 @@ dependencies = [
"serde_json",
]
[[package]]
name = "rustc-demangle"
version = "0.1.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "719b953e2095829ee67db738b3bfa9fa368c94900df327b3f07fe6e794d2fe1f"
[[package]]
name = "rustc-hash"
version = "1.1.0"
@@ -4456,9 +4442,9 @@ checksum = "61697e0a1c7e512e84a621326239844a24d8207b4669b41bc18b32ea5cbf988b"
[[package]]
name = "sequoia-openpgp"
version = "2.1.0"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0e334ce3ec5b9b47d86a80563b3ecec435f59acf37e86058b3b686a42c5a2ba"
checksum = "0620e44a7d514adf7df87b44db235f13b81fed7ddc265adb26f014d42626ac47"
dependencies = [
"aes",
"aes-gcm",
@@ -4499,7 +4485,7 @@ dependencies = [
"p256",
"p384",
"p521",
"rand 0.8.5",
"rand 0.9.2",
"rand_core 0.6.4",
"regex",
"regex-syntax",
@@ -4618,9 +4604,9 @@ dependencies = [
[[package]]
name = "serde_spanned"
version = "1.0.4"
version = "1.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8bbf91e5a4d6315eee45e704372590b30e260ee83af6639d64557f51b067776"
checksum = "6662b5879511e06e8999a8a235d848113e942c9124f211511b16466ee2995f26"
dependencies = [
"serde_core",
]
@@ -4790,7 +4776,7 @@ dependencies = [
"pkcs8",
"rand 0.8.5",
"regex",
"reqwest",
"reqwest 0.12.8",
"rsa",
"rustls-pki-types",
"rustls-webpki",
@@ -4915,12 +4901,12 @@ dependencies = [
[[package]]
name = "socket2"
version = "0.6.0"
version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "233504af464074f9d066d7b5416c5f9b894a5862a6506e306f7b816cdd6f1807"
checksum = "3a766e1110788c36f4fa1c2b71b387a7815aa65f88ce0229841826633d93723e"
dependencies = [
"libc",
"windows-sys 0.59.0",
"windows-sys 0.61.2",
]
[[package]]
@@ -4968,8 +4954,14 @@ name = "strum"
version = "0.27.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f64def088c51c9510a8579e3c5d67c65349dcf755e5479ad3d010aa6454e2c32"
[[package]]
name = "strum"
version = "0.28.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9628de9b8791db39ceda2b119bbe13134770b56c138ec1d3af810d045c04f9bd"
dependencies = [
"strum_macros",
"strum_macros 0.28.0",
]
[[package]]
@@ -4985,6 +4977,18 @@ dependencies = [
"syn 2.0.87",
]
[[package]]
name = "strum_macros"
version = "0.28.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ab85eea0270ee17587ed4156089e10b9e6880ee688791d45a905f5b1ca36f664"
dependencies = [
"heck 0.5.0",
"proc-macro2",
"quote",
"syn 2.0.87",
]
[[package]]
name = "subprocess"
version = "0.2.9"
@@ -5268,29 +5272,26 @@ dependencies = [
[[package]]
name = "tokio"
version = "1.47.1"
version = "1.51.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "89e49afdadebb872d3145a5638b59eb0691ea23e46ca484037cfab3b76b95038"
checksum = "2bd1c4c0fc4a7ab90fc15ef6daaa3ec3b893f004f915f2392557ed23237820cd"
dependencies = [
"backtrace",
"bytes",
"io-uring",
"libc",
"mio",
"parking_lot",
"pin-project-lite",
"signal-hook-registry",
"slab",
"socket2 0.6.0",
"socket2 0.6.3",
"tokio-macros",
"windows-sys 0.59.0",
"windows-sys 0.61.2",
]
[[package]]
name = "tokio-macros"
version = "2.5.0"
version = "2.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e06d43f1345a3bcd39f6a56dbb7dcab2ba47e68e8ac134855e7e2bdbaf8cab8"
checksum = "385a6cb71ab9ab790c5fe8d67f1645e6c450a7ce006a33de03daa956cf70a496"
dependencies = [
"proc-macro2",
"quote",
@@ -5321,9 +5322,9 @@ dependencies = [
[[package]]
name = "tokio-util"
version = "0.7.17"
version = "0.7.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2efa149fe76073d6e8fd97ef4f4eca7b67f599660115591483572e406e165594"
checksum = "9ae9cec805b01e8fc3fd2fe289f89149a9b66dd16786abd8b19cfa7b48cb0098"
dependencies = [
"bytes",
"futures-core",
@@ -5356,17 +5357,17 @@ dependencies = [
[[package]]
name = "toml"
version = "0.9.11+spec-1.1.0"
version = "1.1.2+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f3afc9a848309fe1aaffaed6e1546a7a14de1f935dc9d89d32afd9a44bab7c46"
checksum = "81f3d15e84cbcd896376e6730314d59fb5a87f31e4b038454184435cd57defee"
dependencies = [
"indexmap 2.13.0",
"serde_core",
"serde_spanned",
"toml_datetime 0.7.5+spec-1.1.0",
"toml_datetime 1.1.1+spec-1.1.0",
"toml_parser",
"toml_writer",
"winnow",
"winnow 1.0.1",
]
[[package]]
@@ -5377,9 +5378,9 @@ checksum = "22cddaf88f4fbc13c51aebbf5f8eceb5c7c5a9da2ac40a13519eb5b0a0e8f11c"
[[package]]
name = "toml_datetime"
version = "0.7.5+spec-1.1.0"
version = "1.1.1+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92e1cfed4a3038bc5a127e35a2d360f145e1f4b971b551a2ba5fd7aedf7e1347"
checksum = "3165f65f62e28e0115a00b2ebdd37eb6f3b641855f9d636d3cd4103767159ad7"
dependencies = [
"serde_core",
]
@@ -5392,23 +5393,23 @@ checksum = "41fe8c660ae4257887cf66394862d21dbca4a6ddd26f04a3560410406a2f819a"
dependencies = [
"indexmap 2.13.0",
"toml_datetime 0.6.11",
"winnow",
"winnow 0.7.14",
]
[[package]]
name = "toml_parser"
version = "1.0.6+spec-1.1.0"
version = "1.1.2+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a3198b4b0a8e11f09dd03e133c0280504d0801269e9afa46362ffde1cbeebf44"
checksum = "a2abe9b86193656635d2411dc43050282ca48aa31c2451210f4202550afb7526"
dependencies = [
"winnow",
"winnow 1.0.1",
]
[[package]]
name = "toml_writer"
version = "1.0.6+spec-1.1.0"
version = "1.1.1+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ab16f14aed21ee8bfd8ec22513f7287cd4a91aa92e44edfe2c17ddd004e92607"
checksum = "756daf9b1013ebe47a8776667b466417e2d4c5679d441c26230efd9ef78692db"
[[package]]
name = "tonic"
@@ -5429,7 +5430,7 @@ dependencies = [
"hyper-util",
"percent-encoding",
"pin-project",
"socket2 0.6.0",
"socket2 0.6.3",
"sync_wrapper",
"tokio",
"tokio-stream",
@@ -5469,6 +5470,24 @@ dependencies = [
"tracing",
]
[[package]]
name = "tower-http"
version = "0.6.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d4e6559d53cc268e5031cd8429d05415bc4cb4aefc4aa5d6cc35fbf5b924a1f8"
dependencies = [
"bitflags 2.6.0",
"bytes",
"futures-util",
"http 1.1.0",
"http-body 1.0.1",
"iri-string",
"pin-project-lite",
"tower",
"tower-layer",
"tower-service",
]
[[package]]
name = "tower-layer"
version = "0.3.3"
@@ -5483,9 +5502,9 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
[[package]]
name = "tracing"
version = "0.1.41"
version = "0.1.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "784e0ac535deb450455cbfa28a6f0df145ea1bb7ae51b821cf5e7927fdcfbdd0"
checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
dependencies = [
"log",
"pin-project-lite",
@@ -5495,9 +5514,9 @@ dependencies = [
[[package]]
name = "tracing-attributes"
version = "0.1.29"
version = "0.1.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1b1ffbcf9c6f6b99d386e7444eb608ba646ae452a36b39737deb9663b610f662"
checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da"
dependencies = [
"proc-macro2",
"quote",
@@ -5506,9 +5525,9 @@ dependencies = [
[[package]]
name = "tracing-core"
version = "0.1.34"
version = "0.1.36"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9d12581f227e93f094d3af2ae690a574abb8a2b9b7a96e7cfe9647b2b617678"
checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a"
dependencies = [
"once_cell",
]
@@ -5894,7 +5913,7 @@ version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf221c93e13a30d793f7645a0e7762c55d169dbb0a49671918a2319d289b10bb"
dependencies = [
"windows-sys 0.59.0",
"windows-sys 0.48.0",
]
[[package]]
@@ -6037,6 +6056,15 @@ dependencies = [
"windows-targets 0.52.6",
]
[[package]]
name = "windows-sys"
version = "0.61.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
dependencies = [
"windows-link",
]
[[package]]
name = "windows-targets"
version = "0.42.2"
@@ -6224,6 +6252,12 @@ dependencies = [
"memchr",
]
[[package]]
name = "winnow"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09dac053f1cd375980747450bfc7250c264eaae0583872e845c0c7cd578872b5"
[[package]]
name = "wit-bindgen"
version = "0.51.0"

View File

@@ -41,7 +41,7 @@ serde = { version = "1.0.131", features = ["derive"] }
serde_json = "1.0.73"
# Image pull/unpack
image-rs = { git = "https://github.com/confidential-containers/guest-components", rev = "026694d44d4ec483465d2fa5f80a0376166b174d", features = [
image-rs = { git = "https://github.com/confidential-containers/guest-components", tag = "v0.18.0", features = [
"oci-client-rustls",
"signature-cosign-rustls",
] }

View File

@@ -166,4 +166,14 @@ impl yaml::K8sResource for CronJob {
fn get_sysctls(&self) -> Vec<pod::Sysctl> {
yaml::get_sysctls(&self.spec.jobTemplate.spec.template.spec.securityContext)
}
fn get_pod_security_context(&self) -> Option<&pod::PodSecurityContext> {
self.spec
.jobTemplate
.spec
.template
.spec
.securityContext
.as_ref()
}
}

View File

@@ -167,4 +167,8 @@ impl yaml::K8sResource for DaemonSet {
fn get_sysctls(&self) -> Vec<pod::Sysctl> {
yaml::get_sysctls(&self.spec.template.spec.securityContext)
}
fn get_pod_security_context(&self) -> Option<&pod::PodSecurityContext> {
self.spec.template.spec.securityContext.as_ref()
}
}

View File

@@ -178,4 +178,8 @@ impl yaml::K8sResource for Deployment {
fn get_sysctls(&self) -> Vec<pod::Sysctl> {
yaml::get_sysctls(&self.spec.template.spec.securityContext)
}
fn get_pod_security_context(&self) -> Option<&pod::PodSecurityContext> {
self.spec.template.spec.securityContext.as_ref()
}
}

View File

@@ -167,6 +167,10 @@ impl yaml::K8sResource for Job {
fn get_sysctls(&self) -> Vec<pod::Sysctl> {
yaml::get_sysctls(&self.spec.template.spec.securityContext)
}
fn get_pod_security_context(&self) -> Option<&pod::PodSecurityContext> {
self.spec.template.spec.securityContext.as_ref()
}
}
pub fn pod_name_regex(job_name: String) -> String {

View File

@@ -114,10 +114,12 @@ pub fn get_mount_and_storage(
if let Some(emptyDir) = &yaml_volume.emptyDir {
let settings_volumes = &settings.volumes;
let volume = match emptyDir.medium.as_deref() {
Some("Memory") => &settings_volumes.emptyDir_memory,
_ if settings.cluster_config.encrypted_emptydir => &settings_volumes.emptyDir_encrypted,
_ => &settings_volumes.emptyDir,
let (volume, block_encrypted_emptydir) = match emptyDir.medium.as_deref() {
Some("Memory") => (&settings_volumes.emptyDir_memory, false),
_ if settings.cluster_config.encrypted_emptydir => {
(&settings_volumes.emptyDir_encrypted, true)
}
_ => (&settings_volumes.emptyDir, false),
};
get_empty_dir_mount_and_storage(
@@ -127,6 +129,7 @@ pub fn get_mount_and_storage(
yaml_mount,
volume,
pod_security_context,
block_encrypted_emptydir,
);
} else if yaml_volume.persistentVolumeClaim.is_some() || yaml_volume.azureFile.is_some() {
get_shared_bind_mount(yaml_mount, p_mounts, "rprivate", "rw");
@@ -150,18 +153,42 @@ fn get_empty_dir_mount_and_storage(
yaml_mount: &pod::VolumeMount,
settings_empty_dir: &settings::EmptyDirVolume,
pod_security_context: &Option<pod::PodSecurityContext>,
block_encrypted_emptydir: bool,
) {
debug!("Settings emptyDir: {:?}", settings_empty_dir);
if yaml_mount.subPathExpr.is_none() {
let mut options = settings_empty_dir.options.clone();
if let Some(gid) = pod_security_context.as_ref().and_then(|sc| sc.fsGroup) {
// This matches the runtime behavior of only setting the fsgid if the mountpoint GID is not 0.
// https://github.com/kata-containers/kata-containers/blob/b69da5f3ba8385c5833b31db41a846a203812675/src/runtime/virtcontainers/kata_agent.go#L1602-L1607
if gid != 0 {
options.push(format!("fsgid={gid}"));
// Pod fsGroup in policy must mirror how the shim encodes it on Storage:
// - block-encrypted host emptyDirs become virtio-blk/scsi volumes; the runtime sets
// Storage.fs_group from mount metadata (handleDeviceBlockVolume in kata_agent.go).
// - shared-fs / guest-local emptyDirs use Storage.options: the runtime appends
// fsgid=<host GID> when the volume is not root-owned (handleEphemeralStorage and
// handleLocalStorage in kata_agent.go). Genpolicy uses pod fsGroup when non-zero as
// the usual kubelet-applied GID for that stat.
let pod_gid = pod_security_context.as_ref().and_then(|sc| sc.fsGroup);
let fs_group = if block_encrypted_emptydir {
match pod_gid {
Some(gid) if gid > 0 => protobuf::MessageField::some(agent::FSGroup {
group_id: u32::try_from(gid).unwrap_or_else(|_| {
panic!(
"get_empty_dir_mount_and_storage: securityContext.fsGroup {gid} \
must be <= {}",
u32::MAX
)
}),
..Default::default()
}),
_ => protobuf::MessageField::none(),
}
}
} else {
if let Some(gid) = pod_gid {
if gid != 0 {
options.push(format!("fsgid={gid}"));
}
}
protobuf::MessageField::none()
};
storages.push(agent::Storage {
driver: settings_empty_dir.driver.clone(),
driver_options: settings_empty_dir.driver_options.clone(),
@@ -173,7 +200,7 @@ fn get_empty_dir_mount_and_storage(
} else {
settings_empty_dir.mount_point.clone()
},
fs_group: protobuf::MessageField::none(),
fs_group,
shared: settings_empty_dir.shared,
special_fields: ::protobuf::SpecialFields::new(),
});

View File

@@ -937,6 +937,10 @@ impl yaml::K8sResource for Pod {
fn get_sysctls(&self) -> Vec<Sysctl> {
yaml::get_sysctls(&self.spec.securityContext)
}
fn get_pod_security_context(&self) -> Option<&PodSecurityContext> {
self.spec.securityContext.as_ref()
}
}
impl Container {

View File

@@ -971,6 +971,16 @@ impl AgentPolicy {
);
}
yaml::apply_pod_fs_group_and_supplemental_groups(
&mut process,
resource.get_pod_security_context(),
is_pause_container,
);
debug!(
"get_container_process: after apply_pod_fs_group_and_supplemental_groups: User = {:?}",
&process.User
);
///////////////////////////////////////////////////////////////////////////////////////
// Container-level settings from user's YAML.
yaml_container.get_process_fields(&mut process);

View File

@@ -128,4 +128,8 @@ impl yaml::K8sResource for ReplicaSet {
fn get_sysctls(&self) -> Vec<pod::Sysctl> {
yaml::get_sysctls(&self.spec.template.spec.securityContext)
}
fn get_pod_security_context(&self) -> Option<&pod::PodSecurityContext> {
self.spec.template.spec.securityContext.as_ref()
}
}

View File

@@ -131,4 +131,8 @@ impl yaml::K8sResource for ReplicationController {
fn get_sysctls(&self) -> Vec<pod::Sysctl> {
yaml::get_sysctls(&self.spec.template.spec.securityContext)
}
fn get_pod_security_context(&self) -> Option<&pod::PodSecurityContext> {
self.spec.template.spec.securityContext.as_ref()
}
}

View File

@@ -211,6 +211,10 @@ impl yaml::K8sResource for StatefulSet {
fn get_sysctls(&self) -> Vec<pod::Sysctl> {
yaml::get_sysctls(&self.spec.template.spec.securityContext)
}
fn get_pod_security_context(&self) -> Option<&pod::PodSecurityContext> {
self.spec.template.spec.securityContext.as_ref()
}
}
impl StatefulSet {

View File

@@ -107,6 +107,10 @@ pub trait K8sResource {
// for some of the K8s resource types.
}
fn get_pod_security_context(&self) -> Option<&pod::PodSecurityContext> {
None
}
fn get_sysctls(&self) -> Vec<pod::Sysctl> {
vec![]
}
@@ -388,6 +392,39 @@ fn handle_unused_field(path: &str, silent_unsupported_fields: bool) {
}
}
/// Applies pod `fsGroup` and `supplementalGroups` to `AdditionalGids`.
pub fn apply_pod_fs_group_and_supplemental_groups(
process: &mut policy::KataProcess,
security_context: Option<&pod::PodSecurityContext>,
is_pause_container: bool,
) {
if is_pause_container {
return;
}
let Some(context) = security_context else {
return;
};
if let Some(fs_group) = context.fsGroup {
let gid: u32 = fs_group.try_into().unwrap();
process.User.AdditionalGids.insert(gid);
debug!(
"apply_pod_fs_group_and_supplemental_groups: inserted fs_group = {gid} into AdditionalGids, User = {:?}",
&process.User
);
}
if let Some(supplemental_groups) = &context.supplementalGroups {
supplemental_groups.iter().for_each(|g| {
process.User.AdditionalGids.insert(*g);
});
debug!(
"apply_pod_fs_group_and_supplemental_groups: inserted supplementalGroups = {:?} into AdditionalGids, User = {:?}",
&supplemental_groups, &process.User
);
}
}
pub fn get_process_fields(
process: &mut policy::KataProcess,
must_check_passwd: &mut bool,
@@ -447,27 +484,6 @@ pub fn get_process_fields(
*must_check_passwd = false;
}
if !is_pause_container {
if let Some(fs_group) = context.fsGroup {
let gid = fs_group.try_into().unwrap();
process.User.AdditionalGids.insert(gid);
debug!(
"get_process_fields: inserted fs_group = {gid} into AdditionalGids, User = {:?}",
&process.User
);
}
if let Some(supplemental_groups) = &context.supplementalGroups {
supplemental_groups.iter().for_each(|g| {
process.User.AdditionalGids.insert(*g);
});
debug!(
"get_process_fields: inserted supplementalGroups = {:?} into AdditionalGids, User = {:?}",
&supplemental_groups, &process.User
);
}
}
if let Some(allow) = context.allowPrivilegeEscalation {
process.NoNewPrivileges = !allow
}

View File

@@ -345,12 +345,12 @@
"driver_options": [
"encryption_key=ephemeral"
],
"fs_group": null,
"fs_group": {
"group_id": 1000
},
"fstype": "ext4",
"mount_point": "/run/kata-containers/sandbox/storage/MDAvMDA=",
"options": [
"fsgid=1000"
],
"options": [],
"source": "00/00",
"shared": true
}

View File

@@ -4362,9 +4362,9 @@ checksum = "b6bc1c9ce2b5135ac7f93c72918fc37feb872bdc6a5533a8b85eb4b86bfdae52"
[[package]]
name = "tracing"
version = "0.1.41"
version = "0.1.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "784e0ac535deb450455cbfa28a6f0df145ea1bb7ae51b821cf5e7927fdcfbdd0"
checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
dependencies = [
"pin-project-lite",
"tracing-attributes",
@@ -4373,9 +4373,9 @@ dependencies = [
[[package]]
name = "tracing-attributes"
version = "0.1.29"
version = "0.1.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1b1ffbcf9c6f6b99d386e7444eb608ba646ae452a36b39737deb9663b610f662"
checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da"
dependencies = [
"proc-macro2",
"quote",
@@ -4384,9 +4384,9 @@ dependencies = [
[[package]]
name = "tracing-core"
version = "0.1.34"
version = "0.1.36"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9d12581f227e93f094d3af2ae690a574abb8a2b9b7a96e7cfe9647b2b617678"
checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a"
dependencies = [
"once_cell",
]

View File

@@ -45,7 +45,7 @@ kbs_set_allow_all_resources() {
kbs_set_default_policy() {
kbs_set_resources_policy \
"${COCO_KBS_DIR}/src/policy_engine/opa/default_policy.rego"
"${COCO_KBS_DIR}/sample_policies/default.rego"
}
# Set "deny all" policy to resources.

View File

@@ -22,10 +22,17 @@ setup() {
pod_name="sharevol-kata"
pod_logs_file=""
setup_common || die "setup_common failed"
yaml_file="${pod_config_dir}/pod-empty-dir.yaml"
# Add policy to yaml
policy_settings_dir="$(create_tmp_policy_settings_dir "${pod_config_dir}")"
add_requests_to_policy_settings "${policy_settings_dir}" "ReadStreamRequest"
}
@test "Empty dir volumes" {
local yaml_file
local mount_command
local dd_command
yaml_file="${pod_config_dir}/pod-empty-dir.yaml"
mount_command=(sh -c "mount | grep cache")
add_exec_to_policy_settings "${policy_settings_dir}" "${mount_command[@]}"
@@ -33,11 +40,9 @@ setup() {
dd_command=(sh -c "dd if=/dev/zero of=/tmp/cache/file1 bs=1M count=50; echo $?")
add_exec_to_policy_settings "${policy_settings_dir}" "${dd_command[@]}"
add_requests_to_policy_settings "${policy_settings_dir}" "ReadStreamRequest"
# Add policy to yaml
auto_generate_policy "${policy_settings_dir}" "${yaml_file}"
}
@test "Empty dir volumes" {
# Create the pod
kubectl create -f "${yaml_file}"
@@ -55,20 +60,25 @@ setup() {
local agnhost_name
local agnhost_version
local gid
local image
local logs
local pod_file
local pod_yaml
local pod_yaml_in
local uid
# This is a reproducer of k8s e2e "[sig-storage] EmptyDir volumes when FSGroup is specified [LinuxOnly] [NodeFeature:FSGroup] new files should be created with FSGroup ownership when container is non-root" test
pod_file="${pod_config_dir}/pod-empty-dir-fsgroup.yaml"
pod_yaml_in="${pod_config_dir}/pod-empty-dir-fsgroup.yaml.in"
pod_yaml="${pod_config_dir}/pod-empty-dir-fsgroup.yaml"
agnhost_name="${container_images_agnhost_name}"
agnhost_version="${container_images_agnhost_version}"
image="${agnhost_name}:${agnhost_version}"
export AGNHOST_IMAGE="${agnhost_name}:${agnhost_version}"
envsubst '${AGNHOST_IMAGE}' <"${pod_yaml_in}" >"${pod_yaml}"
# Add policy to yaml
auto_generate_policy "${policy_settings_dir}" "${pod_yaml}"
# Try to avoid timeout by prefetching the image.
sed -e "s#\${agnhost_image}#${image}#" "$pod_file" |\
kubectl create -f -
kubectl create -f "${pod_yaml}"
cmd="kubectl get pods ${pod_name} | grep Completed"
waitForProcess "${wait_time}" "${sleep_time}" "${cmd}"
@@ -90,6 +100,7 @@ setup() {
teardown() {
[ ! -f "$pod_logs_file" ] || rm -f "$pod_logs_file"
[[ -n "${pod_config_dir:-}" ]] && rm -f "${pod_config_dir}/pod-empty-dir-fsgroup.yaml"
delete_tmp_policy_settings_dir "${policy_settings_dir}"
teardown_common "${node}" "${node_start_time:-}"

View File

@@ -69,11 +69,11 @@ EOF"
## Cases for target container
### First container
container_name=$(kubectl exec $pod_name -c $first_container_name -- $env_command | grep CONTAINER_NAME)
container_name=$(kubectl exec $pod_name -c $first_container_name -- $env_command | grep CONTAINER_NAME | tr -d '\r')
[ "$container_name" == "CONTAINER_NAME=$first_container_name" ]
### Second container
container_name=$(kubectl exec $pod_name -c $second_container_name -- $env_command | grep CONTAINER_NAME)
container_name=$(kubectl exec $pod_name -c $second_container_name -- $env_command | grep CONTAINER_NAME | tr -d '\r')
[ "$container_name" == "CONTAINER_NAME=$second_container_name" ]
}

View File

@@ -16,7 +16,7 @@ shim_config_file="/opt/kata/share/defaults/kata-containers/configuration-${KATA_
check_and_skip() {
case "${KATA_HYPERVISOR}" in
qemu-tdx|qemu-coco-dev)
qemu-tdx|qemu-coco-dev|qemu-snp)
if [ "$(uname -m)" == "s390x" ]; then
skip "measured rootfs tests not implemented for s390x"
fi

View File

@@ -29,14 +29,6 @@ setup() {
envsubst < "${pod_yaml_in}" > "${pod_yaml}"
if [ "${TEE}" = "true" ]; then
kernel_params_annotation="io.katacontainers.config.hypervisor.kernel_params"
kernel_params_value="nvrc.smi.srs=1"
set_metadata_annotation "${pod_yaml}" \
"${kernel_params_annotation}" \
"${kernel_params_value}"
fi
policy_settings_dir="$(create_tmp_policy_settings_dir "${pod_config_dir}")"
add_requests_to_policy_settings "${policy_settings_dir}" "ReadStreamRequest"

View File

@@ -10,6 +10,7 @@ load "${BATS_TEST_DIRNAME}/confidential_common.sh"
export KATA_HYPERVISOR="${KATA_HYPERVISOR:-qemu-nvidia-gpu}"
# when using hostPath, ensure directory is writable by container user
export LOCAL_NIM_CACHE="/opt/nim/.cache"
SKIP_MULTI_GPU_TESTS=${SKIP_MULTI_GPU_TESTS:-false}

View File

@@ -35,15 +35,16 @@ setup() {
kubectl wait --for=condition=Ready --timeout=$timeout pod $pod_name
# Check PID from first container
# Strip \r — containers with tty: true return \r\n line endings
first_pid_container=$(kubectl exec $pod_name -c $first_container_name \
-- $ps_command | grep "/pause")
-- $ps_command | grep "/pause" | tr -d '\r')
# Verify that is not empty
check_first_pid=$(echo $first_pid_container | wc -l)
[ "$check_first_pid" == "1" ]
# Check PID from second container
second_pid_container=$(kubectl exec $pod_name -c $second_container_name \
-- $ps_command | grep "/pause")
-- $ps_command | grep "/pause" | tr -d '\r')
# Verify that is not empty
check_second_pid=$(echo $second_pid_container | wc -l)
[ "$check_second_pid" == "1" ]

View File

@@ -10,8 +10,6 @@ load "${BATS_TEST_DIRNAME}/lib.sh"
load "${BATS_TEST_DIRNAME}/tests_common.sh"
setup() {
[ "$(uname -m)" == "aarch64" ] && skip "See: https://github.com/kata-containers/kata-containers/issues/10928"
[[ "${KATA_HYPERVISOR}" == qemu-coco-dev* ]] && skip "Requires CPU hotplug which disabled by static_sandbox_resource_mgmt"
[[ "${KATA_HYPERVISOR}" == "qemu-tdx" ]] && skip "See: https://github.com/kata-containers/kata-containers/issues/12492"
setup_common || die "setup_common failed"
@@ -51,8 +49,6 @@ setup() {
}
teardown() {
[ "$(uname -m)" == "aarch64" ] && skip "See: https://github.com/kata-containers/kata-containers/issues/10928"
[[ "${KATA_HYPERVISOR}" == qemu-coco-dev* ]] && skip "Requires CPU hotplug which disabled by static_sandbox_resource_mgmt"
[[ "${KATA_HYPERVISOR}" == "qemu-tdx" ]] && skip "See: https://github.com/kata-containers/kata-containers/issues/12492"
for pod in "${pods[@]}"; do

View File

@@ -16,14 +16,18 @@ metadata:
# cc_init_data annotation will be added by genpolicy with CDH configuration
# from the custom default-initdata.toml created by create_nim_initdata_file()
spec:
# Explicit user/group/supplementary groups to support nydus guest-pull.
# See issue https://github.com/kata-containers/kata-containers/issues/11162 and
# other references to this issue in the genpolicy source folder.
securityContext:
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
supplementalGroups: [4, 20, 24, 25, 27, 29, 30, 44, 46]
restartPolicy: Never
runtimeClassName: kata
imagePullSecrets:
- name: ngc-secret-instruct
securityContext:
runAsUser: 0
runAsGroup: 0
fsGroup: 0
containers:
- name: ${POD_NAME_INSTRUCT}
image: nvcr.io/nim/meta/llama-3.1-8b-instruct:1.13.1

View File

@@ -14,10 +14,6 @@ spec:
runtimeClassName: kata
imagePullSecrets:
- name: ngc-secret-instruct
securityContext:
runAsUser: 0
runAsGroup: 0
fsGroup: 0
containers:
- name: ${POD_NAME_INSTRUCT}
image: nvcr.io/nim/meta/llama-3.1-8b-instruct:1.13.1

View File

@@ -16,15 +16,18 @@ metadata:
# cc_init_data annotation will be added by genpolicy with CDH configuration
# from the custom default-initdata.toml created by create_nim_initdata_file()
spec:
# Explicit user/group/supplementary groups to support nydus guest-pull.
# See issue https://github.com/kata-containers/kata-containers/issues/11162 and
# other references to this issue in the genpolicy source folder.
securityContext:
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
restartPolicy: Always
runtimeClassName: kata
serviceAccountName: default
imagePullSecrets:
- name: ngc-secret-embedqa
securityContext:
fsGroup: 0
runAsGroup: 0
runAsUser: 0
containers:
- name: ${POD_NAME_EMBEDQA}
image: nvcr.io/nim/nvidia/llama-3.2-nv-embedqa-1b-v2:1.10.1

View File

@@ -10,15 +10,16 @@ metadata:
labels:
app: ${POD_NAME_EMBEDQA}
spec:
# unlike the instruct manifest, this image needs securityContext to
# avoid NVML/GPU permission failures
securityContext:
runAsUser: 1000
runAsGroup: 1000
restartPolicy: Always
runtimeClassName: kata
serviceAccountName: default
imagePullSecrets:
- name: ngc-secret-embedqa
securityContext:
fsGroup: 0
runAsGroup: 0
runAsUser: 0
containers:
- name: ${POD_NAME_EMBEDQA}
image: nvcr.io/nim/nvidia/llama-3.2-nv-embedqa-1b-v2:1.10.1

View File

@@ -15,7 +15,7 @@ spec:
fsGroup: 123
containers:
- name: mounttest-container
image: ${agnhost_image}
image: ${AGNHOST_IMAGE}
command:
- /agnhost
args:
@@ -28,7 +28,7 @@ spec:
- name: emptydir-volume
mountPath: /test-volume
- name: mounttest-container-2
image: ${agnhost_image}
image: ${AGNHOST_IMAGE}
command:
- /agnhost
args:

View File

@@ -148,9 +148,7 @@ install_genpolicy_drop_ins() {
# 20-* OCI version overlay
if [[ "${KATA_HOST_OS:-}" == "cbl-mariner" ]]; then
cp "${examples_dir}/20-oci-1.2.0-drop-in.json" "${settings_d}/"
elif is_k3s_or_rke2; then
cp "${examples_dir}/20-oci-1.2.1-drop-in.json" "${settings_d}/"
elif is_nvidia_gpu_platform || [[ "${KATA_HYPERVISOR}" == "qemu-snp" ]] || [[ "${KATA_HYPERVISOR}" == "qemu-tdx" ]] || [[ -n "${CONTAINER_ENGINE_VERSION:-}" ]]; then
elif is_k3s_or_rke2 || is_nvidia_gpu_platform || [[ "${KATA_HYPERVISOR}" == "qemu-snp" ]] || [[ "${KATA_HYPERVISOR}" == "qemu-tdx" ]] || [[ -n "${CONTAINER_ENGINE_VERSION:-}" ]]; then
cp "${examples_dir}/20-oci-1.3.0-drop-in.json" "${settings_d}/"
fi

View File

@@ -68,12 +68,12 @@ install_userspace_components() {
libnvidia-decode libnvidia-fbc1 libnvidia-encode \
libnvidia-nscq libnvidia-compute nvidia-settings
# Needed for confidential-data-hub runtime dependencies
# Needed for confidential-data-hub and NVAT runtime dependencies
eval "${APT_INSTALL}" cryptsetup-bin dmsetup \
libargon2-1 e2fsprogs
libargon2-1 e2fsprogs libxml2
apt-mark hold cryptsetup-bin dmsetup libargon2-1 \
e2fsprogs
e2fsprogs libxml2
}
setup_apt_repositories() {

View File

@@ -224,6 +224,26 @@ chisseled_gpudirect() {
exit 1
}
chisseled_nvat() {
if [[ "${type}" != "confidential" ]]; then
return
fi
echo "nvidia: chisseling NVAT"
local libdir="lib/${machine_arch}-linux-gnu"
# NVAT shared library (bundled via coco-guest-components tarball)
cp -a "${stage_one}"/usr/local/lib/libnvat.so* "${libdir}"/.
# NVAT runtime dependencies (per ldd on attestation-agent)
cp -a "${stage_one}/${libdir}"/libxml2.so.2* "${libdir}"/.
cp -a "${stage_one}/${libdir}"/libstdc++.so.6* "${libdir}"/.
cp -a "${stage_one}/${libdir}"/liblzma.so.5* "${libdir}"/.
cp -a "${stage_one}/${libdir}"/libicuuc.so.* "${libdir}"/.
cp -a "${stage_one}/${libdir}"/libicudata.so.* "${libdir}"/.
}
setup_nvrc_init_symlinks() {
local nvrc="NVRC-${machine_arch}-unknown-linux-musl"
# make sure NVRC is the init process for the initrd and image case
@@ -358,7 +378,7 @@ coco_guest_components() {
local -r pause_dir="pause_bundle"
mkdir -p "${coco_bin_dir}"
cp -a "${stage_one}/${coco_bin_dir}"/attestation-agent "${coco_bin_dir}/."
cp -a "${stage_one}/${coco_bin_dir}"/attestation-agent-nv "${coco_bin_dir}/attestation-agent"
cp -a "${stage_one}/${coco_bin_dir}"/api-server-rest "${coco_bin_dir}/."
cp -a "${stage_one}/${coco_bin_dir}"/confidential-data-hub "${coco_bin_dir}/."
@@ -418,6 +438,7 @@ setup_nvidia_gpu_rootfs_stage_two() {
done
coco_guest_components
chisseled_nvat
fi
compress_rootfs

View File

@@ -7,17 +7,17 @@
FROM golang:1.24-alpine AS nydus-binary-downloader
# Keep the version here aligned with "ndyus-snapshotter.version"
# in versions.yaml
ARG NYDUS_SNAPSHOTTER_VERSION=v0.15.13
ARG NYDUS_SNAPSHOTTER_REPO=https://github.com/containerd/nydus-snapshotter
COPY versions.yaml /tmp/versions.yaml
RUN \
set -e && \
apk add --no-cache curl yq-go && \
NYDUS_SNAPSHOTTER_VERSION="$(yq eval -e '.externals.nydus-snapshotter.version | explode(.)' /tmp/versions.yaml)" && \
NYDUS_SNAPSHOTTER_REPO="$(yq eval -e '.externals.nydus-snapshotter.url | explode(.)' /tmp/versions.yaml)" && \
mkdir -p /opt/nydus-snapshotter && \
ARCH="$(uname -m)" && \
if [ "${ARCH}" = "x86_64" ]; then ARCH=amd64 ; fi && \
if [ "${ARCH}" = "aarch64" ]; then ARCH=arm64; fi && \
apk add --no-cache curl && \
curl -fOL --progress-bar "${NYDUS_SNAPSHOTTER_REPO}/releases/download/${NYDUS_SNAPSHOTTER_VERSION}/nydus-snapshotter-${NYDUS_SNAPSHOTTER_VERSION}-linux-${ARCH}.tar.gz" && \
tar xvzpf "nydus-snapshotter-${NYDUS_SNAPSHOTTER_VERSION}-linux-${ARCH}.tar.gz" -C /opt/nydus-snapshotter && \
rm "nydus-snapshotter-${NYDUS_SNAPSHOTTER_VERSION}-linux-${ARCH}.tar.gz"
@@ -47,12 +47,16 @@ RUN \
libc6-dev \
musl-tools && \
apt-get clean && rm -rf /var/lib/apt/lists/ && \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain ${RUST_TOOLCHAIN}
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain ${RUST_TOOLCHAIN} && \
rustup component add rustfmt clippy
WORKDIR /kata-deploy
# Build from the repository root so kata-deploy uses the root Cargo workspace:
# docker build -f tools/packaging/kata-deploy/Dockerfile .
WORKDIR /kata
# Copy standalone binary project
COPY binary /kata-deploy
COPY Cargo.toml Cargo.lock ./
COPY src ./src
COPY tools/packaging/kata-deploy/binary ./tools/packaging/kata-deploy/binary
# Install target and run tests based on architecture
# - AMD64/arm64: use musl for fully static binaries
@@ -88,28 +92,39 @@ RUN \
esac; \
echo "${rust_target}" > /tmp/rust_target
# Verify code formatting and run cargo check before tests and build
RUN \
set -e && \
rust_target="$(cat /tmp/rust_target)" && \
echo "Checking code formatting..." && \
cargo fmt -p kata-deploy --check && \
echo "Code formatting check passed!" && \
echo "Running cargo clippy with target ${rust_target}..." && \
cargo clippy -p kata-deploy --all-targets --all-features --release --locked --target "${rust_target}" -- -D warnings && \
echo "Cargo clippy passed!"
# Run tests using --test-threads=1 to prevent environment variable pollution between tests,
# and this is fine as we'll never ever have multiple binaries running at the same time.
RUN \
rust_target="$(cat /tmp/rust_target)"; \
echo "Running binary tests with target ${rust_target}..." && \
RUSTFLAGS="-D warnings" cargo test --target "${rust_target}" -- --test-threads=1 && \
RUSTFLAGS="-D warnings" cargo test -p kata-deploy --target "${rust_target}" -- --test-threads=1 && \
echo "All tests passed!"
RUN \
rust_target="$(cat /tmp/rust_target)"; \
echo "Building kata-deploy binary for ${rust_target}..." && \
RUSTFLAGS="-D warnings" cargo build --release --target "${rust_target}" && \
RUSTFLAGS="-D warnings" cargo build --release -p kata-deploy --target "${rust_target}" && \
mkdir -p /kata-deploy/bin && \
cp "/kata-deploy/target/${rust_target}/release/kata-deploy" /kata-deploy/bin/kata-deploy && \
cp "/kata/target/${rust_target}/release/kata-deploy" /kata-deploy/bin/kata-deploy && \
echo "Cleaning up build artifacts to save disk space..." && \
rm -rf /kata-deploy/target && \
rm -rf /kata/target && \
cargo clean
#### Extract kata artifacts
FROM alpine:3.22 AS artifact-extractor
ARG KATA_ARTIFACTS=kata-static.tar.zst
ARG KATA_ARTIFACTS=tools/packaging/kata-deploy/kata-static.tar.zst
ARG DESTINATION=/opt/kata-artifacts
COPY ${KATA_ARTIFACTS} /tmp/
@@ -222,11 +237,11 @@ COPY --from=runtime-assembler /output/lib/ /lib/
COPY --from=runtime-assembler /output/lib64/ /lib64/
# Copy nydus snapshotter
COPY nydus-snapshotter ${DESTINATION}/nydus-snapshotter
COPY tools/packaging/kata-deploy/nydus-snapshotter ${DESTINATION}/nydus-snapshotter
COPY --from=nydus-binary-downloader /opt/nydus-snapshotter/bin/containerd-nydus-grpc ${DESTINATION}/nydus-snapshotter/
COPY --from=nydus-binary-downloader /opt/nydus-snapshotter/bin/nydus-overlayfs ${DESTINATION}/nydus-snapshotter/
# Copy runtimeclasses and node-feature-rules
COPY node-feature-rules ${DESTINATION}/node-feature-rules
COPY tools/packaging/kata-deploy/node-feature-rules ${DESTINATION}/node-feature-rules
ENTRYPOINT ["/usr/bin/kata-deploy"]

View File

@@ -1,58 +1,38 @@
[package]
name = "kata-deploy"
version = "0.1.0"
authors.workspace = true
edition = "2021"
license.workspace = true
rust-version = "1.90.0"
authors = ["The Kata Containers community <kata-dev@lists.katacontainers.io>"]
license = "Apache-2.0"
[[bin]]
name = "kata-deploy"
path = "src/main.rs"
[dependencies]
# Error handling
anyhow = "1.0"
# Logging
log = "0.4"
anyhow.workspace = true
clap.workspace = true
env_logger = "0.10"
# Command line parsing
clap = { version = "4.5", features = ["derive"] }
# TOML parsing and manipulation
toml_edit = "0.22"
# YAML parsing and manipulation
serde_yaml = "0.9"
# Kubernetes API client
kube = { version = "2.0", features = ["runtime", "derive"] }
k8s-openapi = { version = "0.26", default-features = false, features = [
"v1_33",
] }
# System operations (using nsenter command instead of syscalls)
libc = "0.2"
# JSON serialization
serde_json = "1.0"
# File operations
walkdir = "2"
# String manipulation
regex = "1.10"
# Async runtime (required by kube-rs and for async main)
tokio = { version = "1.38", features = [
kube = { version = "2.0", features = ["runtime", "derive"] }
libc.workspace = true
log.workspace = true
regex.workspace = true
serde_json.workspace = true
serde_yaml = "0.9"
tokio = { workspace = true, features = [
"rt-multi-thread",
"macros",
"signal",
"time",
] }
toml_edit = "0.22"
walkdir = "2"
[dev-dependencies]
tempfile = "3.8"
rstest = "0.18"
rstest.workspace = true
serial_test.workspace = true
tempfile.workspace = true

View File

@@ -76,8 +76,12 @@ pub async fn install_artifacts(config: &Config, container_runtime: &str) -> Resu
// Create the installation directory if it doesn't exist
// fs::create_dir_all handles existing directories gracefully (returns Ok if already exists)
fs::create_dir_all(&config.host_install_dir)
.with_context(|| format!("Failed to create installation directory: {}", config.host_install_dir))?;
fs::create_dir_all(&config.host_install_dir).with_context(|| {
format!(
"Failed to create installation directory: {}",
config.host_install_dir
)
})?;
// Verify the path exists and is a directory (not a file)
let install_path = Path::new(&config.host_install_dir);
@@ -182,7 +186,11 @@ fn write_common_drop_ins(
let kernel_params_content = generate_kernel_params_drop_in(config, shim)?;
if !kernel_params_content.is_empty() {
info!(" - Kernel parameters: configured");
write_drop_in_file(config_d_dir, "30-kernel-params.toml", &kernel_params_content)?;
write_drop_in_file(
config_d_dir,
"30-kernel-params.toml",
&kernel_params_content,
)?;
}
Ok(())
@@ -239,7 +247,12 @@ fn install_custom_runtime_configs(config: &Config, container_runtime: &str) -> R
}
// Generate the common drop-in files (shared with standard runtimes)
write_common_drop_ins(config, &runtime.base_config, &config_d_dir, container_runtime)?;
write_common_drop_ins(
config,
&runtime.base_config,
&config_d_dir,
container_runtime,
)?;
// Copy user-provided drop-in file if provided (at 50-overrides.toml)
if let Some(ref drop_in_src) = runtime.drop_in_file {
@@ -328,10 +341,10 @@ fn copy_artifacts(src: &str, dst: &str) -> Result<()> {
if let Ok(rel) = link_target.strip_prefix(src_path) {
Path::new(dst).join(rel)
} else {
link_target.into()
link_target
}
} else {
link_target.into()
link_target
};
if let Some(parent) = dst_path.parent() {
@@ -342,8 +355,12 @@ fn copy_artifacts(src: &str, dst: &str) -> Result<()> {
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
Err(e) => return Err(e.into()),
}
std::os::unix::fs::symlink(&new_target, &dst_path)
.with_context(|| format!("Failed to create symlink {:?} -> {:?}", dst_path, new_target))?;
std::os::unix::fs::symlink(&new_target, &dst_path).with_context(|| {
format!(
"Failed to create symlink {:?} -> {:?}",
dst_path, new_target
)
})?;
} else {
if let Some(parent) = dst_path.parent() {
fs::create_dir_all(parent)?;
@@ -368,7 +385,7 @@ fn copy_artifacts(src: &str, dst: &str) -> Result<()> {
}
fn set_executable_permissions(dir: &str) -> Result<()> {
let bin_paths = vec!["bin", "runtime-rs/bin"];
let bin_paths = ["bin", "runtime-rs/bin"];
for bin_path in bin_paths.iter() {
let bin_dir = Path::new(dir).join(bin_path);
@@ -459,13 +476,18 @@ fn setup_runtime_directory(config: &Config, shim: &str) -> Result<()> {
// fs::copy follows symlinks and would write to the wrong location
let dest_path = Path::new(&dest_config_file);
if dest_path.exists() || dest_path.is_symlink() {
fs::remove_file(&dest_config_file)
.with_context(|| format!("Failed to remove existing config: {}", dest_config_file))?;
fs::remove_file(&dest_config_file).with_context(|| {
format!("Failed to remove existing config: {}", dest_config_file)
})?;
}
// Copy the base config file
fs::copy(&original_config_file, &dest_config_file)
.with_context(|| format!("Failed to copy config: {} -> {}", original_config_file, dest_config_file))?;
fs::copy(&original_config_file, &dest_config_file).with_context(|| {
format!(
"Failed to copy config: {} -> {}",
original_config_file, dest_config_file
)
})?;
// Add warning comment to inform users about drop-in files
add_kata_deploy_warning(Path::new(&dest_config_file))?;
@@ -484,8 +506,9 @@ fn remove_runtime_directory(config: &Config, shim: &str) -> Result<()> {
);
if Path::new(&runtime_config_dir).exists() {
fs::remove_dir_all(&runtime_config_dir)
.with_context(|| format!("Failed to remove runtime directory: {}", runtime_config_dir))?;
fs::remove_dir_all(&runtime_config_dir).with_context(|| {
format!("Failed to remove runtime directory: {}", runtime_config_dir)
})?;
log::debug!("Removed runtime directory: {}", runtime_config_dir);
}
@@ -514,7 +537,8 @@ async fn configure_shim_config(config: &Config, shim: &str, container_runtime: &
);
let config_d_dir = format!("{}/config.d", runtime_config_dir);
let kata_config_file = Path::new(&runtime_config_dir).join(format!("configuration-{shim}.toml"));
let kata_config_file =
Path::new(&runtime_config_dir).join(format!("configuration-{shim}.toml"));
// The configuration file (symlink) should exist after setup_runtime_directory()
if !kata_config_file.exists() {
@@ -545,8 +569,8 @@ async fn configure_shim_config(config: &Config, shim: &str, container_runtime: &
/// Reads the current value (defaulting to "false" if not found), and if it's not "true",
/// logs the update and sets it to "true".
fn set_toml_bool_to_true(config_file: &Path, path: &str) -> Result<()> {
let current_value = toml_utils::get_toml_value(config_file, path)
.unwrap_or_else(|_| "false".to_string());
let current_value =
toml_utils::get_toml_value(config_file, path).unwrap_or_else(|_| "false".to_string());
if current_value != "true" {
log::debug!(
"Updating {} in {}: old=\"{}\" new=\"true\"",
@@ -636,8 +660,9 @@ fn get_hypervisor_path(config: &Config, shim: &str) -> Result<String> {
if is_qemu_shim(shim) {
// For QEMU shims, use the wrapper script that adds firmware paths
// create_qemu_wrapper_script always returns Some for QEMU shims
create_qemu_wrapper_script(config, shim)?
.ok_or_else(|| anyhow::anyhow!("QEMU wrapper script should always be created for QEMU shims"))
create_qemu_wrapper_script(config, shim)?.ok_or_else(|| {
anyhow::anyhow!("QEMU wrapper script should always be created for QEMU shims")
})
} else {
// For non-QEMU shims, use the appropriate hypervisor binary
let binary = match shim {
@@ -673,20 +698,41 @@ fn generate_installation_prefix_drop_in(config: &Config, shim: &str) -> Result<S
}
// Common paths for all hypervisors
content.push_str(&format!("kernel = \"{}/share/kata-containers/vmlinux.container\"\n", config.dest_dir));
content.push_str(&format!("image = \"{}/share/kata-containers/kata-containers.img\"\n", config.dest_dir));
content.push_str(&format!("initrd = \"{}/share/kata-containers/kata-containers-initrd.img\"\n", config.dest_dir));
content.push_str(&format!(
"kernel = \"{}/share/kata-containers/vmlinux.container\"\n",
config.dest_dir
));
content.push_str(&format!(
"image = \"{}/share/kata-containers/kata-containers.img\"\n",
config.dest_dir
));
content.push_str(&format!(
"initrd = \"{}/share/kata-containers/kata-containers-initrd.img\"\n",
config.dest_dir
));
// QEMU-specific paths (firmware is only relevant for QEMU)
if is_qemu_shim(shim) {
content.push_str(&format!("firmware = \"{}/share/kata-containers/firmware/\"\n", config.dest_dir));
content.push_str(&format!("firmware_volume = \"{}/share/kata-containers/firmware/\"\n", config.dest_dir));
content.push_str(&format!(
"firmware = \"{}/share/kata-containers/firmware/\"\n",
config.dest_dir
));
content.push_str(&format!(
"firmware_volume = \"{}/share/kata-containers/firmware/\"\n",
config.dest_dir
));
}
// Firecracker-specific paths (jailer is only for Firecracker)
if shim == "fc" || shim == "firecracker" {
content.push_str(&format!("jailer_path = \"{}/bin/jailer\"\n", config.dest_dir));
content.push_str(&format!("valid_jailer_paths = [\"{}/bin/jailer\"]\n", config.dest_dir));
content.push_str(&format!(
"jailer_path = \"{}/bin/jailer\"\n",
config.dest_dir
));
content.push_str(&format!(
"valid_jailer_paths = [\"{}/bin/jailer\"]\n",
config.dest_dir
));
}
Ok(content)
@@ -738,16 +784,14 @@ fn get_proxy_value_for_shim(proxy_var: &Option<String>, shim: &str) -> Option<St
match proxy_var {
Some(proxy) if !proxy.is_empty() && proxy.contains('=') => {
// Per-shim format: "qemu-tdx=http://proxy:8080;qemu-snp=http://proxy2:8080"
proxy
.split(';')
.find_map(|m| {
let parts: Vec<&str> = m.splitn(2, '=').collect();
if parts.len() == 2 && parts[0] == shim {
Some(parts[1].to_string())
} else {
None
}
})
proxy.split(';').find_map(|m| {
let parts: Vec<&str> = m.splitn(2, '=').collect();
if parts.len() == 2 && parts[0] == shim {
Some(parts[1].to_string())
} else {
None
}
})
}
Some(proxy) if !proxy.is_empty() => Some(proxy.clone()),
_ => None,
@@ -770,8 +814,8 @@ fn read_base_kernel_params(config: &Config, shim: &str) -> Result<String> {
}
let kernel_params_path = format!("hypervisor.{}.kernel_params", hypervisor_name);
let base_params = toml_utils::get_toml_value(config_path, &kernel_params_path)
.unwrap_or_default();
let base_params =
toml_utils::get_toml_value(config_path, &kernel_params_path).unwrap_or_default();
// Remove surrounding quotes if present
Ok(base_params.trim_matches('"').to_string())
@@ -1100,5 +1144,4 @@ mod tests {
"following the symlink should yield the real content"
);
}
}

View File

@@ -91,7 +91,8 @@ pub async fn configure_snapshotter(
Some(id) => id.as_str(),
None => containerd::get_containerd_pluginid(&paths.config_file)?,
};
let pluginid = containerd::pluginid_for_snapshotter_annotations(runtime_plugin_id, &paths.config_file)?;
let pluginid =
containerd::pluginid_for_snapshotter_annotations(runtime_plugin_id, &paths.config_file)?;
let configuration_file: std::path::PathBuf = if paths.use_drop_in {
// Only add /host prefix if path is not in /etc/containerd (which is mounted from host)

View File

@@ -189,14 +189,11 @@ impl Config {
let default_shim_for_arch = get_arch_var("DEFAULT_SHIM", "qemu", &arch);
// Only use arch-specific variable for allowed hypervisor annotations
let allowed_hypervisor_annotations_for_arch = get_arch_var(
"ALLOWED_HYPERVISOR_ANNOTATIONS",
"",
&arch,
)
.split_whitespace()
.map(|s| s.to_string())
.collect();
let allowed_hypervisor_annotations_for_arch =
get_arch_var("ALLOWED_HYPERVISOR_ANNOTATIONS", "", &arch)
.split_whitespace()
.map(|s| s.to_string())
.collect();
// Only use arch-specific variable for snapshotter handler mapping
let snapshotter_handler_mapping_for_arch =
@@ -208,7 +205,9 @@ impl Config {
let pull_type_mapping_for_arch = get_arch_var_or_base("PULL_TYPE_MAPPING", &arch);
let installation_prefix = env::var("INSTALLATION_PREFIX").ok().filter(|s| !s.is_empty());
let installation_prefix = env::var("INSTALLATION_PREFIX")
.ok()
.filter(|s| !s.is_empty());
let dest_dir = match installation_prefix {
Some(ref prefix) => {
if !prefix.starts_with('/') {
@@ -258,15 +257,12 @@ impl Config {
.map(|s| s.split(',').map(|s| s.trim().to_string()).collect());
// Only use arch-specific variable for experimental force guest pull
let experimental_force_guest_pull_for_arch = get_arch_var(
"EXPERIMENTAL_FORCE_GUEST_PULL",
"",
&arch,
)
.split(',')
.filter(|s| !s.is_empty())
.map(|s| s.trim().to_string())
.collect();
let experimental_force_guest_pull_for_arch =
get_arch_var("EXPERIMENTAL_FORCE_GUEST_PULL", "", &arch)
.split(',')
.filter(|s| !s.is_empty())
.map(|s| s.trim().to_string())
.collect();
// Parse custom runtimes from ConfigMap
let custom_runtimes_enabled =
@@ -531,7 +527,8 @@ impl Config {
use crate::runtime::manager;
// Check if drop-in files can be used based on containerd version
let use_drop_in = manager::is_containerd_capable_of_using_drop_in_files(self, runtime).await?;
let use_drop_in =
manager::is_containerd_capable_of_using_drop_in_files(self, runtime).await?;
let paths = match runtime {
"k0s-worker" | "k0s-controller" => ContainerdPaths {
@@ -556,12 +553,10 @@ impl Config {
// versioned drop-in dir (config.toml.d or config-v3.toml.d). If the import is
// missing we bail; the cluster must configure the template with the import
// (e.g. in tests or via a custom k3s/RKE2 setup). Refs: docs.k3s.io/advanced#configuring-containerd
let container_runtime_version = k8s::get_node_field(
self,
".status.nodeInfo.containerRuntimeVersion",
)
.await
.ok();
let container_runtime_version =
k8s::get_node_field(self, ".status.nodeInfo.containerRuntimeVersion")
.await
.ok();
let use_v3 = k3s_rke2_resolve_use_v3(
&self.containerd_conf_file,
container_runtime_version.as_deref(),
@@ -757,18 +752,15 @@ fn get_arch_var_or_base(base_name: &str, arch: &str) -> Option<String> {
mod tests {
//! Tests for configuration parsing and validation.
//!
//! IMPORTANT: All tests in this crate MUST be run serially (--test-threads=1)
//! because they manipulate shared environment variables. Running tests in parallel
//! will cause race conditions and test failures.
//!
//! Use: cargo test --bin kata-deploy -- --test-threads=1
//! Tests that touch environment variables use `serial_test::serial` so they do not run
//! in parallel within this process. For extra isolation you can still use
//! `cargo test -p kata-deploy config::tests -- --test-threads=1`.
use super::*;
use rstest::rstest;
use serial_test::serial;
// NOTE: These tests modify environment variables which are process-global.
// Run with: cargo test config::tests -- --test-threads=1
// to ensure proper test isolation.
// NOTE: Env-var tests use #[serial] (see above) for safe parallel execution with other modules.
/// Helper to clean up common environment variables used in tests
fn cleanup_env_vars() {
@@ -867,6 +859,7 @@ mod tests {
);
}
#[serial]
#[test]
fn test_get_arch() {
let arch = get_arch().unwrap();
@@ -874,6 +867,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_get_arch_var() {
std::env::set_var("SHIMS_X86_64", "test1 test2");
@@ -887,24 +881,30 @@ mod tests {
#[rstest]
#[case(false, "config.toml.d")]
#[case(true, "config-v3.toml.d")]
#[serial]
fn test_k3s_rke2_drop_in_dir_name(#[case] use_v3: bool, #[case] expected: &str) {
assert_eq!(k3s_rke2_drop_in_dir_name(use_v3), expected);
}
#[serial]
#[test]
fn test_k3s_rke2_rendered_config_path() {
assert_eq!(k3s_rke2_rendered_config_path(), "/etc/containerd/config.toml");
assert_eq!(
k3s_rke2_rendered_config_path(),
"/etc/containerd/config.toml"
);
}
#[rstest]
#[case(
"imports = [\"/var/lib/rancher/k3s/agent/etc/containerd/config.toml.d/*.toml\"]\n",
false,
true,
true
)]
#[case("version = 2\n", false, false)]
#[case("imports = [\"/path/config-v3.toml.d/*.toml\"]", true, true)]
#[case("imports = [\"/path/config.toml.d/*.toml\"]", true, false)]
#[serial]
fn test_k3s_rke2_rendered_has_import(
#[case] content: &str,
#[case] use_v3: bool,
@@ -913,6 +913,7 @@ mod tests {
assert_eq!(k3s_rke2_rendered_has_import(content, use_v3), expected);
}
#[serial]
#[test]
fn test_multi_install_suffix_not_set() {
setup_minimal_env();
@@ -929,6 +930,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_multi_install_suffix_with_value() {
setup_minimal_env();
@@ -950,6 +952,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_multi_install_suffix_different_values() {
let suffixes = ["staging", "prod", "v2", "test123"];
@@ -970,6 +973,7 @@ mod tests {
}
}
#[serial]
#[test]
fn test_multi_install_prefix_and_suffix() {
setup_minimal_env();
@@ -988,6 +992,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_validate_empty_shims_no_custom_runtimes() {
setup_minimal_env();
@@ -1013,6 +1018,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_validate_default_shim_not_in_shims() {
setup_minimal_env();
@@ -1025,6 +1031,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_validate_hypervisor_annotation_invalid_shim() {
setup_minimal_env();
@@ -1041,6 +1048,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_validate_agent_https_proxy_invalid_shim() {
setup_minimal_env();
@@ -1057,6 +1065,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_validate_snapshotter_mapping_invalid_shim() {
setup_minimal_env();
@@ -1067,6 +1076,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_validate_pull_type_mapping_invalid_shim() {
setup_minimal_env();
@@ -1077,6 +1087,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_validate_force_guest_pull_invalid_shim() {
setup_minimal_env();
@@ -1087,6 +1098,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_validate_success() {
setup_minimal_env();
@@ -1106,6 +1118,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_missing_node_name_fails() {
cleanup_env_vars();
@@ -1116,6 +1129,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_empty_node_name_fails() {
setup_minimal_env();
@@ -1125,6 +1139,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_empty_default_shim_fails() {
setup_minimal_env();
@@ -1137,6 +1152,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_whitespace_only_default_shim_fails() {
setup_minimal_env();
@@ -1147,6 +1163,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_whitespace_only_shims_fails() {
setup_minimal_env();
@@ -1156,6 +1173,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_agent_no_proxy_invalid_shim() {
setup_minimal_env();
@@ -1166,6 +1184,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_multi_install_suffix_empty_treated_as_none() {
setup_minimal_env();
@@ -1177,6 +1196,7 @@ mod tests {
cleanup_env_vars();
}
#[serial]
#[test]
fn test_arch_specific_all_variables() {
// Test ALL architecture-specific variables work without base variables

View File

@@ -69,7 +69,10 @@ async fn main() -> Result<()> {
let mut sigterm = match signal(SignalKind::terminate()) {
Ok(s) => s,
Err(e) => {
log::warn!("Failed to register SIGTERM handler: {}, sleeping forever", e);
log::warn!(
"Failed to register SIGTERM handler: {}, sleeping forever",
e
);
std::future::pending::<()>().await;
return Ok(());
}
@@ -174,36 +177,30 @@ async fn install(config: &config::Config, runtime: &str) -> Result<()> {
}
// Validate snapshotter if needed
match config.experimental_setup_snapshotter.as_ref() {
Some(snapshotter) => {
let non_empty_snapshotters: Vec<_> =
snapshotter.iter().filter(|s| !s.is_empty()).collect();
if let Some(snapshotter) = config.experimental_setup_snapshotter.as_ref() {
let non_empty_snapshotters: Vec<_> = snapshotter.iter().filter(|s| !s.is_empty()).collect();
if !non_empty_snapshotters.is_empty() {
if runtime == "crio" {
log::warn!("EXPERIMENTAL_SETUP_SNAPSHOTTER is being ignored!");
log::warn!("Snapshotter is a containerd specific option.");
} else {
for s in &non_empty_snapshotters {
match s.as_str() {
"erofs" => {
runtime::containerd::containerd_erofs_snapshotter_version_check(
config,
)
if !non_empty_snapshotters.is_empty() {
if runtime == "crio" {
log::warn!("EXPERIMENTAL_SETUP_SNAPSHOTTER is being ignored!");
log::warn!("Snapshotter is a containerd specific option.");
} else {
for s in &non_empty_snapshotters {
match s.as_str() {
"erofs" => {
runtime::containerd::containerd_erofs_snapshotter_version_check(config)
.await?;
}
"nydus" => {}
_ => {
return Err(anyhow::anyhow!(
"{s} is not a supported snapshotter by kata-deploy"
));
}
}
"nydus" => {}
_ => {
return Err(anyhow::anyhow!(
"{s} is not a supported snapshotter by kata-deploy"
));
}
}
}
}
}
None => {}
}
runtime::containerd::setup_containerd_config_files(runtime, config).await?;
@@ -212,15 +209,14 @@ async fn install(config: &config::Config, runtime: &str) -> Result<()> {
runtime::configure_cri_runtime(config, runtime).await?;
match config.experimental_setup_snapshotter.as_ref() {
Some(snapshotters) => {
if runtime != "crio" {
if let Some(snapshotters) = config.experimental_setup_snapshotter.as_ref() {
for snapshotter in snapshotters {
artifacts::snapshotters::install_snapshotter(snapshotter, config).await?;
artifacts::snapshotters::configure_snapshotter(snapshotter, runtime, config)
.await?;
}
}
None => {}
}
info!("About to restart runtime: {}", runtime);
@@ -259,17 +255,21 @@ async fn cleanup(config: &config::Config, runtime: &str) -> Result<()> {
config.daemonset_name
);
match config.experimental_setup_snapshotter.as_ref() {
Some(snapshotters) => {
for snapshotter in snapshotters {
info!("Uninstalling snapshotter: {}", snapshotter);
artifacts::snapshotters::uninstall_snapshotter(snapshotter, config).await?;
info!("Successfully uninstalled snapshotter: {}", snapshotter);
if runtime != "crio" {
match config.experimental_setup_snapshotter.as_ref() {
Some(snapshotters) => {
for snapshotter in snapshotters {
info!("Uninstalling snapshotter: {}", snapshotter);
artifacts::snapshotters::uninstall_snapshotter(snapshotter, config).await?;
info!("Successfully uninstalled snapshotter: {}", snapshotter);
}
}
None => {
info!("No experimental snapshotters to uninstall");
}
}
None => {
info!("No experimental snapshotters to uninstall");
}
} else {
info!("Skipping snapshotter uninstall on CRI-O (containerd-specific feature)");
}
info!("Cleaning up CRI runtime configuration");

View File

@@ -144,8 +144,7 @@ fn write_containerd_runtime_config(
config_file,
&format!(
".plugins.{}.runtime_platforms.\"{}\".snapshotter",
CONTAINERD_CRI_IMAGES_PLUGIN_ID,
params.runtime_name
CONTAINERD_CRI_IMAGES_PLUGIN_ID, params.runtime_name
),
snapshotter,
)?;
@@ -280,9 +279,7 @@ pub async fn configure_custom_containerd_runtime(
),
config_path: format!(
"\"{}/share/defaults/kata-containers/custom-runtimes/{}/configuration-{}.toml\"",
config.dest_dir,
custom_runtime.handler,
custom_runtime.base_config
config.dest_dir, custom_runtime.handler, custom_runtime.base_config
),
pod_annotations,
snapshotter,
@@ -339,11 +336,7 @@ pub async fn configure_containerd(config: &Config, runtime: &str) -> Result<()>
let imports_path = ".imports";
let drop_in_path = format!("\"{}\"", paths.drop_in_file);
toml_utils::append_to_toml_array(
Path::new(imports_file),
imports_path,
&drop_in_path,
)?;
toml_utils::append_to_toml_array(Path::new(imports_file), imports_path, &drop_in_path)?;
log::info!("Successfully added drop-in to imports array");
} else {
log::info!("Runtime auto-loads drop-in files, skipping imports");
@@ -369,10 +362,7 @@ pub async fn configure_containerd(config: &Config, runtime: &str) -> Result<()>
config.custom_runtimes.len()
);
for custom_runtime in &config.custom_runtimes {
log::info!(
"Configuring custom runtime: {}",
custom_runtime.handler
);
log::info!("Configuring custom runtime: {}", custom_runtime.handler);
configure_custom_containerd_runtime(config, runtime, custom_runtime).await?;
log::info!(
"Successfully configured custom runtime: {}",
@@ -436,12 +426,14 @@ pub async fn setup_containerd_config_files(runtime: &str, config: &Config) -> Re
Path::new("/host").join(paths.drop_in_file.trim_start_matches('/'))
};
if let Some(parent) = drop_in_path.parent() {
fs::create_dir_all(parent)
.with_context(|| format!("Failed to create K3s/RKE2 drop-in dir: {parent:?}"))?;
fs::create_dir_all(parent).with_context(|| {
format!("Failed to create K3s/RKE2 drop-in dir: {parent:?}")
})?;
}
if !drop_in_path.exists() {
fs::write(&drop_in_path, "")
.with_context(|| format!("Failed to create K3s/RKE2 drop-in file: {drop_in_path:?}"))?;
fs::write(&drop_in_path, "").with_context(|| {
format!("Failed to create K3s/RKE2 drop-in file: {drop_in_path:?}")
})?;
}
}
"k0s-worker" | "k0s-controller" => {
@@ -501,7 +493,10 @@ pub async fn containerd_snapshotter_version_check(config: &Config) -> Result<()>
.map(|s| !s.is_empty())
.unwrap_or(false);
check_containerd_snapshotter_version_support(&container_runtime_version, has_snapshotter_mapping)
check_containerd_snapshotter_version_support(
&container_runtime_version,
has_snapshotter_mapping,
)
}
fn check_containerd_erofs_version_support(container_runtime_version: &str) -> Result<()> {
@@ -609,10 +604,7 @@ mod tests {
use std::path::Path;
use tempfile::NamedTempFile;
fn make_params(
runtime_name: &str,
snapshotter: Option<&str>,
) -> ContainerdRuntimeParams {
fn make_params(runtime_name: &str, snapshotter: Option<&str>) -> ContainerdRuntimeParams {
ContainerdRuntimeParams {
runtime_name: runtime_name.to_string(),
runtime_path: "\"/opt/kata/bin/kata-runtime\"".to_string(),
@@ -673,7 +665,11 @@ mod tests {
/// pluginid_for_snapshotter_annotations maps runtime plugin id to the table where disable_snapshot_annotations lives.
#[rstest]
#[case(CONTAINERD_V3_RUNTIME_PLUGIN_ID, CONTAINERD_CRI_IMAGES_PLUGIN_ID, false)]
#[case(
CONTAINERD_V3_RUNTIME_PLUGIN_ID,
CONTAINERD_CRI_IMAGES_PLUGIN_ID,
false
)]
#[case(CONTAINERD_V2_CRI_PLUGIN_ID, CONTAINERD_CRI_CONTAINERD_TABLE_V2, false)]
#[case(CONTAINERD_LEGACY_CRI_PLUGIN_ID, "", true)]
fn test_pluginid_for_snapshotter_annotations(
@@ -709,9 +705,7 @@ mod tests {
#[rstest]
#[case(CONTAINERD_V3_RUNTIME_PLUGIN_ID)]
#[case(CONTAINERD_V2_CRI_PLUGIN_ID)]
fn test_write_containerd_runtime_config_empty_file_no_leading_newlines(
#[case] pluginid: &str,
) {
fn test_write_containerd_runtime_config_empty_file_no_leading_newlines(#[case] pluginid: &str) {
let file = NamedTempFile::new().unwrap();
let path = file.path();
std::fs::write(path, "").unwrap();
@@ -733,7 +727,12 @@ mod tests {
}
#[rstest]
#[case("containerd://1.6.28", true, false, Some("kata-deploy only supports snapshotter configuration with containerd 1.7 or newer"))]
#[case(
"containerd://1.6.28",
true,
false,
Some("kata-deploy only supports snapshotter configuration with containerd 1.7 or newer")
)]
#[case("containerd://1.6.28", false, true, None)]
#[case("containerd://1.6.0", true, false, None)]
#[case("containerd://1.6.999", true, false, None)]
@@ -750,9 +749,19 @@ mod tests {
) {
let result = check_containerd_snapshotter_version_support(version, has_mapping);
if expect_ok {
assert!(result.is_ok(), "expected ok for version={} has_mapping={}", version, has_mapping);
assert!(
result.is_ok(),
"expected ok for version={} has_mapping={}",
version,
has_mapping
);
} else {
assert!(result.is_err(), "expected err for version={} has_mapping={}", version, has_mapping);
assert!(
result.is_err(),
"expected err for version={} has_mapping={}",
version,
has_mapping
);
if let Some(sub) = expected_error_substring {
assert!(
result.unwrap_err().to_string().contains(sub),

View File

@@ -101,26 +101,65 @@ mod tests {
#[case("qemu-tdx", "/opt/kata", "/opt/kata/share/defaults/kata-containers")]
#[case("fc", "/opt/kata", "/opt/kata/share/defaults/kata-containers")]
#[case("clh", "/opt/kata", "/opt/kata/share/defaults/kata-containers")]
#[case("cloud-hypervisor", "/opt/kata", "/opt/kata/share/defaults/kata-containers/runtime-rs")]
#[case("qemu-runtime-rs", "/opt/kata", "/opt/kata/share/defaults/kata-containers/runtime-rs")]
#[case(
"cloud-hypervisor",
"/opt/kata",
"/opt/kata/share/defaults/kata-containers/runtime-rs"
)]
#[case(
"qemu-runtime-rs",
"/opt/kata",
"/opt/kata/share/defaults/kata-containers/runtime-rs"
)]
#[case("qemu", "/custom/path", "/custom/path/share/defaults/kata-containers")]
#[case("cloud-hypervisor", "/custom/path", "/custom/path/share/defaults/kata-containers/runtime-rs")]
#[case(
"cloud-hypervisor",
"/custom/path",
"/custom/path/share/defaults/kata-containers/runtime-rs"
)]
fn test_get_kata_containers_original_config_path(
#[case] shim: &str,
#[case] base_dir: &str,
#[case] expected: &str,
) {
assert_eq!(get_kata_containers_original_config_path(shim, base_dir), expected);
assert_eq!(
get_kata_containers_original_config_path(shim, base_dir),
expected
);
}
// Tests for get_kata_containers_config_path (per-shim runtime directories)
#[rstest]
#[case("qemu", "/opt/kata", "/opt/kata/share/defaults/kata-containers/runtimes/qemu")]
#[case("qemu-tdx", "/opt/kata", "/opt/kata/share/defaults/kata-containers/runtimes/qemu-tdx")]
#[case("fc", "/opt/kata", "/opt/kata/share/defaults/kata-containers/runtimes/fc")]
#[case("cloud-hypervisor", "/opt/kata", "/opt/kata/share/defaults/kata-containers/runtime-rs/runtimes/cloud-hypervisor")]
#[case("qemu-runtime-rs", "/opt/kata", "/opt/kata/share/defaults/kata-containers/runtime-rs/runtimes/qemu-runtime-rs")]
#[case("qemu", "/custom/path", "/custom/path/share/defaults/kata-containers/runtimes/qemu")]
#[case(
"qemu",
"/opt/kata",
"/opt/kata/share/defaults/kata-containers/runtimes/qemu"
)]
#[case(
"qemu-tdx",
"/opt/kata",
"/opt/kata/share/defaults/kata-containers/runtimes/qemu-tdx"
)]
#[case(
"fc",
"/opt/kata",
"/opt/kata/share/defaults/kata-containers/runtimes/fc"
)]
#[case(
"cloud-hypervisor",
"/opt/kata",
"/opt/kata/share/defaults/kata-containers/runtime-rs/runtimes/cloud-hypervisor"
)]
#[case(
"qemu-runtime-rs",
"/opt/kata",
"/opt/kata/share/defaults/kata-containers/runtime-rs/runtimes/qemu-runtime-rs"
)]
#[case(
"qemu",
"/custom/path",
"/custom/path/share/defaults/kata-containers/runtimes/qemu"
)]
fn test_get_kata_containers_config_path(
#[case] shim: &str,
#[case] base_dir: &str,
@@ -207,7 +246,10 @@ mod tests {
let runtime_path = get_kata_containers_runtime_path(shim, dest_dir);
// Expected paths for Go runtime with per-shim directory
assert_eq!(config_path, "/opt/kata/share/defaults/kata-containers/runtimes/qemu-tdx");
assert_eq!(
config_path,
"/opt/kata/share/defaults/kata-containers/runtimes/qemu-tdx"
);
assert_eq!(original_path, "/opt/kata/share/defaults/kata-containers");
assert_eq!(runtime_path, "/opt/kata/bin/containerd-shim-kata-v2");

View File

@@ -15,9 +15,9 @@ fn parse_toml_path(path: &str) -> Result<Vec<String>> {
let mut parts = Vec::new();
let mut current = String::new();
let mut in_quotes = false;
let mut chars = path.chars().peekable();
let chars = path.chars().peekable();
while let Some(ch) = chars.next() {
for ch in chars {
match ch {
'"' => {
in_quotes = !in_quotes;
@@ -67,11 +67,7 @@ fn split_non_toml_header(content: &str) -> (&str, &str) {
/// Ensures the header ends with a newline before the TOML body.
/// Trims leading newlines from the serialized document to avoid many blank lines
/// when the file was initially empty (e.g. containerd drop-in).
fn write_toml_with_header(
file_path: &Path,
header: &str,
doc: &DocumentMut,
) -> Result<()> {
fn write_toml_with_header(file_path: &Path, header: &str, doc: &DocumentMut) -> Result<()> {
let normalized_header = if header.is_empty() {
String::new()
} else if header.ends_with('\n') {
@@ -214,7 +210,10 @@ pub fn append_to_toml_array(file_path: &Path, path: &str, value: &str) -> Result
// This is the array itself - use .get() to avoid panic on missing key
let key_exists = current.get(part.as_str()).is_some();
if !key_exists {
current.insert(part.as_str(), Item::Value(Value::Array(toml_edit::Array::new())));
current.insert(
part.as_str(),
Item::Value(Value::Array(toml_edit::Array::new())),
);
}
if let Some(Item::Value(Value::Array(arr))) = current.get_mut(part.as_str()) {
let value_item = parse_toml_value(value);
@@ -441,11 +440,7 @@ mod tests {
#[case("", "", "")]
#[case("key = \"value\"\n", "", "key = \"value\"\n")]
#[case("[plugins]\nfoo = 1\n", "", "[plugins]\nfoo = 1\n")]
#[case(
"{{ template \"base\" . }}\n",
"{{ template \"base\" . }}\n",
""
)]
#[case("{{ template \"base\" . }}\n", "{{ template \"base\" . }}\n", "")]
#[case(
"{{ template \"base\" . }}\n[plugins]\nfoo = 1\n",
"{{ template \"base\" . }}\n",
@@ -464,7 +459,11 @@ mod tests {
#[case] expected_toml: &str,
) {
let (header, toml) = split_non_toml_header(input);
assert_eq!(header, expected_header, "header mismatch for input: {:?}", input);
assert_eq!(
header, expected_header,
"header mismatch for input: {:?}",
input
);
assert_eq!(toml, expected_toml, "toml mismatch for input: {:?}", input);
}
@@ -484,7 +483,10 @@ mod tests {
.unwrap();
let content = std::fs::read_to_string(path).unwrap();
assert!(content.starts_with("{{ template \"base\" . }}\n"), "header must be preserved");
assert!(
content.starts_with("{{ template \"base\" . }}\n"),
"header must be preserved"
);
assert!(content.contains("runtime_type"), "value must be written");
let value = get_toml_value(
@@ -580,8 +582,12 @@ mod tests {
&format!("\"io.containerd.{shim}.v2\""),
)
.unwrap();
set_toml_value(path, &format!("{table}.privileged_without_host_devices"), "true")
.unwrap();
set_toml_value(
path,
&format!("{table}.privileged_without_host_devices"),
"true",
)
.unwrap();
}
let content = std::fs::read_to_string(path).unwrap();
@@ -633,7 +639,10 @@ mod tests {
)
.unwrap();
let content = std::fs::read_to_string(path).unwrap();
assert!(content.starts_with(expected_prefix), "header/prefix must be preserved");
assert!(
content.starts_with(expected_prefix),
"header/prefix must be preserved"
);
let body_start = content.strip_prefix(expected_prefix).unwrap();
assert!(
!body_start.starts_with('\n'),
@@ -782,11 +791,7 @@ mod tests {
#[case("test.string_value", "test_string", "test_string")]
#[case("test.bool_value", "true", "true")]
#[case("test.int_value", "42", "42")]
fn test_toml_value_types(
#[case] path: &str,
#[case] value: &str,
#[case] expected: &str,
) {
fn test_toml_value_types(#[case] path: &str, #[case] value: &str, #[case] expected: &str) {
let file = NamedTempFile::new().unwrap();
let file_path = file.path();
std::fs::write(file_path, "").unwrap();
@@ -827,8 +832,8 @@ mod tests {
);
// Test modifying kernel_params on real config
let current = get_toml_value(temp_path, "hypervisor.qemu.kernel_params")
.unwrap_or_default();
let current =
get_toml_value(temp_path, "hypervisor.qemu.kernel_params").unwrap_or_default();
let new_value = format!("{} agent.log=debug", current.trim_matches('"'));
let result = set_toml_value(
temp_path,
@@ -1318,7 +1323,11 @@ kernel_params = "console=hvc0"
"set" => set_toml_value(temp_path, "some.path", "\"value\""),
_ => panic!("unknown op"),
};
assert!(result.is_err(), "Should fail parsing invalid TOML (op={})", op);
assert!(
result.is_err(),
"Should fail parsing invalid TOML (op={})",
op
);
}
#[test]

View File

@@ -37,15 +37,14 @@ pub fn set_yaml_value(file_path: &Path, key_path: &str, value: serde_yaml::Value
} else {
// Navigate/create intermediate mappings
if let Some(map) = current.as_mapping_mut() {
if !map.contains_key(&serde_yaml::Value::String(part.to_string())) {
let key = serde_yaml::Value::String(part.to_string());
if !map.contains_key(&key) {
map.insert(
serde_yaml::Value::String(part.to_string()),
key.clone(),
serde_yaml::Value::Mapping(serde_yaml::Mapping::new()),
);
}
current = map
.get_mut(&serde_yaml::Value::String(part.to_string()))
.unwrap();
current = map.get_mut(&key).unwrap();
} else {
return Err(anyhow::anyhow!("Path component '{part}' is not a mapping"));
}

View File

@@ -25,6 +25,11 @@ shims:
allowedHypervisorAnnotations: []
containerd:
snapshotter: ""
runtimeClass:
# This label is automatically added by gpu-operator. Override it
# if you want to use a different label.
nodeSelector:
nvidia.com/cc.ready.state: "false"
qemu-nvidia-gpu-snp:
enabled: true
@@ -33,11 +38,20 @@ shims:
allowedHypervisorAnnotations: []
containerd:
snapshotter: "nydus"
forceGuestPull: false
crio:
guestPull: true
agent:
httpsProxy: ""
noProxy: ""
runtimeClass:
# These labels are automatically added by gpu-operator and NFD
# respectively. Override if you want to use a different label.
# If you don't have NFD, you need to add the snp label by other
# means to your SNP nodes.
nodeSelector:
nvidia.com/cc.ready.state: "true"
amd.feature.node.kubernetes.io/snp: "true"
qemu-nvidia-gpu-tdx:
enabled: true
@@ -46,11 +60,20 @@ shims:
allowedHypervisorAnnotations: []
containerd:
snapshotter: "nydus"
forceGuestPull: false
crio:
guestPull: true
agent:
httpsProxy: ""
noProxy: ""
runtimeClass:
# These labels are automatically added by gpu-operator and NFD
# respectively. Override if you want to use a different label.
# If you don't have NFD, you need to add the tdx label by other
# means to your TDX nodes.
nodeSelector:
nvidia.com/cc.ready.state: "true"
intel.feature.node.kubernetes.io/tdx: "true"
# Default shim per architecture (prefer NVIDIA GPU shims)
defaultShim:

View File

@@ -41,7 +41,7 @@ updateStrategy:
debug: false
snapshotter:
setup: [] # ["nydus", "erofs"] or []
setup: ["nydus"] # ["nydus", "erofs"] or []
# Shim configuration
# By default (disableAll: false), all shims with enabled: ~ (null) are enabled.
@@ -143,9 +143,8 @@ shims:
runtimeClass:
# This label is automatically added by gpu-operator. Override it
# if you want to use a different label.
# Uncomment once GPU Operator v26.3 is out
# nodeSelector:
# nvidia.com/cc.ready.state: "false"
nodeSelector:
nvidia.com/cc.ready.state: "false"
qemu-nvidia-gpu-snp:
enabled: ~
@@ -153,8 +152,8 @@ shims:
- amd64
allowedHypervisorAnnotations: []
containerd:
snapshotter: ""
forceGuestPull: true
snapshotter: "nydus"
forceGuestPull: false
crio:
guestPull: true
agent:
@@ -166,8 +165,7 @@ shims:
# If you don't have NFD, you need to add the snp label by other
# means to your SNP nodes.
nodeSelector:
# Uncomment once GPU Operator v26.3 is out
# nvidia.com/cc.ready.state: "true"
nvidia.com/cc.ready.state: "true"
amd.feature.node.kubernetes.io/snp: "true"
qemu-nvidia-gpu-tdx:
@@ -176,8 +174,8 @@ shims:
- amd64
allowedHypervisorAnnotations: []
containerd:
snapshotter: ""
forceGuestPull: true
snapshotter: "nydus"
forceGuestPull: false
crio:
guestPull: true
agent:
@@ -189,8 +187,7 @@ shims:
# If you don't have NFD, you need to add the tdx label by other
# means to your TDX nodes.
nodeSelector:
# Uncomment once GPU Operator v26.3 is out
# nvidia.com/cc.ready.state: "true"
nvidia.com/cc.ready.state: "true"
intel.feature.node.kubernetes.io/tdx: "true"
qemu-snp:

View File

@@ -365,6 +365,14 @@ get_latest_ctk_version() {
echo $(get_from_kata_deps ".externals.nvidia.ctk.version")
}
get_latest_nvrc_version() {
echo $(get_from_kata_deps ".externals.nvrc.version")
}
get_latest_nvat_version() {
echo $(get_from_kata_deps ".externals.nvidia.nvat.version")
}
#Install guest image
install_image() {
local variant="${1:-}"
@@ -398,6 +406,8 @@ install_image() {
if [[ "${variant}" == "nvidia-gpu-confidential" ]]; then
latest_artefact+="-$(get_latest_kernel_nvidia_artefact_and_builder_image_version)"
latest_artefact+="-$(get_latest_ctk_version)"
latest_artefact+="-$(get_latest_nvrc_version)"
latest_artefact+="-$(get_latest_nvat_version)"
else
latest_artefact+="-$(get_latest_kernel_artefact_and_builder_image_version)"
fi
@@ -410,6 +420,7 @@ install_image() {
# If we bump the kernel we need to rebuild the image
latest_artefact+="-$(get_latest_kernel_nvidia_artefact_and_builder_image_version)"
latest_artefact+="-$(get_latest_ctk_version)"
latest_artefact+="-$(get_latest_nvrc_version)"
fi
latest_builder_image=""
@@ -506,6 +517,8 @@ install_initrd() {
if [[ "${variant}" == "nvidia-gpu-confidential" ]]; then
latest_artefact+="-$(get_latest_kernel_nvidia_artefact_and_builder_image_version)"
latest_artefact+="-$(get_latest_ctk_version)"
latest_artefact+="-$(get_latest_nvrc_version)"
latest_artefact+="-$(get_latest_nvat_version)"
else
latest_artefact+="-$(get_latest_kernel_artefact_and_builder_image_version)"
fi
@@ -517,6 +530,7 @@ install_initrd() {
# If we bump the kernel we need to rebuild the initrd as well
latest_artefact+="-$(get_latest_kernel_nvidia_artefact_and_builder_image_version)"
latest_artefact+="-$(get_latest_ctk_version)"
latest_artefact+="-$(get_latest_nvrc_version)"
fi
latest_builder_image=""

View File

@@ -11,25 +11,48 @@ set -o nounset
set -o pipefail
set -o errtrace
KATA_DEPLOY_DIR="`dirname ${0}`/../../kata-deploy"
SCRIPT_DIR="$(cd "$(dirname "${0}")" && pwd)"
REPO_ROOT="$(cd "${SCRIPT_DIR}/../../../.." && pwd)"
KATA_DEPLOY_DIR="${REPO_ROOT}/tools/packaging/kata-deploy"
STAGED_ARTIFACT="${KATA_DEPLOY_DIR}/kata-static.tar.zst"
KATA_DEPLOY_ARTIFACT="${1:-"kata-static.tar.zst"}"
REGISTRY="${2:-"quay.io/kata-containers/kata-deploy"}"
TAG="${3:-}"
echo "Copying ${KATA_DEPLOY_ARTIFACT} to ${KATA_DEPLOY_DIR}"
cp ${KATA_DEPLOY_ARTIFACT} ${KATA_DEPLOY_DIR}
# Only remove a staged copy we created (skip when source is already the staged path).
REMOVE_STAGED_ON_EXIT=false
cleanup() {
if [ "${REMOVE_STAGED_ON_EXIT}" = true ]; then
rm -f "${STAGED_ARTIFACT}"
fi
}
trap cleanup EXIT
pushd ${KATA_DEPLOY_DIR}
src_rp="$(realpath -e "${KATA_DEPLOY_ARTIFACT}" 2>/dev/null || true)"
dest_rp="$(realpath -e "${STAGED_ARTIFACT}" 2>/dev/null || true)"
if [ -n "${src_rp}" ] && [ -n "${dest_rp}" ] && [ "${src_rp}" = "${dest_rp}" ]; then
echo "Artifact already at staged path ${STAGED_ARTIFACT}; skipping copy"
else
echo "Copying ${KATA_DEPLOY_ARTIFACT} to ${STAGED_ARTIFACT}"
cp "${KATA_DEPLOY_ARTIFACT}" "${STAGED_ARTIFACT}"
REMOVE_STAGED_ON_EXIT=true
fi
pushd "${REPO_ROOT}"
arch=$(uname -m)
[ "$arch" = "x86_64" ] && arch="amd64"
[ "$arch" = "aarch64" ] && arch="arm64"
# Disable provenance and SBOM so each tag is a single image manifest. quay.io rejects
# pushing multi-arch manifest lists that include attestation manifests ("manifest invalid").
PLATFORM="linux/${arch}"
IMAGE_TAG="${REGISTRY}:kata-containers-$(git rev-parse HEAD)-${arch}"
IMAGE_TAG="${REGISTRY}:kata-containers-$(git -C "${REPO_ROOT}" rev-parse HEAD)-${arch}"
DOCKERFILE="${REPO_ROOT}/tools/packaging/kata-deploy/Dockerfile"
echo "Building the image"
docker buildx build --platform "${PLATFORM}" --provenance false --sbom false \
-f "${DOCKERFILE}" \
--tag "${IMAGE_TAG}" --push .
if [ -n "${TAG}" ]; then
@@ -37,6 +60,7 @@ if [ -n "${TAG}" ]; then
echo "Building the ${ADDITIONAL_TAG} image"
docker buildx build --platform "${PLATFORM}" --provenance false --sbom false \
-f "${DOCKERFILE}" \
--tag "${ADDITIONAL_TAG}" --push .
fi

View File

@@ -14,7 +14,7 @@ ENV PATH="/opt/cargo/bin/:${PATH}"
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN mkdir ${RUSTUP_HOME} ${CARGO_HOME} && chmod -R a+rwX ${RUSTUP_HOME} ${CARGO_HOME}
RUN mkdir ${RUSTUP_HOME} ${CARGO_HOME}
RUN apt-get update && \
apt-get --no-install-recommends install -y \
@@ -38,6 +38,18 @@ RUN apt-get update && \
apt-get clean && rm -rf /var/lib/apt/lists/ && \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain ${RUST_TOOLCHAIN}
ARG NVAT_VERSION
RUN if [ "$(uname -m)" = "x86_64" ] && [ -n "${NVAT_VERSION}" ]; then \
apt-get update && apt-get --no-install-recommends install -y \
build-essential libxml2-dev zlib1g-dev && \
tmpdir=$(mktemp -d) && pushd "$tmpdir" && \
git clone https://github.com/NVIDIA/attestation-sdk && \
pushd attestation-sdk && git fetch --depth=1 origin "${NVAT_VERSION}" && \
git checkout FETCH_HEAD && pushd nv-attestation-sdk-cpp && cmake . && make install && \
mkdir -p /usr/include && ln -sf /usr/local/include/nvat.h /usr/include/nvat.h && ldconfig && \
popd && popd && popd && rm -rf "$tmpdir" && \
apt-get clean && rm -rf /var/lib/apt/lists/; fi
ENV LIBC="gnu"
RUN ARCH=$(uname -m); \
rust_arch=""; \
@@ -50,3 +62,5 @@ RUN ARCH=$(uname -m); \
esac; \
echo "RUST_ARCH=${rust_arch}" > /etc/profile.d/rust.sh; \
rustup target add "${rust_arch}-unknown-linux-${LIBC}"
RUN chmod -R a+rwX ${RUSTUP_HOME} ${CARGO_HOME}

View File

@@ -35,6 +35,22 @@ build_coco_guest_components_from_source() {
DESTDIR="${DESTDIR}/usr/local/bin" TEE_PLATFORM=${TEE_PLATFORM} make install
install -D -m0644 "confidential-data-hub/hub/src/image/ocicrypt_config.json" "${DESTDIR}/etc/ocicrypt_config.json"
if [ -n "${NV_ATTESTER:-}" ]; then
echo "build attestation-agent-nv with nvidia-attester support"
rm "target/${RUST_ARCH}-unknown-linux-${LIBC}/release/attestation-agent"
ATTESTER="${NV_ATTESTER}" NVAT_USE_SYSTEM_LIB=1 RUSTFLAGS="-L /usr/local/lib" \
DESTDIR="${DESTDIR}/usr/local/bin" TEE_PLATFORM=${TEE_PLATFORM} make build
strip "target/${RUST_ARCH}-unknown-linux-${LIBC}/release/attestation-agent"
install -D -m0755 "target/${RUST_ARCH}-unknown-linux-${LIBC}/release/attestation-agent" \
"${DESTDIR}/usr/local/bin/attestation-agent-nv"
mkdir -p "${DESTDIR}/usr/local/lib"
cp -a /usr/local/lib/libnvat.so* "${DESTDIR}/usr/local/lib/"
fi
popd
}

View File

@@ -28,12 +28,16 @@ package_output_dir="${package_output_dir:-}"
[ -n "${coco_guest_components_version}" ] || die "Failed to get coco-guest-components version or commit"
[ -n "${coco_guest_components_toolchain}" ] || die "Failed to get the rust toolchain to build coco-guest-components"
nvat_version="${nvat_version:-}"
[ -n "${nvat_version}" ] || nvat_version=$(get_from_kata_deps ".externals.nvidia.nvat.version" 2>/dev/null || true)
container_image="${COCO_GUEST_COMPONENTS_CONTAINER_BUILDER:-$(get_coco_guest_components_image_name)}"
[ "${CROSS_BUILD}" == "true" ] && container_image="${container_image}-cross-build"
docker pull ${container_image} || \
(docker $BUILDX build $PLATFORM \
--build-arg RUST_TOOLCHAIN="${coco_guest_components_toolchain}" \
--build-arg NVAT_VERSION="${nvat_version}" \
-t "${container_image}" "${script_dir}" && \
# No-op unless PUSH_TO_REGISTRY is exported as "yes"
push_to_registry "${container_image}")
@@ -44,7 +48,8 @@ RESOURCE_PROVIDER="kbs,sev"
# snp-attester and tdx-attester crates require packages only available on x86
# se-attester crate requires packages only available on s390x
case "$(uname -m)" in
x86_64) ATTESTER="snp-attester,tdx-attester,nvidia-attester" ;;
x86_64) ATTESTER="snp-attester,tdx-attester"
NV_ATTESTER="snp-attester,tdx-attester,nvidia-attester" ;;
s390x) ATTESTER="se-attester" ;;
aarch64) ATTESTER="cca-attester" ;;
*) ATTESTER="none" ;;
@@ -56,6 +61,7 @@ docker run --rm -i -v "${repo_root_dir}:${repo_root_dir}" \
--env TEE_PLATFORM=${TEE_PLATFORM:+"all"} \
--env RESOURCE_PROVIDER=${RESOURCE_PROVIDER:-} \
--env ATTESTER=${ATTESTER:-} \
--env NV_ATTESTER=${NV_ATTESTER:-} \
--env coco_guest_components_repo="${coco_guest_components_repo}" \
--env coco_guest_components_version="${coco_guest_components_version}" \
--user "$(id -u)":"$(id -g)" \

View File

@@ -93,7 +93,7 @@ mapping:
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-free-runner / run-k8s-tests (cloud-hypervisor, active)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-zvsi / run-k8s-tests (devmapper, qemu, kubeadm)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-zvsi / run-k8s-tests (nydus, qemu-coco-dev, kubeadm)
# - Kata Containers CI / kata-containers-ci-on-push / run-kata-coco-tests / run-k8s-tests-on-tee (sev-snp, qemu-snp)
- Kata Containers CI / kata-containers-ci-on-push / run-kata-coco-tests / run-k8s-tests-on-tee (sev-snp, qemu-snp)
- Kata Containers CI / kata-containers-ci-on-push / run-kata-coco-tests / run-k8s-tests-coco-nontee (qemu-coco-dev, nydus, guest-pull)
- Kata Containers CI / kata-containers-ci-on-push / run-kata-coco-tests / run-k8s-tests-coco-nontee (qemu-coco-dev-runtime-rs, nydus, guest-pull)
- Kata Containers CI / kata-containers-ci-on-push / run-kata-deploy-tests / run-kata-deploy-tests (qemu, k0s)

View File

@@ -269,6 +269,10 @@ externals:
ctk:
version: "1.18.1-1"
url: "https://github.com/NVIDIA/nvidia-container-toolkit"
nvat:
desc: "NVIDIA Attestation SDK"
version: "2026.03.02"
url: "https://github.com/NVIDIA/attestation-sdk"
busybox:
desc: "The Swiss Army Knife of Embedded Linux"
@@ -288,18 +292,18 @@ externals:
coco-guest-components:
description: "Provides attested key unwrapping for image decryption"
url: "https://github.com/confidential-containers/guest-components/"
version: "ab95914ac84c32a43102463cc0ae330710af47be"
version: "30b552e7841b10e656fa28cf643ed25b9d45e33f"
toolchain: "1.90.0"
coco-trustee:
description: "Provides attestation and secret delivery components"
url: "https://github.com/confidential-containers/trustee"
version: "f5cb8fc1b51b652fc24e2d6b8742cf417805352e"
version: "22788122660d6e9be3e4bf52704282de5fcc0a2a"
# image / ita_image and image_tag / ita_image_tag must be in sync
image: "ghcr.io/confidential-containers/staged-images/kbs"
image_tag: "f5cb8fc1b51b652fc24e2d6b8742cf417805352e"
image_tag: "22788122660d6e9be3e4bf52704282de5fcc0a2a"
ita_image: "ghcr.io/confidential-containers/staged-images/kbs-ita-as"
ita_image_tag: "f5cb8fc1b51b652fc24e2d6b8742cf417805352e-x86_64"
ita_image_tag: "22788122660d6e9be3e4bf52704282de5fcc0a2a-x86_64"
toolchain: "1.90.0"
containerd:
@@ -383,8 +387,6 @@ externals:
url: "https://github.com/dragonflyoss/image-service"
version: "v2.2.3"
# Keep the version here aligned with the NYDUS_SNAPSHOTTER_VERSION
# on tools/packaging/kata-deploy/Dockerfile
nydus-snapshotter:
description: "Snapshotter for Nydus image acceleration service"
url: "https://github.com/containerd/nydus-snapshotter"