Compare commits

...

173 Commits

Author SHA1 Message Date
Fabiano Fidêncio
546f9419e0 build: kernel: Do not expect a modules tarball for vanilla kernel
When I added this I had in mind the period that we still relied on the
SEV module being generated, which we don't do for quite a long time.

This wrong assumption caused the cache to **ALWAYS** fail, increasing
our build time considerably for no reason.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-03-03 12:58:26 +01:00
Dan Mihai
3ea23528a5 docs: require user/group/fsGroup/supplementalGroups
Add a nydus guest-pull limitation explaining that specifying runAsUser,
runAsGroup, fsGroup, and supplementalGroups are required.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-02 23:48:36 +01:00
Steve Horsman
e50324ba5b Merge pull request #12609 from kata-containers/dependabot/go_modules/src/runtime/go.opentelemetry.io/otel/sdk-1.40.0
build(deps): bump go.opentelemetry.io/otel/sdk from 1.35.0 to 1.40.0 in /src/runtime
2026-03-02 16:32:40 +00:00
stevenhorsman
993a4846c8 versions: Bump go to 1.25.7
Now that go 1.26 is out, 1.24 is not supported, so bump to
1.25 as per our policy.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-03-02 16:33:47 +01:00
dependabot[bot]
d95d1796b2 build(deps): bump go.opentelemetry.io/otel/sdk in /src/runtime
Bumps [go.opentelemetry.io/otel/sdk](https://github.com/open-telemetry/opentelemetry-go) from 1.35.0 to 1.40.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.35.0...v1.40.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/sdk
  dependency-version: 1.40.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-02 12:59:21 +00:00
Steve Horsman
501d8d1916 Merge pull request #12596 from kata-containers/remove-install_go
workflow | tests: Remove install go
2026-03-02 12:36:58 +00:00
Steve Horsman
964c91f8fc Merge pull request #12608 from kata-containers/sprt/fix-hostpath-dev-docs
docs: Use more accurate wording for /dev hostPath behavior
2026-03-02 11:50:15 +00:00
Aurélien Bombo
68e67d7f8a docs: Use more accurate wording for /dev hostPath behavior
I got lazy when I first added this section in 5c21b1f, so updating the
language to specify that any non-regular host file (under /dev) qualifies,
not just devices.

This matches the actual code, see:

330bfff4be/src/runtime/virtcontainers/mount.go (L57-L83)

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2026-03-02 11:32:01 +00:00
Steve Horsman
b147cb1319 Merge pull request #12587 from fidencio/topic/runtime-add-configurable-kubelet-root-dir
runtimes: add configurable kubelet root dir
2026-02-28 19:06:14 +00:00
Xuewei Niu
8a4ae090e6 Merge pull request #12513 from lifupan/event_publish
send the task create/start/delete event to containerd
2026-02-28 14:41:46 +08:00
Zvonko Kaiser
afe09803a1 gpu: Ignore OVMF and use the Kernel for proper PCI setup
Sometimes OVFM provides incorrect values to the kernel
we override it by telling the kernel to handle the PCI space setup
like allocating the proper window sizes and assigning the proper busses
to each device.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2026-02-27 22:54:31 +01:00
Manuel Huber
88f746dea8 runtime: nvidia: Use OVMF for NV GPU handler
Shift to using OVMF instead of using SeaBios.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>

Update src/runtime/Makefile

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2026-02-27 22:54:31 +01:00
Zvonko Kaiser
eec397ac08 qemu: Remove PCIe root port BAR reserve sizing
Stop computing and setting mem-reserve and pref64-reserve on PCIe root
ports and switch ports. Remove getBARsMaxAddressableMemory() which
scanned host GPU BARs to pre-calculate these values.

The previous approach only considered GPU devices (IsGPU(), class
0x0302) when scanning for BAR sizes, so devices like NVSwitches (class
0x0680) with their 32MB non-prefetchable BAR0 were not accounted for
and received the 4MB default. Additionally, GetTotalAddressableMemory()
classifies BARs by 32/64-bit address width rather than by the
prefetchable flag that QEMU's mem-reserve vs pref64-reserve maps to.

Modern QEMU introspects VFIO device BARs when they are attached to
root ports and sizes the MMIO windows accordingly. Modern OVMF
(edk2-stable202502+) automatically calculates the 64-bit PCI MMIO
aperture based on the BARs of actually present devices during PCI
enumeration. Omitting the reserve parameters lets QEMU and OVMF
handle MMIO window sizing correctly for all device types including
GPUs, NVSwitches, and NICs without requiring host-side BAR scanning.

This also removes the nvpci dependency from qemu_arch_base.go.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2026-02-27 22:54:31 +01:00
Zvonko Kaiser
bb7fd335f3 qemu: Remove OVMF X-PciMmio64Mb fw_cfg hint
Modern OVMF (edk2-stable202502 and later) automatically sizes the
64-bit PCI MMIO aperture based on the BARs of actually attached
devices during PCI enumeration. The opt/ovmf/X-PciMmio64Mb fw_cfg
hint is no longer needed to ensure large-BAR devices like NVIDIA
GPUs receive adequate MMIO space.

The previous approach was fragile: the runtime scanned host PCI
devices to estimate the required aperture size, but only considered
GPU devices (class 0x0302), missing NVSwitches and other devices
with large BARs. Removing this code avoids confusion about MMIO
sizing responsibility.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2026-02-27 22:54:31 +01:00
Fabiano Fidêncio
330bfff4be kata-deploy: Fix nydus snapshotter config (on v3 config version)
On containerd v3 config, disable_snapshot_annotations must be set under the
images plugin, not the runtime plugin.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-27 18:20:30 +01:00
Fabiano Fidêncio
0a73638744 runtime: add configurable kubelet root dir
Different kubernetes distributions, such as k0s, use a different kubelet
root dir location instead of the default /var/lib/kubelet, so ConfigMap
and Secret volume propagation were failing.

This adds a kubelet_root_dir config option that the go runtime uses when
matching volume paths and kata-deploy now sets it automatically for k0s
via a drop-in file.

runtime-rs does not need this option: it identifies ConfigMap/Secret,
projected, and downward-api volumes by volume-type path segment
(kubernetes.io~configmap, etc.), not by kubelet root prefix.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-27 14:10:57 +01:00
Steve Horsman
2695007ef8 Merge pull request #12584 from stevenhorsman/switch-actionlint-workflow
workflow: Update actionlint workflows
2026-02-27 13:03:58 +00:00
stevenhorsman
66e58d6490 tests: Delete install_go.sh
Having a script to install go is legacy from Jenkins, so
delete it, so there is less code in our repo.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-27 12:42:43 +00:00
stevenhorsman
b71bb47e21 workflow: Use setup-go to install go
Rather than having our own script, just use the github action
to install go when needed.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-27 12:42:43 +00:00
Steve Horsman
3442fc7d07 Merge pull request #12477 from kata-containers/workflow-improvements
workflow: Recommended improvements
2026-02-27 11:57:22 +00:00
Markus Rudy
d9d886b419 agent-policy: read bundle-id from OCI spec rootfs
The host path of bundles is not portable and could be literally anything
depending on containerd configuration, so we can't rely on a specific
prefix when deriving the bundle-id. Instead, we derive the bundle-id
from the target root path in the guest.

NOTE: fixes https://github.com/kata-containers/kata-containers/issues/10065

Signed-off-by: Markus Rudy <mr@edgeless.systems>
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-27 10:24:38 +01:00
Hyounggyu Choi
be5ae7d1e1 Merge pull request #12573 from BbolroC/support-memory-hotplug-go-runtime-s390x
runtime: Support memory hotplug via virtio-mem on s390x
2026-02-27 09:59:40 +01:00
Steve Horsman
c6014ddfe4 Merge pull request #12574 from sathieu/kata-deploy-kubectl-image
kata-deploy: allow to configure kubectl image
2026-02-27 08:42:06 +00:00
Steve Horsman
1048132eb1 Merge pull request #12564 from stevenhorsman/remove-unused-dependencies
Try and remove unused crates
2026-02-26 13:53:44 +00:00
Aurélien Bombo
2a13f33d50 Merge pull request #12565 from microsoft/danmihai1/clh-51.1
versions: update cloud hypervisor to v51.1
2026-02-26 07:52:57 -06:00
Hyounggyu Choi
b1847f9598 tests: Run TestContainerMemoryUpdate() on s390x only with virtio-mem
Let's run `TestContainerMemoryUpdate` on s390x
only when virtio-mem is enabled.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2026-02-26 14:21:34 +01:00
Hyounggyu Choi
b9f3d5aa67 runtime: Support memory hotplug with virtio-mem on s390x
This commit adds logic to properly handle memory hotplug
for QemuCCWVirtio in the ExecMemdevAdd() path.

The new logic is triggered only when virtio-mem is enabled.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2026-02-26 14:21:34 +01:00
Hyounggyu Choi
19771671c2 runtime: Handle virtio-mem resize in hotplugAddMemory()
ResizeMemory() already contains the virtio-mem resize logic.
However, hotplugAddMemory(), which is invoked via a different
path, lacked this handling and always fell back to the pc-dimm
path, even when virtio-mem was configured.

This commit adds virtio-mem resize handling to hotplugAddMemory().
It also adds corresponding unit tests.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2026-02-26 14:21:34 +01:00
Fabiano Fidêncio
8c91e7889c helm-chart: support digest pinning for images
When image.reference or kubectlImage.reference already contains a digest
(e.g. quay.io/...@sha256:...), use the reference as-is instead of
appending :tag. This avoids invalid image strings like 'image@sha256🔤'
when tag is empty and allows users to pin by digest.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-26 13:39:51 +01:00
Mathieu Parent
b61d169472 kata-deploy: allow to configure kubectl image
This can be used to:

- pin tag (current is 20260112)
- pin digest
- use another image

Signed-off-by: Mathieu Parent <mathieu.parent@insee.fr>
2026-02-26 13:12:03 +01:00
stevenhorsman
308442e887 workflow: Update actionlint workflows
The actionlint gh extension is outdated and the wrapping seems
unnecessary when there is a github action that seems to be maintained,
so let's update to use that

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-26 11:52:19 +00:00
stevenhorsman
82c27181d8 kata-deploy: Remove unused crates
cargo machete has identified `serde` and `thiserror` as being unused,
so remove them from Cargo.toml

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-26 09:38:35 +00:00
stevenhorsman
bdbfe9915b kata-ctl: Remove unused crates
cargo machete has identified the follow crates as unused:
- containerd-shim-protos
- safe-path
- strum
- ttrpc

strum is neded (and maybe isn't picked up due to it being
used by macros?), so add it to the ignore list and remove
the rest

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-26 09:38:35 +00:00
stevenhorsman
b4365bdcaa genpolicy: Remove unused crates
`cargo machete` has identified `openssl` and `serde-transcode`
as being un-used. openssl is required, so add it to the ignore
list and just remove serde-transcode

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-26 09:38:35 +00:00
stevenhorsman
382c6d2a2f agent-ctl: Remove unused crates
`log` and `rustjail` are flagged by cargo machete as unused,
so lets remove them to reduce the footprint of crates in this tool

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-26 09:38:35 +00:00
stevenhorsman
e43a17c2ba runtime-rs: Remove unused crates
- Remove unused crates to reduce our size and the work needed
to do updates
- Also update package.metadata.cargo-machete with some crates
that are incorrectly coming up as unused

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-26 09:37:46 +00:00
stevenhorsman
8177a440ca libs: Remove unused crates
Remove unused crates to reduce our size and the work needed
to do updates

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-26 09:37:46 +00:00
stevenhorsman
ed7ef68510 dragonball: Remove unused crates
Remove the crates that cargo machete has assessed as being unused

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-26 09:37:15 +00:00
stevenhorsman
c1b8c6bce6 dragonball: Update cargo-machete config
cargo machete can't understand `host-device = ["dep:vfio-bindings"`,
so tell it to ignore `vfio-bindings` and not suggest it's unused

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-26 09:37:14 +00:00
stevenhorsman
1139a9bb8a trace-forwarder: Try and remove unused crates
I ran cargo machete in trace-forwarder and it suggested that some
of the packages were not used, including a chain with a vulnerability,
so try and remove them to resolve RUSTSEC-2021-0139

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-26 09:37:14 +00:00
Steve Horsman
675c0c3450 Merge pull request #12553 from kata-containers/dependabot/cargo/src/tools/agent-ctl/keccak-0.1.6
build(deps): bump keccak from 0.1.5 to 0.1.6 in /src/tools/agent-ctl
2026-02-26 08:53:57 +00:00
Steve Horsman
9a921bb396 Merge pull request #12575 from kata-containers/build-checks-go-install-setup-go
workflows: Swap our go install for setup-go
2026-02-26 08:51:56 +00:00
Steve Horsman
da0ca483b0 Merge pull request #12572 from fitzthum/bump-trustee
versions: bump Trustee to latest version
2026-02-26 08:48:37 +00:00
Alex Lyn
57b0148356 Merge pull request #12400 from Apokleos/enhance-snp-rs
runtime-rs: Enhance Qemu/SNP Protection
2026-02-26 15:29:33 +08:00
Dan Mihai
2361dc7ca0 tests: k8s: reinstate testing on mariner hosts
Reinstate mariner host testing - including the Agent Policy tests on
these hosts - now that a new CLH version brought in the required fixes.

This reverts commit ea53779b90.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2026-02-25 21:01:25 +00:00
Dan Mihai
7973e4e2a8 runtime: clh: disable nested vCPUs on MSHV
The recently-added nested property is true by default, but is not
supported yet on MSHV.

See cloud-hypervisor/cloud-hypervisor#7408 for additional information.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2026-02-25 21:01:25 +00:00
Dan Mihai
24ac2ccb5c runtime-rs: clh: specify raw image format
Specify raw image format for all guest block devices.

- Attempting to auto-detect the image format from CLH would be riskier
  for the Host.

- Creating a new raw image file, auto-detecting its format, and then
  creating a filesystem from the Guest onto the block device is no
  longer supported by CLH v51. Therefore, Kata CI's k8s-block-volume.bats
  would fail without specifying the raw format when hot plugging its block
  device.

- See cloud-hypervisor/cloud-hypervisor@b3e8e2a for additional information.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2026-02-25 21:01:25 +00:00
Dan Mihai
dc398e801c runtime: clh: specify raw image format
Specify raw image format for all guest block devices.

- Attempting to auto-detect the image format from CLH would be riskier
  for the Host.

- Creating a new raw image file, auto-detecting its format, and then
  creating a filesystem from the Guest onto the block device is no
  longer supported by CLH v51. Therefore, Kata CI's k8s-block-volume.bats
  would fail without specifying the raw format when hot plugging its block
  device.

- See cloud-hypervisor/cloud-hypervisor@b3e8e2a for additional information.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2026-02-25 21:01:25 +00:00
Dan Mihai
0629354ca0 versions: update cloud hypervisor to v51.1
```
v51.1
=====

This is a bug fix release. The following issues have been addressed:

* Fix image_type in OpenAPI definition (#7734)

v51.0
=====

This release has been tracked in v51.0 group of our roadmap project.

Security Fixes

This release fixes a security vulnerability in disk image handling.
Details can be found in GHSA-jmr4-g2hv-mjj6.

* A new `backing_files=on|off` option has been added to `--disk` to
  explicitly control whether QCOW2 backing files are permitted. This
  defaults to `off` to prevent the loading of backing files entirely.
  (#7685)
* Explicit image type specification via the user interface, removing
  reliance on format autodetection (#7728).
* Prevent sector-zero writes for autodetected raw images (#7728).

Significant QCOW2 v3 Improvements

A large number of QCOW2 v3 specification features have been implemented:

* RAW backing file support for QCOW2 overlays (#7570)
* Zero bit in L2 entries (#7627)
* Incompatible feature bit validation (#7612)
* Dirty bit support (#7636)
* Variable refcount widths (1 to 64-bit) (#7633)
* Corrupt bit detection and marking (#7639)
* Autoclear feature bits handling (#7648)
* Thread safety fix for multiple virtio queues (`num_queues > 1`)
  (#7661)
* Correct zero-fill for reads beyond backing file size (#7678)
* Live disk resize support (#7687)

ACPI Generic Initiator Support

ACPI Generic Initiator Affinity (SRAT Type 5) support has been added
to associate VFIO-PCI devices with dedicated memory/CPU-less NUMA
nodes. This enables the guest OS to make NUMA-aware memory allocation
decisions for device workloads. A new `device_id` parameter has been
added to `--numa` for specifying VFIO devices. (#7626)

Block Device DISCARD and WRITE_ZEROES Support

The `virtio-blk` device now supports `DISCARD` and `WRITE_ZEROES`
operations for QCOW2 and RAW image formats. This enables thin
provisioning and efficient space reclamation when guests trim
filesystems. A new `sparse=on|off` option has been added to `--disk` to
control disk space management: `sparse=on` (default) enables thin
provisioning with space reclamation, while `sparse=off` provides thick
provisioning with consistent I/O latency. (#7666)

Notable Performance Improvements

* Transparent Huge Pages (THP) support has been extended to cover
  anonymous shared memory (`shared=on`) via `madvise`. Previously, THP
  was only used for non-shared memory. (#7646)
* The `vhost-user-net` device now uses the default set of vhost-user
  virtio features, including `VIRTIO_F_RING_INDIRECT_DESC`, which
  provides a performance improvement. (#7653)

MSHV Support Improvements

* Optimize CPU state update after emulation by only updating special
  registers when changed (#7603)
* Enable SMT for guests with `threads_per_core > 1` (#7668)
* Stub `save_data_tables()` to unblock VM pause/resume (#7692)
* Handle `GHCB_INFO_SPECIAL_DBGPRINT` VMG exit in SEV-SNP guest exit
  handler (#7703)
* Fix CVM boot failure on MSHV (#7548)
* Fix CPU topology detection for multithreaded configurations (#7576)

Notable Bug Fixes

* Fix VFIO device hot-remove leaving group and container file
  descriptors open, preventing re-add (#7676)
* Fix snapshot restore when backing file is on read-only storage with
  `shared=false` (#7674)
* Enforce `VIRTIO_BLK_F_RO` even if guest does not negotiate it
  (#7705)
* Fix read-only block device FLUSH requests from OVMF preventing VMs
  from booting (#7706)
* Fix vhost-user device not properly dropping unowned file descriptors
  (#7679)
* Fix `vhost-user-block` `get_config` interoperability (#7617)
* Fix vsock TOCTOU race condition by copying packet header from guest
  memory before processing (#7530)
* Fix vsock handling of large TX packets spanning multiple data
  descriptors (#7680)
* Add `gettid()` to all seccomp filters (#7596)
* Fix MAC address parsing that wrongly allowed `+` instead of hex
  characters (#7579)
* Improve UUID parse error message and `--net` fd help text (#7702)
* Fix various inconsistencies in our OpenAPI specification file
  (#7716, #7726)
* Various documentation fixes (#7602, #7606)
```

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2026-02-25 21:01:25 +00:00
Tobin Feldman-Fitzthum
b4b5db2f1c tests: fixup SNP attestation test for new Trustee version
Trustee now returns the binary SNP TCB claims as hex rather than base64
(for consistency with other platforms). Fortunately, the sev-snp-measure
tool has a flag for setting the output type of the launch digest.

I think hex is the default, but let's keep the flag here to be explicit.

Signed-off-by: Tobin Feldman-Fitzthum <tfeldmanfitz@nvidia.com>
2026-02-25 09:57:36 -08:00
Steve Horsman
a655605e8f Merge pull request #12566 from manuelh-dev/mahuber/fail-exp-timeout
tests: Extend fail timeout for failure test
2026-02-25 16:11:53 +00:00
stevenhorsman
856ba08c71 workflows: Swap our go install for setup-go
Unfortunately, due to golang/go#75031, there is an issue
that results in `go: no such tool "covdata"`
with a automatically installed 1.25 toolchain, so
the approach to skip the install_go.sh script (which causes
double install problems) didn't work. Try the alternative approach
of using setup-go action, which should do a more comprehensive job

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-25 13:46:40 +00:00
Alex Lyn
2fb6376a99 dragonball: Reduce warnings in dragonball when using 1.91 rust tools
Some warnings come up when we use bumped rust-1.91, this commit aims to
eliminate warnings.

Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
2026-02-25 21:04:35 +08:00
Alex Lyn
dc87c1f732 runtime-rs: Add support for configurable Qemu/SEV-SNP guest policy
This commit enables the SEV-SNP guest policy to be explicitly
configured via the runtime configuration in runtime-rs.

To provide both ease of use and maximum flexibility, the following
logic is implemented:
1. If the user provides a custom `snp_guest_policy` in the
configuration, this value is passed directly to the QEMU SEV-SNP
guest object.
2. If the user does not specify a policy, the driver defaults to
`0x30000`, matching QEMU's standard default for SEV-SNP guests.

This enhancement allows users to fine-tune security constraints through
the policy bitmask, while ensuring a sensible and functional default
for standard SNP deployments.

Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
2026-02-25 21:04:35 +08:00
Alex Lyn
9fc708ec4f kata-types: Add SNP launch configuration fields to SecurityInfo
This commit introduces three new fields to the `SecurityInfo` struct
to support SEV-SNP (Secure Nested Paging) attestation and measurement
capabilities:

(1) `snp_id_block`: A 96-byte Base64-encoded ID block for the
  SNP_LAUNCH_FINISH command.
(2) `snp_id_auth`: A 4096-byte Base64-encoded authentication structure
  accompanying the ID block.
(3) `snp_guest_policy`: A bitmask for the SNP guest policy, passed to
  the SNP_LAUNCH_START command.

These fields enable users to provide identity information to the SNP
firmware, allowing for remote attestation and verified guest launches.

Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
2026-02-25 21:04:35 +08:00
Alex Lyn
f9ffc95c3c runtime-rs: Introduce a SNP policy field in ObjectSevSnpGuest
A bitmask for the SNP guest policy is introduced in ObjectSevSnpGuest
to help pass to Qemu cmdline.

And defaults to 0x30000 (QEMU's default) to maintain standard behavior
it just looks like as: "policy=0x30000"

Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
2026-02-25 21:04:35 +08:00
Alex Lyn
21e0df4c06 runtime-rs: Add kernel irqchip with split for SNP
Add more param with split when qemu launches for SNP.

Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
2026-02-25 21:04:35 +08:00
Alex Lyn
ebe87d0e6f runtime-rs: Disable memory hotplug setting within SEV-SNP
For SEV-SNP, memory overcommit is not supported. we only set the memory
size.

Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
2026-02-25 21:04:35 +08:00
Alex Lyn
830667c041 runtime-rs: Add two methods for Qemu Memory to control memory set
Introduce two methods to help set max memory and num_slots.

Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
2026-02-25 21:04:35 +08:00
Alex Lyn
d298df7014 kata-types: Add cross-platform host_memory_mib() helper for host memory
Introduce host_memory_mib() with OS-specific implementations
(Linux/Android via nix::sysinfo,
macOS via sysctl) selected at compile time. This improves
portability and allows consistent host memory sizing/validation
across different platforms.

Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
2026-02-25 21:04:26 +08:00
Zvonko Kaiser
7294719e1c Merge pull request #12559 from fidencio/topic/kata-deploy-fix-custom-runtime-no-snapshotter
kata-deploy: a few guard-rails to avoid failures if components are not set in the values.yaml file
2026-02-25 08:03:28 -05:00
dependabot[bot]
528a944b2a build(deps): bump keccak from 0.1.5 to 0.1.6 in /src/tools/agent-ctl
Bumps [keccak](https://github.com/RustCrypto/sponges) from 0.1.5 to 0.1.6.
- [Commits](https://github.com/RustCrypto/sponges/compare/keccak-v0.1.5...keccak-v0.1.6)

---
updated-dependencies:
- dependency-name: keccak
  dependency-version: 0.1.6
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-25 13:02:31 +00:00
Alex Lyn
b3d60698af runtime-rs: move host memory adjustment into MemoryInfo using nix sysinfo
As the memory related information has been serialized at the sandbox
initalization specially at the moment of parsing configuration toml.

This commit aims to refactor MemoryInfo initialization logics:

(1) Remove memory sizing/host-memory adjustment logic from QEMU cmdline
  Memory::new()
(2) Initialize/adjust memory values via kata-types MemoryInfo (single
  source of truth)
(3) Replace sysinfo::System::new_with_specifics with
  nix::sys::sysinfo::sysinfo() to get host RAM

Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
2026-02-25 19:32:44 +08:00
Steve Horsman
7ffb7719b5 Merge pull request #12562 from kata-containers/prep-for-go-1.25-switch
Prep for go 1.25 switch
2026-02-25 11:13:30 +00:00
dependabot[bot]
7cc2e9710b build(deps): bump github.com/BurntSushi/toml in /src/runtime
Bumps [github.com/BurntSushi/toml](https://github.com/BurntSushi/toml) from 1.3.2 to 1.5.0.
- [Release notes](https://github.com/BurntSushi/toml/releases)
- [Commits](https://github.com/BurntSushi/toml/compare/v1.3.2...v1.5.0)

---
updated-dependencies:
- dependency-name: github.com/BurntSushi/toml
  dependency-version: 1.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-25 10:00:36 +01:00
Hyounggyu Choi
2860e68534 kernel: Enable CONFIG_VIRTIO_MEM for s390x
Since QEMU v10.0.0 and Linux v6.13, virtio-mem-ccw is supported.
Let's enable the required kernel configs for s390x.

This commit enables `CONFIG_VIRTIO_MEM` and `CONFIG_MEMORY_HOTREMOVE`
to support memory hotplug in the VM guest.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2026-02-25 08:17:48 +01:00
Tobin Feldman-Fitzthum
88568dd6e0 versions: bump Trustee to latest version
Update Trustee to pickup a few recent features, such as improvements to
TDX attestation configuration, and fixes to our vault/OpenBao backend.

This will also pickup our bump of Trustee to Rust 1.90.0.

We should be able to use this version of Trustee with the current
version of guest-components, which cannot be bumped at the moment due to
development dependencies.

Signed-off-by: Tobin Feldman-Fitzthum <tfeldmanfitz@nvidia.com>
2026-02-24 13:54:44 -08:00
Hyounggyu Choi
78d19a4402 Merge pull request #12569 from BbolroC/fix-assertion-guest-pull-runtime-rs
tests: Improve assertion handling for runtime-rs hypervisor
2026-02-24 16:34:40 +01:00
stevenhorsman
ef1b0b2913 runtime: Fix mismatch in receiver names
Fix: `ST1016: methods on the same type should have the same receiver name`

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
1b2ca678e5 runtime: Fix identifier names
Fix identifiers that are non compliant with go's conventions
e.g. not capitalising initialisations

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
69fea195f9 runtime: Fix arm unit test
I think that c727332b0e
broke the arm unit test by removing the arm specific overrides,
so update the expected output

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
b187983f84 workflows: build_checks: skip the go install if installed
Some of our static checks are hitting issues with duplicate
go versions installed. Given that we in go.mod we set the
version to match our required toolchain, if go is already installed
we can let go handle the toolchain version management instead
of installing a second version

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
8f7a2b3d5d runtime: Add copyright & licenses
Add missing headers

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
9b307a5fa6 metrics: Uncapitalise error strings
Fix `T1005: error strings should not be capitalized (staticcheck)`
This is to comply with go conventitions as errors are normally appended,
so there would be a spurious captialisation in the middle of the message

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
6eb67327d0 tests: Use ReplaceAll over Replace
strings.ReplaceAll was introduced in Go 1.12 as a more readable and self-documenting way to say "replace everything".

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
8fc6280f5e log-parser: Use time.IsZero() to check
Using time.IsZero() to check for uninitialised times is clearer

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
c1117bc831 log-parser: Use ReplaceAll over Replace
strings.ReplaceAll was introduced in Go 1.12 as a more readable and self-documenting way to say "replace everything".

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
8311dffce3 log-parser: Apply De Morgan's law
QF1001: Distributing negation across terms and flipping operators, makes it
easy for humans to process expressions at a time, vs evaluating a whole block
and then flipping it and can allow for earlier exit

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
f24765562d csi-kata-directvolume: Fix error messages
Error messages get appended and prepended, so it's against convention
to end them with punctuation

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
f84b462b95 runtime: Fix typo in comment
Fix `requiered` is a misspelling of `required` (misspell)

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
15813564f7 runtime: Avoid using fmt.Sprintf("%s", x)
It's more efficient and concise to just call .String()

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
a577685a8a runtime: Apply De Morgan's law
QF1001: Distributing negation across terms and flipping operators, makes it
easy for humans to process expressions at a time, vs evaluating a whole block
and then flipping it and can allow for earlier exit

Signed-off-by: stevenhorsman <steven@uk.ibm.com>

fixup: demorgans
2026-02-24 14:33:04 +00:00
stevenhorsman
e86338c9c0 runtime: Remove explicit types in variable declarations
QF1011 - use the short declaration as the type can be inferred

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
f60ee411f0 runtime: Update poorly chosen Duration names
ST1011 - having time.Duration values with variable names of MS/Secs
is misleading

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
6562ec5b61 runtime: Merge conditional assignment
Fix `QF1007: could merge conditional assignment into variable declaration`

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
a0ccb63f47 runtime: Use ReplaceAll over Replace
strings.ReplaceAll was introduced in Go 1.12 as a more readable and self-documenting way to say "replace everything".

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
a78d212dfc kata-monitor: Switch to switch statements
Resolve: `QF1003: could use tagged switch`

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
6f438bfb19 runtime: Improve receiver name
Update from `this` to fix:
```
ST1006: receiver name should be a reflection of its identity; don't use generic names such as "this" or "self" (staticcheck)
```

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
f1960103d1 runtime: Improve split statement
strings.SplitN(s, sep, -1) is functionally identical to strings.Split(s, sep)
as -1 says to return all substrings, so choose the more concise version

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
8cd3aa8c84 runtime: Remove embedded field from selector
GenericDevice is an embedded (anonymous) field in the device struct, so its fields
and methods are "promoted" to the outer struct, so we go straight to it.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
4351a61f67 runtime: Fix error string formatting
Resolve `ST1005: error strings should not end with punctuation or newlines (staticcheck)`

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
312567a137 runtime: Fix double imports
Remove one of the double imports to tidy up the code

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
93c77a7d4e runtime: Improve print statement
fix `QF1012: Use fmt.Fprintf(...) instead of Write([]byte(fmt.Sprintf(...))) (staticcheck)`

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:33:04 +00:00
stevenhorsman
cff8994336 runtime: Switch to switch statements
Resolve: `QF1003: could use tagged switch on major (staticcheck)`
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:22:10 +00:00
stevenhorsman
487f530d89 ci: Update golangci configuration
Add a setting to skip the
`T1005: error strings should not be capitalized (staticcheck)`
rule to avoid impact to our error strings

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 14:22:09 +00:00
Hyounggyu Choi
3d71be3dd3 tests: Improve assertion handling for runtime-rs hypervisor
Since runtime-rs added support for virtio-blk-ccw on s390x in #12531,
the assertion in k8s-guest-pull-image.bats should be generalized
to apply to all hypervisors ending with `-runtime-rs`.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2026-02-24 11:48:07 +01:00
stevenhorsman
5ca4c34a34 kata-monitor: Fix golangci-lint warning
QF1012: Use fmt.Fprintf(...) instead of Write([]byte(fmt.Sprintf(...))) (staticcheck)
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 10:02:48 +00:00
stevenhorsman
2ac89f4569 versions: Update golangci-lint
Bump to the latest version to pick up support for Go 1.25

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-24 10:02:48 +00:00
Manuel Huber
566bb306f1 tests: enable policy for openvpn on nydus
Specify runAsUser, runAsGroup, supplementalGroups values embedded
in the image's /etc/group file explicitly in the security context.
With this, both genpolicy and containerd, which in case of using
nydus guest-pull, lack image introspection capabilities, use the
same values for user/group/additionalG IDs at policy generation
time and at runtime when the OCI spec is passed.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2026-02-24 08:08:15 +01:00
Fupan Li
0bfb6b3c45 Merge pull request #12531 from BbolroC/blkdev-hotplug-s390x-runtime-rs
runtime-rs: Support for block device hotplug on s390x
2026-02-24 13:03:59 +08:00
Fabiano Fidêncio
a0d954cf7c tests: Enable auto-generated policies for experimental_force_guest_pull
We want to run with auto-generated policies when using experimental_force_guest_pull.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-23 22:15:18 +01:00
Manuel Huber
e15c18f05c tests: Extend fail timeout for failure test
Extend the timeout for the assert_pod_fail function call for the
test case "Test we cannot pull a large image that pull time exceeds
createcontainer timeout inside the guest" when the experimental
force guest-pull method is being used. In this method, the image is
first pulled on the host before creating the pod sandbox. While
image pull times can suddenly spike, we already time out in the
assert_pod_fail function before the image is even pulled on the
host.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2026-02-23 12:56:23 -08:00
Hyounggyu Choi
4e533f82e7 tests: Remove skip condition for runtime-rs on s390x in k8s-block-volume
This commit removes the skip condition for qemu-runtime-rs on s390x
in k8s-block-volume.bats.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2026-02-23 09:00:29 +01:00
Hyounggyu Choi
2961914f54 runtime-rs: Support for virtio-blk-ccw devices and hotplug
- Introduced `ccw_addr` field in `BlockConfig` for CCW device addresses
- Updated `CcwSubChannel` to handle CCW addresses and channel itself
- Enhanced `QemuInner` to handle CCW subchannel for hotplug operations
- Handled `virtio-blk-ccw` devices in hotplug_block_device()
- Modified resource management to accommodate `ccw_addr`

Fixes: #10373

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2026-02-23 09:00:29 +01:00
Hyounggyu Choi
e893526fad runtime-rs: Reuse constants from kata-types
Some constants are duplicated in runtime-rs even though they
are already defined in kata-types. Use the definitions from
kata-types as the single source of truth to avoid inconsistencies
between components (e.g. agent and runtime).

This change makes runtime-rs use the constants defined in
kata-types.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2026-02-23 09:00:29 +01:00
Hyounggyu Choi
606d193f65 runtime-rs: Set DRIVER_BLK_CCW_TYPE correctly
`DRIVER_BLK_CCW_TYPE` is defined as `blk-ccw`
in src/libs/kata-types/src/device.rs, so set
the variable in runtime-rs accordingly.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2026-02-23 09:00:29 +01:00
Fabiano Fidêncio
b082cf1708 kata-deploy: validate defaultShim is enabled before propagating it
getDefaultShimForArch previously returned whatever string was set in
defaultShim.<arch> without any validation. A typo, a non-existent shim,
or a shim that is disabled via disableAll would all silently produce a
bogus DEFAULT_SHIM_* env var, causing kata-deploy to fail at runtime.

Guard the return value by checking whether the configured shim is
present in the list of shims that are both enabled and support the
requested architecture. If not, return empty string so the env var is
simply omitted.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-21 14:01:11 +01:00
Fabiano Fidêncio
4ff7f67278 kata-deploy: fix nil pointer when custom runtime omits containerd/crio
Using `$runtime.containerd.snapshotter` and `$runtime.crio.pullType`
panics with a nil pointer error when the containerd or crio block is
absent from the custom runtime definition.

Let's use the `dig` function which safely traverses nested keys and
returns an empty string as the default when any key in the path is
missing.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-21 13:59:41 +01:00
Fabiano Fidêncio
96c20f8baa tests: k8s: set CreateContainerRequest (on free runners) timeout to 600s
Set KubeletConfiguration runtimeRequestTimeout to 600s mainly for CoCo
(Confidential Containers) tests, so container creation (attestation,
policy, image pull, VM start) does not hit the default CRI timeout.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-21 08:44:47 +01:00
Fabiano Fidêncio
9634dfa859 gatekeeper: Update tests name
We need to do so after moving some of the tests to the free runners.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-21 08:44:47 +01:00
Fabiano Fidêncio
a6b7a2d8a4 tests: assert_pod_fail accept RunContainerError and StartError
Treat waiting.reason RunContainerError and terminated.reason StartError/Error
as container failure, so tests that expect guest image-pull failure (e.g.
wrong credentials) pass when the container fails with those states instead
of only BackOff.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-21 08:44:47 +01:00
Fabiano Fidêncio
42d980815a tests: skip k8s-policy-pvc on non-AKS
Otherwise it'll fail as we cannot bind the device.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-21 08:44:47 +01:00
Fabiano Fidêncio
1523c48a2b tests: k8s: Align coco / erofs job declaration
Later on we may even think about merging those, but for now let's at
least make sure the envs used are the same / declared in a similar place
for each job.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-21 08:44:47 +01:00
Fabiano Fidêncio
1b9b53248e tests: k8s: coco: rely more on free runners
Run all CoCo non-TEE variants in a single job on the free runner with an
explicit environment matrix (vmm, snapshotter, pull_type, kbs,
containerd_version).

Here we're testing CoCo only with the "active" version of containerd.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-21 08:44:47 +01:00
Fabiano Fidêncio
1fa3475e36 tests: k8s: rely more on free runners
We were running most of the k8s integration tests on AKS. The ones that
don't actually depend on AKS's environment now run on normal
ubuntu-24.04 GitHub runners instead: we bring up a kubeadm cluster
there, test with both containerd lts and active, and skip attestation
tests since those runtimes don't need them. AKS is left only for the
jobs that do depend on it.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-21 08:44:47 +01:00
Fabiano Fidêncio
2f056484f3 versions: Bump containerd active to 2.2
SSIA

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-21 08:44:47 +01:00
Zvonko Kaiser
6d1eaa1065 Merge pull request #12461 from manuelh-dev/mahuber/guest-pull-bats
tests: enable more scenarios for k8s-guest-pull-image.bats
2026-02-20 08:48:54 -05:00
Zvonko Kaiser
1de7dd58f5 gpu: Add NVLSM daemon
We need to chissel the NVLSM daemon for NVL5 systems

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2026-02-20 11:39:59 +01:00
Zvonko Kaiser
67d154fe47 gpu: Enable NVL5 based platform
NVL5 based HGX systems need ib_umad and
fabricmanager and nvlsm installed.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2026-02-20 11:39:59 +01:00
Dan Mihai
ea53779b90 ci: k8s: temporarily disable mariner host
Disable mariner host testing in CI, and auto-generated policy testing
for the temporary replacements of these hosts (based on ubuntu), to work
around missing:

1. cloud-hypervisor/cloud-hypervisor@0a5e79a, that will allow Kata
   in the future to disable the nested property of guest VPs. Nested
   is enabled by default and doesn't work yet with mariner's MSHV.
2. cloud-hypervisor/cloud-hypervisor@bf6f0f8, exposed by the large
   ttrpc replies intentionally produced by the Kata CI Policy tests.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2026-02-19 20:42:50 +01:00
Dan Mihai
3e2153bbae ci: k8s: easier to modify az aks create command
Make `az aks create` command easier to change when needed, by moving the
arguments specific to mariner nodes onto a separate line of this script.
This change also removes the need for `shellcheck disable=SC2046` here.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2026-02-19 20:42:50 +01:00
Fabiano Fidêncio
cadbf51015 versions: Update Cloud Hypervisor to v50.0
```
This release has been tracked in v50.0 group of our roadmap project.

Configurable Nested Virtualization Option on x86_64
The nested=on|off option has been added to --cpu to allow users
to configure nested virtualization support in the guest on x86_64
hosts (for both KVM and MSHV). The default value is on to maintain
consistency with existing behavior. (#7408)

Compression Support for QCOW2
QCOW2 support has been extended to handle compression clusters based on
zlib and zstd. (#7462)

Notable Performance Improvements
Performance of live migration has been improved via an optimized
implementation of dirty bitmap maintenance. (#7468)

Live Disk Resizing Support for Raw Images
The /vm.resize-disk API has been introduced to allow users to resize block
devices backed by raw images while a guest is running. (#7476)

Developer Experience Improvements
Significant improvements have been made to developer experience and
productivity. These include a simplified root manifest, codified and
tightened Clippy lints, and streamlined workflows for cargo clippy and
cargo test. (#7489)

Improved File-level Locking Support
Block devices now use byte-range advisory locks instead of whole-file
locks. While both approaches prevent multiple Cloud Hypervisor instances
from simultaneously accessing the same disk image with write
permissions, byte-range locks provide better compatibility with network
storage backends. (#7494)

Logging Improvements
Logs now include event information generated by the event-monitor
module. (#7512)

Notable Bug Fixes
* Fix several issues around CPUID in the guest (#7485, #7495, #7508)
* Fix snapshot/restore for Windows Guest (#7492)
* Respect queue size in block performance tests (#7515)
* Fix several Serial Manager issues (#7502)
* Fix several seccomp violation issues (#7477, #7497, #7518)
* Fix various issues around block and qcow (#7526, #7528, #7537, #7546,
  #7549)
* Retrieve MSRs list correctly on MSHV (#7543)
* Fix live migration (and snapshot/restore) with AMX state (#7534)
```

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-19 20:42:50 +01:00
Dan Mihai
d8b403437f static-build: delete cloud-hypervisor directory
This cloud-hypervisor is a directory, so it needs "rm -rf" instead of
"rm -f".

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2026-02-19 20:42:50 +01:00
Manuel Huber
fd340ac91c tests: remove skips for some guest-pull scenarios
Issue 10838 is resolved by the prior commit, enabling the -m
option of the kernel build for confidential guests which are
not users of the measured rootfs, and by commit
976df22119, which ensures
relevant user space packages are present.
Not every confidential guest has the measured rootfs option
enabled. Every confidential guest is assumed to support CDH's
secure storage features, in contrast.

We also adjust test timeouts to account for occasional spikes on
our bare metal runners (e.g., SNP, TDX, s390x).

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2026-02-19 10:10:55 -08:00
Harshitha Gowda
728d8656ee tests: Set sev-snp, qemu-snp CIs as required
run-k8s-tests-on-tee (sev-snp, qemu-snp)

Signed-off-by: Harshitha Gowda <hgowda@amd.com>
2026-02-19 16:41:29 +01:00
Fabiano Fidêncio
855f4dc7fa release: Bump version to 3.27.0
Bump VERSION and helm-charts versions.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-19 14:01:26 +01:00
Markus Rudy
0621e65e74 genpolicy: allow RO and RW for sysfs with privileged container
After containerd 2.0.4, privileged containers handle sysfs mounts a bit
differently, so we can end up with the policy expecting RO and the input
having RW.

The sandbox needs to get privileged mounts when any container in the pod
is privileged, not only when the pause container itself is marked
privileged. So we now compute that and pass it into get_mounts.

One downside: we’re relaxing policy checks (accepting RO/RW mismatch for
sysfs) and giving the pause container privileged mounts whenever the pod
has any privileged workload. For Kata, that means a slightly broader
attack surface for privileged pods—the pause container sees more than it
strictly needs, and we’re being more permissive on sysfs.

It’s a trade-off for compatibility with newer containerd; if you need
maximum isolation, you may want to avoid privileged pods or tighten
policy elsewhere.

Fixes: #12532

Signed-off-by: Markus Rudy <mr@edgeless.systems>
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-19 11:16:50 +01:00
Amulyam24
a22c59a204 kata-deploy: enable kata-remote for ppc64le
When kata-deploy is deployed with cloud-api-adaptor, it
defaults to qemu instead of configuring the remote shim.
Support ppc64le to enable it correctly when shims.remote.enabled=true

Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
2026-02-19 11:14:27 +01:00
Steve Horsman
6a67250397 Merge commit from fork
runtime-go/rs: Disable virtio-pmem for Cloud Hypervisor
2026-02-19 09:00:56 +00:00
Chiranjeevi Uddanti
88203cbf8d tests: Add regression test for sandbox_cgroup_only=false
Add unit test for get_ch_vcpu_tids() and integration test that creates
a pod with sandbox_cgroup_only=false to verify it starts successfully.

Signed-off-by: Chiranjeevi Uddanti <244287281+chiranjeevi-max@users.noreply.github.com>
Co-authored-by: Antigravity <antigravityagent@google.com>
2026-02-18 20:20:14 +01:00
Chiranjeevi Uddanti
9c52f0caa7 runtime-rs/ch: Fix inverted vcpu/tid mapping in get_ch_vcpu_tids
The VcpuThreadIds struct expects a mapping from vcpu_id to thread_id,
but get_ch_vcpu_tids() was inserting (tid, vcpu_id) instead of
(vcpu_id, tid).

This caused move_vcpus_to_sandbox_cgroup() to interpret vcpu IDs
(0, 1, 2...) as process IDs when sandbox_cgroup_only=false, leading
to failed attempts to read /proc/0/status.

Fixes: #12479
Signed-off-by: Chiranjeevi Uddanti <244287281+chiranjeevi-max@users.noreply.github.com>
2026-02-18 20:20:14 +01:00
Aurélien Bombo
8ff9cd1f12 Merge pull request #12455 from ajaypvictor/secret-cm-without-sharedfs
ci: Add integration tests for secret & configmap propagation
2026-02-18 12:06:48 -06:00
Aurélien Bombo
336b922d4f tests/cbl-mariner: Stop disabling NVDIMM explicitly
This is not needed anymore since now disable_image_nvdimm=true for
Cloud Hypervisor.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2026-02-18 11:52:51 -06:00
Aurélien Bombo
48aa077e8c runtime{,-rs}/qemu/arm64: Disable DAX
Enabling full-featured QEMU NVDIMM support on ARM with DAX enabled causes a
kernel panic in caches_clean_inval_pou (see below, different issue from
33b1f07), so we disable DAX in that environment.

[    1.222529] EXT4-fs (pmem0p1): mounted filesystem e5a4892c-dac8-42ee-ba55-27d4ff2f38c3 ro with ordered data mode. Quota mode: disabled.
[    1.222695] VFS: Mounted root (ext4 filesystem) readonly on device 259:1.
[    1.224890] devtmpfs: mounted
[    1.225175] Freeing unused kernel memory: 1920K
[    1.226102] Run /sbin/init as init process
[    1.226164]   with arguments:
[    1.226204]     /sbin/init
[    1.226235]   with environment:
[    1.226268]     HOME=/
[    1.226295]     TERM=linux
[    1.230974] Internal error: synchronous external abort: 0000000096000010 [#1]  SMP
[    1.231963] CPU: 0 UID: 0 PID: 1 Comm: init Tainted: G   M                6.18.5 #1 NONE
[    1.232965] Tainted: [M]=MACHINE_CHECK
[    1.233428] pstate: 43400005 (nZcv daif +PAN -UAO +TCO +DIT -SSBS BTYPE=--)
[    1.234273] pc : caches_clean_inval_pou+0x68/0x84
[    1.234862] lr : sync_icache_aliases+0x30/0x38
[    1.235412] sp : ffff80008000b9a0
[    1.235842] x29: ffff80008000b9a0 x28: 0000000000000000 x27: 00000000019a00e1
[    1.236912] x26: ffff80008000bc08 x25: ffff80008000baf0 x24: fffffdffc0000000
[    1.238064] x23: ffff000001671ab0 x22: ffff000001663480 x21: fffffdffc23401c0
[    1.239356] x20: fffffdffc23401c0 x19: fffffdffc23401c0 x18: 0000000000000000
[    1.240626] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000
[    1.241762] x14: ffffaae8f021b3b0 x13: 0000000000000000 x12: ffffaae8f021b3b0
[    1.242874] x11: ffffffffffffffff x10: 0000000000000000 x9 : 0000ffffbb53c000
[    1.244022] x8 : 0000000000000000 x7 : 0000000000000012 x6 : ffff55178f5e5000
[    1.245157] x5 : ffff80008000b970 x4 : ffff00007fa4f680 x3 : ffff00008d007000
[    1.246257] x2 : 0000000000000040 x1 : ffff00008d008000 x0 : ffff00008d007000
[    1.247387] Call trace:
[    1.248056]  caches_clean_inval_pou+0x68/0x84 (P)
[    1.248923]  __sync_icache_dcache+0x7c/0x9c
[    1.249578]  insert_page_into_pte_locked+0x1e4/0x284
[    1.250432]  insert_page+0xa8/0xc0
[    1.251080]  vmf_insert_page_mkwrite+0x40/0x7c
[    1.251832]  dax_iomap_pte_fault+0x598/0x804
[    1.252646]  dax_iomap_fault+0x28/0x30
[    1.253293]  ext4_dax_huge_fault+0x80/0x2dc
[    1.253988]  ext4_dax_fault+0x10/0x3c
[    1.254679]  __do_fault+0x38/0x12c
[    1.255293]  __handle_mm_fault+0x530/0xcf0
[    1.255990]  handle_mm_fault+0xe4/0x230
[    1.256697]  do_page_fault+0x17c/0x4dc
[    1.257487]  do_translation_fault+0x30/0x38
[    1.258184]  do_mem_abort+0x40/0x8c
[    1.258895]  el0_ia+0x4c/0x170
[    1.259420]  el0t_64_sync_handler+0xd8/0xdc
[    1.260154]  el0t_64_sync+0x168/0x16c
[    1.260795] Code: d2800082 9ac32042 d1000443 8a230003 (d50b7523)
[    1.261756] ---[ end trace 0000000000000000 ]---

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2026-02-18 11:52:43 -06:00
Aurélien Bombo
c727332b0e runtime/qemu/arm64: Align NVDIMM usage on amd64
Nowadays on arm64 we use a modern QEMU version which supports the features we
require for NVDIMM, so we remove the arm64-specific code and use the generic
implementation.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2026-02-18 11:47:53 -06:00
Aurélien Bombo
e17f96251d runtime{,-rs}/clh: Disable virtio-pmem
This disables virtio-pmem support for Cloud Hypervisor by changing
Kata config defaults and removing the relevant code paths.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2026-02-18 11:47:53 -06:00
Zvonko Kaiser
1d09e70233 Merge pull request #12538 from fidencio/topic/kata-deploy-fix-regression-on-hardcopying-symlinks
kata-deploy: preserve symlinks when installing artifacts
2026-02-18 12:44:46 -05:00
Mikko Ylinen
5622ab644b versions: bump QEMU to v10.2.1
v10.2.1 is the latest patch release in v10.2 series. Changes:
https://github.com/qemu/qemu/compare/v10.2.0...v10.2.1

Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
2026-02-18 18:18:52 +01:00
Mikko Ylinen
d68adc54da versions: bump to Linux v6.18.12 (LTS)
Latest changelog in
https://cdn.kernel.org/pub/linux/kernel/v6.x/ChangeLog-6.18.12

Also other changes for 6..11 updates are available.

Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
2026-02-18 18:18:52 +01:00
Fabiano Fidêncio
34336f87c7 kata-deploy: convert install.rs get_hypervisor_name tests to rstest
Use rstest parameterized tests for QEMU variants, other hypervisors,
and unknown/empty shim cases.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-18 12:41:55 +01:00
Fabiano Fidêncio
bb11bf0403 kata-deploy: preserve symlinks when installing artifacts
When copying artifacts from the container to the host, detect source
entries that are symlinks and recreate them as symlinks at the
destination instead of copying the target file.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-18 12:29:14 +01:00
Dan Mihai
eee25095b5 tests: mariner annotations for k8s-openvpn
This test uses YAML files from a different directory than the other
k8s CI tests, so annotations have to be added into these separate
files.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2026-02-18 07:17:04 +01:00
Manuel Huber
4c760fd031 build: add CONFIDENTIAL_GUEST variable for kernel
This change adds the CONFIDENTIAL_GUEST variable to the kernel
build logic. Similar to commit
976df22119, we would like to enable
the cryptsetup functionalities not only when building a measured
root file system, but also when building for a confidential guest.
The current state is that not all confidential guests use a
measured root filesystem, and as a matter of fact, we should
indeed decouple these aspects.

With the current convention, a confidential guest is a user of CDH
with its storage features. A better naming of the
CONFIDENTIAL_GUEST variable could have been a naming related to CDH
storage functionality. Further, the kernel build script's -m
parameter could be improved too - as indicated by this change, not
only measured rootfs builds will need the cryptsetup.conf file.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2026-02-17 12:44:50 -08:00
Manuel Huber
d3742ca877 tests: enable guest pull bats for force guest pull
Similar to k8s-guest-pull-image-authenticated and to
k8s-guest-pull-image-signature, enabling k8s-guest-pull-image to
run against the experimental force guest pull method.
Only k8s-guest-pull-image-encrypted requires nydus.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2026-02-17 12:44:50 -08:00
Markus Rudy
8365afa336 qemu: log exit code after failure
When qemu exits prematurely, we usually see a message like

  msg="Cannot start VM" error="exiting QMP loop, command cancelled"

This is an indirect hint, caused by the QMP server shutting down. It
takes experience to understand what it even means, and it still does not
show what's actually the problem.

With this commit, we're taking the error return from the qemu
subprocess and surface it in the logs, if it's not nil. This means we
automatically capture any non-zero exit codes in the logs.

Signed-off-by: Markus Rudy <mr@edgeless.systems>
2026-02-17 21:03:13 +01:00
Fabiano Fidêncio
f0a0425617 kata-deploy: convert a few toml.rs tests to rstest
Turn test_toml_value_types into a parameterized test with one case per type
(string, bool, int). Merge the two invalid-TOML tests (get and set) into one
rstest with two cases, and the two "not an array" tests into one rstest
with two cases.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-17 09:33:39 +01:00
Fabiano Fidêncio
899005859c kata-deploy: avoid leading/blank lines in written TOML config
When writing containerd drop-in or other TOML (e.g. initially empty file),
the serialized document could start with many newlines.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-17 09:33:39 +01:00
Fabiano Fidêncio
cfa8188cad kata-deploy: convert containerd version support tests to rstest
Replace multiple #[test] functions for snapshotter and erofs version
checks with parameterized #[rstest] #[case] tests for consistency and
easier extension.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-17 09:33:39 +01:00
Fabiano Fidêncio
cadac7a960 kata-deploy: runtime_platform -> runtime_platforms
Fix runtime_platforms typo.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-17 09:33:39 +01:00
Hyounggyu Choi
8bc60a0761 Merge pull request #12521 from fidencio/topic/kata-deploy-auto-add-nfd-tee-labels-to-the-runtime-class
kata-deploy: Add TEE nodeSelectors for TEE shims when NFD is detected
2026-02-16 18:06:18 +01:00
Jacek Tomasiak
8025fa0457 agent: Don't pass empty options to mount
With some older kernels some fs implementations don't handle empty
options strings well. This leads to failures in "setup rootfs" step.
E.g. `cgroup: cgroup2: unknown option ""`.
This is fixed by mapping empty string to `None` before passing to
`nix::mount`.

Signed-off-by: Jacek Tomasiak <jtomasiak@arista.com>
Signed-off-by: Jacek Tomasiak <jacek.tomasiak@gmail.com>
2026-02-16 14:55:59 +01:00
Fabiano Fidêncio
a04df4f4cb kata-deploy: disable provenance/SBOM for quay.io compatibility
Disable provenance and SBOM when building per-arch kata-deploy images so
each tag is a single image manifest. quay.io rejects pushing multi-arch
manifest lists that include attestation manifests (400 manifest invalid).
Add a note in the release script documenting this.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-16 13:32:25 +01:00
Fabiano Fidêncio
0e8e30d6b5 kata-deploy: fix default RuntimeClass + nodeSelectors
The default RuntimeClass (e.g. kata) is meant to point at the default shim
handler (e.g. kata-qemu-$tee). We were building it in a separate block and
only sometimes adding the same TEE nodeSelectors as the shim-specific
RuntimeClass, leading to kata ending up without the SE/SNP/TDX
nodeSelector while kata-qemu-$tee had it.

The fix is to stop duplicating the RuntimeClass definition, having a
single template that renders one RuntimeClass (name, handler, overhead,
nodeSelectors).

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-16 13:09:03 +01:00
Fabiano Fidêncio
80a175d09b kata-deploy: Add TEE nodeSelectors for TEE shims when NFD is detected
When NFD is detected (deployed by the chart or existing in the cluster),
apply shim-specific nodeSelectors only for TEE runtime classes (snp,
tdx, and se).

Non-TEE shims keep existing behavior (e.g. runtimeClass.nodeSelector for
nvidia GPU from f3bba0885 is unchanged).

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-16 12:07:51 +01:00
Fabiano Fidêncio
d000acfe08 infra: fix multi-arch manifest publish
Per-arch images were failing publish-multiarch-manifest with 'X is a manifest
list' because Buildx now enables attestations by default, so each arch tag
became an image index. Use 'docker buildx imagetools create' instead of
'docker manifest create' so we can merge those indexes into the final
multi-arch manifest while keeping provenance and SBOM on per-arch images.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-14 19:49:00 +01:00
Fabiano Fidêncio
02c9a4b23c kata-deploy: Temporarily comment GPU specific labels
We depend on GPU Operator v26.3 release, which is not out yet.
Although we have been testing with it, it's not yet publicly available,
which would break anyone actually trying to use the GPU runtime classes.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-14 09:25:14 +01:00
Fupan Li
2149fc0eee runtime-rs: send the task delete event to containerd
According to shimv2 proto, it should send task delete event to
containerd once a container task delete succesfully.

Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
2026-02-14 12:44:31 +08:00
Fupan Li
d2613025b7 runtime-rs: send the task create event to containerd
According to shimv2 proto, it should send task create event to
containerd once a container task create succesfully.

Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
2026-02-14 12:44:23 +08:00
Fupan Li
499e18c876 runtime-rs: send the task start event to container
According to shimv2 proto, it should send task start event to
containerd once a container task start succesfully.

Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
2026-02-14 12:44:03 +08:00
Ajay Victor
83935e005c ci: Add integration tests for secret & configmap propagation
Enhance k8s-configmap.bats and k8s-credentials-secrets.bats to test that ConfigMap and Secret updates propagate to volume-mounted pods.

- Enhanced k8s-configmap.bats to test ConfigMap propagation
  * Added volume mount test for ConfigMap consumption
  * Added verification that ConfigMap updates propagate to volume-mounted pods

- Enhanced k8s-credentials-secrets.bats to test Secret propagation
  * Added verification that Secret updates propagate to volume-mounted pods

Fixes #8015

Signed-off-by: Ajay Victor <ajvictor@in.ibm.com>
2026-02-14 08:56:21 +05:30
Fabiano Fidêncio
5106e7b341 build: Add gnupg to the agent's builder container
Otherwise we'll fail to check gperf's GPG signing key when needed.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-14 00:33:45 +01:00
stevenhorsman
79b5022a5a kata-ctl: Bump rkyv version to 0.7.46
Bump to remediate RUSTSEC-2026-0001

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-14 00:33:45 +01:00
stevenhorsman
30ebc4241e genpolicy: Bump rkyv version to 0.7.46
Bump to remediate RUSTSEC-2026-0001

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-14 00:33:45 +01:00
stevenhorsman
87d1979c84 agent-ctl: Bump rkyv version to 0.7.46
Bump to remediate RUSTSEC-2026-0001

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-14 00:33:45 +01:00
stevenhorsman
90dbd3f562 agent: Bump rkyv version to 0.7.46
Bump to remediate RUSTSEC-2026-0001

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-14 00:33:45 +01:00
stevenhorsman
7f77948658 versions: Bump rkyv version to 0.7.46
Bump to remediate RUSTSEC-2026-0001

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-14 00:33:45 +01:00
Aurélien Bombo
981f693a88 Merge pull request #11140 from balintTobik/hyperv_warning
runtime: refactor hypervisor devices cgroup creation
2026-02-13 15:16:09 -06:00
Fabiano Fidêncio
d8acc403c8 kata-deploy: set CRI images runtime_platform snapshotter for containerd v3
In containerd config v3 the CRI plugin is split into runtime and images,
and setting the snapshotter only on the runtime plugin is not enough for image
pull/prepare.

The images plugin must have runtime_platform.<runtime>.snapshotter so it
uses the correct snapshotter per runtime (e.g. nydus, erofs).

A PR on the containerd side is open so we can rely on the runtime plugin
snapshotter alone: https://github.com/containerd/containerd/pull/12836

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-13 22:15:02 +01:00
Fabiano Fidêncio
2930c68c0b ci: tdx: properly skip k8s-sandbox-vcpus-allocation.bats
This is a follow-up for 25962e9325

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-13 20:56:08 +01:00
Balint Tobik
295a6a81d0 runtime: refactor hypervisor devices cgroup creation
Separatly added hypervisor devices to cgroup to
omit not relevant warnings and fail if none of them
are available.
Also fix a testcase reload removed kernel modules to later testcases
and skip some tests on ARM because lack of virtualization support
Fixes #6656

Signed-off-by: Balint Tobik <btobik@redhat.com>
2026-02-13 09:23:08 +01:00
stevenhorsman
c5aadada98 workflows: Pin all actions
Previously zizmor only mandated pinning of third-party actions,
but has recommended rolling this out to all actions now.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-12 16:26:45 +00:00
stevenhorsman
cdd7c35c10 workflows: Remove unneeded strategy
In a refactor we've remove the `matrix` section of this strategy, so
the whole section isn't needed any more, so clean this up.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-12 16:26:45 +00:00
stevenhorsman
379eecacb0 dependabot: Add cooldown
It's good practice to get dependabot to wait after a release before bumping
to avoid it bumping to a release done seconds before, which could have
supply-chain security implications, so add a 7 day cooldown to help with this.

See https://blog.yossarian.net/2025/11/21/We-should-all-be-using-dependency-cooldowns

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-02-12 16:26:45 +00:00
404 changed files with 42158 additions and 5627 deletions

View File

@@ -28,3 +28,9 @@ self-hosted-runner:
- s390x-large
- tdx
- ubuntu-24.04-arm
paths:
.github/workflows/**/*.{yml,yaml}:
ignore:
# We use if: false to "temporarily" skip jobs with issues
- 'constant expression "false" in condition'

View File

@@ -15,6 +15,8 @@ updates:
- "/src/tools/trace-forwarder"
schedule:
interval: "daily"
cooldown:
default-days: 7
ignore:
# rust-vmm repos might cause incompatibilities on patch versions, so
# lets handle them manually for now.
@@ -85,8 +87,12 @@ updates:
- "src/tools/csi-kata-directvolume"
schedule:
interval: "daily"
cooldown:
default-days: 7
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "monthly"
cooldown:
default-days: 7

View File

@@ -13,18 +13,13 @@ concurrency:
jobs:
run-actionlint:
name: run-actionlint
env:
GH_TOKEN: ${{ github.token }}
runs-on: ubuntu-24.04
steps:
- name: Checkout the code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
persist-credentials: false
- name: Install actionlint gh extension
run: gh extension install https://github.com/cschleiden/gh-actionlint
- name: Run actionlint
run: gh actionlint
uses: raven-actions/actionlint@e01d1ea33dd6a5ed517d95b4c0c357560ac6f518 # v2.1.1

View File

@@ -47,6 +47,23 @@ jobs:
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install yq
run: |
./ci/install_yq.sh
env:
INSTALL_IN_GOPATH: false
- name: Read properties from versions.yaml
run: |
go_version="$(yq '.languages.golang.version' versions.yaml)"
[ -n "$go_version" ]
echo "GO_VERSION=${go_version}" >> "$GITHUB_ENV"
- name: Setup Golang version ${{ env.GO_VERSION }}
uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0
with:
go-version: ${{ env.GO_VERSION }}
- name: Install dependencies
run: bash tests/integration/cri-containerd/gha-run.sh install-dependencies
env:

View File

@@ -47,8 +47,25 @@ jobs:
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install yq
run: |
./ci/install_yq.sh
env:
INSTALL_IN_GOPATH: false
- name: Read properties from versions.yaml
run: |
go_version="$(yq '.languages.golang.version' versions.yaml)"
[ -n "$go_version" ]
echo "GO_VERSION=${go_version}" >> "$GITHUB_ENV"
- name: Setup Golang version ${{ env.GO_VERSION }}
uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0
with:
go-version: ${{ env.GO_VERSION }}
- name: Install dependencies
run: bash tests/integration/cri-containerd/gha-run.sh
run: bash tests/integration/cri-containerd/gha-run.sh install-dependencies
env:
GH_TOKEN: ${{ github.token }}

View File

@@ -82,11 +82,17 @@ jobs:
./ci/install_yq.sh
env:
INSTALL_IN_GOPATH: false
- name: Install golang
- name: Read properties from versions.yaml
if: contains(matrix.component.needs, 'golang')
run: |
./tests/install_go.sh -f -p
echo "/usr/local/go/bin" >> "$GITHUB_PATH"
go_version="$(yq '.languages.golang.version' versions.yaml)"
[ -n "$go_version" ]
echo "GO_VERSION=${go_version}" >> "$GITHUB_ENV"
- name: Setup Golang version ${{ env.GO_VERSION }}
if: contains(matrix.component.needs, 'golang')
uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0
with:
go-version: ${{ env.GO_VERSION }}
- name: Setup rust
if: contains(matrix.component.needs, 'rust')
run: |

View File

@@ -94,11 +94,19 @@ jobs:
./ci/install_yq.sh
env:
INSTALL_IN_GOPATH: false
- name: Install golang
- name: Read properties from versions.yaml
if: contains(matrix.component.needs, 'golang')
run: |
./tests/install_go.sh -f -p
echo "/usr/local/go/bin" >> "$GITHUB_PATH"
go_version="$(yq '.languages.golang.version' versions.yaml)"
[ -n "$go_version" ]
echo "GO_VERSION=${go_version}" >> "$GITHUB_ENV"
- name: Setup Golang version ${{ env.GO_VERSION }}
if: contains(matrix.component.needs, 'golang')
uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0
with:
go-version: ${{ env.GO_VERSION }}
# Setup-go doesn't work properly with ppc64le: https://github.com/actions/setup-go/issues/648
architecture: ${{ contains(inputs.instance, 'ppc64le') && 'ppc64le' || '' }}
- name: Setup rust
if: contains(matrix.component.needs, 'rust')
run: |

View File

@@ -143,7 +143,7 @@ jobs:
if-no-files-found: error
- name: store-extratarballs-artifact ${{ matrix.asset }}
if: ${{ matrix.asset == 'kernel' || startsWith(matrix.asset, 'kernel-nvidia-gpu') }}
if: ${{ startsWith(matrix.asset, 'kernel-nvidia-gpu') }}
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: kata-artifacts-amd64-${{ matrix.asset }}-modules${{ inputs.tarball-suffix }}
@@ -235,7 +235,6 @@ jobs:
asset:
- busybox
- coco-guest-components
- kernel-modules
- kernel-nvidia-gpu-modules
- pause-image
steps:

View File

@@ -120,15 +120,6 @@ jobs:
retention-days: 15
if-no-files-found: error
- name: store-extratarballs-artifact ${{ matrix.asset }}
if: ${{ matrix.asset == 'kernel' }}
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: kata-artifacts-s390x-${{ matrix.asset }}-modules${{ inputs.tarball-suffix }}
path: kata-build/kata-static-${{ matrix.asset }}-modules.tar.zst
retention-days: 15
if-no-files-found: error
build-asset-rootfs:
name: build-asset-rootfs
runs-on: s390x

View File

@@ -297,6 +297,21 @@ jobs:
AZ_TENANT_ID: ${{ secrets.AZ_TENANT_ID }}
AZ_SUBSCRIPTION_ID: ${{ secrets.AZ_SUBSCRIPTION_ID }}
run-k8s-tests-on-free-runner:
if: ${{ inputs.skip-test != 'yes' }}
needs: publish-kata-deploy-payload-amd64
permissions:
contents: read
uses: ./.github/workflows/run-k8s-tests-on-free-runner.yaml
with:
tarball-suffix: -${{ inputs.tag }}
registry: ghcr.io
repo: ${{ github.repository_owner }}/kata-deploy-ci
tag: ${{ inputs.tag }}-amd64
commit-hash: ${{ inputs.commit-hash }}
pr-number: ${{ inputs.pr-number }}
target-branch: ${{ inputs.target-branch }}
run-k8s-tests-on-arm64:
if: ${{ inputs.skip-test != 'yes' }}
needs: publish-kata-deploy-payload-arm64

View File

@@ -72,7 +72,7 @@ jobs:
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
uses: github/codeql-action/init@4bdb89f48054571735e3792627da6195c57459e2 # v3.31.10
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
@@ -95,6 +95,6 @@ jobs:
make -C src/runtime
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
uses: github/codeql-action/analyze@4bdb89f48054571735e3792627da6195c57459e2 # v3.31.10
with:
category: "/language:${{matrix.language}}"

View File

@@ -31,10 +31,22 @@ jobs:
with:
persist-credentials: false
- name: Install golang
- name: Install yq
run: |
./tests/install_go.sh -f -p
echo "/usr/local/go/bin" >> "${GITHUB_PATH}"
./ci/install_yq.sh
env:
INSTALL_IN_GOPATH: false
- name: Read properties from versions.yaml
run: |
go_version="$(yq '.languages.golang.version' versions.yaml)"
[ -n "$go_version" ]
echo "GO_VERSION=${go_version}" >> "$GITHUB_ENV"
- name: Setup Golang version ${{ env.GO_VERSION }}
uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0
with:
go-version: ${{ env.GO_VERSION }}
- name: Install Rust
run: ./tests/install_rust.sh

View File

@@ -24,10 +24,22 @@ jobs:
fetch-depth: 0
persist-credentials: false
- name: Install golang
- name: Install yq
run: |
./tests/install_go.sh -f -p
echo "/usr/local/go/bin" >> "${GITHUB_PATH}"
./ci/install_yq.sh
env:
INSTALL_IN_GOPATH: false
- name: Read properties from versions.yaml
run: |
go_version="$(yq '.languages.golang.version' versions.yaml)"
[ -n "$go_version" ]
echo "GO_VERSION=${go_version}" >> "$GITHUB_ENV"
- name: Setup Golang version ${{ env.GO_VERSION }}
uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0
with:
go-version: ${{ env.GO_VERSION }}
- name: Docs URL Alive Check
run: |

View File

@@ -16,17 +16,17 @@ jobs:
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
steps:
- uses: actions/configure-pages@v5
- uses: actions/checkout@v5
- uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5.0.0
- uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5.0.1
with:
persist-credentials: false
- uses: actions/setup-python@v5
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: 3.x
- run: pip install zensical
- run: zensical build --clean
- uses: actions/upload-pages-artifact@v4
- uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4.0.0
with:
path: site
- uses: actions/deploy-pages@v4
- uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4.0.5
id: deployment

View File

@@ -27,10 +27,22 @@ jobs:
fetch-depth: 0
persist-credentials: false
- name: Install golang
- name: Install yq
run: |
./tests/install_go.sh -f -p
echo "/usr/local/go/bin" >> "${GITHUB_PATH}"
./ci/install_yq.sh
env:
INSTALL_IN_GOPATH: false
- name: Read properties from versions.yaml
run: |
go_version="$(yq '.languages.golang.version' versions.yaml)"
[ -n "$go_version" ]
echo "GO_VERSION=${go_version}" >> "$GITHUB_ENV"
- name: Setup Golang version ${{ env.GO_VERSION }}
uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0
with:
go-version: ${{ env.GO_VERSION }}
- name: Install govulncheck
run: |

View File

@@ -35,8 +35,6 @@ on:
jobs:
run-cri-containerd:
name: run-cri-containerd-${{ inputs.arch }} (${{ inputs.containerd_version }}, ${{ inputs.vmm }})
strategy:
fail-fast: false
runs-on: ${{ inputs.runner }}
env:
CONTAINERD_VERSION: ${{ inputs.containerd_version }}
@@ -55,6 +53,23 @@ jobs:
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install yq
run: |
./ci/install_yq.sh
env:
INSTALL_IN_GOPATH: false
- name: Read properties from versions.yaml
run: |
go_version="$(yq '.languages.golang.version' versions.yaml)"
[ -n "$go_version" ]
echo "GO_VERSION=${go_version}" >> "$GITHUB_ENV"
- name: Setup Golang version ${{ env.GO_VERSION }}
uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0
with:
go-version: ${{ env.GO_VERSION }}
- name: Install dependencies
timeout-minutes: 15
run: bash tests/integration/cri-containerd/gha-run.sh install-dependencies

View File

@@ -42,17 +42,6 @@ jobs:
strategy:
fail-fast: false
matrix:
host_os:
- ubuntu
vmm:
- clh
- dragonball
- qemu
- qemu-runtime-rs
- cloud-hypervisor
instance-type:
- small
- normal
include:
- host_os: cbl-mariner
vmm: clh
@@ -80,6 +69,7 @@ jobs:
KUBERNETES: "vanilla"
K8S_TEST_HOST_TYPE: ${{ matrix.instance-type }}
GENPOLICY_PULL_METHOD: ${{ matrix.genpolicy-pull-method }}
RUNS_ON_AKS: "true"
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:

View File

@@ -0,0 +1,127 @@
# Run Kubernetes integration tests on free GitHub runners with a locally
# deployed cluster (kubeadm).
name: CI | Run kubernetes tests on free runner
on:
workflow_call:
inputs:
tarball-suffix:
required: false
type: string
registry:
required: true
type: string
repo:
required: true
type: string
tag:
required: true
type: string
pr-number:
required: true
type: string
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
permissions: {}
jobs:
run-k8s-tests:
name: run-k8s-tests
strategy:
fail-fast: false
matrix:
environment: [
{ vmm: clh, containerd_version: lts },
{ vmm: clh, containerd_version: active },
{ vmm: dragonball, containerd_version: lts },
{ vmm: dragonball, containerd_version: active },
{ vmm: qemu, containerd_version: lts },
{ vmm: qemu, containerd_version: active },
{ vmm: qemu-runtime-rs, containerd_version: lts },
{ vmm: qemu-runtime-rs, containerd_version: active },
{ vmm: cloud-hypervisor, containerd_version: lts },
{ vmm: cloud-hypervisor, containerd_version: active },
]
runs-on: ubuntu-24.04
permissions:
contents: read
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
GH_PR_NUMBER: ${{ inputs.pr-number }}
KATA_HOST_OS: ubuntu
KATA_HYPERVISOR: ${{ matrix.environment.vmm }}
KUBERNETES: vanilla
K8S_TEST_HOST_TYPE: baremetal-no-attestation
CONTAINER_ENGINE: containerd
CONTAINER_ENGINE_VERSION: ${{ matrix.environment.containerd_version }}
GH_TOKEN: ${{ github.token }}
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
persist-credentials: false
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: get-kata-tools-tarball
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: kata-tools-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-tools-artifacts
- name: Install kata-tools
run: bash tests/integration/kubernetes/gha-run.sh install-kata-tools kata-tools-artifacts
- name: Remove unnecessary directories to free up space
run: |
sudo rm -rf /usr/local/.ghcup
sudo rm -rf /opt/hostedtoolcache/CodeQL
sudo rm -rf /usr/local/lib/android
sudo rm -rf /usr/share/dotnet
sudo rm -rf /opt/ghc
sudo rm -rf /usr/local/share/boost
sudo rm -rf /usr/lib/jvm
sudo rm -rf /usr/share/swift
sudo rm -rf /usr/local/share/powershell
sudo rm -rf /usr/local/julia*
sudo rm -rf /opt/az
sudo rm -rf /usr/local/share/chromium
sudo rm -rf /opt/microsoft
sudo rm -rf /opt/google
sudo rm -rf /usr/lib/firefox
- name: Deploy k8s (kubeadm)
run: bash tests/integration/kubernetes/gha-run.sh deploy-k8s
- name: Install `bats`
run: bash tests/integration/kubernetes/gha-run.sh install-bats
- name: Deploy Kata
timeout-minutes: 20
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata
- name: Run tests
timeout-minutes: 60
run: bash tests/integration/kubernetes/gha-run.sh run-tests
- name: Report tests
if: always()
run: bash tests/integration/kubernetes/gha-run.sh report-tests
- name: Delete kata-deploy
if: always()
timeout-minutes: 15
run: bash tests/integration/kubernetes/gha-run.sh cleanup

View File

@@ -57,10 +57,24 @@ jobs:
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install golang
- name: Install yq
run: |
./tests/install_go.sh -f -p
echo "/usr/local/go/bin" >> "$GITHUB_PATH"
./ci/install_yq.sh
env:
INSTALL_IN_GOPATH: false
- name: Read properties from versions.yaml
run: |
go_version="$(yq '.languages.golang.version' versions.yaml)"
[ -n "$go_version" ]
echo "GO_VERSION=${go_version}" >> "$GITHUB_ENV"
- name: Setup Golang version ${{ env.GO_VERSION }}
uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0
with:
go-version: ${{ env.GO_VERSION }}
# Setup-go doesn't work properly with ppc64le: https://github.com/actions/setup-go/issues/648
architecture: 'ppc64le'
- name: Prepare the runner for k8s test suite
run: bash "${HOME}/scripts/k8s_cluster_prepare.sh"

View File

@@ -140,165 +140,36 @@ jobs:
strategy:
fail-fast: false
matrix:
vmm:
- qemu-coco-dev
- qemu-coco-dev-runtime-rs
snapshotter:
- nydus
pull-type:
- guest-pull
include:
- pull-type: experimental-force-guest-pull
vmm: qemu-coco-dev
snapshotter: ""
runs-on: ubuntu-22.04
environment: [
{ vmm: qemu-coco-dev, snapshotter: nydus, pull_type: guest-pull },
{ vmm: qemu-coco-dev-runtime-rs, snapshotter: nydus, pull_type: guest-pull },
{ vmm: qemu-coco-dev, snapshotter: "", pull_type: experimental-force-guest-pull },
]
runs-on: ubuntu-24.04
permissions:
id-token: write # Used for OIDC access to log into Azure
contents: read
environment: ci
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
GH_PR_NUMBER: ${{ inputs.pr-number }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
KATA_HYPERVISOR: ${{ matrix.environment.vmm }}
# Some tests rely on that variable to run (or not)
KBS: "true"
# Set the KBS ingress handler (empty string disables handling)
KBS_INGRESS: "aks"
KBS_INGRESS: "nodeport"
KUBERNETES: "vanilla"
PULL_TYPE: ${{ matrix.pull-type }}
PULL_TYPE: ${{ matrix.environment.pull_type }}
AUTHENTICATED_IMAGE_USER: ${{ vars.AUTHENTICATED_IMAGE_USER }}
AUTHENTICATED_IMAGE_PASSWORD: ${{ secrets.AUTHENTICATED_IMAGE_PASSWORD }}
SNAPSHOTTER: ${{ matrix.snapshotter }}
EXPERIMENTAL_FORCE_GUEST_PULL: ${{ matrix.pull-type == 'experimental-force-guest-pull' && matrix.vmm || '' }}
# Caution: current ingress controller used to expose the KBS service
# requires much vCPUs, lefting only a few for the tests. Depending on the
# host type chose it will result on the creation of a cluster with
# insufficient resources.
SNAPSHOTTER: ${{ matrix.environment.snapshotter }}
EXPERIMENTAL_FORCE_GUEST_PULL: ${{ matrix.environment.pull_type == 'experimental-force-guest-pull' && matrix.environment.vmm || '' }}
AUTO_GENERATE_POLICY: "yes"
K8S_TEST_HOST_TYPE: "all"
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
persist-credentials: false
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: get-kata-tools-tarball
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: kata-tools-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-tools-artifacts
- name: Install kata-tools
run: bash tests/integration/kubernetes/gha-run.sh install-kata-tools kata-tools-artifacts
- name: Log into the Azure account
uses: azure/login@a457da9ea143d694b1b9c7c869ebb04ebe844ef5 # v2.3.0
with:
client-id: ${{ secrets.AZ_APPID }}
tenant-id: ${{ secrets.AZ_TENANT_ID }}
subscription-id: ${{ secrets.AZ_SUBSCRIPTION_ID }}
- name: Create AKS cluster
uses: nick-fields/retry@ce71cc2ab81d554ebbe88c79ab5975992d79ba08 # v3.0.2
with:
timeout_minutes: 15
max_attempts: 20
retry_on: error
retry_wait_seconds: 10
command: bash tests/integration/kubernetes/gha-run.sh create-cluster
- name: Install `bats`
run: bash tests/integration/kubernetes/gha-run.sh install-bats
- name: Install `kubectl`
uses: azure/setup-kubectl@776406bce94f63e41d621b960d78ee25c8b76ede # v4.0.1
with:
version: 'latest'
- name: Download credentials for the Kubernetes CLI to use them
run: bash tests/integration/kubernetes/gha-run.sh get-cluster-credentials
- name: Deploy Kata
timeout-minutes: 20
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata-aks
env:
USE_EXPERIMENTAL_SETUP_SNAPSHOTTER: ${{ env.SNAPSHOTTER == 'nydus' }}
AUTO_GENERATE_POLICY: ${{ env.PULL_TYPE == 'experimental-force-guest-pull' && 'no' || 'yes' }}
- name: Deploy CoCo KBS
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-coco-kbs
- name: Install `kbs-client`
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh install-kbs-client
- name: Deploy CSI driver
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh deploy-csi-driver
- name: Run tests
timeout-minutes: 80
run: bash tests/integration/kubernetes/gha-run.sh run-tests
- name: Report tests
if: always()
run: bash tests/integration/kubernetes/gha-run.sh report-tests
- name: Refresh OIDC token in case access token expired
if: always()
uses: azure/login@a457da9ea143d694b1b9c7c869ebb04ebe844ef5 # v2.3.0
with:
client-id: ${{ secrets.AZ_APPID }}
tenant-id: ${{ secrets.AZ_TENANT_ID }}
subscription-id: ${{ secrets.AZ_SUBSCRIPTION_ID }}
- name: Delete AKS cluster
if: always()
timeout-minutes: 15
run: bash tests/integration/kubernetes/gha-run.sh delete-cluster
# Generate jobs for testing CoCo on non-TEE environments with erofs-snapshotter
run-k8s-tests-coco-nontee-with-erofs-snapshotter:
name: run-k8s-tests-coco-nontee-with-erofs-snapshotter
strategy:
fail-fast: false
matrix:
vmm:
- qemu-coco-dev
snapshotter:
- erofs
pull-type:
- default
runs-on: ubuntu-24.04
environment: ci
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
GH_PR_NUMBER: ${{ inputs.pr-number }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
# Some tests rely on that variable to run (or not)
KBS: "false"
# Set the KBS ingress handler (empty string disables handling)
KBS_INGRESS: ""
KUBERNETES: "vanilla"
CONTAINER_ENGINE: "containerd"
CONTAINER_ENGINE_VERSION: "v2.2"
PULL_TYPE: ${{ matrix.pull-type }}
SNAPSHOTTER: ${{ matrix.snapshotter }}
USE_EXPERIMENTAL_SETUP_SNAPSHOTTER: "true"
K8S_TEST_HOST_TYPE: "all"
# We are skipping the auto generated policy tests for now,
# but those should be enabled as soon as we work on that.
AUTO_GENERATE_POLICY: "no"
CONTAINER_ENGINE_VERSION: "active"
GH_TOKEN: ${{ github.token }}
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
@@ -342,8 +213,129 @@ jobs:
- name: Deploy kubernetes
timeout-minutes: 15
run: bash tests/integration/kubernetes/gha-run.sh deploy-k8s
- name: Install `bats`
run: bash tests/integration/kubernetes/gha-run.sh install-bats
- name: Deploy Kata
timeout-minutes: 20
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata
env:
GH_TOKEN: ${{ github.token }}
USE_EXPERIMENTAL_SETUP_SNAPSHOTTER: ${{ matrix.environment.snapshotter == 'nydus' }}
- name: Deploy CoCo KBS
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-coco-kbs
- name: Install `kbs-client`
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh install-kbs-client
- name: Deploy CSI driver
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh deploy-csi-driver
- name: Run tests
timeout-minutes: 80
run: bash tests/integration/kubernetes/gha-run.sh run-tests
- name: Report tests
if: always()
run: bash tests/integration/kubernetes/gha-run.sh report-tests
- name: Delete kata-deploy
if: always()
timeout-minutes: 15
run: bash tests/integration/kubernetes/gha-run.sh cleanup
- name: Delete CoCo KBS
if: always()
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh delete-coco-kbs
- name: Delete CSI driver
if: always()
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh delete-csi-driver
# Generate jobs for testing CoCo on non-TEE environments with erofs-snapshotter
run-k8s-tests-coco-nontee-with-erofs-snapshotter:
name: run-k8s-tests-coco-nontee-with-erofs-snapshotter
strategy:
fail-fast: false
matrix:
vmm:
- qemu-coco-dev
snapshotter:
- erofs
pull-type:
- default
runs-on: ubuntu-24.04
environment: ci
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
GH_PR_NUMBER: ${{ inputs.pr-number }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
# Some tests rely on that variable to run (or not)
KBS: "false"
# Set the KBS ingress handler (empty string disables handling)
KBS_INGRESS: ""
KUBERNETES: "vanilla"
CONTAINER_ENGINE: "containerd"
CONTAINER_ENGINE_VERSION: "active"
PULL_TYPE: ${{ matrix.pull-type }}
SNAPSHOTTER: ${{ matrix.snapshotter }}
USE_EXPERIMENTAL_SETUP_SNAPSHOTTER: "true"
K8S_TEST_HOST_TYPE: "all"
# We are skipping the auto generated policy tests for now,
# but those should be enabled as soon as we work on that.
AUTO_GENERATE_POLICY: "no"
GH_TOKEN: ${{ github.token }}
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
persist-credentials: false
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: get-kata-tools-tarball
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: kata-tools-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-tools-artifacts
- name: Install kata-tools
run: bash tests/integration/kubernetes/gha-run.sh install-kata-tools kata-tools-artifacts
- name: Remove unnecessary directories to free up space
run: |
sudo rm -rf /usr/local/.ghcup
sudo rm -rf /opt/hostedtoolcache/CodeQL
sudo rm -rf /usr/local/lib/android
sudo rm -rf /usr/share/dotnet
sudo rm -rf /opt/ghc
sudo rm -rf /usr/local/share/boost
sudo rm -rf /usr/lib/jvm
sudo rm -rf /usr/share/swift
sudo rm -rf /usr/local/share/powershell
sudo rm -rf /usr/local/julia*
sudo rm -rf /opt/az
sudo rm -rf /usr/local/share/chromium
sudo rm -rf /opt/microsoft
sudo rm -rf /opt/google
sudo rm -rf /usr/lib/firefox
- name: Deploy kubernetes
timeout-minutes: 15
run: bash tests/integration/kubernetes/gha-run.sh deploy-k8s
- name: Install `bats`
run: bash tests/integration/kubernetes/gha-run.sh install-bats
@@ -363,3 +355,13 @@ jobs:
- name: Report tests
if: always()
run: bash tests/integration/kubernetes/gha-run.sh report-tests
- name: Delete kata-deploy
if: always()
timeout-minutes: 15
run: bash tests/integration/kubernetes/gha-run.sh cleanup
- name: Delete CSI driver
if: always()
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh delete-csi-driver

View File

@@ -55,6 +55,6 @@ jobs:
# Upload the results to GitHub's code scanning dashboard (optional).
# Commenting out will disable upload of results to your repo's Code Scanning dashboard
- name: "Upload to code-scanning"
uses: github/codeql-action/upload-sarif@v3
uses: github/codeql-action/upload-sarif@4bdb89f48054571735e3792627da6195c57459e2 # v3.31.10
with:
sarif_file: results.sarif

View File

@@ -126,11 +126,16 @@ jobs:
./ci/install_yq.sh
env:
INSTALL_IN_GOPATH: false
- name: Install golang
- name: Read properties from versions.yaml
run: |
cd "${GOPATH}/src/github.com/${GITHUB_REPOSITORY}"
./tests/install_go.sh -f -p
echo "/usr/local/go/bin" >> "$GITHUB_PATH"
go_version="$(yq '.languages.golang.version' versions.yaml)"
[ -n "$go_version" ]
echo "GO_VERSION=${go_version}" >> "$GITHUB_ENV"
- name: Setup Golang version ${{ env.GO_VERSION }}
uses: actions/setup-go@7a3fe6cf4cb3a834922a1244abfce67bcef6a0c5 # v6.2.0
with:
go-version: ${{ env.GO_VERSION }}
- name: Install system dependencies
run: |
sudo apt-get update && sudo apt-get -y install moreutils hunspell hunspell-en-gb hunspell-en-us pandoc

222
Cargo.lock generated
View File

@@ -44,9 +44,7 @@ version = "0.1.0"
dependencies = [
"anyhow",
"async-trait",
"futures 0.1.31",
"kata-types",
"log",
"logging",
"nix 0.26.4",
"oci-spec 0.8.3",
@@ -141,23 +139,12 @@ version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "435a87a52755b8f27fcf321ac4f04b2802e337c8c4872923137471ec39c37532"
dependencies = [
"event-listener 5.4.1",
"event-listener",
"event-listener-strategy",
"futures-core",
"pin-project-lite",
]
[[package]]
name = "async-channel"
version = "1.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "81953c529336010edd6d8e358f886d9581267795c61b19475b71314bffa46d35"
dependencies = [
"concurrent-queue",
"event-listener 2.5.3",
"futures-core",
]
[[package]]
name = "async-channel"
version = "2.5.0"
@@ -184,21 +171,6 @@ dependencies = [
"slab",
]
[[package]]
name = "async-global-executor"
version = "2.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05b1b633a2115cd122d73b955eadd9916c18c8f510ec9cd1686404c60ad1c29c"
dependencies = [
"async-channel 2.5.0",
"async-executor",
"async-io",
"async-lock",
"blocking",
"futures-lite",
"once_cell",
]
[[package]]
name = "async-io"
version = "2.6.0"
@@ -223,7 +195,7 @@ version = "3.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5fd03604047cee9b6ce9de9f70c6cd540a0520c813cbd49bae61f33ab80ed1dc"
dependencies = [
"event-listener 5.4.1",
"event-listener",
"event-listener-strategy",
"pin-project-lite",
]
@@ -234,14 +206,14 @@ version = "2.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fc50921ec0055cdd8a16de48773bfeec5c972598674347252c0399676be7da75"
dependencies = [
"async-channel 2.5.0",
"async-channel",
"async-io",
"async-lock",
"async-signal",
"async-task",
"blocking",
"cfg-if 1.0.0",
"event-listener 5.4.1",
"event-listener",
"futures-lite",
"rustix 1.1.2",
]
@@ -275,32 +247,6 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "async-std"
version = "1.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2c8e079a4ab67ae52b7403632e4618815d6db36d2a010cfe41b02c1b1578f93b"
dependencies = [
"async-channel 1.9.0",
"async-global-executor",
"async-io",
"async-lock",
"crossbeam-utils",
"futures-channel",
"futures-core",
"futures-io",
"futures-lite",
"gloo-timers",
"kv-log-macro",
"log",
"memchr",
"once_cell",
"pin-project-lite",
"pin-utils",
"slab",
"wasm-bindgen-futures",
]
[[package]]
name = "async-task"
version = "4.7.1"
@@ -447,7 +393,7 @@ version = "1.6.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e83f8d02be6967315521be875afa792a316e28d57b5a2d401897e2a7921b7f21"
dependencies = [
"async-channel 2.5.0",
"async-channel",
"async-task",
"futures-io",
"futures-lite",
@@ -644,29 +590,17 @@ dependencies = [
"containerd-shim-protos",
"kata-sys-util",
"kata-types",
"lazy_static",
"nix 0.26.4",
"oci-spec 0.8.3",
"persist",
"protobuf",
"protocols",
"resource",
"runtime-spec",
"serde_json",
"slog",
"slog-scope",
"strum 0.24.1",
"thiserror 1.0.48",
"tokio",
"ttrpc",
]
[[package]]
name = "common-path"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2382f75942f4b3be3690fe4f86365e9c853c1587d6ee58212cebf6e2a9ccd101"
[[package]]
name = "concurrent-queue"
version = "2.5.0"
@@ -711,7 +645,7 @@ dependencies = [
"async-trait",
"cgroups-rs 0.3.4",
"containerd-shim-protos",
"futures 0.3.28",
"futures",
"go-flag",
"lazy_static",
"libc",
@@ -1044,7 +978,6 @@ dependencies = [
"dbs-interrupt",
"dbs-utils",
"dbs-virtio-devices",
"downcast-rs",
"kvm-bindings",
"kvm-ioctls",
"libc",
@@ -1057,7 +990,6 @@ dependencies = [
"vfio-ioctls",
"virtio-queue",
"vm-memory",
"vmm-sys-util 0.11.1",
]
[[package]]
@@ -1074,7 +1006,6 @@ dependencies = [
name = "dbs-upcall"
version = "0.3.0"
dependencies = [
"anyhow",
"dbs-utils",
"dbs-virtio-devices",
"log",
@@ -1269,12 +1200,6 @@ version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1435fa1053d8b2fbbe9be7e97eca7f33d37b28409959813daefc1446a14247f1"
[[package]]
name = "downcast-rs"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ea835d29036a4087793836fa931b08837ad5e957da9e23886b29586fb9b6650"
[[package]]
name = "dragonball"
version = "0.1.0"
@@ -1295,7 +1220,6 @@ dependencies = [
"dbs-utils",
"dbs-virtio-devices",
"derivative",
"fuse-backend-rs",
"kvm-bindings",
"kvm-ioctls",
"lazy_static",
@@ -1350,6 +1274,18 @@ version = "1.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "66b7e2430c6dff6a955451e2cfc438f09cea1965a9d6f87f7e3b90decc014099"
[[package]]
name = "enum-as-inner"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a1e6a265c649f3f5979b601d26f1d05ada116434c87741c9493cb56218f76cbc"
dependencies = [
"heck 0.5.0",
"proc-macro2",
"quote",
"syn 2.0.104",
]
[[package]]
name = "enumflags2"
version = "0.7.12"
@@ -1403,12 +1339,6 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "event-listener"
version = "2.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0206175f82b8d6bf6652ff7d71a1e27fd2e4efde587fd368662814d6ec1d9ce0"
[[package]]
name = "event-listener"
version = "5.4.1"
@@ -1426,7 +1356,7 @@ version = "0.5.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8be9f3dfaaffdae2972880079a491a1a8bb7cbed0b8dd7a347f668b4150a3b93"
dependencies = [
"event-listener 5.4.1",
"event-listener",
"pin-project-lite",
]
@@ -1554,12 +1484,6 @@ dependencies = [
"vmm-sys-util 0.11.1",
]
[[package]]
name = "futures"
version = "0.1.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a471a38ef8ed83cd6e40aa59c1ffe17db6855c18e3604d9c4ed8c08ebc28678"
[[package]]
name = "futures"
version = "0.3.28"
@@ -1719,18 +1643,6 @@ version = "0.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0cc23270f6e1808e30a928bdc84dea0b9b4136a8bc82338574f23baf47bbd280"
[[package]]
name = "gloo-timers"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bbb143cf96099802033e0d4f4963b19fd2e0b728bcf076cd9cf7f6634f092994"
dependencies = [
"futures-channel",
"futures-core",
"js-sys",
"wasm-bindgen",
]
[[package]]
name = "go-flag"
version = "0.1.0"
@@ -1966,7 +1878,7 @@ dependencies = [
"crossbeam-channel",
"dbs-utils",
"dragonball",
"futures 0.3.28",
"futures",
"go-flag",
"hyper",
"hyperlocal",
@@ -1977,10 +1889,8 @@ dependencies = [
"libc",
"logging",
"nix 0.26.4",
"oci-spec 0.8.3",
"path-clean",
"persist",
"protobuf",
"protocols",
"qapi",
"qapi-qmp",
@@ -1992,7 +1902,6 @@ dependencies = [
"serde",
"serde_json",
"serial_test 2.0.0",
"shim-interface",
"slog",
"slog-scope",
"tempfile",
@@ -2269,8 +2178,6 @@ version = "0.1.0"
dependencies = [
"anyhow",
"byteorder",
"chrono",
"common-path",
"fail",
"hex",
"kata-types",
@@ -2279,11 +2186,9 @@ dependencies = [
"mockall",
"nix 0.26.4",
"oci-spec 0.8.3",
"once_cell",
"pci-ids",
"rand 0.8.5",
"runtime-spec",
"safe-path 0.1.0",
"serde",
"serde_json",
"slog",
@@ -2302,8 +2207,8 @@ dependencies = [
"byte-unit",
"flate2",
"glob",
"hex",
"lazy_static",
"nix 0.26.4",
"num_cpus",
"oci-spec 0.8.3",
"regex",
@@ -2314,18 +2219,10 @@ dependencies = [
"sha2 0.10.9",
"slog",
"slog-scope",
"sysctl",
"sysinfo",
"thiserror 1.0.48",
"toml 0.5.11",
]
[[package]]
name = "kv-log-macro"
version = "1.0.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0de8b303297635ad57c9f5059fd9cee7a47f8e8daa09df0fcd07dd39fb22977f"
dependencies = [
"log",
"toml",
]
[[package]]
@@ -2646,7 +2543,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b65d130ee111430e47eed7896ea43ca693c387f097dd97376bffafbf25812128"
dependencies = [
"bytes",
"futures 0.3.28",
"futures",
"log",
"netlink-packet-core",
"netlink-sys",
@@ -2660,7 +2557,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "416060d346fbaf1f23f9512963e3e878f1a78e707cb699ba9215761754244307"
dependencies = [
"bytes",
"futures 0.3.28",
"futures",
"libc",
"log",
"tokio",
@@ -2817,7 +2714,7 @@ dependencies = [
"log",
"serde",
"serde_json",
"toml 0.5.11",
"toml",
]
[[package]]
@@ -3044,7 +2941,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e785d273968748578931e4dc3b4f5ec86b26e09d9e0d66b55adda7fce742f7a"
dependencies = [
"async-trait",
"futures 0.3.28",
"futures",
"futures-executor",
"headers",
"http",
@@ -3212,11 +3109,9 @@ dependencies = [
"async-trait",
"kata-sys-util",
"kata-types",
"libc",
"safe-path 0.1.0",
"serde",
"serde_json",
"shim-interface",
]
[[package]]
@@ -3626,7 +3521,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7b047adab56acc4948d4b9b58693c1f33fd13efef2d6bb5f0f66a47436ceada8"
dependencies = [
"bytes",
"futures 0.3.28",
"futures",
"log",
"memchr",
"qapi-qmp",
@@ -3908,11 +3803,10 @@ dependencies = [
"agent",
"anyhow",
"async-trait",
"bitflags 2.10.0",
"byte-unit",
"cgroups-rs 0.5.0",
"flate2",
"futures 0.3.28",
"futures",
"hex",
"hypervisor",
"inotify",
@@ -3922,7 +3816,6 @@ dependencies = [
"libc",
"logging",
"netlink-packet-route",
"netlink-sys",
"netns-rs",
"nix 0.26.4",
"oci-spec 0.8.3",
@@ -3945,9 +3838,9 @@ dependencies = [
[[package]]
name = "rkyv"
version = "0.7.45"
version = "0.7.46"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9008cd6385b9e161d8229e1f6549dd23c3d022f132a2ea37ac3a10ac4935779b"
checksum = "2297bf9c81a3f0dc96bc9521370b88f054168c29826a75e89c55ff196e7ed6a1"
dependencies = [
"bitvec",
"bytecheck",
@@ -3963,9 +3856,9 @@ dependencies = [
[[package]]
name = "rkyv_derive"
version = "0.7.45"
version = "0.7.46"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "503d1d27590a2b0a3a4ca4c94755aa2875657196ecbf401a42eff41d7de532c0"
checksum = "84d7b42d4b8d06048d3ac8db0eb31bcb942cbeb709f0b5f2b2ebde398d3038f5"
dependencies = [
"proc-macro2",
"quote",
@@ -4007,7 +3900,6 @@ dependencies = [
"common",
"containerd-shim-protos",
"go-flag",
"logging",
"nix 0.26.4",
"runtimes",
"shim",
@@ -4018,7 +3910,6 @@ dependencies = [
name = "runtime-spec"
version = "0.1.0"
dependencies = [
"libc",
"serde",
"serde_derive",
"serde_json",
@@ -4031,8 +3922,8 @@ dependencies = [
"agent",
"anyhow",
"common",
"containerd-shim-protos",
"hyper",
"hyperlocal",
"hypervisor",
"kata-sys-util",
"kata-types",
@@ -4351,7 +4242,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1c789ec87f4687d022a2405cf46e0cd6284889f1839de292cadeb6c6019506f2"
dependencies = [
"dashmap",
"futures 0.3.28",
"futures",
"lazy_static",
"log",
"parking_lot",
@@ -4365,7 +4256,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0e56dd856803e253c8f298af3f4d7eb0ae5e23a737252cd90bb4f3b435033b2d"
dependencies = [
"dashmap",
"futures 0.3.28",
"futures",
"lazy_static",
"log",
"parking_lot",
@@ -4405,12 +4296,10 @@ dependencies = [
"containerd-shim-protos",
"kata-types",
"logging",
"persist",
"runtimes",
"slog",
"slog-scope",
"tokio",
"tracing",
"ttrpc",
]
@@ -4474,9 +4363,7 @@ dependencies = [
"nix 0.26.4",
"oci-spec 0.8.3",
"protobuf",
"rand 0.8.5",
"runtime-spec",
"runtimes",
"serial_test 0.10.0",
"service",
"sha2 0.10.9",
@@ -4485,11 +4372,8 @@ dependencies = [
"slog-scope",
"slog-stdlog",
"tempfile",
"tests_utils",
"thiserror 1.0.48",
"tokio",
"tracing",
"tracing-opentelemetry",
"unix_socket2",
]
@@ -4499,7 +4383,6 @@ version = "0.1.0"
dependencies = [
"anyhow",
"common",
"logging",
"runtimes",
"tokio",
]
@@ -4793,6 +4676,20 @@ dependencies = [
"syn 2.0.104",
]
[[package]]
name = "sysctl"
version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cca424247104946a59dacd27eaad296223b7feec3d168a6dd04585183091eb0b"
dependencies = [
"bitflags 2.10.0",
"byteorder",
"enum-as-inner",
"libc",
"thiserror 2.0.12",
"walkdir",
]
[[package]]
name = "sysinfo"
version = "0.34.2"
@@ -5083,21 +4980,12 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52a15c15b1bc91f90902347eff163b5b682643aff0c8e972912cca79bd9208dd"
dependencies = [
"bytes",
"futures 0.3.28",
"futures",
"libc",
"tokio",
"vsock",
]
[[package]]
name = "toml"
version = "0.4.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "758664fc71a3a69038656bee8b6be6477d2a6c315a6b81f7081f591bffa4111f"
dependencies = [
"serde",
]
[[package]]
name = "toml"
version = "0.5.11"
@@ -5240,7 +5128,7 @@ dependencies = [
"async-trait",
"byteorder",
"crossbeam",
"futures 0.3.28",
"futures",
"home",
"libc",
"log",
@@ -5442,16 +5330,13 @@ version = "0.1.0"
dependencies = [
"agent",
"anyhow",
"async-std",
"async-trait",
"awaitgroup",
"common",
"containerd-shim-protos",
"futures 0.3.28",
"hypervisor",
"kata-sys-util",
"kata-types",
"lazy_static",
"libc",
"logging",
"nix 0.26.4",
@@ -5467,7 +5352,6 @@ dependencies = [
"slog-scope",
"strum 0.24.1",
"tokio",
"toml 0.4.10",
"tracing",
"url",
"uuid 1.18.1",
@@ -6100,7 +5984,7 @@ dependencies = [
"async-trait",
"blocking",
"enumflags2",
"event-listener 5.4.1",
"event-listener",
"futures-core",
"futures-lite",
"hex",

View File

@@ -1 +1 @@
3.26.0
3.27.0

View File

@@ -187,9 +187,10 @@ different compared to `runc` containers:
into the guest and exposes it directly to the container.
**Mounting guest devices**: When the source path of a hostPath volume is
under `/dev`, and the path either corresponds to a host device or is not
accessible by the Kata shim, the Kata agent bind mounts the source path
directly from the *guest* filesystem into the container.
under `/dev` (or `/dev` itself), and the path corresponds to a
non-regular file (i.e., a device, directory, or any other special file)
or is not accessible by the Kata shim, the Kata agent bind mounts the
source path directly from the *guest* filesystem into the container.
[runtime-config]: /src/runtime/README.md#configuration
[k8s-hostpath]: https://kubernetes.io/docs/concepts/storage/volumes/#hostpath
@@ -226,6 +227,35 @@ Importantly, the default behavior to pass the host devices to a
privileged container is not supported in Kata Containers and needs to be
disabled, see [Privileged Kata Containers](how-to/privileged.md).
## Guest pulled container images
When using features like **nydus guest-pull**, set user/group IDs explicitly in the pod spec.
If the ID values are omitted:
- Your workload might be executed with unexpected user/group ID values, because image layers
may be unavailable to containerd, so image config (including user/group) is not applied.
- If using policy or genpolicy, the generated policy may detect these unexpected values and
reject the creation of workload containers.
Set `securityContext` explicitly. Use **pod-level** `spec.securityContext` (for Pods) or
`spec.template.spec.securityContext` (for controllers like Deployments) and/or **container-level**
`spec.containers[].securityContext`. Include at least:
- `runAsUser` — primary user ID
- `runAsGroup` — primary group ID
- `fsGroup` — volume group ownership (often reflected as a supplemental group)
- `supplementalGroups` — list of additional group IDs (if needed)
Example:
```yaml
# Explicit user/group/supplementary groups to support nydus guest-pull
securityContext:
runAsUser: 0
runAsGroup: 0
fsGroup: 0
supplementalGroups: [1, 2, 3, 4, 6, 10, 11, 20, 26, 27]
```
# Appendices
## The constraints challenge

View File

@@ -49,6 +49,8 @@ In order to allow Kubelet to use containerd (using the CRI interface), configure
EOF
```
For Kata Containers (and especially CoCo / Confidential Containers tests), use at least `--runtime-request-timeout=600s` (10m) so CRI CreateContainerRequest does not time out.
- Inform systemd about the new configuration
```bash

View File

@@ -99,6 +99,9 @@ The [`genpolicy`](../../src/tools/genpolicy/) application can be used to generat
**Warning** Users should review carefully the automatically-generated Policy, and modify the Policy file if needed to match better their use case, before using this Policy.
**Important — User / Group / Supplemental groups for Policy and genpolicy**
When using features like **nydus guest-pull**, set user/group IDs explicitly in the pod spec, as described in [Limitations](../Limitations.md#guest-pulled-container-images).
See the [`genpolicy` documentation](../../src/tools/genpolicy/README.md) and the [Policy contents examples](#policy-contents) for additional information.
## Policy contents

49
src/agent/Cargo.lock generated
View File

@@ -743,12 +743,6 @@ version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5b63caa9aa9397e2d9480a9b13673856c78d8ac123288526c37d7839f2a86990"
[[package]]
name = "common-path"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2382f75942f4b3be3690fe4f86365e9c853c1587d6ee58212cebf6e2a9ccd101"
[[package]]
name = "concurrent-queue"
version = "2.5.0"
@@ -1098,6 +1092,18 @@ dependencies = [
"serde",
]
[[package]]
name = "enum-as-inner"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a1e6a265c649f3f5979b601d26f1d05ada116434c87741c9493cb56218f76cbc"
dependencies = [
"heck 0.5.0",
"proc-macro2",
"quote",
"syn 2.0.101",
]
[[package]]
name = "enumflags2"
version = "0.7.11"
@@ -2102,8 +2108,6 @@ version = "0.1.0"
dependencies = [
"anyhow",
"byteorder",
"chrono",
"common-path",
"fail",
"hex",
"kata-types",
@@ -2112,11 +2116,9 @@ dependencies = [
"mockall",
"nix 0.26.4",
"oci-spec",
"once_cell",
"pci-ids",
"rand",
"runtime-spec",
"safe-path",
"serde",
"serde_json",
"slog",
@@ -2135,8 +2137,8 @@ dependencies = [
"byte-unit",
"flate2",
"glob",
"hex",
"lazy_static",
"nix 0.26.4",
"num_cpus",
"oci-spec",
"regex",
@@ -2147,6 +2149,7 @@ dependencies = [
"sha2 0.10.9",
"slog",
"slog-scope",
"sysctl",
"sysinfo",
"thiserror 1.0.69",
"toml",
@@ -2306,7 +2309,6 @@ name = "mem-agent"
version = "0.2.0"
dependencies = [
"anyhow",
"async-trait",
"chrono",
"maplit",
"nix 0.30.1",
@@ -3488,9 +3490,9 @@ dependencies = [
[[package]]
name = "rkyv"
version = "0.7.45"
version = "0.7.46"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9008cd6385b9e161d8229e1f6549dd23c3d022f132a2ea37ac3a10ac4935779b"
checksum = "2297bf9c81a3f0dc96bc9521370b88f054168c29826a75e89c55ff196e7ed6a1"
dependencies = [
"bitvec",
"bytecheck",
@@ -3506,9 +3508,9 @@ dependencies = [
[[package]]
name = "rkyv_derive"
version = "0.7.45"
version = "0.7.46"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "503d1d27590a2b0a3a4ca4c94755aa2875657196ecbf401a42eff41d7de532c0"
checksum = "84d7b42d4b8d06048d3ac8db0eb31bcb942cbeb709f0b5f2b2ebde398d3038f5"
dependencies = [
"proc-macro2",
"quote",
@@ -3575,7 +3577,6 @@ dependencies = [
name = "runtime-spec"
version = "0.1.0"
dependencies = [
"libc",
"serde",
"serde_derive",
"serde_json",
@@ -4215,6 +4216,20 @@ dependencies = [
"syn 2.0.101",
]
[[package]]
name = "sysctl"
version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cca424247104946a59dacd27eaad296223b7feec3d168a6dd04585183091eb0b"
dependencies = [
"bitflags 2.9.0",
"byteorder",
"enum-as-inner",
"libc",
"thiserror 2.0.12",
"walkdir",
]
[[package]]
name = "sysinfo"
version = "0.34.2"

View File

@@ -857,7 +857,7 @@ fn mount_from(
dest.as_str(),
Some(mount_typ.as_str()),
flags,
Some(d.as_str()),
Some(d.as_str()).filter(|s| !s.is_empty()),
)
.inspect_err(|e| log_child!(cfd_log, "mount error: {:?}", e))?;

View File

@@ -48,7 +48,6 @@ vmm-sys-util = { workspace = true }
virtio-queue = { workspace = true, optional = true }
vm-memory = { workspace = true, features = ["backend-mmap"] }
crossbeam-channel = "0.5.6"
fuse-backend-rs = "0.10.5"
vfio-bindings = { workspace = true, optional = true }
vfio-ioctls = { workspace = true, optional = true }
@@ -86,3 +85,6 @@ host-device = ["dep:vfio-bindings", "dep:vfio-ioctls", "dep:dbs-pci"]
unexpected_cfgs = { level = "warn", check-cfg = [
'cfg(feature, values("test-mock"))',
] }
[package.metadata.cargo-machete]
ignored = ["vfio-bindings"]

View File

@@ -23,24 +23,22 @@ dbs-interrupt = { workspace = true, features = [
"kvm-legacy-irq",
"kvm-msi-irq",
] }
downcast-rs = "1.2.0"
byteorder = "1.4.3"
serde = "1.0.27"
vm-memory = {workspace = true}
kvm-ioctls = {workspace = true}
kvm-bindings = {workspace = true}
vfio-ioctls = {workspace = true}
vfio-bindings = {workspace = true}
vm-memory = { workspace = true }
kvm-ioctls = { workspace = true }
kvm-bindings = { workspace = true }
vfio-ioctls = { workspace = true }
vfio-bindings = { workspace = true }
libc = "0.2.39"
vmm-sys-util = {workspace = true}
virtio-queue = {workspace = true}
dbs-utils = {workspace = true}
virtio-queue = { workspace = true }
dbs-utils = { workspace = true }
[dev-dependencies]
dbs-arch = { workspace = true }
kvm-ioctls = {workspace = true}
kvm-ioctls = { workspace = true }
test-utils = { workspace = true }
nix = { workspace = true }

View File

@@ -11,7 +11,6 @@ keywords = ["dragonball", "secure-sandbox", "devices", "upcall", "virtio"]
readme = "README.md"
[dependencies]
anyhow = "1"
log = "0.4.14"
thiserror = "1"
timerfd = "1.2.0"

View File

@@ -24,8 +24,8 @@ dbs-boot = { workspace = true }
epoll = ">=4.3.1, <4.3.2"
io-uring = "0.5.2"
fuse-backend-rs = { version = "0.10.5", optional = true }
kvm-bindings = { workspace = true}
kvm-ioctls = {workspace = true}
kvm-bindings = { workspace = true }
kvm-ioctls = { workspace = true }
libc = "0.2.119"
log = "0.4.14"
nix = "0.24.3"
@@ -37,19 +37,16 @@ serde = "1.0.27"
serde_json = "1.0.9"
thiserror = "1"
threadpool = "1"
virtio-bindings = {workspace = true}
virtio-queue = {workspace = true}
vmm-sys-util = {workspace = true}
virtio-bindings = { workspace = true }
virtio-queue = { workspace = true }
vmm-sys-util = { workspace = true }
vm-memory = { workspace = true, features = ["backend-mmap"] }
sendfd = "0.4.3"
vhost-rs = { version = "0.6.1", package = "vhost", optional = true }
timerfd = "1.0"
[dev-dependencies]
vm-memory = { workspace = true, features = [
"backend-mmap",
"backend-atomic",
] }
vm-memory = { workspace = true, features = ["backend-mmap", "backend-atomic"] }
test-utils = { workspace = true }
[features]

View File

@@ -439,19 +439,19 @@ pub mod tests {
VirtqDesc { desc }
}
pub fn addr(&self) -> VolatileRef<u64> {
pub fn addr(&self) -> VolatileRef<'_, u64> {
self.desc.get_ref(offset_of!(DescriptorTmp, addr)).unwrap()
}
pub fn len(&self) -> VolatileRef<u32> {
pub fn len(&self) -> VolatileRef<'_, u32> {
self.desc.get_ref(offset_of!(DescriptorTmp, len)).unwrap()
}
pub fn flags(&self) -> VolatileRef<u16> {
pub fn flags(&self) -> VolatileRef<'_, u16> {
self.desc.get_ref(offset_of!(DescriptorTmp, flags)).unwrap()
}
pub fn next(&self) -> VolatileRef<u16> {
pub fn next(&self) -> VolatileRef<'_, u16> {
self.desc.get_ref(offset_of!(DescriptorTmp, next)).unwrap()
}
@@ -513,11 +513,11 @@ pub mod tests {
self.start.unchecked_add(self.ring.len() as GuestUsize)
}
pub fn flags(&self) -> VolatileRef<u16> {
pub fn flags(&self) -> VolatileRef<'_, u16> {
self.ring.get_ref(0).unwrap()
}
pub fn idx(&self) -> VolatileRef<u16> {
pub fn idx(&self) -> VolatileRef<'_, u16> {
self.ring.get_ref(2).unwrap()
}
@@ -525,12 +525,12 @@ pub mod tests {
4 + mem::size_of::<T>() * (i as usize)
}
pub fn ring(&self, i: u16) -> VolatileRef<T> {
pub fn ring(&self, i: u16) -> VolatileRef<'_, T> {
assert!(i < self.qsize);
self.ring.get_ref(Self::ring_offset(i)).unwrap()
}
pub fn event(&self) -> VolatileRef<u16> {
pub fn event(&self) -> VolatileRef<'_, u16> {
self.ring.get_ref(Self::ring_offset(self.qsize)).unwrap()
}
@@ -602,7 +602,7 @@ pub mod tests {
(self.dtable.len() / VirtqDesc::dtable_len(1)) as u16
}
pub fn dtable(&self, i: u16) -> VirtqDesc {
pub fn dtable(&self, i: u16) -> VirtqDesc<'_> {
VirtqDesc::new(&self.dtable, i)
}

View File

@@ -865,11 +865,11 @@ mod tests {
0
);
let config: [u8; 8] = [0; 8];
VirtioDevice::<Arc<GuestMemoryMmap<()>>, QueueSync, GuestRegionMmap>::write_config(
let _ = VirtioDevice::<Arc<GuestMemoryMmap<()>>, QueueSync, GuestRegionMmap>::write_config(
&mut dev, 0, &config,
);
let mut data: [u8; 8] = [1; 8];
VirtioDevice::<Arc<GuestMemoryMmap<()>>, QueueSync, GuestRegionMmap>::read_config(
let _ = VirtioDevice::<Arc<GuestMemoryMmap<()>>, QueueSync, GuestRegionMmap>::read_config(
&mut dev, 0, &mut data,
);
assert_eq!(config, data);

View File

@@ -339,7 +339,7 @@ mod tests {
}
}
pub fn create_event_handler_context(&self) -> EventHandlerContext {
pub fn create_event_handler_context(&self) -> EventHandlerContext<'_> {
const QSIZE: u16 = 256;
let guest_rxvq = GuestQ::new(GuestAddress(0x0010_0000), &self.mem, QSIZE);

View File

@@ -13,13 +13,10 @@ edition = "2018"
[dependencies]
anyhow = "1.0.31"
byteorder = "1.4.3"
chrono = "0.4.0"
common-path = "=1.0.0"
fail = "0.5.0"
lazy_static = "1.4.0"
libc = "0.2.100"
nix = "0.26.4"
once_cell = "1.9.0"
serde = { version = "1.0.138", features = ["derive"] }
serde_json = "1.0.73"
slog = "2.5.2"
@@ -34,10 +31,7 @@ mockall = "0.13.1"
kata-types = { path = "../kata-types" }
oci-spec = { version = "0.8.1", features = ["runtime"] }
runtime-spec = { path = "../runtime-spec" }
safe-path = { path = "../safe-path" }
[dev-dependencies]
num_cpus = "1.13.1"
serial_test = "0.5.1"
tempfile = "3.19.1"
test-utils = { path = "../test-utils" }

View File

@@ -29,12 +29,14 @@ serde-enum-str = "0.4"
sysinfo = "0.34.2"
sha2 = "0.10.8"
flate2 = "1.1"
hex = "0.4"
nix = "0.26.4"
oci-spec = { version = "0.8.1", features = ["runtime"] }
safe-path = { path = "../safe-path", optional = true }
[target.'cfg(target_os = "macos")'.dependencies]
sysctl = "0.7.1"
[dev-dependencies]
tempfile = "3.19.1"
test-utils = { path = "../test-utils" }

View File

@@ -26,7 +26,6 @@
use super::{default, ConfigOps, ConfigPlugin, TomlConfig};
use crate::annotations::KATA_ANNO_CFG_HYPERVISOR_PREFIX;
use crate::{resolve_path, sl, validate_path};
use byte_unit::{Byte, Unit};
use lazy_static::lazy_static;
use regex::RegexSet;
use serde_enum_str::{Deserialize_enum_str, Serialize_enum_str};
@@ -34,7 +33,6 @@ use std::collections::HashMap;
use std::io::{self, Result};
use std::path::Path;
use std::sync::{Arc, Mutex};
use sysinfo::{MemoryRefreshKind, RefreshKind, System};
mod dragonball;
pub use self::dragonball::{DragonballConfig, HYPERVISOR_NAME_DRAGONBALL};
@@ -1007,6 +1005,57 @@ fn default_guest_swap_create_threshold_secs() -> u64 {
60
}
/// Get host memory size in MiB.
/// Retrieves the total physical memory of the host across different platforms.
fn host_memory_mib() -> io::Result<u64> {
// Select a platform-specific implementation via a function pointer.
let get_memory: fn() -> io::Result<u64> = {
#[cfg(target_os = "linux")]
{
|| {
let info = nix::sys::sysinfo::sysinfo().map_err(io::Error::other)?;
Ok(info.ram_total() / (1024 * 1024)) // MiB
}
}
#[cfg(target_os = "macos")]
{
|| {
use sysctl::{Ctl, CtlValue, Sysctl};
let v = Ctl::new("hw.memsize")
.map_err(io::Error::other)?
.value()
.map_err(io::Error::other)?;
let bytes = match v {
CtlValue::S64(x) if x >= 0 => x as u64,
other => {
return Err(io::Error::new(
io::ErrorKind::InvalidData,
format!("unexpected sysctl hw.memsize value type: {:?}", other),
));
}
};
Ok(bytes / (1024 * 1024)) // MiB
}
}
#[cfg(not(any(target_os = "linux", target_os = "macos")))]
{
|| {
Err(io::Error::new(
io::ErrorKind::Unsupported,
"host memory query not implemented on this platform",
))
}
}
};
get_memory()
}
impl MemoryInfo {
/// Adjusts the configuration information after loading from a configuration file.
///
@@ -1018,13 +1067,15 @@ impl MemoryInfo {
self.file_mem_backend,
"Memory backend file {} is invalid: {}"
)?;
if self.default_maxmemory == 0 {
let s = System::new_with_specifics(
RefreshKind::nothing().with_memory(MemoryRefreshKind::everything()),
);
self.default_maxmemory = Byte::from_u64(s.total_memory())
.get_adjusted_unit(Unit::MiB)
.get_value() as u32;
let host_memory = host_memory_mib()?;
if u64::from(self.default_memory) > host_memory {
self.default_memory = host_memory as u32;
}
if self.default_maxmemory == 0 || u64::from(self.default_maxmemory) > host_memory {
self.default_maxmemory = host_memory as u32;
}
Ok(())
}
@@ -1167,6 +1218,29 @@ pub struct SecurityInfo {
#[serde(default)]
pub sev_snp_guest: bool,
/// SNP 'ID Block' and 'ID Authentication Information Structure'.
/// If one of snp_id_block or snp_id_auth is specified, the other must be specified, too.
/// Notice that the default SNP policy of QEMU (0x30000) is used by Kata, if not explicitly
/// set via 'snp_guest_policy' option. The IDBlock contains the guest policy as field, and
/// it must match the value from 'snp_guest_policy' or, if unset, the QEMU default policy.
/// 96-byte, base64-encoded blob to provide the 'ID Block' structure for the
/// SNP_LAUNCH_FINISH command defined in the SEV-SNP firmware ABI (QEMU default: all-zero)
#[serde(default)]
pub snp_id_block: String,
/// 4096-byte, base64-encoded blob to provide the 'ID Authentication Information Structure'
/// for the SNP_LAUNCH_FINISH command defined in the SEV-SNP firmware ABI (QEMU default: all-zero)
#[serde(default)]
pub snp_id_auth: String,
/// SNP Guest Policy, the 'POLICY' parameter to the SNP_LAUNCH_START command.
/// If unset, the QEMU default policy (0x30000) will be used.
/// Notice that the guest policy is enforced at VM launch, and your pod VMs
/// won't start at all if the policy denys it. This will be indicated by a
/// 'SNP_LAUNCH_START' error.
#[serde(default = "default_snp_guest_policy")]
pub snp_guest_policy: u32,
/// Path to OCI hook binaries in the *guest rootfs*.
///
/// This setting does not affect host-side hooks, which must instead be
@@ -1228,6 +1302,10 @@ fn default_qgs_port() -> u32 {
4050
}
fn default_snp_guest_policy() -> u32 {
0x30000
}
impl SecurityInfo {
/// Adjusts the security configuration information after loading from a configuration file.
///

View File

@@ -10,7 +10,6 @@ anyhow = "1.0"
page_size = "0.6"
chrono = "0.4"
tokio = { version = "1.45.1", features = ["full"] }
async-trait = "0.1"
maplit = "1.0"
nix = { version = "0.30.1", features = ["fs", "sched"] }

View File

@@ -9,4 +9,3 @@ license = "Apache-2.0"
serde = "1.0.131"
serde_derive = "1.0.131"
serde_json = "1.0.73"
libc = "0.2.112"

View File

@@ -28,5 +28,4 @@ nix = { workspace = true }
tokio = { workspace = true, features = ["rt", "rt-multi-thread"] }
shim = { path = "crates/shim" }
common = { workspace = true }
logging = { workspace = true }
runtimes = { workspace = true }

View File

@@ -298,7 +298,7 @@ ifneq (,$(CLHCMD))
KERNELTYPE_CLH = uncompressed
KERNEL_NAME_CLH = $(call MAKE_KERNEL_NAME,$(KERNELTYPE_CLH))
KERNELPATH_CLH = $(KERNELDIR)/$(KERNEL_NAME_CLH)
VMROOTFSDRIVER_CLH := virtio-pmem
VMROOTFSDRIVER_CLH := virtio-blk-pci
DEFSANDBOXCGROUPONLY_CLH := true
DEFSTATICRESOURCEMGMT_CLH := false

View File

@@ -22,6 +22,8 @@ rootfs_type = @DEFROOTFSTYPE@
# Block storage driver to be used for the VM rootfs is backed
# by a block device.
#
# virtio-pmem is not supported with Cloud Hypervisor.
vm_rootfs_driver = "@VMROOTFSDRIVER_CLH@"
# Path to the firmware.

View File

@@ -5,13 +5,9 @@ authors = { workspace = true }
edition = { workspace = true }
license = { workspace = true }
[dev-dependencies]
futures = "0.1.27"
[dependencies]
anyhow = { workspace = true }
async-trait = { workspace = true }
log = { workspace = true }
protobuf = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
@@ -31,3 +27,6 @@ protocols = { workspace = true, features = ["async"] }
[features]
default = []
[package.metadata.cargo-machete]
ignored = ["slog-scope"]

View File

@@ -28,8 +28,6 @@ path-clean = "1.0.1"
lazy_static = { workspace = true }
tracing = { workspace = true }
ttrpc = { workspace = true, features = ["async"] }
protobuf = { workspace = true }
oci-spec = { workspace = true }
futures = "0.3.25"
safe-path = "0.1.0"
crossbeam-channel = "0.5.6"
@@ -44,7 +42,6 @@ kata-sys-util = { workspace = true }
kata-types = { workspace = true }
logging = { workspace = true }
protocols = { workspace = true, features = ["async"] }
shim-interface = { workspace = true }
persist = { workspace = true }
ch-config = { workspace = true, optional = true }
tests_utils = { workspace = true }

View File

@@ -118,13 +118,11 @@ impl TryFrom<NamedHypervisorConfig> for VmConfig {
// Note how CH handles the different image types:
//
// - A standard image is specified in PmemConfig.
// - An initrd/initramfs is specified in PayloadConfig.
// - A confidential guest image is specified by a DiskConfig.
// - An image is specified in DiskConfig.
// Note: pmem is not used as it's not properly supported by Cloud Hypervisor.
// - If TDX is enabled, the firmware (`td-shim` [1]) must be
// specified in PayloadConfig.
// - A confidential guest initrd is specified by a PayloadConfig with
// firmware.
//
// [1] - https://github.com/confidential-containers/td-shim
let boot_info = cfg.boot_info;
@@ -140,14 +138,6 @@ impl TryFrom<NamedHypervisorConfig> for VmConfig {
return Err(VmConfigError::NoBootFile);
}
let pmem = if use_initrd || guest_protection_is_tdx(guest_protection_to_use.clone()) {
None
} else {
let pmem = PmemConfig::try_from(&boot_info).map_err(VmConfigError::PmemError)?;
Some(vec![pmem])
};
let payload = Some(
PayloadConfig::try_from((
boot_info.clone(),
@@ -159,7 +149,7 @@ impl TryFrom<NamedHypervisorConfig> for VmConfig {
let mut disks: Vec<DiskConfig> = vec![];
if use_image && guest_protection_is_tdx(guest_protection_to_use.clone()) {
if use_image {
let disk = DiskConfig::try_from(boot_info).map_err(VmConfigError::DiskError)?;
disks.push(disk);
@@ -199,7 +189,6 @@ impl TryFrom<NamedHypervisorConfig> for VmConfig {
fs,
net,
devices: host_devices,
pmem,
disks,
vsock: Some(vsock),
rng,
@@ -1656,7 +1645,6 @@ mod tests {
let (memory_info_confidential_guest, mem_config_confidential_guest) =
make_memory_objects(79, usable_max_mem_bytes, true);
let (_, pmem_config_with_image) = make_bootinfo_pmemconfig_objects(image);
let (machine_info, rng_config) = make_machineinfo_rngconfig_objects(entropy_source);
let payload_firmware = None;
@@ -1664,6 +1652,7 @@ mod tests {
let (boot_info_with_initrd, payload_config_with_initrd) =
make_bootinfo_payloadconfig_objects(kernel, initramfs, payload_firmware, None);
let (_, disk_config_with_image) = make_bootinfo_diskconfig_objects(image);
let (_, disk_config_confidential_guest_image) = make_bootinfo_diskconfig_objects(image);
let boot_info_tdx_image = BootInfo {
@@ -1762,7 +1751,7 @@ mod tests {
vsock: Some(valid_vsock.clone()),
// rootfs image specific
pmem: Some(vec![pmem_config_with_image]),
disks: Some(vec![disk_config_with_image]),
payload: Some(PayloadConfig {
kernel: Some(PathBuf::from(kernel)),

View File

@@ -110,6 +110,16 @@ pub struct DeviceConfig {
pub pci_segment: u16,
}
#[derive(Serialize, Deserialize, Clone, Copy, Debug, PartialEq, Eq, Default)]
pub enum ImageType {
FixedVhd,
Qcow2,
Raw,
Vhdx,
#[default]
Unknown,
}
#[derive(Clone, Debug, PartialEq, Eq, Deserialize, Serialize, Default)]
pub struct DiskConfig {
pub path: Option<PathBuf>,
@@ -135,6 +145,8 @@ pub struct DiskConfig {
pub disable_io_uring: bool,
#[serde(default)]
pub pci_segment: u16,
#[serde(default)]
pub image_type: ImageType,
}
#[derive(Clone, Debug, PartialEq, Eq, Deserialize, Serialize, Default)]

View File

@@ -123,7 +123,12 @@ impl CloudHypervisorInner {
}
}
pub fn set_hypervisor_config(&mut self, config: HypervisorConfig) {
pub fn set_hypervisor_config(&mut self, mut config: HypervisorConfig) {
// virtio-pmem is not supported for Cloud Hypervisor.
if config.boot_info.vm_rootfs_driver == crate::VM_ROOTFS_DRIVER_PMEM {
config.boot_info.vm_rootfs_driver = crate::VM_ROOTFS_DRIVER_BLK.to_string();
}
self.config = config;
}

View File

@@ -27,6 +27,7 @@ use ch_config::ch_api::{
};
use ch_config::convert::DEFAULT_NUM_PCI_SEGMENTS;
use ch_config::DiskConfig;
use ch_config::ImageType;
use ch_config::{net_util::MacAddr, DeviceConfig, FsConfig, NetConfig, VsockConfig};
use kata_sys_util::netns::NetnsGuard;
use kata_types::config::hypervisor::RateLimiterConfig;
@@ -550,6 +551,7 @@ impl TryFrom<BlockConfig> for DiskConfig {
readonly: blkcfg.is_readonly,
num_queues: blkcfg.num_queues,
queue_size: blkcfg.queue_size as u16,
image_type: ImageType::Raw,
..Default::default()
};

View File

@@ -15,7 +15,6 @@ use crate::utils::vm_cleanup;
use crate::utils::{bytes_to_megs, get_jailer_root, get_sandbox_path, megs_to_bytes};
use crate::MemoryConfig;
use crate::VM_ROOTFS_DRIVER_BLK;
use crate::VM_ROOTFS_DRIVER_PMEM;
use crate::{VcpuThreadIds, VmmState};
use anyhow::{anyhow, Context, Result};
use ch_config::ch_api::cloud_hypervisor_vm_netdev_add_with_fds;
@@ -130,12 +129,8 @@ impl CloudHypervisorInner {
let confidential_guest = cfg.security_info.confidential_guest;
// Note that the configuration option hypervisor.block_device_driver is not used.
let rootfs_driver = if confidential_guest {
// PMEM is not available with TDX.
VM_ROOTFS_DRIVER_BLK
} else {
VM_ROOTFS_DRIVER_PMEM
};
// NVDIMM is not supported for Cloud Hypervisor.
let rootfs_driver = VM_ROOTFS_DRIVER_BLK;
let rootfs_type = match cfg.boot_info.rootfs_type.is_empty() {
true => DEFAULT_CH_ROOTFS_TYPE,
@@ -155,6 +150,7 @@ impl CloudHypervisorInner {
&cfg.boot_info.kernel_verity_params,
rootfs_driver,
rootfs_type,
true,
)?;
let mut console_params = if enable_debug {
@@ -1104,7 +1100,7 @@ fn get_guest_protection() -> Result<GuestProtection> {
Ok(guest_protection)
}
// Return a TID/VCPU map from a specified /proc/{pid} path.
// Return a VCPU/TID map from a specified /proc/{pid} path.
fn get_ch_vcpu_tids(proc_path: &str) -> Result<HashMap<u32, u32>> {
const VCPU_STR: &str = "vcpu";
@@ -1147,7 +1143,7 @@ fn get_ch_vcpu_tids(proc_path: &str) -> Result<HashMap<u32, u32>> {
.parse::<u32>()
.map_err(|e| anyhow!(e).context("Invalid vcpu id."))?;
vcpus.insert(tid, vcpu_id);
vcpus.insert(vcpu_id, tid);
}
if vcpus.is_empty() {
@@ -1613,4 +1609,65 @@ mod tests {
assert!(actual_error == expected_error, "{}", msg);
}
}
#[actix_rt::test]
async fn test_get_ch_vcpu_tids_mapping() {
let tmp_dir = Builder::new().prefix("fake-proc-pid").tempdir().unwrap();
let task_dir = tmp_dir.path().join("task");
fs::create_dir_all(&task_dir).unwrap();
#[derive(Debug)]
struct ThreadInfo<'a> {
tid: &'a str,
comm: &'a str,
}
let threads = &[
// Non-vcpu thread, should be skipped.
ThreadInfo {
tid: "1000",
comm: "main_thread\n",
},
ThreadInfo {
tid: "2001",
comm: "vcpu0\n",
},
ThreadInfo {
tid: "2002",
comm: "vcpu1\n",
},
ThreadInfo {
tid: "2003",
comm: "vcpu2\n",
},
];
for t in threads {
let tid_dir = task_dir.join(t.tid);
fs::create_dir_all(&tid_dir).unwrap();
fs::write(tid_dir.join("comm"), t.comm).unwrap();
}
let proc_path = tmp_dir.path().to_str().unwrap();
let result = get_ch_vcpu_tids(proc_path);
let msg = format!("result: {result:?}");
if std::env::var("DEBUG").is_ok() {
println!("DEBUG: {msg}");
}
let vcpus = result.unwrap();
// The mapping must be vcpu_id -> tid.
assert_eq!(vcpus.len(), 3, "non-vcpu threads should be excluded");
assert_eq!(vcpus[&0], 2001, "vcpu 0 should map to tid 2001");
assert_eq!(vcpus[&1], 2002, "vcpu 1 should map to tid 2002");
assert_eq!(vcpus[&2], 2003, "vcpu 2 should map to tid 2003");
assert!(
!vcpus.contains_key(&1000),
"non-vcpu thread should not be in the map"
);
}
}

View File

@@ -13,17 +13,17 @@ use crate::device::DeviceType;
use crate::Hypervisor as hypervisor;
use anyhow::{Context, Result};
use async_trait::async_trait;
pub use kata_types::device::{
DRIVER_BLK_CCW_TYPE as KATA_CCW_DEV_TYPE, DRIVER_BLK_MMIO_TYPE as KATA_MMIO_BLK_DEV_TYPE,
DRIVER_BLK_PCI_TYPE as KATA_BLK_DEV_TYPE, DRIVER_NVDIMM_TYPE as KATA_NVDIMM_DEV_TYPE,
DRIVER_SCSI_TYPE as KATA_SCSI_DEV_TYPE,
};
/// VIRTIO_BLOCK_PCI indicates block driver is virtio-pci based
pub const VIRTIO_BLOCK_PCI: &str = "virtio-blk-pci";
pub const VIRTIO_BLOCK_MMIO: &str = "virtio-blk-mmio";
pub const VIRTIO_BLOCK_CCW: &str = "virtio-blk-ccw";
pub const VIRTIO_PMEM: &str = "virtio-pmem";
pub const KATA_MMIO_BLK_DEV_TYPE: &str = "mmioblk";
pub const KATA_BLK_DEV_TYPE: &str = "blk";
pub const KATA_CCW_DEV_TYPE: &str = "ccw";
pub const KATA_NVDIMM_DEV_TYPE: &str = "nvdimm";
pub const KATA_SCSI_DEV_TYPE: &str = "scsi";
#[derive(Clone, Copy, Debug, Default)]
pub enum BlockDeviceAio {
@@ -95,6 +95,9 @@ pub struct BlockConfig {
/// scsi_addr is of the format SCSI-Id:LUN
pub scsi_addr: Option<String>,
/// CCW device address for virtio-blk-ccw on s390x (e.g., "0.0.0005")
pub ccw_addr: Option<String>,
/// device attach count
pub attach_count: u64,

View File

@@ -150,6 +150,7 @@ impl DragonballInner {
&self.config.boot_info.kernel_verity_params,
&rootfs_driver,
&self.config.boot_info.rootfs_type,
true,
)?;
kernel_params.append(&mut rootfs_params);
}

View File

@@ -90,6 +90,7 @@ impl FcInner {
&self.config.boot_info.kernel_verity_params,
&self.config.blockdev_info.block_device_driver,
&self.config.boot_info.rootfs_type,
true,
)?;
kernel_params.append(&mut rootfs_params);
kernel_params.append(&mut KernelParams::from_string(

View File

@@ -10,8 +10,8 @@ use crate::{
VM_ROOTFS_DRIVER_BLK, VM_ROOTFS_DRIVER_BLK_CCW, VM_ROOTFS_DRIVER_MMIO, VM_ROOTFS_DRIVER_PMEM,
VM_ROOTFS_ROOT_BLK, VM_ROOTFS_ROOT_PMEM,
};
use kata_types::config::LOG_VPORT_OPTION;
use kata_types::config::hypervisor::{parse_kernel_verity_params, VERITY_BLOCK_SIZE_BYTES};
use kata_types::config::LOG_VPORT_OPTION;
use kata_types::fs::{
VM_ROOTFS_FILESYSTEM_EROFS, VM_ROOTFS_FILESYSTEM_EXT4, VM_ROOTFS_FILESYSTEM_XFS,
};
@@ -66,8 +66,7 @@ struct KernelVerityConfig {
}
fn new_kernel_verity_params(params_string: &str) -> Result<Option<KernelVerityConfig>> {
let cfg = parse_kernel_verity_params(params_string)
.map_err(|err| anyhow!(err.to_string()))?;
let cfg = parse_kernel_verity_params(params_string).map_err(|err| anyhow!(err.to_string()))?;
Ok(cfg.map(|params| KernelVerityConfig {
root_hash: params.root_hash,
@@ -145,6 +144,7 @@ impl KernelParams {
kernel_verity_params: &str,
rootfs_driver: &str,
rootfs_type: &str,
use_dax: bool,
) -> Result<Self> {
let mut params = vec![];
@@ -153,16 +153,29 @@ impl KernelParams {
params.push(Param::new("root", VM_ROOTFS_ROOT_PMEM));
match rootfs_type {
VM_ROOTFS_FILESYSTEM_EXT4 => {
params.push(Param::new(
"rootflags",
"dax,data=ordered,errors=remount-ro ro",
));
if use_dax {
params.push(Param::new(
"rootflags",
"dax,data=ordered,errors=remount-ro ro",
));
} else {
params
.push(Param::new("rootflags", "data=ordered,errors=remount-ro ro"));
}
}
VM_ROOTFS_FILESYSTEM_XFS => {
params.push(Param::new("rootflags", "dax ro"));
if use_dax {
params.push(Param::new("rootflags", "dax ro"));
} else {
params.push(Param::new("rootflags", "ro"));
}
}
VM_ROOTFS_FILESYSTEM_EROFS => {
params.push(Param::new("rootflags", "dax ro"));
if use_dax {
params.push(Param::new("rootflags", "dax ro"));
} else {
params.push(Param::new("rootflags", "ro"));
}
}
_ => {
return Err(anyhow!("Unsupported rootfs type {}", rootfs_type));
@@ -346,6 +359,7 @@ mod tests {
struct TestData<'a> {
rootfs_driver: &'a str,
rootfs_type: &'a str,
use_dax: bool,
expect_params: KernelParams,
result: Result<()>,
}
@@ -353,10 +367,11 @@ mod tests {
#[test]
fn test_rootfs_kernel_params() {
let tests = &[
// EXT4
// EXT4 with DAX
TestData {
rootfs_driver: VM_ROOTFS_DRIVER_PMEM,
rootfs_type: VM_ROOTFS_FILESYSTEM_EXT4,
use_dax: true,
expect_params: KernelParams {
params: [
Param::new("root", VM_ROOTFS_ROOT_PMEM),
@@ -370,6 +385,7 @@ mod tests {
TestData {
rootfs_driver: VM_ROOTFS_DRIVER_BLK,
rootfs_type: VM_ROOTFS_FILESYSTEM_EXT4,
use_dax: true,
expect_params: KernelParams {
params: [
Param::new("root", VM_ROOTFS_ROOT_BLK),
@@ -380,14 +396,15 @@ mod tests {
},
result: Ok(()),
},
// XFS
// XFS without DAX
TestData {
rootfs_driver: VM_ROOTFS_DRIVER_PMEM,
rootfs_type: VM_ROOTFS_FILESYSTEM_XFS,
use_dax: false,
expect_params: KernelParams {
params: [
Param::new("root", VM_ROOTFS_ROOT_PMEM),
Param::new("rootflags", "dax ro"),
Param::new("rootflags", "ro"),
Param::new("rootfstype", VM_ROOTFS_FILESYSTEM_XFS),
]
.to_vec(),
@@ -397,6 +414,7 @@ mod tests {
TestData {
rootfs_driver: VM_ROOTFS_DRIVER_BLK,
rootfs_type: VM_ROOTFS_FILESYSTEM_XFS,
use_dax: true,
expect_params: KernelParams {
params: [
Param::new("root", VM_ROOTFS_ROOT_BLK),
@@ -407,10 +425,11 @@ mod tests {
},
result: Ok(()),
},
// EROFS
// EROFS with DAX
TestData {
rootfs_driver: VM_ROOTFS_DRIVER_PMEM,
rootfs_type: VM_ROOTFS_FILESYSTEM_EROFS,
use_dax: true,
expect_params: KernelParams {
params: [
Param::new("root", VM_ROOTFS_ROOT_PMEM),
@@ -424,6 +443,7 @@ mod tests {
TestData {
rootfs_driver: VM_ROOTFS_DRIVER_BLK,
rootfs_type: VM_ROOTFS_FILESYSTEM_EROFS,
use_dax: true,
expect_params: KernelParams {
params: [
Param::new("root", VM_ROOTFS_ROOT_BLK),
@@ -438,6 +458,7 @@ mod tests {
TestData {
rootfs_driver: "foo",
rootfs_type: VM_ROOTFS_FILESYSTEM_EXT4,
use_dax: true,
expect_params: KernelParams {
params: [
Param::new("root", VM_ROOTFS_ROOT_BLK),
@@ -452,6 +473,7 @@ mod tests {
TestData {
rootfs_driver: VM_ROOTFS_DRIVER_BLK,
rootfs_type: "foo",
use_dax: true,
expect_params: KernelParams {
params: [
Param::new("root", VM_ROOTFS_ROOT_BLK),
@@ -466,8 +488,12 @@ mod tests {
for (i, t) in tests.iter().enumerate() {
let msg = format!("test[{i}]: {t:?}");
let result =
KernelParams::new_rootfs_kernel_params("", t.rootfs_driver, t.rootfs_type);
let result = KernelParams::new_rootfs_kernel_params(
"",
t.rootfs_driver,
t.rootfs_type,
t.use_dax,
);
let msg = format!("{msg}, result: {result:?}");
if t.result.is_ok() {
assert!(result.is_ok(), "{}", msg);
@@ -486,6 +512,7 @@ mod tests {
"root_hash=abc,salt=def,data_blocks=1,data_block_size=4096,hash_block_size=4096",
VM_ROOTFS_DRIVER_BLK,
VM_ROOTFS_FILESYSTEM_EXT4,
false,
)?;
let params_string = params.to_string()?;
assert!(params_string.contains("dm-mod.create="));
@@ -496,6 +523,7 @@ mod tests {
"root_hash=abc,data_blocks=1,data_block_size=4096,hash_block_size=4096",
VM_ROOTFS_DRIVER_BLK,
VM_ROOTFS_FILESYSTEM_EXT4,
false,
)
.err()
.expect("expected missing salt error");
@@ -505,6 +533,7 @@ mod tests {
"root_hash=abc,salt=def,data_block_size=4096,hash_block_size=4096",
VM_ROOTFS_DRIVER_BLK,
VM_ROOTFS_FILESYSTEM_EXT4,
false,
)
.err()
.expect("expected missing data_blocks error");
@@ -514,6 +543,7 @@ mod tests {
"root_hash=abc,salt=def,data_blocks=foo,data_block_size=4096,hash_block_size=4096",
VM_ROOTFS_DRIVER_BLK,
VM_ROOTFS_FILESYSTEM_EXT4,
false,
)
.err()
.expect("expected invalid data_blocks error");
@@ -523,6 +553,7 @@ mod tests {
"root_hash=abc,salt=def,data_blocks=1,data_block_size=4096,hash_block_size=4096,badfield",
VM_ROOTFS_DRIVER_BLK,
VM_ROOTFS_FILESYSTEM_EXT4,
false,
)
.err()
.expect("expected invalid entry error");

View File

@@ -179,10 +179,17 @@ impl Kernel {
let mut kernel_params = KernelParams::new(config.debug_info.enable_debug);
if config.boot_info.initrd.is_empty() {
// DAX is disabled on ARM due to a kernel panic in caches_clean_inval_pou.
#[cfg(target_arch = "aarch64")]
let use_dax = false;
#[cfg(not(target_arch = "aarch64"))]
let use_dax = true;
let mut rootfs_params = KernelParams::new_rootfs_kernel_params(
&config.boot_info.kernel_verity_params,
&config.boot_info.vm_rootfs_driver,
&config.boot_info.rootfs_type,
use_dax,
)
.context("adding rootfs/verity params failed")?;
kernel_params.append(&mut rootfs_params);
@@ -249,29 +256,8 @@ struct Memory {
impl Memory {
fn new(config: &HypervisorConfig) -> Memory {
// Move this to QemuConfig::adjust_config()?
let mut mem_size = config.memory_info.default_memory as u64;
let mut max_mem_size = config.memory_info.default_maxmemory as u64;
if let Ok(sysinfo) = nix::sys::sysinfo::sysinfo() {
let host_memory = sysinfo.ram_total() >> 20;
if mem_size > host_memory {
info!(sl!(), "'default_memory' given in configuration.toml is greater than host memory, adjusting to host memory");
mem_size = host_memory
}
if max_mem_size == 0 || max_mem_size > host_memory {
max_mem_size = host_memory
}
} else {
warn!(sl!(), "Failed to get host memory size, cannot verify or adjust configuration.toml's 'default_maxmemory'");
if max_mem_size == 0 {
max_mem_size = mem_size;
};
}
let mem_size = config.memory_info.default_memory as u64;
let max_mem_size = config.memory_info.default_maxmemory as u64;
// Memory sizes are given in megabytes in configuration.toml so we
// need to convert them to bytes for storage.
@@ -293,6 +279,18 @@ impl Memory {
self.memory_backend_file = Some(mem_file.clone());
self
}
#[allow(dead_code)]
fn set_maxmem_size(&mut self, max_size: u64) -> &mut Self {
self.max_size = max_size;
self
}
#[allow(dead_code)]
fn set_num_slots(&mut self, num_slots: u32) -> &mut Self {
self.num_slots = num_slots;
self
}
}
#[async_trait]
@@ -385,7 +383,7 @@ impl ToQemuParams for Cpu {
/// Error type for CCW Subchannel operations
#[derive(Debug)]
#[allow(dead_code)]
enum CcwError {
pub enum CcwError {
DeviceAlreadyExists(String), // Error when trying to add an existing device
#[allow(dead_code)]
DeviceNotFound(String), // Error when trying to remove a nonexistent device
@@ -416,7 +414,7 @@ impl CcwSubChannel {
/// # Returns
/// - `Result<u32, CcwError>`: slot index of the added device
/// or an error if the device already exists
fn add_device(&mut self, dev_id: &str) -> Result<u32, CcwError> {
pub fn add_device(&mut self, dev_id: &str) -> Result<u32, CcwError> {
if self.devices.contains_key(dev_id) {
Err(CcwError::DeviceAlreadyExists(dev_id.to_owned()))
} else {
@@ -435,8 +433,7 @@ impl CcwSubChannel {
/// # Returns
/// - `Result<(), CcwError>`: Ok(()) if the device was removed
/// or an error if the device was not found
#[allow(dead_code)]
fn remove_device(&mut self, dev_id: &str) -> Result<(), CcwError> {
pub fn remove_device(&mut self, dev_id: &str) -> Result<(), CcwError> {
if self.devices.remove(dev_id).is_some() {
Ok(())
} else {
@@ -444,17 +441,30 @@ impl CcwSubChannel {
}
}
/// Formats the CCW address for a given slot
/// Formats the CCW address for a given slot.
/// Uses the 0xfe channel subsystem ID used by QEMU.
///
/// # Arguments
/// - `slot`: slot index
///
/// # Returns
/// - `String`: formatted CCW address (e.g. `fe.0.0000`)
fn address_format_ccw(&self, slot: u32) -> String {
pub fn address_format_ccw(&self, slot: u32) -> String {
format!("fe.{:x}.{:04x}", self.addr, slot)
}
/// Formats the guest-visible CCW address for a given slot.
/// Uses channel subsystem ID 0 (guest perspective).
///
/// # Arguments
/// - `slot`: slot index
///
/// # Returns
/// - `String`: formatted guest-visible CCW address (e.g. `0.0.0000`)
pub fn address_format_ccw_for_virt_server(&self, slot: u32) -> String {
format!("0.{:x}.{:04x}", self.addr, slot)
}
/// Sets the address of the subchannel.
/// # Arguments
/// - `addr`: subchannel address to set
@@ -1869,6 +1879,7 @@ struct ObjectSevSnpGuest {
reduced_phys_bits: u32,
kernel_hashes: bool,
host_data: Option<String>,
policy: u32,
is_snp: bool,
}
@@ -1880,9 +1891,15 @@ impl ObjectSevSnpGuest {
reduced_phys_bits,
kernel_hashes: true,
host_data,
policy: 0x30000,
is_snp,
}
}
fn set_policy(&mut self, policy: u32) -> &mut Self {
self.policy = policy;
self
}
}
#[async_trait]
@@ -1905,6 +1922,7 @@ impl ToQemuParams for ObjectSevSnpGuest {
"kernel-hashes={}",
if self.kernel_hashes { "on" } else { "off" }
));
params.push(format!("policy=0x{:x}", self.policy));
if let Some(host_data) = &self.host_data {
params.push(format!("host-data={host_data}"))
}
@@ -2267,6 +2285,12 @@ impl<'a> QemuCmdLine<'a> {
Ok(qemu_cmd_line)
}
/// Takes ownership of the CCW subchannel, leaving `None` in its place.
/// Used to transfer boot-time CCW state to Qmp for hotplug allocation.
pub fn take_ccw_subchannel(&mut self) -> Option<CcwSubChannel> {
self.ccw_subchannel.take()
}
fn add_monitor(&mut self, proto: &str) -> Result<()> {
let monitor = QmpSocket::new(self.id.as_str(), MonitorProtocol::new(proto))?;
self.devices.push(Box::new(monitor));
@@ -2554,13 +2578,19 @@ impl<'a> QemuCmdLine<'a> {
firmware: &str,
host_data: &Option<String>,
) {
let sev_snp_object =
// For SEV-SNP, memory overcommit is not supported. we only set the memory size.
self.memory.set_maxmem_size(0).set_num_slots(0);
let mut sev_snp_object =
ObjectSevSnpGuest::new(true, cbitpos, phys_addr_reduction, host_data.clone());
sev_snp_object.set_policy(self.config.security_info.snp_guest_policy);
self.devices.push(Box::new(sev_snp_object));
self.devices.push(Box::new(Bios::new(firmware.to_owned())));
self.machine
.set_kernel_irqchip("split")
.set_confidential_guest_support("snp")
.set_nvdimm(false);

View File

@@ -10,6 +10,7 @@ use crate::qemu::qmp::get_qmp_socket_path;
use crate::{
device::driver::ProtectionDeviceConfig, hypervisor_persist::HypervisorState, selinux,
HypervisorConfig, MemoryConfig, VcpuThreadIds, VsockDevice, HYPERVISOR_QEMU,
KATA_BLK_DEV_TYPE, KATA_CCW_DEV_TYPE, KATA_NVDIMM_DEV_TYPE, KATA_SCSI_DEV_TYPE,
};
use crate::utils::{
@@ -21,7 +22,7 @@ use anyhow::{anyhow, Context, Result};
use async_trait::async_trait;
use kata_sys_util::netns::NetnsGuard;
use kata_types::build_path;
use kata_types::config::hypervisor::RootlessUser;
use kata_types::config::hypervisor::{RootlessUser, VIRTIO_BLK_CCW};
use kata_types::rootless::is_rootless;
use kata_types::{
capabilities::{Capabilities, CapabilityBits},
@@ -133,18 +134,18 @@ impl QemuInner {
continue;
}
match block_dev.config.driver_option.as_str() {
"nvdimm" => cmdline.add_nvdimm(
KATA_NVDIMM_DEV_TYPE => cmdline.add_nvdimm(
&block_dev.config.path_on_host,
block_dev.config.is_readonly,
)?,
"ccw" | "blk" | "scsi" => cmdline.add_block_device(
KATA_CCW_DEV_TYPE | KATA_BLK_DEV_TYPE | KATA_SCSI_DEV_TYPE => cmdline.add_block_device(
block_dev.device_id.as_str(),
&block_dev.config.path_on_host,
block_dev
.config
.is_direct
.unwrap_or(self.config.blockdev_info.block_device_cache_direct),
block_dev.config.driver_option.as_str() == "scsi",
block_dev.config.driver_option.as_str() == KATA_SCSI_DEV_TYPE,
)?,
unsupported => {
info!(sl!(), "unsupported block device driver: {}", unsupported)
@@ -285,7 +286,12 @@ impl QemuInner {
let qmp_socket_path = get_qmp_socket_path(self.id.as_str());
match Qmp::new(&qmp_socket_path) {
Ok(qmp) => self.qmp = Some(qmp),
Ok(mut qmp) => {
if let Some(subchannel) = cmdline.take_ccw_subchannel() {
qmp.set_ccw_subchannel(subchannel);
}
self.qmp = Some(qmp);
}
Err(e) => {
error!(sl!(), "couldn't initialise QMP: {:?}", e);
return Err(e);
@@ -842,9 +848,10 @@ impl QemuInner {
qmp.hotplug_network_device(&netdev, &virtio_net_device)?
}
DeviceType::Block(mut block_device) => {
let (pci_path, scsi_addr) = qmp
let block_driver = &self.config.blockdev_info.block_device_driver;
let (pci_path, addr_str) = qmp
.hotplug_block_device(
&self.config.blockdev_info.block_device_driver,
block_driver,
block_device.config.index,
&block_device.config.path_on_host,
&block_device.config.blkdev_aio.to_string(),
@@ -857,8 +864,12 @@ impl QemuInner {
if pci_path.is_some() {
block_device.config.pci_path = pci_path;
}
if scsi_addr.is_some() {
block_device.config.scsi_addr = scsi_addr;
if let Some(addr) = addr_str {
if block_driver == VIRTIO_BLK_CCW {
block_device.config.ccw_addr = Some(addr);
} else {
block_device.config.scsi_addr = Some(addr);
}
}
return Ok(DeviceType::Block(block_device));

View File

@@ -4,12 +4,12 @@
//
use crate::device::pci_path::PciPath;
use crate::qemu::cmdline_generator::{DeviceVirtioNet, Netdev, QMP_SOCKET_FILE};
use crate::qemu::cmdline_generator::{CcwSubChannel, DeviceVirtioNet, Netdev, QMP_SOCKET_FILE};
use crate::utils::get_jailer_root;
use crate::VcpuThreadIds;
use anyhow::{anyhow, Context, Result};
use kata_types::config::hypervisor::VIRTIO_SCSI;
use kata_types::config::hypervisor::{VIRTIO_BLK_CCW, VIRTIO_SCSI};
use kata_types::rootless::is_rootless;
use nix::sys::socket::{sendmsg, ControlMessage, MsgFlags};
use qapi_qmp::{
@@ -50,6 +50,11 @@ pub struct Qmp {
// blocks seem ever to be onlined in the guest by kata-agent.
// Store as u64 to keep up the convention of bytes being represented as u64.
guest_memory_block_size: u64,
// CCW subchannel for s390x device address management.
// Transferred from QemuCmdLine after boot so that hotplug allocations
// continue from where boot-time allocations left off.
ccw_subchannel: Option<CcwSubChannel>,
}
// We have to implement Debug since the Hypervisor trait requires it and Qmp
@@ -76,6 +81,7 @@ impl Qmp {
stream,
)),
guest_memory_block_size: 0,
ccw_subchannel: None,
};
let info = qmp.qmp.handshake().context("qmp handshake failed")?;
@@ -102,6 +108,10 @@ impl Qmp {
.with_context(|| format!("timed out waiting for QMP ready: {}", qmp_sock_path))
}
pub fn set_ccw_subchannel(&mut self, subchannel: CcwSubChannel) {
self.ccw_subchannel = Some(subchannel);
}
pub fn set_ignore_shared_memory_capability(&mut self) -> Result<()> {
self.qmp
.execute(&migrate_set_capabilities {
@@ -605,6 +615,13 @@ impl Qmp {
/// {"execute":"device_add","arguments":{"driver":"scsi-hd","drive":"virtio-scsi0","id":"scsi_device_0","bus":"virtio-scsi1.0"}}
/// {"return": {}}
///
/// Hotplug virtio-blk-ccw block device on s390x
/// # virtio-blk-ccw0
/// {"execute":"blockdev_add", "arguments": {"file":"/path/to/block.image","format":"qcow2","id":"virtio-blk-ccw0"}}
/// {"return": {}}
/// {"execute":"device_add","arguments":{"driver":"virtio-blk-ccw","id":"virtio-blk-ccw0","drive":"virtio-blk-ccw0","devno":"fe.0.0005","share-rw":true}}
/// {"return": {}}
///
#[allow(clippy::too_many_arguments)]
pub fn hotplug_block_device(
&mut self,
@@ -711,6 +728,14 @@ impl Qmp {
blkdev_add_args.insert("lun".to_string(), lun.into());
blkdev_add_args.insert("share-rw".to_string(), true.into());
info!(
sl!(),
"hotplug_block_device(): device_add arguments: bus: {}, id: {}, driver: {}, blkdev_add_args: {:#?}",
"scsi0.0",
node_name,
"scsi-hd",
blkdev_add_args
);
self.qmp
.execute(&qmp::device_add {
bus: Some("scsi0.0".to_string()),
@@ -727,11 +752,60 @@ impl Qmp {
);
Ok((None, Some(scsi_addr)))
} else if block_driver == VIRTIO_BLK_CCW {
let subchannel = self
.ccw_subchannel
.as_mut()
.ok_or_else(|| anyhow!("CCW subchannel not available for virtio-blk-ccw hotplug"))?;
let slot = subchannel
.add_device(&node_name)
.map_err(|e| anyhow!("CCW subchannel add_device failed: {:?}", e))?;
let devno = subchannel.address_format_ccw(slot);
let ccw_addr = subchannel.address_format_ccw_for_virt_server(slot);
blkdev_add_args.insert("devno".to_owned(), devno.clone().into());
blkdev_add_args.insert("share-rw".to_string(), true.into());
info!(
sl!(),
"hotplug_block_device(): CCW device_add: id: {}, driver: {}, blkdev_add_args: {:#?}, ccw_addr: {}",
node_name,
block_driver,
blkdev_add_args,
ccw_addr
);
let device_add_result = self.qmp.execute(&qmp::device_add {
bus: None,
id: Some(node_name.clone()),
driver: block_driver.to_string(),
arguments: blkdev_add_args,
});
if let Err(e) = device_add_result {
// Roll back CCW subchannel state if QMP device_add fails
let _ = subchannel.remove_device(&node_name);
return Err(anyhow!("device_add {:?}", e));
}
info!(
sl!(),
"hotplug CCW block device return ccw address: {:?}", &ccw_addr
);
Ok((None, Some(ccw_addr)))
} else {
let (bus, slot) = self.find_free_slot()?;
blkdev_add_args.insert("addr".to_owned(), format!("{slot:02}").into());
blkdev_add_args.insert("share-rw".to_string(), true.into());
info!(
sl!(),
"hotplug_block_device(): device_add arguments: bus: {}, id: {}, driver: {}, blkdev_add_args: {:#?}",
bus,
node_name,
block_driver,
blkdev_add_args
);
self.qmp
.execute(&qmp::device_add {
bus: Some(bus),

View File

@@ -8,12 +8,10 @@ license = { workspace = true }
[dependencies]
async-trait = { workspace = true }
anyhow = { workspace = true }
libc = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
# Local dependencies
kata-sys-util = { workspace = true }
kata-types = { workspace = true }
shim-interface = { workspace = true }
safe-path = { workspace = true }

View File

@@ -15,7 +15,6 @@ test-utils = { workspace = true }
actix-rt = { workspace = true }
anyhow = { workspace = true }
async-trait = { workspace = true }
bitflags = "2.9.0"
byte-unit = "5.1.6"
cgroups-rs = { version = "0.5.0", features = ["oci"] }
futures = "0.3.11"
@@ -41,7 +40,6 @@ hex = "0.4"
## Dependencies from `rust-netlink`
netlink-packet-route = "0.26"
netlink-sys = "0.8"
rtnetlink = "0.19"
# Local dependencies
@@ -54,3 +52,7 @@ persist = { workspace = true }
tests_utils = { workspace = true }
[features]
[package.metadata.cargo-machete]
ignored = ["slog-scope"]

View File

@@ -429,14 +429,16 @@ impl ResourceManagerInner {
.await
.context("do handle device")?;
// create block device for kata agent,
// if driver is virtio-blk-pci, the id will be pci address.
// create block device for kata agent.
// The device ID is derived from the available address: PCI, SCSI,
// CCW, or virtual path, depending on the driver and configuration.
if let DeviceType::Block(device) = device_info {
// The following would work for drivers virtio-blk-pci and virtio-mmio and virtio-scsi.
let id = if let Some(pci_path) = device.config.pci_path {
pci_path.to_string()
} else if let Some(scsi_address) = device.config.scsi_addr {
scsi_address
} else if let Some(ccw_addr) = device.config.ccw_addr {
ccw_addr
} else {
device.config.virt_path.clone()
};

View File

@@ -100,7 +100,13 @@ impl BlockRootfs {
VIRTIO_BLK_MMIO => {
storage.source = device.config.virt_path;
}
VIRTIO_SCSI | VIRTIO_BLK_CCW | VIRTIO_PMEM => {
VIRTIO_BLK_CCW => {
storage.source = device
.config
.ccw_addr
.ok_or_else(|| anyhow!("CCW address missing for ccw block device"))?;
}
VIRTIO_SCSI | VIRTIO_PMEM => {
return Err(anyhow!(
"Complete support for block driver {} has not been implemented yet",
block_driver

View File

@@ -15,6 +15,10 @@ use crate::{
};
use anyhow::{anyhow, Context, Result};
use kata_sys_util::mount::{get_mount_options, get_mount_path};
use kata_types::device::{
DRIVER_BLK_CCW_TYPE as KATA_CCW_DEV_TYPE, DRIVER_BLK_PCI_TYPE as KATA_BLK_DEV_TYPE,
DRIVER_SCSI_TYPE as KATA_SCSI_DEV_TYPE,
};
use oci_spec::runtime as oci;
use hypervisor::device::DeviceType;
@@ -22,9 +26,6 @@ use hypervisor::device::DeviceType;
pub const DEFAULT_VOLUME_FS_TYPE: &str = "ext4";
pub const KATA_MOUNT_BIND_TYPE: &str = "bind";
pub const KATA_BLK_DEV_TYPE: &str = "blk";
pub const KATA_SCSI_DEV_TYPE: &str = "scsi";
pub fn get_file_name<P: AsRef<Path>>(src: P) -> Result<String> {
let file_name = src
.as_ref()
@@ -104,6 +105,13 @@ pub async fn handle_block_volume(
return Err(anyhow!("block driver is scsi but no scsi address exists"));
}
}
KATA_CCW_DEV_TYPE => {
if let Some(ccw_addr) = device.config.ccw_addr {
ccw_addr.to_string()
} else {
return Err(anyhow!("block driver is ccw but no ccw address exists"));
}
}
_ => device.config.virt_path,
};
device_id = device.device_id;

View File

@@ -11,6 +11,7 @@ lazy_static = { workspace = true }
netns-rs = { workspace = true }
slog = { workspace = true }
slog-scope = { workspace = true }
containerd-shim-protos = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread"] }
tracing = { workspace = true }
tracing-opentelemetry = { workspace = true }
@@ -26,7 +27,6 @@ opentelemetry-jaeger = { version = "0.17.0", features = [
] }
tracing-subscriber = { version = "0.3", features = ["registry", "std"] }
hyper = { workspace = true, features = ["stream", "server", "http1"] }
hyperlocal = { workspace = true }
serde_json = { workspace = true }
nix = "0.25.0"
url = { workspace = true }

View File

@@ -11,20 +11,14 @@ license = { workspace = true }
anyhow = { workspace = true }
async-trait = { workspace = true }
containerd-shim-protos = { workspace = true, features = ["sandbox"] }
lazy_static = { workspace = true }
nix = { workspace = true }
protobuf = { workspace = true }
serde_json = { workspace = true }
slog = { workspace = true }
slog-scope = { workspace = true }
strum = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread", "process", "fs"] }
ttrpc = { workspace = true }
oci-spec = { workspace = true }
# Local dependencies
persist = { workspace = true }
agent = { workspace = true }
kata-sys-util = { workspace = true }
kata-types = { workspace = true }

View File

@@ -6,7 +6,7 @@
use std::sync::Arc;
use anyhow::{Context, Result};
use containerd_shim_protos::events::task::{TaskExit, TaskOOM};
use containerd_shim_protos::events::task::{TaskCreate, TaskDelete, TaskExit, TaskOOM, TaskStart};
use containerd_shim_protos::protobuf::Message as ProtobufMessage;
use tokio::sync::mpsc::{channel, Receiver, Sender};
@@ -49,9 +49,15 @@ impl Message {
const TASK_OOM_EVENT_TOPIC: &str = "/tasks/oom";
const TASK_EXIT_EVENT_TOPIC: &str = "/tasks/exit";
const TASK_START_EVENT_TOPIC: &str = "/tasks/start";
const TASK_CREATE_EVENT_TOPIC: &str = "/tasks/create";
const TASK_DELETE_EVENT_TOPIC: &str = "/tasks/delete";
const TASK_OOM_EVENT_URL: &str = "containerd.events.TaskOOM";
const TASK_EXIT_EVENT_URL: &str = "containerd.events.TaskExit";
const TASK_START_EVENT_URL: &str = "containerd.events.TaskStart";
const TASK_CREATE_EVENT_URL: &str = "containerd.events.TaskCreate";
const TASK_DELETE_EVENT_URL: &str = "containerd.events.TaskDelete";
pub trait Event: std::fmt::Debug + Send {
fn r#type(&self) -> String;
@@ -86,3 +92,45 @@ impl Event for TaskExit {
self.write_to_bytes().context("get exit value")
}
}
impl Event for TaskStart {
fn r#type(&self) -> String {
TASK_START_EVENT_TOPIC.to_string()
}
fn type_url(&self) -> String {
TASK_START_EVENT_URL.to_string()
}
fn value(&self) -> Result<Vec<u8>> {
self.write_to_bytes().context("get start value")
}
}
impl Event for TaskCreate {
fn r#type(&self) -> String {
TASK_CREATE_EVENT_TOPIC.to_string()
}
fn type_url(&self) -> String {
TASK_CREATE_EVENT_URL.to_string()
}
fn value(&self) -> Result<Vec<u8>> {
self.write_to_bytes().context("get create value")
}
}
impl Event for TaskDelete {
fn r#type(&self) -> String {
TASK_DELETE_EVENT_TOPIC.to_string()
}
fn type_url(&self) -> String {
TASK_DELETE_EVENT_URL.to_string()
}
fn value(&self) -> Result<Vec<u8>> {
self.write_to_bytes().context("get delete value")
}
}

View File

@@ -6,14 +6,16 @@
use anyhow::{anyhow, Context, Result};
use common::{
message::Message,
message::{Action, Message},
types::{
ContainerProcess, PlatformInfo, SandboxConfig, SandboxRequest, SandboxResponse,
SandboxStatusInfo, StartSandboxInfo, TaskRequest, TaskResponse, DEFAULT_SHM_SIZE,
ContainerProcess, PlatformInfo, ProcessType, SandboxConfig, SandboxRequest,
SandboxResponse, SandboxStatusInfo, StartSandboxInfo, TaskRequest, TaskResponse,
DEFAULT_SHM_SIZE,
},
RuntimeHandler, RuntimeInstance, Sandbox, SandboxNetworkEnv,
};
use containerd_shim_protos::events::task::{TaskCreate, TaskDelete, TaskStart};
use hypervisor::{
utils::{create_dir_all_with_inherit_owner, create_vmm_user, remove_vmm_user},
Param,
@@ -33,13 +35,13 @@ use netns_rs::{Env, NetNs};
use nix::{sys::statfs, unistd::User};
use oci_spec::runtime as oci;
use persist::sandbox_persist::Persist;
use protobuf::Message as ProtobufMessage;
use resource::{
cpu_mem::initial_size::InitialSizeManager,
network::{dan_config_path, generate_netns_name},
};
use runtime_spec as spec;
use shim_interface::shim_mgmt::ERR_NO_SHIM_SERVER;
use protobuf::Message as ProtobufMessage;
use std::{
collections::HashMap,
env,
@@ -480,6 +482,7 @@ impl RuntimeHandlerManager {
.await
.context("start sandbox in task handler")?;
let bundle = container_config.bundle.clone();
let container_id = container_config.container_id.clone();
let shim_pid = instance
.container_manager
@@ -501,6 +504,19 @@ impl RuntimeHandlerManager {
}
});
let msg_sender = self.inner.read().await.msg_sender.clone();
let event = TaskCreate {
container_id,
bundle,
pid,
..Default::default()
};
let msg = Message::new(Action::Event(Arc::new(event)));
msg_sender
.send(msg)
.await
.context("send task create event")?;
Ok(TaskResponse::CreateContainer(shim_pid))
} else {
self.handler_task_request(req)
@@ -570,6 +586,7 @@ impl RuntimeHandlerManager {
.context("get runtime instance")?;
let sandbox = instance.sandbox.clone();
let cm = instance.container_manager.clone();
let msg_sender = self.inner.read().await.msg_sender.clone();
match req {
TaskRequest::CreateContainer(req) => Err(anyhow!("Unreachable TaskRequest {:?}", req)),
@@ -579,6 +596,20 @@ impl RuntimeHandlerManager {
}
TaskRequest::DeleteProcess(process_id) => {
let resp = cm.delete_process(&process_id).await.context("do delete")?;
if process_id.process_type == ProcessType::Container {
let event = TaskDelete {
id: process_id.container_id().to_string(),
pid: resp.pid.pid,
exit_status: resp.exit_status as u32,
..Default::default()
};
let msg = Message::new(Action::Event(Arc::new(event)));
msg_sender
.send(msg)
.await
.context("send task delete event")?;
}
Ok(TaskResponse::DeleteProcess(resp))
}
TaskRequest::ExecProcess(req) => {
@@ -614,12 +645,28 @@ impl RuntimeHandlerManager {
.context("start process")?;
let pid = shim_pid.pid;
let process_type = process_id.process_type;
let container_id = process_id.container_id().to_string();
tokio::spawn(async move {
let result = sandbox.wait_process(cm, process_id, pid).await;
if let Err(e) = result {
error!(sl!(), "sandbox wait process error: {:?}", e);
}
});
if process_type == ProcessType::Container {
let event = TaskStart {
container_id,
pid,
..Default::default()
};
let msg = Message::new(Action::Event(Arc::new(event)));
msg_sender
.send(msg)
.await
.context("send task start event")?;
}
Ok(TaskResponse::StartProcess(shim_pid))
}

View File

@@ -10,8 +10,6 @@ anyhow = { workspace = true }
async-trait = { workspace = true }
awaitgroup = "0.6.0"
containerd-shim-protos = { workspace = true }
futures = "0.3.19"
lazy_static = { workspace = true }
libc = { workspace = true }
nix = { workspace = true }
protobuf = { workspace = true }
@@ -21,9 +19,7 @@ serde_json = { workspace = true }
slog = { workspace = true }
slog-scope = { workspace = true }
tokio = { workspace = true }
toml = "0.4.2"
url = { workspace = true }
async-std = "1.12.0"
tracing = { workspace = true }
oci-spec = { workspace = true }
strum = { workspace = true }
@@ -48,3 +44,7 @@ cloud-hypervisor = ["hypervisor/cloud-hypervisor"]
# Enable the build-in VMM Dragtonball
dragonball = ["hypervisor/dragonball"]
[package.metadata.cargo-machete]
ignored = ["slog-scope"]

View File

@@ -11,7 +11,6 @@ async-trait = { workspace = true }
slog = { workspace = true }
slog-scope = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread"] }
tracing = { workspace = true }
ttrpc = { workspace = true }
containerd-shim-protos = { workspace = true, features = ["async", "sandbox"] }
containerd-shim = { workspace = true }
@@ -21,4 +20,7 @@ common = { workspace = true }
logging = { workspace = true }
kata-types = { workspace = true }
runtimes = { workspace = true }
persist = { workspace = true }
[package.metadata.cargo-machete]
ignored = ["slog-scope"]

View File

@@ -9,9 +9,8 @@ license = { workspace = true }
[dependencies]
anyhow = { workspace = true }
tokio = { workspace = true, features = [ "rt", "rt-multi-thread" ] }
tokio = { workspace = true, features = ["rt", "rt-multi-thread"] }
# Local dependencies
common = { workspace = true }
logging = { workspace = true }
runtimes = { workspace = true }

View File

@@ -36,8 +36,6 @@ slog-stdlog = "4.1.0"
thiserror = { workspace = true }
tokio = { workspace = true, features = ["rt", "rt-multi-thread"] }
unix_socket2 = "0.5.4"
tracing = { workspace = true }
tracing-opentelemetry = { workspace = true }
oci-spec = { workspace = true }
# Local dependencies
@@ -46,12 +44,7 @@ kata-sys-util = { workspace = true }
logging = { workspace = true }
runtime-spec = { workspace = true }
service = { workspace = true }
runtimes = { workspace = true }
[dev-dependencies]
tempfile = { workspace = true }
rand = { workspace = true }
serial_test = "0.10.0"
# Local dev-dependencies
tests_utils = { workspace = true }

View File

@@ -147,10 +147,14 @@ DEFROOTFSTYPE := $(ROOTFSTYPE_EXT4)
FIRMWAREPATH :=
FIRMWAREVOLUMEPATH :=
FIRMWAREPATH_NV = $(FIRMWAREPATH)
FIRMWARETDVFPATH := $(PREFIXDEPS)/share/ovmf/OVMF.inteltdx.fd
FIRMWARETDVFPATH_NV := $(FIRMWARETDVFPATH)
FIRMWARETDVFVOLUMEPATH :=
FIRMWARESNPPATH := $(PREFIXDEPS)/share/ovmf/AMDSEV.fd
FIRMWARESNPPATH_NV := $(FIRMWARESNPPATH)
KERNELVERITYPARAMS ?= ""
KERNELVERITYPARAMS_NV ?= ""
@@ -272,6 +276,7 @@ DEFVIRTIOFSEXTRAARGS ?= [\"--thread-pool-size=1\", \"--announce-submounts\"]
DEFENABLEIOTHREADS := false
DEFINDEPIOTHREADS := 0
DEFENABLEVHOSTUSERSTORE := false
DEFENABLEVIRTIOMEM ?= false
DEFVHOSTUSERSTOREPATH := $(PKGRUNDIR)/vhost-user
DEFVALIDVHOSTUSERSTOREPATHS := [\"$(DEFVHOSTUSERSTOREPATH)\"]
DEFFILEMEMBACKEND := ""
@@ -288,6 +293,7 @@ DEFSTATICRESOURCEMGMT_NV = true
DEFDISABLEIMAGENVDIMM ?= false
DEFDISABLEIMAGENVDIMM_NV = true
DEFDISABLEIMAGENVDIMM_CLH ?= true
DEFBINDMOUNTS := []
@@ -299,9 +305,11 @@ DEFDANCONF := /run/kata-containers/dans
DEFFORCEGUESTPULL := false
DEFKUBELETROOTDIR := /var/lib/kubelet
# Device cold plug
DEFPODRESOURCEAPISOCK := ""
DEFPODRESOURCEAPISOCK_NV := "/var/lib/kubelet/pod-resources/kubelet.sock"
DEFPODRESOURCEAPISOCK_NV := "$(DEFKUBELETROOTDIR)/pod-resources/kubelet.sock"
SED = sed
@@ -466,8 +474,8 @@ ifneq (,$(QEMUCMD))
KERNELSEPATH = $(KERNELDIR)/$(KERNELSENAME)
# NVIDIA GPU specific options (all should be suffixed by _NV)
# Normal: uncompressed (KERNELTYPE). Confidential: compressed (KERNELCONFIDENTIALTYPE).
KERNELNAME_NV = $(call MAKE_KERNEL_NAME_NV,$(KERNELTYPE))
KERNELTYPE_NV = compressed
KERNELNAME_NV = $(call MAKE_KERNEL_NAME_NV,$(KERNELTYPE_NV))
KERNELPATH_NV = $(KERNELDIR)/$(KERNELNAME_NV)
KERNELNAME_CONFIDENTIAL_NV = $(call MAKE_KERNEL_NAME_NV,$(KERNELCONFIDENTIALTYPE))
KERNELPATH_CONFIDENTIAL_NV = $(KERNELDIR)/$(KERNELNAME_CONFIDENTIAL_NV)
@@ -483,6 +491,9 @@ ifneq (,$(QEMUCMD))
# using an image and /dev is already mounted.
KERNELPARAMS_NV = "cgroup_no_v1=all"
KERNELPARAMS_NV += "devtmpfs.mount=0"
KERNELPARAMS_NV += "pci=realloc"
KERNELPARAMS_NV += "pci=nocrs"
KERNELPARAMS_NV += "pci=assign-busses"
# Setting this to false can lead to cgroup leakages in the host
# Best practice for production is to set this to true
@@ -679,10 +690,13 @@ USER_VARS += KERNELPATH_FC
USER_VARS += KERNELPATH_STRATOVIRT
USER_VARS += KERNELVIRTIOFSPATH
USER_VARS += FIRMWAREPATH
USER_VARS += FIRMWAREPATH_NV
USER_VARS += FIRMWARETDVFPATH
USER_VARS += FIRMWAREVOLUMEPATH
USER_VARS += FIRMWARETDVFVOLUMEPATH
USER_VARS += FIRMWARESNPPATH
USER_VARS += FIRMWARETDVFPATH_NV
USER_VARS += FIRMWARESNPPATH_NV
USER_VARS += MACHINEACCELERATORS
USER_VARS += CPUFEATURES
USER_VARS += TDXCPUFEATURES
@@ -763,6 +777,7 @@ USER_VARS += DEFENABLEANNOTATIONS
USER_VARS += DEFENABLEANNOTATIONS_COCO
USER_VARS += DEFENABLEIOTHREADS
USER_VARS += DEFINDEPIOTHREADS
USER_VARS += DEFENABLEVIRTIOMEM
USER_VARS += DEFSECCOMPSANDBOXPARAM
USER_VARS += DEFENABLEVHOSTUSERSTORE
USER_VARS += DEFVHOSTUSERSTOREPATH
@@ -782,12 +797,14 @@ USER_VARS += DEFSTATICRESOURCEMGMT_NV
USER_VARS += DEFBINDMOUNTS
USER_VARS += DEFCREATECONTAINERTIMEOUT
USER_VARS += DEFDANCONF
USER_VARS += DEFKUBELETROOTDIR
USER_VARS += DEFFORCEGUESTPULL
USER_VARS += DEFVFIOMODE
USER_VARS += DEFVFIOMODE_SE
USER_VARS += BUILDFLAGS
USER_VARS += DEFDISABLEIMAGENVDIMM
USER_VARS += DEFDISABLEIMAGENVDIMM_NV
USER_VARS += DEFDISABLEIMAGENVDIMM_CLH
USER_VARS += DEFCCAMEASUREMENTALGO
USER_VARS += DEFSHAREDFS_QEMU_CCA_VIRTIOFS
USER_VARS += DEFPODRESOURCEAPISOCK

View File

@@ -18,3 +18,6 @@ ifneq (,$(NEEDS_CC_SETTING))
CC := gcc
export CC
endif
# Enable virtio-mem for s390x
DEFENABLEVIRTIOMEM = true

View File

@@ -196,7 +196,7 @@ func indexPageText(w http.ResponseWriter, r *http.Request) {
formatter := fmt.Sprintf("%%-%ds: %%s\n", spacing)
for _, endpoint := range endpoints {
w.Write([]byte(fmt.Sprintf(formatter, endpoint.path, endpoint.desc)))
fmt.Fprintf(w, formatter, endpoint.path, endpoint.desc)
}
}

View File

@@ -63,7 +63,7 @@ func setCPUtype(hypervisorType vc.HypervisorType) error {
cpuType = getCPUtype()
if cpuType == cpuTypeUnknown {
return fmt.Errorf("Unknow CPU Type")
return fmt.Errorf("Unknown CPU Type")
} else if cpuType == cpuTypeIntel {
var kvmIntelParams map[string]string
onVMM, err := vc.RunningOnVMM(procCPUInfo)

View File

@@ -55,18 +55,17 @@ func TestCCCheckCLIFunction(t *testing.T) {
var moduleData []testModuleData
cpuType = getCPUtype()
if cpuType == cpuTypeIntel {
moduleData = []testModuleData{}
switch cpuType {
case cpuTypeIntel:
cpuData = []testCPUData{
{archGenuineIntel, "lm vmx sse4_1", false},
}
moduleData = []testModuleData{}
} else if cpuType == cpuTypeAMD {
case cpuTypeAMD:
cpuData = []testCPUData{
{archAuthenticAMD, "lm svm sse4_1", false},
}
moduleData = []testModuleData{}
}
genericCheckCLIFunction(t, cpuData, moduleData)
@@ -276,7 +275,8 @@ func TestCheckHostIsVMContainerCapable(t *testing.T) {
var moduleData []testModuleData
cpuType = getCPUtype()
if cpuType == cpuTypeIntel {
switch cpuType {
case cpuTypeIntel:
cpuData = []testCPUData{
{"", "", true},
{"Intel", "", true},
@@ -292,7 +292,7 @@ func TestCheckHostIsVMContainerCapable(t *testing.T) {
{filepath.Join(sysModuleDir, "kvm_intel/parameters/nested"), "Y", false},
{filepath.Join(sysModuleDir, "kvm_intel/parameters/unrestricted_guest"), "Y", false},
}
} else if cpuType == cpuTypeAMD {
case cpuTypeAMD:
cpuData = []testCPUData{
{"", "", true},
{"AMD", "", true},
@@ -340,7 +340,7 @@ func TestCheckHostIsVMContainerCapable(t *testing.T) {
// Write the following into the denylist file
// blacklist <mod>
// install <mod> /bin/false
_, err = denylistFile.WriteString(fmt.Sprintf("blacklist %s\ninstall %s /bin/false\n", mod, mod))
_, err = fmt.Fprintf(denylistFile, "blacklist %s\ninstall %s /bin/false\n", mod, mod)
assert.Nil(err)
}
denylistFile.Close()
@@ -348,6 +348,15 @@ func TestCheckHostIsVMContainerCapable(t *testing.T) {
defer func() {
os.Remove(denylistModuleConf)
// reload removed modules
for mod := range archRequiredKernelModules {
cmd := exec.Command(modProbeCmd, mod)
if output, err := cmd.CombinedOutput(); err == nil {
kataLog.WithField("output", string(output)).Info("module loaded")
} else {
kataLog.WithField("output", string(output)).Warn("failed to load module")
}
}
}()
// remove the modules to force a failure
@@ -496,9 +505,10 @@ func TestSetCPUtype(t *testing.T) {
assert.NotEmpty(archRequiredKernelModules)
cpuType = getCPUtype()
if cpuType == cpuTypeIntel {
switch cpuType {
case cpuTypeIntel:
assert.Equal(archRequiredCPUFlags["vmx"], "Virtualization support")
} else if cpuType == cpuTypeAMD {
case cpuTypeAMD:
assert.Equal(archRequiredCPUFlags["svm"], "Virtualization support")
}

View File

@@ -17,7 +17,6 @@ import (
"testing"
"github.com/kata-containers/kata-containers/src/runtime/pkg/katatestutils"
ktu "github.com/kata-containers/kata-containers/src/runtime/pkg/katatestutils"
"github.com/kata-containers/kata-containers/src/runtime/pkg/katautils"
vc "github.com/kata-containers/kata-containers/src/runtime/virtcontainers"
"github.com/sirupsen/logrus"
@@ -509,7 +508,7 @@ func TestCheckCheckCPUAttribs(t *testing.T) {
}
func TestCheckHaveKernelModule(t *testing.T) {
if tc.NotValid(ktu.NeedRoot()) {
if tc.NotValid(katatestutils.NeedRoot()) {
t.Skip(testDisabledAsNonRoot)
}
@@ -638,8 +637,8 @@ func TestCheckCheckKernelModules(t *testing.T) {
func TestCheckCheckKernelModulesUnreadableFile(t *testing.T) {
assert := assert.New(t)
if tc.NotValid(ktu.NeedNonRoot()) {
t.Skip(ktu.TestDisabledNeedNonRoot)
if tc.NotValid(katatestutils.NeedNonRoot()) {
t.Skip(katatestutils.TestDisabledNeedNonRoot)
}
dir := t.TempDir()

View File

@@ -56,9 +56,10 @@ func TestEnvGetEnvInfoSetsCPUType(t *testing.T) {
assert.NotEmpty(archRequiredKernelModules)
cpuType = getCPUtype()
if cpuType == cpuTypeIntel {
switch cpuType {
case cpuTypeIntel:
assert.Equal(archRequiredCPUFlags["vmx"], "Virtualization support")
} else if cpuType == cpuTypeAMD {
case cpuTypeAMD:
assert.Equal(archRequiredCPUFlags["svm"], "Virtualization support")
}

View File

@@ -14,7 +14,6 @@ import (
"path"
"path/filepath"
"runtime"
goruntime "runtime"
"strings"
"testing"
@@ -184,7 +183,7 @@ func genericGetExpectedHostDetails(tmpdir string, expectedVendor string, expecte
}
const expectedKernelVersion = "99.1"
const expectedArch = goruntime.GOARCH
const expectedArch = runtime.GOARCH
expectedDistro := DistroInfo{
Name: "Foo",
@@ -254,7 +253,7 @@ VERSION_ID="%s"
}
}
if goruntime.GOARCH == "arm64" {
if runtime.GOARCH == "arm64" {
expectedHostDetails.CPU.Vendor = "ARM Limited"
expectedHostDetails.CPU.Model = "v8"
}

View File

@@ -55,9 +55,9 @@ var getIPTablesCommand = cli.Command{
return err
}
url := containerdshim.IPTablesUrl
url := containerdshim.IPTablesURL
if isIPv6 {
url = containerdshim.IP6TablesUrl
url = containerdshim.IP6TablesURL
}
body, err := shimclient.DoGet(sandboxID, defaultTimeout, url)
if err != nil {
@@ -108,9 +108,9 @@ var setIPTablesCommand = cli.Command{
return err
}
url := containerdshim.IPTablesUrl
url := containerdshim.IPTablesURL
if isIPv6 {
url = containerdshim.IP6TablesUrl
url = containerdshim.IP6TablesURL
}
if err = shimclient.DoPut(sandboxID, defaultTimeout, url, "application/octet-stream", buf); err != nil {

View File

@@ -62,7 +62,7 @@ var setPolicyCommand = cli.Command{
return err
}
url := containerdshim.PolicyUrl
url := containerdshim.PolicyURL
if err = shimclient.DoPut(sandboxID, defaultTimeout, url, "application/octet-stream", buf); err != nil {
return fmt.Errorf("Error observed when making policy-set request(%s): %s", policyFile, err)

View File

@@ -126,7 +126,7 @@ var resizeCommand = cli.Command{
// Stats retrieves the filesystem stats of the direct volume inside the guest.
func Stats(volumePath string) ([]byte, error) {
sandboxId, err := volume.GetSandboxIdForVolume(volumePath)
sandboxID, err := volume.GetSandboxIDForVolume(volumePath)
if err != nil {
return nil, err
}
@@ -136,8 +136,8 @@ func Stats(volumePath string) ([]byte, error) {
}
urlSafeDevicePath := url.PathEscape(volumeMountInfo.Device)
body, err := shimclient.DoGet(sandboxId, defaultTimeout,
fmt.Sprintf("%s?%s=%s", containerdshim.DirectVolumeStatUrl, containerdshim.DirectVolumePathKey, urlSafeDevicePath))
body, err := shimclient.DoGet(sandboxID, defaultTimeout,
fmt.Sprintf("%s?%s=%s", containerdshim.DirectVolumeStatURL, containerdshim.DirectVolumePathKey, urlSafeDevicePath))
if err != nil {
return nil, err
}
@@ -146,7 +146,7 @@ func Stats(volumePath string) ([]byte, error) {
// Resize resizes a direct volume inside the guest.
func Resize(volumePath string, size uint64) error {
sandboxId, err := volume.GetSandboxIdForVolume(volumePath)
sandboxID, err := volume.GetSandboxIDForVolume(volumePath)
if err != nil {
return err
}
@@ -163,5 +163,5 @@ func Resize(volumePath string, size uint64) error {
if err != nil {
return err
}
return shimclient.DoPost(sandboxId, defaultTimeout, containerdshim.DirectVolumeResizeUrl, "application/json", encoded)
return shimclient.DoPost(sandboxID, defaultTimeout, containerdshim.DirectVolumeResizeURL, "application/json", encoded)
}

View File

@@ -94,11 +94,12 @@ func releaseURLIsValid(url string) error {
func getReleaseURL(currentVersion semver.Version) (url string, err error) {
major := currentVersion.Major
if major == 0 {
switch major {
case 0:
return "", fmt.Errorf("invalid current version: %v", currentVersion)
} else if major == 1 {
case 1:
url = kataLegacyReleaseURL
} else {
default:
url = kataReleaseURL
}

View File

@@ -222,8 +222,8 @@ hypervisor_loglevel = 1
# If false and nvdimm is supported, use nvdimm device to plug guest image.
# Otherwise virtio-block device is used.
#
# nvdimm is not supported when `confidential_guest = true`.
disable_image_nvdimm = @DEFDISABLEIMAGENVDIMM@
# nvdimm is not supported with Cloud Hypervisor or when `confidential_guest = true`.
disable_image_nvdimm = @DEFDISABLEIMAGENVDIMM_CLH@
# Enable hot-plugging of VFIO devices to a root-port.
# The default setting is "no-port"
@@ -491,6 +491,11 @@ create_container_timeout = @DEFCREATECONTAINERTIMEOUT@
# (default: /run/kata-containers/dans)
dan_conf = "@DEFDANCONF@"
# kubelet_root_dir is the kubelet root directory used to match ConfigMap/Secret
# volume paths for propagation. Override for distros that use a different path
# (e.g. k0s: /var/lib/k0s/kubelet).
kubelet_root_dir = "@DEFKUBELETROOTDIR@"
# pod_resource_api_sock specifies the unix socket for the Kubelet's
# PodResource API endpoint. If empty, kubernetes based cold plug
# will not be attempted. In order for this feature to work, the

View File

@@ -382,6 +382,11 @@ create_container_timeout = @DEFCREATECONTAINERTIMEOUT@
# (default: /run/kata-containers/dans)
dan_conf = "@DEFDANCONF@"
# kubelet_root_dir is the kubelet root directory used to match ConfigMap/Secret
# volume paths for propagation. Override for distros that use a different path
# (e.g. k0s: /var/lib/k0s/kubelet).
kubelet_root_dir = "@DEFKUBELETROOTDIR@"
# pod_resource_api_sock specifies the unix socket for the Kubelet's
# PodResource API endpoint. If empty, kubernetes based cold plug
# will not be attempted. In order for this feature to work, the

View File

@@ -670,6 +670,12 @@ dan_conf = "@DEFDANCONF@"
# the container image should be pulled in the guest, without using an external snapshotter.
# This is an experimental feature and might be removed in the future.
experimental_force_guest_pull = @DEFFORCEGUESTPULL@
# kubelet_root_dir is the kubelet root directory used to match ConfigMap/Secret
# volume paths for propagation. Override for distros that use a different path
# (e.g. k0s: /var/lib/k0s/kubelet).
kubelet_root_dir = "@DEFKUBELETROOTDIR@"
# pod_resource_api_sock specifies the unix socket for the Kubelet's
# PodResource API endpoint. If empty, kubernetes based cold plug
# will not be attempted. In order for this feature to work, the

View File

@@ -734,6 +734,11 @@ dan_conf = "@DEFDANCONF@"
# This is an experimental feature and might be removed in the future.
experimental_force_guest_pull = @DEFFORCEGUESTPULL@
# kubelet_root_dir is the kubelet root directory used to match ConfigMap/Secret
# volume paths for propagation. Override for distros that use a different path
# (e.g. k0s: /var/lib/k0s/kubelet).
kubelet_root_dir = "@DEFKUBELETROOTDIR@"
# pod_resource_api_sock specifies the unix socket for the Kubelet's
# PodResource API endpoint. If empty, kubernetes based cold plug
# will not be attempted. In order for this feature to work, the

View File

@@ -99,7 +99,7 @@ kernel_verity_params = "@KERNELVERITYPARAMS_CONFIDENTIAL_NV@"
# Path to the firmware.
# If you want that qemu uses the default firmware leave this option empty
firmware = "@FIRMWARESNPPATH@"
firmware = "@FIRMWARESNPPATH_NV@"
# Path to the firmware volume.
# firmware TDVF or OVMF can be split into FIRMWARE_VARS.fd (UEFI variables
@@ -750,6 +750,11 @@ dan_conf = "@DEFDANCONF@"
# This is an experimental feature and might be removed in the future.
experimental_force_guest_pull = @DEFFORCEGUESTPULL@
# kubelet_root_dir is the kubelet root directory used to match ConfigMap/Secret
# volume paths for propagation. Override for distros that use a different path
# (e.g. k0s: /var/lib/k0s/kubelet).
kubelet_root_dir = "@DEFKUBELETROOTDIR@"
# pod_resource_api_sock specifies the unix socket for the Kubelet's
# PodResource API endpoint. If empty, kubernetes based cold plug
# will not be attempted. In order for this feature to work, the

View File

@@ -76,7 +76,7 @@ kernel_verity_params = "@KERNELVERITYPARAMS_CONFIDENTIAL_NV@"
# Path to the firmware.
# If you want that qemu uses the default firmware leave this option empty
firmware = "@FIRMWARETDVFPATH@"
firmware = "@FIRMWARETDVFPATH_NV@"
# Path to the firmware volume.
# firmware TDVF or OVMF can be split into FIRMWARE_VARS.fd (UEFI variables
@@ -727,6 +727,11 @@ dan_conf = "@DEFDANCONF@"
# This is an experimental feature and might be removed in the future.
experimental_force_guest_pull = @DEFFORCEGUESTPULL@
# kubelet_root_dir is the kubelet root directory used to match ConfigMap/Secret
# volume paths for propagation. Override for distros that use a different path
# (e.g. k0s: /var/lib/k0s/kubelet).
kubelet_root_dir = "@DEFKUBELETROOTDIR@"
# pod_resource_api_sock specifies the unix socket for the Kubelet's
# PodResource API endpoint. If empty, kubernetes based cold plug
# will not be attempted. In order for this feature to work, the

View File

@@ -58,7 +58,7 @@ kernel_verity_params = "@KERNELVERITYPARAMS_NV@"
# Path to the firmware.
# If you want that qemu uses the default firmware leave this option empty
firmware = "@FIRMWAREPATH@"
firmware = "@FIRMWAREPATH_NV@"
# Path to the firmware volume.
# firmware TDVF or OVMF can be split into FIRMWARE_VARS.fd (UEFI variables
@@ -724,6 +724,11 @@ create_container_timeout = @DEFAULTTIMEOUT_NV@
# (default: /run/kata-containers/dans)
dan_conf = "@DEFDANCONF@"
# kubelet_root_dir is the kubelet root directory used to match ConfigMap/Secret
# volume paths for propagation. Override for distros that use a different path
# (e.g. k0s: /var/lib/k0s/kubelet).
kubelet_root_dir = "@DEFKUBELETROOTDIR@"
# pod_resource_api_sock specifies the unix socket for the Kubelet's
# PodResource API endpoint. If empty, kubernetes based cold plug
# will not be attempted. In order for this feature to work, the

View File

@@ -712,6 +712,11 @@ dan_conf = "@DEFDANCONF@"
# This is an experimental feature and might be removed in the future.
experimental_force_guest_pull = @DEFFORCEGUESTPULL@
# kubelet_root_dir is the kubelet root directory used to match ConfigMap/Secret
# volume paths for propagation. Override for distros that use a different path
# (e.g. k0s: /var/lib/k0s/kubelet).
kubelet_root_dir = "@DEFKUBELETROOTDIR@"
# pod_resource_api_sock specifies the unix socket for the Kubelet's
# PodResource API endpoint. If empty, kubernetes based cold plug
# will not be attempted. In order for this feature to work, the

View File

@@ -737,6 +737,11 @@ dan_conf = "@DEFDANCONF@"
# This is an experimental feature and might be removed in the future.
experimental_force_guest_pull = @DEFFORCEGUESTPULL@
# kubelet_root_dir is the kubelet root directory used to match ConfigMap/Secret
# volume paths for propagation. Override for distros that use a different path
# (e.g. k0s: /var/lib/k0s/kubelet).
kubelet_root_dir = "@DEFKUBELETROOTDIR@"
# pod_resource_api_sock specifies the unix socket for the Kubelet's
# PodResource API endpoint. If empty, kubernetes based cold plug
# will not be attempted. In order for this feature to work, the

View File

@@ -719,6 +719,11 @@ dan_conf = "@DEFDANCONF@"
# This is an experimental feature and might be removed in the future.
experimental_force_guest_pull = @DEFFORCEGUESTPULL@
# kubelet_root_dir is the kubelet root directory used to match ConfigMap/Secret
# volume paths for propagation. Override for distros that use a different path
# (e.g. k0s: /var/lib/k0s/kubelet).
kubelet_root_dir = "@DEFKUBELETROOTDIR@"
# pod_resource_api_sock specifies the unix socket for the Kubelet's
# PodResource API endpoint. If empty, kubernetes based cold plug
# will not be attempted. In order for this feature to work, the

View File

@@ -142,7 +142,7 @@ memory_offset = 0
# Please note that this option should be used with the command
# "echo 1 > /proc/sys/vm/overcommit_memory".
# Default false
enable_virtio_mem = false
enable_virtio_mem = @DEFENABLEVIRTIOMEM@
# Disable hotplugging host block devices to guest VMs for container rootfs.
# In case of a storage driver like devicemapper where a container's
@@ -723,6 +723,11 @@ create_container_timeout = @DEFCREATECONTAINERTIMEOUT@
# (default: /run/kata-containers/dans)
dan_conf = "@DEFDANCONF@"
# kubelet_root_dir is the kubelet root directory used to match ConfigMap/Secret
# volume paths for propagation. Override for distros that use a different path
# (e.g. k0s: /var/lib/k0s/kubelet).
kubelet_root_dir = "@DEFKUBELETROOTDIR@"
# pod_resource_api_sock specifies the unix socket for the Kubelet's
# PodResource API endpoint. If empty, kubernetes based cold plug
# will not be attempted. In order for this feature to work, the

View File

@@ -290,6 +290,11 @@ create_container_timeout = @DEFCREATECONTAINERTIMEOUT@
# (default: /run/kata-containers/dans)
dan_conf = "@DEFDANCONF@"
# kubelet_root_dir is the kubelet root directory used to match ConfigMap/Secret
# volume paths for propagation. Override for distros that use a different path
# (e.g. k0s: /var/lib/k0s/kubelet).
kubelet_root_dir = "@DEFKUBELETROOTDIR@"
# pod_resource_api_sock specifies the unix socket for the Kubelet's
# PodResource API endpoint. If empty, kubernetes based cold plug
# will not be attempted. In order for this feature to work, the

View File

@@ -425,6 +425,11 @@ create_container_timeout = @DEFCREATECONTAINERTIMEOUT@
# (default: /run/kata-containers/dans)
dan_conf = "@DEFDANCONF@"
# kubelet_root_dir is the kubelet root directory used to match ConfigMap/Secret
# volume paths for propagation. Override for distros that use a different path
# (e.g. k0s: /var/lib/k0s/kubelet).
kubelet_root_dir = "@DEFKUBELETROOTDIR@"
# pod_resource_api_sock specifies the unix socket for the Kubelet's
# PodResource API endpoint. If empty, kubernetes based cold plug
# will not be attempted. In order for this feature to work, the

View File

@@ -1,14 +1,14 @@
module github.com/kata-containers/kata-containers/src/runtime
// Keep in sync with version in versions.yaml
go 1.24.13
go 1.25.7
// WARNING: Do NOT use `replace` directives as those break dependabot:
// https://github.com/kata-containers/kata-containers/issues/11020
require (
code.cloudfoundry.org/bytefmt v0.0.0-20211005130812-5bb3c17173e5
github.com/BurntSushi/toml v1.5.0
github.com/BurntSushi/toml v1.6.0
github.com/blang/semver v3.5.1+incompatible
github.com/blang/semver/v4 v4.0.0
github.com/container-orchestrated-devices/container-device-interface v0.6.0
@@ -52,11 +52,10 @@ require (
github.com/urfave/cli v1.22.17
github.com/vishvananda/netlink v1.3.1
github.com/vishvananda/netns v0.0.5
gitlab.com/nvidia/cloud-native/go-nvlib v0.0.0-20220601114329-47893b162965
go.opentelemetry.io/otel v1.35.0
go.opentelemetry.io/otel v1.40.0
go.opentelemetry.io/otel/exporters/jaeger v1.0.0
go.opentelemetry.io/otel/sdk v1.35.0
go.opentelemetry.io/otel/trace v1.35.0
go.opentelemetry.io/otel/sdk v1.40.0
go.opentelemetry.io/otel/trace v1.40.0
golang.org/x/oauth2 v0.30.0
golang.org/x/sys v0.40.0
google.golang.org/grpc v1.72.0
@@ -127,9 +126,9 @@ require (
github.com/x448/float16 v0.8.4 // indirect
go.mongodb.org/mongo-driver v1.14.0 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.59.0 // indirect
go.opentelemetry.io/otel/metric v1.35.0 // indirect
go.opentelemetry.io/otel/metric v1.40.0 // indirect
golang.org/x/exp v0.0.0-20241108190413-2d47ceb2692f // indirect
golang.org/x/mod v0.31.0 // indirect
golang.org/x/net v0.49.0 // indirect

View File

@@ -8,8 +8,9 @@ github.com/AdaLogics/go-fuzz-headers v0.0.0-20230811130428-ced1acdcaa24/go.mod h
github.com/AdamKorcz/go-118-fuzz-build v0.0.0-20230306123547-8075edf89bb0 h1:59MxjQVfjXsBpLy+dbd2/ELV5ofnUkUZBvWSC85sheA=
github.com/AdamKorcz/go-118-fuzz-build v0.0.0-20230306123547-8075edf89bb0/go.mod h1:OahwfttHWG6eJ0clwcfBAHoDI6X/LV/15hx/wlMZSrU=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/toml v1.5.0 h1:W5quZX/G/csjUnuI8SUYlsHs9M38FC7znL0lIO+DvMg=
github.com/BurntSushi/toml v1.5.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/BurntSushi/toml v1.6.0 h1:dRaEfpa2VI55EwlIW72hMRHdWouJeRF7TPYhI+AUQjk=
github.com/BurntSushi/toml v1.6.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/Masterminds/semver/v3 v3.4.0 h1:Zog+i5UMtVoCU8oKka5P7i9q9HgrJeGzI9SA1Xbatp0=
github.com/Masterminds/semver/v3 v3.4.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
@@ -265,8 +266,8 @@ github.com/prometheus/common v0.62.0 h1:xasJaQlnWAeyHdUBeGjXmutelfJHWMRr+Fg4QszZ
github.com/prometheus/common v0.62.0/go.mod h1:vyBcEuLSvWos9B1+CyL7JZ2up+uFzXhkqml0W5zIY1I=
github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc=
github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII=
github.com/rogpeppe/go-internal v1.13.1/go.mod h1:uMEvuHeurkdAXX61udpOXGD/AzZDWNMNyH2VO9fmH0o=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/safchain/ethtool v0.6.2 h1:O3ZPFAKEUEfbtE6J/feEe2Ft7dIJ2Sy8t4SdMRiIMHY=
@@ -308,31 +309,29 @@ github.com/xeipuuv/gojsonschema v1.2.0 h1:LhYJRs+L4fBtjZUfuSZIKGeVu0QRy8e5Xi7D17
github.com/xeipuuv/gojsonschema v1.2.0/go.mod h1:anYRn/JVcOK2ZgGU+IjEV4nwlhoK5sQluxsYJ78Id3Y=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
gitlab.com/nvidia/cloud-native/go-nvlib v0.0.0-20220601114329-47893b162965 h1:EXE1ZsUqiUWGV5Dw2oTYpXx24ffxj0//yhTB0Ppv+4s=
gitlab.com/nvidia/cloud-native/go-nvlib v0.0.0-20220601114329-47893b162965/go.mod h1:TBB3sR7/jg4RCThC/cgT4fB8mAbbMO307TycfgeR59w=
go.mongodb.org/mongo-driver v1.14.0 h1:P98w8egYRjYe3XDjxhYJagTokP/H6HzlsnojRgZRd80=
go.mongodb.org/mongo-driver v1.14.0/go.mod h1:Vzb0Mk/pa7e6cWw85R4F/endUC3u0U9jGcNU603k65c=
go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=
go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.59.0 h1:CV7UdSGJt/Ao6Gp4CXckLxVRRsRgDHoI8XjbL3PDl8s=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.59.0/go.mod h1:FRmFuRJfag1IZ2dPkHnEoSFVgTVPUd2qf5Vi69hLb8I=
go.opentelemetry.io/otel v1.0.0/go.mod h1:AjRVh9A5/5DE7S+mZtTR6t8vpKKryam+0lREnfmS4cg=
go.opentelemetry.io/otel v1.35.0 h1:xKWKPxrxB6OtMCbmMY021CqC45J+3Onta9MqjhnusiQ=
go.opentelemetry.io/otel v1.35.0/go.mod h1:UEqy8Zp11hpkUrL73gSlELM0DupHoiq72dR+Zqel/+Y=
go.opentelemetry.io/otel v1.40.0 h1:oA5YeOcpRTXq6NN7frwmwFR0Cn3RhTVZvXsP4duvCms=
go.opentelemetry.io/otel v1.40.0/go.mod h1:IMb+uXZUKkMXdPddhwAHm6UfOwJyh4ct1ybIlV14J0g=
go.opentelemetry.io/otel/exporters/jaeger v1.0.0 h1:cLhx8llHw02h5JTqGqaRbYn+QVKHmrzD9vEbKnSPk5U=
go.opentelemetry.io/otel/exporters/jaeger v1.0.0/go.mod h1:q10N1AolE1JjqKrFJK2tYw0iZpmX+HBaXBtuCzRnBGQ=
go.opentelemetry.io/otel/metric v1.35.0 h1:0znxYu2SNyuMSQT4Y9WDWej0VpcsxkuklLa4/siN90M=
go.opentelemetry.io/otel/metric v1.35.0/go.mod h1:nKVFgxBZ2fReX6IlyW28MgZojkoAkJGaE8CpgeAU3oE=
go.opentelemetry.io/otel/metric v1.40.0 h1:rcZe317KPftE2rstWIBitCdVp89A2HqjkxR3c11+p9g=
go.opentelemetry.io/otel/metric v1.40.0/go.mod h1:ib/crwQH7N3r5kfiBZQbwrTge743UDc7DTFVZrrXnqc=
go.opentelemetry.io/otel/sdk v1.0.0/go.mod h1:PCrDHlSy5x1kjezSdL37PhbFUMjrsLRshJ2zCzeXwbM=
go.opentelemetry.io/otel/sdk v1.35.0 h1:iPctf8iprVySXSKJffSS79eOjl9pvxV9ZqOWT0QejKY=
go.opentelemetry.io/otel/sdk v1.35.0/go.mod h1:+ga1bZliga3DxJ3CQGg3updiaAJoNECOgJREo9KHGQg=
go.opentelemetry.io/otel/sdk/metric v1.34.0 h1:5CeK9ujjbFVL5c1PhLuStg1wxA7vQv7ce1EK0Gyvahk=
go.opentelemetry.io/otel/sdk/metric v1.34.0/go.mod h1:jQ/r8Ze28zRKoNRdkjCZxfs6YvBTG1+YIqyFVFYec5w=
go.opentelemetry.io/otel/sdk v1.40.0 h1:KHW/jUzgo6wsPh9At46+h4upjtccTmuZCFAc9OJ71f8=
go.opentelemetry.io/otel/sdk v1.40.0/go.mod h1:Ph7EFdYvxq72Y8Li9q8KebuYUr2KoeyHx0DRMKrYBUE=
go.opentelemetry.io/otel/sdk/metric v1.40.0 h1:mtmdVqgQkeRxHgRv4qhyJduP3fYJRMX4AtAlbuWdCYw=
go.opentelemetry.io/otel/sdk/metric v1.40.0/go.mod h1:4Z2bGMf0KSK3uRjlczMOeMhKU2rhUqdWNoKcYrtcBPg=
go.opentelemetry.io/otel/trace v1.0.0/go.mod h1:PXTWqayeFUlJV1YDNhsJYB184+IvAH814St6o6ajzIs=
go.opentelemetry.io/otel/trace v1.35.0 h1:dPpEfJu1sDIqruz7BHFG3c7528f6ddfSWfFDVt/xgMs=
go.opentelemetry.io/otel/trace v1.35.0/go.mod h1:WUk7DtFp1Aw2MkvqGdwiXYDZZNvA/1J8o6xRXLrIkyc=
go.opentelemetry.io/otel/trace v1.40.0 h1:WA4etStDttCSYuhwvEa8OP8I5EWu24lkOzp+ZYblVjw=
go.opentelemetry.io/otel/trace v1.40.0/go.mod h1:zeAhriXecNGP/s2SEG3+Y8X9ujcJOTqQ5RgdEJcawiA=
go.uber.org/automaxprocs v1.6.0 h1:O3y2/QNTOdbF+e/dpXNNW7Rx2hZ4sTIPyybbxyNqTUs=
go.uber.org/automaxprocs v1.6.0/go.mod h1:ifeIMSnPZuznNm6jmdzmU3/bfk01Fe2fotchwEFJ8r8=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=

View File

@@ -40,7 +40,6 @@ import (
"github.com/kata-containers/kata-containers/src/runtime/pkg/katautils"
"github.com/kata-containers/kata-containers/src/runtime/pkg/katautils/katatrace"
"github.com/kata-containers/kata-containers/src/runtime/pkg/oci"
vc "github.com/kata-containers/kata-containers/src/runtime/virtcontainers"
"github.com/kata-containers/kata-containers/src/runtime/virtcontainers/pkg/compatoci"
"tags.cncf.io/container-device-interface/pkg/cdi"
)
@@ -52,7 +51,7 @@ var defaultStartManagementServerFunc startManagementServerFunc = func(s *service
shimLog.Info("management server started")
}
func copyLayersToMounts(rootFs *vc.RootFs, spec *specs.Spec) error {
func copyLayersToMounts(rootFs *virtcontainers.RootFs, spec *specs.Spec) error {
for _, o := range rootFs.Options {
if !strings.HasPrefix(o, annotations.FileSystemLayer) {
continue
@@ -75,7 +74,7 @@ func copyLayersToMounts(rootFs *vc.RootFs, spec *specs.Spec) error {
}
func create(ctx context.Context, s *service, r *taskAPI.CreateTaskRequest) (*container, error) {
rootFs := vc.RootFs{}
rootFs := virtcontainers.RootFs{}
if len(r.Rootfs) == 1 {
m := r.Rootfs[0]
rootFs.Source = m.Source
@@ -108,7 +107,7 @@ func create(ctx context.Context, s *service, r *taskAPI.CreateTaskRequest) (*con
}
switch containerType {
case vc.PodSandbox, vc.SingleContainer:
case virtcontainers.PodSandbox, virtcontainers.SingleContainer:
if s.sandbox != nil {
return nil, fmt.Errorf("cannot create another sandbox in sandbox: %s", s.sandbox.ID())
}
@@ -151,7 +150,7 @@ func create(ctx context.Context, s *service, r *taskAPI.CreateTaskRequest) (*con
// 2. If this is not a sandbox infrastructure container, but instead a standalone single container (analogous to "docker run..."),
// then the container spec itself will contain appropriate sizing information for the entire sandbox (since it is
// a single container.
if containerType == vc.PodSandbox {
if containerType == virtcontainers.PodSandbox {
s.config.SandboxCPUs, s.config.SandboxMemMB = oci.CalculateSandboxSizing(ociSpec)
} else {
s.config.SandboxCPUs, s.config.SandboxMemMB = oci.CalculateContainerSizing(ociSpec)
@@ -203,7 +202,7 @@ func create(ctx context.Context, s *service, r *taskAPI.CreateTaskRequest) (*con
defaultStartManagementServerFunc(s, ctx, ociSpec)
}
case vc.PodContainer:
case virtcontainers.PodContainer:
span, ctx := katatrace.Trace(s.ctx, shimLog, "create", shimTracingTags)
defer span.End()
@@ -325,7 +324,7 @@ func checkAndMount(s *service, r *taskAPI.CreateTaskRequest) (bool, error) {
return false, nil
}
if vc.IsNydusRootFSType(m.Type) {
if virtcontainers.IsNydusRootFSType(m.Type) {
// if kata + nydus, do not mount
return false, nil
}
@@ -361,7 +360,7 @@ func doMount(mounts []*containerd_types.Mount, rootfs string) error {
return nil
}
func configureNonRootHypervisor(runtimeConfig *oci.RuntimeConfig, sandboxId string) error {
func configureNonRootHypervisor(runtimeConfig *oci.RuntimeConfig, sandboxID string) error {
userName, err := utils.CreateVmmUser()
if err != nil {
return err
@@ -370,7 +369,7 @@ func configureNonRootHypervisor(runtimeConfig *oci.RuntimeConfig, sandboxId stri
if err != nil {
shimLog.WithFields(logrus.Fields{
"user_name": userName,
"sandbox_id": sandboxId,
"sandbox_id": sandboxID,
}).WithError(err).Warn("configure non root hypervisor failed, delete the user")
if err2 := utils.RemoveVmmUser(userName); err2 != nil {
shimLog.WithField("userName", userName).WithError(err).Warn("failed to remove user")
@@ -398,7 +397,7 @@ func configureNonRootHypervisor(runtimeConfig *oci.RuntimeConfig, sandboxId stri
"user_name": userName,
"uid": uid,
"gid": gid,
"sandbox_id": sandboxId,
"sandbox_id": sandboxID,
}).Debug("successfully created a non root user for the hypervisor")
userTmpDir := path.Join("/run/user/", fmt.Sprint(uid))
@@ -410,7 +409,7 @@ func configureNonRootHypervisor(runtimeConfig *oci.RuntimeConfig, sandboxId stri
}
}
if err = os.Mkdir(userTmpDir, vc.DirMode); err != nil {
if err = os.Mkdir(userTmpDir, virtcontainers.DirMode); err != nil {
return err
}
defer func() {

View File

@@ -34,13 +34,13 @@ import (
const (
DirectVolumePathKey = "path"
AgentUrl = "/agent-url"
DirectVolumeStatUrl = "/direct-volume/stats"
DirectVolumeResizeUrl = "/direct-volume/resize"
IPTablesUrl = "/iptables"
PolicyUrl = "/policy"
IP6TablesUrl = "/ip6tables"
MetricsUrl = "/metrics"
AgentURL = "/agent-url"
DirectVolumeStatURL = "/direct-volume/stats"
DirectVolumeResizeURL = "/direct-volume/resize"
IPTablesURL = "/iptables"
PolicyURL = "/policy"
IP6TablesURL = "/ip6tables"
MetricsURL = "/metrics"
)
var (
@@ -288,13 +288,13 @@ func (s *service) startManagementServer(ctx context.Context, ociSpec *specs.Spec
// bind handler
m := http.NewServeMux()
m.Handle(MetricsUrl, http.HandlerFunc(s.serveMetrics))
m.Handle(AgentUrl, http.HandlerFunc(s.agentURL))
m.Handle(DirectVolumeStatUrl, http.HandlerFunc(s.serveVolumeStats))
m.Handle(DirectVolumeResizeUrl, http.HandlerFunc(s.serveVolumeResize))
m.Handle(IPTablesUrl, http.HandlerFunc(s.ipTablesHandler))
m.Handle(PolicyUrl, http.HandlerFunc(s.policyHandler))
m.Handle(IP6TablesUrl, http.HandlerFunc(s.ip6TablesHandler))
m.Handle(MetricsURL, http.HandlerFunc(s.serveMetrics))
m.Handle(AgentURL, http.HandlerFunc(s.agentURL))
m.Handle(DirectVolumeStatURL, http.HandlerFunc(s.serveVolumeStats))
m.Handle(DirectVolumeResizeURL, http.HandlerFunc(s.serveVolumeResize))
m.Handle(IPTablesURL, http.HandlerFunc(s.ipTablesHandler))
m.Handle(PolicyURL, http.HandlerFunc(s.policyHandler))
m.Handle(IP6TablesURL, http.HandlerFunc(s.ip6TablesHandler))
s.mountPprofHandle(m, ociSpec)
// register shim metrics
@@ -373,7 +373,7 @@ func ClientSocketAddress(id string) (string, error) {
if _, err := os.Stat(socketPath); err != nil {
socketPath = SocketPathRust(id)
if _, err := os.Stat(socketPath); err != nil {
return "", fmt.Errorf("It fails to stat both %s and %s with error %v.", SocketPathGo(id), SocketPathRust(id), err)
return "", fmt.Errorf("it fails to stat both %s and %s with error %v", SocketPathGo(id), SocketPathRust(id), err)
}
}

Some files were not shown because too many files have changed in this diff Show More