Commit Graph

15876 Commits

Author SHA1 Message Date
Fabiano Fidêncio
b96685bf7a
Merge pull request #11153 from fidencio/topic/build-allow-choosing-which-runtime-will-be-built
build: Allow users to build the go, rust, or both runtimes
2025-04-25 08:13:07 +02:00
Fabiano Fidêncio
800c05fffe
Merge pull request #11189 from kata-containers/sprt/fix-create-cluster
temp: ci: Fix AKS cluster creation
2025-04-24 23:01:12 +02:00
Aurélien Bombo
1de466fe84 temp: ci: Fix AKS cluster creation
The AKS CLI recently introduced a regression that prevents using
aks-preview extensions (Azure/azure-cli#31345), and hence create
CI clusters.

To address this, we temporarily hardcode the last known good version of
aks-preview.

Note that I removed the comment about this being a Mariner requirement,
as aks-preview is also a requirement of AKS App Routing, which will
be introduced soon in #11164.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2025-04-24 15:06:14 -05:00
Dan Mihai
706c2e2d68
Merge pull request #11184 from microsoft/danmihai1/retry-genpolicy
ci: retry genpolicy execution
2025-04-24 08:01:22 -07:00
Champ-Goblem
cf4325b535 kernel: Add CONFIG_TASKSTATS (and related) configs
Knowing that the upstream project provides a "ready to use" version of
the kernel, it's good to include an easy way to users to monitor
performance, and that's what we're doing by enabling the TASKSTATS (and
related) kernel configs.

This has been present as part of older kernels, but I couldn't
reasonably find the reason why it's been dropped.

Signed-off-by: Champ-Goblem <cameron@northflank.com>
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
2025-04-24 11:51:21 +02:00
Fabiano Fidêncio
7e9e9263d1 build: Allow users to build the go, rust, or both runtimes
Let's add a RUNTIME_CHOICE env var that can be passed to be build
scripts, which allows the user to select whether they bulld the go
runtime, the rust runtime, or both.

Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
2025-04-24 10:36:26 +02:00
Alex Lyn
8b49564c01
Merge pull request #10610 from Xynnn007/faet-initdata-rbd
Feat | Implement initdata for bare-metal/qemu hypervisor
2025-04-24 09:59:14 +08:00
Alex Lyn
e8f19609b9
Merge pull request #11150 from zvonkok/cdi-annotations
gpu: Fix CDI annotations
2025-04-24 09:58:16 +08:00
Dan Mihai
517d6201f5 ci: retry genpolicy execution
genpolicy is sending more HTTPS requests than other components during
CI so it's more likely to be affected by transient network errors
similar to:

ConnectError(
  "dns error",
  Custom {
     kind: Uncategorized,
     error: "failed to lookup address information: Try again",
  },
)

Note that genpolicy is not the only component hitting network errors
during CI. Recent example from a different component:

"Message:  failed to create containerd task: failed to create shim task:
 failed to async pull blob stream HTTP status server error (502 Bad Gateway)"

This CI change might help just with the genpolicy errors.

Fixes: #11182

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2025-04-23 21:38:12 +00:00
Zvonko Kaiser
3946435291 gpu: Handle VFIO devices with DevicePlugin and CDI
We can provide devices during cold-plug with CDI annotation on a Pod
level and add per container device information wit the device plugin.
Since the sandbox has already attached the VFIO device remove them
from consideration and just apply the inner runtime CDI annotation.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-04-23 21:02:06 +00:00
Zvonko Kaiser
486244b292 gpu: Remove unneeded parsing of CDI devices
The addition of CDI devices is now done for single_container
and pod_sandbox and pod_container before the devmanager creates
the deviceinfos no need for extra parsing.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-04-23 21:02:06 +00:00
Zvonko Kaiser
6713db8990 gpu: Add CDI parsing for Sandbox as well
Extend the CDI parsing for pod_sandbox as well, only single_container
was covered properly.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-04-23 21:02:06 +00:00
Zvonko Kaiser
97f4bcb456 gpu: Remove CDI annotations for outer runtime
After the outer runtime has processed the CDI annotation from the
spec we can delete them since they were converted into Linux
devices in the OCI spec.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-04-23 21:02:06 +00:00
Steve Horsman
6102976d2d
Merge pull request #11178 from stevenhorsman/gperf-mirror
versions: Switch gperf mirror
2025-04-23 20:21:42 +01:00
stevenhorsman
09052faaa0 versions: Switch gperf mirror
Every so often the main gnu site has an outage, so
we can't download gperf. GNU providesthe generic URL https://ftpmirror.gnu.org to
automatically choose a nearby and up-to-date mirror,
so switch to this to help avoid this problem

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-23 15:29:54 +01:00
stevenhorsman
ed56050a99 versions: Bump golangci-lint version
v1.60.0+ is needed for go 1.23 support, so
bump to the current latest 1.x version

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-23 12:37:48 +01:00
stevenhorsman
1c9d7ce0eb ci: cri-containerd: Remove source from install_go.sh
If the correct version of go is already installed then
install_go.sh runs `exit`. When calling this as source from
cri-containerd/gha-run.sh it means all dependencies after
are skipped, so remove this.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-23 12:37:48 +01:00
stevenhorsman
c37840ce80 versions: Bump golang version
Bump golang version to the latest minor 1.23.x release
now that 1.24 has been released and 1.22.x is no longer
stable and receiving security fixes

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-23 12:37:48 +01:00
dependabot[bot]
463fd4eda4
build(deps): bump crossbeam-channel from 0.5.14 to 0.5.15 in /src/agent
Bumps [crossbeam-channel](https://github.com/crossbeam-rs/crossbeam) from 0.5.14 to 0.5.15.
- [Release notes](https://github.com/crossbeam-rs/crossbeam/releases)
- [Changelog](https://github.com/crossbeam-rs/crossbeam/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crossbeam-rs/crossbeam/compare/crossbeam-channel-0.5.14...crossbeam-channel-0.5.15)

---
updated-dependencies:
- dependency-name: crossbeam-channel
  dependency-version: 0.5.15
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-23 11:34:14 +00:00
Steve Horsman
1ffce3ff70
Merge pull request #11173 from stevenhorsman/update-before-install
workflows: Add apt update before install
2025-04-23 12:32:54 +01:00
stevenhorsman
ccfdf59607 workflows: Add apt update before install
Add apt/apt-get updates before we do
apt/apt-get installs to try and help with
issues where we fail to fetch packages

Co-authored-by: Fabiano Fidêncio <fidencio@northflank.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-23 09:06:08 +01:00
Xynnn007
b1c72c7094 test: add integration test for initdata
This test we will test initdata in the following logic
1. Enable image signature verification via kernel commandline
2. Set Trustee address via initdata
3. Pull an image from a banned registry
4. Check if the pulling fails with log `image security validation
failed` the initdata works.

Note that if initdata does not work, the pod still fails to launch. But
the error information is `[CDH] [ERROR]: Get Resource failed` which
internally means that the KBS URL has not been set correctly.

This test now only runs on qemu-coco-dev+x86_64 and qemu-tdx

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
2025-04-23 15:55:04 +08:00
RuoqingHe
ef12dcd7da
Merge pull request #11158 from RuoqingHe/2025-04-15-fix-flag-calc
runtime-rs: Use bitwise or assign for bitflags
2025-04-23 15:20:33 +08:00
alex.lyn
9eb3fcb84b kata-types: Clean up noise caused by unformatted code
For a long time, there has been unformatted code in the kata-types
codebase, for example:
```
if qemu.memory_info.enable_guest_swap {
-                return Err(eother!(
-                    "Qemu hypervisor doesn't support enable_guest_swap"
-                ));
+                return Err(eother!("Qemu hypervisor doesn't support
enable_guest_swap"));
             }
...
-    }, device::DRIVER_NVDIMM_TYPE, eother, resolve_path
+    },
+    device::DRIVER_NVDIMM_TYPE,
+    eother, resolve_path,
-use std::collections::HashMap;
-use anyhow::{Result, anyhow};
+use anyhow::{anyhow, Result};
 use std::collections::hash_map::Entry;
+use std::collections::HashMap;
-/// DRIVER_VFIO_PCI_GK_TYPE is the device driver for vfio-pci 
+/// DRIVER_VFIO_PCI_GK_TYPE is the device driver for vfio-pci
```
This has brought unnecessary difficulties in version maintenance and
commit difficulties. This commit will address this issue.

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2025-04-23 09:40:07 +08:00
alex.lyn
97a1942f86 kata-types: Optimize memory adjuesting by only gathering memory info
The Coniguration initialization was observed to be significantly slow
due to the extensive system information gathering performed by
`sysinfo::System::new_all()`. This function collects data on CPU,
memory, disks, and network, most of which is unnecessary for Kata's
memory adjusting config phase, where only the total system memory is
required.

This commit optimizes the initialization process by implementing a more
targeted approach to retrieve only the total system memory. This avoids
the overhead of collecting a large amount of irrelevant data, resulting
in a noticeable performance improvement.

Fixes #11165

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2025-04-23 09:40:07 +08:00
alex.lyn
3e77377be0 kata-sys-utils: Add test cases for devices
In this, the crate mockall is introduced to help
mock get_all_devices.

Fixes #10556

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2025-04-23 09:32:04 +08:00
alex.lyn
f714b6c049 kata-sys-utils: Add test cases for pci manager
Fixes #10556

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2025-04-23 09:32:04 +08:00
alex.lyn
0cdc05ce0a kata-sys-utils: Introduce method to help handle proper BAR memory
We need more information (BAR memory and other future
ures...)for
PCI devices when vfio devices passed through.
So the method get_bars_max_addressable_memory is introduced for vfio
devices to deduce the memory_reserve and pref64_reserve for NVIDIA

devices. But it will be extended for other devices.

Fixes #10556

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2025-04-23 09:32:04 +08:00
alex.lyn
f5eaaa41d5 kata-sys-utils: Introduce pcilibs to help get pci device info
It's the basic framework for getting information of pci devices.
Currently, we focus on the PCI Max bar memory size, but it'll be
extended in the future.

Fixes #10556

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2025-04-23 09:32:04 +08:00
Ruoqing He
d7f4b6cbef runtime-rs: Use bitwise or assign for bitflags
Use `|=` instead of `+=` while calculating and iterating through a
vector of flags, which makes more sense and prevents situations like
duplicated flags in vector, which would cause problems.

Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
2025-04-22 23:55:11 +00:00
Jakob Naucke
1c3b1f5adb ci: Extend basic s390x tests
Currently, s390x only tests cri-containerd. Partially converge to the
feature set of basic-ci-amd64:

- containerd-sandboxapi
- containerd-stability
- docker

with the appropriate hypervisors.
Do not run tests currently skipped on amd64, as well as

- agent-ctl, which we don't package for s390x
- nerdctl, does not package the `full` image for s390x
- nydus, does not package for s390x

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Co-authored-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2025-04-22 21:34:02 +02:00
Aurélien Bombo
bf93b5daf1
Merge pull request #11113 from Sumynwa/sumsharma/policy_execprocess_container_id
genpolicy: Add container_id & related policy container data to state.
2025-04-22 18:37:58 +01:00
Aurélien Bombo
318c409ed6
Merge pull request #11126 from gkurz/rootfs-systemd-files
rootfs: Don't remove files from the rootfs by default
2025-04-22 18:17:14 +01:00
Aurélien Bombo
12594a9f9e
Merge pull request #11157 from wainersm/make_nontee_job_not_required
ci: demote CoCo non-TEE to non-required from gatekeeper
2025-04-22 18:15:28 +01:00
Greg Kurz
734e7e8c54 rootfs: Don't remove files from the rootfs by default
Recent PR #10732 moved the deletion of systemd files and units that were
deemed uneccessary by 02b3b3b977 from `image_builder.sh` to `rootfs.sh`.
This unfortunately broke `rootfs.sh centos` and `rootfs.sh -r` as used by
some other downstream users like fedora and RHEL, with the following error :

  Warning  FailedCreatePodSandBox  1s (x5 over 63s)  kubelet
   Failed to create pod sandbox: rpc error: code = Unknown
     desc = CreateContainer failed: Establishing a D-Bus connection

Caused by:
    0: I/O error: Connection reset by peer (os error 104)
    1: Connection reset by peer (os error 104)

This is because the aforementioned distros use dbus-broker [1] that requires
systemd-journald to be present.

It is questionable that systemd units or files should be deemed unnecessary
for _all_ distros but this has been around since 2019. There's now also a
long-standing expectation from CI that `make rootfs && make image` does
remove these files.

In order to accomodate all the expectations, add a `-d` flag to `rootfs.sh`
to delete the systemd files and have `make rootfs` to use it.

[1] https://github.com/bus1/dbus-broker

Reported-by: Niteesh Dubey <niteesh@us.ibm.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
2025-04-17 16:53:05 +02:00
Zvonko Kaiser
497ab9faaf
Merge pull request #10999 from zvonkok/rootfs-updates
gpu: Update creation permissions
2025-04-16 10:15:38 -04:00
Wainer dos Santos Moschetta
90397ca4fe ci: demote CoCo non-TEE to non-required from gatekeeper
The CoCo non-TEE job has failed due the removal of an add-on
from AKS, causing KBS to not get installed (see #11156).

The fix should be done in this repo as well as in trustee, which can
take some time. We don't want to hold kata-containers PRs from getting
merged anylonger, so removing the job from required list.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2025-04-15 19:00:30 -03:00
Wainer Moschetta
ff9fb19f11
Merge pull request #11026 from ldoktor/e2e-resources
ci.ocp: Override default runtimeclass CPU resources
2025-04-15 10:33:35 -03:00
Lukáš Doktor
bfdf4e7a6a
ci.ocp: Add peer-pods setup script
this script will be used in a new OCP integration pipeline to monitor
basic workflows of OCP+peer-pods.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-04-15 12:13:22 +02:00
Xynnn007
91bb6b7c34 runtime: add support for io.katacontainers.config.runtime.cc_init_data
io.katacontainers.config.runtime.cc_init_data specifies initdata used by
the pod in base64(gzip(initdata toml)) format. The initdata will be
encapsulated into an initdata image and mount it as a raw block device
to the guest.

The initdata image will be aligned with 512 bytes, which is chosen as a
usual sector size supported by different hypervisors like qemu, clh and
dragonball.

Note that this patch only adds support for qemu hypervisor.

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
2025-04-15 16:35:59 +08:00
Sumedh Sharma
2a17628591 genpolicy: Add container_id & related policy container data to state.
This commit adds changes to add input container_id and related
container data to state after a CreateContainerRequest is allowed. This
helps constrain reference container data for evaluating request
inputs to one instead of matching against every policy container data,
Ex: in ExecProcessRequest inputs.

Fixes #11109

Signed-off-by: Sumedh Sharma <sumsharma@microsoft.com>
2025-04-15 14:02:59 +05:30
Zvonko Kaiser
2f28be3ad9 gpu: Update creation permissions
We need to make sure the device files are created correctly
in the rootfs otherwise kata-agent will apply permission 0o000.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-04-14 21:02:34 +00:00
Fabiano Fidêncio
bfd4b98355
Merge pull request #11142 from fidencio/topic/build-scripts-improvements-for-users
build: User-facing improvements for the build scripts
2025-04-14 19:28:12 +02:00
Fabiano Fidêncio
5e363dc277 virtiofsd: Update to v1.13.1
It's been released for some time already ... and although we did have
the necessary patches in, we better to stick to a released version of
the project.

Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
2025-04-14 13:23:31 +02:00
Fabiano Fidêncio
2fef594f14 build: Allow users to define AGENT_POLICY
This is mostly used for Kata Containers backing up Confidential
Computing use cases, this also has benefits for the normal Kata
Containers use cases, this it's left enabled by default.

However, let's allow users to specify whether or not they want to have
it enabled, as depending on their use-case, it just does not make sense.

Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
2025-04-14 10:02:22 +02:00
Fabiano Fidêncio
5d0688079a build: Allow users to specificy EXTRA_PKGS
Right now we've had some logic to add EXTRA_PKGS, but those were
restrict to the nvidia builds, and would require changing the file
manually.

Let's make sure a user can add this just by specifying an env var.

Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
2025-04-14 10:02:22 +02:00
Fabiano Fidêncio
40a15ac760 build: Allow adding a guest-hook to the rootfs
Kata Containers provides, since forever, a way to run OCI guest-hooks
from the rootfs, as long as the files are dropped in a specific location
defined in the configuration.toml.

However, so far, it's been up to the ones using it to hack the generated
image in order to add those guest hooks, which is far from handy.

Let's add a way for the ones interested on this feature to just drop a
tarball file under the same known build directory, spcificy an env var,
and let the guest hooks be installed during the rootfs build.

Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
2025-04-14 10:02:16 +02:00
RuoqingHe
0b4fea9382
Merge pull request #11134 from stevenhorsman/rust-toolchain
rust: Add rust-toolchain.toml
2025-04-12 15:03:29 +08:00
Steve Horsman
792180a740
Merge pull request #11105 from stevenhorsman/required-tests-process-update
doc: Update required job process
2025-04-11 14:53:27 +01:00
stevenhorsman
93830cbf4d rust: Add rust-toolchain.toml
Add a top-level rust-toolchain.toml with the version
that matches version.yaml to ensure that we stay in sync

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-11 09:24:04 +01:00