Commit Graph

15754 Commits

Author SHA1 Message Date
Zvonko Kaiser
3bd7aeae42 gpu: Add NIM embedding service
This is the second part of the RAG pipeline for embedding

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-04-24 21:21:16 +00:00
Zvonko Kaiser
205d132fce #SQUASH DO NOT PICK 2025-04-24 21:19:10 +00:00
Zvonko Kaiser
d47e79f719 gpu: Disable metrics tests
We are not running the metrics tests anyway for now
lets make room to run the GPU tests.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-04-24 21:19:10 +00:00
Zvonko Kaiser
f7673f4c24 gpu: Add Pod spec for NIM llama
Pod spec for the NIM inferencing service

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-04-24 21:19:06 +00:00
Zvonko Kaiser
c023817a8e gpu: Add NIM bats test
We're running a simple NIM container to test if the GPUs
are working properly

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-04-24 00:07:26 +00:00
Zvonko Kaiser
79bf86d71d gpu: Add run_kubernetes_nv_tests.sh
Replicate what we have for run_tests and run .bats files

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-04-24 00:07:26 +00:00
Zvonko Kaiser
73eb944f5a gpu: Add selector for k8s tests
We want to reuse the current run_tests with GPUs, introduce a var
that will define what to run.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-04-24 00:07:24 +00:00
Zvonko Kaiser
081db20c5f gpu: Add NVIDIA GPU test block for amd64
Once we have the amd64 artifacts we can run some arm64 k8s tests.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-04-23 21:45:59 +00:00
Steve Horsman
6102976d2d
Merge pull request #11178 from stevenhorsman/gperf-mirror
versions: Switch gperf mirror
2025-04-23 20:21:42 +01:00
stevenhorsman
09052faaa0 versions: Switch gperf mirror
Every so often the main gnu site has an outage, so
we can't download gperf. GNU providesthe generic URL https://ftpmirror.gnu.org to
automatically choose a nearby and up-to-date mirror,
so switch to this to help avoid this problem

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-23 15:29:54 +01:00
Steve Horsman
1ffce3ff70
Merge pull request #11173 from stevenhorsman/update-before-install
workflows: Add apt update before install
2025-04-23 12:32:54 +01:00
stevenhorsman
ccfdf59607 workflows: Add apt update before install
Add apt/apt-get updates before we do
apt/apt-get installs to try and help with
issues where we fail to fetch packages

Co-authored-by: Fabiano Fidêncio <fidencio@northflank.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-23 09:06:08 +01:00
RuoqingHe
ef12dcd7da
Merge pull request #11158 from RuoqingHe/2025-04-15-fix-flag-calc
runtime-rs: Use bitwise or assign for bitflags
2025-04-23 15:20:33 +08:00
Ruoqing He
d7f4b6cbef runtime-rs: Use bitwise or assign for bitflags
Use `|=` instead of `+=` while calculating and iterating through a
vector of flags, which makes more sense and prevents situations like
duplicated flags in vector, which would cause problems.

Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
2025-04-22 23:55:11 +00:00
Aurélien Bombo
bf93b5daf1
Merge pull request #11113 from Sumynwa/sumsharma/policy_execprocess_container_id
genpolicy: Add container_id & related policy container data to state.
2025-04-22 18:37:58 +01:00
Aurélien Bombo
318c409ed6
Merge pull request #11126 from gkurz/rootfs-systemd-files
rootfs: Don't remove files from the rootfs by default
2025-04-22 18:17:14 +01:00
Aurélien Bombo
12594a9f9e
Merge pull request #11157 from wainersm/make_nontee_job_not_required
ci: demote CoCo non-TEE to non-required from gatekeeper
2025-04-22 18:15:28 +01:00
Greg Kurz
734e7e8c54 rootfs: Don't remove files from the rootfs by default
Recent PR #10732 moved the deletion of systemd files and units that were
deemed uneccessary by 02b3b3b977 from `image_builder.sh` to `rootfs.sh`.
This unfortunately broke `rootfs.sh centos` and `rootfs.sh -r` as used by
some other downstream users like fedora and RHEL, with the following error :

  Warning  FailedCreatePodSandBox  1s (x5 over 63s)  kubelet
   Failed to create pod sandbox: rpc error: code = Unknown
     desc = CreateContainer failed: Establishing a D-Bus connection

Caused by:
    0: I/O error: Connection reset by peer (os error 104)
    1: Connection reset by peer (os error 104)

This is because the aforementioned distros use dbus-broker [1] that requires
systemd-journald to be present.

It is questionable that systemd units or files should be deemed unnecessary
for _all_ distros but this has been around since 2019. There's now also a
long-standing expectation from CI that `make rootfs && make image` does
remove these files.

In order to accomodate all the expectations, add a `-d` flag to `rootfs.sh`
to delete the systemd files and have `make rootfs` to use it.

[1] https://github.com/bus1/dbus-broker

Reported-by: Niteesh Dubey <niteesh@us.ibm.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
2025-04-17 16:53:05 +02:00
Zvonko Kaiser
497ab9faaf
Merge pull request #10999 from zvonkok/rootfs-updates
gpu: Update creation permissions
2025-04-16 10:15:38 -04:00
Wainer dos Santos Moschetta
90397ca4fe ci: demote CoCo non-TEE to non-required from gatekeeper
The CoCo non-TEE job has failed due the removal of an add-on
from AKS, causing KBS to not get installed (see #11156).

The fix should be done in this repo as well as in trustee, which can
take some time. We don't want to hold kata-containers PRs from getting
merged anylonger, so removing the job from required list.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2025-04-15 19:00:30 -03:00
Wainer Moschetta
ff9fb19f11
Merge pull request #11026 from ldoktor/e2e-resources
ci.ocp: Override default runtimeclass CPU resources
2025-04-15 10:33:35 -03:00
Sumedh Sharma
2a17628591 genpolicy: Add container_id & related policy container data to state.
This commit adds changes to add input container_id and related
container data to state after a CreateContainerRequest is allowed. This
helps constrain reference container data for evaluating request
inputs to one instead of matching against every policy container data,
Ex: in ExecProcessRequest inputs.

Fixes #11109

Signed-off-by: Sumedh Sharma <sumsharma@microsoft.com>
2025-04-15 14:02:59 +05:30
Zvonko Kaiser
2f28be3ad9 gpu: Update creation permissions
We need to make sure the device files are created correctly
in the rootfs otherwise kata-agent will apply permission 0o000.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-04-14 21:02:34 +00:00
Fabiano Fidêncio
bfd4b98355
Merge pull request #11142 from fidencio/topic/build-scripts-improvements-for-users
build: User-facing improvements for the build scripts
2025-04-14 19:28:12 +02:00
Fabiano Fidêncio
5e363dc277 virtiofsd: Update to v1.13.1
It's been released for some time already ... and although we did have
the necessary patches in, we better to stick to a released version of
the project.

Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
2025-04-14 13:23:31 +02:00
Fabiano Fidêncio
2fef594f14 build: Allow users to define AGENT_POLICY
This is mostly used for Kata Containers backing up Confidential
Computing use cases, this also has benefits for the normal Kata
Containers use cases, this it's left enabled by default.

However, let's allow users to specify whether or not they want to have
it enabled, as depending on their use-case, it just does not make sense.

Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
2025-04-14 10:02:22 +02:00
Fabiano Fidêncio
5d0688079a build: Allow users to specificy EXTRA_PKGS
Right now we've had some logic to add EXTRA_PKGS, but those were
restrict to the nvidia builds, and would require changing the file
manually.

Let's make sure a user can add this just by specifying an env var.

Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
2025-04-14 10:02:22 +02:00
Fabiano Fidêncio
40a15ac760 build: Allow adding a guest-hook to the rootfs
Kata Containers provides, since forever, a way to run OCI guest-hooks
from the rootfs, as long as the files are dropped in a specific location
defined in the configuration.toml.

However, so far, it's been up to the ones using it to hack the generated
image in order to add those guest hooks, which is far from handy.

Let's add a way for the ones interested on this feature to just drop a
tarball file under the same known build directory, spcificy an env var,
and let the guest hooks be installed during the rootfs build.

Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
2025-04-14 10:02:16 +02:00
RuoqingHe
0b4fea9382
Merge pull request #11134 from stevenhorsman/rust-toolchain
rust: Add rust-toolchain.toml
2025-04-12 15:03:29 +08:00
Steve Horsman
792180a740
Merge pull request #11105 from stevenhorsman/required-tests-process-update
doc: Update required job process
2025-04-11 14:53:27 +01:00
stevenhorsman
93830cbf4d rust: Add rust-toolchain.toml
Add a top-level rust-toolchain.toml with the version
that matches version.yaml to ensure that we stay in sync

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-11 09:24:04 +01:00
Steve Horsman
ad68cb9afa
Merge pull request #11106 from stevenhorsman/rust-workspace-settings
agent: Inherit rust workspace settings
2025-04-10 09:47:53 +01:00
stevenhorsman
75dc4ce3bf doc: Update required job process
Add information about using required-tests.yaml
as a way to track jobs that are required.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-09 18:13:45 +01:00
Steve Horsman
0dbf4ec39f
Merge pull request #10678 from stevenhorsman/update-gatekeeper-rules-for-md-only-PRs
ci: Update gatekeeper tests for md files
2025-04-09 18:10:05 +01:00
stevenhorsman
d1d60cfe89 ci: Update gatekeeper tests for md files
Update the required-tests.yaml so that .md files only
trigger the static tests, not the build, or CI

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-09 17:55:27 +01:00
Steve Horsman
9b401cd250
Merge pull request #11090 from stevenhorsman/required-test-updates
ci: required-tests fixes/updates
2025-04-09 14:41:57 +01:00
stevenhorsman
576747b060 ci: Skip tests if we only update the required list
When making new tests required, or removing existing tests
from required, this doesn't impact the CI jobs, so we don't need
to run all the tests.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-09 14:22:47 +01:00
stevenhorsman
9a7c5b914e ci: required-tests fixes/updates
- Remove metrics setup job
- Update some truncation typos of job names
- Add shellcheck-required
- Remove the ok-to-test as a required label on the build test
as it isn't needed as a trigger

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-09 14:22:37 +01:00
Xuewei Niu
5774f131ec
Merge pull request #10938 from Apokleos/fix-iommugrp-symlink
runtime-rs: Simplify iommu group base name extraction from symlink
2025-04-09 19:23:48 +08:00
Xuewei Niu
fd9a4548ab
Merge pull request #11129 from RuoqingHe/entend-runtime-rs-workspace
runtime-rs: Extend runtime-rs workspace and centralize local dependencies
2025-04-09 19:23:15 +08:00
stevenhorsman
6603cf7872 agent: Update vsock-exporter to use workspace settings
To reduce duplication, we could update
the vsock-exporter crate to use settings and versions
 from the agent, where applicable.
> [!NOTE]
> In order to use the workspace, this has bumped some crate versions

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-09 12:02:43 +01:00
stevenhorsman
2cb9fd3c69 agent: Update rustjail to use workspace settings
- To reduce duplication, we could update
the rustjail crate to use settings and versions
from the agent, where applicable.
- Also switch to using the derive feature in serde crate
rather than the separate serde_derive to avoid keeping
both versions in sync

> [!NOTE]
> In order to use the workspace, this has bumped
some crate versions

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-09 12:02:43 +01:00
stevenhorsman
655255b50c agent: Update policy to use workspace settings
To reduce duplication, we could update
the policy crate to use settings and versions
from the agent, where applicable.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-09 11:42:05 +01:00
stevenhorsman
1bec432ffa agent: Create workspace package and dependencies
- Create agent workspace dependencies and packge info
so that the packages in the workspace can use them
- Group the local dependencies together for clarity
(like in #11129)

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-09 11:42:00 +01:00
Ruoqing He
28c09ae645 runtime-rs: Put local dependencies into workspace
Put local dependencies into workspace to avoid complex path dependencies all
over the workspace. This gives an overview of local dependencies this workspace
uses, where those crates are located, and simplifies the local dependencies
referencing process.

Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
2025-04-09 07:30:29 +00:00
Ruoqing He
3769ad9c0d runtime-rs: Group local dependencies
Judging by the layout of the `Cargo.toml` files, local dependencies are
intentionally separated from other dependencies, let's enforce it
workspace-wise.

Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
2025-04-09 03:52:16 +00:00
Ruoqing He
abb5fb127b runtime-rs: Extend workspace to cover all crates
Only `shim` and `shim-ctl` are incorporated in `runtime-rs`'s workspace, let's
extend it to cover all crates in `runtime-rs/crates`.

Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
2025-04-09 03:51:48 +00:00
alex.lyn
58bebe332a runtime-rs: Simplify iommu group base name extraction from symlink
Just get base name from iommu group symlink is enough. As the
validation will be handled in subsequent steps when constructing
the full path /sys/kernel/iommu_groups/$iommu_group.

In this PR, it will remove dupicalted validation of iommu_group.

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2025-04-09 09:28:00 +08:00
Steve Horsman
8df271358e
Merge pull request #11128 from stevenhorsman/disable-metrics-jobs
ci: Remove metric jobs
2025-04-08 18:16:35 +01:00
stevenhorsman
e6cca9da6d ci: Remove metric jobs
The metrics runner is broken, so skip the metrics
jobs to stop the CI being stuck waiting.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-04-08 17:55:07 +01:00