Commit Graph

566 Commits

Author SHA1 Message Date
Aurélien Bombo
e43c59a2c6 ci: Add no-op step to compile CSI driver
This adds a no-op build step to compile the CSI driver. The actual compilation
will be implemented in an ulterior PR, so as to ensure we don't break the CI.

Addresses: #10560

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2024-11-21 16:06:55 -06:00
Zvonko Kaiser
0debf77770 gpu: NVIDIA gpu initrd/image build
With each release make sure we ship a GPU enabled rootfs/initrd

Fixes: #6554

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-11-21 18:57:23 +00:00
stevenhorsman
7a8ba14959 release: Bump version to 3.11.0
Bump `VERSION` and helm-chart versions

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-11-18 11:13:15 +00:00
Fabiano Fidêncio
a52ea32b05 build: kernel: Learn how to deal with release candidates
So far we were not prepared to deal with release candidates as those:
* Do not have a sha256sum in the sha256sums provided by the kernel cdn
* Come from a different URL (directly from Linus)
* Have a different suffix (.tar.gz, instead of .tar.xz)

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-11-05 12:26:02 +01:00
Fabiano Fidêncio
9f2d4b2956 build: kernel: Always pass the url to the builder
This doesn't change much on how we're doing things Today, but it
simplifies a lot cases that may be added later on (and will be) like
building -rc kernels.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-11-05 12:26:02 +01:00
Fabiano Fidêncio
cc4006297a build: kernel: Pass the yaml base path instead of the version path
By doing this we can ensure this can be re-used, if needed (and it'll be
needed), for also getting the URL.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-11-05 12:26:02 +01:00
Fabiano Fidêncio
7057ff1cd5 build: kernel: Always pass -f to the kernel builder
-f forces the (re)generaton of the config when doing the setup, which
helps a lot on local development whilst not causing any harm in the CI
builds.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-11-05 12:26:02 +01:00
Sumedh Alok Sharma
4b7aba5c57 agent-ctl: Add support to test kata-agent's container creation APIs.
This commit introduces changes to enable testing kata-agent's container
APIs of CreateContainer/StartContainer/RemoveContainer. The changeset
include:
- using confidential-containers image-rs crate to pull/unpack/mount a
container image. Currently supports only un-authenicated registry pull
- re-factor api handlers to reduce cmdline complexity and handle
request generation logic in tool
- introduce an OCI config template for container creation
- add test case

Fixes #9707

Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
2024-11-01 22:18:54 +05:30
Sumedh Alok Sharma
0adf7a66c3 ci: Use ubuntu for static building of kata tools.
This commit introduces changes to use ubuntu for statically
building kata tools. In the existing CI setup, the tools
currently build only for x86_64 architecture.

It also fixes the build error seen for agent-ctl PR#10395.

Fixes #10441

Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
2024-10-31 13:19:18 +05:30
Fabiano Fidêncio
f19c8cbd02 build: cache: Ensure shim-v2-root_hash.txt is in "${workdir}"
All the oras push logic happens from inside `${workdir}`, while the
root_hash.txt extraction and renaming was not taking this into
consideration.

This was not caught during the manually triggered runs as those do not
perform the oras push.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-28 15:17:16 +01:00
Fabiano Fidêncio
d537932e66 build: shim-v2: Ensure MEASURED_ROOTFS is exported
The approach taken for now is to export MEASURED_ROOTFS=yes on the
workflow files for the architectures using confidential stuff, and leave
the "normal" build without having it set (to avoid any change of
expectation on the current bevahiour).

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-28 12:43:53 +01:00
Fabiano Fidêncio
9c8b20b2bf build: shim-v2: Rebuild if root_hashes do not match
Let's make sure we take the root_hashes into consideration to decide
whether the shim-v2 should or should not be used from the cached
artefacts.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-28 12:43:53 +01:00
Fabiano Fidêncio
9c84998de9 build: cache: Cache root_hash.txt used by the shim-v2
Let's cache the root_hash.txt from the confidential image so we can use
them later on to decide whether there was a rootfs change that would
require shim-v2 to be rebuilt.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-28 12:43:53 +01:00
Fabiano Fidêncio
d2d9792720 build: Don't leave cached component behind if it can't be used
Let's ensure we remove the component and any extra tarball provided by
ORAS in case the cached component cannot be used.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-28 12:43:53 +01:00
stevenhorsman
b22d4429fb release: Bump version to 3.10.1
Fix release to pick up #10463

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-10-25 17:16:09 +01:00
Greg Kurz
3037303e09 release: Bump VERSION to 3.10.0
Let's start the 3.10.0 release.

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-10-22 11:28:15 +02:00
Magnus Kulke
9a33a3413b Merge pull request #10433 from mkulke/mkulke/add-provenance-attestation-for-agent-builds
ci: add provenance attestation for agent artifact
2024-10-18 15:00:18 +02:00
Magnus Kulke
b93f5390ce ci: add provenance attestation for agent artifact
This adds provenance attestation logic for agent binaries that are
published to an oci registry via ORAS.

As a downstream consumer of the kata-agent binary the Peerpod project
needs to verify that the artifact has been built on kata's CI.

To create an attestation we need to know the exact digest of the oci
artifact, at the point when the artifact was pushed.

Therefore we record the full oci image as returned by oras push.

The pushing and tagging logic has been slightly reworked to make this
task less repetetive.

The oras cli accepts multiple tags separated by comma on pushes, so a
push can be performed atomically instead of iterating through tags and
pushing each individually. This removes the risk of partially successful
push operations (think: rate limits on the oci registry).

So far the provenance creation has been only enabled for agent builds on
amd64 and xs390x.

Signed-off-by: Magnus Kulke <magnuskulke@microsoft.com>
2024-10-18 10:24:00 +02:00
Dan Mihai
3622b5e8b4 ci: static_sandbox_resource_mgmt for cbl-mariner
Use the configuration used by AKS (static_sandbox_resource_mgmt=true)
for CI testing on Mariner hosts.

Hopefully pod startup will become more predictable on these hosts -
e.g., by avoiding the occasional hotplug timeouts described by #10413.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-10-10 22:17:39 +00:00
Fabiano Fidêncio
652ba30d4a build: mariner: Remove the ability to build the marine initrd
As mariner has switched to using an image instead of an initrd, let's
just drop the abiliy to build the initrd and avoid keeping something in
the tree that won't be used.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-09 21:42:55 +02:00
Dan Mihai
63c1f81c23 local-build: add rootfs-image-mariner
Kata CI will start testing the new rootfs-image-mariner instead of the
older rootfs-initrd-mariner image.

The "official" AKS images are moving from a rootfs-initrd-mariner
format to the rootfs-image-mariner format. Making the same change in
Kata CI is useful to keep this testing in sync with the AKS settings.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-10-08 17:15:56 +00:00
Aurélien Bombo
78c63c7951 kata-deploy: clean up and fix docs for k0s
* Clarifies instructions for k0s.
* Adds kata-deploy step for each cluster type.
* Removes the old kata-deploy-stable step for vanilla k8s.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2024-09-20 11:59:40 -05:00
Emanuel Lima
a6ee15c5c7 release: Bump VERSION to 3.9.0
Starting the v3.9.0 release

Signed-off-by: Emanuel Lima <emlima@redhat.com>
2024-09-19 10:14:55 -03:00
stevenhorsman
1abeffdac6 kata-deploy: Switch Kubernetes URL
The payload build is failing with:
```
ERROR: failed to solve: process "/bin/sh -c apk --no-cache add bash curl &&
ARCH=$(uname -m) &&
if [ \"${ARCH}\" = \"x86_64\" ]; then ARCH=amd64; fi &&
if [ \"${ARCH}\" = \"aarch64\" ]; then ARCH=arm64; fi &&
DEBIAN_ARCH=${ARCH} &&
if [ \"${DEBIAN_ARCH}\" = \"ppc64le\" ]; then DEBIAN_ARCH=ppc64el; fi &&
curl -fL --progress-bar -o /usr/bin/kubectl https://storage.googleapis.com/kubernetes-release/release/ \
$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/${ARCH}/kubectl &&
chmod +x /usr/bin/kubectl &&
curl -fL --progress-bar -o /usr/bin/jq https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-linux-${DEBIAN_ARCH} &&
chmod +x /usr/bin/jq &&
mkdir -p ${DESTINATION} &&
tar xvf ${WORKDIR}/${KATA_ARTIFACTS} -C ${DESTINATION} &&
rm -f ${WORKDIR}/${KATA_ARTIFACTS} &&
apk del curl &&
apk --no-cache add py3-pip &&
pip install --no-cache-dir yq==3.2.3" did not complete successfully: exit code: 22
```

Looking into this, the problem is that
https://storage.googleapis.com/kubernetes-release/release/v1.31.1/bin/linux/amd64/kubectl
doesn't exist. The [kubectl install doc](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/#install-kubectl-on-linux)
recommends the `dl.k8s.io` site, so let's switch to this.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-09-17 15:35:42 +01:00
Fabiano Fidêncio
d44d66ddf6 kata-deploy: Remove kata-cleanup unneeded vars
As kata-cleanup will only call `reset_runtime()`, there's absolutely no
need to export the other set of environment variables in its yaml file.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-04 19:09:02 +02:00
Fabiano Fidêncio
bb9bcd886a kata-deploy: Add reset_cri_runtime()
This will help to avoid code duplication on what's needed on the helm
and non-helm cases.

The reason it's not been added as part of the commit which adds the
post-delete hook is simply for helping the reviewer (as the diff would
be less readable with this change).

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-03 23:08:22 +02:00
Fabiano Fidêncio
64ccb1645d helm: Add a post-delete hook
Instead of using a lifecycle.preStop hook, as done when we're using
using the helm chat, let's add a post-delete hook to take care of
properly cleaning up the node during when uninstalling kata-deploy.

The reason why the lifecyle.preStop hook would never work on our case is
simply because each helm chart operation follows the Kuberentes
"declarative" approach, meaning that an operation won't wait for its
previous operation to successfully finish before being called, leading
to us trying to access content that's defined by our RBAC, in an
operation that was started before our RBAC was deleted, but having the
RBAC being deleted before the operation actually started.

Unfortunately this hook brings in some code duplicatioon, mainly related
to the RBAC parts, but that's not new as the same happens with our
deamonset.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-09-03 23:08:22 +02:00
Fabiano Fidêncio
0cb93ed1bb kata-deploy: helm: Add INSTALLATION_PREFIX option
This will allow users to properly set the INSTALLATION_PREFIX when
deploying Kata Containers.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-02 20:25:22 +02:00
Fabiano Fidêncio
7be77ebee5 kata-deploy: helm: Stop mounting /opt/kata
It's simply easier if we just use /host/opt/kata instead in our scripts,
which will simplify a lot the logic of adding an INSTALLATION_PREFIX
later on.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-02 09:38:51 +02:00
Fabiano Fidêncio
6ce5e62c48 kata-deploy: Add a $dest_dir var
As we build our binaries with the `/opt/kata` prefix, that's the value
of $dest_dir.

Later in thise series it'll become handy, as we'll introduce a way to
install the Kata Containers artefacts in a different location.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-02 09:36:33 +02:00
Fabiano Fidêncio
a9a1345a31 kata-deploy: Print the action the script was invoked with
This increases debuggability.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-08-22 14:32:33 +02:00
Fabiano Fidêncio
ab493b6028 kata-deploy: Move general logic to the correct actions
therwise we may end up running into unexpected issues when calling the
cleanup option, as the same checks would be done, and files could end up
being copied again, overwriting the original content which was backked
up by the install option.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-22 14:32:29 +02:00
Fabiano Fidêncio
6596012956 kata-deploy: Simplify check for runtime
Let's write the runtime check in a shorter and simpler to read form.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-08-22 14:32:02 +02:00
Fabiano Fidêncio
d03b72f19b kata-deploy: Stop linking binaries to /usr/local/bin
Neither CRI-O nor containerd requires that, and removing such symlinks
makes everything less intrusive from our side.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-18 01:25:12 +02:00
Fabiano Fidêncio
c2393dc467 kata-deploy: Use shim's absolute path for crio's runtime_path
This will allow us, in the future, not have to do symlinks here and
there.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-18 01:25:12 +02:00
Fabiano Fidêncio
58623723b1 kata-deploy: Use runtime_path for containerd
It's already being used with CRi-O, let's simplify what we do and also
use this for containerd, which will allow us to do further cleanups in
the coming patches.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-18 01:25:12 +02:00
Fabiano Fidêncio
b203f715e5 Merge pull request #10170 from beraldoleal/deploy-reset-fix
kata-deploy: fix kata-deploy reset
2024-08-16 16:51:14 +02:00
Beraldo Leal
b843b236e4 kata-deploy: improve kata-deploy script
For the rare cases where containerd_conf_file does not exist, cp could fail
and let the pod in Error state. Let's make it a little bit more robust.

Signed-off-by: Beraldo Leal <bleal@redhat.com>
2024-08-16 07:52:38 -04:00
Fabiano Fidêncio
1f6a8baaf1 Revert "ci: add reset_runtime to cleanup"
This reverts commit 8d9bec2e01, as it
causes issues in the operator and kata-deploy itself, leading to the
node to be NotReady.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-15 16:09:34 +02:00
Steve Horsman
e4c023a9fa Merge pull request #10140 from stevenhorsman/kata-version-in-artefact-version
ci: cache: Include kata version in artefact versions
2024-08-09 11:37:09 +01:00
stevenhorsman
b6a3a3f8fe ci: cache: Include kata version in artefact versions
- At the moment we aren't factoring in the kata version on our caches,
so it means that when we bump this just before release, we don't
rebuilt components that pull in the VERSION content, so the release build
ends up with incorrect versions in it's binaries

Fixes: #10092
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-08-08 14:58:58 +01:00
Zvonko Kaiser
8d9bec2e01 ci: add reset_runtime to cleanup
Adding reset_cleanup to cleanup action so that it is done automatically
without the need to run yet another DS just to reset the runtime.

This is now part of the lifecycle hook when issuing kata-deploy.sh
cleanup

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-08-06 11:57:04 +02:00
Zvonko Kaiser
51690bc157 ci: Use helm to deploy kata-deploy
Rather then modifying the kata-depoy scripts let's use Helm and
create a values.yaml that can be used to render the final templates

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-08-06 11:57:04 +02:00
Zvonko Kaiser
94b3348d3c kata-deploy: Add Helm Chart
For easier handling of kata-deploy we can leverage a Helm chart to get
rid of all the base and overlays for the various components

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-08-06 11:57:04 +02:00
stevenhorsman
66f6ec2919 ci: cache: Pass through RELEASE env
In kata-deploy-binaries.sh we want to understand if we are running
as part of a release, so we need to pass through the RELEASE env
from the workflow, which I missed in
https://github.com/kata-containers/kata-containers/pull/9550

Fixes: #9921
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-07-22 16:39:35 +01:00
Zvonko Kaiser
8eaa2f0dc8 dragonball: Add GPU support
Build a GPU flavoured dragonball kernel

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-07-19 14:48:05 +00:00
Fabiano Fidêncio
b7051890af Merge pull request #9722 from zvonkok/busybox-build
deploy: Add busybox target
2024-07-17 13:47:15 +02:00
Julien Ropé
b83d4e1528 kata-deploy: add storage configuration for cri-o
Make sure that the "skip_mount_home" flag is set in cri-o config.

Fixes: #9878

Signed-off-by: Julien Ropé <jrope@redhat.com>
2024-07-11 10:11:30 +02:00
Gabriela Cervantes
28601b51d2 tools: Remove CI variable in kata deploy in docker script
This PR removes the CI variable in kata deploy in docker script
which was supported it in jenkins environment which is not
longer being supported it.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-07-09 20:04:24 +02:00
Gabriela Cervantes
f2b8c6619d makefile: Remove CI variable from local build makefile
This PR removes the CI variable from the local build makefile as
this was part of the jenkins environment which is not longer supported
it.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-07-09 20:04:24 +02:00