Commit Graph

1435 Commits

Author SHA1 Message Date
Greg Kurz
77c5db6267 Merge pull request #9637 from ldoktor/selective-ci
CI: Select jobs by touched code
2024-10-04 11:29:05 +02:00
Gabriela Cervantes
d7c2b7d13c osbuilder: Remove duplicated arch variable definition
This PR removes duplicated arch variable definition in the rootfs script
as this variable and its value is already defined at the top of the
script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-03 16:22:27 +00:00
Lukáš Doktor
63b6e8a215 ci: Ensure we check the latest workflow run in gatekeeper
with multiple iterations/reruns we need to use the latest run of each
workflow. For that we can use the "run_id" and only update results of
the same or newer run_ids.

To do that we need to store the "run_id". To avoid adding individual
attributes this commit stores the full job object that contains the
status, conclussion as well as other attributes of the individual jobs,
which might come handy in the future in exchange for slightly bigger
memory overhead (still we only store the latest run of required jobs
only).

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-10-03 09:10:45 +02:00
Lukáš Doktor
2ae090b44b ci: Add extra gatekeeper debug output to stderr
which might be useful to assess the amount of querries.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-10-03 09:08:35 +02:00
Lukáš Doktor
2440a39c50 ci: Check required lables before checking tests in gatekeeper
some tests require certain labels before they are executed. When our PR
is not labeled appropriately the gatekeeper detects skipped required
tests and reports a failure. With this change we add "required-labeles"
to the tests mapping and check the expected labels first informing the
user about the missing labeles before even checking the test statuses.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-10-03 09:08:35 +02:00
Lukáš Doktor
dd2878a9c8 ci: Unify character for separating items
the test names are using `;` and regexps were designed to use `,` but
during development simply joined the expressions by `|`. This should
work but might be confusing so let's go with the semi-colon separator
everywhere.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-10-03 09:08:35 +02:00
Wainer dos Santos Moschetta
fdcfac0641 workflows/gatekeeper: export COMMIT_HASH variable
The Github SHA of triggering PR should be exported in the environment
so that gatekeeper can fetch the right workflows/jobs.

Note: by default github will export GITHUB_SHA in the job's environment
but that value cannot be used if the gatekeeper was triggered from a
pull_request_target event, because the SHA correspond to the push
branch.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-10-03 09:08:35 +02:00
Lukáš Doktor
5c1cea1601 ci: Select jobs by touched code
to allow selective testing as well as selective list of required tests
let's add a mapping of required jobs/tests in "skips.py" and a
"gatekeaper" workflow that will ensure the expected required jobs were
successful. Then we can only mark the "gatekeaper" as the required job
and modify the logic to suit our needs.

Fixes: #9237

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-10-03 09:08:33 +02:00
Steve Horsman
0e0cb24387 Merge pull request #10329 from Bickor/webhook-check
tools.kata-webhook: Specify runtime class using configMap
2024-09-23 09:59:12 +01:00
Aurélien Bombo
78c63c7951 kata-deploy: clean up and fix docs for k0s
* Clarifies instructions for k0s.
* Adds kata-deploy step for each cluster type.
* Removes the old kata-deploy-stable step for vanilla k8s.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2024-09-20 11:59:40 -05:00
Martin
b9d88f74ed tools.kata-webhook: Specify runtime class using configMap
The kata webhook requires a configmap to define what runtime class it
should set for the newly created pods. Additionally, the configmap
allows others to modify the default runtime class name we wish to set
(in case the handler is kata but the name of the runtimeclass is
different).

Finally, this PR changes the webhook-check to compare the runtime of the
newly created pod against the specific runtime class in the configmap,
if said confimap doesn't exist, then it will default to "kata".

Signed-off-by: Martin <mheberling@microsoft.com>
2024-09-19 11:51:38 -07:00
Emanuel Lima
a6ee15c5c7 release: Bump VERSION to 3.9.0
Starting the v3.9.0 release

Signed-off-by: Emanuel Lima <emlima@redhat.com>
2024-09-19 10:14:55 -03:00
stevenhorsman
1abeffdac6 kata-deploy: Switch Kubernetes URL
The payload build is failing with:
```
ERROR: failed to solve: process "/bin/sh -c apk --no-cache add bash curl &&
ARCH=$(uname -m) &&
if [ \"${ARCH}\" = \"x86_64\" ]; then ARCH=amd64; fi &&
if [ \"${ARCH}\" = \"aarch64\" ]; then ARCH=arm64; fi &&
DEBIAN_ARCH=${ARCH} &&
if [ \"${DEBIAN_ARCH}\" = \"ppc64le\" ]; then DEBIAN_ARCH=ppc64el; fi &&
curl -fL --progress-bar -o /usr/bin/kubectl https://storage.googleapis.com/kubernetes-release/release/ \
$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/${ARCH}/kubectl &&
chmod +x /usr/bin/kubectl &&
curl -fL --progress-bar -o /usr/bin/jq https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-linux-${DEBIAN_ARCH} &&
chmod +x /usr/bin/jq &&
mkdir -p ${DESTINATION} &&
tar xvf ${WORKDIR}/${KATA_ARTIFACTS} -C ${DESTINATION} &&
rm -f ${WORKDIR}/${KATA_ARTIFACTS} &&
apk del curl &&
apk --no-cache add py3-pip &&
pip install --no-cache-dir yq==3.2.3" did not complete successfully: exit code: 22
```

Looking into this, the problem is that
https://storage.googleapis.com/kubernetes-release/release/v1.31.1/bin/linux/amd64/kubectl
doesn't exist. The [kubectl install doc](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/#install-kubectl-on-linux)
recommends the `dl.k8s.io` site, so let's switch to this.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-09-17 15:35:42 +01:00
Hyounggyu Choi
72471d1a18 local-build: Fix unbound variable for lib_se.sh
As #10315 introduced an `unbound variable` error, this is a
hot-fix for it.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-09-17 10:01:14 +02:00
Hyounggyu Choi
03cd02a006 Merge pull request #10315 from BbolroC/update-ibm-se-doc
doc: Update how-to-run-kata-containers-with-SE-VMs.md
2024-09-16 15:12:18 +02:00
stevenhorsman
705e469696 rootf: Change initrd alpine mirror
The rootfs-initrd build is failing with:
```
fetch https://mirror.math.princeton.edu/pub/alpinelinux//v3.18/main/aarch64/APKINDEX.tar.gz
6684368:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1914:
ERROR: https://mirror.math.princeton.edu/pub/alpinelinux//v3.18/main: Permission denied
```
so try bumping to a newer version of alpine to see
if that helps the issue

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-09-14 18:47:45 +02:00
Hyounggyu Choi
0aae847ae5 tests: Update secure boot image verification for IBM SE
In the latest `s390-tools`, there has been update on how to
verify a secure boot image. A host key revocation list (CRL),
which was optinoal, now becomes mandatory for verification.
This commit updates the relevant scripts and documentation accordingly.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-09-13 14:14:02 +02:00
Alex Lyn
f31839af63 Merge pull request #10253 from teawater/enable_balloon_f_reporting
Add support of dragonball virtio-balloon free page reporting
2024-09-09 17:37:52 +08:00
Fabiano Fidêncio
d44d66ddf6 kata-deploy: Remove kata-cleanup unneeded vars
As kata-cleanup will only call `reset_runtime()`, there's absolutely no
need to export the other set of environment variables in its yaml file.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-04 19:09:02 +02:00
Hui Zhu
9c1b5238b3 kernel/configs: Add ballon and f_reporting to dragonball-experimental
Add CONFIG_PAGE_REPORTING, CONFIG_BALLOON_COMPACTION and
CONFIG_VIRTIO_BALLOON to dragonball-experimental configs to open
dragonball function and free page reporting function.

Signed-off-by: Hui Zhu <teawater@antgroup.com>
2024-09-04 17:25:30 +08:00
Fabiano Fidêncio
bb9bcd886a kata-deploy: Add reset_cri_runtime()
This will help to avoid code duplication on what's needed on the helm
and non-helm cases.

The reason it's not been added as part of the commit which adds the
post-delete hook is simply for helping the reviewer (as the diff would
be less readable with this change).

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-03 23:08:22 +02:00
Fabiano Fidêncio
64ccb1645d helm: Add a post-delete hook
Instead of using a lifecycle.preStop hook, as done when we're using
using the helm chat, let's add a post-delete hook to take care of
properly cleaning up the node during when uninstalling kata-deploy.

The reason why the lifecyle.preStop hook would never work on our case is
simply because each helm chart operation follows the Kuberentes
"declarative" approach, meaning that an operation won't wait for its
previous operation to successfully finish before being called, leading
to us trying to access content that's defined by our RBAC, in an
operation that was started before our RBAC was deleted, but having the
RBAC being deleted before the operation actually started.

Unfortunately this hook brings in some code duplicatioon, mainly related
to the RBAC parts, but that's not new as the same happens with our
deamonset.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-09-03 23:08:22 +02:00
Fabiano Fidêncio
0cb93ed1bb kata-deploy: helm: Add INSTALLATION_PREFIX option
This will allow users to properly set the INSTALLATION_PREFIX when
deploying Kata Containers.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-02 20:25:22 +02:00
Fabiano Fidêncio
7be77ebee5 kata-deploy: helm: Stop mounting /opt/kata
It's simply easier if we just use /host/opt/kata instead in our scripts,
which will simplify a lot the logic of adding an INSTALLATION_PREFIX
later on.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-02 09:38:51 +02:00
Fabiano Fidêncio
6ce5e62c48 kata-deploy: Add a $dest_dir var
As we build our binaries with the `/opt/kata` prefix, that's the value
of $dest_dir.

Later in thise series it'll become handy, as we'll introduce a way to
install the Kata Containers artefacts in a different location.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-02 09:36:33 +02:00
Fabiano Fidêncio
a9a1345a31 kata-deploy: Print the action the script was invoked with
This increases debuggability.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-08-22 14:32:33 +02:00
Fabiano Fidêncio
ab493b6028 kata-deploy: Move general logic to the correct actions
therwise we may end up running into unexpected issues when calling the
cleanup option, as the same checks would be done, and files could end up
being copied again, overwriting the original content which was backked
up by the install option.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-22 14:32:29 +02:00
Fabiano Fidêncio
6596012956 kata-deploy: Simplify check for runtime
Let's write the runtime check in a shorter and simpler to read form.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-08-22 14:32:02 +02:00
Fabiano Fidêncio
d03b72f19b kata-deploy: Stop linking binaries to /usr/local/bin
Neither CRI-O nor containerd requires that, and removing such symlinks
makes everything less intrusive from our side.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-18 01:25:12 +02:00
Fabiano Fidêncio
c2393dc467 kata-deploy: Use shim's absolute path for crio's runtime_path
This will allow us, in the future, not have to do symlinks here and
there.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-18 01:25:12 +02:00
Fabiano Fidêncio
58623723b1 kata-deploy: Use runtime_path for containerd
It's already being used with CRi-O, let's simplify what we do and also
use this for containerd, which will allow us to do further cleanups in
the coming patches.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-18 01:25:12 +02:00
Fabiano Fidêncio
720edbe3fc Merge pull request #10174 from ChengyuZhu6/install_script
tools: install luks-encrypt-storage script by guest-components
2024-08-16 22:04:56 +02:00
Fabiano Fidêncio
b203f715e5 Merge pull request #10170 from beraldoleal/deploy-reset-fix
kata-deploy: fix kata-deploy reset
2024-08-16 16:51:14 +02:00
Beraldo Leal
b3a4cd1a06 Merge pull request #10172 from deagon/fix-typo
osbuilder: fix typo in ubuntu rootfs depends
2024-08-16 08:01:59 -04:00
Beraldo Leal
b843b236e4 kata-deploy: improve kata-deploy script
For the rare cases where containerd_conf_file does not exist, cp could fail
and let the pod in Error state. Let's make it a little bit more robust.

Signed-off-by: Beraldo Leal <bleal@redhat.com>
2024-08-16 07:52:38 -04:00
ChengyuZhu6
aa31a9d3c4 tools: install luks-encrypt-storage script by guest-components
Install luks-encrypt-storage script by guest-components. So that we can maintain a single source and prevent synchronization issues.

Fixes: #10173 -- part I

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-08-16 16:28:20 +08:00
Chengyu Zhu
ba3c484d12 Merge pull request #9999 from ChengyuZhu6/trusted-storage
Trusted image storage
2024-08-16 15:39:50 +08:00
Fabiano Fidêncio
0f3eb2451e Merge pull request #10169 from fidencio/topic/revert-reset_runtime-to-cleanup
Revert "ci: add reset_runtime to cleanup"
2024-08-16 07:29:58 +02:00
Guoqiang Ding
1d21ff9864 osbuilder: fix typo in ubuntu rootfs depends
Remove the duplicate package "xz-utils".

Signed-off-by: Guoqiang Ding <dgq8211@gmail.com>
2024-08-16 11:33:55 +08:00
Beraldo Leal
74662a0721 Merge pull request #10137 from hex2dec/fix-image-warning
tools: Fix container image build warning
2024-08-15 14:45:41 -04:00
Fabiano Fidêncio
1f6a8baaf1 Revert "ci: add reset_runtime to cleanup"
This reverts commit 8d9bec2e01, as it
causes issues in the operator and kata-deploy itself, leading to the
node to be NotReady.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-15 16:09:34 +02:00
ChengyuZhu6
47be9c7c01 osbuilder:rootfs: install init_trusted_storage script
Install init_trusted_storage script if enable MEASURED_ROOTFS.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
Co-authored-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-08-12 16:36:54 +08:00
Steve Horsman
e4c023a9fa Merge pull request #10140 from stevenhorsman/kata-version-in-artefact-version
ci: cache: Include kata version in artefact versions
2024-08-09 11:37:09 +01:00
stevenhorsman
b6a3a3f8fe ci: cache: Include kata version in artefact versions
- At the moment we aren't factoring in the kata version on our caches,
so it means that when we bump this just before release, we don't
rebuilt components that pull in the VERSION content, so the release build
ends up with incorrect versions in it's binaries

Fixes: #10092
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-08-08 14:58:58 +01:00
Archana Shinde
1012449141 Merge pull request #10129 from hex2dec/qemu-aio-native
tools: Support for building qemu with linux aio
2024-08-07 14:32:52 -07:00
Zhiwei Huang
7270a7ba48 tools: Fix container image build warning
All commands within the Dockerfile should use the same casing
(either upper or lower).[1]

[1]: https://docs.docker.com/reference/build-checks/consistent-instruction-casing/

Signed-off-by: Zhiwei Huang <ai.william@outlook.com>
2024-08-07 15:49:01 +08:00
Dan Mihai
2da77c6979 Merge pull request #10068 from burgerdev/genpolicy-test
genpolicy: add crate-scoped integration test
2024-08-06 16:10:46 -07:00
Zvonko Kaiser
8d9bec2e01 ci: add reset_runtime to cleanup
Adding reset_cleanup to cleanup action so that it is done automatically
without the need to run yet another DS just to reset the runtime.

This is now part of the lifecycle hook when issuing kata-deploy.sh
cleanup

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-08-06 11:57:04 +02:00
Zvonko Kaiser
51690bc157 ci: Use helm to deploy kata-deploy
Rather then modifying the kata-depoy scripts let's use Helm and
create a values.yaml that can be used to render the final templates

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-08-06 11:57:04 +02:00
Zvonko Kaiser
94b3348d3c kata-deploy: Add Helm Chart
For easier handling of kata-deploy we can leverage a Helm chart to get
rid of all the base and overlays for the various components

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-08-06 11:57:04 +02:00