Commit Graph

1801 Commits

Author SHA1 Message Date
Magnus Kulke
9a33a3413b Merge pull request #10433 from mkulke/mkulke/add-provenance-attestation-for-agent-builds
ci: add provenance attestation for agent artifact
2024-10-18 15:00:18 +02:00
Magnus Kulke
b93f5390ce ci: add provenance attestation for agent artifact
This adds provenance attestation logic for agent binaries that are
published to an oci registry via ORAS.

As a downstream consumer of the kata-agent binary the Peerpod project
needs to verify that the artifact has been built on kata's CI.

To create an attestation we need to know the exact digest of the oci
artifact, at the point when the artifact was pushed.

Therefore we record the full oci image as returned by oras push.

The pushing and tagging logic has been slightly reworked to make this
task less repetetive.

The oras cli accepts multiple tags separated by comma on pushes, so a
push can be performed atomically instead of iterating through tags and
pushing each individually. This removes the risk of partially successful
push operations (think: rate limits on the oci registry).

So far the provenance creation has been only enabled for agent builds on
amd64 and xs390x.

Signed-off-by: Magnus Kulke <magnuskulke@microsoft.com>
2024-10-18 10:24:00 +02:00
Aurélien Bombo
f13d13c8fa Merge pull request #10416 from microsoft/danmihai1/mariner_static_sandbox_resource_mgmt
ci: static_sandbox_resource_mgmt for cbl-mariner
2024-10-15 10:40:17 -05:00
Sumedh Alok Sharma
bc195d758a ci: Install build dependencies for building agent-ctl with image pull.
Adds dependencies of 'clang' & 'protobuf' to be installed in runners
when building agent-ctl sources having image pull support.

Fixes #10400

Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
2024-10-14 10:36:04 +05:30
Gabriela Cervantes
aae654be80 tools/osbuilder/tests: Remove egrep in test images script
This PR removes egrep command as it has been deprecated and it replaces by
grep in the test images script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-11 17:23:35 +00:00
Dan Mihai
3622b5e8b4 ci: static_sandbox_resource_mgmt for cbl-mariner
Use the configuration used by AKS (static_sandbox_resource_mgmt=true)
for CI testing on Mariner hosts.

Hopefully pod startup will become more predictable on these hosts -
e.g., by avoiding the occasional hotplug timeouts described by #10413.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-10-10 22:17:39 +00:00
Fabiano Fidêncio
036b04094e Merge pull request #10397 from fidencio/topic/build-remove-initrd-mariner-target
build: mariner: Remove the ability to build the marine initrd
2024-10-10 09:44:36 +02:00
Gabriela Cervantes
616eb8b19b packaging: Remove unused variable in build kernel script
This PR removes an unused variable in the build kernel script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-09 20:02:56 +00:00
Fabiano Fidêncio
652ba30d4a build: mariner: Remove the ability to build the marine initrd
As mariner has switched to using an image instead of an initrd, let's
just drop the abiliy to build the initrd and avoid keeping something in
the tree that won't be used.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-09 21:42:55 +02:00
GabyCT
2035d638df Merge pull request #10388 from GabyCT/topic/testimtemp
tools/osbuilder/tests: Add trap statement in test images script
2024-10-09 09:49:45 -06:00
Gabriela Cervantes
e08749ce58 tools/osbuilder/tests: Add trap statement in test images script
This PR adds the trap statement in the test images script to clean up
tmp files.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-08 19:54:23 +00:00
Fabiano Fidêncio
80196c06ad Merge pull request #10390 from microsoft/danmihai1/new-rootfs-image-mariner
local-build: add ability to build rootfs-image-mariner
2024-10-08 21:40:43 +02:00
Dan Mihai
63c1f81c23 local-build: add rootfs-image-mariner
Kata CI will start testing the new rootfs-image-mariner instead of the
older rootfs-initrd-mariner image.

The "official" AKS images are moving from a rootfs-initrd-mariner
format to the rootfs-image-mariner format. Making the same change in
Kata CI is useful to keep this testing in sync with the AKS settings.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-10-08 17:15:56 +00:00
GabyCT
7a38cce73c Merge pull request #10383 from kata-containers/topic/imagevar
image-builder: Remove unused variable
2024-10-08 10:27:03 -06:00
Emanuel Lima
e989e7ee4e build: Fix RPM build fail due to AGENT_POLICY
By checking for AGENT_POLICY we ensure we only try to read
allow-all.rego if AGENT_POLICY is set to "yes"

Signed-off-by: Emanuel Lima <emlima@redhat.com>
2024-10-07 15:43:23 -03:00
Gabriela Cervantes
4cd737d9fd image-builder: Remove unused variable
This PR removes an unused variable in the image builder script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-04 15:56:28 +00:00
Greg Kurz
77c5db6267 Merge pull request #9637 from ldoktor/selective-ci
CI: Select jobs by touched code
2024-10-04 11:29:05 +02:00
Gabriela Cervantes
d7c2b7d13c osbuilder: Remove duplicated arch variable definition
This PR removes duplicated arch variable definition in the rootfs script
as this variable and its value is already defined at the top of the
script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-03 16:22:27 +00:00
Lukáš Doktor
63b6e8a215 ci: Ensure we check the latest workflow run in gatekeeper
with multiple iterations/reruns we need to use the latest run of each
workflow. For that we can use the "run_id" and only update results of
the same or newer run_ids.

To do that we need to store the "run_id". To avoid adding individual
attributes this commit stores the full job object that contains the
status, conclussion as well as other attributes of the individual jobs,
which might come handy in the future in exchange for slightly bigger
memory overhead (still we only store the latest run of required jobs
only).

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-10-03 09:10:45 +02:00
Lukáš Doktor
2ae090b44b ci: Add extra gatekeeper debug output to stderr
which might be useful to assess the amount of querries.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-10-03 09:08:35 +02:00
Lukáš Doktor
2440a39c50 ci: Check required lables before checking tests in gatekeeper
some tests require certain labels before they are executed. When our PR
is not labeled appropriately the gatekeeper detects skipped required
tests and reports a failure. With this change we add "required-labeles"
to the tests mapping and check the expected labels first informing the
user about the missing labeles before even checking the test statuses.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-10-03 09:08:35 +02:00
Lukáš Doktor
dd2878a9c8 ci: Unify character for separating items
the test names are using `;` and regexps were designed to use `,` but
during development simply joined the expressions by `|`. This should
work but might be confusing so let's go with the semi-colon separator
everywhere.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-10-03 09:08:35 +02:00
Wainer dos Santos Moschetta
fdcfac0641 workflows/gatekeeper: export COMMIT_HASH variable
The Github SHA of triggering PR should be exported in the environment
so that gatekeeper can fetch the right workflows/jobs.

Note: by default github will export GITHUB_SHA in the job's environment
but that value cannot be used if the gatekeeper was triggered from a
pull_request_target event, because the SHA correspond to the push
branch.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-10-03 09:08:35 +02:00
Lukáš Doktor
5c1cea1601 ci: Select jobs by touched code
to allow selective testing as well as selective list of required tests
let's add a mapping of required jobs/tests in "skips.py" and a
"gatekeaper" workflow that will ensure the expected required jobs were
successful. Then we can only mark the "gatekeaper" as the required job
and modify the logic to suit our needs.

Fixes: #9237

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-10-03 09:08:33 +02:00
Steve Horsman
0e0cb24387 Merge pull request #10329 from Bickor/webhook-check
tools.kata-webhook: Specify runtime class using configMap
2024-09-23 09:59:12 +01:00
Aurélien Bombo
78c63c7951 kata-deploy: clean up and fix docs for k0s
* Clarifies instructions for k0s.
* Adds kata-deploy step for each cluster type.
* Removes the old kata-deploy-stable step for vanilla k8s.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2024-09-20 11:59:40 -05:00
Martin
b9d88f74ed tools.kata-webhook: Specify runtime class using configMap
The kata webhook requires a configmap to define what runtime class it
should set for the newly created pods. Additionally, the configmap
allows others to modify the default runtime class name we wish to set
(in case the handler is kata but the name of the runtimeclass is
different).

Finally, this PR changes the webhook-check to compare the runtime of the
newly created pod against the specific runtime class in the configmap,
if said confimap doesn't exist, then it will default to "kata".

Signed-off-by: Martin <mheberling@microsoft.com>
2024-09-19 11:51:38 -07:00
Emanuel Lima
a6ee15c5c7 release: Bump VERSION to 3.9.0
Starting the v3.9.0 release

Signed-off-by: Emanuel Lima <emlima@redhat.com>
2024-09-19 10:14:55 -03:00
stevenhorsman
1abeffdac6 kata-deploy: Switch Kubernetes URL
The payload build is failing with:
```
ERROR: failed to solve: process "/bin/sh -c apk --no-cache add bash curl &&
ARCH=$(uname -m) &&
if [ \"${ARCH}\" = \"x86_64\" ]; then ARCH=amd64; fi &&
if [ \"${ARCH}\" = \"aarch64\" ]; then ARCH=arm64; fi &&
DEBIAN_ARCH=${ARCH} &&
if [ \"${DEBIAN_ARCH}\" = \"ppc64le\" ]; then DEBIAN_ARCH=ppc64el; fi &&
curl -fL --progress-bar -o /usr/bin/kubectl https://storage.googleapis.com/kubernetes-release/release/ \
$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/${ARCH}/kubectl &&
chmod +x /usr/bin/kubectl &&
curl -fL --progress-bar -o /usr/bin/jq https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-linux-${DEBIAN_ARCH} &&
chmod +x /usr/bin/jq &&
mkdir -p ${DESTINATION} &&
tar xvf ${WORKDIR}/${KATA_ARTIFACTS} -C ${DESTINATION} &&
rm -f ${WORKDIR}/${KATA_ARTIFACTS} &&
apk del curl &&
apk --no-cache add py3-pip &&
pip install --no-cache-dir yq==3.2.3" did not complete successfully: exit code: 22
```

Looking into this, the problem is that
https://storage.googleapis.com/kubernetes-release/release/v1.31.1/bin/linux/amd64/kubectl
doesn't exist. The [kubectl install doc](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/#install-kubectl-on-linux)
recommends the `dl.k8s.io` site, so let's switch to this.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-09-17 15:35:42 +01:00
Hyounggyu Choi
72471d1a18 local-build: Fix unbound variable for lib_se.sh
As #10315 introduced an `unbound variable` error, this is a
hot-fix for it.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-09-17 10:01:14 +02:00
Hyounggyu Choi
03cd02a006 Merge pull request #10315 from BbolroC/update-ibm-se-doc
doc: Update how-to-run-kata-containers-with-SE-VMs.md
2024-09-16 15:12:18 +02:00
stevenhorsman
705e469696 rootf: Change initrd alpine mirror
The rootfs-initrd build is failing with:
```
fetch https://mirror.math.princeton.edu/pub/alpinelinux//v3.18/main/aarch64/APKINDEX.tar.gz
6684368:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1914:
ERROR: https://mirror.math.princeton.edu/pub/alpinelinux//v3.18/main: Permission denied
```
so try bumping to a newer version of alpine to see
if that helps the issue

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-09-14 18:47:45 +02:00
Hyounggyu Choi
0aae847ae5 tests: Update secure boot image verification for IBM SE
In the latest `s390-tools`, there has been update on how to
verify a secure boot image. A host key revocation list (CRL),
which was optinoal, now becomes mandatory for verification.
This commit updates the relevant scripts and documentation accordingly.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-09-13 14:14:02 +02:00
Alex Lyn
f31839af63 Merge pull request #10253 from teawater/enable_balloon_f_reporting
Add support of dragonball virtio-balloon free page reporting
2024-09-09 17:37:52 +08:00
Fabiano Fidêncio
d44d66ddf6 kata-deploy: Remove kata-cleanup unneeded vars
As kata-cleanup will only call `reset_runtime()`, there's absolutely no
need to export the other set of environment variables in its yaml file.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-04 19:09:02 +02:00
Hui Zhu
9c1b5238b3 kernel/configs: Add ballon and f_reporting to dragonball-experimental
Add CONFIG_PAGE_REPORTING, CONFIG_BALLOON_COMPACTION and
CONFIG_VIRTIO_BALLOON to dragonball-experimental configs to open
dragonball function and free page reporting function.

Signed-off-by: Hui Zhu <teawater@antgroup.com>
2024-09-04 17:25:30 +08:00
Fabiano Fidêncio
bb9bcd886a kata-deploy: Add reset_cri_runtime()
This will help to avoid code duplication on what's needed on the helm
and non-helm cases.

The reason it's not been added as part of the commit which adds the
post-delete hook is simply for helping the reviewer (as the diff would
be less readable with this change).

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-03 23:08:22 +02:00
Fabiano Fidêncio
64ccb1645d helm: Add a post-delete hook
Instead of using a lifecycle.preStop hook, as done when we're using
using the helm chat, let's add a post-delete hook to take care of
properly cleaning up the node during when uninstalling kata-deploy.

The reason why the lifecyle.preStop hook would never work on our case is
simply because each helm chart operation follows the Kuberentes
"declarative" approach, meaning that an operation won't wait for its
previous operation to successfully finish before being called, leading
to us trying to access content that's defined by our RBAC, in an
operation that was started before our RBAC was deleted, but having the
RBAC being deleted before the operation actually started.

Unfortunately this hook brings in some code duplicatioon, mainly related
to the RBAC parts, but that's not new as the same happens with our
deamonset.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-09-03 23:08:22 +02:00
Fabiano Fidêncio
0cb93ed1bb kata-deploy: helm: Add INSTALLATION_PREFIX option
This will allow users to properly set the INSTALLATION_PREFIX when
deploying Kata Containers.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-02 20:25:22 +02:00
Fabiano Fidêncio
7be77ebee5 kata-deploy: helm: Stop mounting /opt/kata
It's simply easier if we just use /host/opt/kata instead in our scripts,
which will simplify a lot the logic of adding an INSTALLATION_PREFIX
later on.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-02 09:38:51 +02:00
Fabiano Fidêncio
6ce5e62c48 kata-deploy: Add a $dest_dir var
As we build our binaries with the `/opt/kata` prefix, that's the value
of $dest_dir.

Later in thise series it'll become handy, as we'll introduce a way to
install the Kata Containers artefacts in a different location.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-02 09:36:33 +02:00
Fabiano Fidêncio
a9a1345a31 kata-deploy: Print the action the script was invoked with
This increases debuggability.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-08-22 14:32:33 +02:00
Fabiano Fidêncio
ab493b6028 kata-deploy: Move general logic to the correct actions
therwise we may end up running into unexpected issues when calling the
cleanup option, as the same checks would be done, and files could end up
being copied again, overwriting the original content which was backked
up by the install option.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-22 14:32:29 +02:00
Fabiano Fidêncio
6596012956 kata-deploy: Simplify check for runtime
Let's write the runtime check in a shorter and simpler to read form.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-08-22 14:32:02 +02:00
Fabiano Fidêncio
d03b72f19b kata-deploy: Stop linking binaries to /usr/local/bin
Neither CRI-O nor containerd requires that, and removing such symlinks
makes everything less intrusive from our side.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-18 01:25:12 +02:00
Fabiano Fidêncio
c2393dc467 kata-deploy: Use shim's absolute path for crio's runtime_path
This will allow us, in the future, not have to do symlinks here and
there.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-18 01:25:12 +02:00
Fabiano Fidêncio
58623723b1 kata-deploy: Use runtime_path for containerd
It's already being used with CRi-O, let's simplify what we do and also
use this for containerd, which will allow us to do further cleanups in
the coming patches.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-18 01:25:12 +02:00
Fabiano Fidêncio
720edbe3fc Merge pull request #10174 from ChengyuZhu6/install_script
tools: install luks-encrypt-storage script by guest-components
2024-08-16 22:04:56 +02:00
Fabiano Fidêncio
b203f715e5 Merge pull request #10170 from beraldoleal/deploy-reset-fix
kata-deploy: fix kata-deploy reset
2024-08-16 16:51:14 +02:00
Beraldo Leal
b3a4cd1a06 Merge pull request #10172 from deagon/fix-typo
osbuilder: fix typo in ubuntu rootfs depends
2024-08-16 08:01:59 -04:00