Commit Graph

1423 Commits

Author SHA1 Message Date
stevenhorsman
1abeffdac6 kata-deploy: Switch Kubernetes URL
The payload build is failing with:
```
ERROR: failed to solve: process "/bin/sh -c apk --no-cache add bash curl &&
ARCH=$(uname -m) &&
if [ \"${ARCH}\" = \"x86_64\" ]; then ARCH=amd64; fi &&
if [ \"${ARCH}\" = \"aarch64\" ]; then ARCH=arm64; fi &&
DEBIAN_ARCH=${ARCH} &&
if [ \"${DEBIAN_ARCH}\" = \"ppc64le\" ]; then DEBIAN_ARCH=ppc64el; fi &&
curl -fL --progress-bar -o /usr/bin/kubectl https://storage.googleapis.com/kubernetes-release/release/ \
$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/${ARCH}/kubectl &&
chmod +x /usr/bin/kubectl &&
curl -fL --progress-bar -o /usr/bin/jq https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-linux-${DEBIAN_ARCH} &&
chmod +x /usr/bin/jq &&
mkdir -p ${DESTINATION} &&
tar xvf ${WORKDIR}/${KATA_ARTIFACTS} -C ${DESTINATION} &&
rm -f ${WORKDIR}/${KATA_ARTIFACTS} &&
apk del curl &&
apk --no-cache add py3-pip &&
pip install --no-cache-dir yq==3.2.3" did not complete successfully: exit code: 22
```

Looking into this, the problem is that
https://storage.googleapis.com/kubernetes-release/release/v1.31.1/bin/linux/amd64/kubectl
doesn't exist. The [kubectl install doc](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/#install-kubectl-on-linux)
recommends the `dl.k8s.io` site, so let's switch to this.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-09-17 15:35:42 +01:00
Hyounggyu Choi
72471d1a18 local-build: Fix unbound variable for lib_se.sh
As #10315 introduced an `unbound variable` error, this is a
hot-fix for it.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-09-17 10:01:14 +02:00
Hyounggyu Choi
03cd02a006
Merge pull request #10315 from BbolroC/update-ibm-se-doc
doc: Update how-to-run-kata-containers-with-SE-VMs.md
2024-09-16 15:12:18 +02:00
stevenhorsman
705e469696
rootf: Change initrd alpine mirror
The rootfs-initrd build is failing with:
```
fetch https://mirror.math.princeton.edu/pub/alpinelinux//v3.18/main/aarch64/APKINDEX.tar.gz
6684368:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1914:
ERROR: https://mirror.math.princeton.edu/pub/alpinelinux//v3.18/main: Permission denied
```
so try bumping to a newer version of alpine to see
if that helps the issue

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-09-14 18:47:45 +02:00
Hyounggyu Choi
0aae847ae5 tests: Update secure boot image verification for IBM SE
In the latest `s390-tools`, there has been update on how to
verify a secure boot image. A host key revocation list (CRL),
which was optinoal, now becomes mandatory for verification.
This commit updates the relevant scripts and documentation accordingly.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-09-13 14:14:02 +02:00
Alex Lyn
f31839af63
Merge pull request #10253 from teawater/enable_balloon_f_reporting
Add support of dragonball virtio-balloon free page reporting
2024-09-09 17:37:52 +08:00
Fabiano Fidêncio
d44d66ddf6 kata-deploy: Remove kata-cleanup unneeded vars
As kata-cleanup will only call `reset_runtime()`, there's absolutely no
need to export the other set of environment variables in its yaml file.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-04 19:09:02 +02:00
Hui Zhu
9c1b5238b3 kernel/configs: Add ballon and f_reporting to dragonball-experimental
Add CONFIG_PAGE_REPORTING, CONFIG_BALLOON_COMPACTION and
CONFIG_VIRTIO_BALLOON to dragonball-experimental configs to open
dragonball function and free page reporting function.

Signed-off-by: Hui Zhu <teawater@antgroup.com>
2024-09-04 17:25:30 +08:00
Fabiano Fidêncio
bb9bcd886a kata-deploy: Add reset_cri_runtime()
This will help to avoid code duplication on what's needed on the helm
and non-helm cases.

The reason it's not been added as part of the commit which adds the
post-delete hook is simply for helping the reviewer (as the diff would
be less readable with this change).

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-03 23:08:22 +02:00
Fabiano Fidêncio
64ccb1645d helm: Add a post-delete hook
Instead of using a lifecycle.preStop hook, as done when we're using
using the helm chat, let's add a post-delete hook to take care of
properly cleaning up the node during when uninstalling kata-deploy.

The reason why the lifecyle.preStop hook would never work on our case is
simply because each helm chart operation follows the Kuberentes
"declarative" approach, meaning that an operation won't wait for its
previous operation to successfully finish before being called, leading
to us trying to access content that's defined by our RBAC, in an
operation that was started before our RBAC was deleted, but having the
RBAC being deleted before the operation actually started.

Unfortunately this hook brings in some code duplicatioon, mainly related
to the RBAC parts, but that's not new as the same happens with our
deamonset.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-09-03 23:08:22 +02:00
Fabiano Fidêncio
0cb93ed1bb
kata-deploy: helm: Add INSTALLATION_PREFIX option
This will allow users to properly set the INSTALLATION_PREFIX when
deploying Kata Containers.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-02 20:25:22 +02:00
Fabiano Fidêncio
7be77ebee5 kata-deploy: helm: Stop mounting /opt/kata
It's simply easier if we just use /host/opt/kata instead in our scripts,
which will simplify a lot the logic of adding an INSTALLATION_PREFIX
later on.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-02 09:38:51 +02:00
Fabiano Fidêncio
6ce5e62c48 kata-deploy: Add a $dest_dir var
As we build our binaries with the `/opt/kata` prefix, that's the value
of $dest_dir.

Later in thise series it'll become handy, as we'll introduce a way to
install the Kata Containers artefacts in a different location.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-02 09:36:33 +02:00
Fabiano Fidêncio
a9a1345a31 kata-deploy: Print the action the script was invoked with
This increases debuggability.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-08-22 14:32:33 +02:00
Fabiano Fidêncio
ab493b6028 kata-deploy: Move general logic to the correct actions
therwise we may end up running into unexpected issues when calling the
cleanup option, as the same checks would be done, and files could end up
being copied again, overwriting the original content which was backked
up by the install option.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-22 14:32:29 +02:00
Fabiano Fidêncio
6596012956 kata-deploy: Simplify check for runtime
Let's write the runtime check in a shorter and simpler to read form.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-08-22 14:32:02 +02:00
Fabiano Fidêncio
d03b72f19b
kata-deploy: Stop linking binaries to /usr/local/bin
Neither CRI-O nor containerd requires that, and removing such symlinks
makes everything less intrusive from our side.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-18 01:25:12 +02:00
Fabiano Fidêncio
c2393dc467
kata-deploy: Use shim's absolute path for crio's runtime_path
This will allow us, in the future, not have to do symlinks here and
there.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-18 01:25:12 +02:00
Fabiano Fidêncio
58623723b1
kata-deploy: Use runtime_path for containerd
It's already being used with CRi-O, let's simplify what we do and also
use this for containerd, which will allow us to do further cleanups in
the coming patches.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-18 01:25:12 +02:00
Fabiano Fidêncio
720edbe3fc
Merge pull request #10174 from ChengyuZhu6/install_script
tools: install luks-encrypt-storage script by guest-components
2024-08-16 22:04:56 +02:00
Fabiano Fidêncio
b203f715e5
Merge pull request #10170 from beraldoleal/deploy-reset-fix
kata-deploy: fix kata-deploy reset
2024-08-16 16:51:14 +02:00
Beraldo Leal
b3a4cd1a06
Merge pull request #10172 from deagon/fix-typo
osbuilder: fix typo in ubuntu rootfs depends
2024-08-16 08:01:59 -04:00
Beraldo Leal
b843b236e4 kata-deploy: improve kata-deploy script
For the rare cases where containerd_conf_file does not exist, cp could fail
and let the pod in Error state. Let's make it a little bit more robust.

Signed-off-by: Beraldo Leal <bleal@redhat.com>
2024-08-16 07:52:38 -04:00
ChengyuZhu6
aa31a9d3c4 tools: install luks-encrypt-storage script by guest-components
Install luks-encrypt-storage script by guest-components. So that we can maintain a single source and prevent synchronization issues.

Fixes: #10173 -- part I

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-08-16 16:28:20 +08:00
Chengyu Zhu
ba3c484d12
Merge pull request #9999 from ChengyuZhu6/trusted-storage
Trusted image storage
2024-08-16 15:39:50 +08:00
Fabiano Fidêncio
0f3eb2451e
Merge pull request #10169 from fidencio/topic/revert-reset_runtime-to-cleanup
Revert "ci: add reset_runtime to cleanup"
2024-08-16 07:29:58 +02:00
Guoqiang Ding
1d21ff9864 osbuilder: fix typo in ubuntu rootfs depends
Remove the duplicate package "xz-utils".

Signed-off-by: Guoqiang Ding <dgq8211@gmail.com>
2024-08-16 11:33:55 +08:00
Beraldo Leal
74662a0721
Merge pull request #10137 from hex2dec/fix-image-warning
tools: Fix container image build warning
2024-08-15 14:45:41 -04:00
Fabiano Fidêncio
1f6a8baaf1 Revert "ci: add reset_runtime to cleanup"
This reverts commit 8d9bec2e01, as it
causes issues in the operator and kata-deploy itself, leading to the
node to be NotReady.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-15 16:09:34 +02:00
ChengyuZhu6
47be9c7c01 osbuilder:rootfs: install init_trusted_storage script
Install init_trusted_storage script if enable MEASURED_ROOTFS.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
Co-authored-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-08-12 16:36:54 +08:00
Steve Horsman
e4c023a9fa
Merge pull request #10140 from stevenhorsman/kata-version-in-artefact-version
ci: cache: Include kata version in artefact versions
2024-08-09 11:37:09 +01:00
stevenhorsman
b6a3a3f8fe ci: cache: Include kata version in artefact versions
- At the moment we aren't factoring in the kata version on our caches,
so it means that when we bump this just before release, we don't
rebuilt components that pull in the VERSION content, so the release build
ends up with incorrect versions in it's binaries

Fixes: #10092
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-08-08 14:58:58 +01:00
Archana Shinde
1012449141
Merge pull request #10129 from hex2dec/qemu-aio-native
tools: Support for building qemu with linux aio
2024-08-07 14:32:52 -07:00
Zhiwei Huang
7270a7ba48 tools: Fix container image build warning
All commands within the Dockerfile should use the same casing
(either upper or lower).[1]

[1]: https://docs.docker.com/reference/build-checks/consistent-instruction-casing/

Signed-off-by: Zhiwei Huang <ai.william@outlook.com>
2024-08-07 15:49:01 +08:00
Dan Mihai
2da77c6979
Merge pull request #10068 from burgerdev/genpolicy-test
genpolicy: add crate-scoped integration test
2024-08-06 16:10:46 -07:00
Zvonko Kaiser
8d9bec2e01
ci: add reset_runtime to cleanup
Adding reset_cleanup to cleanup action so that it is done automatically
without the need to run yet another DS just to reset the runtime.

This is now part of the lifecycle hook when issuing kata-deploy.sh
cleanup

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-08-06 11:57:04 +02:00
Zvonko Kaiser
51690bc157
ci: Use helm to deploy kata-deploy
Rather then modifying the kata-depoy scripts let's use Helm and
create a values.yaml that can be used to render the final templates

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-08-06 11:57:04 +02:00
Zvonko Kaiser
94b3348d3c
kata-deploy: Add Helm Chart
For easier handling of kata-deploy we can leverage a Helm chart to get
rid of all the base and overlays for the various components

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-08-06 11:57:04 +02:00
Zhiwei Huang
d455883b46 tools: Support for building qemu with linux aio
The kata containers hypervisior qemu configuration supports setting
block_device_aio="native", but the kata static build of qemu does
not add the linux aio feature.

The libaio-dev library is a necessary dependency for building qemu
with linux aio.

Fixes: #10130

Signed-off-by: Zhiwei Huang <ai.william@outlook.com>
2024-08-06 14:30:45 +08:00
Markus Rudy
69535e5458 genpolicy: add crate-scoped integration test
Provides a test runner that generates a policy and validates it
with canned requests. The initial set of test cases is mostly for
illustration and will be expanded incrementally.

In order to enable both cross-compilation on Ubuntu test runners as well
as native compilation on the Alpine tools builder, it is easiest to
switch to the vendored openssl-src variant. This builds OpenSSL from
source, which depends on Perl at build time.

Adding the test to the Makefile makes it execute in CI, on a variety of
architectures. Building on ppc64le requires a newer version of the
libz-ng-sys crate.

Fixes: #10061

Signed-off-by: Markus Rudy <mr@edgeless.systems>
2024-08-05 11:52:01 +02:00
Fabiano Fidêncio
388b5b0e58
Revert "ci: Temporarily remove arm64 builds"
This reverts commit e9710332e7, as there
are now 2 arm64-builders (to be expanded to 4 really soon).

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-02 13:53:50 +02:00
Fabiano Fidêncio
08be9c3601
Revert "ci: Temporarily remove arm64 builds -- part II"
This reverts commit c5dad991ce, as there
are now 2 arm64-builders (to be expanded to 4 really soon).

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-02 13:52:53 +02:00
Fabiano Fidêncio
c5dad991ce
ci: Temporarily remove arm64 builds -- part II
Let's remove what we commented out, as publish manifest complains:
```
Created manifest list quay.io/kata-containers/kata-deploy-ci:kata-containers-latest
./tools/packaging/release/release.sh: line 146: --amend: command not found
```

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-01 20:43:28 +02:00
Fabiano Fidêncio
e9710332e7 ci: Temporarily remove arm64 builds
It's been a reasonable time that we're not able to even build arm64
artefacts.

For now I am removing the builds as it doesn't make sense to keep
running failing builds, and those can be re-enabled once we have arm64
machines plugged in that can be used for building the stuff, and
maintainers for those machines.

The `arm-jetson-xavier-nx-01` is also being removed from the runners.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-01 13:30:47 +02:00
Fabiano Fidêncio
5f146e10a1
osbuilder: Add logs for setting up systemd based stuff
This helps us to debug any kind of changes.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-07-26 14:22:45 +02:00
Alex Carter
4a8fb475be
tee: osbuilder: Set /run to use 50% of the image with systemd
Let's ensure at least 50% of the memory is used for /run, as systemd by
default forces it to be 10%, which is way too small even for very small
workloads.

This is only done for the rootfs-confidential image.

Fixes: kata-containers#6775
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
Signed-off-by: Wang, Arron <arron.wang@intel.com>
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.co
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-07-26 14:22:38 +02:00
Zvonko Kaiser
941577ab3b gpu: rootfs ubuntu build expansion
For the GPU build we need go/rust and some other helpers
to build the rootfs.

Always use versions.yaml for the correct and working Rust and golang
version

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-07-23 14:31:35 +00:00
Steve Horsman
d69950e5c6
Merge pull request #10053 from stevenhorsman/release-env-var
ci: cache: Pass through RELEASE env
2024-07-22 21:53:20 +01:00
Dan Mihai
f26d595e5d
Merge pull request #9910 from microsoft/saulparedes/set_policy_rego_via_env
tools: Allow setting policy rego file via
2024-07-22 11:00:30 -07:00
stevenhorsman
66f6ec2919 ci: cache: Pass through RELEASE env
In kata-deploy-binaries.sh we want to understand if we are running
as part of a release, so we need to pass through the RELEASE env
from the workflow, which I missed in
https://github.com/kata-containers/kata-containers/pull/9550

Fixes: #9921
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-07-22 16:39:35 +01:00