Commit Graph

1673 Commits

Author SHA1 Message Date
Fupan Li
2068801b80
Merge pull request #10626 from teawater/ma
Add mem-agent to kata
2024-12-24 14:11:36 +08:00
Guoqiang Ding
75baf75726 packaging: fix the broken usage help
Using the plain usage text instead of the bad variable reference.

Fixes: #9386
Signed-off-by: Guoqiang Ding <dgq8211@gmail.com>
2024-12-20 13:58:40 +08:00
Steve Horsman
99b9ef4e5a
Merge pull request #10675 from stevenhorsman/release-repeat-abort
release: Abort if release version exists
2024-12-19 11:55:44 +00:00
stevenhorsman
e2669d4acc release: Abort if release version exists
In order to check that we don't accidentally overwrite
release artifacts, we should add a check if the release
name already exists and bail if it does.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-12-18 11:04:19 +00:00
Zvonko Kaiser
07d2b00863 qemu: Fix aarch64 build
Building static binaries for aarch64 requires disabling PIE
We get an GOT overflow and the OS libraries are only build with fpic
and not with fPIC which enables unlimited sized GOT tables.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-12-18 03:26:14 +00:00
Zvonko Kaiser
28b57627bd release: Bump version to 3.12.0
Bump VERSION and helm-chart versions

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-12-16 18:41:51 +00:00
Fabiano Fidêncio
7b0c1d0a8c
Merge pull request #10492 from zvonkok/upgrade-qemu-9.1.0
qemu: Upgrade qemu 9.1.2
2024-12-12 08:15:39 +01:00
Zvonko Kaiser
cc9ecedaea qemu: Bump version, new options, add no_patches
We want to have the latest QEMU version available
which is as of this writing v9.1.2

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>

qemu: Add new options for 9.1.2

We need to fence specific options depending on the version
and disable ones that are not needed anymore

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>

qemu: Add no_patches.txt

Since we do not have any patches for this version
let's create the appropriate files.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-12-11 16:32:39 +00:00
Zvonko Kaiser
69ed4bc3b7 qemu: Add depedency
The new QEMU build needs python-tomli, now that we bumped Ubuntu
we can include the needed tomli package

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-12-11 16:32:20 +00:00
Zvonko Kaiser
c82db45eaa qemu: Disable pmem
We're disabling pmem support, it is heavilly broken with
Ubuntu's static build of QEMU and not needed

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-12-11 16:32:19 +00:00
Zvonko Kaiser
a88174e977 qemu: Replace from source build with package
In jammy we have the liburing package available, hence
remove the source build and include the package.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-12-11 16:22:54 +00:00
Zvonko Kaiser
c15f77737a qemu: Bump Ubuntu version in Dockerfile
We need jammy for a new package that is not available in focal

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-12-11 16:22:54 +00:00
Zvonko Kaiser
eef2795226 qemu: Use proper QEMU builder
Do not use hardcoded abs path. Use the deduced rel path.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-12-11 16:22:54 +00:00
Zvonko Kaiser
e604e51b3d qemu: Build as user
We moved all others artifacts to be build as a user,
QEMU should not be the exception

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-12-11 16:22:54 +00:00
Zvonko Kaiser
1d56fd0308 qemu: Remove abs path
We want to stick with the other build scripts and
only use relative paths.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-12-11 16:22:54 +00:00
Fabiano Fidêncio
300a827d03 release: helm: Add the chart as part of the release
So users can simply download the chart and use it accordingly without
the need to download the full repo.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-12-06 11:19:34 +01:00
Hui Zhu
f9c63d20a4 kernel/configs: Add mglru, debugfs and psi to dragonball-experimental
Add mglru, debugfs and psi to dragonball-experimental/mem_agent.conf to
support mem_agent function.

Fixes: #10625

Signed-off-by: Hui Zhu <teawater@antgroup.com>
2024-12-06 09:59:59 +08:00
Fabiano Fidêncio
111082db07
kata-deploy: Add support to multi-installation
This is super useful for development / debugging scenarios, mainly when
dealing with limited hardware availability, as this change allows
multiple people to develop into one single machine, while still using
kata-deploy.

Fixes: #10546

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-12-05 17:42:53 +01:00
Fabiano Fidêncio
0033a0c23a
kata-deploy: Adjust paths for qemu-coco-dev as well
I missed that when working on the INSTALL_PREFIX feature, so adding it
now.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-12-05 17:42:53 +01:00
Fabiano Fidêncio
62b3a07e2f
kata-deploy: helm: Add overlooked INSTALLATION_PREFIX env var
At the same time that INSTALLATION_PREFIX was added, I was working on
the helm changes to properly do the cleanup / deletion when it's
removed.  However, I missed adding the INSTALLATION_PREFIX env var
there. which I'm doing now.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-12-05 17:42:53 +01:00
Steve Horsman
5d96734831
Merge pull request #10572 from ldoktor/gk-stalled-results
ci.gatekeeper: Update existing results
2024-12-04 19:02:14 +00:00
Aurélien Bombo
fb87bf221f ci: Implement build step for CSI driver
This fully implements the compilation step for csi-kata-directvolume.
This component can now be built by the CI running:

 $ cd tools/packaging/kata-deploy/local-build
 $ make csi-kata-directvolume-tarball

A couple notes:

 * When installing the binary, we rename it from directvolplugin to
   csi-kata-directvolume on the fly to make it more readable.
 * We add go to the tools builder Dockerfile to support building this
   tool.
 * I've noticed the file install_libseccomp.sh gets created by the build
   process so I've added it to a .gitignore.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2024-12-03 14:43:36 -06:00
Hyounggyu Choi
9c19d7674a
Merge pull request #10590 from zvonkok/fix-ci
ci: Fix variant for confidential targets
2024-12-02 18:39:52 +01:00
Zvonko Kaiser
20442c0eae ci: Fix variant for confidential targets
The default initrd confidential target will have a
variant=confidential we need to accomodate this
and make sure we also accomodate aaa-xxx-confidential targets.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-12-02 14:21:03 +00:00
Hyounggyu Choi
de3452f8e1 gatekeeper: add run-k8s-tests-on-zvsi(devmapper) to required jobs
As the following CI job has been marked as required:

- kata-containers-ci-on-push / run-k8s-tests-on-zvsi / run-k8s-tests (devmapper, qemu, kubeadm)

we need to add it to the gatekeeper's required job list.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-11-28 12:46:47 +01:00
Fabiano Fidêncio
ca2098f828
build: Allow dummy builds (for when adding a new target)
This will help us to simply allow a new dummy build whenever a new
component is added.

As long as the format `$(call DUMMY,$@)` is followed, we should be good
to go without taking the risk of breaking the CI.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-11-28 11:13:24 +01:00
Aurélien Bombo
16a91fccbe
Merge pull request #10561 from sprt/csi-driver-ci
coco: ci: Lay groundwork for compiling and publishing CSI driver image [1/x]
2024-11-27 10:26:45 -06:00
Fabiano Fidêncio
500508a592 kata-deploy: Use drop-in files whenever it's possible
This will make our lives considerably easier when it comes to cleaning
up content added, while it's also a groundwork needed for having
multiple installations running in parallel.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-11-27 12:27:08 +01:00
Fabiano Fidêncio
6f70ab9169 kata-deploy: Adapt how the containerd version is checked for k0s
Let's actually mount the whole /etc/k0s as /etc/containerd, so we can
easily access the containerd configuration file which has the version in
it, allowing us to parse it instead of just making a guess based on
kubernetes distro being used.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-11-26 16:15:11 +01:00
Fabiano Fidêncio
f5a9aaa100 kata-deploy: Support containerd config version 3
On Ubuntu 24.04, with the distro default containerd, we're already
getting:
```
$ containerd config default | grep "version = "
version = 3
```

With that in mind, let's make sure that we're ready to support this from
the next release.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-11-26 14:01:50 +01:00
Zvonko Kaiser
c3d1b3c5e3
Merge pull request #10464 from zvonkok/nvidia-gpu-rootfs
gpu: NVIDIA GPU initrd/image build
2024-11-25 16:16:42 -05:00
Fabiano Fidêncio
8763a9bc90
Merge pull request #10520 from fidencio/topic/drop-clear-linux-rootfs
osbuilder: Drop Clear Linux
2024-11-25 21:16:03 +01:00
Steve Horsman
322073bea1
Merge pull request #10447 from ldoktor/required-jobs
ci: Required jobs
2024-11-22 09:15:11 +00:00
Lukáš Doktor
e69635b376
ci.gatekeeper: Remove unused variable
this is a left-over from previous way of iterating over jobs.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-11-22 09:27:11 +01:00
Lukáš Doktor
fa7bca4179
ci.gatekeeper: Print the older job id
let's print the also the existing result's id when printing the
information about ignoring older result id to simplify debugging.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-11-22 09:27:11 +01:00
Lukáš Doktor
6c19a067a0
ci.gatekeeper: Update existing results
tha matching run_id means we're dealing with the same job but with
updated results and not with an older job. Update the results in such
case.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-11-22 09:27:09 +01:00
Aurélien Bombo
e43c59a2c6 ci: Add no-op step to compile CSI driver
This adds a no-op build step to compile the CSI driver. The actual compilation
will be implemented in an ulterior PR, so as to ensure we don't break the CI.

Addresses: #10560

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2024-11-21 16:06:55 -06:00
Zvonko Kaiser
0debf77770 gpu: NVIDIA gpu initrd/image build
With each release make sure we ship a GPU enabled rootfs/initrd

Fixes: #6554

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-11-21 18:57:23 +00:00
Jitang Lei
ed4c727c12 osbuilder: Fix build dependency of ubuntu rootfs with Docker
Build ubuntu rootfs with Docker failed with error:
`Unable to find libclang`

Fix this error by adding libclang-dev to the dependency.

Signed-off-by: Jitang Lei <leijitang@outlook.com>
2024-11-21 10:49:27 +08:00
Lukáš Doktor
62e8815a5a
ci: Add documentation to cover mapping format
to help people with adding new entries.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-11-20 17:25:59 +01:00
Lukáš Doktor
64306dc888
ci: Set required-tests according to GH required tests
this should record the current list of required tests from GH.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-11-20 17:25:57 +01:00
Steve Horsman
817438d1f6
Merge pull request #10552 from stevenhorsman/3.11.0-release
release: Bump version to 3.11.0
2024-11-19 09:44:35 +00:00
stevenhorsman
7a8ba14959 release: Bump version to 3.11.0
Bump `VERSION` and helm-chart versions

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-11-18 11:13:15 +00:00
Crypt0s
563a6887e2 kernel: add CONFIG_KEYS=y to enable kernel keyring
KinD checks for the presence of this (and other) kernel configuration
via scripts like
https://blog.hypriot.com/post/verify-kernel-container-compatibility/ or
attempts to directly use /proc/sys/kernel/keys/ without checking to see
if it exists, causing an exit when it does not see it.

Docker/it's consumers apparently expect to be able to use the kernel
keyring and it's associated syscalls from/for containers.

There aren't any known downsides to enabling this except that it would
by definition enable additional syscalls defined in
https://man7.org/linux/man-pages/man7/keyrings.7.html which are
reachable from userspace. This minimally increases the attack surface of
the Kata Kernel, but this attack surface is minimal (especially since
the kernel is most likely being executed by some kind of hypervisor) and
highly restricted compared to the utility of enabling this feature to
get further containerization compatibility.

Signed-off-by: Crypt0s <BryanHalf@gmail.com>
2024-11-15 09:30:06 +01:00
Henrik Schmidt
57a4dbedeb rootfs: Install missing libclang-dev in Ubuntu docker image
Fixes #9444

Signed-off-by: Henrik Schmidt <mrIncompetent@users.noreply.github.com>
2024-11-14 08:48:24 +00:00
Nikos Ch. Papadopoulos
ab80cf8f48 osbuilder: remove redundant env variable
Remove second declaration of GO_HOME in roofs-build ubuntu script.

Signed-off-by: Nikos Ch. Papadopoulos <ncpapad@cslab.ece.ntua.gr>
2024-11-11 19:49:28 +02:00
Fabiano Fidêncio
780b36f477 osbuilder: Drop Clear Linux
The Clear Linux rootfs is not being tested anywhere, and it seems Intel
doesn't have the capacity to review the PRs related to this (combined
with the lack of interested from the rest of the community on reviewing
PRs that are specific to this untested rootfs).

With this in mind, I'm suggesting we drop Clear Linux support and focus
on what we can actually maintain.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-11-11 15:22:55 +01:00
Fabiano Fidêncio
7e6779f3ad
Merge pull request #10488 from fidencio/topic/teach-our-machinery-to-deal-with-rc-kernels
build: kernel: Teach our machinery to deal with -rc kernels
2024-11-05 16:19:57 +01:00
Zvonko Kaiser
a4725034b2
Merge pull request #9480 from zvonkok/build-image-suffix
image: Add suffix to image or initrd depending on the NVIDIA driver version
2024-11-05 09:43:56 -05:00
Fabiano Fidêncio
f7b31ccd6c kernel: bump kata_config_version
Due to the changes done in the previous commits.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-11-05 12:26:57 +01:00
Fabiano Fidêncio
a52ea32b05 build: kernel: Learn how to deal with release candidates
So far we were not prepared to deal with release candidates as those:
* Do not have a sha256sum in the sha256sums provided by the kernel cdn
* Come from a different URL (directly from Linus)
* Have a different suffix (.tar.gz, instead of .tar.xz)

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-11-05 12:26:02 +01:00
Fabiano Fidêncio
9f2d4b2956 build: kernel: Always pass the url to the builder
This doesn't change much on how we're doing things Today, but it
simplifies a lot cases that may be added later on (and will be) like
building -rc kernels.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-11-05 12:26:02 +01:00
Fabiano Fidêncio
ee1a17cffc build: kernel: Take kernel_url into consideration
Let's make sure the kernel_url is actually used whenever it's passed to
the function.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-11-05 12:26:02 +01:00
Fabiano Fidêncio
9a0b501042 build: kernel: Remove tee specific function
As, thankfully, we're relying on upstream kernels for TEEs.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-11-05 12:26:02 +01:00
Fabiano Fidêncio
cc4006297a build: kernel: Pass the yaml base path instead of the version path
By doing this we can ensure this can be re-used, if needed (and it'll be
needed), for also getting the URL.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-11-05 12:26:02 +01:00
Fabiano Fidêncio
7057ff1cd5 build: kernel: Always pass -f to the kernel builder
-f forces the (re)generaton of the config when doing the setup, which
helps a lot on local development whilst not causing any harm in the CI
builds.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-11-05 12:26:02 +01:00
Fabiano Fidêncio
aff3d98ddd builds: ovmf: Workaround Zeex repo becoming private
Let's just do a simple `sed` and **not** use the repo that became
private.

This is not a backport of https://github.com/tianocore/edk2/pull/6402,
but it's a similar approach that allows us to proceed without the need
to pick up a newer version of edk2.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-11-05 11:25:54 +01:00
Zvonko Kaiser
3781526c94 gpu: Add VARIANT to the initrd and image build
We need to know if we're building a nvidia initrd or image
Additionally if we build a regular or confidential VARIANT

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-11-01 18:34:13 +00:00
Zvonko Kaiser
95b69c5732 build: initrd make it coherent to the image build
Add -f for moving the initrd to the correct file path

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-11-01 18:34:13 +00:00
Zvonko Kaiser
3c29c1707d image: Add suffix to image or initrd depending on the NVIDIA driver version
Fixes: #9478

We want to keep track of the driver versions build during initrd/image build so update the artifact_name after the fact.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-11-01 18:34:13 +00:00
Sumedh Alok Sharma
4b7aba5c57 agent-ctl: Add support to test kata-agent's container creation APIs.
This commit introduces changes to enable testing kata-agent's container
APIs of CreateContainer/StartContainer/RemoveContainer. The changeset
include:
- using confidential-containers image-rs crate to pull/unpack/mount a
container image. Currently supports only un-authenicated registry pull
- re-factor api handlers to reduce cmdline complexity and handle
request generation logic in tool
- introduce an OCI config template for container creation
- add test case

Fixes #9707

Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
2024-11-01 22:18:54 +05:30
Sumedh Alok Sharma
0adf7a66c3 ci: Use ubuntu for static building of kata tools.
This commit introduces changes to use ubuntu for statically
building kata tools. In the existing CI setup, the tools
currently build only for x86_64 architecture.

It also fixes the build error seen for agent-ctl PR#10395.

Fixes #10441

Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
2024-10-31 13:19:18 +05:30
Gabriela Cervantes
69ee287e50 packaging: Remove kernel config repo variable as it is unused
This PR removes the kernel config repo variable at the build kernel
script as it is not used.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-29 17:09:52 +00:00
Fabiano Fidêncio
f19c8cbd02
build: cache: Ensure shim-v2-root_hash.txt is in "${workdir}"
All the oras push logic happens from inside `${workdir}`, while the
root_hash.txt extraction and renaming was not taking this into
consideration.

This was not caught during the manually triggered runs as those do not
perform the oras push.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-28 15:17:16 +01:00
Fabiano Fidêncio
d537932e66
build: shim-v2: Ensure MEASURED_ROOTFS is exported
The approach taken for now is to export MEASURED_ROOTFS=yes on the
workflow files for the architectures using confidential stuff, and leave
the "normal" build without having it set (to avoid any change of
expectation on the current bevahiour).

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-28 12:43:53 +01:00
Fabiano Fidêncio
9c8b20b2bf
build: shim-v2: Rebuild if root_hashes do not match
Let's make sure we take the root_hashes into consideration to decide
whether the shim-v2 should or should not be used from the cached
artefacts.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-28 12:43:53 +01:00
Fabiano Fidêncio
9c84998de9
build: cache: Cache root_hash.txt used by the shim-v2
Let's cache the root_hash.txt from the confidential image so we can use
them later on to decide whether there was a rootfs change that would
require shim-v2 to be rebuilt.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-28 12:43:53 +01:00
Fabiano Fidêncio
d2d9792720
build: Don't leave cached component behind if it can't be used
Let's ensure we remove the component and any extra tarball provided by
ORAS in case the cached component cannot be used.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-28 12:43:53 +01:00
stevenhorsman
b22d4429fb release: Bump version to 3.10.1
Fix release to pick up #10463

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-10-25 17:16:09 +01:00
Fabiano Fidêncio
c082b99652
Merge pull request #10439 from microsoft/mahuber/azl-cfg-var
tools: Change PACKAGES var for cbl-mariner
2024-10-23 08:39:49 +02:00
Manuel Huber
a730cef9cf tools: Change PACKAGES var for cbl-mariner
Change the PACKAGES variable for the cbl-mariner rootfs-builder
to use the kata-packages-uvm meta package from
packages.microsoft.com to define the set of packages to be
contained in the UVM.
This aligns the UVM build for the Azure Linux distribution
with the UVM build done for the Kata Containers offering on
Azure Kubernetes Services (AKS).

Signed-off-by: Manuel Huber <mahuber@microsoft.com>
2024-10-22 23:11:42 +00:00
Greg Kurz
3037303e09 release: Bump VERSION to 3.10.0
Let's start the 3.10.0 release.

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-10-22 11:28:15 +02:00
Magnus Kulke
9a33a3413b
Merge pull request #10433 from mkulke/mkulke/add-provenance-attestation-for-agent-builds
ci: add provenance attestation for agent artifact
2024-10-18 15:00:18 +02:00
Magnus Kulke
b93f5390ce ci: add provenance attestation for agent artifact
This adds provenance attestation logic for agent binaries that are
published to an oci registry via ORAS.

As a downstream consumer of the kata-agent binary the Peerpod project
needs to verify that the artifact has been built on kata's CI.

To create an attestation we need to know the exact digest of the oci
artifact, at the point when the artifact was pushed.

Therefore we record the full oci image as returned by oras push.

The pushing and tagging logic has been slightly reworked to make this
task less repetetive.

The oras cli accepts multiple tags separated by comma on pushes, so a
push can be performed atomically instead of iterating through tags and
pushing each individually. This removes the risk of partially successful
push operations (think: rate limits on the oci registry).

So far the provenance creation has been only enabled for agent builds on
amd64 and xs390x.

Signed-off-by: Magnus Kulke <magnuskulke@microsoft.com>
2024-10-18 10:24:00 +02:00
Aurélien Bombo
f13d13c8fa
Merge pull request #10416 from microsoft/danmihai1/mariner_static_sandbox_resource_mgmt
ci: static_sandbox_resource_mgmt for cbl-mariner
2024-10-15 10:40:17 -05:00
Sumedh Alok Sharma
bc195d758a ci: Install build dependencies for building agent-ctl with image pull.
Adds dependencies of 'clang' & 'protobuf' to be installed in runners
when building agent-ctl sources having image pull support.

Fixes #10400

Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
2024-10-14 10:36:04 +05:30
Gabriela Cervantes
aae654be80 tools/osbuilder/tests: Remove egrep in test images script
This PR removes egrep command as it has been deprecated and it replaces by
grep in the test images script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-11 17:23:35 +00:00
Dan Mihai
3622b5e8b4 ci: static_sandbox_resource_mgmt for cbl-mariner
Use the configuration used by AKS (static_sandbox_resource_mgmt=true)
for CI testing on Mariner hosts.

Hopefully pod startup will become more predictable on these hosts -
e.g., by avoiding the occasional hotplug timeouts described by #10413.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-10-10 22:17:39 +00:00
Fabiano Fidêncio
036b04094e
Merge pull request #10397 from fidencio/topic/build-remove-initrd-mariner-target
build: mariner: Remove the ability to build the marine initrd
2024-10-10 09:44:36 +02:00
Gabriela Cervantes
616eb8b19b packaging: Remove unused variable in build kernel script
This PR removes an unused variable in the build kernel script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-09 20:02:56 +00:00
Fabiano Fidêncio
652ba30d4a
build: mariner: Remove the ability to build the marine initrd
As mariner has switched to using an image instead of an initrd, let's
just drop the abiliy to build the initrd and avoid keeping something in
the tree that won't be used.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-09 21:42:55 +02:00
GabyCT
2035d638df
Merge pull request #10388 from GabyCT/topic/testimtemp
tools/osbuilder/tests: Add trap statement in test images script
2024-10-09 09:49:45 -06:00
Gabriela Cervantes
e08749ce58 tools/osbuilder/tests: Add trap statement in test images script
This PR adds the trap statement in the test images script to clean up
tmp files.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-08 19:54:23 +00:00
Fabiano Fidêncio
80196c06ad
Merge pull request #10390 from microsoft/danmihai1/new-rootfs-image-mariner
local-build: add ability to build rootfs-image-mariner
2024-10-08 21:40:43 +02:00
Dan Mihai
63c1f81c23 local-build: add rootfs-image-mariner
Kata CI will start testing the new rootfs-image-mariner instead of the
older rootfs-initrd-mariner image.

The "official" AKS images are moving from a rootfs-initrd-mariner
format to the rootfs-image-mariner format. Making the same change in
Kata CI is useful to keep this testing in sync with the AKS settings.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-10-08 17:15:56 +00:00
GabyCT
7a38cce73c
Merge pull request #10383 from kata-containers/topic/imagevar
image-builder: Remove unused variable
2024-10-08 10:27:03 -06:00
Emanuel Lima
e989e7ee4e
build: Fix RPM build fail due to AGENT_POLICY
By checking for AGENT_POLICY we ensure we only try to read
allow-all.rego if AGENT_POLICY is set to "yes"

Signed-off-by: Emanuel Lima <emlima@redhat.com>
2024-10-07 15:43:23 -03:00
Gabriela Cervantes
4cd737d9fd image-builder: Remove unused variable
This PR removes an unused variable in the image builder script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-04 15:56:28 +00:00
Greg Kurz
77c5db6267
Merge pull request #9637 from ldoktor/selective-ci
CI: Select jobs by touched code
2024-10-04 11:29:05 +02:00
Gabriela Cervantes
d7c2b7d13c osbuilder: Remove duplicated arch variable definition
This PR removes duplicated arch variable definition in the rootfs script
as this variable and its value is already defined at the top of the
script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-03 16:22:27 +00:00
Lukáš Doktor
63b6e8a215
ci: Ensure we check the latest workflow run in gatekeeper
with multiple iterations/reruns we need to use the latest run of each
workflow. For that we can use the "run_id" and only update results of
the same or newer run_ids.

To do that we need to store the "run_id". To avoid adding individual
attributes this commit stores the full job object that contains the
status, conclussion as well as other attributes of the individual jobs,
which might come handy in the future in exchange for slightly bigger
memory overhead (still we only store the latest run of required jobs
only).

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-10-03 09:10:45 +02:00
Lukáš Doktor
2ae090b44b
ci: Add extra gatekeeper debug output to stderr
which might be useful to assess the amount of querries.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-10-03 09:08:35 +02:00
Lukáš Doktor
2440a39c50
ci: Check required lables before checking tests in gatekeeper
some tests require certain labels before they are executed. When our PR
is not labeled appropriately the gatekeeper detects skipped required
tests and reports a failure. With this change we add "required-labeles"
to the tests mapping and check the expected labels first informing the
user about the missing labeles before even checking the test statuses.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-10-03 09:08:35 +02:00
Lukáš Doktor
dd2878a9c8
ci: Unify character for separating items
the test names are using `;` and regexps were designed to use `,` but
during development simply joined the expressions by `|`. This should
work but might be confusing so let's go with the semi-colon separator
everywhere.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-10-03 09:08:35 +02:00
Wainer dos Santos Moschetta
fdcfac0641
workflows/gatekeeper: export COMMIT_HASH variable
The Github SHA of triggering PR should be exported in the environment
so that gatekeeper can fetch the right workflows/jobs.

Note: by default github will export GITHUB_SHA in the job's environment
but that value cannot be used if the gatekeeper was triggered from a
pull_request_target event, because the SHA correspond to the push
branch.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-10-03 09:08:35 +02:00
Lukáš Doktor
5c1cea1601
ci: Select jobs by touched code
to allow selective testing as well as selective list of required tests
let's add a mapping of required jobs/tests in "skips.py" and a
"gatekeaper" workflow that will ensure the expected required jobs were
successful. Then we can only mark the "gatekeaper" as the required job
and modify the logic to suit our needs.

Fixes: #9237

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-10-03 09:08:33 +02:00
Steve Horsman
0e0cb24387
Merge pull request #10329 from Bickor/webhook-check
tools.kata-webhook: Specify runtime class using configMap
2024-09-23 09:59:12 +01:00
Aurélien Bombo
78c63c7951 kata-deploy: clean up and fix docs for k0s
* Clarifies instructions for k0s.
* Adds kata-deploy step for each cluster type.
* Removes the old kata-deploy-stable step for vanilla k8s.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2024-09-20 11:59:40 -05:00
Martin
b9d88f74ed tools.kata-webhook: Specify runtime class using configMap
The kata webhook requires a configmap to define what runtime class it
should set for the newly created pods. Additionally, the configmap
allows others to modify the default runtime class name we wish to set
(in case the handler is kata but the name of the runtimeclass is
different).

Finally, this PR changes the webhook-check to compare the runtime of the
newly created pod against the specific runtime class in the configmap,
if said confimap doesn't exist, then it will default to "kata".

Signed-off-by: Martin <mheberling@microsoft.com>
2024-09-19 11:51:38 -07:00
Emanuel Lima
a6ee15c5c7
release: Bump VERSION to 3.9.0
Starting the v3.9.0 release

Signed-off-by: Emanuel Lima <emlima@redhat.com>
2024-09-19 10:14:55 -03:00
stevenhorsman
1abeffdac6 kata-deploy: Switch Kubernetes URL
The payload build is failing with:
```
ERROR: failed to solve: process "/bin/sh -c apk --no-cache add bash curl &&
ARCH=$(uname -m) &&
if [ \"${ARCH}\" = \"x86_64\" ]; then ARCH=amd64; fi &&
if [ \"${ARCH}\" = \"aarch64\" ]; then ARCH=arm64; fi &&
DEBIAN_ARCH=${ARCH} &&
if [ \"${DEBIAN_ARCH}\" = \"ppc64le\" ]; then DEBIAN_ARCH=ppc64el; fi &&
curl -fL --progress-bar -o /usr/bin/kubectl https://storage.googleapis.com/kubernetes-release/release/ \
$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/${ARCH}/kubectl &&
chmod +x /usr/bin/kubectl &&
curl -fL --progress-bar -o /usr/bin/jq https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-linux-${DEBIAN_ARCH} &&
chmod +x /usr/bin/jq &&
mkdir -p ${DESTINATION} &&
tar xvf ${WORKDIR}/${KATA_ARTIFACTS} -C ${DESTINATION} &&
rm -f ${WORKDIR}/${KATA_ARTIFACTS} &&
apk del curl &&
apk --no-cache add py3-pip &&
pip install --no-cache-dir yq==3.2.3" did not complete successfully: exit code: 22
```

Looking into this, the problem is that
https://storage.googleapis.com/kubernetes-release/release/v1.31.1/bin/linux/amd64/kubectl
doesn't exist. The [kubectl install doc](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/#install-kubectl-on-linux)
recommends the `dl.k8s.io` site, so let's switch to this.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-09-17 15:35:42 +01:00
Hyounggyu Choi
72471d1a18 local-build: Fix unbound variable for lib_se.sh
As #10315 introduced an `unbound variable` error, this is a
hot-fix for it.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-09-17 10:01:14 +02:00
Hyounggyu Choi
03cd02a006
Merge pull request #10315 from BbolroC/update-ibm-se-doc
doc: Update how-to-run-kata-containers-with-SE-VMs.md
2024-09-16 15:12:18 +02:00
stevenhorsman
705e469696
rootf: Change initrd alpine mirror
The rootfs-initrd build is failing with:
```
fetch https://mirror.math.princeton.edu/pub/alpinelinux//v3.18/main/aarch64/APKINDEX.tar.gz
6684368:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1914:
ERROR: https://mirror.math.princeton.edu/pub/alpinelinux//v3.18/main: Permission denied
```
so try bumping to a newer version of alpine to see
if that helps the issue

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-09-14 18:47:45 +02:00
Hyounggyu Choi
0aae847ae5 tests: Update secure boot image verification for IBM SE
In the latest `s390-tools`, there has been update on how to
verify a secure boot image. A host key revocation list (CRL),
which was optinoal, now becomes mandatory for verification.
This commit updates the relevant scripts and documentation accordingly.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-09-13 14:14:02 +02:00
Alex Lyn
f31839af63
Merge pull request #10253 from teawater/enable_balloon_f_reporting
Add support of dragonball virtio-balloon free page reporting
2024-09-09 17:37:52 +08:00
Fabiano Fidêncio
d44d66ddf6 kata-deploy: Remove kata-cleanup unneeded vars
As kata-cleanup will only call `reset_runtime()`, there's absolutely no
need to export the other set of environment variables in its yaml file.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-04 19:09:02 +02:00
Hui Zhu
9c1b5238b3 kernel/configs: Add ballon and f_reporting to dragonball-experimental
Add CONFIG_PAGE_REPORTING, CONFIG_BALLOON_COMPACTION and
CONFIG_VIRTIO_BALLOON to dragonball-experimental configs to open
dragonball function and free page reporting function.

Signed-off-by: Hui Zhu <teawater@antgroup.com>
2024-09-04 17:25:30 +08:00
Fabiano Fidêncio
bb9bcd886a kata-deploy: Add reset_cri_runtime()
This will help to avoid code duplication on what's needed on the helm
and non-helm cases.

The reason it's not been added as part of the commit which adds the
post-delete hook is simply for helping the reviewer (as the diff would
be less readable with this change).

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-03 23:08:22 +02:00
Fabiano Fidêncio
64ccb1645d helm: Add a post-delete hook
Instead of using a lifecycle.preStop hook, as done when we're using
using the helm chat, let's add a post-delete hook to take care of
properly cleaning up the node during when uninstalling kata-deploy.

The reason why the lifecyle.preStop hook would never work on our case is
simply because each helm chart operation follows the Kuberentes
"declarative" approach, meaning that an operation won't wait for its
previous operation to successfully finish before being called, leading
to us trying to access content that's defined by our RBAC, in an
operation that was started before our RBAC was deleted, but having the
RBAC being deleted before the operation actually started.

Unfortunately this hook brings in some code duplicatioon, mainly related
to the RBAC parts, but that's not new as the same happens with our
deamonset.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-09-03 23:08:22 +02:00
Fabiano Fidêncio
0cb93ed1bb
kata-deploy: helm: Add INSTALLATION_PREFIX option
This will allow users to properly set the INSTALLATION_PREFIX when
deploying Kata Containers.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-02 20:25:22 +02:00
Fabiano Fidêncio
7be77ebee5 kata-deploy: helm: Stop mounting /opt/kata
It's simply easier if we just use /host/opt/kata instead in our scripts,
which will simplify a lot the logic of adding an INSTALLATION_PREFIX
later on.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-02 09:38:51 +02:00
Fabiano Fidêncio
6ce5e62c48 kata-deploy: Add a $dest_dir var
As we build our binaries with the `/opt/kata` prefix, that's the value
of $dest_dir.

Later in thise series it'll become handy, as we'll introduce a way to
install the Kata Containers artefacts in a different location.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-09-02 09:36:33 +02:00
Fabiano Fidêncio
a9a1345a31 kata-deploy: Print the action the script was invoked with
This increases debuggability.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-08-22 14:32:33 +02:00
Fabiano Fidêncio
ab493b6028 kata-deploy: Move general logic to the correct actions
therwise we may end up running into unexpected issues when calling the
cleanup option, as the same checks would be done, and files could end up
being copied again, overwriting the original content which was backked
up by the install option.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-22 14:32:29 +02:00
Fabiano Fidêncio
6596012956 kata-deploy: Simplify check for runtime
Let's write the runtime check in a shorter and simpler to read form.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-08-22 14:32:02 +02:00
Fabiano Fidêncio
d03b72f19b
kata-deploy: Stop linking binaries to /usr/local/bin
Neither CRI-O nor containerd requires that, and removing such symlinks
makes everything less intrusive from our side.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-18 01:25:12 +02:00
Fabiano Fidêncio
c2393dc467
kata-deploy: Use shim's absolute path for crio's runtime_path
This will allow us, in the future, not have to do symlinks here and
there.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-18 01:25:12 +02:00
Fabiano Fidêncio
58623723b1
kata-deploy: Use runtime_path for containerd
It's already being used with CRi-O, let's simplify what we do and also
use this for containerd, which will allow us to do further cleanups in
the coming patches.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-18 01:25:12 +02:00
Fabiano Fidêncio
720edbe3fc
Merge pull request #10174 from ChengyuZhu6/install_script
tools: install luks-encrypt-storage script by guest-components
2024-08-16 22:04:56 +02:00
Fabiano Fidêncio
b203f715e5
Merge pull request #10170 from beraldoleal/deploy-reset-fix
kata-deploy: fix kata-deploy reset
2024-08-16 16:51:14 +02:00
Beraldo Leal
b3a4cd1a06
Merge pull request #10172 from deagon/fix-typo
osbuilder: fix typo in ubuntu rootfs depends
2024-08-16 08:01:59 -04:00
Beraldo Leal
b843b236e4 kata-deploy: improve kata-deploy script
For the rare cases where containerd_conf_file does not exist, cp could fail
and let the pod in Error state. Let's make it a little bit more robust.

Signed-off-by: Beraldo Leal <bleal@redhat.com>
2024-08-16 07:52:38 -04:00
ChengyuZhu6
aa31a9d3c4 tools: install luks-encrypt-storage script by guest-components
Install luks-encrypt-storage script by guest-components. So that we can maintain a single source and prevent synchronization issues.

Fixes: #10173 -- part I

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-08-16 16:28:20 +08:00
Chengyu Zhu
ba3c484d12
Merge pull request #9999 from ChengyuZhu6/trusted-storage
Trusted image storage
2024-08-16 15:39:50 +08:00
Fabiano Fidêncio
0f3eb2451e
Merge pull request #10169 from fidencio/topic/revert-reset_runtime-to-cleanup
Revert "ci: add reset_runtime to cleanup"
2024-08-16 07:29:58 +02:00
Guoqiang Ding
1d21ff9864 osbuilder: fix typo in ubuntu rootfs depends
Remove the duplicate package "xz-utils".

Signed-off-by: Guoqiang Ding <dgq8211@gmail.com>
2024-08-16 11:33:55 +08:00
Beraldo Leal
74662a0721
Merge pull request #10137 from hex2dec/fix-image-warning
tools: Fix container image build warning
2024-08-15 14:45:41 -04:00
Fabiano Fidêncio
1f6a8baaf1 Revert "ci: add reset_runtime to cleanup"
This reverts commit 8d9bec2e01, as it
causes issues in the operator and kata-deploy itself, leading to the
node to be NotReady.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-15 16:09:34 +02:00
ChengyuZhu6
47be9c7c01 osbuilder:rootfs: install init_trusted_storage script
Install init_trusted_storage script if enable MEASURED_ROOTFS.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
Co-authored-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2024-08-12 16:36:54 +08:00
Steve Horsman
e4c023a9fa
Merge pull request #10140 from stevenhorsman/kata-version-in-artefact-version
ci: cache: Include kata version in artefact versions
2024-08-09 11:37:09 +01:00
stevenhorsman
b6a3a3f8fe ci: cache: Include kata version in artefact versions
- At the moment we aren't factoring in the kata version on our caches,
so it means that when we bump this just before release, we don't
rebuilt components that pull in the VERSION content, so the release build
ends up with incorrect versions in it's binaries

Fixes: #10092
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-08-08 14:58:58 +01:00
Archana Shinde
1012449141
Merge pull request #10129 from hex2dec/qemu-aio-native
tools: Support for building qemu with linux aio
2024-08-07 14:32:52 -07:00
Zhiwei Huang
7270a7ba48 tools: Fix container image build warning
All commands within the Dockerfile should use the same casing
(either upper or lower).[1]

[1]: https://docs.docker.com/reference/build-checks/consistent-instruction-casing/

Signed-off-by: Zhiwei Huang <ai.william@outlook.com>
2024-08-07 15:49:01 +08:00
Dan Mihai
2da77c6979
Merge pull request #10068 from burgerdev/genpolicy-test
genpolicy: add crate-scoped integration test
2024-08-06 16:10:46 -07:00
Zvonko Kaiser
8d9bec2e01
ci: add reset_runtime to cleanup
Adding reset_cleanup to cleanup action so that it is done automatically
without the need to run yet another DS just to reset the runtime.

This is now part of the lifecycle hook when issuing kata-deploy.sh
cleanup

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-08-06 11:57:04 +02:00
Zvonko Kaiser
51690bc157
ci: Use helm to deploy kata-deploy
Rather then modifying the kata-depoy scripts let's use Helm and
create a values.yaml that can be used to render the final templates

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-08-06 11:57:04 +02:00
Zvonko Kaiser
94b3348d3c
kata-deploy: Add Helm Chart
For easier handling of kata-deploy we can leverage a Helm chart to get
rid of all the base and overlays for the various components

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-08-06 11:57:04 +02:00
Zhiwei Huang
d455883b46 tools: Support for building qemu with linux aio
The kata containers hypervisior qemu configuration supports setting
block_device_aio="native", but the kata static build of qemu does
not add the linux aio feature.

The libaio-dev library is a necessary dependency for building qemu
with linux aio.

Fixes: #10130

Signed-off-by: Zhiwei Huang <ai.william@outlook.com>
2024-08-06 14:30:45 +08:00
Markus Rudy
69535e5458 genpolicy: add crate-scoped integration test
Provides a test runner that generates a policy and validates it
with canned requests. The initial set of test cases is mostly for
illustration and will be expanded incrementally.

In order to enable both cross-compilation on Ubuntu test runners as well
as native compilation on the Alpine tools builder, it is easiest to
switch to the vendored openssl-src variant. This builds OpenSSL from
source, which depends on Perl at build time.

Adding the test to the Makefile makes it execute in CI, on a variety of
architectures. Building on ppc64le requires a newer version of the
libz-ng-sys crate.

Fixes: #10061

Signed-off-by: Markus Rudy <mr@edgeless.systems>
2024-08-05 11:52:01 +02:00
Fabiano Fidêncio
388b5b0e58
Revert "ci: Temporarily remove arm64 builds"
This reverts commit e9710332e7, as there
are now 2 arm64-builders (to be expanded to 4 really soon).

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-02 13:53:50 +02:00
Fabiano Fidêncio
08be9c3601
Revert "ci: Temporarily remove arm64 builds -- part II"
This reverts commit c5dad991ce, as there
are now 2 arm64-builders (to be expanded to 4 really soon).

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-02 13:52:53 +02:00
Fabiano Fidêncio
c5dad991ce
ci: Temporarily remove arm64 builds -- part II
Let's remove what we commented out, as publish manifest complains:
```
Created manifest list quay.io/kata-containers/kata-deploy-ci:kata-containers-latest
./tools/packaging/release/release.sh: line 146: --amend: command not found
```

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-01 20:43:28 +02:00
Fabiano Fidêncio
e9710332e7 ci: Temporarily remove arm64 builds
It's been a reasonable time that we're not able to even build arm64
artefacts.

For now I am removing the builds as it doesn't make sense to keep
running failing builds, and those can be re-enabled once we have arm64
machines plugged in that can be used for building the stuff, and
maintainers for those machines.

The `arm-jetson-xavier-nx-01` is also being removed from the runners.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-08-01 13:30:47 +02:00
Fabiano Fidêncio
5f146e10a1
osbuilder: Add logs for setting up systemd based stuff
This helps us to debug any kind of changes.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-07-26 14:22:45 +02:00
Alex Carter
4a8fb475be
tee: osbuilder: Set /run to use 50% of the image with systemd
Let's ensure at least 50% of the memory is used for /run, as systemd by
default forces it to be 10%, which is way too small even for very small
workloads.

This is only done for the rootfs-confidential image.

Fixes: kata-containers#6775
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
Signed-off-by: Wang, Arron <arron.wang@intel.com>
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.co
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-07-26 14:22:38 +02:00
Zvonko Kaiser
941577ab3b gpu: rootfs ubuntu build expansion
For the GPU build we need go/rust and some other helpers
to build the rootfs.

Always use versions.yaml for the correct and working Rust and golang
version

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-07-23 14:31:35 +00:00
Steve Horsman
d69950e5c6
Merge pull request #10053 from stevenhorsman/release-env-var
ci: cache: Pass through RELEASE env
2024-07-22 21:53:20 +01:00
Dan Mihai
f26d595e5d
Merge pull request #9910 from microsoft/saulparedes/set_policy_rego_via_env
tools: Allow setting policy rego file via
2024-07-22 11:00:30 -07:00
stevenhorsman
66f6ec2919 ci: cache: Pass through RELEASE env
In kata-deploy-binaries.sh we want to understand if we are running
as part of a release, so we need to pass through the RELEASE env
from the workflow, which I missed in
https://github.com/kata-containers/kata-containers/pull/9550

Fixes: #9921
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-07-22 16:39:35 +01:00
Zvonko Kaiser
5765b6e062
Merge pull request #9920 from zvonkok/initrd-builer
gpu: rootfs/initrd build init
2024-07-22 15:06:49 +02:00
Zvonko Kaiser
73bcb09232
Merge pull request #9968 from zvonkok/kernel-gpu-dragonball-6.1.x
dragonball: kernel gpu dragonball 6.1.x
2024-07-22 13:03:14 +02:00
Zvonko Kaiser
3029e6e849 gpu: rootfs/initrd build init
Initramfs expects /init, create symlink only if ${ROOTFS}/init does not exist
Init may be provided by other packages, e.g. systemd or GPU initrd/rootfs

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-07-22 10:19:05 +00:00
Saul Paredes
b7a184a0d8 rootfs: Allow AGENT_POLICY_FILE te be an absolute
path

Don't set AGENT_POLICY_FILE as $script_dir may change

Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
2024-07-21 14:57:41 -07:00
Zvonko Kaiser
8eaa2f0dc8 dragonball: Add GPU support
Build a GPU flavoured dragonball kernel

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-07-19 14:48:05 +00:00
Lei Huang
20f6979d8f build: add kernel config for Nvidia DPU/ConnectX adapter
With Nvidia DPU or ConnectX network adapter, VF can do VFIO passthrough
to guest VM in `guest-kernel` mode. In the guest kernel, the adapter's
driver is required to claim the VFIO device and create network interface.

Signed-off-by: Lei Huang <leih@nvidia.com>
2024-07-18 22:29:16 -07:00
ms-mahuber
ddff762782 tools: Allow setting policy rego file via
environment variable

* Set policy file via env var

* Add restrictive policy file to kata-opa folder

* Change restrictive policy file name

* Change relative default path location

* Add license headers

Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
2024-07-18 15:05:45 -07:00
Fabiano Fidêncio
b7051890af
Merge pull request #9722 from zvonkok/busybox-build
deploy: Add busybox target
2024-07-17 13:47:15 +02:00
Fupan Li
65f2bfb8c4
Merge pull request #9967 from zvonkok/kernel-dragonball-6.1.x
dragonball: kernel dragonball 6.1.x
2024-07-17 14:38:06 +08:00
Xynnn007
1072658219 agent: Enable kata-cc-rustls-tls in image-rs
- Enable the kata-cc-rustls-tls feature in image-rs, so that it
can get resources from the KBS in order to retrieve the registry
credentials.
- Also bump to the latest image-rs to pick up protobuf fixes
- Add libprotobuf-dev dependency to the agent packaging
as it is needed by the new image-rs feature
- Add extra env in the agent make test as the
new version of the anyhow crate has changed the backtrace capture thus unit
tests of kata-agent that compares a raised error with an expected one
would fail. To fix this, we need only panics to have backtraces, thus
set RUST_BACKTRACE=0 for tests due to document
https://docs.rs/anyhow/latest/anyhow/

Fixes #9538

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-07-15 12:00:50 +01:00
Hyounggyu Choi
83b3a681f4
Merge pull request #10010 from BbolroC/osbuilder-bump-fedora-to-40
osbuilder: Bump Fedora to 40
2024-07-15 13:00:28 +02:00
Greg Kurz
203d9e7803
Merge pull request #10000 from littlejawa/kata_deploy_add_storage_config_for_crio
kata-deploy: add storage configuration for cri-o
2024-07-15 12:29:21 +02:00
Hyounggyu Choi
08d2f6bfe4 osbuilder: Bump Fedora to 40
As Fedora 38 has reached EOL, we are encountering 404 errors for s390x, such as:

```
Status code: 404 for https://dl.fedoraproject.org/pub/fedora-secondary/updates/38/Everything/s390x/repodata/repomd.xml
```

Let's bump the OS to the latest version.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-07-15 09:58:54 +02:00
Julien Ropé
b83d4e1528 kata-deploy: add storage configuration for cri-o
Make sure that the "skip_mount_home" flag is set in cri-o config.

Fixes: #9878

Signed-off-by: Julien Ropé <jrope@redhat.com>
2024-07-11 10:11:30 +02:00
Hyounggyu Choi
d94b285189 tests: Enable k8s-confidential-attestation.bats for s390x
For running a KBS with `se-verifier` in service,
specific credentials need to be configured.
(See https://github.com/confidential-containers/trustee/tree/main/attestation-service/verifier/src/se for details.)

This commit introduces two procedures to support IBM SE attestation:

- Prepare required files and directory structure
- Set necessary environment variables for KBS deployment
- Repackage a secure image once the KBS service address is determined

These changes enable `k8s-confidential-attestation.bats` for s390x.

Fixes: #9933

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-07-10 16:18:37 +02:00
Hyounggyu Choi
5d0f74cd70 local-build: Extract build_secure_image() as a separate library
Currently, all functions in `build_se_image.sh` are dedicated to
publishing a payload image. However, `build_secure_image()` is now
also used for repackaging a secure image when a kernel parameter
is reconfigured. This reconfiguration is necessary because the KBS
service address is determined after the initial secure image build.

This commit extracts `build_secure_image()` from `build_se_image.sh`
and creates a separate library, which can be loaded by bats-core.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-07-10 16:18:37 +02:00
Hyounggyu Choi
856a1f72c6 packaging: Set ATTESTER to se-attester for guest components on s390x
This commit allows the guest-components builder to only build se-attester on s390x.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-07-10 16:18:37 +02:00
Gabriela Cervantes
7061272b4e kernel: bump kata config version
This PR bumps the kata config version as the kernel scripts were
modified.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-07-09 20:04:24 +02:00
Gabriela Cervantes
de848c1458 packaging: Remove CI variable from build kernel script
This PR removes the CI variable from build kernel script which
is not longer supported it as this was part of the jenkins
environment.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-07-09 20:04:24 +02:00
Gabriela Cervantes
28601b51d2 tools: Remove CI variable in kata deploy in docker script
This PR removes the CI variable in kata deploy in docker script
which was supported it in jenkins environment which is not
longer being supported it.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-07-09 20:04:24 +02:00
Gabriela Cervantes
f2b8c6619d makefile: Remove CI variable from local build makefile
This PR removes the CI variable from the local build makefile as
this was part of the jenkins environment which is not longer supported
it.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-07-09 20:04:24 +02:00
Gabriela Cervantes
4161fa3792 tools: Remove CI variable in test images script for osbuilder
This PR removes the CI variable in test images script for osbuilder
as this was part of the jenkins environment which is not longer supported
it.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-07-09 20:04:24 +02:00
Greg Kurz
7506d1ec29 tools: Remove CI variable in test config osbuilder script
This PR removes the CI variable in test config osbuilder script
which was supported on the jenkins environment which is not
longer supported it.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
[greg: squash all fixes into a single patch]
Signed-off-by: Greg Kurz <groug@kaod.org>
2024-07-09 20:03:08 +02:00
Gabriela Cervantes
cf2d5ff4c1 scrips: Fix indentation in QAT run script
This PR fixes the indentation of the QAT run script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-07-08 20:23:50 +00:00
Gabriela Cervantes
d53eb61856 QAT: Remove CI variable from QAT run script
This PR removes the CI variable from QAT run script which was used
in the jenkins environment and not longer used.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-07-08 20:16:00 +00:00
Gabriela Cervantes
ff06ef0bbc scripts: Eliminate CI variable as it is not longer used
This PR removes the CI variable which is not longer being used or valid
in the kata containers repository. The CI variable was used when we
were using jenkins and scripts setups which are not longer supported.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-07-08 20:00:30 +00:00
Zvonko Kaiser
7990d3a154 dragonball: Update kata config version
Mandatory update

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-07-04 17:24:16 +00:00
Zvonko Kaiser
26446d1edb dragonball: Update patches
After v5.14 there is no cpu_hotplug_begin function
now cpus_write_lock same for cpu_hotplug_done = cpus_write_unlock

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-07-04 17:23:24 +00:00
Zvonko Kaiser
ad574b7e10 dragonball: Add patches for 6.1.x
Ported the 5.10 patchs to 6.1.x

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-07-04 17:06:39 +00:00
Aurélien Bombo
33d08a8417
Merge pull request #9825 from microsoft/mahuber/main
osbuilder: allow rootfs builds w/o git or version file deps
2024-07-02 09:38:13 -07:00
Amulya Meka
dd12089e0d
Merge pull request #9914 from Amulyam24/qemu-fix
kata-deploy: fix qemu static build on ppc64le
2024-07-02 10:45:03 +05:30
Amulyam24
259ec408b5 kata-deploy: fix qemu static build for v8.2.1 on ppc64le
Do not install the packages librados-dev and librbd-dev as they are not needed for building static qemu.

Add machine option cap-ail-mode-3=off while creating the VM to qemu cmdline.
Fixes: #9893

Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
2024-07-01 14:56:43 +05:30
Manuel Huber
4b2e725d03 rootfs: Install Rust only when necessary
For docker-based builds only install Rust when necessary.
Further, execute the detect Rust version check only when
intending to install Rust.
As of today, this is the case when we intend to build the
agent during rootfs build.

Signed-off-by: Manuel Huber <mahuber@microsoft.com>
2024-06-28 22:19:46 +00:00
Hyounggyu Choi
dd23beeb05 CI: Eliminating dependency on clone_tests_repo()
As part of archiving the tests repo, we are eliminating the dependency on
`clone_tests_repo()`. The scripts using the function is as follows:

- `ci/install_rust.sh`.
- `ci/setup.sh`
- `ci/lib.sh`

This commit removes or replaces the files, and makes an adjustment accordingly.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-06-28 14:52:02 +02:00
Zvonko Kaiser
30ec78b19a rootfs: Fix spurious error
In some DMZ'ed or CI systems the repos are not up to date
and multistrap fails to find the ubuntu-keyring package.
Update the repos to fix this;

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-06-26 11:10:58 +00:00
Hyounggyu Choi
9cb12dfa88 kernel: Add CONFIG_S390_UV_UAPI for s390x
While enabling the attestation for IBM SE, it was observed that
a kernel config `CONFIG_S390_UV_UAPI` is missing.
This config is required to present an ultravisor in the guest VM.
Ths commit adds the missing config.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-06-20 13:15:33 +02:00
Greg Kurz
81972f6ffc
Merge pull request #9149 from ryansavino/upgrade-to-qemu-8.2.1
qemu: upgrade to 8.2.4
2024-06-19 19:10:02 +02:00
Zvonko Kaiser
b1909e940e deploy: Add busybox target
For a minimal initrd/image build we may want to leverage busybox.
This is part number two of the NVIDIA initrd/image build

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-06-18 15:31:00 +00:00
Wainer dos Santos Moschetta
e517167825 kata-deploy: always copy ci/install_yq.sh
To build the build-kata-deploy image, it should be copied ci/install_yq.sh to
tools/packaging/kata-deploy/local-build/dockerbuild as this script will install
yq within the image. Currently, if
tools/packaging/kata-deploy/local-build/dockerbuild/install_yq.sh exists then
make won't copy it again. This can raise problems as, for example, the current
update of yq version (commit c99ba42d) in ci/install_yq.sh won't force the
rebuild of the build-kata-deploy image.

Note: this isn't a problem on a fresh dev or CI environment.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-06-17 12:18:22 -03:00
Manuel Huber
62fd84dfd8 build: allow rootfs builds w/o git or VERSION file deps
We set the VERSION variable consistently across Makefiles to
'unknown'  if the file is empty or not present.
We also use git commands consistently for calculating the COMMIT,
COMMIT_NO variables, not erroring out when building outside of
a git repository.
In create_summary_file we also account for a missing/empty VERSION
file.
This makes e.g. the UVM build process in an environment where we
build outside of git with a minimal/reduced set of files smoother.

Signed-off-by: Manuel Huber <mahuber@microsoft.com>
2024-06-13 22:46:52 +00:00
Ryan Savino
0430794952 qemu: upgrade to 8.2.4
There is a known issue in qemu 7.2.0 that causes kernel-hashes to fail the verification of the launch binaries for the SEV legacy use case.

Upgraded to qemu 8.2.4.
new available features disabled.

Fixes: #9148

Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
2024-06-13 10:19:42 -05:00
Wainer Moschetta
d971e5ae68
Merge pull request #9537 from wainersm/kata-deploy-crio
kata-deploy: configuring CRI-O for guest-pull image pulling
2024-06-12 17:27:00 -03:00
Wainer dos Santos Moschetta
4f398cc969 kata-deploy: add qemu-runtime-rs runtimeClass
Allow kata-deploy to install and configure the qemu-runtime-rs runtimeClass
which ties to qemu hypervisor implementation in rust for the runtime-rs.

Fixes: #9804
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-06-11 12:58:47 -03:00
Amulya Meka
b323afeda9
Merge pull request #9214 from Amulyam24/oras
kata-deploy: install oras using release artefacts on ppc64le
2024-06-05 11:40:55 +05:30
Wainer dos Santos Moschetta
af4f9afb71 kata-deploy: add PULL_TYPE handler for CRI-O
A new PULL_TYPE environment variable is recognized by the kata-deploy's
install script to allow it to configure CRIO-O for guest-pull image pulling
type.

The tests/integration/kubernetes/gha-run.sh change allows for testing it:
```
export PULL_TYPE=guest-pull
cd tests/integration/kubernetes
./gha-run.sh deploy-k8s
```

Fixes #9474
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-06-04 14:02:01 -03:00
Zvonko Kaiser
647560539f
Merge pull request #9769 from zvonkok/initrd-image-no-sudo
ci: remove sudo and make sure artifacts is owned by user
2024-06-04 07:16:51 +02:00
Zvonko Kaiser
a48c084e13 ci: remove sudo and make sure image is owed by user
The image build needs special handling since we're doing a lot of
privileged operations.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-06-03 15:29:06 +00:00
Beraldo Leal
d3a5eb299a tools: bumping kernel config version
Lets make ci happy.

Signed-off-by: Beraldo Leal <bleal@redhat.com>
2024-05-31 13:28:34 -04:00
Beraldo Leal
c99ba42d62 deps: bumping yq to v4.40.7
Since yq frequently updates, let's upgrade to a version from February to
bypass potential issues with versions 4.41-4.43 for now. We can always
upgrade to the newest version if necessary.

Fixes #9354
Depends-on:github.com/kata-containers/tests#5818

Signed-off-by: Beraldo Leal <bleal@redhat.com>
2024-05-31 13:28:34 -04:00
Amulyam24
eadcb868f4 kata-deploy: install oras using release artefacts on ppc64le
We are currently building Oras from source on ppc64le. Now that they offically release the artefacts
for power, consume them to install Oras.

Fixes: #9213

Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
2024-05-31 14:16:14 +05:30