Commit Graph

14760 Commits

Author SHA1 Message Date
Fabiano Fidêncio
829415dfda workflows: Remove the possibility to manually trigger the nightly CI
As a new workflow was added for the cases where developers want to test
their changes in the workflow itself, let's make sure we stop allowing
manual triggers on this workflow, which can lead to a polluted /
misleading weather of the CI.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-23 13:19:45 +02:00
Fabiano Fidêncio
cc093cdfdb workflows: Add a manually trigger "devel" workflow for the CI
This workflow is intended to replace the `workflow_dispatch` trigger
currently present as part of the `ci-nightly.yaml`.

The reasoning behind having this done in this way is because of our good
and old GHA behaviour for `pull_request_target`, which requires a PR to
be merged in order to check the changes in the workflow itself, which
leads to:
* when a change in a workflow is done, developers (should) do:
  * push their branch to the kata-containers repo
  * manually trigger the "nightly" CI in order to ensure the changes
    don't break anything
    * this can result in the "nightly" CI weather being polluted
      * we don't have the guarantee / assurance about the last n nightly
	runs anymore

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-23 13:14:50 +02:00
Fabiano Fidêncio
ca416d8837
Merge pull request #10446 from kata-containers/topic/re-work-shim-v2-build-as-part-of-the-ci-and-release
workflows: Ensure shim-v2 is built as the last asset
2024-10-23 09:27:29 +02:00
Fabiano Fidêncio
c082b99652
Merge pull request #10439 from microsoft/mahuber/azl-cfg-var
tools: Change PACKAGES var for cbl-mariner
2024-10-23 08:39:49 +02:00
Manuel Huber
a730cef9cf tools: Change PACKAGES var for cbl-mariner
Change the PACKAGES variable for the cbl-mariner rootfs-builder
to use the kata-packages-uvm meta package from
packages.microsoft.com to define the set of packages to be
contained in the UVM.
This aligns the UVM build for the Azure Linux distribution
with the UVM build done for the Kata Containers offering on
Azure Kubernetes Services (AKS).

Signed-off-by: Manuel Huber <mahuber@microsoft.com>
2024-10-22 23:11:42 +00:00
Fabiano Fidêncio
67a8665f51 workflows: Ensure shim-v2 is built as the last asset
By doing this we can ensure that whenever the rootfs changes, we'll be
able to get the new root_hash.txt and use it.

This is the very first step to bring the measured rootfs tests back.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-22 14:56:37 +02:00
Greg Kurz
3de6d09a86
Merge pull request #10443 from gkurz/release-3.10.0
release: Bump VERSION to 3.10.0
2024-10-22 14:46:30 +02:00
Greg Kurz
3037303e09 release: Bump VERSION to 3.10.0
Let's start the 3.10.0 release.

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-10-22 11:28:15 +02:00
Fabiano Fidêncio
4c34cfb0ab
Merge pull request #10420 from pmores/add-support-for-virtio-scsi
runtime-rs: support virtio-scsi device in qemu-rs
2024-10-22 11:00:33 +02:00
Pavel Mores
8cdd968092 runtime-rs: support virtio-scsi device in qemu-rs
Semantics are lifted straight out of the go runtime for compatibility.
We introduce DeviceVirtioScsi to represent a virtio-scsi device and
instantiate it if block device driver in the configuration file is set
to virtio-scsi.  We also introduce ObjectIoThread which is instantiated
if the configuration file additionally enables iothreads.

Signed-off-by: Pavel Mores <pmores@redhat.com>
2024-10-22 08:55:54 +02:00
Greg Kurz
91b874f18c
Merge pull request #10421 from Apokleos/hostname-bugfix
kata-agent: fixing bug of unable setting hostname correctly.
2024-10-22 00:26:51 +02:00
alex.lyn
b25538f670 ci: Introduce CI to validate pod hostname
Fixes #10422

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2024-10-21 16:32:56 +01:00
alex.lyn
3dabe0f5f0 kata-agent: fixing bug of unable setting hostname correctly.
When do update_container_namespaces updating namespaces, setting
all UTS(and IPC) namespace paths to None resulted in hostnames
set prior to the update becoming ineffective. This was primarily
due to an error made while aligning with the oci spec: in an attempt
to match empty strings with None values in oci-spec-rs, all paths
were incorrectly set to None.

Fixes #10325

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2024-10-21 16:32:56 +01:00
Steve Horsman
98886a7571
Merge pull request #10437 from mkulke/mkulke/dont-parse-oci-image-for-cached-artifacts
ci: don't parse oci image for cached artifacts
2024-10-21 16:31:23 +01:00
Magnus Kulke
e27d70d47e ci: don't parse oci image for cached artifacts
Moved the parsing of the oci image marker into its own step, since we
only need to perform that for attestation purposes and some cached
images might not have that file in the tarball.

Signed-off-by: Magnus Kulke <magnuskulke@microsoft.com>
2024-10-21 14:50:00 +02:00
Magnus Kulke
9a33a3413b
Merge pull request #10433 from mkulke/mkulke/add-provenance-attestation-for-agent-builds
ci: add provenance attestation for agent artifact
2024-10-18 15:00:18 +02:00
Anastassios Nanos
68d539f5c5
Merge pull request #10435 from nubificus/fix_fc_machineconfig
runtime-rs: Use vCPU and memory values from config
2024-10-18 13:41:20 +01:00
Magnus Kulke
b93f5390ce ci: add provenance attestation for agent artifact
This adds provenance attestation logic for agent binaries that are
published to an oci registry via ORAS.

As a downstream consumer of the kata-agent binary the Peerpod project
needs to verify that the artifact has been built on kata's CI.

To create an attestation we need to know the exact digest of the oci
artifact, at the point when the artifact was pushed.

Therefore we record the full oci image as returned by oras push.

The pushing and tagging logic has been slightly reworked to make this
task less repetetive.

The oras cli accepts multiple tags separated by comma on pushes, so a
push can be performed atomically instead of iterating through tags and
pushing each individually. This removes the risk of partially successful
push operations (think: rate limits on the oci registry).

So far the provenance creation has been only enabled for agent builds on
amd64 and xs390x.

Signed-off-by: Magnus Kulke <magnuskulke@microsoft.com>
2024-10-18 10:24:00 +02:00
Anastassios Nanos
23f5786cca runtime-rs: Use vCPU and memory values from config
Use values from the config for the setup of the microVM.

Fixes: #10434

Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
2024-10-17 23:17:02 +01:00
GabyCT
4ae9317675
Merge pull request #10430 from GabyCT/topic/ciaz
docs: Update CI documentation
2024-10-17 15:09:24 -06:00
GabyCT
b00203ba9b
Merge pull request #10428 from GabyCT/topic/archk8sc
gha: Use a arch_to_golang variable to have uniformity
2024-10-17 11:00:59 -06:00
Chengyu Zhu
cca77f0911
Merge pull request #10412 from stevenhorsman/agent-config-rstest
agent: config: Use rstest for unit tests
2024-10-17 23:01:21 +08:00
Gabriela Cervantes
e3efad8ed2 docs: Update CI documentation
This PR updates the CI documentation referring to the several tests and
in which kind of instances is running them.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-16 19:23:19 +00:00
stevenhorsman
4adb454ed0 agent: config: Use rstest for unit tests
Use rstest for unit test rather than TestData arrays where
possible to make the code more compact, easier to read
and open the possibility to enhance test cases with a
description more easily.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-10-16 16:55:44 +01:00
Gabriela Cervantes
f0e0c74fd4 gha: Use a arch_to_golang variable to have uniformity
This PR replaces the arch uname -m to use the arch_to_golang
variable in the script to have a better uniformity across the script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-15 20:03:09 +00:00
Dan Mihai
69509eff33
Merge pull request #10417 from microsoft/danmihai1/k8s-inotify.bats
tests: k8s-inotify.bats improvements
2024-10-15 11:22:53 -07:00
Dan Mihai
ece0f9690e tests: k8s-inotify: longer pod termination timeout
inotify-configmap-pod.yaml is using: "inotifywait --timeout 120",
so wait for up to 180 seconds for the pod termination to be
reported.

Hopefully, some of the sporadic errors from #10413 will be avoided
this way:

not ok 1 configmap update works, and preserves symlinks
waitForProcess "${wait_time}" "$sleep_time" "${command}" failed

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-10-15 16:01:25 +00:00
Dan Mihai
ccfb7faa1b tests: k8s-inotify.bats: don't leak configmap
Delete the configmap if the test failed, not just on the successful
path.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-10-15 16:01:25 +00:00
Aurélien Bombo
f13d13c8fa
Merge pull request #10416 from microsoft/danmihai1/mariner_static_sandbox_resource_mgmt
ci: static_sandbox_resource_mgmt for cbl-mariner
2024-10-15 10:40:17 -05:00
Aurélien Bombo
c371b4e1ce
Merge pull request #10426 from 3u13r/fix/genpolicy/handle-config-map-binary-data
genpolicy: read binaryData value as String
2024-10-14 21:31:23 -05:00
Leonard Cohnen
c06bf2e3bb genpolicy: read binaryData value as String
While Kubernetes defines `binaryData` as `[]byte`,
when defined in a YAML file the raw bytes are
base64 encoded. Therefore, we need to read the YAML
value as `String` and not as `Vec<u8>`.

Fixes: #10410

Signed-off-by: Leonard Cohnen <lc@edgeless.systems>
2024-10-14 20:03:11 +02:00
Aurélien Bombo
f9b7a8a23c
Merge pull request #10402 from Sumynwa/sumsharma/agent-ctl-dependencies
ci: Install build dependencies for building agent-ctl with image pull.
2024-10-14 10:28:32 -05:00
Sumedh Alok Sharma
bc195d758a ci: Install build dependencies for building agent-ctl with image pull.
Adds dependencies of 'clang' & 'protobuf' to be installed in runners
when building agent-ctl sources having image pull support.

Fixes #10400

Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
2024-10-14 10:36:04 +05:30
Aurélien Bombo
614e21ccfb
Merge pull request #10415 from GabyCT/topic/egreptim
tools/osbuilder/tests: Remove egrep in test images script
2024-10-11 13:47:30 -05:00
Gabriela Cervantes
aae654be80 tools/osbuilder/tests: Remove egrep in test images script
This PR removes egrep command as it has been deprecated and it replaces by
grep in the test images script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-11 17:23:35 +00:00
Dan Mihai
3622b5e8b4 ci: static_sandbox_resource_mgmt for cbl-mariner
Use the configuration used by AKS (static_sandbox_resource_mgmt=true)
for CI testing on Mariner hosts.

Hopefully pod startup will become more predictable on these hosts -
e.g., by avoiding the occasional hotplug timeouts described by #10413.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-10-10 22:17:39 +00:00
Fabiano Fidêncio
02f5fd94bd
Merge pull request #10409 from fidencio/topic/ci-add-ita_image-and-ita_image_tag
kbs: ita: Ensure the proper image / image_tag is used for ITA
2024-10-10 11:46:26 +02:00
Fabiano Fidêncio
cf5d3ed0d4 kbs: ita: Ensure the proper image / image_tag is used for ITA
When dealing with a specific release, it was easier to just do some
adjustments on the image that has to be used for ITA without actually
adding a new entry in the versions.yaml.

However, it's been proven to be more complicated than that when it comes
to dealing with staged images, and we better explicitly add (and
update) those versions altogether to avoid CI issues.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-10 10:01:33 +02:00
Steve Horsman
0c4a7c8771
Merge pull request #10406 from ChengyuZhu6/fix-unit
agent:cdh: fix unit tests about sealed secret
2024-10-10 08:57:28 +01:00
Fabiano Fidêncio
3f7ce1d620
Merge pull request #10401 from stevenhorsman/kbs-deploy-overlays-update
Kbs deploy overlays update
2024-10-10 09:50:19 +02:00
Fabiano Fidêncio
036b04094e
Merge pull request #10397 from fidencio/topic/build-remove-initrd-mariner-target
build: mariner: Remove the ability to build the marine initrd
2024-10-10 09:44:36 +02:00
ChengyuZhu6
65ecac5777 agent:cdh: fix unit tests about sealed secret
The root cause is that the CDH client is a global variable, and unit tests `test_unseal_env` and `test_unseal_file`
share this lock-free global variable, leading to resource contention and destruction.
Merging the two unit tests into one test_sealed_secret will resolve this issue.

Fixes: #10403

Signed-off-by: ChengyuZhu6 <zhucy0405@gmail.com>
2024-10-10 08:38:06 +08:00
ChengyuZhu6
a992feb7f3 Revert "Revert "agent:cdh: unittest for sealed secret as file""
This reverts commit b5142c94b9.

Signed-off-by: ChengyuZhu6 <zhucy0405@gmail.com>
2024-10-10 08:37:06 +08:00
GabyCT
0cda92c6d8
Merge pull request #10407 from GabyCT/topic/fixbuildk
packaging: Remove unused variable in build kernel script
2024-10-09 16:53:45 -06:00
Gabriela Cervantes
616eb8b19b packaging: Remove unused variable in build kernel script
This PR removes an unused variable in the build kernel script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-09 20:02:56 +00:00
Fabiano Fidêncio
652ba30d4a
build: mariner: Remove the ability to build the marine initrd
As mariner has switched to using an image instead of an initrd, let's
just drop the abiliy to build the initrd and avoid keeping something in
the tree that won't be used.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-09 21:42:55 +02:00
Fabiano Fidêncio
59e3ab07e4
Merge pull request #10396 from fidencio/topic/ci-mariner-test-using-mariner-image-instead-of-initrd
ci: mariner: Use the image instead of the initrd
2024-10-09 21:39:44 +02:00
stevenhorsman
b2fb19f8f8 versions: Bump KBS version
Bump to the commit that had the overlays changes we want
to adapt to.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-10-09 17:49:21 +01:00
Fabiano Fidêncio
01a957f7e1
ci: mariner: Stop building mariner initrd
As the mariner image is already in place, and the tests were modified to
use them (as part of this series), let's just stop building it as part
of the CI.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-09 18:23:35 +02:00
Fabiano Fidêncio
091ad2a1b2
ci: mariner: Ensure kernel_params can be set
The reason we're doing this is because mariner image uses, by default,
cgroups default-hierarchy as `unified` (aka, cgroupsv2).

In order to keep the same initrd behaviour for mariner, let's enforce
that `SYSTEMD_CGROUP_ENABLE_LEGACY_FORCE=1
systemd.legacy_systemd_cgroup_controller=yes
systemd.unified_cgroup_hierarchy=0` is passed to the kernel cmdline, at
least for now.

Other tests that are setting `kernel_params` are not running on mariner,
then we're safe taking this path as it's done as part of this PR.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-09 18:23:35 +02:00