Commit Graph

11559 Commits

Author SHA1 Message Date
Peng Tao
f86bfe0da3 runtime/clh: fix image/initrd annotation handling
We should make sure annotations are preferred over
config options in image and initrd path handling.

Fixes: #7705
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
(cherry picked from commit 9fda7059a5)
Signed-off-by: Greg Kurz <groug@kaod.org>
2023-10-18 11:13:27 +02:00
Peng Tao
59fae423b5 runtime/qemu: fix image/initrd annotation handling
Right now if we configure an image annotation and have a config file
setting initrd, the initrd config would override the image annotation.

Add a helper function ImageOrInitrdAssetPath to make sure annotations
are preferred over config options in image and initrd path handling.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
(cherry picked from commit 1a0092d631)
Signed-off-by: Greg Kurz <groug@kaod.org>
2023-10-18 11:13:24 +02:00
Julien Ropé
ef65c5767f kata-agent: use default filemode for block device when it is set to 0
When the FileMode field for the device is unset (0), use a default value instead
to allow the use of the device from the container.
This behaviour is seen from cri-o typically.

Note: this is what runc is doing, which is why regular containers don't have an
issue. This change makes sure kata behaves the same as runc.

Fixes: #7717

Signed-off-by: Julien Ropé <jrope@redhat.com>
(cherry picked from commit 40914b25d4)
Signed-off-by: Greg Kurz <groug@kaod.org>
2023-10-18 11:12:27 +02:00
Xuewei Niu
93609aa0cd deps: Bump dependent crate versions
This pull request is mainly for updating vm-memory and vmm-sys-util.

The affacted crates include:

- vm-memory: from 0.9.0 to 0.10.0
- vmm-sys-util: from 0.10.0 to 0.11.0
- virtio-queue: from 0.6.0 to 0.7.0
- fuse-backend-rs: from 0.10.4 to 0.10.5
- linux-loader: from 0.6.0 to 0.8.0
- nydus-api: from 0.3.0 to 0.3.1
- nydus-rafs: from 0.3.1 to 0.3.2
- nydus-storage: from 0.6.3 to 0.6.4

Fixes: #0000

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
(cherry picked from commit b23c5ed155)
Signed-off-by: Greg Kurz <groug@kaod.org>
2023-10-18 11:07:12 +02:00
Fabiano Fidêncio
4b542c2fcf
Merge pull request #8022 from fidencio/topic/stable-3.2-backport-everything-tests-related
stable-3.2: Backport everything related to the tests
2023-10-11 16:11:41 +02:00
Gabriela Cervantes
7ff98daecf gha: Add install dependencies for stability tests
This PR adds the install dependencies for stability tests.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit e786b2b019)
2023-10-11 10:36:33 +02:00
Gabriela Cervantes
ef49db59f7 gha: Add general dependencies to stability tests
This PR adds the general dependencies to stability tests.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 84e3d884e4)
2023-10-11 10:36:25 +02:00
Gabriela Cervantes
a818f628d7 tests: Add soak parallel stability test
This PR adds the soak parallel stability test.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit dec3951ca5)
2023-10-11 10:36:19 +02:00
Gabriela Cervantes
602c56c0d7 tests: Enable soak parallel test
This PR enables the soak parallel test for stability test.

Fixes #8153

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 0f04d527d9)
2023-10-11 10:36:12 +02:00
Wainer dos Santos Moschetta
a195539307 ci: k8s: set KUBERNETES default value
The KUBERNETES variable is mostly used by kata-deploy whether to apply
k3s specific deployments or not. It is used to select the type of
kubernetes to be installed (k3s, k0s, rancher...etc) and it is always
set on CI. Running the script locally we want to set a value by default
to avoid `KUBERNETES: unbound variable` errors.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit e669282c25)
2023-10-11 10:36:03 +02:00
Wainer dos Santos Moschetta
c4456c21d9 tests: run k8s-volume on a given node
This test can give false-positive on a multi-node cluster. Changed it to
use the new get_one_kata_node() and the modified exec_host() to run the
setup commands on a given node (that has kata installed) and ensure the
test pod is scheduled at that same node.

Fixes #7619
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit c30c3ff185)
2023-10-11 10:35:58 +02:00
Wainer dos Santos Moschetta
58ad833300 tests: run k8s-file-volume on a given node
This test can give false-positive on a multi-node cluster. Changed it to
use the new get_one_kata_node() and the modified exec_host() to run the
setup commands on a given node (that has kata installed) and ensure the
test pod is scheduled at that same node.

Fixes #7619
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit 666993da8d)
2023-10-11 10:35:52 +02:00
Wainer dos Santos Moschetta
a54bdd00d5 tests: exec_host() now gets the node name
The exec_host() simply fails on cluster with multi-nodes because
`kubectl get node -o name" will return a list o names. Moreover, it will
return control nodes names which usually don't have kata installed.

Fixes #7619
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit 3a00fc9101)
2023-10-11 10:35:43 +02:00
Wainer dos Santos Moschetta
0eaf81c1a2 tests: add get_one_kata_node() to tests_common.sh
The introduced get_one_kata_node() returns the first node that
has the kata-runtime=true label, i.e., supposedly a node with
kata installed.

This is useful for tests that should run on a determined worker
node on a multi-nodes cluster.

Fixes #7619
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit 61c9c17bff)
2023-10-11 10:35:34 +02:00
Wainer dos Santos Moschetta
5f2c7c78ff ci: k8s: set KATA_HYPERVISOR default value
Let KATA_HYPERVISOR be qemu by default in gh-run.sh as this variable
is required to tweak some configurations of kata-deploy.

Fixes #7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit 68f083c4d0)
2023-10-11 10:35:28 +02:00
Wainer dos Santos Moschetta
7fceb21362 ci: k8s: configurable deploy kata timeout
The deploy-kata() of gha-run.sh will wait for 10 minutes for the kata
deploy installation finish. This allow users of the script to overwrite
that value by exporting the KATA_DEPLOY_WAIT_TIMEOUT environment
variable.

Fixes #7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit 6677a61fe4)
2023-10-11 10:35:20 +02:00
Wainer dos Santos Moschetta
c4b0f1f31b ci: k8s: shellcheck fixes to gha-run.sh
Fixed a couple of warns shellcheck emitted and disabled others:
 * SC2154 (var is referenced but not assigned)
 * SC2086 (Double quote to prevent globbing and word splitting)

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit 200e542921)
2023-10-11 10:35:10 +02:00
Wainer dos Santos Moschetta
6fb40ad47d kata-deploy: re-format kata-[deploy|cleanup].yaml
The .tests/integration/kubernetes/gh-run.sh script run `yq write` a
couple of times to edit the kata-[deploy|cleanup].yaml, resulting
on the file being formatted again. This is annoying because leaves
the git tree dirty.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit 4af78be13a)
2023-10-11 10:35:04 +02:00
Wainer dos Santos Moschetta
5cd2e947dc ci: k8s: run_tests() for kcli
The only difference to the other platforms is that it needs to
export KUBECONFIG.

Fixes #7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit d54e6d9cda)
2023-10-11 10:34:53 +02:00
Wainer dos Santos Moschetta
56cebfb485 ci: k8s: add deploy-kata-kcli() to gh-run.sh
The cleanup-kcli() behaves like other deploy kata for
bare-metal (e.g. sev, tdx...etc) except that KUBECONFIG
should be exported.

Fixes #7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit c2ef1f0fb0)
2023-10-11 10:34:46 +02:00
Wainer dos Santos Moschetta
6b76d21568 ci: k8s: add cleanup-kcli() to gha-run.sh
The cleanup-kcli() behaves like other clean up for bare-metal (e.g. sev,
tdx...etc) except that KUBECONFIG should be exported.

Fixes #7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit d2be8eef1a)
2023-10-11 10:34:40 +02:00
Wainer dos Santos Moschetta
308ce26438 ci: k8s: set default image for deploy_kata()
On CI workflows the variables DOCKER_REGISTRY, DOCKER_REPO and
DOCKER_TAG are exported to match the built image. However, when running
the script outside of CI context, a developer might just use the latest
image which in this case will be
`quay.io/kata-containers/kata-deploy-ci:kata-containers-latest`.

Fixes #7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit cbb9aa15b6)
2023-10-11 10:34:33 +02:00
Wainer dos Santos Moschetta
c3b91ed394 ci: k8s: create k8s clusters with kcli
Adapted the gha-run.sh script to create a Kubernetes cluster locally
using the kcli tool.

Use `./gha-run.sh create-cluster-kcli` to create it, and
`./gha-run.sh delete-cluster-kcli` to delete.

Fixes #7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit 89bef7d036)
2023-10-11 10:34:27 +02:00
David Esparza
33791f0944 metrics: stops kata components and k8s deployment when test finishes
This PR adds a trap whenever the scrip exits, it deletes the iperf
k8s deployment and k8s services, and deletes the kata components.

This way, when the script finishes, it verifies that there are
indeed no kata components still running.

Fixes: #8126

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
(cherry picked from commit bba34910df)
2023-10-11 10:34:18 +02:00
Peng Tao
621e6e6d8c gha: combine coco jobs into a single yaml
So that we don't risk exceeding the GHA 20 rerefenced yaml files limit
that easy.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
(cherry picked from commit 954d40cce5)
2023-10-11 10:34:05 +02:00
Peng Tao
fe52c0900c gha: combine basic amd64 jobs into a single yaml
GHA has an undocumented limitation that there can be at most 20
referenced yamls in a single yaml file. We workaround it by combining
multiple jobs into a single yaml file.

Fixes: #8161
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
(cherry picked from commit b60e0a9b57)
2023-10-11 10:33:56 +02:00
Aurélien Bombo
301a7d94e3 gha: ci: Revert tracing test PR to unbreak CI
Revert "Merge pull request #8115 from fidencio/topic/ci-add-tracing-tests"

This unbreaks CI as seen in https://github.com/kata-containers/kata-containers/actions/runs/6434757133

Fixes: #8161

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
(cherry picked from commit e9bd852113)
2023-10-11 10:33:49 +02:00
Fabiano Fidêncio
c1da29b9b1 ci: Port runk tests to this repo
I'm basically moving the runk tests from the tests repo to this one, and
I'm adding the "Signed-off-by:" of every single contributor the tests.

Fixes: #8116

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Chen Yiyang <cyyzero@qq.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit da91c9df88)
2023-10-06 15:27:51 +02:00
Fabiano Fidêncio
63be808730 ci: Add placeholder for runk tests
The runk test has been executed as part of the former "ubuntu" jenkins
CI.

We're porting it to GHA and running it against LTS containerd.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 7f23772763)
2023-10-06 15:27:42 +02:00
Fabiano Fidêncio
6541969a83 ci: Move tracing tests here
I'm basically moving the tracing tests from the tests repo to this one,
and I'm adding the "Signed-off-by:" of every single contributor to the
tests.

Fixes: #8114

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: yaoyinnan <yaoyinnan@foxmail.com>
(cherry picked from commit 9205acc3d2)
2023-10-06 15:27:34 +02:00
Fabiano Fidêncio
5d232c8143 ci: Add placeholder for tracing tests
The tracing tests are currently running as part of the Jenkins CI with
the following setups:
* Container Engines: containerd
* VMMs: QEMU | Cloud Hypervisor
* Snapshotters: overlayfs | devmapper

We'll be restricting those tests to be running on LTS version of
containerd, without devmapper.

As it's known due to our GHA limitation, this is just a placeholder and
the tests will actually be added in the next interations.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 3bb2923e5d)
2023-10-06 15:27:25 +02:00
Fabiano Fidêncio
619ef169fb ci: Create a function to install docker
This will be re-used in other tests as well.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 2c3bf406dc)
2023-10-06 15:27:19 +02:00
David Esparza
16e31dd409 metrics: Use jq tool to pretty-print json metrics output
This PR enables the use of jq pretty-print feature to
improve the formatting of metric results json files.

Fixes: #8081

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
(cherry picked from commit 8c498ef5ee)
2023-10-06 15:27:12 +02:00
David Esparza
1f9a4e908f metrics: Enables FIO test for kata containers
FIO benchmark is enabled to measure IO in Kata
at different latencies using containerd client,
in order to complement the CI metrics testing set.

This PR asl deprecated the previous Fio bench
based on k8s.

Fixes: #8080

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
(cherry picked from commit a2159a6361)
2023-10-06 15:27:02 +02:00
Gabriela Cervantes
fe4f72e0a1 gha: Add containerd stability tests to ci yaml
This PR adds containerd stability tests to ci yaml.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 0f2dc8c675)
2023-10-06 15:26:56 +02:00
Gabriela Cervantes
7963298ba2 gha: Add stability gha run script
This PR adds the stability gha run script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 85d290a048)
2023-10-06 15:26:51 +02:00
Gabriela Cervantes
a4e0929054 gha: Add stability tests workflow for gha
This PR adds the stability test workflow for gha for the kata CI.

Fixes #8107

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 54f0c8f88e)
2023-10-06 15:26:42 +02:00
Fabiano Fidêncio
be3a3c221b gha: arm64: Ensure the builder is arm64-builder
Otherwise we'll use any arm64 machine that's added as a runner, and
whenever new machines are added those may end up being only used for
running some specific set of the tests.

Fixes: #8109

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 119f03de26)
2023-10-06 15:25:43 +02:00
Fabiano Fidêncio
f20164dc75 packaging: tools: Remove set -x leftover
This was used for debugging, and ended up being merged with that.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 560bbffb57)
2023-10-06 15:24:25 +02:00
Fabiano Fidêncio
1941d87b84 packaging: release: Mention newly added images
We've added two new containerd builder images recently, one for the
components under `src/tools` and another one for the Kata Containers
agent.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 18fa483d90)
2023-10-06 15:24:19 +02:00
Fabiano Fidêncio
95da1c71ec packaging: tools: Fix container image env var name
This should be TOOLS_CONTAINER_BUILDER instead of
VIRTIOFSD_CONTAINER_BUILDER.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit ca3b888371)
2023-10-06 15:24:11 +02:00
Fabiano Fidêncio
508016fca1 packaging: Allow passing the TOOLS_CONTAINER_BUILDER
This follows what we've been doing for all the components we're
building, but was missed as part of #8077.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 5ca66795c7)
2023-10-06 15:24:05 +02:00
Fabiano Fidêncio
bb1efe0d46 packaging: stable-3.2: Remove everything related to agent policy
Agent policy is not part of the stable-3.2 branch, and will never be.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-10-06 15:23:57 +02:00
Fabiano Fidêncio
892c9f2f03 gha: Build the kata-agent as part of our workflows
The kata-agent binary won't be released, just built so it can be used,
later on,  as part of our tests and as part of the rootfs build.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 02acef9575)
2023-10-06 15:23:42 +02:00
Fabiano Fidêncio
a586b8c581 packaging: Build the kata-agent
Let's add the needed functions to start building the kata-agent, with or
without the OPA support.

For now this build is not used as part of the rootfs build, but later on
this will (not as part of this series, though).

Fixes: #8099

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 5208386ab1)
2023-10-06 15:20:54 +02:00
Fabiano Fidêncio
766a5fa118 agent: Allow specifying DESTDIR and AGENT_POLICY via env vars
This will help to build the agent binary as part of the kata-deploy
localbuild, as we need to pass the DESTDIR to where the agent will be
installed, and also whether we're building the agent with policy support
enabled or not.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 1727487eef)

 Conflicts:
	src/agent/Makefile

The conflict happened as AGENT_POLICY is not part of the stable-3.2,
thus I've manually removed it.
2023-10-06 15:20:13 +02:00
Fabiano Fidêncio
050a4260b9 packaging: Add get_agent_image_name()
This will be used for building the kata-agent.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 45c1188839)
2023-10-06 15:19:19 +02:00
Fabiano Fidêncio
3770b200a8 gha: Fix k0s deployment
The tests are failing when setting up k0s, and that happens because we
download a kubectl binary matching the kubernetes version k0s is using,
and we do that by:
```
sudo k0s kubectl version --short 2>/dev/null | ...
```

With kubectl 1.28, which is now the default on k0s, `kubectl version
--short` has been removed, leading us to an empty stringm causing then
the error in the CI.

Fixes: #8105

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 70e7ec3e23)
2023-10-06 15:19:04 +02:00
Gabriela Cervantes
cf254bc4ee tests: Add general stability fixes
This PR adds general stability fixes.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 6339605a14)
2023-10-06 15:18:55 +02:00
Gabriela Cervantes
1edf2d9bc1 tests: Add agent stability test
This PR adds the agent stability test to stability test.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit fd19f4082f)
2023-10-06 15:18:50 +02:00