Commit Graph

11543 Commits

Author SHA1 Message Date
Wainer dos Santos Moschetta
c4b0f1f31b ci: k8s: shellcheck fixes to gha-run.sh
Fixed a couple of warns shellcheck emitted and disabled others:
 * SC2154 (var is referenced but not assigned)
 * SC2086 (Double quote to prevent globbing and word splitting)

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit 200e542921)
2023-10-11 10:35:10 +02:00
Wainer dos Santos Moschetta
6fb40ad47d kata-deploy: re-format kata-[deploy|cleanup].yaml
The .tests/integration/kubernetes/gh-run.sh script run `yq write` a
couple of times to edit the kata-[deploy|cleanup].yaml, resulting
on the file being formatted again. This is annoying because leaves
the git tree dirty.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit 4af78be13a)
2023-10-11 10:35:04 +02:00
Wainer dos Santos Moschetta
5cd2e947dc ci: k8s: run_tests() for kcli
The only difference to the other platforms is that it needs to
export KUBECONFIG.

Fixes #7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit d54e6d9cda)
2023-10-11 10:34:53 +02:00
Wainer dos Santos Moschetta
56cebfb485 ci: k8s: add deploy-kata-kcli() to gh-run.sh
The cleanup-kcli() behaves like other deploy kata for
bare-metal (e.g. sev, tdx...etc) except that KUBECONFIG
should be exported.

Fixes #7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit c2ef1f0fb0)
2023-10-11 10:34:46 +02:00
Wainer dos Santos Moschetta
6b76d21568 ci: k8s: add cleanup-kcli() to gha-run.sh
The cleanup-kcli() behaves like other clean up for bare-metal (e.g. sev,
tdx...etc) except that KUBECONFIG should be exported.

Fixes #7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit d2be8eef1a)
2023-10-11 10:34:40 +02:00
Wainer dos Santos Moschetta
308ce26438 ci: k8s: set default image for deploy_kata()
On CI workflows the variables DOCKER_REGISTRY, DOCKER_REPO and
DOCKER_TAG are exported to match the built image. However, when running
the script outside of CI context, a developer might just use the latest
image which in this case will be
`quay.io/kata-containers/kata-deploy-ci:kata-containers-latest`.

Fixes #7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit cbb9aa15b6)
2023-10-11 10:34:33 +02:00
Wainer dos Santos Moschetta
c3b91ed394 ci: k8s: create k8s clusters with kcli
Adapted the gha-run.sh script to create a Kubernetes cluster locally
using the kcli tool.

Use `./gha-run.sh create-cluster-kcli` to create it, and
`./gha-run.sh delete-cluster-kcli` to delete.

Fixes #7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit 89bef7d036)
2023-10-11 10:34:27 +02:00
David Esparza
33791f0944 metrics: stops kata components and k8s deployment when test finishes
This PR adds a trap whenever the scrip exits, it deletes the iperf
k8s deployment and k8s services, and deletes the kata components.

This way, when the script finishes, it verifies that there are
indeed no kata components still running.

Fixes: #8126

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
(cherry picked from commit bba34910df)
2023-10-11 10:34:18 +02:00
Peng Tao
621e6e6d8c gha: combine coco jobs into a single yaml
So that we don't risk exceeding the GHA 20 rerefenced yaml files limit
that easy.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
(cherry picked from commit 954d40cce5)
2023-10-11 10:34:05 +02:00
Peng Tao
fe52c0900c gha: combine basic amd64 jobs into a single yaml
GHA has an undocumented limitation that there can be at most 20
referenced yamls in a single yaml file. We workaround it by combining
multiple jobs into a single yaml file.

Fixes: #8161
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
(cherry picked from commit b60e0a9b57)
2023-10-11 10:33:56 +02:00
Aurélien Bombo
301a7d94e3 gha: ci: Revert tracing test PR to unbreak CI
Revert "Merge pull request #8115 from fidencio/topic/ci-add-tracing-tests"

This unbreaks CI as seen in https://github.com/kata-containers/kata-containers/actions/runs/6434757133

Fixes: #8161

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
(cherry picked from commit e9bd852113)
2023-10-11 10:33:49 +02:00
Fabiano Fidêncio
c1da29b9b1 ci: Port runk tests to this repo
I'm basically moving the runk tests from the tests repo to this one, and
I'm adding the "Signed-off-by:" of every single contributor the tests.

Fixes: #8116

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Chen Yiyang <cyyzero@qq.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit da91c9df88)
2023-10-06 15:27:51 +02:00
Fabiano Fidêncio
63be808730 ci: Add placeholder for runk tests
The runk test has been executed as part of the former "ubuntu" jenkins
CI.

We're porting it to GHA and running it against LTS containerd.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 7f23772763)
2023-10-06 15:27:42 +02:00
Fabiano Fidêncio
6541969a83 ci: Move tracing tests here
I'm basically moving the tracing tests from the tests repo to this one,
and I'm adding the "Signed-off-by:" of every single contributor to the
tests.

Fixes: #8114

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: yaoyinnan <yaoyinnan@foxmail.com>
(cherry picked from commit 9205acc3d2)
2023-10-06 15:27:34 +02:00
Fabiano Fidêncio
5d232c8143 ci: Add placeholder for tracing tests
The tracing tests are currently running as part of the Jenkins CI with
the following setups:
* Container Engines: containerd
* VMMs: QEMU | Cloud Hypervisor
* Snapshotters: overlayfs | devmapper

We'll be restricting those tests to be running on LTS version of
containerd, without devmapper.

As it's known due to our GHA limitation, this is just a placeholder and
the tests will actually be added in the next interations.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 3bb2923e5d)
2023-10-06 15:27:25 +02:00
Fabiano Fidêncio
619ef169fb ci: Create a function to install docker
This will be re-used in other tests as well.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 2c3bf406dc)
2023-10-06 15:27:19 +02:00
David Esparza
16e31dd409 metrics: Use jq tool to pretty-print json metrics output
This PR enables the use of jq pretty-print feature to
improve the formatting of metric results json files.

Fixes: #8081

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
(cherry picked from commit 8c498ef5ee)
2023-10-06 15:27:12 +02:00
David Esparza
1f9a4e908f metrics: Enables FIO test for kata containers
FIO benchmark is enabled to measure IO in Kata
at different latencies using containerd client,
in order to complement the CI metrics testing set.

This PR asl deprecated the previous Fio bench
based on k8s.

Fixes: #8080

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
(cherry picked from commit a2159a6361)
2023-10-06 15:27:02 +02:00
Gabriela Cervantes
fe4f72e0a1 gha: Add containerd stability tests to ci yaml
This PR adds containerd stability tests to ci yaml.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 0f2dc8c675)
2023-10-06 15:26:56 +02:00
Gabriela Cervantes
7963298ba2 gha: Add stability gha run script
This PR adds the stability gha run script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 85d290a048)
2023-10-06 15:26:51 +02:00
Gabriela Cervantes
a4e0929054 gha: Add stability tests workflow for gha
This PR adds the stability test workflow for gha for the kata CI.

Fixes #8107

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 54f0c8f88e)
2023-10-06 15:26:42 +02:00
Fabiano Fidêncio
be3a3c221b gha: arm64: Ensure the builder is arm64-builder
Otherwise we'll use any arm64 machine that's added as a runner, and
whenever new machines are added those may end up being only used for
running some specific set of the tests.

Fixes: #8109

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 119f03de26)
2023-10-06 15:25:43 +02:00
Fabiano Fidêncio
f20164dc75 packaging: tools: Remove set -x leftover
This was used for debugging, and ended up being merged with that.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 560bbffb57)
2023-10-06 15:24:25 +02:00
Fabiano Fidêncio
1941d87b84 packaging: release: Mention newly added images
We've added two new containerd builder images recently, one for the
components under `src/tools` and another one for the Kata Containers
agent.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 18fa483d90)
2023-10-06 15:24:19 +02:00
Fabiano Fidêncio
95da1c71ec packaging: tools: Fix container image env var name
This should be TOOLS_CONTAINER_BUILDER instead of
VIRTIOFSD_CONTAINER_BUILDER.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit ca3b888371)
2023-10-06 15:24:11 +02:00
Fabiano Fidêncio
508016fca1 packaging: Allow passing the TOOLS_CONTAINER_BUILDER
This follows what we've been doing for all the components we're
building, but was missed as part of #8077.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 5ca66795c7)
2023-10-06 15:24:05 +02:00
Fabiano Fidêncio
bb1efe0d46 packaging: stable-3.2: Remove everything related to agent policy
Agent policy is not part of the stable-3.2 branch, and will never be.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-10-06 15:23:57 +02:00
Fabiano Fidêncio
892c9f2f03 gha: Build the kata-agent as part of our workflows
The kata-agent binary won't be released, just built so it can be used,
later on,  as part of our tests and as part of the rootfs build.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 02acef9575)
2023-10-06 15:23:42 +02:00
Fabiano Fidêncio
a586b8c581 packaging: Build the kata-agent
Let's add the needed functions to start building the kata-agent, with or
without the OPA support.

For now this build is not used as part of the rootfs build, but later on
this will (not as part of this series, though).

Fixes: #8099

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 5208386ab1)
2023-10-06 15:20:54 +02:00
Fabiano Fidêncio
766a5fa118 agent: Allow specifying DESTDIR and AGENT_POLICY via env vars
This will help to build the agent binary as part of the kata-deploy
localbuild, as we need to pass the DESTDIR to where the agent will be
installed, and also whether we're building the agent with policy support
enabled or not.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 1727487eef)

 Conflicts:
	src/agent/Makefile

The conflict happened as AGENT_POLICY is not part of the stable-3.2,
thus I've manually removed it.
2023-10-06 15:20:13 +02:00
Fabiano Fidêncio
050a4260b9 packaging: Add get_agent_image_name()
This will be used for building the kata-agent.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 45c1188839)
2023-10-06 15:19:19 +02:00
Fabiano Fidêncio
3770b200a8 gha: Fix k0s deployment
The tests are failing when setting up k0s, and that happens because we
download a kubectl binary matching the kubernetes version k0s is using,
and we do that by:
```
sudo k0s kubectl version --short 2>/dev/null | ...
```

With kubectl 1.28, which is now the default on k0s, `kubectl version
--short` has been removed, leading us to an empty stringm causing then
the error in the CI.

Fixes: #8105

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 70e7ec3e23)
2023-10-06 15:19:04 +02:00
Gabriela Cervantes
cf254bc4ee tests: Add general stability fixes
This PR adds general stability fixes.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 6339605a14)
2023-10-06 15:18:55 +02:00
Gabriela Cervantes
1edf2d9bc1 tests: Add agent stability test
This PR adds the agent stability test to stability test.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit fd19f4082f)
2023-10-06 15:18:50 +02:00
Gabriela Cervantes
a8eec39559 tests: Add cassandra stress in stability tests
This PR adds the cassandra stress at the stability tests.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 215577032f)
2023-10-06 15:18:43 +02:00
Gabriela Cervantes
240c584ae2 tests: Add stressng dockerfile for stability tests
This PR adds the stressng dockerfile for stability tests.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit f2d3ea988d)
2023-10-06 15:18:38 +02:00
Gabriela Cervantes
e95d3b1be5 tests: Add stressor CPU test for stability tests
This PR adds the stressor CPU test for stability tests.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 6493aa309e)
2023-10-06 15:18:32 +02:00
Gabriela Cervantes
4393f553e9 metrics: Add stability test for kata CI
This PR adds the stability test for kata containers repository.

Fixes #8084

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit ef68a3a36b)
2023-10-06 15:18:26 +02:00
Gabriela Cervantes
362adea8cd metrics: Fix general check static warnings
This PR fixes general check static warnings.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit d7def8317a)
2023-10-06 15:18:15 +02:00
Gabriela Cervantes
16c349e76c docs: Update url in kata vra document
This PR updates the url in kata vra document.

Fixes #8065

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 928553d1ba)
2023-10-06 15:18:08 +02:00
Fabiano Fidêncio
5800be5029 ci: Build src/tools components as part of our tests / releases
Build those as part of our CI and release workflows.

Fixes #5520 #5348

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit a6b1f5e21b)
2023-10-06 15:17:43 +02:00
Fabiano Fidêncio
41b509e0a6 kata-deploy: Build components from src/tools
Let's add targets and actually enable users and oursevles to build those
components in the same way we build the rest of the project.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 501a168a81)
2023-10-06 15:17:36 +02:00
Fabiano Fidêncio
a5d7ba6662 static-build: Add scripts to build content from src/tools
As we'd like to ship the content from src/tools, we need to build them
in the very same way we build the other components, and the first step
is providing scripts that can build those inside a container.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 6ef42db5ec)
2023-10-06 15:17:28 +02:00
Fabiano Fidêncio
d503daf75e packaging: Add get_tools_image_name()
This will be used for building all the (rust) components from src/tools.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 4d08ec29bc)
2023-10-06 15:17:22 +02:00
Fabiano Fidêncio
b2e432c024 packaging: Use git abbreviated hash
This will make it easier to build images that rely on several
directories hashes.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 98097c96de)
2023-10-06 15:17:15 +02:00
Gabriela Cervantes
c22fdb46e3 metrics: Increase qemu jitter value
This PR increases qemu jitter value.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 8d66ef5185)
2023-10-06 15:17:07 +02:00
Gabriela Cervantes
8a1af8689b metrics: Increase jitter value for clh
This PR increases jitter value for clh.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 5600e28b54)
2023-10-06 15:16:59 +02:00
Gabriela Cervantes
f3fcf6cbf9 metrics: Add checkmetrics for latency test
This PR adds the checkmetrics for latency test.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 8cb7df1bed)
2023-10-06 15:16:53 +02:00
Gabriela Cervantes
ce03e9f97a metrics: Add qemu latency value limit
This PR adds the qemu latency value limit for kata metrics.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit e90440ae24)
2023-10-06 15:16:47 +02:00
Gabriela Cervantes
cd82a351bd metrics: Add latency value limits for kata CI
This PR adds latency value limits for kata CI.

Fixes #8067

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit a74a8f8a9d)
2023-10-06 15:16:39 +02:00