Commit Graph

1573 Commits

Author SHA1 Message Date
Wainer Moschetta
38088a934b Merge pull request #9184 from wainersm/fix_kata_deploy_bats
tests/kata-deploy: fix checker for kata-deploy running
2024-03-04 20:50:37 -03:00
GabyCT
77d048da4d Merge pull request #9065 from wainersm/ci_install_kbs
CI: Install KBS on k8s for attestation tests
2024-03-04 16:59:01 -06:00
Gabriela Cervantes
5d50262422 docs: Add general tests documentation in main README
This PR adds the general tests documentation in main README of the
kata containers repository.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-03-04 21:53:01 +00:00
Gabriela Cervantes
d5fa2bebd5 docs: Add general README for tests section
This PR adds general README documentation for the tests section
in the kata containers repository.

Fixes #9209

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-03-04 21:50:37 +00:00
GabyCT
4dea9019ab Merge pull request #9126 from GabyCT/topic/addartifactsk
gha: Storing artifacts for logs of k8s tests garm
2024-03-04 15:41:54 -06:00
Gabriela Cervantes
fc5e040d96 scripts: Apply general fixes to variables in gha-run script
This PR applies general fixes to variables in gha-run script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-03-04 18:54:15 +00:00
Fupan Li
07e0cf1855 CI: fix the issue of ci failure on crio
PR #8760 tentatively tried to have the shim to run in its own mount
namespace for the sake of improving isolation between the sandbox and
the host. Thus crio storage drivers shouldn't create a PRIVATE
bind mount on their home directory. Otherwise, the container's rootfs
mount wouldn't be propagated to kata runtime's mount namespace, and
kata runtime couldn't access the container's rootfs files.

So, when kata cooperated with crio, crio should set
skip_mount_home=true for its storage overlay.

Fixes: #9028

Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
2024-03-03 20:53:36 +08:00
Wainer dos Santos Moschetta
2c24977cb1 tests/k8s: allow to overwrite the cluster name
_print_cluster_name() create a string based information like the
pull request number and commit SHA. However, when you are developing the
scripts you might want to use an arbitrary name, so it was introduced
the $AKS_NAME variable that once exported it will overwrite the
generated name.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-02 12:42:35 -03:00
Wainer dos Santos Moschetta
5e4b7bbd04 tests/k8s: expose KBS service externally
Until this point the deployed KBS service is only reachable from within
the cluster. This introduces a generic mechanism to apply an Ingress
configuration to expose the service externally.

The first implemened ingress is for AKS. In case the HTTP application
routing isn't enabled in the cluster (this is required for ingress), an
add-on is applied.

It was added the get_cluster_specific_dns_zone() and
enable_cluster_http_application_routing() helper functions
to gha-run-k8s-common.sh.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-02 12:42:35 -03:00
Wainer dos Santos Moschetta
e1e0b94975 tests/k8s: introduce the CoCo kbs library
Introduce the tests/integration/kubernetes/confidential_kbs.sh library
that contains functions to manage the KBS on CI. Initially implemented
the kbs_k8s_deploy() and kbs_k8s_delete() functions to, respectively,
deploy and delete KBS on Kubernetes. Also hooked those functions in the
tests/integration/kubernetes/gha-run.sh script to follow the convention
of running commands from Github Workflows:

$ .tests/integration/kubernetes/gha-run.sh deploy-coco-kbs
$ .tests/integration/kubernetes/gha-run.sh delete-coco-kbs

Fixes #9058
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-02 12:39:26 -03:00
Wainer dos Santos Moschetta
6a28c94d99 tests/k8s: add a kustomize installer
Kustomize has been used on some of our internal components (e.g.
kata-deploy) to manage k8s deployments. On CI it has been used
the `sed` tool to edit kustomization.yaml files, but `kustomize` is
more suitable for that purpose. So in order to use that tool on CI
scripts in the future, this commit introduces the `install_kustomize()`
function that is going to download and install the binary in
/usr/local/bin in case it's found on $PATH.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-02 12:39:26 -03:00
GabyCT
4a0cfc4e3f Merge pull request #9199 from GabyCT/topic/enablecri
gha: Enable cri-containerd tests for cloud hypervisor runtime-rs
2024-03-01 12:23:16 -06:00
Gabriela Cervantes
7299dbdb43 gha: Store journalctl logs
This PR stores the journalctl logs.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-03-01 15:17:20 +00:00
Gabriela Cervantes
342d3a320d gha: Add collect artifacts function in gha-run script
This PR adds the collect artifacts function in gha-run script for
the kubernetes tests.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-03-01 15:17:20 +00:00
Greg Kurz
dc6bda19bf Merge pull request #9179 from gkurz/fix-k8s-sandbox-vcpus-allocation-check
tests: k8s: Adapt k8s-sandbox-vcpus-allocation.bats to kubernetes v1.29
2024-03-01 15:55:07 +01:00
Wainer dos Santos Moschetta
24c163e6e1 tests/kata-deploy: fix checker for kata-deploy running
Currently, the checking for kata-deploy is running assume that the
daemonset scheduled at least one pod, however it might not had and the
kubectl wait command fails due to "error: no matching resources found".

On CI I've observed that fail intermittently. I suspect the service
account kata-deploy-sa take a while to show up then no kata-deploy is
scheduled in meanwhile.

Changed the checker logic to use waitForProcess() to keep testing if it is
already running, or hit the timeout (still 10m).

Fixes #9183
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-02-29 22:26:27 -03:00
Gabriela Cervantes
beb592b309 gha: Enable cri-containerd tests for cloud hypervisor runtime-rs
This PR enables the cri-containerd tests for cloud hypervisor runtime-rs.

Fixes #9198

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-29 20:18:16 +00:00
Gabriela Cervantes
0f595cf15b gha: General variable fixes to gha-run script
This PR adds general variable fixes to gha-run script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-29 18:15:27 +00:00
Greg Kurz
f3442cdef9 tests: k8s: Adapt k8s-sandbox-vcpus-allocation.bats to kubernetes v1.29
Kubernetes v1.29 introduced a new `PodReadyToStartContainers` condition
that gets inserted at index 0 in the conditions array. This means that
the expected `PodCompleted` reason can now be either at index 0 with
kubernetes v1.28 and older or at index 1 starting with kubernetes v1.29.
This is fragile at best since the `kubectl wait` doesn't allow to combine
multiple checks. Also, checking the reason is dubious as it doesn't really
tell if the pods have actually completed or not.

Check the pod phase to be `Succeeded` instead, this guarantees that :

> All containers in the Pod have terminated in success, and will not
> be restarted.

Fixes #9178

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-02-29 17:00:31 +01:00
Greg Kurz
f89120662d tests: k8s: Wait for all pods concurrently
A single invocation of `kubectl wait` can handle all pods.

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-02-29 17:00:31 +01:00
Gabriela Cervantes
3cd319fcc2 scripts: General fixes to the gha-run script
This PR implements general fixes to the gha-run script for the
cri-containerd tests.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-28 19:32:51 +00:00
Gabriela Cervantes
5a498948c8 scripts: Skip cri-containerd in gha-run script
This PR skips the cri-containerd in gha-run script for cloud hypervisor
runtime-rs.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-28 19:30:38 +00:00
Wainer Moschetta
c4b8270073 Merge pull request #9009 from wainersm/runk_bats
tests/runk: fix the "run ps command" flaky test
2024-02-28 15:58:36 -03:00
Wainer Moschetta
129ce84705 Merge pull request #9116 from wainersm/ci_install_kbs-workflow
gha: k8s: prepare AKS workflow to install the CoCo KBS
2024-02-28 14:43:41 -03:00
Wainer dos Santos Moschetta
b44e0c4e7c gha: k8s: prepare AKS workflow to install the CoCo KBS
Changed the "run k8s tests on AKS" workflows to get the CoCo KBS
installed so that we can run attestation tests.

The plan is to run attestation tests only on a subset of non-TEE jobs
initially, so this commit restricts to install KBS only on kata-qemu
configuration. Actually at this point it is added only stubs commands
to tests/integration/kubernetes/gha-run.sh that should be implemented
in a future commit.

Fixes #9058
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-02-27 13:51:15 -03:00
Wainer dos Santos Moschetta
0f8c36d990 tests/nydus: refactor the teardown()
This refactor the teardown() of tests/integration/nydus/nydus_tests.sh:

* Moved boilerplate code that kill process to a loop;
* Doesn't leave teardown() if a process failed to get killed, so that
  other clean up routines are ran;
* Check if the pid exist then attempt to kill the process, so avoid this
  misleading message:
```
Usage:
 kill [options] <pid> [...]

Options:
 <pid> [...]            send signal to every <pid> listed
 -<signal>, -s, --signal <signal>
                        specify the <signal> to be sent
 -q, --queue <value>    integer value to be sent with the signal
 -l, --list=[<signal>]  list all signal names, or convert one to a name
 -L, --table            list all signal names in a nice table

 -h, --help     display this help and exit
 -V, --version  output version information and exit

For more details see kill(1).
```

Fixes #8948
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-02-26 11:21:43 -03:00
Wainer dos Santos Moschetta
0f0ce9a81b tests/runk: replace the busybox image
It's recommended to avoid images from docker.io to avoid errors related
with hitting the pull limits that happens mostly on bare-metal machines.

So this replaced the docker.io's busybox with
quay.io/prometheus/busybox.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-02-26 11:11:05 -03:00
Wainer dos Santos Moschetta
bba8b5b2b4 tests/runk: fix flaky test
The "run ps command" test has failed once in a while because it doesn't
wait the sh command to start within the container, consequently `ps`
won't report the amount of lines expected.

Fixes #8975
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-02-26 11:09:29 -03:00
Wainer dos Santos Moschetta
8a606eb94d tests/runk: convert to bats
Migrated runk tests from pure shell script to bats to be consistent with
other test suites.

The install_dependencies() will install the bats tool locally.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-02-26 11:09:23 -03:00
GabyCT
4f3c83cd12 Merge pull request #9115 from GabyCT/topic/adddief
scripts: Add an enhanced die function
2024-02-23 12:03:02 -06:00
Steve Horsman
dfa6e932bb Merge pull request #9122 from ChengyuZhu6/snapshotter-clean
gha: try to cleanup nydus snapshotter before deploying it
2024-02-22 13:30:04 +00:00
ChengyuZhu6
8ab3894dc5 gha: try to cleanup nydus snapshotter before deploying it
CI failed to deploy nydus snapshotter because it was not cleaned up last time.
So we can try to cleanup nydus snapshotter before deploying it.

Fixes: #9121

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-22 18:51:14 +08:00
Dan Mihai
b3c3f992ab tests: k8s: common clean-up on teardown
teardown() gets executed after each test case, so there is no need to
clean-up before teardown.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:08 +00:00
Dan Mihai
9c164698d3 tests: k8s: k8s-optional-empty-configmap policy
Auto-generate policy for k8s-optional-empty-configmap.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:08 +00:00
Dan Mihai
74a52c6d25 tests: k8s: k8s-oom.bats auto-generated policy
Auto-generate policy for k8s-oom.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:08 +00:00
Dan Mihai
26a77d67f4 tests: k8s: k8s-number-cpus auto-generated policy
Auto-generate policy for k8s-number-cpus.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:08 +00:00
Dan Mihai
9cbdce15fd tests: k8s: k8s-memory.bats auto-generated policy
Auto-generate policy for k8s-memory.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:08 +00:00
Dan Mihai
40209cc0b7 tests: k8s: k8s-limit-range auto-generated policy
Auto-generate policy for k8s-limit-range.bats.

Also, fix teardown() namespace.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:08 +00:00
Dan Mihai
df3c0318c6 tests: k8s: add set_namespace_to_policy_settings
Add set_namespace_to_policy_settings() for changing the pod namespace
in genpolicy settings.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:08 +00:00
Dan Mihai
6e14ce93c9 tests: k8s-kill-all-process-in-container policy
Auto-generate policy for k8s-kill-all-process-in-container.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Dan Mihai
fad7ba0aea tests: k8s: k8s-job.bats auto-generated policy
Auto-generate policy for 8s-job.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Dan Mihai
41c2bcbdc5 tests: k8s: k8s-file-volume auto-generated policy
Auto-generate policy for k8s-file-volume.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Dan Mihai
81e641814f tests: k8s: k8s-cpu-ns auto-generated policy
Auto-generate policy for k8s-cpu-ns.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Dan Mihai
bc6d3fc238 tests: k8s: k8s-env.bats auto-generated policy
Auto-generate policy for k8s-env.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Dan Mihai
0a4fc071ac tests: k8s: k8s-custom-dns auto-generated policy
Auto-generate policy for k8s-custom-dns.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Dan Mihai
f693f49e92 tests: k8s: k8s-credentials-secrets policy
Auto-generate policy for k8s-credentials-secrets.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Dan Mihai
d3d27bbb5b tests: k8s: k8s-configmap auto-generated policy
Auto-generate policy for k8s-configmap.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Dan Mihai
b318535536 tests: k8s: auto-generate k8s-caps.bats policy
Auto-generated policy for k8s-caps.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Chelsea Mafrica
241a56989a Merge pull request #9090 from GabyCT/topic/pulldockerimage
gha: docker: Pull docker image as part of the dependencies
2024-02-20 14:28:53 -08:00
GabyCT
64c09fe6c5 Merge pull request #9088 from GabyCT/topic/fixnydus
gha: nydus: Fix indentation in gha run script
2024-02-20 14:09:54 -06:00