Commit Graph

285 Commits

Author SHA1 Message Date
Greg Kurz
dc6bda19bf Merge pull request #9179 from gkurz/fix-k8s-sandbox-vcpus-allocation-check
tests: k8s: Adapt k8s-sandbox-vcpus-allocation.bats to kubernetes v1.29
2024-03-01 15:55:07 +01:00
Gabriela Cervantes
0f595cf15b gha: General variable fixes to gha-run script
This PR adds general variable fixes to gha-run script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-29 18:15:27 +00:00
Greg Kurz
f3442cdef9 tests: k8s: Adapt k8s-sandbox-vcpus-allocation.bats to kubernetes v1.29
Kubernetes v1.29 introduced a new `PodReadyToStartContainers` condition
that gets inserted at index 0 in the conditions array. This means that
the expected `PodCompleted` reason can now be either at index 0 with
kubernetes v1.28 and older or at index 1 starting with kubernetes v1.29.
This is fragile at best since the `kubectl wait` doesn't allow to combine
multiple checks. Also, checking the reason is dubious as it doesn't really
tell if the pods have actually completed or not.

Check the pod phase to be `Succeeded` instead, this guarantees that :

> All containers in the Pod have terminated in success, and will not
> be restarted.

Fixes #9178

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-02-29 17:00:31 +01:00
Greg Kurz
f89120662d tests: k8s: Wait for all pods concurrently
A single invocation of `kubectl wait` can handle all pods.

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-02-29 17:00:31 +01:00
Gabriela Cervantes
3cd319fcc2 scripts: General fixes to the gha-run script
This PR implements general fixes to the gha-run script for the
cri-containerd tests.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-28 19:32:51 +00:00
Gabriela Cervantes
5a498948c8 scripts: Skip cri-containerd in gha-run script
This PR skips the cri-containerd in gha-run script for cloud hypervisor
runtime-rs.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-28 19:30:38 +00:00
Wainer Moschetta
c4b8270073 Merge pull request #9009 from wainersm/runk_bats
tests/runk: fix the "run ps command" flaky test
2024-02-28 15:58:36 -03:00
Wainer Moschetta
129ce84705 Merge pull request #9116 from wainersm/ci_install_kbs-workflow
gha: k8s: prepare AKS workflow to install the CoCo KBS
2024-02-28 14:43:41 -03:00
Wainer dos Santos Moschetta
b44e0c4e7c gha: k8s: prepare AKS workflow to install the CoCo KBS
Changed the "run k8s tests on AKS" workflows to get the CoCo KBS
installed so that we can run attestation tests.

The plan is to run attestation tests only on a subset of non-TEE jobs
initially, so this commit restricts to install KBS only on kata-qemu
configuration. Actually at this point it is added only stubs commands
to tests/integration/kubernetes/gha-run.sh that should be implemented
in a future commit.

Fixes #9058
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-02-27 13:51:15 -03:00
Wainer dos Santos Moschetta
0f8c36d990 tests/nydus: refactor the teardown()
This refactor the teardown() of tests/integration/nydus/nydus_tests.sh:

* Moved boilerplate code that kill process to a loop;
* Doesn't leave teardown() if a process failed to get killed, so that
  other clean up routines are ran;
* Check if the pid exist then attempt to kill the process, so avoid this
  misleading message:
```
Usage:
 kill [options] <pid> [...]

Options:
 <pid> [...]            send signal to every <pid> listed
 -<signal>, -s, --signal <signal>
                        specify the <signal> to be sent
 -q, --queue <value>    integer value to be sent with the signal
 -l, --list=[<signal>]  list all signal names, or convert one to a name
 -L, --table            list all signal names in a nice table

 -h, --help     display this help and exit
 -V, --version  output version information and exit

For more details see kill(1).
```

Fixes #8948
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-02-26 11:21:43 -03:00
Wainer dos Santos Moschetta
0f0ce9a81b tests/runk: replace the busybox image
It's recommended to avoid images from docker.io to avoid errors related
with hitting the pull limits that happens mostly on bare-metal machines.

So this replaced the docker.io's busybox with
quay.io/prometheus/busybox.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-02-26 11:11:05 -03:00
Wainer dos Santos Moschetta
bba8b5b2b4 tests/runk: fix flaky test
The "run ps command" test has failed once in a while because it doesn't
wait the sh command to start within the container, consequently `ps`
won't report the amount of lines expected.

Fixes #8975
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-02-26 11:09:29 -03:00
Wainer dos Santos Moschetta
8a606eb94d tests/runk: convert to bats
Migrated runk tests from pure shell script to bats to be consistent with
other test suites.

The install_dependencies() will install the bats tool locally.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-02-26 11:09:23 -03:00
Steve Horsman
dfa6e932bb Merge pull request #9122 from ChengyuZhu6/snapshotter-clean
gha: try to cleanup nydus snapshotter before deploying it
2024-02-22 13:30:04 +00:00
ChengyuZhu6
8ab3894dc5 gha: try to cleanup nydus snapshotter before deploying it
CI failed to deploy nydus snapshotter because it was not cleaned up last time.
So we can try to cleanup nydus snapshotter before deploying it.

Fixes: #9121

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-22 18:51:14 +08:00
Dan Mihai
b3c3f992ab tests: k8s: common clean-up on teardown
teardown() gets executed after each test case, so there is no need to
clean-up before teardown.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:08 +00:00
Dan Mihai
9c164698d3 tests: k8s: k8s-optional-empty-configmap policy
Auto-generate policy for k8s-optional-empty-configmap.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:08 +00:00
Dan Mihai
74a52c6d25 tests: k8s: k8s-oom.bats auto-generated policy
Auto-generate policy for k8s-oom.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:08 +00:00
Dan Mihai
26a77d67f4 tests: k8s: k8s-number-cpus auto-generated policy
Auto-generate policy for k8s-number-cpus.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:08 +00:00
Dan Mihai
9cbdce15fd tests: k8s: k8s-memory.bats auto-generated policy
Auto-generate policy for k8s-memory.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:08 +00:00
Dan Mihai
40209cc0b7 tests: k8s: k8s-limit-range auto-generated policy
Auto-generate policy for k8s-limit-range.bats.

Also, fix teardown() namespace.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:08 +00:00
Dan Mihai
df3c0318c6 tests: k8s: add set_namespace_to_policy_settings
Add set_namespace_to_policy_settings() for changing the pod namespace
in genpolicy settings.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:08 +00:00
Dan Mihai
6e14ce93c9 tests: k8s-kill-all-process-in-container policy
Auto-generate policy for k8s-kill-all-process-in-container.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Dan Mihai
fad7ba0aea tests: k8s: k8s-job.bats auto-generated policy
Auto-generate policy for 8s-job.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Dan Mihai
41c2bcbdc5 tests: k8s: k8s-file-volume auto-generated policy
Auto-generate policy for k8s-file-volume.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Dan Mihai
81e641814f tests: k8s: k8s-cpu-ns auto-generated policy
Auto-generate policy for k8s-cpu-ns.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Dan Mihai
bc6d3fc238 tests: k8s: k8s-env.bats auto-generated policy
Auto-generate policy for k8s-env.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Dan Mihai
0a4fc071ac tests: k8s: k8s-custom-dns auto-generated policy
Auto-generate policy for k8s-custom-dns.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Dan Mihai
f693f49e92 tests: k8s: k8s-credentials-secrets policy
Auto-generate policy for k8s-credentials-secrets.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Dan Mihai
d3d27bbb5b tests: k8s: k8s-configmap auto-generated policy
Auto-generate policy for k8s-configmap.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Dan Mihai
b318535536 tests: k8s: auto-generate k8s-caps.bats policy
Auto-generated policy for k8s-caps.bats.

Fixes: #9072

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-21 18:08:07 +00:00
Chelsea Mafrica
241a56989a Merge pull request #9090 from GabyCT/topic/pulldockerimage
gha: docker: Pull docker image as part of the dependencies
2024-02-20 14:28:53 -08:00
GabyCT
64c09fe6c5 Merge pull request #9088 from GabyCT/topic/fixnydus
gha: nydus: Fix indentation in gha run script
2024-02-20 14:09:54 -06:00
Fabiano Fidêncio
3468ac3b6e ci: k8s: Fix checks used to skip confidential tests
This has been introduced by 53bc4a432b,
where the condition was changed.

The correct condition is:
* If the list of supported tees does not contain the kata hypervisor
  and the list of supported non tees does not contain the kata
  hypervisor.

The error is that we were checking whether kata-hypervisor would contain
the list of supported tees, and that would almost always be false
(unless in the case where the list had an one and only one element).

Fixes: #9055 -- part II

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-18 10:10:45 +01:00
Hyounggyu Choi
8b3f7f353d CI|k8s: Skip vcpu allocation test for s390x
A test `vcpu allocation k8s test` exhibits different behavior on s390x
For more details, please refer to issue #9093.
This commit is to make the test skipped until the issue is resolved on
the platform.

Fixes: #9093

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-02-15 12:26:35 +01:00
GabyCT
9cf343779f Merge pull request #9062 from GabyCT/topic/nonteet
tests: Add ability to run non-TEE environments
2024-02-13 14:28:07 -06:00
Gabriela Cervantes
598c77409a gha: docker: Pull docker image as part of the dependencies
This PR pulls the docker image needed for the test as part of the dependencies
in order to avoid failures of timeouts mainly because the image was not
properly download it and it is unable to find it.

Fixes #9089

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-13 17:48:31 +00:00
Gabriela Cervantes
53bc4a432b tests: Add ability to run non-TEE environments
This PR adds the ability to run k8s confidential tests in a
non-TEE environment.

Fixes #9055

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-13 17:27:55 +00:00
Gabriela Cervantes
54d1f34650 gha: nydus: Fix indentation in gha run script
This PR fixes the indentation in gha run script for nydus.

Fixes #9087

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-13 16:53:28 +00:00
Fabiano Fidêncio
3877a9f49a ci: Clean up kata-deploy ds before starting the tests
This will ensure no leftovers are in the node, which has been cause the
TDX CI to fail every now and then.

Fixes: #9081

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-13 14:10:44 +01:00
GabyCT
00be9ae872 Merge pull request #9070 from microsoft/danmihai1/debug-containers
tests: k8s: avoid deleting unrelated pods
2024-02-12 15:24:15 -06:00
Dan Mihai
a21ca9b7c9 tests: k8s: avoid deleting unrelated pods
Delete the debugger pod created during the test, rather than already
existing debugger pods.

Also, send the output of "kubectl delete" to stderr, just in case it's
useful for debugging.

Fixes: #9069

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-09 22:48:41 +00:00
Dan Mihai
a054462eb7 Merge pull request #9051 from microsoft/danmihai1/k8s-copy-file
tests: k8s: k8s-copy-file auto-generated policy
2024-02-09 12:30:49 -08:00
ChengyuZhu6
97fbf360cc gha: Cleanup nydus snapshotter by the daemonset
Cleanup nydus snapshotter by the daemonset.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-02-09 14:47:13 +01:00
ChengyuZhu6
43b04fd0c0 gha: Deploy nydus snapshotter by the daemonset
We can use daemonset to deploy nydus snapshotter, which will decrease
one manual step both for Kata Containers and Confidential Containers CI.

Fixes: #8584

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-02-09 14:47:09 +01:00
Gabriela Cervantes
0b508f301b tests:k8s: make add_kernel_initrd_anotations function generic
This PR replaces the add_kernel_initrd_annotations_to_yaml function
more generic so later can be used for other components.

Fixes #9054

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-08 19:30:43 +00:00
Dan Mihai
f139c7dc60 tests: k8s: k8s-copy-file auto-generated policy
Auto-generate policy for k8s-copy-file.bats.

Fixes: #9050

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-08 13:26:05 +00:00
Dan Mihai
1179306afa tests: k8s: additional policy testing utilities
1. add_requests_to_policy_settings allows one or more ttrpc requests
   from the Host to the Guest. Example:

add_requests_to_policy_settings "${policy_settings_dir}" \
   "ReadStreamRequest" "WriteStreamRequest"

2. add_copy_from_host_to_policy_settings allows executing on the Guest
   the commands initiated behind the scenes by "kubectl cp" from the
   Host to the Guest. Example:

add_copy_from_host_to_policy_settings "${policy_settings_dir}"

3. add_copy_from_guest_to_policy_settings allows executing on the Guest
   the commands initiated behind the scenes by "kubectl cp" from the
   Guest to the Host. Example:

add_copy_from_guest_to_policy_settings "${policy_settings_dir}" \
   "/tmp/file.txt"

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-08 13:25:41 +00:00
Dan Mihai
6b5e57f7c7 tests: k8s: address PR review feedback
1. Rename install_kata_common to install_kata_core.

2. Add TODO for better way to install the Kata tools.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-07 18:51:56 +00:00
Dan Mihai
dd16bc393f tests: k8s: k8s-attach-handlers generated policy
Automatically generate the test policy for k8s-attach-handlers.bats,
if AUTO_GENERATE_POLICY is enabled.

Steps:

- Create a temporary directory for the current test and copy the
  common genpolicy settings into this new directory.

- Change genpolicy settings in the temp directory to allow the
  "kubectl exec" command that this test needs. (For CoCo, exec is
  blocked by the default policy settings)

- Auto-generate the policy for the test YAML file.

- Test as usual, using the YAML file.

- Clean-up the temporary settings described above.

Fixes: #8921

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-07 02:26:03 +00:00