708 Commits

Author SHA1 Message Date
Gabriela Cervantes
0066aebd84 tests: setup: Improve setup script for kubernetes tests
This PR makes general improvements like definition of variables and
the use of them to improve the general setup script for kubernetes
tests.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-06-12 19:39:54 +00:00
GabyCT
0c5849b68b Merge pull request #9809 from microsoft/danmihai1/yq-breaking-change
tests: k8s: use newer yq command line format
2024-06-10 16:29:59 -06:00
Wainer Moschetta
ade69e44f9 Merge pull request #9785 from BbolroC/kubectl-retry
CI: Introduce retry mechanism for kubectl in gha-run.sh
2024-06-10 18:33:34 -03:00
Dan Mihai
32198620a9 tests: k8s: use newer yq command line format
Fix the recent collision between:
- https://github.com/kata-containers/kata-containers/pull/9377
- https://github.com/kata-containers/kata-containers/pull/9725

One enabled a newer yq command line format and the other used the older
format. Both passed CI because they were not tested together.

Fixes: #9789

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-06-10 18:48:25 +00:00
Dan Mihai
079a0a017c Merge pull request #9557 from portersrc/ci-debug-output-nydus-pod
CI: describe pod on k8s-create-pod wait failure
2024-06-10 08:17:54 -07:00
Hyounggyu Choi
8ff128dda8 CI: Introduce retry mechanism for kubectl in gha-run.sh
Frequent errors have been observed during k8s e2e tests:

- The connection to the server 127.0.0.1:6443 was refused - did you specify the right host or port?
- Error from server (ServiceUnavailable): the server is currently unable to handle the request
- Error from server (NotFound): the server could not find the requested resource

These errors can be resolved by retrying the kubectl command.

This commit introduces a wrapper function in common.sh that runs kubectl up to 3 times
with a 5-second interval. Initially, this change only covers gha-run.sh for Kubernetes.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-06-07 18:24:19 +02:00
Fabiano Fidêncio
81c221c1b4 ci: k8s: tdx: Re-enable volume tests
It seems I was very lose on disabling some of the tests, and the issues
I faced could be related to other instabilities in the CI.

Let's re-enable this one, following what was done for the SEV, SNP, and
coco-qemu-dev.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-06-07 18:13:36 +02:00
Fabiano Fidêncio
9db9d35198 ci: k8s: tdx: Re-enable projected-volume tests
It seems I was very lose on disabling some of the tests, and the issues
I faced could be related to other instabilities in the CI.

Let's re-enable this one, following what was done for the SEV, SNP, and
coco-qemu-dev.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-06-07 18:12:36 +02:00
Fabiano Fidêncio
f6a6cba8ca ci: k8s: tdx: Re-enable nested-configmap-secret tests
It seems I was very lose on disabling some of the tests, and the issues
I faced could be related to other instabilities in the CI.

Let's re-enable this one, following what was done for the SEV, SNP, and
coco-qemu-dev.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-06-07 18:12:06 +02:00
Fabiano Fidêncio
957d0cccf6 ci: k8s: tdx: Re-enable inotify tests
It seems I was very lose on disabling some of the tests, and the issues
I faced could be related to other instabilities in the CI.

Let's re-enable this one, following what was done for the SEV, SNP, and
coco-qemu-dev.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-06-07 18:10:39 +02:00
Fabiano Fidêncio
fc6f662ae0 ci: k8s: tdx: Re-enable credentials-secrets tests
It seems I was very lose on disabling some of the tests, and the issues
I faced could be related to other instabilities in the CI.

Let's re-enable this one, following what was done for the SEV, SNP, and
coco-qemu-dev.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-06-07 18:08:29 +02:00
Fabiano Fidêncio
fde457589e ci: kbs: tdx: Enable basic attestation tests
Let's stop skipping the CDH tests for TDX, as know we should have an
environmemnt where it can run and should pass. :-)

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-06-07 12:18:50 +02:00
Fabiano Fidêncio
cac525059e ci: kbs: tdx: Use the hostname ip instead of localhost for the PCCS
We must ensure we use the host ip to connect to the PCCS running on the
host side, instead of using localhost (which has a different meaning
from inside the KBS pod).

The reason we're using `hostname -i` isntead of the helper functions, is
because the helper functions need the coco-kbs deployed for them to
work, and what we do is before the deployment.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-06-07 12:18:07 +02:00
Chris Porter
4172ccb3a0 CI: describe pod on k8s-create-pod wait failure
This is generally useful debug output on test failures,
and specifically this has been useful for nydus-related
issues recently.

Signed-off-by: Chris Porter <porter@ibm.com>
2024-06-05 12:37:53 -04:00
Fabiano Fidêncio
138ef2c55f Merge pull request #9678 from AdithyaKrishnan/main
TEEs: Skip a few CI tests for SEV/SNP
2024-06-04 23:42:51 +02:00
GabyCT
ba30f0804a Merge pull request #9770 from GabyCT/topic/fixvad
tests: Use variable definition for better uniformity
2024-06-04 15:23:34 -06:00
Wainer dos Santos Moschetta
af4f9afb71 kata-deploy: add PULL_TYPE handler for CRI-O
A new PULL_TYPE environment variable is recognized by the kata-deploy's
install script to allow it to configure CRIO-O for guest-pull image pulling
type.

The tests/integration/kubernetes/gha-run.sh change allows for testing it:
```
export PULL_TYPE=guest-pull
cd tests/integration/kubernetes
./gha-run.sh deploy-k8s
```

Fixes #9474
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-06-04 14:02:01 -03:00
GabyCT
6c2e8bed77 Merge pull request #9725 from 3u13r/feat/genpolicy/filter-by-runtime
genpolicy: add ability to filter for runtimeClassName
2024-06-04 10:06:14 -06:00
Gabriela Cervantes
cafba23f3e tests: Use variable definition for better uniformity
This PR replaces the name to use a variable that is already defined
to have a better uniformity across the general script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-06-04 15:49:27 +00:00
Hyounggyu Choi
3aff6c5bd8 CI: Retry fetching node_start_time when it is empty
It was observed that the `node_start_time` value is sometimes empty,
leading to a test failure.

This commit retries fetching the value up to 3 times.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-06-04 15:41:15 +02:00
Wainer Moschetta
b5561074c3 Merge pull request #9377 from beraldoleal/yqbump
deps: bumping yq to v4.40.7
2024-06-03 14:34:58 -03:00
Ryan Savino
72dc823059 tests: k8s: sev: snp: skip "setting sysctl" test
This test fails when using `shared_fs=none` with the nydus snapshotter.
Issue tracked here: #9666
Skipping for now.

Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
2024-06-03 01:14:17 -05:00
Ryan Savino
3f3be54893 tests: k8s: sev: snp: skip initContainers shared vol test
This test is failing due to the initContainers not being properly
handled with the guest image pulling.
Issue tracked here: #9668
Skipping for now.

Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
2024-06-03 01:14:17 -05:00
Ryan Savino
35dfb730ce tests: k8s: sev: snp: skip "kill all processes in container" test
This test fails when using `shared_fs=none` with the nydus napshotter,
Issue tracked here: #9664
Skipping for now.

Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
2024-06-03 01:14:16 -05:00
Ryan Savino
62cc1dec4c tests: replace docker debug alpine image with ghcr
docker alpine latest image is rate limited.
Need to use ghcr.io image.

Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
2024-06-03 01:14:16 -05:00
ChengyuZhu6
1820b02993 tests: replace busybox from docker with quay in guest pull
To prevent download failures caused by high traffic to the Docker image,
opt for quay.io/prometheus/busybox:latest over docker.io/library/busybox:latest .

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-06-03 01:14:16 -05:00
Ryan Savino
6c646dc96d tests: k8s: sev: snp: add runtime annotation for sev and snp
sev and snp cases added to the KATA_HYPERVISOR switch.

Signed-off-by: Ryan Savino <ryan.savino@amd.com>
2024-06-03 01:14:16 -05:00
Ryan Savino
668959408d tests: ensure kata_deploy cleanup even if namespace deletion fails
the test cluster namespace deletion failing causes kata_deploy to not get cleaned up.

Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
2024-06-03 01:14:15 -05:00
Markus Rudy
13310587ed genpolicy: check requested devices
CreateContainerRequest objects can specify devices to be created inside
the guest VM. This change ensures that requested devices have a
corresponding entry in the PodSpec.

Devices that are added to the pod dynamically, for example via the
Device Plugin architecture, can be allowlisted globally by adding their
definition to the settings file.

Fixes: #9651
Signed-off-by: Markus Rudy <mr@edgeless.systems>
2024-05-31 22:05:49 +02:00
Wainer Moschetta
f093c4c190 Merge pull request #9754 from wainersm/qemu_coco_dev-enable_policy_tests
tests/k8s: enable policy tests for qemu-coco-dev
2024-05-31 15:09:25 -03:00
Beraldo Leal
ba5d2e54c2 tests: remove object separation mark from eof
End of file should not end with --- mark. This will confuse tools like
yq and kubectl that might think this is another object.

Signed-off-by: Beraldo Leal <bleal@redhat.com>
2024-05-31 13:28:34 -04:00
Beraldo Leal
c99ba42d62 deps: bumping yq to v4.40.7
Since yq frequently updates, let's upgrade to a version from February to
bypass potential issues with versions 4.41-4.43 for now. We can always
upgrade to the newest version if necessary.

Fixes #9354
Depends-on:github.com/kata-containers/tests#5818

Signed-off-by: Beraldo Leal <bleal@redhat.com>
2024-05-31 13:28:34 -04:00
Greg Kurz
7886ed6670 Merge pull request #9751 from wainersm/k8s_print_logs_on_fail
tests/k8s: print logs on fail only (k8s-confidential-attestation.bats)
2024-05-31 14:47:27 +02:00
Fabiano Fidêncio
03a7cf4b02 ci: k8s: Skip empty dir tests also for TDX
Wainer noticed this is failing for the coco-qemu-dev case, and decided
to skip it, notifying me that he didn't fully understand why it was not
failing on TDX.

Turns out, though, this is also failing on TDX, and we need to skip it
there as well.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-05-31 09:59:46 +02:00
Leonard Cohnen
1d1690e2a4 genpolicy: add ability to filter for runtimeClassName
Add the CLI flag --runtime-class-names, which is used during
policy generation. For resources that can define a
runtimeClassName (e.g., Pods, Deployments, ReplicaSets,...)
the value must have any of the --runtime-class-names as
prefix, otherwise the resource is ignored.

This allows to run genpolicy on larger yaml
files defining many different resources and only generating
a policy for resources which will be deployed in a
confidential context.

Signed-off-by: Leonard Cohnen <lc@edgeless.systems>
2024-05-31 03:17:02 +02:00
Wainer dos Santos Moschetta
3333f8ddfd tests/k8s: enable policy tests for qemu-coco-dev
So qemu-coco-dev is on pair with the TEE configurations.

Fixes: #9753
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-05-30 21:51:15 -03:00
Wainer Moschetta
83fa813700 Merge pull request #9694 from wainersm/qemu_coco_dev-k8s-guest-pull
tests: enable guest-pull on all k8s tests for the qemu-coco-dev configuration
2024-05-30 21:48:11 -03:00
Wainer dos Santos Moschetta
55ae98eb28 tests/k8s: print logs on fail only (k8s-confidential-attestation.bats)
Use the variable BATS_TEST_COMPLETED which is defined by the bats framework
when the test finishes. `BATS_TEST_COMPLETED=` (empty) means the test failed,
so the node syslogs will be printed only at that condition.

Fixes: #9750
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-05-30 17:19:33 -03:00
Wainer dos Santos Moschetta
3e18fe7805 tests/k8s: skip file volume tests for qemu-coco-dev
This test fails with qemu-coco-dev configuration and guest-pull image pull.

Issue: #9667
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-05-30 14:50:59 -03:00
Wainer dos Santos Moschetta
5faf9ca344 ci: pin the nydus-snapshotter image version
It's cloning the nydus-snapshotter repo from the version specified in
versions.yaml, however, the deployment files are set to pull in the
latest version of the snapshotter image. With this version we are
pinning the image version too.

This is a temporary fix as it should be better worked out at nydus-snapshotter
project side.

Fixes: #9742
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-05-30 11:21:16 -03:00
Wainer dos Santos Moschetta
37894923c1 tests/k8s: skip empty dir volumes tests for qemu-coco-dev
This test fails with qemu-coco-dev configuration and guest-pull image pull.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-05-29 18:37:24 -03:00
Wainer dos Santos Moschetta
79a8b31ec5 tests/k8s: skip shared volume tests for qemu-coco-dev
This test fails with qemu-coco-dev configuration and guest-pull image pull.

Issue: #9668
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-05-29 18:37:24 -03:00
Wainer dos Santos Moschetta
aa1a37081e tests/k8s: skip sysctls tests for qemu-coco-dev
This test fails with qemu-coco-dev configuration and guest-pull image pull.

Issue: #9666
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-05-29 18:37:24 -03:00
Wainer dos Santos Moschetta
0e81ced9f1 tests/k8s: skip kill-all-process tests for qemu-coco-dev
This test fails with qemu-coco-dev configuration and guest-pull image pull.

Issue: #9664
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-05-29 18:37:24 -03:00
Wainer dos Santos Moschetta
18896efa3c tests/k8s: skip seccomp tests for qemu-coco-dev
This test fails with qemu-coco-dev configuration and guest-pull image pull.
Unlike other tests that I've seen failing on this scenario, k8s-seccomp.bats
fails after a couple of consecutive executions, so it's that kind of failure
that happens once in a while.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-05-29 18:37:24 -03:00
Wainer dos Santos Moschetta
b62ad71c43 tests/k8s: add runtime handler annotation for qemu-coco-dev
This will enable the k8s tests to leverage guest pulling when
PULL_TYPE=guest-pull for qemu-coco-dev runtimeclass.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-05-29 18:37:24 -03:00
Wainer dos Santos Moschetta
089c7ad84a tests/k8s: add runtime handler annotation only for guest-pull
The runtime handler annotation is required for Kubernetes <= 1.28 and
guest-pull pull type. So leverage $PULL_TYPE (which is exported by CI jobs)
to conditionally apply the annotation.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-05-29 18:37:24 -03:00
Fabiano Fidêncio
dbd0d4a090 gha: Only do preventive cleanups for baremetal
This takes a few minutes that could be saved, so let's avoid doing this
on all the platforms, but simply do this when it's needed (the baremetal
use case).

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-05-27 05:05:06 +02:00
Fabiano Fidêncio
ee2ef0641c tests: k8s: Allow passing "all" to run all the tests
Currently only "baremetal" runs all the tests, but we could easily run
"all" locally or using the github provided runners, even when not using
a "baremetal" system.

The reason I'd like to have a differentiation between "all" and
"baremetal" is because "baremetal" may require some cleanup, which "all"
can simply skip if testing against a fresh created VM.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-05-27 05:05:06 +02:00
Fabiano Fidêncio
556227cb51 tests: Add the possibility to deploy k0s / rke2
For now we've only exposed the option to deploy kata-deploy for k3s and
vanilla kubernetes when using containerd.

However, I do need to also deploy k0s and rke2 for an internal CI, and
having those exposed here do not hurt, and allow us to easily expand the
CI at any time in the future.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-05-27 05:05:06 +02:00