Commit Graph

798 Commits

Author SHA1 Message Date
ChengyuZhu6
cd6a84cfc5 kata-deploy: Setting up snapshotters per runtime handler
Setting up snapshotters per runtime handler as the commit
(6cc6ca5a7f) described.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-03-19 18:05:59 +01:00
Chelsea Mafrica
2c50d3c393 Merge pull request #9278 from wainersm/github_env_fix
tests: fix nounset error with $GITHUB_ENV
2024-03-14 16:39:13 -07:00
Dan Mihai
6094f1e31d Merge pull request #9250 from microsoft/danmihai1/k8s-pid-ns2
tests: k8s: k8s-pid-ns.bats auto-generated policy
2024-03-14 10:10:24 -07:00
Wainer dos Santos Moschetta
981f95df55 tests: fix nounset error with $GITHUB_ENV
Initialize $GITHUB_ENV to avoid nounset error when running the scripts locally
out of Github Actions.

Fixed commit 9ba5e3d2a8

Fixes #9217
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-13 14:57:38 -03:00
Dan Mihai
ac27caf1b4 Merge pull request #9248 from microsoft/danmihai1/k8s-exec.bats2
tests: k8s: k8s-exec.bats auto-generated policy
2024-03-13 09:21:12 -07:00
Dan Mihai
e8c2a45ce0 tests: k8s: k8s-pid-ns.bats auto-generated policy
Auto-generate policy for k8s-pid-ns.bats.

Fixes: #9249

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-03-12 22:34:46 +00:00
Alex Lyn
a116b252c8 Merge pull request #9236 from jodh-intel/docs-improve-install-details
docs: install: Simplify instructions
2024-03-12 14:29:38 +08:00
Dan Mihai
88b7a44271 tests: k8s: k8s-exec.bats auto-generated policy
Auto-generate policy for k8s-exec.bats.

Fixes: #9247

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-03-08 17:48:20 +00:00
GabyCT
35d8f82232 Merge pull request #9242 from GabyCT/topic/enabldebugnerd
gha: Add collect artifacts step to nerdctl workflow
2024-03-07 13:34:40 -06:00
Wainer Moschetta
91998af173 Merge pull request #9114 from wainersm/ci_kbs_cli
CI: add KBS utilities for attestation tests
2024-03-07 16:34:03 -03:00
Wainer dos Santos Moschetta
8ea9ac515e tests/k8s: update kbs repository
Recently confidential-containers/kbs repository was renamed to
confidential-containers/trustee. Github will automatically resolve the
old URL but we better adjust it in code.

The trustee repository will be cloned to $COCO_TRUSTEE_DIR. Adjusted
file paths and pushd/popd's to use $COCO_KBS_DIR
($COCO_TRUSTEE_DIR/kbs).

On versions.yaml changed from `coco-kbs` to `coco-trustee` as in the
future we might need other trustee components, so keeping it generic.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-07 11:20:36 +00:00
Wainer dos Santos Moschetta
c669567cd3 tests/k8s: add utils to set KBS policies
Added the kbs_set_resources_policy() function to set the KBS policy. Also the
kbs_set_allow_all_resources() and kbs_set_deny_all_resources to set the
"allow all" and "deny all" policy, respectively.

Fixes #9056
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-07 11:20:36 +00:00
Wainer dos Santos Moschetta
6f0d38094d tests/k8s: add utils to set KBS resources
Added utility functions to manage resources in KBS:
- kbs_set_resource(), where the resource data is passed via argument
- kbs_set_resource_from_file(), where the resource data is found in a
  file

Fixes #9056
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-07 11:20:36 +00:00
Wainer dos Santos Moschetta
2a374422c5 tests/k8s: add function to install kbs-client
Added kbs_install_cli function to build and install the kbs-client
executable if not present into the system.

Removed the stub from gha-run.sh; now the install kbs-client in the
.github/workflows/run-kata-deploy-tests-on-aks.yaml will effectively
install the executable.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-07 11:20:36 +00:00
Wainer dos Santos Moschetta
e410aef4fa tests/k8s: add utils to get kbs service address
Added functions to return the service host, port or full-qualified
HTTP address, respectively, kbs_k8s_svc_host(), kbs_k8s_svc_port(),
and kbs_k8s_svc_http_addr().

Fixes #9056
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-07 11:20:36 +00:00
Dan Mihai
c08b696d9e tests: k8s: k8s-shared-volume generated policy
Auto-generate policy for k8s-shared-volume.bats.

Fixes: #9096

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-03-07 05:57:30 +00:00
Dan Mihai
b24758fad8 tests: k8s: k8s-scale-nginx auto-generated policy
Auto-generate policy for k8s-scale-nginx.bats.

Fixes: #9096

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-03-07 05:57:30 +00:00
Dan Mihai
af9ac8d194 tests: k8s: k8s-replication auto-generated policy
Auto-generate policy for k8s-replication.bats.

Fixes: #9096

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-03-07 05:57:30 +00:00
Dan Mihai
56689c6800 tests: k8s: k8s-qos-pods auto-generated policy
Auto-generate policy for k8s-qos-pods.bats.

Fixes: #9096

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-03-07 05:57:30 +00:00
Dan Mihai
0179f53469 tests: k8s: k8s-parallel auto-generated policy
Auto-generate policy for k8s-parallel.bats.

Fixes: #9096

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-03-07 05:57:30 +00:00
Gabriela Cervantes
94fdcda7f7 scripts: Add collect artifacts function in nerdctl gha run script
This PR adds the collect artifacts function in nerdctl gha run script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-03-06 19:48:12 +00:00
James O. D. Hunt
b1d4cbd9d1 utils: spell-checker: Fix grep warning
Fix the `grep(1)` warning caused by the unnecessary escaping of the
hash/sharp symbol.

Fixes: #9235.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2024-03-06 13:21:15 +00:00
James O. D. Hunt
a67ed2f1c2 tests: Add k3s artifacts
The k3s distribution of k8s uses an embedded version of containerd and
configures it to log to a file, not the journal. Hence, although we
collect the journal as a test artifact, we also need to collect the
actual log files for containerd.

Also collect the k3s containerd config files to help with debugging.

Fixes: #9104.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2024-03-05 17:54:20 +00:00
Wainer dos Santos Moschetta
9ba5e3d2a8 gha: export start_time to collect artifacts properly
The jobs running on garm will collect journal information. The data gathered
is based on the time the tests started running. The $start_time is
exported on run_tests() and used in collect_artifacts(). It happens that
run_tests() and collect_artifacts() are called on different steps of the
workflow and the environment variables aren't preserved between them,
i.e, $start_time exported on the first step is not available on the
subsequents.

To solve that issue, let's save $start_time in the file pointed out by
$GITHUB_ENV that Github actions uses to export variables. In case $GITHUB_ENV is
empty then probably it is running locally outside of Github, so it won't
save the start time value.

Fixes #9217
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-05 12:15:20 -03:00
Greg Kurz
0320198889 Merge pull request #9206 from lifupan/main
CI: fix the issue of ci failure on crio
2024-03-05 09:52:13 +01:00
Wainer Moschetta
38088a934b Merge pull request #9184 from wainersm/fix_kata_deploy_bats
tests/kata-deploy: fix checker for kata-deploy running
2024-03-04 20:50:37 -03:00
GabyCT
77d048da4d Merge pull request #9065 from wainersm/ci_install_kbs
CI: Install KBS on k8s for attestation tests
2024-03-04 16:59:01 -06:00
Gabriela Cervantes
5d50262422 docs: Add general tests documentation in main README
This PR adds the general tests documentation in main README of the
kata containers repository.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-03-04 21:53:01 +00:00
Gabriela Cervantes
d5fa2bebd5 docs: Add general README for tests section
This PR adds general README documentation for the tests section
in the kata containers repository.

Fixes #9209

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-03-04 21:50:37 +00:00
GabyCT
4dea9019ab Merge pull request #9126 from GabyCT/topic/addartifactsk
gha: Storing artifacts for logs of k8s tests garm
2024-03-04 15:41:54 -06:00
Gabriela Cervantes
fc5e040d96 scripts: Apply general fixes to variables in gha-run script
This PR applies general fixes to variables in gha-run script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-03-04 18:54:15 +00:00
Fupan Li
07e0cf1855 CI: fix the issue of ci failure on crio
PR #8760 tentatively tried to have the shim to run in its own mount
namespace for the sake of improving isolation between the sandbox and
the host. Thus crio storage drivers shouldn't create a PRIVATE
bind mount on their home directory. Otherwise, the container's rootfs
mount wouldn't be propagated to kata runtime's mount namespace, and
kata runtime couldn't access the container's rootfs files.

So, when kata cooperated with crio, crio should set
skip_mount_home=true for its storage overlay.

Fixes: #9028

Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
2024-03-03 20:53:36 +08:00
Wainer dos Santos Moschetta
2c24977cb1 tests/k8s: allow to overwrite the cluster name
_print_cluster_name() create a string based information like the
pull request number and commit SHA. However, when you are developing the
scripts you might want to use an arbitrary name, so it was introduced
the $AKS_NAME variable that once exported it will overwrite the
generated name.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-02 12:42:35 -03:00
Wainer dos Santos Moschetta
5e4b7bbd04 tests/k8s: expose KBS service externally
Until this point the deployed KBS service is only reachable from within
the cluster. This introduces a generic mechanism to apply an Ingress
configuration to expose the service externally.

The first implemened ingress is for AKS. In case the HTTP application
routing isn't enabled in the cluster (this is required for ingress), an
add-on is applied.

It was added the get_cluster_specific_dns_zone() and
enable_cluster_http_application_routing() helper functions
to gha-run-k8s-common.sh.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-02 12:42:35 -03:00
Wainer dos Santos Moschetta
e1e0b94975 tests/k8s: introduce the CoCo kbs library
Introduce the tests/integration/kubernetes/confidential_kbs.sh library
that contains functions to manage the KBS on CI. Initially implemented
the kbs_k8s_deploy() and kbs_k8s_delete() functions to, respectively,
deploy and delete KBS on Kubernetes. Also hooked those functions in the
tests/integration/kubernetes/gha-run.sh script to follow the convention
of running commands from Github Workflows:

$ .tests/integration/kubernetes/gha-run.sh deploy-coco-kbs
$ .tests/integration/kubernetes/gha-run.sh delete-coco-kbs

Fixes #9058
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-02 12:39:26 -03:00
Wainer dos Santos Moschetta
6a28c94d99 tests/k8s: add a kustomize installer
Kustomize has been used on some of our internal components (e.g.
kata-deploy) to manage k8s deployments. On CI it has been used
the `sed` tool to edit kustomization.yaml files, but `kustomize` is
more suitable for that purpose. So in order to use that tool on CI
scripts in the future, this commit introduces the `install_kustomize()`
function that is going to download and install the binary in
/usr/local/bin in case it's found on $PATH.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-02 12:39:26 -03:00
GabyCT
4a0cfc4e3f Merge pull request #9199 from GabyCT/topic/enablecri
gha: Enable cri-containerd tests for cloud hypervisor runtime-rs
2024-03-01 12:23:16 -06:00
Gabriela Cervantes
7299dbdb43 gha: Store journalctl logs
This PR stores the journalctl logs.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-03-01 15:17:20 +00:00
Gabriela Cervantes
342d3a320d gha: Add collect artifacts function in gha-run script
This PR adds the collect artifacts function in gha-run script for
the kubernetes tests.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-03-01 15:17:20 +00:00
Greg Kurz
dc6bda19bf Merge pull request #9179 from gkurz/fix-k8s-sandbox-vcpus-allocation-check
tests: k8s: Adapt k8s-sandbox-vcpus-allocation.bats to kubernetes v1.29
2024-03-01 15:55:07 +01:00
Wainer dos Santos Moschetta
24c163e6e1 tests/kata-deploy: fix checker for kata-deploy running
Currently, the checking for kata-deploy is running assume that the
daemonset scheduled at least one pod, however it might not had and the
kubectl wait command fails due to "error: no matching resources found".

On CI I've observed that fail intermittently. I suspect the service
account kata-deploy-sa take a while to show up then no kata-deploy is
scheduled in meanwhile.

Changed the checker logic to use waitForProcess() to keep testing if it is
already running, or hit the timeout (still 10m).

Fixes #9183
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-02-29 22:26:27 -03:00
Gabriela Cervantes
beb592b309 gha: Enable cri-containerd tests for cloud hypervisor runtime-rs
This PR enables the cri-containerd tests for cloud hypervisor runtime-rs.

Fixes #9198

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-29 20:18:16 +00:00
Gabriela Cervantes
0f595cf15b gha: General variable fixes to gha-run script
This PR adds general variable fixes to gha-run script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-29 18:15:27 +00:00
Greg Kurz
f3442cdef9 tests: k8s: Adapt k8s-sandbox-vcpus-allocation.bats to kubernetes v1.29
Kubernetes v1.29 introduced a new `PodReadyToStartContainers` condition
that gets inserted at index 0 in the conditions array. This means that
the expected `PodCompleted` reason can now be either at index 0 with
kubernetes v1.28 and older or at index 1 starting with kubernetes v1.29.
This is fragile at best since the `kubectl wait` doesn't allow to combine
multiple checks. Also, checking the reason is dubious as it doesn't really
tell if the pods have actually completed or not.

Check the pod phase to be `Succeeded` instead, this guarantees that :

> All containers in the Pod have terminated in success, and will not
> be restarted.

Fixes #9178

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-02-29 17:00:31 +01:00
Greg Kurz
f89120662d tests: k8s: Wait for all pods concurrently
A single invocation of `kubectl wait` can handle all pods.

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-02-29 17:00:31 +01:00
Gabriela Cervantes
3cd319fcc2 scripts: General fixes to the gha-run script
This PR implements general fixes to the gha-run script for the
cri-containerd tests.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-28 19:32:51 +00:00
Gabriela Cervantes
5a498948c8 scripts: Skip cri-containerd in gha-run script
This PR skips the cri-containerd in gha-run script for cloud hypervisor
runtime-rs.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-28 19:30:38 +00:00
Wainer Moschetta
c4b8270073 Merge pull request #9009 from wainersm/runk_bats
tests/runk: fix the "run ps command" flaky test
2024-02-28 15:58:36 -03:00
Wainer Moschetta
129ce84705 Merge pull request #9116 from wainersm/ci_install_kbs-workflow
gha: k8s: prepare AKS workflow to install the CoCo KBS
2024-02-28 14:43:41 -03:00
Wainer dos Santos Moschetta
b44e0c4e7c gha: k8s: prepare AKS workflow to install the CoCo KBS
Changed the "run k8s tests on AKS" workflows to get the CoCo KBS
installed so that we can run attestation tests.

The plan is to run attestation tests only on a subset of non-TEE jobs
initially, so this commit restricts to install KBS only on kata-qemu
configuration. Actually at this point it is added only stubs commands
to tests/integration/kubernetes/gha-run.sh that should be implemented
in a future commit.

Fixes #9058
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-02-27 13:51:15 -03:00