Commit Graph

326 Commits

Author SHA1 Message Date
Fabiano Fidêncio
19eb45a27d Merge pull request #8484 from ChengyuZhu6/guest-pull
Merge basic guest pull image code to main
2024-03-19 23:15:39 +01:00
Fabiano Fidêncio
8911d3565f gha: tests: Filter out confidential tests for aarch64 / ppc64le
Those two architectures are not TEE capable, thus we can just skip
running those tests there.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-03-19 18:06:01 +01:00
Fabiano Fidêncio
d14e9802b6 gha: k8s: Set {https,no}_proxy correctly for TDX
This is needed as the TDX machine is hosted inside Intel and relies on
proxies in order to connect to the external world.  Not having those set
causes issues when pulling the image inside the guest.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-03-19 18:06:00 +01:00
ChengyuZhu6
e23737a103 gha: refactor code with yq for better clarity
refactor code with yq for better clarity:

Before:
```bash
yq write -i "${tools_dir}/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml"
'spec.template.spec.containers[0].env[7].value' "${KATA_HYPERVISOR}:${SNAPSHOTTER}"
```

After:
```bash
yq write -i \
  "${tools_dir}/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml" \
  'spec.template.spec.containers[0].env[7].value' \
  "${KATA_HYPERVISOR}:${SNAPSHOTTER}"
```

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-03-19 18:06:00 +01:00
ChengyuZhu6
2c0bc8855b tests: Make sure to install yq before using it
Make sure to install yq before using it to modify YAML files.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-03-19 18:06:00 +01:00
ChengyuZhu6
c52b356482 tests: add guest pull image test
Add a test case of pulling image inside the guest for confidential
containers.

Signed-off-by: Da Li Liu <liudali@cn.ibm.com>
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: Georgina Kinge <georgina.kinge@ibm.com>
Co-authored-by: Megan Wright <Megan.Wright@ibm.com>
2024-03-19 18:06:00 +01:00
ChengyuZhu6
e8c4effc07 tests: refactor the check for hypervisor to a function
Extract two reusable functions for confidential tests in confidential_common.sh

- check_hypervisor_for_confidential_tests: verifies if the input hypervisor supports confidential tests.
- confidential_setup: performs the common setup for confidential tests.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Co-authored-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-03-19 18:06:00 +01:00
ChengyuZhu6
cd6a84cfc5 kata-deploy: Setting up snapshotters per runtime handler
Setting up snapshotters per runtime handler as the commit
(6cc6ca5a7f) described.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-03-19 18:05:59 +01:00
Hyounggyu Choi
b381743dd5 CI|k8s: Handle skipped tests with a comment for filter_out_per_arch
This commit updates `filter_k8s_test.sh` to handle skipped tests that
include comments. In addition to the existing parameter expansion,
the following expansions have been added:

- Removal of a comment
- Stripping of trailing spaces

Fixes: #9304

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-03-19 17:21:25 +01:00
Chelsea Mafrica
2c50d3c393 Merge pull request #9278 from wainersm/github_env_fix
tests: fix nounset error with $GITHUB_ENV
2024-03-14 16:39:13 -07:00
Dan Mihai
6094f1e31d Merge pull request #9250 from microsoft/danmihai1/k8s-pid-ns2
tests: k8s: k8s-pid-ns.bats auto-generated policy
2024-03-14 10:10:24 -07:00
Wainer dos Santos Moschetta
981f95df55 tests: fix nounset error with $GITHUB_ENV
Initialize $GITHUB_ENV to avoid nounset error when running the scripts locally
out of Github Actions.

Fixed commit 9ba5e3d2a8

Fixes #9217
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-13 14:57:38 -03:00
Dan Mihai
ac27caf1b4 Merge pull request #9248 from microsoft/danmihai1/k8s-exec.bats2
tests: k8s: k8s-exec.bats auto-generated policy
2024-03-13 09:21:12 -07:00
Dan Mihai
e8c2a45ce0 tests: k8s: k8s-pid-ns.bats auto-generated policy
Auto-generate policy for k8s-pid-ns.bats.

Fixes: #9249

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-03-12 22:34:46 +00:00
Dan Mihai
88b7a44271 tests: k8s: k8s-exec.bats auto-generated policy
Auto-generate policy for k8s-exec.bats.

Fixes: #9247

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-03-08 17:48:20 +00:00
GabyCT
35d8f82232 Merge pull request #9242 from GabyCT/topic/enabldebugnerd
gha: Add collect artifacts step to nerdctl workflow
2024-03-07 13:34:40 -06:00
Wainer Moschetta
91998af173 Merge pull request #9114 from wainersm/ci_kbs_cli
CI: add KBS utilities for attestation tests
2024-03-07 16:34:03 -03:00
Wainer dos Santos Moschetta
8ea9ac515e tests/k8s: update kbs repository
Recently confidential-containers/kbs repository was renamed to
confidential-containers/trustee. Github will automatically resolve the
old URL but we better adjust it in code.

The trustee repository will be cloned to $COCO_TRUSTEE_DIR. Adjusted
file paths and pushd/popd's to use $COCO_KBS_DIR
($COCO_TRUSTEE_DIR/kbs).

On versions.yaml changed from `coco-kbs` to `coco-trustee` as in the
future we might need other trustee components, so keeping it generic.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-07 11:20:36 +00:00
Wainer dos Santos Moschetta
c669567cd3 tests/k8s: add utils to set KBS policies
Added the kbs_set_resources_policy() function to set the KBS policy. Also the
kbs_set_allow_all_resources() and kbs_set_deny_all_resources to set the
"allow all" and "deny all" policy, respectively.

Fixes #9056
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-07 11:20:36 +00:00
Wainer dos Santos Moschetta
6f0d38094d tests/k8s: add utils to set KBS resources
Added utility functions to manage resources in KBS:
- kbs_set_resource(), where the resource data is passed via argument
- kbs_set_resource_from_file(), where the resource data is found in a
  file

Fixes #9056
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-07 11:20:36 +00:00
Wainer dos Santos Moschetta
2a374422c5 tests/k8s: add function to install kbs-client
Added kbs_install_cli function to build and install the kbs-client
executable if not present into the system.

Removed the stub from gha-run.sh; now the install kbs-client in the
.github/workflows/run-kata-deploy-tests-on-aks.yaml will effectively
install the executable.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-07 11:20:36 +00:00
Wainer dos Santos Moschetta
e410aef4fa tests/k8s: add utils to get kbs service address
Added functions to return the service host, port or full-qualified
HTTP address, respectively, kbs_k8s_svc_host(), kbs_k8s_svc_port(),
and kbs_k8s_svc_http_addr().

Fixes #9056
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-07 11:20:36 +00:00
Dan Mihai
c08b696d9e tests: k8s: k8s-shared-volume generated policy
Auto-generate policy for k8s-shared-volume.bats.

Fixes: #9096

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-03-07 05:57:30 +00:00
Dan Mihai
b24758fad8 tests: k8s: k8s-scale-nginx auto-generated policy
Auto-generate policy for k8s-scale-nginx.bats.

Fixes: #9096

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-03-07 05:57:30 +00:00
Dan Mihai
af9ac8d194 tests: k8s: k8s-replication auto-generated policy
Auto-generate policy for k8s-replication.bats.

Fixes: #9096

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-03-07 05:57:30 +00:00
Dan Mihai
56689c6800 tests: k8s: k8s-qos-pods auto-generated policy
Auto-generate policy for k8s-qos-pods.bats.

Fixes: #9096

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-03-07 05:57:30 +00:00
Dan Mihai
0179f53469 tests: k8s: k8s-parallel auto-generated policy
Auto-generate policy for k8s-parallel.bats.

Fixes: #9096

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-03-07 05:57:30 +00:00
Gabriela Cervantes
94fdcda7f7 scripts: Add collect artifacts function in nerdctl gha run script
This PR adds the collect artifacts function in nerdctl gha run script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-03-06 19:48:12 +00:00
James O. D. Hunt
a67ed2f1c2 tests: Add k3s artifacts
The k3s distribution of k8s uses an embedded version of containerd and
configures it to log to a file, not the journal. Hence, although we
collect the journal as a test artifact, we also need to collect the
actual log files for containerd.

Also collect the k3s containerd config files to help with debugging.

Fixes: #9104.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2024-03-05 17:54:20 +00:00
Wainer dos Santos Moschetta
9ba5e3d2a8 gha: export start_time to collect artifacts properly
The jobs running on garm will collect journal information. The data gathered
is based on the time the tests started running. The $start_time is
exported on run_tests() and used in collect_artifacts(). It happens that
run_tests() and collect_artifacts() are called on different steps of the
workflow and the environment variables aren't preserved between them,
i.e, $start_time exported on the first step is not available on the
subsequents.

To solve that issue, let's save $start_time in the file pointed out by
$GITHUB_ENV that Github actions uses to export variables. In case $GITHUB_ENV is
empty then probably it is running locally outside of Github, so it won't
save the start time value.

Fixes #9217
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-05 12:15:20 -03:00
Wainer Moschetta
38088a934b Merge pull request #9184 from wainersm/fix_kata_deploy_bats
tests/kata-deploy: fix checker for kata-deploy running
2024-03-04 20:50:37 -03:00
GabyCT
77d048da4d Merge pull request #9065 from wainersm/ci_install_kbs
CI: Install KBS on k8s for attestation tests
2024-03-04 16:59:01 -06:00
GabyCT
4dea9019ab Merge pull request #9126 from GabyCT/topic/addartifactsk
gha: Storing artifacts for logs of k8s tests garm
2024-03-04 15:41:54 -06:00
Gabriela Cervantes
fc5e040d96 scripts: Apply general fixes to variables in gha-run script
This PR applies general fixes to variables in gha-run script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-03-04 18:54:15 +00:00
Wainer dos Santos Moschetta
5e4b7bbd04 tests/k8s: expose KBS service externally
Until this point the deployed KBS service is only reachable from within
the cluster. This introduces a generic mechanism to apply an Ingress
configuration to expose the service externally.

The first implemened ingress is for AKS. In case the HTTP application
routing isn't enabled in the cluster (this is required for ingress), an
add-on is applied.

It was added the get_cluster_specific_dns_zone() and
enable_cluster_http_application_routing() helper functions
to gha-run-k8s-common.sh.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-02 12:42:35 -03:00
Wainer dos Santos Moschetta
e1e0b94975 tests/k8s: introduce the CoCo kbs library
Introduce the tests/integration/kubernetes/confidential_kbs.sh library
that contains functions to manage the KBS on CI. Initially implemented
the kbs_k8s_deploy() and kbs_k8s_delete() functions to, respectively,
deploy and delete KBS on Kubernetes. Also hooked those functions in the
tests/integration/kubernetes/gha-run.sh script to follow the convention
of running commands from Github Workflows:

$ .tests/integration/kubernetes/gha-run.sh deploy-coco-kbs
$ .tests/integration/kubernetes/gha-run.sh delete-coco-kbs

Fixes #9058
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-02 12:39:26 -03:00
GabyCT
4a0cfc4e3f Merge pull request #9199 from GabyCT/topic/enablecri
gha: Enable cri-containerd tests for cloud hypervisor runtime-rs
2024-03-01 12:23:16 -06:00
Gabriela Cervantes
7299dbdb43 gha: Store journalctl logs
This PR stores the journalctl logs.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-03-01 15:17:20 +00:00
Gabriela Cervantes
342d3a320d gha: Add collect artifacts function in gha-run script
This PR adds the collect artifacts function in gha-run script for
the kubernetes tests.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-03-01 15:17:20 +00:00
Greg Kurz
dc6bda19bf Merge pull request #9179 from gkurz/fix-k8s-sandbox-vcpus-allocation-check
tests: k8s: Adapt k8s-sandbox-vcpus-allocation.bats to kubernetes v1.29
2024-03-01 15:55:07 +01:00
Wainer dos Santos Moschetta
24c163e6e1 tests/kata-deploy: fix checker for kata-deploy running
Currently, the checking for kata-deploy is running assume that the
daemonset scheduled at least one pod, however it might not had and the
kubectl wait command fails due to "error: no matching resources found".

On CI I've observed that fail intermittently. I suspect the service
account kata-deploy-sa take a while to show up then no kata-deploy is
scheduled in meanwhile.

Changed the checker logic to use waitForProcess() to keep testing if it is
already running, or hit the timeout (still 10m).

Fixes #9183
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-02-29 22:26:27 -03:00
Gabriela Cervantes
beb592b309 gha: Enable cri-containerd tests for cloud hypervisor runtime-rs
This PR enables the cri-containerd tests for cloud hypervisor runtime-rs.

Fixes #9198

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-29 20:18:16 +00:00
Gabriela Cervantes
0f595cf15b gha: General variable fixes to gha-run script
This PR adds general variable fixes to gha-run script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-29 18:15:27 +00:00
Greg Kurz
f3442cdef9 tests: k8s: Adapt k8s-sandbox-vcpus-allocation.bats to kubernetes v1.29
Kubernetes v1.29 introduced a new `PodReadyToStartContainers` condition
that gets inserted at index 0 in the conditions array. This means that
the expected `PodCompleted` reason can now be either at index 0 with
kubernetes v1.28 and older or at index 1 starting with kubernetes v1.29.
This is fragile at best since the `kubectl wait` doesn't allow to combine
multiple checks. Also, checking the reason is dubious as it doesn't really
tell if the pods have actually completed or not.

Check the pod phase to be `Succeeded` instead, this guarantees that :

> All containers in the Pod have terminated in success, and will not
> be restarted.

Fixes #9178

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-02-29 17:00:31 +01:00
Greg Kurz
f89120662d tests: k8s: Wait for all pods concurrently
A single invocation of `kubectl wait` can handle all pods.

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-02-29 17:00:31 +01:00
Gabriela Cervantes
3cd319fcc2 scripts: General fixes to the gha-run script
This PR implements general fixes to the gha-run script for the
cri-containerd tests.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-28 19:32:51 +00:00
Gabriela Cervantes
5a498948c8 scripts: Skip cri-containerd in gha-run script
This PR skips the cri-containerd in gha-run script for cloud hypervisor
runtime-rs.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-28 19:30:38 +00:00
Wainer Moschetta
c4b8270073 Merge pull request #9009 from wainersm/runk_bats
tests/runk: fix the "run ps command" flaky test
2024-02-28 15:58:36 -03:00
Wainer Moschetta
129ce84705 Merge pull request #9116 from wainersm/ci_install_kbs-workflow
gha: k8s: prepare AKS workflow to install the CoCo KBS
2024-02-28 14:43:41 -03:00
Wainer dos Santos Moschetta
b44e0c4e7c gha: k8s: prepare AKS workflow to install the CoCo KBS
Changed the "run k8s tests on AKS" workflows to get the CoCo KBS
installed so that we can run attestation tests.

The plan is to run attestation tests only on a subset of non-TEE jobs
initially, so this commit restricts to install KBS only on kata-qemu
configuration. Actually at this point it is added only stubs commands
to tests/integration/kubernetes/gha-run.sh that should be implemented
in a future commit.

Fixes #9058
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-02-27 13:51:15 -03:00