Commit Graph

501 Commits

Author SHA1 Message Date
Fabiano Fidêncio
058f068d67 Merge pull request #9020 from BbolroC/ok-to-test-static-checks-but-x86
gha: Run static-checks on self-hosted runners conditionally
2024-02-06 19:30:21 +01:00
Wainer Moschetta
f1ca5d1563 Merge pull request #8953 from ChengyuZhu6/ci-guest-pull
gha: Enable nydus snapshotter in CoCo ci tests
2024-02-06 09:36:59 -03:00
Amulya Meka
354a3093fa Merge pull request #9019 from Amulyam24/k8s-fix
gha: add GOPATH env var to the ppc64le k8s workflow
2024-02-06 11:01:49 +05:30
Hyounggyu Choi
40b2b2a43a gha: Run static-checks on self-hosted runners conditionally
Due to the restrictions on instance provisioning for self-hosted runners, performing
static checks (36 jobs at the time of writing) on them each time a PR is updated could
significantly burden them, consequently slowing down the entire CI system. To address
this, the decision is to trigger these checks only when an 'ok-to-test' label is added.
Meanwhile, the checks for x86_64, which are supported by GitHub-hosted runners, will
remain unchanged.

Fixes: #8998

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-02-05 15:24:21 +01:00
ChengyuZhu6
a214bd8d13 gha: Enable nydus snapshotter in CoCo ci tests
This PR is a split of #8585.
make the changes on the Github workflows, and the skeleton to deploy_snapshotter()
and cleanup_snapshotter() in tests/integration/kubernetes/gha-run.sh in this commit.

After initially merging this patch to trigger CI jobs for CoCo, which will begin executing
the dummy functions deploy_snapshotter() and cleanup_snapshotter(), the implementation details for these functions
remain in #8585. Our subsequent step involves transferring this logic to the PR #8484, enabling the PR to undergo CI testing prior to its merge.

Fixes: #8997

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-02-05 18:51:59 +08:00
Fabiano Fidêncio
1362918ff0 Merge pull request #9011 from fidencio/topic/switch-to-using-the-confidential-rootfs
runtime: Replace TEE specific initrd / image for the confidential one
2024-02-05 10:43:12 +01:00
Amulyam24
e59d005568 gha: add GOPATH env var to the ppc64le k8s workflow
The filtering of testing cases installs/uses yq and expects GOPATH to be present. Hence, add it to the workflow.

Fixes: #9018

Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
2024-02-05 10:30:10 +05:30
ChengyuZhu6
f354beb253 static-checks: Install clang in the ci environments
To test PR #8484, the compilation process for the kata-agent relies on clang.
There have been encountered failures on ARM, s390x, and ppc64le architectures:
ppc64le: https://github.com/kata-containers/kata-containers/actions/runs/7754082828/job/21146689026?pr=8484
s390x: https://github.com/kata-containers/kata-containers/actions/runs/7754082828/job/21146689401?pr=8484
arm: https://github.com/kata-containers/kata-containers/actions/runs/7754082828/job/21146689026?pr=8484

Fixes: #9004

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-02-04 17:00:19 +08:00
Fabiano Fidêncio
d4a9856a84 gha: Remove SEV / SNP / TDX images / initrds
We can remove this now that we're relying on the confidential one.

Fixes: #9010

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-03 13:22:07 +01:00
Fabiano Fidêncio
a9f8888c15 packaging: Add confidential image / initrd
Let's use a single rootfs image / initrd for confidential workloads,
instead of having those split for different TEEs.

We can easily do this now as the soon-to-be-added guest-components can
be built in a generic way.

Fixes: #8982

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-03 00:58:52 +01:00
Fabiano Fidêncio
dc027e39d6 gha: Remove TEE specific kernel build targets
We're using the confidential kernel instead from now on.

Fixes: #8981 -- part I

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-02 21:12:41 +01:00
GabyCT
40d9a65601 Merge pull request #8996 from GabyCT/topic/addclhr
gha: k8s: Add cloud-hypervisor (runtime-rs) support
2024-02-02 09:48:35 -06:00
Gabriela Cervantes
6771ca463b gha: k8s: Add cloud-hypervisor (runtime-rs) support
This PR adds the Cloud Hypervisor driver, integrated with the runtime-rs,
as part of the kubernetes tests different with devmapper.

Fixes #8995

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-01 21:22:56 +00:00
Amulyam24
f8585db8d9 gha: add kubernetes tests workflow for ppc64le
This PR adds workflow for running kubernetes test suite on ppc64le.

It uses scripts to create and delete the cluster using kubeadm as none of the current cluster creation tools are supported on Power.

Fixes: #7950

Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
2024-02-01 12:23:11 +05:30
Fabiano Fidêncio
34d51b05f8 gha: cri-o: Bump runners to 22.04
This will *not* solve the CRI-O CI breakage but will give us an
environment where we could get it to run locally.

Fixes: #8935 -- part I

Thanks to Julien Ropé for trying to reproduce the issues I faced on
https://github.com/kata-containers/kata-containers/issues/8935 in an
Ubuntu 22.04 system.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-30 14:17:06 +01:00
Greg Kurz
d687b601f1 Merge pull request #8933 from fidencio/topic/package-coco-guest-components
packaging: Build coco-guest-components
2024-01-29 16:34:06 +01:00
Fabiano Fidêncio
448c0aaecb gha: azure: Set the correct subscription to the account
Due to the changes done in the CI, we need to set the correct
subscription to be used with the account from now on, otherwise we'd end
up using CoCo subscription.

Fixes: #8946

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-29 15:00:38 +01:00
Fabiano Fidêncio
644abde35c packaging: coco-guest-components: Allow building the project
The Confidential Containers guest-components will, in the very short
future, be part of the Kata Containers rootfs that's used by the
Confidential Containers usecase.

This commit introduces the ability to, standalone, build the component
locally and as part of our CI, and this can be done by calling:
`make coco-guest-components-tarball`

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Linda Yu <linda.yu@intel.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: Jakob Naucke <jakob.naucke@ibm.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
Co-authored-by: zhouliang121 <liang.a.zhou@linux.alibaba.com>
Co-authored-by: Alex Carter <alex.carter@ibm.com>
Co-authored-by: Suraj Deshmukh <suraj.deshmukh@microsoft.com>
Co-authored-by: Xynnn007 <xynnn@linux.alibaba.com>
2024-01-26 19:36:01 +01:00
Hyounggyu Choi
ee072e8a06 Merge pull request #8926 from fidencio/topic/cache-the-agent-for-non-x86_64
gha: Cache the agent for non-x86_64 arches
2024-01-26 18:04:33 +01:00
Fabiano Fidêncio
08a082ca47 gha: Cache the agent for non-x86_64 arches
Those are not yet being cached for no reason, and they better be as
it'll allow us to save a considerable amount of time building the
rootfs.

Fixes: #8917

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-26 12:02:26 +01:00
Hyounggyu Choi
ab462a4b89 tests: Add IBM SE to the basic confidential test
The existing confidential basic test titled `Test unencrypted
confidential container launch success and verify that we are
running in a secure enclave` has been updated to incorporate
IBM Secure Execution (`qemu-se`).
Previously, a secure image was absent from kata-deploy, hindering
the inclusion of IBM SE in the test.
Thanks to the #6755 update, it is now possible to test the TEE.

This modification extends the existing test by introducing
`qemu-se`. The specific changes are outlined below:

- Add an additional test `cc-se-e2e-tests` to s390x nightly
- Expansion of `REMOTE_COMMAND_PER_HYPERVISOR` for `qemu-se`
- Temporary exclusion of two test cases currently incompatible with IBM SE
(`cpu-ns` is a common issue across all TEEs, while `inotify`
will be addressed in a subsequent pull request).

Fixes: #8913

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-01-26 06:04:39 +01:00
Zvonko Kaiser
4b8d79c1f6 gpu: remove GHA target first then remove the obsoleted Makefile targets
Lets remove the GHA target actions first so the the follow-up PR #8874 tests are succeeding.

Fixes: #8900

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-01-24 11:43:39 +00:00
Greg Kurz
4516f38165 Merge pull request #8872 from zvonkok/nvidia-gpu-confidential
gpu: Add NVIDIA GPU Confidential kernel target
2024-01-23 09:22:27 +01:00
Dan Mihai
3d2ec5c919 Merge pull request #8857 from microsoft/danmihai1/k8s-gha
gha: get ready to install genpolicy
2024-01-22 08:29:24 -08:00
Zvonko Kaiser
4fc34323ae gpu: Add NVIDIA GPU Confidential kernel target
This is a follow up to the work of minimizing targets, unifying TDX,SNP builds for NVIDIA GPUs

Fixes: #8828

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-01-22 14:58:57 +00:00
Dan Mihai
ea9c659d36 gha: get ready to install genpolicy
The changes to install and test genpolicy must come later, after CI
picks up these gha changes.

Fixes: #8856

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-01-19 23:37:49 +00:00
Amulyam24
14934c7b0d github: run static checks on ppc64le
This PR adds ppc64le runner to the static-checks workflow.

Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
2024-01-18 16:31:13 +01:00
Hyounggyu Choi
8061a49ca5 kata-ctl: Clean up a test leftover file explicitely
It was observed that a tmporary file `/tmp/kata_hybrid_vsock02.hvsock`
for test_setup_hvsock_failed() is not removed from time to time.
This leads to a test failure for the same test next time due to the
file permission on a self-hosted runner.
This commit is to explicitely delete the file before the check starts.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-01-18 16:31:13 +01:00
Hyounggyu Choi
354cbede9c GHA: Enable static check for s390x
As part of the CI migration from Jenkins to GitHub Action, a CI job named
`kata-containers-2.0-ubuntu-s390x-unit-PR` is covered by the static check.
This commit is to enable the check for s390x by incorporating a runner
`s390x` with the corresponding workflow.

Fixes: #8482

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-01-18 16:31:13 +01:00
Jianyong Wu
a10779bf0b GHA: enable static check on arm64
This is to add a runner for arm64 to the workflow.

Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2024-01-18 16:31:11 +01:00
Dan Mihai
201eec628a tools: genpolicy static checks
Package genpolicy and enable static checks for it.

Fixes: #8813

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-01-15 16:49:58 +00:00
GabyCT
2ffb161958 Merge pull request #8763 from stevenhorsman/fix-backport-check-hub
Fix backport check hub
2024-01-11 15:15:12 -06:00
GabyCT
69be050ff9 Merge pull request #8657 from WenyuanLau/8656/Fix_StratoVirt_on_gha_metrics
gha: Fix the failure of gha metrics for StratoVirt
2024-01-11 11:41:25 -06:00
Fabiano Fidêncio
8b2f43a2c2 build: Add "confidential" kernel
We're using a Kernel based on v6.7, which should include all te
patches needed for SEV / SNP / TDX.

By doing this, later on, we'll be able to stop building the specific
kernel for each one of the targets we have for the TEEs.

Let's note that we've introduced the "confidential" target for the
kernel builder script, while the TEE specific builds are being kept as
they're -- at least for now.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-09 14:35:45 -03:00
stevenhorsman
6bac3323be workflows: Update backport-label to use gh-utils.sh
- hub is deprecated, so use the new gh-utils.sh script that wraps the github cli instead

Fixes: #8125
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-01-04 16:48:34 +00:00
Greg Kurz
1cfcc80018 Merge pull request #8664 from amshinde/remove-ignore-paths-ga
github-actions: Remove ignore paths for required CI checks
2023-12-18 12:49:21 +01:00
Archana Shinde
510bc36a77 github-actions: Remove ignore paths for required CI checks
If a PR contains files from the ignore-paths, these actions do not run
as intended. However, the actions are make as required. And there does
not seem to be a way to mark these as non-required in that case.
As a result a PR containing the files from the ignore-paths remains
stalled.
Hence remove the ignore-paths until github provides a way to mark
actions that are skipped due to ignore-paths as non-required/passed.

Fixes: #8663

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2023-12-15 15:12:20 -08:00
Liu Wenyuan
61fe20cf9a gha: Fix some of gha metrics failure for StratoVirt
Update the Speed & Density metric tests baseline for StratoVirt
and re-enable them, and skip other metric tests temporarily.

Fixes: #8656

Signed-off-by: Liu Wenyuan <liuwenyuan9@huawei.com>
2023-12-15 17:45:01 +08:00
Steve Horsman
04de6eb4fd Merge pull request #8674 from ChengyuZhu6/fix_statis_check
static-checks: Add some dependencies to static checks for CoCo features
2023-12-14 16:47:01 +00:00
ChengyuZhu6
dfad0e6622 .github: fix the failure without devicemapper for host sharing
fix error when running checks and tests:
error: failed to run custom build command for `devicemapper-sys v0.1.5`
fatal error: 'libdevmapper.h' file not found

thread 'main' panicked at 'Could not generate dm.h bindings:
ClangDiagnostic("dm.h:2:10: fatal error: 'libdevmapper.h' file not found\n")',
/home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/devicemapper-sys-0.1.5/build.rs:24:10
  stack backtrace:
     0: rust_begin_unwind
               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:593:5
     1: core::panicking::panic_fmt
               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/panicking.rs:67:14
     2: core::result::unwrap_failed
               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/result.rs:1651:5
     3: core::result::Result<T,E>::expect
     4: build_script_build::main
     5: core::ops::function::FnOnce::call_once
  note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
warning: build failed, waiting for other jobs to finish...
make: *** [../../utils.mk:177: standard_rust_check] Error 101

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-12-14 20:47:47 +08:00
ChengyuZhu6
983479748f .github: fix error when making checks for CoCo guest pull
Fix error when making checks:
```
error: failed to run custom build command for `image-rs v0.1.0
(https://github.com/confidential-containers/guest-components?tag=v0.8.0#e849dc89)`

Caused by:
  process didn't exit successfully: `/home/runner/work/kata-containers/kata-containers/src/
  agent/target/release/build/image-rs-fd932206d09362b7/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-changed=./protos/getresource.proto
  cargo:rerun-if-changed=./protos

  --- stderr
  thread 'main' panicked at 'Could not find `protoc` installation and this build crate cannot proceed without
  this knowledge. If `protoc` is installed and this crate had trouble finding
  it, you can set the `PROTOC` environment variable with the specific path to your
  installed `protoc` binary.If you're on debian, try `apt-get install protobuf-compiler`
  or download it from https://github.com/protocolbuffers/protobuf/releases
```

Fixes #8673

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-12-14 20:47:42 +08:00
Amulyam24
0db820fa01 gha: add a post cleanup script for cri-containerd ppc64le workflow
This PR identifies and adds an action to cleanup the ppc64le self hosted runner.

Fixes: #8666

Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
2023-12-14 13:46:47 +05:30
Hyounggyu Choi
05e278de5b GHA: Put all the preliminary steps into pre-action for s390x
This is to introduce a pre-action to all the workflows for building artifacts.
The action could take care of tasks such as cleaning up files and reinstalling
packages, which prevents a workflow from getting affected by the environment.

This also includes the removal of the step `Adjust a permission for repo`,
because it could be incorporated into the action.

Fixes: #8648

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-12-13 13:24:40 +01:00
GabyCT
43410e1918 Merge pull request #8560 from GabyCT/topic/enablek8srs
gha: k8s: Add cloud-hypervisor (runtime-rs) support
2023-12-11 09:42:49 -06:00
Hyounggyu Choi
636eef8907 GHA: make secrets inherited for build-kata-static-tarball-s390x
This is to make GHA secrets inherited for the workflow titled
`build-kata-static-tarball-s390x` to configure an environment
variable `CI_HKD_PATH` for a `build-asset-boot-image-se` step.

Fixes: #8611

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-12-08 13:55:45 +01:00
Hyounggyu Choi
588f639a69 Merge pull request #6755 from BbolroC/add-se-artifacts-to-main
packaging: Add IBM Z SE artifacts to main
2023-12-08 05:17:38 +01:00
Hyounggyu Choi
f38c7f14c5 gha: remove build redundancy of kernel and rootfs-initrd
It is to remove the build redundancy of `kernel` and `rootfs-initrd` by making `boot-image-se` built based on them at the second build stage.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-12-07 20:05:40 +01:00
Gabriela Cervantes
acee3d8438 gha: k8s: Add cloud-hypervisor (runtime-rs) support
This PR adds the Cloud Hypervisor driver, integrated with the runtime-rs,
as part of the kubernetes tests.

Fixes #8559

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-12-07 16:33:59 +00:00
Gabriela Cervantes
e70b2ea95d gha: nerdctl: Enable cloud hypervisor runtime-rs for nerdctl CI
This PR enables the cloud hypervisor runtime-rs for the nerdctl
gha CI.

Fixes #8603

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-12-07 16:24:36 +00:00
Fabiano Fidêncio
2cd003156e Merge pull request #8573 from fidencio/topic/gha-add-a-timeout-for-tests
gha: basic-ci: Add a timeout for the tests
2023-12-05 22:20:49 +01:00