Compare commits

..

1166 Commits

Author SHA1 Message Date
Fabiano Fidêncio
6c2a2a14fe Merge pull request #8415 from fidencio/topic/CCv0-update-release-workflow
cc: gha: kernel-snp-experimental is not a valid tag
2023-11-09 15:36:05 +01:00
Fabiano Fidêncio
87e6a61cf7 cc: gha: kernel-snp-experimental is not a valid tag
We fixed this as part of the after-push PR, but we forgot to have this
one fixed.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-11-09 15:22:57 +01:00
Steve Horsman
8de1f8e19f Merge pull request #8400 from littlejawa/fix_pullimage_in_guest_for_crio
CC | agent: fix annotation name to allow cri-o support of CoCo pullimage in guest
2023-11-08 18:55:47 +00:00
Julien Ropé
06308fbf96 agent: fix annotation to allow cri-o support of CoCo pullimage in guest
The pullimage handler behaves differently for the sandbox and container,
and it uses the annotations to differentiate between the two.
The annotation name used by crio for container-type being different than
the containerd one, we need to check both.

Fixes: #8399

Signed-off-by: Julien Ropé <jrope@redhat.com>
2023-11-08 16:07:33 +01:00
Steve Horsman
4089d48b45 Merge pull request #8353 from portersrc/set-v0.8.0-tag-imagers-aa-tdshim
CCv0: Set v0.8.0 tag for image-rs, AA and td-shim
2023-11-06 16:50:10 +00:00
Chris Porter
1dadbb337b CCv0: Set v0.8.0 tag for image-rs, AA and td-shim
The release candidates for image-rs, AA, and td-shim have
now been tagged with v0.8.0. Point the respective toml
and yaml files to this tag.

Fixes: #8352
Signed-off-by: Chris Porter <porter@ibm.com>
2023-11-01 15:26:52 -05:00
Steve Horsman
424de1cbfa Merge pull request #7852 from surajssd/update-aa-imagers-tdshim2
CCv0: Update image-rs, AA and td-shim
2023-10-31 09:00:15 +00:00
Suraj Deshmukh
f1b4b95b3f CCv0: Update image-rs, AA and td-shim
- image-rs & AA: e5e6e69ae0ea01393a125cde8e2772ad98aba39c
- td-shim: 3f18d9f3f51c9428c034cf01b45ab94bcd0d1622

Fixes: #7580

Signed-off-by: Suraj Deshmukh <suraj.deshmukh@microsoft.com>
2023-10-30 14:32:09 +00:00
Steve Horsman
135c166b8e Merge pull request #8222 from mattarnoatibm/enable-image-rs-signature-simple-xrss
CC | agent: Enable signature-simple-xrss
2023-10-19 16:20:57 +01:00
Steve Horsman
cf6169de4c Merge pull request #8220 from microsoft/danmihai1/CCv0-default-policy
rootfs: add default agent policy file
2023-10-19 15:43:33 +01:00
Dan Mihai
159bc2713f rootfs: add default agent policy file
Restricting access to agent endpoints using agent-config.toml is
expected to be deprecated in the main branch. Therefore, in
preparation of merging this script with its main branch version,
install default settings for main branch's kata-opa service.

coco-default.rego blocks access to the same kata agent endpoints
that are blocked by agent-config.toml. For additional information,
search for "default-policy.rego" in main branch's rootfs.sh.

Fixes: #8219

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2023-10-17 13:16:45 +00:00
Steve Horsman
3bd72b9a0d Merge pull request #8224 from stevenhorsman/CCv0-bump-guest-components-917d5cf
versions: Bump guest-components
2023-10-13 22:21:25 +01:00
Steve Horsman
0eda83fa52 Merge pull request #8211 from BbolroC/sealed_secret_s390x
cc|rootfs: Define SEALED_SECRET for cc-rootfs-initrd-tarball
2023-10-13 15:53:57 +01:00
Matthew Arnold
2b3eb00db7 agent: enable signature-simple-xrss
Bump version of image-rs and reqwest.

Build image-rs with the signature-simple-xrss feature flag.

Fixes: #8221

Signed-off-by: Matthew Arnold <mattarno@uk.ibm.com>
2023-10-13 15:34:51 +01:00
stevenhorsman
7f89d291ee versions: Bump guest-components
- Bump guest components to pick up fix to issue in
https://github.com/confidential-containers/guest-components/pull/374

Fixes: #8223
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-10-13 14:19:26 +01:00
Steve Horsman
69b820a9bd Merge pull request #8209 from yoheiueda/CCv0-remote-secret
CCv0: runtime: fix k8s secret issue with remote hypervisor
2023-10-12 18:23:35 +01:00
Hyounggyu Choi
981f0a1f0f cc|rootfs: Define SEALED_SECRET for cc-rootfs-initrd-tarball
This is to define `SEALED_SECRET` as yes for a make target `cc-rootfs-initrd-tarball`,
which makes a service `confidential-data-hub` available with an initrd-based VM creation.

Fixes: #8210

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-10-12 14:28:46 +02:00
Yohei Ueda
d9c9040474 runtime: fix k8s secret issue with remote hyp
kata-shim CCv0 does not propagate dynamically
updated k8s secret values due to incorrect
file name matching. This patch fixes the the wrong file
name matching for k8s secret volume paths.

Note that this problem has already fixed in the main
branch.

Fixes: #8208
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
2023-10-12 15:07:10 +09:00
Steve Horsman
64392c9a87 Merge pull request #8173 from bpradipt/gpu-fix
CC | config: Enable guestHook for remote hyp
2023-10-10 17:15:02 +01:00
Fabiano Fidêncio
548e4c1667 Merge pull request #8195 from fidencio/topic/CC-bump-td-shim
versions: bump td-shim version
2023-10-10 14:03:45 +02:00
Fabiano Fidêncio
a1eab1248c versions: bump td-shim version
We're doing so in order to avoid errors like the ones from here:
https://github.com/kata-containers/kata-containers/actions/runs/6468115049/job/17559485246

I was able to locally build it, but as we're not testing CLH + TDX
lately, running CI on this will be useless.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-10-10 13:53:40 +02:00
Fabiano Fidêncio
cf8f8bc2cf Merge pull request #8194 from fidencio/topic/CC-do-not-use-cache-when-building-the-payload
CC | payload-after-push: Bypass cache
2023-10-10 12:35:12 +02:00
Fabiano Fidêncio
18b378dbb4 payload-after-push: Bypass cache
We're facing errors in the operator CI, which are related to the
measured rootfs.

For now, let's skip the cache builds (as those were dropped anyways for
this branch) and ensure we do a clean build, and then check if the
problem persists.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-10-10 12:31:33 +02:00
Pradipta Banerjee
6be56addd8 config: Enable guestHook for remote hyp
The guesthook config was missing which prevented handling
of GPUs with remote hypervisor

Fixes: #8172

Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
2023-10-09 06:30:10 +00:00
Steve Horsman
1ad87faaf4 Merge pull request #8053 from ChengyuZhu6/metadata
CC | runtime: merge metadata and annotation in imagepullvolume
2023-09-26 15:30:34 +01:00
Steve Horsman
35b7b0379c Merge pull request #8050 from mattarnoatibm/CCv0-set-policy-path-from-agent-config
CC | policy: fix setting the policy path from agent config
2023-09-26 10:47:43 +01:00
Fabiano Fidêncio
1f0cc490bd Merge pull request #7715 from fidencio/topic/CCv0-configure-snapshotter-as-part-of-kata-deploy
CC: kata-deploy: Set the snapshotter in the containerd runtime config
2023-09-26 11:16:03 +02:00
ChengyuZhu6
0ad5dff3b8 runtime: merge metadata and annotation in imagepullvolume
merge metadata and annotation in imagepullvolume

Fixes #8060

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-26 08:51:44 +08:00
Fabiano Fidêncio
5e6cecf01a kata-deploy: Remove cri_handler = cc
This won't be used anymore as we won't be depending on the forked
version of containerd.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-25 22:58:59 +02:00
Fabiano Fidêncio
9fb0eb4076 CC: kata-deploy: Set the snapshotter in the containerd runtime config
This is a patch that should **NOT** be forward ported to main, as there
we want to take a cleaner approach on configuring specific snapshotters
for specific runtime handlers.

However, for CC, for the v0.8.0 release of CC, this is good enough as it
is, and it'll allow us to set one snapshotter for all the deployments
done with the CoCo Operator.

This is the Kata Containers counterpart of the work, and there's still
work to be done on the Confidential Containers in order to make it work
as expected, as:
* Confidential Containers Operator has to expose to the users which
  snapshotter will be configured
* Confidential Containers Opereator, specifically the pre-install hook,
  will have to take care of actually installing and configuring the
  snapshotter, so it can be used.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-25 22:57:18 +02:00
Matthew Arnold
7fc822896c policy: fix setting the policy path from agent config
Fix setting the image service policy path when there
is a policy path in the agent config.

Fixes #8049

Signed-off-by: Matthew Arnold <mattarno@uk.ibm.com>
2023-09-25 17:00:13 +01:00
Fabiano Fidêncio
b588c1128c Merge pull request #7676 from ChengyuZhu6/pull_image_in_guest
CC | image pulling in the guest without forked containerd
2023-09-20 21:11:24 +02:00
Fabiano Fidêncio
7ee7ca2b31 Merge pull request #8021 from fidencio/topic/CC-fix-sev-tdx-rootfs-cached-artefacts
CC | cache: Fix rootfs-image-tdx and rootfs-initrd-sev cached artefacts
2023-09-20 19:00:38 +02:00
Fabiano Fidêncio
3dbbbc88ac cache: Fix rootfs-image-tdx and rootfs-initrd-sev cached artefacts
The name of the tarballs changed on main, but we didn't follow up
changing this on the CCv0 branch. :-/

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-20 18:49:27 +02:00
Fabiano Fidêncio
6ae4951a8c Merge pull request #8010 from arronwy/reduce_binary_size
CC | osbuild: Reduce guest components binary size with strip
2023-09-20 13:32:39 +02:00
ChengyuZhu6
e18a425fbb static-build: Fix arch error on nydus build
Fix the arch error when downloading the nydus tarball.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Signed-off-by: Steven Horsman <steven@uk.ibm.com>
(cherry picked from commit f6df3d6efb)
2023-09-20 16:09:38 +08:00
Fabiano Fidêncio
e14424011c Merge pull request #7998 from fidencio/topic/CC-add-forked-vanilla-entries-for-containerd
CC | version: Specify forked / vanilla entries of containerd
2023-09-20 10:06:04 +02:00
Wang, Arron
58e8eed807 osbuild: Reduce guest components binary size with strip
Guest rootfs will aligned to 128M, we may exceed the rootfs
with several megabytes but the rootfs will add 128M.

Fixes: #8009

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2023-09-20 15:07:21 +08:00
Fabiano Fidêncio
6e784fb6b3 Merge pull request #7948 from stevenhorsman/api-server-rest-rootfs
osbuilder: Update api-server-rest
2023-09-20 08:47:04 +02:00
Fabiano Fidêncio
b07dfbe213 versions: Bump nydus and nydus-snapshotter to its latest release
As we need https://github.com/containerd/nydus-snapshotter/pull/530 in.

Fixes #7984

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-19 22:23:15 +02:00
Fabiano Fidêncio
6329c7e290 versions: Specify forked / vanilla entries of containerd
As we'll need to test with both the vanilla and the forked versions of
containerd, let's make sure we'll specify both entries as part of our
versions.yaml file, and we can read whatever we need accordingly as part
of our tests jobs.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-19 22:23:09 +02:00
Fabiano Fidêncio
d5d0befc61 build: Pass RUSTUP_UPDATE_ROOT & RUSTUP_DIST_SERVER down
This may help us to decrease the amount of issues we're having with the
TDX CI.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-18 21:05:11 +02:00
ChengyuZhu6
87b3f6a63c runtime: add functions to handle ImageGuestPull to storage
Add functions to handle ImageGuestPull of KataVirtualVolume.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-18 21:05:11 +02:00
ChengyuZhu6
53ea36d3f5 agent: Introduce ImagePullHandler to support IMAGE_GUEST_PULL volume
As we do not employ a forked containerd, we utilize the KataVirtualVolume
which storing the image url supplied by snapshotter as an integral part of `CreateContainer`.
Within this process, we store the image information in rootfs.storage and pass this image url through `CreateContainerRequest`.
This approach distinguishes itself from the use of `PullImageRequest`, as rootfs.storage is already set and initialized at this stage.
To maintain clarity and avoid any need for modification to the `OverlayfsHandler`,we introduce the `ImagePullHandler`.
This dedicated handler is responsible for orchestrating the image-pulling logic within the guest environment.
This logic encompasses tasks such as calling the image-rs to download and unpack the image into `/run/kata-containers/{container_id}/images`,
followed by a bind mount to `/run/kata-containers/{container_id}`.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-18 21:05:11 +02:00
ChengyuZhu6
d0ac25f5c5 agent: redefine functions of pulling image in the guest
Without using forked containerd, the kata-agent wouldn't receive the `PullImageRequest`.
To using nydus-snapshotter, kata-agent can pass the image url and container id to image-rs
to handle pulling image.So we need to redefine functions of pulling image in the guest to support
both PullImageRequest and remote snapshotter.
1) Extract codes for setting proxy environment variables into a separate function `set_proxy_env_vars`.
2) Create a separate function `handle_attestation_agent` to handle attestation agent
   initialization.
3) Create a separate function `common_image_pull` for image pull logic.
4) Extract codes for unpacking pause image into a separate function `unpack_pause_image` and pass the necessary parameters to customize the behavior.

Fixes #7790

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Jiang Liu <gerry@linux.alibaba.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
Co-authored-by: wllenyj <wllenyj@linux.alibaba.com>
Co-authored-by: jordan9500 <jordan.jackson@ibm.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
2023-09-18 21:05:11 +02:00
stevenhorsman
0ffc9c02a2 agent: bump image-rs version
- Bump image-rs to stay aligned with guest-components version

Fixes: #7947
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-18 11:01:10 +01:00
stevenhorsman
560e21d8e3 versions: Bump guest-components
- Bump guest-components to pick up the new api-server-rest Makefile

Fixes: #7947
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-18 11:00:20 +01:00
Greg Kurz
87b8164add Merge pull request #7951 from BbolroC/reduce-redundancy-se-image
cc|gha: remove build redundancy of kernel and cc-rootfs-initrd for s390x
2023-09-18 10:12:24 +02:00
Fabiano Fidêncio
6b3c63570e Merge pull request #7981 from BbolroC/fix-agent-double-build-for-initrd
cc|osbuilder: Correct a typo in the initrd-image
2023-09-16 00:39:35 +02:00
Hyounggyu Choi
5e5e78cd6f cc|osbuilder: Correct a typo in the initrd-image
This PR is to prevent rootfs.sh from running twice by fixing the typo `initrd-image`.

Fixes: #7980

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-09-15 19:18:31 +02:00
Hyounggyu Choi
4f1d631af1 cc|gha: remove build redundancy of kernel and cc-rootfs-initrd for s390x
This PR is to remove the build redundancy of `kernel` and `cc-rootfs-initrd` by making `cc-se-image` built based on them at the second build stage.

Fixes: #7949

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-09-14 16:48:34 +02:00
stevenhorsman
4662a7a942 osbuilder: Update api-server-rest
- Switch api-server-rest to use the Makefile rather than
directly calling cargo for multi-platform support and decoupling

Fixes: #7947
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-14 11:24:54 +01:00
Fabiano Fidêncio
ee15a389de Merge pull request #7688 from ChengyuZhu6/image_sharing_dmverity
CC | support dm-verity tarfs disk image in CoCo
2023-09-13 20:52:01 +02:00
Fabiano Fidêncio
b71443a7ae Merge pull request #7878 from ChengyuZhu6/check_rust
CC | osbuilder: check rust enviornment before building agent and AA
2023-09-13 17:15:59 +02:00
ChengyuZhu6
a533c974f9 agent: enable verity feature in image-rs
update image-rs to support verity feature

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 18:38:51 +08:00
Fabiano Fidêncio
d0b69651ce Merge pull request #7917 from stevenhorsman/cherry-pick-cgroups-fixes
agent: optimize the code of systemd cgroup manager
2023-09-13 09:53:46 +02:00
ChengyuZhu6
8f38dcb850 osbuilder: check rust enviornment before building agent and AA
We should configure the Rust environment when AGENT_SOURCE_BIN is empty or AA_KBC is not empty.

Fixes #7877

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 15:22:49 +08:00
Fabiano Fidêncio
f462c69484 Merge pull request #7880 from ChengyuZhu6/udev
image-builder: fix udev error when using docker to build image
2023-09-13 07:47:35 +02:00
ChengyuZhu6
622bd4e370 agent: create directories to mount filesystem by overlay
When creating a container with a raw disk image using virtio-blk,
the guest does not have the upper directory and worker directory present.
Therefore, it is necessary to create these directories before mounting the filesystem with overlay.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 10:01:12 +08:00
ChengyuZhu6
72c9f62b70 agent: introduce DmVerityHandler to support dm-verity volume
We utilize the KataVirtualVolume which storing the dm-verity info
and the path of disk image on the host supplied by snapshotter as an integral part of `CreateContainer`.
Within this process, we copy the verity info and the disk image path to mount slice to create a block device by virtio-blk.
Then storing the `lowerdir` in rootfs.storage which is the mountpoint of the verity path through `CreateContainerRequest`.
To maintain clarity and avoid any need for modification to the `VirtioBlkPciHandler`,we introduce the `DmVerityHandler`.
This dedicated handler is responsible for calling image-rs to create verity device and mount the device to the `lowerdir` within the guest environment.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 10:01:11 +08:00
ChengyuZhu6
fd33309475 image-builder: fix udev error when using docker to build image
Incorporate the `DM_VERITY` parameter when building the image with docker.

Fixes #7879

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 09:34:22 +08:00
ChengyuZhu6
e36c2b6249 runtime: support to create VirtualVolume rootfs storages
1) Creating storage for each `extraoption` in rootFs.Options,
and then aggregates all storages  into `containerStorages`.
2) Creating storage for other data volumes and push them into `volumeStorages`.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 09:30:30 +08:00
ChengyuZhu6
5ad3eba8b1 runtime: redefine and add functions to handle VirtualVolume to storage
1) Extract function `handleBlockVolume` to create Storage only.
2) Add functions to handle KataVirtualVolume device and construct
   corresponding storages.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 09:30:30 +08:00
ChengyuZhu6
29eb2c02d9 runtime: extend SharedFile to support mutiple storage devices
To enhance the construction and administration of `Katavirtualvolume` storages,
this commit expands the 'sharedFile' structure to manage both
rootfs storages(`containerStorages`) including `Katavirtualvolume` and other data volumes storages(`volumeStorages`).

NOTE: `volumeStorages` is intended for future extensions to support Kubernetes data volumes.
Currently, `KataVirtualVolume` is exclusively employed for container rootfs, hence only `containerStorages` is actively utilized.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 09:30:30 +08:00
ChengyuZhu6
bedd536461 runtime: add functions to create devices in KataVirtualVolume
The snapshotter will place `KataVirtualVolume` information
into 'rootfs.options' and commence with the prefix 'io.katacontainers.volume='.
The purpose of this commit is to transform the encapsulated KataVirtualVolume data into device information.

Fixes #7792

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Feng Wang <feng.wang@databricks.com>
Co-authored-by: Samuel Ortiz <sameo@linux.intel.com>
Co-authored-by: Wedson Almeida Filho <walmeida@microsoft.com>
2023-09-13 09:30:30 +08:00
ChengyuZhu6
d788d4af2f runtime: Add KataVirtualVolume struct in runtime
Add the corresponding data structure in the runtime part according to
kata-containers/kata-containers/pull/7698.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 09:30:30 +08:00
Steve Horsman
51c665a09c Merge pull request #7918 from stevenhorsman/CCv0-bump-to-rust-1.72
CCv0 bump to rust 1.72
2023-09-12 20:20:40 +01:00
stevenhorsman
36431de30f versions: Bump rust version
Bump rust to 1.72.0 to test what extra warnings/issues we get

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
9ebb91f94c runk: Fix rust unecessary mut error
- Fix `error: variable does not need to be mutable`
in rust 1.72

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
f7fd2c1dfc kata-ctl: useless-vec warning
- Fix clippy::useless-vec warning

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
c0668ef7eb kata-ctl: Resolve non-minimal-cfg warning
- In rust 1.72, clippy warned clippy::non-minimal-cfg
as the cfg has only one condition, so doesn't
need to be wrapped in the any combinator.

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
0a33d27c30 agent-ctl: Allow clippy lint
- Allow `clippy::redundant-closure-call`
which has issues with the guard function passed into
the `run_if_auto_values` macro

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
48465d0547 runtime-rs: Fix useless-vec warning
Fix clippy::useless-vec warning

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
cb7cc1d708 runtime-rs: Remove mut
Fix `error: variable does not need to be mutable`

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
6e508ae322 dragonball: Allow ambiguous-glob-reexports
The bindgen generated code is triggering lots of
ambiguous-glob-reexports warnings in rust 1.70+

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
91916ed118 dragonball: Resolve non-minimal-cfg warning
- In rust 1.72, clippy warned clippy::non-minimal-cfg
as the cfg has only one condition, so doesn't
need to be wrapped in the all combinators.

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
d1d49675a0 agent: config: Allow clippy lint
- Allow `clippy::redundant-closure-call` in `from_cmdline`
which has issues with the guard function passed into
the `parse_cmdline_param` macro

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
3416e104b8 agent: config: Fix useles-vec warning
Fix clippy::useless-vec warning

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
495b9825e5 libs: Fix clippy unnecesary hashes error
- Fix error: unnecessary hashes around raw string literal

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 14:19:41 +01:00
stevenhorsman
da8d4a4584 agent: Vendor
run make vendor on agent to update

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 12:54:54 +01:00
Yuan-Zhuo
03aceccb1b agent: optimize the code of systemd cgroup manager
1. Directly support CgroupManager::freeze through systemd API.
2. Avoid always passing unit_name by storing it into DBusClient.
3. Realize CgroupManager::destroy more accurately by killing systemd unit rather than stop it.
4. Ignore no such unit error when destroying systemd unit.
5. Update zbus version and corresponding interface file.

Acknowledgement: error handling for no such systemd unit error refers to

Fixes: #7080, #7142, #7143, #7166

Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
(cherry picked from commit 470d065415)
2023-09-12 12:01:40 +01:00
Fabiano Fidêncio
70e3dc5acb Merge pull request #7891 from fidencio/topic/CC-update-kernel-to-the-latest-lts-plus-bring-in-erofs-patches
CC | Update kernel to the latest LTS release (v6.1.52) and bring in erofs patches needed for the CC work
2023-09-11 16:31:00 +02:00
Fabiano Fidêncio
bbe0db55ac kernel: Add erofs patches needed for CC related work
All the patches have already been merged upstream and they've just been
cherry-picked to this branch.

Fixes: #7885

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit fde34610cd)

 Conflicts:
	tools/packaging/kernel/kata_config_version
2023-09-11 13:21:21 +02:00
Fabiano Fidêncio
bea936b4a8 versions: Bump kernel to the latest LTS release (6.1.52)
We're bumping here in order to make our lives easier backporting EROFS
patches needed for the CC related work.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit dc6a4588a2)
2023-09-11 13:20:56 +02:00
Fabiano Fidêncio
3b212ec8ed Merge pull request #7883 from kata-containers/revert-7847-topic/CC-versions-update-kernel-to-6.4.14-stable
Revert "CC | versions: Bump to 6.4.14 stable kernel"
2023-09-11 12:24:44 +02:00
Fabiano Fidêncio
08b87a4eab Revert "CC | versions: Bump to 6.4.14 stable kernel" 2023-09-11 09:01:53 +02:00
Archana Shinde
911ab9c306 Merge pull request #7847 from fidencio/topic/CC-versions-update-kernel-to-6.4.14-stable
CC | versions: Bump to 6.4.14 stable kernel
2023-09-08 16:53:30 -07:00
Fabiano Fidêncio
d70ed93173 Merge pull request #7570 from LindaYu17/CCv0
CC | add sealed secret support in Kata
2023-09-07 09:46:33 +02:00
Biao Lu
13943fb81b agent: launch api-server-rest
If 'rest_api' is configured, start api-server-rest after
attestation-agent and confidential-data-hub.

Fixes: #7555

Signed-off-by: Biao Lu <biao.lu@intel.com>
2023-09-07 09:55:46 +08:00
Biao Lu
e865359f4e osbuilder: add api-server-rest in rootfs
Integrate api-server-rest into rootfs image.

Fixes: #7555

Signed-off-by: Biao Lu <biao.lu@intel.com>
2023-09-07 09:55:46 +08:00
Biao lu
47c28923d7 agent: Add config for api-server-rest
Add configuration for 'rest api server'.
Optional configurations are
  'agent.rest_api=attestation' will enable attestation api
  'agent.rest_api=resource' will enable resource api
  'agent.rest_api=all' will enable all (attestation and resource) api

Fixes: #7555

Signed-off-by: Biao lu <biao.lu@intel.com>
2023-09-07 09:55:46 +08:00
Linda Yu
212229df83 runtime: add sealed secret configuration
Fixes: #7555

Signed-off-by: Linda Yu <linda.yu@intel.com>
2023-09-07 09:55:46 +08:00
Linda Yu
f1573b4747 agent: unittest for sealed secret as file in kata
Fixes: #7555

Signed-off-by: Linda Yu <linda.yu@intel.com>
2023-09-07 09:55:46 +08:00
Linda Yu
d7873e5251 agent: support sealed secret as file in kata
Fixes: #7555

Signed-off-by: Linda Yu <linda.yu@intel.com>
2023-09-07 09:55:40 +08:00
Linda Yu
c60adedf99 agent: add feature for confidential data hub (cdh)
Fixes: #7555

Signed-off-by: Linda Yu <linda.yu@intel.com>
2023-09-07 09:00:25 +08:00
Linda Yu
9c02722d46 agent: unittest for sealed secret as env in kata
Fixes: #7555

Signed-off-by: Linda Yu <linda.yu@intel.com>
2023-09-07 09:00:25 +08:00
Linda Yu
75def881e5 agent: support sealed secret as env in kata
Fixes: #7555

Signed-off-by: Linda Yu <linda.yu@intel.com>
2023-09-07 09:00:25 +08:00
Biao Lu
5316839165 agent: launch confidential-data-hub
confidential-data-hub depends attestation-agent, and
confidential-data-hab need to start before rpc server, so move the
function 'init_attestation_agent' from image_rpc.rs to main.rs and
launch confidential-data-hub after 'init_attestation_agent'.

Fixes: #7544

Signed-off-by: Biao Lu <biao.lu@intel.com>
2023-09-07 08:59:31 +08:00
Fabiano Fidêncio
7cc29708a4 Merge pull request #7851 from BbolroC/hotfix-dockerbuild-s390x
CCv0: packaging: do not install docker-compose-plugin for s390x|ppc64le
2023-09-06 15:40:46 +02:00
Hyounggyu Choi
c3a8ce53e6 CCv0: packaging: do not install docker-compose-plugin for s390x|ppc64le
This PR is to skip installing docker-compose-plugin while buiding a `build-kata-deploy` image for s390x|ppc64le.
It is a temporary solution to fix current CI failures for s390x regarding `hash sum mismatch`.

Fixes: #7848
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
(cherry picked from commit 2efda20c77)
2023-09-06 13:16:16 +02:00
Fabiano Fidêncio
f64041e686 kernel: Add more configs to the whitelist
This is a partial backport of 8115a0522d,
which added those configs to the whitelist.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-06 11:55:37 +02:00
Fabiano Fidêncio
57001431b4 versions: Bump to 6.4.14 stable kernel
This kernel update is needed in order to get the latest and greatest
commits related to EROFS, which will be used for allowing sharing the
container images between the guest and host for Confidential Containers
using the tarfs mode of EROFS.

We're removing a few options here, because:
* SECURITY_SELINUX_CHECKREQPROT_VALUE was deprecated as part of
  a7e4676e8e2c.
* CONFIG_IP_NF_TARGET_CLUSTERIP was removed as part of 9db5d918e2c0.
* CONFIG_NET_SCH_CBQ was removed as part of 051d44209842.

Fixes: #7845
Backports: #7846

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-06 11:03:31 +02:00
Biao Lu
b4092023bf osbuilder: add confidential-data-hub in rootfs
Fixes: #7544

Signed-off-by: Biao Lu <biao.lu@intel.com>
2023-09-06 10:57:34 +08:00
Biao Lu
acd0a75efd agent: rootfs: add sealed-secret in Makefile
When set SEALED_SECRET to "yes", the kata-agent is built with
sealed-secret capability, default value is "no".

Fixes: #7544

Signed-off-by: Biao Lu <biao.lu@intel.com>
2023-09-06 10:57:34 +08:00
Biao Lu
4e3a1ebcaf protocols: add support sealed_secret
To call CDH ttrpc API, 'unseal_secret' for 'sealed_secret', add
protocol file and generate ttrpc code.

Fixes: #7544

Signed-off-by: Biao Lu <biao.lu@intel.com>
2023-09-06 10:57:34 +08:00
Fabiano Fidêncio
83b020f4a3 Merge pull request #7826 from jiangliu/cherry2
CC | cherry-pick #7819 and #7821 from main branch
2023-09-03 00:33:56 +02:00
Jiang Liu
f45ee1fe1d agent: refine StorageDeviceGeneric::cleanup()
Refine StorageDeviceGeneric::cleanup() to improve safety.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-09-02 17:05:29 +08:00
Jiang Liu
1f4facdfe9 agent: implement StorageDeviceGeneric::cleanup()
Refactor cleanup_sandbox_storage as StorageDeviceGeneric::cleanup().

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-09-02 17:05:16 +08:00
Jiang Liu
d955f9dcf8 types: make StorageDevice::cleanup() return possible error code
Make StorageDevice::cleanup() return possible error code.

Fixes: #7818

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-09-02 17:04:55 +08:00
Jiang Liu
039fde2b66 agent: move StorageDeviceGeneric from kata-types into agent
Move StorageDeviceGeneric from kata-types into agent, so we can
refactor code later.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-09-02 17:04:36 +08:00
Jiang Liu
eac38d1a05 agent: avoid possible leakage of storage device
When a storage device is used by more than one container, the second
and forth instances will cause storage device reference count leakage,
thus cause storage device leakage. The reason is:
add_storages() will increase reference count of existing storage device,
but forget to add the device to the `mount_list` array, thus leak the
reference count.

Fixes: #7820

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-09-02 17:04:16 +08:00
Fabiano Fidêncio
bb4f0a9263 Merge pull request #7801 from fidencio/topic/error-out-if-attestation-agent-fails-to-build
CC | rootfs: Fail in case attestation-agent fails to build
2023-08-31 08:08:49 +02:00
Fabiano Fidêncio
af413550dd Merge pull request #7797 from stevenhorsman/bump-guest-components-2d8dcd3
versions: Bump guest-components
2023-08-30 18:03:50 +02:00
Fabiano Fidêncio
badba8058c rootfs: Fail in case attestation-agent fails to build
Today I learned, I must say.

When running a basic script, such as:
```bash
 #/usr/bin/env bash

 set -o errexit
 set -o pipefail
 set -o errtrace

 cat junk && echo "hello"
 echo "didn't fail"

 cat junk
 echo "hello"
 echo "didn't fail"
```

One will get as a result:
```bash
cat: junk: No such file or directory
didn't fail
cat: junk: No such file or directory
```

Meaning that although there was an error on `cat junk && echo "hello"`,
and the `echo "hello"` part was not executed, an error was not reported
for that failure.

On the second part, though, it just breaks and returns an error as
expected.

Small scripts aside, this is exactly what was happening with the
attestation-agent, where a `make ... && make install ...` was being
called, make was failing but not actually breaking the script.

Let's change the logic and avoid such situations in the future, as it
caused our CI to be broken for quite some time without a simple way to
detect that line in the huge amount of logs left behind.

Here goes a reference to the documentation:
```
-e      Exit immediately if a pipeline (which may consist
        of a single simple command), a list, or a compound
        command (see SHELL GRAMMAR above), exits with a
        non-zero status.  The shell does not exit if the
        command that fails is part of the command list
        immediately following a while or until keyword,
        part of the test following the if or elif reserved
        words, part of any command executed in a && or ||
        list except the command following the final && or
        ||, any command in a pipeline but the last, or if
        the command's return value is being inverted with
        !.  If a compound command other than a subshell
        returns a non-zero status because a command failed
        while -e was being ignored, the shell does not
        exit.  A trap on ERR, if set, is executed before
        the shell exits.  This option applies to the shell
        environment and each subshell environment
        separately (see COMMAND EXECUTION ENVIRONMENT
        above), and may cause subshells to exit before
        executing all the commands in the subshell.

        If a compound command or shell function executes
        in a context where -e is being ignored, none of
        the commands executed within the compound command
        or function body will be affected by the -e
        setting, even if -e is set and a command returns a
        failure status.  If a compound command or shell
        function sets -e while executing in a context
        where -e is ignored, that setting will not have
        any effect until the compound command or the
        command containing the function call completes.
```

This comes from https://www.man7.org/linux/man-pages/man1/bash.1.html

Fixes: #7793

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-30 17:27:27 +02:00
stevenhorsman
c888facd24 versions: Bump guest-components
Bump image-rs and attestation-agent to use the latest guest-components
with the rust clap version fix

Fixes: #7580
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-08-30 14:33:35 +01:00
Jiang Liu
de1fe7bed0 Merge pull request #7767 from jiangliu/pick-7602-2
CC | cherry pick #7602 from main into CCv0
2023-08-29 01:39:32 +08:00
Jiang Liu
412e8554f3 agent: simplify storage device by removing StorageDeviceObject
Simplify storage device implementation by removing StorageDeviceObject.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:20 +08:00
Jiang Liu
d5483aaf7c agent: move storage device related code into dedicated files
Move storage device related code into dedicated files.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:20 +08:00
Jiang Liu
3b1af40b16 agent: refine storage related code a bit
Refine storage related code by:
- remove the STORAGE_HANDLER_LIST
- define type alias
- move code near to its caller

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:19 +08:00
Jiang Liu
331e35bc1a agent: switch to new storage subsystem
Switch to new storage subsystem to create a StorageDevice for each
storage object.

Fixes: #7614

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:19 +08:00
Jiang Liu
aee71b16f1 kata-types: introduce StorageDevice and StorageHandlerManager
Introduce StorageDevice and StorageHandlerManager, which will be used
to refine storage device management for kata-agent.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:19 +08:00
Jiang Liu
0d5a9eaeff agent: simplify the way to manage storage object
Simplify the way to manage storage objects, and introduce
StorageStateCommon structures for coming extensions.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:19 +08:00
Jiang Liu
6dcd164c4e sys-util: support more mount flags in parse_mount_options()
Support more mount flags in parse_mount_options().

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:19 +08:00
Jiang Liu
cf15777edd agent: use create_mount_destination() from kata-sys-util
Use create_mount_destination() from kata-sys-util crate to reduce
redundant code.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:19 +08:00
Jiang Liu
7f12e27a68 types: add more mount related constants
Add more mount related constants.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:19 +08:00
Jiang Liu
67748bde6c Merge pull request #7763 from jiangliu/pick-7602
CC | cherry-pick #7602 from main branch
2023-08-28 09:50:29 +08:00
Jiang Liu
b6218beef6 agent: use function from kata-sys-utils to reduce code
Use function get_linux_mount_info() from kata-sys-util crate to share
common code.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-25 22:23:03 +08:00
Fabiano Fidêncio
dfb38245e7 Merge pull request #7749 from fidencio/topic/CC-backport-fix-to-allow-building-the-images
CC | Backport | local-build: Remove GID before creating group
2023-08-24 14:23:18 +02:00
Jeremi Piotrowski
4417641803 local-build: Remove GID before creating group
docker install now creates a group with gid 999 which happens to match what we
need to get docker-in-docker to work. Remove the group first as we don't need
it.

Fixes: #7726
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
(cherry picked from commit 3b881fbc0e)
2023-08-24 14:17:58 +02:00
Fabiano Fidêncio
9d4ec379b1 Merge pull request #7748 from fidencio/topic/CC-kata-deploy-dont-try-to-remove-opt-kata
CC | backport | kata-deploy: Don't try to remove /opt/kata
2023-08-24 14:15:25 +02:00
Fabiano Fidêncio
a7f01b4456 kata-deploy: Don't try to remove /opt/kata
The directory is a host path mount and cannot be removed from within the
container.  What we actually want to remove is whatever is inside that
directory.

This may raise errors like:
```
rm: cannot remove '/opt/kata/': Device or resource busy
```

Fixes: #7746

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-24 14:01:45 +02:00
Jiang Liu
bb644ee3ed Merge pull request #7744 from jiangliu/pick-7698
CC | cherry-pick PR #7698 into CCv0
2023-08-24 15:31:42 +08:00
Fabiano Fidêncio
aef93c7aaf Merge pull request #7576 from surajssd/update-aa-imagers-tdshim
Update AA, image-rs td-shim
2023-08-24 08:38:29 +02:00
Jiang Liu
15d1b2431c kata-types: implement serde methods for KataVirtualVolume
Implement serilization/deserialization methods for KataVirtualVolume.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-24 13:02:56 +08:00
Jiang Liu
61340c3d63 kata-types: validate KataVirtualVolume object
Implement method validate() for KataVirtualVolume to validate message
format.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-24 13:02:42 +08:00
Jiang Liu
6d07df4b15 kata-types: implement two conversion helpers for KataVirtualVolume
Enable conversions from NydusExtraOptions/DirectVolumeMountInfo to
KataVirtualVolume.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-24 13:02:28 +08:00
Jiang Liu
7e553e6707 kata-types: introduce KataVirtualVolume
Introduce structure KataVirtualVolume to to encapsulate information
for extra mount options and direct volumes, so we could build a common
infrastructure to handle these cases.

Fixes: #7699

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-24 13:02:11 +08:00
Fabiano Fidêncio
edf51c83c0 Merge pull request #7739 from fidencio/topic/CC-fix-uninstall-issues
CC | backport | kata-deploy: Avoid failing on content removal
2023-08-24 00:16:06 +02:00
Fabiano Fidêncio
b64891c5f5 kata-deploy: Avoid failing on content removal
We can simply use `rm -f` all over the place and avoid the container
returning any error.

Fixes: #7733

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 5cba38c175)
2023-08-23 20:05:01 +02:00
Steve Horsman
0e9a8f22ca Merge pull request #7601 from ChengyuZhu6/install_dmsetup
CC | tools: Install dependencies with dm-verity in rootfs
2023-08-23 17:24:43 +01:00
Suraj Deshmukh
d8953498c6 CCv0: Update image-rs, AA and td-shim
- image-rs & AA: 3d8192f8d3efab041916ea4d60e32248ac6ec43d
- td-shim: 35c8ec33311877f0711412fd34cee929ae57e80e

Fixes: #7580

Signed-off-by: Suraj Deshmukh <suraj.deshmukh@microsoft.com>
2023-08-23 14:34:07 +00:00
Jiang Liu
cfba372f17 Merge pull request #7635 from jiangliu/image-service-singleton
CC | move image service related code into image-rpc.rs
2023-08-21 22:01:46 +08:00
Jiang Liu
f218a3104e agent/image: move image service related code into image-rpc.rs
Move image service related code into image-rpc.rs, to simplify
maintenance.

Fixes: #7633

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Co-authored-by: wllenyj <wllenyj@linux.alibaba.com>
Co-authored-by: jordan9500 <jordan.jackson@ibm.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
2023-08-21 19:29:09 +08:00
Jiang Liu
81980388d4 agent/image: export the image service singleton instance
Export the image service singleton instance.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-21 19:28:44 +08:00
Jiang Liu
624d3c063a agent/image: syntax only change to image service implementation
Syntax only change to image service implementation.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-21 19:28:32 +08:00
Fabiano Fidêncio
2a084ecbef Merge pull request #7682 from sprt/backport-ci-fixes
CC | kata-deploy: Properly create default runtime class
2023-08-18 07:47:39 +02:00
Aurélien Bombo
bc685665c6 tests: k8s: Call ensure_yq() in setup.sh
It wasn't the `common.bash` import in `run_kubernetes_tests.sh` causing
the yq error so let's try this instead.

Reference: https://github.com/kata-containers/kata-containers/actions/runs/5674941359/job/15379797568#step:10:341

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-08-17 10:28:58 -07:00
Aurélien Bombo
723c44a7c4 kata-deploy: Properly create default runtime class
The default `kata` runtime class would get created with the `kata`
handler instead of `kata-$KATA_HYPERVISOR`. This made Kata use the wrong
hypervisor and broke CI.

Fixes: #7681

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-08-17 10:28:58 -07:00
ChengyuZhu6
d053f848b4 tools: Install the dependencies with dm-verity
Fixes #7636

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-08-16 21:47:52 +08:00
Suraj Deshmukh
32d347aa25 tools/static-checks: Install clang
Without this library the builds are failing with the following error:

```
...
error: failed to run custom build command for `devicemapper-sys v0.1.5`

Caused by: process didn't exit successfully:
    `/kata-containers/src/agent/target/release/build/devicemapper-sys-d8eae524a127e049/build-script-build`
    (exit status: 101) --- stderr thread 'main' panicked at 'Unable to
    find libclang: "couldn't find any valid shared libraries matching:
    ['libclang.so', 'libclang-*.so', 'libclang.so.*', 'libclang-*.so.*'],
    set the `LIBCLANG_PATH` environment variable to a path where one of
    these files can be found (invalid: [])"',
    /root/.cargo/registry/src/github.com-1ecc6299db9ec823/bindgen-0.63.0/./lib.rs:2338:31
```

Fixes: #7580

Signed-off-by: Suraj Deshmukh <suraj.deshmukh@microsoft.com>
2023-08-16 13:12:49 +00:00
Suraj Deshmukh
1ec85d7485 static-checks: Install devmapper libraries
After image-rs added the image-block-device integrity check using
dm-verity a new dependency is now needed, so install that.

Refer the following PR for more information:
https://github.com/confidential-containers/guest-components/pull/270

Fixes: #7580

Signed-off-by: Suraj Deshmukh <suraj.deshmukh@microsoft.com>
2023-08-16 13:12:49 +00:00
Fabiano Fidêncio
3930a62c41 Merge pull request #7639 from fidencio/topic/merge-from-main-Aug-13th
CC | Merge from main to CCv0 - Aug 14th, 2023
2023-08-14 14:25:39 +02:00
Fabiano Fidêncio
516468815e cc: Merge from main to CCv0 - Aug 14th
Conflicts:
	src/agent/src/rpc.rs

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-14 09:22:03 +02:00
Fabiano Fidêncio
30f504e962 Merge pull request #7589 from stevenhorsman/ccv0-remote-hypervsior-annotations
Re-added hypervisor annotations
2023-08-10 11:05:57 +02:00
Xuewei Niu
6b48ac63ba deps: Bump dependent crate versions
This pull request is mainly for updating vm-memory and vmm-sys-util.

The affacted crates include:

- vm-memory: from 0.9.0 to 0.10.0
- vmm-sys-util: from 0.10.0 to 0.11.0
- virtio-queue: from 0.6.0 to 0.7.0
- fuse-backend-rs: from 0.10.4 to 0.10.5
- linux-loader: from 0.6.0 to 0.8.0
- nydus-api: from 0.3.0 to 0.3.1
- nydus-rafs: from 0.3.1 to 0.3.2
- nydus-storage: from 0.6.3 to 0.6.4

Fixes: #0000

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
(cherry picked from commit b23c5ed155)
2023-08-08 14:17:31 +01:00
stevenhorsman
a0ebfbf18a runtime: Re-added hypervisor annotations
- Add support for setting the sandbox name and namespace
in the hypervisor config, which is needed in the remote hypervisor
implementation to get the pod name and namespace for the remote pod
create request

Fixes: #7588
Co-authored-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Co-authored-by: Yohei Ueda <yohei@jp.ibm.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-08-08 14:04:37 +01:00
Fabiano Fidêncio
d0abf45ed1 Merge pull request #7564 from fidencio/topic/merge-from-main-Aug-7th
CC | Merge from main to CCv0 -- Aug 7th, 2023
2023-08-07 23:08:12 +02:00
Fabiano Fidêncio
5f5e05a77f CC: Merge from main to CCv0 - Aug 7th, 2023
Conflicts:
	src/runtime/pkg/containerd-shim-v2/create.go
	tools/packaging/kata-deploy/local-build/kata-deploy-binaries.sh
	tools/packaging/scripts/lib.sh

Fixes: #7563
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-07 11:12:04 +02:00
Fabiano Fidêncio
0bee9f199d Merge pull request #7549 from fidencio/topic/add-missing-runtimeclasses
CC | Add missing runtime classes
2023-08-07 10:52:04 +02:00
Fabiano Fidêncio
2df6cb7609 kata-deploy: Add missing kata-remote runtimeclass
It's CCv0 specific for now, and it's needed as the Operator is now
delegating the runtimeclass creation to the kata-deploy daemonset.

Fixes: #7550

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-07 10:24:21 +02:00
Fabiano Fidêncio
83e866a37d kata-deploy: Add missing kata-qemu-se runtimeclass
It's CCv0 specific for now, and it's needed as the Operator is now
delegating the runtimeclass creation to the kata-deploy daemonset.

Fixes: #7550

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-05 20:23:54 +02:00
Fabiano Fidêncio
bde0e72da5 kata-deploy: Add missing kata-clh-tdx runtimeclass
It's CCv0 specific for now, and it's needed as the Operator is now
delegating the runtimeclass creation to the kata-deploy daemonset.

Fixes: #7550

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-05 20:23:42 +02:00
Fabiano Fidêncio
44eb19841a cc: gha: Fix rootfs relatex names
Steve pointed this out, and I was able to get it fixed as part of
cc-payload-amd64.yaml but I missed the cc-payload-after-push-amd64.yaml
one.

Fixes: #7433

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-05 09:20:18 +02:00
Fabiano Fidêncio
d5d9f03e85 Merge pull request #7513 from fidencio/topic/merge-from-main-Aug-1st
CCv0 | CCv0: Merge from main -- August 1st
2023-08-05 09:11:16 +02:00
Fabiano Fidêncio
7164ced4dc CCv0: Merge from main -- August 1st
Conflicts:
	src/runtime/pkg/katautils/config.go
	src/runtime/virtcontainers/container.go
	src/runtime/virtcontainers/hypervisor.go
	src/runtime/virtcontainers/qemu_arch_base.go
	src/runtime/virtcontainers/sandbox.go
	tests/integration/kubernetes/gha-run.sh
	tests/integration/kubernetes/setup.sh
	tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml
	tools/packaging/kata-deploy/local-build/kata-deploy-binaries.sh
	tools/packaging/kata-deploy/scripts/kata-deploy.sh
	tools/packaging/kernel/kata_config_version
	versions.yaml

Fixes: #7433

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-04 22:15:09 +02:00
Fabiano Fidêncio
431c3630f2 Merge pull request #7390 from ChengyuZhu6/add_to_configure_request_timeout
runtime: Configure the image request timeout to handle large workloads
2023-07-28 18:54:57 +02:00
Fabiano Fidêncio
78522c5802 Merge pull request #7484 from fidencio/topic/CCv0-converge-build-and-payload-scripts-follow-up-9
kata-deploy: We need shim to an array, not a string
2023-07-28 15:54:42 +02:00
Fabiano Fidêncio
288296dacd kata-deploy: We need shim to an array, not a string
In order to do so, we need the `()` around the `shim_{arch}`.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 15:51:45 +02:00
Fabiano Fidêncio
607c87ef94 Merge pull request #7482 from fidencio/topic/CCv0-converge-build-and-payload-scripts-follow-up-8
kata-deploy: Fix if-elif-else-statement
2023-07-28 10:59:03 +02:00
Fabiano Fidêncio
40e678164a kata-deploy: Fix if-elif-else statement
We were doing "if - else  if - else", while bash expects "if - elif -
else", and that should never have happened in the first place, but it
happend as part of b8b73939ea

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-28 10:36:57 +02:00
Fabiano Fidêncio
3b957c7ec3 Merge pull request #7477 from fidencio/topic/CCv0-converge-build-and-payload-scripts-follow-up-7
CC | kata-deploy: Use different shim arrays for different arches
2023-07-27 23:00:00 +02:00
Fabiano Fidêncio
b8b73939ea kata-deploy: Use different shim arrays for different arches
On main we will not have this problem as we can easily configure which
shims will be installed according to an environment variable passed to
the kata-deploy.yaml file.

However, on CCV0, at least for now, we better keep the list of shims
separated by architecture, as we've found out that s390x CoCo Operator
CI is breaking because we try to install a shim that's not even built
for that architecture (dragonball).

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-27 22:57:42 +02:00
ChengyuZhu6
57b932c127 kata-runtime: Add configurable image request timeout
Add ImageRequestTimeout field in the config struct, set RequestTimeout
by configured image request timeout, add image_request_timeout to
default configuration files, add image request timeout to annotations
and add image timeout annotation to sandbox config documentation.

exp:

configure the image request timout in the configuration:
[image]
image_request_timeout = 300

configure the image request timeout in the yaml:
annotations:
      "io.katacontainers.config.runtime.image_request_timeout": "300"

Fixes: #7389

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-07-27 18:18:54 +02:00
Fabiano Fidêncio
e0bcb39ee7 Merge pull request #7471 from fidencio/topic/CCv0-converge-build-and-payload-scripts-follow-up-6
CC | kata-deploy: Add the runtime-classes that are not yet on main
2023-07-27 18:12:37 +02:00
Fabiano Fidêncio
03478ad064 kata-deploy: Add the runtime-classes that are not yet on main
This is another piece that got dropped as part of
6f552b010c and is causing regressions on
the operator tests.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-27 17:02:18 +02:00
Fabiano Fidêncio
61cbae6c39 Merge pull request #7463 from fidencio/topic/CCv0-converge-build-and-payload-scripts-follow-up-5
cc: kata-deploy: Configure cri-handler="cc"
2023-07-26 14:00:28 +02:00
Fabiano Fidêncio
e35b4cc9fb Merge pull request #7461 from jepio/kata-deploy-make-exec
CC | kata-deploy: Add executable bit to script
2023-07-26 13:00:25 +02:00
Fabiano Fidêncio
eff98f5795 cc: kata-deploy: Configure cri-handler="cc"
This has been mistakenly dropped as part of
441399df1f

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-26 12:57:03 +02:00
Jeremi Piotrowski
5ae7a74846 kata-deploy: Add executable bit to script
We need the executable bit set because it is preserved into the
runtime-payload-ci image.

Fixes: #7460
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-26 12:07:37 +02:00
Hyounggyu Choi
46f04d762f Merge pull request #7448 from fidencio/topic/CCv0-converge-build-and-payload-scripts-follow-up-4
cc: cache: shim-v2: Re-enable cached artefact
2023-07-26 08:18:53 +02:00
Fabiano Fidêncio
9ea161577f Merge pull request #7450 from BbolroC/fix-prefix-to-kata
CCv0: fix a prefix to kata for IBM SE image build
2023-07-26 00:23:40 +02:00
Hyounggyu Choi
a578266b26 CCv0: fix prefix to kata for IBM SE image build
This is to change a prefix from `confidential-containers` to `kata` for IBM SE image build.

Fixes: #7444

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-07-25 22:54:18 +02:00
Hyounggyu Choi
78262695d2 Merge pull request #7445 from BbolroC/remove-cc-from-kernel-ibm-se
CCv0: Remove `cc` from kernel for IBM SE image build
2023-07-25 21:52:01 +02:00
Fabiano Fidêncio
f6197f60b4 cc: cache: shim-v2: Re-enable cached artefact
Now that the shim-v2 for CCv0 has been rebuilt with the correct path,
let's re-enable the cache.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 21:48:20 +02:00
Fabiano Fidêncio
ce926439f0 Merge pull request #7443 from fidencio/topic/CCv0-converge-build-and-payload-scripts-follow-up-3
CC| cc: cache: Enable more cached components back and ensure shim-v2 can be properly cached
2023-07-25 20:32:02 +02:00
Hyounggyu Choi
1093e71cc5 CCv0: Remove cc from kernel for IBM SE image build
This is a quick fix for the error on IBM SE image build.

Fixes: #7444

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-07-25 19:46:31 +02:00
Fabiano Fidêncio
ad8c96b6c0 cc: cache: Enable more cached components after rebuild was done
Let's re-enabled caching for the following components, as those were
rebuilt with the new prefix:
* cc-rootfs-image
* cc-rootfs-initrd
* cc-tdx-rootfs-image
* cc-tdx-td-shim
* cc-sev-rootfs-initrd

"cc-se-image" was part of the list, but we never had a target for it.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 18:33:27 +02:00
Fabiano Fidêncio
2213660bf3 cc: cache: shim-v2: Allow root_hashes to be downloaded
We should not return, in case cache is not used, before actually
downloading the root_hash_*.txt provided by the other components,
otherwise the job used to do the caching will always fail.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 18:33:27 +02:00
Fabiano Fidêncio
7f1226ae2b Merge pull request #7440 from fidencio/topic/CCv0-converge-build-and-payload-scripts-follow-up-2
CC | Adjust cache URL for TDX components
2023-07-25 18:33:08 +02:00
Fabiano Fidêncio
471e23cb12 cc: kernel-tdx: Ensure we try the cache for the specific CC version
Otherwise we'd have to build the component every single time as the main
version is different from the CC one.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 13:06:43 +02:00
Fabiano Fidêncio
3f309fad01 cc: qemu-tdx: Ensure we try the cache for the specific CC version
Otherwise we'd have to build the component every single time as the main
version is different from the CC one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 13:01:15 +02:00
Fabiano Fidêncio
d03685004e cc: tdvf: Ensure we try the cache for the specific CC version
Otherwise we'd have to build the component every single time as the main
version is different from the CC one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 13:01:15 +02:00
Fabiano Fidêncio
0749022f8c cc: clh: Ensure we try the cache for the specific CC version
Otherwise we'd have to build the component every single time as the main
version is different from the CC one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 12:41:17 +02:00
Fabiano Fidêncio
19a86aa072 Merge pull request #7434 from fidencio/topic/CCv0-converge-build-and-payload-scripts-follow-up-1
gha: Fix typo in the workflow name
2023-07-25 10:02:11 +02:00
Fabiano Fidêncio
a40ea43413 gha: Fix typo in the workflow name
Kudos to Magnus who has a better vision than I do.
exprimental -> experimental

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 09:56:18 +02:00
Fabiano Fidêncio
eb82e0d4b9 Merge pull request #7402 from fidencio/topic/CCv0-converge-build-and-payload-scripts
CCv0 | Ensure kata-deploy scripts from CCv0 are as close to main as possible
2023-07-25 08:45:16 +02:00
Fabiano Fidêncio
068e535b9d runtime: tdx: Adjust QEMU TDX path
We need to use qemu-system-x86_64-tdx-experimental instead of
qemu-system-x86_64-tdx.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:39:52 +02:00
Fabiano Fidêncio
7204b991e7 kata-deploy-binaries: kernel_cache: Take module_dir into account
`module_dir` has been passed to the function but was never assigned to a
var, leading to errors when trying to use it.

Fixes: #7416

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit d4eba36980)
2023-07-25 00:19:26 +02:00
Fabiano Fidêncio
b8abd6bfee kata-deploy-binaries: Adjust TDVF edk2 tarball name
We must use "edk2-staging-tdx" instead of "edk2-tdx".  The reason for
that is versions diverging between main and CCv0.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:18:35 +02:00
Fabiano Fidêncio
0f022d5771 guest-image: Update kernel_module_dir to main sev kernel
As we're building SEV kernel from the main branch, we can stop relying
on the path produced by the one from the CCv0 branch (which is now
removed).

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:18:35 +02:00
Fabiano Fidêncio
344921849c kata-deploy-binaries: Temporarily disable using cached components
We need to rebuild those with the appropriate path.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:21 +02:00
Fabiano Fidêncio
507a89bb32 gha: cc-payload: Adjust to using main componenets
Again, it'll make our lives easier in the near future.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:21 +02:00
Fabiano Fidêncio
ef6c0be984 kata-depkoy-binarues: Add tarballs from main to the cc target
Same as the others, it'll help us in the merges.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
20a523f81b kata-deloy-binaries: Get rid of cc_prefix
We'll be using prefix (/opt/kata) from now on, as it simplifies things
on our side.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
4d0b319a8b kata-deploy-binaries: Remove CC OVMF / TDVF
Let's just rely on whatever we have on main.  The big execption here is
TDVF, but we have a big note saying to not update the version n this
branch.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
8d1e1d4b0a kata-deploy-binaries: Remove CC kernel builds
We can simply rely on those coming from main.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
3fa936e492 kata-deploy-binaires: Remove CC virtiofsd build
We can simply ship the one from main.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
f62a88f179 kata-deploy-binaries: Remove CC hypervisor builds
We can just rely on the hypervisors builds from `main`, with the TDX one
being the only discrepancy here.

However, we have a big note in the versions.yaml to **not** update the
TDX hypervisor versions on this branch, so we should be good.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
6f552b010c kata-deploy: Make sure kata-deploy handles kata-deploy-cc content
This will also help us immensely on main -> CCv0 merges

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
4e883fc5be versions: Converge to the same asset names used on main
This will make things easier in the future `main -> CCv0` merges.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:16 +02:00
Steve Horsman
16d6e37196 Merge pull request #7418 from wainersm/release_0.7.0-fix
workflows: fix malformed CC payload s390x yaml
2023-07-21 20:52:17 +01:00
Wainer dos Santos Moschetta
b366dc6015 workflows: fix malformed CC payload s390x yaml
The measured_boot matrix parameter should be a list.

Fixes #7400
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2023-07-21 16:09:11 -03:00
Fabiano Fidêncio
f5420e5cf1 Merge pull request #7401 from fitzthum/bump-070
CC | Update to v0.7.0 tags
2023-07-21 16:56:30 +02:00
Tobin Feldman-Fitzthum
e61974d81f versions: update td-shim to v0.7.0
v0.7.0 of td-shim has been released. Let's update to it.

Fixes: #7400

Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
2023-07-20 15:55:50 -05:00
Tobin Feldman-Fitzthum
e41306abad versions: update attestation-agent to v0.7.0
v0.7.0 of guest-components has been released, so let's
use the new tag for the attestation-agent

Fixes: #7400

Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
2023-07-20 15:53:54 -05:00
Tobin Feldman-Fitzthum
c02b6713bc agent: update image-rs to v0.7.0
v0.7.0 of image-rs has been tagged. Update to it.

Fixes: #7400

Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
2023-07-20 15:49:35 -05:00
Wainer Moschetta
9e036108b4 Merge pull request #7391 from jepio/manual-cc-push-action
gha: ci: cc-payload-after-push: Add workflow_dispatch trigger
2023-07-20 11:56:36 -03:00
Jeremi Piotrowski
2eee54de77 gha: ci: cc-payload-after-push: Add workflow_dispatch trigger
Allow triggering this action manually, since we noticed it being skipped on
push exactly when we needed it the most.

Fixes: #7353
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-20 15:33:22 +02:00
Fabiano Fidêncio
db0071422b Merge pull request #7332 from zvonkok/CCv0
CCv0: Adding CDI support for cold and hot-plug of VFIO devices
2023-07-19 22:34:28 +02:00
Fabiano Fidêncio
9a45a0a16f Merge pull request #7354 from jepio/update-deps-0.7.0
CC | Update deps 0.7.0 (rc)
2023-07-19 11:42:36 +02:00
Zvonko Kaiser
3b9f8fdbcb CCv0: Adding CDI support for cold and hot-plug of VFIO devices
We need to do proper sandbox sizing when we're doing cold-plug introduce CDI,
the de-facto standard for enabling devices in containers. containerd
will pass-through annotations for accumulated CPU,Memory and now CDI
devices. With that information sandbox sizing can be derived correctly.

Fixes: #7331

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2023-07-19 06:55:58 +00:00
Wainer dos Santos Moschetta
94b3ab8339 versions: migrate out of k8s.gcr.io
The k8s.gcr.io is deprecated for a while now and has been redirected to
registry.k8s.io. However on some bare-metal machines in our testing
pools that redirection is not working, so let's just replace the
registries.

Fixes #6461
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-18 15:17:59 +02:00
Jeremi Piotrowski
ef8b5ca32d versions: Update td-shim to 0.7.0-rc
Update td-shim to the commit that will become the v0.7.0-rc tag.

Fixes: #7353
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-18 15:17:56 +02:00
Jeremi Piotrowski
484e363e67 versions: Update attestation-agent to 0.7.0-rc
Update attestation-agent to the commit that will become the v0.7.0 tag.

Fixes: #7353
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-18 15:17:56 +02:00
Jeremi Piotrowski
a192971d72 agent: Update image-rs to 0.7.0-rc
Update image-rs, which is part of the guest-components repo, to the commit that
will become the v0.7.0 tag.

Fixes: #7353
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-18 15:17:56 +02:00
Wainer Moschetta
b2fdaf2e13 Merge pull request #7300 from stevenhorsman/CCv0-merge-10th-july
CCv0: Merge main into CCv0 branch
2023-07-18 09:42:43 -03:00
Steve Horsman
6f2c95a2ea Merge pull request #7351 from fitzthum/fix-offload
local-build: always use DEFSERVICEOFFLOAD for CoCo
2023-07-17 11:07:24 +01:00
Tobin Feldman-Fitzthum
5a78d90797 local-build: always use DEFSERVICEOFFLOAD for CoCo
DEFSERVICEOFFLOAD controls whether images are pulled inside
the guest. This should always be set for CoCo, not just
when we use MEASURED_ROOTFS.

Fixes: #7350

Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
2023-07-14 16:24:03 -05:00
Steve Horsman
109980b24f Merge pull request #7287 from Xynnn007/dep-guest-components
CCv0 | Update guest components dependency version
2023-07-12 16:51:54 +01:00
stevenhorsman
e16235584c agent: Update logger
`sl` was switched from a macro to a function,
so update the CoCo specifics uses of it

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-07-11 21:10:51 +01:00
stevenhorsman
68a364abfa agent: Reflect AGENT_CONFIG change
AGENT_CONFIG was changed to not be a lazy type, so
we need to remove the .read().await calls on it

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-07-11 20:12:50 +01:00
stevenhorsman
15647a000e runtime: Ignore cyclomatic complexity
Ignore cyclomatic complexity failure. I have fixed this in my PR waiting
to forward port remote-hypervisor support into main

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-07-11 19:55:36 +01:00
stevenhorsman
7188a60e25 runtime: Fix bad merge
- Fix the HotPlug type

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-07-11 19:47:45 +01:00
stevenhorsman
1c058d1d91 packaging: Bump kernel version
- Bump kernel version to reflect that they are changes
- We've some how gone out of sync with main, so just add a +

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-07-11 15:08:39 +01:00
stevenhorsman
f4d7011f3b CCv0: Merge main into CCv0 branch
- Merge remote-tracking branch 'upstream/main' into CCv0
- Note excludes 532755ce31 due to incompatiblity

Fixes: #7278
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-07-11 14:45:58 +01:00
stevenhorsman
e17587b023 Revert "tests: Build Mariner rootfs initrd"
This reverts commit 532755ce31.
2023-07-11 14:27:53 +01:00
Xynnn007
70c4df6d47 agent: update image-rs version
Update image-rs dep version the same as attestation-agent

Fixes: #7285

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
2023-07-11 15:19:50 +08:00
Xynnn007
7deafacdee version: Update attestation-agent
- Update attestation-agent to work with the KBS/AS to be released.

Fixes: #7285

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
2023-07-11 15:18:12 +08:00
Steve Horsman
f8a4966aaa Merge pull request #7266 from stevenhorsman/kata-deploy-remote-annotations
kata-deploy: Remove remote annotation logic
2023-07-07 16:18:34 +01:00
stevenhorsman
82d44320e6 kata-deploy: Remove remote annotation logic
Remove the logic that made the kata-remote containerd config not support
io.katacontainers annotations

Fixes: #7265
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-07-07 14:08:00 +01:00
Steve Horsman
61f3e58abf Merge pull request #7257 from stevenhorsman/remote-config-instance-type-support
config:  Update remote hypervisor config
2023-07-07 11:21:34 +01:00
stevenhorsman
335a456425 config: Update remote hypervisor config
- Add annotation enablement for machine_type, default_memory and
default_vcpus
- Remove note that says that cpu and memory settings are ignored.

Fixes: #7256
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-07-07 08:37:46 +01:00
Fabiano Fidêncio
80df87321f Merge pull request #7246 from fidencio/topic/cc-gha-fix-measured-rootfs-builds-follow-up-again-follow-up
cc: gha: Avoid building twice the components set as "include"
2023-07-06 14:12:04 +02:00
Fabiano Fidêncio
0425c4fe7c cc: gha: Avoid building twice the components set as "include"
Those components are being built twice, one as part of the normal matrix
assets, and the second one as part of the include.

Fixes: #7235

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 12:55:11 +02:00
Fabiano Fidêncio
a2218f0a99 Merge pull request #7244 from fidencio/topic/cc-gha-fix-measured-rootfs-builds-follow-up-again
cc: gha: Export MEASURE_ROOTFS=yes for rootfs-image builds
2023-07-06 12:06:02 +02:00
Fabiano Fidêncio
8d47e34558 cc: gha: Export MEASURE_ROOTFS=yes for rootfs-image builds
We need to export MEASURED_ROOTFS=yes for the rootfs-image builds, as
shown here[0], otherwise the root_hash.txt file won't be generated.

A huge thanks to Choi for quickly finding this out.

Fixes: #7235

[0]:
https://github.com/kata-containers/kata-containers/blob/CCv0/tools/osbuilder/image-builder/image_builder.sh#L507,

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 10:50:17 +02:00
Fabiano Fidêncio
a40cd94a66 Merge pull request #7241 from fidencio/topic/cc-gha-fix-measured-rootfs-builds-follow-up
cc: gha: Matrix "include" cannot be a list
2023-07-05 23:23:11 +02:00
Fabiano Fidêncio
5751781879 cc: gha: Matrix "include" cannot be a list
TIL that when using "include" for a matrix we must duplicate the value
we're overriding for each element we want that to happen.

Fixes: #7235

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-05 21:46:51 +02:00
Fabiano Fidêncio
2ae98093f6 Merge pull request #7237 from fidencio/topic/cc-gha-fix-measured-rootfs-builds
cc: gha: Pass MEASURED_ROOTFS to the artefacts build
2023-07-05 20:27:44 +02:00
Fabiano Fidêncio
8bc30b8a24 cc: gha: Pass MEASURED_ROOTFS to the artefacts build
Since the measured rootfs work has been merged to main, and then
brought in to the CCv0 via the weekly merge, we have introduced a few
regressions related to how we build it / use it.

This PR attempts to make sure the artefacts are properly built, using
GitHub Actions, so the feature can be used with the operator.

Fixes: #7235

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-05 17:10:39 +02:00
Fabiano Fidêncio
2cf48e4a55 Merge pull request #7234 from fidencio/topic/fix-cc-operator-uninstall-issue
cc: kata-deploy: Adapt the SNP's QEMU name
2023-07-05 13:39:35 +02:00
Fabiano Fidêncio
701a0defb6 cc: kata-deploy: Adapt the SNP's QEMU name
SNP's QEMU has changed its name some time ago and, due to that, we have
been leaving the new binary behind during the uninstall process, which
lead to the Operator hanging when uninstalling.

Fixes: #7233

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-05 12:04:14 +02:00
Fabiano Fidêncio
191071b10b Merge pull request #7228 from stevenhorsman/update-dependencies
Update dependencies after guest-components repo updates
2023-07-05 08:55:39 +02:00
stevenhorsman
aadc68633e agent: Update image-rs
- Update location and version of image-rs after
the repo merge

Fixes: #7152
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-07-04 17:07:00 +01:00
stevenhorsman
6e67d52242 version: Update attestation-agent
- Update attestation-agent to reflect the new URL and version

Fixes: #7152
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-07-04 16:56:27 +01:00
stevenhorsman
1fc8f7040f rootfs: Update attestation-agent install logic
- Update attestation-agent install now the create
is in the guest-components repo

Fixes: #7152
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-07-04 16:55:54 +01:00
Steve Horsman
bef48f4a9e Merge pull request #7205 from stevenhorsman/CCv0-merge-28th-june
CCv0: Merge main into CCv0 branch
2023-07-03 19:17:15 +01:00
stevenhorsman
e75435944f guest-image: Add failure check
Handle unknown image_initrd_suffix better

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-07-03 12:32:43 +01:00
stevenhorsman
75bdc94636 packaging: Bump kernel config_version
Bump kata_config_version as upstream has kernel changes

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-07-03 12:32:43 +01:00
stevenhorsman
1fefa3790a packaging: Fix bad merges
- Fix issues found with auto-merged

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-06-30 10:30:43 +01:00
stevenhorsman
51eb0c5130 runtime: SEV sysconfig fix
- SEV and SNP need a different sysconfig path

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-06-29 20:52:57 +01:00
stevenhorsman
6fee9fbe4e CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #7083
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-06-29 10:05:59 +01:00
Steve Horsman
70e6e40a8d Merge pull request #7134 from stevenhorsman/CCv0-merge-19th-june
CCv0: Merge main into CCv0 branch
2023-06-27 09:16:49 +01:00
GabyCT
4b8229c252 Merge pull request #7141 from bpradipt/fix-7140
runtime: Add support for key annotations to remote hyp
2023-06-23 16:47:20 -06:00
Steve Horsman
4ff3afc59d Merge pull request #6707 from Xynnn007/feat-policy-uri
agent: add container launch control parameters from kernel commandline
2023-06-21 17:02:46 +01:00
Pradipta Banerjee
004f07f076 runtime: Add support for key annotations to remote hyp
In order to support different pod VM instance type via
remote hypervisor implementation (cloud-api-adaptor),
we need to pass machine_type, default_vcpus
and default_memory annotations to cloud-api-adaptor.

The cloud-api-adaptor then uses these annotations to spin
up the appropriate cloud instance.

Reference PR for cloud-api-adaptor
https://github.com/confidential-containers/cloud-api-adaptor/pull/1088

Fixes: #7140
Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
2023-06-21 20:22:36 +05:30
stevenhorsman
5a4a89c108 runtime: Remove duplicated variables
Remove duplicated variables that were in `CCv0` and merged in from main

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-06-20 15:01:54 +01:00
stevenhorsman
a7fce537c7 kata-deploy: Add MEASURED_ROOTFS check
- Check MEASURED_ROOTFS is set before adding the cc_rootfs_verity
config

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-06-20 14:48:54 +01:00
stevenhorsman
6350f49baf agent-ctl: Re-vendor
Re-vendor after bad merge

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-06-19 11:58:43 +01:00
stevenhorsman
7fc10b975f agent: re-vendor
Re-vendor after bad merge

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-06-19 11:37:35 +01:00
stevenhorsman
64a27d962b CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #7083
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-06-19 11:24:03 +01:00
Hyounggyu Choi
128f0282e1 Merge pull request #7122 from BbolroC/set-default-aa-kbc-for-cc-initrd
CC: Set default value for AA_KBC for cc_rootfs_initrd_tarball
2023-06-16 14:13:02 +02:00
Hyounggyu Choi
952f3ed057 CC: Set default value for AA_KBC for cc_rootfs_initrd_tarball
This is to set a default value for `AA_KBC` for the make target `cc_rootfs_initrd_tarball`.

Fixes: #7121

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-06-16 12:01:23 +02:00
Fabiano Fidêncio
940d126bbb Merge pull request #7070 from UnmeshDeodhar/fix-qemu-snp-prefix-and-name
Fix qemu snp prefix and name
2023-06-15 18:18:07 +02:00
Unmesh Deodhar
f4ee2a622f runtime: Update snp qemu command name
Main merge back to CCv0 caused snp qemu build to move from install_qemu to install_qemu_experimental.
Thus, reflecting this change into the qemu snp command.

Fixes: #7059

Signed-Off-By: Unmesh Deodhar <udeodhar@amd.com>
2023-06-12 12:34:42 -05:00
Unmesh Deodhar
d06507c5aa packaging: Fix qemu experimental build for cc
Qemu for SNP is experimental. Thus, when building QEMU for SNP we need to create a builder that builds experimental qemu for CC.

Fixes: #7059

Signed-Off-By: Unmesh Deodhar <udeodhar@amd.com>
2023-06-12 12:34:41 -05:00
Steve Horsman
d742fe37a9 Merge pull request #6970 from yoheiueda/remote-pod-annotation
CCv0: kata-deploy-cc: Ignore pod annotations in kata-remote
2023-06-12 09:39:10 +01:00
Steve Horsman
8330fdeadf Merge pull request #7060 from UnmeshDeodhar/fix-qemu-build-snp
local-build: Fix qemu build for SNP
2023-06-08 20:40:57 +01:00
Unmesh Deodhar
827c98ddab local-build: Fix qemu build for SNP
Qemu entry for SNP was changed in the versions.yaml resulting into the incorrect qemu build for SNP.

Fixes: #7059

Signed-Off-By: Unmesh Deodhar <udeodhar@amd.com>
2023-06-07 11:16:50 -05:00
Steve Horsman
338e18e4fd Merge pull request #7037 from sprt/sprt/cc06-kata-deps
CCv0: Update dependencies to v0.6.0 for release
2023-06-06 18:00:44 +01:00
Aurélien Bombo
600d55692f versions: Update td-shim to v0.6.0
Updates td-shim in preparation for the CoCo v0.6.0 release.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-06-06 15:10:11 +01:00
Aurélien Bombo
944d7ab2c1 versions: Update attestation-agent to v0.6.0
Updates attestation-agent in preparation for the CoCo v0.6.0 release.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-06-06 15:10:11 +01:00
Aurélien Bombo
24528cf61c agent: Update image-rs to v0.6.0
Updates image-rs in preparation for the CoCo v0.6.0 release.

Fixes: #7036

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-06-06 15:10:11 +01:00
Steve Horsman
6288ed7202 Merge pull request #7038 from stevenhorsman/agent-ctl-fix-cherry-pick
kata-sys-util: Fix rust compile errors
2023-06-06 14:19:35 +01:00
stevenhorsman
07000e8622 kata-sys-util: Fix rust compile errors
- Bump version of rand to avoid error
- Fix based on
 https://github.com/kata-containers/kata-containers/pull/7033
 , but not directly cherry-picked due to vendor
 conflicts

Fixes: #7032
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-06-06 11:33:36 +01:00
Steve Horsman
947466ce98 Merge pull request #7023 from Xynnn007/dep-update-060
CCv0: update dependencies for CoCo release 0.6.0
2023-06-02 15:28:12 +01:00
Steve Horsman
0de75d5138 Merge pull request #6847 from Jordan9500/cosign-s390x
agent: Update image-rs to support cosign s390x
2023-06-02 11:54:01 +01:00
Xynnn007
9e9a045b08 versions: update td-shim dependency
In preparation for CoCo 0.6.0 release, updated td-shim to commit
3252047213b2c580c21bdc52f67e8515ca1e374a

Fixes #7022

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
2023-06-02 17:36:33 +08:00
Xynnn007
1fc89c1440 versions: update attestation-agent dependency
In preparation for CoCo 0.6.0 release, updated attestation-agent to
commit aa1d3c510350cd2f2668aca374abba19e2b73b3f

Fixes #7022

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
2023-06-02 17:36:21 +08:00
Jordan Jackson
55d293386a agent: Update image-rs to support cosign s390x
Content of commit
Update Cargo.toml of kata-agent
Change the features to use new naming convention
Run make vendor, to fix the static checks
Update image-rs, step4 of release checklist

Fixes: #6635
Signed-off-by: Jordan Jackson <jordan.jackson@ibm.com>
2023-06-01 10:03:13 +01:00
Fabiano Fidêncio
9272165bc7 Merge pull request #6885 from stevenhorsman/CCv0-merge-18-may
CCv0: Merge main into CCv0 branch
2023-05-30 13:54:04 +02:00
Fabiano Fidêncio
bdb214aa34 runtimne: Add back the IMAGETDXPATH
This was mistakenly removed as part of the rebase.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-30 10:17:43 +02:00
stevenhorsman
8b7b88f341 runtime: Update FIRMWARETDVFPATH
Correct path

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-05-30 10:13:29 +02:00
Yohei Ueda
5cb8221977 kata-deploy-cc: Ignore annotations in kata-remote
Pod annotations (io.katacontainers.*) are not meaningful
for the remote hypervisor. This patch disables pod annotations
in the kata-remote settings of the containerd configuration.

Fixes: #6345
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
2023-05-26 17:04:21 +09:00
stevenhorsman
cd118ae216 versions: Clean up qemu
- Remove not-needed qemu-tdx-experimental

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-05-26 07:58:04 +01:00
stevenhorsman
38e7c2be62 doc: Update ccv0.sh to reflect changes
- Update prefix
- Add new requirement environment variables
- Use new ci scripts

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-05-25 20:09:36 +01:00
stevenhorsman
0032b8bfbf static_build: sev ovmf
- Update sev omvf to use main code base

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-05-25 19:36:25 +01:00
stevenhorsman
66ca2f1bc4 qemu: static-check disable
Disable gocyclo on large complex function in CCv0 branch

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-05-25 17:05:16 +01:00
stevenhorsman
c87c8ffce5 runtime: Fix bad merge
- Re-add removed CC features from sandbox.go

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-05-25 16:30:01 +01:00
GabyCT
4ab93e662c Merge pull request #6963 from dborquez/CCv0
kata-deploy: Remove trailing and leading white spaces in kata-deploy-binaries.sh
2023-05-25 09:22:18 -06:00
stevenhorsman
33143eb342 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: kata-containers#5645
Depends-on: github.com/kata-containers/kata-containers#6885

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-05-25 16:17:59 +01:00
Steve Horsman
ba263d2b04 Merge pull request #6956 from stevenhorsman/kata-deploy-cc-fixes
kata-deploy-cc: Port fixes
2023-05-25 10:26:21 +01:00
David Esparza
fb40ad37a9 kata-deploy: Remove trailing and leading white spaces in kata-deploy-binaries
This PR removes unwanted white spaces in order to fix the format
of the kata-deploy-binaries script.

Fixes: #6962

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-05-24 17:35:16 -06:00
Steve Horsman
499ec324a9 Merge pull request #6933 from fidencio/topic/cc-fix-image-used-with-qemu-tdx
CC | runtime: config: Fix image path for QEMU TDX
2023-05-24 21:15:28 +01:00
Fabiano Fidêncio
370811b017 runtime: Fix TDVF configuration with QEMU TDX
Instead of setting:
```
firmware = "/path/to/OVMF.fd"
firmware_volume = "/path/to/OVMF_VARS.fd"
```

We should either be setting:
```
firmware = "/path/to/OVMF.fd"
```
Or:

```
firmware = "/path/to/OVMF_CODE.fd"
firmware_volume = "/path/to/OVMF_VARS.fd"
```

I'm taking the approach to setting up the latter, as that's what's been
tested as part of our TDX CI.

Fixes: #4926

This patch is the same as #4927, but it ended up reverted somewhere in
the CCv0 -> main process, or in the attempts to fix TDX after that.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-24 19:01:44 +02:00
stevenhorsman
b1fc929191 kata-deploy-cc: Port fixes
Port over fixes from `kata-deploy` into `kata-deploy-cc`
- https://github.com/kata-containers/kata-containers/pull/6829
- https://github.com/kata-containers/kata-containers/pull/6913
- https://github.com/kata-containers/kata-containers/pull/6648

Fixes: #6955
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-05-24 17:14:03 +01:00
Fabiano Fidêncio
efb0ac55c8 runtime: config: tdx: Enable service_offload
This also as mistakenly overwritten by the `main` -> `CCv0` merge.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-24 07:57:49 +02:00
Fabiano Fidêncio
8b4b233358 runtime: config: Fix image path for QEMU TDX
The rebase from `main` to `CCv0` ended up overwriting the image path
that should be used for QEMU, in the CCv0 branch.

Fixes: #6932

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-24 07:57:22 +02:00
Fabiano Fidêncio
585edca06d Merge pull request #6931 from stevenhorsman/ccv0-cherry-pick-build-script
Ccv0 cherry pick build script
2023-05-23 19:59:02 +02:00
Fabiano Fidêncio
248aa0cb8c Merge pull request #6927 from stevenhorsman/ccv0-cherry-pick-fedora-bump
CC | osbuilder: Bump fedora image version
2023-05-23 16:22:27 +02:00
Fabiano Fidêncio
3506de5d21 local-build: Standardise what's set for the local build scripts
We've a discrepancy on what's set along the scripts used to build the
Kata Cotainers artefacts locally.

Some of those were missing a way to easily debug them in case of a
failure happens, but one specific one (build-and-upload-payload.sh)
could actually silently fail.

All of those have been changed as part of this commut.

Fixes: #6908

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit ae24dc73c1)
2023-05-23 09:19:46 +01:00
Fabiano Fidêncio
3661d57bea kata-deploy: Use apt-key.gpg from k8s.io
We're facing some issues to download / use the public key provided by
google for installing kubernetes as part of the kata-deploy image.
```
The following signatures couldn't be verified because the public key is
not available: NO_PUBKEY B53DC80D13EDEF05
Reading package lists... Done
W: GPG error: https://packages.cloud.google.com/apt kubernetes-xenial
   InRelease: The following signatures couldn't be verified because the
   public key is not available: NO_PUBKEY B53DC80D13EDEF05 E: The
   repository 'https://apt.kubernetes.io kubernetes-xenial InRelease' is
   not signed.
N: Updating from such a repository can't be done securely, and is
   therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user
   configuration details.
```

Let's work this around following the suggestion made by @dims, at:
https://github.com/kubernetes/k8s.io/pull/4837#issuecomment-1446426585

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 636539bf0c)
2023-05-23 09:18:45 +01:00
Fabiano Fidêncio
43139934cd Merge pull request #6919 from fidencio/topic/cc-fix-qemu-tdx-location
runtime: Fix QEMU cmdline for TDX
2023-05-22 23:32:11 +02:00
stevenhorsman
e8f8179d62 osbuilder: Bump fedora image version
- Swap out an EoL fedora image for the latest

Fixes: #6923
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
(cherry picked from commit b8ffcd1b9b)
2023-05-22 20:42:22 +01:00
Fabiano Fidêncio
1f9ed94d74 runtime: Fix QEMU cmdline for TDX
This commit should've been part of the series that reverted a bunch of
TDX changes that are not compatible with the TDX stack we're using in
the Jenkins CI machine.

The change made here is in order to match what's been undone here:
c29e5036a6

Fixes: #6884

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-22 11:29:46 +02:00
Steve Horsman
2ba029f3e1 Merge pull request #6902 from fidencio/topic/cc-bring-back-root-hashes-logic
CC | local-build: Bring back root_hash logic
2023-05-19 18:21:39 +01:00
Fabiano Fidêncio
f27f29d8b8 local-build: Bring back root_hash logic
I cannot easily pin-point which commit dropped it, but my gut feeling is
that it's the result of an erroneous conflict resolution when merging
content from main to the CCv0 branch.

Regardless of when / why it happened, as the root_hash logic ended up
being dropped, workflows that depend on that are now failing.

With everything said in mind, let's bring the logic back.

Fixes: #6901

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-19 15:18:50 +02:00
Fabiano Fidêncio
6763c41d7e Merge pull request #6886 from fidencio/topic/cc-stick-to-2022ww44-for-tdx
CC: tdx: Stick to the 2022ww44 TDX stack for the CCv0 branch
2023-05-19 11:55:13 +02:00
Fabiano Fidêncio
ac99d2d63e Merge pull request #6869 from fidencio/topic/cc-use-static-sandbox-resource-mgmt-for-TEEs
CC | runtime: Use static_sandbox_resource_mgmt=true for TEEs
2023-05-19 08:56:15 +02:00
Fabiano Fidêncio
c29e5036a6 Revert "versions: Move QEMU TDX to its own experimental entry"
This reverts commit 800ee5cd88.

As the Jenkins TDX CI is running on a system with a TDX stack called
"2022ww44", we should keep the QEMU / kernel / OVMF versions matching
what's provided in that stack.

The reason we were able to update this on `main` is because the GHA TDX
CI is running on a TDX stack called "2023ww01", but we have decided to
NOT take the bullet, NOT updating the Jenkins CI in order to avoid
unexepected breakages.

This regression was introduced as part of the last CCv0 merge to main,
and would've been caught by the CI, and should've been caught by the
reviewer (myself :-)), but CI was having a hard time to even build the
compoenents and I wrote in the PR and I'm quoting it here: "I rather
deal with possible breakages on this later on, than block this PR to get
in." ... and here we are. :-)

Fixes: #6884

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-18 19:47:10 +02:00
Fabiano Fidêncio
7f13329c6a Revert "versions: Update QEMU TDX version"
This reverts commit 3018c9ad51.

As the Jenkins TDX CI is running on a system with a TDX stack called
"2022ww44", we should keep the QEMU / kernel / OVMF versions matching
what's provided in that stack.

The reason we were able to update this on `main` is because the GHA TDX
CI is running on a TDX stack called "2023ww01", but we have decided to
NOT take the bullet, NOT updating the Jenkins CI in order to avoid
unexepected breakages.

This regression was introduced as part of the last CCv0 merge to main,
and would've been caught by the CI, and should've been caught by the
reviewer (myself :-)), but CI was having a hard time to even build the
compoenents and I wrote in the PR and I'm quoting it here: "I rather
deal with possible breakages on this later on, than block this PR to get
in." ... and here we are. :-)

Fixes: #6884

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-18 19:47:06 +02:00
Fabiano Fidêncio
7bf6ba11c6 Revert "versions: Move Kernel TDX to its own experimental entry"
This reverts commit 20ab2c2420.

As the Jenkins TDX CI is running on a system with a TDX stack called
"2022ww44", we should keep the QEMU / kernel / OVMF versions matching
what's provided in that stack.

The reason we were able to update this on `main` is because the GHA TDX
CI is running on a TDX stack called "2023ww01", but we have decided to
NOT take the bullet, NOT updating the Jenkins CI in order to avoid
unexepected breakages.

This regression was introduced as part of the last CCv0 merge to main,
and would've been caught by the CI, and should've been caught by the
reviewer (myself :-)), but CI was having a hard time to even build the
compoenents and I wrote in the PR and I'm quoting it here: "I rather
deal with possible breakages on this later on, than block this PR to get
in." ... and here we are. :-)

Fixes: #6884

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-18 12:56:37 +02:00
Fabiano Fidêncio
4bdf56a3f8 Revert "versions: Update Kernel TDX version"
This reverts commit f33345c311.

As the Jenkins TDX CI is running on a system with a TDX stack called
"2022ww44", we should keep the QEMU / kernel / OVMF versions matching
what's provided in that stack.

The reason we were able to update this on `main` is because the GHA TDX
CI is running on a TDX stack called "2023ww01", but we have decided to
NOT take the bullet, NOT updating the Jenkins CI in order to avoid
unexepected breakages.

This regression was introduced as part of the last CCv0 merge to main,
and would've been caught by the CI, and should've been caught by the
reviewer (myself :-)), but CI was having a hard time to even build the
compoenents and I wrote in the PR and I'm quoting it here: "I rather
deal with possible breakages on this later on, than block this PR to get
in." ... and here we are. :-)

Fixes: #6884

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-18 12:56:22 +02:00
Fabiano Fidêncio
82e6b65f63 Partially revert "versions: Bump OVMF for TDX"
This partially reverts commit 054174d3e6

As the Jenkins TDX CI is running on a system with a TDX stack called
"2022ww44", we should keep the QEMU / kernel / OVMF versions matching
what's provided in that stack.

The reason we were able to update this on `main` is because the GHA TDX
CI is running on a TDX stack called "2023ww01", but we have decided to
NOT take the bullet, NOT updating the Jenkins CI in order to avoid
unexepected breakages.

This regression was introduced as part of the last CCv0 merge to main,
and would've been caught by the CI, and should've been caught by the
reviewer (myself :-)), but CI was having a hard time to even build the
compoenents and I wrote in the PR and I'm quoting it here: "I rather
deal with possible breakages on this later on, than block this PR to get
in." ... and here we are. :-)

Fixes: #6884

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-18 12:55:50 +02:00
Fabiano Fidêncio
fe6e918ddc Revert "virtcontainers: Drop check for the tdx CPU flag"
This reverts commit 25b3cdd38c.

As the Jenkins TDX CI is running on a system with a TDX stack called
"2022ww44", we should keep the QEMU / kernel / OVMF versions matching
what's provided in that stack.

The reason we were able to update this on `main` is because the GHA TDX
CI is running on a TDX stack called "2023ww01", but we have decided to
NOT take the bullet, NOT updating the Jenkins CI in order to avoid
unexepected breakages.

This regression was introduced as part of the last CCv0 merge to main,
and would've been caught by the CI, and should've been caught by the
reviewer (myself :-)), but CI was having a hard time to even build the
compoenents and I wrote in the PR and I'm quoting it here: "I rather
deal with possible breakages on this later on, than block this PR to get
in." ... and here we are. :-)

Fixes: #6884

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-18 12:55:20 +02:00
Fabiano Fidêncio
2962d8db45 Revert "runtime/qemu: Drop "kvm-type=tdx""
This reverts commit ed145365ec.

As the Jenkins TDX CI is running on a system with a TDX stack called
"2022ww44", we should keep the QEMU / kernel / OVMF versions matching
what's provided in that stack.

The reason we were able to update this on `main` is because the GHA TDX
CI is running on a TDX stack called "2023ww01", but we have decided to
NOT take the bullet, NOT updating the Jenkins CI in order to avoid
unexepected breakages.

This regression was introduced as part of the last CCv0 merge to main,
and would've been caught by the CI, and should've been caught by the
reviewer (myself :-)), but CI was having a hard time to even build the
compoenents and I wrote in the PR and I'm quoting it here: "I rather
deal with possible breakages on this later on, than block this PR to get
in." ... and here we are. :-)

Fixes: #6884

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-18 12:55:03 +02:00
Fabiano Fidêncio
eee621f431 Revert "govmm: Set "sept-ve-disable=on""
This reverts commit 3c5ffb0c85.

As the Jenkins TDX CI is running on a system with a TDX stack called
"2022ww44", we should keep the QEMU / kernel / OVMF versions matching
what's provided in that stack.

The reason we were able to update this on `main` is because the GHA TDX
CI is running on a TDX stack called "2023ww01", but we have decided to
NOT take the bullet, NOT updating the Jenkins CI in order to avoid
unexepected breakages.

This regression was introduced as part of the last CCv0 merge to main,
and would've been caught by the CI, and should've been caught by the
reviewer (myself :-)), but CI was having a hard time to even build the
compoenents and I wrote in the PR and I'm quoting it here: "I rather
deal with possible breakages on this later on, than block this PR to get
in." ... and here we are. :-)

Fixes: #6884

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-18 12:54:46 +02:00
Fabiano Fidêncio
e005dfa718 Revert "govmm: Directly pass the firmware using -bios with TDX"
This reverts commit 3e15800199.

As the Jenkins TDX CI is running on a system with a TDX stack called
"2022ww44", we should keep the QEMU / kernel / OVMF versions matching
what's provided in that stack.

The reason we were able to update this on `main` is because the GHA TDX
CI is running on a TDX stack called "2023ww01", but we have decided to
NOT take the bullet, NOT updating the Jenkins CI in order to avoid
unexepected breakages.

This regression was introduced as part of the last CCv0 merge to main,
and would've been caught by the CI, and should've been caught by the
reviewer (myself :-)), but CI was having a hard time to even build the
compoenents and I wrote in the PR and I'm quoting it here: "I rather
deal with possible breakages on this later on, than block this PR to get
in." ... and here we are. :-)

Fixes: #6884

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-18 12:50:11 +02:00
Steve Horsman
a4ecf5a3e4 Merge pull request #6862 from stevenhorsman/CCv0-cherry-pick-image-tag-fix-2
deploy: fix shell script error
2023-05-18 09:12:03 +01:00
Fabiano Fidêncio
1222800f7c runtime: Use static_sandbox_resource_mgmt=true for TEEs
When this option is enabled the runtime will attempt to determine the
appropriate sandbox size (memory, CPU) before booting the virtual
machine.

As TEEs do not support memory and CPU hotplug, this approach must be
used.

Fixes: #6818

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-17 20:50:45 +02:00
Fabiano Fidêncio
f07b27d445 Merge pull request #6559 from stevenhorsman/CCv0-merge-30th-mar
CCv0: Merge main into CCv0 branch
2023-05-17 17:00:39 +02:00
stevenhorsman
401b7947e3 deploy: fix shell script error
- Remove local introduced by bad copy-paste

Fixes: #6814
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
(cherry picked from commit 1a3f8fc1a2)
2023-05-17 11:10:53 +01:00
Fabiano Fidêncio
c618578908 Merge pull request #6832 from stevenhorsman/CCv0-cherry-pick-image-tag-fix
deploy: Fix arch in image tag
2023-05-16 16:11:07 +02:00
stevenhorsman
e6f7ee218c deploy: Fix arch in image tag
`uname -m` produces `x86_64`, but container image convention
is to use `amd64`, so update this in the tag

Fixes: #6820
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
(cherry picked from commit 2856d3f23d)
2023-05-15 10:21:44 +01:00
Fabiano Fidêncio
39d54713f2 Merge pull request #6815 from stevenhorsman/multi-arch-runtime-payload
actions:  multi-arch runtime-payload
2023-05-12 14:17:41 +02:00
stevenhorsman
057db564fd actions: multi-arch runtime-payload
- Create multi-arch manifests for the ci and release runtime-payload
that are tagged with the commit, for use in the operator

Fixes: #6814
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-05-12 10:54:49 +01:00
Fabiano Fidêncio
3b9ae6716c Merge pull request #6774 from bpradipt/fix-6730
CC | runtime: Add configurable context timeout for StopVM
2023-05-04 15:06:08 +02:00
Pradipta Banerjee
1f0d709be6 CC: Add configurable context timeout for StopVM in remote hyp
Add configurable context timeout for StopVM in remote hypervisor
similar to StartVM

Fixes: #6730

Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
2023-05-04 10:42:30 +00:00
stevenhorsman
db26632123 packaging: update builder_registry
Use `CC_BUILDER_REGISTRY`

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-05-03 15:08:41 +01:00
stevenhorsman
9bdd9af898 agent: Update image_rpc
- Update image_rpc to reflect the changes in code gen

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-04-27 15:12:13 +01:00
stevenhorsman
8b8e71fa65 protocols: Update type
- Fix merge issue

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-04-27 11:25:41 +01:00
stevenhorsman
dbe1fd9436 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #6558
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-04-27 09:42:44 +01:00
Steve Horsman
796c8d5015 Merge pull request #6710 from Jordan9500/cherry-pick-cgroupv2-fix
rustjail: Use CPUWeight with systemd and CgroupsV2
2023-04-25 15:28:29 +01:00
Greg Kurz
bc101b7f56 rustjail: Use CPUWeight with systemd and CgroupsV2
The CPU shares property belongs to CgroupsV1. CgroupsV2 uses CPU weight
instead. The correct value is computed in the latter case but it is passed
to systemd using the legacy property. Systemd rejects the request and the
agent exists with the following error :

        Value specified in CPUShares is out of range: unknown

Replace the "shares" wording with "weight" in the CgroupsV2 code to
avoid confusions. Use the "CPUWeight" property since this is what
systemd expects in this case.

Fixes #6636

References:

https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html#CPUWeight=weight
https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html#systemd%20252
https://github.com/containers/crun/blob/main/crun.1.md#cpu-controller

Signed-off-by: Greg Kurz <groug@kaod.org>
(cherry picked from commit c1fbaae8d6)
2023-04-24 15:43:27 +01:00
Xynnn007
03d8391404 agent: add container launch control parameters from kernel commandline
`image_policy_file` is the URI of the image security policy file which
is used to control the allowlist and denylist of the containers to be
run inside the guest.

`image_registry_auth_file` is the URI of the auth file which is needed
to access an private registry.

`simple_signing_sigstore_config` is a configuration file that shows
where the signature files are when simple signing is enabled.

All the config items support both `kbs://..` scheme, `file://..` scheme,
or a direct absolute path `/...`, which means either the file is to be
fetched from a remote KBS that `aa_kbc_params` points to, or from the
local filesystem of the guest.

Fixes: #6640

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
2023-04-24 17:25:19 +08:00
Fabiano Fidêncio
6abfb9dead Merge pull request #6661 from fidencio/topic/update-deps-for-v0.5.0-release
Update deps for the v0.5.0 release of Confidential Containers
2023-04-14 17:12:14 +02:00
Fabiano Fidêncio
a3b2358c77 agent: Update image-rs to v0.5.1
Let's update image-rs to the latest released version of the project.

Fixes: #6650

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-04-14 15:20:35 +02:00
Fabiano Fidêncio
d05ec9cb12 versions: Update attestation-agent to v0.5.0
Let's update the attestation-agent to the latest released version of the
project.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-04-14 11:27:43 +02:00
Fabiano Fidêncio
2289e53cbe versions: Update td-shim to v0.5.0
Let's update td-shim to the lataest released version of the project.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-04-14 11:26:10 +02:00
Fabiano Fidêncio
e58ccb632f Merge pull request #6651 from wainersm/coco_release_0.5
CCv0: update dependencies for CoCo release 0.5.0
2023-04-13 22:55:05 +02:00
Wainer dos Santos Moschetta
2556052848 versions: update td-shim dependency
In preparation for CoCo 0.5 release, updated td-shim to
commit 10568bab569bc40034cc973f26fbb0a768dcc3e3

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2023-04-12 18:22:53 -03:00
Wainer dos Santos Moschetta
6cd74b4530 versions: update attestation-agent dependency
In preparation for CoCo 0.5 release, updated attestation-agent to
commit c939d211fe5ac497715008e36161aff20cabb6e6

Fixes #6650
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2023-04-12 18:22:53 -03:00
Wainer dos Santos Moschetta
2bd2b12a0a agent: update image-rs dependency
In preparation for CoCo 0.5 release, updated image-rs to
commit b28eaae.

Fixes #6650
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2023-04-12 18:22:33 -03:00
Steve Horsman
dfa7ed06eb Merge pull request #6617 from yoheiueda/remote-static-resmgmt
CCv0: runtime: Use static_sandbox_resource_mgmt by default in remote hypervisor
2023-04-12 07:49:11 +01:00
Fabiano Fidêncio
ca51331a5f Merge pull request #6602 from Alex-Carter01/snp-qemu-x86-workflow
CC | gha: add SNP components to amdx86_64 payload workflows
2023-04-06 20:17:29 +02:00
Jeremi Piotrowski
3db0c6bcac Merge pull request #6625 from BbolroC/revive-runtime-config-s390x
CC: Revive removed runtime config for IBM SE
2023-04-06 15:59:22 +02:00
Fabiano Fidêncio
b936ddc139 Merge pull request #6569 from jepio/jepio/6404-test
CC | agent: add support for ttrpc + dependency on test PR
2023-04-06 14:15:44 +02:00
Hyounggyu Choi
b73f26440e CC: Revive removed runtime config for IBM SE
This is just to get the removed runtime config for IBM SE back again.

Fixes: #6624

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-04-06 20:18:28 +09:00
Yohei Ueda
938447803b runtime: Use static resource in remote hypervisor
This patch updates the template configuration file for
the remote hypervisor to set static_sandbox_resource_mgmt
to be true.  The remote hypervisor uses the peer pod config
to determine the sandbox size, so requires this to be set to
true by default.

Fixes: #6616
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
2023-04-06 16:53:05 +09:00
Hyounggyu Choi
bb2a35ac88 Merge pull request #6601 from BbolroC/cc-rootfs-initrd-s390x
CC: Add cc-rootfs-initrd to payload image for s390x
2023-04-06 16:04:06 +09:00
Alex Carter
ee20810a78 gha: Add SNP components to the cc-payload workflows
Adding SNP components needed to the x86 payload push and release payloads.

QEMU is needed in both the after-push payload and release payload, while OVMF is only
missing from the release workflow.

Fixes: #6600

Signed-Off-By: Alex Carter <AlexCarter@ibm.com>
2023-04-05 17:54:28 -04:00
Hyounggyu Choi
47322c32bf CC: Add cc-rootfs-initrd to payload image for s390x
This is to add an artifact named `cc-rootfs-initrd` to a payload image
because it is identified that the artifact is required to run a cc-operator
e2e test.

Fixes: #6544

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-04-05 00:59:18 +09:00
Xynnn007
38037dbe1f osbuilder: Support attestation-agent[cc_kbc] TDX dependencies
attestation-agent depends on tdx-attest-rs when cc_kbc is enabled, which
depends on libtdx-attest.so. Include the dev package in build container,
and the runtime package in the built rootfs.

The build of tdx-attest-sys (which is a dep of tdx-attest-rs) uses
bindgen, which requires libclang so install that in the build container
as well.

We specify the tdx stack DCAP v1.15

Fixes: #6519

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
2023-04-04 10:26:57 +02:00
Jeremi Piotrowski
5c27f27af0 Merge pull request #6135 from Alex-Carter01/snp-config
config: Add SEV SNP config
2023-03-31 20:10:08 +02:00
Jeremi Piotrowski
c8f615b204 Merge pull request #6062 from Alex-Carter01/snp-qemu
kata-deploy: SNP version of Qemu
2023-03-31 20:09:57 +02:00
Alex Carter
28fd92c478 kata-deploy: SNP version of Qemu
An SNP runtimeclass needs a build of Qemu from https://github.com/AMDESE/qemu/tree/snp-v3.

So a new target needs to be added to add it to a kata-deploy bundle.

Building requires a qemu no_patches file

Fixes: #6061
Signed-Off-By: Alex Carter <alex.carter@ibm.com>
2023-03-31 14:58:38 +00:00
Alex Carter
2b5c84b9ec config: Add SEV SNP config
Adding SNP template for config, and values to the runtime makefile to generate the output toml.

Based on [Joana's SNP Guide](https://github.com/kata-containers/kata-containers/blob/main/docs/how-to/how-to-run-kata-containers-with-SNP-VMs.md).

Fixes: #6112
Signed-Off-By: Alex Carter <alex.carter@ibm.com>
2023-03-31 13:51:09 +00:00
Jeremi Piotrowski
1da81308c7 dep: Add depends-on entry for tests PR
Depends-on:github.com/kata-containers/tests#5579

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-31 13:36:31 +02:00
Steve Horsman
2b8e241179 Merge pull request #5850 from Alex-Carter01/snp-ovmf
CC: kata-deploy add x86 ovmf build for SNP
2023-03-31 11:41:20 +01:00
Christophe de Dinechin
10920e2398 Merge pull request #6545 from BbolroC/se-image-in-payload
CC: Introduce secure image in payload image
2023-03-31 11:16:39 +02:00
Steve Horsman
a4781e2cc9 Merge pull request #6342 from bpradipt/fsnotify
runtime: propagate configmap, secrets, downward-api etc changes for remote-hyp
2023-03-31 09:03:44 +01:00
Fabiano Fidêncio
c7c29ee722 Merge pull request #6552 from BbolroC/kata-qemu-se-deploy
CC|kata-deploy: Add kata-qemu-se runtimeclass
2023-03-31 09:58:47 +02:00
Pradipta Banerjee
3081cd5f8e runtime: propagate configmap/secrets etc changes for remote-hyp
For remote hypervisor, the configmap, secrets, downward-api or project-volumes are
copied from host to guest. This patch watches for changes to the host files
and copies the changes to the guest.

Note that configmap updates takes significantly longer than updates via downward-api.
This is similar across runc and Kata runtimes.

Fixes: #6341

Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Signed-off-by: Julien Ropé <jrope@redhat.com>
2023-03-30 03:14:52 +00:00
Xynnn007
93f23dcd2a osbuilder: add ttrpc as default for attestation-agent
`ttrpc=true` parameter tells the Makefile of attestation-agent
to build the attestation-agent with ttrpc support

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
2023-03-30 09:12:07 +08:00
Xynnn007
30981f81b5 dep: use newest version of attestation-agent
Newest AA supports ttrpc, and it will be built when
generating guest image.

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
2023-03-30 09:11:50 +08:00
Xynnn007
fcea177d96 agent: add support for ttrpc
This commit brings ttrpc of image-rs. It will use the
lightweight underlying ttrpc to interact between kata-agent
and attestation-agent.

Also, this PR brings a patch for `oci-distribution`,
because two dependencies of `image-rs` depends on different
versions of `oci-distribution`, which will cause that
`image-rs` can not be built. We need a specified version of
`oci-distribution` to unify.

Fixes #6219

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
2023-03-30 09:11:50 +08:00
Hyounggyu Choi
f73a5f4b1f CC|kata-deploy: Add kata-qemu-se runtimeclass
This is to add a new element `qemu-se` to the shims for a new runtime
class `kata-qemu-se`.

Fixes: #6549

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-03-29 22:32:29 +09:00
Alex Carter
594aa178b3 CC: kata-deploy add x86 ovmf build for SNP
SNP needs two builds of ovmf: the AmdSev build and the normal x86_64 build.

Adds target for vanilla ovmf build for snp

Adding another make target / kata-deploy function, and fixing the ovmf builder so these builds dont overlap.

Fixes: #5849

Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
2023-03-29 13:22:43 +00:00
Hyounggyu Choi
48b0dd66ec CC: Introduce secure image in payload image
This is a preliminary work to establish an e2e test for a new runtime
class kata-qemu-se (IBM secure execution).

Fixes: #6544

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-03-29 18:51:28 +09:00
Fabiano Fidêncio
4c810c7933 Merge pull request #6457 from arronwy/multiple_same_images
image_rpc: Add container_count for containers without cid
2023-03-28 10:16:48 +02:00
Fabiano Fidêncio
121892ec61 Merge pull request #6532 from fidencio/topic/cc-tdx-use-a-specific-dcap-version
rootfs: Ensure we use a compatible version of dcap with rats-tls
2023-03-27 19:57:59 +02:00
Fabiano Fidêncio
bda8160224 rootfs: Ensure we use a compatible version of dcap with rats-tls
DCAP has upgraded to 1.16, which is not compatible with the host OS used
as part of our CI (2022ww44).  Let's ensure DCAP 1.15 is used instead.

Fixes: #6529

Signed-off-by: Wang, Arron <arron.wang@intel.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-27 17:07:17 +02:00
Steve Horsman
4b027ba664 Merge pull request #6506 from Megan-Wright/CCv0
CCv0: Merge main into CCv0 branch
2023-03-27 14:42:03 +01:00
Steve Horsman
b1afabdef0 Merge pull request #6249 from stevenhorsman/attestation-agent-bump
osbuilder: Make attestation-agent build
2023-03-23 12:26:51 +00:00
Steve Horsman
1474053196 Merge pull request #6350 from stevenhorsman/remote-config
config: Add initial remote hypervisor config
2023-03-21 13:44:08 +00:00
Steve Horsman
7ea4357723 Merge pull request #6352 from stevenhorsman/kata-deploy
kata-deploy: Add cri-o support
2023-03-21 13:43:25 +00:00
stevenhorsman
ddc2f1c331 osbuilder: Make attestation-agent build
Update the build to use the attestation-agent makefile to build it, so
we can pick up the enhancements there

Fixes: #6253
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-03-20 17:58:58 +00:00
stevenhorsman
6a7c39824a kata-deploy: Add cri-o support
Add support for cri-o into kata-deploy and merge in CAA features

Fixes: #6351
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by:: Pradipta Banerjee <pradipta.banerjee@gmail.com>
2023-03-20 17:58:43 +00:00
stevenhorsman
7c9a791d67 config: Add initial remote hypervisor config
Remote hypervisor template config

Fixes: #6349
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-03-20 17:58:27 +00:00
Fabiano Fidêncio
d12e53b5a1 Merge pull request #6503 from jepio/jepio/static-checks-workflow-improvements-ccv0
CC | static checks workflow improvements
2023-03-20 18:50:15 +01:00
Megan Wright
f31c907f46 Fix bad merge 2023-03-20 13:51:10 +00:00
Megan Wright
42978f3e83 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #6504
Signed-off-by: Megan Wright <megan.wright@ibm.com>
2023-03-20 13:23:49 +00:00
Jeremi Piotrowski
269c5f5877 workflows: static-checks: Free disk space before running checks
We've been seeing the 'sudo make test' job occasionally run out of space in
/tmp, which is part of the root filesystem. Removing dotnet and
`AGENT_TOOLSDIRECTORY` frees around 10GB of space and in my tests the job still
has 13GB of space left after running.

Fixes: #6401
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-20 12:19:35 +01:00
Jeremi Piotrowski
70ed46169c workflows: static-checks: Set GOPATH only once
{{ runner.workspace }}/kata-containers and {{ github.workspace }} resolve to
the same value, but they're being used multiple times in the workflow. Remove
multiple definitions and define the GOPATH var at job level once.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-20 12:19:35 +01:00
Jeremi Piotrowski
e201544145 tools/osbuilder/tests: Remove TRAVIS variable
The last remaining user of the TRAVIS variable in this repo is
tools/osbuilder/tests and it is only used to skip spinning up VMs. Travis
didn't support virtualization and the same is true for github actions hosted
runners. Replace the variable with KVM_MISSING and determine availability of
/dev/kvm at runtime.

TRAVIS is also used by '.ci/setup.sh' in kata-containers/tests to reduce the
set of dependencies that gets installed, but this is also in the process of
being removed.

Fixes: #3544
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-20 12:19:35 +01:00
Jeremi Piotrowski
cdb2efe003 workflows: static-checks: Remove TRAVIS_XXX variables
These variables are unused since we don't use travis CI. This also allows to
remove two steps:

- 'Setup GOPATH' only printed variables
- 'Setup travis reference' modified some shell local variables that don't have
  any influence on the rest of the steps

The TRAVIS var is still used by tools/osbuilder/tests to determine if
virtualization is available.

Fixes: #3544
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-20 12:19:35 +01:00
Steve Horsman
4bd6103253 Merge pull request #6491 from jepio/jepio/kata-deploy-push-more-space
CC | kata-deploy-push more space
2023-03-20 10:34:51 +00:00
Fabiano Fidêncio
20452402b5 Merge pull request #6498 from fidencio/topic/cc-switch-back-the-permission-of-cache-components-to-0755
cache_components: Switch back the permission to 755
2023-03-20 09:09:22 +01:00
Fabiano Fidêncio
77783153b6 cache_components: Switch back the permission to 755
2b41dbe broke all the cached jobs as it changed the permission of the
cache_components.sh file from 755 to 644.

Fixes: #6497

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-17 23:50:47 +01:00
Fabiano Fidêncio
45759e82b1 Merge pull request #6474 from UnmeshDeodhar/fix-sev-cached-kernel-bug
packaging: fix sev kernel caching
2023-03-17 23:01:40 +01:00
Jeremi Piotrowski
26f80ff468 workflows: kata-deploy-push: Remove unneeded tools from runner
`make kata-tarball` needs a lot of disk space and github action runners don't
have that much of it. Remove unneeded tools from the runner, which frees
another ~10GB of space.

Fixes: #6490
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-17 17:46:38 +01:00
Jeremi Piotrowski
a394798b75 kata-deploy: Remove temporary directory after creating tarball
The various tarballs are unpacked into a temporary directory, and then that
directory is compressed into kata-static.tar.xz. After we have the tarball,
there is no reason to keep the temporary directory. Dispose of it as the last
step.

Fixes: #6490
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-17 17:43:42 +01:00
Unmesh Deodhar
2b41dbe00f packaging: fix sev kernel caching
Check if tarball exists. If yes, skip the tarball create command.

Fixes: #6473

signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
2023-03-17 10:17:54 -05:00
Fabiano Fidêncio
9c16268b0a Merge pull request #6486 from fidencio/topic/cache-artefacts-fix-kernel-version
CC | kernel: Ensure `kata_config_version` is taken into account
2023-03-17 10:17:09 +01:00
Fabiano Fidêncio
b3a2caa527 kernel: Ensure kata_config_version is taken into account
We need to ensure `kata_config_version` is taken into account when:
* consuming a cached kernel, otherwise we may introduce changes to a
  kernel that will never be validated as part of the PR
* caching the kernel, otherwise we won't update the artefacts if just a
  config is changed

Fixes: #6485

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-16 17:44:06 +01:00
Steve Horsman
c3059d0bd9 Merge pull request #6207 from BbolroC/s390x-prepare-se-image
CC: Prepare secure execution image tarball for s390x
2023-03-15 14:33:59 +00:00
Steve Horsman
5da86f3e25 Merge pull request #6440 from GeorginaKin/CCv0
CCv0: Merge main into CCv0 branch
2023-03-15 10:40:55 +00:00
Fabiano Fidêncio
0bcbbfe44f Merge pull request #5665 from jimcadden/online_sev_kbc
CC | runtime: Add online-kbs to kernel params
2023-03-14 21:19:37 +01:00
Wang, Arron
dd1c22211c image_rpc: Add container_count for containers without cid
If cid is empty, we will use image name as default cid, to
support multiple containers with same image, we need append
unique id to the image name.

Fixes: #6456

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2023-03-14 16:01:27 +08:00
Fabiano Fidêncio
c9d6b46ea0 Merge pull request #6447 from fidencio/topic/kata-deploy-cc-switch-to-using-ubuntu
kata-deploy-cc: Switch to using an ubuntu image
2023-03-13 17:39:40 +01:00
Georgina Kinge
77e5351102 Bumping the kata config version
Bumping the version to 101cc to fix the static checks

Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2023-03-13 16:21:21 +00:00
Georgina Kinge
818569be79 Reverting container image names
Reverting the container image names to pick up the lib.sh methods.

Signed-off-by: Georgina Kinge georgina.kinge@ibm.com
Co-authored-by: Steve Horsman <steven@uk.ibm.com>
2023-03-13 16:21:21 +00:00
Fabiano Fidêncio
1eeed31e23 kata-deploy-cc: Switch to using Ubuntu image
This will help us in several ways:
* The first one is not using an image that's close to be EOLed, and
  which doesn't officially provide multi-arch images.
* The second is getting closer to what's been already done on main.
* The third is simplifying the logic to build the payload image.

Fixes: #6446

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-10 20:38:30 +01:00
Fabiano Fidêncio
a854355bea kata-deploy: Use more descriptive names for IMG_{TAG,NAME}
On main we're using BASE_IMAGE_TAG and BASE_IMAGE_NAME to avoid
any kind of confusion.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-10 15:54:29 +01:00
Georgina Kinge
eba66de646 Revert "runtime: clh: Re-generate the client code"
Reverting as cloud-hypervisor shouldn't be upgraded in CC.

Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2023-03-10 11:28:04 +00:00
Wainer Moschetta
a6bbc8af4e Merge pull request #6416 from UnmeshDeodhar/remove-duplicate-sev-kernel-build
packaging: fix sev kernel build
2023-03-09 15:51:33 -03:00
Georgina Kinge
b5526b4ab6 Reverting cloud-hypervisor version
Reverting the cloud-hypervisor bump back to v29

Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2023-03-09 16:25:39 +00:00
Georgina Kinge
ec6ce46e7e CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #6428
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2023-03-09 16:14:53 +00:00
Hyounggyu Choi
f831098852 CC: Prepare secure execution image tarball for s390x
This is to prepare a secure image tarball to run a confidential
container for IBM Z SE(TEE).

Fixes: #6206

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-03-09 13:55:48 +09:00
Unmesh Deodhar
9afa38f11b packaging: add kernel modules dir
Create a directory where kernel modules will be extracted.

Fixes: #6418

Signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
2023-03-07 10:15:43 -06:00
Unmesh Deodhar
7267237660 packaging: remove duplicate sev kernel build
Remove duplicate SEV kernel builds.

Fixes: #6415

Signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
2023-03-07 10:15:41 -06:00
Fabiano Fidêncio
129e43d1ea Merge pull request #6382 from jepio/jepio/deps-v0.4.0-tag
CCv0: set dependencies to v0.4.0 tag
2023-03-01 21:57:14 +01:00
Fabiano Fidêncio
75a698be6e Merge pull request #6397 from jepio/jepio/cleanup-runtime-rs
kata-deploy-cc: Cleanup runtime-rs
2023-03-01 20:51:09 +01:00
Fabiano Fidêncio
e6cd11221d Merge pull request #6389 from UnmeshDeodhar/fix-initrd-build
packaging: fix initrd build
2023-03-01 20:12:45 +01:00
Unmesh Deodhar
a1b772c4d3 packaging: fix initrd build
Fixing initrd build by providing correct suffix.

Fixes: #6388

Signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
2023-03-01 17:31:11 +01:00
Jeremi Piotrowski
990cfa7559 kata-deploy-cc: Cleanup runtime-rs
/opt/confidential-containers/runtime-rs needs to be cleaned up, otherwise
containerd post-uninstall script fails due to weird logic in `rmdir
--ignore-fail-on-non-empty`.

Fixes: #6396
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-01 16:58:59 +01:00
Fabiano Fidêncio
8ed7b676c9 Merge pull request #6392 from fidencio/topic/CC-do-not-install-docker-as-part-of-the-actions
CC | workflows: Do not install docker
2023-03-01 16:57:10 +01:00
Fabiano Fidêncio
924f9fdbc7 workflows: Do not install docker
The latest ubuntu runners already have docker installed and trying to
install it manually will cause the following issue:
```
Run curl -fsSL https://test.docker.com/ -o test-docker.sh
Warning: the "docker" command appears to already exist on this system.

If you already have Docker installed, this script can cause trouble, which is
why we're displaying this warning and provide the opportunity to cancel the
installation.

If you installed the current Docker package using this script and are using it
again to update Docker, you can safely ignore this message.

You may press Ctrl+C now to abort this script.
+ sleep 20
+ sudo -E sh -c apt-get update -qq >/dev/null
E: The repository 'https://packages.microsoft.com/ubuntu/22.04/prod jammy Release' is no longer signed.
```

Fixes: #6390

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-02-28 23:52:32 +01:00
Jim Cadden
4510aeaa91 runtime: Adds annotations for SEV/kbs controls at the pod level
Note: only for online-kbs configuration

Fixes #5782

Signed-off-by: Jim Cadden <jcadden@ibm.com>
2023-02-28 10:51:46 -05:00
Jim Cadden
4eb88d6a74 runtime: Add online-kbs to kernel params
In online-kbs attestation the guest is given the location of the
keybroker server to connect after launch. This patch appends the
IP:Port of the online-kbs to the kernel params of the guest.

Patch also simplifies the kbs config into "mode" = offline/online,
and updates SEV config variable names and default values

Fixes: #5661 #5715

Signed-off-by: Jim Cadden <jcadden@ibm.com>
2023-02-28 10:51:46 -05:00
Jeremi Piotrowski
dd274ab51a versions: update dependencies to v0.4.0 tags
This covers td-shim, attestation-agent and image-rs.

Fixes: #6366
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-02-28 09:28:07 +01:00
Fabiano Fidêncio
f46da42847 Merge pull request #6360 from jepio/jepio/update-deps-for-0.4.0
CCv0: update deps for 0.4.0
2023-02-28 08:24:38 +01:00
Fabiano Fidêncio
9e6ce36ad6 Merge pull request #6363 from UnmeshDeodhar/fix-sev-kernel-caching
packaging: Fix sev cache kernel issue
2023-02-27 12:57:26 +01:00
Jeremi Piotrowski
bcc4adb955 versions: Update td-shim and aa to 0.4.0 rc
This is the latest td-shim commit and the latest known working
attestation-agent commit.

Fixes: #6366
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-02-24 07:19:00 +00:00
Jeremi Piotrowski
bd61f61952 agent: Update image-rs to v0.4.0 rc
Features were renamed, so switch both arches to the katacc* feature.
Testing showed that "signature-simple" feature in image-rs is needed on
x86_64, so add that too. This image-rs commit does not include the
latest ocicrypt-rs and attestation-agent code itself.

Fixes: #6366
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-02-24 07:17:53 +00:00
Unmesh Deodhar
46a6fe75dc packaging: Fix sev cache kernel issue
Fix cache kernel issue introduced by #6177

Fixes: #6362

Signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
2023-02-23 23:35:25 -06:00
Fabiano Fidêncio
2d525bbf1b Merge pull request #6177 from UnmeshDeodhar/CC-cache-sev-artifacts
Adding caching capability for SEV kernel and initrd
2023-02-23 17:49:41 +01:00
Fabiano Fidêncio
98de5921fd Merge pull request #6338 from Alex-Carter01/sev-ovmf-rename
CC: Rename SEV firmware to AMDSEV.fd
2023-02-22 20:30:26 +01:00
Unmesh Deodhar
2313b7fa8c packaging: Adding caching capability for SEV kernel
Cache and reuse kernel tarball whenever possible.

Fixes: #6176

Signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-02-22 18:14:01 +01:00
Alex Carter
eebd24bec4 CC: Rename SEV firmware to AMDSEV.fd
Renaming the output binary from the AmdSevPkg from OVMF.fd to AMDSEV.fd so it does not conflict with the base x86_64 build.

Changing install name in ovmf static builder and the location in the sev config file.

Fixes: #6337
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
2023-02-22 16:05:28 +00:00
Unmesh Deodhar
321b061a53 packaging: Adding caching capability for SEV kernel and initrd
Cache and reuse initrd binary whenever possible.

Fixes: #6176

Signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
2023-02-22 13:47:52 +01:00
Fabiano Fidêncio
3e3a1bab70 Merge pull request #6242 from Megan-Wright/CCv0
CCv0: Merge main into CCv0 branch
2023-02-22 09:15:31 +01:00
Megan Wright
241a5a490f Fix bad merge 2023-02-21 14:23:17 +00:00
stevenhorsman
25aa03aefe static-checks: Update base branch for CCv0
Update the base branch we are compariing against for CCv0

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-02-21 13:02:12 +00:00
Megan Wright
e85a6f77c6 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #6241
Signed-off-by: Megan Wright megan.wright@ibm.com
2023-02-21 11:53:30 +00:00
Fabiano Fidêncio
ad889489a0 Merge pull request #6277 from openanolis/chao/add_is_tdx_enable
CC | Dragonball TDX: add is_tdx_enabled to identify tdx VM type
2023-02-16 06:37:10 +01:00
Chao Wu
337f19f0b2 Dragonball: add is_tdx_enabled to identify tdx VM type
In order to disable or enable some features when running tdx vms, we
need to add is_tdx_enabled() function to identify whether the VM
confidiential type is TDX.

fixes: #6276

Signed-off-by: fengshifang <fengshifang@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2023-02-14 22:39:45 +08:00
Hyounggyu Choi
4e99777a6e Merge pull request #6279 from BbolroC/fix-gha-permission-s390x
CC|action: Place permission adjustment for s390x kata-payload
2023-02-14 13:23:41 +01:00
Hyounggyu Choi
fa00ef0206 CC|action: Place permission adjustment for s390x kata-payload
It is just to place a missing stage for permission adjustment in the
cc-payload-after-push-s390x workflow.

Fixes #6278

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-02-14 10:26:06 +01:00
Fabiano Fidêncio
b74e84e123 Merge pull request #6243 from openanolis/chao/tdx_1_vm_type
CC |  Dragonball: add confidential_vm_type for TDX
2023-02-14 08:42:31 +01:00
Fabiano Fidêncio
89e9af2ef7 Merge pull request #6274 from BbolroC/exclude-s390x-ppc64le-from-runtime-rs
CCv0: shim-v2: Only build runtime-rs for the supported arches
2023-02-14 08:11:20 +01:00
Hyounggyu Choi
e546e9532e CCv0: runtime-rs: Include target install in conditional branch
A Makefile target `install` should be included in the conditional branch
as default and test.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
(cherry picked from commit 4139d68d51)
2023-02-13 23:52:17 +01:00
Fabiano Fidêncio
4aac40b92a CCv0: runtime-rs: Improve s390x error message
Nothing much to add, let's just make the message more clear.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit c071355359)
2023-02-13 23:52:12 +01:00
Fabiano Fidêncio
4eb4564a5e CCv0: runtime-rs: Don't try to build on Power
As done for s390x, let's just skip the runtime-rs build for Power.

Fixes: #6142

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 4e2db96ef7)
2023-02-13 23:52:02 +01:00
Hyounggyu Choi
a7d8c7788d Merge pull request #6245 from BbolroC/fix-protoc-s390x
CCv0: install protobuf-compiler explicitly in shim-v2 Dockerfile
2023-02-13 16:02:40 +01:00
Archana Shinde
56e5dc7cce Merge pull request #6233 from jepio/jepio/remove-duplicate-env
osbuilder: remove duplicate KATA_BUILD_CC entry
2023-02-09 11:46:28 -08:00
Fabiano Fidêncio
01e56a7c6d Merge pull request #5796 from niteeshkd/CCv0
CCv0: Enable 'policy' for SNP container
2023-02-09 20:24:25 +01:00
Chao Wu
39a6990cb7 TDX: add confidential_vm_type for TDX
add confidential_vm_type to let Dragonball create a TDX VM.

fixes: #6246

Signed-off-by: fengshifang <fengshifang@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2023-02-09 19:24:41 +08:00
Hyounggyu Choi
10603e3def CCv0: install protobuf-compiler explicitly in shim-v2 Dockerfile
This is to install a missing binary protoc in shim-v2 Dockerfile.

Fixes: #6244

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-02-09 11:45:32 +01:00
Jeremi Piotrowski
f5febe4fe1 osbuilder: remove duplicate KATA_BUILD_CC entry
KATA_BUILD_CC is already getting passed 10 lines above.

Fixes: #6237
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-02-07 18:41:01 +01:00
Georgina Kinge
b95440712d Merge pull request #6182 from GeorginaKin/CCv0
CCv0: Merge main into CCv0 branch
2023-02-07 15:53:41 +00:00
Fabiano Fidêncio
55dcaa22e1 Merge pull request #6158 from fidencio/topic/CCv0-enable-runtime-rs
CC | Take runtime-rs into consideration when building and caching the shim-v2 tarball
2023-02-07 15:34:02 +01:00
Fabiano Fidêncio
84d6b3dea4 Merge pull request #6217 from stevenhorsman/cherry-pick-docker-fix
CI: Set docker version to v20.10 in ubuntu:20.04 for s390x|ppc64le
2023-02-07 13:38:23 +01:00
Hyounggyu Choi
b399c72690 CI: Set docker version to v20.10 in ubuntu:20.04 for s390x|ppc64le
This is to make a docker version to v20.10 in docker upstream image ubuntu:20.04 for s390x and ppc64le.

Fixes: #6211
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Cherry-picked: f49b89b
2023-02-04 09:51:17 +00:00
Steve Horsman
6378fcb0be Merge pull request #6191 from ryansavino/downgrade-qemu-to-7.1.0-CCv0
[CCv0] versions: downgrade qemu from 7.2.0 to 7.1.0
2023-02-03 18:22:52 +00:00
Ryan Savino
d6c12476fb versions: downgrade qemu from 7.2.0 to 7.1.0
A known bug in qemu 7.2.0 causes a problem handling the kernel hashes argument and causes SEV container launching to fail.

Fixes: #6189

Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
2023-02-02 08:23:43 -06:00
Niteesh Dubey
9ef28e3e5e CCv0: Enable 'policy' for SNP container
includes conflict resolution in qemu_amd64.go
Signed-off-by: Niteesh Dubey <niteesh@linux.ibm.com>
2023-02-02 04:21:56 +00:00
Georgina Kinge
5b4075d18a CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #6181
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2023-02-01 10:10:40 +00:00
Fabiano Fidêncio
7345c14caa caching: Take runtime-rs into consideration when caching shim-v2
Now that TDX work will start coming for runtime-rs, let's also take it
into consideration when caching the shim-v2 tarball.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-30 14:12:26 +01:00
Fabiano Fidêncio
0f43ec8ff7 Revert "packaging: Don't build runtime-rs if no RUST_VERSION is provided"
This reverts commit 299829aec0, as we'll
start adding TDX support for the runtime-rs.
2023-01-30 14:00:31 +01:00
Niteesh Dubey
55b36212e6 CCv0: Remove qemu paramter 'policy' for SNP container
Removed the qemu paramter 'policy' (and also dh-cert-file, session-file, kernel-hashes=on)
for SNP container.

Fixes: #5795

Signed-off-by: Niteesh Dubey <niteesh@linux.ibm.com>
2023-01-27 17:18:58 +00:00
Steve Horsman
d3547814e5 Merge pull request #6104 from stevenhorsman/remove-skopeo-umoci
CC: Remove skopeo umoci
2023-01-27 10:31:33 +00:00
Fabiano Fidêncio
420203f054 Merge pull request #6120 from UnmeshDeodhar/CC-Cache-SEV-firmware
Adding caching capability for SEV ovmf
2023-01-27 08:47:30 +01:00
Steve Horsman
ee35096407 Merge pull request #6127 from Megan-Wright/CCv0
CCv0: Merge main into CCv0 branch
2023-01-26 12:22:15 +00:00
Megan Wright
c81071bb68 agent: update vendor packages
Update vendor packages after merge
Signed-off-by: Megan Wright <megan.wright@ibm.com>
2023-01-25 14:30:16 +00:00
stevenhorsman
3fd6c91ec6 doc: Update signature files path
Update the link to the signature files after x86 and s390x introduced

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-01-25 11:37:56 +00:00
stevenhorsman
e777860a0c osbuilder: Make skopeo version externalised
- Remove the hardcoded skopeo version arg
- Switch to read from version.yaml and pass via --build-arg

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-01-25 11:37:56 +00:00
stevenhorsman
9c75d43aef kata-deploy: Update cache version
- Remove skopeo & umoci from cache version now they aren't built
into the rootfs image

Fixes: #3970
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-01-25 11:37:56 +00:00
stevenhorsman
1bfbeee1dd versions: Remove umoci version
- Remove umoci entry from versions
- Update the usage of skopeo to control the tooling we use to build
the pause image
Fixes: #

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-01-25 11:37:56 +00:00
stevenhorsman
01a88fab2e doc: Remove umoci reference
Remove umoci from doc

Fixes: #3970
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-01-25 11:37:56 +00:00
stevenhorsman
ebffb7bc9b kata-deploy: Remove skopeo and umoci
- Now we don't need skopeo and umoci in the rootfs
remove the option from kata-deploy

Fixes: #3970
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-01-25 11:37:56 +00:00
stevenhorsman
dd2d6a94f1 osbuilder: Remove option to build skopeo, umoci
Now we don't need to have skopeo and umoci in the rootfs
remove the code that optionally builds and installs them

Fixes: #3970
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-01-25 11:37:56 +00:00
stevenhorsman
ee5fb8c998 agent: Remove container_policy_file config
- Remove the container_policy_file config parameter as it was only used
by the skopeo code path

Fixes: #3970
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-01-25 11:37:56 +00:00
stevenhorsman
dea1891851 agent: Remove skopeo & umoci
Remove code paths that use skopeo to pull images & umoci to unpack them

Fixes: #3970
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-01-25 11:37:56 +00:00
Megan Wright
36cd741943 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #6126
Signed-off-by: Megan Wright megan.wright@ibm.com
2023-01-25 11:23:18 +00:00
Unmesh Deodhar
32a6bf570a cache_components: Adding caching capability for SEV ovmf
These changed will be consumed by SEV firmware caching job in the CI. This will help in reducing the CI runtime.

Fixes: #6119

Signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
2023-01-24 15:00:18 -06:00
Steve Horsman
49b89c915c Merge pull request #6117 from stevenhorsman/image-pull-logs
agent: Improve logging of pull image
2023-01-24 08:53:37 +00:00
stevenhorsman
0fc7b4b74d agent: Improve logging of pull image
- Add agent log if pull image fails

Fixes: #6118

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-01-23 17:09:39 +00:00
Fabiano Fidêncio
b91337afef Merge pull request #6066 from fidencio/topic/CC-update-dependencies-for-release
CC | Update dependencies for the v0.3.0 release
2023-01-13 10:06:02 +01:00
Fabiano Fidêncio
7415227645 versions: Bump attestation-agent to v0.3.0
The attestation-agent had its v0.3.0 release earlier Today, following
the Confidential Containers v0.3.0 release process.

Let's bump it on our side, as we've already tested the version that
became this release.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-12 19:10:40 +01:00
Fabiano Fidêncio
e87b887589 build: Update image-rs to v0.3.0
image-rs has released its v0.3.0 release earlier Today, following the
v0.3.0 Confidential Containers release process.

The v0.3.0 is based on exactly the same commit we've been using already,
so no changes are expected for us.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-12 18:30:59 +01:00
Fabiano Fidêncio
a758cb0f7f versions: Update TD-Shim version to v0.3.0
TD-Shim has released its v0.3.0 release earlier Today, following the
Confidential Containers v0.3.0 release.

Let's update it here.  We need to also bump the toolchain to using the
nightly-2022-11-15

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-12 18:19:30 +01:00
Steve Horsman
d1993d2c90 Merge pull request #6057 from stevenhorsman/CCv-no-netns
runtime: fix up disable_netns handling
2023-01-12 13:19:48 +00:00
Peng Tao
d7ca0393d1 runtime: fix up disable_netns handling
With `disable_netns=true`, we should never scan the sandbox netns which
is the host netns in such case.

Fixes: #6021
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Cherry-picked: 12fd6ff
2023-01-12 08:57:20 +00:00
Steve Horsman
3a46a80fc2 Merge pull request #6034 from fidencio/topic/CC-consider-toolchain-version-for-td-shim-builder-image
CC | packaging: Take toolchain version when using the td-shim builder image
2023-01-11 17:26:08 +00:00
Steve Horsman
cbf2753a28 Merge pull request #6038 from BbolroC/make-target-cc-payload
CC: Support s390x for target cc-payload in Makefile
2023-01-11 17:23:28 +00:00
Hyounggyu Choi
c7148dc2c7 CC: Support s390x for target cc-payload in Makefile
This is to adjust dependencies for a target cc-payload based on
architecture.

Fixes: #6028

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-01-11 13:25:33 +01:00
Fabiano Fidêncio
8125f5dc97 packaging: Take toolchain version when using the td-shim builder image
As the toolchain is installed in the image itself, we *must* take the
toolchain into consideration when deciding whether to use a cached image
or building a new one.

Fixes: #6033

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-11 12:14:48 +01:00
Fabiano Fidêncio
03631a4360 Merge pull request #6025 from Alex-Carter01/online_kbc_debug
osbuilder: copy efi_secret module for online_sev_kbc
2023-01-10 23:11:44 +01:00
Alex Carter
db22befd78 osbuilder: copy efi_secret module for online_sev_kbc
Bug fix for #5651. Faulty bash syntax let a initrd build complete, but not copy the kernel module.
This change fixes the if logic to work as an 'or' as intended.

Fixes: #6024
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
2023-01-10 19:04:46 +00:00
Fabiano Fidêncio
ed59cebcbf Merge pull request #6012 from GeorginaKin/CCv0
CCv0: Merge main into CCv0 branch
2023-01-09 22:51:33 +01:00
Georgina Kinge
73532eb400 runtime: fixing merge conflicts
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2023-01-09 14:22:32 +00:00
Georgina Kinge
e8c63ccc4d CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #6011
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2023-01-09 12:02:48 +00:00
Fabiano Fidêncio
f23b087145 Merge pull request #5977 from stevenhorsman/foreign-CC
CC | agent: unset `CC` for cross-build
2023-01-04 18:36:48 +01:00
James Tumber
f4ececc8ac agent: unset CC for cross-build
When `HOST_ARCH` != `ARCH` unset `CC`

Specifying a foreign CC is incompatible with building libgit2. Thus after the RUSTFLAGS linker
has been set we can safely unset CC to avoid passing this value through the build.

Fixes: #5890

Signed-off-by: James Tumber <james.tumber@ibm.com>
Cherry-picked: 087515a
2023-01-04 15:17:25 +00:00
Fabiano Fidêncio
3db9100a5c Merge pull request #5651 from Alex-Carter01/online-sev-kbc
osbuilder: Switch to online_sev_kbc
2023-01-04 14:41:59 +01:00
Steve Horsman
de999429ce Merge pull request #5927 from dubek/sev-es-guest
CCv0: runtime: Support launching SEV-ES guests
2023-01-04 09:37:27 +00:00
Alex Carter
1b86be65f2 osbuilder: Switch to online_sev_kbc
Adds AA_KBC option in rootfs builder to specify online_sev_kbc into the initrd.
Guid and secret type for sev updated in shim makefile to generate default config
KBC URI will be specified via kernel_params
Also changing the default option for sev in the local build scipts
Making sure sev guest kernel module is copied into the initrd. Will also eventually be needed for SNP

Fixes: #5650
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
2023-01-04 04:04:13 +00:00
Fabiano Fidêncio
715fd0c933 Merge pull request #5969 from fidencio/topic/bump-image-rs
CC | agent: Update image-rs to bring stream pulling support
2023-01-03 22:09:51 +01:00
Fabiano Fidêncio
2a975c6546 Merge pull request #5939 from stevenhorsman/sev-initrd-suffix
CC | config: Add sev suffix to initrd
2023-01-03 21:37:33 +01:00
Fabiano Fidêncio
7da3635595 Merge pull request #5910 from Alex-Carter01/module_rootfs_fix
osbuilder: fix kernel module copy logic in rootfs builder
2023-01-03 17:34:20 +01:00
Fabiano Fidêncio
f1d6e64adf agent: Update image-rs to bring stream pulling support
Image layers stream pulling support has been merged into image-rs, and
we're now pulling an image-rs version that contains the merged code.

See: https://github.com/confidential-containers/image-rs/pull/96

Fixes: #5968

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 19:56:00 +01:00
Fabiano Fidêncio
474e37c286 runk: Fix needless_borrow warning
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to needless_borrow.

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 19:56:00 +01:00
Fabiano Fidêncio
c5be78a03d agent: Fix CCv0 specific unnecessary_lazy_evaluations
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to unnecessary_lazy_evaluations.

Let's fix them all here.

For more info about the warnings, please, take a look at:

https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_lazy_evaluations

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 19:56:00 +01:00
Fabiano Fidêncio
60a8a5bf4a agent: Fix CCv0 specific "match-like-matches-macro" warning
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to "match-like-matches-macro".

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#match_like_matches_macro

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 19:56:00 +01:00
Fabiano Fidêncio
eaf72daa80 agent: Fix CCv0 specific needless_borrow warnings
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to needless_borrow.

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 19:56:00 +01:00
Fabiano Fidêncio
853a3e0fa0 runtime-rs: Fix clippy::bool-to-int-with-if warnings
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to boolean to int conversion using if.

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#bool_to_int_with_if

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 19:56:00 +01:00
Fabiano Fidêncio
f5549de9cf runtime-rs: Fix needless_borrow warnings
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to needless_borrow.

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 19:56:00 +01:00
Fabiano Fidêncio
4d05ab8022 runtime-rs: Allow clippy:box_default warnings
As the rust toolchain version bump to its 1.66.0 release raised a
warning about using Box::default() instead of specifying a type.

For now that's something we don't need to change, so let's ignore such
warning in this very specific case.

See:
https://rust-lang.github.io/rust-clippy/master/index.html#box_default

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 17:15:46 +01:00
Fabiano Fidêncio
38c43f4e32 runtime-rs: Fix unnecessary_cast warnings
As we bumped the rust toolchain to 1.66.0, some new warnings have been
 raised due to unnecessary_cast.

 Let's fix them all here.

 For more info about the warnings, please, take a look at:
 https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_cast

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 16:37:24 +01:00
Fabiano Fidêncio
bdb94fa35b dragonball: Allow question_mark warning in allocate_device_resources()
As the rust toolchain version bump to its 1.66.0 release raised a
warning about the code being able to be refactored to use `?`.

For now that's something we don't need to change, so let's ignore such
warning in this very specific case.

See:
https://rust-lang.github.io/rust-clippy/master/index.html#question_mark

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 16:37:24 +01:00
Fabiano Fidêncio
c7eb516bc0 dragonball: Fix unnecessary_cast warnings
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to unnecessary_cast.

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_cast

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 16:37:24 +01:00
Fabiano Fidêncio
1085fac5c2 agent: Allow clippy::question_mark warning in Namespace{}
As the rust toolchain version bump to its 1.66.0 release raised a
warning about the code being able to be refactored to use `?`.

For now that's something we don't need to change, so let's ignore such
warning in this very specific case.

See:
https://rust-lang.github.io/rust-clippy/master/index.html#question_mark

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 15:34:02 +01:00
Fabiano Fidêncio
c3c9e1b4a3 agent: Fix explicit_auto_deref warnings
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to explicit_auto_deref.

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#explicit_auto_deref

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 15:33:58 +01:00
Fabiano Fidêncio
20be612cd1 agent: Fix needless_borrow warnings
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to needless_borrow.

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 15:33:53 +01:00
Fabiano Fidêncio
083e3f26ed rustjail: Fix unnecessary_cast warnings
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to unnecessary_cast.

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_cast

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 15:33:48 +01:00
Fabiano Fidêncio
1f391ef511 rustjail: Fix needless_borrow warnings
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to needless_borrow.

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 15:33:40 +01:00
Fabiano Fidêncio
bc71ca0d73 kata-types: Fix unnecessary_cast warnings
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to unnecessary_cast.

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_cast

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 15:33:33 +01:00
Fabiano Fidêncio
ac4c0ff9d6 kata-types: Fix needless_borrow warnings
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to needless_borrow.

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 15:33:29 +01:00
Fabiano Fidêncio
036e82a164 safe-path: Fix needless_borrow warnings
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to needless_borrow.

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 15:33:23 +01:00
Fabiano Fidêncio
4b4ecd0cba kata-sys-util: Fix "match-like-matches-macro" warning
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to "match-like-matches-macro".

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#match_like_matches_macro

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 15:33:13 +01:00
Fabiano Fidêncio
c16c1bde86 kata-sys-util: Fix unnecessary_cast warnings
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to unnecessary_cast.

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_cast

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 15:33:08 +01:00
Fabiano Fidêncio
021201005d kata-sys-util: Fix needless_borrow warnings
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to needless_borrow.

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 15:33:01 +01:00
Fabiano Fidêncio
716b22a458 logging: Allow clippy::type-complexity warning
As the rust toolchain version bump to its 1.66.0 release raised a
warning about the type complexity used for the closure, and that's
something we don't want to change, let's ignore such warning in this
very specific case.

See:
https://rust-lang.github.io/rust-clippy/master/index.html#type_complexity

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 15:32:52 +01:00
Fabiano Fidêncio
738d2d9736 logging: Fix needless_borrow warnings
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to needless_borrow.

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 15:32:47 +01:00
Fabiano Fidêncio
869b9d15a8 protocols: Fix unnecessary_cast warnings
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to unnecessary_cast.

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_cast

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 15:32:30 +01:00
Fabiano Fidêncio
eda857d898 protocols: Fix needless_borrow warnings
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to needless_borrow.

Let's fix them all here.

For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 15:31:12 +01:00
Fabiano Fidêncio
7c79806ff0 versions: Update the rust toolchain to 1.66.0
We're doing the bump on main, as we'll need this as part of the CCv0
branch due to the dependencies we have there.

Link to the 1.66.0 release:
https://github.com/rust-lang/rust/blob/master/RELEASES.md#version-1660-2022-12-15

Fixes: #5966
Backport: #5967

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-01-02 15:30:59 +01:00
Dov Murik
adec86cc40 runtime: Support launching SEV-ES guests
The `sev_guest_policy` configuration field distinguishes between SEV and
SEV-ES guests (according to standard AMD SEV policy values).

Modify the kata runtime to detect SEV-ES guests and calculate calculate
the expected launch digest taking into account the number of VCPUs and
their CPU signature (model/family/stepping).

Fixes: #5471

Signed-off-by: Dov Murik <dovmurik@linux.ibm.com>
2022-12-21 21:29:45 +00:00
stevenhorsman
ca2cb3faff config: Add sev suffix to initrd
- Add new sev initrd name and path to point to the correct name of
the file built

Fixes: #5938
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-12-21 16:25:04 +00:00
Steve Horsman
769f91de8b Merge pull request #5933 from Megan-Wright/CCv0
CC: Merge main into CCv0 branch
2022-12-21 13:35:57 +00:00
Steve Horsman
70b617db98 Merge pull request #5937 from stevenhorsman/remote-hypervisor-selinux
runtime: Update the remote hypervisor config
2022-12-21 10:17:55 +00:00
stevenhorsman
3ef2fd1784 runtime: Update the remote hypervisor config
Add the SELinux setting to ensure it is passed through to the remote
hypervisor

Fixes: #5936

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-12-20 13:11:54 +00:00
Megan Wright
e2341c1b33 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #5932
Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-12-20 09:34:27 +00:00
Steve Horsman
60beb49698 Merge pull request #5929 from stevenhorsman/fix-version-comment
version: Fix formatting
2022-12-19 13:33:27 +00:00
stevenhorsman
c66a26ac09 version: Fix formatting
Add space before start of comment

Fixes: #5928
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-12-19 10:04:58 +00:00
Fabiano Fidêncio
1ee684021d Merge pull request #5906 from GeorginaKin/CCv0
CCv0: Merge main into CCv0 branch
2022-12-17 22:31:45 +01:00
Wainer Moschetta
a851982f71 Merge pull request #5893 from Alex-Carter01/ovmf-fix
CC: static build switching sev ovmf to CC fork
2022-12-16 09:58:32 -03:00
Steve Horsman
e217628eb5 Merge pull request #5581 from dubek/sev-es-launch-digest
CCv0: runtime: pkg/sev: Support SEV-ES launch digest calculation
2022-12-16 11:38:45 +00:00
Alex Carter
8c96bd5c7c CC: static build switching sev ovmf to CC fork
Switching sev build of ovmf to the cc fork until patches are upstreamed.

Adding build for dependencies

Fixes: #5892
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
2022-12-15 16:46:25 +00:00
Alex Carter
bd17c7143c osbuilder: fix kernel module copy logic in rootfs builder
the switch to cases lets AA_KBC to be parsed correctly.

There will be an addition to the offline_sev_kbc case to do the same for online_sev_kbc

There will also be an addition for SNP

Fixes: #5909
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
2022-12-15 15:46:39 +00:00
Georgina Kinge
05311ffa37 runtime: Add disable_guest_selinux to CCv0
Adding blocks for disable_guest_selinux to extra CCv0 files.

Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-12-15 15:03:07 +00:00
Steve Horsman
c6f1a1e42a Merge pull request #5897 from BbolroC/runtime-payload-multiarch
CC: Enable multi-arch build for runtime-payload image
2022-12-15 10:01:25 +00:00
Fabiano Fidêncio
813f8b7e93 Merge pull request #5896 from fidencio/topic/CC-use-pre-built-containerd
versions: Use prebuilt containerd artefacts
2022-12-14 17:19:26 +01:00
Georgina Kinge
0701e171e2 agent: removing sysinfo during merge
Removed the sysinfo that got left behind during merge

Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-12-14 15:36:13 +00:00
Georgina Kinge
332a425145 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #5905
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-12-14 14:55:23 +00:00
Hyounggyu Choi
843b5afb3d CC: Enable multi-arch build for runtime-payload image
This is to enable quay.io/confidential-containers/runtime-payload for
s390x on top of amd64.

Fixes: #5894

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2022-12-14 11:38:27 +01:00
Fabiano Fidêncio
94ea07e42b versions: Use prebuilt containerd artefacts
In order to avoid building the containerd binaries on every single run
of our CI, let's simply rely on the fact that now we're releasing the
"same" artefacts containerd normally releases from our own fork.

See:
https://github.com/confidential-containers/containerd/releases/tag/v1.6.8.1

Fixes: #5895

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-14 10:11:48 +01:00
Fabiano Fidêncio
c2d1ea770a Merge pull request #5835 from fidencio/topic/CC-cache-rootfs
CC | Cache and use rootfs whenever it's possible
2022-12-13 20:50:48 +01:00
Fabiano Fidêncio
aef3e5184b kata-deploy-binaries: Use cached shim-v2 when possible
As done for different components, let's also use a cached version of the
shim-v2 whenever it's possible.

Fixes: #5838

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-12 20:07:32 +01:00
Fabiano Fidêncio
5d6ca191ce Merge pull request #5886 from fidencio/topic/CC-fix-pushing-the-shim-v2-build-image-to-quay
actions: Fix pushing the shim-v2 build image to quay.io
2022-12-12 19:32:33 +01:00
Gabriela Cervantes
de69f9c832 cache_components: Add the ability to cache the shim-v2
In order to cache the shim-v2 we're considering the the cached component
can be used if:
* There were no changes in the runtime directory
* There were no changes in the golang version used
* There were no changes in the rust version used
  * We don't build the rust agent, but better be prepared for the future
* There were no changes in the following files that are provided by the
  rootfs builds:
  * root_hash_vanilla.txt
  * root_hash_tdx.txt

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-12 19:12:26 +01:00
Fabiano Fidêncio
d9dd1ac9ec kata-deploy-binaries: Use cached rootfs when possible
As done for different components, let's also use a cached version of
the rootfs whenever it's possible.

Fixes: #5433

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-12 18:03:01 +01:00
Fabiano Fidêncio
60f0fc996e actions: Fix pushing the shim-v2 build image to quay.io
e1f075dc60 reworked the action so the
shim-v2 was split out of the matrix build.  With that done I ended up
not realising I'd need to log into the quay.io as one step of the
build-asset-cc-shim-v2 job.

Fixes: #5885

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-12 17:58:46 +01:00
Fabiano Fidêncio
4d835f6089 cache_components: Add the ability to cache the rootfs
This is the most complex part to cache, as the cached component can be
only used if:
* There were no changes in the agent
* There were no changes in the libs (used by the agent)
* There were no changes in the rootfs build scripts
* There is no change in the version of the following components:
  * attestation-agent (part of the rootfs)
  * gperf (used to build libseccomp)
  * libseccomp (used to build the agent)
  * pause image (part of the rootfs)
  * skopeo (part of the rootfs)
  * umoci (part of the rootfs)
  * rust (used to build the kata-containers and attestation agents)

We're relying on the last commit merged on places related to the rootfs
generation  and using that as the rootfs version and that should be good
enough for what we need.

Apart from everything already mentioned, we've also added the ability to
cache the `root_hash_vanilla.txt` and `root_hash_tdx.txt` files, as
those are needed for when building the shim-v2, in order to have
measured boot working there.

It's important to note that we've added the ability to cache *both*
files, and I've taken that path as the shim-v2 cache work (which will
come soon) relies on both files.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-12 16:07:45 +01:00
Fabiano Fidêncio
d3e0190786 Merge pull request #5876 from fidencio/topic/CC-rework-build-dependencies-so-measured-rootfs-can-be-used
Rework how we build the payload so measured boot can be used with the Operator
2022-12-12 16:03:45 +01:00
Fabiano Fidêncio
aa351bc989 kata-deploy-binaries: Print extra_opts when building shim-v2
This will help us, in the future, to debug any possible issue related to
the measured rootfs arguments passed to the shim during the build time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-12 13:54:56 +01:00
Fabiano Fidêncio
e1f075dc60 actions: Consider root_hash_*.txt for the CC actions
The ability to do a measured boot has been overlooked when releasing the
payload consumed by the Confidential Containers project, and this
happened as we depend, at the shim-v2 build time, of a `root_hash_*.txt`
generated in the `tools/osbuilder/` directory, which is then used to add
a specific parameter to the `kernel_params` in the Kata Containers
configuration files.

With everything said above, the best way we can ensure this is done is
by saving those files during the rootfs build, download them during the
shim-v2 build (which *must* happen only after the rootfs builds happen),
and correctly use them there.

Fixes: #5847

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-12 13:54:56 +01:00
Fabiano Fidêncio
788f1e6d61 config: Adjust TDXKERNELPARAMS for different VMMs
As Cloud Hypervisor and QEMU are using different rootfs images (the
former with `offline_fs_kbc` as aa_kbc, and the latter with `eaa_kbc`),
we need to differentiate the kernel parameters passed to each one of
those, as the `root_hash.txt` file used for measured boot will differ
according to the rootfs used.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-12 13:54:56 +01:00
Fabiano Fidêncio
24c7066425 config: Drop $(ROOTMEASURECONFIG) from SEV kernel param
This is not used at all, as SEV relies on an initrd instead of relying
on an image.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-12 13:54:56 +01:00
Fabiano Fidêncio
0a1de98179 build_image: config: Allow passing a specific root_hash_suffix
By doing this we can ensure that when building different rootfs-images
we won't end up overring the `root_hash.txt` file.

Plus, this will help us later in this series to pass the correct
argument to be used with the respective image.

Nothing's been done for SEV as it uses a initrd instead of an image.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-12 13:54:52 +01:00
Fabiano Fidêncio
2d48300bf8 build_image: Don't add root_hash.txt to the final tarball
Adding the `root_hash.txt` to the final tarball doesn't bring any
benefit to the project, as the file dependency is for building the
shim-v2 and passing the correct measurement for the kernel command line.

It's important to mention that when building shim-v2, it doesn't look
for the file in `/opt/confidential-containers/share/kata-containers`,
bur rather in the `${repo_root_dir}/tools/osbuilder/`, as shown here:
ac3683e26e/tools/packaging/kata-deploy/local-build/kata-deploy-binaries.sh (L228-L232)

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-09 21:01:54 +01:00
Fabiano Fidêncio
de4f1231c0 Merge pull request #5852 from BbolroC/payload-390x-fix
CC: Fix issues on cc-payload-ci workflow for s390x
2022-12-09 20:26:08 +01:00
Fabiano Fidêncio
bd753b5834 Merge pull request #5863 from fidencio/topic/CC-rework-tdx-config-files
CC | Rework TDX runtime configuration files
2022-12-09 16:52:21 +01:00
Fabiano Fidêncio
a75e83ab41 config: Drop clh-tdx-eaa-kbc configuration file
It turns out that there's more work needed to be done on the Cloud
Hypervisor side so we can fully support EAA_KBC with it.

For now, let's remove the configuration as the tests are not currently
passing when using it, and stick to the `offline_fs_kbc` and its
specific image for the Cloud Hypervisor + TDX case.

Fixes: #5862

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-08 13:29:15 +01:00
Fabiano Fidêncio
788c327cac config: Set qemu-tdx as the one with eaa_kbc support
The `qemu-tdx` configuration is tied to using `offline_fs_kbc` as the
aa_kbc, which is something we're moving away from.

With this in mind, let's rename the `qemu-tdx-eaa-kbc` to `qemu-tdx` and
decrease the amount of the way too many configurations that we ship.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-08 13:09:21 +01:00
Fabiano Fidêncio
7257470fdc Merge pull request #5855 from Megan-Wright/CCv0
CC: Merge main into CCv0 branch
2022-12-08 12:10:57 +01:00
Megan Wright
7c1b3f0b61 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #5854
Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-12-07 11:15:50 +00:00
Hyounggyu Choi
6e6842c815 CC: Fix issues on cc-payload-ci workflow for s390x
This is to differentiate an artifact name between amd64 and s390x and add a
virtiofsd target for s390x.

Fixes: #5851

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2022-12-06 21:59:53 +01:00
Fabiano Fidêncio
9ac9f99c3a Merge pull request #5846 from fidencio/topic/fix-install_cc_tee_kernel
kernel: Export KATA_BUILD_CC in install_cc_tee_kernel()
2022-12-06 16:37:50 +01:00
Fabiano Fidêncio
265347993d Merge pull request #5826 from GeorginaKin/CCv0
CCv0: Merge main into CCv0 branch
2022-12-06 16:29:02 +01:00
Steve Horsman
ac3683e26e Merge pull request #5783 from BbolroC/multi-arch-payload-ci
CC: Add cc-payload-ci workflow for s390x
2022-12-06 12:15:19 +00:00
Fabiano Fidêncio
32e91d9d8a kernel: Export KATA_BUILD_CC in install_cc_tee_kernel()
As already done for install_cc_kernel(), let's ensure we export
KATA_BUILD_CC=yes as part of the install_cc_tee_kernel.

This is used to generate the hash of the devices in the initramfs.

Fixes: #5845

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-06 11:47:46 +01:00
Fabiano Fidêncio
e271eba4e0 kernel: Only print build info when not using cached artefacts
Let's move the info about building initramfs to *after* trying to
install the cached components.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-06 11:47:41 +01:00
GabyCT
7830e94e8c Merge pull request #5827 from GabyCT/topic/cachevirtofsdccc
CCv0: use virtiofsd cache
2022-12-05 09:31:31 -06:00
Hyounggyu Choi
69b037d53e CC: Add cc-payload-ci workflow for s390x
This is to adjust the existing cc-payload-ci workflow for s390x.

Fixes: #5660

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2022-12-05 14:23:23 +01:00
Fabiano Fidêncio
846c765c89 Merge pull request #5834 from mattarnoatibm/agent-target-dependencies
agent: Fix s390x agent build
2022-12-05 14:15:28 +01:00
Gabriela Cervantes
beceb3c844 CCv0: use virtiofsd cache
This PR allow us to use the virtiofsd cache tarball instead of
building it from source.

Fixes #5356

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-12-05 12:41:53 +01:00
Fabiano Fidêncio
bbdd3b35d3 Merge pull request #5828 from fidencio/topic/cached-components-add-support-for-caching-firmwares
CC | Add support for caching firmwares
2022-12-05 12:34:18 +01:00
Matthew Arnold
43c9b8fb5d agent: Fix s390x agent build
Exclude the image-rs cosign feature when the build target
is the s390x architecture.

Change Cargo to use workspace resolver 2 so that conditional
include for the image-rs crate is resolved correctly for different
targets.

Update cargo lock.

Fixes: #5582

Signed-off-by: Matthew Arnold <mattarno@uk.ibm.com>
2022-12-05 10:50:05 +00:00
Fabiano Fidêncio
3b6dd03b04 kata-deploy-binaruies: Use wget instead of curl for cached components
It seems that the Kata Containers jenkins may be very slow to reach from
behind the firewall, causing TDX machine to fail downloading some of the
cached artefacts.

With this in mind, let's switch to using wget for this specific case.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-05 10:24:22 +01:00
Fabiano Fidêncio
56d5d5932d kata-deploy-binaries: Avoid pushd / popd if not needed
Let's avoid getting into a dir and risking not being able to leave that
dir in case something fails.

Instead, let's just stay in the current dir and move the final tarball
to the exoected directory in case all the checks go as expected.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-05 10:24:22 +01:00
Fabiano Fidêncio
724108a817 kata-deploy-binaries: Fix getting TDVF version
It's under the externals sections, not under assets.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-05 10:24:22 +01:00
Fabiano Fidêncio
5f2eb63574 kata-deploy-binaryes: Adapt td-shim version for its cached version
With the cached version we're concatenating the td-shim version with the
toolchain version used to build the project.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-05 10:24:22 +01:00
Fabiano Fidêncio
316a4cfc8e cache_components: Add more document to the kernel / qemu options
Let's add a documentation about the environment variables that can be
used with the `-k` and `-q` options.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-05 10:24:19 +01:00
Fabiano Fidêncio
8d03bc7e4b cache_components: Remove unused qemu_script_dir
The `qemu_script_dir` is a leftover from before the rework on how we
cache the components.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-03 02:22:49 +01:00
Fabiano Fidêncio
fdd6826d40 cache_components: Add support for caching firmwares
As we're already doing for some components, let's also add support for
caching firmwares. TD-Shim and TDVF are the ones supported for now.

Fixes: #5360, #5361

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-03 02:22:39 +01:00
stevenhorsman
578ee62da2 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #5824
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-12-02 16:15:39 +00:00
Fabiano Fidêncio
d910a79a3f Merge pull request #5823 from fidencio/topic/cached-components-fix-clh-tarball-name
CC | packaging: Fix the cloud hypervisor cached logic
2022-12-02 15:43:58 +01:00
Fabiano Fidêncio
2b72f8337c kata-deploy-binaries: Fix OVMF/TDVF component_version
We should use {} instead of () when passing the component version to the
install_cached_component() function.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-02 14:07:45 +01:00
Fabiano Fidêncio
f93e0c1541 packaging: Fix typos in cache_components.sh
asserts -> assets
stastic -> static

Those were not caught during the first merge of the series as we didn't
have CI jobs testing for the TEE artefacts.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-02 12:45:20 +01:00
Fabiano Fidêncio
08bea6c900 packaging: Fix checksum logic
Unless the checksum fails, we should *not* call cleanup_and_return.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-02 12:03:19 +01:00
Fabiano Fidêncio
7e8e54ce7d packaging: Drop the TDX cloud hypervisor target
We're currently building Cloud Hypervusor with thE TDX feature
regardless of using with TDX or not.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-02 10:24:05 +01:00
Fabiano Fidêncio
b9dc23033a packaging: Fix cloud-hypervisor version cache
The name of the asset was wrong, "cloud-hypervisor" instead of
"hypervisor.cloud_hypervsior", generating an empty "latest" file.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-02 10:23:16 +01:00
Fabiano Fidêncio
dab8e5673e packaging: Fix the cloud hypervisor tarball name
The final cloud hypervisor tarball name is either
kata-static-cc-cloud-hypervisor.tar.xz or
kata-static-cc-tdx-cloud-hypervisor.tar.xz, meaning it uses
"cloud-hypervisor" instead of "clh" in the name.

Fixes: #5816

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-02 09:23:08 +01:00
Fabiano Fidêncio
72d2f19c98 Merge pull request #5784 from fidencio/topic/cached-components
Re-work the way we cache components for the CCv0 branch
2022-12-01 23:19:59 +01:00
Fabiano Fidêncio
0eb2b1f58c kata-deploy-binaries: Check for cached version
Let's check for the cached version of the components as part of the
kata-deploy-binaries.sh as here we already have the needed info for
checking whether a component is cached or not, and to use it without
depending on changes made on each one of the builder scripts.

Fixes: #5816

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-01 13:37:57 +01:00
Fabiano Fidêncio
3c79af1ee1 cache_components: Cache the final tarball
Instead of caching files generated during the component build, let's
cache the final tarball generated for each component.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-01 01:12:34 +01:00
Fabiano Fidêncio
8a4e771144 packaging: Add functions to generate component's image name
Let's do this as the component name will be re-used later on, when we
start checking whether a cached component needs to be rebuilt or not.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-12-01 01:12:34 +01:00
Fabiano Fidêncio
02f6af9e1e packaging: Move repo_root_dir to lib.sh
This is used in several parts of the code, and can have a single
declaration as part of the `lib.sh` file, which is already imported by
all the places where it's used.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-30 23:03:13 +01:00
Fabiano Fidêncio
58480aac34 packaging: Move calc_qemu_files_sha256sum() to lib.sh
We're going to use this function from different places, so we better
move it to lib.sh and avoid rewriting it.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-30 22:01:39 +01:00
Fabiano Fidêncio
5022a0d2c2 packaging: Don´t print sha256sum as part of sha256sum_from_files()
If you're directly using the output of this function, the info message
will show up as part of the string, and that's not what we want.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-30 22:00:44 +01:00
Fabiano Fidêncio
2c6b3d114c initramfs: get_from_kata_deps does't require a "kata_version" arg
It's been dropped for a long time now, as part of
a09e58fa80.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-30 22:00:02 +01:00
Fabiano Fidêncio
54e39dfcdf Merge pull request #5733 from Jordan9500/workdir-ccv0-fix
CC | agent: Update the merge_oci_process function to change cwd
2022-11-30 18:19:58 +01:00
Fabiano Fidêncio
94d0c3c97d Merge pull request #5767 from Megan-Wright/CCv0
CC: Merge main into CCv0 branch
2022-11-30 18:18:45 +01:00
Fabiano Fidêncio
e922c73f0c Merge pull request #5680 from stevenhorsman/authenticated-registry
CC | agent: Add auth reg support to the agent
2022-11-30 11:17:17 +01:00
Jordan Jackson
0264584935 agent: Update the merge_oci_process function to change cwd
Change the if statement to check if the CWD is set to /
Add unit tests for the correct merging of working directory
in the container and image process

Note: there is an outstanding question about one test case
Format code

Fixes: #5721

Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Signed-off-by: Jordan Jackson <jordan.jackson@ibm.com>
2022-11-29 09:20:22 +00:00
Dov Murik
6a65803f75 runtime: pkg/sev: Support SEV-ES launch digest calculation
This includes contructing VMSA pages, parsing OVMF footer table to fetch
the AP reset EIP address, and allowing different vcpu types.

Fixes: #5471

Signed-off-by: Dov Murik <dovmurik@linux.ibm.com>
2022-11-28 18:08:50 +00:00
Fabiano Fidêncio
db56d897ab Merge pull request #5744 from Jordan9500/env-ccv0-fix
CC | agent: Update the merge_oci_process to properly manage the env variables
2022-11-28 14:13:30 +01:00
Fabiano Fidêncio
32fa47e59e Merge pull request #5771 from fidencio/topic/CC-export-kernel-version
CC | build: export kernel_version to be used with its build script
2022-11-25 15:51:55 +01:00
Fabiano Fidêncio
8baee93fca build: Fix kernel/build.sh identation
Simply replace space by tabs in one line that was off.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-25 13:50:24 +01:00
Fabiano Fidêncio
8a6533501f build: export kernel_version to be used with its build script
Now that we're caching the kernel, we're relying on the kernel version
being exported.  This is already done for the CC kernel, but not for the
TEE specific ones.

Fixes: #5770

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-25 13:50:17 +01:00
Fabiano Fidêncio
56f798eac2 Merge pull request #5618 from GabyCT/topic/cachekernel
CCv0: Use cached kernel
2022-11-25 13:08:53 +01:00
Fabiano Fidêncio
95f1872319 Merge pull request #5711 from BbolroC/cc-builders-s390x
CC: Fix errors on building cc-builders on s390x
2022-11-25 12:13:50 +01:00
Jordan Jackson
5f2d81e490 agent: Update the merge_oci_process to properly manage the env variables
Loop through the images enviroment variables, checking if it exists
inside the target. If it does then do not append it.
Add unit tests for correctly merging the env variables of the pod yaml
and image itself in the container and image process
Format code

Fixes: #5730

Signed-off-by: Jordan Jackson <jordan.jackson@ibm.com>
2022-11-25 10:47:39 +00:00
Megan Wright
a71de1e5e9 runtime: Update vendor modules
Update containerd vendor modules after merge
Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-11-25 10:27:06 +00:00
Megan Wright
a8509821dd CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #5743
Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-11-25 09:55:44 +00:00
Fabiano Fidêncio
f72fe061eb tools: Pass kernel version to build script
This PR passes the kernel version to the build script.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-24 22:22:51 +00:00
Fabiano Fidêncio
4f9ce2cde0 Merge pull request #5746 from fidencio/topic/CC-revendor-containerd-to-v1.6.8
CC | runtime: Revendor containerd to v1.6.8
2022-11-24 19:22:53 +01:00
Fabiano Fidêncio
47a0a796bf Merge pull request #5671 from GabyCT/topic/fixqemucached
tools: Improve cached artifacts for qemu
2022-11-24 17:34:41 +01:00
Fabiano Fidêncio
92fa3329db runtime: Revendor containerd to v1.6.8
As we bumped containerd dependency to v1.6.8, let's also do the
re-vendor of its code on the runtime side.

Fixes: #5745

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-24 17:02:51 +01:00
stevenhorsman
bd7266ba66 agent: Add auth reg support
If the attestation-agent is used then enable image_client_auth
to enable the attempt to get registry credentials for the pull

Fixes: #5652

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-11-24 15:47:13 +00:00
Gabriela Cervantes
3e325f8da7 tools: Improve cached artifacts for qemu
This PR improves the cached artifacts for qemu components.

Fixes #5670

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-11-24 15:40:34 +01:00
Hyounggyu Choi
f24c47ea47 CC: Fix errors on building cc-builders on s390x
This is to fix errors on building cc-builders images on s390x.

Fixes: #5677

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2022-11-24 15:28:39 +01:00
Fabiano Fidêncio
4ba26bf86a Merge pull request #5698 from GeorginaKin/CCv0
CCv0: Merge main into CCv0 branch
2022-11-24 14:55:05 +01:00
stevenhorsman
c23c93edb2 runtime: Go formatter
Add new line between copyright and package

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-11-24 09:56:31 +00:00
stevenhorsman
bd153436dc runtime: Update deprecate package
- Switch io/ioutil to os

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-11-24 09:32:38 +00:00
stevenhorsman
c11f7733c8 version: Update version of containerd
Update version of containerd after
Fabiano's containerd bump PR:
https://github.com/confidential-containers/containerd/pull/6

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-11-23 14:26:30 +00:00
Gabriela Cervantes
e92b2e3ef1 CCv0: Use cached kernel
This PR enables the use for the cached kernel.

Fixes #5355

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-11-22 21:08:56 +00:00
Georgina Kinge
5da7d42e87 runtime: fix go.sum merge errors
Some packages got messed up during the merge

Fixes: #5696
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-11-18 17:24:59 +00:00
Georgina Kinge
5b5a1c3cb3 agent: fix Cargo.lock after merge
Fixing up the Cargo.lock file after some merge conflicts

Fixes: #5696
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-11-18 17:24:59 +00:00
Georgina Kinge
357da1f46d CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #5696
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-11-18 14:13:41 +00:00
Fabiano Fidêncio
4c26d8e16a Merge pull request #5547 from GabyCT/topic/clhcached
CCv0: Use cached cloud hypervisor
2022-11-16 09:53:50 +01:00
Gabriela Cervantes
f16b7410f7 CCv0: Use cached cloud hypervisor
This PR allows us to use the cached for the cloud hypervisor.

Fixes #5354

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-11-15 22:52:20 +00:00
Fabiano Fidêncio
ac74ce88de Merge pull request #5629 from fidencio/topic/CC-fix-cc-tdx-rootfs-image-tarball-target-name
CC | packaging: Fix `cc-tdx-rootfs-image-tarball` target invocation
2022-11-10 16:02:20 +01:00
Fabiano Fidêncio
360bdea3ad packaging: Fix cc-tdx-rootfs-image-tarball target invocation
The incorrect name causes `make cc-payload` to fail, as
`cc-tdx-rootfs-tarball` is a non existent target.

Fixes: #5628

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-09 12:57:02 +01:00
Fabiano Fidêncio
5b7009f2f9 Merge pull request #5600 from BbolroC/image-rs-s390x
CC: Make agent build differently for image-rs on s390x
2022-11-08 07:57:25 +01:00
Hyounggyu Choi
b4d3a79b56 CC: Make agent build differently for image-rs on s390x
This is just to keep the support for s390x without the cosign
verification while looking for a solution for #5582.

Fixes: #5599

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2022-11-07 18:37:24 +01:00
Fabiano Fidêncio
38b5818813 Merge pull request #5587 from fidencio/topic/bump-confidential-containers-dependencies
CC | Bump Confidential Containers dependencies
2022-11-07 15:54:23 +01:00
Fabiano Fidêncio
4d0658e3fa agent: Bump image-rs to v0.2.0
image-rs tagged its v0.2.0 release, let's bump it here as we're about to
release the payload for the v0.2.0 Confidential Containers release.

Fixes: #5593

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-07 12:45:15 +01:00
Fabiano Fidêncio
45857859ec versions: Bump td-shim to v0.2.0
Let's bump the td-shim to its `v0.2.0` release.
Together with the bump, let's also adapt its build scripts so we're able
to build the `v0.2.0` as part of our infra.

Fixes: #5593

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-07 12:44:58 +01:00
Fabiano Fidêncio
542b42ad58 versions: Bump attestation-agent to v0.2.0
The attestation-agent v0.2.0 has been released, let's bump it here and
ensure we use the new release as part of what will become the payload
for the Confidential Containers v0.2.0 release.

Fixes: #5593

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-07 10:54:36 +01:00
Fabiano Fidêncio
d45f7e54c4 Merge pull request #5592 from fidencio/topic/CC-backport-static-checks-split
CC | github: Parallelise static checks
2022-11-07 10:47:46 +01:00
James O. D. Hunt
62c5e4e9a9 snap: Unbreak docker install
It appears that _either_ the GitHub workflow runners have changed their
environment, or the Ubuntu archive has changed package dependencies,
resulting in the following error when building the snap:

```
Installing build dependencies: bc bison build-essential cpio curl docker.io ...

    :

The following packages have unmet dependencies:
docker.io : Depends: containerd (>= 1.2.6-0ubuntu1~)
E: Unable to correct problems, you have held broken packages.
```

This PR uses the simplest solution: install the `containerd` and `runc`
packages. However, we might want to investigate alternative solutions in
the future given that the docker and containerd packages seem to have
gone wild in the Ubuntu GitHub workflow runner environment. If you
include the official docker repo (which the snap uses), a _subset_ of
the related packages is now:

- `containerd`
- `containerd.io`
- `docker-ce`
- `docker.io`
- `moby-containerd`
- `moby-engine`
- `moby-runc`
- `runc`

Fixes: #5545.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
(cherry picked from commit 990e6359b7)
2022-11-07 09:12:33 +01:00
James O. D. Hunt
5f122a0760 snap: Use metadata for dependencies
Rather than hard-coding the package manager into the docker part,
use the `build-packages` section to specify the parts package
dependencies in a distro agnostic manner.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
(cherry picked from commit ca69a9ad6d)
2022-11-07 09:12:21 +01:00
Fabiano Fidêncio
98121472da github: Parallelise static checks
Although introducing an awful amount of code duplication, let's
parallelise the static checks in order to reduce its time and the space
used in the VMs running those.

While I understand there may be ways to make the whole setup less
repetitive and error prone, I'm taking the approach of:
* Make it work
* Make it right
* Make it fast

So, it's clear that I'm only attempting to make it work, and I'd
appreciate community help in order to improve the situation here.  But,
for now, this is a stopgap solution.

JFYI, the time needed for run the tests on the `main` branch went down
from ~110 minutes to ~60 minutes.  Plus, we're not running those on a
single VM anymore, which decreases the change to hit the space limit.

Reference: https://github.com/kata-containers/kata-containers/actions/runs/3393468605/jobs/5640842041

Ideally, each one of the following tests should be also split into
smaller tests, each test for one component, for instance.
* static-checks
* compiler-checks
* unit-tests
* unit-tests-as-root

Fixes: #5585

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 40d514aa2c)
2022-11-07 08:03:45 +01:00
Fabiano Fidêncio
656a3e06a7 Merge pull request #5580 from fidencio/topic/CC-build-and-provide-yet-another-tdx-runtime-class
CC | config: Add specific config for TDX + EAA KBC
2022-11-04 13:22:36 +01:00
Fabiano Fidêncio
1b93cd1661 Merge pull request #5546 from jimcadden/allow_empty_keysets
CC | runtime: Enable kernel hashes for all SEV guests
2022-11-04 13:22:01 +01:00
Jim Cadden
411482bf19 runtime: Enable kernel hashes for all SEV guests
This commit adds the `kernel-hashes=on` flag to the QEMU command line
for all SEV guests (previously, this was only enabled for SEV guests
with `guest_pre_attestation=on`. This change allows the AmdSev firmware
to be used for both encrypted and non-encrypted container images.

**Note:** This change makes the AmdSev OVMF build a requirement for all
SEV guests. The standard host OVMF package will no longer work.

Fixes #5307.

Signed-off-by: Jim Cadden <jcadden@ibm.com>
2022-11-03 08:23:17 -04:00
Steve Horsman
3dd655d60d Merge pull request #5337 from dubek/runtime-add-sev-tests
CCv0: runtime/pkg/sev: Add unit tests; allow measurement without kernel
2022-11-03 11:15:16 +00:00
Fabiano Fidêncio
1f799d6a29 config: Add AGENT_AA_KBC_PARAMS to the kernel config
Let's ensure we add the option for the user, at build time, to set the
AGENT_AA_KBC_PARAMS passed to the agent, via the kernel command line.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-03 11:37:14 +01:00
Fabiano Fidêncio
b14921937a config: Add specific config for TDX + EAA KBC
As we're switching TDX to using EAA KBC instead of OfflineFS KBC, let's
add the configuration files needed for testing this before we fully
switch TDX to using such an image.

Fixes: #5563

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-03 11:35:45 +01:00
Fabiano Fidêncio
c72fdaf916 packaging: Build TDX specific guest image
The specific TDX image relies on having EAA KBC, instead of using the
default `offline_fs_kbc`.

This image is, with this commit, built and distributed, but not yet used
by TDX specific configurations, which will be done in a follow-up
commit.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-03 11:35:45 +01:00
Fabiano Fidêncio
05db886e27 osbuilder: Remove non-needed packages at the end
Instead of removing the non-needed packages under `/usr/share` and then
installing new components, let's make sure we do the removal at the end
of our script.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-03 11:35:45 +01:00
Fabiano Fidêncio
af75ce04c1 osbuilder: Manually add universe repo
Let's do that instead of updating and installing the
`software-properties-common` package, as it reduces the final size of
the image.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-03 11:35:45 +01:00
Fabiano Fidêncio
68956ad127 osbuilder: Add /etc/tdx-attest.conf when using eaa_kbc
First of all, EAA KBC is only used with TDX, thus we can safely assume
that eaa_kbc means TDX, at least for now.

A `/etc/tdx-attest.conf` file, with the data "port=4050" is needed as
that's the default configuration for the Quote Generation Service (QGS)
which is present on the guest side.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-03 11:35:45 +01:00
Fabiano Fidêncio
6f16071359 guest-image: Allow passing the image / initrd suffix
This will become very handy by the moment we start building different
images targetting different TEEs.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-03 11:35:45 +01:00
Fabiano Fidêncio
95fbe46891 Merge pull request #5552 from stevenhorsman/sig-ver-param
Sig ver param
2022-11-03 11:22:31 +01:00
stevenhorsman
360e01c0f4 agent: Set image_client security_validate
Replace hard-coded aa_kbc_param check to set the image_client's
security_validate, with reading the setting from the agent config

Fixes: #4888
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-11-02 21:28:33 +00:00
Fabiano Fidêncio
b6bf1c3f2c Merge pull request #5574 from fidencio/topic/CC-increase-clh-timeouts
CC | clh: Increase the timeouts when using Conf Guests
2022-11-02 21:45:01 +01:00
Fabiano Fidêncio
76ef07a22d clh: Increase the timeouts when using Conf Guests
Launching a pod with measured boot enabled seems to be taking longer
than expected with Cloud Hypervisor, which leads to hitting a timeout
limit.

Let's double those timeout limits for now.

Fixes: #5576

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-11-02 18:36:21 +01:00
stevenhorsman
9aa4afee63 runtime: Disable signature verification in config
Add agent.enable_signature_verification=false to the kernel_params
default config to get backwards compatibility in config.
Note the the agent config will default this setting to true for security
reasons if it's unset

Fixes: #4888
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-11-02 17:21:50 +00:00
stevenhorsman
46a6c52ef4 agent: Add enable_signature_verification config
- Add a new agent config parameter enable_signature_verification which
defaults to true for security reasons
- Add unit tests to check parsing and defaults

Fixes: #4888
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-11-02 17:21:50 +00:00
Fabiano Fidêncio
5b1df532da Merge pull request #5562 from fitzthum/params_cmdline
CC | Agent: Allow agent config to be overwritten
2022-11-02 18:20:01 +01:00
Tobin Feldman-Fitzthum
d71e7bbd59 Agent: Allow agent config to be overwritten
Allows parameters in the agent config file to be overwritten
by the kernel commandline. Does not change trust model since
the commandline is measured.

Makes sure to set endpoints_allowed correctly.

Fixes: #5173

Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-11-02 13:27:23 +00:00
Steve Horsman
5f5b4f7da9 Merge pull request #5374 from Alex-Carter01/AA-tag-version
osbuilder: specify Attestation Agent tag for rootfs build
2022-11-01 17:42:35 +00:00
Megan Wright
3623c033c7 Merge pull request #5554 from Megan-Wright/CCv0
CCv0: Merge main into CCv0 branch
2022-11-01 16:42:45 +00:00
Fabiano Fidêncio
fe9ea1351f Merge pull request #5542 from arronwy/image-rs
CC | agent: Bump pinned version of image-rs to support cosign signature verification
2022-11-01 12:49:33 +01:00
Megan Wright
61ec234b6a CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #5553
Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-11-01 10:55:33 +00:00
Fabiano Fidêncio
8f56ad57ad Merge pull request #5544 from fidencio/topic/CC-install-rats-tls-tdx
CC | osbuilder: Install rats-tls-tdx
2022-11-01 09:39:09 +01:00
Fabiano Fidêncio
aa4d803e35 Merge pull request #5533 from arronwy/aa_kbc_params
CC | config: Export aa_kbc_params to be set in guest kernel command line
2022-10-31 22:37:20 +01:00
GabyCT
04f0fcc5eb Merge pull request #5398 from GabyCT/topic/qemucccached
CCv0: Use cached cc qemu tarball
2022-10-31 14:25:52 -06:00
Gabriela Cervantes
72a13f6064 CCv0: Use cached cc qemu tarball
This PR implements the use of a cached cc qemu tarball to speed up
the CI and avoid building the cc qemu tarball when it is not
necessary.

Fixes #5363

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-10-31 18:32:17 +00:00
Fabiano Fidêncio
68c962601c Merge pull request #5537 from GabyCT/updateclh
CC  backport tools: Fix indentation of build static clh script
2022-10-31 18:55:04 +01:00
Wang, Arron
112a3d2bae config: Export aa_kbc_params to be set in guest kernel command line
As we discussed in #5178, user need set aa_kbc_params config without
modify kata guest image, since kernel params is also measured in TEE
boot flow, we make aa_kbc_params can be parsed through kernel cmdline.

Fixes: #5178

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2022-10-31 22:26:02 +08:00
Fabiano Fidêncio
a1571721dd osbuilder: Install rats-tls-tdx
Inclavare released a rats-tls-tdx package, which we depend on for using
verdictd.

Let's install it when using EAA_KBC, as already done for the rats-tls
package.

One thin to note here is that rats-tls-tdx depends on libtdx-attest,
which depends on libprotobuf-c1, thus we had to add the intel-sgx repo
together with enabling the universe channel.

Fixes: #5543

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-31 13:11:37 +01:00
Fabiano Fidêncio
5885f005f1 ubuntu: 20.04 is focal, not bionic
Let's use the correct OS version name for Ubuntu 20.04.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-31 10:46:41 +01:00
Fabiano Fidêncio
30990872f1 ubuntu: Ensure RATS-TLS is only installed on 20.04
We're currently using Ubuntu 20.04 as the base for the Ubuntu rootfs,
meaning that right now there's no issue with the approach currently
taken.  However, if we do a bump of an Ubuntu version, we could face
issues as the rats-tls package is only provided for Ubuntu 20.04.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-31 09:26:23 +01:00
Wang, Arron
27affb2a63 agent: Bump pinned version of image-rs to support cosign signature
To support cosign signature verification.

Fix build warning in signal.rs:
error: unused `tokio::sync::MutexGuard` that must be used
  --> src/signal.rs:27:9
   |
27 |         rustjail::container::WAIT_PID_LOCKER.lock().await;
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: `-D unused-must-use` implied by `-D warnings`
   = note: if unused the Mutex will immediately unlock

Fixes: #5541

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2022-10-31 14:44:15 +08:00
Gabriela Cervantes
1b8b2f9dce tools: Fix indentation of build static clh script
This Pr removes single spaces and fix the indentation of the script.

Fixes #5528

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-10-28 19:43:01 +00:00
Alex Carter
63e6abfa9d osbuilder: specify Attestation Agent tag for rootfs build
Add tag entry to the attestation agent entry of the versions file.
Checkout tag commit after cloning AA in rootfs builder.

Fixes: #5373
Fixes: kata-containers#5373
Signed-Off-By: Alex Carter <alex.carter@ibm.com>
2022-10-27 16:21:37 +00:00
Dov Murik
423778aec7 runtime: sev: Add unittests; allow measurement without kernel
1. Add unit tests for pkg/sev
2. Allow CalculateLaunchDigest to calculate launch digest without direct
   booted kernel (and, therefore, without initrd and kernel cmdline).
   This mode is currently not used in kata.

Fixes: #5456

Signed-off-by: Dov Murik <dovmurik@linux.ibm.com>
2022-10-27 16:45:53 +03:00
Fabiano Fidêncio
b30deca617 Merge pull request #5518 from fidencio/topic/CC-allow-passing-a-builder-image-to-the-kata-deploy-scripts
CC | packaging: Allow passing a container builder to the scripts
2022-10-27 14:07:33 +02:00
Fabiano Fidêncio
64009be3d7 packaging: Allow passing a container builder to the scripts
This, combined with the effort of caching builder images *and* only
performing the build itself inside the builder images, is the very first
step for reproducible builds for the project.

Reproducible builds are quite important when we talk about Confidential
Containers, as users may want to verify the content used / provided by
the CSPs, and this is the first step towards that direction.

Fixes: #5517

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-26 14:42:14 +02:00
Fabiano Fidêncio
2ac01cac0d Merge pull request #5458 from Megan-Wright/CCv0
CCv0: Merge main into CCv0 branch
2022-10-26 08:53:09 +02:00
Fabiano Fidêncio
6cd021ce86 Merge branch 'CCv0' into CCv0 2022-10-26 01:20:54 +02:00
Fabiano Fidêncio
855e63a121 Merge pull request #5508 from fidencio/topic/CC-fix-up-issues-with-cached-images
CC | Fix up issues with cached images
2022-10-26 00:30:12 +02:00
Fabiano Fidêncio
86905cdcdf packaging: Append $(uname -m) to the images tags
We need to do that in order to avoid trying to use the image in an
architecture which is not yet supported (such as trying to use the x6_64
image on a s390x machine)

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 21:58:01 +02:00
Fabiano Fidêncio
c720869eef initramfs: Build dependencies as part of the Dockerfile
This will help to not have to build those on every CI run, and rather
take advantage of the cached image.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 21:06:45 +02:00
Fabiano Fidêncio
c3cb65d0bb Revert "packaging: Use existing image for the kata-deploy-build"
This reverts commit c1aac0cdea.

The reason this has to be reverted is because we cannot cache an image
that has a specific user, uid, gid, docker_host_id, and expect that to
work equally on different machines.  Unfortunately, this is one of the
images that cannot be cached at all.
2022-10-25 20:24:38 +02:00
Fabiano Fidêncio
959dc3226b Revert "packaging: Add infra to push the kata-deploy builder image"
This reverts commit fe8b246ae4.

The reason this has to be reverted is because we cannot cache an image
that has a specific user, uid, gid, docker_host_id, and expect that to
work equally on different machines.  Unfortunately, this is one of the
images that cannot be cached at all.
2022-10-25 20:23:28 +02:00
Fabiano Fidêncio
44414e1708 lib.sh: Fix get_last_modification()
The ${file} path is an absolute path, as /home/fidencio/..., while the
result of the `git status --porcelain` is a path relative to the
${repo_root_dir}.  Because of this, the logic to adding `-dirty` to the
image name would never work.

Let's fix this by removing the ${repo_root_dir} from the ${file} when
grepping for it.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 20:21:03 +02:00
Fabiano Fidêncio
712177a337 Merge pull request #5482 from fidencio/topic/CC-cache-images-used-to-build-the-artefacts
CC | Avoid rebuilding the containers used for building the artefacts
2022-10-25 17:55:13 +02:00
Wang, Arron
25a961f5f3 initramfs: Add build script to generate initramfs
The init.sh in initramfs will parse the verity scheme,
roothash, root device and setup the root device accordingly.

Fixes: #5135

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2022-10-25 14:23:09 +01:00
Fabiano Fidêncio
c916c98ab5 actions: Push the builder images as part of the payload generation
Let's take advantge of an existing action that publishes the payload
after each pull request, to also publish the "builder images" used to
build each one of the artefacts.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
111ad87828 packaging: Add infra to push the initramfs builder image
Let's add the needed infra for only building and pushing the initramfs
builder image to the Kata Containers' quay.io registry.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
ebf6c83839 packaging: Use exissting image to build the initramfs
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder for the initramds.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
94807e73e7 packaging: Don't remove QEMU image
Now that the QEMU builder image provides only the environment used for
building QEMU, let's ensure it doesn't get removed.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
d4db7ed3c8 packaging: Add infra to push the QEMU builder image
Let's add the needed infra for only building and pushing the QEMU
builder image to the Kata Containers' quay.io registry.

Fixes: #5481

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
9e1df04e66 packaging: Use existing image to build QEMU
Let's first try to pull a pre-existsing image, instead of building our
own, to be used as a builder image for QEMU.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
b26cd250c8 qemu: Re-work static-build Dockerfile
Differently than every single other bit that's part of our repo, QEMU
has been using a single Dockerfile that prepares an environment where
the project can be built, but *also* building the project as part of
that very same Dockerfile.

This is a problem, for several different reasons, including:
* It's very hard to have a reproducible build if you don't have an
  archived image of the builder
* One cannot cache / ipload the image of the builder, as that contains
  already a specific version of QEMU
* Every single CI run we end up building the builder image, which
  includes building dependencies (such as liburing)

Let's split the logic into a new build script, and pass the build script
to be executed inside the builder image, which will be only responsible
for providing an environment where QEMU can be built.

Fixes: #5464
Backports: #5465

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
a036584ed9 packaging: Add infra to push the virtiofsd builder image
Let's add the needed infra for only building and pushing the virtiofsd
builder image to the Kata Containers' quay.io registry.

Fixes: #5480

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
29f64d6181 packaging: Use existing image to build virtiofsd
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the virtiofsd.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
9ba01f36de virtiofsd: Pass the expected toolchain to the build container
Let's ensure we're building virtiofsd with a specific toolchain that's
known to not cause any issues, instead of always using the latest one.

On each bump of the virtiofsd, we'll make sure to adjust this according
to what's been used by the virtiofsd community.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
42fd229f26 packaging: Add infra to push the td-shim builder image
Let's add the needed infra for only building and pushing the td-shim
builder image to the Kata Containers' quay.io registry.

Fixes: #5479

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
55cdd92b57 packaging: Use existing image to build td-shim
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the td-shim.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
ca8abc6cae packaging: Add infra to push the shim-v2 builder image
Let's add the needed infra for only building and pushing the shim-v2
builder image to the Kata Containers' quay.io registry.

Fixes: #5478

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
1c1034255a packaging: Use existing image to build the shim-v2
Let's try to pull a pre-existing image, instead of building our own, to
be used as a builder for the shim-v2.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
92d5dbb20c packaging: Add infra to push the OVMF builder image
Let's add the needed infra for building and pushing the OVMF builder
image to the Kata Containers' quay.io registry.

Fixes: #5477

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
5cef4d9837 packaging: Use existing image to build OVMF
Let's first try to pull a pre-existing image, instead of buildinf our
own, to be used as a builder image for OVMF.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
31a13e8081 packaging: Add infra to push the kernel builder image
Let's add the needed infra for only building and pushing the kernel
builder image to the Kata Containers' quay.io registry.

Fixes: #5476

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
3cd900da6d packaging: Use existing image to build the kernel
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the kernel.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
fe8b246ae4 packaging: Add infra to push the kata-deploy builder image
Let's add the needed infra for only building and pushing the image used
to build the kata-deploy artefacts to the Kata Containers' quay.io
registry.

Fixes: #5475

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
c1aac0cdea packaging: Use existing image for the kata-deploy-build
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the kata-deploy artefacts.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
a6c0bf8823 packaging: Add push_to_registry()
This function will push a specific tag to a registry, whenever the
PUSH_TO_REGISTRY environment variable is set, otherwise it's a no-op.

This will be used in the future to avoid replicating that logic in every
builder used by the kata-deploy scripts.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
b1454dbcaa packaging: Add get_last_modification()
Let's add a function to get the hash of the last commit modifying a
specific file.

This will help to avoid writing `git rev-list ...` into every single
build script used by the kata-deploy.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:16:59 +02:00
Fabiano Fidêncio
fa1bf8f75c packaging: Add and export CC_BUILDER_REGISTRY
CC_BUILD_REGISTRY, which points to quay.io/kata-containers/cc-builder,
will be used for storing the builder images used to build the artefacts
via the kata-deploy scripts.

The plan is to tag, whenever it's possible and makes sense, images like:
* ${CC_BUILDER_REGISTRY}:kernel-${sha}
* ${CC_BUILDER_REGISTRY}:qemu-${sha}
* ${CC_BUILDER_REGISTRY}:ovmf-${sha}
* ${CC_BUILDER_REGISTRY}:shim-v2-${go-toolchain}-{rust-toolchain}-${sha}
* ${CC_BUILDER_REGISTRY}:td-shim-${toolchain}-${sha}
* ${CC_BUILDER_REGISTRY}:virtiofsd-${toolchain}-${sha}

Where ${sha} is the sha of the last commit modifying the Dockerfile used
by the builder.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:03:06 +02:00
Fabiano Fidêncio
fb89a83c89 Merge pull request #5136 from arronwy/initramfs
initramfs: Add build script to generate initramfs
2022-10-25 15:01:22 +02:00
snir911
18283fd65a Merge pull request #5192 from kata-containers/CCv0-validate-hp-size
CCv0: agent: validate hugepage size is supported
2022-10-23 08:15:16 +03:00
Fabiano Fidêncio
5ddbce0746 Merge pull request #5483 from fidencio/topic/CC-do-not-build-runtime-rs
CC | packaging: Don't build runtime-rs if no RUST_VERSION is provided
2022-10-21 12:06:16 +02:00
Wang, Arron
a5dd0cd3ab initramfs: Add build script to generate initramfs
The init.sh in initramfs will parse the verity scheme,
roothash, root device and setup the root device accordingly.

Fixes: #5135

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2022-10-21 17:38:54 +08:00
Fabiano Fidêncio
53bcaf0547 Merge pull request #5314 from anakrish/anakrish-ccv0-optimize-integrity-no-wipe
CCv0: Optimize integrity device creation by avoiding a full device clear
2022-10-21 09:37:44 +02:00
Fabiano Fidêncio
299829aec0 packaging: Don't build runtime-rs if no RUST_VERSION is provided
As the CCv0 effort is not using the runtime-rs, let's add a mechanism to
avoid building it.

The easiest way to do so, is to simply *not* build the runtime-rs if the
RUST_VERSION is not provided, and then not providing the RUST_VERSION as
part of the cc-shim-v2-tarball target.

Fixes: #5462

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-20 21:54:45 +02:00
Fabiano Fidêncio
2ca6319f18 Merge pull request #5466 from fidencio/topic/CC-fix-runtime-payload-ci-registry-address
CC | actions: Fix runtime-payload-ci registry address
2022-10-20 07:29:50 +02:00
Fabiano Fidêncio
323fb9cfe8 Merge pull request #5459 from fidencio/topic/add-github-action-to-automate-CC-release
CC | action: Automate CC payload release
2022-10-19 23:37:40 +02:00
Fabiano Fidêncio
b3bd4e432c actions: Fix runtime-payload-ci registry address
There was a typo in the registry name, which should be
quay.io/confidential-containers/runtime-payload-ci instead of
quay.io/repository/confidential-containers/runtime-payload-ci

Fixes: #5469

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-19 22:41:04 +02:00
Fabiano Fidêncio
65de96e774 Merge pull request #5461 from fidencio/topic/generate-a-CI-tarball-after-every-push
CC | actions: Publish a payload on every CCv0 push
2022-10-19 21:49:12 +02:00
Fabiano Fidêncio
74b2ab001d action: Automate CC payload release
Let's create a GitHub action to automate the Kata Containers payload
generation for the Confidential Containers project.

This GitHub action builds the artefacts (in parallel), merges them into
a single tarball, generates the payload with the resulting tarball, and
uploads the payload to the Confidential Containers quay.io.

It expects the tags to be used to be in the `CC-x.y.z` format, with x,
y, and z being numbers.

Fixes: #5330

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-19 19:40:30 +02:00
Fabiano Fidêncio
4648d8bec7 actions: Publish a payload on every CCv0 push
Let's have a GitHub action to publish the Kata Containers payload, after
every push to the CCv0 branch, to the Confidential Containers
`runtime-payload-ci` registry.

The intention of this action is to allow developers to test new
features, and easily bisect breakages that could've happened during the
development process.  Ideally we'd have a CI/CD pipeline where every
single change would be tested with the operator, but we're not yet
there.  In any case, this work would still be needed. :-)

It's very important to mention that this should be carefully considered
on whether it should or should not be merged back to `main`, as the flow
of PRs there is way higher than what we currently have as part of the
CCv0 branch.

Fixes: #5460

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-19 19:09:49 +02:00
Fabiano Fidêncio
c57f8ff669 packaging: Expand the vars on build-and-upload-payload.sh
Just for the sake of avoiding issues in the future.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-19 13:58:08 +02:00
Fabiano Fidêncio
f4437980b4 packaging: Allow passing an extra tag to build-and-upload-payload.sh
Let's modify the script so we allow passing an extra tag, which will be
used as part of the Kata Containers pyload for Confidential Containers
CI GitHub action.

With this we can pass a `latest` tag, which will make things easier for
the integration on the operator side.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-19 13:58:05 +02:00
Fabiano Fidêncio
54544dd617 packaging: Allow passing registry to build-and-upload-payload.sh
Let's make the registry an optional argument to be passed to the
`kata-deploy-build-and-upload-payload.sh` script, defaulting to the
official Confidential Containers payload registry.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-19 12:59:38 +02:00
Megan Wright
a06c6dd861 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #5457
Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-10-19 11:01:13 +01:00
Fabiano Fidêncio
f087380180 Merge pull request #5437 from fidencio/topic/backport-virtiofsd-build-inside-a-container
CC | backport | virtiofsd: Build inside a container
2022-10-18 19:57:17 +02:00
Fabiano Fidêncio
411888796b packaging: Fix kata-deploy-binaries-in-docker.sh with podman
Instad of bailing out whenever the docker group doesn't exist, just
consider podman is being used, and set the docker_gid to the user's gid.

Also, let's ensure to pass `--privileged` to the container, so
`/run/podman/podman.socket` (which is what `/var/run/docker.sock` points
to) can be passed to the container.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-18 15:11:20 +02:00
Fabiano Fidêncio
07e76c7392 packaging: Pass RUST_VERSION when building shim-v2
We need to pass the RUST_VERSION, in the same way done for GO_VERSION,
as nowadays both the go and the rust runtime are built.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-18 12:23:26 +02:00
Fabiano Fidêncio
821dd63710 snap: Build virtiofsd using the kata-deploy scripts
Let's build virtiofsd using the kata-deploy build scripts, which
simplifies and unifies the way we build our components.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 0bc5baafb9)
2022-10-15 10:42:43 +02:00
Fabiano Fidêncio
8797a87bfd snap: Create a task for installing docker
Let's have the docker installation / configuration as part of its own
task, which can be set as a dependency of other tasks whcih may or may
not depend on docker.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit cb4ef4734f)
2022-10-15 10:42:33 +02:00
Fabiano Fidêncio
babd9924c6 virtiofsd: Build inside a container
When moving to building the CI artefacts using the kata-deploy scripts,
we've noticed that the build would fail on any machine where the tarball
wasn't officially provided.

This happens as rust is missing from the 1st layer container.  However,
it's a very common practice to leave the 1st layer container with the
minimum possible dependencies and install whatever is needed for
building a specific component in a 2nd layer container, which virtiofsd
never had.

In this commit we introduce the second layer containers (yes,
comtainers), one for building virtiofsd using musl, and one for building
virtiofsd using glibc.  The reason for taking this approach was to
actually simplify the scripts and avoid building the dependencies
(libseccomp, libcap-ng) using musl libc.

Fixes: #5425

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 7e5941c578)
2022-10-15 10:41:45 +02:00
Fabiano Fidêncio
f0041f01ed Merge pull request #5422 from wedsonaf/verify_cid
image_rpc: always call `verify_cid` after determining the cid
2022-10-14 15:43:30 +02:00
Fabiano Fidêncio
842d278206 Merge pull request #5427 from fidencio/topic/CC-update-TDX-kernel-and-qemu-repos
CC | backport TDX kernel and qemu repo updates
2022-10-14 12:19:06 +02:00
Fabiano Fidêncio
af4f3cdfbd versions: Update TDX QEMU
The previously used repo will be removed by Intel, as done with the one
used for TDX kernel.  The TDX team has already worked on providing the
patches that were hosted atop of the QEMU commit with the following hash
4c127fdbe81d66e7cafed90908d0fd1f6f2a6cd0 as a tarball in the
https://github.com/intel/tdx-tools repo, see
https://github.com/intel/tdx-tools/pull/162.

On the Kata Containers side, in order to simplify the process and to
avoid adding hundreds of patches to our repo, we've revived the
https://github.com/kata-containers/qemu repo, and created a branch and a
tag with those hundreds of patches atop of the QEMU commit hash
4c127fdbe81d66e7cafed90908d0fd1f6f2a6cd0.  The branch is called
4c127fdbe81d66e7cafed90908d0fd1f6f2a6cd0-plus-TDX-v3.1 and the tag is
called TDX-v3.1.

Knowing the whole background, let's switch the repo we're getting the
TDX QEMU from.

Fixes: #5419

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 35d52d30fd)
2022-10-14 10:03:33 +02:00
Fabiano Fidêncio
bfc3c45854 versions: Update TDX kernel
The previously used repo has been removed by Intel.  As this happened,
the TDX team worked on providing the patches that were hosted atop of
the v5.15 kernel as a tarball present in the
https://github.com/intel/tdx-tools repos, see
https://github.com/intel/tdx-tools/pull/161.

On the Kata Containers side, in order to simplify the process and to
avoid adding ~1400 kernel patches to our repo, we've revived the
https://github.com/kata-containers/linux repo, and created a branch and
a tag with those ~1400 patches atop of the v5.15.  The branch is called
v5.15-plus-TDX, and the tag is called 5.15-plus-TDX (in order to avoid
having to change how the kernel builder script deals with versioning).

Knowing the whole background, let's switch the repo we're getting the
TDX kernel from.

Fixes: #5326

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 9eb73d543a)
2022-10-14 10:03:26 +02:00
Fabiano Fidêncio
e42fce6ece Merge pull request #5407 from wedsonaf/fail-gracefully
image_rpc: Make `init_attestation_agent` fail gracefully
2022-10-13 18:57:35 +02:00
Fabiano Fidêncio
938de13c50 Merge pull request #5400 from wedsonaf/indirection
image_rpc: avoid double and triple indirections
2022-10-13 18:57:19 +02:00
Fabiano Fidêncio
c1fa5d60b7 Merge pull request #5149 from arronwy/kernel
kernel: Integrate initramfs into Guest kernel
2022-10-13 18:02:56 +02:00
Fabiano Fidêncio
325bafa7d8 Merge pull request #5169 from arronwy/configuration
config: Add root hash value and measure config to kernel params
2022-10-13 18:02:26 +02:00
Wedson Almeida Filho
cde438ceb7 image_rpc: always call verify_cid after determining the cid
Prior to this patch, we were missing a call to `verify_cid` when the cid
was derived from the image path, which meant that the host could specify
something like "prefix/..", and we would use ".." as the cid. Paths
derived from this (e.g., `bundle_path`) would not be at the intended
tree.

This patch factors the code out of `pull_image` so that it can be more
easily tested. Tests are added for a number of cases.

Fixes #5421

Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
2022-10-13 03:59:18 +01:00
Wedson Almeida Filho
c7e4548fc2 image_rpc: Make init_attestation_agent fail gracefully
Without this, a failure in `init_attestion_agent` would lead to a panic.

Fixes #5406

Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
2022-10-12 16:39:25 +01:00
Wedson Almeida Filho
2e90c62c31 image_rpc: avoid double and triple indirections
This also slightly improves readability by decluttering the function
declaration and call site.

Fixes #5405

Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
2022-10-12 16:24:05 +01:00
Fabiano Fidêncio
7dd8b78d44 Merge pull request #5328 from GeorginaKin/CCv0
CCv0: Merge main into CCv0 branch
2022-10-10 22:11:59 +02:00
stevenhorsman
d70fe49b28 runtime: Merge fixes
Update tests to fix merge errors

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-10-10 18:49:30 +01:00
Georgina Kinge
7de2cecfff CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #5327
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-10-10 11:12:22 +01:00
Georgina Kinge
760c899efd regenerate go.sum
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-10-07 15:50:29 +01:00
Georgina Kinge
c1fff85805 fix syntax error
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-10-07 15:50:29 +01:00
Steve Horsman
cc0561bcff Merge pull request #5319 from yoheiueda/ignore-image-ttrpc-async-rs
CCv0: Add image_ttrpc_async.rs to .gitignore
2022-10-07 11:23:18 +01:00
Fabiano Fidêncio
8e34807d49 Merge pull request #5324 from fidencio/topic/CC-backport-fix-for-kata-deploy-build-behind-proxy
CC | packaging: Mount $HOME/.docker in the 1st layer container
2022-10-06 16:55:02 +02:00
Anand Krishnamoorthi
4906228701 CCv0: Optimize integrity device creation by avoiding full device clear
Based on https://gitlab.com/cryptsetup/cryptsetup/-/issues/525

1. When --no-wipe is used, the device will have invalid checksums
2. mkfs.ext4 would fail on an un-wiped device due to reads of pages with
   invalid checksums
3. To make mkfs.ext4 work
   - Perform a dry run to figure out which sectors (pages) mkfs.ext4 will
     write to.
   - Perform directe writes to these pages to ensure that they will have
     valid checksums
   - Invoke mkfs.ext4 again to perform initialization
4 Use lazy_journal_init option with mkfs.ext4 to lazily initialize the journal.
  According to the man pages,
  "This speeds up file system initialization  noticeably, but carries some small
   risk if the system crashes before the  journal  has  been overwritten  entirely
   one time."
  Since the storage is ephemeral, not expected to survive a system crash/power cycle,
  it is safe to use lazy_journal_init.

Fixes #5329

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2022-10-05 12:29:54 -07:00
Fabiano Fidêncio
f476470c37 packaging: Mount $HOME/.docker in the 1st layer container
In order to ensure that the proxy configuration is passed to the 2nd
layer container, let's ensure the $HOME/.docker/config.json file is
exposed inside the 1st layer container.

For some reason which I still don't fully understand exporting
https_proxy / http_proxy / no_proxy was not enough to get those
variables exported to the 2nd layer container.

In this commit we're creating a "$HOME/.docker" directory, and removing
it after the build, in case it doesn't exist yet.  The reason we do this
is to avoid docker not running in case "$HOME/.docker" doesn't exist.

This was not tested with podman, but if there's an issue with podman,
the issue was already there beforehand and should be treated as a
different problem than the one addressed in this commit.

Fixes: #5077

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 4da743f90b)
2022-10-05 21:19:52 +02:00
Georgina Kinge
8c3846d431 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #5327
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-10-05 16:34:02 +01:00
Yohei Ueda
f8ad25e875 agent: Add image_ttrpc_async.rs to .gitignore
Fixes: #5318

Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
2022-10-04 17:33:50 +09:00
Snir Sheriber
e410c04622 agent: validate hugepage size is supported
before setting a limit, otherwise paths may not be found.
guest supporting different hugepage size is more likely with peer-pods where
podvm may use different flavor.

Fixes: #5191
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2022-09-28 17:23:46 +03:00
Wainer Moschetta
8ad86e2ec9 Merge pull request #5238 from stevenhorsman/don't-default-umoci-in-rootfs
osbuilder: don't pull in umoci with attestation
2022-09-26 10:38:16 -03:00
stevenhorsman
c9af89d094 osbuilder: don't pull in umoci with attestation
Umoci is not longer required if we have the attestation-agent, so don't
override the user input

Fixes: #5237
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-09-23 09:33:35 +01:00
Wainer Moschetta
fb56efd658 Merge pull request #5202 from stevenhorsman/remove-skopeo-from-deploy
packaging: Remove skopeo & umoci
2022-09-22 21:31:20 -03:00
stevenhorsman
77176cd7b9 packaging: Remove skopeo & umoci
- Remove skopeo and umoci from the CC image

Fixes: #4889
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-09-20 14:28:40 +01:00
Steve Horsman
f80723a483 Merge pull request #5167 from Alex-Carter01/sev-initrd-rename
kata-deploy: rename make target for sev initrd
2022-09-20 10:50:33 +01:00
Wainer Moschetta
63f931b719 Merge pull request #5127 from Alex-Carter01/depmod-rootfs
CC use depmod in rootfs builder for kernel modules with docker support
2022-09-19 09:33:33 -03:00
Wainer Moschetta
ad49a11761 Merge pull request #5038 from Alex-Carter01/sev-kernel-makefile
CC | packaging: Build SEV capable kernel + efi_secret module
2022-09-19 09:32:33 -03:00
Alex Carter
59537ceb2d osbuilder: CC, add kmod to rootfs builder docker environment
To use depmod in the rootfs builder, the docker environment will require kmod.

Fixes: kata-containers#5125

Signed-Off-By: Alex Carter <alex.carter@ibm.com>
2022-09-16 10:13:10 -04:00
Alex Carter
50a959e6cf osbuilder: CC use depmod in rootfs builder for kernel modules
Using depmod when adding kernel modules to get dependencies.

Needed for the efi secret module for sev.

Fixes: #5125

Signed-Off-By: Alex Carter <alex.carter@ibm.com>
2022-09-16 10:12:49 -04:00
Wainer Moschetta
c53f063acd Merge pull request #5024 from Alex-Carter01/rootfs-agent-config
CC | rootfs builder: add agent config file to rootfs for offline_sev_kbc
2022-09-16 10:25:30 -03:00
stevenhorsman
6233fa95d1 doc: Add new endpoints
Add new endpoints to the sample confidential config for the agent

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-09-16 07:50:11 +02:00
stevenhorsman
31164eeedf doc: workaround link checker
- Update crictl to remove backticks

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-09-16 07:50:11 +02:00
stevenhorsman
ef381b084d vendor: Bump image-rs version
- Bump the image-rs version to pick up the signing scheme updates

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-09-16 07:50:11 +02:00
stevenhorsman
812818d381 agent: Enable security_validate if required
If we are using the offline_fs_kbc and have created a resource json
then switch security_validate on the image_client to enable
the signature verification feature for image-rs

Fixes: #4581
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-09-16 07:50:11 +02:00
stevenhorsman
216c066795 doc: Update the agent build
Update the agent build to get around the nix & glibc linker problems
by running the libseccomp installation first

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-09-16 07:50:11 +02:00
stevenhorsman
3aebead189 doc: Update ccv0 doc and scripts
- Update the doc and scripts to reflect that skopeo isn't mandatory
for signature verification any longer
- Update the script to default the aa_kbc to offline_fs_kbc

Fixes: #4581
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-09-16 07:50:11 +02:00
Alex Carter
9f643ac9c8 osbuilder: add CC's agent config file to rootfs for offline_sev_kbc
Adds default config file.
Adds case in rootfs.sh to copy config.

Fixes kata-containers#5023

Fixes: #5023

Signed-Off-By: Alex Carter <alex.carter@ibm.com>
2022-09-15 18:26:29 +00:00
Wang, Arron
8225457dd5 config: Add root hash value and measure config to kernel params
After we have a guest kernel with builtin initramfs which
provide the rootfs measurement capability and Kata rootfs
image with hash device, we need set related root hash value
and measure config to the kernel params in kata configuration file.

Fixes: #5168

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2022-09-15 10:39:57 +08:00
Alex Carter
056d85de2c kata-deploy: rename make target for sev initrd
The sev initrd target had been changed to "cc-sev-rootfs-initrd".
This was good discussion as part of #5120.
I failed to rename it from "cc-sev-initrd-image" in kata-deploy-binaries.
The script will fail for a bad build target.

Fixes: #5166

Signed-Off-By: Alex Carter <alex.carter@ibm.com>
2022-09-14 23:34:33 +00:00
Wainer Moschetta
30460044a5 Merge pull request #5130 from Alex-Carter01/qemu-sev-runtimeclass
CC, kata-deploy: add kata-qemu-sev runtimeclass
2022-09-14 19:47:23 -03:00
Wainer Moschetta
895150e383 Merge pull request #5120 from Alex-Carter01/initrd-efi-dep
CC, packaging: add sev-initrd target, and add efi_secret module to sev's initrd
2022-09-14 08:03:55 -03:00
Alex Carter
69230fac97 packaging: add efi secret module to rootfs for sev
Adds the efi_secret kernel module to the sev initrd.

Adds a rootfs flag for kernel module based on the AA_KBC.

Finding the kernel module in the local build based on kernel version and kernel config version.

Moved kernel config version checking function from kernel builder to lib script.

Fixes: #5118

Signed-Off-By: Alex Carter <alex.carter@ibm.com>
2022-09-13 11:18:16 -04:00
Alex Carter
fa11294b0f packaging: add CC's sev-initrd target
Adds a make target, and a function in the kata-deploy-binaries script.
In the spirit of avoiding code duplication, making the cc-initrd function more generic.

Fixes: #5118

Signed-Off-By: Alex Carter <alex.carter@ibm.com>
2022-09-13 11:17:54 -04:00
Wainer Moschetta
9b49a6ddc6 Merge pull request #5026 from Alex-Carter01/sev-ovmf-makefile
CC| packaging: Add AmdSev OVMF target for use in payload images
2022-09-13 11:11:51 -03:00
Steve Horsman
65b4261aa8 Merge pull request #3697 from yoheiueda/CCv0-peerpod
Support Peer Pod VMs
2022-09-13 11:14:44 +01:00
Alex Carter
502a78730b packaging: Build SEV capable kernel + efi_secret module
Adds a new make target for an sev kernel which can be built and put into payload bundles for the operator.

Currently not including this sev kernel target in the cc payload bundle.

Unfortunately having to breakflow from using the generic cc_tee_kernel functions in either the kata-deploy-binaries or build-kernel.
Largely based on using an upstreamed kernel release, meaning the url is the defaul cdn, and e.g. we use version rather than tag.
The upside of this is that we can use the sha sum checking functionality from the generic get_kernel function.

CC label in title removed for commit message check.

Fixes: #5037

Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
2022-09-12 15:52:50 -04:00
Alex Carter
93a1780bdb packaging: Add CC's AmdSev OVMF target for use in payload images
Currently leaving the cc-sev-ovmf-tarball target out of the cc payload.
I was not sure where discussion had landed on the number of payload bundles.
e.g. could be included in a cc bundle along with tdx support or create an SEV bundle.

Fixes: kata-containers#5025

Fixes: #5025

Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
2022-09-12 12:59:11 -04:00
Wang, Arron
9373ec7d80 kernel: Integrate initramfs into Guest kernel
Integrate initramfs into guest kernel as one binary,
which will be measured by the firmware together.

Fixes: #5148

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2022-09-09 13:35:37 +08:00
Yohei Ueda
5422a056f2 runtime: Copy shared files recursively
This patch enables recursive file copying
when filesystem sharing is not used.

Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
2022-09-09 11:22:26 +09:00
Yohei Ueda
de232b8030 agent: support copying of directories and symlinks
This patch allows copying of directories and symlinks when
static file copying is used between host and guest. This change is
necessary to support recursive file copying between shim and agent.

Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
2022-09-09 11:07:34 +09:00
Lei Li
c3e6b66051 runtime: Support privileged containers in peer pod VM
This patch fixes the issue of running containers
with privileged as true.

See the discussion at this URL for the details.
https://github.com/confidential-containers/cloud-api-adaptor/issues/111

Author:    Lei Li <cdlleili@cn.ibm.com>
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
2022-09-09 11:07:34 +09:00
Yohei Ueda
f9278f22c3 runtime: Support the remote hypervisor type
This patch adds the support of the remote hypervisor type.
Shim opens a Unix domain socket specified in the config file,
and sends TTPRC requests to a external process to control
sandbox VMs.

Fixes #4482

Co-authored-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
2022-09-09 11:07:28 +09:00
Yohei Ueda
150e8aba6d runtime: Add hypervisor proto to support peer pod VMs
This patch adds a protobuf definiton of the remote hypervisor type.

Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
2022-09-09 10:18:23 +09:00
Fabiano Fidêncio
55c8c7226d Merge pull request #5134 from Megan-Wright/CCv0
CCv0: Merge main into CCv0 branch
2022-09-08 19:35:45 +02:00
Alex Carter
7849c7977c kata-deploy: add CC's kata-qemu-sev runtimeclass
After adding an SEV QEMU config file (#4850), need to configure containerd to select this when appropriate based on a new runtimeclass.

Adds to the configuration of containerd so the correct config is selected.

Fixes: #4851

Signed-Off-By: Alex Carter <alex.carter@ibm.com>
2022-09-08 15:19:45 +00:00
Fabiano Fidêncio
61fd408594 Merge pull request #5096 from ryansavino/sev-kernel-build-fix-ccv0
kernel: fix kernel tarball name for SEV for CCv0
2022-09-08 11:43:09 +02:00
Fabiano Fidêncio
b11b6e3756 Merge pull request #5129 from fidencio/topic/cc-simplify-kata-deploy-cleanup
kata-deploy-cc: Simplify cleanup
2022-09-07 15:41:06 +02:00
Megan Wright
2e7e81b8d8 CCv0: Revert version of tokio in Cargo.lock file
Revert version of tokio back to 1.20.1

Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-09-07 11:38:14 +01:00
Megan Wright
9ad37bfd90 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #5132
Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-09-07 09:44:08 +01:00
Fabiano Fidêncio
c17a6f1b53 kata-deploy-cc: Simplify cleanup
Let's remove the whole content from:
* /opt/confidential-containers/libexec
* /opt/confidential-containers/share

And then manually remove the binaries under bin directory` as the
pre-install hook will drop binaries there.

Finally, let's call a `rmdir -p /opt/confidential-containers/bin` which
should take care of the cleanup in case no pre-install hook is used, and
let's make sure we pass `--ignore-fail-on-non-empty` so we don't fail
when using a pre-install hook.

Fixes: #5128

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-09-07 00:05:45 +02:00
Fabiano Fidêncio
f5a6522398 Merge pull request #5121 from fidencio/topic/kata-deploy-fix-containerd-shim-v2-location
kata-deploy: Fix `containerd-shim-kata-v2` location
2022-09-06 19:28:00 +02:00
Ryan Savino
9cad7fb045 kernel: fix kernel tarball name for SEV
'linux-' prefix needed for tarball name in SEV case. Output to same file name.

Fixes: #5094

Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
2022-09-06 11:03:55 -05:00
Fabiano Fidêncio
124c0e7af4 kata-deploy: Fix containerd-shim-kata-v2 location
For Confidential Containers the file is present at
`/opt/confidential-containers` instead of `/opt/kata`.

Fixes: #5119

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-09-06 16:32:28 +02:00
Fabiano Fidêncio
3c7fe93997 Merge pull request #5045 from fidencio/topic/kata-deploy-misc-fixes
CC | kata-deploy: Misc fixes related to building the payload
2022-09-05 20:28:02 +02:00
Fabiano Fidêncio
01e29fc1fd kata-deploy-cc: Do a serial build of the payload
Every now and then we've been hitting issues with parallel builds.  in
order to not rely on lucky for the first release, let's do a serial
build of the payload image.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-09-05 18:20:24 +02:00
Fabiano Fidêncio
fb54dfd648 docs: How to generate the payload for the CC Operator
Let's add the documentation on how to generate the Kata Containers
payload, based in the CCv0 branch, that's consumed by the Confidential
Containers Operator.

Fixes: #5041

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-09-05 18:20:24 +02:00
Fabiano Fidêncio
012a76d098 kata-deploy: Rename image target to cc-payload
The `image` target is only used by and only present in the `CCv0`
branch, and it's name is misleading. :-)

Let's rename it (and the scripts used by it) to mention payload rather
than image, and to actually build the cc related tarballs instead of the
"vanilla" Kata Containers tarballs.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-09-05 16:21:17 +02:00
Fabiano Fidêncio
48c0cf5b5d kata-deploy: Adjust build & upload script
Let's adjust the `kata-deploy-build-and-upload-image.sh` to build the
image following the `kata-containers-${commit}` tag pattern, and to push
it to the quay.io/confidential-containers/runtime-payload repo.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-09-05 14:32:07 +02:00
Fabiano Fidêncio
518137f781 kata-deploy-cc: Try to remove /opt/confidential-containers
Let's try to remove the /opt/confidential-containers directory.  If it's
not empty, let's not bother force removing it, as the pre-install script
also drops files to the very same directory.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-09-05 14:29:57 +02:00
Fabiano Fidêncio
fb711e0e8e kata-deploy-cc: Improve shim backup / restore
We're currently backing up and restoring all the possible shim files,
but the default one ("containerd-shim-kata-v2").

Let's ensure this is also backed up and restored.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-09-05 14:29:57 +02:00
Fabiano Fidêncio
a43f95d01b kata-deploy: Rely on the configure config path
Instead of passing a `KATA_CONF_FILE` environament variable, let's rely
on the configured (in the container engine) config path, as both
containerd and CRI-O support it, and we're using this for both of them.

This is a "backport" of f7ccf92dc8, from
the original `kata-deploy.sh` to the one used for Confidential
Containers.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-09-05 14:29:57 +02:00
Fabiano Fidêncio
f684d00d50 kata-deploy-cc: Simplify the script
As containerd is the only supported container engine, let's simplify the
script and, at the same time, make it clear that other container engines
are not supported yet.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-09-05 14:29:57 +02:00
Fabiano Fidêncio
1f610ea5cc packaging: Improve qemu build image handling
Let's create the QEMU build image based on the version of QEMU used, so
if we happen to have a parallel build we ensure different images are
being used.

Also, let's ensure the image gets remove after the build.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-09-05 09:47:59 +02:00
Fabiano Fidêncio
abe89586c6 qemu: Keep passing BUILD_SUFFIX
In the commit 54d6d01754 we ended up
removing the BUILD_SUFFIX argument passed to QEMU as it only seemed to
be used to generate the HYPERVISOR_NAME and PKGVERSION, which were added
as arguments to the dockerfile.

However, it turns out BUILD_SUFFIX is used by the `qemu-build-post.sh`
script, so it can rename the QEMU binary accordingly.

Let's just bring it back.

Fixes: #5078

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 373dac2dbb)
2022-09-05 09:39:39 +02:00
Ryan Savino
a8feee68a8 qemu: create no_patches.txt file for SPR-BKC-QEMU-v2.5
Patches failing without the no_patches.txt file for SPR-BKC-QEMU-v2.5.

Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
(cherry picked from commit 59e3850bfd)
2022-09-05 09:39:25 +02:00
Ryan Savino
a2d9633dad qemu: fix tdx qemu tarball directories
Dockerfile cannot decipher multiple conditional statements in the main RUN call.
Cannot segregate statements in Dockerfile with '{}' braces without wrapping entire statement in 'bash -c' statement.
Dockerfile does not support setting variables by bash command.
Must set HYPERVISOR_NAME and PKGVERSION from parent script: build-base-qemu.sh

Fixes: #5078

Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
(cherry picked from commit 54d6d01754)
2022-09-05 09:39:03 +02:00
Fabiano Fidêncio
b6873f9581 kata-deploy: Add td-shim as part of the cc-tarball
4cf502fb20 added the ability to build
TD-Shim, but forgot to have it added as part of the cc-tarball target.

Fixes: #5042

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-09-05 08:39:37 +02:00
Fabiano Fidêncio
b9c0f7fb09 Merge pull request #5056 from GeorginaKin/CCv0
CCv0: Merge main into CCv0 branch
2022-09-02 13:40:23 +02:00
Fabiano Fidêncio
981c0b1646 Merge pull request #5013 from ryansavino/add-sev-config
config: Added SEV config
2022-09-02 11:36:45 +02:00
Ryan Savino
656d72bd74 config: Added SEV config
Added default sev kata config template.
Added required default variables in Makefile.

Fixes #5012
Fixes #5008

Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
2022-09-01 09:40:14 -05:00
Georgina Kinge
683755483b runtime: Remove duplicate fields
Remove duplicate fields produced from bad merge

Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-08-31 16:55:37 +01:00
Georgina Kinge
e46364ad98 vendor: regenerating vendor files
Fix vendor files after conflicts

Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-08-31 15:23:15 +01:00
Georgina Kinge
9931d4cbf0 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #5054
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-08-31 15:01:40 +01:00
Fabiano Fidêncio
20b999c479 Merge pull request #4967 from arronwy/generate_root_hash
CC | image-build: generate root hash as an separate partition for rootfs
2022-08-31 13:18:39 +02:00
Fabiano Fidêncio
2e77eb4bdb Merge pull request #5021 from fidencio/topic/add-https_proxy-to-the-agent-config
CC | docs: Add https_proxy to confidential agent config
2022-08-30 19:36:24 +02:00
Fabiano Fidêncio
512a92a543 docs: Add https_proxy to confidential agent config
The agent configuration file, which is part of the docs, is used by the
confidential containers CIs and, right now, cannot be run behind a
firewall, which is exactly how the TDX CIs are reunning, as https_proxy
is not set there.

Fixes: #5020
Depends-on: github.com/kata-containers/tests#5080

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-29 19:04:48 +02:00
Fabiano Fidêncio
e528b63f4f Merge pull request #4984 from fidencio/topic/cc-add-runtime-configuration-for-clh-plus-tdx
CC | Build and ship CLH TDX (and all the needed artefacts)
2022-08-26 08:19:30 +02:00
Fabiano Fidêncio
c362257142 kata-deploy-cc: Add CLH support to be used with TDX
As the previous commit added a new runtime class to be used with TDX,
let's make sure this gets shipped and configured as part of the
kata-deploy-cc script, which is used by the Confidential Containers
Operator.

This commit also cleans up all the extra artefacts that will be
installed in order to run the CLH TDX workloads.

Fixes: #4833
Depends-on: github.com/kata-containers/tests#5070

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-25 23:42:03 +02:00
Fabiano Fidêncio
0f4b5c08fe runtime: Add configuration file for CLH TDX
Let's add a new configuration file for using a cloud hypervisor (and all
the needed artefacts) that are TDX capable.

This PR extends the Makefile in order to provide variables to be set
during the build time that are needed for the proper configuration of
the VMM, such as:
* Specific kernel parameters to be used with TDX
* Specific kernel features to be used when using TDX
* Artefacts path for the artefacts built to be used with TDX
  * Kernel
  * TD-Shim

The reason we don't hack into the current Cloud Hypervisor configuration
file is because we want to ship both configurations, with for the
non-TEE use case and one for the TDX use case.

It's important to note that the Cloud Hypervisor used upstream is
already built with TDX support.

Fixes: #4831

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-25 23:42:03 +02:00
Fabiano Fidêncio
433ee7c92a Merge pull request #5002 from fidencio/topic/cc-backport-4999
CC | kernel: Ignore CONFIG_SPECULATION_MITIGATIONS for older kernels
2022-08-25 20:28:27 +02:00
Fabiano Fidêncio
4d1c0a3235 kernel: Ignore CONFIG_SPECULATION_MITIGATIONS for older kernels
TDX kernel is based on a kernel version which doesn't have the
CONFIG_SPECULATION_MITIGATIONS option.

Having this in the allow list for missing configs avoids a breakage in
the TDX CI.

Fixes: #4998

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-25 17:45:30 +02:00
Fabiano Fidêncio
20129dea87 Merge pull request #4986 from fidencio/topic/cc-drop-force_tdx_guest-kernel-parameter
CC | config: Drop `force_tdx_guest` unneeded parameter
2022-08-25 14:23:14 +02:00
Fabiano Fidêncio
6125587750 Merge pull request #4997 from fidencio/topic/cc-backport-fixes-for-cloud-hypervisor-plus-tdx
CC | Backport fixes for Cloud Hypervisor + TDX
2022-08-25 12:14:22 +02:00
Fabiano Fidêncio
cfece9b796 qemu: Drop unnecessary tdx_guest kernel parameter
With the current TDX kernel used with Kata Containers, `tdx_guest` is
not needed, as TDX_GUEST is now a kernel configuration.

With this in mind, let's just drop the kernel parameter.

Fixes: #4981

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-25 09:49:42 +02:00
Fabiano Fidêncio
f454bcdef1 clh: Use HVC console with TDX
As right now the TDX guest kernel doesn't support "serial" console,
let's switch to using HVC in this case.

Fixes: #4980

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-25 09:49:38 +02:00
Fabiano Fidêncio
0364184f90 clh: Avoid crashing when memory hotplug is not allowed
The runtime will crash when trying to resize memory when memory hotplug
is not allowed.

This happens because we cannot simply set the hotplug amount to zero,
leading is to not set memory hotplug at all, and later then trying to
access the value of a nil pointer.

Fixes: #4979

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-25 09:49:36 +02:00
Fabiano Fidêncio
c61d075cf7 clh: Increase API and SandboxStop timeouts for TDX
While doing tests using `ctr`, I've noticed that I've been hitting those
timeouts more frequently than expected.

Till we find the root cause of the issue (which is *not* in the Kata
Containers), let's increase the timeouts when dealing with a
Confidential Guest.

Fixes: #4978

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-25 09:49:33 +02:00
Fabiano Fidêncio
132d0e9927 clh: Lift the sharedFS restriction used with TDX
When booting the TDX kernel with `tdx_disable_filter`, as it's been done
for QEMU, VirtioFS can work without any issues.

Whether this will be part of the upstream kernel or not is a different
story, but it easily could make it there as Cloud Hypervisor relies on
the VIRTIO_F_IOMMU_PLATFORM feature, which forces the guest to use the
DMA API, making these devices compatible with TDX.

See Sebastien Boeuf's explanation of this in the
3c973fa7ce208e7113f69424b7574b83f584885d commit:
"""
By using DMA API, the guest triggers the TDX codepath to share some of
the guest memory, in particular the virtqueues and associated buffers so
that the VMM and vhost-user backends/processes can access this memory.
"""

Fixes: #4977

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-25 09:49:27 +02:00
Fabiano Fidêncio
4e9972fb50 config: Drop force_tdx_guest unneeded parameter
The `force_tdx_guest` kernel parameter was only needed in the early
development stages of the TDX kernel driver.  We can safely drop it with
the kernel version we've been currently using.

Fixes: #4985

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-24 21:56:48 +02:00
GabyCT
ccf21299cc Merge pull request #4971 from Megan-Wright/CCv0
CCv0: Merge main into CCv0 branch
2022-08-24 09:13:01 -05:00
Megan Wright
c6c8018730 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0
Fixes: #4970
Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-08-24 11:18:46 +01:00
Wang, Arron
2611779255 image-build: generate root hash as an separate partition for rootfs
Generate rootfs hash data during creating the kata rootfs,
current kata image only have one partition, we add another
partition as hash device to save hash data of rootfs data blocks.

Fixes: #4966

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2022-08-23 19:44:57 +08:00
GabyCT
4960f43ef6 Merge pull request #4960 from wainersm/CCv0-kata-deploy-skopeo
CCv0 | kata-deploy: allow to build the image without skopeo
2022-08-22 10:15:55 -05:00
Wainer dos Santos Moschetta
80a831e537 kata-deploy: allow to build the image without skopeo
The local-build script should honor the value of SKOPEO exported in the
environment so that it will be able to build the image without skopeo
inside. This remove the hard-coded "SKOPEO=yes".

Fixes #4959
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2022-08-19 18:22:10 -03:00
Wang, Arron
4831193bde agent: initialize trusted storage device
Initialize the trusted stroage when the device is defined
as "/dev/trusted_store" with shell script as first step.

Fixes: #4882

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2022-08-19 11:25:36 +02:00
Wang, Arron
813e36e615 agent: Support data_integrity option for trusted storage
After enable data integrity for trusted storage, the initialize
time will take three times more and IO performance will drop more than
30%, the default value will be NOT enabled but add this config to
allow the user to enable if they care more strict security.

Fixes: #4882

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2022-08-19 11:25:36 +02:00
Wang, Arron
ad4a811c39 image: Use guest provided pause image
By default the pause image and runtime config will provided
by host side, this may have potential security risks when the
host config a malicious pause image, then we will use the pause
image packaged in the rootfs.

Fixes: #4882

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2022-08-19 11:25:36 +02:00
Steve Horsman
96c47df5d8 Merge pull request #4947 from GeorginaKin/CCv0
CCv0: Merge main into CCv0 branch
2022-08-18 10:36:47 +01:00
stevenhorsman
73566bb4b9 agent: Convert image service to be async
With the runtime-rs changes the agent services need to be asynchronous,
so attempt to update the image_service to match this

Co-authored-by: Georgina Kinge <georgina.kinge@ibm.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-08-17 17:01:29 +01:00
Georgina Kinge
acb7a16522 runtime-rs: update LinuxCgroup type
Update translation code to match CCv0 changes

Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-08-17 15:38:36 +01:00
Georgina Kinge
bb9bbc7523 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4943
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-08-17 10:40:39 +01:00
Fabiano Fidêncio
bb196d56ca Merge pull request #4762 from arronwy/cryptsetup
CC | Add cryptsetup support in Guest kernel and rootfs
2022-08-17 08:51:14 +02:00
Fabiano Fidêncio
5df9cadc63 Merge pull request #4769 from arronwy/integrate_pause
CC | Integrate pause image inside rootfs
2022-08-16 16:46:53 +02:00
Fabiano Fidêncio
59566c0f69 Merge pull request #4927 from fidencio/topic/fix-tdvf-configuration-for-qemu-tdx
CC | runtime: Fix TDVF configuration with QEMU TDX
2022-08-16 15:56:14 +02:00
Fabiano Fidêncio
dc2e8cd317 runtime: Fix TDVF configuration with QEMU TDX
Instead of setting:
```
firmware = "/path/to/OVMF.fd"
firmware_volume = "/path/to/OVMF_VARS.fd"
```

We should either be setting:
```
firmware = "/path/to/OVMF.fd"
```
Or:

```
firmware = "/path/to/OVMF_CODE.fd"
firmware_volume = "/path/to/OVMF_VARS.fd"
```

I'm taking the approach to setting up the latter, as that's what's been
tested as part of our TDX CI.

Fixes: #4926

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-16 11:36:28 +02:00
Arron Wang
edf3cba463 CCv0: Add cryptsetup support in Guest kernel and rootfs
Add required kernel config for dm-crypt/dm-integrity/dm-verity
and related crypto config.

Add userspace command line tools for disk encryption support
and ext4 file system utilities.

Fixes: #4761

Signed-off-by: Arron Wang <arron.wang@intel.com>
2022-08-15 10:55:41 +08:00
Wang, Arron
75b9f3fa3c osbuilder: Integrate pause image inside rootfs
For CoCo stack, the pause image is managed by host side,
then it may configure a malicious pause image, we need package
a pause image inside the rootfs and don't the pause image from host.

Fixes: #4768

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2022-08-15 10:38:17 +08:00
Fabiano Fidêncio
72691ed6af Merge pull request #4866 from Megan-Wright/CCv0
CCv0: Merge main into CCv0 branch
2022-08-11 15:04:12 +02:00
Megan Wright
d08bb20e98 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4864
Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-08-11 11:16:02 +01:00
Steve Horsman
81e065f7bd Merge pull request #4868 from stevenhorsman/bump-image-rs
agent: Update pinned verison of image-rs
2022-08-11 10:52:50 +01:00
stevenhorsman
668672643f agent: Update pinned verison of image-rs
Pick up a new verison of image-rs as the pinned version depended on a
version of ocicrypt-rs that doesn't build anymore

Fixes: #4867
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-08-11 09:03:51 +01:00
Fabiano Fidêncio
df7529ee18 Merge pull request #4270 from confidential-containers-demo/sev_attestation_pr
runtime: Add support for SEV pre-attestation
2022-08-11 09:30:26 +02:00
Jim Cadden
a87698fe56 runtime: Add support for SEV pre-attestation
AMD SEV pre-attestation is handled by the runtime before the guest is
launched. Guest VM is started paused and the runtime communicates with a
remote keybroker service (e.g., simple-kbs) to validate the attestation
measurement and to receive launch secret. Upon validation, the launch
secret is injected into guest memory and the VM is started.

Fixes: #4280
Signed-off-by: Jim Cadden <jcadden@ibm.com>
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Signed-off-by: Dov Murik <dovmurik@linux.ibm.com>
2022-08-10 14:00:14 -04:00
Fabiano Fidêncio
5ade87c16e Merge pull request #4839 from fidencio/topic/cc-add-runtime-configuration-for-qemu-plus-tdx
CC | Build and ship QEMU TDX (and all the needed artefacts)
2022-08-10 19:08:01 +02:00
Fabiano Fidêncio
bff9f90d24 packaging: Add QEMU TDX targets as part of the cc targets
Let's add the QEMU TDX targets to be generated together with the cc
targets, when calling `make cc-tarball`.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-08 10:02:54 +02:00
Fabiano Fidêncio
0b34a8a186 kata-deploy-cc: Add QEMU support to be used with TDX
As the previous commit added a new runtime class to be used with TDX,
let's make sure this gets shipped and configured as part of the
kata-deploy-cc script, which is used by the Confidential Containers
Operator.

This commit also cleans up all the extra artefacts that will be
installed in order to run the QEMU TDX workloads.

Fixes: #4832

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-08 10:00:33 +02:00
Fabiano Fidêncio
89a5faef7a runtime: Add configuration file for QEMU TDX
Let's add a new configuration file for using a QEMU (and all the needed
artefacts) that are TDX capable.

This PR extends the Makefile in order to provide variables to be set
during the build time that are needed for the proper configuration of
the VMM, such as:
* Specific kernel parameters to be used with TDX
* Specific kernel features to be used when using TDX
* Artefacts path for the artefacts built to be used with TDX
  * QEMU
  * Kernel
  * TDVF

The reason we don't hack into the current QEMU configuration file is
because we want to ship both configurations, with for the non-TEE use
case and one for the TDX use case.

Fixes: #4830

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-08 09:54:48 +02:00
GabyCT
4f49423c91 Merge pull request #4811 from fidencio/topic/cc-build-and-ship-tdvf-and-td-shim
CC | packaging: Build and ship TDVF and TD-shim
2022-08-03 15:50:46 -05:00
Fabiano Fidêncio
4cf502fb20 packaging: Build and ship TD-shim
We're adding a new target for building TD-shim, a firmware used with
Cloud Hypervisor to start TDX capable VMs for CC.

Fixes: #4780

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-03 15:36:36 +02:00
Fabiano Fidêncio
9d0d5b9361 packaging: Build and ship TDVF
We're adding a new target for building a TDVF, a firmware used with QEMU
to start TDX capable VMs for CC.

Fixes: #4625

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-03 15:24:55 +02:00
Fabiano Fidêncio
04be5521d0 packaging: Reorganise TDX targets
Let's simply add the tdx targets after the CC ones.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-03 15:10:03 +02:00
Fabiano Fidêncio
554dff20c3 td-shim: Adjust final tarball location
Let's create the td-shim tarball in the directory where the script was
called from, instead of doing it in the $DESTDIR.

This aligns with the logic being used for creating / extracting the
tarball content, which is already in use by the kata-deploy local build
scripts.

Fixes: #4809

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-03 15:10:03 +02:00
Fabiano Fidêncio
9259646235 ovmf: Adjust final tarball location
Let's create the OVMF tarball in the directory where the script was
called from, instead of doing it in the $DESTDIR.

This aligns with the logic being used for creating / extracting the
tarball content, which is already in use by the kata-deploy local build
scripts.

Fixes: #4808

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-08-03 15:10:02 +02:00
Georgina Kinge
91cae52fe3 Merge pull request #4801 from GeorginaKin/CCv0
CCv0: Merge main into CCv0 branch
2022-08-03 11:58:04 +01:00
Georgina Kinge
a924faeead CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4800
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-08-03 09:44:12 +01:00
Jim Cadden
a51164f314 runtime: Add sev utility package
Package used to compute SEV attestation launch measurement

Fixes: #4280
Signed-off-by: Jim Cadden <jcadden@ibm.com>
2022-08-02 15:18:31 -04:00
Jim Cadden
e672401f7f runtime: Add support for simple-kbs protocol
Simple-kbs keybroker protocol is used by runtime for SEV(-ES)
pre-attestation. Includes protobuf module.

Fixes: #4280
Signed-off-by: Jim Cadden <jcadden@ibm.com>
2022-08-02 15:18:31 -04:00
Fabiano Fidêncio
a19321f5ad Merge pull request #4751 from Megan-Wright/CCv0
CCv0: Merge main into CCv0 branch
2022-08-02 19:24:02 +02:00
Megan Wright
7f71cdc290 Fix clippy errors 2022-08-02 10:26:18 +01:00
Megan Wright
c13380ba69 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4750
Depends-on: github.com/kata-containers/tests#4971
Signed-off-by: Megan Wright megan.wright@ibm.com
2022-08-01 16:29:50 +01:00
Georgina Kinge
6a77af527f Merge pull request #4697 from GeorginaKin/CCv0
CCv0: Merge main into CCv0 branch
2022-07-21 11:59:23 +01:00
GabyCT
401e69eab0 Merge pull request #4682 from stevenhorsman/CCv0-remove-skopeo-doc
doc: Remove skopeo requirement from doc
2022-07-20 14:05:28 -05:00
Georgina Kinge
c8d783e5ef CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4696
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-07-20 09:39:59 +01:00
stevenhorsman
69b1a072f2 doc: Remove skopeo requirement from doc
We don't need skopeo to get the encrypted container image
scenario working, so remove that instruction from the doc

Fixes: #4587
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-07-18 12:54:50 +01:00
Fabiano Fidêncio
da7ba2ef71 Merge pull request #4660 from fidencio/topic/cc-build-and-ship-tdx-clh
CC | packaging: Build and ship Cloud Hypervisor
2022-07-14 12:24:05 +02:00
Fabiano Fidêncio
8488d02c23 packaging: Build and ship Cloud Hypervisor
We're adding a new target for building a TDX capable Cloud Hypervisor
for CC.

As the current version of Cloud Hypervisor is already built with TDX
support, we just rely on calling the same `install_cc_clh()` function,
as done for the non-tee `cc` target.

Fixes: #4659

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-07-14 10:13:09 +02:00
Fabiano Fidêncio
64a9363925 Merge pull request #4652 from Megan-Wright/CCv0
CCv0: Merge main into CCv0 branch
2022-07-13 17:46:46 +02:00
Megan Wright
f4979a9aa5 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4651
Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-07-13 14:32:08 +01:00
Fabiano Fidêncio
1485634e28 Merge pull request #4640 from fidencio/topic/cc-build-and-ship-tdx-qemu
cc | packaging: Allow building a TDX capable QEMU
2022-07-13 15:03:59 +02:00
Fabiano Fidêncio
be165c40f9 packaging: Allow building a TDX capable QEMU
We're adding a new target for building a TDX capable QEMU for CC.
This commit, differently than b307531c29,
introduces support for building the artefacts that are TEE specific.

Fixes: #4623

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-07-12 21:41:19 +02:00
Fabiano Fidêncio
6d9d8e0660 packaging: Add a "-" in the dir name if $BUILD_DIR is available
Currently $BUILD_DIR will be used to create a directory as:
/opt/kata/share/kata-qemu${BUILD_DIR}

It means that when passing a BUILD_DIR, like "foo", a name would be
built like /opt/kata/share/kata-qemufoo
We should, instead, be building it as /opt/kata/share/kata-qemu-foo.

Fixes: #4638

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-07-12 21:41:16 +02:00
Fabiano Fidêncio
d2f17ee55a packaging: Use the $BUILD_SUFFIX when renaming the qemu binary
Instead of always naming the binary as "-experimental", let's take
advantage of the $BUILD_SUFFIX that's already passed and correctly name
the binary according to it.

Fixes: #4638

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-07-12 15:12:21 +02:00
Fabiano Fidêncio
6f79928df7 Merge pull request #4631 from fidencio/topic/cc-build-kernel-with-tdx-support
CC | packaging: Allow building a TDX capable kernel
2022-07-12 12:08:15 +02:00
Fabiano Fidêncio
516ed240f4 packaging: Allow building a TDX capable kernel
We're adding a new target for building a TDX capable kernel for CC.
This commit, differently than c4cc16efcd,
introduces support for building the artefacts that are TEE specific.

Fixes: #4622

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-07-11 14:35:06 +02:00
Fabiano Fidêncio
423162d2aa kernel: Allow passing the URL to download the tarball
Passing the URL to be used to download the kernel tarball is useful in
various scenarios, mainly when doing a downstream build, thus let's add
this new option.

This new option also works around a known issue of the Dockerfile used
to build the kernel not having `yq` installed.

Fixes: #4629

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-07-11 14:30:39 +02:00
Fabiano Fidêncio
965c0b1ad2 kernel: Deduplicate code used for building TEE kernels
There's no need to have the entire function for building SEV / TDX
duplicated.

Let's remove those functions and create a `get_tee_kernel()` which takes
the TEE as the argument.

Fixes: #4627

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-07-11 14:30:39 +02:00
Fabiano Fidêncio
e8902bb373 Merge pull request #4619 from fidencio/topic/cc-introduce-kata-deploy-cc
CC | Introduce kata-deploy-cc
2022-07-08 15:06:42 +02:00
Fabiano Fidêncio
df0cc78e57 kata-deploy-cc: Introduce the new kata-deploy for CC
Although I don't like the duplication introduced here, it's (at least
for now) way cleaner to have a specific daemonset for the Confidential
Containers effort.

As soon as we have all the bits and pieces upstreamed (kernel, QEMU, and
specific dependencies for each one of the TEEs), we'll be easily able to
get rid of this one.  However, for now, focusing on this different set
of files will make our lives easier.

This new daemonset includes the configurations needed for containerd in
order to use the `cc` specific `cri_handler`, which is not and will not
be upstream on the containerd side.

Note, CRI-O is **not** supported for now.

Fixes: #4620

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-07-08 11:12:26 +02:00
Fabiano Fidêncio
441399df1f kata-deploy: Get back to the original kata-deploy.sh
Instead of hacking the original `kata-deploy.sh` script, let's add a
totally new folder where we'll be adding content that's CC related.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-07-07 11:06:50 +02:00
Fabiano Fidêncio
ec20089c1b Merge pull request #4603 from GeorginaKin/CCv0
CCv0: Merge main into CCv0 branch
2022-07-07 10:24:57 +02:00
Georgina Kinge
9d524b29ad CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4602
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-07-06 14:27:15 +01:00
GabyCT
889ed4f14b Merge pull request #4570 from fidencio/topic/cc-add-a-build-target-for-unmodified-components
CC | packaging: Add a build target for all the "unmodified" components
2022-07-01 11:29:27 -05:00
Fabiano Fidêncio
097fe823e5 packaging: Show the new options of kata-deploy-binaries.sh
We've added a bunch of new options related to Confidential Containers
builds as part of the kata-deploy-binaries.sh.  Let's make sure those
are displayed to the users of the script when it's called with --help.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-30 21:10:25 +02:00
Fabiano Fidêncio
07bdf75913 packaging: Add a cc option for kata-deploy-binaries.sh
Similar to what we have with the `all` option, let's also add a `cc`
one, allowing others to easily call the script and build all the `cc`
related components.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-30 21:10:25 +02:00
Fabiano Fidêncio
1ba29c3e0c packaging: Add a cc-tarball target to build cc related tarballs
Quite similar to the `kata-tarball` target, let's add a `cc-tarball`
target so we can build all the CC related tarballs in a single command,
with all the tarballs being merged together in the end.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-30 21:10:25 +02:00
Fabiano Fidêncio
226abc4a47 packaging: Add a cc-parallel target to build cc related tarballs
Quite similar to the `all-parallel` target, let's add a `cc-parallel`
target so we can build all the CC related tarballs in parallel.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-30 21:10:25 +02:00
Fabiano Fidêncio
dc5f0c7d0c packaging: Add a cc target to build all the CC related tarballs
Quite similar to the `all` target, let's add a `cc` target so we can
build all the CC related tarballs.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-30 21:10:25 +02:00
Fabiano Fidêncio
186cec6889 packaging: Allow building virtiofsd for CC
We're adding a new target for building virtiofsd for CC, but it's
important to note that the only difference between this one and the
"vanilla" build is the installation path.

Moreover, virtiofsd will **NOT** be used by the CC effort, but as the
very first release target doesn't include TEE support, let's not force
those who want to give it a try to setup devicemapper.

Fixes: #4569

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-30 21:10:14 +02:00
Fabiano Fidêncio
b307531c29 packaging: Allow building QEMU for CC
We're adding a new target for building QEMU for CC, but it's important
to note that the only difference between this one and the "vanilla"
build is the installation path.

The reason we're taking this approach is because the first release
target for CC doesn't include TEE support.

We had to also include a new builder for QEMU, a specific one for CC, as
for now that's the easiest way to override the prefix in a way that
we'll be easily able to expand the script to support TEE capable builds
in the very near future.

Fixes: #4568

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-30 21:10:08 +02:00
Fabiano Fidêncio
c4cc16efcd packaging: Allow building the Kernel for CC
We're adding a new target for building the Kernel for CC, but it's
important to note that the only difference between this one and the
"vanilla" build is the installation path.

The reason we're taking this approach is because the first release
target for CC doesn't include TEE support.

Fixes: #4567

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-30 21:09:55 +02:00
Fabiano Fidêncio
caabd54b6e packaging: Allow building Cloud Hypervisor for CC
We're adding a new target for building Cloud Hypervisor for CC, but it's
important to note that the only difference between this one and the
"vanilla" build is the installation path.

The reasons we're taking this approach are:
* Cloud Hypervisor, for the `main` and `stable` branches, is already
  built with TDX support.
* The first target for the CC release doesn't include TEE support.

Fixes: #4566

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-30 21:09:35 +02:00
Fabiano Fidêncio
8f6eca517a Merge pull request #4565 from fidencio/topic/cc-build-the-shim-with-the-needed-configurations
CC | Build the shim-v2 with the needed configurations for Confidential Containers usage
2022-06-30 20:42:43 +02:00
Fabiano Fidêncio
df486533fa packaging: Allow building a CC shim-v2 capable runtime
Let's add a new build target for our local-build scripts, cc-shim-v2,
and use it to build Kata Containers properly configured for the CC
use-case.

Fixes: #4564
Depends-on: github.com/kata-containers/tests#4895

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-30 13:21:21 +02:00
Fabiano Fidêncio
7ceeeba9a2 static-build: Allow removing shipped shim-v2 configs
Let's add a new "REMOVE_VMM_CONFIGS" environment variable that can be
passsed to the script responsible for building Kata Containers.

Right now this is not useful for the `main` or `stable` branch, but for
the CC release we only have been working and testing with QEMU and Cloud
Hypervisor.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-30 13:06:43 +02:00
Fabiano Fidêncio
d4d178359b runtime: Expose DEFSERVICEOFFLOAD build option
For the CC build we need to enable such a flag, and the cleaner way to
do so is exposing it in the Makefile and, later on, making sure its
correct value to the build script.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-30 12:21:10 +02:00
Fabiano Fidêncio
e23322b95c packaging: Allow passing EXTRA_OPTS to the shim-v2 build
While this has never been needed for the `main` and `stable` releases,
for the coming CC release we need to pass a few extra options when
building the shim.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-30 12:16:44 +02:00
GabyCT
3a655c4198 Merge pull request #4558 from fidencio/topic/cc-build-rootfs-image-with-the-needed-tools
CC | Build the rootfs image with skopeo, umoci, and using an offline_fs_kbc
2022-06-29 17:05:55 -05:00
GabyCT
995c14d429 Merge pull request #4552 from fidencio/topic/cc-always-use-ubuntu-as-rootfs
CC | versions: Always use ubuntu as rootfs for CC
2022-06-29 17:04:11 -05:00
Fabiano Fidêncio
a438d6114b packaging: Allow building a CC capable image
Let's add a new build target for our local-build scripts,
cc-rootfs-image-tarball, and use it to build an image that has skopeo
and umoci embedded in, and that using the offline_fs_kbc as the
attenstation agent KBC.

Fixes: #4557

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-29 21:09:33 +02:00
Fabiano Fidêncio
335ddd5876 packaging: Do not unset env vars needed for CC
SKOPEO, UMOCI, and AA_KBC have been unset so far as we have not been
generating rootfs images that would be used for CC as part of our
workflow.

Now, as we're targetting the first release of the operator with the CCv0
branch, let's stop unsetting those and start taking advantage of our
tools to help us building a CC capable image.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-29 19:05:11 +02:00
Georgina Kinge
6c2b9f67d7 Merge pull request #4556 from GeorginaKin/CCv0
CCv0: Merge main into CCv0 branch
2022-06-29 15:59:56 +01:00
Georgina Kinge
eb9836ff8e runtime: add image import back in
Putting image import back in after removal during merge conflict

Fixes: #4555
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-06-29 14:13:18 +01:00
Georgina Kinge
bda68b16f1 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4555
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-06-29 13:22:22 +01:00
Fabiano Fidêncio
03170c2651 versions: Always use ubuntu as rootfs for CC
As we're still depending on components that are only being tested on
Ubuntu, let's make sure the VM image distributed is exactly the same
we've been testing.

Fixes: #4551

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-29 11:25:23 +02:00
Fabiano Fidêncio
38b61bb743 Merge pull request #4537 from GeorginaKin/CCv0
CCv0: Refactor ccv0.sh to remove duplicate code
2022-06-29 08:17:35 +02:00
Georgina Kinge
74a748f36e CCv0: Refactor ccv0.sh to remove duplicate code
Refactored ccv0.sh to remove rootfs code now in lib.sh

Fixes: #4512

Co-authored-by: Megan Wright Megan.Wright@ibm.com
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-06-28 14:09:23 +01:00
Steve Horsman
c6a5814a91 Merge pull request #4490 from Megan-Wright/CCv0-containerd-1.6.6-rebase
CCv0: Update CC containerd version
2022-06-27 15:59:09 +01:00
stevenhorsman
761786324e CCv0: Increase timeout
Increase crictl timout to stop pod creation failing on v1.24.1

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-06-27 10:19:05 +01:00
Megan Wright
d6924182f3 vendor: Update CC containerd version
Re-vendor to pickup confidential-containers containerd 1.6.6 rebase
Fixes: #4489

Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-06-27 10:19:05 +01:00
Megan Wright
94a6edcfa3 versions: Update containerd version
Update kata-containers version to reflect 1.6.6 rebase of
containerd
Fixes: #4489

Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-06-27 10:14:32 +01:00
Fabiano Fidêncio
e52d6b1d0b Merge pull request #4529 from GeorginaKin/CCv0
CCv0: Refactor ccv0.sh to use bats test
2022-06-27 09:39:27 +02:00
Fabiano Fidêncio
54f47cceaa Merge pull request #4518 from fidencio/topic/pin-a-specific-version-of-image-rs
CCv0 | agent: Pin a specific version of image-rs
2022-06-27 07:51:44 +02:00
Steve Horsman
ebacd986bb Merge pull request #4532 from stevenhorsman/CCv0-PS1-unbound
CCv0: Fix PS1 unbound error
2022-06-25 18:08:24 +01:00
stevenhorsman
305532db02 CCv0: Fix PS1 unbound error
export PS1, so it is bound

Fixes: #4531
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-06-24 17:27:43 +01:00
Georgina Kinge
20ef9e9f5c CCv0: Refactor ccv0.sh to use bats test
Refactored ccv0.sh to utilise new automated tests for pulling encrypted images and creating a pod.

Fixes: #4512
Depends-on: github.com/kata-containers/tests#4866

Co-authored-by: Megan Wright <Megan.Wright@ibm.com>
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-06-24 15:43:15 +01:00
Fabiano Fidêncio
40b1c79c97 agent: Pin a specific version of image-rs
Let's pin a specific version of image-rs, one that pins a specific
version of ocicrypt-rs on their side, and ensure we don't fall into
issues by consuming the content from main on those repos, and also
helping to ensure reproducible builds from our side.

Fixes: #4517

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-24 14:01:53 +02:00
Fabiano Fidêncio
62864b5041 agent: Update Cargo.lock
Let's update the Cargo.lock file to bring in all the new dependencies
and to decrease the diff after pinning a specific version of image-rs.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-24 14:01:09 +02:00
Fabiano Fidêncio
404515f568 image_rpc: Fix "single-char-pattern" clippy warning
```
error: single-character string constant used as pattern
   --> src/image_rpc.rs:199:36
    |
199 |                 cid = v[0].replace(":", "_");
    |                                    ^^^ help: try using a `char` instead: `':'`
    |
    = note: `-D clippy::single-char-pattern` implied by `-D warnings`
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#single_char_pattern
```

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-24 14:00:17 +02:00
Fabiano Fidêncio
d21c3c340d random: Fix "nonminimal-bool" clippy warning
The error shown below was caught during a dependency bump in the CCv0
branch, but we better fix it here first.
```
error: this boolean expression can be simplified
  --> src/random.rs:85:21
   |
85 |             assert!(!ret.is_ok());
   |                     ^^^^^^^^^^^^ help: try: `ret.is_err()`
   |
   = note: `-D clippy::nonminimal-bool` implied by `-D warnings`
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#nonminimal_bool

error: this boolean expression can be simplified
  --> src/random.rs:93:17
   |
93 |         assert!(!ret.is_ok());
   |                 ^^^^^^^^^^^^ help: try: `ret.is_err()`
   |
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#nonminimal_bool
```

Fixes: #4523

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-24 13:50:13 +02:00
Fabiano Fidêncio
386af028be netlink: Fix "or-fun-call" clippy warnings
The error shown below was caught during a dependency bump in the CCv0
branch, but we better fix it here first.
```
error: use of `ok_or` followed by a function call
   --> src/netlink.rs:526:14
    |
526 |             .ok_or(anyhow!(nix::Error::EINVAL))?;
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try this: `ok_or_else(|| anyhow!(nix::Error::EINVAL))`
    |
    = note: `-D clippy::or-fun-call` implied by `-D warnings`
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#or_fun_call
error: use of `ok_or` followed by a function call
   --> src/netlink.rs:615:49
    |
615 |         let v = u8::from_str_radix(split.next().ok_or(anyhow!(nix::Error::EINVAL))?, 16)?;
    |                                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try this: `ok_or_else(|| anyhow!(nix::Error::EINVAL))`
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#or_fun_call
```

Fixes: #4523

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-24 13:50:13 +02:00
Steve Horsman
0782f4a43b Merge pull request #4514 from stevenhorsman/CCv0-update-crictl-pod-config
doc: Update crictl pod-config
2022-06-24 08:38:39 +01:00
Georgina Kinge
676b1d6048 Merge pull request #4508 from GeorginaKin/ccv0-main-merge
CCv0: Merge main into CCv0 branch
2022-06-22 19:28:53 +01:00
stevenhorsman
2e5c4a9245 CCv0: Update script to use new lib method
- Update `ccv0.sh` to use the new lib method which updates the CC pod config yaml
to add a a unique id
for compatibility with crictl 1.24.0+

Fixes: #4867
Depends-on: github.com/kata-containers/tests#4867
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-06-22 16:48:55 +01:00
stevenhorsman
8f8c2215f4 doc: Update crictl pod-config
- Ensure that our documented crictl pod config file contents have
uid  and namespace fields for compatibility with crictl 1.24+

Fixes: #4513
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-06-22 16:48:55 +01:00
Georgina Kinge
4f80ea1962 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4507
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-06-22 10:06:27 +01:00
Georgina Kinge
cfa3e1e933 Merge pull request #4354 from GeorginaKin/CCv0
CCv0: Refactor signature verification PoC changes
2022-06-20 13:01:07 +01:00
Georgina Kinge
a1e16ff6e0 CCv0: Refactor signature verification PoC changes
Removes signature verficiation files and related code

Fixes: #4111

Signed-off-by: Georgina Kinge <Georgina.Kinge@ibm.com>
Co-authored-by: Megan Wright <megan.wright@ibm.com>
2022-06-20 10:58:36 +01:00
Steve Horsman
dc92e134aa Merge pull request #4466 from stevenhorsman/rootfs-aa-build
osbuilder: Fix attestation-agent build
2022-06-16 17:23:05 +01:00
stevenhorsman
4af8f0a999 doc: Update encrypted image doc
Encrypted image support with offline_fs_kbc mode
of the attesation-agent, currently required skopeo
so update the doc to clarify this

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-06-16 14:53:10 +01:00
stevenhorsman
9c73babdb2 osbuilder: Fix attestation-agent build
Fix of the attestation-agent build and install issue in rootfs.sh

Fixes: #4465
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-06-16 14:24:09 +01:00
Fabiano Fidêncio
0b065444fc Merge pull request #4461 from Megan-Wright/CCv0
CCv0: Merge main into CCv0 branch
2022-06-16 13:39:48 +02:00
Megan Wright
245fa7caf5 vendor: Revert containerd version
Switch containerd to confidential containers fork
Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-06-16 10:56:51 +01:00
Megan Wright
eeff63375f CCv0: Merge main into CCv0 branch
Merge in snap fix

Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-06-16 10:55:42 +01:00
Megan Wright
94695869b0 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4460
Signed-off-by: Megan-Wright <megan.wright@ibm.com>
2022-06-15 11:05:51 +01:00
Steve Horsman
d3fe110765 Merge pull request #4426 from Megan-Wright/CCv0
CCv0: Merge main into CCv0 branch
2022-06-13 14:20:04 +01:00
Megan Wright
2300521c1b CCv0: Compatibility update for merge
Added ImageClient support to agent_cmd_sandbox_set_ip_tables and agent_cmd_sandbox_get_ip_tables

Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-06-08 16:36:24 +01:00
Megan Wright
aa9d875a8d CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4424
Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-06-08 15:51:18 +01:00
Fabiano Fidêncio
6e399dcb61 Merge pull request #4348 from GeorginaKin/CCv0-main-merge
CCv0: Merge main into CCv0 branch
2022-06-01 22:21:10 +02:00
Steve Horsman
be223b1db5 Merge pull request #4329 from stevenhorsman/ccv0.sh-non-root-refactor
CCv0: Fix permissions of kata-containers repo
2022-05-31 13:53:31 +01:00
Georgina Kinge
7eb74e51be CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4345
Signed-off-by: Georgina Kinge <Georgina.Kinge@ibm.com>
2022-05-31 13:50:38 +01:00
stevenhorsman
b772cc6b45 CCv0: Fix permissions of kata-containers repo
- Solve `fatal: unsafe repository` ownership error by using `lib.sh`
code to check out the kata-containers repo
- Update `~/rustup` and repo directory ownership to `${USER}`
in order to allow subsequent build steps to work as a non-root
user

Fixes: #4241
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-05-30 16:53:31 +01:00
Tim Zhang
b08ea1fd99 agent: remove bin oci-kata-agent
Fixes: #4291

Signed-off-by: Tim Zhang <tim@hyper.sh>
2022-05-30 16:02:29 +02:00
Tim Zhang
5eb109c6da runk: merge oci-kata-agent into runk
Merge two bins into one.

Fixes: #4291

Signed-off-by: Tim Zhang <tim@hyper.sh>
2022-05-30 16:02:29 +02:00
Jianyong Wu
92a7b2f5f0 qemu: remove virtiofsd option in qemu config
As virtiofsd will be built base on rust, "virtiofsd" option is no longer
needed in qemu.

Fixes: #4258
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2022-05-30 16:02:29 +02:00
Jianyong Wu
6a51c6615a virtiofsd: build rust based virtiofsd from source for non-x86_64
Based on @fidencio's opoinon,
On Arm: static build virtiofsd using musl lib;
on ppc64 & s390: static build virtiofsd using gnu lib;

Fixes: #4258
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2022-05-30 16:02:29 +02:00
Yibo Zhuang
d9e7966714 agent: fix direct-assigned volume stats
The current implementation of walking the
disks to match with the requested volume path
in agent doesn't work because the volume path
provided by the shim to the agent is the mount
path within the guest and not the device name.
The current logic is trying to match the
device name to the volume path which will never
match.

This change will simplify the
get_volume_capacity_stats and
get_volume_inode_stats to just call statfs and
get the bytes and inodes usage of the volume
path directly.

Fixes: #4297

Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
2022-05-30 16:02:29 +02:00
Yibo Zhuang
e708ef3c7d runtime: direct-volume stats use correct name
Today the shim does a translation when doing
direct-volume stats where it takes the source and
returns the mount path within the guest.

The source for a direct-assigned volume is actually
the device path on the host and not the publish
volume path.

This change will perform a lookup of the mount info
during direct-volume stats to ensure that the
device path is provided to the shim for querying
the volume stats.

Fixes: #4297

Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
2022-05-30 16:02:29 +02:00
Yibo Zhuang
44c6d5bcea runtime: direct-volume stats update to use GET parameter
The go default http mux AFAIK doesn’t support pattern
routing so right now client is padding the url
for direct-volume stats with a subpath of the volume
path and this will always result in 404 not found returned
by the shim.

This change will update the shim to take the volume
path as a GET query parameter instead of a subpath.
If the parameter is missing or empty, then return
400 BadRequest to the client.

Fixes: #4297

Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
2022-05-30 16:02:29 +02:00
Yibo Zhuang
e68cb28129 runtime: fix incorrect Action function for direct-volume stats
The action function expects a function that returns error
but the current direct-volume stats Action returns
(string, error) which is invalid.

This change fixes the format and print out the stats from
the command instead.

Fixes: #4293

Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
2022-05-30 16:02:29 +02:00
Rafael Fonseca
322c6dab66 runtime: sync docstrings with function names
The functions were renamed but their docstrings were not.

Fixes #4006

Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
2022-05-30 16:02:29 +02:00
Rafael Fonseca
4d5e446643 runtime: remove duplicate 'types' import
Fallout of 09f7962ff

Fixes #4285

Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
2022-05-30 16:02:29 +02:00
Snir Sheriber
7040b297c5 docs: fix annotations example
annotation value should always be quoted, regardless to its type

Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2022-05-30 16:02:29 +02:00
Snir Sheriber
a48d13f68d runtime: allow annotation configuration to use_legacy_serial
and update the docs and test

Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2022-05-30 16:02:29 +02:00
Gabriela Cervantes
756a07537c docs: Remove clear containers reference in README
This PR removes the clear containers reference as this is not longer
being used and is deprecated at the rootfs builder README.

Fixes #4278

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-05-30 16:02:29 +02:00
Snir Sheriber
060fed814c qemu: allow using legacy serial device for the console
This allows to get guest early boot logs which are usually
missed when virtconsole is used.
- It utilizes previous work on the govmm side:
https://github.com/kata-containers/govmm/pull/203
- unit test added

Fixes: #4237
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2022-05-30 16:02:29 +02:00
Snir Sheriber
5453128159 qemu: treat console kernel params within appendConsole
as it is tightly coupled with the appended console device
additionally have it tested

Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2022-05-30 16:02:29 +02:00
Zvonko Kaiser
79a060ac68 runtime: Adding the correct detection of mediated PCIe devices
Fixes #4212

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2022-05-30 16:02:29 +02:00
Steve Horsman
c84be3c6cd Merge pull request #4287 from stevenhorsman/CCv0-virtiofsd-fix
CCv0: Fix ccv0.sh to install virtiofsd
2022-05-20 09:13:29 +01:00
stevenhorsman
69122d2a05 CCv0: Fix ccv0.sh to install virtiofsd
- Add call to install_virtiofsd.sh
- Remove the qemu build in the cloud_hypervisor path

Fixes: #4286
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-05-19 18:22:16 +01:00
Steve Horsman
01c878e293 Merge pull request #4277 from GeorginaKin/CCv0
CCv0: Merge main into CCv0 branch
2022-05-19 08:53:12 +01:00
Georgina Kinge
dd78e4915c CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4275
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-05-18 11:19:22 +01:00
Steve Horsman
1cda87bea7 Merge pull request #4221 from GeorginaKin/CCv0
CCv0: Refactored ccv0.sh to use new shared library
2022-05-16 09:22:13 +01:00
Georgina Kinge
d50f98b603 CCv0: Refactored ccv0.sh to use new shared library
Refactoring ccv0.sh to use integration/confidential/lib.sh

Fixes: #4132
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
Co-authored-by: Megan Wright <Megan.Wright@ibm.com>
2022-05-13 14:31:57 +01:00
Steve Horsman
e64c2244f2 Merge pull request #4242 from stevenhorsman/git-errors-ccv0-non-root
doc: Allow ccv0 to run as non-root user
2022-05-13 13:33:53 +01:00
Steve Horsman
e318023ed9 Merge pull request #4236 from GeorginaKin/CCv0-main-merge
CCv0: Merge main into CCv0 branch
2022-05-13 09:08:22 +01:00
stevenhorsman
b89af0b373 doc: Allow ccv0 to run as non-root user
Change all the tests repo interactions to be done as sudo

Fixes: #4241
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-05-12 18:20:55 +01:00
Georgina Kinge
cc560cb85a CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4235
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-05-12 17:09:34 +01:00
Georgina Kinge
a259b1360d CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4235
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-05-12 09:45:01 +01:00
Fabiano Fidêncio
6cd4497b66 Merge pull request #4201 from Megan-Wright/CCv0
CCv0: Merge main into CCv0 branch
2022-05-04 17:45:27 +02:00
Megan Wright
ef1ae5bc93 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4200
Signed-off-by: Megan Wright <megan.wright@.ibm.com>
2022-05-04 11:26:50 +01:00
Steve Horsman
9b27329281 Merge pull request #4158 from GeorginaKin/CCv0-main-merge
CCv0: Merge main into CCv0 branch
2022-05-03 16:40:04 +01:00
Georgina Kinge
67015ac1d7 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4157
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-04-27 10:39:08 +01:00
Fabiano Fidêncio
2d67b1ee1d Merge pull request #4116 from Megan-Wright/CCv0
CCv0: Merge main into CCv0 branch
2022-04-20 14:05:51 +02:00
Megan Wright
738ae8c60e CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4115
Signed-off-by: Megan-Wright <megan.wright.ibm.com>
2022-04-20 11:32:31 +01:00
Steve Horsman
e231501558 Merge pull request #4092 from Megan-Wright/CCv0
CCv0: Merge main into CCv0 branch
2022-04-19 17:43:39 +01:00
Megan Wright
a36e9ba87f CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4090
Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-04-13 09:54:32 +01:00
Steve Horsman
e4bae434de Merge pull request #4049 from GeorginaKin/CCv0-main-merge
CCv0: Merge main into CCv0 branch
2022-04-07 17:54:25 +01:00
Georgina Kinge
8add48d759 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #4047
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2022-04-07 10:58:17 +01:00
Fabiano Fidêncio
1f22f9ca38 Merge pull request #3926 from Jakob-Naucke/cc-cross
CCv0 | Multistrap & enable cross-building guest
2022-04-06 20:16:21 +02:00
Fabiano Fidêncio
c95dd8f57e Merge pull request #4039 from fidencio/wip/ccv0.sh-also-support-cloud-hypervisor
CCv0 | ccv0.sh: Expand to also using Cloud Hypervisor
2022-04-06 18:06:54 +02:00
Fabiano Fidêncio
bdb0f6b471 how-to,ccv0: Reword the full build sentence
Let's reword the sentence so it's easier for someone who's not a native
nor familiar with the project to understand.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-04-06 14:56:55 +02:00
Fabiano Fidêncio
c5b39c5686 ccv0.sh: Expand to also using Cloud Hypervisor
Right now the script only support QEMU, but there's not a reason to do
that, mainly considering we already have the tests parity in the CIs
between QEMU and Clouud Hypervisor.

With this in mind, let's expand this script to also using Cloud
Hypervisor.

Whether this script should use QEMU or Cloud Hypervisor is defined
according to the KATA_HYPERVISOR environment variable.

Fixes: #4038

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-04-06 14:56:55 +02:00
Fabiano Fidêncio
5fdbdaafd3 ccv0: Don't use the QEMU process to get the sandbox ID
Instead, rely on the conntainerd-shim-kata-v2 process, as this makes
this script VMM agnostic.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-04-05 19:39:17 +02:00
Fabiano Fidêncio
8fe5b97c2b Merge pull request #4026 from fidencio/wip/ccv0-expose-service-offload-option-to-clh
CCv0 | clh: Expose service offload configuration
2022-04-01 15:21:39 +02:00
Fabiano Fidêncio
1e78f5e66a clh: Expose service offload configuration
This configuration option is valid for all the hypervisor that are going
to be used with the confidential containers effort, thus exposing the
configuration option for Cloud Hypervisor as well.

Fixes: #4022

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 98750d792b)
2022-04-01 14:15:09 +02:00
Fabiano Fidêncio
252044613d Merge pull request #3957 from arronwy/support_https_proxy
CCv0: agent: Support https_proxy config for image download in guest
2022-04-01 11:43:50 +02:00
Steve Horsman
51383243b7 Merge pull request #4000 from stevenhorsman/sig-ver-updates
CCv0: Refactor image verification doc and tests
2022-04-01 08:23:42 +01:00
stevenhorsman
4e7b6306b4 CCv0: Refactor image verification doc and tests
Refactor image verification documentation to be more user
focussed, using crictl rather than agent-ctl and re-using the
integration test config files

Fixes: #3958
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-03-30 11:30:43 +01:00
Christophe de Dinechin
fd20824a00 Merge pull request #3945 from stevenhorsman/update-logging-doc
docs: Update to reflect logging changes
2022-03-30 11:50:51 +02:00
Jakob Naucke
8fbf6c4e14 osbuilder: Multistrap Ubuntu
Use `multistrap` for building Ubuntu rootfs. Adds support for building
for foreign architectures using the `ARCH` environment variable
(including umoci).
In the process, the Ubuntu rootfs workflow is vastly simplified.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2022-03-28 16:13:00 +02:00
Arron Wang
35360d4ad6 agent: Support https_proxy/no_proxy config for image download in guest
Containerd can support set a proxy when downloading images with a environment variable.
For CC stack, image download is offload to the kata agent, we need support similar feature.
Current we add https_proxy and no_proxy, http_proxy is added since it is insecure.

Fixes #3956

Signed-off-by: Arron Wang <arron.wang@intel.com>
2022-03-28 16:38:09 +08:00
Jakob Naucke
578678e051 packaging: Enable cross-building agent
Requires setting ARCH and CC.

- Add CC linker option for building agent.
- Set host for building libseccomp.

Fixes: #3681
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2022-03-25 17:57:47 +01:00
Jakob Naucke
527d741c07 osbuilder: Fix use of LIBC in rootfs.sh
- Add a doc comment
- Pass to build container, e.g. to build x86_64 with glibc (would
  always use musl)

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2022-03-25 17:57:47 +01:00
Jakob Naucke
e167237b13 osbuilder: Simplify Rust installation
no double export, direct target

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2022-03-25 17:57:47 +01:00
Jakob Naucke
77ea087ae7 osbuilder: Remove musl installations
Remove a lot of cruft of musl installations -- we needed those for the
Go agent, but Rustup just takes care of everything. aarch64 on
Debian-based & Alpine is an exception -- create a symlink
`aarch64-linux-musl-gcc` to `musl-tools`'s `musl-gcc` or `gcc` on
Alpine. This is unified -- arch-specific Dockerfiles are removed.

Furthermore, we should keep it in Ubuntu for supporting the offline SEV
KBC. We also keep it in Clear Linux, as that runs our internal checks,
but it is e.g. not shipped in CentOS Stream 9.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2022-03-25 17:57:47 +01:00
Jakob Naucke
207e325a0d osbuilder: apk add --no-cache
Hadolint DL3019. If you're wondering why this is in this PR, that's
because I touch the file later, and we're only triggering the lints for
changed files.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2022-03-25 17:57:45 +01:00
stevenhorsman
6d5a329535 docs: Update to reflect logging changes
- We've updated the CC logging scripts to log to the journal
rather than a socket, so remove socat scripts and instructions
to reflect this

Fixes: #3928
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-03-25 11:33:49 +00:00
James O. D. Hunt
da10350d9b Merge pull request #3921 from Jakob-Naucke/agent-config-dir
CCv0 | github: Use `/etc/kata-containers/agent.toml`
2022-03-25 09:33:37 +00:00
Fabiano Fidêncio
480c4d9716 Merge pull request #3948 from Megan-Wright/CCv0
CCv0: Merge main into CCv0 branch
2022-03-24 09:45:11 +01:00
Megan Wright
ef8ba4bbec CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #3931
Signed-off-by: Megan Wright megan.wright@ibm.com
2022-03-23 17:01:38 +00:00
Steve Horsman
fa3aced1ac Merge pull request #3899 from Megan-Wright/CCv0
CCv0: Refactor CCv0.sh to use integration test lib.sh
2022-03-22 15:47:44 +00:00
Jakob Naucke
3b7955a02d github: Use /etc/kata-containers/agent.toml
for config, as per suggestion from @jodh-intel in #3243.
- Uses the pre-established `kata-containers` folder which we can also
  use for more
- Makes it clear the agent is used

Also, use curl instead of wget for uniformity.

Fixes: #3920
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2022-03-21 12:06:31 +01:00
Megan Wright
ead111abf7 CCv0: Refactor CCv0.sh to use integration test lib.sh
Updates ccv0.sh to remove duplicated code
Fixes: #3898
Signed-off-by: Megan Wright <megan.wright@ibm.com>
Signed-off-by: Georgina Kinge  <georgina.kinge@ibm.com>
2022-03-18 15:31:02 +00:00
Garrett Mahin
039a15efff doc: fix filename typo
Corrects a filename typo in cleanup cluster part
of kata-deploy README.md

Fixes: #3869
Signed-off-by: Garrett Mahin <garrett.mahin@gmail.com>
2022-03-17 15:11:20 +01:00
Jack Hance
a5b72720d2 rustjail: optimization, merged several writelns into one
Optimized several writelns by merging them into one in src/utils.rs

Fixes: #3772

Signed-off-by: Jack Hance <jack.hance@ndsu.edu>
2022-03-17 15:11:20 +01:00
Bo Chen
f921688c8c versions: Upgrade to Cloud Hypervisor v22.1
This is a bug fix release. The following issues have been addressed:
1) VFIO ioctl reordering to fix MSI on AMD platforms; 2) Fix virtio-net
control queue.

Details can be found: https://github.com/cloud-hypervisor/cloud-hypervisor/releases/tag/v22.1

Fixes: #3872

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-03-17 15:11:20 +01:00
James O. D. Hunt
fc7ffe8cfc scripts: Change here document delimiters
Fix the outstanding scripts using non standard shell here document delimiters.

This should have been caught by
https://github.com/kata-containers/tests/pull/3937, but there is a bug
in the checker which is fixed on
https://github.com/kata-containers/tests/pull/4569.

Fixes: #3864.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-03-17 15:11:20 +01:00
James O. D. Hunt
ff36e6a72e manager: Change here documents to use standard delimiter
All scripts should use `EOF` as the shell here document delimiter as
this is checked by the static checker.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-03-17 15:11:20 +01:00
James O. D. Hunt
b34374d554 manager: Add options to change self test behaviour
Added new `kata-manager` options to control the self-test behaviour. By
default, after installation the manager will run a test to ensure a Kata
Containers container can be created. New options allow:

- The self test to be disabled.
- Only the self test to be run (no installation).

These features allow changes to be made to the installed system before
the self test is run.

Fixes: #3851.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-03-17 15:11:20 +01:00
James O. D. Hunt
f00724a1aa manager: Add option to enable component debug
Added a `-d` option to `kata-manager` to enable Kata Containers
and containerd debug.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-03-17 15:11:20 +01:00
James O. D. Hunt
d4041f1a62 manager: Whitespace fix
Remove additional blank line in the `kata-manager`.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-03-17 15:11:20 +01:00
James O. D. Hunt
1d47e893cb manager: Create containerd link
Make the `kata-manager` create a `containerd` link to ensure the
downloaded containerd systemd service file can find the daemon when
using the GitHub packaged version of containerd.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-03-17 15:11:20 +01:00
Braden Rayhorn
64fbf8435c agent: add tests for get_memory_info function
Add test coverage for get_memory_info function in src/rpc.rs. Includes
some minor refactoring of the function.

Fixes #3837

Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
2022-03-17 15:11:20 +01:00
James O. D. Hunt
d42bc8c76f CI: Update GHA secret name
Change the secret used by the GitHub Action  that adds the PR size
label to one with the correct set of privileges.

Fixes: #3856.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-03-17 15:11:20 +01:00
Gabriela Cervantes
307e30fd1b docs: Update contact link in runtime README
This PR updates the contact link in the runtime README document.

Fixes #3854

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-03-17 15:11:20 +01:00
Julio Montes
f54d999c3a kernel: fix cve-2022-0847
bump guest kernel version to fix cve-2022-0847 "Dirty Pipe"

fixes #3852

Signed-off-by: Julio Montes <julio.montes@intel.com>
2022-03-17 15:11:20 +01:00
James O. D. Hunt
91af844497 CI: Create GHA to add PR sizing label
Created a new GitHub Action workflow file that adds a sizing label to
each PR.

Fixes: #3841.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-03-17 15:11:20 +01:00
Eric Ernst
1e79f7c9d9 release: Revert kata-deploy changes after 2.4.0-rc0 release
As 2.4.0-rc0 has been released, let's switch the kata-deploy / kata-cleanup
tags back to "latest", and re-add the kata-deploy-stable and the
kata-cleanup-stable files.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-03-17 15:11:20 +01:00
Eric Ernst
ac65feeae3 release: Kata Containers 2.4.0-rc0
- Enhancement: fix comments/logs and delete not used function
- storage: make k8s emptyDir volume creation location configurable
- Implement direct-assigned volume
- Bump containerd to 1.6.1
- experimentally enable vcpu hotplug and virtio-mem on arm64 in kernel part
- versions: Upgrade to Cloud Hypervisor v22.0
- katatestutils: remove distro constraints
- Minor fixes for the `disable_block_device_use` comments
- clh: stop virtofsd if clh fails to boot up the vm
- clh: tdx: Don't use sharedFS with Confidential Guests
- runtime: Build golang components with extra security options
- snap: Use git clone depth 1 for QEMU and dependencies
- snap: Don't build cloud-hypevisor on ppc64le
- build: always reset ARCH after getting it
- virtcontainers: remove temp dir created for vsock in test code
- docs: Add unit testing presentation
- virtcontainers: Use available s390x hugepages
- Update QEMU >= 6.1.0 in configure-hypervisor.sh
- Fix monitor listen address
- snap: clh: Re-use kata-deploy script here
- osbuilder: Add CentOS Stream rootfs
- runtime: Gofmt fixes
- Update `confidential_guest` comments
- cleanup runtime pkgs for Darwin build, add basic Darwin build/unit test
- docs: Update Readme document
- runtime: use Cmd.StdoutPipe instead of self-created pipe
- docs: Developer-Guide build a custom Kata agent with musl
- kata-agent: Fix mismatching error of cgroup and mountinfo.
- runtime, config: make selinux configurable
- Fix unbound variable / typo on error mesage
- clh: Add TDX support
- virtcontainers: Do not add a virtio-rng-ccw device
- kata-monitor: fix collecting metrics for sandboxes not started through CRI
- runtime: fix package declaration for ppc64le
- Make the hypervisor framework not Linux specific
- kata-deploy: Simplify Dockerfile and support s390x
- Support nerdctl OCI hooks
- shim: log events for CRI-O
- docs: Update contributing link
- kata-deploy: Use (kata with) qemu as the default shim-v2 binary
- kata-monitor: simplify sandbox cache management and attach kubernetes POD metadata to metrics
- nydus: add lazyload support for kata with clh
- kernel: remove SYS_SUPPORTS_HUGETLBFS from powerpc fragments
- packaging: Use `patch` for applying patches
- virtcontainers: Remove duplicated assert messages in utils test code
- versions: add nydus-snapshotter
- docs: Update limitations document
- packaging: support qemu-tdx
- Kata manager fix install
- versions: Linux 5.15.x
- trace-forwarder/agent-ctl: run cargo fmt/clippy in make check
- docs: Improve top-level README
- runtime: use github.com/mdlayher/vsock@v1.1.0
- tools: Build cloud-hypervisor with "--features tdx"
- virtiofsd: Use "-o announce_submounts"
- feature: hugepages support
- tools: clh: Allow to set when to build from sources and the build flags passed down to cargo
- docs: Remove docker run and shared memory from limitations
- versions: Udpate Cloud Hypervisor to 55479a64d237
- kernel: add missing config fragment for TDx
- runtime: The index variable is initialized multiple times in for
- scripts: fix a typo while to check build_type
- versions: bump CRI-O to its 1.23 release
- feature(nydusd): add nydusd support to introduce lazyload ability
- docs: Fix relative links in Markdown
- kernel: support TDx
- device: Actually update PCIDEVICE_ environment variables for the guest
- docs: Update link to EFK stack docs
- runtime: support QEMU SGX
- snap: update qemu version to 6.1.0 for arm
- Release process related fixes
- openshift-ci: switch to CentOS Stream
- virtcontainers: Split the rootless package into OS specific parts
- runtime: suppport split firmware
- kata-deploy: for testing, make sure we use the PR branch
- docs: Remove Zun documentation with kata containers
- agent: Fix execute_hook() args error
- workflows: stop checking revert commit

84dff440 release: Adapt kata-deploy for 2.4.0-rc0
b257e0e5 rustjail: delete function signal in BaseContainer
d647b28b agent: delete meaningless FIXME comment
1b34494b runtime: fix invalid comments for pkg/resourcecontrol
afc567a9 storage: make k8s emptyDir creation configurable
e76519af runtime: small refactor to improve readability
7e5f11a5 vendor: Update containerd to 1.6.1
42771fa7 runtime: don't set socket and thread for arm/virt
8828ef41 kernel: add arm experimental kernel build support
8a9007fe config: remove 2 config as they are removed in 5.15
1b6f7401 kernel: add arm experimental patches to support vcpu hotplug and virtio-mem
f905161b runtime: mount direct-assigned block device fs only once
27fb4902 agent: add get volume stats handler in agent
ea51ef1c runtime: forward the stat and resize requests from shimv2 to kata agent
c39281ad runtime: update container creation to work with direct assigned volumes
4e00c237 agent: add grpc interface for stat and resize operations
e9b5a255 runtime: add stat and resize APIs to containerd-shim-v2
6e0090ab runtime: persist direct volume mount info
fa326b4e runtime: augment kata-runtime CLI to support direct-assigned volume
b8844fb8 versions: Upgrade to Cloud Hypervisor v22.0
af804734 clh: stop virtofsd if clh fails to boot up the vm
97951a2d clh: Don't use SharedFS with Confidential Guests
c30b3a9f clh: Adding a volume is not supported without SharedFS
f889f1f9 clh: introduce supportsSharedFS()
54d27ed7 clh: introduce loadVirtiofsDaemon()
ae2221ea clh: introduce stopVirtiofsDaemon()
e8bc26f9 clh: introduce setupVirtiofsDaemon()
413b3b47 clh: introduce createVirtiofsDaemon()
55cd0c89 runtime: Build golang components with extra security options
76e4f6a2 Revert "hypervisors: Confidential Guests do not support Device hotplug"
fa8b9392 config: qemu: Fix disable_block_device_use comments
9615c8bc config: fc: Don't expose disable_block_device_use
c1fb4bb7 snap: Don't build cloud-hypevisor on ppc64le
58913694 snap: Use git clone depth 1 for QEMU and dependencies
b27c7f40 docs: Add unit testing presentation
e64c54a2 monitor: Listen to localhost only by default
e6350d3d monitor: Fix build options
a67b93bb snap: clh: Re-use kata-deploy script here
f31125fe version: Bump cloud-hypervisor to b0324f85571c441f
54d0a672 subsystem: build
edf20766 docs: Update Readme document
eda8ea15 runtime: Gofmt fixes
4afb278f ci: add github action to exercise darwin build, unit tests
e355a718 container: file is not linux specific
b31876ee device-manager: move linux-only test to a linux-only file
6a5c6344 resourcecontrol: SystemdCgroup check is not necessarily linux specific
cc58cf69 resourcecontrol: convert stats dev_t to unit64types
5be188cc utils: Add darwin stub
ad044919 virtcontainers: Convert stats dev_t to uint64
56751089 katautils: Use a syscall wrapper for the hook JSON state
7d64ae7a runtime: Add a syscall wrapper package
abc681ca katautils: Add Darwin stub for the netNS API
de574662 config: Expand confidential_guest comments
641d475f config: clh: Use "Intel TDX" instead of just "TDX"
0bafa2de config: clh: Mention supported TEEs
81ed269e runtime: use Cmd.StdoutPipe instead of self-created pipe
8edca8bb kata-agent: Fix mismatching error of cgroup and mountinfo.
a9ba7c13 clh: Fix typo on HotplugRemoveDevice
827ab82a tools: clh: Fix unbound variable
082d538c runtime: make selinux configurable
1103f5a4 virtcontainers: Use FilesystemSharer for sharing the containers files
533c1c0e virtcontainers: Keep all filesystem sharing prep code to sandbox.go
61590bbd virtcontainers: Add a Linux implementation for the FilesystemSharer
03fc1cbd virtcontainers: Add a filesystem sharing interface
72434333 clh: Add TDX support
a13b4d5a clh: Add firmware to the config file
a8827e0c hypervisors: Confidential Guests do not support NVDIMM
f50ff9f7 hypervisors: Confidential Guests do not support Memory hotplug
df8ffecd hypervisors: Confidential Guests do not support Device hotplug
28c4c044 hypervisors: Confidential Guests do not support VCPUs hotplug
29ee870d clh: Add confidential_guest to the config file
9621c596 clh: refactor image / initrd configuration set
dcdc412e clh: use common kernel params from the hypervisor code
4c164afb versions: Update Cloud Hypervisor to 5343e09e7b8db
b2a65f90 virtcontainers: Use available s390x hugepages
cb4230e6 runtime: fix package declaration for ppc64le
fec26f8e kata-monitor: trivial: rename symbols & labels
9fd4e551 runtime: Move the resourcecontrol package one layer up
823faee8 virtcontainers: Rename the cgroups package
0d1a7da6 virtcontainers: Rename and clean the cgroup interface
ad10e201 virtcontainers: cgroups: Move non Linux routine to utils.go
d49d0b6f virtcontainers: cgroups: Define a cgroup interface
3ac52e81 kata-monitor: fix updating sandbox cache at startup
160bb621 kata-monitor: bump version to 0.3.0
1a3381b0 docs: Developer-Guide build a custom Kata agent with musl
f6fc1621 shim: log events for CRI-O
1d68a08f docs: Update contributing link
9123fc09 kata-deploy: Simplify Dockerfile and support s390x
11220f05 kata-deploy: Use (kata with) qemu as the default shim-v2 binary
3175aad5 virtiofs-nydus: add lazyload support for kata with clh
94b831eb virtcontainers: remove temp dir created for vsock in test code
8cc1b186 kernel: remove SYS_SUPPORTS_HUGETLBFS from powerpc fragments
5c9d2b41 packaging: Use `patch` for applying patches
5b3fb6f8 kernel: Build SGX as part of the vanilla kernel
2c35d8cb workflows: Stop building the experimental kernel
32e7845d snap: Build vanilla kernel for all arches
27de212f runtime: Always add network endpoints from the pod netns
1cee0a94 virtcontainers: Remove duplicated assert messages in utils test code
6c1d149a docs: Update limitations document
7c4ee6ec packaging/qemu: create no_patches file for qemu-tdx
d47c488b versions: add qemu tdx section
77c29bfd container: Remove VFIO lazy attach handling
7241d618 versions: add nydus-snapshotter
26b3f001 virtcontainers: Split hypervisor into Linux and OS agnostic bits
fa0e9dc6 virtcontainers: Make all Linux VMMs only build on Linux
c91035d0 virtcontainers: Move non QEMU specific constants to hypervisor.go
10ae0591 virtcontainers: Move guest protection definitions to hypervisor.go
b28d0274 virtcontainers: Make max vCPU config less QEMU specific
a5f6df6a govmm: Define the number of supported vCPUs per architecture
a6b40151 tools: clh: Remove unused variables
5816c132 tools: Build cloud-hypervisor with "--features tdx"
e6060cb7 versions: Linux 5.15.x
9818cf71 docs: Improve top-level and runtime README
36c3fc12 agent: support hugepages for containers
81a8baa5 runtime: add hugepages support
7df677c0 runtime: Update calculateSandboxMemory to include Hugepages Limit
948a2b09 tools: clh: Ensure the download binary is executable
72bf5496 agent: handle hook process result
80e8dbf1 agent: valid envs for hooks
4f96e3ea katautils: Pass the nerdctl netns annotation to the OCI hooks
a871a33b katautils: Run the createRuntime hooks
d9dfce14 katautils: Run the preStart hook in the host namespace
6be6d0a3 katautils: Pass the OCI annotations back to the called OCI hooks
493ebc8c utils: Update kata manager docs
34b2e67d utils: Added more kata manager cli options
714c9f56 utils: Improve containerd configuration
c464f326 utils: kata-manager: Force containerd sym link creation
4755d004 utils: Fix unused parameter
601be4e6 utils: Fix containerd installation
ae21fcc7 utils: Fix Kata tar archive check
f4d1e45c utils: Add kata-manager CLI options for kata and containerd
395cff48 docs: Remove docker run and shared memory from limitations
e07545a2 tools: clh: Allow passing down a build flag
55cdef22 tools: clh: Add the possibility to always build from sources
3f87835a utils: Switch kata manager to use getopts
4bd945b6 virtiofsd: Use "-o announce_submounts"
37df1678 build: always reset ARCH after getting it
3a641b56 katatestutils: remove distro constraints
90fd625d versions: Udpate Cloud Hypervisor to 55479a64d237
573a37b3 osbuilder: Add CentOS Stream rootfs
f10642c8 osbuilder: Source .cargo/env before checking Rust
955d359f kernel: add missing config fragment for TDx
734b618c agent-ctl: run cargo fmt/clippy in make check
12c37faf trace-forwarder: add make check for Rust
c1ce67d9 runtime: use github.com/mdlayher/vsock@v1.1.0
42a878e6 runtime: The index variable is initialized multiple times in for
1797b3eb packaging/kernel: build TDX guest kernel
98752529 versions: add url and tag for tdx kernel
bc8464e0 packaging/kernel: add option -s option
2d9f89ae feature(nydusd): add nydusd support to introduse lazyload ability
b19b6938 docs: Fix relative links in Markdown
9590874d device: Update PCIDEVICE_ environment variables for the guest
7b7f426a device: Keep host to VM PCI mapping persistently
0b2bd641 device: Rework update_spec_pci() to update_env_pci()
982f14fa runtime: support QEMU SGX
40aa43f4 docs: Update link to EFK stack docs
54e1faec scripts: fix a typo while to check build_type
07b9d93f virtcontainer: Simplify the sandbox network creation flow
2c7087ff virtcontainers: Make all endpoints Linux only
49d2cde1 virtcontainers: Split network tests into generic and OS specific parts
0269077e virtcontainers: Remove the netlink package dependency from network.go
7fca5792 virtcontainers: Unify Network endpoints management interface
c67109a2 virtcontainers: Remove the Network PostAdd method
e0b26443 virtcontainers: Define a Network interface
5e119e90 virtcontainers: Rename the Network structure fields and methods
b858d0de virtcontainers: Make all Network fields private
49eee79f virtcontainers: Remove the NetworkNamespace structure
844eb619 virtcontainers: Have CreateVM use a Network reference
d7b67a7d virtcontainers: Network API cleanups and simplifications
2edea883 virtcontainers: Make the Network structure manage endpoints
8f48e283 virtcontainers: Expand the Network structure
5ef522f7 runtime: check kvm module `sev` correctly
419d8134 snap: update qemu version to 6.1.0 for arm
00722187 docs: update Release-Process.md
496bc10d tools: check for yq before using it
88a70d32 Revert "workflows: Ensure a label change re-triggers the actions"
a9bebb31 openshift-ci: switch to CentOS Stream
89047901 kata-deploy-push: only run if PR modifying tools path
7ffe9e51 virtcontainers: Do not add a virtio-rng-ccw device
1f29478b runtime: suppport split firmware
24796d2f kata-deploy: for testing, make sure we use the PR branch
1cc1c8d0 docs: Remove images from Zun documentation
5861e52f docs: Remove Zun documentation with kata containers
903a6a45 versions: Bump critools to its 1.23 release
63eb1158 versions: bump CRI-O to its 1.23 release
5083ae65 workflows: stop checking revert commit
14e7f52a virtcontainers: Split the rootless package into OS specific parts
ab447285 kata-monitor: add kubernetes pod metadata labels to metrics
834e199e kata-monitor: drop unused functions
7516a8c5 kata-monitor: rework the sandbox cache sync with the container manager
e78d80ea kata-monitor: silently ignore CHMOD events on the sandboxes fs
e9eb34ce kata-monitor: improve debug logging
4fc4c76b agent: Fix execute_hook() args error

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-03-17 15:11:20 +01:00
Eric Ernst
fbb2f0afd0 release: Adapt kata-deploy for 2.4.0-rc0
kata-deploy files must be adapted to a new release.  The cases where it
happens are when the release goes from -> to:
* main -> stable:
  * kata-deploy-stable / kata-cleanup-stable: are removed

* stable -> stable:
  * kata-deploy / kata-cleanup: bump the release to the new one.

There are no changes when doing an alpha release, as the files on the
"main" branch always point to the "latest" and "stable" tags.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-03-17 15:11:20 +01:00
Fabiano Fidêncio
5e05de2a51 tools: release: Do not consider release candidates as stable releases
During the release of 2.4.0-rc0 @egernst noticed an incositency in the
way we handle release tags, as release candidates are being taken as
"stable" releases, while both the kata-deploy tests and the release
action consider this as "latest".

Ideally we should have our own tag for "release candidate", but that's
something that could and should be discussed more extensively outside of
the scope of this quick fix.

For now, let's align the code generating the PR for bumping the release
with what we already do as part of the release action and kata-deploy
test, and tag "-rc"  as latest, regardless of which branch it's coming
from.

Fixes: #3847

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-03-17 15:11:20 +01:00
David Gibson
fd8e162f25 agent: Verify that we allocated as many hugepages as we need
allocate_hugepages() writes to the kernel sysfs file to allocate hugepages
in the Kata VM.  However, even if the write succeeds, it's not certain that
the kernel will actually be able to allocate as many hugepages as we
requested.

This patch reads back the file after writing it to check if we were able to
allocate all the required hugepages.

fixes #3816

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-03-17 15:11:20 +01:00
David Gibson
e3adbf95a5 agent: Don't attempt to create directories for hugepage configuration
allocate_hugepages() constructs the path for the sysfs directory containing
hugepage configuration, then attempts to create this directory if it does
not exist.

This doesn't make sense: sysfs is a view into kernel configuration, if the
kernel has support for the hugepage size, the directory will already be
there, if it doesn't, trying to create it won't help.

For the same reason, attempting to create the "nr_hugepages" file
itself is pointless, so there's no reason to call
OpenOptions::create(true).

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-03-17 15:11:20 +01:00
Steve Horsman
341e098329 Merge pull request #3861 from arronwy/enable_image_rs
image: Enable image-rs crate to pull image inside guest
2022-03-16 15:57:32 +00:00
Arron Wang
deee3cf4a2 oci: Make device type optional in LinuxDeviceCgroup
Follow the oci spec to compatiable with config file generated
by other implementation:
6969a0a09a/specs-go/config.go (L401)

Fixes: #3860

Signed-off-by: Arron Wang <arron.wang@intel.com>
2022-03-15 20:36:16 +08:00
Arron Wang
c7a7fc1267 image: Enable image-rs crate to pull image inside guest
Image-rs crate image pull/decrypt/decompression/unpack/mount
features are ready now.

With image-rs pull_image API, the downloaded container image layers
will store at IMAGE_RS_WORK_DIR, and generated bundle dir with rootfs
and config.json will be saved under CONTAINER_BASE/cid directory.

Fixes: #3860

Signed-off-by: Arron Wang <arron.wang@intel.com>
2022-03-14 16:24:26 +08:00
GabyCT
6fdafd47ef Merge pull request #3828 from liangzhou121/agent-aa-1
CCv0: Update AA's launch command
2022-03-08 14:09:12 -06:00
Steve Horsman
8327fcd5c2 Merge pull request #3844 from stevenhorsman/CCv0
CCv0: Merge main into CCv0 branch
2022-03-07 13:59:09 +00:00
stevenhorsman
7b4fd200ca agent-ctl: Add image client
- Add image-client to the new get volume stats code in CCv0

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-03-07 12:36:01 +00:00
stevenhorsman
51a9de8079 runtime: Fix bad merge
Fix missing brace from bad merge

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-03-07 12:36:01 +00:00
stevenhorsman
75e2e5ab46 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #3843
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-03-07 11:09:12 +00:00
Steve Horsman
6637d92cb2 Merge pull request #3832 from stevenhorsman/CCv0
CCv0: Update containerd version to be based on 1.6.1
2022-03-06 13:34:38 +00:00
stevenhorsman
6da2eac059 runtime: re-vendor containerd
- Re-vendor to the cc containerd 1.6.1 rebase

Fixes: #3830
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-03-04 16:10:00 +00:00
stevenhorsman
035f539bbf versions: Update containerd
Update cc containerd's version

Fixes: #3830
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-03-04 16:09:35 +00:00
zhouliang121
5451c8da4a CCv0: Update AA's launch command
1.Update AA's launch command according to latest implementation
2.Enable get_resource port which will be used by signature verification

Fixes: #3827
Signed-off-by: zhouliang121 <liang.a.zhou@linux.alibaba.com>
2022-03-04 14:26:31 +08:00
GabyCT
c9cbdd085f Merge pull request #3808 from stevenhorsman/CCv0
CCv0: Merge main into CCv0 branch
2022-03-03 14:55:37 -06:00
stevenhorsman
39d6b826c1 runtime: Add cc compatibility to refactor
The create rootfs code has been refactored,
try and add back in the CC behaviour

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-03-03 15:08:36 +00:00
stevenhorsman
fe52465bdb doc: Fix typo
Fix typo in script

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-03-02 17:25:36 +00:00
stevenhorsman
4decf30b3e CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #3807
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-03-02 15:02:54 +00:00
GabyCT
433a5de354 Merge pull request #3739 from stevenhorsman/CCv0
CCv0: Merge main into CCv0 branch - 21st Feb
2022-02-23 10:37:53 -06:00
stevenhorsman
2656b466b9 agent-ctl: Ignore clippy warning
- Clippy checks were introduced that cause a warning
for a function with more than 7 arguments.
The image service addition means handle_cmd
has 8 and re-factoring it would take us further
away from main, so ignore for now

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-02-21 17:37:07 +00:00
stevenhorsman
fca91c4fa7 CCv0: attempt to fix lint problem
- golint complained about field alignment, so try
moving field up in the struct

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-02-21 16:06:07 +00:00
stevenhorsman
e7e4ba9fc4 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #3738
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-02-21 15:22:18 +00:00
Fabiano Fidêncio
1559e5390c Merge pull request #3665 from stevenhorsman/CCv0-merge-main-14-feb
CCv0: Merge main into CCv0 branch
2022-02-17 15:14:34 +01:00
Steve Horsman
45e1268c70 Merge pull request #3638 from stevenhorsman/ccv0-encrypted
doc: Add doc and script for ssh-demo sample
2022-02-17 09:10:56 +00:00
stevenhorsman
e636e67e92 doc: Add doc and script for ssh-demo sample
- Add scripts and documentation to build, configure and test
the ssh-demo encrypted image sample in Kubernetes

Fixes: #3637

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-02-14 19:52:36 +00:00
stevenhorsman
8cc68970ed CCv0: Fix bad merge
I seem to have lost the image_rpc `use` statement in the merge
so re-add back in

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-02-14 18:02:05 +00:00
stevenhorsman
7159a35d20 agent: Move proto
- Move image proto to reflect main's refactoring of
protos into src/libs

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-02-14 17:08:09 +00:00
stevenhorsman
9f3b2aaf6a CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #3573
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-02-14 16:25:09 +00:00
Steve Horsman
aae311caaf Merge pull request #3640 from stevenhorsman/CCv0-containerd-rebase
CCv0: Update to new containerd
2022-02-14 12:57:06 +00:00
stevenhorsman
b185e6e704 runtime: kata updates based on containerd changes
Compatibility fixes with containerd updates

Fixes: #3658

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-02-14 10:59:51 +00:00
root
4cbcc23a55 vendor: Re-vendor based on updated containerd
Re-vendor runtime based on confidential-containers/containerd
fork being updated branch

Fixes: #3658

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-02-14 10:58:46 +00:00
stevenhorsman
79a7da4e72 version: version.yaml update
- Update version to point to new containerd fork branch name

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-02-14 10:55:59 +00:00
Steve Horsman
46522a3e46 Merge pull request #3569 from stevenhorsman/ccv0-kubernetes-e2e
CCv0 | doc: Add k8s e2e doc and scripts
2022-02-08 11:02:14 +00:00
stevenhorsman
9d4cd77560 doc: Add k8s e2e doc and scripts
- Add scripts and documentation to build, configure and test
created a Kata CC unencrypted container using Kubernetes
- Switch test images to quay.io as image_rpc.rs has some
problems with docker.io?
- Update documentation to better fit the kata documentation
requirements and fix typos

- Fixes: #3511

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-02-07 17:44:00 +00:00
Steve Horsman
e71592d5da Merge pull request #3564 from stevenhorsman/ccv0-improvements
CCv0 | doc: Improve ccv0.sh and CCv0 how-to document
2022-02-07 13:02:20 +00:00
stevenhorsman
f68f73a849 CCv0: Add network debug packages
Add extra packages for debugging purposes

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-01-31 18:31:15 +00:00
stevenhorsman
f19811df43 doc: Improve ccv0.sh and CCv0 how-to document
General doc enchancements including:
- Change `cd`s for `pushd` and `popd`s
- Remove hard coded architectures
- Tighten up the security where we `chmod 777`
- Add support for not running as source
- Updates so it doesn't do `ctr pull` if the image is on the
 local system already
- Doc and Test running as non-root user (covered by #2879)
- Update doc to match image_rpc changes

Fixes: #3549
Fixes: #2879
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-01-31 09:33:05 +00:00
Steve Horsman
d5a351877d Merge pull request #3517 from stevenhorsman/crictl-ccv0-e2e-doc
CCv0 | doc: Add crictl e2e cc doc
2022-01-28 14:13:44 +00:00
stevenhorsman
c68f0360f0 doc: Add crictl e2e cc doc
- Add scripts and documentation to build, configure and test
created a Kata CC unencrypted container using crictl
- Update documentation to better fit the kata documentation requirements

- Fixes: #3510

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-01-28 09:35:45 +00:00
Steve Horsman
a3d8bc3346 Merge pull request #3377 from stevenhorsman/CCv0
CCv0: Merge main into CCv0 branch
2022-01-28 09:17:35 +00:00
stevenhorsman
d85ef3f6a5 agent: Update ocicrypt-rs package
- Update the ocicrypt-rs package after
https://github.com/containers/ocicrypt-rs/issues/16
was fixed

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-01-27 10:44:18 +00:00
stevenhorsman
5f7115eca8 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #3376
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-01-04 10:29:03 +00:00
Pradipta Banerjee
39d438763c Merge pull request #3346 from Jakob-Naucke/runtime-payload
CCv0 | actions: Push demo image to runtime-payload
2021-12-24 08:44:52 +05:30
Jakob Naucke
18e6267730 actions: Push demo image to runtime-payload
Push the demo image to `quay.io/confidential-containers/runtime-payload`
(which, as opposed to `.../kata-demo`, existed all along).

Fixes: #3345
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-23 15:57:20 +01:00
Jakob Naucke
bb7a722ce0 Merge pull request #3199 from Jakob-Naucke/deploy-demo
github: Add workflow for deploying a CCv0 demo
2021-12-21 11:30:25 +01:00
Jakob Naucke
f5e6961dcb kata-deploy: Configure Kata & containerd for CCv0
Introduce kata-cc runtime class, shim & config
- Specify cri_handler in containerd config
- Specify to use initrd
- Specify kernel_params according to guest config
- Specify service_offload

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-20 18:05:46 +01:00
Jakob Naucke
a570b6a0a6 github: Add workflow for deploying a CCv0 demo
using the offline FS KBC [1] and keys from the SSH demo [2]. The
workflow is adapted from `main:kata-deploy-test.yaml`. The image
deployed here is _not_ for a trusted execution environment.

[1] - https://github.com/confidential-containers/attestation-agent/tree/main/src/kbc_modules/offline_fs_kbc
[2] - https://github.com/confidential-containers/documentation/tree/main/demos/ssh-demo

Fixes: #3198
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-20 18:05:46 +01:00
Jakob Naucke
3c79630b87 docs: Create sample config for confidential agent
Basic config, no debug endpoints, no exec/reseed. Uses the
`$AA_KBC_PARAMS` variable to be used with `envsubst`.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-20 18:05:46 +01:00
Jakob Naucke
a2926324f5 kata-deploy: realpath INCLUDE_ROOTFS for Docker
Run `realpath` on `INCLUDE_ROOTFS` so it is not required to provide a
full path. This simplifies the required GitHub Actions workflow, as
GitHub's `env` cannot use shell expansions, as well as the usability
overall.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-20 18:05:46 +01:00
Bin Liu
637b519c5a Merge pull request #3301 from stevenhorsman/CCv0
CCv0: Merge main into CCv0 branch
2021-12-20 17:34:32 +08:00
stevenhorsman
822efa344d CCv0: Merge
- Re-vendor after merge due to static check error

Fixes: #3295

Signed-off-by: stevenhorsman steven@uk.ibm.com
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-12-17 13:55:51 +00:00
stevenhorsman
bf6b1102db doc: Update ccv0
- Update script to reflect merge of directory changes

- Fixes: #3295

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-12-17 11:00:33 +00:00
stevenhorsman
85bb1e5e45 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #3295

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-12-17 10:33:30 +00:00
Steve Horsman
934f1f6a8b Merge pull request #3260 from stevenhorsman/CCv0
CCv0: Merge main into CCv0 branch
2021-12-15 11:26:15 +00:00
stevenhorsman
e1ba87408c CCv0: Merge main into CCv0 branch
Weekly merge of new commits in main into CCv0

Fixes: #3259

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-12-13 14:15:32 +00:00
Fabiano Fidêncio
a7b6d16c88 Merge pull request #3210 from stevenhorsman/CCv0
CCv0: Merge main into CCv0 branch
2021-12-06 21:42:34 +01:00
stevenhorsman
ff572fa219 CCv0: re-vendor code
Ran `CARGO_HOME="/tmp/.cargo" make vendor` after static checks failed

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-12-06 12:00:35 +00:00
stevenhorsman
15494d7e06 CCv0: Merge main into CCv0 branch
Resolve merge conflicts during merge

Fixes: #3209
Depends-on: github.com/kata-containers/tests#4256

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-12-06 11:12:47 +00:00
Steve Horsman
6f2d89ef6f Merge pull request #3171 from stevenhorsman/skopeo-not-default
Stop skopeo being installed by default
2021-12-03 15:54:44 +00:00
stevenhorsman
bb66dbdccc osbuilder: Stop skopeo being installed by default
With the new rust image pull service skopeo we can parameterise whether to build
and install skopeo and turn it off by default if we don't need
signature verification support

Fixes: #3170

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-12-03 14:53:39 +00:00
stevenhorsman
95ab38ae54 doc: Stop skopeo being installed by default
With the new rust image pull service skopeo we can parameterise whether to build
and install skopeo and turn it off by default if we don't need
signature verification support

Fixes: #3170

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-12-03 14:53:39 +00:00
Jakob Naucke
389bbcb183 kata-deploy: Support separate Skopeo spec
The initrd build process now supports facultatively installing Skopeo
while still installing Umoci. Mirror this change in the respective
kata-deploy process.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-03 14:53:39 +00:00
Jakob Naucke
af91084887 Merge pull request #3151 from liangzhou121/osbuilder-dev-1
CCv0|osbuilder: integrate AA's EAA KBC into ubuntu rootfs image
2021-12-03 09:37:29 +01:00
zhouliang121
365acda643 CCv0|osbuilder: add EAA KBC into ubuntu rootfs
Integrate EAA KBC into ubuntu rootfs image.
Fix build failure if build with AA_KBC=eaa_kbc option.

Fixes: #3167
Signed-off-by: zhouliang121 <liang.a.zhou@linux.alibaba.com>
2021-12-02 12:07:45 +08:00
Arron Wang
263bbe937f osbuilder: Add build dependency for openssl and protobuf
The openssl-sys crate will automatically detect:
https://docs.rs/openssl/0.10.16/openssl/#automatic

Signed-off-by: Arron Wang <arron.wang@intel.com>
2021-12-01 23:42:50 +01:00
Samuel Ortiz
378a4ce7a2 workflow: Fix Rust vendoring step for git crates
The CI runner fails to clone the git crates as it probably is confused
about its CARGO_HOME value. That prevents vendoring to succeed as the
runner has nothing to copy over to the vendoring code.

We fix that by temporarily setting CARGO_HOME to tmpfs, only for the
vendoring step. It's hackish.

Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
2021-12-01 23:42:50 +01:00
Arron Wang
3454785c9b uevent: Fix clippy warning
unneeded `return` statement

Signed-off-by: Arron Wang <arron.wang@intel.com>
2021-12-01 23:42:50 +01:00
Samuel Ortiz
77ba9dcdbb versions: Update to Rust 2021 edition
We use tonic to build GRPC client to talk with attestation agent,
and tonic require newer version of rust.

Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
2021-12-01 23:42:50 +01:00
Arron Wang
48ecac3c98 image_rpc: replace colon with underline for image store path
':' will have special meaning for umoci during upack, then we
do not use it as part of the image store path

Signed-off-by: Arron Wang <arron.wang@intel.com>
2021-12-01 23:42:50 +01:00
Arron Wang
7a1085fb74 image_rpc: Add rustified image pulling/decryption
Fixes: #3041

Signed-off-by: Arron Wang <arron.wang@intel.com>
2021-12-01 23:42:50 +01:00
Arron Wang
6237b9991c image_rpc: Fix the image store path
the "latest" is the tag info, real data will store in image_oci directory

Fixes: #3041

Signed-off-by: Arron Wang <arron.wang@intel.com>
2021-12-01 23:42:50 +01:00
Steve Horsman
855878b803 Merge pull request #3133 from stevenhorsman/CCv0
CCv0: Merge main into `ccv0`
2021-12-01 14:45:37 +00:00
Jakob Naucke
f740f97bed kata-deploy: Support CCv0
For the initrd build, add makeopts for $SKOPEO_UMOCI and $AA_KBC. Use
the $INCLUDE_ROOTFS variable to specify a directory of files that should
be recursively merged into the guest.

Fixes: #3126
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-01 08:58:04 +01:00
Jakob Naucke
69d157f78b CCv0: Use Ubuntu as base for initrd
As discussed in #2908, Ubuntu is used as a base for CCv0 for building
umoci in the guest. Currently, CCv0 only works with initrd, so this only
applies to initrd.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-01 08:58:04 +01:00
stevenhorsman
0214eac4c5 static-checks: Allow Merge commit to be >75 chars
Some generated merge commit messages are >75 chars
Allow these to not trigger the subject line length failure

Fixes: #3132
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-26 13:19:57 +00:00
stevenhorsman
8962cce365 CCv0: Merge main into CCv0
Weekly merge of main branch into CCv0 26th November

Fixes: #3132
Depends-on: github.com/kata-containers/tests#4226

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-26 11:14:24 +00:00
Tobin Feldman-Fitzthum
7c41af4082 Agent: Attestation Agent Integration
Pull an encrypted image using the Attestation Agent as
a keyprovider.

Fixes: #3022

Signed-off-by: Tobin Feldman-Fitzthum <tobin@linux.ibm.com>
Co-authored-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-11-24 23:32:11 +01:00
Steve Horsman
27c0dc260c Merge pull request #3084 from stevenhorsman/CCv0
CCv0: Merge main into CCv0
2021-11-22 17:44:09 +00:00
stevenhorsman
7152448f21 ci: Add Merge message to commit check
Add the "Merge pull request (kata-containers)?#<x> from" message to the
subsystem check to allow commit check on merges between branches to work

Fixes: #3085
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-22 16:21:24 +00:00
stevenhorsman
d17aaba475 agent: Add PoC container signature validation
- Add new agent configuration policy path parameter
- Update agent pull image to use the policy path if specified and
otherwise fall back to the accept all policy
- Remove the double copy of the image during pulling
- Ensure that temporary directories are always removed

Fixes: #2682
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-22 11:51:05 +01:00
stevenhorsman
6f1bdd7079 doc: Add PoC container signature validation
- Document how to test the signature validation with
a number of different scenarios and test images
- Update ccv0.sh to add policy_path to kernel_params

Fixes: #2682

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-22 11:51:05 +01:00
stevenhorsman
7670792f97 osbuilder: Add PoC container signature validation
- Add hardcoded gpg, signature and polict files
- Modify rootfs.sh to put these in the correct place in the kata image
if skopeo and umoci are being used

Fixes: #2682

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-22 11:51:05 +01:00
root
e3c8c9023f vendor: Re-vendor after merging main
Ran `make -C src/runtime handle_vendor` after merge

Fixes: #3085

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-19 15:41:05 +00:00
stevenhorsman
ea34b30839 Merge remote-tracking branch 'upstream/main' into CCv0
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-19 14:37:33 +00:00
Samuel Ortiz
3f68265d60 versions: Switch to the Alibaba containerd branch
It contains the initial ImageService implementation.

Fixes #3009

Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
2021-11-18 22:41:05 +01:00
wllenyj
ab6b1cbfe9 agent: Append the container OCI spec with the image bundle one
We do not get a root filesystem path from the agent when creating a
new container for which the container image was not pulled by
containerd. That prevents the agent from creating the container.

To fix that, we populate the container root path with the internal
rootfs path by fetching the containerd added image name annotation and
mapping it back to a path through our image hash map.

Fixes #3009

Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
2021-11-18 22:41:05 +01:00
wllenyj
5691e66e1b shim: Fix CreateContainer for the confidential computing case
In the confidential computing scenario, there is no Image
information on the host, so skip handling Rootfs at
CreateContainer.

Fixes #3009

Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
2021-11-18 22:41:05 +01:00
wllenyj
e502fb23ea shim: Use ImageService to be compatible with containerd
- Replace containerd to `confidential-containers/containerd` in go.mod
- Use separate ImageService to support PullImage

Fixes #3009

Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
2021-11-18 22:41:05 +01:00
wllenyj
e1fcc2529c shim: Upgrade containerd to v1.6.0-beta.2
The version `v1.6.0-beta.2` released support for shim service,
which is needed for our implementation of ImageService.

Fixes #3009

Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
2021-11-18 22:41:05 +01:00
Fabiano Fidêncio
d22706c060 Merge pull request #3040 from wllenyj/rework-agent-ctl
CCv0|agent-ctl: fix compile error
2021-11-15 13:32:01 +01:00
wllenyj
429ab089f7 agent-ctl: fix complie error
Since the `utils::get_option` interface is modified,
PullImage needs to adapt to this modification in CCv0 branch.

Fixes #3044

Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
2021-11-15 14:28:40 +08:00
stevenhorsman
0a1d7893ff Merge remote-tracking branch 'upstream/main' into CCv0
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-12 09:46:51 +00:00
stevenhorsman
6c3e03958a Merge branch 'CCv0' of github.com:kata-containers/kata-containers into CCv0
Keep the CCvo branch up to date with changes from main
2021-11-12 09:36:39 +00:00
wllenyj
08c34f2ac2 runtime: Use a separate ImageService to support PullImage
Compatible with the agent's ImageService.

Fixes #2984

Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
2021-11-10 16:10:21 +01:00
wllenyj
dfb8c965e6 agent: Separate the ImageService from the AgentService
Use a separate ImageService to support PullImage.

Fixes #2984

Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
2021-11-10 16:10:21 +01:00
Jakob Naucke
23bd6fe5da Merge pull request #2908 from Jakob-Naucke/install-aa
osbuilder: Build attestation-agent and friends
2021-11-10 16:09:54 +01:00
Jakob Naucke
13f6418c46 osbuilder: Build Skopeo, umoci, attestation-agent
When the environment variable $SKOPEO_UMOCI is set to "yes", Skopeo and
umoci are built inside the guest build container and installed to the
guest rootfs. The respective build- and runtime dependencies are added.
This respects the (existing) $LIBC variable (gnu/musl) and avoids issues
with glibc mismatches.
This is currently only supported for Ubuntu guests, as the system Golang
packages included in the versions of other distros that we use are too
old to build these packages, and re-enabling installing Golang from
golang.org is cumbersome, given especially that it is unclear how long
we will keep using Skopeo and umoci.

Additionally, when the environment variable $AA_KBC is set,
attestation-agent (with that KBC) is included.

This replaces some logic in ccv0.sh that is removed.

Fixes: #2907
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-11-09 18:04:21 +01:00
stevenhorsman
4762da105c Merge remote-tracking branch 'upstream/main' into CCv0 2021-11-08 14:58:30 +00:00
stevenhorsman
9b34595ad0 version: Update CCv0 fork of containerd used
- Update CCv0 to use the new confidential containers fork of containerd
- Start using the current-CCv0 branch

Fixes #2947

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-05 14:50:32 +00:00
stevenhorsman
d57648eb13 doc: Add pull image support to shim
- Add a ctr shim pull image function to CCv0.sh
- Add information on how to use it in the how-to doc

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-05 14:50:32 +00:00
Dave Hay
02f6db595c shim: Add PullImage support
Add a new PullImage endpoint to the shim API.
Add new PullImage functions to the virtcontainers files, which allows
the PullImage endpoint on the agent to be called.
Update the containerd vendor files to support new PullImage API changes.

Fixes #2651

Signed-off-by: Dave Hay <david_hay@uk.ibm.com>
Co-authored-by: ashleyrobertson <ashleyro@uk.ibm.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
2021-11-05 14:50:31 +00:00
Fabiano Fidêncio
18834810e6 kata-deploy: point to the -cc:v0 image
Just in case someone thinks about using kata-deploy directly from this
branch, let's point to the `-cc:v0`image.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2021-11-05 14:50:29 +00:00
Fabiano Fidêncio
7854cf007a kata-deploy: Add a basic script to build & upload a CCv0 image
This is a dirty hack, that we should expand later so we can pass one or
n number of repos where we'll upload our images, and use it as part of
the release scripts.

For now, however, let's just do this quick & dirty hack so we can
present the CCv0 demo using the operator, even knowing that the
kubernetes part of the work is not done yet and that the demo itself
will be done connecting to a node and doing all the shenanigans
manually.

Fixes: #2854

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2021-11-05 14:49:45 +00:00
stevenhorsman
f2bdd846fd doc: Update base rootfs to be ubuntu
- Update the CCv0 demo script to use ubuntu instead of fedora
- Update the extra packages to reflect the apt vs dnf namings
- Build and add the skopeo binary to the rootfs image
- Minor kubernetes init fix

Fixes #2849

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-05 14:49:45 +00:00
stevenhorsman
85d455791c osbuilder: update base rootfs to be ubuntu
- Revert fedora OS changes made in #2556 as we aren't using it anymore.
- They should be done in main under #2116

Fixes #2849

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-05 14:49:45 +00:00
stevenhorsman
03cf3aa168 agent: update base rootfs to be ubuntu
- Add skopeo --insecure-policy tag to reflect that ubuntu doesn't
create a default container policy

Fixes #2849

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-05 14:49:45 +00:00
stevenhorsman
c624e7fd97 agent: Add authenticated pull image support
Add source credentials field to pull_image endpoint
If field is not blank, send to skopeo in image pull command
Add source_creds to agentl-ctl pull command

Fixes: #2653
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-05 14:49:45 +00:00
stevenhorsman
522b9e33c3 doc: Add doc for authenticated pull image
Add support for a new source credentials environment variable in the
test script
Add documentation of it into the how-to guide

Fixes #2653

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-05 14:49:45 +00:00
stevenhorsman
40c406506b agent: Add CreateContainer support for pre-pulled bundle
- Add a check in setup_bundle to see if the bundle already exists
and if it does then skip the setup.

Fixes: #2617

Co-authored-by: Dave Hay <david_hay@uk.ibm.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-05 14:49:42 +00:00
stevenhorsman
500356ace1 doc: Add Create Container test and doc
Refactor ccv0.sh so it's easier to add agent-ctl commands
Add agent create container agent-ctl method to ccv0.sh
Update doc to reflect changes

Fixes #2617

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-05 14:49:20 +00:00
stevenhorsman
87c632baf2 version: Add support for using containerd PR
Add a pr_id field to the cri-containerd config in versions.yaml
so the CI scripts can use this in the CCv0 builds

Fixes #2576

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-05 14:49:20 +00:00
stevenhorsman
76b70a7a82 doc: Add doc and scripts for CCv0 agent PullImage
This commit add documentation and a script to help people to build, run,
test and demo the CCv0 changes around PullImage on guest.
It is currently limited to the Agent pullimage, but can be expanded
as more code is shared.

Fixes #2574

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2021-11-05 14:49:20 +00:00
Georgina Kinge
af44b7a591 agent: Add PullImage endpoint
This commit adds the PullImge endpoint to the agent
and the agent-ctl command to test it.

Fixes: #2509

Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
2021-11-05 14:49:20 +00:00
3833 changed files with 282360 additions and 462340 deletions

View File

@@ -1,24 +0,0 @@
# Copyright (c) 2024 Red Hat
#
# SPDX-License-Identifier: Apache-2.0
#
# Configuration file with rules for the actionlint tool.
#
self-hosted-runner:
# Labels of self-hosted runner that linter should ignore
labels:
- arm64-builder
- garm-ubuntu-2004
- garm-ubuntu-2004-smaller
- garm-ubuntu-2204
- garm-ubuntu-2304
- garm-ubuntu-2304-smaller
- garm-ubuntu-2204-smaller
- k8s-ppc64le
- metrics
- ppc64le
- sev
- sev-snp
- s390x
- s390x-large
- tdx

View File

@@ -8,7 +8,7 @@
script_dir=$(dirname "$(readlink -f "$0")")
parent_dir=$(realpath "${script_dir}/../..")
cidir="${parent_dir}/ci"
source "${cidir}/../tests/common.bash"
source "${cidir}/lib.sh"
cargo_deny_file="${script_dir}/action.yaml"

View File

@@ -15,7 +15,7 @@ concurrency:
jobs:
pr_wip_check:
runs-on: ubuntu-22.04
runs-on: ubuntu-latest
name: WIP Check
steps:
- name: WIP Check

View File

@@ -0,0 +1,104 @@
name: Add backport label
on:
pull_request:
types:
- opened
- synchronize
- reopened
- edited
- labeled
- unlabeled
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
check-issues:
if: ${{ github.event.label.name != 'auto-backport' }}
runs-on: ubuntu-latest
steps:
- name: Checkout code to allow hub to communicate with the project
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
uses: actions/checkout@v3
- name: Install hub extension script
run: |
pushd $(mktemp -d) &>/dev/null
git clone --single-branch --depth 1 "https://github.com/kata-containers/.github" && cd .github/scripts
sudo install hub-util.sh /usr/local/bin
popd &>/dev/null
- name: Determine whether to add label
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CONTAINS_AUTO_BACKPORT: ${{ contains(github.event.pull_request.labels.*.name, 'auto-backport') }}
id: add_label
run: |
pr=${{ github.event.pull_request.number }}
linked_issue_urls=$(hub-util.sh \
list-issues-for-pr "$pr" |\
grep -v "^\#" |\
cut -d';' -f3 || true)
[ -z "$linked_issue_urls" ] && {
echo "::error::No linked issues for PR $pr"
exit 1
}
has_bug=false
for issue_url in $(echo "$linked_issue_urls")
do
issue=$(echo "$issue_url"| awk -F\/ '{print $NF}' || true)
[ -z "$issue" ] && {
echo "::error::Cannot determine issue number from $issue_url for PR $pr"
exit 1
}
labels=$(hub-util.sh list-labels-for-issue "$issue")
label_names=$(echo $labels | jq -r '.[].name' || true)
if [[ "$label_names" =~ "bug" ]]; then
has_bug=true
break
fi
done
has_backport_needed_label=${{ contains(github.event.pull_request.labels.*.name, 'needs-backport') }}
has_no_backport_needed_label=${{ contains(github.event.pull_request.labels.*.name, 'no-backport-needed') }}
echo "add_backport_label=false" >> $GITHUB_OUTPUT
if [ $has_backport_needed_label = true ] || [ $has_bug = true ]; then
if [[ $has_no_backport_needed_label = false ]]; then
echo "add_backport_label=true" >> $GITHUB_OUTPUT
fi
fi
# Do not spam comment, only if auto-backport label is going to be newly added.
echo "auto_backport_added=$CONTAINS_AUTO_BACKPORT" >> $GITHUB_OUTPUT
- name: Add comment
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') && steps.add_label.outputs.add_backport_label == 'true' && steps.add_label.outputs.auto_backport_added == 'false' }}
uses: actions/github-script@v6
with:
script: |
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: 'This issue has been marked for auto-backporting. Add label(s) backport-to-BRANCHNAME to backport to them'
})
# Allow label to be removed by adding no-backport-needed label
- name: Remove auto-backport label
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') && steps.add_label.outputs.add_backport_label == 'false' }}
uses: andymckay/labeler@e6c4322d0397f3240f0e7e30a33b5c5df2d39e90
with:
remove-labels: "auto-backport"
repo-token: ${{ secrets.GITHUB_TOKEN }}
- name: Add auto-backport label
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') && steps.add_label.outputs.add_backport_label == 'true' }}
uses: andymckay/labeler@e6c4322d0397f3240f0e7e30a33b5c5df2d39e90
with:
add-labels: "auto-backport"
repo-token: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -17,7 +17,7 @@ concurrency:
jobs:
add-new-issues-to-backlog:
runs-on: ubuntu-22.04
runs-on: ubuntu-latest
steps:
- name: Install hub
run: |
@@ -39,7 +39,7 @@ jobs:
popd &>/dev/null
- name: Checkout code to allow hub to communicate with the project
uses: actions/checkout@v4
uses: actions/checkout@v2
- name: Add issue to issue backlog
env:

View File

@@ -18,19 +18,10 @@ concurrency:
jobs:
add-pr-size-label:
runs-on: ubuntu-22.04
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ github.event.pull_request.base.ref }}
uses: actions/checkout@v1
- name: Install PR sizing label script
run: |

33
.github/workflows/auto-backport.yaml vendored Normal file
View File

@@ -0,0 +1,33 @@
on:
pull_request_target:
types: ["labeled", "closed"]
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
backport:
name: Backport PR
runs-on: ubuntu-latest
if: |
github.event.pull_request.merged == true
&& contains(github.event.pull_request.labels.*.name, 'auto-backport')
&& (
(github.event.action == 'labeled' && github.event.label.name == 'auto-backport')
|| (github.event.action == 'closed')
)
steps:
- name: Backport Action
uses: sqren/backport-github-action@v8.9.2
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
auto_backport_label_prefix: backport-to-
- name: Info log
if: ${{ success() }}
run: cat /home/runner/.backport/backport.info.log
- name: Debug log
if: ${{ failure() }}
run: cat /home/runner/.backport/backport.debug.log

View File

@@ -1,380 +0,0 @@
name: CI | Basic amd64 tests
on:
workflow_call:
inputs:
tarball-suffix:
required: false
type: string
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
run-cri-containerd:
strategy:
# We can set this to true whenever we're 100% sure that
# the all the tests are not flaky, otherwise we'll fail
# all the tests due to a single flaky instance.
fail-fast: false
matrix:
containerd_version: ['lts', 'active']
vmm: ['clh', 'dragonball', 'qemu', 'stratovirt', 'cloud-hypervisor', 'qemu-runtime-rs']
runs-on: ubuntu-22.04
env:
CONTAINERD_VERSION: ${{ matrix.containerd_version }}
GOPATH: ${{ github.workspace }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install dependencies
run: bash tests/integration/cri-containerd/gha-run.sh install-dependencies
- name: get-kata-tarball
uses: actions/download-artifact@v4
with:
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-artifacts
- name: Install kata
run: bash tests/integration/cri-containerd/gha-run.sh install-kata kata-artifacts
- name: Run cri-containerd tests
timeout-minutes: 10
run: bash tests/integration/cri-containerd/gha-run.sh run
run-containerd-stability:
strategy:
fail-fast: false
matrix:
containerd_version: ['lts', 'active']
vmm: ['clh', 'cloud-hypervisor', 'dragonball', 'qemu', 'stratovirt']
runs-on: ubuntu-22.04
env:
CONTAINERD_VERSION: ${{ matrix.containerd_version }}
GOPATH: ${{ github.workspace }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install dependencies
run: bash tests/stability/gha-run.sh install-dependencies
- name: get-kata-tarball
uses: actions/download-artifact@v4
with:
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-artifacts
- name: Install kata
run: bash tests/stability/gha-run.sh install-kata kata-artifacts
- name: Run containerd-stability tests
timeout-minutes: 15
run: bash tests/stability/gha-run.sh run
run-nydus:
strategy:
# We can set this to true whenever we're 100% sure that
# the all the tests are not flaky, otherwise we'll fail
# all the tests due to a single flaky instance.
fail-fast: false
matrix:
containerd_version: ['lts', 'active']
vmm: ['clh', 'qemu', 'dragonball', 'stratovirt']
runs-on: ubuntu-22.04
env:
CONTAINERD_VERSION: ${{ matrix.containerd_version }}
GOPATH: ${{ github.workspace }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install dependencies
run: bash tests/integration/nydus/gha-run.sh install-dependencies
- name: get-kata-tarball
uses: actions/download-artifact@v4
with:
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-artifacts
- name: Install kata
run: bash tests/integration/nydus/gha-run.sh install-kata kata-artifacts
- name: Run nydus tests
timeout-minutes: 10
run: bash tests/integration/nydus/gha-run.sh run
run-runk:
runs-on: ubuntu-22.04
env:
CONTAINERD_VERSION: lts
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install dependencies
run: bash tests/integration/runk/gha-run.sh install-dependencies
- name: get-kata-tarball
uses: actions/download-artifact@v4
with:
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-artifacts
- name: Install kata
run: bash tests/integration/runk/gha-run.sh install-kata kata-artifacts
- name: Run runk tests
timeout-minutes: 10
run: bash tests/integration/runk/gha-run.sh run
run-tracing:
strategy:
fail-fast: false
matrix:
vmm:
- clh # cloud-hypervisor
- qemu
# TODO: enable me when https://github.com/kata-containers/kata-containers/issues/9763 is fixed
# TODO: Transition to free runner (see #9940).
if: false
runs-on: garm-ubuntu-2204-smaller
env:
KATA_HYPERVISOR: ${{ matrix.vmm }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install dependencies
run: bash tests/functional/tracing/gha-run.sh install-dependencies
- name: get-kata-tarball
uses: actions/download-artifact@v4
with:
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-artifacts
- name: Install kata
run: bash tests/functional/tracing/gha-run.sh install-kata kata-artifacts
- name: Run tracing tests
timeout-minutes: 15
run: bash tests/functional/tracing/gha-run.sh run
run-vfio:
strategy:
fail-fast: false
matrix:
vmm:
- clh
- qemu
# TODO: enable with clh when https://github.com/kata-containers/kata-containers/issues/9764 is fixed
# TODO: enable with qemu when https://github.com/kata-containers/kata-containers/issues/9851 is fixed
# TODO: Transition to free runner (see #9940).
if: false
runs-on: garm-ubuntu-2304
env:
GOPATH: ${{ github.workspace }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install dependencies
run: bash tests/functional/vfio/gha-run.sh install-dependencies
- name: get-kata-tarball
uses: actions/download-artifact@v4
with:
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-artifacts
- name: Run vfio tests
timeout-minutes: 15
run: bash tests/functional/vfio/gha-run.sh run
run-docker-tests:
strategy:
# We can set this to true whenever we're 100% sure that
# all the tests are not flaky, otherwise we'll fail them
# all due to a single flaky instance.
fail-fast: false
matrix:
vmm:
- clh
- qemu
- dragonball
- cloud-hypervisor
runs-on: ubuntu-22.04
env:
KATA_HYPERVISOR: ${{ matrix.vmm }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install dependencies
run: bash tests/integration/docker/gha-run.sh install-dependencies
- name: get-kata-tarball
uses: actions/download-artifact@v4
with:
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-artifacts
- name: Install kata
run: bash tests/integration/docker/gha-run.sh install-kata kata-artifacts
- name: Run docker smoke test
timeout-minutes: 5
run: bash tests/integration/docker/gha-run.sh run
run-nerdctl-tests:
strategy:
# We can set this to true whenever we're 100% sure that
# all the tests are not flaky, otherwise we'll fail them
# all due to a single flaky instance.
fail-fast: false
matrix:
vmm:
- clh
- dragonball
- qemu
- cloud-hypervisor
runs-on: ubuntu-22.04
env:
KATA_HYPERVISOR: ${{ matrix.vmm }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install dependencies
run: bash tests/integration/nerdctl/gha-run.sh install-dependencies
- name: get-kata-tarball
uses: actions/download-artifact@v4
with:
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-artifacts
- name: Install kata
run: bash tests/integration/nerdctl/gha-run.sh install-kata kata-artifacts
- name: Run nerdctl smoke test
timeout-minutes: 5
run: bash tests/integration/nerdctl/gha-run.sh run
- name: Collect artifacts ${{ matrix.vmm }}
if: always()
run: bash tests/integration/nerdctl/gha-run.sh collect-artifacts
continue-on-error: true
- name: Archive artifacts ${{ matrix.vmm }}
uses: actions/upload-artifact@v4
with:
name: nerdctl-tests-garm-${{ matrix.vmm }}
path: /tmp/artifacts
retention-days: 1
run-kata-agent-apis:
strategy:
fail-fast: false
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install dependencies
run: bash tests/functional/kata-agent-apis/gha-run.sh install-dependencies
- name: get-kata-tarball
uses: actions/download-artifact@v4
with:
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-artifacts
- name: Install kata
run: bash tests/functional/kata-agent-apis/gha-run.sh install-kata kata-artifacts
- name: Run kata agent api tests with agent-ctl
run: bash tests/functional/kata-agent-apis/gha-run.sh run

View File

@@ -1,113 +0,0 @@
on:
workflow_call:
inputs:
instance:
required: true
type: string
name: Build checks
jobs:
check:
runs-on: ${{ inputs.instance }}
strategy:
fail-fast: false
matrix:
component:
- agent
- dragonball
- runtime
- runtime-rs
- agent-ctl
- kata-ctl
- runk
- trace-forwarder
- genpolicy
command:
- "make vendor"
- "make check"
- "make test"
- "sudo -E PATH=\"$PATH\" make test"
include:
- component: agent
component-path: src/agent
- component: dragonball
component-path: src/dragonball
- component: runtime
component-path: src/runtime
- component: runtime-rs
component-path: src/runtime-rs
- component: agent-ctl
component-path: src/tools/agent-ctl
- component: kata-ctl
component-path: src/tools/kata-ctl
- component: runk
component-path: src/tools/runk
- component: trace-forwarder
component-path: src/tools/trace-forwarder
- install-libseccomp: no
- component: agent
install-libseccomp: yes
- component: runk
install-libseccomp: yes
- component: genpolicy
component-path: src/tools/genpolicy
steps:
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE $HOME
sudo rm -rf $GITHUB_WORKSPACE/* && echo "GITHUB_WORKSPACE removed" || { sleep 10 && sudo rm -rf $GITHUB_WORKSPACE/*; }
sudo rm -f /tmp/kata_hybrid* # Sometime we got leftover from test_setup_hvsock_failed()
- name: Checkout the code
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install yq
run: |
./ci/install_yq.sh
env:
INSTALL_IN_GOPATH: false
- name: Install golang
if: ${{ matrix.component == 'runtime' }}
run: |
./tests/install_go.sh -f -p
echo "/usr/local/go/bin" >> $GITHUB_PATH
- name: Install rust
if: ${{ matrix.component != 'runtime' }}
run: |
./tests/install_rust.sh
echo "${HOME}/.cargo/bin" >> $GITHUB_PATH
- name: Install musl-tools
if: ${{ matrix.component != 'runtime' }}
run: sudo apt-get -y install musl-tools
- name: Install devicemapper
if: ${{ matrix.command == 'make check' && matrix.component == 'agent' }}
run: sudo apt-get -y install libdevmapper-dev
- name: Install libseccomp
if: ${{ matrix.command != 'make vendor' && matrix.command != 'make check' && matrix.install-libseccomp == 'yes' }}
run: |
libseccomp_install_dir=$(mktemp -d -t libseccomp.XXXXXXXXXX)
gperf_install_dir=$(mktemp -d -t gperf.XXXXXXXXXX)
./ci/install_libseccomp.sh "${libseccomp_install_dir}" "${gperf_install_dir}"
echo "Set environment variables for the libseccomp crate to link the libseccomp library statically"
echo "LIBSECCOMP_LINK_TYPE=static" >> $GITHUB_ENV
echo "LIBSECCOMP_LIB_PATH=${libseccomp_install_dir}/lib" >> $GITHUB_ENV
- name: Install protobuf-compiler
if: ${{ matrix.command != 'make vendor' && (matrix.component == 'agent' || matrix.component == 'runk' || matrix.component == 'genpolicy' || matrix.component == 'agent-ctl') }}
run: sudo apt-get -y install protobuf-compiler
- name: Install clang
if: ${{ matrix.command == 'make check' && (matrix.component == 'agent' || matrix.component == 'agent-ctl') }}
run: sudo apt-get -y install clang
- name: Setup XDG_RUNTIME_DIR for the `runtime` tests
if: ${{ matrix.command != 'make vendor' && matrix.command != 'make check' && matrix.component == 'runtime' }}
run: |
XDG_RUNTIME_DIR=$(mktemp -d /tmp/kata-tests-$USER.XXX | tee >(xargs chmod 0700))
echo "XDG_RUNTIME_DIR=${XDG_RUNTIME_DIR}" >> $GITHUB_ENV
- name: Running `${{ matrix.command }}` for ${{ matrix.component }}
run: |
cd ${{ matrix.component-path }}
${{ matrix.command }}
env:
RUST_BACKTRACE: "1"
SKIP_GO_VERSION_CHECK: "1"

View File

@@ -16,206 +16,94 @@ on:
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
build-asset:
runs-on: ubuntu-22.04
permissions:
contents: read
packages: write
id-token: write
attestations: write
runs-on: ubuntu-latest
strategy:
matrix:
asset:
- agent
- agent-ctl
- cloud-hypervisor
- cloud-hypervisor-glibc
- coco-guest-components
- firecracker
- genpolicy
- kata-ctl
- kata-manager
- kernel
- kernel-confidential
- kernel-sev
- kernel-dragonball-experimental
- kernel-tdx-experimental
- kernel-nvidia-gpu
- kernel-nvidia-gpu-confidential
- kernel-nvidia-gpu-snp
- kernel-nvidia-gpu-tdx-experimental
- nydus
- ovmf
- ovmf-sev
- pause-image
- qemu
- qemu-snp-experimental
- stratovirt
- qemu-tdx-experimental
- rootfs-image
- rootfs-image-confidential
- rootfs-image-mariner
- rootfs-image-tdx
- rootfs-initrd
- rootfs-initrd-confidential
- runk
- trace-forwarder
- rootfs-initrd-mariner
- rootfs-initrd-sev
- shim-v2
- tdvf
- virtiofsd
stage:
- ${{ inputs.stage }}
exclude:
- asset: cloud-hypervisor-glibc
stage: release
env:
PERFORM_ATTESTATION: ${{ matrix.asset == 'agent' && inputs.push-to-registry == 'yes' && 'yes' || 'no' }}
steps:
- name: Login to Kata Containers quay.io
if: ${{ inputs.push-to-registry == 'yes' }}
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- uses: actions/checkout@v4
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0 # This is needed in order to keep the commit ids history
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Build ${{ matrix.asset }}
id: build
run: |
make "${KATA_ASSET}-tarball"
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
mkdir -p kata-build && cp "${build_dir}"/kata-static-${KATA_ASSET}*.tar.* kata-build/.
sudo cp -r "${build_dir}" "kata-build"
env:
KATA_ASSET: ${{ matrix.asset }}
TAR_OUTPUT: ${{ matrix.asset }}.tar.gz
PUSH_TO_REGISTRY: ${{ inputs.push-to-registry }}
ARTEFACT_REGISTRY: ghcr.io
ARTEFACT_REGISTRY_USERNAME: ${{ github.actor }}
ARTEFACT_REGISTRY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}
TARGET_BRANCH: ${{ inputs.target-branch }}
RELEASE: ${{ inputs.stage == 'release' && 'yes' || 'no' }}
- name: Parse OCI image name and digest
id: parse-oci-segments
if: ${{ env.PERFORM_ATTESTATION == 'yes' }}
run: |
oci_image="$(<"build/${{ matrix.asset }}-oci-image")"
echo "oci-name=${oci_image%@*}" >> "$GITHUB_OUTPUT"
echo "oci-digest=${oci_image#*@}" >> "$GITHUB_OUTPUT"
- uses: oras-project/setup-oras@v1
if: ${{ env.PERFORM_ATTESTATION == 'yes' }}
with:
version: "1.2.0"
# for pushing attestations to the registry
- uses: docker/login-action@v3
if: ${{ env.PERFORM_ATTESTATION == 'yes' }}
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: actions/attest-build-provenance@v1
if: ${{ env.PERFORM_ATTESTATION == 'yes' }}
with:
subject-name: ${{ steps.parse-oci-segments.outputs.oci-name }}
subject-digest: ${{ steps.parse-oci-segments.outputs.oci-digest }}
push-to-registry: true
- name: store-artifact ${{ matrix.asset }}
if: ${{ matrix.stage != 'release' || (matrix.asset != 'agent' && matrix.asset != 'coco-guest-components' && matrix.asset != 'pause-image') }}
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v3
with:
name: kata-artifacts-amd64-${{ matrix.asset }}${{ inputs.tarball-suffix }}
name: kata-artifacts-amd64${{ inputs.tarball-suffix }}
path: kata-build/kata-static-${{ matrix.asset }}.tar.xz
retention-days: 15
if-no-files-found: error
build-asset-shim-v2:
runs-on: ubuntu-22.04
needs: build-asset
steps:
- name: Login to Kata Containers quay.io
if: ${{ inputs.push-to-registry == 'yes' }}
uses: docker/login-action@v3
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0 # This is needed in order to keep the commit ids history
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Build shim-v2
id: build
run: |
make "${KATA_ASSET}-tarball"
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
mkdir -p kata-build && cp "${build_dir}"/kata-static-${KATA_ASSET}*.tar.* kata-build/.
env:
KATA_ASSET: shim-v2
TAR_OUTPUT: shim-v2.tar.gz
PUSH_TO_REGISTRY: ${{ inputs.push-to-registry }}
ARTEFACT_REGISTRY: ghcr.io
ARTEFACT_REGISTRY_USERNAME: ${{ github.actor }}
ARTEFACT_REGISTRY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}
TARGET_BRANCH: ${{ inputs.target-branch }}
RELEASE: ${{ inputs.stage == 'release' && 'yes' || 'no' }}
- name: store-artifact shim-v2
uses: actions/upload-artifact@v4
with:
name: kata-artifacts-amd64-shim-v2${{ inputs.tarball-suffix }}
path: kata-build/kata-static-shim-v2.tar.xz
retention-days: 15
retention-days: 1
if-no-files-found: error
create-kata-tarball:
runs-on: ubuntu-22.04
needs: [build-asset, build-asset-shim-v2]
runs-on: ubuntu-latest
needs: build-asset
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: get-artifacts
uses: actions/download-artifact@v4
uses: actions/download-artifact@v3
with:
pattern: kata-artifacts-amd64-*${{ inputs.tarball-suffix }}
name: kata-artifacts-amd64${{ inputs.tarball-suffix }}
path: kata-artifacts
merge-multiple: true
- name: merge-artifacts
run: |
./tools/packaging/kata-deploy/local-build/kata-deploy-merge-builds.sh kata-artifacts versions.yaml
- name: store-artifacts
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v3
with:
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-static.tar.xz
retention-days: 15
retention-days: 1
if-no-files-found: error

View File

@@ -16,150 +16,84 @@ on:
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
build-asset:
runs-on: arm64-builder
runs-on: arm64
strategy:
matrix:
asset:
- agent
- cloud-hypervisor
- firecracker
- kernel
- kernel-dragonball-experimental
- nydus
- qemu
- stratovirt
- rootfs-image
- rootfs-initrd
- shim-v2
- virtiofsd
steps:
- name: Login to Kata Containers quay.io
if: ${{ inputs.push-to-registry == 'yes' }}
uses: docker/login-action@v3
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0 # This is needed in order to keep the commit ids history
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Build ${{ matrix.asset }}
run: |
make "${KATA_ASSET}-tarball"
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
mkdir -p kata-build && cp "${build_dir}"/kata-static-${KATA_ASSET}*.tar.* kata-build/.
env:
KATA_ASSET: ${{ matrix.asset }}
TAR_OUTPUT: ${{ matrix.asset }}.tar.gz
PUSH_TO_REGISTRY: ${{ inputs.push-to-registry }}
ARTEFACT_REGISTRY: ghcr.io
ARTEFACT_REGISTRY_USERNAME: ${{ github.actor }}
ARTEFACT_REGISTRY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}
TARGET_BRANCH: ${{ inputs.target-branch }}
RELEASE: ${{ inputs.stage == 'release' && 'yes' || 'no' }}
- name: store-artifact ${{ matrix.asset }}
if: ${{ inputs.stage != 'release' || matrix.asset != 'agent' }}
uses: actions/upload-artifact@v4
with:
name: kata-artifacts-arm64-${{ matrix.asset }}${{ inputs.tarball-suffix }}
path: kata-build/kata-static-${{ matrix.asset }}.tar.xz
retention-days: 15
if-no-files-found: error
build-asset-shim-v2:
runs-on: arm64-builder
needs: build-asset
steps:
- name: Login to Kata Containers quay.io
if: ${{ inputs.push-to-registry == 'yes' }}
uses: docker/login-action@v3
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0 # This is needed in order to keep the commit ids history
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Build shim-v2
run: |
make "${KATA_ASSET}-tarball"
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
mkdir -p kata-build && cp "${build_dir}"/kata-static-${KATA_ASSET}*.tar.* kata-build/.
env:
KATA_ASSET: shim-v2
TAR_OUTPUT: shim-v2.tar.gz
PUSH_TO_REGISTRY: ${{ inputs.push-to-registry }}
ARTEFACT_REGISTRY: ghcr.io
ARTEFACT_REGISTRY_USERNAME: ${{ github.actor }}
ARTEFACT_REGISTRY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}
TARGET_BRANCH: ${{ inputs.target-branch }}
RELEASE: ${{ inputs.stage == 'release' && 'yes' || 'no' }}
- name: store-artifact shim-v2
uses: actions/upload-artifact@v4
with:
name: kata-artifacts-arm64-shim-v2${{ inputs.tarball-suffix }}
path: kata-build/kata-static-shim-v2.tar.xz
retention-days: 15
if-no-files-found: error
create-kata-tarball:
runs-on: arm64-builder
needs: [build-asset, build-asset-shim-v2]
stage:
- ${{ inputs.stage }}
steps:
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- uses: actions/checkout@v4
- name: Login to Kata Containers quay.io
if: ${{ inputs.push-to-registry == 'yes' }}
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
fetch-depth: 0 # This is needed in order to keep the commit ids history
- name: Build ${{ matrix.asset }}
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
make "${KATA_ASSET}-tarball"
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
sudo cp -r "${build_dir}" "kata-build"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: get-artifacts
uses: actions/download-artifact@v4
KATA_ASSET: ${{ matrix.asset }}
TAR_OUTPUT: ${{ matrix.asset }}.tar.gz
PUSH_TO_REGISTRY: ${{ inputs.push-to-registry }}
- name: store-artifact ${{ matrix.asset }}
uses: actions/upload-artifact@v3
with:
pattern: kata-artifacts-arm64-*${{ inputs.tarball-suffix }}
name: kata-artifacts-arm64${{ inputs.tarball-suffix }}
path: kata-build/kata-static-${{ matrix.asset }}.tar.xz
retention-days: 1
if-no-files-found: error
create-kata-tarball:
runs-on: arm64
needs: build-asset
steps:
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
- name: get-artifacts
uses: actions/download-artifact@v3
with:
name: kata-artifacts-arm64${{ inputs.tarball-suffix }}
path: kata-artifacts
merge-multiple: true
- name: merge-artifacts
run: |
./tools/packaging/kata-deploy/local-build/kata-deploy-merge-builds.sh kata-artifacts versions.yaml
- name: store-artifacts
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v3
with:
name: kata-static-tarball-arm64${{ inputs.tarball-suffix }}
path: kata-static.tar.xz
retention-days: 15
retention-days: 1
if-no-files-found: error

View File

@@ -1,171 +0,0 @@
name: CI | Build kata-static tarball for ppc64le
on:
workflow_call:
inputs:
stage:
required: false
type: string
default: test
tarball-suffix:
required: false
type: string
push-to-registry:
required: false
type: string
default: no
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
build-asset:
runs-on: ppc64le
strategy:
matrix:
asset:
- agent
- kernel
- qemu
- rootfs-initrd
- virtiofsd
stage:
- ${{ inputs.stage }}
steps:
- name: Prepare the self-hosted runner
run: |
${HOME}/scripts/prepare_runner.sh
sudo rm -rf $GITHUB_WORKSPACE/*
- name: Login to Kata Containers quay.io
if: ${{ inputs.push-to-registry == 'yes' }}
uses: docker/login-action@v3
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0 # This is needed in order to keep the commit ids history
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Build ${{ matrix.asset }}
run: |
make "${KATA_ASSET}-tarball"
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
mkdir -p kata-build && cp "${build_dir}"/kata-static-${KATA_ASSET}*.tar.* kata-build/.
env:
KATA_ASSET: ${{ matrix.asset }}
TAR_OUTPUT: ${{ matrix.asset }}.tar.gz
PUSH_TO_REGISTRY: ${{ inputs.push-to-registry }}
ARTEFACT_REGISTRY: ghcr.io
ARTEFACT_REGISTRY_USERNAME: ${{ github.actor }}
ARTEFACT_REGISTRY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}
TARGET_BRANCH: ${{ inputs.target-branch }}
RELEASE: ${{ inputs.stage == 'release' && 'yes' || 'no' }}
- name: store-artifact ${{ matrix.asset }}
if: ${{ inputs.stage != 'release' || matrix.asset != 'agent' }}
uses: actions/upload-artifact@v4
with:
name: kata-artifacts-ppc64le-${{ matrix.asset }}${{ inputs.tarball-suffix }}
path: kata-build/kata-static-${{ matrix.asset }}.tar.xz
retention-days: 1
if-no-files-found: error
build-asset-shim-v2:
runs-on: ppc64le
needs: build-asset
steps:
- name: Prepare the self-hosted runner
run: |
${HOME}/scripts/prepare_runner.sh
sudo rm -rf $GITHUB_WORKSPACE/*
- name: Login to Kata Containers quay.io
if: ${{ inputs.push-to-registry == 'yes' }}
uses: docker/login-action@v3
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0 # This is needed in order to keep the commit ids history
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Build shim-v2
run: |
make "${KATA_ASSET}-tarball"
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
mkdir -p kata-build && cp "${build_dir}"/kata-static-${KATA_ASSET}*.tar.* kata-build/.
env:
KATA_ASSET: shim-v2
TAR_OUTPUT: shim-v2.tar.gz
PUSH_TO_REGISTRY: ${{ inputs.push-to-registry }}
ARTEFACT_REGISTRY: ghcr.io
ARTEFACT_REGISTRY_USERNAME: ${{ github.actor }}
ARTEFACT_REGISTRY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}
TARGET_BRANCH: ${{ inputs.target-branch }}
RELEASE: ${{ inputs.stage == 'release' && 'yes' || 'no' }}
- name: store-artifact shim-v2
uses: actions/upload-artifact@v4
with:
name: kata-artifacts-ppc64le-shim-v2${{ inputs.tarball-suffix }}
path: kata-build/kata-static-shim-v2.tar.xz
retention-days: 1
if-no-files-found: error
create-kata-tarball:
runs-on: ppc64le
needs: [build-asset, build-asset-shim-v2]
steps:
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: get-artifacts
uses: actions/download-artifact@v4
with:
pattern: kata-artifacts-ppc64le-*${{ inputs.tarball-suffix }}
path: kata-artifacts
merge-multiple: true
- name: merge-artifacts
run: |
./tools/packaging/kata-deploy/local-build/kata-deploy-merge-builds.sh kata-artifacts versions.yaml
- name: store-artifacts
uses: actions/upload-artifact@v4
with:
name: kata-static-tarball-ppc64le${{ inputs.tarball-suffix }}
path: kata-static.tar.xz
retention-days: 1
if-no-files-found: error

View File

@@ -16,226 +16,81 @@ on:
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
build-asset:
runs-on: s390x
permissions:
contents: read
packages: write
id-token: write
attestations: write
strategy:
matrix:
asset:
- agent
- coco-guest-components
- kernel
- kernel-confidential
- pause-image
- qemu
- rootfs-image
- rootfs-image-confidential
- rootfs-initrd
- rootfs-initrd-confidential
- shim-v2
- virtiofsd
env:
PERFORM_ATTESTATION: ${{ matrix.asset == 'agent' && inputs.push-to-registry == 'yes' && 'yes' || 'no' }}
stage:
- ${{ inputs.stage }}
steps:
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- name: Login to Kata Containers quay.io
if: ${{ inputs.push-to-registry == 'yes' }}
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- uses: actions/checkout@v4
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0 # This is needed in order to keep the commit ids history
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Build ${{ matrix.asset }}
id: build
run: |
make "${KATA_ASSET}-tarball"
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
mkdir -p kata-build && cp "${build_dir}"/kata-static-${KATA_ASSET}*.tar.* kata-build/.
sudo cp -r "${build_dir}" "kata-build"
sudo chown -R $(id -u):$(id -g) "kata-build"
env:
KATA_ASSET: ${{ matrix.asset }}
TAR_OUTPUT: ${{ matrix.asset }}.tar.gz
PUSH_TO_REGISTRY: ${{ inputs.push-to-registry }}
ARTEFACT_REGISTRY: ghcr.io
ARTEFACT_REGISTRY_USERNAME: ${{ github.actor }}
ARTEFACT_REGISTRY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}
TARGET_BRANCH: ${{ inputs.target-branch }}
RELEASE: ${{ inputs.stage == 'release' && 'yes' || 'no' }}
- name: Parse OCI image name and digest
id: parse-oci-segments
if: ${{ env.PERFORM_ATTESTATION == 'yes' }}
run: |
oci_image="$(<"build/${{ matrix.asset }}-oci-image")"
echo "oci-name=${oci_image%@*}" >> "$GITHUB_OUTPUT"
echo "oci-digest=${oci_image#*@}" >> "$GITHUB_OUTPUT"
# for pushing attestations to the registry
- uses: docker/login-action@v3
if: ${{ env.PERFORM_ATTESTATION == 'yes' }}
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: actions/attest-build-provenance@v1
if: ${{ env.PERFORM_ATTESTATION == 'yes' }}
with:
subject-name: ${{ steps.parse-oci-segments.outputs.oci-name }}
subject-digest: ${{ steps.parse-oci-segments.outputs.oci-digest }}
push-to-registry: true
- name: store-artifact ${{ matrix.asset }}
if: ${{ inputs.stage != 'release' || (matrix.asset != 'agent' && matrix.asset != 'coco-guest-components' && matrix.asset != 'pause-image') }}
uses: actions/upload-artifact@v4
with:
name: kata-artifacts-s390x-${{ matrix.asset }}${{ inputs.tarball-suffix }}
path: kata-build/kata-static-${{ matrix.asset }}.tar.xz
retention-days: 15
if-no-files-found: error
build-asset-boot-image-se:
runs-on: s390x
needs: build-asset
steps:
- uses: actions/checkout@v4
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: get-artifacts
uses: actions/download-artifact@v4
with:
pattern: kata-artifacts-s390x-*${{ inputs.tarball-suffix }}
path: kata-artifacts
merge-multiple: true
- name: Place a host key document
run: |
mkdir -p "host-key-document"
cp "${CI_HKD_PATH}" "host-key-document"
env:
CI_HKD_PATH: ${{ secrets.CI_HKD_PATH }}
- name: Build boot-image-se
run: |
base_dir=tools/packaging/kata-deploy/local-build/
cp -r kata-artifacts ${base_dir}/build
# Skip building dependant artifacts of boot-image-se-tarball
# because we already have them from the previous build
sed -i 's/\(^boot-image-se-tarball:\).*/\1/g' ${base_dir}/Makefile
make boot-image-se-tarball
build_dir=$(readlink -f build)
sudo cp -r "${build_dir}" "kata-build"
sudo chown -R $(id -u):$(id -g) "kata-build"
env:
HKD_PATH: "host-key-document"
- name: store-artifact boot-image-se
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v3
with:
name: kata-artifacts-s390x${{ inputs.tarball-suffix }}
path: kata-build/kata-static-boot-image-se.tar.xz
path: kata-build/kata-static-${{ matrix.asset }}.tar.xz
retention-days: 1
if-no-files-found: error
build-asset-shim-v2:
runs-on: s390x
needs: build-asset
steps:
- name: Login to Kata Containers quay.io
if: ${{ inputs.push-to-registry == 'yes' }}
uses: docker/login-action@v3
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0 # This is needed in order to keep the commit ids history
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Build shim-v2
id: build
run: |
make "${KATA_ASSET}-tarball"
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
mkdir -p kata-build && cp "${build_dir}"/kata-static-${KATA_ASSET}*.tar.* kata-build/.
env:
KATA_ASSET: shim-v2
TAR_OUTPUT: shim-v2.tar.gz
PUSH_TO_REGISTRY: ${{ inputs.push-to-registry }}
ARTEFACT_REGISTRY: ghcr.io
ARTEFACT_REGISTRY_USERNAME: ${{ github.actor }}
ARTEFACT_REGISTRY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}
TARGET_BRANCH: ${{ inputs.target-branch }}
RELEASE: ${{ inputs.stage == 'release' && 'yes' || 'no' }}
- name: store-artifact shim-v2
uses: actions/upload-artifact@v4
with:
name: kata-artifacts-s390x-shim-v2${{ inputs.tarball-suffix }}
path: kata-build/kata-static-shim-v2.tar.xz
retention-days: 15
if-no-files-found: error
create-kata-tarball:
runs-on: s390x
needs: [build-asset, build-asset-boot-image-se, build-asset-shim-v2]
needs: build-asset
steps:
- uses: actions/checkout@v4
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: get-artifacts
uses: actions/download-artifact@v4
uses: actions/download-artifact@v3
with:
pattern: kata-artifacts-s390x-*${{ inputs.tarball-suffix }}
name: kata-artifacts-s390x${{ inputs.tarball-suffix }}
path: kata-artifacts
merge-multiple: true
- name: merge-artifacts
run: |
./tools/packaging/kata-deploy/local-build/kata-deploy-merge-builds.sh kata-artifacts versions.yaml
- name: store-artifacts
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v3
with:
name: kata-static-tarball-s390x${{ inputs.tarball-suffix }}
path: kata-static.tar.xz
retention-days: 15
retention-days: 1
if-no-files-found: error

View File

@@ -6,6 +6,7 @@ on:
- edited
- reopened
- synchronize
paths-ignore: [ '**.md', '**.png', '**.jpg', '**.jpeg', '**.svg', '/docs/**' ]
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
@@ -13,12 +14,12 @@ concurrency:
jobs:
cargo-deny-runner:
runs-on: ubuntu-22.04
runs-on: ubuntu-latest
steps:
- name: Checkout Code
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
uses: actions/checkout@v4
uses: actions/checkout@v3
- name: Generate Action
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
run: bash cargo-deny-generator.sh

View File

@@ -0,0 +1,170 @@
name: CI | Publish CC runtime payload for amd64
on:
workflow_call:
inputs:
target-arch:
required: true
type: string
jobs:
build-asset:
runs-on: ubuntu-latest
strategy:
matrix:
measured_rootfs:
- no
asset:
- cloud-hypervisor
- qemu
- virtiofsd
- kernel-sev
- ovmf-sev
- ovmf
- qemu-snp-experimental
- qemu-tdx-experimental
- rootfs-initrd-sev
- cc-tdx-td-shim
- tdvf
include:
- measured_rootfs: yes
asset: kernel
- measured_rootfs: yes
asset: kernel-tdx-experimental
- measured_rootfs: yes
asset: cc-rootfs-image
- measured_rootfs: yes
asset: rootfs-image-tdx
steps:
- name: Login to Kata Containers quay.io
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- uses: actions/checkout@v3
with:
fetch-depth: 0 # This is needed in order to keep the commit ids history
- name: Build ${{ matrix.asset }}
run: |
USE_CACHE="no" make "${KATA_ASSET}-tarball"
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
sudo cp -r "${build_dir}" "kata-build"
env:
KATA_ASSET: ${{ matrix.asset }}
TAR_OUTPUT: ${{ matrix.asset }}.tar.gz
PUSH_TO_REGISTRY: yes
MEASURED_ROOTFS: ${{ matrix.measured_rootfs }}
- name: store-artifact ${{ matrix.asset }}
uses: actions/upload-artifact@v3
with:
name: kata-artifacts
path: kata-build/kata-static-${{ matrix.asset }}.tar.xz
retention-days: 1
if-no-files-found: error
- name: store-artifact root_hash_tdx.txt
uses: actions/upload-artifact@v3
with:
name: root_hash_tdx.txt
path: tools/osbuilder/root_hash_tdx.txt
retention-days: 1
if-no-files-found: ignore
- name: store-artifact root_hash_vanilla.txt
uses: actions/upload-artifact@v3
with:
name: root_hash_vanilla.txt
path: tools/osbuilder/root_hash_vanilla.txt
retention-days: 1
if-no-files-found: ignore
build-asset-cc-shim-v2:
runs-on: ubuntu-latest
needs: build-asset
steps:
- name: Login to Kata Containers quay.io
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- uses: actions/checkout@v3
- name: Get root_hash_tdx.txt
uses: actions/download-artifact@v3
with:
name: root_hash_tdx.txt
path: tools/osbuilder/
- name: Get root_hash_vanilla.txt
uses: actions/download-artifact@v3
with:
name: root_hash_vanilla.txt
path: tools/osbuilder/
- name: Build cc-shim-v2
run: |
USE_CACHE="no" make cc-shim-v2-tarball
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
sudo cp -r "${build_dir}" "kata-build"
env:
PUSH_TO_REGISTRY: yes
MEASURED_ROOTFS: yes
- name: store-artifact cc-shim-v2
uses: actions/upload-artifact@v3
with:
name: kata-artifacts
path: kata-build/kata-static-cc-shim-v2.tar.xz
retention-days: 1
if-no-files-found: error
create-kata-tarball:
runs-on: ubuntu-latest
needs: [build-asset, build-asset-cc-shim-v2]
steps:
- uses: actions/checkout@v3
- name: get-artifacts
uses: actions/download-artifact@v3
with:
name: kata-artifacts
path: kata-artifacts
- name: merge-artifacts
run: |
./tools/packaging/kata-deploy/local-build/kata-deploy-merge-builds.sh kata-artifacts
- name: store-artifacts
uses: actions/upload-artifact@v3
with:
name: kata-static-tarball
path: kata-static.tar.xz
retention-days: 1
if-no-files-found: error
kata-payload:
needs: create-kata-tarball
runs-on: ubuntu-latest
steps:
- name: Login to Confidential Containers quay.io
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.COCO_QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.COCO_QUAY_DEPLOYER_PASSWORD }}
- uses: actions/checkout@v3
- name: get-kata-tarball
uses: actions/download-artifact@v3
with:
name: kata-static-tarball
- name: build-and-push-kata-payload
id: build-and-push-kata-payload
run: |
./tools/packaging/kata-deploy/local-build/kata-deploy-build-and-upload-payload.sh \
$(pwd)/kata-static.tar.xz "quay.io/confidential-containers/runtime-payload-ci" \
"kata-containers-${{ inputs.target-arch }}"

View File

@@ -0,0 +1,207 @@
name: CI | Publish CC runtime payload for s390x
on:
workflow_call:
inputs:
target-arch:
required: true
type: string
jobs:
build-asset:
runs-on: s390x
strategy:
matrix:
measured_rootfs:
- no
asset:
- qemu
- cc-rootfs-initrd
- virtiofsd
include:
- measured_rootfs: yes
asset: kernel
- measured_rootfs: yes
asset: cc-rootfs-image
steps:
- name: Login to Kata Containers quay.io
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- uses: actions/checkout@v3
with:
fetch-depth: 0 # This is needed in order to keep the commit ids history
- name: Build ${{ matrix.asset }}
run: |
USE_CACHE="no" make "${KATA_ASSET}-tarball"
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
sudo cp -r "${build_dir}" "kata-build"
sudo chown -R $(id -u):$(id -g) "kata-build"
env:
KATA_ASSET: ${{ matrix.asset }}
TAR_OUTPUT: ${{ matrix.asset }}.tar.gz
PUSH_TO_REGISTRY: yes
MEASURED_ROOTFS: ${{ matrix.measured_rootfs }}
- name: store-artifact ${{ matrix.asset }}
uses: actions/upload-artifact@v3
with:
name: kata-artifacts-s390x
path: kata-build/kata-static-${{ matrix.asset }}.tar.xz
retention-days: 1
if-no-files-found: error
- name: store-artifact root_hash_vanilla.txt
uses: actions/upload-artifact@v3
with:
name: root_hash_vanilla.txt-s390x
path: tools/osbuilder/root_hash_vanilla.txt
retention-days: 1
if-no-files-found: ignore
build-asset-cc-shim-v2:
runs-on: s390x
needs: build-asset
steps:
- name: Login to Kata Containers quay.io
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- uses: actions/checkout@v3
- name: Get root_hash_vanilla.txt
uses: actions/download-artifact@v3
with:
name: root_hash_vanilla.txt-s390x
path: tools/osbuilder/
- name: Build cc-shim-v2
run: |
USE_CACHE="no" make cc-shim-v2-tarball
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
sudo cp -r "${build_dir}" "kata-build"
env:
PUSH_TO_REGISTRY: yes
MEASURED_ROOTFS: yes
- name: store-artifact cc-shim-v2
uses: actions/upload-artifact@v3
with:
name: kata-artifacts-s390x
path: kata-build/kata-static-cc-shim-v2.tar.xz
retention-days: 1
if-no-files-found: error
build-asset-cc-se-image:
runs-on: s390x
needs: build-asset
steps:
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- uses: actions/checkout@v3
- name: get-artifacts
uses: actions/download-artifact@v3
with:
name: kata-artifacts-s390x
path: kata-artifacts
- name: Place a host key document
run: |
mkdir -p "host-key-document"
cp "${CI_HKD_PATH}" "host-key-document"
env:
CI_HKD_PATH: ${{ secrets.CI_HKD_PATH }}
- name: Build cc-se-image
run: |
base_dir=tools/packaging/kata-deploy/local-build/
cp -r kata-artifacts ${base_dir}/build
# Skip building dependant artifacts of cc-se-image-tarball
# because we already have them from the previous build
sed -i 's/\(^cc-se-image-tarball:\).*/\1/g' ${base_dir}/Makefile
USE_CACHE="no" make cc-se-image-tarball
build_dir=$(readlink -f build)
sudo cp -r "${build_dir}" "kata-build"
sudo chown -R $(id -u):$(id -g) "kata-build"
env:
HKD_PATH: "host-key-document"
- name: store-artifact cc-se-image
uses: actions/upload-artifact@v3
with:
name: kata-artifacts-s390x
path: kata-build/kata-static-cc-se-image.tar.xz
retention-days: 1
if-no-files-found: error
create-kata-tarball:
runs-on: s390x
needs: [build-asset, build-asset-cc-shim-v2, build-asset-cc-se-image]
steps:
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- uses: actions/checkout@v3
- name: get-artifacts
uses: actions/download-artifact@v3
with:
name: kata-artifacts-s390x
path: kata-artifacts
- name: merge-artifacts
run: |
./tools/packaging/kata-deploy/local-build/kata-deploy-merge-builds.sh kata-artifacts
- name: store-artifacts
uses: actions/upload-artifact@v3
with:
name: kata-static-tarball-s390x
path: kata-static.tar.xz
retention-days: 1
if-no-files-found: error
kata-payload:
needs: create-kata-tarball
runs-on: s390x
steps:
- name: Login to Confidential Containers quay.io
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.COCO_QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.COCO_QUAY_DEPLOYER_PASSWORD }}
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- uses: actions/checkout@v3
- name: get-kata-tarball
uses: actions/download-artifact@v3
with:
name: kata-static-tarball-s390x
- name: build-and-push-kata-payload
id: build-and-push-kata-payload
run: |
./tools/packaging/kata-deploy/local-build/kata-deploy-build-and-upload-payload.sh \
$(pwd)/kata-static.tar.xz "quay.io/confidential-containers/runtime-payload-ci" \
"kata-containers-${{ inputs.target-arch }}"

View File

@@ -0,0 +1,47 @@
name: CI | Publish Kata Containers payload for Confidential Containers
on:
push:
branches:
- CCv0
workflow_dispatch:
jobs:
build-assets-amd64:
uses: ./.github/workflows/cc-payload-after-push-amd64.yaml
with:
target-arch: amd64
secrets: inherit
build-assets-s390x:
uses: ./.github/workflows/cc-payload-after-push-s390x.yaml
with:
target-arch: s390x
secrets: inherit
publish:
runs-on: ubuntu-latest
needs: [build-assets-amd64, build-assets-s390x]
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Login to Confidential Containers quay.io
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.COCO_QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.COCO_QUAY_DEPLOYER_PASSWORD }}
- name: Push commit multi-arch manifest
run: |
docker manifest create quay.io/confidential-containers/runtime-payload-ci:kata-containers-${GITHUB_SHA} \
--amend quay.io/confidential-containers/runtime-payload-ci:kata-containers-${GITHUB_SHA}-amd64 \
--amend quay.io/confidential-containers/runtime-payload-ci:kata-containers-${GITHUB_SHA}-s390x
docker manifest push quay.io/confidential-containers/runtime-payload-ci:kata-containers-${GITHUB_SHA}
- name: Push latest multi-arch manifest
run: |
docker manifest create quay.io/confidential-containers/runtime-payload-ci:kata-containers-latest \
--amend quay.io/confidential-containers/runtime-payload-ci:kata-containers-amd64 \
--amend quay.io/confidential-containers/runtime-payload-ci:kata-containers-s390x
docker manifest push quay.io/confidential-containers/runtime-payload-ci:kata-containers-latest

154
.github/workflows/cc-payload-amd64.yaml vendored Normal file
View File

@@ -0,0 +1,154 @@
name: Publish Kata Containers payload for Confidential Containers (amd64)
on:
workflow_call:
inputs:
target-arch:
required: true
type: string
jobs:
build-asset:
runs-on: ubuntu-latest
strategy:
matrix:
measured_rootfs:
- no
asset:
- cloud-hypervisor
- qemu
- virtiofsd
- kernel-sev
- ovmf-sev
- ovmf
- qemu-snp-experimental
- qemu-tdx-experimental
- rootfs-initrd-sev
- cc-tdx-td-shim
- tdvf
include:
- measured_rootfs: yes
asset: kernel
- measured_rootfs: yes
asset: kernel-tdx-experimental
- measured_rootfs: yes
asset: cc-rootfs-image
- measured_rootfs: yes
asset: rootfs-image-tdx
steps:
- uses: actions/checkout@v3
- name: Build ${{ matrix.asset }}
run: |
make "${KATA_ASSET}-tarball"
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
sudo cp -r "${build_dir}" "kata-build"
env:
KATA_ASSET: ${{ matrix.asset }}
TAR_OUTPUT: ${{ matrix.asset }}.tar.gz
MEASURED_ROOTFS: ${{ matrix.measured_rootfs }}
- name: store-artifact ${{ matrix.asset }}
uses: actions/upload-artifact@v3
with:
name: kata-artifacts
path: kata-build/kata-static-${{ matrix.asset }}.tar.xz
retention-days: 1
if-no-files-found: error
- name: store-artifact root_hash_tdx.txt
uses: actions/upload-artifact@v3
with:
name: root_hash_tdx.txt
path: tools/osbuilder/root_hash_tdx.txt
retention-days: 1
if-no-files-found: ignore
- name: store-artifact root_hash_vanilla.txt
uses: actions/upload-artifact@v3
with:
name: root_hash_vanilla.txt
path: tools/osbuilder/root_hash_vanilla.txt
retention-days: 1
if-no-files-found: ignore
build-asset-cc-shim-v2:
runs-on: ubuntu-latest
needs: build-asset
steps:
- uses: actions/checkout@v3
- name: Get root_hash_tdx.txt
uses: actions/download-artifact@v3
with:
name: root_hash_tdx.txt
path: tools/osbuilder/
- name: Get root_hash_vanilla.txt
uses: actions/download-artifact@v3
with:
name: root_hash_vanilla.txt
path: tools/osbuilder/
- name: Build cc-shim-v2
run: |
make cc-shim-v2-tarball
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
sudo cp -r "${build_dir}" "kata-build"
env:
MEASURED_ROOTFS: yes
- name: store-artifact cc-shim-v2
uses: actions/upload-artifact@v3
with:
name: kata-artifacts
path: kata-build/kata-static-cc-shim-v2.tar.xz
retention-days: 1
if-no-files-found: error
create-kata-tarball:
runs-on: ubuntu-latest
needs: [build-asset, build-asset-cc-shim-v2]
steps:
- uses: actions/checkout@v3
- name: get-artifacts
uses: actions/download-artifact@v3
with:
name: kata-artifacts
path: kata-artifacts
- name: merge-artifacts
run: |
./tools/packaging/kata-deploy/local-build/kata-deploy-merge-builds.sh kata-artifacts
- name: store-artifacts
uses: actions/upload-artifact@v3
with:
name: kata-static-tarball
path: kata-static.tar.xz
retention-days: 1
if-no-files-found: error
kata-payload:
needs: create-kata-tarball
runs-on: ubuntu-latest
steps:
- name: Login to quay.io
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.COCO_QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.COCO_QUAY_DEPLOYER_PASSWORD }}
- uses: actions/checkout@v3
- name: get-kata-tarball
uses: actions/download-artifact@v3
with:
name: kata-static-tarball
- name: build-and-push-kata-payload
id: build-and-push-kata-payload
run: |
./tools/packaging/kata-deploy/local-build/kata-deploy-build-and-upload-payload.sh \
$(pwd)/kata-static.tar.xz \
"quay.io/confidential-containers/runtime-payload" \
"kata-containers-${{ inputs.target-arch }}"

142
.github/workflows/cc-payload-s390x.yaml vendored Normal file
View File

@@ -0,0 +1,142 @@
name: Publish Kata Containers payload for Confidential Containers (s390x)
on:
workflow_call:
inputs:
target-arch:
required: true
type: string
jobs:
build-asset:
runs-on: s390x
strategy:
matrix:
measured_rootfs:
- no
asset:
- qemu
- virtiofsd
include:
- measured_rootfs: yes
asset: kernel
- measured_rootfs: yes
asset: cc-rootfs-image
steps:
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- uses: actions/checkout@v3
- name: Build ${{ matrix.asset }}
run: |
make "${KATA_ASSET}-tarball"
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
sudo cp -r "${build_dir}" "kata-build"
env:
KATA_ASSET: ${{ matrix.asset }}
TAR_OUTPUT: ${{ matrix.asset }}.tar.gz
MEASURED_ROOTFS: ${{ matrix.measured_rootfs }}
- name: store-artifact ${{ matrix.asset }}
uses: actions/upload-artifact@v3
with:
name: kata-artifacts-s390x
path: kata-build/kata-static-${{ matrix.asset }}.tar.xz
retention-days: 1
if-no-files-found: error
- name: store-artifact root_hash_vanilla.txt
uses: actions/upload-artifact@v3
with:
name: root_hash_vanilla.txt-s390x
path: tools/osbuilder/root_hash_vanilla.txt
retention-days: 1
if-no-files-found: ignore
build-asset-cc-shim-v2:
runs-on: s390x
needs: build-asset
steps:
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- uses: actions/checkout@v3
- name: Get root_hash_vanilla.txt
uses: actions/download-artifact@v3
with:
name: root_hash_vanilla.txt-s390x
path: tools/osbuilder/
- name: Build cc-shim-v2
run: |
make cc-shim-v2-tarball
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
sudo cp -r "${build_dir}" "kata-build"
env:
MEASURED_ROOTFS: yes
- name: store-artifact cc-shim-v2
uses: actions/upload-artifact@v3
with:
name: kata-artifacts-s390x
path: kata-build/kata-static-cc-shim-v2.tar.xz
retention-days: 1
if-no-files-found: error
create-kata-tarball:
runs-on: s390x
needs: [build-asset, build-asset-cc-shim-v2]
steps:
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- uses: actions/checkout@v3
- name: get-artifacts
uses: actions/download-artifact@v3
with:
name: kata-artifacts-s390x
path: kata-artifacts
- name: merge-artifacts
run: |
./tools/packaging/kata-deploy/local-build/kata-deploy-merge-builds.sh kata-artifacts
- name: store-artifacts
uses: actions/upload-artifact@v3
with:
name: kata-static-tarball-s390x
path: kata-static.tar.xz
retention-days: 1
if-no-files-found: error
kata-payload:
needs: create-kata-tarball
runs-on: s390x
steps:
- name: Login to quay.io
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.COCO_QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.COCO_QUAY_DEPLOYER_PASSWORD }}
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- uses: actions/checkout@v3
- name: get-kata-tarball
uses: actions/download-artifact@v3
with:
name: kata-static-tarball-s390x
- name: build-and-push-kata-payload
id: build-and-push-kata-payload
run: |
./tools/packaging/kata-deploy/local-build/kata-deploy-build-and-upload-payload.sh \
$(pwd)/kata-static.tar.xz \
"quay.io/confidential-containers/runtime-payload" \
"kata-containers-${{ inputs.target-arch }}"

46
.github/workflows/cc-payload.yaml vendored Normal file
View File

@@ -0,0 +1,46 @@
name: Publish Kata Containers payload for Confidential Containers
on:
push:
tags:
- 'CC\-[0-9]+.[0-9]+.[0-9]+'
jobs:
build-assets-amd64:
uses: ./.github/workflows/cc-payload-amd64.yaml
with:
target-arch: amd64
secrets: inherit
build-assets-s390x:
uses: ./.github/workflows/cc-payload-s390x.yaml
with:
target-arch: s390x
secrets: inherit
publish:
runs-on: ubuntu-latest
needs: [build-assets-amd64, build-assets-s390x]
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Login to Confidential Containers quay.io
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.COCO_QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.COCO_QUAY_DEPLOYER_PASSWORD }}
- name: Push commit multi-arch manifest
run: |
docker manifest create quay.io/confidential-containers/runtime-payload:kata-containers-${GITHUB_SHA} \
--amend quay.io/confidential-containers/runtime-payload:kata-containers-${GITHUB_SHA}-amd64 \
--amend quay.io/confidential-containers/runtime-payload:kata-containers-${GITHUB_SHA}-s390x
docker manifest push quay.io/confidential-containers/runtime-payload:kata-containers-${GITHUB_SHA}
- name: Push latest multi-arch manifest
run: |
docker manifest create quay.io/confidential-containers/runtime-payload:kata-containers-latest \
--amend quay.io/confidential-containers/runtime-payload:kata-containers-amd64 \
--amend quay.io/confidential-containers/runtime-payload:kata-containers-s390x
docker manifest push quay.io/confidential-containers/runtime-payload:kata-containers-latest

View File

@@ -1,19 +0,0 @@
name: Kata Containers CoCo Stability Tests Weekly
on:
schedule:
- cron: '0 0 * * 0'
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
kata-containers-ci-on-push:
uses: ./.github/workflows/ci-weekly.yaml
with:
commit-hash: ${{ github.sha }}
pr-number: "weekly"
tag: ${{ github.sha }}-weekly
target-branch: ${{ github.ref_name }}
secrets: inherit

View File

@@ -1,13 +0,0 @@
name: Kata Containers CI (manually triggered)
on:
workflow_dispatch:
jobs:
kata-containers-ci-on-push:
uses: ./.github/workflows/ci.yaml
with:
commit-hash: ${{ github.sha }}
pr-number: "dev"
tag: ${{ github.sha }}-dev
target-branch: ${{ github.ref_name }}
secrets: inherit

View File

@@ -1,21 +0,0 @@
on:
schedule:
- cron: '0 5 * * *'
name: Nightly CI for s390x
jobs:
check-internal-test-result:
runs-on: s390x
strategy:
fail-fast: false
matrix:
test_title:
- kata-vfio-ap-e2e-tests
- cc-se-e2e-tests
steps:
- name: Fetch a test result for {{ matrix.test_title }}
run: |
file_name="${TEST_TITLE}-$(date +%Y-%m-%d).log"
/home/${USER}/script/handle_test_log.sh download $file_name
env:
TEST_TITLE: ${{ matrix.test_title }}

View File

@@ -2,6 +2,7 @@ name: Kata Containers Nightly CI
on:
schedule:
- cron: '0 0 * * *'
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
@@ -14,5 +15,4 @@ jobs:
commit-hash: ${{ github.sha }}
pr-number: "nightly"
tag: ${{ github.sha }}-nightly
target-branch: ${{ github.ref_name }}
secrets: inherit

View File

@@ -13,27 +13,19 @@ on:
- synchronize
- reopened
- labeled
paths-ignore:
- 'docs/**'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
skipper:
if: ${{ contains(github.event.pull_request.labels.*.name, 'ok-to-test') }}
uses: ./.github/workflows/gatekeeper-skipper.yaml
with:
commit-hash: ${{ github.event.pull_request.head.sha }}
target-branch: ${{ github.event.pull_request.base.ref }}
kata-containers-ci-on-push:
needs: skipper
if: ${{ needs.skipper.outputs.skip_build != 'yes' }}
if: ${{ contains(github.event.pull_request.labels.*.name, 'ok-to-test') }}
uses: ./.github/workflows/ci.yaml
with:
commit-hash: ${{ github.event.pull_request.head.sha }}
pr-number: ${{ github.event.pull_request.number }}
tag: ${{ github.event.pull_request.number }}-${{ github.event.pull_request.head.sha }}
target-branch: ${{ github.event.pull_request.base.ref }}
skip-test: ${{ needs.skipper.outputs.skip_test }}
secrets: inherit

View File

@@ -1,86 +0,0 @@
name: Run the CoCo Kata Containers Stability CI
on:
workflow_call:
inputs:
commit-hash:
required: true
type: string
pr-number:
required: true
type: string
tag:
required: true
type: string
target-branch:
required: false
type: string
default: ""
jobs:
build-kata-static-tarball-amd64:
uses: ./.github/workflows/build-kata-static-tarball-amd64.yaml
with:
tarball-suffix: -${{ inputs.tag }}
commit-hash: ${{ inputs.commit-hash }}
target-branch: ${{ inputs.target-branch }}
publish-kata-deploy-payload-amd64:
needs: build-kata-static-tarball-amd64
uses: ./.github/workflows/publish-kata-deploy-payload-amd64.yaml
with:
tarball-suffix: -${{ inputs.tag }}
registry: ghcr.io
repo: ${{ github.repository_owner }}/kata-deploy-ci
tag: ${{ inputs.tag }}-amd64
commit-hash: ${{ inputs.commit-hash }}
target-branch: ${{ inputs.target-branch }}
secrets: inherit
build-and-publish-tee-confidential-unencrypted-image:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Kata Containers ghcr.io
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Docker build and push
uses: docker/build-push-action@v5
with:
tags: ghcr.io/kata-containers/test-images:unencrypted-${{ inputs.pr-number }}
push: true
context: tests/integration/kubernetes/runtimeclass_workloads/confidential/unencrypted/
platforms: linux/amd64
file: tests/integration/kubernetes/runtimeclass_workloads/confidential/unencrypted/Dockerfile
run-kata-coco-stability-tests:
needs: [publish-kata-deploy-payload-amd64, build-and-publish-tee-confidential-unencrypted-image]
uses: ./.github/workflows/run-kata-coco-stability-tests.yaml
with:
registry: ghcr.io
repo: ${{ github.repository_owner }}/kata-deploy-ci
tag: ${{ inputs.tag }}-amd64
commit-hash: ${{ inputs.commit-hash }}
pr-number: ${{ inputs.pr-number }}
target-branch: ${{ inputs.target-branch }}
secrets: inherit

View File

@@ -11,14 +11,6 @@ on:
tag:
required: true
type: string
target-branch:
required: false
type: string
default: ""
skip-test:
required: false
type: string
default: no
jobs:
build-kata-static-tarball-amd64:
@@ -26,7 +18,6 @@ jobs:
with:
tarball-suffix: -${{ inputs.tag }}
commit-hash: ${{ inputs.commit-hash }}
target-branch: ${{ inputs.target-branch }}
publish-kata-deploy-payload-amd64:
needs: build-kata-static-tarball-amd64
@@ -37,97 +28,31 @@ jobs:
repo: ${{ github.repository_owner }}/kata-deploy-ci
tag: ${{ inputs.tag }}-amd64
commit-hash: ${{ inputs.commit-hash }}
target-branch: ${{ inputs.target-branch }}
secrets: inherit
build-kata-static-tarball-arm64:
uses: ./.github/workflows/build-kata-static-tarball-arm64.yaml
with:
tarball-suffix: -${{ inputs.tag }}
commit-hash: ${{ inputs.commit-hash }}
target-branch: ${{ inputs.target-branch }}
publish-kata-deploy-payload-arm64:
needs: build-kata-static-tarball-arm64
uses: ./.github/workflows/publish-kata-deploy-payload-arm64.yaml
with:
tarball-suffix: -${{ inputs.tag }}
registry: ghcr.io
repo: ${{ github.repository_owner }}/kata-deploy-ci
tag: ${{ inputs.tag }}-arm64
commit-hash: ${{ inputs.commit-hash }}
target-branch: ${{ inputs.target-branch }}
secrets: inherit
build-kata-static-tarball-s390x:
uses: ./.github/workflows/build-kata-static-tarball-s390x.yaml
with:
tarball-suffix: -${{ inputs.tag }}
commit-hash: ${{ inputs.commit-hash }}
target-branch: ${{ inputs.target-branch }}
secrets: inherit
build-kata-static-tarball-ppc64le:
uses: ./.github/workflows/build-kata-static-tarball-ppc64le.yaml
with:
tarball-suffix: -${{ inputs.tag }}
commit-hash: ${{ inputs.commit-hash }}
target-branch: ${{ inputs.target-branch }}
publish-kata-deploy-payload-s390x:
needs: build-kata-static-tarball-s390x
uses: ./.github/workflows/publish-kata-deploy-payload-s390x.yaml
with:
tarball-suffix: -${{ inputs.tag }}
registry: ghcr.io
repo: ${{ github.repository_owner }}/kata-deploy-ci
tag: ${{ inputs.tag }}-s390x
commit-hash: ${{ inputs.commit-hash }}
target-branch: ${{ inputs.target-branch }}
secrets: inherit
publish-kata-deploy-payload-ppc64le:
needs: build-kata-static-tarball-ppc64le
uses: ./.github/workflows/publish-kata-deploy-payload-ppc64le.yaml
with:
tarball-suffix: -${{ inputs.tag }}
registry: ghcr.io
repo: ${{ github.repository_owner }}/kata-deploy-ci
tag: ${{ inputs.tag }}-ppc64le
commit-hash: ${{ inputs.commit-hash }}
target-branch: ${{ inputs.target-branch }}
secrets: inherit
build-and-publish-tee-confidential-unencrypted-image:
runs-on: ubuntu-22.04
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v2
- name: Login to Kata Containers ghcr.io
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Docker build and push
uses: docker/build-push-action@v5
uses: docker/build-push-action@v4
with:
tags: ghcr.io/kata-containers/test-images:unencrypted-${{ inputs.pr-number }}
push: true
@@ -135,112 +60,71 @@ jobs:
platforms: linux/amd64, linux/s390x
file: tests/integration/kubernetes/runtimeclass_workloads/confidential/unencrypted/Dockerfile
run-kata-monitor-tests:
if: ${{ inputs.skip-test != 'yes' }}
needs: build-kata-static-tarball-amd64
uses: ./.github/workflows/run-kata-monitor-tests.yaml
with:
tarball-suffix: -${{ inputs.tag }}
commit-hash: ${{ inputs.commit-hash }}
target-branch: ${{ inputs.target-branch }}
run-k8s-tests-on-aks:
if: ${{ inputs.skip-test != 'yes' }}
needs: publish-kata-deploy-payload-amd64
uses: ./.github/workflows/run-k8s-tests-on-aks.yaml
with:
tarball-suffix: -${{ inputs.tag }}
registry: ghcr.io
repo: ${{ github.repository_owner }}/kata-deploy-ci
tag: ${{ inputs.tag }}-amd64
commit-hash: ${{ inputs.commit-hash }}
pr-number: ${{ inputs.pr-number }}
target-branch: ${{ inputs.target-branch }}
secrets: inherit
run-k8s-tests-on-amd64:
if: ${{ inputs.skip-test != 'yes' }}
needs: publish-kata-deploy-payload-amd64
uses: ./.github/workflows/run-k8s-tests-on-amd64.yaml
with:
registry: ghcr.io
repo: ${{ github.repository_owner }}/kata-deploy-ci
tag: ${{ inputs.tag }}-amd64
commit-hash: ${{ inputs.commit-hash }}
pr-number: ${{ inputs.pr-number }}
target-branch: ${{ inputs.target-branch }}
secrets: inherit
run-kata-coco-tests:
if: ${{ inputs.skip-test != 'yes' }}
run-k8s-tests-on-sev:
needs: [publish-kata-deploy-payload-amd64, build-and-publish-tee-confidential-unencrypted-image]
uses: ./.github/workflows/run-kata-coco-tests.yaml
uses: ./.github/workflows/run-k8s-tests-on-sev.yaml
with:
registry: ghcr.io
repo: ${{ github.repository_owner }}/kata-deploy-ci
tag: ${{ inputs.tag }}-amd64
commit-hash: ${{ inputs.commit-hash }}
pr-number: ${{ inputs.pr-number }}
target-branch: ${{ inputs.target-branch }}
secrets: inherit
run-k8s-tests-on-zvsi:
if: ${{ inputs.skip-test != 'yes' }}
needs: [publish-kata-deploy-payload-s390x, build-and-publish-tee-confidential-unencrypted-image]
uses: ./.github/workflows/run-k8s-tests-on-zvsi.yaml
run-k8s-tests-on-snp:
needs: [publish-kata-deploy-payload-amd64, build-and-publish-tee-confidential-unencrypted-image]
uses: ./.github/workflows/run-k8s-tests-on-snp.yaml
with:
registry: ghcr.io
repo: ${{ github.repository_owner }}/kata-deploy-ci
tag: ${{ inputs.tag }}-s390x
tag: ${{ inputs.tag }}-amd64
commit-hash: ${{ inputs.commit-hash }}
pr-number: ${{ inputs.pr-number }}
target-branch: ${{ inputs.target-branch }}
secrets: inherit
run-k8s-tests-on-ppc64le:
if: ${{ inputs.skip-test != 'yes' }}
needs: publish-kata-deploy-payload-ppc64le
uses: ./.github/workflows/run-k8s-tests-on-ppc64le.yaml
run-k8s-tests-on-tdx:
needs: [publish-kata-deploy-payload-amd64, build-and-publish-tee-confidential-unencrypted-image]
uses: ./.github/workflows/run-k8s-tests-on-tdx.yaml
with:
registry: ghcr.io
repo: ${{ github.repository_owner }}/kata-deploy-ci
tag: ${{ inputs.tag }}-ppc64le
tag: ${{ inputs.tag }}-amd64
commit-hash: ${{ inputs.commit-hash }}
pr-number: ${{ inputs.pr-number }}
target-branch: ${{ inputs.target-branch }}
run-metrics-tests:
if: ${{ inputs.skip-test != 'yes' }}
needs: build-kata-static-tarball-amd64
uses: ./.github/workflows/run-metrics.yaml
with:
tarball-suffix: -${{ inputs.tag }}
commit-hash: ${{ inputs.commit-hash }}
target-branch: ${{ inputs.target-branch }}
run-basic-amd64-tests:
if: ${{ inputs.skip-test != 'yes' }}
run-cri-containerd-tests:
needs: build-kata-static-tarball-amd64
uses: ./.github/workflows/basic-ci-amd64.yaml
uses: ./.github/workflows/run-cri-containerd-tests.yaml
with:
tarball-suffix: -${{ inputs.tag }}
commit-hash: ${{ inputs.commit-hash }}
target-branch: ${{ inputs.target-branch }}
run-cri-containerd-tests-s390x:
if: ${{ inputs.skip-test != 'yes' }}
needs: build-kata-static-tarball-s390x
uses: ./.github/workflows/run-cri-containerd-tests-s390x.yaml
run-nydus-tests:
needs: build-kata-static-tarball-amd64
uses: ./.github/workflows/run-nydus-tests.yaml
with:
tarball-suffix: -${{ inputs.tag }}
commit-hash: ${{ inputs.commit-hash }}
target-branch: ${{ inputs.target-branch }}
run-cri-containerd-tests-ppc64le:
if: ${{ inputs.skip-test != 'yes' }}
needs: build-kata-static-tarball-ppc64le
uses: ./.github/workflows/run-cri-containerd-tests-ppc64le.yaml
run-vfio-tests:
needs: build-kata-static-tarball-amd64
uses: ./.github/workflows/run-vfio-tests.yaml
with:
tarball-suffix: -${{ inputs.tag }}
commit-hash: ${{ inputs.commit-hash }}
target-branch: ${{ inputs.target-branch }}

View File

@@ -1,31 +0,0 @@
name: Cleanup dangling Azure resources
on:
schedule:
- cron: "0 0 * * *"
workflow_dispatch:
jobs:
cleanup-resources:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- name: Log into Azure
env:
AZ_APPID: ${{ secrets.AZ_APPID }}
AZ_PASSWORD: ${{ secrets.AZ_PASSWORD }}
AZ_TENANT_ID: ${{ secrets.AZ_TENANT_ID }}
AZ_SUBSCRIPTION_ID: ${{ secrets.AZ_SUBSCRIPTION_ID }}
run: bash tests/integration/kubernetes/gha-run.sh login-azure
- name: Install Python dependencies
run: |
pip3 install --user --upgrade \
azure-identity==1.16.0 \
azure-mgmt-resource==23.0.1
- name: Cleanup resources
env:
AZ_SUBSCRIPTION_ID: ${{ secrets.AZ_SUBSCRIPTION_ID }}
CLEANUP_AFTER_HOURS: 24 # Clean up resources created more than this many hours ago.
run: python3 tests/cleanup_resources.py

View File

@@ -18,9 +18,7 @@ env:
jobs:
commit-message-check:
runs-on: ubuntu-22.04
env:
PR_AUTHOR: ${{ github.event.pull_request.user.login }}
runs-on: ubuntu-latest
name: Commit Message Check
steps:
- name: Get PR Commits
@@ -34,10 +32,7 @@ jobs:
#
# Revert "<original-subject-line>"
#
# The format of a re-re-vert commit as follows:
#
# Reapply "<original-subject-line>"
filter_out_pattern: '^Revert "|^Reapply "'
filter_out_pattern: '^Revert "'
- name: DCO Check
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
@@ -52,16 +47,16 @@ jobs:
commits: ${{ steps.get-pr-commits.outputs.commits }}
- name: Check Subject Line Length
if: ${{ (env.PR_AUTHOR != 'dependabot[bot]') && !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') && ( success() || failure() ) }}
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') && ( success() || failure() ) }}
uses: tim-actions/commit-message-checker-with-regex@v0.3.1
with:
commits: ${{ steps.get-pr-commits.outputs.commits }}
pattern: '^.{0,75}(\n.*)*$'
pattern: '^.{0,75}(\n.*)*$|^Merge pull request (?:kata-containers)?#[\d]+ from.*'
error: 'Subject too long (max 75)'
post_error: ${{ env.error_msg }}
- name: Check Body Line Length
if: ${{ (env.PR_AUTHOR != 'dependabot[bot]') && !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') && ( success() || failure() ) }}
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') && ( success() || failure() ) }}
uses: tim-actions/commit-message-checker-with-regex@v0.3.1
with:
commits: ${{ steps.get-pr-commits.outputs.commits }}
@@ -91,11 +86,22 @@ jobs:
error: 'Body line too long (max 150)'
post_error: ${{ env.error_msg }}
- name: Check Subsystem
if: ${{ (env.PR_AUTHOR != 'dependabot[bot]') && !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') && ( success() || failure() ) }}
- name: Check Fixes
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') && ( success() || failure() ) }}
uses: tim-actions/commit-message-checker-with-regex@v0.3.1
with:
commits: ${{ steps.get-pr-commits.outputs.commits }}
pattern: '^[\s\t]*[^:\s\t]+[\s\t]*:'
pattern: '\s*Fixes\s*:?\s*(#\d+|github\.com\/kata-containers\/[a-z-.]*#\d+)|^\s*release\s*:'
flags: 'i'
error: 'No "Fixes" found'
post_error: ${{ env.error_msg }}
one_pass_all_pass: 'true'
- name: Check Subsystem
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') && ( success() || failure() ) }}
uses: tim-actions/commit-message-checker-with-regex@v0.3.1
with:
commits: ${{ steps.get-pr-commits.outputs.commits }}
pattern: '^[\s\t]*[^:\s\t]+[\s\t]*:|^Merge pull request (?:kata-containers)?#[\d]+ from.*'
error: 'Failed to find subsystem in subject'
post_error: ${{ env.error_msg }}

View File

@@ -5,6 +5,7 @@ on:
- edited
- reopened
- synchronize
paths-ignore: [ '**.md', '**.png', '**.jpg', '**.jpeg', '**.svg', '/docs/**' ]
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
@@ -18,8 +19,8 @@ jobs:
- name: Install Go
uses: actions/setup-go@v2
with:
go-version: 1.22.2
go-version: 1.19.3
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v2
- name: Build utils
run: ./ci/darwin-test.sh

124
.github/workflows/deploy-ccv0-demo.yaml vendored Normal file
View File

@@ -0,0 +1,124 @@
on:
issue_comment:
types: [created, edited]
name: deploy-ccv0-demo
jobs:
check-comment-and-membership:
runs-on: ubuntu-latest
if: |
github.event.issue.pull_request
&& github.event_name == 'issue_comment'
&& github.event.action == 'created'
&& startsWith(github.event.comment.body, '/deploy-ccv0-demo')
steps:
- name: Check membership
uses: kata-containers/is-organization-member@1.0.1
id: is_organization_member
with:
organization: kata-containers
username: ${{ github.event.comment.user.login }}
token: ${{ secrets.GITHUB_TOKEN }}
- name: Fail if not member
run: |
result=${{ steps.is_organization_member.outputs.result }}
if [ $result == false ]; then
user=${{ github.event.comment.user.login }}
echo Either ${user} is not part of the kata-containers organization
echo or ${user} has its Organization Visibility set to Private at
echo https://github.com/orgs/kata-containers/people?query=${user}
echo
echo Ensure you change your Organization Visibility to Public and
echo trigger the test again.
exit 1
fi
build-asset:
runs-on: ubuntu-latest
needs: check-comment-and-membership
strategy:
matrix:
asset:
- cloud-hypervisor
- firecracker
- kernel
- qemu
- rootfs-image
- rootfs-initrd
- shim-v2
steps:
- uses: actions/checkout@v2
- name: Prepare confidential container rootfs
if: ${{ matrix.asset == 'rootfs-initrd' }}
run: |
pushd include_rootfs/etc
curl -LO https://raw.githubusercontent.com/confidential-containers/documentation/main/demos/ssh-demo/aa-offline_fs_kbc-keys.json
mkdir kata-containers
envsubst < docs/how-to/data/confidential-agent-config.toml.in > kata-containers/agent.toml
popd
env:
AA_KBC_PARAMS: offline_fs_kbc::null
- name: Build ${{ matrix.asset }}
run: |
make "${KATA_ASSET}-tarball"
build_dir=$(readlink -f build)
# store-artifact does not work with symlink
sudo cp -r "${build_dir}" "kata-build"
env:
AA_KBC: offline_fs_kbc
INCLUDE_ROOTFS: include_rootfs
KATA_ASSET: ${{ matrix.asset }}
TAR_OUTPUT: ${{ matrix.asset }}.tar.gz
- name: store-artifact ${{ matrix.asset }}
uses: actions/upload-artifact@v2
with:
name: kata-artifacts
path: kata-build/kata-static-${{ matrix.asset }}.tar.xz
if-no-files-found: error
create-kata-tarball:
runs-on: ubuntu-latest
needs: build-asset
steps:
- uses: actions/checkout@v2
- name: get-artifacts
uses: actions/download-artifact@v2
with:
name: kata-artifacts
path: kata-artifacts
- name: merge-artifacts
run: |
./tools/packaging/kata-deploy/local-build/kata-deploy-merge-builds.sh kata-artifacts
- name: store-artifacts
uses: actions/upload-artifact@v2
with:
name: kata-static-tarball
path: kata-static.tar.xz
kata-deploy:
needs: create-kata-tarball
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: get-kata-tarball
uses: actions/download-artifact@v2
with:
name: kata-static-tarball
- name: build-and-push-kata-deploy-ci
id: build-and-push-kata-deploy-ci
run: |
tag=$(echo $GITHUB_REF | cut -d/ -f3-)
pushd $GITHUB_WORKSPACE
git checkout $tag
pkg_sha=$(git rev-parse HEAD)
popd
mv kata-static.tar.xz $GITHUB_WORKSPACE/tools/packaging/kata-deploy/kata-static.tar.xz
docker build --build-arg KATA_ARTIFACTS=kata-static.tar.xz -t quay.io/confidential-containers/runtime-payload:$pkg_sha $GITHUB_WORKSPACE/tools/packaging/kata-deploy
docker login -u ${{ secrets.QUAY_DEPLOYER_USERNAME }} -p ${{ secrets.QUAY_DEPLOYER_PASSWORD }} quay.io
docker push quay.io/confidential-containers/runtime-payload:$pkg_sha
mkdir -p packaging/kata-deploy
ln -s $GITHUB_WORKSPACE/tools/packaging/kata-deploy/action packaging/kata-deploy/action
echo "::set-output name=PKG_SHA::${pkg_sha}"

View File

@@ -5,7 +5,7 @@ on:
name: Docs URL Alive Check
jobs:
test:
runs-on: ubuntu-22.04
runs-on: ubuntu-20.04
# don't run this action on forks
if: github.repository_owner == 'kata-containers'
env:
@@ -14,7 +14,7 @@ jobs:
- name: Install Go
uses: actions/setup-go@v2
with:
go-version: 1.22.2
go-version: 1.19.3
env:
GOPATH: ${{ runner.workspace }}/kata-containers
- name: Set env
@@ -22,10 +22,15 @@ jobs:
echo "GOPATH=${{ github.workspace }}" >> $GITHUB_ENV
echo "${{ github.workspace }}/bin" >> $GITHUB_PATH
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v2
with:
fetch-depth: 0
path: ./src/github.com/${{ github.repository }}
- name: Setup
run: |
cd ${GOPATH}/src/github.com/${{ github.repository }} && ./ci/setup.sh
env:
GOPATH: ${{ runner.workspace }}/kata-containers
# docs url alive check
- name: Docs URL Alive Check
run: |

View File

@@ -1,52 +0,0 @@
name: Skipper
# This workflow sets various "skip_*" output values that can be used to
# determine what workflows/jobs are expected to be executed. Sample usage:
#
# skipper:
# uses: ./.github/workflows/gatekeeper-skipper.yaml
# with:
# commit-hash: ${{ github.event.pull_request.head.sha }}
# target-branch: ${{ github.event.pull_request.base.ref }}
#
# your-workflow:
# needs: skipper
# if: ${{ needs.skipper.outputs.skip_build != 'yes' }}
on:
workflow_call:
inputs:
commit-hash:
required: true
type: string
target-branch:
required: false
type: string
default: ""
outputs:
skip_build:
value: ${{ jobs.skipper.outputs.skip_build }}
skip_test:
value: ${{ jobs.skipper.outputs.skip_test }}
skip_static:
value: ${{ jobs.skipper.outputs.skip_static }}
jobs:
skipper:
runs-on: ubuntu-latest
outputs:
skip_build: ${{ steps.skipper.outputs.skip_build }}
skip_test: ${{ steps.skipper.outputs.skip_test }}
skip_static: ${{ steps.skipper.outputs.skip_static }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- id: skipper
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
run: |
python3 tools/testing/gatekeeper/skips.py | tee -a "$GITHUB_OUTPUT"
shell: /usr/bin/bash -x {0}

View File

@@ -1,44 +0,0 @@
name: Gatekeeper
# Gatekeeper uses the "skips.py" to determine which job names/regexps are
# required for given PR and waits for them to either complete or fail
# reporting the status.
on:
pull_request_target:
types:
- opened
- synchronize
- reopened
- labeled
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
gatekeeper:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
- id: gatekeeper
env:
TARGET_BRANCH: ${{ github.event.pull_request.base.ref }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
COMMIT_HASH: ${{ github.event.pull_request.head.sha }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: |
#!/usr/bin/env bash -x
mapfile -t lines < <(python3 tools/testing/gatekeeper/skips.py -t)
export REQUIRED_JOBS="${lines[0]}"
export REQUIRED_REGEXPS="${lines[1]}"
export REQUIRED_LABELS="${lines[2]}"
echo "REQUIRED_JOBS: $REQUIRED_JOBS"
echo "REQUIRED_REGEXPS: $REQUIRED_REGEXPS"
echo "REQUIRED_LABELS: $REQUIRED_LABELS"
python3 tools/testing/gatekeeper/jobs.py
exit $?
shell: /usr/bin/bash -x {0}

View File

@@ -12,10 +12,10 @@ concurrency:
jobs:
kata-deploy-runtime-classes-check:
runs-on: ubuntu-22.04
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v3
- name: Ensure the split out runtime classes match the all-in-one file
run: |
pushd tools/packaging/kata-deploy/runtimeclasses/

View File

@@ -13,7 +13,7 @@ on:
jobs:
move-linked-issues-to-in-progress:
runs-on: ubuntu-22.04
runs-on: ubuntu-latest
steps:
- name: Install hub
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
@@ -38,17 +38,7 @@ jobs:
- name: Checkout code to allow hub to communicate with the project
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ github.event.pull_request.base.ref }}
uses: actions/checkout@v2
- name: Move issue to "In progress"
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
@@ -62,10 +52,11 @@ jobs:
grep -v "^\#" |\
cut -d';' -f3 || true)
# PR doesn't have any linked issues, handle it only if it exists
# PR doesn't have any linked issues
# (it should, but maybe a new user forgot to add a "Fixes: #XXX" commit).
[ -z "$linked_issue_urls" ] && {
echo "::warning::No linked issues for PR $pr"
exit 0
echo "::error::No linked issues for PR $pr"
exit 1
}
project_name="Issue backlog"

View File

@@ -3,10 +3,11 @@ on:
push:
branches:
- main
workflow_dispatch:
- stable-*
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
build-assets-amd64:
@@ -14,7 +15,6 @@ jobs:
with:
commit-hash: ${{ github.sha }}
push-to-registry: yes
target-branch: ${{ github.ref_name }}
secrets: inherit
build-assets-arm64:
@@ -22,7 +22,6 @@ jobs:
with:
commit-hash: ${{ github.sha }}
push-to-registry: yes
target-branch: ${{ github.ref_name }}
secrets: inherit
build-assets-s390x:
@@ -30,15 +29,6 @@ jobs:
with:
commit-hash: ${{ github.sha }}
push-to-registry: yes
target-branch: ${{ github.ref_name }}
secrets: inherit
build-assets-ppc64le:
uses: ./.github/workflows/build-kata-static-tarball-ppc64le.yaml
with:
commit-hash: ${{ github.sha }}
push-to-registry: yes
target-branch: ${{ github.ref_name }}
secrets: inherit
publish-kata-deploy-payload-amd64:
@@ -48,8 +38,7 @@ jobs:
commit-hash: ${{ github.sha }}
registry: quay.io
repo: kata-containers/kata-deploy-ci
tag: kata-containers-latest-amd64
target-branch: ${{ github.ref_name }}
tag: kata-containers-amd64
secrets: inherit
publish-kata-deploy-payload-arm64:
@@ -59,8 +48,7 @@ jobs:
commit-hash: ${{ github.sha }}
registry: quay.io
repo: kata-containers/kata-deploy-ci
tag: kata-containers-latest-arm64
target-branch: ${{ github.ref_name }}
tag: kata-containers-arm64
secrets: inherit
publish-kata-deploy-payload-s390x:
@@ -70,30 +58,18 @@ jobs:
commit-hash: ${{ github.sha }}
registry: quay.io
repo: kata-containers/kata-deploy-ci
tag: kata-containers-latest-s390x
target-branch: ${{ github.ref_name }}
secrets: inherit
publish-kata-deploy-payload-ppc64le:
needs: build-assets-ppc64le
uses: ./.github/workflows/publish-kata-deploy-payload-ppc64le.yaml
with:
commit-hash: ${{ github.sha }}
registry: quay.io
repo: kata-containers/kata-deploy-ci
tag: kata-containers-latest-ppc64le
target-branch: ${{ github.ref_name }}
tag: kata-containers-s390x
secrets: inherit
publish-manifest:
runs-on: ubuntu-22.04
needs: [publish-kata-deploy-payload-amd64, publish-kata-deploy-payload-arm64, publish-kata-deploy-payload-s390x, publish-kata-deploy-payload-ppc64le]
runs-on: ubuntu-latest
needs: [publish-kata-deploy-payload-amd64, publish-kata-deploy-payload-arm64, publish-kata-deploy-payload-s390x]
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v3
- name: Login to Kata Containers quay.io
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
@@ -101,7 +77,8 @@ jobs:
- name: Push multi-arch manifest
run: |
./tools/packaging/release/release.sh publish-multiarch-manifest
env:
KATA_DEPLOY_IMAGE_TAGS: "kata-containers-latest"
KATA_DEPLOY_REGISTRIES: "quay.io/kata-containers/kata-deploy-ci"
docker manifest create quay.io/kata-containers/kata-deploy-ci:kata-containers-latest \
--amend quay.io/kata-containers/kata-deploy-ci:kata-containers-amd64 \
--amend quay.io/kata-containers/kata-deploy-ci:kata-containers-arm64 \
--amend quay.io/kata-containers/kata-deploy-ci:kata-containers-s390x
docker manifest push quay.io/kata-containers/kata-deploy-ci:kata-containers-latest

View File

@@ -17,34 +17,23 @@ on:
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
kata-payload:
runs-on: ubuntu-22.04
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: get-kata-tarball
uses: actions/download-artifact@v4
uses: actions/download-artifact@v3
with:
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
- name: Login to Kata Containers quay.io
if: ${{ inputs.registry == 'quay.io' }}
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
@@ -52,7 +41,7 @@ jobs:
- name: Login to Kata Containers ghcr.io
if: ${{ inputs.registry == 'ghcr.io' }}
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}

View File

@@ -17,38 +17,27 @@ on:
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
kata-payload:
runs-on: arm64-builder
runs-on: arm64
steps:
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- uses: actions/checkout@v4
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: get-kata-tarball
uses: actions/download-artifact@v4
uses: actions/download-artifact@v3
with:
name: kata-static-tarball-arm64${{ inputs.tarball-suffix }}
- name: Login to Kata Containers quay.io
if: ${{ inputs.registry == 'quay.io' }}
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
@@ -56,7 +45,7 @@ jobs:
- name: Login to Kata Containers ghcr.io
if: ${{ inputs.registry == 'ghcr.io' }}
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}

View File

@@ -1,75 +0,0 @@
name: CI | Publish kata-deploy payload for ppc64le
on:
workflow_call:
inputs:
tarball-suffix:
required: false
type: string
registry:
required: true
type: string
repo:
required: true
type: string
tag:
required: true
type: string
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
kata-payload:
runs-on: ppc64le
steps:
- name: Prepare the self-hosted runner
run: |
${HOME}/scripts/prepare_runner.sh
sudo rm -rf $GITHUB_WORKSPACE/*
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: get-kata-tarball
uses: actions/download-artifact@v4
with:
name: kata-static-tarball-ppc64le${{ inputs.tarball-suffix }}
- name: Login to Kata Containers quay.io
if: ${{ inputs.registry == 'quay.io' }}
uses: docker/login-action@v3
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- name: Login to Kata Containers ghcr.io
if: ${{ inputs.registry == 'ghcr.io' }}
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: build-and-push-kata-payload
id: build-and-push-kata-payload
run: |
./tools/packaging/kata-deploy/local-build/kata-deploy-build-and-upload-payload.sh \
$(pwd)/kata-static.tar.xz \
${{ inputs.registry }}/${{ inputs.repo }} ${{ inputs.tag }}

View File

@@ -17,34 +17,27 @@ on:
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
kata-payload:
runs-on: s390x
steps:
- uses: actions/checkout@v4
- name: Adjust a permission for repo
run: |
sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: get-kata-tarball
uses: actions/download-artifact@v4
uses: actions/download-artifact@v3
with:
name: kata-static-tarball-s390x${{ inputs.tarball-suffix }}
- name: Login to Kata Containers quay.io
if: ${{ inputs.registry == 'quay.io' }}
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
@@ -52,7 +45,7 @@ jobs:
- name: Login to Kata Containers ghcr.io
if: ${{ inputs.registry == 'ghcr.io' }}
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}

View File

@@ -10,45 +10,39 @@ jobs:
build-kata-static-tarball-amd64:
uses: ./.github/workflows/build-kata-static-tarball-amd64.yaml
with:
push-to-registry: yes
stage: release
secrets: inherit
kata-deploy:
needs: build-kata-static-tarball-amd64
runs-on: ubuntu-22.04
runs-on: ubuntu-latest
steps:
- name: Login to Kata Containers docker.io
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to Kata Containers quay.io
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: get-kata-tarball
uses: actions/download-artifact@v4
uses: actions/download-artifact@v3
with:
name: kata-static-tarball-amd64
- name: build-and-push-kata-deploy-ci-amd64
id: build-and-push-kata-deploy-ci-amd64
run: |
# We need to do such trick here as the format of the $GITHUB_REF
# We need to do such trick here as the format of the $GITHUB_REF
# is "refs/tags/<tag>"
tag=$(echo $GITHUB_REF | cut -d/ -f3-)
if [ "${tag}" = "main" ]; then
tag=$(./tools/packaging/release/release.sh release-version)
tags=(${tag} "latest")
else
tags=(${tag})
fi
tags=($tag)
tags+=($([[ "$tag" =~ "alpha"|"rc" ]] && echo "latest" || echo "stable"))
for tag in ${tags[@]}; do
./tools/packaging/kata-deploy/local-build/kata-deploy-build-and-upload-payload.sh \
$(pwd)/kata-static.tar.xz "docker.io/katadocker/kata-deploy" \

View File

@@ -10,45 +10,39 @@ jobs:
build-kata-static-tarball-arm64:
uses: ./.github/workflows/build-kata-static-tarball-arm64.yaml
with:
push-to-registry: yes
stage: release
secrets: inherit
kata-deploy:
needs: build-kata-static-tarball-arm64
runs-on: arm64-builder
runs-on: arm64
steps:
- name: Login to Kata Containers docker.io
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to Kata Containers quay.io
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: get-kata-tarball
uses: actions/download-artifact@v4
uses: actions/download-artifact@v3
with:
name: kata-static-tarball-arm64
- name: build-and-push-kata-deploy-ci-arm64
id: build-and-push-kata-deploy-ci-arm64
run: |
# We need to do such trick here as the format of the $GITHUB_REF
# We need to do such trick here as the format of the $GITHUB_REF
# is "refs/tags/<tag>"
tag=$(echo $GITHUB_REF | cut -d/ -f3-)
if [ "${tag}" = "main" ]; then
tag=$(./tools/packaging/release/release.sh release-version)
tags=(${tag} "latest")
else
tags=(${tag})
fi
tags=($tag)
tags+=($([[ "$tag" =~ "alpha"|"rc" ]] && echo "latest" || echo "stable"))
for tag in ${tags[@]}; do
./tools/packaging/kata-deploy/local-build/kata-deploy-build-and-upload-payload.sh \
$(pwd)/kata-static.tar.xz "docker.io/katadocker/kata-deploy" \

View File

@@ -1,64 +0,0 @@
name: Publish Kata release artifacts for ppc64le
on:
workflow_call:
inputs:
target-arch:
required: true
type: string
jobs:
build-kata-static-tarball-ppc64le:
uses: ./.github/workflows/build-kata-static-tarball-ppc64le.yaml
with:
push-to-registry: yes
stage: release
secrets: inherit
kata-deploy:
needs: build-kata-static-tarball-ppc64le
runs-on: ppc64le
steps:
- name: Prepare the self-hosted runner
run: |
bash ${HOME}/scripts/prepare_runner.sh
sudo rm -rf $GITHUB_WORKSPACE/*
- name: Login to Kata Containers docker.io
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to Kata Containers quay.io
uses: docker/login-action@v3
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- uses: actions/checkout@v4
- name: get-kata-tarball
uses: actions/download-artifact@v4
with:
name: kata-static-tarball-ppc64le
- name: build-and-push-kata-deploy-ci-ppc64le
id: build-and-push-kata-deploy-ci-ppc64le
run: |
# We need to do such trick here as the format of the $GITHUB_REF
# is "refs/tags/<tag>"
tag=$(echo $GITHUB_REF | cut -d/ -f3-)
if [ "${tag}" = "main" ]; then
tag=$(./tools/packaging/release/release.sh release-version)
tags=(${tag} "latest")
else
tags=(${tag})
fi
for tag in ${tags[@]}; do
./tools/packaging/kata-deploy/local-build/kata-deploy-build-and-upload-payload.sh \
$(pwd)/kata-static.tar.xz "docker.io/katadocker/kata-deploy" \
"${tag}-${{ inputs.target-arch }}"
./tools/packaging/kata-deploy/local-build/kata-deploy-build-and-upload-payload.sh \
$(pwd)/kata-static.tar.xz "quay.io/kata-containers/kata-deploy" \
"${tag}-${{ inputs.target-arch }}"
done

View File

@@ -10,45 +10,39 @@ jobs:
build-kata-static-tarball-s390x:
uses: ./.github/workflows/build-kata-static-tarball-s390x.yaml
with:
push-to-registry: yes
stage: release
secrets: inherit
kata-deploy:
needs: build-kata-static-tarball-s390x
runs-on: s390x
steps:
- name: Login to Kata Containers docker.io
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to Kata Containers quay.io
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: get-kata-tarball
uses: actions/download-artifact@v4
uses: actions/download-artifact@v3
with:
name: kata-static-tarball-s390x
- name: build-and-push-kata-deploy-ci-s390x
id: build-and-push-kata-deploy-ci-s390x
run: |
# We need to do such trick here as the format of the $GITHUB_REF
# We need to do such trick here as the format of the $GITHUB_REF
# is "refs/tags/<tag>"
tag=$(echo $GITHUB_REF | cut -d/ -f3-)
if [ "${tag}" = "main" ]; then
tag=$(./tools/packaging/release/release.sh release-version)
tags=(${tag} "latest")
else
tags=(${tag})
fi
tags=($tag)
tags+=($([[ "$tag" =~ "alpha"|"rc" ]] && echo "latest" || echo "stable"))
for tag in ${tags[@]}; do
./tools/packaging/kata-deploy/local-build/kata-deploy-build-and-upload-payload.sh \
$(pwd)/kata-static.tar.xz "docker.io/katadocker/kata-deploy" \

View File

@@ -1,189 +1,179 @@
name: Release Kata Containers
name: Publish Kata release artifacts
on:
workflow_dispatch
push:
tags:
- '[0-9]+.[0-9]+.[0-9]+*'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
release:
runs-on: ubuntu-22.04
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Create a new release
run: |
./tools/packaging/release/release.sh create-new-release
env:
GH_TOKEN: ${{ github.token }}
build-and-push-assets-amd64:
needs: release
uses: ./.github/workflows/release-amd64.yaml
with:
target-arch: amd64
secrets: inherit
build-and-push-assets-arm64:
needs: release
uses: ./.github/workflows/release-arm64.yaml
with:
target-arch: arm64
secrets: inherit
build-and-push-assets-s390x:
needs: release
uses: ./.github/workflows/release-s390x.yaml
with:
target-arch: s390x
secrets: inherit
build-and-push-assets-ppc64le:
needs: release
uses: ./.github/workflows/release-ppc64le.yaml
with:
target-arch: ppc64le
secrets: inherit
publish-multi-arch-images:
runs-on: ubuntu-22.04
needs: [build-and-push-assets-amd64, build-and-push-assets-arm64, build-and-push-assets-s390x, build-and-push-assets-ppc64le]
runs-on: ubuntu-latest
needs: [build-and-push-assets-amd64, build-and-push-assets-arm64, build-and-push-assets-s390x]
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v3
- name: Login to Kata Containers docker.io
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to Kata Containers quay.io
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.QUAY_DEPLOYER_USERNAME }}
password: ${{ secrets.QUAY_DEPLOYER_PASSWORD }}
- name: Get the image tags
- name: Push multi-arch manifest
run: |
release_version=$(./tools/packaging/release/release.sh release-version)
echo "KATA_DEPLOY_IMAGE_TAGS=$release_version latest" >> "$GITHUB_ENV"
# tag the container image we created and push to DockerHub
tag=$(echo $GITHUB_REF | cut -d/ -f3-)
tags=($tag)
tags+=($([[ "$tag" =~ "alpha"|"rc" ]] && echo "latest" || echo "stable"))
# push to quay.io and docker.io
for tag in ${tags[@]}; do
docker manifest create quay.io/kata-containers/kata-deploy:${tag} \
--amend quay.io/kata-containers/kata-deploy:${tag}-amd64 \
--amend quay.io/kata-containers/kata-deploy:${tag}-arm64 \
--amend quay.io/kata-containers/kata-deploy:${tag}-s390x
- name: Publish multi-arch manifest on docker.io and quay.io
run: |
./tools/packaging/release/release.sh publish-multiarch-manifest
env:
KATA_DEPLOY_REGISTRIES: "quay.io/kata-containers/kata-deploy docker.io/katadocker/kata-deploy"
docker manifest create docker.io/katadocker/kata-deploy:${tag} \
--amend docker.io/katadocker/kata-deploy:${tag}-amd64 \
--amend docker.io/katadocker/kata-deploy:${tag}-arm64 \
--amend docker.io/katadocker/kata-deploy:${tag}-s390x
docker manifest push quay.io/kata-containers/kata-deploy:${tag}
docker manifest push docker.io/katadocker/kata-deploy:${tag}
done
upload-multi-arch-static-tarball:
needs: [build-and-push-assets-amd64, build-and-push-assets-arm64, build-and-push-assets-s390x, build-and-push-assets-ppc64le]
runs-on: ubuntu-22.04
needs: publish-multi-arch-images
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set KATA_STATIC_TARBALL env var
- uses: actions/checkout@v3
- name: install hub
run: |
tarball=$(pwd)/kata-static.tar.xz
echo "KATA_STATIC_TARBALL=${tarball}" >> "$GITHUB_ENV"
wget -q -O- https://github.com/mislav/hub/releases/download/v2.14.2/hub-linux-amd64-2.14.2.tgz | \
tar xz --strip-components=2 --wildcards '*/bin/hub' && sudo mv hub /usr/local/bin/hub
- name: Download amd64 artifacts
uses: actions/download-artifact@v4
- name: download-artifacts-amd64
uses: actions/download-artifact@v3
with:
name: kata-static-tarball-amd64
- name: Upload amd64 static tarball to GitHub
- name: push amd64 static tarball to github
run: |
./tools/packaging/release/release.sh upload-kata-static-tarball
env:
GH_TOKEN: ${{ github.token }}
ARCHITECTURE: amd64
tag=$(echo $GITHUB_REF | cut -d/ -f3-)
tarball="kata-static-$tag-amd64.tar.xz"
mv kata-static.tar.xz "$GITHUB_WORKSPACE/${tarball}"
pushd $GITHUB_WORKSPACE
echo "uploading asset '${tarball}' for tag: ${tag}"
GITHUB_TOKEN=${{ secrets.GIT_UPLOAD_TOKEN }} hub release edit -m "" -a "${tarball}" "${tag}"
popd
- name: Download arm64 artifacts
uses: actions/download-artifact@v4
- name: download-artifacts-arm64
uses: actions/download-artifact@v3
with:
name: kata-static-tarball-arm64
- name: Upload arm64 static tarball to GitHub
- name: push arm64 static tarball to github
run: |
./tools/packaging/release/release.sh upload-kata-static-tarball
env:
GH_TOKEN: ${{ github.token }}
ARCHITECTURE: arm64
tag=$(echo $GITHUB_REF | cut -d/ -f3-)
tarball="kata-static-$tag-arm64.tar.xz"
mv kata-static.tar.xz "$GITHUB_WORKSPACE/${tarball}"
pushd $GITHUB_WORKSPACE
echo "uploading asset '${tarball}' for tag: ${tag}"
GITHUB_TOKEN=${{ secrets.GIT_UPLOAD_TOKEN }} hub release edit -m "" -a "${tarball}" "${tag}"
popd
- name: Download s390x artifacts
uses: actions/download-artifact@v4
- name: download-artifacts-s390x
uses: actions/download-artifact@v3
with:
name: kata-static-tarball-s390x
- name: Upload s390x static tarball to GitHub
- name: push s390x static tarball to github
run: |
./tools/packaging/release/release.sh upload-kata-static-tarball
env:
GH_TOKEN: ${{ github.token }}
ARCHITECTURE: s390x
- name: Download ppc64le artifacts
uses: actions/download-artifact@v4
with:
name: kata-static-tarball-ppc64le
- name: Upload ppc64le static tarball to GitHub
run: |
./tools/packaging/release/release.sh upload-kata-static-tarball
env:
GH_TOKEN: ${{ github.token }}
ARCHITECTURE: ppc64le
tag=$(echo $GITHUB_REF | cut -d/ -f3-)
tarball="kata-static-$tag-s390x.tar.xz"
mv kata-static.tar.xz "$GITHUB_WORKSPACE/${tarball}"
pushd $GITHUB_WORKSPACE
echo "uploading asset '${tarball}' for tag: ${tag}"
GITHUB_TOKEN=${{ secrets.GIT_UPLOAD_TOKEN }} hub release edit -m "" -a "${tarball}" "${tag}"
popd
upload-versions-yaml:
needs: release
runs-on: ubuntu-22.04
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Upload versions.yaml to GitHub
run: |
./tools/packaging/release/release.sh upload-versions-yaml-file
- uses: actions/checkout@v3
- name: upload versions.yaml
env:
GH_TOKEN: ${{ github.token }}
GITHUB_TOKEN: ${{ secrets.GIT_UPLOAD_TOKEN }}
run: |
tag=$(echo $GITHUB_REF | cut -d/ -f3-)
pushd $GITHUB_WORKSPACE
versions_file="kata-containers-$tag-versions.yaml"
cp versions.yaml ${versions_file}
hub release edit -m "" -a "${versions_file}" "${tag}"
popd
upload-cargo-vendored-tarball:
needs: release
runs-on: ubuntu-22.04
needs: upload-multi-arch-static-tarball
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Generate and upload vendored code tarball
- uses: actions/checkout@v3
- name: generate-and-upload-tarball
run: |
./tools/packaging/release/release.sh upload-vendored-code-tarball
env:
GH_TOKEN: ${{ github.token }}
tag=$(echo $GITHUB_REF | cut -d/ -f3-)
tarball="kata-containers-$tag-vendor.tar.gz"
pushd $GITHUB_WORKSPACE
bash -c "tools/packaging/release/generate_vendor.sh ${tarball}"
GITHUB_TOKEN=${{ secrets.GIT_UPLOAD_TOKEN }} hub release edit -m "" -a "${tarball}" "${tag}"
popd
upload-libseccomp-tarball:
needs: release
runs-on: ubuntu-22.04
needs: upload-cargo-vendored-tarball
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Download libseccomp tarball and upload it to GitHub
run: |
./tools/packaging/release/release.sh upload-libseccomp-tarball
- uses: actions/checkout@v3
- name: download-and-upload-tarball
env:
GH_TOKEN: ${{ github.token }}
publish-release:
needs: [ build-and-push-assets-amd64, build-and-push-assets-arm64, build-and-push-assets-s390x, build-and-push-assets-ppc64le, publish-multi-arch-images, upload-multi-arch-static-tarball, upload-versions-yaml, upload-cargo-vendored-tarball, upload-libseccomp-tarball ]
runs-on: ubuntu-22.04
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Publish a release
GITHUB_TOKEN: ${{ secrets.GIT_UPLOAD_TOKEN }}
GOPATH: ${HOME}/go
run: |
./tools/packaging/release/release.sh publish-release
env:
GH_TOKEN: ${{ github.token }}
pushd $GITHUB_WORKSPACE
./ci/install_yq.sh
tag=$(echo $GITHUB_REF | cut -d/ -f3-)
versions_yaml="versions.yaml"
version=$(${GOPATH}/bin/yq read ${versions_yaml} "externals.libseccomp.version")
repo_url=$(${GOPATH}/bin/yq read ${versions_yaml} "externals.libseccomp.url")
download_url="${repo_url}/releases/download/v${version}"
tarball="libseccomp-${version}.tar.gz"
asc="${tarball}.asc"
curl -sSLO "${download_url}/${tarball}"
curl -sSLO "${download_url}/${asc}"
# "-m" option should be empty to re-use the existing release title
# without opening a text editor.
# For the details, check https://hub.github.com/hub-release.1.html.
hub release edit -m "" -a "${tarball}" "${tag}"
hub release edit -m "" -a "${asc}" "${tag}"
popd

View File

@@ -0,0 +1,58 @@
# Copyright (c) 2020 Intel Corporation
#
# SPDX-License-Identifier: Apache-2.0
#
name: Ensure PR has required porting labels
on:
pull_request_target:
types:
- opened
- reopened
- labeled
- unlabeled
branches:
- main
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
check-pr-porting-labels:
runs-on: ubuntu-latest
steps:
- name: Install hub
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
run: |
HUB_ARCH="amd64"
HUB_VER=$(curl -sL "https://api.github.com/repos/github/hub/releases/latest" |\
jq -r .tag_name | sed 's/^v//')
curl -sL \
"https://github.com/github/hub/releases/download/v${HUB_VER}/hub-linux-${HUB_ARCH}-${HUB_VER}.tgz" |\
tar xz --strip-components=2 --wildcards '*/bin/hub' && \
sudo install hub /usr/local/bin
- name: Checkout code to allow hub to communicate with the project
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
uses: actions/checkout@v2
- name: Install porting checker script
run: |
# Clone into a temporary directory to avoid overwriting
# any existing github directory.
pushd $(mktemp -d) &>/dev/null
git clone --single-branch --depth 1 "https://github.com/kata-containers/.github" && cd .github/scripts
sudo install pr-porting-checks.sh /usr/local/bin
popd &>/dev/null
- name: Stop PR being merged unless it has a correct set of porting labels
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
env:
GITHUB_TOKEN: ${{ secrets.KATA_GITHUB_ACTIONS_TOKEN }}
run: |
pr=${{ github.event.number }}
repo=${{ github.repository }}
pr-porting-checks.sh "$pr" "$repo"

View File

@@ -1,67 +0,0 @@
name: CI | Run cri-containerd tests on ppc64le
on:
workflow_call:
inputs:
tarball-suffix:
required: false
type: string
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
run-cri-containerd:
strategy:
# We can set this to true whenever we're 100% sure that
# the all the tests are not flaky, otherwise we'll fail
# all the tests due to a single flaky instance
fail-fast: false
matrix:
containerd_version: ['active']
vmm: ['qemu']
runs-on: ppc64le
env:
CONTAINERD_VERSION: ${{ matrix.containerd_version }}
GOPATH: ${{ github.workspace }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
steps:
- name: Adjust a permission for repo
run: sudo chown -R $USER:$USER $GITHUB_WORKSPACE
- name: Prepare the self-hosted runner
run: |
bash ${HOME}/scripts/prepare_runner.sh cri-containerd
sudo rm -rf $GITHUB_WORKSPACE/*
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install dependencies
run: bash tests/integration/cri-containerd/gha-run.sh install-dependencies
- name: get-kata-tarball
uses: actions/download-artifact@v4
with:
name: kata-static-tarball-ppc64le${{ inputs.tarball-suffix }}
path: kata-artifacts
- name: Install kata
run: bash tests/integration/cri-containerd/gha-run.sh install-kata kata-artifacts
- name: Run cri-containerd tests
run: bash tests/integration/cri-containerd/gha-run.sh run
- name: Cleanup actions for the self hosted runner
run: ${HOME}/scripts/cleanup_runner.sh

View File

@@ -8,45 +8,31 @@ on:
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
run-cri-containerd:
strategy:
# We can set this to true whenever we're 100% sure that
# the all the tests are not flaky, otherwise we'll fail
# all the tests due to a single flaky instance
fail-fast: false
fail-fast: true
matrix:
containerd_version: ['active']
vmm: ['qemu', 'qemu-runtime-rs']
runs-on: s390x-large
containerd_version: ['lts', 'active']
vmm: ['clh', 'qemu']
runs-on: garm-ubuntu-2204
env:
CONTAINERD_VERSION: ${{ matrix.containerd_version }}
GOPATH: ${{ github.workspace }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install dependencies
run: bash tests/integration/cri-containerd/gha-run.sh install-dependencies
- name: get-kata-tarball
uses: actions/download-artifact@v4
uses: actions/download-artifact@v3
with:
name: kata-static-tarball-s390x${{ inputs.tarball-suffix }}
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-artifacts
- name: Install kata

View File

@@ -2,9 +2,6 @@ name: CI | Run kubernetes tests on AKS
on:
workflow_call:
inputs:
tarball-suffix:
required: false
type: string
registry:
required: true
type: string
@@ -20,10 +17,6 @@ on:
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
run-k8s-tests:
@@ -36,28 +29,10 @@ jobs:
- clh
- dragonball
- qemu
- qemu-runtime-rs
- stratovirt
- cloud-hypervisor
instance-type:
- small
- normal
include:
- host_os: cbl-mariner
vmm: clh
instance-type: small
genpolicy-pull-method: oci-distribution
auto-generate-policy: yes
- host_os: cbl-mariner
vmm: clh
instance-type: small
genpolicy-pull-method: containerd
auto-generate-policy: yes
- host_os: cbl-mariner
vmm: clh
instance-type: normal
auto-generate-policy: yes
runs-on: ubuntu-22.04
runs-on: ubuntu-latest
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
@@ -65,31 +40,11 @@ jobs:
GH_PR_NUMBER: ${{ inputs.pr-number }}
KATA_HOST_OS: ${{ matrix.host_os }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
KUBERNETES: "vanilla"
USING_NFD: "false"
K8S_TEST_HOST_TYPE: ${{ matrix.instance-type }}
GENPOLICY_PULL_METHOD: ${{ matrix.genpolicy-pull-method }}
AUTO_GENERATE_POLICY: ${{ matrix.auto-generate-policy }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: get-kata-tarball
uses: actions/download-artifact@v4
with:
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-artifacts
- name: Install kata
run: bash tests/integration/kubernetes/gha-run.sh install-kata-tools kata-artifacts
- name: Download Azure CLI
run: bash tests/integration/kubernetes/gha-run.sh install-azure-cli
@@ -100,7 +55,6 @@ jobs:
AZ_APPID: ${{ secrets.AZ_APPID }}
AZ_PASSWORD: ${{ secrets.AZ_PASSWORD }}
AZ_TENANT_ID: ${{ secrets.AZ_TENANT_ID }}
AZ_SUBSCRIPTION_ID: ${{ secrets.AZ_SUBSCRIPTION_ID }}
- name: Create AKS cluster
timeout-minutes: 10
@@ -118,7 +72,7 @@ jobs:
- name: Deploy Kata
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata-aks
- name: Run tests
timeout-minutes: 60
run: bash tests/integration/kubernetes/gha-run.sh run-tests

View File

@@ -1,107 +0,0 @@
name: CI | Run kubernetes tests on amd64
on:
workflow_call:
inputs:
registry:
required: true
type: string
repo:
required: true
type: string
tag:
required: true
type: string
pr-number:
required: true
type: string
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
run-k8s-tests-amd64:
strategy:
fail-fast: false
matrix:
vmm:
- clh #cloud-hypervisor
- dragonball
- fc #firecracker
- qemu
- cloud-hypervisor
container_runtime:
- containerd
snapshotter:
- devmapper
k8s:
- k3s
include:
- vmm: qemu
container_runtime: crio
snapshotter: ""
k8s: k0s
runs-on: ubuntu-22.04
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
PR_NUMBER: ${{ inputs.pr-number }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
KUBERNETES: ${{ matrix.k8s }}
KUBERNETES_EXTRA_PARAMS: ${{ matrix.container_runtime != 'crio' && '' || '--cri-socket remote:unix:///var/run/crio/crio.sock --kubelet-extra-args --cgroup-driver="systemd"' }}
SNAPSHOTTER: ${{ matrix.snapshotter }}
USING_NFD: "false"
K8S_TEST_HOST_TYPE: all
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Configure CRI-O
if: matrix.container_runtime == 'crio'
run: bash tests/integration/kubernetes/gha-run.sh setup-crio
- name: Deploy ${{ matrix.k8s }}
run: bash tests/integration/kubernetes/gha-run.sh deploy-k8s
- name: Configure the ${{ matrix.snapshotter }} snapshotter
if: matrix.snapshotter != ''
run: bash tests/integration/kubernetes/gha-run.sh configure-snapshotter
- name: Deploy Kata
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata
- name: Install `bats`
run: bash tests/integration/kubernetes/gha-run.sh install-bats
- name: Run tests
timeout-minutes: 30
run: bash tests/integration/kubernetes/gha-run.sh run-tests
- name: Collect artifacts ${{ matrix.vmm }}
if: always()
run: bash tests/integration/kubernetes/gha-run.sh collect-artifacts
continue-on-error: true
- name: Archive artifacts ${{ matrix.vmm }}
uses: actions/upload-artifact@v4
with:
name: k8s-tests-${{ matrix.vmm }}-${{ matrix.snapshotter }}-${{ matrix.k8s }}-${{ matrix.instance }}-${{ inputs.tag }}
path: /tmp/artifacts
retention-days: 1
- name: Delete kata-deploy
if: always()
run: bash tests/integration/kubernetes/gha-run.sh cleanup

View File

@@ -1,82 +0,0 @@
name: CI | Run kubernetes tests on Power(ppc64le)
on:
workflow_call:
inputs:
registry:
required: true
type: string
repo:
required: true
type: string
tag:
required: true
type: string
pr-number:
required: true
type: string
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
run-k8s-tests:
strategy:
fail-fast: false
matrix:
vmm:
- qemu
k8s:
- kubeadm
runs-on: k8s-ppc64le
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
PR_NUMBER: ${{ inputs.pr-number }}
GOPATH: ${{ github.workspace }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
KUBERNETES: ${{ matrix.k8s }}
USING_NFD: "false"
TARGET_ARCH: "ppc64le"
steps:
- name: Prepare the self-hosted runner
run: |
bash ${HOME}/scripts/prepare_runner.sh kubernetes
sudo rm -rf $GITHUB_WORKSPACE/*
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install golang
run: |
./tests/install_go.sh -f -p
echo "/usr/local/go/bin" >> $GITHUB_PATH
- name: Prepare the runner for k8s cluster creation
run: bash ${HOME}/scripts/k8s_cluster_cleanup.sh
- name: Create k8s cluster using kubeadm
run: bash ${HOME}/scripts/k8s_cluster_create.sh
- name: Deploy Kata
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata-kubeadm
- name: Run tests
timeout-minutes: 30
run: bash tests/integration/kubernetes/gha-run.sh run-tests
- name: Delete cluster and post cleanup actions
run: bash ${HOME}/scripts/k8s_cluster_cleanup.sh

View File

@@ -0,0 +1,52 @@
name: CI | Run kubernetes tests on SEV
on:
workflow_call:
inputs:
registry:
required: true
type: string
repo:
required: true
type: string
tag:
required: true
type: string
pr-number:
required: true
type: string
commit-hash:
required: false
type: string
jobs:
run-k8s-tests:
strategy:
fail-fast: false
matrix:
vmm:
- qemu-sev
runs-on: sev
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
PR_NUMBER: ${{ inputs.pr-number }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
KUBECONFIG: /home/kata/.kube/config
USING_NFD: "false"
steps:
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
- name: Deploy Kata
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata-sev
- name: Run tests
timeout-minutes: 30
run: bash tests/integration/kubernetes/gha-run.sh run-tests
- name: Delete kata-deploy
if: always()
run: bash tests/integration/kubernetes/gha-run.sh cleanup-sev

View File

@@ -0,0 +1,52 @@
name: CI | Run kubernetes tests on SEV-SNP
on:
workflow_call:
inputs:
registry:
required: true
type: string
repo:
required: true
type: string
tag:
required: true
type: string
pr-number:
required: true
type: string
commit-hash:
required: false
type: string
jobs:
run-k8s-tests:
strategy:
fail-fast: false
matrix:
vmm:
- qemu-snp
runs-on: sev-snp
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
PR_NUMBER: ${{ inputs.pr-number }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
KUBECONFIG: /home/kata/.kube/config
USING_NFD: "false"
steps:
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
- name: Deploy Kata
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata-snp
- name: Run tests
timeout-minutes: 30
run: bash tests/integration/kubernetes/gha-run.sh run-tests
- name: Delete kata-deploy
if: always()
run: bash tests/integration/kubernetes/gha-run.sh cleanup-snp

View File

@@ -0,0 +1,51 @@
name: CI | Run kubernetes tests on TDX
on:
workflow_call:
inputs:
registry:
required: true
type: string
repo:
required: true
type: string
tag:
required: true
type: string
pr-number:
required: true
type: string
commit-hash:
required: false
type: string
jobs:
run-k8s-tests:
strategy:
fail-fast: false
matrix:
vmm:
- qemu-tdx
runs-on: tdx
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
PR_NUMBER: ${{ inputs.pr-number }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
USING_NFD: "true"
steps:
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
- name: Deploy Kata
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata-tdx
- name: Run tests
timeout-minutes: 30
run: bash tests/integration/kubernetes/gha-run.sh run-tests
- name: Delete kata-deploy
if: always()
run: bash tests/integration/kubernetes/gha-run.sh cleanup-tdx

View File

@@ -1,141 +0,0 @@
name: CI | Run kubernetes tests on IBM Cloud Z virtual server instance (zVSI)
on:
workflow_call:
inputs:
registry:
required: true
type: string
repo:
required: true
type: string
tag:
required: true
type: string
pr-number:
required: true
type: string
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
run-k8s-tests:
strategy:
fail-fast: false
matrix:
snapshotter:
- overlayfs
- devmapper
- nydus
vmm:
- qemu
- qemu-runtime-rs
- qemu-coco-dev
k8s:
- k3s
include:
- snapshotter: devmapper
pull-type: default
using-nfd: true
deploy-cmd: configure-snapshotter
- snapshotter: nydus
pull-type: guest-pull
using-nfd: false
deploy-cmd: deploy-snapshotter
exclude:
- snapshotter: overlayfs
vmm: qemu
- snapshotter: overlayfs
vmm: qemu-coco-dev
- snapshotter: devmapper
vmm: qemu-runtime-rs
- snapshotter: devmapper
vmm: qemu-coco-dev
- snapshotter: nydus
vmm: qemu
- snapshotter: nydus
vmm: qemu-runtime-rs
runs-on: s390x-large
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
PR_NUMBER: ${{ inputs.pr-number }}
GH_PR_NUMBER: ${{ inputs.pr-number }}
KATA_HOST_OS: "ubuntu"
KATA_HYPERVISOR: ${{ matrix.vmm }}
KUBERNETES: ${{ matrix.k8s }}
PULL_TYPE: ${{ matrix.pull-type }}
SNAPSHOTTER: ${{ matrix.snapshotter }}
USING_NFD: ${{ matrix.using-nfd }}
TARGET_ARCH: "s390x"
AUTHENTICATED_IMAGE_USER: ${{ secrets.AUTHENTICATED_IMAGE_USER }}
AUTHENTICATED_IMAGE_PASSWORD: ${{ secrets.AUTHENTICATED_IMAGE_PASSWORD }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Set SNAPSHOTTER to empty if overlayfs
run: echo "SNAPSHOTTER=" >> $GITHUB_ENV
if: ${{ matrix.snapshotter == 'overlayfs' }}
- name: Set KBS and KBS_INGRESS if qemu-coco-dev
run: |
echo "KBS=true" >> $GITHUB_ENV
echo "KBS_INGRESS=nodeport" >> $GITHUB_ENV
if: ${{ matrix.vmm == 'qemu-coco-dev' }}
- name: Deploy ${{ matrix.k8s }}
run: bash tests/integration/kubernetes/gha-run.sh deploy-k8s
# qemu-runtime-rs only works with overlayfs
# See: https://github.com/kata-containers/kata-containers/issues/10066
- name: Configure the ${{ matrix.snapshotter }} snapshotter
run: bash tests/integration/kubernetes/gha-run.sh ${{ matrix.deploy-cmd }}
if: ${{ matrix.snapshotter != 'overlayfs' }}
- name: Deploy Kata
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata-zvsi
- name: Uninstall previous `kbs-client`
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh uninstall-kbs-client
if: ${{ matrix.vmm == 'qemu-coco-dev' }}
- name: Deploy CoCo KBS
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-coco-kbs
if: ${{ matrix.vmm == 'qemu-coco-dev' }}
- name: Install `kbs-client`
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh install-kbs-client
if: ${{ matrix.vmm == 'qemu-coco-dev' }}
- name: Run tests
timeout-minutes: 60
run: bash tests/integration/kubernetes/gha-run.sh run-tests
- name: Delete kata-deploy
if: always()
run: bash tests/integration/kubernetes/gha-run.sh cleanup-zvsi
- name: Delete CoCo KBS
if: always()
run: |
if [ "${KBS}" == "true" ]; then
bash tests/integration/kubernetes/gha-run.sh delete-coco-kbs
fi

View File

@@ -1,113 +0,0 @@
name: CI | Run Kata CoCo k8s Stability Tests
on:
workflow_call:
inputs:
registry:
required: true
type: string
repo:
required: true
type: string
tag:
required: true
type: string
pr-number:
required: true
type: string
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
# Generate jobs for testing CoCo on non-TEE environments
run-stability-k8s-tests-coco-nontee:
strategy:
fail-fast: false
matrix:
vmm:
- qemu-coco-dev
snapshotter:
- nydus
pull-type:
- guest-pull
runs-on: ubuntu-latest
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
GH_PR_NUMBER: ${{ inputs.pr-number }}
KATA_HOST_OS: ${{ matrix.host_os }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
# Some tests rely on that variable to run (or not)
KBS: "true"
# Set the KBS ingress handler (empty string disables handling)
KBS_INGRESS: "aks"
KUBERNETES: "vanilla"
PULL_TYPE: ${{ matrix.pull-type }}
AUTHENTICATED_IMAGE_USER: ${{ secrets.AUTHENTICATED_IMAGE_USER }}
AUTHENTICATED_IMAGE_PASSWORD: ${{ secrets.AUTHENTICATED_IMAGE_PASSWORD }}
SNAPSHOTTER: ${{ matrix.snapshotter }}
USING_NFD: "false"
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Download Azure CLI
run: bash tests/integration/kubernetes/gha-run.sh install-azure-cli
- name: Log into the Azure account
run: bash tests/integration/kubernetes/gha-run.sh login-azure
env:
AZ_APPID: ${{ secrets.AZ_APPID }}
AZ_PASSWORD: ${{ secrets.AZ_PASSWORD }}
AZ_TENANT_ID: ${{ secrets.AZ_TENANT_ID }}
AZ_SUBSCRIPTION_ID: ${{ secrets.AZ_SUBSCRIPTION_ID }}
- name: Create AKS cluster
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh create-cluster
- name: Install `bats`
run: bash tests/integration/kubernetes/gha-run.sh install-bats
- name: Install `kubectl`
run: bash tests/integration/kubernetes/gha-run.sh install-kubectl
- name: Download credentials for the Kubernetes CLI to use them
run: bash tests/integration/kubernetes/gha-run.sh get-cluster-credentials
- name: Deploy Snapshotter
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh deploy-snapshotter
- name: Deploy Kata
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata-aks
- name: Deploy CoCo KBS
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-coco-kbs
- name: Install `kbs-client`
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh install-kbs-client
- name: Run stability tests
timeout-minutes: 300
run: bash tests/stability/gha-stability-run.sh run-tests
- name: Delete AKS cluster
if: always()
run: bash tests/integration/kubernetes/gha-run.sh delete-cluster

View File

@@ -1,323 +0,0 @@
name: CI | Run kata coco tests
on:
workflow_call:
inputs:
registry:
required: true
type: string
repo:
required: true
type: string
tag:
required: true
type: string
pr-number:
required: true
type: string
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
run-k8s-tests-on-tdx:
strategy:
fail-fast: false
matrix:
vmm:
- qemu-tdx
snapshotter:
- nydus
pull-type:
- guest-pull
runs-on: tdx
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
PR_NUMBER: ${{ inputs.pr-number }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
KUBERNETES: "vanilla"
USING_NFD: "true"
KBS: "true"
K8S_TEST_HOST_TYPE: "baremetal"
KBS_INGRESS: "nodeport"
SNAPSHOTTER: ${{ matrix.snapshotter }}
PULL_TYPE: ${{ matrix.pull-type }}
AUTHENTICATED_IMAGE_USER: ${{ secrets.AUTHENTICATED_IMAGE_USER }}
AUTHENTICATED_IMAGE_PASSWORD: ${{ secrets.AUTHENTICATED_IMAGE_PASSWORD }}
ITA_KEY: ${{ secrets.ITA_KEY }}
AUTO_GENERATE_POLICY: "yes"
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Deploy Snapshotter
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh deploy-snapshotter
- name: Deploy Kata
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata-tdx
- name: Uninstall previous `kbs-client`
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh uninstall-kbs-client
- name: Deploy CoCo KBS
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-coco-kbs
- name: Install `kbs-client`
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh install-kbs-client
- name: Run tests
timeout-minutes: 100
run: bash tests/integration/kubernetes/gha-run.sh run-tests
- name: Delete kata-deploy
if: always()
run: bash tests/integration/kubernetes/gha-run.sh cleanup-tdx
- name: Delete Snapshotter
if: always()
run: bash tests/integration/kubernetes/gha-run.sh cleanup-snapshotter
- name: Delete CoCo KBS
if: always()
run: bash tests/integration/kubernetes/gha-run.sh delete-coco-kbs
run-k8s-tests-on-sev:
strategy:
fail-fast: false
matrix:
vmm:
- qemu-sev
snapshotter:
- nydus
pull-type:
- guest-pull
runs-on: sev
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
PR_NUMBER: ${{ inputs.pr-number }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
KUBECONFIG: /home/kata/.kube/config
KUBERNETES: "vanilla"
USING_NFD: "false"
K8S_TEST_HOST_TYPE: "baremetal"
SNAPSHOTTER: ${{ matrix.snapshotter }}
PULL_TYPE: ${{ matrix.pull-type }}
AUTHENTICATED_IMAGE_USER: ${{ secrets.AUTHENTICATED_IMAGE_USER }}
AUTHENTICATED_IMAGE_PASSWORD: ${{ secrets.AUTHENTICATED_IMAGE_PASSWORD }}
AUTO_GENERATE_POLICY: "yes"
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Deploy Snapshotter
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh deploy-snapshotter
- name: Deploy Kata
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata-sev
- name: Run tests
timeout-minutes: 50
run: bash tests/integration/kubernetes/gha-run.sh run-tests
- name: Delete kata-deploy
if: always()
run: bash tests/integration/kubernetes/gha-run.sh cleanup-sev
- name: Delete Snapshotter
if: always()
run: bash tests/integration/kubernetes/gha-run.sh cleanup-snapshotter
run-k8s-tests-sev-snp:
strategy:
fail-fast: false
matrix:
vmm:
- qemu-snp
snapshotter:
- nydus
pull-type:
- guest-pull
runs-on: sev-snp
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
PR_NUMBER: ${{ inputs.pr-number }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
KUBECONFIG: /home/kata/.kube/config
KUBERNETES: "vanilla"
USING_NFD: "false"
KBS: "true"
KBS_INGRESS: "nodeport"
K8S_TEST_HOST_TYPE: "baremetal"
SNAPSHOTTER: ${{ matrix.snapshotter }}
PULL_TYPE: ${{ matrix.pull-type }}
AUTHENTICATED_IMAGE_USER: ${{ secrets.AUTHENTICATED_IMAGE_USER }}
AUTHENTICATED_IMAGE_PASSWORD: ${{ secrets.AUTHENTICATED_IMAGE_PASSWORD }}
AUTO_GENERATE_POLICY: "yes"
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Deploy Snapshotter
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh deploy-snapshotter
- name: Deploy Kata
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata-snp
- name: Uninstall previous `kbs-client`
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh uninstall-kbs-client
- name: Deploy CoCo KBS
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-coco-kbs
- name: Install `kbs-client`
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh install-kbs-client
- name: Run tests
timeout-minutes: 50
run: bash tests/integration/kubernetes/gha-run.sh run-tests
- name: Delete kata-deploy
if: always()
run: bash tests/integration/kubernetes/gha-run.sh cleanup-snp
- name: Delete Snapshotter
if: always()
run: bash tests/integration/kubernetes/gha-run.sh cleanup-snapshotter
- name: Delete CoCo KBS
if: always()
run: bash tests/integration/kubernetes/gha-run.sh delete-coco-kbs
# Generate jobs for testing CoCo on non-TEE environments
run-k8s-tests-coco-nontee:
strategy:
fail-fast: false
matrix:
vmm:
- qemu-coco-dev
snapshotter:
- nydus
pull-type:
- guest-pull
runs-on: ubuntu-22.04
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
GH_PR_NUMBER: ${{ inputs.pr-number }}
KATA_HOST_OS: ${{ matrix.host_os }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
# Some tests rely on that variable to run (or not)
KBS: "true"
# Set the KBS ingress handler (empty string disables handling)
KBS_INGRESS: "aks"
KUBERNETES: "vanilla"
PULL_TYPE: ${{ matrix.pull-type }}
AUTHENTICATED_IMAGE_USER: ${{ secrets.AUTHENTICATED_IMAGE_USER }}
AUTHENTICATED_IMAGE_PASSWORD: ${{ secrets.AUTHENTICATED_IMAGE_PASSWORD }}
SNAPSHOTTER: ${{ matrix.snapshotter }}
USING_NFD: "false"
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Download Azure CLI
run: bash tests/integration/kubernetes/gha-run.sh install-azure-cli
- name: Log into the Azure account
run: bash tests/integration/kubernetes/gha-run.sh login-azure
env:
AZ_APPID: ${{ secrets.AZ_APPID }}
AZ_PASSWORD: ${{ secrets.AZ_PASSWORD }}
AZ_TENANT_ID: ${{ secrets.AZ_TENANT_ID }}
AZ_SUBSCRIPTION_ID: ${{ secrets.AZ_SUBSCRIPTION_ID }}
- name: Create AKS cluster
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh create-cluster
- name: Install `bats`
run: bash tests/integration/kubernetes/gha-run.sh install-bats
- name: Install `kubectl`
run: bash tests/integration/kubernetes/gha-run.sh install-kubectl
- name: Download credentials for the Kubernetes CLI to use them
run: bash tests/integration/kubernetes/gha-run.sh get-cluster-credentials
- name: Deploy Snapshotter
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh deploy-snapshotter
- name: Deploy Kata
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata-aks
- name: Deploy CoCo KBS
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-coco-kbs
- name: Install `kbs-client`
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh install-kbs-client
- name: Run tests
timeout-minutes: 60
run: bash tests/integration/kubernetes/gha-run.sh run-tests
- name: Delete AKS cluster
if: always()
run: bash tests/integration/kubernetes/gha-run.sh delete-cluster

View File

@@ -1,91 +0,0 @@
name: CI | Run kata-deploy tests on AKS
on:
workflow_call:
inputs:
registry:
required: true
type: string
repo:
required: true
type: string
tag:
required: true
type: string
pr-number:
required: true
type: string
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
run-kata-deploy-tests:
strategy:
fail-fast: false
matrix:
host_os:
- ubuntu
vmm:
- clh
- dragonball
- qemu
- qemu-runtime-rs
include:
- host_os: cbl-mariner
vmm: clh
runs-on: ubuntu-22.04
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
GH_PR_NUMBER: ${{ inputs.pr-number }}
KATA_HOST_OS: ${{ matrix.host_os }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
KUBERNETES: "vanilla"
USING_NFD: "false"
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Download Azure CLI
run: bash tests/functional/kata-deploy/gha-run.sh install-azure-cli
- name: Log into the Azure account
run: bash tests/functional/kata-deploy/gha-run.sh login-azure
env:
AZ_APPID: ${{ secrets.AZ_APPID }}
AZ_PASSWORD: ${{ secrets.AZ_PASSWORD }}
AZ_TENANT_ID: ${{ secrets.AZ_TENANT_ID }}
AZ_SUBSCRIPTION_ID: ${{ secrets.AZ_SUBSCRIPTION_ID }}
- name: Create AKS cluster
timeout-minutes: 10
run: bash tests/functional/kata-deploy/gha-run.sh create-cluster
- name: Install `bats`
run: bash tests/functional/kata-deploy/gha-run.sh install-bats
- name: Install `kubectl`
run: bash tests/functional/kata-deploy/gha-run.sh install-kubectl
- name: Download credentials for the Kubernetes CLI to use them
run: bash tests/functional/kata-deploy/gha-run.sh get-cluster-credentials
- name: Run tests
run: bash tests/functional/kata-deploy/gha-run.sh run-tests
- name: Delete AKS cluster
if: always()
run: bash tests/functional/kata-deploy/gha-run.sh delete-cluster

View File

@@ -1,69 +0,0 @@
name: CI | Run kata-deploy tests on GARM
on:
workflow_call:
inputs:
registry:
required: true
type: string
repo:
required: true
type: string
tag:
required: true
type: string
pr-number:
required: true
type: string
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
run-kata-deploy-tests:
strategy:
fail-fast: false
matrix:
vmm:
- clh
- qemu
k8s:
- k0s
- k3s
- rke2
# TODO: There are a couple of vmm/k8s combination failing (https://github.com/kata-containers/kata-containers/issues/9854)
# and we will put the entire kata-deploy-tests on GARM on maintenance.
# TODO: Transition to free runner (see #9940).
if: false
runs-on: garm-ubuntu-2004-smaller
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
PR_NUMBER: ${{ inputs.pr-number }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
KUBERNETES: ${{ matrix.k8s }}
USING_NFD: "false"
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Deploy ${{ matrix.k8s }}
run: bash tests/functional/kata-deploy/gha-run.sh deploy-k8s
- name: Install `bats`
run: bash tests/functional/kata-deploy/gha-run.sh install-bats
- name: Run tests
run: bash tests/functional/kata-deploy/gha-run.sh run-tests

View File

@@ -1,64 +0,0 @@
name: CI | Run kata-monitor tests
on:
workflow_call:
inputs:
tarball-suffix:
required: false
type: string
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
run-monitor:
strategy:
fail-fast: false
matrix:
vmm:
- qemu
container_engine:
- crio
- containerd
# TODO: enable when https://github.com/kata-containers/kata-containers/issues/9853 is fixed
#include:
# - container_engine: containerd
# containerd_version: lts
exclude:
# TODO: enable with containerd when https://github.com/kata-containers/kata-containers/issues/9761 is fixed
- container_engine: containerd
vmm: qemu
runs-on: ubuntu-22.04
env:
CONTAINER_ENGINE: ${{ matrix.container_engine }}
#CONTAINERD_VERSION: ${{ matrix.containerd_version }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install dependencies
run: bash tests/functional/kata-monitor/gha-run.sh install-dependencies
- name: get-kata-tarball
uses: actions/download-artifact@v4
with:
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-artifacts
- name: Install kata
run: bash tests/functional/kata-monitor/gha-run.sh install-kata kata-artifacts
- name: Run kata-monitor tests
run: bash tests/functional/kata-monitor/gha-run.sh run

View File

@@ -8,10 +8,6 @@ on:
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
setup-kata:
@@ -20,19 +16,12 @@ jobs:
env:
GOPATH: ${{ github.workspace }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: get-kata-tarball
uses: actions/download-artifact@v4
uses: actions/download-artifact@v3
with:
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-artifacts
@@ -43,12 +32,9 @@ jobs:
run-metrics:
needs: setup-kata
strategy:
# We can set this to true whenever we're 100% sure that
# the all the tests are not flaky, otherwise we'll fail
# all the tests due to a single flaky instance.
fail-fast: false
fail-fast: true
matrix:
vmm: ['clh', 'qemu', 'stratovirt']
vmm: ['clh', 'qemu']
max-parallel: 1
runs-on: metrics
env:
@@ -79,14 +65,11 @@ jobs:
- name: run iperf test
run: bash tests/metrics/gha-run.sh run-test-iperf
- name: run latency test
run: bash tests/metrics/gha-run.sh run-test-latency
- name: make metrics tarball ${{ matrix.vmm }}
run: bash tests/metrics/gha-run.sh make-tarball-results
- name: archive metrics results ${{ matrix.vmm }}
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v3
with:
name: metrics-artifacts-${{ matrix.vmm }}
path: results-${{ matrix.vmm }}.tar.gz

42
.github/workflows/run-nydus-tests.yaml vendored Normal file
View File

@@ -0,0 +1,42 @@
name: CI | Run nydus tests
on:
workflow_call:
inputs:
tarball-suffix:
required: false
type: string
commit-hash:
required: false
type: string
jobs:
run-nydus:
strategy:
fail-fast: true
matrix:
containerd_version: ['lts', 'active']
vmm: ['clh', 'qemu', 'dragonball']
runs-on: garm-ubuntu-2204
env:
CONTAINERD_VERSION: ${{ matrix.containerd_version }}
GOPATH: ${{ github.workspace }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
steps:
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
- name: Install dependencies
run: bash tests/integration/nydus/gha-run.sh install-dependencies
- name: get-kata-tarball
uses: actions/download-artifact@v3
with:
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-artifacts
- name: Install kata
run: bash tests/integration/nydus/gha-run.sh install-kata kata-artifacts
- name: Run nydus tests
run: bash tests/integration/nydus/gha-run.sh run

View File

@@ -1,46 +0,0 @@
name: CI | Run runk tests
on:
workflow_call:
inputs:
tarball-suffix:
required: false
type: string
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
jobs:
run-runk:
runs-on: ubuntu-22.04
env:
CONTAINERD_VERSION: lts
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: Install dependencies
run: bash tests/integration/runk/gha-run.sh install-dependencies
- name: get-kata-tarball
uses: actions/download-artifact@v4
with:
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-artifacts
- name: Install kata
run: bash tests/integration/runk/gha-run.sh install-kata kata-artifacts
- name: Run runk tests
run: bash tests/integration/runk/gha-run.sh run

37
.github/workflows/run-vfio-tests.yaml vendored Normal file
View File

@@ -0,0 +1,37 @@
name: CI | Run vfio tests
on:
workflow_call:
inputs:
tarball-suffix:
required: false
type: string
commit-hash:
required: false
type: string
jobs:
run-vfio:
strategy:
fail-fast: false
matrix:
vmm: ['clh', 'qemu']
runs-on: garm-ubuntu-2204
env:
GOPATH: ${{ github.workspace }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
steps:
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit-hash }}
- name: Install dependencies
run: bash tests/functional/vfio/gha-run.sh install-dependencies
- name: get-kata-tarball
uses: actions/download-artifact@v3
with:
name: kata-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-artifacts
- name: Run vfio tests
run: bash tests/functional/vfio/gha-run.sh run

View File

@@ -1,17 +0,0 @@
name: 'Automatically close stale PRs'
on:
schedule:
- cron: '0 0 * * *'
workflow_dispatch:
jobs:
stale:
runs-on: ubuntu-22.04
steps:
- uses: actions/stale@v9
with:
stale-pr-message: 'This PR has been opened without with no activity for 180 days. Comment on the issue otherwise it will be closed in 7 days'
days-before-pr-stale: 180
days-before-pr-close: 7
days-before-issue-stale: -1
days-before-issue-close: -1

View File

@@ -0,0 +1,37 @@
on:
pull_request:
types:
- opened
- edited
- reopened
- synchronize
paths-ignore: [ '**.md', '**.png', '**.jpg', '**.jpeg', '**.svg', '/docs/**' ]
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
name: Static checks dragonball
jobs:
test-dragonball:
runs-on: dragonball
env:
RUST_BACKTRACE: "1"
steps:
- uses: actions/checkout@v3
- name: Set env
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
run: |
echo "GOPATH=${{ github.workspace }}" >> $GITHUB_ENV
- name: Install Rust
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
run: |
./ci/install_rust.sh
echo PATH="$HOME/.cargo/bin:$PATH" >> $GITHUB_ENV
- name: Run Unit Test
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
run: |
cd src/dragonball
cargo version
rustc --version
sudo -E env PATH=$PATH LIBC=gnu SUPPORT_VIRTUALIZATION=true make test

View File

@@ -1,34 +0,0 @@
on:
pull_request:
types:
- opened
- synchronize
- reopened
- labeled # a workflow runs only when the 'ok-to-test' label is added
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
name: Static checks self-hosted
jobs:
skipper:
if: ${{ contains(github.event.pull_request.labels.*.name, 'ok-to-test') }}
uses: ./.github/workflows/gatekeeper-skipper.yaml
with:
commit-hash: ${{ github.event.pull_request.head.sha }}
target-branch: ${{ github.event.pull_request.base.ref }}
build-checks:
needs: skipper
if: ${{ needs.skipper.outputs.skip_static != 'yes' }}
strategy:
fail-fast: false
matrix:
instance:
- "arm-no-k8s"
- "s390x"
- "ppc64le"
uses: ./.github/workflows/build-checks.yaml
with:
instance: ${{ matrix.instance }}

View File

@@ -12,114 +12,84 @@ concurrency:
name: Static checks
jobs:
skipper:
uses: ./.github/workflows/gatekeeper-skipper.yaml
with:
commit-hash: ${{ github.event.pull_request.head.sha }}
target-branch: ${{ github.event.pull_request.base.ref }}
check-kernel-config-version:
needs: skipper
if: ${{ needs.skipper.outputs.skip_static != 'yes' }}
runs-on: ubuntu-22.04
steps:
- name: Checkout the code
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Ensure the kernel config version has been updated
run: |
kernel_dir="tools/packaging/kernel/"
kernel_version_file="${kernel_dir}kata_config_version"
modified_files=$(git diff --name-only origin/$GITHUB_BASE_REF..HEAD)
if git diff --name-only origin/$GITHUB_BASE_REF..HEAD "${kernel_dir}" | grep "${kernel_dir}"; then
echo "Kernel directory has changed, checking if $kernel_version_file has been updated"
if echo "$modified_files" | grep -v "README.md" | grep "${kernel_dir}" >>"/dev/null"; then
echo "$modified_files" | grep "$kernel_version_file" >>/dev/null || ( echo "Please bump version in $kernel_version_file" && exit 1)
else
echo "Readme file changed, no need for kernel config version update."
fi
echo "Check passed"
fi
build-checks:
needs: skipper
if: ${{ needs.skipper.outputs.skip_static != 'yes' }}
uses: ./.github/workflows/build-checks.yaml
with:
instance: ubuntu-22.04
build-checks-depending-on-kvm:
runs-on: ubuntu-22.04
needs: skipper
if: ${{ needs.skipper.outputs.skip_static != 'yes' }}
strategy:
fail-fast: false
matrix:
component:
- runtime-rs
include:
- component: runtime-rs
command: "sudo -E env PATH=$PATH LIBC=gnu SUPPORT_VIRTUALIZATION=true make test"
- component: runtime-rs
component-path: src/dragonball
steps:
- name: Checkout the code
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install system deps
run: |
sudo apt-get install -y build-essential musl-tools
- name: Install yq
run: |
sudo -E ./ci/install_yq.sh
env:
INSTALL_IN_GOPATH: false
- name: Install rust
run: |
export PATH="$PATH:/usr/local/bin"
./tests/install_rust.sh
- name: Running `${{ matrix.command }}` for ${{ matrix.component }}
run: |
export PATH="$PATH:${HOME}/.cargo/bin"
cd ${{ matrix.component-path }}
${{ matrix.command }}
env:
RUST_BACKTRACE: "1"
static-checks:
runs-on: ubuntu-22.04
needs: skipper
if: ${{ needs.skipper.outputs.skip_static != 'yes' }}
runs-on: garm-ubuntu-2004
strategy:
fail-fast: false
matrix:
cmd:
- "make vendor"
- "make static-checks"
- "make check"
- "make test"
- "sudo -E PATH=\"$PATH\" make test"
env:
RUST_BACKTRACE: "1"
target_branch: ${{ github.base_ref }}
GOPATH: ${{ github.workspace }}
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
path: ./src/github.com/${{ github.repository }}
- name: Install yq
run: |
cd ${GOPATH}/src/github.com/${{ github.repository }}
./ci/install_yq.sh
env:
INSTALL_IN_GOPATH: false
- name: Install golang
run: |
cd ${GOPATH}/src/github.com/${{ github.repository }}
./tests/install_go.sh -f -p
echo "/usr/local/go/bin" >> $GITHUB_PATH
- name: Install system dependencies
run: |
sudo apt-get -y install moreutils hunspell hunspell-en-gb hunspell-en-us pandoc
- name: Run check
run: |
export PATH=${PATH}:${GOPATH}/bin
cd ${GOPATH}/src/github.com/${{ github.repository }} && ${{ matrix.cmd }}
- name: Free disk space
run: |
sudo rm -rf /usr/share/dotnet
sudo rm -rf "$AGENT_TOOLSDIRECTORY"
- name: Checkout code
uses: actions/checkout@v3
with:
fetch-depth: 0
path: ./src/github.com/${{ github.repository }}
- name: Install dependencies
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
build-essential \
haveged \
libdevmapper-dev \
clang
- name: Install Go
uses: actions/setup-go@v3
with:
go-version: 1.19.3
- name: Check kernel config version
run: |
cd "${{ github.workspace }}/src/github.com/${{ github.repository }}"
kernel_dir="tools/packaging/kernel/"
kernel_version_file="${kernel_dir}kata_config_version"
modified_files=$(git diff --name-only origin/CCv0..HEAD)
if git diff --name-only origin/CCv0..HEAD "${kernel_dir}" | grep "${kernel_dir}"; then
echo "Kernel directory has changed, checking if $kernel_version_file has been updated"
if echo "$modified_files" | grep -v "README.md" | grep "${kernel_dir}" >>"/dev/null"; then
echo "$modified_files" | grep "$kernel_version_file" >>/dev/null || ( echo "Please bump version in $kernel_version_file" && exit 1)
else
echo "Readme file changed, no need for kernel config version update."
fi
echo "Check passed"
fi
- name: Set PATH
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
run: |
echo "${{ github.workspace }}/bin" >> $GITHUB_PATH
- name: Setup
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
run: |
cd ${GOPATH}/src/github.com/${{ github.repository }} && ./ci/setup.sh
- name: Installing rust
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
run: |
cd ${GOPATH}/src/github.com/${{ github.repository }} && ./ci/install_rust.sh
PATH=$PATH:"$HOME/.cargo/bin"
rustup target add x86_64-unknown-linux-musl
rustup component add rustfmt clippy
- name: Setup seccomp
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
run: |
libseccomp_install_dir=$(mktemp -d -t libseccomp.XXXXXXXXXX)
gperf_install_dir=$(mktemp -d -t gperf.XXXXXXXXXX)
cd ${GOPATH}/src/github.com/${{ github.repository }} && ./ci/install_libseccomp.sh "${libseccomp_install_dir}" "${gperf_install_dir}"
echo "Set environment variables for the libseccomp crate to link the libseccomp library statically"
echo "LIBSECCOMP_LINK_TYPE=static" >> $GITHUB_ENV
echo "LIBSECCOMP_LIB_PATH=${libseccomp_install_dir}/lib" >> $GITHUB_ENV
- name: Run check
if: ${{ !contains(github.event.pull_request.labels.*.name, 'force-skip-ci') }}
run: |
export PATH=$PATH:"$HOME/.cargo/bin"
export XDG_RUNTIME_DIR=$(mktemp -d /tmp/kata-tests-$USER.XXX | tee >(xargs chmod 0700))
cd ${GOPATH}/src/github.com/${{ github.repository }} && ${{ matrix.cmd }}

1
.gitignore vendored
View File

@@ -15,4 +15,3 @@ src/agent/protocols/src/*.rs
!src/agent/protocols/src/lib.rs
build
src/tools/log-parser/kata-log-parser
tools/packaging/static-build/agent/install_libseccomp.sh

View File

@@ -1,4 +1,4 @@
# Copyright (c) 2019-2023 Intel Corporation
# Copyright (c) 2019 Intel Corporation
#
# SPDX-License-Identifier: Apache-2.0
#
@@ -9,83 +9,4 @@
# Order in this file is important. Only the last match will be
# used. See https://help.github.com/articles/about-code-owners/
/CODEOWNERS @kata-containers/codeowners
VERSION @kata-containers/release
# The versions database needs careful handling
versions.yaml @kata-containers/release @kata-containers/ci @kata-containers/tests
Makefile* @kata-containers/build
*.mak @kata-containers/build
*.mk @kata-containers/build
# Documentation related files could also appear anywhere
# else in the repo.
*.md @kata-containers/documentation
*.drawio @kata-containers/documentation
*.jpg @kata-containers/documentation
*.png @kata-containers/documentation
*.svg @kata-containers/documentation
*.bash @kata-containers/shell
*.sh @kata-containers/shell
**/completions/ @kata-containers/shell
Dockerfile* @kata-containers/docker
/ci/ @kata-containers/ci
*.bats @kata-containers/tests
/tests/ @kata-containers/tests
*.rs @kata-containers/rust
*.go @kata-containers/golang
/utils/ @kata-containers/utils
# FIXME: Maybe a new "protocol" team would be better?
#
# All protocol changes must be reviewed.
# Note, we include all subdirs, including the vendor dir, as at present there are no .proto files
# in the vendor dir. Later we may have to extend this matching rule if that changes.
/src/libs/protocols/*.proto @kata-containers/architecture-committee @kata-containers/builder @kata-containers/packaging
# GitHub Actions
/.github/workflows/ @kata-containers/action-admins @kata-containers/ci
/ci/ @kata-containers/ci @kata-containers/tests
/docs/ @kata-containers/documentation
/src/agent/ @kata-containers/agent
/src/runtime*/ @kata-containers/runtime
/src/runtime/ @kata-containers/golang
src/runtime-rs/ @kata-containers/rust
src/libs/ @kata-containers/rust
src/dragonball/ @kata-containers/dragonball
/tools/osbuilder/ @kata-containers/builder
/tools/packaging/ @kata-containers/packaging
/tools/packaging/kernel/ @kata-containers/kernel
/tools/packaging/kata-deploy/ @kata-containers/kata-deploy
/tools/packaging/qemu/ @kata-containers/qemu
/tools/packaging/release/ @kata-containers/release
**/vendor/ @kata-containers/vendoring
# Handle arch specific files last so they match more specifically than
# the kernel packaging files.
**/*aarch64* @kata-containers/arch-aarch64
**/*arm64* @kata-containers/arch-aarch64
**/*amd64* @kata-containers/arch-amd64
**/*x86-64* @kata-containers/arch-amd64
**/*x86_64* @kata-containers/arch-amd64
**/*ppc64* @kata-containers/arch-ppc64le
**/*s390x* @kata-containers/arch-s390x
*.md @kata-containers/documentation

View File

@@ -1,4 +1,4 @@
# Copyright (c) 2020-2023 Intel Corporation
# Copyright (c) 2020 Intel Corporation
#
# SPDX-License-Identifier: Apache-2.0
#
@@ -18,6 +18,7 @@ TOOLS =
TOOLS += agent-ctl
TOOLS += kata-ctl
TOOLS += log-parser
TOOLS += log-parser-rs
TOOLS += runk
TOOLS += trace-forwarder
@@ -42,7 +43,7 @@ generate-protocols:
# Some static checks rely on generated source files of components.
static-checks: static-checks-build
bash tests/static-checks.sh github.com/kata-containers/kata-containers
bash ci/static-checks.sh
docs-url-alive-check:
bash ci/docs-url-alive-check.sh

View File

@@ -123,7 +123,7 @@ The table below lists the core parts of the project:
| [agent](src/agent) | core | Management process running inside the virtual machine / POD that sets up the container environment. |
| [`dragonball`](src/dragonball) | core | An optional built-in VMM brings out-of-the-box Kata Containers experience with optimizations on container workloads |
| [documentation](docs) | documentation | Documentation common to all components (such as design and install documentation). |
| [tests](tests) | tests | Excludes unit tests which live with the main code. |
| [tests](https://github.com/kata-containers/tests) | tests | Excludes unit tests which live with the main code. |
### Additional components
@@ -137,22 +137,17 @@ The table below lists the remaining parts of the project:
| [kata-debug](tools/packaging/kata-debug/README.md) | infrastructure | Utility tool to gather Kata Containers debug information from Kubernetes clusters. |
| [`agent-ctl`](src/tools/agent-ctl) | utility | Tool that provides low-level access for testing the agent. |
| [`kata-ctl`](src/tools/kata-ctl) | utility | Tool that provides advanced commands and debug facilities. |
| [`log-parser-rs`](src/tools/log-parser-rs) | utility | Tool that aid in analyzing logs from the kata runtime. |
| [`trace-forwarder`](src/tools/trace-forwarder) | utility | Agent tracing helper. |
| [`runk`](src/tools/runk) | utility | Standard OCI container runtime based on the agent. |
| [`ci`](.github/workflows) | CI | Continuous Integration configuration files and scripts. |
| [`ocp-ci`](ci/openshift-ci/README.md) | CI | Continuous Integration configuration for the OpenShift pipelines. |
| [`ci`](https://github.com/kata-containers/ci) | CI | Continuous Integration configuration files and scripts. |
| [`katacontainers.io`](https://github.com/kata-containers/www.katacontainers.io) | Source for the [`katacontainers.io`](https://www.katacontainers.io) site. |
| [`Webhook`](tools/testing/kata-webhook/README.md) | utility | Example of a simple admission controller webhook to annotate pods with the Kata runtime class |
### Packaging and releases
Kata Containers is now
[available natively for most distributions](docs/install/README.md#packaged-installation-methods).
## General tests
See the [tests documentation](tests/README.md).
## Metrics tests
See the [metrics documentation](tests/metrics/README.md).

View File

@@ -1 +1 @@
3.10.1
3.2.0-rc0

View File

@@ -1,343 +0,0 @@
# Kata Containers CI
> [!WARNING]
> While this project's CI has several areas for improvement, it is constantly
> evolving. This document attempts to describe its current state, but due to
> ongoing changes, you may notice some outdated information here. Feel free to
> modify/improve this document as you use the CI and notice anything odd. The
> community appreciates it!
## Introduction
The Kata Containers CI relies on [GitHub Actions][gh-actions], where the actions
themselves can be found in the `.github/workflows` directory, and they may call
helper scripts, which are located under the `tests` directory, to actually
perform the tasks required for each test case.
## The different workflows
There are a few different sets of workflows that are running as part of our CI,
and here we're going to cover the ones that are less likely to get rotten. With
this said, it's fair to advise that if the reader finds something that got
rotten, opening an issue to the project pointing to the problem is a nice way to
help, and providing a fix for the issue is a very encouraging way to help.
### Jobs that run automatically when a PR is raised
These are a bunch of tests that will automatically run as soon as a PR is
opened, they're mostly running on "cost free" runners, and they do some
pre-checks to evaluate that your PR may be okay to start getting reviewed.
Mind, though, that the community expects the contributors to, at least, build
their code before submitting a PR, which the community sees as a very fair
request.
Without getting into the weeds with details on this, those jobs are the ones
responsible for ensuring that:
- The commit message is in the expected format
- There's no missing Developer's Certificate of Origin
- Static checks are passing
### Jobs that require a maintainer's approval to run
These are the required tests, and our so-called "CI". These require a
maintainer's approval to run as parts of those jobs will be running on "paid
runners", which are currently using Azure infrastructure.
Once a maintainer of the project gives "the green light" (currently by adding an
`ok-to-test` label to the PR, soon to be changed to commenting "/test" as part
of a PR review), the following tests will be executed:
- Build all the components (runs on free cost runners, or bare-metal depending on the architecture)
- Create a tarball with all the components (runs on free cost runners, or bare-metal depending on the architecture)
- Create a kata-deploy payload with the tarball generated in the previous step (runs on free costs runner, or bare-metal depending on the architecture)
- Run the following tests:
- Tests depending on the generated tarball
- Metrics (runs on bare-metal)
- `docker` (runs on cost free runners)
- `nerdctl` (runs on cost free runners)
- `kata-monitor` (runs on cost free runners)
- `cri-containerd` (runs on cost free runners)
- `nydus` (runs on cost free runners)
- `vfio` (runs on cost free runners)
- Tests depending on the generated kata-deploy payload
- kata-deploy (runs on cost free runners)
- Tests are performed using different "Kubernetes flavors", such as k0s, k3s, rke2, and Azure Kubernetes Service (AKS).
- Kubernetes (runs in Azure small and medium instances depending on what's required by each test, and on TEE bare-metal machines)
- Tests are performed with different runtime engines, such as CRI-O and containerd.
- Tests are performed with different snapshotters for containerd, namely OverlayFS and devmapper.
- Tests are performed with all the supported hypervisors, which are Cloud Hypervisor, Dragonball, Firecracker, and QEMU.
For all the tests relying on Azure instances, real money is being spent, so the
community asks for the maintainers to be mindful about those, and avoid abusing
them to merely debug issues.
## The different runners
In the previous section we've mentioned using different runners, now in this section we'll go through each type of runner used.
- Cost free runners: Those are the runners provided by Github itself, and
those are fairly small machines with virtualization capabilities enabled.
- Azure small instances: Those are runners which have virtualization
capabilities enabled, 2 CPUs, and 8GB of RAM. These runners have a "-smaller"
suffix to their name.
- Azure normal instances: Those are runners which have virtualization
capabilities enabled, 4 CPUs, and 16GB of RAM. These runners are usually
`garm` ones with no "-smaller" suffix.
- Bare-metal runners: Those are runners provided by community contributors,
and they may vary in architecture, size and virtualization capabilities.
Builder runners don't actually require any virtualization capabilities, while
runners which will be actually performing the tests must have virtualization
capabilities and a reasonable amount for CPU and RAM available (at least
matching the Azure normal instances).
## Adding new tests
Before someone decides to add a new test, we strongly recommend them to go
through [GitHub Actions Documentation][gh-actions],
which will provide you a very sensible background on how to read and understand
current tests we have, and also become familiar with how to write a new test.
On the Kata Containers land, there are basically two sets of tests: "standalone"
and "part of something bigger".
The "standalone" tests, for example the commit message check, won't be covered
here as they're better covered by the GitHub Actions documentation pasted above.
The "part of something bigger" is the more complicated one and not so
straightforward to add, so we'll be focusing our efforts on describing the
addition of those.
> [!NOTE]
> TODO: Currently, this document refers to "tests" when it actually means the
> jobs (or workflows) of GitHub. In an ideal world, except in some specific cases,
> new tests should be added without the need to add new workflows. In the
> not-too-distant future (hopefully), we will improve the workflows to support
> this.
### Adding a new test that's "part of something bigger"
The first important thing here is to align expectations, and we must say that
the community strongly prefers receiving tests that already come with:
- Instructions how to run them
- A proven run where it's passing
There are several ways to achieve those two requirements, and an example of that
can be seen in PR #8115.
With the expectations aligned, adding a test consists in:
- Adding a new yaml file for your test, and ensure it's called from the
"bigger" yaml. See the [Kata Monitor test example][monitor-ex01].
- Adding the helper scripts needed for your test to run. Again, use the [Kata Monitor script as example][monitor-ex02].
Following those examples, the community advice during the review, and even
asking the community directly on Slack are the best ways to get your test
accepted.
## Running tests
### Running the tests as part of the CI
If you're a maintainer of the project, you'll be able to kick in the tests by
yourself. With the current approach, you just need to add the `ok-to-test`
label and the tests will automatically start. We're moving, though, to use a
`/test` command as part of a GitHub review comment, which will simplify this
process.
If you're not a maintainer, please, send a message on Slack or wait till one of
the maintainers reviews your PR. Maintainers will then kick in the tests on
your behalf.
In case a test fails and there's the suspicion it happens due to flakiness in
the test itself, please, create an issue for us, and then re-run (or asks
maintainers to re-run) the tests following these steps:
- Locate which tests is failing
- Click in "details"
- In the top right corner, click in "Re-run jobs"
- And then in "Re-run failed jobs"
- And finally click in the green "Re-run jobs" button
> [!NOTE]
> TODO: We need figures here
### Running the tests locally
In this section, aligning expectations is also something very important, as one
will not be able to run the tests exactly in the same way the tests are running
in the CI, as one most likely won't have access to an Azure subscription.
However, we're trying our best here to provide you with instructions on how to
run the tests in an environment that's "close enough" and will help you to debug
issues you find with the current tests, or even provide a proof-of-concept to
the new test you're trying to add.
The basic steps, which we will cover in details down below are:
1. Create a VM matching the configuration of the target runner
2. Generate the artifacts you'll need for the test, or download them from a
current failed run
3. Follow the steps provided in the action itself to run the tests.
Although the general overview looks easy, we know that some tricks need to be
shared, and we'll go through the general process of debugging one non-Kubernetes
and one Kubernetes specific test for educational purposes.
One important thing to note is that "Create a VM" can be done in innumerable
different ways, using the tools of your choice. For the sake of simplicity on
this guide, we'll be using `kcli`, which we strongly recommend in case you're a
non-experienced user, and happen to be developing on a Linux box.
For both non-Kubernetes and Kubernetes cases, we'll be using PR #8070 as an
example, which at the time this document is being written serves us very well
the purpose, as you can see that we have `nerdctl` and Kubernetes tests failing.
## Debugging tests
### Debugging a non Kubernetes test
As shown above, the `nerdctl` test is failing.
As a developer you can go ahead to the details of the job, and expand the job
that's failing in order to gather more information.
But when that doesn't help, we need to set up our own environment to debug
what's going on.
Taking a look at the `nerdctl` test, which is located here, you can easily see
that it runs-on a `garm-ubuntu-2304-smaller` virtual machine.
The important parts to understand are `ubuntu-2304`, which is the OS where the
test is running on; and "smaller", which means we're running it on a machine
with 2 CPUs and 8GB of RAM.
With this information, we can go ahead and create a similar VM locally using `kcli`.
```bash
$ sudo kcli create vm -i ubuntu2304 -P disks=[60] -P numcpus=2 -P memory=8192 -P cpumodel=host-passthrough debug-nerdctl-pr8070
```
In order to run the tests, you'll need the "kata-tarball" artifacts, which you
can build your own using "make kata-tarball" (see below), or simply get them
from the PR where the tests failed. To download them, click on the "Summary"
button that's on the top left corner, and then scroll down till you see the
artifacts, as shown below.
Unfortunately GitHub doesn't give us a link that we can download those from
inside the VM, but we can download them on our local box, and then `scp` the
tarball to the newly created VM that will be used for debugging purposes.
> [!NOTE]
> Those artifacts are only available (for 15 days) when all jobs are finished.
Once you have the `kata-static.tar.xz` in your VM, you can login to the VM with
`kcli ssh debug-nerdctl-pr8070`, go ahead and then clone your development branch
```bash
$ git clone --branch feat_add-fc-runtime-rs https://github.com/nubificus/kata-containers
```
Add the upstream as a remote, set up your git, and rebase your branch atop of the upstream main one
```bash
$ git remote add upstream https://github.com/kata-containers/kata-containers
$ git remote update
$ git config --global user.email "you@example.com"
$ git config --global user.name "Your Name"
$ git rebase upstream/main
```
Now copy the `kata-static.tar.xz` into your `kata-containers/kata-artifacts` directory
```bash
$ mkdir kata-artifacts
$ cp ../kata-static.tar.xz kata-artifacts/
```
> [!NOTE]
> If you downloaded the .zip from GitHub you need to uncompress first to see `kata-static.tar.xz`
And finally run the tests following what's in the yaml file for the test you're
debugging.
In our case, the `run-nerdctl-tests-on-garm.yaml`.
When looking at the file you'll notice that some environment variables are set,
such as `KATA_HYPERVISOR`, and should be aware that, for this particular example,
the important steps to follow are:
Install the dependencies
Install kata
Run the tests
Let's now run the steps mentioned above exporting the expected environment variables
```bash
$ export KATA_HYPERVISOR=dragonball
$ bash ./tests/integration/nerdctl/gha-run.sh install-dependencies
$ bash ./tests/integration/nerdctl/gha-run.sh install-kata
$ bash tests/integration/nerdctl/gha-run.sh run
```
And with this you should've been able to reproduce exactly the same issue found
in the CI, and from now on you can build your own code, use your own binaries,
and have fun debugging and hacking!
### Debugging a Kubernetes test
Steps for debugging the Kubernetes tests are very similar to the ones for
debugging non-Kubernetes tests, with the caveat that what you'll need, this
time, is not the `kata-static.tar.xz` tarball, but rather a payload to be used
with kata-deploy.
In order to generate your own kata-deploy image you can generate your own
`kata-static.tar.xz` and then take advantage of the following script. Be aware
that the image generated and uploaded must be accessible by the VM where you'll
be performing your tests.
In case you want to take advantage of the payload that was already generated
when you faced the CI failure, which is considerably easier, take a look at the
failed job, then click in "Deploy Kata" and expand the "Final kata-deploy.yaml
that is used in the test" section. From there you can see exactly what you'll
have to use when deploying kata-deploy in your local cluster.
> [!NOTE]
> TODO: WAINER TO FINISH THIS PART BASED ON HIS PR TO RUN A LOCAL CI
## Adding new runners
Any admin of the project is able to add or remove GitHub runners, and those are
the folks you should rely on.
If you need a new runner added, please, tag @ac in the Kata Containers slack,
and someone from that group will be able to help you.
If you're part of that group and you're looking for information on how to help
someone, this is simple, and must be done in private. Basically what you have to
do is:
- Go to the kata-containers/kata-containers repo
- Click on the Settings button, located in the top right corner
- On the left panel, under "Code and automation", click on "Actions"
- Click on "Runners"
If you want to add a new self-hosted runner:
- In the top right corner there's a green button called "New self-hosted runner"
If you want to remove a current self-hosted runner:
- For each runner there's a "..." menu, where you can just click and the
"Remove runner" option will show up
## Known limitations
As the GitHub actions are structured right now we cannot: Test the addition of a
GitHub action that's not triggered by a pull_request event as part of the PR.
[gh-actions]: https://docs.github.com/en/actions
[monitor-ex01]: https://github.com/kata-containers/kata-containers/commit/a3fb067f1bccde0cbd3fd4d5de12dfb3d8c28b60
[monitor-ex02]: https://github.com/kata-containers/kata-containers/commit/489caf1ad0fae27cfd00ba3c9ed40e3d512fa492

View File

@@ -7,6 +7,6 @@
set -e
cidir=$(dirname "$0")
source "${cidir}/../tests/common.bash"
source "${cidir}/lib.sh"
run_docs_url_alive_check

View File

@@ -1,182 +0,0 @@
#!/bin/bash
# Copyright (c) 2020 Intel Corporation
# Copyright (c) 2024 IBM Corporation
#
# SPDX-License-Identifier: Apache-2.0
set -o errexit
set -o errtrace
set -o nounset
set -o pipefail
[ -n "${DEBUG:-}" ] && set -o xtrace
script_name=${0##*/}
#---------------------------------------------------------------------
die()
{
echo >&2 "$*"
exit 1
}
usage()
{
cat <<EOF
Usage: $script_name [OPTIONS] [command] [arguments]
Description: Utility to expand the abilities of the GitHub CLI tool, gh.
Command descriptions:
list-issues-for-pr List issues linked to a PR.
list-labels-for-issue List labels, in json format for an issue
Commands and arguments:
list-issues-for-pr <pr>
list-labels-for-issue <issue>
Options:
-h Show this help statement.
-r <owner/repo> Optional <org/repo> specification. Default: 'kata-containers/kata-containers'
Examples:
- List issues for a Pull Request 123 in kata-containers/kata-containers repo
$ $script_name list-issues-for-pr 123
EOF
}
list_issues_for_pr()
{
local pr="${1:-}"
local repo="${2:-kata-containers/kata-containers}"
[ -z "$pr" ] && die "need PR"
local commits=$(gh pr view ${pr} --repo ${repo} --json commits --jq .commits[].messageBody)
[ -z "$commits" ] && die "cannot determine commits for PR $pr"
# Extract the issue number(s) from the commits.
#
# This needs to be careful to take account of lines like this:
#
# fixes 99
# fixes: 77
# fixes #123.
# Fixes: #1, #234, #5678.
#
# Note the exclusion of lines starting with whitespace which is
# specifically to ignore vendored git log comments, which are whitespace
# indented and in the format:
#
# "<git-commit> <git-commit-msg>"
#
local issues=$(echo "$commits" |\
egrep -v "^( | )" |\
egrep -i "fixes:* *(#*[0-9][0-9]*)" |\
tr ' ' '\n' |\
grep "[0-9][0-9]*" |\
sed 's/[.,\#]//g' |\
sort -nu || true)
[ -z "$issues" ] && die "cannot determine issues for PR $pr"
echo "# Issues linked to PR"
echo "#"
echo "# Fields: issue_number"
local issue
echo "$issues"|while read issue
do
printf "%s\n" "$issue"
done
}
list_labels_for_issue()
{
local issue="${1:-}"
[ -z "$issue" ] && die "need issue number"
local labels=$(gh issue view ${issue} --repo kata-containers/kata-containers --json labels)
[ -z "$labels" ] && die "cannot determine labels for issue $issue"
printf "$labels"
}
setup()
{
for cmd in gh jq
do
command -v "$cmd" &>/dev/null || die "need command: $cmd"
done
}
handle_args()
{
setup
local show_all="false"
local opt
while getopts "ahr:" opt "$@"
do
case "$opt" in
a) show_all="true" ;;
h) usage && exit 0 ;;
r) repo="${OPTARG}" ;;
esac
done
shift $(($OPTIND - 1))
local repo="${repo:-kata-containers/kata-containers}"
local cmd="${1:-}"
case "$cmd" in
list-issues-for-pr) ;;
list-labels-for-issue) ;;
"") usage && exit 0 ;;
*) die "invalid command: '$cmd'" ;;
esac
# Consume the command name
shift
local issue=""
local pr=""
case "$cmd" in
list-issues-for-pr)
pr="${1:-}"
list_issues_for_pr "$pr" "${repo}"
;;
list-labels-for-issue)
issue="${1:-}"
list_labels_for_issue "$issue"
;;
*) die "impossible situation: cmd: '$cmd'" ;;
esac
exit 0
}
main()
{
handle_args "$@"
}
main "$@"

22
ci/install_go.sh Executable file
View File

@@ -0,0 +1,22 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019 Intel Corporation
#
# SPDX-License-Identifier: Apache-2.0
set -e
cidir=$(dirname "$0")
source "${cidir}/lib.sh"
clone_tests_repo
new_goroot=/usr/local/go
pushd "${tests_repo_dir}"
# Force overwrite the current version of golang
[ -z "${GOROOT}" ] || rm -rf "${GOROOT}"
.ci/install_go.sh -p -f -d "$(dirname ${new_goroot})"
[ -z "${GOROOT}" ] || sudo ln -sf "${new_goroot}" "${GOROOT}"
go version
popd

View File

@@ -7,10 +7,12 @@
set -o errexit
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
script_name="$(basename "${BASH_SOURCE[0]}")"
cidir=$(dirname "$0")
source "${cidir}/lib.sh"
source "${script_dir}/../tests/common.bash"
clone_tests_repo
source "${tests_repo_dir}/.ci/lib.sh"
# The following variables if set on the environment will change the behavior
# of gperf and libseccomp configure scripts, that may lead this script to
@@ -23,11 +25,11 @@ workdir="$(mktemp -d --tmpdir build-libseccomp.XXXXX)"
# Variables for libseccomp
libseccomp_version="${LIBSECCOMP_VERSION:-""}"
if [ -z "${libseccomp_version}" ]; then
libseccomp_version=$(get_from_kata_deps ".externals.libseccomp.version")
libseccomp_version=$(get_version "externals.libseccomp.version")
fi
libseccomp_url="${LIBSECCOMP_URL:-""}"
if [ -z "${libseccomp_url}" ]; then
libseccomp_url=$(get_from_kata_deps ".externals.libseccomp.url")
libseccomp_url=$(get_version "externals.libseccomp.url")
fi
libseccomp_tarball="libseccomp-${libseccomp_version}.tar.gz"
libseccomp_tarball_url="${libseccomp_url}/releases/download/v${libseccomp_version}/${libseccomp_tarball}"
@@ -36,11 +38,11 @@ cflags="-O2"
# Variables for gperf
gperf_version="${GPERF_VERSION:-""}"
if [ -z "${gperf_version}" ]; then
gperf_version=$(get_from_kata_deps ".externals.gperf.version")
gperf_version=$(get_version "externals.gperf.version")
fi
gperf_url="${GPERF_URL:-""}"
if [ -z "${gperf_url}" ]; then
gperf_url=$(get_from_kata_deps ".externals.gperf.url")
gperf_url=$(get_version "externals.gperf.url")
fi
gperf_tarball="gperf-${gperf_version}.tar.gz"
gperf_tarball_url="${gperf_url}/${gperf_tarball}"
@@ -48,64 +50,65 @@ gperf_tarball_url="${gperf_url}/${gperf_tarball}"
# We need to build the libseccomp library from sources to create a static library for the musl libc.
# However, ppc64le and s390x have no musl targets in Rust. Hence, we do not set cflags for the musl libc.
if ([ "${arch}" != "ppc64le" ] && [ "${arch}" != "s390x" ]); then
# Set FORTIFY_SOURCE=1 because the musl-libc does not have some functions about FORTIFY_SOURCE=2
cflags="-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -O2"
# Set FORTIFY_SOURCE=1 because the musl-libc does not have some functions about FORTIFY_SOURCE=2
cflags="-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -O2"
fi
die() {
msg="$*"
echo "[Error] ${msg}" >&2
exit 1
msg="$*"
echo "[Error] ${msg}" >&2
exit 1
}
finish() {
rm -rf "${workdir}"
rm -rf "${workdir}"
}
trap finish EXIT
build_and_install_gperf() {
echo "Build and install gperf version ${gperf_version}"
mkdir -p "${gperf_install_dir}"
curl -sLO "${gperf_tarball_url}"
tar -xf "${gperf_tarball}"
pushd "gperf-${gperf_version}"
# Unset $CC for configure, we will always use native for gperf
CC= ./configure --prefix="${gperf_install_dir}"
make
make install
export PATH=$PATH:"${gperf_install_dir}"/bin
popd
echo "Gperf installed successfully"
echo "Build and install gperf version ${gperf_version}"
mkdir -p "${gperf_install_dir}"
curl -sLO "${gperf_tarball_url}"
tar -xf "${gperf_tarball}"
pushd "gperf-${gperf_version}"
# gperf is a build time dependency of libseccomp and not to be used in the target.
# Unset $CC since that might point to a cross compiler.
CC= ./configure --prefix="${gperf_install_dir}"
make
make install
export PATH=$PATH:"${gperf_install_dir}"/bin
popd
echo "Gperf installed successfully"
}
build_and_install_libseccomp() {
echo "Build and install libseccomp version ${libseccomp_version}"
mkdir -p "${libseccomp_install_dir}"
curl -sLO "${libseccomp_tarball_url}"
tar -xf "${libseccomp_tarball}"
pushd "libseccomp-${libseccomp_version}"
[ "${arch}" == $(uname -m) ] && cc_name="" || cc_name="${arch}-linux-gnu-gcc"
CC=${cc_name} ./configure --prefix="${libseccomp_install_dir}" CFLAGS="${cflags}" --enable-static --host="${arch}"
make
make install
popd
echo "Libseccomp installed successfully"
echo "Build and install libseccomp version ${libseccomp_version}"
mkdir -p "${libseccomp_install_dir}"
curl -sLO "${libseccomp_tarball_url}"
tar -xf "${libseccomp_tarball}"
pushd "libseccomp-${libseccomp_version}"
[ "${arch}" == $(uname -m) ] && cc_name="" || cc_name="${arch}-linux-gnu-gcc"
CC=${cc_name} ./configure --prefix="${libseccomp_install_dir}" CFLAGS="${cflags}" --enable-static --host="${arch}"
make
make install
popd
echo "Libseccomp installed successfully"
}
main() {
local libseccomp_install_dir="${1:-}"
local gperf_install_dir="${2:-}"
local libseccomp_install_dir="${1:-}"
local gperf_install_dir="${2:-}"
if [ -z "${libseccomp_install_dir}" ] || [ -z "${gperf_install_dir}" ]; then
die "Usage: ${0} <libseccomp-install-dir> <gperf-install-dir>"
fi
if [ -z "${libseccomp_install_dir}" ] || [ -z "${gperf_install_dir}" ]; then
die "Usage: ${0} <libseccomp-install-dir> <gperf-install-dir>"
fi
pushd "$workdir"
# gperf is required for building the libseccomp.
build_and_install_gperf
build_and_install_libseccomp
popd
pushd "$workdir"
# gperf is required for building the libseccomp.
build_and_install_gperf
build_and_install_libseccomp
popd
}
main "$@"

16
ci/install_rust.sh Executable file
View File

@@ -0,0 +1,16 @@
#!/usr/bin/env bash
# Copyright (c) 2019 Ant Financial
#
# SPDX-License-Identifier: Apache-2.0
#
set -e
cidir=$(dirname "$0")
source "${cidir}/lib.sh"
clone_tests_repo
pushd ${tests_repo_dir}
.ci/install_rust.sh ${1:-}
popd

19
ci/install_vc.sh Executable file
View File

@@ -0,0 +1,19 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018 Intel Corporation
#
# SPDX-License-Identifier: Apache-2.0
set -e
cidir=$(dirname "$0")
vcdir="${cidir}/../src/runtime/virtcontainers/"
source "${cidir}/lib.sh"
export CI_JOB="${CI_JOB:-default}"
clone_tests_repo
if [ "${CI_JOB}" != "PODMAN" ]; then
echo "Install virtcontainers"
make -C "${vcdir}" && chronic sudo make -C "${vcdir}" install
fi

View File

@@ -5,8 +5,6 @@
# SPDX-License-Identifier: Apache-2.0
#
[ -n "$DEBUG" ] && set -o xtrace
# If we fail for any reason a message will be displayed
die() {
msg="$*"
@@ -14,48 +12,21 @@ die() {
exit 1
}
function verify_yq_exists() {
local yq_path=$1
local yq_version=$2
local expected="yq (https://github.com/mikefarah/yq/) version $yq_version"
if [ -x "${yq_path}" ] && [ "$($yq_path --version)"X == "$expected"X ]; then
return 0
else
return 1
fi
}
# Install the yq yaml query package from the mikefarah github repo
# Install via binary download, as we may not have golang installed at this point
function install_yq() {
local yq_pkg="github.com/mikefarah/yq"
local yq_version=v4.40.7
local precmd=""
local yq_path=""
local yq_version=3.4.1
INSTALL_IN_GOPATH=${INSTALL_IN_GOPATH:-true}
if [ "${INSTALL_IN_GOPATH}" == "true" ]; then
if [ "${INSTALL_IN_GOPATH}" == "true" ];then
GOPATH=${GOPATH:-${HOME}/go}
mkdir -p "${GOPATH}/bin"
yq_path="${GOPATH}/bin/yq"
local yq_path="${GOPATH}/bin/yq"
else
yq_path="/usr/local/bin/yq"
fi
if verify_yq_exists "$yq_path" "$yq_version"; then
echo "yq is already installed in correct version"
return
fi
if [ "${yq_path}" == "/usr/local/bin/yq" ]; then
# Check if we need sudo to install yq
if [ ! -w "/usr/local/bin" ]; then
# Check if we have sudo privileges
if ! sudo -n true 2>/dev/null; then
die "Please provide sudo privileges to install yq"
else
precmd="sudo"
fi
fi
fi
[ -x "${yq_path}" ] && [ "`${yq_path} --version`"X == "yq version ${yq_version}"X ] && return
read -r -a sysInfo <<< "$(uname -sm)"
@@ -104,9 +75,9 @@ function install_yq() {
## NOTE: ${var,,} => gives lowercase value of var
local yq_url="https://${yq_pkg}/releases/download/${yq_version}/yq_${goos}_${goarch}"
${precmd} curl -o "${yq_path}" -LSsf "${yq_url}"
curl -o "${yq_path}" -LSsf "${yq_url}"
[ $? -ne 0 ] && die "Download ${yq_url} failed"
${precmd} chmod +x "${yq_path}"
chmod +x "${yq_path}"
if ! command -v "${yq_path}" >/dev/null; then
die "Cannot not get ${yq_path} executable"

149
ci/lib.sh Normal file
View File

@@ -0,0 +1,149 @@
#
# Copyright (c) 2018 Intel Corporation
#
# SPDX-License-Identifier: Apache-2.0
set -o nounset
export tests_repo="${tests_repo:-github.com/kata-containers/tests}"
export tests_repo_dir="$GOPATH/src/$tests_repo"
export branch="${target_branch:-main}"
# Clones the tests repository and checkout to the branch pointed out by
# the global $branch variable.
# If the clone exists and `CI` is exported then it does nothing. Otherwise
# it will clone the repository or `git pull` the latest code.
#
clone_tests_repo()
{
if [ -d "$tests_repo_dir" ]; then
[ -n "${CI:-}" ] && return
# git config --global --add safe.directory will always append
# the target to .gitconfig without checking the existence of
# the target, so it's better to check it before adding the target repo.
local sd="$(git config --global --get safe.directory ${tests_repo_dir} || true)"
if [ -z "${sd}" ]; then
git config --global --add safe.directory ${tests_repo_dir}
fi
pushd "${tests_repo_dir}"
git checkout "${branch}"
git pull
popd
else
git clone -q "https://${tests_repo}" "$tests_repo_dir"
pushd "${tests_repo_dir}"
git checkout "${branch}"
popd
fi
}
run_static_checks()
{
clone_tests_repo
# Make sure we have the targeting branch
git remote set-branches --add origin "${branch}"
git fetch -a
bash "$tests_repo_dir/.ci/static-checks.sh" "$@"
}
run_docs_url_alive_check()
{
clone_tests_repo
# Make sure we have the targeting branch
git remote set-branches --add origin "${branch}"
git fetch -a
bash "$tests_repo_dir/.ci/static-checks.sh" --docs --all "github.com/kata-containers/kata-containers"
}
run_get_pr_changed_file_details()
{
clone_tests_repo
# Make sure we have the targeting branch
git remote set-branches --add origin "${branch}"
git fetch -a
source "$tests_repo_dir/.ci/lib.sh"
get_pr_changed_file_details
}
# Check if the 1st argument version is greater than and equal to 2nd one
# Version format: [0-9]+ separated by period (e.g. 2.4.6, 1.11.3 and etc.)
#
# Parameters:
# $1 - a version to be tested
# $2 - a target version
#
# Return:
# 0 if $1 is greater than and equal to $2
# 1 otherwise
version_greater_than_equal() {
local current_version=$1
local target_version=$2
smaller_version=$(echo -e "$current_version\n$target_version" | sort -V | head -1)
if [ "${smaller_version}" = "${target_version}" ]; then
return 0
else
return 1
fi
}
# Build a IBM zSystem secure execution (SE) image
#
# Parameters:
# $1 - kernel_parameters
# $2 - a source directory where kernel and initrd are located
# $3 - a destination directory where a SE image is built
#
# Return:
# 0 if the image is successfully built
# 1 otherwise
build_secure_image() {
kernel_params="${1:-}"
install_src_dir="${2:-}"
install_dest_dir="${3:-}"
if [ ! -f "${install_src_dir}/vmlinuz.container" ] ||
[ ! -f "${install_src_dir}/kata-containers-initrd.img" ]; then
cat << EOF >&2
Either kernel or initrd does not exist or is mistakenly named
A file name for kernel must be vmlinuz.container (raw binary)
A file name for initrd must be kata-containers-initrd.img
EOF
return 1
fi
cmdline="${kernel_params} panic=1 scsi_mod.scan=none swiotlb=262144"
parmfile="$(mktemp --suffix=-cmdline)"
echo "${cmdline}" > "${parmfile}"
chmod 600 "${parmfile}"
[ -n "${HKD_PATH:-}" ] || (echo >&2 "No host key document specified." && return 1)
cert_list=($(ls -1 $HKD_PATH))
declare hkd_options
eval "for cert in ${cert_list[*]}; do
hkd_options+=\"--host-key-document=\\\"\$HKD_PATH/\$cert\\\" \"
done"
command -v genprotimg > /dev/null 2>&1 || { apt update; apt install -y s390-tools; }
extra_arguments=""
genprotimg_version=$(genprotimg --version | grep -Po '(?<=version )[^-]+')
if ! version_greater_than_equal "${genprotimg_version}" "2.17.0"; then
extra_arguments="--x-pcf '0xe0'"
fi
eval genprotimg \
"${extra_arguments}" \
"${hkd_options}" \
--output="${install_dest_dir}/kata-containers-secure.img" \
--image="${install_src_dir}/vmlinuz.container" \
--ramdisk="${install_src_dir}/kata-containers-initrd.img" \
--parmfile="${parmfile}" \
--no-verify # no verification for CI testing purposes
build_result=$?
rm -f "${parmfile}"
if [ $build_result -eq 0 ]; then
return 0
else
return 1
fi
}

View File

@@ -1,149 +0,0 @@
OpenShift CI
============
This directory contains scripts used by
[the OpenShift CI](https://github.com/openshift/release/tree/master/ci-operator/config/kata-containers/kata-containers)
pipelines to monitor selected functional tests on OpenShift.
There are 2 pipelines, history and logs can be accessed here:
* [main - currently supported OCP](https://prow.ci.openshift.org/job-history/gs/origin-ci-test/logs/periodic-ci-kata-containers-kata-containers-main-e2e-tests)
* [next - currently under development OCP](https://prow.ci.openshift.org/job-history/gs/origin-ci-test/logs/periodic-ci-kata-containers-kata-containers-main-next-e2e-tests)
Running openshift-tests on OCP with kata-containers manually
============================================================
To run openshift-tests (or other suites) with kata-containers one can use
the kata-webhook. To deploy everything you can mimic the CI pipeline by:
```bash
#!/bin/bash -e
# Setup your kubectl and check it's accessible by
kubectl nodes
# Deploy kata (set KATA_DEPLOY_IMAGE to override the default kata-deploy-ci:latest image)
./test.sh
# Deploy the webhook
KATA_RUNTIME=kata-qemu cluster/deploy_webhook.sh
```
This should ensure kata-containers as well as kata-webhook are installed and
working. Before running the openshift-tests it's (currently) recommended to
ignore some security features by:
```bash
#!/bin/bash -e
oc adm policy add-scc-to-group privileged system:authenticated system:serviceaccounts
oc adm policy add-scc-to-group anyuid system:authenticated system:serviceaccounts
oc label --overwrite ns default pod-security.kubernetes.io/enforce=privileged pod-security.kubernetes.io/warn=baseline pod-security.kubernetes.io/audit=baseline
```
Now you should be ready to run the openshift-tests. Our CI only uses a subset
of tests, to get the current ``TEST_SKIPS`` see
[the pipeline config](https://github.com/openshift/release/tree/master/ci-operator/config/kata-containers/kata-containers).
Following steps require the [openshift tests](https://github.com/openshift/origin)
being cloned and built in the current directory:
```bash
#!/bin/bash -e
# Define tests to be skipped (see the pipeline config for the current version)
TEST_SKIPS="\[sig-node\] Security Context should support seccomp runtime/default\|\[sig-node\] Variable Expansion should allow substituting values in a volume subpath\|\[k8s.io\] Probing container should be restarted with a docker exec liveness probe with timeout\|\[sig-node\] Pods Extended Pod Container lifecycle evicted pods should be terminal\|\[sig-node\] PodOSRejection \[NodeConformance\] Kubelet should reject pod when the node OS doesn't match pod's OS\|\[sig-network\].*for evicted pods\|\[sig-network\].*HAProxy router should override the route\|\[sig-network\].*HAProxy router should serve a route\|\[sig-network\].*HAProxy router should serve the correct\|\[sig-network\].*HAProxy router should run\|\[sig-network\].*when FIPS.*the HAProxy router\|\[sig-network\].*bond\|\[sig-network\].*all sysctl on whitelist\|\[sig-network\].*sysctls should not affect\|\[sig-network\] pods should successfully create sandboxes by adding pod to network"
# Get the list of tests to be executed
TESTS="$(./openshift-tests run --dry-run --provider "${TEST_PROVIDER}" "${TEST_SUITE}")"
# Store the list of tests in /tmp/tsts file
echo "${TESTS}" | grep -v "$TEST_SKIPS" > /tmp/tsts
# Remove previously-existing temporarily files as well as previous results
OUT=RESULTS/tmp
rm -Rf /tmp/*test* /tmp/e2e-*
rm -R $OUT
mkdir -p $OUT
# Run the tests ignoring the monitor health checks
./openshift-tests run --provider azure -o "$OUT/job.log" --junit-dir "$OUT" --file /tmp/tsts --max-parallel-tests 5 --cluster-stability Disruptive --run '^\[sig-node\].*|^\[sig-network\]'
```
[!NOTE]
Note we are ignoring the cluster stability checks because our public cloud is
not that stable and running with VMs instead of containers results in minor
stability issues. Some of the old monitor stability tests do not reflect
the ``--cluster-stability`` setting, one should simply ignore these. If you
get a message like ``invariant was violated`` or ``error: failed due to a
MonitorTest failure``, it's usually an indication that only those kind of
tests failed but the real tests passed. See
[wrapped-openshift-tests.sh](https://github.com/openshift/release/blob/master/ci-operator/config/kata-containers/kata-containers/wrapped-openshift-tests.sh)
for details how our pipeline deals with that.
[!TIP]
To compare multiple results locally one can use
[junit2html](https://github.com/inorton/junit2html) tool.
Best-effort kata-containers cleanup
===================================
If you need to cleanup the cluster after testing, you can use the
``cleanup.sh`` script from the current directory. It tries to delete all
resources created by ``test.sh`` as well as ``cluster/deploy_webhook.sh``
ignoring all failures. The primary purpose of this script is to allow
soft-cleanup after deployment to test different versions without
re-provisioning everything.
[!WARNING]
Do not rely on this script in production, return codes are not checked!**
Bisecting e2e tests failures
============================
Let's say the OCP pipeline passed running with
``quay.io/kata-containers/kata-deploy-ci:kata-containers-d7afd31fd40e37a675b25c53618904ab57e74ccd-amd64``
but failed running with
``quay.io/kata-containers/kata-deploy-ci:kata-containers-9f512c016e75599a4a921bd84ea47559fe610057-amd64``
and you'd like to know which PR caused the regression. You can either run with
all the 60 tags between or you can utilize the [bisecter](https://github.com/ldoktor/bisecter)
to optimize the number of steps in between.
Before running the bisection you need a reproducer script. Sample one called
``sample-test-reproducer.sh`` is provided in this directory but you might
want to copy and modify it, especially:
* ``OCP_DIR`` - directory where your openshift/release is located (can be exported)
* ``E2E_TEST`` - openshift-test(s) to be executed (can be exported)
* behaviour of SETUP (returning 125 skips the current image tag, returning
>=128 interrupts the execution, everything else reports the tag as failure
* what should be executed (perhaps running the setup is enough for you or
you might want to be looking for specific failures...)
* use ``timeout`` to interrupt execution in case you know things should be faster
Executing that script with the GOOD commit should pass
``./sample-test-reproducer.sh quay.io/kata-containers/kata-deploy-ci:kata-containers-d7afd31fd40e37a675b25c53618904ab57e74ccd-amd64``
and fail when executed with the BAD commit
``./sample-test-reproducer.sh quay.io/kata-containers/kata-deploy-ci:kata-containers-9f512c016e75599a4a921bd84ea47559fe610057-amd64``.
To get the list of all tags in between those two PRs you can use the
``bisect-range.sh`` script
```bash
./bisect-range.sh d7afd31fd40e37a675b25c53618904ab57e74ccd 9f512c016e75599a4a921bd84ea47559fe610057
```
[!NOTE]
The tagged images are only built per PR, not for individual commits. See
[kata-deploy-ci](https://quay.io/kata-containers/kata-deploy-ci) to see the
available images.
To find out which PR caused this regression, you can either manually try the
individual commits or you can simply execute:
```bash
bisecter start "$(./bisect-range.sh d7afd31fd40 9f512c016)"
OCP_DIR=/path/to/openshift/release bisecter run ./sample-test-reproducer.sh
```
[!NOTE]
If you use ``KATA_WITH_SYSTEM_QEMU=yes`` you might want to deploy once with
it and skip it for the cleanup. That way you might (in most cases) test
all images with a single MCP update instead of per-image MCP update.
[!TIP]
You can check the bisection progress during/after execution by running
``bisecter log`` from the current directory. Before starting a new
bisection you need to execute ``bisecter reset``.

View File

@@ -1,27 +0,0 @@
#!/bin/bash
# Copyright (c) 2024 Red Hat, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
if [ "$#" -gt 2 ] || [ "$#" -lt 1 ] ; then
echo "Usage: $0 GOOD [BAD]"
echo "Prints list of available kata-deploy-ci tags between GOOD and BAD commits (by default BAD is the latest available tag)"
exit 255
fi
GOOD="$1"
[ -n "$2" ] && BAD="$2"
ARCH=amd64
REPO="quay.io/kata-containers/kata-deploy-ci"
TAGS=$(skopeo list-tags "docker://$REPO")
# Only amd64
TAGS=$(echo "$TAGS" | jq '.Tags' | jq "map(select(endswith(\"$ARCH\")))" | jq -r '.[]')
# Sort by git
SORTED=""
[ -n "$BAD" ] && LOG_ARGS="$GOOD~1..$BAD" || LOG_ARGS="$GOOD~1.."
for TAG in $(git log --merges --pretty=format:%H --reverse $LOG_ARGS); do
[[ "$TAGS" =~ "$TAG" ]] && SORTED+="
kata-containers-$TAG-$ARCH"
done
# Comma separated tags with repo
echo "$SORTED" | tail -n +2 | sed -e "s@^@$REPO:@" | paste -s -d, -

View File

@@ -1,59 +0,0 @@
#!/bin/bash
#
# Copyright (c) 2024 Red Hat, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
# This script tries to removes most of the resources added by `test.sh` script
# from the cluster.
scripts_dir=$(dirname $0)
deployments_dir=${scripts_dir}/cluster/deployments
configs_dir=${scripts_dir}/configs
source ${scripts_dir}/lib.sh
# Set to 'yes' if you want to configure SELinux to permissive on the cluster
# workers.
#
SELINUX_PERMISSIVE=${SELINUX_PERMISSIVE:-no}
# Enable workaround for OCP 4.13 https://github.com/kata-containers/kata-containers/pull/9206
#
WORKAROUND_9206_CRIO=${WORKAROUND_9206_CRIO:-no}
# Ignore errors as we want best-effort-approach here
trap - ERR
# Delete webhook resources
oc delete -f "${scripts_dir}/../../tools/testing/kata-webhook/deploy"
oc delete -f "${scripts_dir}/cluster/deployments/configmap_kata-webhook.yaml.in"
# Delete potential smoke-test resources
oc delete -f "${scripts_dir}/smoke/service.yaml"
oc delete -f "${scripts_dir}/smoke/service_kubernetes.yaml"
oc delete -f "${scripts_dir}/smoke/http-server.yaml"
# Delete test.sh resources
oc delete -f "${deployments_dir}/relabel_selinux.yaml"
if [[ "$WORKAROUND_9206_CRIO" == "yes" ]]; then
oc delete -f "${deployments_dir}/workaround-9206-crio-ds.yaml"
oc delete -f "${deployments_dir}/workaround-9206-crio.yaml"
fi
[ ${SELINUX_PERMISSIVE} == "yes" ] && oc delete -f "${deployments_dir}/machineconfig_selinux.yaml.in"
# Delete kata-containers
pushd "$katacontainers_repo_dir/tools/packaging/kata-deploy"
oc delete -f kata-deploy/base/kata-deploy.yaml
oc -n kube-system wait --timeout=10m --for=delete -l name=kata-deploy pod
oc apply -f kata-cleanup/base/kata-cleanup.yaml
echo "Wait for all related pods to be gone"
( repeats=1; for i in $(seq 1 600); do
oc get pods -l name="kubelet-kata-cleanup" --no-headers=true -n kube-system 2>&1 | grep "No resources found" -q && ((repeats++)) || repeats=1
[ "$repeats" -gt 5 ] && echo kata-cleanup finished && break
sleep 1
done) || { echo "There are still some kata-cleanup related pods after 600 iterations"; oc get all -n kube-system; exit -1; }
oc delete -f kata-cleanup/base/kata-cleanup.yaml
oc delete -f kata-rbac/base/kata-rbac.yaml
oc delete -f runtimeclasses/kata-runtimeClasses.yaml

View File

@@ -1,6 +0,0 @@
# Copyright (c) 2020 Red Hat, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
SELINUX=permissive
SELINUXTYPE=targeted

View File

@@ -1,36 +0,0 @@
#!/bin/bash
#
# Copyright (c) 2021 Red Hat, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
# This script builds the kata-webhook and deploys it in the test cluster.
#
# You should export the KATA_RUNTIME variable with the runtimeclass name
# configured in your cluster in case it is not the default "kata-ci".
#
set -e
set -o nounset
set -o pipefail
script_dir="$(realpath $(dirname $0))"
webhook_dir="${script_dir}/../../../tools/testing/kata-webhook"
source "${script_dir}/../lib.sh"
KATA_RUNTIME=${KATA_RUNTIME:-kata-ci}
pushd "${webhook_dir}" >/dev/null
# Build and deploy the webhook
#
info "Builds the kata-webhook"
./create-certs.sh
info "Deploys the kata-webhook"
oc apply -f deploy/
info "Override our KATA_RUNTIME ConfigMap"
RUNTIME_CLASS="${KATA_RUNTIME}" \
envsubst < "${script_dir}/deployments/configmap_kata-webhook.yaml.in" \
| oc apply -f -
# Check the webhook was deployed and is working.
RUNTIME_CLASS="${KATA_RUNTIME}" ./webhook-check.sh
popd >/dev/null

View File

@@ -1,13 +0,0 @@
# Copyright (c) 2021 Red Hat, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
# Instruct the daemonset installer to configure Kata Containers to use the
# host kernel.
#
apiVersion: v1
kind: ConfigMap
metadata:
name: ci.kata.installer.kernel
data:
host_kernel: "yes"

View File

@@ -1,14 +0,0 @@
# Copyright (c) 2021 Red Hat, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
# Instruct the daemonset installer to configure Kata Containers to use the
# system QEMU.
#
apiVersion: v1
kind: ConfigMap
metadata:
name: ci.kata.installer.qemu
data:
qemu_path: /usr/libexec/qemu-kvm
host_kernel: "yes"

View File

@@ -1,12 +0,0 @@
# Copyright (c) 2021 Red Hat, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
# Apply customizations to the kata-webhook.
#
apiVersion: v1
kind: ConfigMap
metadata:
name: kata-webhook
data:
runtime_class: ${RUNTIME_CLASS}

View File

@@ -1,9 +0,0 @@
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: worker
name: 50-enable-sandboxed-containers-extension
spec:
extensions:
- sandboxed-containers

View File

@@ -1,23 +0,0 @@
# Copyright (c) 2020 Red Hat, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
# Configure SELinux on worker nodes.
---
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: worker
name: 51-kata-selinux
spec:
config:
ignition:
version: 2.2.0
storage:
files:
- contents:
source: data:text/plain;charset=utf-8;base64,${SELINUX_CONF_BASE64}
filesystem: root
mode: 0644
path: /etc/selinux/config

View File

@@ -1,40 +0,0 @@
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: relabel-selinux-daemonset
namespace: kube-system
spec:
selector:
matchLabels:
app: restorecon
template:
metadata:
labels:
app: restorecon
spec:
serviceAccountName: kata-deploy-sa
hostPID: true
containers:
- name: relabel-selinux-container
image: alpine
securityContext:
privileged: true
command: ["/bin/sh", "-c", "
set -e;
echo Starting the relabel;
nsenter --target 1 --mount bash -xc '
command -v semanage &>/dev/null || { echo Does not look like a SELINUX cluster, skipping; exit 0; };
for ENTRY in \
\"/(.*/)?opt/kata/bin(/.*)?\" \
\"/(.*/)?opt/kata/runtime-rs/bin(/.*)?\" \
\"/(.*/)?opt/kata/share/kata-.*(/.*)?(/.*)?\" \
\"/(.*/)?opt/kata/share/ovmf(/.*)?\" \
\"/(.*/)?opt/kata/share/tdvf(/.*)?\" \
\"/(.*/)?opt/kata/libexec(/.*)?\";
do
semanage fcontext -a -t qemu_exec_t \"$ENTRY\" || semanage fcontext -m -t qemu_exec_t \"$ENTRY\" || { echo \"Error in semanage command\"; exit 1; }
done;
restorecon -v -R /opt/kata || { echo \"Error in restorecon command\"; exit 1; }
';
echo NSENTER_FINISHED_WITH: $?;
sleep infinity"]

View File

@@ -1,28 +0,0 @@
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: workaround-9206-crio-ds
spec:
selector:
matchLabels:
app: workaround-9206-crio-ds
template:
metadata:
labels:
app: workaround-9206-crio-ds
spec:
containers:
- name: workaround-9206-crio-ds
image: alpine
volumeMounts:
- name: host-dir
mountPath: /tmp/config
securityContext:
runAsUser: 0
privileged: true
command: ["/bin/sh", "-c", "while [ ! -f '/tmp/config/10-workaround-9206-crio' ]; do sleep 1; done; echo 'Config file present'; sleep infinity"]
volumes:
- name: host-dir
hostPath:
path: /etc/crio/crio.conf.d/

View File

@@ -1,18 +0,0 @@
---
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: worker
name: 10-workaround-9206-crio
spec:
config:
ignition:
version: 2.2.0
storage:
files:
- contents:
source: data:text/plain;charset=utf-8;base64,W2NyaW9dCnN0b3JhZ2Vfb3B0aW9uID0gWwoJIm92ZXJsYXkuc2tpcF9tb3VudF9ob21lPXRydWUiLApdCg==
filesystem: root
mode: 0644
path: /etc/crio/crio.conf.d/10-workaround-9206-crio

View File

@@ -1,245 +0,0 @@
#!/bin/bash
#
# Copyright (c) 2020 Red Hat, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
# This script installs the built kata-containers in the test cluster,
# and configure a runtime.
scripts_dir=$(dirname $0)
deployments_dir=${scripts_dir}/deployments
configs_dir=${scripts_dir}/configs
source ${scripts_dir}/../lib.sh
# Set to 'yes' if you want to configure SELinux to permissive on the cluster
# workers.
#
SELINUX_PERMISSIVE=${SELINUX_PERMISSIVE:-no}
# Set to 'yes' if you want to configure Kata Containers to use the system's
# QEMU (from the RHCOS extension).
#
KATA_WITH_SYSTEM_QEMU=${KATA_WITH_SYSTEM_QEMU:-no}
# Set to 'yes' if you want to configure Kata Containers to use the host kernel.
#
KATA_WITH_HOST_KERNEL=${KATA_WITH_HOST_KERNEL:-no}
# kata-deploy image to be used to deploy the kata (by default use CI image
# that is built for each pull request)
#
KATA_DEPLOY_IMAGE=${KATA_DEPLOY_IMAGE:-quay.io/kata-containers/kata-deploy-ci:kata-containers-latest}
# Enable workaround for OCP 4.13 https://github.com/kata-containers/kata-containers/pull/9206
#
WORKAROUND_9206_CRIO=${WORKAROUND_9206_CRIO:-no}
# Leverage kata-deploy to install Kata Containers in the cluster.
#
apply_kata_deploy() {
local deploy_file="tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml"
pushd "$katacontainers_repo_dir"
sed -ri "s#(\s+image:) .*#\1 ${KATA_DEPLOY_IMAGE}#" "$deploy_file"
info "Applying kata-deploy"
oc apply -f tools/packaging/kata-deploy/kata-rbac/base/kata-rbac.yaml
oc label --overwrite ns kube-system pod-security.kubernetes.io/enforce=privileged pod-security.kubernetes.io/warn=baseline pod-security.kubernetes.io/audit=baseline
oc apply -f "$deploy_file"
oc -n kube-system wait --timeout=10m --for=condition=Ready -l name=kata-deploy pod
info "Adding the kata runtime classes"
oc apply -f tools/packaging/kata-deploy/runtimeclasses/kata-runtimeClasses.yaml
popd
}
# Wait all worker nodes reboot.
#
# Params:
# $1 - timeout in seconds (default to 900).
#
wait_for_reboot() {
local delta="${1:-900}"
local sleep_time=60
declare -A BOOTIDS
local workers=($(oc get nodes | \
awk '{if ($3 == "worker") { print $1 } }'))
# Get the boot ID to compared it changed over time.
for node in ${workers[@]}; do
BOOTIDS[$node]=$(oc get -o jsonpath='{.status.nodeInfo.bootID}'\
node/$node)
echo "Wait $node reboot"
done
echo "Set timeout to $delta seconds"
timer_start=$(date +%s)
while [ ${#workers[@]} -gt 0 ]; do
sleep $sleep_time
now=$(date +%s)
if [ $(($timer_start + $delta)) -lt $now ]; then
echo "Timeout: not all workers rebooted"
return 1
fi
echo "Checking after $(($now - $timer_start)) seconds"
for i in ${!workers[@]}; do
current_id=$(oc get \
-o jsonpath='{.status.nodeInfo.bootID}' \
node/${workers[i]})
if [ "$current_id" != ${BOOTIDS[${workers[i]}]} ]; then
echo "${workers[i]} rebooted"
unset workers[i]
fi
done
done
}
wait_mcp_update() {
local delta="${1:-3600}"
local sleep_time=30
# The machineconfigpool is fine when all the workers updated and are ready,
# and none are degraded.
local ready_count=0
local degraded_count=0
local machine_count=$(oc get mcp worker -o jsonpath='{.status.machineCount}')
if [[ -z "$machine_count" && "$machine_count" -lt 1 ]]; then
warn "Unabled to obtain the machine count"
return 1
fi
echo "Set timeout to $delta seconds"
local deadline=$(($(date +%s) + $delta))
# The ready count might not have changed yet, so wait a little.
while [[ "$ready_count" != "$machine_count" && \
"$degraded_count" == 0 ]]; do
# Let's check it hit the timeout (or not).
local now=$(date +%s)
if [ $deadline -lt $now ]; then
echo "Timeout: not all workers updated" >&2
return 1
fi
sleep $sleep_time
ready_count=$(oc get mcp worker \
-o jsonpath='{.status.readyMachineCount}')
degraded_count=$(oc get mcp worker \
-o jsonpath='{.status.degradedMachineCount}')
echo "check machineconfigpool - ready_count: $ready_count degraded_count: $degraded_count"
done
[ $degraded_count -eq 0 ]
}
# Enable the RHCOS extension for the Sandboxed Containers.
#
enable_sandboxedcontainers_extension() {
info "Enabling the RHCOS extension for Sandboxed Containers"
local deployment_file="${deployments_dir}/machineconfig_sandboxedcontainers_extension.yaml"
oc apply -f ${deployment_file}
oc get -f ${deployment_file} || \
die "Sandboxed Containers extension machineconfig not found"
wait_mcp_update || die "Failed to update the machineconfigpool"
}
# Print useful information for debugging.
#
# Params:
# $1 - the pod name
debug_pod() {
local pod="$1"
info "Debug pod: ${pod}"
oc describe pods "$pod"
oc logs "$pod"
}
# Wait for all pods of the app label to contain expected message
#
# Params:
# $1 - app labela
# $2 - expected pods count (>=1)
# $3 - message to be present in the logs
# $4 - timeout (60)
# $5 - namespace (the current one)
wait_for_app_pods_message() {
local app="$1"
local pod_count="$2"
local message="$3"
local timeout="$4"
local namespace="$5"
[ -z "$pod_count" ] && pod_count=1
[ -z "$timeout" ] && timeout=60
[ -n "$namespace" ] && namespace=" -n $namespace "
local pod
local pods
local i
SECONDS=0
while :; do
pods=($(oc get pods -l app="$app" --no-headers=true $namespace | awk '{print $1}'))
[ "${#pods}" -ge "$pod_count" ] && break
if [ "$SECONDS" -gt "$timeout" ]; then
echo "Unable to find ${pod_count} pods for '-l app=\"$app\"' in ${SECONDS}s (${pods[@]})"
return -1
fi
done
for pod in "${pods[@]}"; do
while :; do
local log=$(oc logs $namespace "$pod")
echo "$log" | grep "$message" -q && echo "Found $(echo "$log" | grep "$message") in $pod's log ($SECONDS)" && break;
if [ "$SECONDS" -gt "$timeout" ]; then
echo -n "Message '$message' not present in '${pod}' pod of the '-l app=\"$app\"' "
echo "pods after ${SECONDS}s (${pods[@]})"
echo "Pod $pod's output so far:"
echo "$log"
return -1
fi
sleep 1;
done
done
}
oc config set-context --current --namespace=default
worker_nodes=$(oc get nodes | awk '{if ($3 == "worker") { print $1 } }')
num_nodes=$(echo $worker_nodes | wc -w)
[ $num_nodes -ne 0 ] || \
die "No worker nodes detected. Something is wrong with the cluster"
if [ "${KATA_WITH_SYSTEM_QEMU}" == "yes" ]; then
# QEMU is deployed on the workers via RCHOS extension.
enable_sandboxedcontainers_extension
oc apply -f ${deployments_dir}/configmap_installer_qemu.yaml
fi
if [ "${KATA_WITH_HOST_KERNEL}" == "yes" ]; then
oc apply -f ${deployments_dir}/configmap_installer_kernel.yaml
fi
apply_kata_deploy
# Set SELinux to permissive mode
if [ ${SELINUX_PERMISSIVE} == "yes" ]; then
info "Configuring SELinux"
if [ -z "$SELINUX_CONF_BASE64" ]; then
export SELINUX_CONF_BASE64=$(echo \
$(cat $configs_dir/selinux.conf|base64) | \
sed -e 's/\s//g')
fi
envsubst < ${deployments_dir}/machineconfig_selinux.yaml.in | \
oc apply -f -
oc get machineconfig/51-kata-selinux || \
die "SELinux machineconfig not found"
# The new SELinux configuration will trigger another reboot.
wait_for_reboot
fi
if [[ "$WORKAROUND_9206_CRIO" == "yes" ]]; then
info "Applying workaround to enable skip_mount_home in crio on OCP 4.13"
oc apply -f "${deployments_dir}/workaround-9206-crio.yaml"
oc apply -f "${deployments_dir}/workaround-9206-crio-ds.yaml"
wait_for_app_pods_message workaround-9206-crio-ds "$num_nodes" "Config file present" 1200 || echo "Failed to apply the workaround, proceeding anyway..."
fi
# FIXME: Remove when https://github.com/kata-containers/kata-containers/pull/8417 is resolved
# Selinux context is currently not handled by kata-deploy
oc apply -f ${deployments_dir}/relabel_selinux.yaml
wait_for_app_pods_message restorecon "$num_nodes" "NSENTER_FINISHED_WITH:" 120 "kube-system" || echo "Failed to treat selinux, proceeding anyway..."

View File

@@ -4,7 +4,7 @@
#
# This is the build root image for Kata Containers on OpenShift CI.
#
FROM quay.io/centos/centos:stream9
FROM quay.io/centos/centos:stream8
RUN yum -y update && \
yum -y install \

View File

@@ -1,20 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2023 Red Hat
#
# SPDX-License-Identifier: Apache-2.0
#
# Ensure GOPATH set
if command -v go > /dev/null; then
export GOPATH=${GOPATH:-$(go env GOPATH)}
else
# if go isn't installed, set default location for GOPATH
export GOPATH="${GOPATH:-$HOME/go}"
fi
lib_dir=$(dirname "${BASH_SOURCE[0]}")
source "$lib_dir/../../tests/common.bash"
export katacontainers_repo=${katacontainers_repo:="github.com/kata-containers/kata-containers"}
export katacontainers_repo_dir="${GOPATH}/src/${katacontainers_repo}"

View File

@@ -1,94 +0,0 @@
#!/bin/bash
#
# Copyright (c) 2020 Red Hat, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
# Run a smoke test.
#
script_dir=$(dirname $0)
source ${script_dir}/lib.sh
pod='http-server'
# Create a pod.
#
info "Creating the ${pod} pod"
[ -z "$KATA_RUNTIME" ] && die "Please set the KATA_RUNTIME first"
envsubst < "${script_dir}/smoke/${pod}.yaml.in" | \
oc apply -f - || \
die "failed to create ${pod} pod"
# Check it eventually goes to 'running'
#
wait_time=600
sleep_time=5
cmd="oc get pod/${pod} -o jsonpath='{.status.containerStatuses[0].state}' | \
grep running > /dev/null"
info "Wait until the pod gets running"
waitForProcess $wait_time $sleep_time "$cmd" || timed_out=$?
if [ -n "$timed_out" ]; then
oc describe pod/${pod}
oc delete pod/${pod}
die "${pod} not running"
fi
info "${pod} is running"
# Add a file with the hello message
#
hello_file=/tmp/hello
hello_msg='Hello World'
oc exec ${pod} -- sh -c "echo $hello_msg > $hello_file"
info "Creating the service and route"
if oc apply -f ${script_dir}/smoke/service.yaml; then
# Likely on OCP, use service
is_ocp=1
host=$(oc get route/http-server-route -o jsonpath={.spec.host})
port=80
else
# Likely on plain kubernetes, test using another container
is_ocp=0
info "Failed to create service, likely not on OCP, trying via NodePort"
oc apply -f "${script_dir}/smoke/service_kubernetes.yaml"
# For some reason kcli's cluster lists external IP as internal IP, try both
host=$(oc get nodes -o jsonpath='{.items[0].status.addresses[?(@.type=="ExternalIP")].address}')
[ -z "$host"] && host=$(oc get nodes -o jsonpath='{.items[0].status.addresses[?(@.type=="InternalIP")].address}')
port=$(oc get service/http-server-service -o jsonpath='{.spec.ports[0].nodePort}')
fi
info "Wait for the HTTP server to respond"
tempfile=$(mktemp)
check_cmd="curl -vvv '${host}:${port}${hello_file}' 2>&1 | tee -a '$tempfile' | grep -q '$hello_msg'"
if waitForProcess 60 1 "${check_cmd}"; then
test_status=0
info "HTTP server is working"
else
test_status=1
echo "::error:: HTTP server not working"
echo "::group::Output of the \"curl -vvv '${host}:${port}${hello_file}'\""
cat "${tempfile}"
echo "::endgroup::"
echo "::group::Describe kube-system namespace"
oc describe -n kube-system all
echo "::endgroup::"
echo "::group::Descibe current namespace"
oc describe all
echo "::endgroup::"
info "HTTP server is unreachable"
fi
rm -f "$tempfile"
# Delete the resources.
#
info "Deleting the service/route"
if [ "$is_ocp" -eq 0 ]; then
oc delete -f ${script_dir}/smoke/service_kubernetes.yaml
else
oc delete -f ${script_dir}/smoke/service.yaml
fi
info "Deleting the ${pod} pod"
oc delete pod/${pod} || test_status=$?
exit $test_status

Some files were not shown because too many files have changed in this diff Show More