Commit Graph

12445 Commits

Author SHA1 Message Date
Dan Mihai
159bc2713f rootfs: add default agent policy file
Restricting access to agent endpoints using agent-config.toml is
expected to be deprecated in the main branch. Therefore, in
preparation of merging this script with its main branch version,
install default settings for main branch's kata-opa service.

coco-default.rego blocks access to the same kata agent endpoints
that are blocked by agent-config.toml. For additional information,
search for "default-policy.rego" in main branch's rootfs.sh.

Fixes: #8219

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2023-10-17 13:16:45 +00:00
Steve Horsman
3bd72b9a0d
Merge pull request #8224 from stevenhorsman/CCv0-bump-guest-components-917d5cf
versions: Bump guest-components
2023-10-13 22:21:25 +01:00
Steve Horsman
0eda83fa52
Merge pull request #8211 from BbolroC/sealed_secret_s390x
cc|rootfs: Define SEALED_SECRET for cc-rootfs-initrd-tarball
2023-10-13 15:53:57 +01:00
stevenhorsman
7f89d291ee versions: Bump guest-components
- Bump guest components to pick up fix to issue in
https://github.com/confidential-containers/guest-components/pull/374

Fixes: #8223
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-10-13 14:19:26 +01:00
Steve Horsman
69b820a9bd
Merge pull request #8209 from yoheiueda/CCv0-remote-secret
CCv0: runtime: fix k8s secret issue with remote hypervisor
2023-10-12 18:23:35 +01:00
Hyounggyu Choi
981f0a1f0f cc|rootfs: Define SEALED_SECRET for cc-rootfs-initrd-tarball
This is to define `SEALED_SECRET` as yes for a make target `cc-rootfs-initrd-tarball`,
which makes a service `confidential-data-hub` available with an initrd-based VM creation.

Fixes: #8210

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-10-12 14:28:46 +02:00
Yohei Ueda
d9c9040474
runtime: fix k8s secret issue with remote hyp
kata-shim CCv0 does not propagate dynamically
updated k8s secret values due to incorrect
file name matching. This patch fixes the the wrong file
name matching for k8s secret volume paths.

Note that this problem has already fixed in the main
branch.

Fixes: #8208
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
2023-10-12 15:07:10 +09:00
Steve Horsman
64392c9a87
Merge pull request #8173 from bpradipt/gpu-fix
CC | config: Enable guestHook for remote hyp
2023-10-10 17:15:02 +01:00
Fabiano Fidêncio
548e4c1667
Merge pull request #8195 from fidencio/topic/CC-bump-td-shim
versions: bump td-shim version
2023-10-10 14:03:45 +02:00
Fabiano Fidêncio
a1eab1248c versions: bump td-shim version
We're doing so in order to avoid errors like the ones from here:
https://github.com/kata-containers/kata-containers/actions/runs/6468115049/job/17559485246

I was able to locally build it, but as we're not testing CLH + TDX
lately, running CI on this will be useless.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-10-10 13:53:40 +02:00
Fabiano Fidêncio
cf8f8bc2cf
Merge pull request #8194 from fidencio/topic/CC-do-not-use-cache-when-building-the-payload
CC | payload-after-push: Bypass cache
2023-10-10 12:35:12 +02:00
Fabiano Fidêncio
18b378dbb4 payload-after-push: Bypass cache
We're facing errors in the operator CI, which are related to the
measured rootfs.

For now, let's skip the cache builds (as those were dropped anyways for
this branch) and ensure we do a clean build, and then check if the
problem persists.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-10-10 12:31:33 +02:00
Pradipta Banerjee
6be56addd8 config: Enable guestHook for remote hyp
The guesthook config was missing which prevented handling
of GPUs with remote hypervisor

Fixes: #8172

Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
2023-10-09 06:30:10 +00:00
Steve Horsman
1ad87faaf4
Merge pull request #8053 from ChengyuZhu6/metadata
CC | runtime: merge metadata and annotation in imagepullvolume
2023-09-26 15:30:34 +01:00
Steve Horsman
35b7b0379c
Merge pull request #8050 from mattarnoatibm/CCv0-set-policy-path-from-agent-config
CC | policy: fix setting the policy path from agent config
2023-09-26 10:47:43 +01:00
Fabiano Fidêncio
1f0cc490bd
Merge pull request #7715 from fidencio/topic/CCv0-configure-snapshotter-as-part-of-kata-deploy
CC: kata-deploy: Set the snapshotter in the containerd runtime config
2023-09-26 11:16:03 +02:00
ChengyuZhu6
0ad5dff3b8 runtime: merge metadata and annotation in imagepullvolume
merge metadata and annotation in imagepullvolume

Fixes #8060

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-26 08:51:44 +08:00
Fabiano Fidêncio
5e6cecf01a kata-deploy: Remove cri_handler = cc
This won't be used anymore as we won't be depending on the forked
version of containerd.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-25 22:58:59 +02:00
Fabiano Fidêncio
9fb0eb4076 CC: kata-deploy: Set the snapshotter in the containerd runtime config
This is a patch that should **NOT** be forward ported to main, as there
we want to take a cleaner approach on configuring specific snapshotters
for specific runtime handlers.

However, for CC, for the v0.8.0 release of CC, this is good enough as it
is, and it'll allow us to set one snapshotter for all the deployments
done with the CoCo Operator.

This is the Kata Containers counterpart of the work, and there's still
work to be done on the Confidential Containers in order to make it work
as expected, as:
* Confidential Containers Operator has to expose to the users which
  snapshotter will be configured
* Confidential Containers Opereator, specifically the pre-install hook,
  will have to take care of actually installing and configuring the
  snapshotter, so it can be used.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-25 22:57:18 +02:00
Matthew Arnold
7fc822896c policy: fix setting the policy path from agent config
Fix setting the image service policy path when there
is a policy path in the agent config.

Fixes #8049

Signed-off-by: Matthew Arnold <mattarno@uk.ibm.com>
2023-09-25 17:00:13 +01:00
Fabiano Fidêncio
b588c1128c
Merge pull request #7676 from ChengyuZhu6/pull_image_in_guest
CC | image pulling in the guest without forked containerd
2023-09-20 21:11:24 +02:00
Fabiano Fidêncio
7ee7ca2b31
Merge pull request #8021 from fidencio/topic/CC-fix-sev-tdx-rootfs-cached-artefacts
CC | cache: Fix rootfs-image-tdx and rootfs-initrd-sev cached artefacts
2023-09-20 19:00:38 +02:00
Fabiano Fidêncio
3dbbbc88ac cache: Fix rootfs-image-tdx and rootfs-initrd-sev cached artefacts
The name of the tarballs changed on main, but we didn't follow up
changing this on the CCv0 branch. :-/

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-20 18:49:27 +02:00
Fabiano Fidêncio
6ae4951a8c
Merge pull request #8010 from arronwy/reduce_binary_size
CC | osbuild: Reduce guest components binary size with strip
2023-09-20 13:32:39 +02:00
ChengyuZhu6
e18a425fbb static-build: Fix arch error on nydus build
Fix the arch error when downloading the nydus tarball.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Signed-off-by: Steven Horsman <steven@uk.ibm.com>
(cherry picked from commit f6df3d6efb)
2023-09-20 16:09:38 +08:00
Fabiano Fidêncio
e14424011c
Merge pull request #7998 from fidencio/topic/CC-add-forked-vanilla-entries-for-containerd
CC | version: Specify forked / vanilla entries of containerd
2023-09-20 10:06:04 +02:00
Wang, Arron
58e8eed807 osbuild: Reduce guest components binary size with strip
Guest rootfs will aligned to 128M, we may exceed the rootfs
with several megabytes but the rootfs will add 128M.

Fixes: #8009

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2023-09-20 15:07:21 +08:00
Fabiano Fidêncio
6e784fb6b3
Merge pull request #7948 from stevenhorsman/api-server-rest-rootfs
osbuilder: Update api-server-rest
2023-09-20 08:47:04 +02:00
Fabiano Fidêncio
b07dfbe213 versions: Bump nydus and nydus-snapshotter to its latest release
As we need https://github.com/containerd/nydus-snapshotter/pull/530 in.

Fixes #7984

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-19 22:23:15 +02:00
Fabiano Fidêncio
6329c7e290 versions: Specify forked / vanilla entries of containerd
As we'll need to test with both the vanilla and the forked versions of
containerd, let's make sure we'll specify both entries as part of our
versions.yaml file, and we can read whatever we need accordingly as part
of our tests jobs.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-19 22:23:09 +02:00
Fabiano Fidêncio
d5d0befc61 build: Pass RUSTUP_UPDATE_ROOT & RUSTUP_DIST_SERVER down
This may help us to decrease the amount of issues we're having with the
TDX CI.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-18 21:05:11 +02:00
ChengyuZhu6
87b3f6a63c runtime: add functions to handle ImageGuestPull to storage
Add functions to handle ImageGuestPull of KataVirtualVolume.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-18 21:05:11 +02:00
ChengyuZhu6
53ea36d3f5 agent: Introduce ImagePullHandler to support IMAGE_GUEST_PULL volume
As we do not employ a forked containerd, we utilize the KataVirtualVolume
which storing the image url supplied by snapshotter as an integral part of `CreateContainer`.
Within this process, we store the image information in rootfs.storage and pass this image url through `CreateContainerRequest`.
This approach distinguishes itself from the use of `PullImageRequest`, as rootfs.storage is already set and initialized at this stage.
To maintain clarity and avoid any need for modification to the `OverlayfsHandler`,we introduce the `ImagePullHandler`.
This dedicated handler is responsible for orchestrating the image-pulling logic within the guest environment.
This logic encompasses tasks such as calling the image-rs to download and unpack the image into `/run/kata-containers/{container_id}/images`,
followed by a bind mount to `/run/kata-containers/{container_id}`.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-18 21:05:11 +02:00
ChengyuZhu6
d0ac25f5c5 agent: redefine functions of pulling image in the guest
Without using forked containerd, the kata-agent wouldn't receive the `PullImageRequest`.
To using nydus-snapshotter, kata-agent can pass the image url and container id to image-rs
to handle pulling image.So we need to redefine functions of pulling image in the guest to support
both PullImageRequest and remote snapshotter.
1) Extract codes for setting proxy environment variables into a separate function `set_proxy_env_vars`.
2) Create a separate function `handle_attestation_agent` to handle attestation agent
   initialization.
3) Create a separate function `common_image_pull` for image pull logic.
4) Extract codes for unpacking pause image into a separate function `unpack_pause_image` and pass the necessary parameters to customize the behavior.

Fixes #7790

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Jiang Liu <gerry@linux.alibaba.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
Co-authored-by: wllenyj <wllenyj@linux.alibaba.com>
Co-authored-by: jordan9500 <jordan.jackson@ibm.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
2023-09-18 21:05:11 +02:00
stevenhorsman
0ffc9c02a2 agent: bump image-rs version
- Bump image-rs to stay aligned with guest-components version

Fixes: #7947
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-18 11:01:10 +01:00
stevenhorsman
560e21d8e3 versions: Bump guest-components
- Bump guest-components to pick up the new api-server-rest Makefile

Fixes: #7947
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-18 11:00:20 +01:00
Greg Kurz
87b8164add
Merge pull request #7951 from BbolroC/reduce-redundancy-se-image
cc|gha: remove build redundancy of kernel and cc-rootfs-initrd for s390x
2023-09-18 10:12:24 +02:00
Fabiano Fidêncio
6b3c63570e
Merge pull request #7981 from BbolroC/fix-agent-double-build-for-initrd
cc|osbuilder: Correct a typo in the initrd-image
2023-09-16 00:39:35 +02:00
Hyounggyu Choi
5e5e78cd6f cc|osbuilder: Correct a typo in the initrd-image
This PR is to prevent rootfs.sh from running twice by fixing the typo `initrd-image`.

Fixes: #7980

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-09-15 19:18:31 +02:00
Hyounggyu Choi
4f1d631af1 cc|gha: remove build redundancy of kernel and cc-rootfs-initrd for s390x
This PR is to remove the build redundancy of `kernel` and `cc-rootfs-initrd` by making `cc-se-image` built based on them at the second build stage.

Fixes: #7949

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-09-14 16:48:34 +02:00
stevenhorsman
4662a7a942 osbuilder: Update api-server-rest
- Switch api-server-rest to use the Makefile rather than
directly calling cargo for multi-platform support and decoupling

Fixes: #7947
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-14 11:24:54 +01:00
Fabiano Fidêncio
ee15a389de
Merge pull request #7688 from ChengyuZhu6/image_sharing_dmverity
CC | support dm-verity tarfs disk image in CoCo
2023-09-13 20:52:01 +02:00
Fabiano Fidêncio
b71443a7ae
Merge pull request #7878 from ChengyuZhu6/check_rust
CC | osbuilder: check rust enviornment before building agent and AA
2023-09-13 17:15:59 +02:00
ChengyuZhu6
a533c974f9 agent: enable verity feature in image-rs
update image-rs to support verity feature

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 18:38:51 +08:00
Fabiano Fidêncio
d0b69651ce
Merge pull request #7917 from stevenhorsman/cherry-pick-cgroups-fixes
agent: optimize the code of systemd cgroup manager
2023-09-13 09:53:46 +02:00
ChengyuZhu6
8f38dcb850 osbuilder: check rust enviornment before building agent and AA
We should configure the Rust environment when AGENT_SOURCE_BIN is empty or AA_KBC is not empty.

Fixes #7877

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 15:22:49 +08:00
Fabiano Fidêncio
f462c69484
Merge pull request #7880 from ChengyuZhu6/udev
image-builder: fix udev error when using docker to build image
2023-09-13 07:47:35 +02:00
ChengyuZhu6
622bd4e370 agent: create directories to mount filesystem by overlay
When creating a container with a raw disk image using virtio-blk,
the guest does not have the upper directory and worker directory present.
Therefore, it is necessary to create these directories before mounting the filesystem with overlay.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 10:01:12 +08:00
ChengyuZhu6
72c9f62b70 agent: introduce DmVerityHandler to support dm-verity volume
We utilize the KataVirtualVolume which storing the dm-verity info
and the path of disk image on the host supplied by snapshotter as an integral part of `CreateContainer`.
Within this process, we copy the verity info and the disk image path to mount slice to create a block device by virtio-blk.
Then storing the `lowerdir` in rootfs.storage which is the mountpoint of the verity path through `CreateContainerRequest`.
To maintain clarity and avoid any need for modification to the `VirtioBlkPciHandler`,we introduce the `DmVerityHandler`.
This dedicated handler is responsible for calling image-rs to create verity device and mount the device to the `lowerdir` within the guest environment.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 10:01:11 +08:00
ChengyuZhu6
fd33309475 image-builder: fix udev error when using docker to build image
Incorporate the `DM_VERITY` parameter when building the image with docker.

Fixes #7879

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 09:34:22 +08:00