Commit Graph

12426 Commits

Author SHA1 Message Date
Fabiano Fidêncio
9fb0eb4076 CC: kata-deploy: Set the snapshotter in the containerd runtime config
This is a patch that should **NOT** be forward ported to main, as there
we want to take a cleaner approach on configuring specific snapshotters
for specific runtime handlers.

However, for CC, for the v0.8.0 release of CC, this is good enough as it
is, and it'll allow us to set one snapshotter for all the deployments
done with the CoCo Operator.

This is the Kata Containers counterpart of the work, and there's still
work to be done on the Confidential Containers in order to make it work
as expected, as:
* Confidential Containers Operator has to expose to the users which
  snapshotter will be configured
* Confidential Containers Opereator, specifically the pre-install hook,
  will have to take care of actually installing and configuring the
  snapshotter, so it can be used.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-25 22:57:18 +02:00
Fabiano Fidêncio
b588c1128c
Merge pull request #7676 from ChengyuZhu6/pull_image_in_guest
CC | image pulling in the guest without forked containerd
2023-09-20 21:11:24 +02:00
Fabiano Fidêncio
7ee7ca2b31
Merge pull request #8021 from fidencio/topic/CC-fix-sev-tdx-rootfs-cached-artefacts
CC | cache: Fix rootfs-image-tdx and rootfs-initrd-sev cached artefacts
2023-09-20 19:00:38 +02:00
Fabiano Fidêncio
3dbbbc88ac cache: Fix rootfs-image-tdx and rootfs-initrd-sev cached artefacts
The name of the tarballs changed on main, but we didn't follow up
changing this on the CCv0 branch. :-/

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-20 18:49:27 +02:00
Fabiano Fidêncio
6ae4951a8c
Merge pull request #8010 from arronwy/reduce_binary_size
CC | osbuild: Reduce guest components binary size with strip
2023-09-20 13:32:39 +02:00
ChengyuZhu6
e18a425fbb static-build: Fix arch error on nydus build
Fix the arch error when downloading the nydus tarball.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Signed-off-by: Steven Horsman <steven@uk.ibm.com>
(cherry picked from commit f6df3d6efb)
2023-09-20 16:09:38 +08:00
Fabiano Fidêncio
e14424011c
Merge pull request #7998 from fidencio/topic/CC-add-forked-vanilla-entries-for-containerd
CC | version: Specify forked / vanilla entries of containerd
2023-09-20 10:06:04 +02:00
Wang, Arron
58e8eed807 osbuild: Reduce guest components binary size with strip
Guest rootfs will aligned to 128M, we may exceed the rootfs
with several megabytes but the rootfs will add 128M.

Fixes: #8009

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2023-09-20 15:07:21 +08:00
Fabiano Fidêncio
6e784fb6b3
Merge pull request #7948 from stevenhorsman/api-server-rest-rootfs
osbuilder: Update api-server-rest
2023-09-20 08:47:04 +02:00
Fabiano Fidêncio
b07dfbe213 versions: Bump nydus and nydus-snapshotter to its latest release
As we need https://github.com/containerd/nydus-snapshotter/pull/530 in.

Fixes #7984

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-19 22:23:15 +02:00
Fabiano Fidêncio
6329c7e290 versions: Specify forked / vanilla entries of containerd
As we'll need to test with both the vanilla and the forked versions of
containerd, let's make sure we'll specify both entries as part of our
versions.yaml file, and we can read whatever we need accordingly as part
of our tests jobs.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-19 22:23:09 +02:00
Fabiano Fidêncio
d5d0befc61 build: Pass RUSTUP_UPDATE_ROOT & RUSTUP_DIST_SERVER down
This may help us to decrease the amount of issues we're having with the
TDX CI.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-18 21:05:11 +02:00
ChengyuZhu6
87b3f6a63c runtime: add functions to handle ImageGuestPull to storage
Add functions to handle ImageGuestPull of KataVirtualVolume.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-18 21:05:11 +02:00
ChengyuZhu6
53ea36d3f5 agent: Introduce ImagePullHandler to support IMAGE_GUEST_PULL volume
As we do not employ a forked containerd, we utilize the KataVirtualVolume
which storing the image url supplied by snapshotter as an integral part of `CreateContainer`.
Within this process, we store the image information in rootfs.storage and pass this image url through `CreateContainerRequest`.
This approach distinguishes itself from the use of `PullImageRequest`, as rootfs.storage is already set and initialized at this stage.
To maintain clarity and avoid any need for modification to the `OverlayfsHandler`,we introduce the `ImagePullHandler`.
This dedicated handler is responsible for orchestrating the image-pulling logic within the guest environment.
This logic encompasses tasks such as calling the image-rs to download and unpack the image into `/run/kata-containers/{container_id}/images`,
followed by a bind mount to `/run/kata-containers/{container_id}`.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-18 21:05:11 +02:00
ChengyuZhu6
d0ac25f5c5 agent: redefine functions of pulling image in the guest
Without using forked containerd, the kata-agent wouldn't receive the `PullImageRequest`.
To using nydus-snapshotter, kata-agent can pass the image url and container id to image-rs
to handle pulling image.So we need to redefine functions of pulling image in the guest to support
both PullImageRequest and remote snapshotter.
1) Extract codes for setting proxy environment variables into a separate function `set_proxy_env_vars`.
2) Create a separate function `handle_attestation_agent` to handle attestation agent
   initialization.
3) Create a separate function `common_image_pull` for image pull logic.
4) Extract codes for unpacking pause image into a separate function `unpack_pause_image` and pass the necessary parameters to customize the behavior.

Fixes #7790

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Jiang Liu <gerry@linux.alibaba.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
Co-authored-by: wllenyj <wllenyj@linux.alibaba.com>
Co-authored-by: jordan9500 <jordan.jackson@ibm.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
2023-09-18 21:05:11 +02:00
stevenhorsman
0ffc9c02a2 agent: bump image-rs version
- Bump image-rs to stay aligned with guest-components version

Fixes: #7947
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-18 11:01:10 +01:00
stevenhorsman
560e21d8e3 versions: Bump guest-components
- Bump guest-components to pick up the new api-server-rest Makefile

Fixes: #7947
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-18 11:00:20 +01:00
Greg Kurz
87b8164add
Merge pull request #7951 from BbolroC/reduce-redundancy-se-image
cc|gha: remove build redundancy of kernel and cc-rootfs-initrd for s390x
2023-09-18 10:12:24 +02:00
Fabiano Fidêncio
6b3c63570e
Merge pull request #7981 from BbolroC/fix-agent-double-build-for-initrd
cc|osbuilder: Correct a typo in the initrd-image
2023-09-16 00:39:35 +02:00
Hyounggyu Choi
5e5e78cd6f cc|osbuilder: Correct a typo in the initrd-image
This PR is to prevent rootfs.sh from running twice by fixing the typo `initrd-image`.

Fixes: #7980

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-09-15 19:18:31 +02:00
Hyounggyu Choi
4f1d631af1 cc|gha: remove build redundancy of kernel and cc-rootfs-initrd for s390x
This PR is to remove the build redundancy of `kernel` and `cc-rootfs-initrd` by making `cc-se-image` built based on them at the second build stage.

Fixes: #7949

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-09-14 16:48:34 +02:00
stevenhorsman
4662a7a942 osbuilder: Update api-server-rest
- Switch api-server-rest to use the Makefile rather than
directly calling cargo for multi-platform support and decoupling

Fixes: #7947
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-14 11:24:54 +01:00
Fabiano Fidêncio
ee15a389de
Merge pull request #7688 from ChengyuZhu6/image_sharing_dmverity
CC | support dm-verity tarfs disk image in CoCo
2023-09-13 20:52:01 +02:00
Fabiano Fidêncio
b71443a7ae
Merge pull request #7878 from ChengyuZhu6/check_rust
CC | osbuilder: check rust enviornment before building agent and AA
2023-09-13 17:15:59 +02:00
ChengyuZhu6
a533c974f9 agent: enable verity feature in image-rs
update image-rs to support verity feature

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 18:38:51 +08:00
Fabiano Fidêncio
d0b69651ce
Merge pull request #7917 from stevenhorsman/cherry-pick-cgroups-fixes
agent: optimize the code of systemd cgroup manager
2023-09-13 09:53:46 +02:00
ChengyuZhu6
8f38dcb850 osbuilder: check rust enviornment before building agent and AA
We should configure the Rust environment when AGENT_SOURCE_BIN is empty or AA_KBC is not empty.

Fixes #7877

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 15:22:49 +08:00
Fabiano Fidêncio
f462c69484
Merge pull request #7880 from ChengyuZhu6/udev
image-builder: fix udev error when using docker to build image
2023-09-13 07:47:35 +02:00
ChengyuZhu6
622bd4e370 agent: create directories to mount filesystem by overlay
When creating a container with a raw disk image using virtio-blk,
the guest does not have the upper directory and worker directory present.
Therefore, it is necessary to create these directories before mounting the filesystem with overlay.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 10:01:12 +08:00
ChengyuZhu6
72c9f62b70 agent: introduce DmVerityHandler to support dm-verity volume
We utilize the KataVirtualVolume which storing the dm-verity info
and the path of disk image on the host supplied by snapshotter as an integral part of `CreateContainer`.
Within this process, we copy the verity info and the disk image path to mount slice to create a block device by virtio-blk.
Then storing the `lowerdir` in rootfs.storage which is the mountpoint of the verity path through `CreateContainerRequest`.
To maintain clarity and avoid any need for modification to the `VirtioBlkPciHandler`,we introduce the `DmVerityHandler`.
This dedicated handler is responsible for calling image-rs to create verity device and mount the device to the `lowerdir` within the guest environment.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 10:01:11 +08:00
ChengyuZhu6
fd33309475 image-builder: fix udev error when using docker to build image
Incorporate the `DM_VERITY` parameter when building the image with docker.

Fixes #7879

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 09:34:22 +08:00
ChengyuZhu6
e36c2b6249 runtime: support to create VirtualVolume rootfs storages
1) Creating storage for each `extraoption` in rootFs.Options,
and then aggregates all storages  into `containerStorages`.
2) Creating storage for other data volumes and push them into `volumeStorages`.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 09:30:30 +08:00
ChengyuZhu6
5ad3eba8b1 runtime: redefine and add functions to handle VirtualVolume to storage
1) Extract function `handleBlockVolume` to create Storage only.
2) Add functions to handle KataVirtualVolume device and construct
   corresponding storages.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 09:30:30 +08:00
ChengyuZhu6
29eb2c02d9 runtime: extend SharedFile to support mutiple storage devices
To enhance the construction and administration of `Katavirtualvolume` storages,
this commit expands the 'sharedFile' structure to manage both
rootfs storages(`containerStorages`) including `Katavirtualvolume` and other data volumes storages(`volumeStorages`).

NOTE: `volumeStorages` is intended for future extensions to support Kubernetes data volumes.
Currently, `KataVirtualVolume` is exclusively employed for container rootfs, hence only `containerStorages` is actively utilized.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 09:30:30 +08:00
ChengyuZhu6
bedd536461 runtime: add functions to create devices in KataVirtualVolume
The snapshotter will place `KataVirtualVolume` information
into 'rootfs.options' and commence with the prefix 'io.katacontainers.volume='.
The purpose of this commit is to transform the encapsulated KataVirtualVolume data into device information.

Fixes #7792

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Feng Wang <feng.wang@databricks.com>
Co-authored-by: Samuel Ortiz <sameo@linux.intel.com>
Co-authored-by: Wedson Almeida Filho <walmeida@microsoft.com>
2023-09-13 09:30:30 +08:00
ChengyuZhu6
d788d4af2f runtime: Add KataVirtualVolume struct in runtime
Add the corresponding data structure in the runtime part according to
kata-containers/kata-containers/pull/7698.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-09-13 09:30:30 +08:00
Steve Horsman
51c665a09c
Merge pull request #7918 from stevenhorsman/CCv0-bump-to-rust-1.72
CCv0 bump to rust 1.72
2023-09-12 20:20:40 +01:00
stevenhorsman
36431de30f versions: Bump rust version
Bump rust to 1.72.0 to test what extra warnings/issues we get

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
9ebb91f94c runk: Fix rust unecessary mut error
- Fix `error: variable does not need to be mutable`
in rust 1.72

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
f7fd2c1dfc kata-ctl: useless-vec warning
- Fix clippy::useless-vec warning

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
c0668ef7eb kata-ctl: Resolve non-minimal-cfg warning
- In rust 1.72, clippy warned clippy::non-minimal-cfg
as the cfg has only one condition, so doesn't
need to be wrapped in the any combinator.

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
0a33d27c30 agent-ctl: Allow clippy lint
- Allow `clippy::redundant-closure-call`
which has issues with the guard function passed into
the `run_if_auto_values` macro

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
48465d0547 runtime-rs: Fix useless-vec warning
Fix clippy::useless-vec warning

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
cb7cc1d708 runtime-rs: Remove mut
Fix `error: variable does not need to be mutable`

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
6e508ae322 dragonball: Allow ambiguous-glob-reexports
The bindgen generated code is triggering lots of
ambiguous-glob-reexports warnings in rust 1.70+

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
91916ed118 dragonball: Resolve non-minimal-cfg warning
- In rust 1.72, clippy warned clippy::non-minimal-cfg
as the cfg has only one condition, so doesn't
need to be wrapped in the all combinators.

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
d1d49675a0 agent: config: Allow clippy lint
- Allow `clippy::redundant-closure-call` in `from_cmdline`
which has issues with the guard function passed into
the `parse_cmdline_param` macro

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
3416e104b8 agent: config: Fix useles-vec warning
Fix clippy::useless-vec warning

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 15:16:50 +01:00
stevenhorsman
495b9825e5 libs: Fix clippy unnecesary hashes error
- Fix error: unnecessary hashes around raw string literal

Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 14:19:41 +01:00
stevenhorsman
da8d4a4584 agent: Vendor
run make vendor on agent to update

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-09-12 12:54:54 +01:00