Commit Graph

12405 Commits

Author SHA1 Message Date
Fabiano Fidêncio
57001431b4 versions: Bump to 6.4.14 stable kernel
This kernel update is needed in order to get the latest and greatest
commits related to EROFS, which will be used for allowing sharing the
container images between the guest and host for Confidential Containers
using the tarfs mode of EROFS.

We're removing a few options here, because:
* SECURITY_SELINUX_CHECKREQPROT_VALUE was deprecated as part of
  a7e4676e8e2c.
* CONFIG_IP_NF_TARGET_CLUSTERIP was removed as part of 9db5d918e2c0.
* CONFIG_NET_SCH_CBQ was removed as part of 051d44209842.

Fixes: #7845
Backports: #7846

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-09-06 11:03:31 +02:00
Biao Lu
b4092023bf osbuilder: add confidential-data-hub in rootfs
Fixes: #7544

Signed-off-by: Biao Lu <biao.lu@intel.com>
2023-09-06 10:57:34 +08:00
Biao Lu
acd0a75efd agent: rootfs: add sealed-secret in Makefile
When set SEALED_SECRET to "yes", the kata-agent is built with
sealed-secret capability, default value is "no".

Fixes: #7544

Signed-off-by: Biao Lu <biao.lu@intel.com>
2023-09-06 10:57:34 +08:00
Biao Lu
4e3a1ebcaf protocols: add support sealed_secret
To call CDH ttrpc API, 'unseal_secret' for 'sealed_secret', add
protocol file and generate ttrpc code.

Fixes: #7544

Signed-off-by: Biao Lu <biao.lu@intel.com>
2023-09-06 10:57:34 +08:00
Fabiano Fidêncio
83b020f4a3
Merge pull request #7826 from jiangliu/cherry2
CC | cherry-pick #7819 and #7821 from main branch
2023-09-03 00:33:56 +02:00
Jiang Liu
f45ee1fe1d agent: refine StorageDeviceGeneric::cleanup()
Refine StorageDeviceGeneric::cleanup() to improve safety.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-09-02 17:05:29 +08:00
Jiang Liu
1f4facdfe9 agent: implement StorageDeviceGeneric::cleanup()
Refactor cleanup_sandbox_storage as StorageDeviceGeneric::cleanup().

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-09-02 17:05:16 +08:00
Jiang Liu
d955f9dcf8 types: make StorageDevice::cleanup() return possible error code
Make StorageDevice::cleanup() return possible error code.

Fixes: #7818

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-09-02 17:04:55 +08:00
Jiang Liu
039fde2b66 agent: move StorageDeviceGeneric from kata-types into agent
Move StorageDeviceGeneric from kata-types into agent, so we can
refactor code later.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-09-02 17:04:36 +08:00
Jiang Liu
eac38d1a05 agent: avoid possible leakage of storage device
When a storage device is used by more than one container, the second
and forth instances will cause storage device reference count leakage,
thus cause storage device leakage. The reason is:
add_storages() will increase reference count of existing storage device,
but forget to add the device to the `mount_list` array, thus leak the
reference count.

Fixes: #7820

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-09-02 17:04:16 +08:00
Fabiano Fidêncio
bb4f0a9263
Merge pull request #7801 from fidencio/topic/error-out-if-attestation-agent-fails-to-build
CC | rootfs: Fail in case attestation-agent fails to build
2023-08-31 08:08:49 +02:00
Fabiano Fidêncio
af413550dd
Merge pull request #7797 from stevenhorsman/bump-guest-components-2d8dcd3
versions: Bump guest-components
2023-08-30 18:03:50 +02:00
Fabiano Fidêncio
badba8058c rootfs: Fail in case attestation-agent fails to build
Today I learned, I must say.

When running a basic script, such as:
```bash
 #/usr/bin/env bash

 set -o errexit
 set -o pipefail
 set -o errtrace

 cat junk && echo "hello"
 echo "didn't fail"

 cat junk
 echo "hello"
 echo "didn't fail"
```

One will get as a result:
```bash
cat: junk: No such file or directory
didn't fail
cat: junk: No such file or directory
```

Meaning that although there was an error on `cat junk && echo "hello"`,
and the `echo "hello"` part was not executed, an error was not reported
for that failure.

On the second part, though, it just breaks and returns an error as
expected.

Small scripts aside, this is exactly what was happening with the
attestation-agent, where a `make ... && make install ...` was being
called, make was failing but not actually breaking the script.

Let's change the logic and avoid such situations in the future, as it
caused our CI to be broken for quite some time without a simple way to
detect that line in the huge amount of logs left behind.

Here goes a reference to the documentation:
```
-e      Exit immediately if a pipeline (which may consist
        of a single simple command), a list, or a compound
        command (see SHELL GRAMMAR above), exits with a
        non-zero status.  The shell does not exit if the
        command that fails is part of the command list
        immediately following a while or until keyword,
        part of the test following the if or elif reserved
        words, part of any command executed in a && or ||
        list except the command following the final && or
        ||, any command in a pipeline but the last, or if
        the command's return value is being inverted with
        !.  If a compound command other than a subshell
        returns a non-zero status because a command failed
        while -e was being ignored, the shell does not
        exit.  A trap on ERR, if set, is executed before
        the shell exits.  This option applies to the shell
        environment and each subshell environment
        separately (see COMMAND EXECUTION ENVIRONMENT
        above), and may cause subshells to exit before
        executing all the commands in the subshell.

        If a compound command or shell function executes
        in a context where -e is being ignored, none of
        the commands executed within the compound command
        or function body will be affected by the -e
        setting, even if -e is set and a command returns a
        failure status.  If a compound command or shell
        function sets -e while executing in a context
        where -e is ignored, that setting will not have
        any effect until the compound command or the
        command containing the function call completes.
```

This comes from https://www.man7.org/linux/man-pages/man1/bash.1.html

Fixes: #7793

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-30 17:27:27 +02:00
stevenhorsman
c888facd24 versions: Bump guest-components
Bump image-rs and attestation-agent to use the latest guest-components
with the rust clap version fix

Fixes: #7580
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-08-30 14:33:35 +01:00
Jiang Liu
de1fe7bed0
Merge pull request #7767 from jiangliu/pick-7602-2
CC | cherry pick #7602 from main into CCv0
2023-08-29 01:39:32 +08:00
Jiang Liu
412e8554f3 agent: simplify storage device by removing StorageDeviceObject
Simplify storage device implementation by removing StorageDeviceObject.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:20 +08:00
Jiang Liu
d5483aaf7c agent: move storage device related code into dedicated files
Move storage device related code into dedicated files.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:20 +08:00
Jiang Liu
3b1af40b16 agent: refine storage related code a bit
Refine storage related code by:
- remove the STORAGE_HANDLER_LIST
- define type alias
- move code near to its caller

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:19 +08:00
Jiang Liu
331e35bc1a agent: switch to new storage subsystem
Switch to new storage subsystem to create a StorageDevice for each
storage object.

Fixes: #7614

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:19 +08:00
Jiang Liu
aee71b16f1 kata-types: introduce StorageDevice and StorageHandlerManager
Introduce StorageDevice and StorageHandlerManager, which will be used
to refine storage device management for kata-agent.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:19 +08:00
Jiang Liu
0d5a9eaeff agent: simplify the way to manage storage object
Simplify the way to manage storage objects, and introduce
StorageStateCommon structures for coming extensions.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:19 +08:00
Jiang Liu
6dcd164c4e sys-util: support more mount flags in parse_mount_options()
Support more mount flags in parse_mount_options().

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:19 +08:00
Jiang Liu
cf15777edd agent: use create_mount_destination() from kata-sys-util
Use create_mount_destination() from kata-sys-util crate to reduce
redundant code.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:19 +08:00
Jiang Liu
7f12e27a68 types: add more mount related constants
Add more mount related constants.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-28 09:53:19 +08:00
Jiang Liu
67748bde6c
Merge pull request #7763 from jiangliu/pick-7602
CC | cherry-pick #7602 from main branch
2023-08-28 09:50:29 +08:00
Jiang Liu
b6218beef6 agent: use function from kata-sys-utils to reduce code
Use function get_linux_mount_info() from kata-sys-util crate to share
common code.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-25 22:23:03 +08:00
Fabiano Fidêncio
dfb38245e7
Merge pull request #7749 from fidencio/topic/CC-backport-fix-to-allow-building-the-images
CC | Backport | local-build: Remove GID before creating group
2023-08-24 14:23:18 +02:00
Jeremi Piotrowski
4417641803 local-build: Remove GID before creating group
docker install now creates a group with gid 999 which happens to match what we
need to get docker-in-docker to work. Remove the group first as we don't need
it.

Fixes: #7726
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
(cherry picked from commit 3b881fbc0e)
2023-08-24 14:17:58 +02:00
Fabiano Fidêncio
9d4ec379b1
Merge pull request #7748 from fidencio/topic/CC-kata-deploy-dont-try-to-remove-opt-kata
CC | backport | kata-deploy: Don't try to remove /opt/kata
2023-08-24 14:15:25 +02:00
Fabiano Fidêncio
a7f01b4456 kata-deploy: Don't try to remove /opt/kata
The directory is a host path mount and cannot be removed from within the
container.  What we actually want to remove is whatever is inside that
directory.

This may raise errors like:
```
rm: cannot remove '/opt/kata/': Device or resource busy
```

Fixes: #7746

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-08-24 14:01:45 +02:00
Jiang Liu
bb644ee3ed
Merge pull request #7744 from jiangliu/pick-7698
CC | cherry-pick PR #7698 into CCv0
2023-08-24 15:31:42 +08:00
Fabiano Fidêncio
aef93c7aaf
Merge pull request #7576 from surajssd/update-aa-imagers-tdshim
Update AA, image-rs td-shim
2023-08-24 08:38:29 +02:00
Jiang Liu
15d1b2431c kata-types: implement serde methods for KataVirtualVolume
Implement serilization/deserialization methods for KataVirtualVolume.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-24 13:02:56 +08:00
Jiang Liu
61340c3d63 kata-types: validate KataVirtualVolume object
Implement method validate() for KataVirtualVolume to validate message
format.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-24 13:02:42 +08:00
Jiang Liu
6d07df4b15 kata-types: implement two conversion helpers for KataVirtualVolume
Enable conversions from NydusExtraOptions/DirectVolumeMountInfo to
KataVirtualVolume.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-24 13:02:28 +08:00
Jiang Liu
7e553e6707 kata-types: introduce KataVirtualVolume
Introduce structure KataVirtualVolume to to encapsulate information
for extra mount options and direct volumes, so we could build a common
infrastructure to handle these cases.

Fixes: #7699

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-24 13:02:11 +08:00
Fabiano Fidêncio
edf51c83c0
Merge pull request #7739 from fidencio/topic/CC-fix-uninstall-issues
CC | backport | kata-deploy: Avoid failing on content removal
2023-08-24 00:16:06 +02:00
Fabiano Fidêncio
b64891c5f5 kata-deploy: Avoid failing on content removal
We can simply use `rm -f` all over the place and avoid the container
returning any error.

Fixes: #7733

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 5cba38c175)
2023-08-23 20:05:01 +02:00
Steve Horsman
0e9a8f22ca
Merge pull request #7601 from ChengyuZhu6/install_dmsetup
CC | tools: Install dependencies with dm-verity in rootfs
2023-08-23 17:24:43 +01:00
Suraj Deshmukh
d8953498c6
CCv0: Update image-rs, AA and td-shim
- image-rs & AA: 3d8192f8d3efab041916ea4d60e32248ac6ec43d
- td-shim: 35c8ec33311877f0711412fd34cee929ae57e80e

Fixes: #7580

Signed-off-by: Suraj Deshmukh <suraj.deshmukh@microsoft.com>
2023-08-23 14:34:07 +00:00
Jiang Liu
cfba372f17
Merge pull request #7635 from jiangliu/image-service-singleton
CC | move image service related code into image-rpc.rs
2023-08-21 22:01:46 +08:00
Jiang Liu
f218a3104e agent/image: move image service related code into image-rpc.rs
Move image service related code into image-rpc.rs, to simplify
maintenance.

Fixes: #7633

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Co-authored-by: wllenyj <wllenyj@linux.alibaba.com>
Co-authored-by: jordan9500 <jordan.jackson@ibm.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
2023-08-21 19:29:09 +08:00
Jiang Liu
81980388d4 agent/image: export the image service singleton instance
Export the image service singleton instance.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-21 19:28:44 +08:00
Jiang Liu
624d3c063a agent/image: syntax only change to image service implementation
Syntax only change to image service implementation.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-08-21 19:28:32 +08:00
Fabiano Fidêncio
2a084ecbef
Merge pull request #7682 from sprt/backport-ci-fixes
CC | kata-deploy: Properly create default runtime class
2023-08-18 07:47:39 +02:00
Aurélien Bombo
bc685665c6 tests: k8s: Call ensure_yq() in setup.sh
It wasn't the `common.bash` import in `run_kubernetes_tests.sh` causing
the yq error so let's try this instead.

Reference: https://github.com/kata-containers/kata-containers/actions/runs/5674941359/job/15379797568#step:10:341

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-08-17 10:28:58 -07:00
Aurélien Bombo
723c44a7c4 kata-deploy: Properly create default runtime class
The default `kata` runtime class would get created with the `kata`
handler instead of `kata-$KATA_HYPERVISOR`. This made Kata use the wrong
hypervisor and broke CI.

Fixes: #7681

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-08-17 10:28:58 -07:00
ChengyuZhu6
d053f848b4 tools: Install the dependencies with dm-verity
Fixes #7636

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-08-16 21:47:52 +08:00
Suraj Deshmukh
32d347aa25
tools/static-checks: Install clang
Without this library the builds are failing with the following error:

```
...
error: failed to run custom build command for `devicemapper-sys v0.1.5`

Caused by: process didn't exit successfully:
    `/kata-containers/src/agent/target/release/build/devicemapper-sys-d8eae524a127e049/build-script-build`
    (exit status: 101) --- stderr thread 'main' panicked at 'Unable to
    find libclang: "couldn't find any valid shared libraries matching:
    ['libclang.so', 'libclang-*.so', 'libclang.so.*', 'libclang-*.so.*'],
    set the `LIBCLANG_PATH` environment variable to a path where one of
    these files can be found (invalid: [])"',
    /root/.cargo/registry/src/github.com-1ecc6299db9ec823/bindgen-0.63.0/./lib.rs:2338:31
```

Fixes: #7580

Signed-off-by: Suraj Deshmukh <suraj.deshmukh@microsoft.com>
2023-08-16 13:12:49 +00:00
Suraj Deshmukh
1ec85d7485
static-checks: Install devmapper libraries
After image-rs added the image-block-device integrity check using
dm-verity a new dependency is now needed, so install that.

Refer the following PR for more information:
https://github.com/confidential-containers/guest-components/pull/270

Fixes: #7580

Signed-off-by: Suraj Deshmukh <suraj.deshmukh@microsoft.com>
2023-08-16 13:12:49 +00:00