Commit Graph

12991 Commits

Author SHA1 Message Date
yaoyinnan
9aa1ed805a runtime: add SingleContainer when obtaining OCI Spec
When creating a cgroup, add a SingleContainer when obtaining the OCI Spec to apply to ctr, podman, etc.

Fixes: #5240

Signed-off-by: yaoyinnan <35447132+yaoyinnan@users.noreply.github.com>
2024-01-31 15:24:07 +08:00
yaoyinnan
b0b8523cea runtime: modify ValidCgroupPath unit test
Modify ValidCgroupPath unit test.

Fixes: #8930

Signed-off-by: yaoyinnan <35447132+yaoyinnan@users.noreply.github.com>
2024-01-31 14:37:17 +08:00
yaoyinnan
feed5c8ff9 runtime: merged ValidCgroupPath method
Merged ValidCgroupPath method to handle cgroupv1 and cgroupv2.

Fixes: #8930

Signed-off-by: yaoyinnan <35447132+yaoyinnan@users.noreply.github.com>
2024-01-31 14:37:13 +08:00
yaoyinnan
864389c524 runtime-rs: report error on missing or empty fields in configuration
Removed the setting of default values for runtime fields. Added explicit checks for missing or empty fields, reporting errors with clear messages.

Fixes: #8838

Signed-off-by: yaoyinnan <35447132+yaoyinnan@users.noreply.github.com>
2024-01-31 12:46:17 +08:00
Wainer dos Santos Moschetta
abc2fcd88f kata-deploy: fix deprecations on kustomization files
By running `kustomize edit fix` on those files they have changed
deprecated instructions ('bases' and 'patchesStrategicMerge') as well as
'apiVersion' and 'kind' were added.

Fixes #8268
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-01-30 18:41:03 -03:00
Lukáš Doktor
4876eadd2f
tools: Add reference to the kata webhook's README
The newly added webhook is a new component and oughst to be linked from
the main README file.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-01-30 19:05:56 +01:00
Lukáš Doktor
b0b7748f30
ci/openshift-ci: Correct the lib location
correct the lib file locations after the move from
tests->kata-containers repo and add a minimized version of the
".ci/lib.sh" library into the "ci/openshift-ci" as we don't really
utilize all of the features.

Fixes: #8653

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-01-30 19:05:56 +01:00
Lukáš Doktor
4c58478536
ci/openshift-ci: Move openshift-ci from the tests repo
Move the f15be37d9bef58a0128bcba006f8abb3ea13e8da version of scripts
required for openshift-ci from "kata-containers/tests/.ci/openshift-ci"
into "kata-containers/kata-containers/ci/openshift-ci" and required
webhook+libs into "kata-containers/kata-containers/tools/testing" as is
to simplify verification, the different location handling will be added
in following commit.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-01-30 19:05:55 +01:00
Kvlil
3fd5628771 dragonball: fix noop-method-call warning
The `noop-method-call` is a rustc lint that has existed since v1.52.0.
This lint has been moved to the warn by default lint level since v1.73.0.
Therefore build is failing with this version and above.
This commit removes the unnecessary call to `<&T as Deref>::deref` on `T: !Deref`.

Fixes: #8586

Signed-off-by: Kvlil <kalil.pelissier@gmail.com>
2024-01-30 17:16:49 +00:00
Wainer Moschetta
bf54a02e16
Merge pull request #8924 from microsoft/danmihai1/pod-nested-configmap-secret
genpolicy: fix ConfigMap volume mount paths
2024-01-30 14:09:41 -03:00
Gabriela Cervantes
78b517ccc8 tests: Re-arranged nerdctl tests
This PR re-arranged the nerdctl tests to avoid random failures.
In this PR first will run the tests with RunC and then with the kata hypervisor.
This PR tries to avoid the random failures that is happening with cloud-hypervisor
and clh.

Fixes #8963

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-01-30 16:07:12 +00:00
Dan Mihai
d12875ee66 genpolicy: ignore volume configMap optional field
The auto-generated Policy already allows these volumes to be mounted,
regardless if they are:
- Present, or
- Missing and optional

Fixes: #8893

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-01-30 15:32:37 +00:00
Fabiano Fidêncio
7a83e6dc14
Merge pull request #8959 from fidencio/topic/crio-bump-runners-to-2204
gha: cri-o: Bump runners to 22.04
2024-01-30 14:27:40 +01:00
Fabiano Fidêncio
34d51b05f8
gha: cri-o: Bump runners to 22.04
This will *not* solve the CRI-O CI breakage but will give us an
environment where we could get it to run locally.

Fixes: #8935 -- part I

Thanks to Julien Ropé for trying to reproduce the issues I faced on
https://github.com/kata-containers/kata-containers/issues/8935 in an
Ubuntu 22.04 system.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-30 14:17:06 +01:00
Xuewei Niu
7e10000b6f
Merge pull request #8928 from yaoyinnan/8927/fix/unused-DriverInfo
runtime-rs: fix unused driverInfo error
2024-01-30 20:39:10 +08:00
Hyounggyu Choi
f3bc6e4155 packaging: Use Ubuntu 20.04 for building an agent
This involves using Ubuntu 20.04 as a build environment for an agent to match with a runtime environment.

Fixes: #8955

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-01-30 10:22:14 +01:00
Pavel Mores
d53edbd0a5 runtime-rs: collect qemu stderr and log it in shim log
Qemu stderr monitoring runs in its own asynchronous green thread.
For that, `stderr` is taken out of the Child representing the qemu child
process to avoid partial move and make it possible for the main thread
still to call functions on QemuInner::qemu_process (e.g. kill(), id()).

Fixes #8937

Signed-off-by: Pavel Mores <pmores@redhat.com>
2024-01-30 09:09:05 +01:00
Pavel Mores
684d740122 runtime-rs: switch qemu child process management from std to tokio
We'll want to capture qemu's stderr in parallel with normal runtime-rs
execution.  Tokio's primitives make this much easier than std's.  This
also makes child process management more consistent across runtime-rs
(i.e. virtiofsd child process is already launched and managed using tokio).

Some changes were necessary due to tokio functions being slightly different
from their std counterparts.  Child::kill() is now async and Child::id()
now returns an Option.

Signed-off-by: Pavel Mores <pmores@redhat.com>
2024-01-30 09:07:14 +01:00
Dan Mihai
6a8f46f3b8
Merge pull request #8918 from microsoft/danmihai1/metadata
genpolicy: optional PodTemplateSpec metadata field
2024-01-29 12:36:30 -08:00
Dan Mihai
60ac3048e9 genpolicy: fix ConfigMap volume mount paths
Allow Kata CI's pod-nested-configmap-secret.yaml to work with
genpolicy and current cbl-mariner images:

1. Ignore the optional type field of Secret input YAML files.

   It's possible that CoCo will need a more sophisticated Policy
   for Secrets, but this change at least unblocks CI testing for
   already-existing genpolicy features.

2. Adapt the value of the settings field below to fit current CI
   images for testing on cbl-mariner Hosts:

    "kata_config": {
        "confidential_guest": false
    },

    Switching this value from true to false instructs genpolicy to
    expect ConfigMap volume mounts similar to:

        "configMap": {
            "mount_type": "bind",
            "mount_source": "$(sfprefix)",
            "mount_point": "^$(cpath)/watchable/$(bundle-id)-[a-z0-9]{16}-",
            "driver": "watchable-bind",
            "fstype": "bind",
            "options": [
                "rbind",
                "rprivate",
                "ro"
            ]
        },

    instead of:

        "confidential_configMap": {
            "mount_type": "bind",
            "mount_source": "$(sfprefix)",
            "mount_point": "$(sfprefix)",
            "driver": "local",
            "fstype": "bind",
            "options": [
                "rbind",
                "rprivate",
                "ro"
            ]
        }
    },

    This settings change unblocks CI testing for ConfigMaps.

Simple sanity testing for these changes:

genpolicy -u -y pod-nested-configmap-secret.yaml

kubectl apply -f pod-nested-configmap-secret.yaml

kubectl get pods | grep config
nested-configmap-secret-pod 1/1     Running   0          26s

Fixes: #8892

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-01-29 16:13:47 +00:00
Gabriela Cervantes
31813cf8d8 metrics: Update packages for TensorFlow ResNet Int8 Dockerfile
This PR updates the required packages for the TensorFlow ResNet50
Int8 Dockerfile.

Fixes #8950

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-01-29 16:11:09 +00:00
Fabiano Fidêncio
087856f26c
Merge pull request #8934 from microsoft/danmihai1/nodeName
genpolicy: ignore the nodeName field
2024-01-29 16:57:59 +01:00
Greg Kurz
d687b601f1
Merge pull request #8933 from fidencio/topic/package-coco-guest-components
packaging: Build coco-guest-components
2024-01-29 16:34:06 +01:00
Zvonko Kaiser
a9348fa35b
Merge pull request #8375 from zvonkok/opa-binary-fix
arm64: agent_policy build always pulls amd64 opa binary
2024-01-29 15:10:10 +01:00
Fabiano Fidêncio
5ea6a29c37
Merge pull request #8947 from fidencio/topic/gha-pass-down-AZ_SUBSCRIPTION_ID
gha: azure: Set the correct subscription to the account
2024-01-29 15:07:06 +01:00
Fabiano Fidêncio
448c0aaecb
gha: azure: Set the correct subscription to the account
Due to the changes done in the CI, we need to set the correct
subscription to be used with the account from now on, otherwise we'd end
up using CoCo subscription.

Fixes: #8946

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-29 15:00:38 +01:00
Pavel Mores
b52a398469 runtime-rs: move creation of VM path from start_vm() to prepare_vm()
This fixes a flaw pointed out in review of PR #8185.  Creation of the
directory semantically fits better into VM preparation than VM launch.

Signed-off-by: Pavel Mores <pmores@redhat.com>
2024-01-27 13:46:35 +01:00
Fabiano Fidêncio
98dc2d4c52
rootfs: agent: Initialise AGENT_SOURCE_BIN & AGENT_TARBALL
Otherwise those would be unbound if not passed.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-26 19:58:41 +01:00
Fabiano Fidêncio
5e57e0235e
rootfs: agent: Fix build with AGENT_SOURCE_BIN
We need to actually check that the env var is not empty. :-)
This was introduced by 8307718842.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-26 19:58:20 +01:00
Fabiano Fidêncio
fbfc880eb6
rootfs: Add COCO_GUEST_COMPONENTS_TARBALL env var
This env ver will serve us to pass the Confidential Containers
guest-components tarball to the rootfs builder, which will then just
unpack the content into the rootfs.

Fixes: #8848 -- part I

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Linda Yu <linda.yu@intel.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: Jakob Naucke <jakob.naucke@ibm.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
Co-authored-by: zhouliang121 <liang.a.zhou@linux.alibaba.com>
Co-authored-by: Alex Carter <alex.carter@ibm.com>
Co-authored-by: Suraj Deshmukh <suraj.deshmukh@microsoft.com>
Co-authored-by: Xynnn007 <xynnn@linux.alibaba.com>
2024-01-26 19:58:19 +01:00
Fabiano Fidêncio
644abde35c
packaging: coco-guest-components: Allow building the project
The Confidential Containers guest-components will, in the very short
future, be part of the Kata Containers rootfs that's used by the
Confidential Containers usecase.

This commit introduces the ability to, standalone, build the component
locally and as part of our CI, and this can be done by calling:
`make coco-guest-components-tarball`

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Linda Yu <linda.yu@intel.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: Jakob Naucke <jakob.naucke@ibm.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
Co-authored-by: zhouliang121 <liang.a.zhou@linux.alibaba.com>
Co-authored-by: Alex Carter <alex.carter@ibm.com>
Co-authored-by: Suraj Deshmukh <suraj.deshmukh@microsoft.com>
Co-authored-by: Xynnn007 <xynnn@linux.alibaba.com>
2024-01-26 19:36:01 +01:00
Hyounggyu Choi
ee072e8a06
Merge pull request #8926 from fidencio/topic/cache-the-agent-for-non-x86_64
gha: Cache the agent for non-x86_64 arches
2024-01-26 18:04:33 +01:00
Dan Mihai
076869aa39 genpolicy: ignore the nodeName field
Validating the node name is currently outside the scope of the CoCo
policy.

This change unblocks testing using Kata CI's test-pod-file-volume.yaml
and pv-pod.yaml.

Fixes: #8888

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-01-26 16:30:55 +00:00
Dan Mihai
ef1ee81f81
Merge pull request #8909 from microsoft/danmihai1/main-shareProcessNamespace
genpolicy: add shareProcessNamespace support
2024-01-26 05:49:19 -08:00
yaoyinnan
9b7c5c69cf runtime-rs: fix unused driverInfo error
Remove the unused DriverInfo declaration or integrate it into the codebase where applicable.

Fixes: #8927
Signed-off-by: yaoyinnan <35447132+yaoyinnan@users.noreply.github.com>
2024-01-26 19:59:52 +08:00
Greg Kurz
f41fa7557a
Merge pull request #8914 from BbolroC/basic-e2e-ibm-se
tests: Add IBM SE to the basic confidential test
2024-01-26 12:32:32 +01:00
Fabiano Fidêncio
08a082ca47
gha: Cache the agent for non-x86_64 arches
Those are not yet being cached for no reason, and they better be as
it'll allow us to save a considerable amount of time building the
rootfs.

Fixes: #8917

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-26 12:02:26 +01:00
Fabiano Fidêncio
a7c68225aa
Merge pull request #8916 from fidencio/topic/packaging-reuse-already-built-agent
packaging:  Don't always build the kata-agent
2024-01-26 12:00:55 +01:00
Fabiano Fidêncio
95c569b0a6
packaging: Add safe.directory to the git config
Otherwise building as root will not work, as demonstrated by the arm64
CI.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-26 09:44:43 +01:00
Hyounggyu Choi
ab462a4b89 tests: Add IBM SE to the basic confidential test
The existing confidential basic test titled `Test unencrypted
confidential container launch success and verify that we are
running in a secure enclave` has been updated to incorporate
IBM Secure Execution (`qemu-se`).
Previously, a secure image was absent from kata-deploy, hindering
the inclusion of IBM SE in the test.
Thanks to the #6755 update, it is now possible to test the TEE.

This modification extends the existing test by introducing
`qemu-se`. The specific changes are outlined below:

- Add an additional test `cc-se-e2e-tests` to s390x nightly
- Expansion of `REMOTE_COMMAND_PER_HYPERVISOR` for `qemu-se`
- Temporary exclusion of two test cases currently incompatible with IBM SE
(`cpu-ns` is a common issue across all TEEs, while `inotify`
will be addressed in a subsequent pull request).

Fixes: #8913

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-01-26 06:04:39 +01:00
GabyCT
c13a63c8ba
Merge pull request #8905 from zvonkok/enable-tpm
qemu: enable TPM
2024-01-25 14:52:00 -06:00
GabyCT
aa958adf90
Merge pull request #8904 from GabyCT/topic/buildbq
tools: Use defined variable in build base qemu script
2024-01-25 13:51:44 -06:00
GabyCT
36fc2fd83f
Merge pull request #8876 from GabyCT/topic/dockerrestfp
metrics: Update packages needed for ResNet50 FP32 Dockerfile
2024-01-25 13:51:16 -06:00
Dan Mihai
8ad5459beb genpolicy: optional PodTemplateSpec metadata field
Add metadata containing the Policy annotation if the user didn't
provide any metadata in the input yaml file.

For a simple sanity test using a Kata CI YAML file:

genpolicy -u -y job.yaml

kubectl apply -f job.yaml

kubectl get pods | grep job
job-pi-test-64dxs 0/1     Completed   0          14s

Fixes: #8891

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-01-25 19:06:59 +00:00
Fabiano Fidêncio
dd49479829
packaging: Don't build the agent if not needed
Let's start relying on the already cached agent to be deployed inside
the rootfs.  By doing this we save a lot of time in our CI, and we have
a better way, for developers, to play with changes in the agent.

Fixes: #8915

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-25 19:41:33 +01:00
Fabiano Fidêncio
21fd7e6dfd
packaging: Fail in case oras can't find an artefact
It just means the component is not cached, and that it must be built in
the usual way.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-25 19:41:32 +01:00
Fabiano Fidêncio
eb7a33ee71
rootfs: Always strip the agent binary
Let's always do this, regardless of where the agent is coming from.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-25 19:41:32 +01:00
Fabiano Fidêncio
f23451de01
rootfs: Add xz as a dep
As we'll be untarring the agent tarball (and any other component that
may be part of the rootfs) into the rootfs, we have to have xz
installed.

For debian and ubuntu the package is called xz-utils; for centos,
alpine and cbl-mariner the package is called xz.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-25 19:41:32 +01:00
Fabiano Fidêncio
8307718842
rootfs: Add AGENT_TARBALL env var
This env var will serve us to pass the agent tarball to the rootfs
builder, which will then just unpack the content into the rootfs instead
of building the agent again.

AGENT_TARBALL and AGENT_SOURCE_BIN should never be used together.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-25 19:41:32 +01:00
Fabiano Fidêncio
5b0d0687e5
packaging: agent: Allow building in all arches
We're moving away from alpine and using ubuntu in order to be able to
build the agent for all the architectures we need.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-25 19:41:32 +01:00