Commit Graph

13147 Commits

Author SHA1 Message Date
ChengyuZhu6
965da9bc9b
runtime: support to pass image information to guest by KataVirtualVolume
support to pass image information to guest by KataVirtualVolumeImageGuestPullType
in KataVirtualVolume, which will be used to pull image on the guest.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-03-19 17:22:36 +01:00
ChengyuZhu6
cfd14784a0
agent: Introduce ImagePullHandler to support IMAGE_GUEST_PULL volume
As we do not employ a forked containerd in confidential-containers, we utilize the KataVirtualVolume
which storing the image information as an integral part of `CreateContainer`.
Within this process, we store the image information in rootfs.storage and pass this image url through `CreateContainerRequest`.
This approach distinguishes itself from the use of `PullImageRequest`, as rootfs.storage is already set and initialized at this stage.
To maintain clarity and avoid any need for modification to the `OverlayfsHandler`,we introduce the `ImagePullHandler`.
This dedicated handler is responsible for orchestrating the image-pulling logic within the guest environment.
This logic encompasses tasks such as calling the image-rs to download and unpack the image into `/run/kata-containers/{container_id}/images`,
followed by a bind mount to `/run/kata-containers/{container_id}`.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-03-19 17:22:36 +01:00
ChengyuZhu6
462051b067
agent/image: merge container spec for images pulled inside guest
When being passed an image name through a container annotation,
merge its corresponding bundle OCI specification and process into the passed container creation one.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Arron Wang <arron.wang@intel.com>
Co-authored-by: Jiang Liu <gerry@linux.alibaba.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: wllenyj <wllenyj@linux.alibaba.com>
Co-authored-by: jordan9500 <jordan.jackson@ibm.com>
2024-03-19 17:22:36 +01:00
ChengyuZhu6
cec1916196
agent: Support https_proxy/no_proxy config for image download in guest
Containerd can support set a proxy when downloading images with a environment variable.
For CC stack, image download is offload to the kata agent, we need support similar feature.
Current we add https_proxy and no_proxy, http_proxy is not added since it is insecure.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Arron Wang <arron.wang@intel.com>
2024-03-19 17:22:36 +01:00
ChengyuZhu6
9cddd5813c
agent/image: Enable image-rs crate to pull image inside guest
With image-rs pull_image API, the downloaded container image layers
will store at IMAGE_RS_WORK_DIR, and generated bundle dir with rootfs
and config.json will be saved under CONTAINER_BASE/cid directory.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Arron Wang <arron.wang@intel.com>
Co-authored-by: Jiang Liu <gerry@linux.alibaba.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: wllenyj <wllenyj@linux.alibaba.com>
2024-03-19 17:22:36 +01:00
ChengyuZhu6
2b3a00f848
agent: export the image service singleton instance
Export the image service singleton instance.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Jiang Liu <gerry@linux.alibaba.com>
Co-authored-by: Xynnn007 <xynnn@linux.alibaba.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: wllenyj <wllenyj@linux.alibaba.com>
2024-03-19 17:22:36 +01:00
ChengyuZhu6
1f1ca6187d
agent: Introduce ImageService
Introduce structure ImageService, which will be used to pull images
inside the guest.

Fixes: #8103

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
co-authored-by: wllenyj <wllenyj@linux.alibaba.com>
co-authored-by: stevenhorsman <steven@uk.ibm.com>
2024-03-19 17:22:33 +01:00
Chelsea Mafrica
42dfe0e8d1
Merge pull request #9286 from jodh-intel/agent-show-enabled-features
agent: Show features enabled at build time
2024-03-19 08:54:49 -07:00
Wainer Moschetta
e6501aa4ad
Merge pull request #9229 from ldoktor/ocp-ci
ocp.ci: Various fixes and improvements to the OCP pipeline
2024-03-19 11:13:01 -03:00
James O. D. Hunt
46aec0f15a
Merge pull request #9293 from jodh-intel/kata-manager-fix-containerd-for-docker
kata-manager: Fix Docker install
2024-03-19 10:06:44 +00:00
Fabiano Fidêncio
e0a6b6449f
Merge pull request #9302 from BbolroC/fix-permission-issue-on-s390x-runners
gha: Place pre-action on s390x runner for kata-deploy during release
2024-03-19 10:42:23 +01:00
Hyounggyu Choi
f2bc819644 gha: Place pre-action on s390x runner for kata-deploy during release
This is to place a pre-action step for the kata-deploy job in order to
clean up the github workspace directory before checking out the repo.

Fixes: #9301

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-03-19 10:18:38 +01:00
Alex Lyn
7af2df408e
Merge pull request #9295 from likebreath/0318/fix_clh_default_netconfig
runtime-rs: ch: Provide valid default value for NetConfig
2024-03-19 15:17:18 +08:00
Xuewei Niu
99d0e5fff8
Merge pull request #9270 from zvonkok/kata-agent-bind-mount
kata-agent: optional bind flag
2024-03-19 10:39:23 +08:00
Bo Chen
ad4262e86b runtime-rs: ch: Provide valid default value for NetConfig
The current default value of IP `0.0.0.0` with mask `0.0.0.0` will cause
ioctl error when being used to create and configure TAP device, with
newer version of Cloud Hypervisor [1]. This patch replaces them with
valid value that are the same as the Go-lang runtime [2].

[1] https://github.com/cloud-hypervisor/cloud-hypervisor/pull/5924
[2] e3f7852738/src/runtime/virtcontainers/pkg/cloud-hypervisor/client/model_net_config.go (L40-L57)

Fixes: #9254

Signed-off-by: Bo Chen <chen.bo@intel.com>
2024-03-18 15:47:58 -07:00
Fabiano Fidêncio
e3f7852738
Merge pull request #9289 from fidencio/topic/releases-follow-up-IV
releases: Simply the release in order to avoid pushing a commit updating the VERSION file
2024-03-18 17:38:58 +01:00
James O. D. Hunt
a6c3f75872 kata-manager: Fix Docker install
Fix the Docker install by removing the second (erroneous) call to
`containerd_installed()` in `handle_docker()`.

Without this fix, installing using Docker (`-D`) will work *iff* you
already have containerd installed. However, if you do not have
containerd installed, the `containerd_installed()` function returns 1,
which exits the script as we're running with `set -e`, leaving a broken
Docker installation.

> **Note:** containerd is installed via Docker's `get-docker.sh` script.

Fixes: #9292.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2024-03-18 14:08:35 +00:00
stevenhorsman
0ab8e61a64 release: Remove release type from arch release
Now we don't have minor and major releases and
we are now generating a new version
in the release workflow, we can
tidy up the arch specific releases workflows to remove
the extra required inputs

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-03-18 12:27:57 +00:00
Greg Kurz
3cfc1b6ba7 releases: Adjust documentation to the new workflow
This drops the documentation of the legacy release scripts and adds
a quick description of the scripts of the new workflow. It also
highlights the bump of the `VERSION` file.

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-03-18 12:57:02 +01:00
Greg Kurz
76c640767e releases: Drop Makefile
It isn't used anymore.

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-03-18 12:54:00 +01:00
Greg Kurz
bfe19e68e8 kata-deploy: Adapt test-kata.sh to the new release workflow
All releases are now created in the `main` branch following
the very same workflow. No need to special case pre-releases.

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-03-18 12:54:00 +01:00
Fabiano Fidêncio
12578f11bc
releases: Assume VERSION has the correct version to be released
This is done in order to avoid having to push a commit to the main
branch, which is against the defined rules on GitHub.

By doing this, we need to educate ourselves to always bump the VERSION
file as soon as a release is cut out.

As a side effect of this change, we can drop the release-major and
release-minor workflows, as those are not needed anymore.

Fixes: #9064 - part IV

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-03-16 13:30:58 +01:00
Fabiano Fidêncio
8ce50269fe
release: Bump the VERSION file to the next release number
3.3.0 it will be.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-03-16 13:21:27 +01:00
Xuewei Niu
9f512c016e
Merge pull request #9282 from gkurz/runtime-rs-fds-for-qemu
runtime-rs: Consolidate the handling of fds passed to QEMU
2024-03-16 10:26:11 +08:00
Greg Kurz
1e526a4769 runtime-rs: Consolidate the handling of fds passed to QEMU
File descriptors that are passed to QEMU need some special care.
We want them to be closed when the QEMU process is started. But
at the same time, it is required that the associated rust File
structures, either coming from the` std::fs` or the `tokio::fs`
crates, are still in scope when the QEMU process is forked. This
is currently achieved by keeping File structures in variables
at the outer scope of `start_vm()`. This scheme is currently
duplicated, with similar justifications in the corresponding
comments.

Consolidate all this handling in one place with a more generic
explanation.

Fixes #9281

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-03-15 16:14:59 +01:00
James O. D. Hunt
9ef59488d9 agent: Show features enabled at build time
The agent now has a number of optional build-time features that can be
enabled.

Add details of these features to the following areas:

- Version output (`kata-agent --version`)
- Announce message (so that the details are always added to the journal
  at agent startup).
- The response message returned by the ttRPC `GetGuestDetails()` API.

Fixes: #9285.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2024-03-15 13:29:21 +00:00
Chelsea Mafrica
2c50d3c393
Merge pull request #9278 from wainersm/github_env_fix
tests: fix nounset error with $GITHUB_ENV
2024-03-14 16:39:13 -07:00
Greg Kurz
6a112cc7a5 runtime-rs: Fix missing dependency
Some previous contribution missed to run cargo clippy.
Fix the dependency now so that it doesn't cause noise
in future contributions.

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-03-14 23:19:38 +01:00
Dan Mihai
b3b00e00a6
Merge pull request #9246 from microsoft/danmihai/default-env
genpolicy: default env if image doesn't have env
2024-03-14 11:01:43 -07:00
Dan Mihai
6094f1e31d
Merge pull request #9250 from microsoft/danmihai1/k8s-pid-ns2
tests: k8s: k8s-pid-ns.bats auto-generated policy
2024-03-14 10:10:24 -07:00
Zvonko Kaiser
c15e19c806 kata-agent: optional bind flag
Fixes: #9269

From https://github.com/opencontainers/runtime-spec/blob/main/config.md#mounts
type (string, OPTIONAL) The type of the filesystem to be mounted.
bind may be only specified in the oci spec options -> flags update r#type
The agent will ignore bind mounts if they are only specified in the OCI spec options and not in the flags.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-03-14 14:42:01 +00:00
Wainer dos Santos Moschetta
981f95df55 tests: fix nounset error with $GITHUB_ENV
Initialize $GITHUB_ENV to avoid nounset error when running the scripts locally
out of Github Actions.

Fixed commit 9ba5e3d2a8

Fixes #9217
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-03-13 14:57:38 -03:00
Dan Mihai
ac27caf1b4
Merge pull request #9248 from microsoft/danmihai1/k8s-exec.bats2
tests: k8s: k8s-exec.bats auto-generated policy
2024-03-13 09:21:12 -07:00
Steve Horsman
8f4cbd49d7
Merge pull request #9263 from Amulyam24/gha-fixes
gha: ensure that the self hosted runner is in desired state before running the workflow
2024-03-13 10:49:29 +00:00
Amulyam24
3f4b24be8b gha: ensure that self hosted runner is prepared before running the workflow
This PR ensures that the self hosted runner is prepared by taking
necesary actions before running the workflow. The script prepare_runner.sh
checks the following:
1. Ensure that containerd/docker is up and running
2. Make sure that the repository workspace is cleaned up and has no conflicts
3. Remove/cleanup any leftover files from the previous runs

Fixes: #9262

Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
2024-03-13 14:20:10 +05:30
Alex Lyn
410afcc913
Merge pull request #8866 from Apokleos/netdev-qemu-rs
runtime-rs: add netdev params to cmdline for qemu-rs.
2024-03-13 13:07:43 +08:00
Dan Mihai
e8c2a45ce0 tests: k8s: k8s-pid-ns.bats auto-generated policy
Auto-generate policy for k8s-pid-ns.bats.

Fixes: #9249

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-03-12 22:34:46 +00:00
Lukáš Doktor
46e62eecb1
ci.ocp: Log the full grepped line rather than the expected msg
we are grepping for an expected message but it might contain extra bits
of information fruitful for later debugging. Let's include it in the
output and the full log in case of an error.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-03-12 17:03:46 +01:00
Lukáš Doktor
7ff2eb508e
ci.ocp: Increase the mcp update timeout
we're hitting this timeout quite often, looks like newer OCP takes
longer to reconfigure. Increase the timeout to 1200.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-03-12 16:38:04 +01:00
Lukáš Doktor
cc02329fd1
ci.ocp: Add a cleanup script
This script doesn't serve as a complete cleanup, but it can be used as a
best-effort cleaner between deploying different versions of
kata-containers on the same OCP cluster.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-03-12 16:38:04 +01:00
Lukáš Doktor
b811ee0650
ci.ocp: Allow to override the kata-deploy image
sometimes we want to test a different than the latest image (eg. when
verifying a PR via ghcr images or when bisecting a failure over older
builds). Let's add a KATA_DEPLOY_IMAGE variable for that while keeping
the latest image by default.

Fixes: #9228

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-03-12 16:38:04 +01:00
Lukáš Doktor
2936503b24
ci.ocp: Always replace the kata-deploy image in OCP pipeline
previously we only replaced the image when the previously defined one
matched the "old_img". This is good to avoid modifying developers custom
changes, but it might lead to hard-to-debug issues when the image stays
different. Let's ensure we always replace the image with the one we
asked for.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-03-12 16:38:04 +01:00
Lukáš Doktor
6525c94065
ci.ocp: Add a workaround to optionally enable skip_mount_home
the latest upstream kata-containers requires the skip_mount_home to be
enabled, which is default on OCP 4.14+ but disabled on OCP 4.13-. Let's
use a "WORKAROUND_9206_CRIO" (called by kata-containers GH issue)
variable to allow users to enable this treatement when needed.

Related to: #9206

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-03-12 16:38:04 +01:00
Lukáš Doktor
739d627b4e
ci.ocp: Turn selinux relabel failures into warnings
Instead of failing the pipeline let's proceed with an error message that
selinux setup failed so, in case of a later failure, we know what might
have caused it while keeping the coverage in case of a false setup
issue.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-03-12 16:38:04 +01:00
Lukáš Doktor
76c452d4e0
ci.ocp: Wait for all pods to finish the work
previously we only waited for a random pod to finish the selinux
relabel, which could be error-prone. Let's wait for all of the podst to
contain the expected message.

Increase the timeout to 120s as some pods might take a little bit longer
to finish.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-03-12 16:34:56 +01:00
Lukáš Doktor
f7febd07a0
ci.ocp: Allow to re-apply the selinux workaround
in case we re-apply the selinux workaround or if user had already
existing similar rule the relabel_selinux was failing. Let's allow it to
modify the existing rules as well to avoid such issues.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-03-12 16:02:21 +01:00
Lukáš Doktor
fbbea68f1f
ci.ocp: Ignore selinux setup on non-selinux cluster
improve our selinux workaround to work well on non-selinux clusters.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-03-12 16:02:20 +01:00
Alex Lyn
e2ae8ba79b runtime-rs: add network device into Qemu's cmdline
It will open tuntap device and vhost-net device
and store device files.

Fixes: #8865

Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
2024-03-12 22:28:54 +08:00
Alex Lyn
d3bca4597e runtime-rs: add open_named_tuntap to open a named tuntap device.
The open_named_tuntap function is designed as a public function to
open a tuntap device with the specified name. However, in order to
reference existing methods in dbs_utils, we still need to keep the
reference "path = "../../../dragonball/src/dbs_utils" in dependencies
and cannot hide it.

Fixes: #8865

Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
2024-03-12 22:26:32 +08:00
Alex Lyn
005b333976 runtime-rs: add network helpers and impl ToQemuParams
Add network helpers and impl ToQemuParams trait to build
netdev params which are put into cmdline for Qemu VM running.

Fixes: #8865

Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
2024-03-12 22:25:39 +08:00