Commit Graph

13370 Commits

Author SHA1 Message Date
Chao Wu
5a4b858ece Dragonballl: introduce MTRR regs support
MTRR, or Memory-Type Range Registers are a group of x86 MSRs providing a way to control access
 and cache ability of physical memory regions.
During our test in runtime-rs + Dragonball, we found out that this register support is a must
for passthrough GPU running CUDA application, GPU needs that information to properly use GPU memory.

fixes: #9310
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2024-03-20 14:18:16 +08:00
Fabiano Fidêncio
19eb45a27d
Merge pull request #8484 from ChengyuZhu6/guest-pull
Merge basic guest pull image code to main
2024-03-19 23:15:39 +01:00
Hyounggyu Choi
6e782826c7
Merge pull request #9305 from BbolroC/handle-comment-for-skipped-tests
CI|k8s: Handle skipped tests with a comment for filter_out_per_arch
2024-03-19 22:54:03 +01:00
Fabiano Fidêncio
8911d3565f
gha: tests: Filter out confidential tests for aarch64 / ppc64le
Those two architectures are not TEE capable, thus we can just skip
running those tests there.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-03-19 18:06:01 +01:00
Fabiano Fidêncio
d14e9802b6
gha: k8s: Set {https,no}_proxy correctly for TDX
This is needed as the TDX machine is hosted inside Intel and relies on
proxies in order to connect to the external world.  Not having those set
causes issues when pulling the image inside the guest.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-03-19 18:06:00 +01:00
Fabiano Fidêncio
291b14bfb5
kata-deploy: Add the ability to set {https,no}_proxy if needed
Let's make sure those two proxy settings are respected, as those will be
widely used when pulling the image inside the guest on the Confidential
Containers case.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-03-19 18:06:00 +01:00
ChengyuZhu6
5bad18f9c9
agent: set https_proxy/no_proxy before initializing agent policy
When the https_proxy/no_proxy settings are configured alongside agent-policy enabled, the process of pulling image in the guest will hang.
This issue could stem from the instantiation of `reqwest`’s HTTP client at the time of agent-policy initialization,
potentially impacting the effectiveness of the proxy settings during image guest pulling.
Given that both functionalities use `reqwest`, it is advisable to set https_proxy/no_proxy prior to the initialization of agent-policy.

Fixes: #9212

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-03-19 18:06:00 +01:00
ChengyuZhu6
db9f18029c
README: Add https_proxy and no_proxy to agent README
Add agent.https_proxy and agent.no_proxy to the table in the agent README.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-03-19 18:06:00 +01:00
ChengyuZhu6
e23737a103
gha: refactor code with yq for better clarity
refactor code with yq for better clarity:

Before:
```bash
yq write -i "${tools_dir}/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml"
'spec.template.spec.containers[0].env[7].value' "${KATA_HYPERVISOR}:${SNAPSHOTTER}"
```

After:
```bash
yq write -i \
  "${tools_dir}/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml" \
  'spec.template.spec.containers[0].env[7].value' \
  "${KATA_HYPERVISOR}:${SNAPSHOTTER}"
```

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-03-19 18:06:00 +01:00
ChengyuZhu6
2c0bc8855b
tests: Make sure to install yq before using it
Make sure to install yq before using it to modify YAML files.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-03-19 18:06:00 +01:00
ChengyuZhu6
c52b356482
tests: add guest pull image test
Add a test case of pulling image inside the guest for confidential
containers.

Signed-off-by: Da Li Liu <liudali@cn.ibm.com>
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: Georgina Kinge <georgina.kinge@ibm.com>
Co-authored-by: Megan Wright <Megan.Wright@ibm.com>
2024-03-19 18:06:00 +01:00
ChengyuZhu6
e8c4effc07
tests: refactor the check for hypervisor to a function
Extract two reusable functions for confidential tests in confidential_common.sh

- check_hypervisor_for_confidential_tests: verifies if the input hypervisor supports confidential tests.
- confidential_setup: performs the common setup for confidential tests.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Co-authored-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-03-19 18:06:00 +01:00
ChengyuZhu6
6e5e4e55d0
rootfs: add ca file to guest rootfs
To access the URL, the component to pull image in the guest needs to send a request to the remote.
Therefore, we need to add CA to the rootfs.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-03-19 18:06:00 +01:00
ChengyuZhu6
8724d7deeb
packaging: Enable to build agent with PULL_TYPE feature
Enable to build kata-agent with PULL_TYPE feature.

We build kata-agent with guest-pull feature by default, with PULL_TYPE set to default.
This doesn't affect how kata shares images by virtio-fs. The snapshotter controls the image pulling in the guest.
Only the nydus snapshotter with proxy mode can activate this feature.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-03-19 18:06:00 +01:00
ChengyuZhu6
cd6a84cfc5
kata-deploy: Setting up snapshotters per runtime handler
Setting up snapshotters per runtime handler as the commit
(6cc6ca5a7f) described.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-03-19 18:05:59 +01:00
ChengyuZhu6
ba242b0198
runtime: support different cri container type check
To support handle image-guest-pull block volume from different CRIs, including cri-o and containerd.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-03-19 18:05:59 +01:00
ChengyuZhu6
874d83b510
agent/image: Use guest provided pause image
By default the pause image and runtime config will provided
by host side, this may have potential security risks when the
host config a malicious pause image, then we will use the pause
image packaged in the rootfs.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Arron Wang <arron.wang@intel.com>
Co-authored-by: Julien Ropé <jrope@redhat.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
2024-03-19 18:05:59 +01:00
ChengyuZhu6
c269b9e8c6
agent: Add guest-pull feature for kata-agent
Add "guest-pull" feature option to determine that the related dependencies
would be compiled if the feature is enabled.

By default, agent would be built with default-pull feature, which would
support all pull types, including sharing images by virtio-fs and
pulling images in the guest.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-03-19 18:05:59 +01:00
Aurélien
192250c52e
Merge pull request #9299 from sprt/sprt/mariner-normal-tests
ci: aks: also run tests in normal instance for Mariner
2024-03-19 11:34:20 -05:00
ChengyuZhu6
965da9bc9b
runtime: support to pass image information to guest by KataVirtualVolume
support to pass image information to guest by KataVirtualVolumeImageGuestPullType
in KataVirtualVolume, which will be used to pull image on the guest.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-03-19 17:22:36 +01:00
ChengyuZhu6
cfd14784a0
agent: Introduce ImagePullHandler to support IMAGE_GUEST_PULL volume
As we do not employ a forked containerd in confidential-containers, we utilize the KataVirtualVolume
which storing the image information as an integral part of `CreateContainer`.
Within this process, we store the image information in rootfs.storage and pass this image url through `CreateContainerRequest`.
This approach distinguishes itself from the use of `PullImageRequest`, as rootfs.storage is already set and initialized at this stage.
To maintain clarity and avoid any need for modification to the `OverlayfsHandler`,we introduce the `ImagePullHandler`.
This dedicated handler is responsible for orchestrating the image-pulling logic within the guest environment.
This logic encompasses tasks such as calling the image-rs to download and unpack the image into `/run/kata-containers/{container_id}/images`,
followed by a bind mount to `/run/kata-containers/{container_id}`.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-03-19 17:22:36 +01:00
ChengyuZhu6
462051b067
agent/image: merge container spec for images pulled inside guest
When being passed an image name through a container annotation,
merge its corresponding bundle OCI specification and process into the passed container creation one.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Arron Wang <arron.wang@intel.com>
Co-authored-by: Jiang Liu <gerry@linux.alibaba.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: wllenyj <wllenyj@linux.alibaba.com>
Co-authored-by: jordan9500 <jordan.jackson@ibm.com>
2024-03-19 17:22:36 +01:00
ChengyuZhu6
cec1916196
agent: Support https_proxy/no_proxy config for image download in guest
Containerd can support set a proxy when downloading images with a environment variable.
For CC stack, image download is offload to the kata agent, we need support similar feature.
Current we add https_proxy and no_proxy, http_proxy is not added since it is insecure.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Arron Wang <arron.wang@intel.com>
2024-03-19 17:22:36 +01:00
ChengyuZhu6
9cddd5813c
agent/image: Enable image-rs crate to pull image inside guest
With image-rs pull_image API, the downloaded container image layers
will store at IMAGE_RS_WORK_DIR, and generated bundle dir with rootfs
and config.json will be saved under CONTAINER_BASE/cid directory.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Arron Wang <arron.wang@intel.com>
Co-authored-by: Jiang Liu <gerry@linux.alibaba.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: wllenyj <wllenyj@linux.alibaba.com>
2024-03-19 17:22:36 +01:00
ChengyuZhu6
2b3a00f848
agent: export the image service singleton instance
Export the image service singleton instance.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Jiang Liu <gerry@linux.alibaba.com>
Co-authored-by: Xynnn007 <xynnn@linux.alibaba.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: wllenyj <wllenyj@linux.alibaba.com>
2024-03-19 17:22:36 +01:00
ChengyuZhu6
1f1ca6187d
agent: Introduce ImageService
Introduce structure ImageService, which will be used to pull images
inside the guest.

Fixes: #8103

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
co-authored-by: wllenyj <wllenyj@linux.alibaba.com>
co-authored-by: stevenhorsman <steven@uk.ibm.com>
2024-03-19 17:22:33 +01:00
Hyounggyu Choi
b381743dd5 CI|k8s: Handle skipped tests with a comment for filter_out_per_arch
This commit updates `filter_k8s_test.sh` to handle skipped tests that
include comments. In addition to the existing parameter expansion,
the following expansions have been added:

- Removal of a comment
- Stripping of trailing spaces

Fixes: #9304

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-03-19 17:21:25 +01:00
Chelsea Mafrica
42dfe0e8d1
Merge pull request #9286 from jodh-intel/agent-show-enabled-features
agent: Show features enabled at build time
2024-03-19 08:54:49 -07:00
Wainer Moschetta
e6501aa4ad
Merge pull request #9229 from ldoktor/ocp-ci
ocp.ci: Various fixes and improvements to the OCP pipeline
2024-03-19 11:13:01 -03:00
James O. D. Hunt
46aec0f15a
Merge pull request #9293 from jodh-intel/kata-manager-fix-containerd-for-docker
kata-manager: Fix Docker install
2024-03-19 10:06:44 +00:00
Fabiano Fidêncio
e0a6b6449f
Merge pull request #9302 from BbolroC/fix-permission-issue-on-s390x-runners
gha: Place pre-action on s390x runner for kata-deploy during release
2024-03-19 10:42:23 +01:00
Hyounggyu Choi
f2bc819644 gha: Place pre-action on s390x runner for kata-deploy during release
This is to place a pre-action step for the kata-deploy job in order to
clean up the github workspace directory before checking out the repo.

Fixes: #9301

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-03-19 10:18:38 +01:00
Alex Lyn
7af2df408e
Merge pull request #9295 from likebreath/0318/fix_clh_default_netconfig
runtime-rs: ch: Provide valid default value for NetConfig
2024-03-19 15:17:18 +08:00
Xuewei Niu
99d0e5fff8
Merge pull request #9270 from zvonkok/kata-agent-bind-mount
kata-agent: optional bind flag
2024-03-19 10:39:23 +08:00
Aurélien Bombo
71a1be9c57 ci: aks: also run tests in normal instance for Mariner
Currently we're only running the small instance tests. This adds the
normal instance tests as well.

Fixes: #9298

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2024-03-18 23:33:17 +00:00
Bo Chen
ad4262e86b runtime-rs: ch: Provide valid default value for NetConfig
The current default value of IP `0.0.0.0` with mask `0.0.0.0` will cause
ioctl error when being used to create and configure TAP device, with
newer version of Cloud Hypervisor [1]. This patch replaces them with
valid value that are the same as the Go-lang runtime [2].

[1] https://github.com/cloud-hypervisor/cloud-hypervisor/pull/5924
[2] e3f7852738/src/runtime/virtcontainers/pkg/cloud-hypervisor/client/model_net_config.go (L40-L57)

Fixes: #9254

Signed-off-by: Bo Chen <chen.bo@intel.com>
2024-03-18 15:47:58 -07:00
Fabiano Fidêncio
e3f7852738
Merge pull request #9289 from fidencio/topic/releases-follow-up-IV
releases: Simply the release in order to avoid pushing a commit updating the VERSION file
2024-03-18 17:38:58 +01:00
James O. D. Hunt
a6c3f75872 kata-manager: Fix Docker install
Fix the Docker install by removing the second (erroneous) call to
`containerd_installed()` in `handle_docker()`.

Without this fix, installing using Docker (`-D`) will work *iff* you
already have containerd installed. However, if you do not have
containerd installed, the `containerd_installed()` function returns 1,
which exits the script as we're running with `set -e`, leaving a broken
Docker installation.

> **Note:** containerd is installed via Docker's `get-docker.sh` script.

Fixes: #9292.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2024-03-18 14:08:35 +00:00
stevenhorsman
0ab8e61a64 release: Remove release type from arch release
Now we don't have minor and major releases and
we are now generating a new version
in the release workflow, we can
tidy up the arch specific releases workflows to remove
the extra required inputs

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-03-18 12:27:57 +00:00
Greg Kurz
3cfc1b6ba7 releases: Adjust documentation to the new workflow
This drops the documentation of the legacy release scripts and adds
a quick description of the scripts of the new workflow. It also
highlights the bump of the `VERSION` file.

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-03-18 12:57:02 +01:00
Greg Kurz
76c640767e releases: Drop Makefile
It isn't used anymore.

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-03-18 12:54:00 +01:00
Greg Kurz
bfe19e68e8 kata-deploy: Adapt test-kata.sh to the new release workflow
All releases are now created in the `main` branch following
the very same workflow. No need to special case pre-releases.

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-03-18 12:54:00 +01:00
Fabiano Fidêncio
12578f11bc
releases: Assume VERSION has the correct version to be released
This is done in order to avoid having to push a commit to the main
branch, which is against the defined rules on GitHub.

By doing this, we need to educate ourselves to always bump the VERSION
file as soon as a release is cut out.

As a side effect of this change, we can drop the release-major and
release-minor workflows, as those are not needed anymore.

Fixes: #9064 - part IV

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-03-16 13:30:58 +01:00
Fabiano Fidêncio
8ce50269fe
release: Bump the VERSION file to the next release number
3.3.0 it will be.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-03-16 13:21:27 +01:00
Xuewei Niu
9f512c016e
Merge pull request #9282 from gkurz/runtime-rs-fds-for-qemu
runtime-rs: Consolidate the handling of fds passed to QEMU
2024-03-16 10:26:11 +08:00
Greg Kurz
1e526a4769 runtime-rs: Consolidate the handling of fds passed to QEMU
File descriptors that are passed to QEMU need some special care.
We want them to be closed when the QEMU process is started. But
at the same time, it is required that the associated rust File
structures, either coming from the` std::fs` or the `tokio::fs`
crates, are still in scope when the QEMU process is forked. This
is currently achieved by keeping File structures in variables
at the outer scope of `start_vm()`. This scheme is currently
duplicated, with similar justifications in the corresponding
comments.

Consolidate all this handling in one place with a more generic
explanation.

Fixes #9281

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-03-15 16:14:59 +01:00
James O. D. Hunt
9ef59488d9 agent: Show features enabled at build time
The agent now has a number of optional build-time features that can be
enabled.

Add details of these features to the following areas:

- Version output (`kata-agent --version`)
- Announce message (so that the details are always added to the journal
  at agent startup).
- The response message returned by the ttRPC `GetGuestDetails()` API.

Fixes: #9285.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2024-03-15 13:29:21 +00:00
Chelsea Mafrica
2c50d3c393
Merge pull request #9278 from wainersm/github_env_fix
tests: fix nounset error with $GITHUB_ENV
2024-03-14 16:39:13 -07:00
Greg Kurz
6a112cc7a5 runtime-rs: Fix missing dependency
Some previous contribution missed to run cargo clippy.
Fix the dependency now so that it doesn't cause noise
in future contributions.

Signed-off-by: Greg Kurz <groug@kaod.org>
2024-03-14 23:19:38 +01:00
Dan Mihai
b3b00e00a6
Merge pull request #9246 from microsoft/danmihai/default-env
genpolicy: default env if image doesn't have env
2024-03-14 11:01:43 -07:00