Commit Graph

12951 Commits

Author SHA1 Message Date
Fabiano Fidêncio
741ed1c8bd
Merge pull request #9001 from fidencio/topic/fix-cache-for-confidential-kernel-part-III
packaging: Don't build the confidential / sev kernel twice -- part III
2024-02-02 15:19:41 +01:00
Wainer Moschetta
424fbfe58f
Merge pull request #8654 from ldoktor/openshift-tests
ci/openshift-ci: Move openshift-ci from the tests repo here
2024-02-02 10:40:30 -03:00
Fabiano Fidêncio
2ff3f0afc6
packaging: Remove trailing whitespace from extra_tarballs arg
This was overlooked during the reviews.

Fixes: #6415 -- part III

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-02 12:42:02 +01:00
Fabiano Fidêncio
228bc48c73
packaging: Fix kernel confidential name
It should be "kernel-confidential" instead of "kernel".

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-02 12:42:02 +01:00
Fabiano Fidêncio
31b21093b0
packaging: Pass the kernel flavour to get_kernel_modules_dir
I made this a required argument during the series and ended up
forgetting to add that while calling the function.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-02 12:42:02 +01:00
Fabiano Fidêncio
51b1df2333
packaging: Fix typo to get the extra_tarballs path
It should've been  "${m#*:}" instead of "${m#&:}".

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-02 12:41:54 +01:00
Fabiano Fidêncio
53e8461db2
Merge pull request #9000 from fidencio/topic/fix-pushing-artefacts-to-registry
packaging: Fix pushing artefacts to the registry
2024-02-02 10:21:40 +01:00
Fabiano Fidêncio
0b221b5618
packaging: Fix pushing artefacts to the registry
This issues was introduced due to a typo not caught during reviews on
e5bca90274.

Fixes: #6415 -- part II

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-02 10:13:11 +01:00
Wenyuan Liu
cb888516c1
Merge pull request #8760 from fadecoder/reduce_go_runtime_mounts
runtime: Reduce the mount points with namespace isolation
2024-02-02 16:54:44 +08:00
Greg Kurz
d1a26ead94
Merge pull request #8454 from BbolroC/compile-with-qemu-s390x
runtime-rs: make compilation for QEMU on s390x
2024-02-02 09:29:32 +01:00
Fabiano Fidêncio
0520b272a3
Merge pull request #8987 from fidencio/topic/fix-cache-for-confidential-kernel
packaging: cache: Fix caching kernels which rely on extra modules
2024-02-02 09:10:52 +01:00
Amulya Meka
e4252a3fe2
Merge pull request #8957 from Amulyam24/add-k8s-test-ppc64le
gha: add kubernetes tests workflow for ppc64le
2024-02-02 10:22:00 +05:30
Fabiano Fidêncio
b2f1235e3c
Merge pull request #8994 from sprt/sprt/switch-aks-eastus
ci: aks: switch from eastus2 to eastus region
2024-02-02 00:09:40 +01:00
Hyounggyu Choi
bb6f5073aa runtime-rs: Allow compilation for s390x
Until now, runtime-rs couldn't be compiled on s390x.
We need to lift those restrictions in Makefile first.

Fixes: #8446

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-02-01 23:48:15 +01:00
Dan Mihai
6f1062b5d6
Merge pull request #8966 from microsoft/danmihai1/k8s-sandbox-vcpus-allocation
genpolicy: ignore empty YAML as input
2024-02-01 13:51:02 -08:00
Dan Mihai
8f9c92c0ee
Merge pull request #8977 from microsoft/danmihai1/default-namespace
genpolicy: support non-default namespace name
2024-02-01 13:50:33 -08:00
Gabriela Cervantes
6771ca463b gha: k8s: Add cloud-hypervisor (runtime-rs) support
This PR adds the Cloud Hypervisor driver, integrated with the runtime-rs,
as part of the kubernetes tests different with devmapper.

Fixes #8995

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-02-01 21:22:56 +00:00
Aurélien Bombo
0ace31f041 ci: aks: switch from eastus2 to eastus region
This addresses an internal AKS issue that intermittently prevents
clusters from getting created. The fix has been rolled out to eastus but
not yet eastus2, so we unblock the CI by switching. No downsides in
general.

This supersedes #8990.

Fixes: #8989

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2024-02-01 19:22:42 +00:00
Hyounggyu Choi
8fcee6e6ec runtime-rs: Use Persist::restore() of QEMU for VirtSandbox
It fails to compile virt_container because Dragonball is only
used in the implementation of the trait method Persist::restore().
As the hypervisor is not compiled on s390x and QEMU implements
the trait method, this commit is to let the method use QEMUi's.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-02-01 18:02:10 +01:00
Hyounggyu Choi
56aef3741d runtime-rs: Exclude hypervisors plugins except QEMU for s390x
Dragonball and cloud-hypervisor are not supported on s390x.
We need to exclude the plugins for these hypervisors from compilation.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-02-01 18:02:10 +01:00
Fabiano Fidêncio
5d2906c36a
packaging: Bump the kata config kernel version
Just to make sure we won't use cached components.

Fixes: #6415

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-01 16:57:15 +01:00
Fabiano Fidêncio
d2ea11dbff
packaging: Use the cached kernel modules
Till now we didn't have a logic to consume the kernel modules cached
tarball.  Let's make sure those are consumed as it'll save us a
reasonable amount of build time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-01 16:57:15 +01:00
Fabiano Fidêncio
e5bca90274
packaging: Cache the kernel modules
This will save us a lot of time, as right now the CI is rebuilding the
kernel for absolutely no reason.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-01 16:55:21 +01:00
Fabiano Fidêncio
f481f58659
packaging: Create the tarball for the kernel modules
Let's start doing this for the confidential kernels (and also for SEV,
till it gets removed).

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-01 16:55:20 +01:00
Fabiano Fidêncio
a58caca723
packaging: Take extra tarballs in install_cached_tarball_component()
This allows us to add a map, in the format of:
`"tarball1_name:tarball1_path tarball2_name:tarball2_path ..."`

With this we have a base to start doing a better job when caching extra
artefacts, like kernel modules.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-01 16:55:20 +01:00
Fabiano Fidêncio
33ac5468fe
packaging: Add function to get the kernel modules directory
Right now this is just being added but not used yet.  The idea is to use
this to both cache and later on untar the kernel modules needed for some
of the kernel targets we have (specifically looking at the confidential
one).

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-01 16:55:20 +01:00
Zhigang Wang
9317e23df1 mount: Reduce the mount points with namespace isolation
This patch can reduce load on systemd process, and
increase the k8s deployment density when using go runtime.

Fixes: #8758

Signed-off-by: Zhigang Wang <wangzhigang17@huawei.com>
Signed-off-by: Liu Wenyuan <liuwenyuan9@huawei.com>
2024-02-01 18:34:24 +08:00
Fabiano Fidêncio
ed6816e29f
kata-manager: Add support for nerdctl installation
As already done for docker, let's also add support for installing
nerdctl + kata containers.

For now, at least for now, we are explicitly not allowing the
combination of installing both docker and nerdctl in the same
installation in order to reduce the script complexity.

Also, nerdctl installation, for now, is limited to x86_64 and aarch64 as
those are the only architectures that nerdctl releases a "full" package
for.

Fixes: #8358

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-01 09:19:35 +01:00
Xuewei Niu
2332552c8f
Merge pull request #7483 from frezcirno/passfd_io_feature
runtime-rs: improving io performance using dragonball's vsock fd passthrough
2024-02-01 14:53:53 +08:00
Amulyam24
f8585db8d9 gha: add kubernetes tests workflow for ppc64le
This PR adds workflow for running kubernetes test suite on ppc64le.

It uses scripts to create and delete the cluster using kubeadm as none of the current cluster creation tools are supported on Power.

Fixes: #7950

Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
2024-02-01 12:23:11 +05:30
Alex Lyn
cf26c16017
Merge pull request #8931 from yaoyinnan/8930/feat/merge-ValidCgroupPath
runtime: merged ValidCgroupPath method
2024-02-01 12:53:55 +08:00
Alex Lyn
a157fc3b74
Merge pull request #8974 from yaoyinnan/5240/fix/cgroup-parallel
runtime: add SingleContainer when obtaining OCI Spec
2024-02-01 11:43:02 +08:00
Alex Lyn
1b8f3ce28a
Merge pull request #8929 from yaoyinnan/8838/fix/error-message
runtime-rs: report error on missing or empty fields in configuration
2024-02-01 11:02:30 +08:00
Dan Mihai
09ea0eed9d genpolicy: ignore empty YAML as input
Kata CI's pod-sandbox-vcpus-allocation.yaml ends with "---", so the
empty YAML document following that line should be ignored.

To test this fix:

genpolicy -u -y pod-sandbox-vcpus-allocation.yaml

Fixes: #8895

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-02-01 02:22:21 +00:00
Dan Mihai
befef119ff
Merge pull request #8941 from malt3/genpolicy-flags
genpolicy: allow separate paths for rules and settings files
2024-01-31 18:14:12 -08:00
GabyCT
6db1cd5f65
Merge pull request #8964 from GabyCT/topic/fixnerdcltt
tests: Re-arranged nerdctl tests
2024-01-31 15:02:54 -06:00
Dan Mihai
21125baec3
Merge pull request #8962 from microsoft/danmihai1/config-map-optional2
genpolicy: ignore volume configMap optional field
2024-01-31 12:29:30 -08:00
Fabiano Fidêncio
39a64d1447
Merge pull request #8269 from wainersm/kata-deploy_deprecated
kata-deploy: fix deprecations on kustomization files
2024-01-31 20:02:01 +01:00
Hyounggyu Choi
9c0312d466
Merge pull request #8956 from BbolroC/agent-build-fix-s390x-ppc64le
packaging: Use Ubuntu 20.04 for building an agent
2024-01-31 18:23:16 +01:00
Greg Kurz
8b1dc06971
Merge pull request #8938 from pmores/log-qemus-stderr-in-shim-log
runtime-rs: Log qemu's stderr in shim log
2024-01-31 18:04:28 +01:00
Dan Mihai
f0339a79a6 genpolicy: support non-default namespace name
Allow users to specify in genpolicy-settings.json a default cluster
namespace other than "default". For example, Kata CI uses as default
namespace: "kata-containers-k8s-tests".

Fixes: #8976

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-01-31 15:47:01 +00:00
Zixuan Tan
222de4f684 agent: Fix a race condition in passfd_io.rs
There is a race condition in agent HVSOCK_STREAMS hashmap, where a
stream may be taken before it is inserted into the hashmap. This patch
add simple retry logic to the stream consumer to alleviate this issue.

Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
2024-01-31 21:07:48 +08:00
Zixuan Tan
6e4d4c329a agent,runtime-rs: Add license header to passfd_io.rs
Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
2024-01-31 21:07:48 +08:00
Zixuan Tan
1206de2c23 agent: Use pipes as stdout/stderr of container process
Linux forbids opening an existing socket through /proc/<pid>/fd/<fd>,
making some images relying on the special file /dev/stdout(stderr),
/proc/self/fd/1(2) fail to boot in passfd io mode, where the
stdout/stderr of a container process is a vsock socket.

For back compatibility, a pipe is introduced between the process
and the socket, and its read end is set as stdout/stderr of the
container process instead of the socket. The agent will do the
forwarding between the pipe and the socket.

Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
2024-01-31 21:07:48 +08:00
Zixuan Tan
f6710610d1 agent,runtime-rs,runk: fix fmt and clippy warnings
Fix rustfmt and clippy warnings detected by CI.

Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
2024-01-31 21:07:48 +08:00
Zixuan Tan
89be42a177 runtime-rs: open stdout and stderr fifos NONBLOCK
This patch adds O_NONBLOCK flag when open stdout and stderr FIFOs
to avoid blocking.

Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
2024-01-31 21:07:48 +08:00
Zixuan Tan
3eb4bed957 agent: use biased select to avoid data loss
This patch uses a biased select to avoid stdin data loss in case of
CloseStdinRequest.

Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
2024-01-31 21:07:48 +08:00
Zixuan Tan
7874ef5fd2 agent: set stdout/err vsock stream as blocking before passing to child
In passfd io mode, when not using a terminal, the stdout/stderr vsock
streams are directly used as the stdout/stderr of the child process.
These streams are non-blocking by default.

The stdout/stderr of the process should be blocking, otherwise
the process may encounter EAGAIN error when writing to stdout/stderr.

Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
2024-01-31 21:07:48 +08:00
Fupan Li
cfb262d02f container: keep the io connection when pass fd to hybrid vsock
We want the io connection keep connected when the containerd closed
the io pipe, thus it can be attached on the io stream.

Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
2024-01-31 21:07:48 +08:00
Fupan Li
4a762fcfdd dbs: hybrid stream support keep the connection when local closed
Support the hybrid fd passthrough mode with passing pipe fd,
which can specify this connection kept even when the pipe
peer closed, and this connection can be reget wich re-opening
the pipe.

Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
2024-01-31 21:07:48 +08:00