Commit Graph

10154 Commits

Author SHA1 Message Date
Eric Ernst
dc42f0a33b
Merge pull request #6411 from wlan0/empty-dir
Add support for ephemeral mounts to occupy entire sandbox's memory
2023-03-13 20:07:27 -07:00
Fabiano Fidêncio
ab0bd7a1ee
Merge pull request #6292 from fidencio/topic/runtime-rs-small-fixes
runtime-rs: fix default kernel location and add more default config paths
2023-03-13 16:53:30 +01:00
Fabiano Fidêncio
40f4eef535 build: Use the correct kernel name
When calling `MAKE_KERNEL_NAME` we're considering the default kernel
name will be `vmlinux.container` or `vmlinuz.container`, which is not
the case as the runtime-rs, when used with dragonball, relies on the
`vmlinu[zx]-dragonball-experimental.container` kernel.

Other hypervisors will have to introduce a similar
`MAKE_KERNEL_NAME_${HYPERVISOR}` to adapt this to the kernel they want
to use, similarly to what's already done for the go runtime.

By doing this we also ensure that no changes in the configuration file
will be required to run runtime-rs, with dragonball, as part of our CI
or as part of kata-deploy.

Fixes: #6290

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-13 13:47:20 +01:00
James O. D. Hunt
ae9be1d94b
Merge pull request #5840 from tzY15368/feat-runtimers-direct-vol
Implement direct-volume commands handler for shim-mgmt
2023-03-13 07:58:40 +00:00
Chelsea Mafrica
4b877b0a3e
Merge pull request #6426 from openanolis/runtime-rs-resize-pty
bugfix: modify tty_win info in runtime when handling ResizePtyRequest
2023-03-10 14:08:41 -08:00
Sidhartha Mani
a6c67a161e
runtime: add support for ephemeral mounts to occupy entire sandbox memory
On hotplug of memory as containers are started, remount all ephemeral mounts with size option set to the total sandbox memory

Fixes: #6417

Signed-off-by: Sidhartha Mani <sidhartha_mani@apple.com>
2023-03-10 13:36:02 -08:00
James O. D. Hunt
99a4eaa898
Merge pull request #6443 from openanolis/runtime-rs-get-netns
bugfix: add get_ns_path API for Hypervisor
2023-03-10 20:16:22 +00:00
Fabiano Fidêncio
44bc222ca4
Merge pull request #5578 from Richardhongyu/main
runtime-rs: add the missing default trait
2023-03-10 18:01:43 +01:00
Li Hongyu
844bf053b2 runtime-rs: add the missing default trait
Some structs in the runtime-rs don't implement Default trait.
This commit adds the missing Default.

Fixes: #5463

Signed-off-by: Li Hongyu <lihongyu1999@bupt.edu.cn>
2023-03-10 08:19:56 +00:00
Yushuo
e7bca62c32 bugfix: modify tty_win info in runtime when handling ResizePtyRequest
Currently, we only create the new exec process in runtime, this will cause error
when the following requests needing to be handled:

- Task: exec process
- Task: resize process pty
- ...

The agent do not do_exec_process when we handle ExecProcess, thus we can not find
any process information in the guest when we handle ResizeProcessPty. This will
report an error.

In this commit, the handling process is modified to the:
* Modify process tty_win information in runtime
* If the exec process is not running, we just return. And the truly pty_resize will
happen when start_process

Fixes: #6248

Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
2023-03-10 14:33:51 +08:00
Tingzhou Yuan
30e235f0a1 runtime-rs: impl volume-resize trait for sandbox
Implements resize-volume handlers in shim-mgmt,
trait for sandbox and add RPC calls to agent.
Note the actual rpc handler for the resize request is currently not
implemented, refer to issue #3694.

Fixes #5369

Signed-off-by: Tingzhou Yuan <tzyuan15@bu.edu>
2023-03-10 01:27:06 -05:00
Yushuo
e029988bc2 bugfix: add get_ns_path API for Hypervisor
For external hypervisors(qemu, cloud-hypervisor, ...), the ns they launch vm in
is different from internal hypervisor(dragonball). And when we doing CreateContainer
hook, we will rely on the netns path. So we add a get_ns_path API.

Fixes: #6442

Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
2023-03-10 13:57:00 +08:00
Tingzhou Yuan
42b8867148 runtime-rs: impl volume-stats trait for sandbox
Implements get-volume-stats trait for sandbox,
handler for shim-mgmt and add RPC calls to
agent. Also added type conversions in trans.rs

Fixes #5369

Signed-off-by: Tingzhou Yuan <tzyuan15@bu.edu>
2023-03-10 00:48:02 -05:00
Christophe de Dinechin
7566a7eae4
Merge pull request #6432 from fidencio/topic/simplify-get-last-modification
packaging: Simplify get_last_modification()
2023-03-09 10:57:58 +01:00
Fabiano Fidêncio
43ce3f7588 packaging: Simplify get_last_modification()
There's no need to pass repo_root_dir to get_last_modification() as the
variable used everywhere is exported from that very same file.

Fixes: #6431

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-08 21:22:03 +01:00
Fabiano Fidêncio
33c5c49719 packaging: Move repo_root_dir to lib.sh
This is used in several parts of the code, and can have a single
declaration as part of the `lib.sh` file, which is already imported by
all the places where it's used.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-08 21:10:53 +01:00
James O. D. Hunt
614d1817ce
Merge pull request #6410 from tg5788re/kata-manager-use-runtime-checks
utils: Make kata-manager.sh runs checks
2023-03-08 09:55:03 +00:00
Chao Wu
fef268a7de
Merge pull request #6413 from xuejun-xj/xuejun/pmu
dragonball: support pmu on aarch64
2023-03-08 14:24:31 +08:00
Steve Horsman
cc1821fb8b
Merge pull request #6409 from Sig00rd/patch-1
docs: fix typo in key filename in AWS installation guide
2023-03-07 15:19:46 +00:00
Fabiano Fidêncio
861552c305
Merge pull request #6414 from jepio/jepio/backport-3.1-rustjail-systemd-cgroup-fix-6331
backport rustjail systemd cgroup fix #6331 to 3.1
2023-03-07 12:51:08 +01:00
Sidhartha Mani
16e2c3cc55
agent: implement update_ephemeral_mounts api
- implement update_ephemeral_mounts rpc
- for each mountpoint passed in, remount it with new options

Signed-off-by: Sidhartha Mani <sidhartha_mani@apple.com>
2023-03-06 13:44:14 -08:00
Sidhartha Mani
3896c7a22b
protocol: add updateEphemeralMounts proto
- adds a new rpc call to the agent service named `updateEphemeralMounts`
- this call takes a list of grpc.Storage objects

Signed-off-by: Sidhartha Mani <sidhartha_mani@apple.com>
2023-03-06 13:43:47 -08:00
Jeremi Piotrowski
23488312f5 agent: always use cgroupfs when running as init
The logic to decide which cgroup driver is used is currently based on the
cgroup path that the host provides. This requires host and guest to use the
same cgroup driver. If the guest uses kata-agent as init, then systemd can't be
used as the cgroup driver. If the host requests a systemd cgroup, this
currently results in a rustjail panic:

  thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: I/O error: No such file or directory (os error 2)

  Caused by:
      No such file or directory (os error 2)', rustjail/src/cgroups/systemd/manager.rs:44:51
  stack backtrace:
     0:     0x7ff0fe77a793 - std::backtrace_rs::backtrace::libunwind::trace::h8c197fa9a679d134
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
     1:     0x7ff0fe77a793 - std::backtrace_rs::backtrace::trace_unsynchronized::h9ee19d58b6d5934a
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
     2:     0x7ff0fe77a793 - std::sys_common::backtrace::_print_fmt::h4badc450600fc417
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:65:5
     3:     0x7ff0fe77a793 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::had334ddb529a2169
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:44:22
     4:     0x7ff0fdce815e - core::fmt::write::h1aa7694f03e44db2
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/fmt/mod.rs:1209:17
     5:     0x7ff0fe74e0c4 - std::io::Write::write_fmt::h61b2bdc565be41b5
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/io/mod.rs:1682:15
     6:     0x7ff0fe77cd3f - std::sys_common::backtrace::_print::h4ec69798b72ff254
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:47:5
     7:     0x7ff0fe77cd3f - std::sys_common::backtrace::print::h0e6c02048dec3c77
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:34:9
     8:     0x7ff0fe77c93f - std::panicking::default_hook::{{closure}}::hcdb7e705dc37ea6e
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:267:22
     9:     0x7ff0fe77d9b8 - std::panicking::default_hook::he03a933a0f01790f
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:286:9
    10:     0x7ff0fe77d9b8 - std::panicking::rust_panic_with_hook::he26b680bfd953008
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:688:13
    11:     0x7ff0fe77d482 - std::panicking::begin_panic_handler::{{closure}}::h559120d2dd1c6180
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:579:13
    12:     0x7ff0fe77d3ec - std::sys_common::backtrace::__rust_end_short_backtrace::h36db621fc93b005a
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:137:18
    13:     0x7ff0fe77d3c1 - rust_begin_unwind
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:575:5
    14:     0x7ff0fda52ee2 - core::panicking::panic_fmt::he7679b415d25c5f4
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/panicking.rs:65:14
    15:     0x7ff0fda53182 - core::result::unwrap_failed::hb71caff146724b6b
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/result.rs:1791:5
    16:     0x7ff0fe5bd738 - <rustjail::cgroups::systemd::manager::Manager as rustjail::cgroups::Manager>::apply::hd46958d9d807d2ca
    17:     0x7ff0fe606d80 - <rustjail::container::LinuxContainer as rustjail::container::BaseContainer>::start::{{closure}}::h1de806d91fcb878f
    18:     0x7ff0fe604a76 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h1749c148adcc235f
    19:     0x7ff0fdc0c992 - kata_agent::rpc::AgentService::do_create_container::{{closure}}::{{closure}}::hc1b87a15dfdf2f64
    20:     0x7ff0fdb80ae4 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h846a8c9e4fb67707
    21:     0x7ff0fe3bb816 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h53de16ff66ed3972
    22:     0x7ff0fdb519cb - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h1cbece980286c0f4
    23:     0x7ff0fdf4019c - <tokio::future::poll_fn::PollFn<F> as core::future::future::Future>::poll::hc8e72d155feb8d1f
    24:     0x7ff0fdfa5fd8 - tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut::h0a407ffe2559449a
    25:     0x7ff0fdf033a1 - tokio::runtime::task::raw::poll::h1045d9f1db9742de
    26:     0x7ff0fe7a8ce2 - tokio::runtime::scheduler::multi_thread::worker::Context::run_task::h4924ae3464af7fbd
    27:     0x7ff0fe7afb85 - tokio::runtime::task::raw::poll::h5c843be39646b833
    28:     0x7ff0fe7a05ee - std::sys_common::backtrace::__rust_begin_short_backtrace::ha7777c55b98a9bd1
    29:     0x7ff0fe7a9bdb - core::ops::function::FnOnce::call_once{{vtable.shim}}::h27ec83c953360cdd
    30:     0x7ff0fe7801d5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hed812350c5aef7a8
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/alloc/src/boxed.rs:1987:9
    31:     0x7ff0fe7801d5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hc7df8e435a658960
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/alloc/src/boxed.rs:1987:9
    32:     0x7ff0fe7801d5 - std::sys::unix:🧵:Thread:🆕:thread_start::h575491a8a17dbb33
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys/unix/thread.rs:108:17

Forward the value of "init_mode" to AgentService, so that we can force cgroupfs
when systemd is unavailable.

Fixes: #5779
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-06 20:34:21 +01:00
Jeremi Piotrowski
8546387348 agent: determine value of use_systemd_cgroup before LinuxContainer::new()
Right now LinuxContainer::new() gets passed a CreateOpts struct, but then
modifies the use_systemd_cgroup field inside that struct. Pull the cgroups path
parsing logic into do_create_container, so that CreateOpts can be immutable in
LinuxContainer::new. This is just moving things around, there should be no
functional changes.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-06 20:34:21 +01:00
Jeremi Piotrowski
736aae47a4 rustjail: print type of cgroup manager
Since the cgroup manager is wrapped in a dyn now, the print in
LinuxContainer::new has been useless and just says "CgroupManager". Extend the
Debug trait for 'dyn Manager' to print the type of the cgroup manager so that
it's easier to debug issues.

Fixes: #5779
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-06 20:34:21 +01:00
Fabiano Fidêncio
0749657c73
Merge pull request #6359 from singhwang/main
main | kata-deploy: Fix kata deploy arm64 image build error
2023-03-06 16:48:03 +01:00
Fabiano Fidêncio
dbae281924 workflows: Properly set the kata-tarball architecture
Let's make sure the kata-tarball architecture upload / downloaded / used
is exactly the same one that we need as part of the architecture we're
using to generate the image.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-06 13:18:51 +01:00
Fabiano Fidêncio
76b4591e2b tools: Adjust the build-and-upload-payload.sh script
Now that we've switched the base container image to using Ubuntu instead
of CentOS, we don't need any kind of extra logic to correctly build the
image for different architectures, as Ubuntu is a multi-arch image that
supports all the architectures we're targetting.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-06 13:18:51 +01:00
SinghWang
cd2aaeda2a kata-deploy: Switch to using an ubuntu image
Let's make sure we use a multi-arch image for building kata-deploy.
A few changes were also added in order to get systemd working inside the
kata-deploy image, due to the switch from CentOS to Ubuntu.

Fixes: #6358
Signed-off-by: SinghWang <wangxin_0611@126.com>
2023-03-06 13:18:51 +01:00
Szymon Fugas
2d43e13102
docs: fix typo in AWS installation guide
Fixes referring to previously created key file with .pen extension instead of .pem.

Fixes: #6412
Signed-off-by: Sig00rd <sfugas@virtuslab.com>
2023-03-06 13:18:08 +01:00
xuejun-xj
760f78137d dragonball: support pmu on aarch64
This commit adds support for pmu virtualization on aarch64. The
initialization of pmu is in the following order:
1. Receive pmu parameter(vpmu_feature) from runtime-rs to determine the
VpmuFeatureLevel.
2. Judge whether to initialize pmu devices and add pmu device node into
fdt on aarch64, according to VpmuFeatureLevel.

Fixes: #6168

Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
2023-03-06 18:55:13 +08:00
Fabiano Fidêncio
93a40cb35e
Merge pull request #6402 from fidencio/topic/yet-more-fixes-for-the-publish-kata-deploy-payload-work
workflows: Yet more fixes for publishing the kata-deploy payload after every PR merged
2023-03-06 10:43:32 +01:00
Fabiano Fidêncio
df35f8f885
Merge pull request #6331 from jepio/jepio/fix-agent-init-cgroups
rustjail: fix cgroup handling in agent-init mode
2023-03-05 20:29:40 +01:00
Fabiano Fidêncio
98d611623f
Merge pull request #6361 from etrunko/main
runtime/Makefile: Fix install-containerd-shim-v2 dependency
2023-03-04 13:47:11 +01:00
Fabiano Fidêncio
9bc7bef3d6 kata-deploy: Fix path to the Dockerfile
As part of bd1ed26c8d, we've pointed to
the Dockerfile that's used in the CC branch, which is wrong.

For what we're doing on main, we should be pointing to the one under the
`kata-deploy` folder, and not the one under the non-existent
`kata-deploy-cc` one.

Fixes: #6343

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-04 12:18:38 +01:00
Fabiano Fidêncio
78ba363f8e kata-deploy: Use different images for s390x and aarch64
As the image provided as part of registry.centos.org is not a multi-arch
one, at least not for CentOS 7, we need to expand the script used to
build the image to pass images that are known to work for s390x (ClefOS)
and aarch64 (CentOS, but coming from dockerhub).

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-04 12:18:32 +01:00
Fabiano Fidêncio
6267909501 kata-deploy: Allow passing BASE_IMAGE_{NAME,TAG}
Let's break the IMAGE build parameter into BASE_IMAGE_NAME and
BASE_IMAGE_TAG, as it makes it easier to replace the default CentOS
image by something else.

Spoiler alert, the default CentOS image is **not** multi-arch, and we do
want to support at least aarch64 and s390x in the near term future.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-04 12:16:41 +01:00
tg5788re
f8e44172f6 utils: Make kata-manager.sh runs checks
Updated the `kata-manager.sh` script to make it run all the checks on
the host system before attempting to create a container. If any checks
fail, they will indicate to the user what the problem is in a clearer
manner than those reported by the container manager.

Fixes: #6281.

Signed-off-by: tg5788re <jfokugas@gmail.com>
2023-03-03 09:56:12 -06:00
Chelsea Mafrica
ebe916b372
Merge pull request #6355 from yanggangtony/fix-wrong-notes
fix wrong notes for func GetSandboxesStoragePathRust()
2023-03-03 07:55:54 -08:00
Zhongtao Hu
60bb9d114a
Merge pull request #6399 from yipengyin/fix-cleanup
fix(runtime-rs): add exited state to ensure cleanup
2023-03-03 17:41:16 +08:00
Chao Wu
6fc4c8b099
Merge pull request #5788 from openanolis/runtime-rs-ocihook
runtime-rs: add oci hook support
2023-03-03 01:06:21 +08:00
James O. D. Hunt
4a7a859592
Merge pull request #6377 from pembek01/remove-cgroupsv2-check
utils: Remove kata-manager.sh cgroups v2 check
2023-03-02 17:00:46 +00:00
Fabiano Fidêncio
b20d5289cb
Merge pull request #6400 from fidencio/topic/fixes-for-generating-the-kata-deploy-payload
workflows:  Fixes for the `payload-after-push` action
2023-03-02 14:20:24 +01:00
Yipeng Yin
8030e469b2 fix(runtime-rs): add exited state to ensure cleanup
Set process status to exited at end of io wait, which indicate process
exited only, but stop process has not been finished. Otherwise, the
cleanup_container will be skipped.

Fixes: #6393

Signed-off-by: Yipeng Yin <yinyipeng@bytedance.com>
2023-03-02 18:14:20 +08:00
Fabiano Fidêncio
7d292d7fc3 workflows: Fix the path of imported workflows
In `payload-after-push.yaml` we ended up mentioning cc-*.yaml workflows,
which are non existent in the main branch.

Let's adapt the name to the correct ones.

Fixes: #6343

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-02 10:18:10 +01:00
Fabiano Fidêncio
e07162e79d workflows: Fix action name
We have a few actions in the `payload-after-push.*.yaml` that are
referring to Confidential Containers, but they should be referring to
Kata Containers instead.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-02 10:17:18 +01:00
Chao Wu
572c385774
Merge pull request #6269 from openanolis/chao/update_dragonball_version
Dragonball: update dependencies
2023-03-02 17:15:39 +08:00
Fabiano Fidêncio
7286f8f706
Merge pull request #6391 from fidencio/topic/do-not-install-docker-as-part-of-the-actions
workflows: Do not install docker
2023-03-02 10:12:15 +01:00
Fabiano Fidêncio
7201279647
Merge pull request #6344 from fidencio/topic/generate-a-kata-deploy-payload-on-each-PR-merged
workflows: Publish kata-deploy payload after a merge
2023-03-02 09:02:34 +01:00
Chao Wu
dd2713521e Dragonball: update dependencies
Since rust-vmm and dragonball-sandbox has introduced several updates
such as vPMU support for aarch64, we also need to update Dragonball
dependencies to include those changes.

Update:
virtio-queue to v0.6.0
kvm-ioctls to v0.12.0
dbs-upcall to v0.2.0
dbs-virtio-devices to v0.2.0
kvm-bindings to v0.6.0

Also, several aarch64 features are updated because of dependencies
changes:
1. update vcpu hotplug API.
2. update vpmu related API.
3. adjust unit test cases for aarch64 Dragonball.

fixes: #6268

Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2023-03-02 14:53:04 +08:00