Commit Graph

9768 Commits

Author SHA1 Message Date
Pavel Mores
96553e8bd2 runtime: Add documentation of drop-in config file fragments
Added user manual for the drop-in config file fragments feature.

Signed-off-by: Pavel Mores <pmores@redhat.com>
2022-06-29 10:56:53 +02:00
Pavel Mores
c656457e90 runtime: Add tests of drop-in config file decoding
The tests ensure that interactions between drop-ins and the base
configuration.toml and among drop-ins themselves work as intended,
basically that files are evaluated in the correct order (base file
first, then drop-ins in alphabetical order) and the last one to set
a specific key wins.

Signed-off-by: Pavel Mores <pmores@redhat.com>
2022-06-29 09:54:39 +02:00
Pavel Mores
99f5ca80fc runtime: Plug drop-in decoding into decodeConfig()
Fixes #4108

Signed-off-by: Pavel Mores <pmores@redhat.com>
2022-06-29 09:54:38 +02:00
Pavel Mores
0f9856c465 runtime: Scan drop-in directory, read files and decode them
updateFromDropIn() uses the infrastructure built by previous commits to
ensure no contents of 'tomlConfig' are lost during decoding.   To do
this, we preserve the current contents of our tomlConfig in a clone and
decode a drop-in into the original.  At this point, the original
instance is updated but its Agent and/or Hypervisor fields are
potentially damaged.

To merge, we update the clone's Agent/Hypervisor from the original
instance.   Now the clone has the desired Agent/Hypervisor and the
original instance has the rest, so to finish, we just need to move the
clone's Agent/Hypervisor to the original.

Signed-off-by: Pavel Mores <pmores@redhat.com>
2022-06-29 09:54:38 +02:00
Pavel Mores
2c1efcc697 runtime: Add helpers to copy fields between tomlConfig instances
These functions take a TOML key - an array of individual components,
e.g. ["agent" "kata" "enable_tracing"], as returned by BurntSushi - and
two 'tomlConfig' instances.  They copy the value of the struct field
identified by the key from the source instance to the target one if
necessary.

This is only done if the TOML key points to structures stored in
maps by 'tomlConfig', i.e. 'hypervisor' and 'agent'.  Nothing needs to
be done in other cases.

Signed-off-by: Pavel Mores <pmores@redhat.com>
2022-06-29 09:54:38 +02:00
Pavel Mores
20f11877be runtime: Add framework to manipulate config structs via reflection
For 'tomlConfig' substructures stored in Golang maps - 'hypervisor' and
'agent' - BurntSushi doesn't preserve their previous contents as it does
for substructures stored directly (e.g. 'runtime').  We use reflection
to work around this.

This commit adds three primitive operations to work with struct fields
identified by their `toml:"..."` tags - one to get a field value, one to
set a field value and one to assign a source struct field value to the
corresponding field of a target.

Signed-off-by: Pavel Mores <pmores@redhat.com>
2022-06-29 09:54:38 +02:00
liubin
ab5f1c9564 shim: set a non-zero return code if the wait process call failed.
Return code is an int32 type, so if an error occurred, the default value
may be zero, this value will be created as a normal exit code.

Set return code to 255 will let the caller(for example Kubernetes) know
that there are some problems with the pod/container.

Fixes: #4419

Signed-off-by: liubin <liubin0329@gmail.com>
2022-06-29 12:33:32 +08:00
Zhongtao Hu
07231b2f3f runtime-rs:refactor network model with netlink
add unit test for tcfilter

Fixes: #4289
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-29 11:38:23 +08:00
Zhongtao Hu
c8a9052063 build: format files
add Enter at the end of the file

Fixes: #4379
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-29 11:19:10 +08:00
Zhongtao Hu
242992e3de build: put install methods in utils.mk
put install methods in utils.mk to avoid duplication

Fixes: #4379
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-29 11:19:10 +08:00
Zhongtao Hu
8a697268d0 build: makefile for dragonball config
use makefile to generate dragonball config file

Fixes: #4379
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-29 11:19:07 +08:00
Zhongtao Hu
9c526292e7 runtime-rs:refactor network model with netlink
refactor tcfilter with netlink

Fixes: #4289
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-29 11:03:29 +08:00
Eric Ernst
e5be5cb086 runtime: device: cleanup outdated comments
Prior device config move didn't update the comments. Let's address this,
and make sure comments match the new path...

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-06-28 18:22:28 -07:00
Eric Ernst
5f936f268f virtcontainers: config validation is host specific
Ideally this config validation would be in a seperate package
(katautils?), but that would introduce circular dependency since we'd
call it from vc, and it depends on vc types (which, shouldn't be vc, but
probably a hypervisor package instead).

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-06-28 18:22:28 -07:00
Fabiano Fidêncio
323271403e virtcontainers: Remove unused function
While working on the previous commits, some of the functions become
non-used.  Let's simply remove them.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-28 21:19:24 +02:00
Fabiano Fidêncio
0939f5181b config: Expose default_maxmemory
Expose the newly added `default_maxmemory` to the project's Makefile and
to the configuration files.

Fixes: #4516

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-28 21:19:24 +02:00
Fabiano Fidêncio
58ff2bd5c9 clh,qemu: Adapt to using default_maxmemory
Let's adapt Cloud Hypervisor's and QEMU's code to properly behave to the
newly added `default_maxmemory` config.

While implementing this, a change of behaviour (or a bug fix, depending
on how you see it) has been introduced as if a pod requests more memory
than the amount avaiable in the host, instead of failing to start the
pod, we simply hotplug the maximum amount of memory available, mimicing
better the runc behaviour.

Fixes: #4516

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-28 21:19:24 +02:00
Fabiano Fidêncio
ad055235a5
Merge pull request #4547 from GabyCT/topic/removeunuseddocker
packaging: Remove unused kata docker configure script
2022-06-28 20:09:15 +02:00
GabyCT
b2c0387993
Merge pull request #4130 from surajssd/add-cgroup-driver-info
kata-with-k8s: Add cgroupDriver for containerd
2022-06-28 10:30:18 -05:00
GabyCT
12c1b9e6d6
Merge pull request #4536 from Tim-0731-Hzt/runtime-rs-kata-main
runtime-rs: Merge Main into runtime-rs branch
2022-06-28 10:27:35 -05:00
Gabriela Cervantes
1a78c3df2e packaging: Remove unused kata docker configure script
This PR removes an unused kata configure docker script which was used
in packaging for kata 1.x but not longer being used in kata 2.x

Fixes #4546

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-06-28 15:10:39 +00:00
Zhongtao Hu
f3907aa127 runtime-rs:Merge remote-tracking branch 'origin/main' into runtime-rs-newv
Fixes:#4536
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-28 20:58:40 +08:00
Bin Liu
badbbcd8be
Merge pull request #4400 from openanolis/anolis/dragonball-2
runtime-rs: built-in Dragonball sandbox part II - vCPU manager
2022-06-28 20:41:36 +08:00
Tim Zhang
916ffb75d7
Merge pull request #4432 from liubin/fix/4420-binary-log
shim: support shim v2 logging plugin
2022-06-28 16:29:07 +08:00
Fabiano Fidêncio
afdc960424 hypervisor: Add default_maxmemory configuration
Let's add a `default_maxmemory` configuration, which allows the admins
to set the maximum amount of memory to be used by a VM, considering the
initial amount + whatever ends up being hotplugged via the pod limits.

By default this value is 0 (zero), and it means that the whole physical
RAM is the limit.

Fixes: #4516

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-28 08:32:15 +02:00
Bin Liu
4e30e11b31 shim: support shim v2 logging plugin
Now kata shim only supports stdout/stderr of fifo from
containerd/CRI-O, but shim v2 supports logging plugins,
and nerdctl default will use the binary schema for logs.

This commit will add the others type of log plugins:

- file
- binary

In case of binary, kata shim will receive a stdout/stderr like:

binary:///nerdctl?_NERDCTL_INTERNAL_LOGGING=/var/lib/nerdctl/1935db59

That means the nerdctl process will handle the logs(stdout/stderr)

Fixes: #4420

Signed-off-by: Bin Liu <bin@hyper.sh>
2022-06-28 13:54:22 +08:00
Eric Ernst
bdf5e5229b virtcontainers: validate hypervisor config outside of hypervisor itself
Depending on the user of it, the hypervisor from hypervisor interface
could have differing view on what is valid or not. To help decouple,
let's instead check the hypervisor config validity as part of the
sandbox creation, rather than as part of the CreateVM call within the
hypervisor interface implementation.

Fixes: #4251

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-06-27 11:53:41 -07:00
Eric Ernst
469e098543 katautils: don't do validation when loading hypervisor config
Policy for whats valid/invalid within the config varies by VMM, host,
and by silicon architecture. Let's keep katautils simple for just
translating a toml to the hypervisor config structure, and leave
validation to virtcontainers.

Without this change, we're doing duplicate validation.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-06-27 10:13:26 -07:00
Chao Wu
71db2dd5b8 hotplug: add room for future acpi hotplug mechanism
In order to support ACPI hotplug in the future with the cooperative work
from the Kata community, we add ACPI feature and dbs-upcall feature to
add room for ACPI hotplug.

Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2022-06-27 21:52:36 +08:00
Zizheng Bian
8bb00a3dc8 dragonball: fix a bug when generating kernel boot args
We should refuse to generate boot args when hotplugging, not cold starting.

Signed-off-by: Zizheng Bian <zizheng.bian@linux.alibaba.com>
2022-06-27 18:12:50 +08:00
Chao Wu
2aedd4d12a doc: add document for vCPU, api and device
Create the document for vCPU and api.

Add some detail in the device document.

Fixes: #4257

Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2022-06-27 18:12:50 +08:00
wllenyj
bec22ad01f dragonball: add api module
It is used to define the vmm communication interface.

Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
2022-06-27 18:12:50 +08:00
wllenyj
07f44c3e0a dragonball: add vcpu manager
Manage vcpu related operations.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
2022-06-27 18:12:48 +08:00
wllenyj
78c9718752 dragonball: add upcall support
Upcall is a direct communication tool between VMM and guest developed
upon vsock. It is used to implement device hotplug.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Signed-off-by: Zizheng Bian <zizheng.bian@linux.alibaba.com>
2022-06-27 17:04:47 +08:00
wllenyj
7d1953b52e dragonball: add vcpu
Virtual CPU manager for virtual machines.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
2022-06-27 17:04:42 +08:00
wllenyj
468c73b3cb dragonball: add kvm context
KVM operation context for virtual machines.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
2022-06-27 16:02:06 +08:00
Bin Liu
27b1bb5ed9
Merge pull request #4467 from egernst/device-pkg
device package cleanup/refactor
2022-06-27 14:40:53 +08:00
Eric Ernst
e32bf53318 device: deduplicate state structures
Before, we maintained almost identical structures between our persist
API and what we keep for our devices, with the persist API being a
slight subset of device structures.

Let's deduplicate this, now that persist is importing device package.
Json unmarshal of prior persist structure will work fine, since it was
an exact subset of fields.

Fixes: #4468

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-06-26 21:31:29 -07:00
Eric Ernst
f97d9b45c8 runtime: device/persist: drop persist dependency from device pkgs
Rather than have device package depend on persist, let's define the
(almost duplicate) structures within device itself, and have the Kata
Container's persist pkg import these.

This'll help avoid unecessary dependencies within our core packages.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-06-26 21:31:29 -07:00
Eric Ernst
f9e96c6506 runtime: device: move to top level package
Let's move device package to runtime/pkg instead of being buried under
virtcontainers.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-06-26 21:31:29 -07:00
Bin Liu
3880e0c077 agent: refactor reading file timing for debugging
In the original code, reads mountstats file and return
the content in the error, but at this time the file maybe
changed, we should return the file content that parsed
line by line to check why there is not a fstype option.

Fixes: #4246

Signed-off-by: Bin Liu <bin@hyper.sh>
2022-06-26 21:27:43 -07:00
Archana Shinde
2488a0f6c0
Merge pull request #4439 from amshinde/update-kernel-to-5.15.46
versions: Update kernel to latest LTS version 5.15.48
2022-06-24 11:03:32 -07:00
Fabiano Fidêncio
083ca5f217
Merge pull request #4505 from yoheiueda/agent-debug-build
agent: Allow BUILD_TYPE=debug
2022-06-24 14:04:23 +02:00
Fabiano Fidêncio
03fca8b459
Merge pull request #4526 from fidencio/topic/fix-clippy-warnings-and-update-agent-vendored-code
Fix clippy warnings and update agent's vendored code
2022-06-24 14:02:28 +02:00
Fabiano Fidêncio
c70d3a2c35 agent: Update the dependencies
Let's run a `cargo update` and ensure the deps are up-to-date before we
cut the "-rc0" release.

Fixes: #4525

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-24 11:37:25 +02:00
Fabiano Fidêncio
612fd79bae random: Fix "nonminimal-bool" clippy warning
The error shown below was caught during a dependency bump in the CCv0
branch, but we better fix it here first.
```
error: this boolean expression can be simplified
  --> src/random.rs:85:21
   |
85 |             assert!(!ret.is_ok());
   |                     ^^^^^^^^^^^^ help: try: `ret.is_err()`
   |
   = note: `-D clippy::nonminimal-bool` implied by `-D warnings`
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#nonminimal_bool

error: this boolean expression can be simplified
  --> src/random.rs:93:17
   |
93 |         assert!(!ret.is_ok());
   |                 ^^^^^^^^^^^^ help: try: `ret.is_err()`
   |
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#nonminimal_bool
```

Fixes: #4523

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-24 11:37:05 +02:00
Fabiano Fidêncio
d4417f210e netlink: Fix "or-fun-call" clippy warnings
The error shown below was caught during a dependency bump in the CCv0
branch, but we better fix it here first.
```
error: use of `ok_or` followed by a function call
   --> src/netlink.rs:526:14
    |
526 |             .ok_or(anyhow!(nix::Error::EINVAL))?;
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try this: `ok_or_else(|| anyhow!(nix::Error::EINVAL))`
    |
    = note: `-D clippy::or-fun-call` implied by `-D warnings`
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#or_fun_call
error: use of `ok_or` followed by a function call
   --> src/netlink.rs:615:49
    |
615 |         let v = u8::from_str_radix(split.next().ok_or(anyhow!(nix::Error::EINVAL))?, 16)?;
    |                                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try this: `ok_or_else(|| anyhow!(nix::Error::EINVAL))`
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#or_fun_call
```

Fixes: #4523

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-24 11:37:01 +02:00
Archana Shinde
93874cb3bb packaging: Restrict kernel patches applied to top-level dir
The apply_patches.sh script applies all patches in the patches
directory,  as well as subdirectories. This means if there is a sub-dir
called "experimental" under a major kernel version directory,
experimental patches would be applied to the default kernel supported by
Kata.
We did not come accross this issue earlier as typically the experimental
kernel version was different from the default kernel.
With both the default kernel and the arm-experimental kernel having the
same major kernel version (5.15.x) at this time, trying to update the
kernel patch version revealed that arm-experimental patches were being
applied to the default kernel.

Restricting the patches to be applied to the top level directory will
solve the issue. The apply_patches script should ignore any
sub-directories meant for experimental patches.

Fixes #4520

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2022-06-23 10:43:52 -07:00
Archana Shinde
07b1367c2b versions: Update kernel to latest LTS version 5.15.48
This brings in a few security fixes.
Removing arm patches related to virtio-mem that are no longer required
as they have been merged.

Fixes #4438

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2022-06-23 10:43:52 -07:00
Fabiano Fidêncio
133528dd14
Merge pull request #4503 from amshinde/multi-queue-block
block: Leverage multiqueue for virtio-block
2022-06-23 12:17:11 +02:00