Commit Graph

9129 Commits

Author SHA1 Message Date
Eric Ernst
e32bf53318 device: deduplicate state structures
Before, we maintained almost identical structures between our persist
API and what we keep for our devices, with the persist API being a
slight subset of device structures.

Let's deduplicate this, now that persist is importing device package.
Json unmarshal of prior persist structure will work fine, since it was
an exact subset of fields.

Fixes: #4468

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-06-26 21:31:29 -07:00
Eric Ernst
f97d9b45c8 runtime: device/persist: drop persist dependency from device pkgs
Rather than have device package depend on persist, let's define the
(almost duplicate) structures within device itself, and have the Kata
Container's persist pkg import these.

This'll help avoid unecessary dependencies within our core packages.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-06-26 21:31:29 -07:00
Eric Ernst
f9e96c6506 runtime: device: move to top level package
Let's move device package to runtime/pkg instead of being buried under
virtcontainers.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-06-26 21:31:29 -07:00
Bin Liu
3880e0c077 agent: refactor reading file timing for debugging
In the original code, reads mountstats file and return
the content in the error, but at this time the file maybe
changed, we should return the file content that parsed
line by line to check why there is not a fstype option.

Fixes: #4246

Signed-off-by: Bin Liu <bin@hyper.sh>
2022-06-26 21:27:43 -07:00
Archana Shinde
2488a0f6c0
Merge pull request #4439 from amshinde/update-kernel-to-5.15.46
versions: Update kernel to latest LTS version 5.15.48
2022-06-24 11:03:32 -07:00
Fabiano Fidêncio
083ca5f217
Merge pull request #4505 from yoheiueda/agent-debug-build
agent: Allow BUILD_TYPE=debug
2022-06-24 14:04:23 +02:00
Fabiano Fidêncio
03fca8b459
Merge pull request #4526 from fidencio/topic/fix-clippy-warnings-and-update-agent-vendored-code
Fix clippy warnings and update agent's vendored code
2022-06-24 14:02:28 +02:00
Fabiano Fidêncio
c70d3a2c35 agent: Update the dependencies
Let's run a `cargo update` and ensure the deps are up-to-date before we
cut the "-rc0" release.

Fixes: #4525

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-24 11:37:25 +02:00
Fabiano Fidêncio
612fd79bae random: Fix "nonminimal-bool" clippy warning
The error shown below was caught during a dependency bump in the CCv0
branch, but we better fix it here first.
```
error: this boolean expression can be simplified
  --> src/random.rs:85:21
   |
85 |             assert!(!ret.is_ok());
   |                     ^^^^^^^^^^^^ help: try: `ret.is_err()`
   |
   = note: `-D clippy::nonminimal-bool` implied by `-D warnings`
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#nonminimal_bool

error: this boolean expression can be simplified
  --> src/random.rs:93:17
   |
93 |         assert!(!ret.is_ok());
   |                 ^^^^^^^^^^^^ help: try: `ret.is_err()`
   |
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#nonminimal_bool
```

Fixes: #4523

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-24 11:37:05 +02:00
Fabiano Fidêncio
d4417f210e netlink: Fix "or-fun-call" clippy warnings
The error shown below was caught during a dependency bump in the CCv0
branch, but we better fix it here first.
```
error: use of `ok_or` followed by a function call
   --> src/netlink.rs:526:14
    |
526 |             .ok_or(anyhow!(nix::Error::EINVAL))?;
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try this: `ok_or_else(|| anyhow!(nix::Error::EINVAL))`
    |
    = note: `-D clippy::or-fun-call` implied by `-D warnings`
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#or_fun_call
error: use of `ok_or` followed by a function call
   --> src/netlink.rs:615:49
    |
615 |         let v = u8::from_str_radix(split.next().ok_or(anyhow!(nix::Error::EINVAL))?, 16)?;
    |                                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try this: `ok_or_else(|| anyhow!(nix::Error::EINVAL))`
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#or_fun_call
```

Fixes: #4523

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-24 11:37:01 +02:00
Archana Shinde
93874cb3bb packaging: Restrict kernel patches applied to top-level dir
The apply_patches.sh script applies all patches in the patches
directory,  as well as subdirectories. This means if there is a sub-dir
called "experimental" under a major kernel version directory,
experimental patches would be applied to the default kernel supported by
Kata.
We did not come accross this issue earlier as typically the experimental
kernel version was different from the default kernel.
With both the default kernel and the arm-experimental kernel having the
same major kernel version (5.15.x) at this time, trying to update the
kernel patch version revealed that arm-experimental patches were being
applied to the default kernel.

Restricting the patches to be applied to the top level directory will
solve the issue. The apply_patches script should ignore any
sub-directories meant for experimental patches.

Fixes #4520

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2022-06-23 10:43:52 -07:00
Archana Shinde
07b1367c2b versions: Update kernel to latest LTS version 5.15.48
This brings in a few security fixes.
Removing arm patches related to virtio-mem that are no longer required
as they have been merged.

Fixes #4438

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2022-06-23 10:43:52 -07:00
Fabiano Fidêncio
133528dd14
Merge pull request #4503 from amshinde/multi-queue-block
block: Leverage multiqueue for virtio-block
2022-06-23 12:17:11 +02:00
Fabiano Fidêncio
f186a52b16
Merge pull request #4511 from fidencio/topic/add-config-efi-to-the-tdx-kernel
kernel: Add CONFIG_EFI=y as part of the TDX fragments
2022-06-23 12:15:30 +02:00
Yohei Ueda
1b7d36fdb0
agent: Allow BUILD_TYPE=debug
The cargo command creates debug build binaries, when the --release
option is not specified. Specifying --debug option causes an error.
This patch specifies --release option when BUILD_TYPE=release,
and does not specify any build type option when BUILD_TYPE=debug.

Fixes #4504

Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
2022-06-23 13:54:32 +09:00
Fabiano Fidêncio
9ff10c0830 kernel: Add CONFIG_EFI=y as part of the TDX fragments
Otherwise `./build-kernel.sh -x tdx setup` will fail with the following
error:
```
$ ./build-kernel.sh -x tdx setup
INFO: Config version: 92
INFO: Kernel version: tdx-guest-v5.15-4
INFO: kernel path does not exist, will download kernel
INFO: Apply patches from
/home/ffidenci/go/src/github.com/kata-containers/kata-containers/tools/packaging/kernel/patches/tdx-guest-v5.15-4.x
INFO: Found 0 patches
INFO: Enabling config for 'tdx' confidential guest protection
INFO: Constructing config from fragments:
/home/ffidenci/go/src/github.com/kata-containers/kata-containers/tools/packaging/kernel/configs/fragments/x86_64/.config

WARNING: unmet direct dependencies detected for UNACCEPTED_MEMORY
  Depends on [n]: EFI [=n] && EFI_STUB [=n]
  Selected by [y]:
  - INTEL_TDX_GUEST [=y] && HYPERVISOR_GUEST [=y] && X86_64 [=y] &&
    CPU_SUP_INTEL [=y] && PARAVIRT [=y] && SECURITY [=y] &&
     X86_X2APIC[=y]
INFO: Some CONFIG elements failed to make the final .config:
INFO: Value requested for CONFIG_EFI_STUB not in final .config
INFO: Generated config file can be found in
/home/ffidenci/go/src/github.com/kata-containers/kata-containers/tools/packaging/kernel/configs/fragments/x86_64/.config
ERROR: Failed to construct requested .config file
ERROR: failed to find default config
```

Fixes: #4510

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-22 15:21:30 +02:00
Fabiano Fidêncio
78e27de6c3
Merge pull request #4358 from zvonkok/memreserve
runtime: Add heuristic to get the right value(s) for mem-reserve
2022-06-22 13:41:23 +02:00
Archana Shinde
e227b4c404 block: Leverage multiqueue for virtio-block
Similar to network, we can use multiple queues for virtio-block
devices. This can help improve storage performance.
This commit changes the number of queues for block devices to
the number of cpus for cloud-hypervisor and qemu.

Today the default number of cpus a VM starts with is 1.
Hence the queues used will be 1. This change will help
improve performance when the default cold-plugged cpus is greater
than one by changing this in the config file. This may also help
when we use the sandboxing feature with k8s that passes down
the sum of the resources required down to Kata.

Fixes #4502

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2022-06-21 12:38:53 -07:00
Eric Ernst
72049350ae
Merge pull request #4288 from fengwang666/enable-qemu-sandbox
runtime: enable sandbox feature on qemu
2022-06-21 09:22:26 -07:00
GabyCT
8eac22ac53
Merge pull request #4495 from Amulyam24/snap-fix
snap: fix snap build on ppc64le
2022-06-21 09:21:23 -05:00
Zvonko Kaiser
e7e7dc9dfe runtime: Add heuristic to get the right value(s) for mem-reserve
Fixes: #2938

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2022-06-21 03:44:28 -07:00
Bin Liu
e422730c7f
Merge pull request #4497 from GabyCT/topic/removeunusedref
packaging: Remove unused publish kata image script
2022-06-21 17:46:45 +08:00
James O. D. Hunt
e11fcf7d3c
Merge pull request #4168 from Champ-Goblem/patch/fix-chronyd-failure-on-boot
rootfs: Fix chronyd.service failing on boot
2022-06-21 09:43:13 +01:00
Gabriela Cervantes
c7dd10e5ed packaging: Remove unused publish kata image script
This PR removes unused the publish kata image script which
was used on kata 1.x when we had OBS packages which are not
longer used on kata 2.x

Fixes #4496

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-06-20 14:43:39 +00:00
Amulyam24
0bbbe70687 snap: fix snap build on ppc64le
Fixes the syntax error while building rustdeps.

Fixes: #4494

Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
2022-06-20 19:26:27 +05:30
Fabiano Fidêncio
6fd40085ef
Merge pull request #4484 from cmaf/tracing-update-rootspan-name
tracing: Remove whitespace from root span
2022-06-20 08:37:45 +02:00
Fupan Li
98f041ed8e
Merge pull request #4486 from openanolis/runtime-rs-merge-main
runtime-rs: runtime-rs merge main
2022-06-20 13:52:14 +08:00
Bin Liu
2c1b68d6e4
Merge pull request #4481 from zvonkok/fix-action
workflow: Removing man-db, workflow kept failing
2022-06-20 11:10:48 +08:00
Chao Wu
86123f49f2 Merge branch 'main' into runtime-rs
In order to keep update with the main, we will update runtime-rs every
week.

Fixes: #4485
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2022-06-20 10:01:58 +08:00
Liang Zhou
ef925d40ce runtime: enable sandbox feature on qemu
Enable "-sandbox on" in qemu can introduce another protect layer
on the host, to make the secure container more secure.

The default option is disable because this feature may introduce some
performance cost, even though user can enable
/proc/sys/net/core/bpf_jit_enable to reduce the impact.

Fixes: #2266

Signed-off-by: Feng Wang <feng.wang@databricks.com>
2022-06-17 15:30:46 -07:00
Chelsea Mafrica
28995301b3 tracing: Remove whitespace from root span
Remove space from root span name to follow camel casing of other tracing
span names in the runtime and to make parsing easier in testing.

Fixes #4483

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2022-06-17 12:07:37 -07:00
Zvonko Kaiser
9941588c00 workflow: Removing man-db, workflow kept failing
Fixes: #4480

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2022-06-17 04:55:12 -07:00
wllenyj
e89e6507a4 dragonball: add signal handler
Used to register dragonball's signal handler.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
2022-06-16 17:31:58 +08:00
Fabiano Fidêncio
f30fe86dc1
Merge pull request #4456 from Bevisy/fixIssue4454
docs: Update outdated URLs and keep them available
2022-06-16 10:26:24 +02:00
Bin Liu
553ec46115
Merge pull request #4436 from alex-matei/fix/sandbox-mem-overflow
runtime: fix error when trying to parse sandbox sizing annotations
2022-06-16 11:18:24 +08:00
James O. D. Hunt
0d33b28802
Merge pull request #4459 from jodh-intel/snap-fix-cli-options
snap: Fix debug cli option
2022-06-15 17:10:15 +01:00
James O. D. Hunt
9766a285a4
Merge pull request #4422 from snir911/dependabot_bumps
deps: Resolve dependabot bumps of containerd, crossbeam-utils, regex
2022-06-15 15:57:53 +01:00
James O. D. Hunt
90a7763ac6 snap: Fix debug cli option
`snap`/`snapcraft` seems to have changed recently. Since `snap`
auto-updates all `snap` packages and since we use the `snapcraft` `snap`
for building snaps, this is impacting all our CI jobs which now show:

```
Installing Snapcraft for Linux…
snapcraft 7.0.4 from Canonical* installed

Run snapcraft -d snap --destructive-mode
Usage: snapcraft [options] command [args]...
Try 'snapcraft pack -h' for help.
Error: unrecognized arguments: -d
Error: Process completed with exit code 1.
```

Move the debug option to make it a sub-command (long) option to resolve
this issue.

Fixes: #4457.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-06-15 10:00:56 +01:00
James O. D. Hunt
d06dd8fcdc
Merge pull request #4312 from fidencio/topic/pass-the-tuntap-fd-to-clh
Allow Cloud Hypervisor to run under the `container_kvm_t`
2022-06-15 09:37:49 +01:00
Binbin Zhang
a305bafeef docs: Update outdated URLs and keep them available
By comparing the content of the old url and the new url,
ensure that their content is consistent and does not contain ambiguities

Fixes: #4454

Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
2022-06-15 16:34:28 +08:00
Archana Shinde
185360cb9a
Merge pull request #4452 from GabyCT/topic/updatedeveloperguide
docs: Update containerd url link
2022-06-14 16:13:35 -07:00
Chelsea Mafrica
db2a4d6cdf
Merge pull request #4441 from liubin/fix/refactor-reading-mountstat-log
agent: refactor reading file timing for debugging
2022-06-14 14:18:14 -07:00
Gabriela Cervantes
bee7703436 docs: Update containerd url link
This PR updates the containerd url link in the Developer Guide.

Fixes #4451

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-06-14 15:35:03 +00:00
Fabiano Fidêncio
ac5dbd8598 clh: Improve logging related to the net dev addition
Let's improve the log so we make it clear that we're only *actually*
adding the net device to the Cloud Hypervisor configuration when calling
our own version of VmAddNetPut().

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-14 10:53:09 +00:00
Fabiano Fidêncio
0b75522e1f network: Set queues to 1 to ensure we get the network fds
We want to have the file descriptors of the opened tuntap device to pass
them down to the VMMs, so the VMMs don't have to explicitly open a new
tuntap device themselves, as the `container_kvm_t` label does not allow
such a thing.

With this change we ensure that what's currently done when using QEMU as
the hypervisor, can be easily replicated with other VMMs, even if they
don't support multiqueue.

As a side effect of this, we need to close the received file descriptors
in the code of the VMMs which are not going to use them.

Fixes: #3533

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-14 10:53:09 +00:00
Fabiano Fidêncio
93b61e0f07 network: Add FFI_NO_PI to the netlink flags
Adding FFI_NO_PI to the netlink flags causes no harm to the supported
and tested hypervisors as when opening the device by its name Cloud
Hypervisor[0], Firecracker[1], and QEMU[2] do set the flag already.

However, when receiving the file descriptor of an opened tutap device
Cloud Hypervisor is not able to set the flag, leaving the guest without
connectivity.

To avoid such an issue, let's simply add the FFI_NO_PI flag to the
netlink flags and ensure, from our side, that the VMMs don't have to set
it on their side when dealing with an already opened tuntap device.

Note that there's a PR opened[3] just for testing that this change
doesn't cause any breakage.

[0]: e52175c2ab/net_util/src/tap.rs (L129)
[1]: b6d6f71213/src/devices/src/virtio/net/tap.rs (L126)
[2]: 3757b0d08b/net/tap-linux.c (L54)
[3]: https://github.com/kata-containers/kata-containers/pull/4292

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-14 10:53:09 +00:00
Fabiano Fidêncio
bf3ddc125d clh: Pass the tuntap fds down to Cloud Hypervisor
This is basically a no-op right now, as:
* netPair.TapInterface.VMFds is nil
* the tap name is still passed to Cloud Hypervisor, which is the Cloud
  Hypervisor's first choice when opening a tap device.

In the very near future we'll stop passing the tap name to Cloud
Hypervisor, and start passing the file descriptors of the opened tap
instead.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-14 10:53:09 +00:00
Fabiano Fidêncio
55ed32e924 clh: Take care of the VmAdNetdPut request ourselves
Knowing that VmAddNetPut works as expected, let's switch to manually
building the request and writing it to the appropriate socket.

By doing this it gives us more flexibility to, later on, pass the file
descriptor of the tuntap device to Cloud Hypervisor, as openAPI doesn't
support such operation (it has no notion of SCM Rights).

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-14 10:53:09 +00:00
Fabiano Fidêncio
01fe09a4ee clh: Hotplug the network devices
Instead of creating the VM with the network device already plugged in,
let's actually add the network device *after* the VM is created, but
*before* the Vm is actually booted.

Although it looks like it doesn't make any functional difference between
what's done in the past and what this commit introduces, this will be
used to workaround a limitation on OpenAPI when it comes to passing down
the network device's file descriptor to Cloud Hypervisor, so Cloud
Hypervisor can use it instead of opening the device by its name on the
VMM side.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-14 10:51:02 +00:00
Fabiano Fidêncio
2e07538334 clh: Expose VmAddNetPut
VmAddNetPut is the API provided by the Cloud Hypervisor client (auto
generated) code to hotplug a new network device to the VM.

Let's expose it now as it'll be used as part this series, mostly to
guide the reviewer through the process of what we have to do, as later
on, spoiler alert, it'll end up being removed.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-14 10:27:30 +00:00