Commit Graph

6963 Commits

Author SHA1 Message Date
Chelsea Mafrica
39cd05e0bb runtime: tracing: Use root context to stop tracing
Call StopTracing with s.rootCtx, which is the root context for tracing,
instead of s.ctx, which is parent to a subset of trace spans.

Fixes #2661

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2021-09-17 11:39:13 -07:00
Feng Wang
1cfe59304d runtime: Run QEMU using a non-root user/group
A random generated user/group is used to start QEMU VMM process.
The /dev/kvm group owner is also added to the QEMU process to grant it access.

Fixes #2444

Signed-off-by: Feng Wang <feng.wang@databricks.com>
2021-09-17 11:28:44 -07:00
wangyongchao.bj
fd98373850 runtime: update .gitignore file cleare the vc shim config
update .gitignore file, remove the follow configurations:
/virtcontainers/shim/mock/cc-shim/cc-shim
/virtcontainers/shim/mock/kata-shim/kata-shim
/virtcontainers/shim/mock/shim

Fixes: #2670

Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
2021-09-17 15:25:28 +08:00
Hui Zhu
fff82b4ef5
Merge pull request #2628 from bergwolf/runtime-reorg
runtime: refactor commandline code directory
2021-09-17 10:37:22 +08:00
Chelsea Mafrica
6159ef3499
Merge pull request #2626 from YchauWang/wyc-vc-api02
virtcontainers: update VC HypervisorConfig API add three lost fields
2021-09-16 16:46:27 -07:00
Peng Tao
067c44d0b6 runtime: fix UT build failure
storeContainer has been removed.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-09-16 19:42:02 +08:00
Jakob Naucke
9353cd77fd
runtime: Remove outdated TestStoreContainer
Due to #2332 being merged after running tests for #2604, and the latter
being merged now, a test for the now removed `storeContainer` was added.
Remove it.

Fixes: #2652
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-09-16 12:26:37 +02:00
Peng Tao
9a311a2b58 docs: fix invalid kernel dax doc url
And use a released version instead of the master branch so that it no
longer gets invalidated.

Depends-on: github.com/kata-containers/kata-containers#2645
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-09-16 17:19:18 +08:00
Peng Tao
e7c42fbc76 runtime: unify generated config
We don't need to maintain two generated config.go and even have
duplicates between them.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-09-16 17:19:18 +08:00
Peng Tao
4f7cc18622 runtime: refactor commandline code directory
Move all command line code to `cmd` and move containerd-shim-v2 to pkg.

Fixes: #2627
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-09-16 17:19:18 +08:00
Samuel Ortiz
7bf96d2457
Merge pull request #2604 from Amulyam24/container_tests
virtcontainers: add unit tests for container.go
2021-09-16 11:02:16 +02:00
Samuel Ortiz
9ed024e0bf
Merge pull request #2649 from likebreath/0916/clh_hugepages
runtime: clh: Enable hugepages support
2021-09-16 10:57:34 +02:00
David Gibson
b46adbc527
Merge pull request #2428 from dgibson/simplify-mount-storage
agent: Simplify mount point creation
2021-09-16 14:43:29 +10:00
David Gibson
9d3cd9841f agent/mount: Remove unused ensure_destination_exists()
The only remaining callers of ensure_destination_exists() are in its own
unit tests.  So, just remove it.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-16 12:24:47 +10:00
David Gibson
64aa562355 agent: Correct mount point creation
mount_storage() first makes sure the mount point for the storage volume
exists.  It uses fs::create_dir_all() in the case of 9p or virtiofs volumes
otherwise ensure_destination_exists().  But.. ensure_destination_exists()
boils down to an fs::create_dir_all() in most cases anyway.  The only case
it doesn't is for a bind fstype, where it creates a file instead of a
directory.  But, that's not correct anyway because we need to create either
a file or a directory depending on the source of the bind mount, which
ensure_destination_exists() doesn't know.

The 9p/virtiofs paths also check if the mountpoint exists before calling
fs::create_dir_all(), which is unnecessary (fs::create_dir_all already
handles that case).

mount_storage() does have the information to know what we need to create,
so have it explicitly call ensure_destination_file_exists() for the bind
mount to a non-directory case, and fs::create_dir_all() in all other cases.

fixes #2390

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-16 12:24:47 +10:00
David Gibson
08d7aebc28 agent/mount: Split out regular file case from ensure_destination_exists()
ensure_destination_exists() can create either a directory or a regular file
depending on the arguments.  This patch extracts the regular file specific
option into its own helper: ensure_destination_file_exists().  This:
 - Avoids doing some steps in the directory case (they're already handled
   by create_dir_all())
 - Enables some further future cleanups

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-16 12:24:47 +10:00
David Gibson
9fa3beff4f agent: Remove unnecessary BareMount structure
struct Baremount contains the information necessary to make a new mount.
As a datastructure, however, it's pointless, since every user just
constructs it, immediately calls the BareMount::mount() method then
discards the structure.

Simplify the code by making this a direct function call baremount().

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-16 12:24:47 +10:00
David Gibson
49282854f1 agent: Simplify BareMount::mount by using nix::mount::mount
BareMount::mount does some complicated marshalling and uses unsafe code to
call into the mount(2) system call.  However, we're already using the nix
crate which provides a more Rust-like wrapper for mount(2).  We're even
already using nix::mount::umount and nix::mount::MsFlags from the same
module.

In the same way, we can replace the direct usage of libc::umount() with
nix::mount::umount() in one of the tests.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-16 12:24:47 +10:00
David Gibson
bac849ecba
Merge pull request #2634 from dgibson/newer-rust
versions: Allow newer Rust versions
2021-09-16 12:23:37 +10:00
Bo Chen
d00decc97d runtime: clh: Enable hugepages support
This patch adds the configuration option that allows to use hugepages
with Cloud Hypervisor guests.

Fixes: #2648

Signed-off-by: Bo Chen <chen.bo@intel.com>
2021-09-15 10:43:57 -07:00
GabyCT
2a26c2397d
Merge pull request #2645 from dgibson/query-cpus
runtime/qemu: Move from query-cpus to query-cpus-fast
2021-09-15 10:35:03 -05:00
David Gibson
64bb803fcf runtime/qemu: Move from query-cpus to query-cpus-fast
We recently updated to using qemu-6.1 (from qemu 5.2).  Unfortunately one
breaking change in qemu 6.0 wasn't caught by the CI.

The query-cpus QMP command has been removed, replaced by query-cpus-fast
(which has been available since qemu 2.12).  govmm already had support for
query-cpus-fast, we just weren't using it, so the change is quite easy.

fixes #2643

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-15 16:41:26 +10:00
David Gibson
e7deee948a
Merge pull request #2502 from dgibson/qemu-6.1
Update Kata to use qemu-6.1
2021-09-15 11:06:14 +10:00
David Gibson
25ac3524c9 versions: Allow newer Rust versions
Rust 1.47.0 which is the latest we note as tested in versions.yaml is now
getting fairly old - many current distros have newer versions (e.g.
Rust 1.54.0 in Fedora 34).  Bring this more up to date.

Note that this is only updating the 'newest-version', not the minimum
required version.

The new version changes the name of the 'clippy::unknown_clipp_lints'
option to simply 'unknown_lints' so we need to change that as well to avoid
warnings.

fixes #2633

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-15 08:58:28 +10:00
Eric Ernst
e4cb6cbfbb
Merge pull request #2332 from sameo/topic/host-cgroups
Host cgroups improvements and simplifications
2021-09-14 09:09:10 -07:00
David Gibson
851d5f8613 tests: Correct heading in static checks test
The github static checks action has a section heading called "Building
rust".  It doesn't actually build rust, though, just installs it with
rustup.  Correct the misleading message.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-14 20:17:27 +10:00
Tim Zhang
842c76cb40
Merge pull request #2359 from teawater/swap_doc
Add doc for guest swap
2021-09-14 15:54:10 +08:00
Samuel Ortiz
4b7e4a4c70 runtime: Vendoring update
Due to the libcontainer dependencies removal.

Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
2021-09-14 07:09:34 +02:00
Samuel Ortiz
8d9d6e6af0 docs: Host cgroups documentation update
Update according to the new sandbox/overhead cgroup split.

Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
2021-09-14 07:09:34 +02:00
Samuel Ortiz
9bed2ade0f virtcontainers: Convert to the new cgroups package API
The new API is based on containerd's cgroups package.
With that conversion we can simpligy the virtcontainers sandbox code and
also uniformize our cgroups external API dependency. We now only depend
on containerd/cgroups for everything cgroups related.

Depends-on: github.com/kata-containers/tests#3805
Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2021-09-14 07:09:34 +02:00
Samuel Ortiz
b42ed39349 virtcontainers: cgroups: Add a containerd API based cgroups package
Eventually, we will convert the virtcontainers and the whole Kata
runtime code base to only rely on that package.

This will make Kata only depends on the simpler containerd cgroups API.

Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
2021-09-14 07:09:34 +02:00
Samuel Ortiz
f17752b0dc virtcontainers: container: Do not create and manage container host cgroups
The only process we are adding there is the container host one, and
there is no such thing anymore.

Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
2021-09-14 07:09:33 +02:00
Samuel Ortiz
dc7e9bce73 virtcontainers: sandbox: Host cgroups partitioning
This is a simplification of the host cgroup handling by partitioning the
host cgroups into 2: A sandbox cgroup and an overhead cgroup.

The sandbox cgroup is always created and initialized. The overhead
cgroup is only available when sandbox_cgroup_only is unset, and is
unconstrained on all controllers. The goal of having an overhead cgroup
is to be more flexible on how we manage a pod overhead. Having such
cgroup will allow for setting a fixed overhead per pod, for a subset of
controllers, while at the same time not having the pod being accounted
for those resources.

When sandbox_cgroup_only is not set, we move all non vCPU threads
to the overhead cgroup and let them run unconstrained. When it is set,
all pod related processes and threads will run in the sandbox cgroup.

Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
2021-09-14 07:09:29 +02:00
Samuel Ortiz
f811026c77 virtcontainers: Unconditionally create the sandbox cgroup manager
Regardless of the sandbox_cgroup_only setting, we create the sandbox
cgroup manager and set the sandbox cgroup path at the same time.

Without doing this, the hypervisor constraint routine is mostly a NOP as
the sandbox state cgroup path is not initialized.

Fixes #2184

Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
2021-09-14 07:05:57 +02:00
wangyongchao.bj
a6066404f7 virtcontainers: update VC HypervisorConfig API add three lost fields
Sync the virtcontainers api.md document, add `ConfidentialGuest` `EntropySourceList` `GuestSwap` three
 fields to the HypervisorConfig API.

Fixes #2625

Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
2021-09-14 10:42:54 +08:00
wangyongchao.bj
bb18cd475c virtcontainers: update VC SandboxConfig API add SandboxBindMounts field
sync the virtcontainers api.md document, add SandboxBindMounts field to the SandboxConfig API.
And update the order of the SandboxConfig API fields.

Fixes #2621

Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
2021-09-14 09:56:47 +08:00
Eric Ernst
967db0cbcc
Merge pull request #2544 from likebreath/0831/upgrade_clh_v18.0
versions: Upgrade to Cloud Hypervisor v18.0
2021-09-13 11:27:45 -07:00
Fabiano Fidêncio
9381f23ccf
Merge pull request #2613 from sameo/topic/runtime-readme
runtime: Fix README link
2021-09-13 17:44:56 +02:00
Binbin Zhang
58e77a3c13 sandbox: Allow the device to be accessed,such as /dev/null and /dev/urandom
If the device has no permission, such as /dev/null, /dev/urandom,
it needs to be added into cgroup.

Fixes: #2615

Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
2021-09-13 20:47:16 +08:00
Samuel Ortiz
057eb80ac9
Merge pull request #2596 from jongwu/qemu_mak
qemu: remove default config for arm64.
2021-09-13 11:23:35 +02:00
Samuel Ortiz
75ef8c243a
Merge pull request #2603 from Bevisy/main-2539
sandbox: Add device permissions such as /dev/null to cgroup
2021-09-13 11:04:51 +02:00
Samuel Ortiz
62a1a6f827
Merge pull request #2593 from nubificus/fix_fc_vcpu_thread
virtcontainers: fc: parse vcpuID correctly
2021-09-13 09:23:53 +02:00
Hui Zhu
d67a414b2b src/runtime/README.md: Fix URL of Licence
Fix URL of Licence of src/runtime/README.md.

Fixes: #2326

Signed-off-by: Hui Zhu <teawater@gmail.com>
2021-09-13 09:11:42 +08:00
Samuel Ortiz
13b8bb0c74 runtime: Fix README link
The LICENSE file lives in the project's root.

Fixes #2612

Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
2021-09-11 09:44:40 +02:00
David Gibson
25670d3058 packaging/qemu: Update qemu-exerimental version to v6.1.0
This brings it back into line with the normal qemu version.  We refer to
v6.1.0 by full SHA in versions.yaml, rather than the tag, so that
apply_patches.sh sees it as different and applies the virtiofs DAX patches
which is what the experimental version is actually about having.

The virtiofs DAX patches themselves are updated to the version from
https://gitlab.com/virtio-fs/qemu, virtio-fs-dev branch as of commit
3620cb0a.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-11 16:43:26 +10:00
David Gibson
041a513f80 versions: Update qemu to v6.1.0
We need qemu-6.1 for ACPI PCI hotplug support for the q35 machine.  At the
moment qemu will use SHPC hotplug under the PCIe to PCI bridge on q35.
SHPC is too slow to use for our purposes (it requires a 5s delay).

Update the qemu version to v6.1.0.  This leaves the experimental version
*older* than the normal version, but we'll fix that up later.

We also need to tweak the snapcraft.yaml, since the location for configs
has changed in the new qemu version.

fixes #1691

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-11 16:43:22 +10:00
Anastassios Nanos
62baa48ef5 virtcontainers: fc: parse vcpuID correctly
In getThreadIDs(), the cpuID variable is derived from a string that
already contains a whitespace. As a result, strings.SplitAfter returns
the cpuID with a leading space. This makes any go variant of string to int
fail (strconv.ParseInt() in our case). This patch makes sure that the
leading space character is removed so the string passed to
strconv.ParseInt() is "CPUID" and not " CPUID".

This has been caused by a change in the naming scheme of vcpu threads
for Firecracker after v0.19.1.

Fixes: #2592

Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
2021-09-10 09:39:56 +00:00
David Gibson
81de2d476b packaging: Correct error message in apply_patches.sh
If the script doesn't find a patches directory it expects, it gives an
error saying to create a dummy 'no_patches' file if you really don't want
any patches applied for that version.

But actual practice in the tree is to call the dummy file 'no_patches.txt'
rather than simply 'no_patches'.  Correct the message to match existing
practice.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-10 11:19:10 +10:00
Bo Chen
f785ff0bf2 virtcontainers: clh: Revert the workaround incorrect default values
Given the fix to the bugs of the openapi spec file is included in the
Cloud Hypervisor v18.0 [1], this patch reverts the workaround we carried
in the CLH driver.

This reverts commit 932ee41b3f.

[1] https://github.com/cloud-hypervisor/cloud-hypervisor/pull/3029

Signed-off-by: Bo Chen <chen.bo@intel.com>
2021-09-09 14:52:53 -07:00
Bo Chen
0e0e59dc5f virtcontainers: clh: Re-generate the client code
This patch re-generates the client code for Cloud Hypervisor v18.0.
Note: The client code of cloud-hypervisor's (CLH) OpenAPI is
automatically generated by openapi-generator [1-2].

[1] https://github.com/OpenAPITools/openapi-generator
[2] https://github.com/kata-containers/kata-containers/blob/main/src/runtime/virtcontainers/pkg/cloud-hypervisor/README.md

Signed-off-by: Bo Chen <chen.bo@intel.com>
2021-09-09 14:51:55 -07:00