As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to "match-like-matches-macro".
Let's fix them all here.
For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#match_like_matches_macro
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to unnecessary_cast.
Let's fix them all here.
For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_cast
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to needless_borrow.
Let's fix them all here.
For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the rust toolchain version bump to its 1.66.0 release raised a
warning about the type complexity used for the closure, and that's
something we don't want to change, let's ignore such warning in this
very specific case.
See:
https://rust-lang.github.io/rust-clippy/master/index.html#type_complexity
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to needless_borrow.
Let's fix them all here.
For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to unnecessary_cast.
Let's fix them all here.
For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_cast
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Substitution in the yq install script doesn't like zsh, and additionally
the version of yq we're using doesn't have a darwin/arm64 build so grab
the amd64 version and let rosetta work its magic.
Additionally swap to abspath from readlink -m for the printing of what binaries
to install, as the -m flag doesn't exist on the BSD variant, and this
should be the same behavior.
Fixes: #5970
Signed-off-by: Danny Canter <danny@dcantah.dev>
As we bumped the rust toolchain to 1.66.0, some new warnings have been
raised due to needless_borrow.
Let's fix them all here.
For more info about the warnings, please, take a look at:
https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Was about to change `urandomdev` to a constant when I realized it's
intentionally mutable so it can be mocked in tests. There's other
comments to the same effect so clarify here as well.
Fixes: #5965
Signed-off-by: Danny Canter <danny@dcantah.dev>
In order to avoid resource leak, we need to remove upcall client in vm
and vcpu manager when stopping vm.
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
We have to execute some hooks both in host and guest. And in
/libs/kata-sys-util/src/hooks.rs, the coomon operations are implemented.
In this commit, we are going to refactor the code of guest hooks using
code in /libs/kata-sys-util/src/hooks.rs. At the same time, we move
function valid_env to kata-sys-util to make it usable by both agent and
runtime.
Fixes: #5857
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
This commit will call `error_for_status` after `send`, this call
will generate errors if status code between 400-499 and 500-599.
And sometime access github.com will fail, in this case we can
skip the test to prevent the CI failing.
Fixes: #5948
Signed-off-by: Bin Liu <bin@hyper.sh>
kata-ctl depends on runtime-rs, and this commit:
fbf294da3f
added a new dependency named shim-interface, this Cargo.lock should be updated too.
Signed-off-by: Bin Liu <bin@hyper.sh>
The current check framwork is specific for x86. Refactor the code
to let it arch-agnostic.
Fixes: #5923
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
In order to avoid cloning, changed the signature of
`ShareFsMount::share_rootfs`, `ShareFsMount::share_volume`, and
`ShareFsMount::umount_rootfs` to receive a reference to a config.
Fixes: #5898
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Fixed issues where shared volumes couldn't umount correctly.
The rootfs of each container is cleaned up after the container is killed, except
for `NydusRootfs`. `ShareFsRootfs::cleanup()` calls
`VirtiofsShareMount::umount_rootfs()` to umount mount points shared to the
guest, and umounts the bundle rootfs.
Fixes: #5898
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
If the serial path is given, legacy_manager should create socket console
based on that path. Or the console should be created based on stdio.
Fixes: #5914
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
enable start container from bundle in this way
$ ls ./bundle
config.json rootfs
$ sudo ctr run -d --runtime io.containerd.kata.v2 --config bundle/config.json test_kata
Fixes:#5872
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Fixed formatting for check release options
Fixes: #5345
Signed-off-by: Alex <alee23@bu.edu>
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
The client side is moved to libs. This is to solve the problem
that including clients will bring about messy dependencies.
Fixes: #5874
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
After building the binary as usual with `cargo build` run it as follows.
It needs a configuration.toml in which only qemu keys `path`, `kernel`
and `initrd` will initially need to be set. Point them to respective
files e.g. from a kata distribution tarball.
It also needs to be launched from an exported container bundle
directory. One can be created by running
mkdir rootfs
podman export $(podman create busybox) | tar -C ./rootfs -xvf -
runc spec -b .
in a suitable directory.
Then launch the program like this:
KATA_CONF_FILE=/path/to/configuration-qemu.toml /path/to/shim-ctl
Fixes: #5817
Signed-off-by: Pavel Mores <pmores@redhat.com>
This does almost literally nothing so far apart from getting and setting
HypervisorConfig. It's mostly copied from/inspired by dragonball.
Signed-off-by: Pavel Mores <pmores@redhat.com>
DEFAULT_REGISTRY pre-registers many metrics that we don't need or have duplicated.
This PR uses a custom register for metrics without interference and ensures that
the registration process is executed only once when the program is running.
Fixes: #5255
Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
None of the host namespace paths make sense in the guest. Let's clear
them all before sending the spec to the agent.
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
We should test is_pid_namespace_enabled before amending the container
spec, where the pid namespace path is cleared and resulting
sandbox_pidns to always being false.
Fixes: #5881
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Strings in Rust don't have \0 at the end, but C does, which leads to `umount2`
in the libc can't get the correct path. Besides, calling `nix::mount::umount2`
to avoid using an unsafe block is a robust solution.
Fixes: #5871
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Standalone share fs should add virtiofs device in setup_device_before_start_vm
and return the storages to mount the directory in guest. And it uses
hypervisor's jailer root directly instead of jail config.
Besides, we tweaked the parameter, so it adapts to rust version virtiofsd
now. And its cache policy which forbids caching is "never" now, instead of
"none". Hence, we change the default cache mode.
Fixes: #5655
Signed-off-by: Yipeng Yin <yinyipeng@bytedance.com>
Cgroup manager for a container will always be created.
Thus, dropping the option for LinuxContainer.cgroup_manager
is feasible and could simplify the code.
Fixes: #5778
Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
Use pidfd_open and poll on newer versions of Linux to wait
for the process to exit. For older versions use existing wait logic
Fixes: #5617
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Fixed the issue when using nonblocking, the `tokio::io::copy()` needing
to handle EAGAIN, resulting in high CPU usage.
Fixes: #5740
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
Removed the `Debug` trait for the `ShareFs` and etc. Renamed
`ShareFsMount::upgrade()` and `ShareFsMount::downgrade()` to
`upgrade_to_rw()` and `downgrade_to_ro()`. Protected `mounted_info_set`
with a mutex to avoid race conditions.
Fixes: #5588
Signed-off-by: Xuewei Niu <justxuewei@apache.org>
This commit implemented umonut controls and permission controls. When a volume
is no longer referenced, it will be umounted immediately. When a volume mounted
with readonly permission and a new coming container needs readwrite permission,
the volume should be upgraded to readwrite permission. On the contrary, if a
volume with readwrite permission and no container needs readwrite, then the
volume should be downgraded.
Fixes: #5588
Signed-off-by: Xuewei Niu <justxuewei@apache.org>
Implemented bind mount related managment on the sandbox side, involving bind
mount a volume if it's not mounted before, upgrade permission to readwrite if
there is a new container needs.
Fixes: #5588
Signed-off-by: Xuewei Niu <justxuewei@apache.org>
Also added crate `runtime-rs/crates/runtimes` as dependency as it's
immediately depended upon by the `direct-volume` feature, see issue
5341 and PR 5467.
Fixes#5810
Signed-off-by: Tingzhou Yuan <tzyuan15@bu.edu>
Now we are supporting two runtime/shim, the go version,
and the rust version, for debug purposes, we can
add an identification in the version info
to tell us which runtime/shim is used.
Fixes: #5806
Signed-off-by: Bin Liu <bin@hyper.sh>
Pass SELinux policy for containers to the agent if `disable_guest_selinux`
is set to `false` in the runtime configuration. The `container_t` type
is applied to the container process inside the guest by default.
Users can also set a custom SELinux policy to the container process using
`guest_selinux_label` in the runtime configuration. This will be an
alternative configuration of Kubernetes' security context for SELinux
because users cannot specify the policy in Kata through Kubernetes's security
context. To apply SELinux policy to the container, the guest rootfs must
be CentOS that is created and built with `SELINUX=yes`.
Fixes: #4812
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
The kata-agent supports SELinux for containers inside the guest
to comply with the OCI runtime specification.
Fixes: #4812
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
This commit re-implements `start` operation by leveraging the agent codes.
Currently, `runk` has own `start` mechanism even if the agent already
has the feature to handle starting a container. This worsen the maintainability
and `runk` cannot keep up with the changes on the agent side easily.
Hence, `runk` replaces own implementations with agent's ones.
Fixes: #5648
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
For now, we can check if host support running kata by check if "/dev/kvm"
exist on aarch64.
Fixes: #5768
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
When using source code to compile runtime-rs,make the
documentation point out the detailed environment build
and compilation methods to avoid errors caused by related
dependent packages.
Fixes:#5757
Signed-off-by: Chen Taotao <chentt10@chinatelecom.cn>
The displayed commit message and version message are partially duplicated.
Remove the version number from the commit display message.
Fixes:#5735
Signed-off-by: Chen Taotao <chentt10@chinatelecom.cn>
Some rootfs put iptables-save and iptables-restore
under /usr/sbin instead of /sbin. This pr checks both
and returns the one exist.
Fixes: #5608
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
We have starting to use golang 1.19, some features are
not supported later, so run `go fix` to fix them.
Fixes: #5750
Signed-off-by: Bin Liu <bin@hyper.sh>
Use MkdirAll instead of Mkdir so it doesn't generate an
error when the folder is created by another process
Fixes#5713
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
As the increase of the I/O intensive tasks, two issues could be caused:
1. When the future is blocked, the current thread (which is in the network namespace)
might be take over by other tasks. After the future is finished, the thread take over
the current task might not be in the pod network namespace
2. When finish setting up the network, the current thread will be set back to the host namsapce.
But the task which be taken over would still stay in the pod network namespace
To avoid that, we need to block the future on the current thread.
Fixes:#5728
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
When the user tried to add new devices to the VM, there is no error info for the invalid
device. This PR adds a log record to the `appendDevices` for the invalid device of the
qemu config.
Fixes: #5719
Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
Let's follow the binary bump used in the CI and also bump the vendored
version of containerd to v1.6.8.
Fixes: #5722
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Rename old VsockConfig to HybridVsockConfig. And add VsockConfig to
support vhost-vsock. We follow kata's old way to try random vhost fd
for 50 times to generate uniqe fd.
Fixes: #5654
Signed-off-by: Yipeng Yin <yinyipeng@bytedance.com>
test_ip_tables test depends on iptables tools. But we can't
ensure these tools are exist. it's better to skip the test
if there is no such tools.
Fixes: #5697
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
The default vhost-user-fs queue-size of qemu is 128 now. Set it to 1024
by default which is same as clh. Also make this value configurable.
Fixes: #5694
Signed-off-by: liyuxuan.darfux <liyuxuan.darfux@bytedance.com>
Add functionality to get kernel version and related unit tests.
This is intended to be used in the kata-env command going forward.
Fixes: #5688
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This patch re-generates the client code for Cloud Hypervisor v28.0.
Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by openapi-generator.
Fixes: #5683
Signed-off-by: Bo Chen <chen.bo@intel.com>
```
14:13:15 parse.go:306:5: S1009: should omit nil check; len() for github.com/kata-containers/kata-containers/src/tools/log-parser.kvPairs is defined as zero (gosimple)
14:13:15 if pairs == nil || len(pairs) == 0 {
14:13:15 ^
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It seems that bumping the version of golang and golangci-lint new format
changes are required.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The package has been deprecated as part of 1.16 and the same
functionality is now provided by either the io or the os package.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
So that we get the latest language fixes.
There is little use to maitain compiler backward compatibility.
Let's just set the default golang version to the latest 1.19.2.
Fixes: #5494
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Since the upstream rust-vmm is changing its dependency style towards
caret requirements in these days (more information:
rust-vmm/vm-memory#199) and it breaks Dragonball compilation frequently.
rust-vmm is expected to finish the changes this week and in order to not
break Kata CI due to Dragonball's compilation error, we will add
Cargo.lock file into /src/dragonball first and remove it later when
rust-vmm is stable.
fixes: #5657
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Through proactively checking if Cloud Hypervisor process is dead,
this patch provides a faster path for isClhRunning
Fixes: #5623
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Use atomic operations instead of acquiring a mutex in isClhRunning.
This stops isClhRunning from generating a deadlock by trying to
reacquire an already-acquired lock when called via StopVM->terminate.
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Avoid executing StopVM concurrently when virtiofs dies as a result of clh
being stopped in StopVM.
Fixes: #5622
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Since the upstream of vmm-sys-utils upgraded to 0.11.0, some crates
automatically upgrade to v0.11.0, and some stay at v0.10.0 ( depending
on how they write version dependency in Cargo toml` which causes the
compile error in runtime-rs.
In order to fix this problem, we need to upgrade all vmm-sys-util
dependencies in runtime-rs to v0.11.0.
fixes: #5636
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Support the handlers in runtime, which are used by kata-ctl iptables series of commands in runtime.
Fixes: #5370
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
The cargo patch in the cargo.toml seems to cause the whole runtime-rs
building time longer and also makes it harder to build runtime-rs in an
environment without the network
We should delete all patches from the cargo.toml file and publish all
the crates that was once patched.
fixes: #5614#5527#5526#5449
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
In the current Dragonball code, mem_file_path config is not used when
hugetlbfs is enabled.
In this commit we add mem_file_path into hugetlbfs enable process.
fixes: #5566
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
1. be able to check does hypervisor support use block device, block
device hotplug, multi-queue, and share file
2. be able to set the hypervisor capability of using block device, block
device hotplug, multi-queue, and share file
Fixes: #5569
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
The 'config' argument to ShareVirtioFsStandalone::new() is now actually
used, taking care of an explicit TODO.
If a shared path doesn't exist in ShareVirtioFsStandalone::virtiofsd_args()
it is now created instead of returning an error, thus following
ShareVirtioFsInline's suit.
The '-o vhost_user_socket=...' command line argument doesn't seem to be
supported by newer versions of virtiofsd so we replace it with
'--socket-path' which should be functionally equivalent according to docs.
Fixes#5572
Signed-off-by: Pavel Mores <pmores@redhat.com>
The libseccomp crate was upgraded to v0.3.0 by 4696ead,
but `Cargo.lock` of runk wasn't updated by mistake.
So, this commit updates `Cargo.lock` of runk to the latest dependencies.
Fixes: #5487
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Ignore an error handling that is triggered when the kill command is called
with `--all option` to the stopped container.
High-level container runtimes such as containerd call the kill command with
`--all` option in order to terminate all processes inside the container
even if the container already is stopped. Hence, a low-level runtime
should allow `kill --all` regardless of the container state like runc.
This commit reverts to the previous behavior.
Fixes: #5555
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Fix the issue where share volumes always have readwrite permission even if
readonly permission is enough.
Fixes: #5549
Signed-off-by: Xuewei Niu <justxuewei@apache.org>
In the documentation test, the name shim has multiple potential
sources of import, now give it a clear source.
Fixes: #5535
Signed-off-by: Chen TaoTao <chentt10@chinatelecom.cn>
This function is meant to be used before operations
such as accessing network to make sure those operations
are not performed as a privilged user.
Fixes: #5331
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
1. Implemented a rust module for operating cgroups through systemd with the help of zbus (src/agent/rustjail/src/cgroups/systemd).
2. Add support for optional cgroup configuration through fs and systemd at agent (src/agent/rustjail/src/container.rs).
3. Described the usage and supported properties of the agent systemd cgroup (docs/design/agent-systemd-cgroup.md).
Fixes: #4336
Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
An API change, done a long time ago, has been exposed on Cloud
Hypervisor and we should update it on the Kata Containers side to ensure
it doesn't affect Cloud Hypervisor CI and because the change is needed
for an upcoming work to get QAT working with Cloud Hypervisor.
Fixes: #5492
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Sometimes we will face EEXIST error when adding arp neighbour.
Using NLM_F_REPLACE replace NLM_F_EXCL will avoid fail if the
entry exists.
See https://man7.org/linux/man-pages/man7/netlink.7.htmlFixes: #4895
Signed-off-by: Bin Liu <bin@hyper.sh>
Add a basic s390x cpu check for the "sie" feature to be present.
Also re-enable cpu check testing.
Fixes: #5438
Signed-off-by: Hendrik Brueckner <brueckner@linux.ibm.com>
For s390x, use native-tls for reqwest because the rustls-tls/ring
dependency is not available for s390x.
Also exclude s390x, powerpc64le, and aarch64 from running the cpu
check due to the lack of the arch-specific implementation. In this
case, rust complains about unused functions in src/check.rs (both
normal and test context).
Fixes: #5438
Co-authored-by: James O. D. Hunt <james.o.hunt@intel.com>
Signed-off-by: Hendrik Brueckner <brueckner@linux.ibm.com>
before setting a limit, otherwise paths may not be found.
guest supporting different hugepage size is more likely with peer-pods where
podvm may use different flavor.
Fixes: #5191
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
In runtime-rs makefile, we use
```
```
to let make help print out help information for variables and targets,
but later commits forgot this rule.
So we need to follow the previous rule and change the current comments.
fixes: #5413
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
s390x apparently does not support rust-tls, which is required by the
network check (due to the `reqwest` crate dependency).
Disable the network check on s390x until we can find a solution to the
problem.
> **Note:**
>
> This fix is assumed to be a temporary one until we find a solution.
> Hence, I have not moved the network check code (which should be entirely
> generic) into an architecture specific module.
Fixes: #5435.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Rework the architecture-specific `check()` call by moving all the
conditional logic out of the function.
Fixes: #5402.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This change improves the command descriptions for kata-ctl and can avoid certain confusions in command functionality.
Fixes#5411
Signed-off-by: Tingzhou Yuan <tzyuan15@bu.edu>
Make this file conform to the standard rust layout conventions and
simplify the code as recommended by `clippy`.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Build using the rust TLS implementation rather than the system ones.
This resolves the `reqwest` crate build failure: it doesn't appear to
build against the native libssl libraries due to Kata defaulting to
using the musl libc.
Fixes: #5387.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This kata-ctl argument returns the latest stable Kata
release by hitting github.com.
Adds check-version unit tests.
Fixes: #11
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Switch from the functional version of `clap` to the declarative
methodology.
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Commit-edited-by: James O. D. Hunt <james.o.hunt@intel.com>
Add architecture-specific code for x86_64 and generic calls handling
checks for CPU flags and attributes.
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Changed the `panic!()` call to a `compile_error!()` one to ensure it
fires at compile time rather than runtime.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add framework for different architectures for check. In the existing
kata-runtime check, the network checks do not appear to be
architecture-specific while the kernel module, cpu, and kvm checks do
have separate implementations for different architectures.
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Add more unit tests cases to --version argument.
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Commit-edited-by: James O. D. Hunt <james.o.hunt@intel.com>
As we're switching to using the rust version of the kata-ctl, lets
provide with its own entry in the kata-ctl command line.
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Commit-edited-by: James O. D. Hunt <james.o.hunt@intel.com>
handle_events for EventManager doesn't take max_events as arguments, so
we need to update the comments for it.
p.s. max_events is defined when initializing the EventManager.
fixes: #5382
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
With default_maxvcpus = 0 and default_vcpus = 1 settings, the
default_vcpus will be set to 0 and leads to starting fail.
The default_maxvcpus is not set correctly when it is set to 0,
and the default_vcpus is set to 0.
The correct action is setting default_maxvcpus to the max number
of CPUs or MAX_DRAGONBALL_VCPUS, and the default_vcpus should be
set to the desired value if the valuse is between 0 and
default_maxvcpus.
Fixes: #5110
Signed-off-by: Bin Liu <bin@hyper.sh>
ESRCH usually means the process has exited. In this case,
the execution should continue to kill remaining container processes.
Fixes: #5366
Signed-off-by: Feng Wang <feng.wang@databricks.com>
[Fix up cargo updates]
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Currently ACRN hypervisor support in Kata2.x releases is broken.
This commit re-enables ACRN hypervisor support and also refactors
the code so as to remove dependency on Sandbox.
Fixes#3027
Signed-off-by: Vijay Dhanraj <vijay.dhanraj@intel.com>
The kata agent adds a reference for each storage object before mount
and skip mount again if the storage object is known. We need to
remove the object reference if mount fails.
Fixes: #5364
Signed-off-by: Feng Wang <feng.wang@databricks.com>
The containerd stats method and metrics API are broken with Kata 2.5.x, the stats fail to load and the metrics API responds with status code 500
This seems to be down to the conversion from the stats reported by the agent RPC `StatsContainer` where the field `Pagesize` is not
completed by the `setHugetlbStats` method. In the case where multiple sized tables stats are reported, this causes containerd to register two metrics
with the same label set, rather than each being partitioned by the `page` label.
Fixes: #5316
Signed-off-by: Champ-Goblem <cameron@northflank.com>
The new way to boot from TDX firmware (e.g. td-shim) is using the
combination of '--platform tdx=on' with '--firmware tdshim'.
Fixes: #5309
Signed-off-by: Bo Chen <chen.bo@intel.com>
`kernel_irqchip` option doesn't seem to bring any benefits and, on the
contrary, its usage cause issues when using the microvm machine type.
With this in mind, let's remove it.
Fixes: #1984, #4386
Signed-off-by: norbjd <norbjd@users.noreply.github.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add support for ephemeral storage and k8s emptydir.
Depends-on:github.com/kata-containers/tests#5161
Fixes: #4730
Signed-off-by: Bin Liu <bin@hyper.sh>
Cmdline struct update in the latest linux-loader lib and its as_str
method is changed to as_cstring, thus we need fix it according whereas
the old as_str method is used.
Fixes: #5287
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
There are two duplicated mentioning of the rust libraries in README.md.
Let's just remove them all as the section is intended to list out core
Kata components rather than general libraries.
Fixes: #5275
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Since linux loader is updated in the Dragonball and the api for Cmdline
has been changed ( as_str() changed to as_cstring() ), we need to update
unit test in Dragonball.
fixes: #5277
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Add close_io to shim and call agent's close_stdin in close_io.
Depends-on:github.com/kata-containers/tests#5155
Fixes: #4729
Signed-off-by: Bin Liu <bin@hyper.sh>
Since linux-loader 0.4.0 and 0.5.0 is yanked due to null terminator bug,
we need to update linux-loader to 0.6.0.
And as_str() function should also be changed.
fixes: #5253
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
For stream RPCs:
- write_stdin
- read_stdout
- read_stderr
there should be no timeout (by setting it to 0).
Fixes: #5249
Signed-off-by: Bin Liu <bin@hyper.sh>
The qmp command of hotplug cpu failed error was hidden. It didn't friendly for
the user tracing the hotplug cpu error. The PR help us to improve the hotplug
cpu error log. Add real qemu command error log for `failed to hot add vCPUs`.
Through the error message, we can get the reason of the failed qmp command
for hotplug cpu operation.
Fixes: #5234
Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
In src/runtime-rs/crates/hypervisor/src/device/vfio.rs,
the path of new_id is defined as a const, but unbind is used
as a local variable, they should be unified to const.
Fixes: #5189
Signed-off-by: Bin Liu <bin@hyper.sh>
This is based on a patch from @niteeshkd that adds a config
parameter to choose between AMD SEV and SEV-SNP VMs as the
confidential guest type in case both types are supported. SEV is
the default.
Signed-off-by: Joana Pecholt <joana.pecholt@aisec.fraunhofer.de>
This commit adds AMD SEV-SNP as a confidential guest option to the
runtime. Information on required components such as OVMF, QEMU and
a kernel supporting SEV-SNP are defined in the versions file and
corresponding configs are added.
Note: The CPU model 'host' provided by the current SNP-QEMU does
not support all SNP capabilities yet, which is why this option is
changed to EPYC-v4.
Note: The guest's physical address space reduction specified with
ReducedPhysBits is 1. Details are can be found in Section 15.34.6
here https://www.amd.com/system/files/TechDocs/24593.pdfFixes#4437
Signed-off-by: Joana Pecholt <joana.pecholt@aisec.fraunhofer.de>
Adds initrd configuration option to the configuration.toml that is
generated for the setup using QEMU.
Signed-off-by: Joana Pecholt <joana.pecholt@aisec.fraunhofer.de>
Some device types have the same definition, they can be implemented
by macro to reduce code.
And this commit also deleted the `peer_name` field of the structs that
is never been used.
Fixes: #5170
Signed-off-by: Bin Liu <bin@hyper.sh>
The user name will be used to delete the user instead of relying on
uid lookup because uid can be reused.
Fixes: #5155
Signed-off-by: Feng Wang <feng.wang@databricks.com>
Enable seccomp support in `runk` by default.
Due to this, `runk` is built with `gnu libc` by default
because the building `runk` with statically linked the `libseccomp`
and `musl` requires additional configurations.
Also, general container runtimes are built with `gnu libc` as
dynamically linked binaries by default.
The user can disable seccomp by `make SECCOMP=no`.
Fixes: #4896
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Let client side support timeout if the timeout value is set.
If timeout not set, execute directly.
Fixes: #5114
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
"inline-virtio-fs" is newly supported by kata 3.0 as a "shared_fs" type,
it should be described in configuration file.
"inline-virtio-fs" is the same as "virtio-fs", but it is running in
the same process of shim, does not need an external virtiofsd process.
Fixes: #5102
Signed-off-by: Bin Liu <bin@hyper.sh>
Add client side function(public), to establish http connections (PUT,
POST, GET) to the long standing shim mgmt server.
Fixes: #5114
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
Add agent-url to its handler. The general framework of registering URL
handlers is done.
Fixes: #5114
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
Add shim management http server and boot it as a light-weight thread
when the sandbox is created.
Fixes: #5114
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
period should have a type of u64, and quota should be i64, the
function of getting CPU period and quota from annotations should
use the same data type as function return type.
Fixes: #5100
Signed-off-by: Bin Liu <bin@hyper.sh>
Kata 3.0 introduced 3 new configurations under runtime section:
name="virt_container"
hypervisor_name="dragonball"
agent_name="kata"
Blank values will lead to starting to fail.
Adding default values will make user easy to migrate to kata 3.0.
Fixes: #5098
Signed-off-by: Bin Liu <bin@hyper.sh>
Refactor the container builder code (`InitContainer` and `ActivatedContainer`)
to make it easier to understand and to maintain.
The details:
1. Separate the existing `builder.rs` into an `init_builder.rs` and
`activated_builder.rs` to make them easy to read and maintain.
2. Move the `create_linux_container` function from the `builder.rs` to
`container.rs` because it is shared by the both files.
3. Some validation functions such as `validate_spec` from `builder.rs`
to `utils.rs` because they will be also used by other components as
utilities in the future.
Fixes: #5033
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
In some case the call of cleanup from shim to service manager will fail,
and the shim process will continue to running, that will make process leak.
This commit will force shutdown the shim process in case of any errors in
service crate.
Fixes: #5087
Signed-off-by: Bin Liu <bin@hyper.sh>
Following the instructions in guidance doc will result in the ECONNREFUSED,
thus we need to keep the unix socket address in the two commands consistent.
Fixes: #5085
Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
amend_spec do two works:
- modify the spec
- check if the pid namespace is enabled
This make it confusable. So split it into two functions.
Fixes: #5062
Signed-off-by: Bin Liu <bin@hyper.sh>
Augment the mock hypervisor so that we can validate that ACPI memory hotplug
is carried out as expected.
We'll augment the number of memory slots in the hypervisor config each
time the memory of the hypervisor is changed. In this way we can ensure
that large memory hotplugs are broken up into appropriately sized
pieces in the unit test.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
If we're using ACPI hotplug for memory, there's a limitation on the
amount of memory which can be hotplugged at a single time.
During hotplug, we'll allocate memory for the memmap for each page,
resulting in a 64 byte per 4KiB page allocation. As an example, hotplugging 12GiB
of memory requires ~192 MiB of *free* memory, which is about the limit
we should expect for an idle 256 MiB guest (conservative heuristic of 75%
of provided memory).
From experimentation, at pod creation time we can reliably add 48 times
what is provided to the guest. (a factor of 48 results in using 75% of
provided memory for hotplug). Using prior example of a guest with 256Mi
RAM, 256 Mi * 48 = 12 Gi; 12GiB is upper end of what we should expect
can be hotplugged successfully into the guest.
Note: It isn't expected that we'll need to hotplug large amounts of RAM
after workloads have already started -- container additions are expected
to occur first in pod lifecycle. Based on this, we expect that provided
memory should be freely available for hotplug.
If virtio-mem is being utilized, there isn't such a limitation - we can
hotplug the max allowed memory at a single time.
Fixes: #4847
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
It'll be useful to get the total memory provided to the guest
(hotplugged + coldplugged). We'll use this information when calcualting
how much memory we can add at a time when utilizing ACPI hotplug.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Updates versions of crossbeam-channel because 0.52.0 is a yanked package
(creators mark version as not for release except as a dependency for
another package)
Updates chrono to use >0.42.0 to avoid:
https://rustsec.org/advisories/RUSTSEC-2020-0159
Updates lz4-sys.
Signed-off-by: Derek Lee <derlee@redhat.com>
Adds oci under the src/libs workplace.
oci shares a Cargo.lock file with the rest of src/libs but was not
listed as a member of the workspace.
There is no clear reason why it is not included in the workspace, so
adding it so cargo-deny stop complaining
Signed-off-by: Derek Lee <derlee@redhat.com>
One of the checks done by cargo-deny is ensuring all crates have a valid
license. As the rust programs import each other, cargo.toml files
without licenses trigger the check. While I could disable this check
this would be bad practice.
This adds an Apache-2.0 license in the Cargo.toml files.
Some of these files already had a header comment saying it is an Apache
license. As the entire project itself is under an Apache-2.0 license, I
assumed all individual components would also be covered under that
license.
Signed-off-by: Derek Lee <derlee@redhat.com>
Somewhere is lack of log info, add more details about
the storage and log when error will help understand
what happened.
Fixes: #4962
Signed-off-by: Bin Liu <bin@hyper.sh>
Users can specify the kernel module to be loaded through the agent
configuration in kata configuration file or in pod anotation file.
And information of those modules will be sent to kata agent when
sandbox is created.
Fixes: #4894
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
With the current TDX kernel used with Kata Containers, `tdx_guest` is
not needed, as TDX_GUEST is now a kernel configuration.
With this in mind, let's just drop the kernel parameter.
Fixes: #4981
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As right now the TDX guest kernel doesn't support "serial" console,
let's switch to using HVC in this case.
Fixes: #4980
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The runtime will crash when trying to resize memory when memory hotplug
is not allowed.
This happens because we cannot simply set the hotplug amount to zero,
leading is to not set memory hotplug at all, and later then trying to
access the value of a nil pointer.
Fixes: #4979
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
While doing tests using `ctr`, I've noticed that I've been hitting those
timeouts more frequently than expected.
Till we find the root cause of the issue (which is *not* in the Kata
Containers), let's increase the timeouts when dealing with a
Confidential Guest.
Fixes: #4978
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add cli message for init command to tell the user
not to run this command directly.
Fixes: #4367
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
When booting the TDX kernel with `tdx_disable_filter`, as it's been done
for QEMU, VirtioFS can work without any issues.
Whether this will be part of the upstream kernel or not is a different
story, but it easily could make it there as Cloud Hypervisor relies on
the VIRTIO_F_IOMMU_PLATFORM feature, which forces the guest to use the
DMA API, making these devices compatible with TDX.
See Sebastien Boeuf's explanation of this in the
3c973fa7ce208e7113f69424b7574b83f584885d commit:
"""
By using DMA API, the guest triggers the TDX codepath to share some of
the guest memory, in particular the virtqueues and associated buffers so
that the VMM and vhost-user backends/processes can access this memory.
"""
Fixes: #4977
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Move delete logic to `libcontainer` crate to make the code clean
like other commands.
Fixes: #4975
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Let's swith to depending on podman which also simplies indirect
dependency on kubernetes components. And it helps to avoid cri-o
security issues like CVE-2022-1708 as well.
Fixes: #4972
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
So that we bump several indirect dependencies like crossbeam-channel,
crossbeam-utils to bring in fixes to known security issues like CVE-2020-15254.
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Replaces instances of anyhow!(nix::Error::EINVAL) with other messages to
make it easier to debug.
Fixes#954
Signed-off-by: Derek Lee <derlee@redhat.com>
To make cgroup v1 and v2 works well, I use `cgroups::cgroup` in
`Container` to manager cgroup now. `CgroupManager` in rustjail has some
drawbacks. Frist, methods in Manager traits are not visiable. So we need
to modify rustjail and make them public. Second, CgrupManager.cgroup is
private too, and it can't be serialized. We can't load/save it in
status file. One solution is adding getter/setter in rustjail, then
create `cgroup` and set it when loading status. In order to keep the
modifications to a minimum in rustjail, I use `cgroups::cgroup`
directly. Now it can work on cgroup v1 or v2, since cgroup-rs do this
stuff.
Fixes: #4364#4821
Signed-off-by: Chen Yiyang <cyyzero@qq.com>
To plug drop-in support into existing config-loading code in a robust
way, more specifically to create a single point where this needs to be
handled, load_from_file() and load_raw_from_file() were refactored.
Seeing as the original implemenations of both functions were identical
apart from adjust_config() calls in load_from_file(), load_from_file()
was reimplemented in terms of load_raw_from_file().
Fixes #4771
Signed-off-by: Pavel Mores <pmores@redhat.com>
The central function being added here is load() which takes a path to a
base config file and uses it to load the base config file itself, find
the corresponding drop-in directory (get_dropin_dir_path()), iterate
through its contents (update_from_dropins()) and load each drop-in in
turn and merge its contents with the base file (update_from_dropin()).
Also added is a test of load() which mirrors the corresponding test in
the golang runtime (TestLoadDropInConfiguration() in config_test.go).
Signed-off-by: Pavel Mores <pmores@redhat.com>
This is the core functionality of merging config file fragments into the
base config file. Our TOML parser crate doesn't seem to allow working
at the level of TomlConfig instances like BurntSushi, used in the Golang
runtime, does so we implement the required functionality at the level of
toml::Value trees.
Tests to verify basic requirements are included. Values set by a base
config file and not touched by a subsequent drop-in should be preserved.
Drop-in config file fragments should be able to change values set by the
base config file and add settings not present in the base. Conversion
of a merged tree into a mock TomlConfig-style structure is tested as
well.
Signed-off-by: Pavel Mores <pmores@redhat.com>
We are not spinning up any L2 guests in vm factory, so the L1 guest
migration is expected to work even with VMX.
See https://www.linux-kvm.org/page/Nested_GuestsFixes: #4050
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
More and more Rust code is introduced, the test utils original in agent
should be made easy to share, move it into a new crate will make it
easy to share between different crates.
Fixes: #4925
Signed-off-by: Bin Liu <bin@hyper.sh>
vergen is a build dependency, but it is not being used.
we are processing ver/commit hash by make command, but not by vergen.
Fixes: #4920
Signed-off-by: Bin Liu <bin@hyper.sh>
Static resource management should be default to false. If default to be
true, later update sandbox operation, e.g. resize, will not work.
Fixes: #4742
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
The root span should exist the duration of the trace. Defer ending span
until the end of the trace instead of end of function. Add the span to
the service struct to do so.
Fixes#4902
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
As route model is used for specific internal scenario, and it's not for
the general requirement.
Fixes:#4838
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
"An empty [workspace] can be used with a package to conveniently create a
workspace with the package and all of its path dependencies", according
to the https://doc.rust-lang.org/cargo/reference/workspaces.html
This is also matches with the suggestion provided by the Cargo itself,
due to the errors faced with the Cloud Hypervisor CI:
```
10:46:23 this may be fixable by adding `go/src/github.com/kata-containers/kata-containers/src/tools/agent-ctl` to the `workspace.members` array of the manifest located at: /tmp/jenkins/workspace/kata-containers-2-clh-PR/Cargo.toml
10:46:23 Alternatively, to keep it out of the workspace, add the package to the `workspace.exclude` array, or add an empty `[workspace]` table to the package's manifest.
```
Fixes: #4843
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
If the API server is not ready, the mount call will fail, so before
mounting share fs, we should wait the nydusd is started and
the API server is ready.
Fixes: #4710
Signed-off-by: liubin <liubin0329@gmail.com>
Signed-off-by: Bin Liu <bin@hyper.sh>
Instead of passing a bunch of arguments to qmp functions for
adding block devices, use govmm BlockDevice structure to reduce these.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Get rid of redundant return values from function.
args and blockdevArgs used to return different values to maintain
compatilibity between qemu versions. These are exactly the same now.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This configuration will allow users to choose between different
I/O backends for qemu, with the default being io_uring.
This will allow users to fallback to a different I/O mechanism while
running on kernels olders than 5.1.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
To keep runtime-rs up to date, we will merge main into runtime-rs every
week.
Fixes:kata-containers#4822
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
github.com/kata-containers/tests#4986.To avoid returning an error when
running the ci, we just skip the test if the arch is s390x
Fixes: #4816
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
github.com/kata-containers/tests#4986.To avoid returning an error when running the ci, we just skip the build
process if the arch is s390x
Fixes: #4816
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
io_uring was introduced as a new kernel IO interface in kernel 5.1.
It is designed for higher performance than the older Linux AIO API.
This feature was added in qemu 5.0.
Fixes#4645
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
ps command supprot two formats, `json` and `table`. `json` format just
outputs pids in the container. `table` format will use `ps` utilty in
the host, search and output all processes in the container. Add a struct
`container` to represent a spawned container. Move the `kill`
implemention from kill.rs as a method of `container`.
Fixes: #4361
Signed-off-by: Chen Yiyang <cyyzero@qq.com>
Linux 5.14 supports core scheduling to have better security control
for SMT siblings. This PR supports that.
Fixes: #4429
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
To keep runtime-rs up to date, we will merge main into runtime-rs every
week.
Fixes: #4790
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Bump rtnetlink version from 0.8.0 to 0.11.0. Use rtnetlinks's API to
add neighbors and fix issues to adapt new verson of rtnetlink.
Fixes: #4607
Signed-off-by: Xuewei Niu <justxuewei@apache.org>
To keep runtime-rs up to date, we will merge main into runtime-rs every
week.
Fixes:#4776
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Fixes#4764
versions: update rust version to fix ccv0 attestation-agent build error
static-checks: kata tools, libs, and agent fixes
Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
In some cases do_create_container may return an error, mostly due to
`container.start(process)` call. This commit will do some rollback
works if this function failed.
Fixes: #4749
Signed-off-by: Bin Liu <bin@hyper.sh>
In qemu.StopVM(), if debug is enabled, the shim will dump logs
from qemu.log, but users don't know which logs are from qemu.log
and shim itself. Adding some additional messages will
help users to distinguish these logs.
Fixes: #4745
Signed-off-by: Bin Liu <bin@hyper.sh>
We can log all RPC calls to the agent for debugging purposes
to check which RPC is called, which can help us to understand
the container lifespan.
Fixes: #4738
Signed-off-by: liubin <liubin0329@gmail.com>
when the default_vcpus is greater than the default_maxvcpus, the default
vcpu number should be set equal to the default_maxvcpus.
Fixes: #4712
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
When run with configuration `sandbox_cgroup_only=false`, we will call
`gen_overhead_path()` as the overhead path. The `cgroup-rs` will push
the path with the subsystem prefix by `PathBuf::push()`. When the path
has prefix “/” it will act as root path, such as
```
let mut path = PathBuf::from("/tmp");
path.push("/etc");
assert_eq!(path, PathBuf::from("/etc"));
```
So we shoud not set overhead path with prefix "/".
Fixes: #4687
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
During use, there will be cases where the container is in the stop state
and get another stop. In this case, the second stop needs to be ignored.
Fixes: #4683
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
Update dbs-xxx commit ID for aarch64 in runtime-rs/Cargo.toml file to add
dependencies for aarch64.
Fixes: #4676
Signed-off-by: xuejun-xj <jiyunxue@alibaba.linux.com>
Module anyhow::anyhow is only used on x86_64 architecture in
crates/hypervisor/src/device/vfio.rs file.
Fixes: #4676
Signed-off-by: xuejun-xj <jiyunxue@alibaba.linux.com>
As a built in VMM, Path, jailer path, ctlpath are not needed for
Dragonball. So we don't generate those value in Makefile.
Fixes: #4677
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
New nix is supporting UMOUNT_NOFOLLOW, upgrade nix
version to use this flag instead of the self-defined flag.
Fixes: #4670
Signed-off-by: liubin <liubin0329@gmail.com>
When a container terminated, we should make sure there's no processes
left after destroying the container.
Before this commit, kata-agent depended on the kernel's pidns
to destroy all of the process in a container after the 1 process
exit in a container. This is true for those container using a
separated pidns, but for the case of shared pidns within the
sandbox, the container exit wouldn't trigger the pidns terminated,
and there would be some daemon process left in this container, this
wasn't expected.
Fixes: #4663
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
1. support annotation for runtime.name, hypervisor_name, agent_name.
2. fix parse memory from annotation
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Change get_mounts to get paths from a borrowed argument rather than
calling get_paths a second time.
Fixes#3768
Signed-off-by: Derek Lee <derlee@redhat.com>
Fix the warning "unused import: `dbs_arch::gic::Error as GICError`" and
"unused import: `dbs_arch::gic::GICDevice`" in file src/vm/mod.rs when
compiling.
Fixes: #4544
Signed-off-by: xuejun-xj <jiyunxue@alibaba.linux.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
As string "com1", "com2" and "rtc" are used in two files
(device_manager/mod.rs and device_manager/legacy.rs), we use public
const variables COM1, COM2 and RTC to replace them respectively.
Fixes: #4544
Signed-off-by: xuejun-xj <jiyunxue@alibaba.linux.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Implement generate_virtio_device_info() and
get_virtio_mmio_device_info() functions su support the mmio_device_info
member, which is used by FDT.
Fixes: #4544
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
remove redundant by_id in get_vm_by_id_mut and get_vm_by_id. They are
optimized to get_vm_mut and get_vm.
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Since cpu topology could tell whether hyper thread is enabled or not, we
removed ht_enabled config from VmConfigInfo
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Change error name from `StartMicrovm` to `StartMicroVm`,
`StartMicrovmError` to `StartMicroVmError`.
Besides, we fix a compile error in config_manager.
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
`exec` will execute a command inside a container which exists and is not
frozon or stopped. *Inside* means that the new process share namespaces
and cgroup with the container init process. Command can be specified by
`--process` parameter to read from a file, or from other parameters such
as arg, env, etc. In order to be compatible with `create`/`run`
commands, I refactor libcontainer. `Container` in builder.rs is divided
into `InitContainer` and `ActivatedContainer`. `InitContainer` is used
for `create`/`run` command. It will load spec from given bundle path.
`ActivatedContainer` is used by `exec` command, and will read the
container's status file, which stores the spec and `CreateOpt` for
creating the rustjail::LinuxContainer. Adapt the spec by replacing the
process with given options and updating the namesapces with some paths
to join the container. I also rename the `ContainerContext` as
`ContainerLauncher`, which is only used to spawn process now. It uses
the `LinuxContaier` in rustjail as the runner. For `create`/`run`, the
`launch` method will create a new container and run the first process.
For `exec`, the `launch` method will spawn a process which joins a
container.
Fixes#4363
Signed-off-by: Chen Yiyang <cyyzero@qq.com>
Enable Kata runtime to handle `disable_selinux` flag properly in order
to be able to change the status by the runtime configuration whether the
runtime applies the SELinux label to VMM process.
Fixes: #4599
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
We add microvm start related support in thie pull request.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
The vm struct to manage resources and control states of an virtual
machine instance.
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
add create_vcpu() function in vcpu test unit for aarch64
Fixes: #4445
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Replace `libc::setgroups()`, `libc::fchown()`, and `libc::sethostname()`
functions with nix crate ones for safety and maintainability.
Fixes: #4579
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Run the OCI `poststart` hooks must be called after the
user-specified process is executed but before the `start`
operation returns in accordance with OCI runtime spec.
Fixes: #4575
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Some clients like nerdctl may pass mount type of none for volumes/bind mounts,
this will lead to container start fails.
Referring to runc, it overwrites the mount type to bind and ignores the input value.
Fixes: #4548
Signed-off-by: liubin <liubin0329@gmail.com>
For runC, send the signal to the init process directly.
For kata, we try to send `SIGKILL` instead of `SIGTERM` when the process
has not installed the handler for `SIGTERM`.
The `is_signal_handled` function determine which signal the container
process has been handled. But currently `is_signal_handled` is only
catching (SigCgt). While the container process is ignoring (SigIgn) or
blocking (SigBlk) also should not be converted from the `SIGTERM` to
`SIGKILL`. For example, when using terminationGracePeriodSeconds the k8s
will send SIGTERM first and then send `SIGKILL`, in this case, the
container ignores the `SIGTERM`, so we should send the `SIGTERM` not the
`SIGKILL` to the container.
Fixes: #4478
Signed-off-by: quanweiZhou <quanweiZhou@linux.alibaba.com>
The tests ensure that interactions between drop-ins and the base
configuration.toml and among drop-ins themselves work as intended,
basically that files are evaluated in the correct order (base file
first, then drop-ins in alphabetical order) and the last one to set
a specific key wins.
Signed-off-by: Pavel Mores <pmores@redhat.com>
updateFromDropIn() uses the infrastructure built by previous commits to
ensure no contents of 'tomlConfig' are lost during decoding. To do
this, we preserve the current contents of our tomlConfig in a clone and
decode a drop-in into the original. At this point, the original
instance is updated but its Agent and/or Hypervisor fields are
potentially damaged.
To merge, we update the clone's Agent/Hypervisor from the original
instance. Now the clone has the desired Agent/Hypervisor and the
original instance has the rest, so to finish, we just need to move the
clone's Agent/Hypervisor to the original.
Signed-off-by: Pavel Mores <pmores@redhat.com>
These functions take a TOML key - an array of individual components,
e.g. ["agent" "kata" "enable_tracing"], as returned by BurntSushi - and
two 'tomlConfig' instances. They copy the value of the struct field
identified by the key from the source instance to the target one if
necessary.
This is only done if the TOML key points to structures stored in
maps by 'tomlConfig', i.e. 'hypervisor' and 'agent'. Nothing needs to
be done in other cases.
Signed-off-by: Pavel Mores <pmores@redhat.com>
For 'tomlConfig' substructures stored in Golang maps - 'hypervisor' and
'agent' - BurntSushi doesn't preserve their previous contents as it does
for substructures stored directly (e.g. 'runtime'). We use reflection
to work around this.
This commit adds three primitive operations to work with struct fields
identified by their `toml:"..."` tags - one to get a field value, one to
set a field value and one to assign a source struct field value to the
corresponding field of a target.
Signed-off-by: Pavel Mores <pmores@redhat.com>
Return code is an int32 type, so if an error occurred, the default value
may be zero, this value will be created as a normal exit code.
Set return code to 255 will let the caller(for example Kubernetes) know
that there are some problems with the pod/container.
Fixes: #4419
Signed-off-by: liubin <liubin0329@gmail.com>
Prior device config move didn't update the comments. Let's address this,
and make sure comments match the new path...
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Ideally this config validation would be in a seperate package
(katautils?), but that would introduce circular dependency since we'd
call it from vc, and it depends on vc types (which, shouldn't be vc, but
probably a hypervisor package instead).
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
While working on the previous commits, some of the functions become
non-used. Let's simply remove them.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Expose the newly added `default_maxmemory` to the project's Makefile and
to the configuration files.
Fixes: #4516
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's adapt Cloud Hypervisor's and QEMU's code to properly behave to the
newly added `default_maxmemory` config.
While implementing this, a change of behaviour (or a bug fix, depending
on how you see it) has been introduced as if a pod requests more memory
than the amount avaiable in the host, instead of failing to start the
pod, we simply hotplug the maximum amount of memory available, mimicing
better the runc behaviour.
Fixes: #4516
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add a `default_maxmemory` configuration, which allows the admins
to set the maximum amount of memory to be used by a VM, considering the
initial amount + whatever ends up being hotplugged via the pod limits.
By default this value is 0 (zero), and it means that the whole physical
RAM is the limit.
Fixes: #4516
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Now kata shim only supports stdout/stderr of fifo from
containerd/CRI-O, but shim v2 supports logging plugins,
and nerdctl default will use the binary schema for logs.
This commit will add the others type of log plugins:
- file
- binary
In case of binary, kata shim will receive a stdout/stderr like:
binary:///nerdctl?_NERDCTL_INTERNAL_LOGGING=/var/lib/nerdctl/1935db59
That means the nerdctl process will handle the logs(stdout/stderr)
Fixes: #4420
Signed-off-by: Bin Liu <bin@hyper.sh>
Depending on the user of it, the hypervisor from hypervisor interface
could have differing view on what is valid or not. To help decouple,
let's instead check the hypervisor config validity as part of the
sandbox creation, rather than as part of the CreateVM call within the
hypervisor interface implementation.
Fixes: #4251
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Policy for whats valid/invalid within the config varies by VMM, host,
and by silicon architecture. Let's keep katautils simple for just
translating a toml to the hypervisor config structure, and leave
validation to virtcontainers.
Without this change, we're doing duplicate validation.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
In order to support ACPI hotplug in the future with the cooperative work
from the Kata community, we add ACPI feature and dbs-upcall feature to
add room for ACPI hotplug.
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
It is used to define the vmm communication interface.
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Upcall is a direct communication tool between VMM and guest developed
upon vsock. It is used to implement device hotplug.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Signed-off-by: Zizheng Bian <zizheng.bian@linux.alibaba.com>
Before, we maintained almost identical structures between our persist
API and what we keep for our devices, with the persist API being a
slight subset of device structures.
Let's deduplicate this, now that persist is importing device package.
Json unmarshal of prior persist structure will work fine, since it was
an exact subset of fields.
Fixes: #4468
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Rather than have device package depend on persist, let's define the
(almost duplicate) structures within device itself, and have the Kata
Container's persist pkg import these.
This'll help avoid unecessary dependencies within our core packages.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
In the original code, reads mountstats file and return
the content in the error, but at this time the file maybe
changed, we should return the file content that parsed
line by line to check why there is not a fstype option.
Fixes: #4246
Signed-off-by: Bin Liu <bin@hyper.sh>
Let's run a `cargo update` and ensure the deps are up-to-date before we
cut the "-rc0" release.
Fixes: #4525
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The error shown below was caught during a dependency bump in the CCv0
branch, but we better fix it here first.
```
error: this boolean expression can be simplified
--> src/random.rs:85:21
|
85 | assert!(!ret.is_ok());
| ^^^^^^^^^^^^ help: try: `ret.is_err()`
|
= note: `-D clippy::nonminimal-bool` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#nonminimal_bool
error: this boolean expression can be simplified
--> src/random.rs:93:17
|
93 | assert!(!ret.is_ok());
| ^^^^^^^^^^^^ help: try: `ret.is_err()`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#nonminimal_bool
```
Fixes: #4523
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The error shown below was caught during a dependency bump in the CCv0
branch, but we better fix it here first.
```
error: use of `ok_or` followed by a function call
--> src/netlink.rs:526:14
|
526 | .ok_or(anyhow!(nix::Error::EINVAL))?;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try this: `ok_or_else(|| anyhow!(nix::Error::EINVAL))`
|
= note: `-D clippy::or-fun-call` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#or_fun_call
error: use of `ok_or` followed by a function call
--> src/netlink.rs:615:49
|
615 | let v = u8::from_str_radix(split.next().ok_or(anyhow!(nix::Error::EINVAL))?, 16)?;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try this: `ok_or_else(|| anyhow!(nix::Error::EINVAL))`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#or_fun_call
```
Fixes: #4523
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The cargo command creates debug build binaries, when the --release
option is not specified. Specifying --debug option causes an error.
This patch specifies --release option when BUILD_TYPE=release,
and does not specify any build type option when BUILD_TYPE=debug.
Fixes#4504
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
Similar to network, we can use multiple queues for virtio-block
devices. This can help improve storage performance.
This commit changes the number of queues for block devices to
the number of cpus for cloud-hypervisor and qemu.
Today the default number of cpus a VM starts with is 1.
Hence the queues used will be 1. This change will help
improve performance when the default cold-plugged cpus is greater
than one by changing this in the config file. This may also help
when we use the sandboxing feature with k8s that passes down
the sum of the resources required down to Kata.
Fixes#4502
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Enable "-sandbox on" in qemu can introduce another protect layer
on the host, to make the secure container more secure.
The default option is disable because this feature may introduce some
performance cost, even though user can enable
/proc/sys/net/core/bpf_jit_enable to reduce the impact.
Fixes: #2266
Signed-off-by: Feng Wang <feng.wang@databricks.com>
Remove space from root span name to follow camel casing of other tracing
span names in the runtime and to make parsing easier in testing.
Fixes#4483
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Used to register dragonball's signal handler.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
By comparing the content of the old url and the new url,
ensure that their content is consistent and does not contain ambiguities
Fixes: #4454
Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
Let's improve the log so we make it clear that we're only *actually*
adding the net device to the Cloud Hypervisor configuration when calling
our own version of VmAddNetPut().
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We want to have the file descriptors of the opened tuntap device to pass
them down to the VMMs, so the VMMs don't have to explicitly open a new
tuntap device themselves, as the `container_kvm_t` label does not allow
such a thing.
With this change we ensure that what's currently done when using QEMU as
the hypervisor, can be easily replicated with other VMMs, even if they
don't support multiqueue.
As a side effect of this, we need to close the received file descriptors
in the code of the VMMs which are not going to use them.
Fixes: #3533
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Adding FFI_NO_PI to the netlink flags causes no harm to the supported
and tested hypervisors as when opening the device by its name Cloud
Hypervisor[0], Firecracker[1], and QEMU[2] do set the flag already.
However, when receiving the file descriptor of an opened tutap device
Cloud Hypervisor is not able to set the flag, leaving the guest without
connectivity.
To avoid such an issue, let's simply add the FFI_NO_PI flag to the
netlink flags and ensure, from our side, that the VMMs don't have to set
it on their side when dealing with an already opened tuntap device.
Note that there's a PR opened[3] just for testing that this change
doesn't cause any breakage.
[0]: e52175c2ab/net_util/src/tap.rs (L129)
[1]: b6d6f71213/src/devices/src/virtio/net/tap.rs (L126)
[2]: 3757b0d08b/net/tap-linux.c (L54)
[3]: https://github.com/kata-containers/kata-containers/pull/4292
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is basically a no-op right now, as:
* netPair.TapInterface.VMFds is nil
* the tap name is still passed to Cloud Hypervisor, which is the Cloud
Hypervisor's first choice when opening a tap device.
In the very near future we'll stop passing the tap name to Cloud
Hypervisor, and start passing the file descriptors of the opened tap
instead.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Knowing that VmAddNetPut works as expected, let's switch to manually
building the request and writing it to the appropriate socket.
By doing this it gives us more flexibility to, later on, pass the file
descriptor of the tuntap device to Cloud Hypervisor, as openAPI doesn't
support such operation (it has no notion of SCM Rights).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Instead of creating the VM with the network device already plugged in,
let's actually add the network device *after* the VM is created, but
*before* the Vm is actually booted.
Although it looks like it doesn't make any functional difference between
what's done in the past and what this commit introduces, this will be
used to workaround a limitation on OpenAPI when it comes to passing down
the network device's file descriptor to Cloud Hypervisor, so Cloud
Hypervisor can use it instead of opening the device by its name on the
VMM side.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
VmAddNetPut is the API provided by the Cloud Hypervisor client (auto
generated) code to hotplug a new network device to the VM.
Let's expose it now as it'll be used as part this series, mostly to
guide the reviewer through the process of what we have to do, as later
on, spoiler alert, it'll end up being removed.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
So far this has been done for x86_64. Now that the support for building
and testing has been added for all arches, let's do the second part of
the switch.
We're still not done yet for powerpc, as some a virtifosd crash on the
rust version has been found by the maintainer.
Fixes: #4258, #4260
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
metrics system is added for collecting Dragonball metrics to analyze the
system.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
In the original code, reads mountstats file and return
the content in the error, but at this time the file maybe
changed, we should return the file content that parsed
line by line to check why there is not a fstype option.
Fixes: #4246
Signed-off-by: Bin Liu <bin@hyper.sh>
Changed bitsize for parsing functions to 64-bit in order to avoid
parsing errors.
Fixes#4435
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Revert this patch, after dragonball-sandbox is ready. And all
subsequent implementations are submitted.
Fixes: #4257
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Update Dragonball Readme to fix style problem and add github issue for
TODOs.
Add document for devices in dragonball. This is the document for the
current dragonball device status and we'll keep updating it when we
introduce more devices in later pull requets.
Fixes: #4257
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Device manager to manage IO devices for a virtual machine. And added
DeviceManagerTx to provide operation transaction for device management,
added DeviceManagerContext to operation context for device management.
Fixes: #4257
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
It is used for holding guest kernel configuration information.
Fixes: #4257
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
It is used for managing a group of configuration information.
Fixes: #4257
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
The legacy devices manager is used for managing legacy devices.
Fixes: #4257
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Console manager to manage frontend and backend console devcies.
A virtual console are composed up of two parts: frontend in virtual
machine and backend in host OS. A frontend may be serial port,
virtio-console etc, a backend may be stdio or Unix domain socket. The
manager connects the frontend with the backend.
Fixes: #4257
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
1. support oom event
2. use ContainerProcess to store container_id and exec_id
3. support stats
Fixes: #3785
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
1. service: Responsible for processing services, such as task service, image service
2. Responsible for implementing different runtimes, such as Virt-container,
Linux-container, Wasm-container
Fixes: #3785
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
1. support async.
2. update ttrpc and protobuf
update ttrpc to 0.6.0
update protobuf to 2.23.0
3. support trans from oci
Fixes: #3746
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
Provide functions to execute OCI hooks.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Bin Liu <bin@hyper.sh>
Signed-off-by: Huamin Tang <huamin.thm@alibaba-inc.com>
Signed-off-by: Lei Wang <wllenyj@linux.alibaba.com>
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
Add function to detect and update K8s emptyDir volume.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Qingyuan Hou <qingyuan.hou@linux.alibaba.com>
Introduce get_devid() to get major/minor number of a block device.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
Implement reflink_copy() to copy file by reflink, and fallback to normal
file copy.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
Add utilities to parse NUMA information.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Qingyuan Hou <qingyuan.hou@linux.alibaba.com>
Signed-off-by: Simon Guo <wei.guo.simon@linux.alibaba.com>
Add utilities to manipulate cgroup, currently only v1 is supported.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: He Rongguang <herongguang@linux.alibaba.com>
Signed-off-by: Jiahuan Chao <jhchao@linux.alibaba.com>
Signed-off-by: Qingyuan Hou <qingyuan.hou@linux.alibaba.com>
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
Signed-off-by: Tim Zhang <tim@hyper.sh>
Add some wrappers for mount and fs syscall.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Bin Liu <bin@hyper.sh>
Signed-off-by: Fupan Li <lifupan@gmail.com>
Signed-off-by: Huamin Tang <huamin.thm@alibaba-inc.com>
Signed-off-by: Lei Wang <wllenyj@linux.alibaba.com>
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
The kata-sys-util crate is a collection of modules that provides helpers
and utilities used by multiple Kata Containers components.
Fixes: #3305
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
1. modify default values for hypervisor
2. change the variable name
3. check the min memory limit
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
1. Some Nit problems are fixed
2. Make the code more readable
3. Modify some implementation details
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
If there is a parse error when we are trying to get the annotations, we
will return Result<Option<type>> to handle that.
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
loading from empty string is only used to identity that the config is
not initialized yet, so Option<TomlConfig> is a better option
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Some annotations are used to override hypervisor configurations, and you
know it's dangerous. We must be careful when overriding hypervisor configuration
by annotations, to avoid security flaws.
There are two existing mechanisms to prevent attacks by annotations:
1) config.hypervisor.enable_annotations defines the allowed annotation
keys for config.hypervisor.
2) config.hyperisor.xxxx_paths defines allowd values for specific keys.
The access methods for config.hypervisor.xxx enforces the permisstion
checks for above rules.
To update conifg, traverse the annotation hashmap,check if the key is enabled in hypervisor or not.
If it is enabled. For path related annotation, check whether it is valid or not
before updating conifg. For cpu and memory related annotation, check whether it
is more than or less than the limitation for DB and qemu beforing updating config.
If it is not enabled, there will be three possibilities, agent related
annotation, runtime related annotation and hypervisor related annotation
but not enabled. The function will handle agent and runtime annotation
first, then the option left will be the invlaid hypervisor, err message
will be returned.
add more edge cases tests for updating config
clean up unused functions, delete unused files and fix warnings
Fixes: #3523
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
The TomlConfig structure is a parsed form of Kata configuration file,
but it's a little inconveneient to access those configuration
information directly. So introduce a wrapper KataConfig to easily
access those configuration information.
Two singletons of KataConfig is provided:
- KATA_DEFAULT_CONFIG: the original version directly loaded from Kata
configuration file.
- KATA_ACTIVE_CONFIG: the active version is the KATA_DEFAULT_CONFIG
patched by annotations.
So the recommended to way to use these two singletons:
- Load TomlConfig from configuration file and set it as the default one.
- Clone the default one and patch it with values from annotations.
- Use the default one for permission checks, such as to check for
allowed annotation keys/values.
- The patched version may be set as the active one or passed to clients.
- The clients directly accesses information from the active/passed one,
and do not need to check annotation for override.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Add structures to load Kata agent configuration from configuration files.
Also define a mechanism for vendor to extend the Kata configuration
structure.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Add structures to load Kata hypevisor configuration from configuration
files. Also define a mechanisms to:
1) for hypervisors to handle the configuration info.
2) for vendor to extend the Kata configuration structure.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Add structures to load Kata runtime configuration from configuration
files. Also define a mechanism for vendor to extend the Kata
configuration structure.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Add kata-types crate to host constants and data types shared by multiple
Kata Containers components.
Fixes: #3305
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Fupan Li <lifupan@gmail.com>
Signed-off-by: Huamin Tang <huamin.thm@alibaba-inc.com>
Signed-off-by: Lei Wang <wllenyj@linux.alibaba.com>
Signed-off-by: yanlei <yl.on.the.way@gmail.com>
Introduce a wrapper writer `LogWriter` which converts every line written
to it into a log record.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Wei Yang <wei.yang1@linux.alibaba.com>
Signed-off-by: yanlei <yl.on.the.way@gmail.com>
Add FileRotator to rotate log files.
The FileRotator structure may be used as writer for create_logger()
and limits the storage space occupied by log files.
Fixes: #3304
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Wei Yang <wei.yang1@linux.alibaba.com>
Signed-off-by: yanlei <yl.on.the.way@gmail.com>
Convert libs into a Cargo workspace, so all libraries could share the
build infrastructure.
Fixes#3282
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Add more detail to the `kata-monitor` doc to allow an admin to make a
more informed decision about where and how to run the daemon.
Fixes: #4416.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
GetOOMEvent is a blocking call that will fail if
the container exit, in this case, it's not an error or warning.
Changing the log level for logs in case of GetOOMEvent call fails
will reduce log noise in a large cluster that has pods
creating/deleting frequently.
Fixes: #4376
Signed-off-by: Bin Liu <bin@hyper.sh>
Add a new `Examples` section to the `agent-ctl` docs giving some
examples of how to use the tool with QEMU and stand-alone.
Fixes: #4414.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The `agent-ctl` and `trace-forwarder` tools make use of
`anyhow::Context` to provide additional call site information on error.
However, previously neither tool was using the "alternate debug" format
to display the error, meaning full error output was not displayed.
Fixes: #4411.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Since #902 the `io.katacontainers.config.hypervisor` pod annotations
have only been permitted if explicitly allowed in the global
configuration. The default global configuration allows no such
annotations. That's important because several of those annotations
would cause Kata to execute arbitrary binaries, and so were wildly
unsafe.
However, this is inconvenient for the
`io.katacontainers.config.hypervisor.enable_iommu` annotation
specifically, which controls whether the sandbox VM includes a vIOMMU.
A guest side vIOMMU is necessary to implement VFIO passthrough devices
with `vfio_mode = vfio`, so enabling that mode of operation currently
requires a global configuration change, and can't just be enabled
per-pod.
Unlike some of the other hypervisor annotations, the `enable_iommu`
annotation is quite safe. By default the vIOMMU is not present, so
allowing a user to override it for a pod only improves their
facilities for isolation. Even if the global default were changed to
enable the vIOMMU, that doesn't compel the guest kernel to use it, so
allowing a user to disable the vIOMMU doesn't materially affect
isolation either.
Therefore, allow the io.katacontainers.config.hypervisor.enable_iommu
annotation to work in the default configurations.
fixes#4330
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
destroy() method should ignore the ESRCH error from signal::kill
and continue the operation as ESRCH is often considered harmless.
Fixes: #4359
Signed-off-by: Feng Wang <feng.wang@databricks.com>
Set thestop container force flag to true so that the container state is always set to
“StateStopped” after the container wait goroutine is finished. This is necessary for
the following delete container step to succeed.
Fixes: #4359
Signed-off-by: Feng Wang <feng.wang@databricks.com>
Support list sub-command. It will traverse the root directory, parse
status file and print basic information of containers. Behavior and
print format consistent with runc. To handle race with runk delete
or system user modify, the loop will continue to traverse when errors
are encountered.
Fixes: #4362
Signed-off-by: Chen Yiyang <cyyzero@qq.com>
The `kata-agent` passes its standard I/O file descriptors
through to the container process that will be launched
by `runk` without manipulation or modification in order to
allow the container process can handle its I/O operations.
Fixes: #4327
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
In linux 5.14 and hopefully some backports, core scheduling allows processes to
be co scheduled within the same domain on SMT enabled systems.
Containerd impl sets the core sched domain when launching a shim. This
allows a clean way for each shim(container/pod) to be in its own domain and any
additional containers, (v2 pods) be be launched with the same domain as well as
any exec'd process added to the container.
kernel docs: https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/core-scheduling.html
For Kata specifically, we will look for SCHED_CORE environment variable
to be set to indicate we shuold create a new schedule core domain.
This is equivalent to the containerd shim's PR: e48bbe8394Fixes: #4309
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Signed-off-by: Michael Crosby <michael@thepasture.io>
While end users can connect directly to the shim, let's provide a way to
easily get/set iptables from kata-runtime itself.
Fixes: #4080
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Without this, potential errors are silently dropped. Let's ensure we
return the error code as well as potenial data from the response.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Before, we had a mix of slash, etc. Unfortunately, when cleaning URL
paths, serve mux seems to mangle the request method, resulting in each
request being a GET (instead of PUT or POST).
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Add two endpoints: ip6tables, iptables.
Each url handler supports GET and PUT operations. PUT expects
the requests' data to be []bytes, and to contain iptable information in
format to be consumed by iptables-restore.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Introduce get/set iptable handling. We add a sandbox API for getting and
setting the IPTables within the guest. This routes it from sandbox
interface, through kata-agent, ultimately making requests to the guest
agent.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Update the agent protocol definition to introduce support for setting
and getting iptables from the guest.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
This enables tests for the kata-agent for runk that is built
with standard-oci-runtime feature in CI.
Fixes: #4351
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Get user's home dir using `nix::unistd` crate instead of `utils` crate,
and remove useless code from agent.
Fixes: #4209
Signed-off-by: Xuewei Niu <justxuewei@apache.org>
runk always launches containers with detached mode,
so users have to use a console socket with run or
create operation when a terminal is used.
If users set `terminal` to `true` in `config.json` and
try to launch a container without specifying a console
socket, runk returns an error with a message early.
Fixes: #4324
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
This commit enables runk to handle `root.path` in `config.json`
properly even if the path is specified by a relative path that
includes the single (`.`) or the double (`..`) dots.
For example, with a bundle at `/to/bundle` and a rootfs directly
under `/to/bundle` such as `/to/bundle/{bin,dev,etc,home,...}`,
the `root.path` value can be either `/to/bundle` or just `.`.
This behavior conforms to OCI runtime spec.
Accordingly, a bundle path managed by runk's status file
(`status.json`) always is statically stored as a canonical path.
Previously, a bundle path has been got by `oci_state()` of rustjail's
API that returns the path as the parent directory path of a rootfs
(`root.path`). In case of the kata-agent, this works properly because
the kata containers assume that the rootfs path is always
`/to/bundle/rootfs`. However in case of standard OCI runtimes,
a rootfs can be placed anywhere under a bundle, so the rootfs path
doesn't always have to be at a `/to/bundle/rootfs`.
Fixes: #4334
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
This release has been tracked through the v24.0 project.
virtio-iommu specification describes how a device can be attached by default
to a bypass domain. This feature is particularly helpful for booting a VM with
guest software which doesn't support virtio-iommu but still need to access
the device. Now that Cloud Hypervisor supports this feature, it can boot a VM
with Rust Hypervisor Firmware or OVMF even if the virtio-block device exposing
the disk image is placed behind a virtual IOMMU.
Multiple checks have been added to the code to prevent devices with identical
identifiers from being created, and therefore avoid unexpected behaviors at boot
or whenever a device was hot plugged into the VM.
Sparse mmap support has been added to both VFIO and vfio-user devices. This
allows the device regions that are not fully mappable to be partially mapped.
And the more a device region can be mapped into the guest address space, the
fewer VM exits will be generated when this device is accessed. This directly
impacts the performance related to this device.
A new serial_number option has been added to --platform, allowing a user to
set a specific serial number for the platform. This number is exposed to the
guest through the SMBIOS.
* Fix loading RAW firmware (#4072)
* Reject compressed QCOW images (#4055)
* Reject virtio-mem resize if device is not activated (#4003)
* Fix potential mmap leaks from VFIO/vfio-user MMIO regions (#4069)
* Fix algorithm finding HOB memory resources (#3983)
* Refactor interrupt handling (#4083)
* Load kernel asynchronously (#4022)
* Only create ACPI memory manager DSDT when resizable (#4013)
Deprecated features will be removed in a subsequent release and users should
plan to use alternatives
* The mergeable option from the virtio-pmem support has been deprecated
(#3968)
* The dax option from the virtio-fs support has been deprecated (#3889)
Fixes: #4317
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The current implementation of walking the
disks to match with the requested volume path
in agent doesn't work because the volume path
provided by the shim to the agent is the mount
path within the guest and not the device name.
The current logic is trying to match the
device name to the volume path which will never
match.
This change will simplify the
get_volume_capacity_stats and
get_volume_inode_stats to just call statfs and
get the bytes and inodes usage of the volume
path directly.
Fixes: #4297
Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
Today the shim does a translation when doing
direct-volume stats where it takes the source and
returns the mount path within the guest.
The source for a direct-assigned volume is actually
the device path on the host and not the publish
volume path.
This change will perform a lookup of the mount info
during direct-volume stats to ensure that the
device path is provided to the shim for querying
the volume stats.
Fixes: #4297
Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
The go default http mux AFAIK doesn’t support pattern
routing so right now client is padding the url
for direct-volume stats with a subpath of the volume
path and this will always result in 404 not found returned
by the shim.
This change will update the shim to take the volume
path as a GET query parameter instead of a subpath.
If the parameter is missing or empty, then return
400 BadRequest to the client.
Fixes: #4297
Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
The action function expects a function that returns error
but the current direct-volume stats Action returns
(string, error) which is invalid.
This change fixes the format and print out the stats from
the command instead.
Fixes: #4293
Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
The documentation of the bufio package explicitly says
"Err returns the first non-EOF error that was encountered by the
Scanner."
When io.EOF happens, `Err()` will return `nil` and `Scan()` will return
`false`.
Fixes#4079
Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
This allows to get guest early boot logs which are usually
missed when virtconsole is used.
- It utilizes previous work on the govmm side:
https://github.com/kata-containers/govmm/pull/203
- unit test added
Fixes: #4237
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
In the is_signal_handled function, when parsing the hex string returned
from `/proc/<pid>/status` the space/tab character after the colon
is not removed.
This patch trims the result of SigCgt so that
all whitespace characters are removed. It also extends the existing
test cases to check for this scenario.
Fixes: #4250
Signed-off-by: Champ-Goblem <cameron@northflank.com>
As now we build and ship the rust version of virtiofsd, which is not
tied to QEMU, we need to update its default location to match with where
we're installing this binary.
Fixes: #4249
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
go-test.sh by default adds the -v option to 'go test' meaning that output
will be printed from all the passing tests as well as any failing ones.
This results in a lot of output in which it's often difficult to locate the
failing tests you're interested in.
So, remove -v from the default flags.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
One of the responsibilities of the go-test.sh script is setting up the
default flags for 'go test'. This is constructed across several different
places in the script using several unneeded intermediate variables though.
Consolidate all the flag construction into one place.
fixes#4190
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
go-test.sh changes behaviour based on both the $CI and $KATA_DEV_MODE
variables, but not in a way that makes a lot of sense.
If either one is set it uses the test_coverage path, instead of the
test_local path. That collects coverage information, as the name
suggests, but it also means it runs the tests twice as root and
non-root, which is very non-obvious.
It's not clear what use case the test_local path is for at all.
Developer local builds will typically have $KATA_DEV_MODE set and CI
builds will have $CI set. There's essentially no downside to running
coverage all the time - it has little impact on the test runtime.
In addition, if *both* $CI and $KATA_DEV_MODE are set, the script
refuses to run things as root, considering it "unsafe". While having
both set might be unwise in a general sense, there's not really any
way running sudo can be any more unsafe than it is with either one
set.
So, simplify everything by just always running the test_coverage path.
This leaves the test_local path unused, so we can remove it entirely.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
go-test.sh accepts subcommands, however invoking it in the usual way via
the Makefile doesn't use them. In fact the only remaining subcommand is
"help" and we already have another way of getting the usage information
(-h or --help). We don't need a second way, so just drop subcommand
handling.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
go-test.sh defaults to testing all the packages listed by go list, except
for a number filtered out. It turns out that none of those filters are
necessary any more:
* We've long required a Go newer than 1.9 which means the vendor filter
isn't needed
* The agent filter doesn't do anything now that we've moved to the Kata
2.x unified repo
* The tests filters don't hit anything on the list of modules in
src/runtime (which is the only user of the script)
But since we don't need to filter anything out any more, we don't even need
to iterate through a list ourselves. We can simply pass "./..." directly
to go test and it will iterate through all the sub-packages itself.
Interestingly this more than doubles the speed of "make test" for me - I
suspect because go test's internal paralellism works better over a larger
pool of tests.
This also lets us remove handling of non-existent coverage files from
test_go_package(), since with default options we will no longer test packages without tests
by default. If the user explicitly requests testing of a package with no
tests, then failing makes sense.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The go-test.sh script has an explicit chmod command, run as root, to
set the mode of the temporary coverage files to 0644. AFAICT the
point of this is specifically the 004 bit allowing world read access,
so that we can then merge the temporary coverage file into the main
coverage file.
That's a convoluted way of doing things. Instead we can just run the tail
command which reads the temporary file as the same user that generated it.
In addition, go-test.sh became root to remove that temporary coverage
file. This is not necessary, since deleting a regular file just requires
write access to the directory, not the file itself.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The html-coverage option to this script doesn't really alter behaviour
it just does the same thing as normal coverage, then converts the
report to HTML. That conversion is a single command, plus a chmod to
make the final output mode 0644. That overrides any umask the user
has set, which doesn't seem like a policy decision this script should
be making.
Nothing in the kata-containers or tests repository uses this, so it doesn't
really make sense to keep this logic inside this script.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
In addition to coverage.txt, the go-test.sh script creates
coverage.txt.tmp files while running. These are temporary and
certainly shouldn't be committed, so add them to the gitignore file.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The go unit tests for the runtime are invoked by the helper script
ci/go-test.sh. Which calls the run_go_test() function in ci/lib.sh. Which
calls into .ci/go-test.sh from the tests repository.
But.. the runtime is the only user of this script, and generally stuff for
unit tests (rather than functional or integration tests) lives in the main
repository, not the tests repository.
So, just move the actual script into src/runtime. A change to remove it
from the tests repo will follow.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
to the kata-containers repo under the src/tools/log-parser folder
and vendor the modules
Fixes: #4100
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
We're currently hitting a race condition on the Cloud Hypervisor's
driver code when quickly removing and adding a block device.
This happens because the device removal is an asynchronous operation,
and we currently do *not* monitor events coming from Cloud Hypervisor to
know when the device was actually removed. Together with this, the
sandbox code doesn't know about that and when a new device is attached
it'll quickly assign what may be the very same ID to the new device,
leading to the Cloud Hypervisor's driver trying to hotplug a device with
the very same ID of the device that was not yet removed.
This is, in a nutshell, why the tests with Cloud Hypervisor and
devmapper have been failing every now and then.
The workaround taken to solve the issue is basically *not* passing down
the device ID to Cloud Hypervisor and simply letting Cloud Hypervisor
itself generate those, as Cloud Hypervisor does it in a manner that
avoids such conflicts. With this addition we have then to keep a map of
the device ID and the Cloud Hypervisor's generated ID, so we can
properly remove the device.
This workaround will probably stay for a while, at least till someone
has enough cycles to implement a way to watch the device removal event
and then properly act on that. Spoiler alert, this will be a complex
change that may not even be worth it considering the race can be avoided
with this commit.
Fixes: #4176
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add test coverage for the functions find_process and online_resources in src/sandbox.rs.
Fixes#4085Fixes#4136
Signed-off-by: Jack Hance <jack.hance@ndsu.edu>
Today in agent watchers, when we copy files/symlinks
or create directories, the ownership of the source path
is not preserved which can lead to permission issues.
In copy, ensure that we do a chown of the source path
uid/gid to the destination file/symlink after copy to
ensure that ownership matches the source ownership.
fs::copy() takes care of setting the permissions.
For directory creation, ensure that we set the
permissions of the created directory to the source
directory permissions and also perform a chown of the
source path uid/gid to ensure directory ownership
and permissions matches to the source.
Fixes: #4188
Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
runk uses liboci-cli crate to parse command line options,
but liboci-cli does not support --all option for kill command,
though this is the runtime spec behavior.
But crictl will issue kill --all command when stopping containers,
as a workaround, we use a custom kill command instead of the one
provided by liboci-cli.
Fixes: #4182
Signed-off-by: Bin Liu <bin@hyper.sh>
The default runtime for io.containerd.runc.v2 is runc,
to use runk, the containerd configuration should set the
default runtime to runk or add BinaryName options for the
runtime.
Fixes: #4177
Signed-off-by: Bin Liu <bin@hyper.sh>
Update the `agent-ctl` docs to show how to use a VSOCK local address
when running the agent and the tool in the same environment. This is an
alternative to using a Unix socket.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Unbreak the `agent-ctl` tool connecting to the agent with a Unix domain
socket.
It appears that [1] changed the behaviour of connecting to the agent
using a local Unix socket (which is not used by Kata under normal
operation).
The change can be seen by reverting to commit
72b8144b56 (the one before [1]) and
running the agent manually as:
```bash
$ sudo KATA_AGENT_SERVER_ADDR=unix:///tmp/foo.socket target/x86_64-unknown-linux-musl/release/kata-agent
```
Before [1], in another terminal we see this:
```bash
$ sudo lsof -U 2>/dev/null |grep foo|awk '{print $9}'
@/tmp/foo.socket@
```
But now, we see the following:
```bash
$ sudo lsof -U 2>/dev/null |grep foo|awk '{print $9}'
@/tmp/foo.socket
```
Note the last byte which represents a nul (`\0`) value.
The `agent-ctl` tool used to add that trailing nul but now it seems to not
be needed, so this change removes it, restoring functionality. No
external changes are necessary so the `agent-ctl` tool can connect to
the agent as below like this:
```bash
$ cargo run -- -l debug connect --server-address "unix://@/tmp/foo.socket" --bundle-dir "$bundle_dir" -c Check -c GetGuestDetails
```
[1] - https://github.com/kata-containers/kata-containers/issues/3124Fixes: #4164.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
With everything implemented, let's now expose the disk rate limiter
configuration options in the Cloud Hypervisor configuration file.
Fixes: #4139
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
With everything implemented, let's now expose the net rate limiter
configuration options in the Cloud Hypervisor configuration file.
Fixes: #4017
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The notion of "built-in rate limiter" was added as part of
bd8658e362, and that commit considered
that only Firecracker had a built-in rate limiter, which I think was the
case when that was introduced (mid 2020).
Nowadays, however, Cloud Hypervisor takes advantage of the very same crate
used by Firecraker to do I/O throttling.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's take advantage of the newly added DiskRateLimiter* options and
apply those to the network device configuration.
The logic here is identical to the one already present in the Network
part of Cloud Hypervisor's driver.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the newly added disk rate limiter configurations to the Cloud
Hypervisor's hypervisor configuration.
Right now those are not used anywhere, and there's absolutely no way the
users can set those up. That's coming later in this very same series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is the disk counterpart of the what was introduced for the network
as part of the previous commits in this series.
The newly added fields are:
* DiskRateLimiterBwMaxRate, defined in bits per second, which is used to
control the network I/O bandwidth at the VM level.
* DiskRateLimiterBwOneTimeBurst, also defined in bits per second, which
is used to define an *initial* max rate, which doesn't replenish.
* DiskRateLimiterOpsMaxRate, the operations per second equivalent of the
DiskRateLimiterBwMaxRate.
* DiskRateLimiterOpsOneTimeBurst, the operations per second equivalent of
the DiskRateLimiterBwOneTimeBurst.
For now those extra fields have only been added to the hypervisor's
configuration and they'll be used in the coming patches of this very
same series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's take advantage of the newly added NetRateLimiter* options and
apply those to the network device configuration.
The logic here is quite similar to the one already present in the
Firecracker's driver, with the main difference being the single Inbound
/ Outbound MaxRate and the presence of both Bandwidth and Operations
rate limiter.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Firecracker's driver doesn't expose the RefillTime option of the rate
limiter to the user. Instead, it uses a contant value of 1000
miliseconds (1 second).
As we're following Firecracker's driver implementation, let's expose
create a new constant, use it as part of the Firecracker's driver, and
later on re-use it as part of the Cloud Hypervisor's driver.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Firecracker's revertBytes function, now called "RevertBytes", can be
exposed as part of the virtcontainers' utils file, as this function will
be reused by Cloud Hypervisor, when adding the rate limiter logic there.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the newly added network rate limiter configurations to the
Cloud Hypervisor's hypervisor configuration.
Right now those are not used anywhere, and there's absolutely no way the
users can set those up. That's coming later in this very same series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In a similar way to what's already exposed as RxRateLimiterMaxRate and
TxRateLimiterMaxRate, let's add four new fields to the Hypervisor's
configuration.
The values added are related to bandwidth and operations rate limiters,
which have to be added so we can expose I/O throttling configurations to
users using Cloud Hypervisor as their preferred VMM.
The reason we cannot simply re-use {Rx,Tx}RateLimiterMaxRate is because
Cloud Hypervisor exposes a single MaxRate to be used for both inbound
and outbound queues.
The newly added fields are:
* NetRateLimiterBwMaxRate, defined in bits per second, which is used to
control the network I/O bandwidth at the VM level.
* NetRateLimiterBwOneTimeBurst, also defined in bits per second, which
is used to define an *initial* max rate, which doesn't replenish.
* NetRateLimiterOpsMaxRate, the operations per second equivalent of the
NetRateLimiterBwMaxRate.
* NetRateLimiterOpsOneTimeBurst, the operations per second equivalent of
the NetRateLimiterBwOneTimeBurst.
For now those extra fields have only been added to the hypervisor's
configuration and they'll be used in the coming patches of this very
same series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add tests for parse_mount_table function in rustjail/src/mount.rs.
Includes some minor refactoring improve the testability of the
function and improve its error values.
Fixes: #4082
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
Add a Rust-based standard OCI container runtime based on
Kata agent.
You can build and install runk as follows:
```sh
$ cd src/tools/runk
$ make
$ sudo make install
$ runk --help
```
Fixes: #2784
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Generate an oci-kata-agent which is a customized agent to be
called from runk which is a Rust-based standard OCI container
runtime based on Kata agent.
Fixes: #2784
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
The type of MemorySwappiness in runtime is uint64, and the type of swappiness in agent is int64,
if we set max uint64 in runtime and pass it to agent, the value will be equal to -1. We should
modify the type of swappiness to u64
Fixes: #4123
Signed-off-by: holyfei <yangfeiyu20092010@163.com>
Currently EnableMockTesting() takes no arguments and will always place the
mock storage in the fixed location /tmp/vc/mockfs. This means that one
test run can interfere with the next one if anything isn't cleaned up
(and there are other bugs which means that happens). If if those were
fixed this would allow developers testing on the same machine to interfere
with each other.
So, allow the mockfs to be placed at an arbitrary place given as a
parameter to EnableMockTesting(). In TestMain() we place it under our
existing temporary directory, so we don't need any additional cleanup just
for the mockfs.
fixes#4140
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently MockFSInit always creates the mockfs at the fixed path
/tmp/vc/mockfs. This change allows it to be initialized at any path
given as a parameter. This allows the tests in fs_test.go to be
simplified, because the by using a temporary directory from
t.TempDir(), which is automatically cleaned up, we don't need to
manually trigger initTestDir() (which is misnamed, it's actually a
cleanup function).
For now we still use the fixed path when auto-creating the mockfs in
MockAutoInit(), but we'll change that later.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
virtcontainers/persist/fs/mockfs.go defines a mock filesystem type for
testing. A global variable in virtcontainers/persist/manager.go is used to
force use of the mock fs rather than a normal one.
This patch moves the global, and the EnableMockTesting() function which
sets it into mockfs.go. This is slightly cleaner to begin with, and will
allow some further enhancements.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
storagePathSuffix defines the file path suffix - "vc" - used for
Kata's persistent storage information, as a private constant. We
duplicate this information in fc.go which also needs it.
Export it from fs.go instead, so it can be used in fc.go.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
A number of unit tests under virtcontainers/factory use
MockStorageRootPath() as a general purpose temporary directory. This
doesn't make sense: the mockfs driver isn't even in use here since we only
call EnableMockTesting for the pase virtcontainers package, not the
subpackages.
Instead use t.TempDir() which is for exactly this purpose. As a bonus it
also handles the cleanup, so we don't need MockStorageDestroy any more.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
There are several tests in mount_test.go which perform a sample bind
mount. These need a corresponding unmount to clean up afterwards or
attempting to delete the temporary files will fail due to the existing
mountpoint. Most of them had such an unmount, but
TestBindMountInvalidPgtypes was missing one.
In addition, the existing unmounts where done inconsistently - one was
simply inline (so wouldn't be executed if the test fails too early) and one
is a defer. Change them all to use the t.Cleanup mechanism.
For the dummy mountpoint files, rather than cleaning them up after the
test, the tests were removing them at the beginning of the test. That
stops the test being messed up by a previous run, but messily. Since
these are created in a private temporary directory anyway, if there's
something already there, that indicates a problem we shouldn't ignore.
In fact we don't need to explicitly remove these at all - they'll be
removed along with the rest of the private temporary directory.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The tests in hook_test.go run a mock hook binary, which does some debug
logging to /tmp/mock_hook.log. Currently we don't clean up those logs
when the tests are done. Use a test cleanup function to do this.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
SetupOCIConfigFile creates a temporary directory with os.MkDirTemp(). This
means the callers need to register a deferred function to remove it again.
At least one of them was commented out meaning that a /temp/katatest-
directory was leftover after the unit tests ran.
Change to using t.TempDir() which as well as better matching other parts of
the tests means the testing framework will handle cleaning it up.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Several tests in kata_agent_test.go create /tmp/mountPoint as a dummy
directory to mount. This is not cleaned up after the test. Although it
is in /tmp, that's still a little messy and can be confusing to a user.
In addition, because it uses the same name every time, it allows for one
run of the test to interfere with the next.
Use the built in t.TempDir() to use an automatically named and deleted
temporary directory instead.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
There are always path(symlink) based attacks, so the `safe-path` crate
tries to provde some mechanisms to harden path resolution related code.
Fixes: #3451
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Move the assert_result macro to the shared test_utils file
so that it is not duplicated in individual files.
Fixes: #4093
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
During container exit, the agent tries to remove all the mount point directories,
which can fail if it's a readonly filesytem (e.g. device mapper). This commit ignores
the removal failure and logs a warning message.
Fixes: #4043
Signed-off-by: Feng Wang <feng.wang@databricks.com>
This is to fix a test failure for the
kata-containers-2.0-ubuntu-20.04-s390x-main-baseline jenkins job
Fixes: #4088
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
kata-monitor allows to get data profiles from the kata shim
instances running on the same node by acting as a proxy
(e.g., http://$NODE_ADDRESS:8090/debug/pprof/?sandbox=$MYSANDBOXID).
In order to proxy the requests and the responses to the right shim,
kata-monitor requires to pass the sandbox id via a query string in the
url.
The profiling index page proxied by kata-monitor contains the link to all
the data profiles available. All the links anyway do not contain the
sandbox id included in the request: the links result then broken when
accessed through kata-monitor.
This happens because the profiling index page comes from the kata shim,
which will not include the query string provided in the http request.
Let's add on-the-fly the sandbox id in each href tag returned by the kata
shim index page before providing the proxied page.
Fixes: #4054
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
The scanner reads nothing from viriofsd stderr pipe, because param
'--syslog' rediercts stderr to syslog. So there is no need to write
scanner.Text() to kata log
Fixes: #4063
Signed-off-by: Zhuoyu Tie <tiezhuoyu@outlook.com>
Add test coverage for mount_to_rootfs function in src/mount.rs.
Includes minor refactoring to make function more easily testable.
Fixes#4073
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
Add test coverage for is_signal_handled function in rpc.rs. Includes
refactors to make the function testable and handle additional cases.
Fixes#3939
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
Add test coverage for update_container_namespaces function
in src/rpc.rs. Includes minor refactor to make function easier
to test.
Fixes#4034
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
Adding two functions set_ownership and
recursive_ownership_change to support changing group id
ownership for a mounted volume.
The set_ownership will be called in common_storage_handler
after mount_storage performs the mount for the volume.
set_ownership will be a noop if the FSGroup field in the
Storage struct is not set which indicates no chown will be
performed. If FSGroup field is specified, then it will
perform the recursive walk of the mounted volume path to
change ownership of all files and directories to the
desired group id. It will also configure the SetGid bit
so that files created the directory will have group
following parent directory group.
If the fsGroupChangePolicy is on root mismatch,
then the group ownership will be skipped if the root
directory group id alreasy matches the desired group
id and if the SetGid bit is also set on the root directory.
This is the same behavior as what
Kubelet does today when performing the recursive walk
to change ownership.
Fixes#4018
Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
The fsGroup will be specified by the fsGroup key in
the direct-assign mountinfo metadate field.
This will be set when invoking the kata-runtime
binary and providing the key, value pair in the metadata
field. Similarly, the fsGroupChangePolicy will also
be provided in the mountinfo metadate field.
Adding an extra fields FsGroup and FSGroupChangePolicy
in the Mount construct for container mount which will
be populated when creating block devices by parsing
out the mountInfo.json.
And in handleDeviceBlockVolume of the kata-agent client,
it checks if the mount FSGroup is not nil, which
indicates that fsGroup change is required in the guest,
and will provide the FSGroup field in the protobuf to
pass the value to the agent.
Fixes#4018
Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
This change adds two fields to the Storage pb
FSGroup which is a group id that the runtime
specifies to indicate to the agent to perform a
chown of the mounted volume to the specified
group id after mounting is complete in the guest.
FSGroupChangePolicy which is a policy to indicate
whether to always perform the group id ownership
change or only if the root directory group id
does not match with the desired group id.
These two fields will allow CSI plugins to indicate
to Kata that after the block device is mounted in
the guest, group id ownership change should be performed
on that volume.
Fixes#4018
Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
Add some links to rendered webpages for better user experience,
let users can jump to pages only by clicking links in browsers.
Fixes: #4061
Signed-off-by: bin <bin@hyper.sh>
Add test coverage for the parse_mount_flags_and_options function
in src/mount.rs.
Fixes#4056
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
virtiofsd's debug will be enabled if hypervisor's debug has been
enabled, this will generate too many noisy logs from virtiofsd.
Unbind the relationship of log level between virtiofsd and
hypervisor, if users want to see debug log of virtiofsd,
can set it by:
virtio_fs_extra_args = ["-o", "log_level=debug"]
Fixes: #3303
Signed-off-by: bin <bin@hyper.sh>
This allows the kata-agent to be rebuilt when Cargo "features" is
changed. The Makefile for the agent do not need to specify the
sources for prerequisites by having Cargo check for the sources
changes.
Fixes: #4052
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
The vhost-user-blk can be hotplugged on the PCI bridge successfully on
X86, but failed on Arm. However, hotplugging it on Root Port as a PCIe
device can work well on ARM.
Open the "pcie_root_port" in configuration.toml is needed.
Fixes: #4019
Signed-off-by: Jaylyn Ren <jaylyn.ren@arm.com>
After running make in src/agent, the git working area will be changed:
Untracked files:
(use "git add <file>..." to include in what will be committed)
src/libs/protocols/src/csi.rs
The generated file by `build.rs` should be ignored in git.
Fixes: #3959
Signed-off-by: bin <bin@hyper.sh>
Add test coverage for do_write_stream function of AgentService
in src/rpc.rs. Includes minor refactoring to make function more
easily testable.
Fixes#3984
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
This configuration option is valid for all the hypervisor that are going
to be used with the confidential containers effort, thus exposing the
configuration option for Cloud Hypervisor as well.
Fixes: #4022
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The nix::sys::signal::Signal package api cannot deal with SIGRTMIN+3,
directly use libc function to send the signal.
Fixes: #3990
Signed-off-by: Wang Xingxing <stellarwxx@163.com>
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.
This commit also updates the unit test advice to use `T.TempDir` to
create temporary directory in tests.
Fixes: #3924
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
(default: "/run/containerd/containerd.sock") is duplicated when
printing kata-monitor usage:
[root@kubernetes ~]# kata-monitor --help
Usage of kata-monitor:
-listen-address string
The address to listen on for HTTP requests. (default ":8090")
-log-level string
Log level of logrus(trace/debug/info/warn/error/fatal/panic). (default "info")
-runtime-endpoint string
Endpoint of CRI container runtime service. (default: "/run/containerd/containerd.sock") (default "/run/containerd/containerd.sock")
the golang flag package takes care of adding the defaults when printing
usage. Remove the explicit print of the value so that it would not be
printed on screen twice.
Fixes: #3998
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
getOOMEvents is a long-waiting call, it will retry when failed.
For cases of agent shutdown, the retry should stop.
When the agent hasn't detected agent has died, we can also check
whether the error is "ttrpc: closed".
Fixes: #3815
Signed-off-by: bin <bin@hyper.sh>
Modify the 2Mib in the comment to 4Mib.
VirtioMem is set by configuration file or annotation. And setupVirtioMem is called only when VirtioMem is true.
Fixes: #3750
Signed-off-by: yaoyinnan <yaoyinnan@foxmail.com>
For the library `procfs`, the unit of values in `CpuTime` is ticks,
and we do not know how many ticks per second from metrics because the
`tps` in `CpuTime` is private.
But there are some implements in `CpuTime` for getting these values,
e.g., `user_ms()` for `user`, and `nice_ms()` for `nice`. With these
values, accurate time can be obtained.
Fixes: #3979
Acked-by: zhaojizhuang <571130360@qq.com>
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
Previously, it was not permitted to have neither an initrd nor an image.
However, this is the exact config to use for Secure Execution, where the
initrd is part of the image to be specified as `-kernel`. Require the
configuration of no initrd for Secure Execution.
Also
- remove redundant code for image/initrd checking -- no need to check in
`newQemuHypervisorConfig` (calling) when it is also checked in
`getInitrdAndImage` (called)
- use `QemuCCWVirtio` constant when possible
Fixes: #3922
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
src/runtime/virtcontainers/hook/mock contains a simple example hook in Go.
The only thing this is used for is for some tests in
src/runtime/pkg/katautils/hook_test.go. It doesn't really have anything
to do with the rest of the virtcontainers package.
So, move it next to the test code that uses it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We've now removed the need to install the mock hook binary for unit tests.
However, it turns out that managing that was the *only* thing that the
install and uninstall targets in the virtcontainers Makefile handled.
So, remove them.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Running unit tests should generally have minimal dependencies on
things outside the build tree. It *definitely* shouldn't modify
system wide things outside the build tree. Currently the runtime
"make test" target does so, though.
Several of the tests in src/runtime/pkg/katautils/hook_test.go require a
sample hook binary. They expect this hook in
/usr/bin/virtcontainers/bin/test/hook, so the makefile, as root, installs
the test binary to that location.
Go tests automatically run within the package's directory though, so
there's no need to use a system wide path. We can use a relative path to
the binary build within the tree just as easily.
fixes#3941
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The VC_BIN_DIR variable in the virtcontainers Makefile is almost unused.
It's used to generate TEST_BIN_DIR, and it's created in the install target.
However, we also create TEST_BIN_DIR, which is a subdirectory of VC_BIN_DIR
with mkdir -p, so it will necessarily create VC_BIN_DIR along the way.
So we can drop the unnecessary mkdir and expand the definition of
VC_BIN_DIR in the definition of TEST_BIN_DIR.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The INSTALL_EXEC and UNINSTALL_EXEC definitions from the virtcontainers
Makefile (unlike those from the runtime Makefile in the parent directory)
are entirely unused. Remove them.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The check-go-test target passes the path to the mock hook test binary to
go-test.sh when it invokes it. But go-test.sh just calls run_go_test from
ci/lib.sh, which invokes a script from the tests repo *without* any
parameters.
That is, this parameter is ignored anyway, so remove it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently kata shim v2 doesn't translate ESRCH signal, causing container
fail to stop and shim leak.
Fixes: #3874
Signed-off-by: Feng Wang <feng.wang@databricks.com>
Currently, the block driver option is specifed by hard coding, maybe it
is better to use const string variables instead of hard coded strings.
Another modification is to remove duplicate consts for virtio driver in
manager.go.
Fixes: #3321
Signed-off-by: Jason Zhang <zhanghj.lc@inspur.com>
Add test coverage for get_memory_info function in src/rpc.rs. Includes
some minor refactoring of the function.
Fixes#3837
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
This change introduces the `disable_guest_empty_dir` config option,
which allows the user to change whether a Kubernetes emptyDir volume is
created on the guest (the default, for performance reasons), or the host
(necessary if you want to pass data from the host to a guest via an
emptyDir).
Fixes#2053
Signed-off-by: Evan Foster <efoster@adobe.com>
Let's bring in the latest release of Containerd, 1.6.1, released on
March 2nd, 2022.
With this, we take the opportunity to remove containerd/api reference as
we shouldn't need a separate module only for the API.
Here's the list of changes needed in the code due to the bump:
* stop using `grpc.WithInsecure()` as it's been deprecated
- use `grpc.WithTransportCredentials(insecure.NewCredentials())`
instead
Fixes: #3820
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As this is just a initial vcpu hotplug support, thread and socket has
not been supported. So, don't set socket and thread when hotadd cpu for
arm/virt.
Fixes: #3280
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Mount the direct-assigned block device fs only once and keep a refcount
in the guest. Also use the ro flag inside the options field to determine
whether the block device and filesystem should be mounted as ro
Fixes: #3454
Signed-off-by: Feng Wang <feng.wang@databricks.com>
Translate the volume path from host-known path to guest-known path
and forward the request to kata agent.
Fixes: #3454
Signed-off-by: Feng Wang <feng.wang@databricks.com>
During the container creation, it will parse the mount info file
of the direct assigned volumes and update the in memory mount object.
Fixes: #3454
Signed-off-by: Feng Wang <feng.wang@databricks.com>
Add GetVolumeStats and ResizeVolume APIs for the runtime to query stat
and resize fs in the guest.
Fixes: #3454
Signed-off-by: Feng Wang <feng.wang@databricks.com>
To query fs stats and resize fs, the requests need to be passed to
kata agent through containerd-shim-v2. So we're adding to rest APIs
on the shim management endpoint.
Also refactor shim management client to its own go file.
Fixes: #3454
Signed-off-by: Feng Wang <feng.wang@databricks.com>
In the direct assigned volume scenario, Kata Containers persists
the information required for managing the volume inside the guest
on host filesystem.
Fixes: #3454
Signed-off-by: Feng Wang <feng.wang@databricks.com>
Add commands to add, remove, resize and get stats of a direct-assigned volume.
These commands are expected to be consumed by CSI.
Fixes: #3454
Signed-off-by: Feng Wang <feng.wang@databricks.com>
If, for some reason, we're able to launch cloud hypervisor but not able
to boot the VM up, the virtiofsd process would be left behind.
Let's ensure, via defer, that we stop virtiofsd in case of errors.
Fixes: #3819
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
kata-containers/pulls#3771 added TDX support for Cloud Hypervisor, but
two big things got overlooked while doing that.
1. virtio-fs, as of now, cannot be part of the trust boundary, so the
Confidential Guest will not be using it.
2. virtio-block hotplug should be enabled in order to use virtio-block
for the rootfs (used with the devmapper plugin).
When trying to use cloud-hypervisor with TDX using virtio-fs, we're
facing the following error on the guest kernel:
```
virtiofs virtio2: device must provide VIRTIO_F_ACCESS_PLATFORM
```
After checking and double-checking with virtiofs and cloud-hypervisor
developers, it happens as confidential containers might put some
limitations on the device, so it can't access all of the guests' memory
and that's where this restriction seems to be coming from. Vivek
mentioned that virtiofsd do not support VIRTIO_F_ACCESS_PLATFORM (aka
VIRTIO_F_IOMMU_PLATFORM) yet, and that for ecrypted guests virtiofs may
not be the best solution at the moment.
@sboeuf put this in a very nice way: "if the virtio-fs driver doesn't
support VIRTIO_F_ACCESS_PLATFORM, then the pages corresponding to the
virtqueues and the buffers won't be marked as SHARED, meaning the VMM
won't have access to it".
Interestingly enough, it works with QEMU, and it may be due to some
change done on the patched QEMU that @devimc is packaging, but we won't
take the path to figure out what was the change and patch
cloud-hypervisor on the same way, because of 1.
Fixes: #3810
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As mounting volumes into the guest requires SharedFS setup, let's ensure
we error out if trying to do so in a situation where SharedFS is not
supported.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
supportsSharedFS() is a new method to be used to ensure that no SharedFS
specifics are called when, for a reason or another, Cloud Hypervisor is
in a mode where SharedFSs are not supported.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Similarly to the `createVirtiofsDaemon` and `stopVirtiofsDaemon` methos,
let's introduce and use loadVirtiofsDaemon, at it'll also be handy later
in this series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Similary to the `createVirtiofsDaemon` method, let's introduce and use
its counterpart, as it'll also be handy later in this series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Similarly to what's been done with the `createVirtiofsDaemon`, let's
create a `setupVirtiofsDaemon` one.
It will also become handy later in this series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's introduce and use a new `createVirtiofsDaemon` method. Its name
says it all, and it'll be handy later in this series when, spoiler
alert, SharedFS cannot be used (in such cases as in Confidential
Guests).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit df8ffecde0, as device
hotplug *is* supported and, more than that, is very much needed when
using virtio-blk instead of virtio-fs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
allocate_hugepages() writes to the kernel sysfs file to allocate hugepages
in the Kata VM. However, even if the write succeeds, it's not certain that
the kernel will actually be able to allocate as many hugepages as we
requested.
This patch reads back the file after writing it to check if we were able to
allocate all the required hugepages.
fixes#3816
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
allocate_hugepages() constructs the path for the sysfs directory containing
hugepage configuration, then attempts to create this directory if it does
not exist.
This doesn't make sense: sysfs is a view into kernel configuration, if the
kernel has support for the hugepage size, the directory will already be
there, if it doesn't, trying to create it won't help.
For the same reason, attempting to create the "nr_hugepages" file
itself is pointless, so there's no reason to call
OpenOptions::create(true).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
virtio-fs, instead of virtio-9p, is the default shared file system type
in case virtio-blk is not used.
Fixes: #3813
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Relying on virtio-block is the *only* way to use Firecracker with Kata
Containers, as shared FS (virtio-{fs,fs-nydus,9p}) is not supported by
Firecracker.
As configuration doesn't make sense to be exposed, we hardcode the
`false` value in the Firecracker configuration structure.
Fixes: #3813
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Change `kata-monitor` to listen to port `8090` on the local interface
only by default.
> **Note:**
>
> This is a breaking change as previously it listened on all interfaces.
Fixes: #3795.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Removed redundant and duplicated build options to build
`kata-monitor` the same way as the other components:
- `CGO_ENABLED=0` is not necessary.
- `-buildmode=exe` is not necessary since `BUILDFLAGS` already sets the
build mode.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
- Mostly blank lines after `+build` -- see
https://pkg.go.dev/go/build@go1.14.15 -- this is, to date, enforced by
`gofmt`.
- 1.17-style go:build directives are also added.
- Spaces in govmm/vmm_s390x.go
Fixes: #3769
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
This utility function is also used to check the spec that will run in
the guest - no need for this to be linux specific.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Their types may differ on various host OSes, but
unix.Major|Minor always takes a uint64
Depends-on: github.com/kata-containers/tests#4516
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Add a stub for utils_darwin to facilitate building this package on
Darwin. We can probably drop this empty stub if we have better
abstraction for the various parts of virtcontainers that call it
today...
Fixes:# 3777
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
We need to convert them to uint64 as their types may differ on various
host OSes, but unix.Major|Minor takes a uint64 regardless.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Let's clarify that an error will be reported in case confidential_guest
is enabled, but the hardware where Kata Containers is running doesn't
provide the required feature set.
Fixes: #3787
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's use "Intel TDX" rather than just "TDX", as it can ease the
understanding of the terminology.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's mention the supported TEEs to be used with confidential guests.
Right now, Cloud Hyperisor supports only Intel TDX, used together with
TD Shim.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Nydusd uses a bufio.Scanner to check if nydusd process has
existed, but stderr/stdout passed to Cmd is self-created pipe,
this pipe will not be closed if the process start failing.
Use standard Cmd.StdoutPipe can close the stdout and kata shim
will detect the existence of the nydusd process, then call cmd.Wait to
reap the process' resources.
Fixes: #3783
Signed-off-by: bin <bin@hyper.sh>
The content about systemd in "/proc/self/cgroup" is as:
1:name=systemd:/kubepods/pod1815643d-3789-4e4e-aaf4-00de024912e1/0e15a65bd5f7b30a0b818d90706212354d8b3f0998a1495473c3be9a24706ccf
and in "/prol/self/mountinfo" is as:
30 29 0:26 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime shared:6 - cgroup cgroup rw,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd
The keys extracted from the two files are the same as "name=systemd". So no need to rename the key to "systemd".
Fixes: #3385
Signed-off-by: sailorvii <challengingway@hotmail.com>
A copy and paste mistake was made and the error on HotplugRemoveDevice()
should be about removal and not about addition.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
removes --tags selinux handling in the makefile (part of it introduced here: d78ffd6)
and makes selinux configurable via configuration.toml
Fixes: #3631
Signed-off-by: Tanweer Noor <tnoor@apple.com>
Switching to the generic FilesystemSharer brings 2 majors improvements:
1. Remove container and sandbox specific code from kata_agent.go
2. Allow for non Linux implementations to provide ways to share
container files and root filesystems with the Kata Linux guest.
Fixes#3622
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
With the Linux implementation of the FilesystemSharer interface, we can
now remove all host filesystem sharing code from kata_agent and keep it
where it belongs: sandbox.go.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
This gathers the current kata agent and container filesystem sharing
code into a FilesystemSharer implementation.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Filesystem sharing here means the ability to share some parts of the
host filesystem with the guest. It's mostly about sharing files and
container bundle root filesystems.
In order to allow for different file and rootfs sharing implementations,
we define a FilesystemSharer interface.
This interface provides a preparation step, where concrete
implementations will be able to e.g. prepare the host filesysstem.
Then it provides 2 methods, one for sharing any file (regular file or a
directory) and another one for sharing a container root filesystem
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Let's enable TDX support for Cloud Hypervisor, using td-shim as its
desired firmware.
Fixes: #3632
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
"firmware" option was already present for a while, but it's never been
exposed to the configuration file before.
Let's do it now as it can be used, in combination with the newly added
confidential_guest option, to boot a guest VM using the so called
`td-shim`[0] with Cloud Hypervisor.
[0]: https://github.com/confidential-containers/td-shim
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
NVDIMM is also not supported with Confidential Guests and Virtio Block
devices should be used instead.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Similarly to VCPUs and Device hotplug, Confidential Guests also do not
support Memory hotplug.
Let's make it clear in the documentation and guard the code on both QEMU
and Cloud Hypervisor side to ensure we don't advertise Memory hotplug as
being supported when running Confidential Guests.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Similarly to VCPUs hotplug, Confidential Guests also do not support
Device hotplug.
Let's make it clear in the documentation and guard the code on both QEMU
and Cloud Hypervisor side to ensure we don't advertise Device hotplug as
being supported when running Confidential Guests.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As confidential guests do not support VCPUs hotplug, let's set the
"DefaultMaxVCPUs" value to "NumVCPUs".
The reason to do this is to ensure that guests will be started with the
correct amount of VCPUs, without giving to the guest with all the
possible VCPUs the host could provide.
One clear side effect of this limitation is that workloads that would
require more VCPUs on their yaml definition will not run on this
scenario.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
ConfidentialGuest is an option already present and exposed for QEMU,
which is used for using Kata Containers together with different sorts of
Guest Protections, such as TDX and SEV for x86_64, PEF for ppc64le, and
SE for s390x.
Right now we error out in case confidential_guest is enabled, as we will
be implementing the needed blocks for this as part of this series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is a small code refactor removing a deadcode based the checks
already done in the generic hypervisor abstraction.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The hypervisor code already defines 3 common kernel root params for the
following cases:
* NVDIMM
* NVDIMM without DAX support
* Virtio Block
As parameters used for cloud-hypervisor have an overlap with the ones
provided by the NVDIMM case, let's take advantage of that.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's bump the Cloud Hypervisor version to 5343e09e7b8db, as that brings
a few fixes we're interested in, such as:
* hypervisor, vmm: Handle TDX hypercalls with INVALID_OPERAND
- https://github.com/cloud-hypervisor/cloud-hypervisor/pull/3723
- This is needed for the TDX support on the cloud hypervisor driver,
which is part of this very same series.
* openapi: Update the PciBdf types
- https://github.com/cloud-hypervisor/cloud-hypervisor/pull/3748
- This is needed due to a change in a DeviceNode field, which would
cause a marshalling / demarshalling error when running with a
version of cloud-hypervisor that includes the TDX fixes mentioned
above.
* scripts: dev_cli: Don't quote $features_build
* scripts: dev_cli: Add --features option
- https://github.com/cloud-hypervisor/cloud-hypervisor/pull/3773
- This is needed due to changes in the scripts used to build Cloud
Hypervisor, which are used as part of Kata Containers CIs and
github actions.
Due to this change, we're also adapting the build scripts as part
of this very same commit.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
in TestHandleHugepages. On s390x, hugepage sizes must be set at boot, so
test with any that are present (default is 1M).
Depends-on: github.com/kata-containers/kata-containers#3770
Fixes: #3763
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
We introduced collection of sandboxes metadata from the CRI that will be
attached to the sandbox metrics: this will allow to immediately match
sandboxes metrics with CRI workloads.
Rename the symbols from *Kube* to *CRI* as the metadata will be there
every time pods are created through CRI, also if kubernetes is not
installed (e.g., 'crictl runp').
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
To resourcecontrol, and make it consistent with the fact that cgroups
are a Linux implementation of the ResourceController interface.
Fixes: #3601
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
We call it a ResourceController, and we make it not so Linux specific.
Now the Linux implementations is the cgroups one.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
We now rely on fs events only to update the sandbox cache. This is not
true anyway for sandboxes already present at kata-monitor startup: we
just retrieve the list and add them in the cache only when we get their
CRI metadata. If CRI metadata is not available we will never add them to
the sandbox cache.
Fix this by immediately adding the sandboxes we find at startup time to
the sandbox cache.
Fixes: #3705
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
Since kata-monitor now:
- relies on fs events *only* to update the sandbox cache
- adds CRI meta-data as labels (CRI pod name, namespace and uid)
it deserves a version bump.
Note that while we could let kata-monitor match the runtime version,
kata-monitor will usually work flawlessy with different kata shim
releases: so it makes sense to keep kata-monitor version separated.
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
CRI-O start shim process without setting TTRPC_ADDRESS,
that the forwarding events goroutine will get errors.
For CRI-O runtime, we can log the events to log file.
Fixes: #3733
Signed-off-by: bin <bin@hyper.sh>
As kata with qemu has supported lazyload, so this pr aims to
bring lazyload ability to kata with clh.
Fixes#3654
Signed-off-by: luodaowen.backend <luodaowen.backend@bytedance.com>
As the container runtime, we're never inspecting, adding or configuring
host networking endpoints.
Make sure we're always do that by wrapping addSingleEndpoint calls into
the pod network namespace.
Fixes#3661
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Keep all the OS agnostic bits in the hypervisor.go and
hypervisor_ARCH.go files.
Fixes#3614
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Some of them (e.g. QEMU) can run on other OSes (e.g. Darwin) but the
current virtcontainers implementation is Linux specific.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Even though it's still actually defined as the QEMU upper bound,
it's now abstracted away through govmm.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Various improvements to the top-level README file:
- Moved the following sections from the runtime's README to the
top-level README:
- License
- Platform support / Hardware requirements
- Added the following sections to the top-level README:
- Configuration
- Hypervisors
- Improved formatting of the Documentation section in the top-level
README.
- Removed some unused named links from the top-level README.
Also improvements to the runtime README:
- Removed confusing mention of the old 1.x runtime name.
- Clarify the binary name for the 2.x runtime and the utility program.
> **Note:**
>
> We cannot currently link to the AMD website as that site's
> configuration causes the CI static checks to fail. See
> https://github.com/kata-containers/tests/issues/4401Fixes: #3557.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Mount hugepage directories and configure the requested number of hugepages
dynamically by writing to sysfs files
Port from:
78b307b5bdFixes: #3342
Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Signed-off-by: bin <bin@hyper.sh>
Support hugepages and port from:
96dbb2e8f0Fixes: #3342
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Signed-off-by: bin <bin@hyper.sh>
Current hook process is handled by just calling
unwrap() on it, sometime it will cause panic.
By handling all Result type and check the error can
avoid panic.
Fixes: #3649
Signed-off-by: bin <bin@hyper.sh>
Envs contain null-byte will cause running hooks to panic,
this commit will filter envs and only pass valid envs to hooks.
Fixes: #3667
Signed-off-by: bin <bin@hyper.sh>
The OCI spec is very specific about it:
"The prestart hooks MUST be executed in the runtime namespace."
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
That allows us to amend those annotations with information that could be
used when running those hooks.
For example nerdctl will use those annotations to resolve the networking
namespace path in where to run the CNI plugin, i.e. the created pod
networking namespace.
Fixes#3629
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
German Maglione, one of the current virtio-fs developers, has brought to
our attention that using "announce-submounts" could help us to prevent
inode number collisions.
This feature was introduced a year ago or so by Hanna Reitz as part of
the 08dce386e77eb9ab044cb118e5391dc9ae11c5a8, and as we already mandate
QEMU >= 6.1.0, let's take advantage of that.
Fixes: #3507
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When building with `ARCH=x86_64`, the previous `Makefile` will use it
without checking and cause:
Makefile:319: *** "ERROR: No hypervisors known for architecture x86_64 (looked for: acrn firecracker qemu cloud-hypervisor)". Stop.
This commit fix the above issue by checking `ARCH` no matter where it
is assigned.
Fixes: #3444
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
The distro constraint parses os release files, which may not contain
distro version(VERSION_ID field), for example rolling release distributions
like Debian testing, archlinux.
These distro constraints are not used anyway, so removing them instead
of fixing the complex version detection.
Fixes: #1864
Signed-off-by: Shengjing Zhu <zhsj@debian.org>
Let's update cloud-hypervisor to a version that exposes the TDx support
via the OpenAPI's auto-generated code.
Fixes: #3663
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Change the variables `mountTypeFieldIdx := 8`, `mntDestIdx := 4` and `netNsMountType := "nsfs"` to const.
And unify the variable naming style, modify `mntDestIdx` to `mountDestIdx`.
Fixes: #3646
Signed-off-by: yaoyinnan <yaoyinnan@foxmail.com>
Pulling image is the most time-consuming step in the container lifecycle. This PR
introduse nydus to kata container, it can lazily pull image when container start. So it
can speed up kata container create and start.
Fixes#2724
Signed-off-by: luodaowen.backend <luodaowen.backend@bytedance.com>
Relative links within this repository allow for easier navigation to
the corresponding file / directory in the current commit / for the
selected version.
Link text was slightly changed / fixed in
- docs/Unit-Test-Advice.md
- docs/how-to/how-to-run-docker-with-kata.md
Fixes#3045
Signed-off-by: Daniel Höxtermann <daniel@hxtm.dev>
In commit 78dff468bf1 we introduced logic to rewrite PCIDEVICE_ environment
variables for the container so that they contain correct addresses for the
Kata VM rather than for the host. Unfortunately, we never actually invoked
the function to do this.
It turns out we need to do this not only at container creation time, but
also for environment variables supplied to processes exec-ed into the
container after creation (e.g. with crictl exec). Add calls to make both
those updates.
fixes#3634
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
add_devices() generates a mapping of host to guest PCI addresses which is
used to update some environment variables for the workload. Currently it
just does this locally, but it turns out we're going to need the same map
again in order to correct environment variables for processes exec-ed into
the existing container.
Move the map to the sandbox structure so we can keep it around for those
later uses.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This function updates PCIDEVICE_ environment variables (such as those
supplied by the Kubernetes SR-IOV plugin) in the OCI spec to be correct
for the Kata VM, rather than for the host.
We neglected to actually call this function, however, and it turns out that
when we do, we need to do things slightly different. We actually need to
adjust envionment variables both in the OCI spec when creating a container
and also in the variables supplied for exec-ing a new process within an
existing container.
Adjust the function so that it can be used for both these cases.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We don't need to call NewNetwork() twice, and we can have the VM factory
case return immediatly. That makes the code more readable.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Move the netlink dependent code into network_linux.go.
Other OSes will have to provide the same functions.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
And only have AddEndpoints/RemoveEndpoints for all cases (single
endpoint vs all of them, hotplug or not).
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
We are converting the Network structure into an interface, so that
different host OSes can have different networking implementations for
Kata.
One step into that direction is to rename all the Network structure
fields and methods to something that is less Linux networking namespace
specific. This will make the Network interface naming consistent.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
We are replacing the NetworkingNamespace structure with the Network
one, so we should have the hypervisor interface switching to it as well.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Remove unused parameters.
Reduce the number of parameters by deriving some of them (e.g. a
networking config) from their outer structure (e.g. a Sandbox
reference).
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Endpoints creations, attachement and hotplug are bound to the networking
namespace described through the Network structure.
Making them Network methods is natural and simplifies the code.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
For simplicity sake, there should only be one networking structure per
sandbox, as opposed to two (Network and NetworkingNamespace) currently.
This commit start expanding the Network structure in order to eventually
make it the single representation of a virtcontainers sandbox
networking.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Runtime now accepts both `1` and `Y` as valid values for
kvm_amd module parameter kvm_amd.sev.
Fixes#3273
Signed-off-by: Pierre Kohler <pierre.kohler@cysec.systems>
On s390x, skip adding a virtio-rng device. The on-chip CPACF provides
entropy instead. For Confidential Containers, when using Secure
Execution, entropy attacks on virtio-rng are mitigated.
Fixes: #3598
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
firmware can be split into FIRMWARE_VARS.fd (UEFI variables as
configuration) and FIRMWARE_CODE.fd (UEFI program image). UEFI
variables can be customized per each user while UEFI code is kept same.
fixes#3583
Signed-off-by: Julio Montes <julio.montes@intel.com>
Move the netns specific bits into a Linux specific file.
Fixes: #3607
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
There are software and hardware architectures which do not support
dynamically adjusting the CPU and memory resources associated with a
sandbox. For these, today, they rely on "default CPU" and "default
memory" configuration options for the runtime, either set by annotation
or by the configuration toml on disk.
In the case of a single container (launched by ctr, or something like
"docker run"), we could allow for sizing the VM correctly, since all of
the information is already available to us at creation time.
In the sandbox / pod container case, it is possible for the upper layer
container runtime (ie, containerd or crio) could send a specific
annotation indicating the total workload resource requirements
associated with the sandbox creation request.
In the case of sizing information not being provided, we will follow
same behavior as today: start the VM with (just) the default CPU/memory.
If this information is provided, we'll track this as Workload specific
resources, and track default sizing information as Base resources. We
will update the hypervisor configuration to utilize Base+Workload
resources, thus starting the VM with the appropriate amount of CPU and
memory.
In this scenario (we start the VM with the "right" amount of
CPU/Memory), we do not want to update the VM resources when containers
are added, or adjusted in size.
This functionality is introduced behind a configuration flag,
`static_sandbox_resource_mgmt`. This is defaulted to false for all
configurations except Firecracker, which is set to true.
This'll greatly improve UX for folks who are utilizing
Kata with a VMM or hardware architecture that doesn't support hotplug.
Note, users will still be unable to do in place vertical pod autoscaling
or other dynamic container/pod sizing with this enabled.
Fixes: #3264
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Add the POD metadata we get from the container manager to the metrics by
adding more labels.
Fixes: #3551
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
Kata-monitor detects started and terminated kata pods by monitoring the
vc/sbs fs (this makes sense since we will have to access that path to
access the sockets there to get the metrics from the shim).
While kata-monitor updates its sandbox cache based on the sbs fs events,
it will schedule also a sync with the container manager via the CRI in
order to sync the list of sandboxes there.
The container manager will be the ultimate source of truth, so we will
stick with the response from the container manager, removing the
sandboxes not reported from the container manager.
May happen anyway that when we check the container manager, the new kata
pod is not reported yet, and we will remove it from the kata-monitor pod
cache. If we don't get any new kata pod added or removed, we will not
check with the container manager again, missing reporting metrics about
that kata pod.
Let's stick with the sbs fs as the source of truth: we will update the
cache just following what happens on the sbs fs.
At this point we may have also decided to drop the container manager
connection... better instead to keep it in order to get the kube pod
metadata from it, i.e., the kube UID, Name and Namespace associated with
the sandbox.
Every time we get a new sandbox from the sbs fs we will try to retrieve the
pod metadata associated with it.
Right now we just attach the container manager sandbox id as a label to
the exposed metrics, making hard to link the metrics to the running pod
in the kubernetes cluster.
With kubernetes pod metadata we will be able to add them as labels to map
explicitly the metrics to the kubernetes workloads.
Fixes: #3550
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
We currently WARN about unexpected fs events, which includes CHMOD
operations (which should be actually expected...).
Just ignore all the fs events we don't care about without any warn.
We dump all the events with debug log in any case.
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
Improve debug log formatting of the sandbox cache update process.
Move raw and tracing logs from the DEBUG to the TRACE log level.
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
The OCI container spec specifies a limit of -1 signifies
unlimited memory. Update the sandbox memory calculator
to reflect this part of the spec.
Fixes: #3512
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
1. The hook.args[0] is the hook binary name which shouldn't be included
in the Command.args.
2. Add new unit tests
Fixes: #2610
Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
no explicit PCI test, just switch path depending on architecture
(CCW for s390x, PCI for others). Also fixes an unknown variable error.
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
When no options are passed to -ldflags, it passes
incorrect values(in this case, $BUILDFLAGS) to it.
Fix passing empty values by passing $KATA_LDFLAGS
in quotes.
Fixes: #3521
Signed-off-by: Amulya Meka <amulmek1@in.ibm.com>
For now a bunch of tests are simply not working.
Let's skip them all, and re-enable them once
kata-containers/kata-containers/issues/3500 gets fixed.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Both projects follow the same license, Apache-2.0, but the header saying
that comes from govmm is different from the one expected for the tests
present on the kata-containers repo.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
govet checks have been ignored on govmm repo, but those are enabled on
kata-containers one. So, in order to avoid failing our CIs let's just
keep ignoring the checks for the govmm structs and have an issue opened
for fixing it whenever someone has cycles to do it.
The important bit here is, we're not making anything worse that it
already is. :-)
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
govmm, from now on, should follow the same guidelines from contributing,
copying, and etc as kata-containers does.
The go.mod is not needed anymore as the project lives inside the
runtime.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's stop using govmm from kata-containers/govmm and let's start using
it from our own repo.
Fixes: #3495
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
`enable_swap` option was added long time ago to add
`-realtime mlock=off` to the QEMU's command line.
Kata now supports QEMU 6, `-realtime` option has been deprecated and
`mlock=on` is causing unexpected behaviors in kata.
This patch removes support for `enable_swap`, `-realtime` and `mlock=`
since they are causing bugs in kata.
Signed-off-by: Julio Montes <julio.montes@intel.com>
for linking. Required for basic KVM checks on some kernels (e.g. the
one RHEL is currently shipping), cf.
6621441db5/target/s390x/kvm/meson.build (L15-L16).
Fixes: #3469
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
bring SGX support and other fixes
shortlog:
8939b0f qemu: add support for SGX
b17f073 qemu: update readonly flag for block devices
f971801 qemu: only set wait parameter for server mode socket based
char device
82cc01d qemu: Fix 32 bit int overflow in test file
1d1a231 qemu: Add support for legacy serial device
9a2bbed qemu: Remove -realtime in favor of -overcommit
fe83c20 qemu: Add support for --no-shutdown Knob
1ed5271 qmp: wait for POWERDOWN event in ExecuteSystemPowerdown()
fixes#3080
Signed-off-by: Julio Montes <julio.montes@intel.com>
Generated protocols files should not be inclued in Git repo.
And also add Cargo.lock in oci/protocols directory to .gitignore.
Fixes: #3422
Signed-off-by: bin <bin@hyper.sh>
When Sandbox AddInterface() is called, it may fail after endpoint.HotAttach,
we'd better rollback and call save() in the end.
Fixes: #3419
Signed-off-by: yangfeiyu <yangfeiyu20102011@163.com>
After the protocols are moved to upper libs (PR3355),
the runtime protocol generation is broken. This fixes it.
Fixes: #3414
Signed-off-by: Feng Wang <feng.wang@databricks.com>
This PR removes the reference of how to use disable_new_netns
configuration with docker as for kata 2.0 we are not supporting docker
and this information was used for kata 1.x
Fixes#3400
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
move the protocols to upper libs thus it can
be shared between agent and other rust runtime.
Depends-on: github.com/kata-containers/tests#4306
Fixes: #3348
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Move the oci crate to upper libs thus it can be
shared between agent and other rust runtimes.
Fixes: #3348
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Update our containerd vendoring. In particular, we're interested in
grabbing the updated annotation definitions for defining sandbox sizing.
- go get github.com/containerd/containerd@v1.6.0-beta.4
- edit go.mod to remove containerd v1.5.8 replacement directive
- go mod vendor
- go mod tidy
Fixes: #3276
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Restore Debian as a rootfs.
1. revert of #3154, but some change
2. update debian version to 10.11
3. update `libstdc++-6-dev` to `libstdc++-8-dev`
4. changes discarded in QAT are not restored
Fixes: #3372
Signed-off-by: zhaojizhuang <571130360@qq.com>
The tokio's spawn will only create an future async task
instead of a new real thread, thus executing unshare to
create a new namespace in tokio's async task would make
the agent process to join in the new created namespace,
which isn't expected.
Thus, we'd better to to the unshare in a real thread to
prevent moving the agent process into a new namespace.
Fixes: #3369
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
For calls from shim to agent, the return error will be processed like this:
match self.do_start_container(req).await {
Err(e) => Err(ttrpc_error(ttrpc::Code::INTERNAL, e.to_string())),
Ok(_) => Ok(Empty::new()),
}
The e.to_string() return only a part of the error(for example set by context()),
this may lead lack of information.
The `format!("{:?}", err)` will return more info.
Fixes: #3353
Signed-off-by: bin <bin@hyper.sh>
We already have a `mount_path` local Path variable which holds the mount
point.
Use it instead of creating a new `mount_point` variable with identical
type and content.
Fixes: #3332
Signed-off-by: Dov Murik <dovmurik@linux.ibm.com>
Make it clear when reading the table in the agent's "Change the agent
API" documentation that the commands in the "Generation method" column
should be run in the agent repo.
Fixes: #3317.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add span name to logging error to help with debugging when the context
is not set before the span is created.
Fixes#3289
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Move the architecture document into a new `docs/design/architecture/` directory
in preparation for splitting it into more manageable pieces.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
not clear why this was commented out before -- ensure that we set
approprate annotation on the sandbox container's annotations to indicate
this is a sandbox.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Today we assume that if the CRI/upper layer doesn't provide a container
type annotation, it should be treated as a sandbox. Up to this point, a
sandbox with a pause container in CRI context and a single container
(ala ctr run) are treated the same.
For VM sizing and container constraining, it'll be useful to know if
this is a sandbox or if this is a single container.
In updating this, we cleanup the type handling tests and we update the
containerd annotations vendoring.
Fixes: #2926
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Some new attributes are added to hypervisor config:
- VMStorePath
- RunStorePath
- SharedPath
These attributes should be handled in two places:
- reset when check the new hypervisor's config is suitable
to the base config.
- copy from new hypervisor's config when create new VM
Fixes: #3193
Signed-off-by: bin <bin@hyper.sh>
The latest release of openapi-generator v5.3.0 contains the fix for
`dropping err` bug [1]. This patch also re-generated the client code of
Cloud Hypervisor to have the bug fixed.
[1] https://github.com/OpenAPITools/openapi-generator/pull/10275Fixes: #3201
Signed-off-by: Bo Chen <chen.bo@intel.com>
vhost-net is disabled in the rootless kata runtime feature, which has been abandoned since kata 2.0.
I reused the rootless flag for nonroot hypervisor and would like to enable vhost-net.
Fixes#3182
Signed-off-by: Feng Wang <feng.wang@databricks.com>
In function `update_target`, if the updated source is a directory,
we should create the corresponding directory.
Fixes: #3140
Signed-off-by: bin <bin@hyper.sh>
This PR updates the comments in the configuration.toml to point to
the current kata containers repository instead of the kata 1.x.
Fixes#3163
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Currently we do not have debian as part of the kata CI as we
do not have a mantainer, this PR removes debian as a supported
rootfs in order to have only the distros that we are supporting
and mantainining.
Fixes#3153
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Wrap `nix` `Error`'s in an `anyhow` error for consistency with the way
`rustjail` handles errors.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Replace `Result` values that use a "bare" `nix` `Error` like this:
```rust
return Err(nix::Error::EINVAL.into());
```
... to the following which wraps the nix` error in an `anyhow` call for
consistency with the other errors returned by `rustjail`:
```rust
return Err(anyhow!(nix::Error::EINVAL));
```
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Remove a bare `return` from a test function. This looks wrong but isn't
because the callers are all tests that just wait for a state change
caused by this test function.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Running `cargo audit` showed that the `nix` package for the agent and
the `rustjail` and `vsock-exporter` local crates need to be updated to
resolve rust security issue
[RUSTSEC-2021-0119](https://rustsec.org/advisories/RUSTSEC-2021-0119).
Hence, bumped `nix` to the latest version (which required changes to
work with the new, simpler `errno` handling).
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Run `cargo update` to update to the latest crate dependency versions.
The agent is an application so this includes expanding the partially
specified semvers to full semver values for the following crates,
which makes those crates consistent with the other agent dependencies:
- `futures`
- `regex`
- `scan_fmt`
- `tokio`
Fixes: #3124.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Remove the format specifier in the `"failed to get VFIO group"` error
returned by `vfio_device_handler()`.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Some tests in sandbox.rs need root user to run, because they need create
directories under /run/agent directories, actually this is a limit
that shouldn't be there. By using a temp directory for test containers
will not need run tests as root user.
Fixes: #3122
Signed-off-by: bin <bin@hyper.sh>
Kata agent logs unknown system calls given by seccomp profiles
in advance before the log file descriptor closes.
Fixes: #2957
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
os.Exit() will terminate program immediately, the defer functions
won't be executed, so we add defer functions again before os.Exit().
Refer to https://pkg.go.dev/os#ExitFixes: #3059
Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
Replace some `unwrap()` and `expect()` calls with code to return the
error to the caller.
Fixes: #3011.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Improved the `verify_cid()` function that validates container ID's by
removing the need for an `unwrap()`.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Change `baremount()` to accept `Path` values rather than string values
since:
- `Path` is more natural given the function deals with paths.
- This minimises the caller having to convert between string and `Path`
types, which simplifies the surrounding code.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The unrecognized option: 'deny-warnings' args caused `make optimize` failed.
Fixed the Makefile of the agent project, make sure the `make optimize` command
execute correctly. This PR modify the rustc args from '--deny-warnings' to
'--deny warnings'.
Fixes: #3104
Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
Function parameters in test config is not used. This
commit will add under score before variable name
in test config.
Fixes: #3091
Signed-off-by: bin <bin@hyper.sh>
1. use ci/go-test.sh to replace the direct call to go test
2. fix data race test
3. install hook whether it is root or not
Fixes#1494
Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
This'll end up moving to hypervisors pkg, but let's stop using virtLog,
instead introduce hvLogger.
Fixes: #2884
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Today the hypervisor code in vc relies on persist pkg for two things:
1. To get the VM/run store path on the host filesystem,
2. For type definition of the Load/Save functions of the hypervisor
interface.
For (1), we can simply remove the store interface from the hypervisor
config and replace it with just the path, since this is all we really
need. When we create a NewHypervisor structure, outside of the
hypervisor, we can populate this path.
For (2), rather than have the persist pkg define the structure, let's
let the hypervisor code (soon to be pkg) define the structure. persist
API already needs to call into hypervisor anyway; let's allow us to
define the structure.
We'll probably want to look at following similar pattern for other parts
of vc that we want to make independent of the persist API.
In doing this, we started an initial hypervisors pkg, to hold these
types (avoid a circular dependency between virtcontainers and persist
pkg). Next step will be to remove all other dependencies and move the
hypervisor specific code into this pkg, and out of virtcontaienrs.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Today, acrn relies on sandbox level information, as well as a store
provided by common parts of the hypervisor. As we cleanup the
abstractions within our runtime, we need to ensure that there aren't
cross dependencies between the sandbox, the persistence logic and the
hypervisor.
Ensure that ACRN still compiles, but remove the setSandbox usage as
well as persist driver setup.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
We noticed s390x test failures on several of the watcher unit tests.
Discovered that on s390 in particular, if we update a file in quick
sucecssion, the time stampe on the file would not be unique between the
writes. Through testing, we observe that a 20 millisecond delay is very
reliable for being able to observe the timestamp update. Let's ensure we
have this delay between writes for our tests so our tests are more
reliable.
In "the real world" we'll be polling for changes every 2 seconds, and
frequency of filesystem updates will be on order of minutes and days,
rather that microseconds.
Fixes: #2946
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
The k8s SR-IOV plugin, when it assigns a VFIO device to a container, adds
an variable of the form PCIDEVICE_<identifier> to the container's
environment, so that the payload knows which device is which. The contents
of the variable gives the PCI address of the device to use.
Kata allows VFIO devices to be passed in to a Kata container, however it
runs within a VM which has a different PCI topology. In order for the
payload to find the right device, the environment variables therefore need
to be converted to list the guest PCI addresses instead of the host PCI
addresses.
fixes#2897
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
update_spec_devices() takes a bunch of updates for the device entries in
the OCI spec and applies them, adjusting things in both the linux.devices
and linux.resources.devices sections of the spec.
It's important that each entry in the spec only be updated once. Currently
we ensure this by first creating an index of where the entries are, then
consulting that as we apply each update, so that earlier updates don't
cause us to incorrectly detect an entry as being relevant to a later
update. This method works, but it's quite awkward.
This inverts the loop structure in update_spec_devices() to make this
clearer. Instead of stepping through each update and finding the relevant
entries in the spec to change, we step through each entry in the spec and
find the relevant update. This makes it structurally clear that we're only
updating each entry once.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We have a test case commented as testing the case where linux.devices is
empty in the OCI spec. While it's true that linux.devices is empth in this
example, the reason it fails isn't specifically because it's empty but
because it doesn't contain a device for the update we're trying to apply.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
update_spec_devices() explicitly checks for being called with an empty
container path and fails. We have a unit test to verify this behaviour.
But while an empty container_path probably does mean something has gone
wrong elsewhere, that's also true of any number of other bad paths. Having
an empty string here doesn't prevent what we're doing in this function
making sense - we can compare it to the strings in the OCI spec perfectly
well (though more likely we simply won't find it there).
So, there's no real reason to check this one particular odd case.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The DevIndex data structure keeps track of devices in the OCI
specification. We used to carry it around to quite a lot of
functions, but it's now used only within update_spec_devices(). That
means we can simplify things a bit by just open coding the maps we
need, rather than declaring a special type.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
update_spec_device() adjusts the OCI spec for device differences
between the host and guest. It is called repeatedly for each device
we need to alter. These calls are now all in a single loop in
add_devices(), so it makes more sense to move the loop into a renamed
update_spec_devices() and process all the fixups in one call.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently the DevNumUpdate structure is created with a path to a
device node in the VM, which is then used by update_spec_device().
However the only piece of information that update_spec_device()
actually needs is the VM side major and minor numbers for the device.
We can determine those when we create the DevNumUpdate structure.
This means we detect errors earlier and as a bonus we don't need to
make a copy of the vm path string.
Since that change requires updating 2 of the log statements, we take the
opportunity to update all the log statements to structured style.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
For each device in the OCI spec we need to update it to reflect the guest
rather than the host. We do this with additional device information
provided by the runtime. There should only be one update for each device
though, if there are multiple, something has gone horribly wrong.
Detect and report this situation, for safety.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
As we process container devices in the agent, we repeatedly call
update_spec_device() to adjust the OCI spec as necessary for differences
between the host and the VM. This means that for the whole of a pretty
complex call graph, the spec is in a partially-updated state - neither
fully as it was on the host, not fully as it will be for the container
within the VM.
Worse, it's not discernable from the contents itself which parts of the
spec have already been updated and which have not. We used to have real
bugs because of this, until the DevIndex structure was introduced, but that
means a whole, fairly complex, parallel data structure needs to be passed
around this call graph just to keep track of the state we're in.
Start simplifying this by having the device handler functions not directly
update the spec, but instead return an update structure describing the
change they need. Once all the devices are added, add_devices() will
process all the updates as a batch.
Note that collecting the updates in a HashMap, rather than a simple Vec
doesn't make a lot of sense in the current code, but will reduce churn
in future changes which make use of it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently update_spec_device() takes parameters 'vm_path' and 'final_path'
to give it the information it needs to update a single device in the OCI
spec for the guest. This bundles these parameters into a single structure
type describing the updates to a single device. This doesn't accomplish
much immediately, but will allow a number of further cleanups.
At the same time we change the representation of vm_path from a Unicode
string to a std::path::Path, which is a bit more natural since we are
performing file operations on it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
virtio_blk_device_handler(), virtio_blk_ccw_device_handler() and
virtio_scsi_device_handler() all take a clone of their 'device' parameter.
They appear to do this in order to get a mutable copy in which they can
update the vm_path field.
However, the copy is dropped at the end of the function, so the only thing
that's used in it is the vm_path field passed to update_spec_device()
afterwards.
We can avoid the clone by just using a local variable for the vm_path.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
update_spec_device() takes a 'final_path' parameter which gives the
name the device should be given in the "inner" OCI spec. We need this
for VFIO devices where the name the payload sees needs to match the
VM's IOMMU groups. However, in all other cases (for now, and maybe
forever), this is the same as the original 'container_path' given in
the input OCI spec. To make this clearer and simplify callers, make
this parameter an Option, and only update the device name if it is
non-None.
Additionally, update_spec_device() needs to call to_string() on
update_path to get an owned version. Rust convention[0] is to let the
caller decide whether it should copy, or just give an existing owned
version to the function. Change from &str to String to allow that; it
doesn't buy us anything right now, but will make some things a little
nicer in future.
[0] https://rust-lang.github.io/api-guidelines/flexibility.html?highlight=clone#caller-decides-where-to-copy-and-place-data-c-caller-control
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
update_spec_device() takes a 'host_path' parameter which it uses to locate
the device to correct in the OCI spec. Although this will usually be the
path of the device on the host, it doesn't have to be - a traditional
runtime like runc would create a device node of that name in the container
with the given (host) major and minor numbers. To clarify that, rename it
to 'container_path'.
We also update the block comment to explain the distinctions more
carefully. Finally we update some variable names in tests to match.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This error is returned if we have information for a device from the
runtime, but a matching device does not appear in the OCI spec. However,
the name for the device we print is the name from the VM, rather than the
name from the container which is what we actually expect in the spec.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
update_spec_devices() includes an unsafe block, in order to call the libc
functions to get the major and minor numbers from a device ID. However,
the nix crate already has a safe wrapper for this function, which we use in
other places in the file.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
- Even a directory could be a symlink - check for this. This is very
common when using configmaps/secrets
- Add unit test to better mimic a configmap, configmap update
- We would never remove directories before. Let's ensure that these are
added to the watched_list, and verify in unit tests
- Update unit tests which exercise maximum number of files per entry. There's a change
in behavior now that we consider directories/symlinks watchable as well.
For these tests, it means we support one less file in a watchable mount.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
The current implementation just copies the file, dereferencing any
simlinks in the process. This results in symlinks no being preserved,
and a change in layout relative to the mount that we are making
watchable.
What we want is something like "cp -d"
This isn't available in a crate, so let's go ahead and introduce a copy
function which will create a symlink with same relative path if the
source file is a symlink. Regular files are handled with the standard
fs::copy.
Introduce a unit test to verify symlinks are now handled appropriately.
Fixes: #2950
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
If a file/directory doesn't exist, os.Stat() returns an
error. Assert the returned value with os.IsNotExist() to
prevent it from failing.
Fixes: #2920
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
When the container didn't had a tty console, it would be in a same
process group with the kata-agent, which wasn't expected. Thus,
create a new session for the container process.
Fixes: #3063
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Many of these functions are just used on one place throughout the rest
of the code base. If we create hypervisor package, newtork package, etc, we may want to
parse this out.
Fixes: #3049
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
This will be useful at runtime level; no need for oci or uuid to be subpkg of
virtcontainers.
While at it, ensure we run gofmt on the changed files.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Update the agent README by removing the historical details about the
conversion from golang to rust which (occurred at the start of Kata 2.x
development) and replacing it with information that developers and
testers should find more useful.
Fixes: #3056.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The fact that we need to "bridge" the endpoint is a bit irrelevant. To
be consistent with the rest of the endpoints, let's just call this
"macvlan"
Fixes: #3050
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
The thread monitor will check if the agent and the VMM are alive every
second in a blocking thread. The Cloud hypervisor API server is
single-threaded, if the monitor does a `check()`, while a slow request
is still in progress, the monitor check() method will timeout. The
monitor thread will stop all the shim-v2 execution.
This commit modifies the monitor thread to make it check the status of
the hypervisor after 5 seconds. Additionally, the `check()` method from
cloud-hypervisor will use the method `clh.isClhRunning(timeout)` with a
10 seconds timeout. The monitor function does no timeout, so even if
`hypervisor.check()` takes more 10 seconds, the isClhRunning method
handles errors doing a VmmPing and retry in case of errors until the
timeout is reached.
Reduce the time to the next check to 5 should not affect any functionality,
but it will reduce the overhead polling the hypervisor.
Fixes: #2777
Signed-off-by: Carlos Venegas <jose.carlos.venegas.munoz@intel.com>
There are two types packages under virtcontainers, and the
virtcontainers/pkg/types has a few codes, merging them into
one can make it easy for outstanding and using types package.
Fixes: #3031
Signed-off-by: bin <bin@hyper.sh>
As github.com/containerd/cgroups doesn't support scope
units which are essential in some cases lets create
the cgroups manually and load it trough the cgroups
api
This is currently done only when there's single sandbox
cgroup (sandbox_cgroup_only=true), otherwise we set it
as static cgroup path as it used to be (until a proper
soultion for overhead cgroup under systemd will be
suggested)
Fixes: #2868
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
Update the `agent-ctl` tool to allow API fields to be specified in JSON
format, either directly on the command-line, or via a file URI.
This feature is made possible by enabling `serde` support in the agent
`protocols` crate. Careful use of the `serde` macros allows the
`agent-ctl` tool to accept _partially_ specified API objects in JSON
format; fields that are not specified are set to the default value for
their respective types.
`build.rs` changes based on work by Fupan.
Fixes: #2978.
Contributions-by: Fupan Li <lifupan@gmail.com>
Contributions-by: Bin Liu <bin@hyper.sh>
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This reverts commit 76f16fd1a7 to bring
back cri-containerd crioptions parsing so that kata works with older
containerd versions like v1.3.9 and v1.4.6.
Fixes: #2999
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Protection types like tdxProtection or seProtection were marked nolint,
remove this. As a side effect, ARM needs dummy tests for these.
Fixes: #2801
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Remove the `StartTracing` and `StopTracing` agent APIs that toggle
dynamic tracing. This is not supported in Kata 2.x, as documented in the
[tracing proposals document](https://github.com/kata-containers/kata-containers/pull/2062).
Fixes: #2985.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
If Kata agent cannot resolve the system calls given by seccomp profiles,
the agent ignores the system calls and continues to run without an error.
Fixes: #2957
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Upgrade from v0.20.0 to v1.0.0, first stable release.
Git log
4bfa0034 Release prep v1.0.0-RC3 (2218)
c7ae470a Refactor SDK span creation and implementation (2213)
db317fce Verify and update OTLP trace exporter documentation (2053)
04de34a2 Update the website getting started docs (2203)
a7b9d021 Rename metric instruments to match feature-freeze API specification (2202)
1f527a52 Update trace API config creation functions (2212)
361a2096 Fix RC2 header in changelog (2215)
e209ee75 chore(exporter/zipkin): improves logging on invalid collector. (2191)
c0c5ef65 Fix typos in resource.go. (2201)
abf6afe0 Update otel example guide (2210)
3b05ba02 Bump actions/setup-go from 2.1.3 to 2.1.4 (2206)
bcd7ff7b Bump codecov/codecov-action from 2.0.2 to 2.0.3 (2205)
c912b179 Print JSON objects to stdout without a wrapping array (2196)
add511c1 Make WithoutTimestamps work (2195)
85c27e01 Bump github.com/golangci/golangci-lint from 1.41.1 to 1.42.0 in /internal/tools (2199)
bf6500b3 Bump google.golang.org/grpc from 1.39.1 to 1.40.0 in /exporters/otlp/otlptrace (2184)
9392af96 Bump google.golang.org/grpc in /exporters/otlp/otlptrace/otlptracegrpc (2185)
c95694dc Bump google.golang.org/grpc from 1.39.1 to 1.40.0 in /example/otel-collector (2183)
0528fa66 Bump google.golang.org/grpc from 1.39.1 to 1.40.0 in /exporters/otlp/otlpmetric (2186)
3a26ed21 Deprecate the oteltest package (2188)
c885435f Website: support GH page links to canonical src (2189)
6da20a27 Add cross-module test coverage (2182)
dfc866bd Support capturing stack trace (2163)
41588fea Deprecate the attribute.Any function (2181)
4e8d667f Support a single Resource per MeterProvider in the SDK (2120)
a8bb0bf8 Make the tracetest.SpanRecorder concurrent safe (2178)
87d09df3 Deprecate Array attribute in favor of *Slice types (2162)
df384a9a Move InstrumentKind into the new metric/sdkapi package (2091)
1cb5cdca Unify the OTLP attribute transform (2170)
a882ee37 Clarify the attribute package documentation and order/grouping (2168)
5d25c4d2 Add support for int32 in attribute.Any (2169)
2b0e139e Refactor attributes benchmark tests (2167)
4c7470d9 Bump google.golang.org/grpc from 1.39.0 to 1.39.1 in /exporters/otlp/otlptrace (2176)
990c534a Bump google.golang.org/grpc in /example/otel-collector (2172)
b45c9d31 Bump google.golang.org/grpc from 1.39.0 to 1.39.1 in /exporters/otlp/otlpmetric (2174)
a3d4ff5c Deprecated the bridge/opencensus/utils package (2166)
b1d1d529 Move OC bridge integration tests to own mod (2165)
89a9489c Add OC bridge internal unit tests (2164)
56c743ba Allow global ErrorHandler to be set multiple times (2160)
d18c135f Add OpenCensus bridge internal package (2146)
fcf945a4 Just a little typo fix in code documentation. (2159)
59a82eba Update version.go (2157)
21d4686f Add ErrorHandlerFunc to simplify creating ErrorHandlers (2149)
23cb9396 Remove `internal/semconv-gen` (2155)
39acab32 Fix code sample in otel.GetTraceProvider (2147)
2b1bb29e Update OpenCensus bridge docs with limitations (2145)
fd7c327b Fix Jaeger exporter agent port default value and docs (2131)
b8561785 fix(2138): add guard to constructOTResources to return an empty resource (2139)
11f62640 Add a SpanRecorder to the sdk/trace/tracetest (2132)
fd9de7ec rename assertsocketbuffersize.go to *_test (2136)
a6b4d90c nit doc fix (2135)
79398418 pre-release v1.0.0-RC2 (2133)
2501e0fd Use semconv.SchemaURL in STDOUT exporter example (2134)
ef03dbc9 Bump codecov/codecov-action from 1 to 2.0.2 (2129)
bbe6ca40 Deprecate oteltest.Harness for removal (2123)
7a624ac2 Deprecated the oteltest.TraceStateFromKeyValues function (2122)
ece1879f Removed dropped link's attributes field from API package (2118)
03902d98 Rename sdk/trace/tracetest test.go -> exporter.go (2128)
cb607b0a Unify OTLP exporter retry logic (2095)
abe22437 API: create new linked span from current context (2115)
db81d4aa Update internal/global/trace testing (2111)
7f10ef72 Remove propagation testing types from oteltest (2116)
25d739b0 Remove resource.WithBuiltinDetectors() which has not been maintained (2097)
d57c5a56 Remove several metrics test helpers (2105)
49359495 Simplify trace_context tests (2108)
56d42011 Simplify trace context benchmark test (2109)
63dfe64a Correct status transform in OTLP exporter (2102)
9b1a5f70 Performance improvement: avoid creating multiple same read-only objects (2104)
ab78dbd0 Update release URL (2106)
647af3a0 Pre release experimental metrics v0.22.0 (2101)
0a562337 Fixed OS type value for DragonFly BSD (2092)
62c21ffb Bump golang.org/x/tools from 0.1.4 to 0.1.5 in /internal/tools (2096)
4a3da55a Ensure sample code in website_docs getting started page works (2094)
d3063a3d Update otel.Meter to global.Meter in Getting Started Document.(2087) (2093)
00a1ec5f Add documentation guidelines and improve Jaeger exporter readme (2082)
12f737c7 oteltest: ensure valid SpanContext created for span started WithNewRoot (2073)
484258eb OS description attribute detector (1840)
d8c9a955 Bump google.golang.org/grpc from 1.38.0 to 1.39.0 in /example/otel-collector (2054)
4ffdf034 Add @pellard as an Approver (2047)
1a74b399 Bump google.golang.org/protobuf from 1.26.0 to 1.27.0 in /exporters/otlp/otlpmetric (2040)
57c2e8fb Bump golang.org/x/tools from 0.1.3 to 0.1.4 in /internal/tools (2036)
7cff31a9 Bump google.golang.org/protobuf from 1.26.0 to 1.27.0 in /exporters/otlp/otlptrace (2035)
9e8f523d when using WithNewRoot, don't use the parent context for sampling (2032)
62af6c70 semconv-gen: fix capitalization at word boundaries, add stability/deprecation indicators (2033)
0bceed7e Fix docs on otel-collector example (2034)
6428cd69 Update doc.go (2030)
311a6396 fix documentation for trace.Status (2029)
16f83ce6 export ToZipkinSpanModels for use outside this library (2027)
d5d4c87f Add HTTP metrics exporter for OTLP (2022)
d6e8f60f Bump github.com/golangci/golangci-lint from 1.40.1 to 1.41.1 in /internal/tools (2023)
51dbe3cb Remove deprecated exporters (2020)
257ef7fc Update project status in README (2017)
ced177b7 Pre-release 1.0.0-RC1 (2013)
694c9a41 Interface stability documentation (2012)
39fe8092 Add span.TracerProvider() (2009)
d020e1a2 Add more tests for go.opentelemetry.io/otel/trace package. (2004)
6d4a38f1 replace WithSyncer with WithBatcher in opencensus example (2007)
c30cd1d0 Split stdout exporter into stdouttrace and stdoutmetric (2005)
80ca2b1e otlp: mark unix endpoints to work without transport security (2001)
65140985 Update codecov ignore (2006)
3be9813d Deprecate the exporters in the "trace" and "metric" sub-directories (1993)
377f7ce4 remove WithTrace* options from otlptrace exporters (1997)
b33edaa5 OTLP metrics gRPC exporter (1991)
64b640cc Remove old OTLP exporter (1990)
7728a521 Remove dependency on metrics packages (1988)
135ac4b6 Moved internal/tools duplicated findRepoRoot function to common package (1978)
cdf67ddf Update semantic conventions to v1.4.0, move to versioned package (1987)
4883cb11 Refactor exporter creation functions (1985)
87cc1e1f Test BatchSpanProcessor export timeout directly (1982)
7ffe2845 Added inputPath validation to semconv-gen (1986)
a113856a Add caveat about installing opencensus bridge (1983)
741cb9a3 Fix generator.go call typo in RELEASING.md (1977)
7a0cee7b Replaces golint by revive and fix newly reported linter issues (1946)
46d9687a Add Schema URL support to Resource (1938)
0827aa62 Use mock server as jaeger agent listener. (1930)
20886012 Bugfix jaeger exporter test panic (1973)
4bf6150f Add baggage implementation based on the W3C and OpenTelemetry specification (1967)
bbe2b8a3 Bump github.com/itchyny/gojq from 0.12.3 to 0.12.4 in /internal/tools (1971)
4949bf05 Bump github.com/cenkalti/backoff/v4 from 4.1.0 to 4.1.1 in /exporters/otlp/otlptrace (1972)
015b4c17 Bump github.com/cenkalti/backoff/v4 from 4.1.0 to 4.1.1 in /exporters/otlp (1970)
13eb12ac Bump github.com/prometheus/client_golang from 1.10.0 to 1.11.0 in /exporters/metric/prometheus (1974)
2371bb0a add otlp trace http exporter (1963)
a75ade4e sdk/resource: honor OTEL_SERVICE_NAME in fromEnv resource detector (1969)
aed45802 Bump go.opentelemetry.io/proto/otlp from 0.8.0 to 0.9.0 in /exporters/otlp/otlptrace (1959)
c4ebae6a Bump go.opentelemetry.io/proto/otlp (1960)
b1d2be3b Bump google.golang.org/grpc from 1.37.1 to 1.38.0 in /exporters/otlp/otlptrace (1958)
f6daea5e Generate semantic conventions according to specification latest tagged version (1933)
435a63b3 Bump github.com/google/go-cmp from 0.5.5 to 0.5.6 (1954)
6c46af66 Bump github.com/google/go-cmp from 0.5.5 to 0.5.6 in /exporters/trace/jaeger (1953)
4d294853 Bump actions/cache from 2.1.5 to 2.1.6 (1952)
dfe2b6f1 OTLP trace gRPC exporter (1922)
5a8f7ff7 Bump go.opentelemetry.io/proto/otlp from 0.8.0 to 0.9.0 in /exporters/otlp (1943)
bd935866 Add schema URL support to Tracer (1889)
c1f460e0 Update API configs. (1921)
270cc603 Small fixes on some Span method's documentation headers (1950)
8603b902 Fix typo in doc (1949)
acbb1882 Bump google.golang.org/grpc from 1.37.1 to 1.38.0 in /exporters/otlp (1942)
b1621501 Add codecov badge (1940)
ea1434c3 Fix some golint issues (1947)
0eeb8f87 Refactor Tracestate (1931)
d3b12808 Add Passthrough example (1912)
f06cace6 Add @MadVikingGod as a project Approver (1923)
ab5facb3 Bump github.com/golangci/golangci-lint in /internal/tools (1925)
d23cc61b Refactor configs (1882)
6324adaa Add tracer option argument to global Tracer function (1902)
035fc650 Do not include authentication information in the http.url attribute (1919)
d8ac212c Fix sporadic test failure in otlp exporter http driver (1906)
a3df00f4 Create .gitattributes (1920)
fb88e926 Bump google.golang.org/grpc from 1.37.0 to 1.37.1 in /exporters/otlp (1914)
1982dc46 Bump google.golang.org/grpc in /example/prom-collector (1915)
1759c630 Bump github.com/golangci/golangci-lint in /internal/tools (1916)
7342aa47 Bump google.golang.org/grpc in /example/otel-collector (1913)
21c16418 Add support for scheme in OTEL_EXPORTER_OTLP_ENDPOINT (1886)
5cb62636 Semantic Convention generation tooling (1891)
6219221f Move the unit package to the metric module (1903)
63e0ecfc Implement global default non-recording span (1901)
b6d5442f Remove the Tracer method from the Span API (1900)
ae85fab3 Document functional options (1899)
cabf0c07 Fix default Jaeger collector endpoint (1898)
1e3fa3a3 Bump go.opentelemetry.io/proto/otlp from 0.7.0 to 0.8.0 in /exporters/otlp (1872)
696af787 Bump github.com/benbjohnson/clock from 1.0.3 to 1.1.0 in /sdk/metric (1532)
97eea6c3 Fix some golint issues (1894)
79d9852e fix container port mismatch issue (1895)
d20e7228 CI builds validate against last two versions of Go, dropping 1.14 and adding 1.16 (1865)
cbcd4b1a Redefine ExportSpans of SpanExporter with ReadOnlySpan (1873)
c99d5e99 Split large jaeger span batch to admire the udp packet size limit (1853)
42a84509 Unembed SpanContext (1877)
b7d02db1 Add Status type to SDK (1874)
f90d0d93 Update README (1876)
a1349944 Update resource.go (1871)
f40cad5e Add markdown link check configuration and action (1869)
9bc28f6b Fix existing markdown lint issues (1866)
08f4c270 Add documentation for tracer.Start() (1864)
2bd4840c remove Set.Encoded(Encoder) enconding cache (1855)
7674eebf Removed different types of Detectors for Resources. (1810)
f92a6d83 Implement retry policy for the OTLP/gRPC exporter (1832)
ec75390f Fix BSP context done tests (1863)
8e55f10a Move the Event type from the API to the SDK (1846)
e399d355 drop failed to exporter batches and return error when forcing flush a span processor (1860)
f6a9279a Honor context deadline or cancellation in SimpleSpanProcessor.Shutdown (1856)
aeef8e00 Add markdown lint GitHub action (1849)
d4c8ffad Replace spaces to tabs in Go code snippets (1854)
cb097250 fixed typo (1857)
392a44fa Refine configuration design docs (1841)
62cd933d Handle Resource env error when non-nil (1851)
24a91628 Document the SSP is not for production use (1844)
ec26ac23 Update RELEASING.md (1843)
8eb0bb99 Fix golint issue caused by typo (1847)
ca130e54 Markdownlint (1842)
1144a83d Small typo fixes to existing CHANGELOG entries (1839)
e6086958 Update website_docs to v0.20.0 (1838)
0f4e454c Change NewSplitDriver paramater and initialization (1798)
92551d39 Prerelease v1.0.0 (2250)
61839133 zipkin: remove no-op WithSDKOptions (2248)
568e7556 Set Schema URL when exporting traces to OTLP (2242)
ec26b556 Fix RC tags in docs (2239)
767ce26c Bump github.com/itchyny/gojq from 0.12.4 to 0.12.5 in /internal/tools (2216)
fe7058da adding NewNoopMeterProvider to follow trace api (2237)
c338a5ef Bump github.com/golangci/golangci-lint from 1.42.0 to 1.42.1 in /internal/tools (2236)
ef126f5c Remove deprecated Array from attribute package (2235)
360d1302 Add tests for nil *Resource (2227)
9e7812d1 Remove the deprecated oteltest package (2234)
486afd34 Remove the deprecated bridge/opencensus/utils pkg (2233)
eaacfaa8 Fix slice-valued attributes when used as map keys (2223)
df2bdbba Fix the import comments of otelpconfig (2224)
7aae2a02 otlptrace: Document supported environment variables (2222)
Fixes#2591
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Update OpenTelemetry from v0.15.0 to v0.20.0.
Git log
02d8bdd5 Release v0.20.0 (1837)
aa66fe75 OS and Process resource detectors (1788)
7374d679 Fix Links documents (1835)
856f5b84 Add feature request issue template (1831)
0fdc3d78 Remove bundler from Jaeger exporter (1830)
738ef11e Fix flaky global ErrorHandler delegation test (1829)
e43d9c00 Update Default Value for Jaeger Exporter Endpoint (1824)
0032bd64 Fix default merging of resource attributes from environment variable (1785)
96c5e4ba Add SpanProcessor example for Span annotation on start (1733)
543c8144 Remove the WithSDKOptions from the Jaeger exporter (1825)
66389ad6 Update function docs in sdk.go (1826)
70bc9eb3 Adds support for timeout on the otlp/gRPC exporter (1821)
081cc61d Update Jaeger exporter convenience functions (1822)
1b9f16d3 Remove the WithDisabled option from Jaeger exporter (1806)
6867faa0 Bump actions/cache from v2.1.4 to v2.1.5 (1818)
a2bf04dc Build context pipeline in Jaeger upload process (1809)
2de86f23 Remove locking from Jaeger exporter shutdown/export (1807)
4f9fec29 Add ExportSpans benchmark to Jaeger exporter (1805)
d9566abe Fix OTLP testing flake: signal connection from mock collector (1816)
a2cecb6e add support for env var configuration to otlp/gRPC (1811)
d616df61 Fix flaky OTLP exporter reconnect test (1814)
b09df84a Changes stdout to expose the `*sdktrace.TracerProvider` (1800)
04890608 Remove options field from Jaeger exporter (1808)
6db20e00 Remove the abandoned Process struct in Jaeger exporter (1804)
086abf34 docs: use test example to document prometheus.InstallNewPipeline (1796)
d0cea04b Bump google.golang.org/api from 0.43.0 to 0.44.0 in /exporters/trace/jaeger (1792)
99c477fe Fixed typo for default service name in Jaeger Exporter (1797)
95fd8f50 Bump google.golang.org/grpc from 1.36.1 to 1.37.0 in /exporters/otlp (1791)
9b251644 Zipkin Exporter: Use default resouce's serviceName as default serivce name (1777) (1786)
4d141e47 Add k8s.node.name and k8s.node.uid to semconv (1789)
5c99a34c Fix golint issue caused by incorrect comment (1795)
c5d006c0 Update Jaeger environment variables (1752)
58432808 add NewExportPipeline and InstallNewPipeline for otlp (1373)
7d8e6bd7 Zipkin Exporter: Adjust span transformation to comply with the spec (1688)
2817c091 Merge sdk/export/trace into sdk/trace (1778)
c61e654c Refactor prometheus exporter tests to match file headers as well (1470)
23422c56 Remove process config for Jaeger exporter (1776)
0d49b592 Add test to check bsp ignores `OnEnd` and `ForceFlush` post Shutdown` (1772)
e9aaa04b Record links/events attribute drops independently (1771)
5bbfc22c Make ExportSpans for Jaeger Exporter honor deadline (1773)
0786fe32 Add Bug report issue templates (1775)
3c7facee Add `ExportTimeout` option to batch span processor (1755)
c6b92d5b Make TraceFlags spec-compliant (1770)
ee687ca5 Bump github.com/itchyny/gojq from 0.12.2 to 0.12.3 in /internal/tools (1774)
52a24774 add support for configuring tls certs via env var to otlp/HTTP (1769)
35cfbc7e Update precedence of event name in Jaeger exporter (1768)
33699d24 Adds semantic conventions for exceptions (1492)
928e3c38 Modify ForceFlush to abort after timeout/cancellation (1757)
3947cab4 Fix testCollectorEndpoint typo and add tag assertions in jaeger_test (1753)
ecc635dc add website docs (1747)
07a8d195 Fix Jaeger span status reporting and unify tag keys (1761)
4fa35c90 add partial support for env var config to otlp/HTTP (1758)
bf180d0f improve OTLP/gRPC connection errors (1737)
d575865b Fix span IsRecording when not sampling (1750)
20c93b01 Update SamplingParameters (1749)
97501a3f Update SpanSnapshot to use parent SpanContext (1748)
604b05cb Store current Span instead of local and remote SpanContext in context.Context (1731)
c61f4b6d Set @lizthegrey to emeritus status (1745)
b1342fec Bump github.com/golangci/golangci-lint in /internal/tools (1743)
54e1bd19 Bump google.golang.org/api from 0.41.0 to 0.43.0 in /exporters/trace/jaeger (1741)
4d25b6a2 Bump github.com/prometheus/client_golang from 1.9.0 to 1.10.0 in /exporters/metric/prometheus (1740)
0a47b66f Bump google.golang.org/grpc from 1.36.0 to 1.36.1 in /exporters/otlp (1739)
26f006b8 Reinstate @paivagustavo as an Approver (1734)
382c7ced Remove hasRemoteParent field from SDK span (1728)
862a5a68 Remove setting error status while recording error with Span from oteltest package (1729)
6defcfdf Remove links on NewRoot spans (1726)
a9b2f851 upgrade thrift to v0.14.1 in jaeger exporter (1712)
5a6a854d Bump google.golang.org/protobuf from 1.25.0 to 1.26.0 in /exporters/otlp (1724)
23486213 Migrate to using go.opentelemetry.io/proto/otlp (1713)
5d559b40 Remove makeSamplingDecision func (1711)
e24702da Update the TraceContext.Extract docs (1720)
9d4eb1f6 Update dates in CHANGELOG.md for 2021 releases (1723)
2b4fa968 Release v0.19.0 (1710)
4beb7041 sdk/trace: removing ApplyConfig and Config (1693)
1d42be16 Rename WithDefaultSampler TracerProvider option to WithSampler and update docs (1702)
860d5d86 Add flag to determine whether SpanContext is remote (1701)
0fe65e6b Comply with OpenTelemetry attributes specification (1703)
88884351 Bump google.golang.org/api from 0.40.0 to 0.41.0 in /exporters/trace/jaeger (1700)
345f264a breaking(zipkin): removes servicName from zipkin exporter. (1697)
62cbf0f2 Populate Jaeger's Span.Process from Resource (1673)
28eaaa9a Add a test to prove the Tracer is safe for concurrent calls (1665)
8b1be11a Rename resource pkg label vars and methods (1692)
a1539d44 OpenCensus metric exporter bridge (1444)
77aa218d Fix issue #1490, apply same logic as in the SDK (1687)
9d3416cc Fix synchronization issues in global trace delegate implementation (1686)
58f69f09 Span status from HTTP code: Do not set status message if it can be inferred (1681)
9c305bde Flush metric events prior to shutdown in OTLP example (1678)
66b1135a Fix CHANGELOG (1680)
90bd4ab5 Update employer information for maintainers (1683)
36841913 Remove WithRecord() option from trace.SpanOption when starting a span (1660)
65c7de20 Remove trace prefix from NoOp src files. (1679)
e88a091a Make SpanContext Immutable (1573)
d75e2680 Avoid overriding configuration of tracer provider (1633)
2b4d5ac3 Bump github.com/golangci/golangci-lint in /internal/tools (1671)
150b868d Bump github.com/google/go-cmp from 0.5.4 to 0.5.5 (1667)
76aa924e Fix the examples target info messaging (1676)
a3aa9fda Bump github.com/itchyny/gojq from 0.12.1 to 0.12.2 in /internal/tools (1672)
a5edd79e Removed setting error status while recording err as span event (1663)
e9814758 chore(zipkin): improves zipkin example to not to depend on timeouts. (1566)
3dc91f2d Add ForceFlush method to TracerProvider (1608)
bd0bba43 exporter: swap pusher for exporter (1656)
56904859 Update the SimpleSpanProcessor (1612)
a7f7abac SpanStatus description set only when status code is set to Error (1662)
05252f40 Jaeger Exporter: Fix minor mapping discrepancies (1626)
238e7c61 Add non-empty string check for attribute keys (1659)
e9b9aca8 Add tests for propagation of Sampler Tracestate changes (1655)
875a2583 Add docs on when reviews should be cleared (1556)
7153ef2d Add HTTP/JSON to the otlp exporter (1586)
62e2a0f7 Unexport the simple and batch SpanProcessors (1638)
992837f1 Add TracerProvider tests to oteltest harness (1607)
bb4c297e Pre release v0.18.0 (1635)
712c3dcc Fix makefile ci target and coverage test packages (1634)
841d2a58 Rename local var new to not collide with builtin (1610)
13938ab5 Update SpanProcessor docs (1611)
e25503a0 Add compatibility tests to CI (1567)
1519d959 Use reasonable interval in sdktrace.WithBatchTimeout (1621)
7d4496e0 Pass metric labels when transforming to gaugeArray (1570)
6d4a5e0d Bump google.golang.org/grpc from 1.35.0 to 1.36.0 in /exporters/otlp (1619)
a93393a0 Bump google.golang.org/grpc in /example/prom-collector (1620)
e499ca86 Fix validation for tracestate with vendor and add tests (1581)
43886e52 Make timestamps sequential in lastvalue agg check (1579)
37688ef6 revent end-users from implementing some interfaces (1575)
85e696d2 Updating documentation with an working example for creating NewExporter (1513)
562eb28b Unify the Added sections of the unreleased changes (1580)
c4cf1aff Fix Windows build of Jaeger tests (1577)
4a163bea Fix stdout TestStdoutTimestamp failure with sleep (1572)
bd4701eb Stagger timestamps in exact aggregator tests (1569)
b94cd4b2 add code attributes to semconv package (1558)
78c06cef Update docs from gitter to slack for communication (1554)
1307c911 Remove vendor exclude from license-check (1552)
5d2636e5 Bump github.com/golangci/golangci-lint in /internal/tools (1565)
d7aff473 Vendor Thrift dependency (1551)
298c5a14 Update span limits to conform with OpenTelemetry specification (1535)
ecf65d79 Rename otel/label -> otel/attribute (1541)
1b5b6621 Remove resampling on span.SetName (1545)
8da52996 fix: grpc reconnection (1521)
3bce9c97 Add Keys() method to propagation.TextMapCarrier (1544)
0b1a1c72 Make oteltest.SpanRecorder into a concrete type (1542)
7d0e3e52 SDK span no modification after ended (1543)
7de3b58c Remove extra labels types (1314)
73194e44 Bump google.golang.org/api from 0.39.0 to 0.40.0 in /exporters/trace/jaeger (1536)
8fae0a64 Create resource.Default() with required attributes/default values (1507)
76f93422 Release v0.17.0 (1534)
9b242bc4 Organize API into Go modules based on stability and dependencies (1528)
e50a1c8c Bump actions/cache from v2 to v2.1.4 (1518)
a6aa7f00 Bump google.golang.org/api from 0.38.0 to 0.39.0 in /exporters/trace/jaeger (1517)
38efc875 Code Improvement - Error strings should not be capitalized (1488)
6b340501 Update default branch name (1505)
b39fd052 nit: Fix comment to be up-to-date (1510)
186c2953 Fix golint error of package comment form (1487)
9308d662 Bump google.golang.org/api from 0.37.0 to 0.38.0 in /exporters/trace/jaeger (1506)
1952d7b6 Reverse order of attribute precedence when merging two Resources (1501)
ad7b4715 Remove build flags for runtime/trace support (1498)
4bf4b690 Remove inaccurate and unnecessary import comment (1481)
7e19eb6a Bump google.golang.org/api from 0.36.0 to 0.37.0 in /exporters/trace/jaeger (1504)
c6a4406a Bump github.com/golangci/golangci-lint in /internal/tools (1503)
9524ac09 Update workflows to include main branch as trigger (1497)
c066f15e Bump github.com/gogo/protobuf from 1.3.1 to 1.3.2 in /internal/tools (1478)
894e0240 Bump github.com/golangci/golangci-lint in /internal/tools (1477)
71ffba39 Bump google.golang.org/grpc from 1.34.0 to 1.35.0 in /exporters/otlp (1471)
515809a8 Bump github.com/itchyny/gojq from 0.12.0 to 0.12.1 in /internal/tools (1472)
3e96ad1e gitignore: remove unused example path (1474)
c5622777 Histogram aggregator functional options (1434)
0df8cd62 Rename Makefile.proto to avoid interpretation as proto file (1468)
979ff51f Bump github.com/stretchr/testify from 1.6.1 to 1.7.0 (1453)
1df8b3b8 Bump github.com/gogo/protobuf from 1.3.1 to 1.3.2 in /exporters/otlp (1456)
4c30a90a Bump github.com/stretchr/testify from 1.6.1 to 1.7.0 in /sdk (1455)
5a9f8f6e Bump github.com/stretchr/testify from 1.6.1 to 1.7.0 in /exporters/stdout (1454)
7786f34c Bump github.com/stretchr/testify from 1.6.1 to 1.7.0 in /exporters/trace/zipkin (1457)
4352a7a6 Bump github.com/stretchr/testify from 1.6.1 to 1.7.0 in /exporters/otlp (1460)
6990b3b3 Bump github.com/stretchr/testify from 1.6.1 to 1.7.0 in /exporters/metric/prometheus (1461)
7af40d22 Bump github.com/stretchr/testify from 1.6.1 to 1.7.0 in /exporters/trace/jaeger (1463)
f16f1892 Bump google.golang.org/grpc in /example/otel-collector (1465)
fe363be3 Move Span Event to API (1452)
43922240 Bump google.golang.org/grpc in /example/prom-collector (1466)
0aadfb27 Prepare release v0.16.0 (1464)
207587b6 Metric histogram aggregator: Swap in SynchronizedMove to avoid allocations (1435)
c29c6fd1 Shutdown underlying span exporter while shutting down BatchSpanProcessor (1443)
dfece3d2 Combine the Push and Pull metric controllers (1378)
74deeddd Handle tracestate in TraceContext propagator (1447)
49f699d6 Remove Quantile aggregation, DDSketch aggregator; add Exact timestamps (1412)
9c949411 Rename internal/testing to internal/internaltest (1449)
8d809814 Move gRPC driver to a subpackage and add an HTTP driver (1420)
9332af1b Bump github.com/golangci/golangci-lint in /internal/tools (1445)
5ed96e92 Update exporters/otlp Readme.md (1441)
bc9cb5e3 Switch CircleCI badge to GitHub Actions (1440)
716ad082 Remove CircleCI config (1439)
0682db1e Adding Security Workflows to GitHub Actions (2/2): gosec workflow (1429)
11f732b8 Adding Security Workflows to GitHub Actions (1/2): codeql workflow (1428)
40f1c003 Add Tracestate into the SamplingResult struct (1432)
db06c8d1 Flush metric events before shutdown in collector example (1438)
f6f458e1 Fix golint issue caused by typo in trace.go (1436)
fe9d1f7e Use uint64 Count consistently in metric aggregation (1430)
3a337d0b Bump github.com/golangci/golangci-lint in /internal/tools (1433)
1e4c8321 cleanup: drop the removed examples in gitignore (1427)
5c9221cf Unify endpoint API that related to OTel exporter (1401)
045c3ffe Build scripts: Replace mapfile with read loop for old bash versions (1425)
2def8c3d Add Versioning Documentation (1388)
6bcd1085 Bump github.com/itchyny/gojq from 0.11.2 to 0.12.0 in /internal/tools (1424)
38e76efe Add a split protocol driver for otlp exporter (1418)
439cd313 Add TraceState to SpanContext in API (1340)
35215264 Split connection management away from exporter (1369)
add9d933 Bump github.com/prometheus/client_golang from 1.8.0 to 1.9.0 in /exporters/metric/prometheus (1414)
93d426a1 Add @dashpole as a project Approver (1410)
6fe20ef3 Fix small typo (1409)
b22d0d70 Mention the getting started guide (1406)
3fb80fb2 Fix duplicate checkout action in GitHub workflow (1407)
2051927b Correct CI workflow syntax (1403)
f11a86f7 Fix typo in comment (1402)
bdf87a78 Migrate CircleCI ci.yml workflow to GitHub Actions (1382)
4e59dd1f Bump google.golang.org/grpc from 1.32.0 to 1.34.0 in /example/otel-collector (1400)
83513f70 Bump google.golang.org/api from 0.32.0 to 0.36.0 in /exporters/trace/jaeger (1398)
a354fc41 Bump github.com/prometheus/client_golang from 1.7.1 to 1.8.0 in /exporters/metric/prometheus (1397)
3528e42c Bump google.golang.org/grpc from 1.32.0 to 1.34.0 in /exporters/otlp (1396)
af114baf Call otel.Handle with non-nil errors (1384)
c3c4273e Add RO/RW span interfaces (1360)
Fixes#2591
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Ensure the tests in the local `logging` crate are run for all consumers
of it.
Additionally, add a new test which checks that output is generated by a
range of different log level `slog` macros. This is designed to ensure
debug level output is always available for the consumers of the
`logging` crate.
Fixes: #2969.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
In later versions of OpenTelemetry label.Any() is deprecated. Create
addTag() to handle type assertions of values. Change AddTag() to
variadic function that accepts multiple keys and values.
Fixes#2547
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Raise the `slog` maximum log level feature for release code from `info`
to `debug` by changing the `slog` maximum level features in the shared
`logging` crate. This allows the consumers of the `logging` crate (the
agent, the `trace-forwarder` and the `agent-ctl` tool) to produce debug
output when their debug options are enabled. Currently, those options
will essentially be a NOP (unless using a debug version of the code).
Testing showed that setting the `slog` maximum level features in the
rust manifest files for the consumers of the `logging` crate has no
impact: those values are ignored, so they have been removed and replaced
with a comment stating the levels are set in the `logging` crate.
Fixes: #2966.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
There are some issues with Makefile for runtime:
- default target can't be used as a dependent of other targets.
- empty target `check`
And also add two targets for locally development/tests.
- lint: run golangci-lint
- pre-commit: run lint and test
Fixes: #2942
Signed-off-by: bin <bin@hyper.sh>
Use `dup3` system call instead of `dup2` in unit tests of seccomp
because `dup2` is obsolete on aarch64.
Fixes: #2939
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
We only added span.End() in the main process of the shim2 Shutdown method.
The "Shutdown" span would keep alive, when the containers number is not 0.
This PR make sure the "Shutdown" trace span have a correct end.
Fixes: #2930
Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
Add -failfast option to let test exit on error, but -failfast option
can't cross package, so there is a for loop used to test on all packages
in src/runtime, and the parallel number is set to 1, this may lead test
to be slow.
Fixes: #1997
Signed-off-by: bin <bin@hyper.sh>
Virtcontainers API document functions weren't sync with the codes Sandbox and VCImpl.
And we have two functions named `CreateSandbox` functions, diff by one parameter,
very confused. So this pr sync the codes to api documents.
Fixes: #2928
Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
Cloud hypervisor is only supporting virtio-blk, this PR removes comments
that make a wrong reference of other features that are not supported
by clh.
Fixes#2924
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Update the trace forwarder README to remove the quotes around the socket
path, which makes manipulating that path easier.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Current handling of read-only mounts is a little tricky.
However, a clearer solution can be used here:
1. make a private ro bind mount at privateDest to the mount source
2. make a bind mount at mountDest to the mount created in step 1
3. umount the private bind mount created in step 1
One important aspect is that the mount in step 2 is duplicated from
the one we created in step 1. So the MS_RDONLY flag is properly
preserved in all mounts created in the propagtion.
Fixes: #2205
Depends-on: github.com/kata-containers/tests#4106
Signed-off-by: Yujia Qiao <rapiz3142@gmail.com>
In order to pass CI test of aarch64, it is necessary to run
`ci/install_libseccomp.sh` before ruuning unit tests in
`jenkins_job_build.sh`.
However, `ci/install_libseccomp.sh` is not available
until PR #1788 including this commit is merged in the mainline.
Therefore, we disable seccomp feature on aarch64 temporarily.
After #1788 lands and CI is fixed, this commit will be reverted.
Fixes: #1476
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
The kata-agent supports seccomp feature based on the OCI runtime specification.
This seccomp capability in the kata-agent is enabled by default.
However, it is not enforced by default: users need to enable that by setting
`disable_guest_seccomp` to `false` in the main configuration file.
Fixes: #1476
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
All endpoint names share the `Request` suffix.
Also, the current list is based on functions, not requests.
Fixes#2916
Reported-by: Jakob Naucke <jakob.naucke@ibm.com>
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
This changed valid() in hypervisor to check the case where both
initrd and image path are set; in this case it returns an error.
Fixes#1868
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Fix `-l <log-level>` for the trace forwarder which didn't work
previously as it lacked the magic Cargo configuration.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Hybrid VSOCK requires `root` privileges to access the sandbox-specific
host-side AF_UNIX socket created by the hypervisor (CLH or FC). However,
once the socket has been bound, privileges can be dropped, allowing the
forwarder to run as user `nobody`.
Fixes: #2905.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Rather than generating a potentially misleading error message if the
socket bind fails, perform an explicit check for `root` for Hybrid
VSOCK.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Updated the trace forwarder README to ensure the real socket path is
created, not the template socket path returned by `kata-runtime env`.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Show available guest protections in the
`kata-runtime env` output. Also bump the formatVersion.
Fixes: #1982
Signed-off-by: Yujia Qiao <rapiz3142@gmail.com>
Add functions to return guestProtection as a string slice, which
can be then used in `kata-runtime env` output.
Signed-off-by: Yujia Qiao <rapiz3142@gmail.com>
uevents with action=remove was ignored causing the agent to reuse stale
data in the device map. This patch adds handling of such uevents.
Fixes#2405
Signed-off-by: Haitao Li <lihaitao@gmail.com>
On a conventional (e.g. runc) container, passing in a VFIO group device,
/dev/vfio/NN, will result in the same VFIO group device being available
within the container.
With Kata, however, the VFIO device will be bound to the guest kernel's
driver (if it has one), possibly appearing as some other device (or a
network interface) within the guest.
This add a new `vfio_mode` option to alter this. If set to "vfio" it will
instruct the agent to remap VFIO devices to the VFIO driver within the
guest as well, meaning they will appear as VFIO devices within the
container.
Unlike a runc container, the VFIO devices will have different names to the
host, since the names correspond to the IOMMU groups of the guest and those
can't be remapped with namespaces.
For now we keep 'guest-kernel' as the value in the default configuration
files, to maintain current Kata behaviour. In future we should change this
to 'vfio' as the default. That will make Kata's default behaviour more
closely resemble OCI specified behaviour.
fixes#693
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently constrainGRPCSpec always removes VFIO devices from the OCI
container spec which will be used for the inner container. For
upcoming support for VFIO devices in DPDK usecases we'll need to not
do that.
As a preliminary to that, add an extra parameter to the function to
control whether or not it will remove the VFIO devices from the spec.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
"constraint" is a noun, "constrain" is the associated verb, which makes
more sense in this context.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
In order to support DPDK workloads, we need to change the way VFIO devices
will be handled in Kata containers. However, the current method, although
it is not remotely OCI compliant has real uses. Therefore, introduce a new
runtime configuration field "vfio_mode" to control how VFIO devices will be
presented to the container.
We also add a new sandbox annotation -
io.katacontainers.config.runtime.vfio_mode - to override this on a
per-sandbox basis.
For now, the only allowed value is "guest-kernel" which refers to the
current behaviour where VFIO devices added to the container will be bound
to whatever driver in the VM kernel claims them.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Add and adjust the vfio devices in the inner container spec so that
rustjail will create device nodes for them.
In order to do that, we also need to make sure the VFIO device node is
ready within the guest VM first. That may take (slightly) longer than
just the underlying PCI device(s) being ready, because vfio-pci needs
to initialize. So, add a helper function that will wait for a
specific VFIO device node to be ready, using the existing uevent
listening mechanism. It also returns the device node name for the
device (though in practice it will always /dev/vfio/NN where NN is the
group number).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Many device nodes go directly under /dev, however some are conventionally
placed in subdirectories under /dev. For example /dev/vfio/vfio or
/dev/pts/ptmx.
Currently, attempting to pass such a device into a Kata container will fail
because mknod() will get an ENOENT because the parent directory is missing
(or an equivalent error for bind_dev()).
Correct that by making subdirectories as necessary in create_devices().
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
For each user supplied device, create_devices() checks that the given path
actually is in /dev, by checking that its path starts with /dev and does
not contain "..".
However, this has subtle errors because it's interpreting the path as a raw
string without considering separators. It will accept the path /devfoo
which it should not, while it will not accept the valid (though weird)
paths /dev/... and /dev/a..b.
Correct this by using std::path::Path methods designed for the purpose.
Having done this, it's trivial to also generate the relative path that
mknod_dev() or bind_dev() will need, so do that at the same time.
We also move this logic into a helper function so that we can add some unit
tests for it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Both these functions take the absolute path from LinuxDevice and drop the
leading '/' to make a relative path. They do that with a simple
&dev.path[1..]. That can be technically incorrect in some edge cases such
as a path with redundant /s like "//dev//sda".
To handle cases like that, have the explicit relative path passed into
these functions. For now we calculate it in the same buggy way, but we'll
fix that shortly.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
create_devices() within the rustjail module is responsible for creating
device nodes within the (inner) containers. Errors that occur here will
be propagated up, but are likely to be low level failures of mknod() - e.g.
ENOENT or EACCESS - which won't be very useful without context when
reported all the way up to the runtime without the context of what we were
trying to do.
Add some anyhow context information giving the details of the device we
were trying to create when it failed.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently, update_spec_device() assumes that the proper device path in the
(inner) container is the same as the device path specified in the outer OCI
spec on the host.
Usually that's correct. However for VFIO group devices we actually need
the container to see the VM's device path, since it's normal to correlate
that with IOMMU group information from sysfs which will be different in the
guest and which we can't namespace away.
So, add an extra "final_path" parameter to update_spec_device() to allow
callers to chose the device path that should be used for the inner
container. All current callers pass the same thing as container_path, but
that will change in future.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
update_spec_device_list() is used to update the container configuration to
change device major/minor numbers configured by the Kata client based on
host details to values suitable for the sandbox VM, which may differ. It
takes a 'device' object, but the only things it actually uses from there
are container_path and vm_path.
Refactor this as update_spec_device(), taking the host and guest paths to
the device as explicit parameters. This makes the function more
self-contained and will enable some future extensions.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Each VFIO device passed into the guest could represent a whole IOMMU group
of devices on the host. Since these devices aren't DMA isolated from each
other, they must appear as the same IOMMU group in the guest as well.
The VMM should enforce that for us, but double check it, since things can't
work otherwise. This also means we determine the guest IOMMU group for the
VFIO device, which we'll be needing later.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
For upcoming VFIO extensions we'll need to work with the IOMMU groups of
VFIO devices. This helps us towards that by adding pci_iommu_group() to
retrieve the IOMMU group (if any) of a given PCI device.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
VFIO devices can be added to a Kata container and they will be passed
through to the sandbox guest. However, inside the guest those devices
will bind to a native guest driver, so they will no longer appear as VFIO
devices within the guest. This behaviour differs from runc or other
conventional container runtimes.
This code allows the agent to match the behaviour of other runtimes,
if instructed to by kata-runtime. VFIO devices it's informed about
with the "vfio" type instead of the existing "vfio-gk" type will be
rebound to the vfio-pci driver within the guest.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
For better VFIO support, we're going to need to take control of which guest
driver controls specific guest devices. To assist with that, add the
pci_driver_override() function to force a specific guest device to be
bound to a specific guest driver.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Use `"c".to_string` in the device type of `dev/full`
in order to consistent with the coding style of other devices
Fixes: #2890
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
create_tmpfs won't pass as the race condition in watcher umount. quote
James's words here:
1. Rust runs all tests in parallel.
2. Mounts are a process-wide, not a per-thread resource.
The only test that calls watcher.mount() is create_tmpfs().
However, other tests create BindWatcher objects.
3. BindWatcher's drop() implementation calls self.cleanup(),
which calls unmount for the mountpoint create_tmpfs() asserts.
4. The other tests are calling unmount whenever a BindWatcher goes
out of scope.
To avoid that issue, let the tests using BindWatcher in watcher and
sandbox.rs run sequentially.
Fixes: #2809
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
DefaultMaxVCPUs may be larger than the defaultMaxQemuVCPUs that should
be checked and avoided.
Fixes: #2809
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
The physical current vcpu number should not be used directly as the
largest vcpu number is limited to defaultMaxQemuVCPUs.
Here, a new helper is introduced in pkg/katautils/config.go to get
current vcpu number.
Fixes: #2809
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
The current kernel version parse lib can't process suffix '+', as the
modified kernel version will add '+' as suffix, thus panic will occur.
For example, if the current kernel version is "5.14.0-rc4+", test
TestHostNetworkingRequested will panic:
--- FAIL: TestHostNetworkingRequested (0.00s)
panic: &{DistroName:ubuntu DistroVersion:18.04
KernelVersion:5.11.0-rc3+ Issue: Passed:[] Failed:[] Debug:true
ActualEUID:0}: failed to check test constraints: error: Build meta data
is empty
Here, remove the suffix '+' in kernel version fix helper.
Fixes: #2809
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Export the top level hypervisor type
s/hypervisor/Hypervisor
Fixes: #2880
Signed-off-by: Manohar Castelino <mcastelino@apple.com>
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Last of a series of commits to export the top level
hypervisor generic methods.
s/createSandbox/CreateVM
Fixes#2880
Signed-off-by: Manohar Castelino <mcastelino@apple.com>
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Export commonly used hypervisor fields and utility functions.
These need to be exposed to allow the hypervisor to be consumed
externally.
Note: This does not change the hypervisor interface definition.
Those changes will be separate commits.
Signed-off-by: Manohar Castelino <mcastelino@apple.com>
This PR includes these optimize changes:
- Remove the dependency on the container engine.
The old code uses runc to generate config.json and
Docker to export rootfs, that will be heavy and need
additional dependency.
Using a fixed config for busybox image can avoid
the heavy processing above.
- Moved duplicate code to pkg/katatestutils package
Fixes: #2752
Signed-off-by: bin <bin@hyper.sh>
cri-containerd project has been merged into containerd repo, and
we should not reference it any more in code and docs.
This commit will use containerd package instead of cri-containerd
package.
Fixes: #2791
Signed-off-by: bin <bin@hyper.sh>
When running the TestIoCopy test, on some occasions, the test
runs too quick, and closes the stdin pipe before the ioCopy()
routine start to read from it. This causes a SIGSEGV error.
To fix this issue, I am adding additional read/write tests before
closing the pipes. As the read operation waits for the writer to
be done, this actually synchronizes the threads and make sure
the final tests (with closed pipes) works as expected.
Fixes: #2042
Signed-off-by: Julien Ropé <jrope@redhat.com>
The tracing tags for api.go contain `"packages"` as a tag name,
whereas all other tags contain `"package"`.
Fixes: #2847
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
Display a pseudo path to the sandbox socket in the output of
`kata-runtime env` for those hypervisors that use Hybrid VSOCK.
The path is not a real path since the command does not create a sandbox.
The output includes a `{ID}` tag which would be replaced with the real
sandbox ID (name) when the sandbox was created.
This feature is only useful for agent tracing with the trace forwarder
where the configured hypervisor uses Hybrid VSOCK.
Note that the features required a new `setConfig()` method to be added
to the `hypervisor` interface. This isn't normally needed as the
specified hypervisor configuration passed to `setConfig()` is also
passed to `createSandbox()`. However the new call is required by
`kata-runtime env` to display the correct socket path for Firecracker.
The new method isn't wholly redundant for the main code path though as
it's now used by each hypervisor's `createSandbox()` call.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add support for Hybrid VSOCK. Unlike standard vsock (`vsock(7)`), under
hybrid VSOCK, the hypervisor creates a "master" *UNIX* socket on the
host. For guest-initiated VSOCK connections (such as the Kata agent uses
for agent tracing), the hypervisor will then attempt to open a VSOCK
port-specific variant of the socket which it expects a server to be
listening on. Running the trace forwarder with the new `--socket-path`
option and passing it the Hypervisor specific master UNIX socket path,
the trace forwarder will listen on the VSOCK port-specific socket path
to handle Kata agent traces.
For further details and examples, see the README or run the
trace forwarder with `--help`.
Fixes: #2786.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Adding a route that already exists should not be a reason for the agent to fail
booting and thus preventing the sandbox to start.
Fixes#2712
Signed-off-by: zhaojizhuang <571130360@qq.com>
The 'quiet' kernel parameter can avoid guest kernel logs while booting,
which can reduce boot time.
Fix: #2820
Signed-off-by: Bo Chen <chen.bo@intel.com>
We will need to have console output from the guest only for debugging
purposes. As a result, we can turn-off both the serial and
virtio-console devices by default for better boot time.
Fixes: #2820
Signed-off-by: Bo Chen <chen.bo@intel.com>
Variables in rust will be dropped at the end of the function.
In function real_main the trace will be shut down by `tracer::end_tracing()`,
but at this time the root span is in an active state, so this root span
will not be sent to the trace collector.
This can be fixed by dropping the root span manually.
Fixes: #2812
Signed-off-by: bin <bin@hyper.sh>
The variable for 'name' in config-settings.go.in was previously
hardcoded as "kata". In e7c42fb it was changed to the runtime name,
which is "kata-runtime". Add a variable to specify a syslog identifier
for consistency for tests and documentation that use it.
Fixes#2806
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Update the sandbox dir clean up logic to be more appropriate
Add different seeds for randInt() method
Fixes#2770
Signed-off-by: Feng Wang <feng.wang@databricks.com>
This patch adds an option "disable_seccomp" to the config
hypervisor.clh, from which users can disable the `seccomp`
feature from Cloud Hypervisor when needed (for debugging purposes).
Fixes: #2782
Signed-off-by: Bo Chen <chen.bo@intel.com>
This patch enables the `seccomp` feature from Cloud Hypervisor which
provides fine-grained allowed syscalls for each of its worker
threads. It brings important security benefits, while would increase
memory footprint.
Fixes: #2782
Signed-off-by: Bo Chen <chen.bo@intel.com>
Shim management server is running in a go routine, in test mode
this will cause the directory where the listen socket
file(/run/vc/sbs/777-77-77777777/shim-monitor.sock) in leak
after the tests finished.
Fixes: #2805
Signed-off-by: bin <bin@hyper.sh>
wait_for_pci_device() waits for the PCI device at the given path to become
ready, but it doesn't currently give you any meaningful handle on that
device.
Change the signature, so that it returns the PCI address of the device.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Add a new pci::Address type which represents a guest PCI address in
DDDD:BB:SS.F form.
fixes#2745
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
pci::Slot represents a PCI slot. However, in all cases where we use it, we
actually care about addressing a specific PCI function. So, at the moment
we can only refer to function 0 in each slot.
Replace pci::Slot with pci::SlotFn to represent both the slot and function.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This commit does two chagnes:
- move code for managing temp users to rootless.go.
- use common function in qemu.go when shutdown the VM.
Fixes: #2759
Signed-off-by: bin <bin@hyper.sh>
From the endpoints string described through the configuration file, we
build a hash set of allowed enpoints. If a configuration files does not
include an endpoints section, we assume all endpoints are not allowed.
If there is no configuration file, then all endpoints are allowed.
Then for every ttrpc request, we check if the name of the endpoint is
part of the hashset. If it is not, then we return ttrcp::UNIMPLEMENTED.
Fixes: #1837
Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
When the kernel command line includes a agent.config_file=<path> entry,
then we will try to override the default confiuguration values with the
ones we parse from a TOML file at <path>.
As the configuration file overrides the default values, we need to go
through a simplified builder that convert a set of Option<> fields into
the actual AgentConfig structure.
Fixes: #1837
Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
They will define the list of endpoints that an agent supports.
They're empty and non actionable for now.
Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
A single constructor setting default value is a typical pattern for a
Default implementation.
Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
Even CCA, which is the confidential compute archtecture, has not been
ready, add a empty implementation to avoid static check error.
Fixes: #2789
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Suggested-by: Fabiano Fidêncio <fidencio@redhat.com>
Exclude from lint checking for it is ultimately only used in
architecture-specific code.
Fixes: #2273
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Bump containerd to v1.5.7 in order to bring in a fix for CVE-2021-41103,
"insufficiently restricted permissions ons plugins directories
(https://github.com/advisories/GHSA-c2h3-6mxw-7mvq)".
dependabot found a potential security vulnerability and raised a PR to
fix it. However, dependabot does not properly follows nor understands
the needed of our CIs (mainly related to formatting the PR and whatnot),
thus I'm re-raising it.
Fixes: #2796
Supersedes: #2787
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
Not all routes have either a gateway or a destination IP.
Interface routes, where the source, destination and gateway are undefined,
will default to IP v4 with the current is_ipv6() check even when they
are v6 routes.
We use the provided gRPC Route.Family field instead. This field is built
from the host netlink messages, and is a reliable way of finding out
a route's IP family.
Fixes: #2768
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Our check for the IP family is working as long as we have either a
gateway or a destination IP. Some routes are missing both.
The RT netlink messages provide the IP family information for each
route, so we can carry that piece of information up to the guest. That
will allow for a more reliable route IP family determination.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
We need to be able to get the IP family from the netlink route meesages,
and the Route.Family field only got recently added to the netlink
package.
The update generates static check warnings about the call for
nethandler.Delete() being deprecated in favor of a Close() call instead.
So we include the s/Delete()/Close()/ change as part of this PR.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Reduce the cloud-hypervisor log level from `Debug` to `Info` when hypervisor
debug is enabled. This is required since `Debug` level:
- Is overkill for debugging hypervisor failures.
- Effectively hides the output from the guest kernel and userland: CLH
generates so much output that the output from the guest gets "lost in
the noise" (experiments show that for each full CLH debug message, at most
1 _byte_ of guest output is displayed).
Fixes: #2726.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
modify the make script of the check-go-static, changing the `./cli` path to `./cmd/kata-runtime`
Fixes: #2765
Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
Run tests sometimes generate pkg/containerd-shim-v2/monitor_address,
and `git status` will treat it as a new file.
Package containerd-shim-v2 has moved to pkg/containerd-shim-v2,
the monitor_address in .gitignore should be updated too.
Fixes: #2762
Signed-off-by: bin <bin@hyper.sh>
It seems the client (crio) can send multiple requests to stop the Kata VM,
resulting a nil reference if the uid has already been cleaned up by a different thread.
Fixes#2743
Signed-off-by: Feng Wang <feng.wang@databricks.com>
Since we now have "unix://" kind of socket returned by the
SocketAddress() function, there is no more need to build the sandbox
storage path dynamically to keep OS compatibility.
Fixes: #2738
Suggested-by: Christophe de Dinechin <dinechin@redhat.com>
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
Rectify the values of testModuleData with the correct
types in TestCCCheckCLiFunction in kata-check_(!x86)_test.go
Fixes: #2735
Signed-off-by: Amulya Meka <amulmek1@in.ibm.com>
The agent initiates a PCI rescan from two places. One is triggered
for each virtio-blk PCI device, and one is triggered unconditionally
when we start a new container.
The PCI bus rescan code was added long time ago in Clear Containers due to
lack of ACPI support in QEMU 2.9 + q35. Since Kata routinely plugs devices
under a PCIe-to-PCI bridge, that left SHPC as the only available hotplug
mechanism.
However, while Kata was using SHPC on the qemu side, it wasn't actually
using it on the guest side. Due to a quirk of our guest kernel
configuration, the SHPC driver never bound to the bridge, and *no* hotplug
was working at all. To work around that, Kata was forcing the rescan
manually, which would discover the new device. That was very fragile (we
were arguably relying on a kernel bug). Even if we were using SHPC
propertly, it includes a mandatory 5s delay during plug operations
(designed for physical cards and human operators), which makes it
unsuitable quick start up.
Worse, the forced PCI rescans could race with either SHPC or PCIe native
hotplug sequences, causing several problems. In some cases this could put
the device into an entirely broken state where it wouldn't respond to
config space accesses at all.
Since pull request #2323 was merged, we have instead used ACPI hotplug
which is both fast, and more solid in terms of semantics and races. So,
the forced PCI rescans are no longer necessary. Remove them all.
fixes#683
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
do_add_swap() has some mildly complex code to translate the PCI path of
a virtio-blk device (where the swap will reside) into a /dev path. However,
the device module already has get_virtio_blk_pci_device_name() which does
exactly that. The existing code has some further advantages: it uses
more precise matching of the sysfs paths, and if necessary it will wait for
the device to be added to the guest.
While we're there, remove an unnecessary 'as u8' from the PCI path
construction: pci::Path::new() already accepts anything which implements
TryInfo<u8>, which u32 certainly does.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We send information about several kinds of devices to the agent so
that it can apply specific handling. We don't currently do this with
VFIO devices. However we need to do that so that the agent can
properly wait for VFIO devices to be ready (previously it did that
using a PCI rescan which may not be reliable and has some very bad
side effects).
This patch collates and sends the relevant information.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Both appendBlockDevice and appendVhostUserBlkDevice start by using
GetDeviceByID to lookup the api.Device object corresponding to their
ContainerDevice object. However their common caller, appendDevices() has
already done this.
This changes it so the looked up api.Device is passed to the individual
append*Device() functions. This slightly reduces duplicated work, but more
importantly it makes it clearer that append*Device() don't need to check
for a nil result from GetDeviceByID, since the caller has already done
that.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
For several device types which correspond to a PCI device in the guest
we record the device's PCI path in the guest. We don't currently do
that for VFIO devices, but we're going to need to for better handling
of SR-IOV devices.
To accomplish this, we have to determine the guest PCI path from the
information the VMM gives us:
For qemu, we query the slot of the device and its bridge from QMP.
For cloud-hypervisor, the device add interface gives us a guest PCI
address. In fact this represents a design error in the clh API -
there's no way it can really know the guest PCI address in general.
It works in this case, because clh doesn't use PCI bridges, so the
device will always be on the root bus. Based on that, the PCI path is
simply the device's slot number.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
hotplugVFIODevice() has several different paths depending if we're
plugging into a root port or a PCIE<->PCI bridge and if we're using a
regular or mediated VFIO device.
We're going to want some common code on the successful exit path here,
so refactor the function to allow that without duplication.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently, VFIO devices attached to a Kata container aren't described to
the agent at all. We essentially just hope they're ready by the time
we've entered the container proper, which is usually the case because of
the PCI rescan - but that causes other problems.
This adds a new device type to the agent representing VFIO devices. The
agent will use its existing uevent watching mechanisms to wait for the
associated guest PCI device to appear before proceeding.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently the constants giving the names for each device/driver type in
the protocol are in mount.rs, and used in device.rs. Since these constants
are inherently related to, well, devices, it makes more sense to put them
in device.rs and use them from mount.rs.
This will become even more so with planned extensions which will add some
device types that will not be used in mount.rs at all.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
A few "go fmt" errors appear to have crept it. Clean them up with
"go fmt ./..." in the src/runtime directory.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We would only create the target when updating files. We need to make
sure that we create the target if the source is a directory. Without
this, we'll fail to start a container that utilizes an empty configmap,
for example.
Add unit tests for this.
Fixes: #2638
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
We now support any container engine CRI compliant. Let's bump the
kata-monitor version to 0.2.0.
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
This commit stops the container engine polling in favor of
the kata sandbox storage path monitoring.
The pod cache list is now refreshed based on fs events and synced with
the container engine only when needed.
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
When the container engine is different than containerd or CRI-O we
lack proper detection of kata workloads and consider all the pods as
kata ones.
Instead of querying the container engine for the lower level runtime
used in each pod, check if a directory matching the pod exists in
the virtualcontainers sandboxes storage path.
This provides a container engine independent way to check for kata pods.
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
Under certain circumstances[0] Kata will attempt to use SHPC hotplug
for PCI devices on the guest. In fact we explicitly enable SHPC on
our PCI to PCI bridges, regardless of the qemu default.
SHPC was designed a long, long time ago for physical hotplugging and
works very poorly for a virtual environment. In particular it has a
mandatory 5s delay to allow a (real, human) operator to back out the
operation if they press a button by mistake. This alone makes it
unusable for a fast start up application like Kata.
Worse, the agent forces a PCI rescan during startup. That will race
with the SHPC hotplug operation causing the device to go into a bad
state where config space can't be accessed from the guest at all.
The only reason we've sort of gotten away with this is that our
default guest kernel configuration triggers what's arguably a kernel
bug effectively disabling SHPC. That makes the agent rescan the only
reason we see the new device.
Now that we require a qemu >=6.1, which includes ACPI PCI hotplug on
the q35 machine, we can explicitly disable SHPC in all cases. It's
nothing but trouble.
fixes#2174
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
qemuArchBase.appendBridges is never actually used, because the bare
qemuArchBase type is itself never used (outside of unit tests). Instead
*all* the subclasses of qemuArchBase override appendBridges() to call
the very similar, but not identical genericAppendBridges. So, we can
remove the qemuArchBase.appendBridges implementation.
Furthermore, all those subclasses override appendBridges() in exactly
the same way, and so we can remove *those* definitions and replace the
base class qemuArchBase appendBridges() with that version, calling
genericAppendBridges().
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Update to commit 1b60b536f3, in particular to get extensions to
allow IO and memory window reservations to be set on PCI bridges.
https://github.com/kata-containers/govmm/pull/201
Git log:
de039da govmm/qemu: Let IO/memory reservations be specified for bridge devices
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
There are `DeviceToDeviceCgroup` and `deviceToDeviceCgroup` two functions,
creating a `specs.LinuxDeviceCgroup` object. We clear the new function `deviceToDeviceCgroup`.
Fixes: #2694
Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
A discussion on the Linux kernel mailing list [1] exposed that virtiofsd makes a
core assumption that the file systems being shared are not accessible by any
non-privileged user. We currently create the `shared` directory in the sandbox
with the default `0750` permissions, which gives read and directory traversal
access to the group. There is no real good reason for a non-root user to access
the shared directory, and this is potentially dangerous.
Fixes: #2589
[1]: https://lore.kernel.org/linux-fsdevel/YTI+k29AoeGdX13Q@redhat.com/
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
Retrieve the absolute sandbox storage path. We will soon need this to
monitor the creation/deletion of new kata sandboxes.
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
The storage path we use to collect the sandbox files is defined in the
virtcontainers/persist/fs package.
We create the runtime socket in that storage path, by hardcoding the
full path in the SocketAddress() function in the runtime package.
This commit splits the hardcoded path by the socket address path so that
the runtime package will be able to provide the storage path to all the
components that may need it.
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
In order to retrieve the list of sandboxes, we poll the container engine
every 15 seconds via the CRI. Once we have the list we have to inspect
each pod to find out the kata ones.
This commit extend the sandbox cache to keep track of all the pods,
marking the kata ones, so that during the next polling only the new
sandboxes should be inspected to figure out which ones are using the
kata runtime.
Fixes: #2563
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
this is an unexpected event (likely a change in how containerd/cri-o
record the lower level runtime in the pod) and should be more visible:
raise the log level to "warning".
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
Change logger in Trace call in newContainer from sandbox.Logger() to
nil. Passing nil will cause an error to be logged by kataTraceLogger
instead of the sandbox logger, which will avoid having the log message
report it as part of the sandbox subsystem when it is part of the
container subsystem.
The kataTraceLogger will not log it as related to the container
subsystem, but since the container logger has not been created at this
point, and we already use the kataTraceLogger in other instances where a
subsystem's logger has not been created yet, this PR makes the call
consistent with other code.
Fixes#2665
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Call StopTracing with s.rootCtx, which is the root context for tracing,
instead of s.ctx, which is parent to a subset of trace spans.
Fixes#2661
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
A random generated user/group is used to start QEMU VMM process.
The /dev/kvm group owner is also added to the QEMU process to grant it access.
Fixes#2444
Signed-off-by: Feng Wang <feng.wang@databricks.com>
The only remaining callers of ensure_destination_exists() are in its own
unit tests. So, just remove it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
mount_storage() first makes sure the mount point for the storage volume
exists. It uses fs::create_dir_all() in the case of 9p or virtiofs volumes
otherwise ensure_destination_exists(). But.. ensure_destination_exists()
boils down to an fs::create_dir_all() in most cases anyway. The only case
it doesn't is for a bind fstype, where it creates a file instead of a
directory. But, that's not correct anyway because we need to create either
a file or a directory depending on the source of the bind mount, which
ensure_destination_exists() doesn't know.
The 9p/virtiofs paths also check if the mountpoint exists before calling
fs::create_dir_all(), which is unnecessary (fs::create_dir_all already
handles that case).
mount_storage() does have the information to know what we need to create,
so have it explicitly call ensure_destination_file_exists() for the bind
mount to a non-directory case, and fs::create_dir_all() in all other cases.
fixes#2390
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
ensure_destination_exists() can create either a directory or a regular file
depending on the arguments. This patch extracts the regular file specific
option into its own helper: ensure_destination_file_exists(). This:
- Avoids doing some steps in the directory case (they're already handled
by create_dir_all())
- Enables some further future cleanups
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
struct Baremount contains the information necessary to make a new mount.
As a datastructure, however, it's pointless, since every user just
constructs it, immediately calls the BareMount::mount() method then
discards the structure.
Simplify the code by making this a direct function call baremount().
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
BareMount::mount does some complicated marshalling and uses unsafe code to
call into the mount(2) system call. However, we're already using the nix
crate which provides a more Rust-like wrapper for mount(2). We're even
already using nix::mount::umount and nix::mount::MsFlags from the same
module.
In the same way, we can replace the direct usage of libc::umount() with
nix::mount::umount() in one of the tests.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This patch adds the configuration option that allows to use hugepages
with Cloud Hypervisor guests.
Fixes: #2648
Signed-off-by: Bo Chen <chen.bo@intel.com>
We recently updated to using qemu-6.1 (from qemu 5.2). Unfortunately one
breaking change in qemu 6.0 wasn't caught by the CI.
The query-cpus QMP command has been removed, replaced by query-cpus-fast
(which has been available since qemu 2.12). govmm already had support for
query-cpus-fast, we just weren't using it, so the change is quite easy.
fixes#2643
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Rust 1.47.0 which is the latest we note as tested in versions.yaml is now
getting fairly old - many current distros have newer versions (e.g.
Rust 1.54.0 in Fedora 34). Bring this more up to date.
Note that this is only updating the 'newest-version', not the minimum
required version.
The new version changes the name of the 'clippy::unknown_clipp_lints'
option to simply 'unknown_lints' so we need to change that as well to avoid
warnings.
fixes#2633
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The new API is based on containerd's cgroups package.
With that conversion we can simpligy the virtcontainers sandbox code and
also uniformize our cgroups external API dependency. We now only depend
on containerd/cgroups for everything cgroups related.
Depends-on: github.com/kata-containers/tests#3805
Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Eventually, we will convert the virtcontainers and the whole Kata
runtime code base to only rely on that package.
This will make Kata only depends on the simpler containerd cgroups API.
Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
The only process we are adding there is the container host one, and
there is no such thing anymore.
Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
This is a simplification of the host cgroup handling by partitioning the
host cgroups into 2: A sandbox cgroup and an overhead cgroup.
The sandbox cgroup is always created and initialized. The overhead
cgroup is only available when sandbox_cgroup_only is unset, and is
unconstrained on all controllers. The goal of having an overhead cgroup
is to be more flexible on how we manage a pod overhead. Having such
cgroup will allow for setting a fixed overhead per pod, for a subset of
controllers, while at the same time not having the pod being accounted
for those resources.
When sandbox_cgroup_only is not set, we move all non vCPU threads
to the overhead cgroup and let them run unconstrained. When it is set,
all pod related processes and threads will run in the sandbox cgroup.
Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
Regardless of the sandbox_cgroup_only setting, we create the sandbox
cgroup manager and set the sandbox cgroup path at the same time.
Without doing this, the hypervisor constraint routine is mostly a NOP as
the sandbox state cgroup path is not initialized.
Fixes#2184
Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
Sync the virtcontainers api.md document, add `ConfidentialGuest` `EntropySourceList` `GuestSwap` three
fields to the HypervisorConfig API.
Fixes#2625
Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
sync the virtcontainers api.md document, add SandboxBindMounts field to the SandboxConfig API.
And update the order of the SandboxConfig API fields.
Fixes#2621
Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
If the device has no permission, such as /dev/null, /dev/urandom,
it needs to be added into cgroup.
Fixes: #2615
Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
In getThreadIDs(), the cpuID variable is derived from a string that
already contains a whitespace. As a result, strings.SplitAfter returns
the cpuID with a leading space. This makes any go variant of string to int
fail (strconv.ParseInt() in our case). This patch makes sure that the
leading space character is removed so the string passed to
strconv.ParseInt() is "CPUID" and not " CPUID".
This has been caused by a change in the naming scheme of vcpu threads
for Firecracker after v0.19.1.
Fixes: #2592
Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
Given the fix to the bugs of the openapi spec file is included in the
Cloud Hypervisor v18.0 [1], this patch reverts the workaround we carried
in the CLH driver.
This reverts commit 932ee41b3f.
[1] https://github.com/cloud-hypervisor/cloud-hypervisor/pull/3029
Signed-off-by: Bo Chen <chen.bo@intel.com>
adds the default devices for unix such as /dev/null, /dev/urandom to
the container's resource cgroup spec
Fixes: #2539
Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
There is no need to keep multiple copies of the license file in
different directory. We can just use the top level one for the project.
Fixes: #2553
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Use ExecuteBlockdevAddWithDriverCache with swap in
hotplugAddBlockDevice to handle swap file cannot work OK with
ExecuteBlockdevAddWithCache issue.
Fixes: #2548
Signed-off-by: Hui Zhu <teawater@antfin.com>
Where possible, move attributes added with AddTag() to Trace() call to
reduce the amount of code used for tracing.
Fixes#2512
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Two default values defined in the 'cloud-hypervisor.yaml' have typo, and this
patch manually overwrites them with the correct value as a workaround
before the corresponding fix is landed to Cloud Hypervisor upstream.
Signed-off-by: Bo Chen <chen.bo@intel.com>
With the updated openapi-generator, the client code now handles optional
attributes correctly, and ensures to assign the right default
values. This patch enables to use those constructors to make sure the
proper default values being used.
Signed-off-by: Bo Chen <chen.bo@intel.com>
The general Trace() function accepts one map as a set of tags. Modify it
to accept multiple sets of tags so that additional ones can be added at
Trace() and not as a subsequent call.
Additionally, we should not iterate over the maps unless tracing tracing
is enabled.
Fixes#2512
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Tracing tags are stored inconsistently throughout the runtime. Change
all instances of tracing tags to variables.
Fixes#2512
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
The client code (and APIs) for Cloud Hypervisor has been changed
dramatically due to the upgrade to `openapi-generator` v5.2.1. This
patch migrate the Cloud Hypervisor driver in the kata-runtime to use
those updated APIs.
The main change from the client code is that it now uses "pointer" type
to represent "optional" attributes from the input openapi specification
file.
Signed-off-by: Bo Chen <chen.bo@intel.com>
the katatestutils GoDoc url stilled using the kata 1.x branch url. This PR fixed the
url from kata-containers/runtime/pkg/katatestutils to
kata-containers/kata-containers/src/runtime/pkg/katatestutils
Fixes: #2500
Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
To improve the quality and correctness of the auto-generated code, this
patch upgrade the `openapi-generator` to its latest stable release
v5.2.1.
Fixes: #2487
Signed-off-by: Bo Chen <chen.bo@intel.com>
Generate `config-generated.go` file under src/runtime/cli/containerd-shim-kata-v2 before excuting test or coverage.
Fixes#2479
Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
Now that we have enabled CI tests for jailed firecracker and we have
fixed the issue with removing the block storage device #2387, we
should leverage the full power of firecracker and enable jailer by
default.
Fixes: #2455
Signed-off-by: Jack Rieck <jack.rieck@sendgrid.com>
The call to Trace() in runHooks() should return a context so that
subsequent calls to runHook() produce properly ordered trace spans.
Fixes#2423
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
inotify/watchable-mount changes...
- Allow up to 16 files. It isn't that uncommon to have 3 files in a secret.
In Kubernetes, this results in 9 files in the mount (the presented files,
which are symlinks to the latest files, which are symlinks to actual files
which are in a seperate hidden directoy on the mount). Bumping from eight to 16 will
help ensure we can support "most" secret/tokens, and is still a pretty
small number to scan...
- Now we will only replace the watched storage with a bindmount if we observe
that there are too many files or if its too large. Since the scanning/updating is racy,
we should expect that we'll occassionally run into errors (ie, a file
deleted between scan / update). Rather than stopping and making a bind
mount, continue updating, as the changes will be updated the next time
check is called for that entry (every 2 seconds today).
To facilitate the 'oversized' handling, we create specific errors for too large
or too many files, and handle these specific errors when scanning the storage entry.
- When handling an oversided mount, do not remove the prior files -- we'll just
overwrite them with the bindmount. This'll help avoid the files
disappearing from the user, avoid racy cleanup and simplifies the flow.
Similarly, only mark it as a non-watched storage device after the
bindmount is created successfully.
- When creating bind mount, make sure destination exists. If we hadn't
had a successful scan before, this wouldn't exist and the mount would
fail. Update logic and unit test to cover this.
- In several spots, we were returning when there was an error (both in
scan and update). For update case, let's just log an warning and continue;
since the scan/update is racy, we should expect that we'll have
transient errors which should resolve the next time the watcher runs.
Fixes: #2402
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
changed the document project url in the using-vpp-and-kata.md and
runtime experimental README.md files.
Fixes: #2418
Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
just search for the "kata" substring in the runtime value and log at
info level when the runtime name/type is not found.
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
We keep the container engine info in the sandbox cache map, as the value
associated to the pod id (the key). Since we used that in
getMonitorAddress() only (which is gone) we can avoid storing that
information. Let's drop it.
Keep the map structure and the [put,delete]IfExists functions as we may
want to move to an event based cache update process sooner or later, and
we will need those.
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
since the shim socket path is statically defined in the containerd-shimv2
code, we don't need to retrieve the socket name from the filesystem:
construct the socket name using the containerd-shimv2 code.
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
kata-monitor switched from containerd client to CRI. Update the
dependencies and vendored code.
go mod tidy
go mod vendor
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
kata-monitor uses containerd client to retrieve information from the
container engine. This makes kata-monitor work with the containerd
container engine only.
Bin Liu (bin <bin@hyper.sh>) worked on a kata-monitor version able
to talk to any container engine leveraging the standard CRI[1].
Here, the original work of Bin Lui has been adapted on the current
kata-monitor to make it container engine independent.
[1] https://github.com/liubin/kata-containers/tree/fix/1030-use-cri-in-kata-monitorFixes: #1030
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
When running a firecracker instance jailed, block devices
are not removed correctly, as the jailerRoot path is not
stripped from the PATCH command sent to the FC API.
This patch differentiates the jailed case from the non-jailed
one and allows the firecracker instance to be properly
terminated.
Fixes#2387
Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
The CRI-O integration test suite has two tests that fail because they search for
"not found" in the error message, but we emit "is not exist".
Change the error message to match the expectations of the test suite.
Fixes: #2036
Reported-by: Julien Ropé <jrope@redhat.com>
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
Update to commit 3c64244cbb, in particular to get these fixes which
are needed to work with qemu-6.0 and later:
https://github.com/kata-containers/govmm/pull/192https://github.com/kata-containers/govmm/pull/194
Git log
d27256f (qmp: Don't use deprecated 'props' field for object-add, 2021-08-03)
d8cdf9a (qemu: Drop support for versions older than 5.0, 2021-08-03)
1b02192 (Use 'host_device' driver for blockdev backends, 2021-07-29)
9518675 (add support for "sandbox" feature to qemu, 2021-07-20)
335fa81 (qemu: fix golangci-lint errors, 2021-07-21)
61b6378 (.github/workflows: reimplement github actions CI, 2021-07-21)
9d6e797 (go: support go modules, 2021-07-21)
0d21263 (qemu: support read-only nvdimm, 2021-07-21)
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Kata uses the 'kernel_irqchip' machine option to qemu. By default it
uses it in what qemu calls the "short-form boolean" with no parameter.
That style was deprecated by qemu between 5.2 and 6.0 (commit
ccd3b3b8112b) and effectively removed entirely between 6.0 and 6.1
(commit d8fb7d0969d5).
Update ourselves for newer qemus by using an explicit
"kernel_irqchip=on".
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The swappiness is not right if just set
io.katacontainers.container.resource.swappiness:
$ pod_yaml=pod.yaml
$ container_yaml=container.yaml
$ image="quay.io/prometheus/busybox:latest"
$ cat << EOF > "${pod_yaml}"
metadata:
name: busybox-sandbox1
EOF
$ cat << EOF > "${container_yaml}"
metadata:
name: busybox-killed-vmm
annotations:
io.katacontainers.container.resource.swappiness: "100"
image:
image: "$image"
command:
- top
EOF
$ sudo crictl pull $image
$ podid=$(sudo crictl runp $pod_yaml)
$ cid=$(sudo crictl create $podid $container_yaml $pod_yaml)
$ sudo crictl start $cid
crictl exec $cid cat /sys/fs/cgroup/memory/memory.swappiness
60
The cause of this issue is there are two elements store the resources
infomation. They are c.config.Resources for calculateSandboxMemory and
c.GetPatchedOCISpec() for agent.
This add initConfigResourcesMemory to Container and call it in
newContainer to handle the issue.
Fixes: #2372
Signed-off-by: Hui Zhu <teawater@antfin.com>
Although the OCI specification does not explictly requires that, we
should create the process CWD if it does not exist, before chdir'ing
to it. Without that fizx, the kata-agent fails to create a container
and returns a grpc error when it's trying to change the containerd
working directory to an non existing folder.
runc, the OCI runtime reference implementation, also creates the process
CWD when it's not part of the container rootfs.
Fixes#2374
Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
When enabling tracing with Cloud Hypervisor, we end up establishing 2
connections to 2 different HTTP servers: The Cloud Hypervisor API one
that runs over a UNIX socket and the Jaeger endpoint running over UDP.
Both connections use the default HTTP golang client instance, and thus
share the same transport layer. As the Cloud Hypervisor implementation
sets it up to be over a Unix socket, the jaeger uploader ends up going
through that transport as well, and sending its spans to the Cloud
Hypervisor API server.
We fix that by giving the Cloud Hypervisor implementation its own HTTP
client instance and we avoid sharing it with anything else in the shim.
Fixes#2364
Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
Removed all TOCs now that GitHub auto-generates them.
Also updated the documentation requirements doc removing the requirement
to add a TOC.
Fixes: #2022.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The old ones are carrying CVEs, do not use them.
PS: In order to update the modules, we're running `make handle_vendor`
target from the runtime's Makefile. This is now part of the CI and
ensures that the vendored code is up-to-date. It's important to note
that older versions of golang may generate different results for those,
but those versions are not supported anymore, so we're good to go with
what we have in the CI (1.15 and 1.16).
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
Since the old ones are carrying CVEs. Do not use them.
PS: In order to update the modules, we're running `make handle_vendor`
target from the runtime's Makefile. This is now part of the CI and
ensures that the vendored code is up-to-date. It's important to note
that older versions of golang may generate different results for those,
but those versions are not supported anymore, so we're good to go with
what we have in the CI (1.15 and 1.16).
Fixes: #2338
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
There is a new "ReadOnly" option added to nvdimm device in qemu
and now added to kata. However, qemu used for arm64 is a little
old and has no this feature. Here we remove this feature for arm.
Fixes: #2320
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
This PR updates the experimental documentation with the proper reference
to kata 2.x
Fixes#2317
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Pior our bump to runc 1.0.1 the manager's Set() would take a Config as
its parameter. Now it takes the Resources directly.
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
Previously part of the "system" namespace, the RunningInUserNS() has
been moved to the "userns" namespace.
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
Dependabot brought to us attention that we were still vendoring the runc
code which was affected by CVE-2021-30465.
Although the vulnerability doesn't seem to affect kata-containers, we
better keep our dependencies up-to-date anyways. With this in mind,
let's bump our runc dependency to the latest release.
Fixes: #2309
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
Improve security by making rootfs image read-only, nobody
will be able to modify it from the guest.
fixes#1916
Signed-off-by: Julio Montes <julio.montes@intel.com>
Bring read-only nvdimm support
Shortlog:
335fa81 qemu: fix golangci-lint errors
61b6378 .github/workflows: reimplement github actions CI
9d6e797 go: support go modules
0d21263 qemu: support read-only nvdimm
ff34d28 qemu: Consistent parameter building
Signed-off-by: Julio Montes <julio.montes@intel.com>
Since the monitor socket used the unix socket path file,
which needed to be cleaned after the pod terminated,
thus put it into the sandbox data directory, and it
would be cleaned up once the sandbox termianted.
Fixes: #2269
Signed-off-by: fupan.lfp <fupan.lfp@antgroup.com>
This commit add option "enable_guest_swap" to config hypervisor.qemu.
It will enable swap in the guest. Default false.
When enable_guest_swap is enabled, insert a raw file to the guest as the
swap device if the swappiness of a container (set by annotation
"io.katacontainers.container.resource.swappiness") is bigger than 0.
The size of the swap device should be
swap_in_bytes (set by annotation
"io.katacontainers.container.resource.swap_in_bytes") - memory_limit_in_bytes.
If swap_in_bytes is not set, the size should be memory_limit_in_bytes.
If swap_in_bytes and memory_limit_in_bytes is not set, the size should be
default_memory.
Fixes: #2201
Signed-off-by: Hui Zhu <teawater@antfin.com>
This commit add code to handle the annotations
"io.katacontainers.container.resource.swappiness" and
"io.katacontainers.container.resource.swap_in_bytes".
It will set the value of "io.katacontainers.resource.swappiness" to
c.config.Resources.Memory.Swappiness and set the value of
"io.katacontainers.resource.swap_in_bytes" to
c.config.Resources.Memory.Swap.
Fixes: #2201
Signed-off-by: Hui Zhu <teawater@antfin.com>
ocispec.Annotations is dropped in ContainerConfig.
This commit let it to be set to containerConfig.Annotations in
ContainerConfig.
Fixes: #2201
Signed-off-by: Hui Zhu <teawater@antfin.com>
addSwap will create a swap file, hotplug it to hypervisor as a special
block device and let agent to setup it in the guest kernel.
removeSwap will remove the swap file.
Just QEMU support addSwap.
Fixes: #2201
Signed-off-by: Hui Zhu <teawater@antfin.com>
Add new fuction AddSwap. When agent get AddSwap, it will get the device
name from PCIPath and set the device as the swap device.
Fixes: #2201
Signed-off-by: Hui Zhu <teawater@antfin.com>
'FLAGS' hash map has bool to indicate if the flag should be cleared or
not. But in parse_mount_flags_and_options() we set the flag even 'clear'
is true. This results in a 'rw' mount being mounted as 'MS_RDONLY'.
Fixes: #2262
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
Seems that at least some versions of container, when using ConifgPath,
still rely on the runtime options and its APIs from the not in use
anymore github.com/containerd/cri-containerd/pkg/api/runtimeoptions/v1.
The fact backward compat breaks when moving from the old to the new
runtime options, which happened as part of f60641a6e6d, strongly feels
like a containerd bug. Regardless, we can easily work this around on
our side without much hassle.
Just by importing old runtime options the unmarshalling doesn't break
anymore and we can easily check whether getting the options fails or not
and fallback to the old way if it does.
Fixes: #2258
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
Commit 32c9ae1388 upgrade the
containerd vendor, which used the socket path to replace
the abstract socket address for socket listen and dial, and
there's an bug in containerd's abstract socket dialing.
Thus we should replace our monitor and exec socket server
with the socket path to fix this issue.
Fixes: #2238
Signed-off-by: fupan.lfp <fupan.lfp@antgroup.com>
It's better to check whether the destination file exists
before creating them, if it had been existed, then return
directly.
Fixes: #2247
Signed-off-by: fupan.lfp <fupan.lfp@antgroup.com>
Update to latest tokio to address RUSTSEC-2021-0072:
Task dropped in wrong thread when aborting `LocalSet` task
Update the toml to specify just 1.x for the tokio version.
Fixes: #2165
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Make the vsock-exporter async totally using tokio runtime.
And delay the timing of the connection to trace-forwarder so that
it is easy to reconnect when the connection was broken.
Fixes: #2234
Signed-off-by: Tim Zhang <tim@hyper.sh>
This has a similar intent as the go code, but not totally equal. For
the go code we want to ensure that the vendored code is up-to-date,
while here we want to ensure that `cargo vendor` actually works.
We happened to release a few tarballs where `cargo vendor` didn't work
and it causes some pain for downstream maintainers.
Related: #2159
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
This has a similar intent as the go code, but not totally equal. For
the go code we want to ensure that the vendored code is up-to-date,
while here we want to ensure that `cargo vendor` actually works.
We happened to release a few tarballs where `cargo vendor` didn't work
and it causes some pain for downstream maintainers.
Related: #2159
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
Let's add this target so we can actually enforce, as part of the static
checks (which will be added in a follow-up commit), that our vendored go
code is up-to-date.
Related: #2159
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
This will help us to ensure that we always update the vendored code when
needed. Right now we've been lacking behind and we tend to realise
something change during the next mandatory update, which is not exactly
optimal.
Related: #2159
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
Use a defined error variable replade inplace error, and shortcut
for handling errors returned from function calls.
Fixes: #2187
Signed-off-by: bin <bin@hyper.sh>
When checking clh's binary path if valid, return error even
though the error is not a IsNotExist error.
And add errors to log filed when errors occurred.
Fixes: #2208
Signed-off-by: bin <bin@hyper.sh>
Removes custom trace functions defined across the repo and creates
a single trace function in a new katatrace package. Also moves
span tag management into this package and provides a function to
dynamically add a tag at runtime, such as a container id, etc.
Fixes#1162
Signed-off-by: Benjamin Porter <bporter816@gmail.com>
get_mounts() parses /proc/self/mountinfo in order to get the mountpoints
for various cgroup filesystems. One of the entries in mountinfo is the
"device" for each filesystem, but for virtual filesystems like /proc, /sys
and cgroups, the device entry is arbitrary. Depending on the exact rootfs
setup, it can end up being "-".
This breaks get_mounts() because it uses " - " as a separator. There
really is a " - " separator in mountinfo, but in this case the device entry
shows up as a second one. Fix this, by changing a split to a splitn, which
will effectively only consider the first " - " in the line.
While we're there, make the warning message more useful, by having it
actually show which line it wasn't able to parse.
fixes#2182
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We'd better stop the virtiofsd specifically after stop qemu,
instead of depending on the qemu's termination to notify virtiofsd
to exit.
Fixes: #2211
Signed-off-by: fupan.lfp <fupan.lfp@antgroup.com>
For some reason our static check started to get opinionated about code
that's been there for ages.
One of the suggestions is to improve:
```
INFO: Running golangci-lint on /home/fidencio/go/src/github.com/kata-containers/kata-containers/src/runtime/containerd-shim-v2
utils_test.go:76:36: S1039: unnecessary use of fmt.Sprintf (gosimple)
testDir, err = ioutil.TempDir("", fmt.Sprintf("shimV2-"))
```
And that's what this PR is about.
Fixes: #2204
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
For some reason our static check started to get opinionated about code
that's been there for ages.
One of the suggestions is to improve:
```
INFO: Running golangci-lint on /home/fidencio/go/src/github.com/kata-containers/kata-containers/src/runtime/virtcontainers/pkg/agent/protocols/client
client.go:431:2: S1017: should replace this `if` statement with an unconditional `strings.TrimPrefix` (gosimple)
if strings.HasPrefix(sock, "mock:") {
```
And that's what this PR is about.
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
First of all, cpuset.go just comes from kubernetes and we shouldn't be
doing much with this file apart from updating it every now and then
(but that's material for another PR).
Right now, due to some change on the static checks we use as part of our
CI, we started getting issues as:
```
INFO: Running golangci-lint on /home/fidencio/go/src/github.com/kata-containers/kata-containers/src/runtime/virtcontainers/pkg/cpuset
cpuset.go:60:2: SA4005: ineffective assignment to field Builder.done (staticcheck)
b.done = true
```
For those, let's just ignore the lint and move on.
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
Forward-port of https://github.com/kata-containers/agent/pull/600.
Enable virtio-blk-ccw devices in agent (virtio-blk for s390x, already
enabled in runtime).
Fixes: #2026
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Added a `--dump-only` option which disables forwarding of trace spans.
This essentially makes the forwarder a NOP but can be useful for testing
purposes.
Fixes: #2132.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Since we only send an shutdown qmp command to qemu when do
stopSandbox, and didn't wait until qemu process's exit, thus
we'd better to make sure it had exited when shimv2 terminated.
Thus here to do the last cleanup of the hypervisor.
Fixes: #2198
Signed-off-by: fupan.lfp <fupan.lfp@antgroup.com>
It looks like the version check for cloud hypervisor (clh) was added
initially when clh was actively evolving its API. We no longer need the
version check as clh API has been fairly stable for its recent releases.
Fixes: #1991
Signed-off-by: Bo Chen <chen.bo@intel.com>
This commit clean up config parsing and testing code to make it a bit more easy to maintain.
- Adds `with_context` from anyhow to include the underlying error. This helps to understand what exactly went wrong.
- Uses ensure and bail as a shorter alternative for `if` checks.
- TestData in test_parse_cmdline is now implements Default to reduce boilerplate code
- Remove `make_err` as it doesn’t make any sense.
Fixes: #2177
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
Use which to find the full path of exe before run execute_hook
to avoid error: 'No such file or directory'
Fixes: #2172
Signed-off-by: Tim Zhang <tim@hyper.sh>
For some simple functions that only process memory data(list/hashmap),
they don't need to be instrumented.
And sometime they may generate non-parent spans, if they are called from
daemon-style "threads".
Fixes: #1968
Signed-off-by: bin <bin@hyper.sh>
Pass span context through ttRPC metadata, that
agent can get parent from the context to create
new sub-spans.
Fixes: #1968
Signed-off-by: bin <bin@hyper.sh>
Use common functions wrapping logic of getting values
from annotations, parsing bool/uint32/uint64 and setting
to struct fields.
Fixes: #2082
Signed-off-by: bin <bin@hyper.sh>
Since the rpc spec used an interface to represen the ErrnoRet,
thus the transform function of OCItoGRPC should take care of
this case.
Depends-on: github.com/kata-containers/tests#3629
Fixes: #1441
Signed-off-by: fupan.lfp <fupan.lfp@antgroup.com>
Fix the mismatch bring by the upgrading of vendor of containerd,
cgroup and runtime spec.
Fixes: #1441
Signed-off-by: fupan.lfp <fupan.lfp@antgroup.com>
Since the latest containerd's shimv2 had changed the socket
from abstract unix socket to path unix socket, thus we'd
better to update the vendor to match with the latest containerd.
containerd from v1.3.9, v1.4.3 and v1.5.0 used the path unix socket
instead of abstract socket, thus kata wouldn's support the
containerd's version older than them.
Fixes: #1441
short logs:
15d9703d6 Remove ARM64 releases from release notes
5d2e8e86d Revert "Release artifacts for Linux ARM64"
7942ae68b Revert "Specify seccomp target arch for CC"
3187b6dc8 tests: Adds consumed memory stats test
969ec8949 Specify seccomp target arch for CC
c19b7b64d RELEASES.md: recommend alternatives for deprecated
features
8a62aa1c3 Deprecate built-in aufs snapshotter
4e7915f80 CI: allow Go 1.13 for Docker/Moby compatibility
8e589e873 Vagrantfile: update to Fedora 34
5847340a7 tests: Refactors container image usage
9f43eade6 Prepare v1.5.0-rc.3 release notes
4c7b960cb prow needs some additional setup for docker buildx
2e4c1d4b7 Use the multi-arch version of the test images
4e00c4b65 integration tests needs lsof
177273680 Add script to build test images
1b5d59dfe Add multi-arch support for test images
78e529727 add integration tests
2b0e6cdd4 Separate jobs for build and test for openlab/arm64
cdd075853 Release artifacts for Linux ARM64
efcb18742 Add unit tests for PID NamespaceMode_TARGET validation
b48f27df6 Support PID NamespaceMode_TARGET
909660ea9 process: use the unbuffered channel as the done signal
0f332dadd Update cgroups for regenerated protos
391b123a5 adds quiet option for ref
ab1654d0e Fix PushHandler cannot push image that contains duplicated
blobs
00f8d32ef add not found debug out for check cmd; update usage
55734b1c5 Prepare 1.5.0-rc.2 release notes
3ef337ae3 Update containerd vendors to tags
fbe1e140f Update Go to 1.16.3
c1d1edbad gha: use sudo -E in some places to prevent dropping
env-vars
7966a6652 Cleanup code
5d79d3adb go.mod: update kubernetes to v1.20.6
1c03c377e go.mod: github.com/containerd/fifo v1.0.0
12a2a2108 go.mod: github.com/google/uuid v1.2.0
3292ea586 pkg/seccomp: use sync.Once to speed up IsEnabled
00b5c99b1 pkg/seccomp: simplify IsEnabled, update doc
6dd29c25f go.mod: github.com/containerd/aufs
330a2a809 go.mod: github.com/containerd/zfs
34780d67a runtime/shim: check the namespace flag first
c3dde8c4b freebsd: add zfs to the default plugins
b431fe4fc freebsd: don't run shim delete in deleted dir
1f4192daf freebsd: exclude v1 runtimes
cb1580937 metadata: improve deleting a non-empty namespace's error
message
5bf84034d Remove junit test result processor
b83d04f91 Add variable names to runtime's interface definitions
993b86399 Add shim start opts
9e576b889 Optimize backoff
5c02688b5 converter: use OpenWriter helper function
fcf3b275f Add lock for ListPids
fdb76f55d Fix backword-compatibility issue of non-versioned config
file
d21fe4625 adds log for each failed host and status not found on host
8a4cbabc6 Reimport windows layers when comitting snapshots
2de38a926 fix(windows): create debug npipe failure
41fc516a2 docs/rootless.md: recommend "easy way" over "hard way"
864a3322b go.mod: github.com/containerd/go-cni v1.0.2
ee34caccb go.mod: github.com/Microsoft/go-winio v0.4.17
d478676d3 go.mod: github.com/containerd/imgcrypt v1.1.1
1dd45d51c go.mod: github.com/containerd/typeurl v1.0.2
abd4be07a fix the 404 url
978ebbef6 Prepare 1.5.0-rc.1 release
ce116d4c5 go.mod: github.com/containerd/imgcrypt
v1.1.1-0.20210412181126-0bed51b9522c
0550c3233 containerd-stress: add snapshotter option for stress test
to use
8a04bd052 address recent runtimes config confusion
c4778fe1b go.mod: github.com/containernetworking/plugins v0.9.1
5ce35ac39 devmapper: log pool status when mkfs fails
75097b8ca hcsshim seems to have been updated
9ad087947 Switch all our tests to version 2
e96d2a5d9 Revert "remove two very old no longer used runtime
options"
14f357b90 CI: update crun to 0.19
294331060 go.mod: github.com/containerd/console v1.0.2
bb6c0c2de Add more bolt utils
0ad8c0a16 Decouple shim start from task creation
c7504987e Implement windowsDiff.Compare via hcsshim/pkg/ociwclayer
a64a76846 Replace inline applyWindowsLayer using hcsshim
149fa366f Don't tease the logger with a %-less format string
b399e2ef6 Don't lose Compare failure if aborting diff upload fails
36bf3f0e8 go.mod: github.com/Microsoft/hcsshim v0.8.16
8e1a8ecd8 Prepare v1.5.0-rc.0
45df696bf Fix return event publishing error
4bc8f692f optimize cri redirect logs
9bc8d63c9 cri/server: use containerd/oci instead of
libcontainer/devices
dd16b006e merge in the move to the new options type
9144ce967 shows our runc.v2 default options in the containerd
default config
3d20fa930 fix TestSetOOMScoreBoundaries
4d4117415 Change CRI config runtime options type
21ebeef74 integration: use busybox:1.32.0 since latest is
unavailable
f9bcf4a8a add section link
d4be6aa8f rm mirror defaults; doc registry deprecations
7bb73da6b runtime/v2/shim: remove unused SetScore() and remove
sys.OOMScoreMaxKillable
91e7d21ee sys: add AdjustOOMScore() utility
44240116a sys: add boundary checks to SetOOMScore()
ace1912bb sys: use assert for error checks in OOM tests
6e7271522 sys: add missing pre-condition checks in tests
badd60d3f sys: un-export runningPrivileged(), remove
runningUnprivileged()
21a175860 go.mod github.com/klauspost/compress v1.11.13
58c5fd09e re-enable cri test
da998c81e move to gcr.io/k8s-staging-cri-tools test images
8ba8533bd pkg/cri/opts.WithoutRunMount -> oci.WithoutRunMount
92ea98eda cri-cni-release: add imgcrypt binaries (v1.1.0)
4c1fa5719 remotes/docker: Only return "already exists" on push when
the upload was successful
0186a329e remove two very old no longer used runtime options
58a07754a Temporarily disable cri-tools critest
7ae0a60fb Add OCI ref.name to unique key in remotes handler
5ada2f74a Keep host order as defined in TOML file
d9ff8ebef support multi-arch images for windows via ctr
af1e2af72 ci: upload junit formatted test results
6866b36ab Add workaround to keep docker hosts structs private
c54d92c79 image: use generic decompressor for calculating DiffID
1faca349e integration/client: rename package to "client"
6fc9e4500 synchronize replace rules in integration/client go.mod
with main go.mod
9e19a2984 Fix hosts test on Windows
3f406d4af Cleanup vendor
d56b49c13 Rewrite Docker hosts parser
e1f51ba73 Use os.File#Seek() to get the size of a block device
ddd4298a1 Migrate current TOML code to github.com/pelletier/go-toml
499c2f7d4 Vendor github.com/pelletier/go-toml
61c749036 integration/util: remove dependency on k8s.io/klog/v2
d9765f7bf Extend default timeout for nested VM integration run
5e94745f2 ctr: add --user for task exec
f8c2f0475 remotes/ctr: allow to limit max concurrent uploads like
downloads
4674ad7be Ignore some tests on darwin
55450e773 Run unit tests on CI for MacOS
311e326a1 Add CI job to cross compile all the things
10a498c7c Update go-winio to fix compile error on armv7
1a9c6f557 Revendor zfs to to fix integer overflow
1fd3d12f9 `go mod tidy` the client integration test module
da7d96ba3 Clean up WCOW layers after tests in the correct order
9ad87b9ba adds critools-version
72b7f4bab task: allow checkpoint on pause state
e4b9b1038 Make CRI registry docs more clear
ec4d7736d Increase timeout for linux integration tests
eb7c7c71e Fix oom tests on non Linux
708299ca4 Move RunningInUserNS() to its own package
0886ceaea Fix reference ordering in CRI image store
bf9db47e8 add caller info to the testHook
305b42583 use happy-eyeballs for port-forwarding
22ef69d77 Support HTTP debug in ctr
01765d097 night ci fix: add packages for ubuntu 20.04
8cdc1f13b go.mod: github.com/containerd/zfs
v0.0.0-20210322090317-0e92c2247fb7
30e1e66e5 runtime/v2: Fix defer cleanup
33776ada0 Use specific image for user namespaces tests
7704fe72d Specifically mention "mkfs.ext4" on the error from the
command
1410220d8 Fix error log when copy file
fe787efa2 Fix error log when kill shim
8d8c15ca5 contentproxy: ensure grpc stream is closed on commit
6e343f25e Switch test image to a non rate-limited manifest list
9fdc96c09 runtime/v2: add comment for checkCopyShimLogError
24602e7a9 change default runtime for containerd-stress app
8731888ec Re-enable CRIU tests by not using overlayfs snapshotter
b520428b5 Fix CRIU
4e76bcf06 gofmt -s -w all the things
569023fd5 go.mod: github.com/containerd/nri
v0.0.0-20210316161719-dbaa18c31c14
0e1f59e89 go.mod: github.com/containerd/zfs
v0.0.0-20210315114300-dde8f0fda960
ffff68866 upgrade pause image to 3.5 for non-root
88d3881e1 go.mod: github.com/containerd/fifo
v0.0.0-20210316144830-115abcc95a1d
a22c43fa4 go.mod: github.com/containerd/aufs
v0.0.0-20210316121734-20793ff83c97
f6f861736 go.mod: github.com/containerd/btrfs
v0.0.0-20210316141732-918d888fb676
460b35236 go.mod: kubernetes v1.20.4
5e484c961 runtime/v2/runc: fix the defer cleanup of the NewContainer
e6086d9c0 Prepare release notes for v1.5.0-beta.4
34b7a5f09 Update mailmap
ba8f9845e move overlay-checks to an overlayutils package
7776e5ef2 Support adding devices by dir
d895118c7 runtime/v2/runc: fix leaking socket path
a76cefd12 plugin status should be skip, not error
766e7953a Change dgst to digest in debug
4e8b2f309 rootfs: fix the error handling of the createInitLayer
d3ad7f390 cmd/ctr: use e.g. in the command usage
231bbdc37 cmd/ctr: fix export command
ecb881e5e add imgcrypt stream processors to the default config
ac2726e12 cmd/containerd: deduplicate config*.go
9a7ca39cb defaults: add DefaultConfigDir
8f863afd3 Use net.IP.IsLoopback() to match loopback addresses
eabd9b98b runtime: ignore file-already-closed error if dead shim
Signed-off-by: fupan.lfp <fupan.lfp@antgroup.com>
Our dependencies already bring several versions of nix, we should avoid
adding even more fragementation.
Fixes#2114
Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
To workaround virtiofs' lack of inotify support, we'll special case
particular mounts which are typically watched, and pass on information
to the agent so it can ensure that the mount presented to the container
is indeed watchable (see applicable agent commit).
This commit will:
- identify watchable mounts based on file count and mount source
- create a watchable-bind storage object for these mounts to
communicate intent to the agent
- update the OCI spec to take the updated watchable mount source into account
Unit tests added and updated for the newly introduced
functionality/functions.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Add support for watchable-bind storage driver. When watchable-bind storage
is present, the agent will create a watchable path in a tmpfs, and poll the
watchable-bind source to keep this new mount-point up to date.
This poll will allow the agent to present the mount-point to the
container, allowing for inotify usage by the container workload.
If a mount becomes too large, either in file count or in overall size,
we want to stop treating it as watchable, and instead just treat as a
bindmount. This'll help avoid DoS by growing tmpfs too large, as well
as limiting time spent scanning files. If a watchable-bind grows beyond
8 files (arbitrary sane number for certs/secrets) or 1MB (limit on ConfigMap size),
we treat it as a normal bind.
Fixes: #1879
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
agent: watcher: SandboxStorages check loop cleanup
There's no reason to pass the paths; they can be
determined when they are actually used.
Let's make the return values more comparable to the other mount handling
functions (we'll add storage object in future commit), and pass the mount maps as
function parameters.
...No functional changes here...
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Architectures that do not support memory hotplugging will fail when
memory limits are set because that amount is hotplugged. Issue a warning
instead. The long-term solution is virtio-mem.
Fixes: #1412
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
After create an container/exec successfully, containerd
would wait it immediately, and if start it failed, there
is no chance to send value to exitCh, thus the wait goroutine
would blocked for ever and had no chance to exit.
Fixes: #2087
Signed-off-by: fupan.lfp <fupan.lfp@antgroup.com>
Currently the virtio-mem device is hotplugged on the root bus.
This doesn't work for PCIe machines like q35.
Hotplug the virtio-mem device into the pci bridge instead.
Fixes#1953
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
Keeping around two different x86 machines has no added value
and require more tests and maintenance. Prefer the q35 machine
since it has more features and drop the pc machine.
Fixes#1953
Depends-on: github.com/kata-containers/tests#3586
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
We should update golang proto files.
These changes are updated using libprotoc v3.6.1.
Fixes: #2064
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
This PR updates the correct url for kata installation guides in kata 2.x
Fixes#2069
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
To print the correct value of kernel parameters, the log field
value should not be a function name. And for that qemuArchBase
doesn't contain debug flag, so the log contains debug/non-debug
parameters.
Fixes: #2048
Signed-off-by: bin <bin@hyper.sh>
`memory_offset` is used to increase the maximum amount of memory
supported in a VM, this offset is equal to the NVDIMM/PMEM device that
is hot added, in real use case workloads such devices are bigger than
4G, which is the current limit (uint32).
fixes#2006
Signed-off-by: Julio Montes <julio.montes@intel.com>
It's hard to visually scan over the list currently.
Therefore, we should sort the list alphabetically to scan easily.
Fixes: #1999
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Since SEV support has been added, an implementation mistake was also
added to TestQemuAmd64AppendProtectionDevice.
appendProtectionDevice() will, as it name says, append the protection
device to whatever was there previously. So, when SEV was added, we
broke the comparison done for TDX as we didn't append the expected
output for TDX with what we already had for SEV.
This should be enough to get the tests passing.
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
This PR removes old links that were used in kata 1.x but not
longer valid for kata 2.x
Fixes#2019
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The call to Trace() in runHooks() does not return a context; fix this so
that the subsequent calls to runHook() produces a properly ordered trace
span.
Fixes#2001
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Add the following mount options to catch up with the runtime spec
- silent
- loud
- (no)acl
- (no)iversion
- (no)lazytime
Fixes: #1999
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
We need to fix some agent's code to conform to the latest nix crate
to be able to use new features of the nix.
Fixes: #1987
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Remove storeSandbox() at the end of createSandboxFromConfig(),
because this callchain createSandboxFromConfig -> createContainers
has already calls storeSandbox().
This can improve the startup speed of the container,
even just for a little.
Fixes: #1980
Signed-off-by: Liang Zhou <zhoul110@chinatelecom.cn>
Secure Execution is a confidential computing technology on s390x (IBM Z
& LinuxONE). Enable the correspondent virtualization technology in QEMU
(where it is referred to as "Protected Virtualization").
- Introduce enableProtection and appendProtectionDevice functions for
QEMU s390x.
- Introduce CheckCmdline to check for "prot_virt=1" being present on the
kernel command line.
- Introduce CPUFacilities and avilableGuestProtection for hypervisor
s390x to check for CPU support.
Fixes: #1771
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Previously, all consts were in single lines in
virtcontainers/qemu_s390x.go. Put them into a const block.
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
See also: little-dude/netlink#165
Fixes: #1952
Because the author of netlink has no time to maintain the crate
(https://github.com/little-dude/netlink/issues/161), so we
need to switch the dependency to github temporarily.
Signed-off-by: Tim Zhang <tim@hyper.sh>
Trace spans erroneously set the network model to default in all cases.
Add function to return network model string and use it to set attribute
in spans.
Fixes#1878
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Add sandbox, container, and hypervisor IDs to trace spans. Note that
some spans in sandbox.go are created with a trace() call from api.go.
These spans have additional attributes set after span creation to
overwrite the api attributes.
Fixes#1878
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Implement an openTelemetry custom exporter that sends trace spans to a
VSOCK socket. A VSOCK-to-span converter (such as the Kata trace
forwarder) needs to be running on the host to allow systems like Jaeger
to capture the trace spans.
By default, tracing is not enabled (meaning a NOP tracer is used). To
activate tracing, set the `agent.kata.enable_tracing=true` in the
configuration file.
The type of tracing this change introduces is "static isolated"
tracing. See [1] for further details.
> **Note:**
>
> This change only provides the foundational changes for agent
> tracing work. The feature is _not_ yet complete since it does
> not yet show the correct trace hierarchy.
Fixes: #60.
[1] - https://github.com/kata-containers/agent/blob/master/TRACING.md
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Export proc stats for virtiofsd.
This commit only adds for hypervisors that have support for it.
- qemu
- cloud-hypervisor
Fixes: #1926
Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
It is in real life usage as we put non constrained sandbox processes
(like shim) in a separate cgroup path.
Fixes: #1944
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
CloudHypervisor is using virtiofsd.go to manage virtiofsd process,
but qemu has its code in qemu.go. This commit let qemu to re-use
code in virtiofsd.go to reduce code and improve maintenanceability.
Fixes: #1933
Signed-off-by: bin <bin@hyper.sh>
When update memory limit, we should adapt the write sequence
for memory and swap memory, so it won't fail because
the new value and the old value don't fit kernel's
validation.
Fixes: #1917
Signed-off-by: fupan.lfp <fupan.lfp@antgroup.com>
The Rust standard library had suppressed the default SIGPIPE
behavior, see https://github.com/rust-lang/rust/pull/13158.
Since the parent's signal handler would be inherited by it's child
process, thus we should re-enable the standard SIGPIPE behavior as a
workaround.
Fixes: #1887
Signed-off-by: fupan.lfp <fupan.lfp@antgroup.com>
Span attributes (tags) are not consistent in runtime tracing, so
designate and use core attributes such source, package, subsystem, and
type as span metadata for more understandable output.
Use WithAttributes() during span creation to reduce calls to
SetAttributes().
Modify Trace() in katautils to accept slice of attributes so multiple
functions using different attributes can use it.
Fixes#1852
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
In kata-runtime check sub-command, checks cgroups and SandboxCgroupOnly
to show message if the SandboxCgroupOnly is not set to true
and cgroup v2 is used.
Fixes: #1927
Signed-off-by: bin <bin@hyper.sh>
Use the tracing crate to create automatic trace spans for the _majority_
of top-level modules.
Note that not all functions in the top-level modules can be traced:
- Some functions cannot be traced due to the requirement that all
function parameters implement the `Debug` trait. In some cases (such
as `netlink.rs`), objects are being passed that are defined in
different crates and which do not implement `Debug`.
- Some functions may never return (`signal.rs`).
- Some functions are inlined.
- Some functions are very simple getter/setter functions.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This commit will fix two problems:
- Virtiofsd process ID returned to the caller will always be 0,
the pid var is never being assigned a value.
- Socket listen fd may leak in case of failure of starting virtiofsd process.
This is a port of be9ca0d58bFixes: #1931
Signed-off-by: bin <bin@hyper.sh>
Protected Execution Facility(PEF) is the confidential computing
technology on ppc64le. This PR adds the support for it in Kata.
Also re-vendor govmm for the latest changes.
Fixes: #1881
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
When starting a container and dropping all capabilities,
the init child process has no permission to read the exec.fifo
file because the parent set the file mode 0o622. So change the exec.fifo file mode to 0o644.
fixes#1913
Signed-off-by: quanweiZhou <quanweiZhou@linux.alibaba.com>
Without this, if the shim dies, we will not have a reliable way to
identify what mounts should be cleaned up if `containerd-shim-kata-v2
cleanup` is called for the sandbox.
Before this, if you `ctr run` with a sandbox bindmount defined and SIGKILL the
containerd-shim-kata-v2, you'll notice the sandbox bindmount left on
host.
With this change, the shim is able to get the sandbox bindmount
information from disk and do the appropriate cleanup.
Fixes#1896
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
If for any reason there's an error when trying to setup the sandbox
bindmounts, make sure we roll back any mounts already created when
setting up the sandbox.
Without this, we'd leave shared directory mount and potentially
sandbox-bindmounts on the host.
Fixes: #1895
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
we can have the following case,
1. start kata container with factory feature, this need kata-runtime
config to enable factory and use initrd as base image.
2. start a kata container.
3. cd /root; cd /run/vc/vm/template dir, this will make
/run/vc/vm/template to be in used.
4. destroy vm template with kata-runtime factory destroy , and check
the template mountpoint.
we can see the template mountpoints will add everytime we repeat the above steps .
[root@centos1 template]# mount |grep template
[root@centos1 template]# docker run -ti --rm --runtime untrusted-runtime --net none busybox echo
[root@centos1 template]# cd /root; cd /run/vc/vm/template/
[root@centos1 template]# /kata/bin/kata-runtime factory destroy
vm factory destroyed
[root@centos1 template]# mount |grep template
tmpfs on /run/vc/vm/template type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=2105344k)
[root@centos1 template]# docker run -ti --rm --runtime untrusted-runtime --net none busybox echo
[root@centos1 template]# cd /root; cd /run/vc/vm/template/
[root@centos1 template]# /kata/bin/kata-runtime factory destroy
vm factory destroyed
[root@centos1 template]# mount |grep template
tmpfs on /run/vc/vm/template type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=2105344k)
tmpfs on /run/vc/vm/template type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=2105344k)
Fixes: #938
Signed-off-by: Shukui Yang <keloyangsk@gmail.com>
While evaluating the possibility of having kata-agent statically linked
to the GNU libc, we've ended up facing some issues with prctl.
When debugging the issues, we figured out that the crate hasn't been
maintained since 2015 and that the capctl one is a good 1:1 replacement
for what we need.
Fixes: #1844
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
covertool has no active since 2018 and is not compatible with go1.16
../vendor/github.com/dlespiau/covertool/pkg/cover/cover.go:76:29: cannot use f (type dummyTestDeps) as type testing.testDeps in argument to testing.MainStart:
dummyTestDeps does not implement testing.testDeps (missing SetPanicOnExit0 method)
Fixes: #1862
Signed-off-by: Shengjing Zhu <zhsj@debian.org>
Otherwise we might block delete and create orphan containers.
Fixes: #1039
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Since the propagation flags couldn't be combinted with the
standard mount flags, and they should be used with the remount,
thus it's better to split them from the standard mount flags.
Fixes: #1699
Signed-off-by: fupan.lfp <fupan.lfp@antgroup.com>
Following the fix for #1713, adding a unit test for ioCopy() that
verifies that data is properly copied from source to destination
whatever the order in which the pipes are closed.
Fixes#1831
Signed-off-by: Julien Ropé <jrope@redhat.com>
For easier debug, let's add subcommand to kata-runtime for gathering
metrics associated with a given sandbox.
kata-runtime metrics --sandbox-id foobar
Fixes: #1815
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Gathering stats for a given sandbox is pretty useful; let's export a
function from katamonitor pkg to do this.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
(1) Add an accessor function, SocketAddress, to the shim-v2 code for
determining the shim's abstract domain socket address, given the sandbox
ID.
(2) In kata monitor, create a function, BuildShimClient, for obtaining the appropriate
http.Client for communicating with the shim's monitoring endpoint.
(3) Update the kata CLI and kata-monitor code to make use of these.
(4) Migrate some kata monitor methods to be functions, in order to ease
future reuse.
(5) drop unused namespace from functions where it is no longer needed.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Instead of having something like
"/containerd-shim/$namespace/$sandboxID/shim-monitor.sock", let's change
the approach to:
* create the file in a more neutral location "/run/vc", instead of
"/containerd-shim";
* drop the namespace, as the sandboxID should be unique;
* remove ".sock" from the socket name.
This will result on a name that looks like:
"/run/vc/$sandboxID/shim-monitor"
Fixes: #497
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
Define the structure and functions needed to support confidential
guests, this commit doesn't add support for any specific technology,
support for TDX, SEV, PEF and others will be added in following
commits.
Signed-off-by: Julio Montes <julio.montes@intel.com>
Define config options to enable or disable confidential computing and
its features, for example:
* Image service offloading
* Image decryption keys
Signed-off-by: Julio Montes <julio.montes@intel.com>
This commint include two types of fixes for comments
in src/runtime/containerd-shim-v2/start.go.
- Update comment for calling of watchOOMEvents.
- Comments without heading spaces.
Fixes: #1750
Signed-off-by: bin <bin@hyper.sh>
Got:
FATA[0000] run pod sandbox: rpc error: code = Unknown desc = failed to
create containerd task: Add 189759MB virtio-mem-pci fail QMP command
failed: backend memory size must be multiple of 0x200000: unknown
This commit let sizeMB be multiple of 2Mib to fix the issue.
Fixes: #1796
Signed-off-by: Hui Zhu <teawater@antfin.com>
Some applications may fail if NOFILE limit is set to unlimited.
Although in some environments this value is explicitly overridden,
lets set it to a more sane value in case it doesn't.
Fixes#1715
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
`CPUFlags` returns a map with all the CPU flags, these CPU flags
may help us to identiry whether a system support confidential computing
or not.
Signed-off-by: Julio Montes <julio.montes@intel.com>
In file src/agent/rustjail/src/validator.rs,
these two functions are not used:
- get_namespace_path
- check_host_ns
Fixes: #1783
Signed-off-by: bin <bin@hyper.sh>
Quotes from the cloud-hypervisor release v15.0:
This release is the first in a new version numbering scheme to represent that
we believe Cloud Hypervisor is maturing and entering a period of stability.
With this new release we are beginning our new stability guarantees.
Other highlights from the latest release include: 1) Network device rate
limiting; 2) Support for runtime control of `virtio-net` guest offload;
3) `--api-socket` supports file descriptor parameter; 4) Bug fixes on
`virtio-pmem`, PCI BARs alignment, `virtio-net`, etc.; 5) Deprecation of
the "LinuxBoot" protocol for ELF and bzImage in the coming release.
Details can be found: https://github.com/cloud-hypervisor/cloud-hypervisor/releases/tag/v15.0
Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by `openapi-generator` [1-2]. As the API changes do not
impact usages in Kata, no additional changes in kata's runtime are
needed to work with the current version of cloud-hypervisor.
[1] https://github.com/OpenAPITools/openapi-generator
[2] https://github.com/kata-containers/kata-containers/blob/main/src/runtime/virtcontainers/pkg/cloud-hypervisor/README.mdFixes: #1779
Signed-off-by: Bo Chen <chen.bo@intel.com>
Allow and configure vhost-user-fs devices (virtio-fs) on s390x. As a
consequence, appendVhostUserDevice now takes a context, which affects
its signature for other architectures.
Fixes: #1753
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
This reverts commit 7f60911333.
Patch allowed other vhost user devices besides FS not supported on s390x
and failed to attach a CCW device number, which results in the
inavailability to use more devices after vhost-user-fs-ccw.
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
cgroupsCreate will just keep the CPU resources infomation but not the
others.
Set it to c.config.Resources will clean most of resources of the
container.
This commit remove it to handle the issue.
Fixes: #1758
Signed-off-by: Hui Zhu <teawater@antfin.com>
The pointer that send to newContainer in CreateContainer and
createContainers is not the pointer that point to the address in
s.config.Containers.
This commit fix this issue.
Fixes: #1758
Signed-off-by: Hui Zhu <teawater@antfin.com>
If the QEMU VMM fails to launch, we currently fail to kill virtiofsd,
resulting in leftover processes running on the host. Let's make sure we
kill these, and explicitly cleanup the virtiofs socket on the
filesystem.
Ideally we'll migrate QEMU to utilize the same virtiofsd interface that
CLH uses, but let's fix this bug as a first step.
Fixes: #1755
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Now enabling enable_pprof for individual pods is supported,
but not documented.
This commit will add per-Pod Kata configurations for `enable_pprof`
in file `docs/how-to/how-to-set-sandbox-config-kata.md`
Fixes: #1744
Signed-off-by: bin <bin@hyper.sh>
Users can specify extra arguments for virtiofsd in a pod spec using the
io.katacontainers.config.hypervisor.virtio_fs_extra_args annontation.
However, this annotation was ignored so far by the runtime. This commit
fixes the issue by processing the annotation value (if present) and
translating it to the corresponding hypervisor configuration item.
Fixes#1523
Signed-off-by: Pavel Mores <pmores@redhat.com>
Parametr builtIn is not used in function updateRuntimeConfigAgent,
delete it from updateRuntimeConfigAgent and LoadConfiguration
function signature.
Fixes: #1731
Signed-off-by: bin <bin@hyper.sh>
It would be undesirable to be given an annotation like "/dev/null".
Filter out bad annotation values.
Fixes: #1043
Suggested-by: James O. D. Hunt <james.o.hunt@intel.com>
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
For k8s emptyDir volume, a specific fsGroup would
be set for it, thus guest should get this fsGroup
from runtime and set it properly on the EphemeralStorage
volume in guest.
Fixes: #1580
Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
For k8s emptyDir volume, a specific fsGroup would
be set for it, thus runtime should pass this fsGroup
for EphemeralStorage to guest and set it properly on
the emptyDir volume in guest.
Fixes: #1580
Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
Currently we implement the Default trait for NamespaceType. It doesn't
really make sense to have a default for this type though - you really need
to know what type of namespace you're setting. In fact the Default
implementation is never used, so we can just drop it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We had some code that initialized a Uevent to the default value, then set
specific fields to various values. This can be accomplished inside the one
initialized using the ..Default::default() syntax. Making this change
stops clippy from complaining.
fixes#1611
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We have one place where we create an empty vector then immediately push
something into it. We can do this in one step using the vec![] macro,
which stops clippy complaining.
fixes#1611
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The various type implementing the UeventMatcher trait have new() methods
which return a Result<>, however none of them can actually fail. This is
a leftover from their development where some versions could fail to
initialize. Remove the unneccessary wrappers to silence clippy.
fixes#1611
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently these are in all-caps, to match typical capitalization of IPC,
UTS and PID in the world at large. However, this violates Rust's
capitalization conventions and makes clippy complain.
fixes#1611
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Clippy (in Rust 1.51 at least) has some complaints about this closure
inside execute_hook() because it uses explicit returns in some places
where it doesn't need them, because they're the last expression in the
function.
That isn't necessarily obvious from a glance, but we can make clippy happy
and also make things a little clearer: first we replace a somewhat verbose
'match' using Option::ok_or_else(), then rearrange the remaining code to
put all the error path first with an explicit return then the "happy" path
as the stright line exit with an implicit return.
fixes#1611
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
PathBuf is an owned, mutable Path. We don't need those properties in
get_value_from_cgroup() so we can use a Path instead. This may be slightly
safer, and definitely stops clippy (version 1.51 at least) from
complaining.
fixes#1611
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
DEFAULT_ALLOWED_DEVICES and DEFAULT_DEVICES are essentially global
constant lists. They're implemented as a lazy_static! initialized Vec
values.
The code to initialize them creates an empty Vec then pushes values
onto it. We can simplify this a bit by using the vec! macro. This
might be slightly more efficient, and it definitely stops recent
clippy versions (e.g. 1.51) from complaining about it.
fixes#1611
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Recent versions of clippy (e.g. in Rust 1.51) complain about a number
of names in the oci crate, which don't obey Rust's normal CamelCasing
conventions.
It's pretty clear that these don't obey the usual rules because they
are attempting to preserve conventional casing of existing acronyms
they incorporate ("VM", "POSIX", etc.). However, it's been my
experience that matching the case and name conventions of your
environs is more important than matching case with external norms.
Therefore, this patch changes all the identifiers in the oci crate to
match Rust conventions. Their users in the rustjail crate are updated
to match.
fixes#1611
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This comment appears to be connected specifically with this function, but
has some other items separating it for no particular reason. It also has
a typo. Correct both.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Functions in rustjail deal with both the local oci module's data structure
and the protocol::oci module's data structure. Since these both cover the
OCI container config they are quite similar and have many identically named
types.
To avoid conflicts, we import many things from those modules with altered
names. However the names we use oci* and grpc* don't fit the normal Rust
capitalization convention for types.
However by renaming the import of the 'protocols::oci' module itself to
'grpc', we can actually get rid of the many renames by just qualifying at
each use site with only a very small increase in verbosity. As a bonus
this gets rid of multiple 'use' items scattered through the file.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Do not close the tty as part of the stdout redirection routine.
The close is already happening a couple lines below, after all routines
have finished.
Fixes#1713
Signed-off-by: Julien Ropé <jrope@redhat.com>
The situation is not a IPC scene, pipe(2) is too heavy.
We have tokio::sync:⌚:channel after tokio has been introduced.
The channel has better performance and easy to use.
Fixes: #1721
Signed-off-by: Tim Zhang <tim@hyper.sh>
Update:
- Make the type of errnoRet in oci.proto oneof
- Update seccomp_grpc_to_oci that can set errnoRet as EPREM if the
value is empty.
- Update the oci.pb.go based on the above fixes
- Add seccomp errnoRet and flags option to configs in rustjail
Fixes: #1719
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Remove the prohibition of vhost-user devices on s390x, which are by now
supported (e.g. vhost-user-fs-ccw). As a consequence,
appendVhostUserDevice no longer needs an error in its signature.
This enables virtio-fs support on s390x.
Fixes: #1469
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Update GoVMM to get memory backend support for non-DIMM setups. This is
necessary for virtio-fs on s390x.
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
register_memory_event_v2() includes a closure spawned as an async task
with tokio. At the end of that closure, there's a test for a closed fd
exiting if so. But this is right at the end of the closure when it was
about to exit anyway, so this does nothing.
This code was originally an explicit thread, converted to a tokio task
by 332fa4c "agent: switch to async runtime". It looks like there was an
error during conversion, where this logic was accidentally moved out of the
while loop above, where it makes a lot more sense.
Put it back into the loop.
fixes#1702
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Rather than relying on the system clock, use a channel timeout to avoid
problems if the system time changed.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Fixed logic used to handle static agent tracing.
For a standard (untraced) hypervisor shutdown, the runtime kills the VM
process once the workload has finished. But if static agent tracing is
enabled, the agent running inside the VM is responsible for the
shutdown. The existing code handled this scenario but did not wait for
the hypervisor process to end. The outcome of this being that the
console watcher thread was killed too early.
Although not a problem for an untraced system, if static agent tracing
was enabled, the logs from the hypervisor would be truncated, missing the
crucial final stages of the agents shutdown sequence.
The fix necessitated adding a new parameter to the `stopSandbox()` API,
which if true requests the runtime hypervisor logic simply to wait for
the hypervisor process to exit rather than killing it.
Fixes: #1696.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Previously, the hypervisors were sending a signal and then checking to
see if the process had died by sending the magic null signal (`0`). However,
that doesn't work as it was written: the logic was assuming sending the
null signal to a process that was dead would return `ESRCH`, but it
doesn't: you first need to you `wait(2)` for the process before sending
that signal. This means that previously, all affected hypervisors would
appear to take `timeout` seconds to end, even though they had _already_
finished.
Now, the hypervisors true end time will be seen as we wait for the
processes before sending the null signal to ensure the process has
finished.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Refactored some of the hypervisors to remove the duplicated code used to
trigger a shutdown.
Also added some unit tests.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Currently runtime and agent special case virtio-blk devices under clh,
ostensibly because the PCI address information is not available in that
case.
In fact, cloud-hypervisor's VmAddDiskPut API does return a PciDeviceInfo,
which includes a PCI address. That API is broken, because PCI addressing
depends on guest (firmware or OS) actions that the hypervisor won't know
about. clh only gets away with this because it only uses a single PCI root
and never uses PCI bridges, in which case the guest addresses are
accurately predictable: they always have domain and bus zero.
Until https://github.com/kata-containers/kata-containers/pull/1190, Kata
couldn't handle PCI addressing unless there was exactly one bridge, which
might be why this was actually special-cased for clh.
With #1190 merged, we can handle more general PCI paths, and we can derive
a trivial (one element) PCI path from the information that the clh API
gives us. We can use that to remove this special case.
fixes#1431
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Change the "moo FAILURE" message shown in a couple of the unit tests to
"moo message". This means that searching for unrelated failures in the
test output by looking for "FAIL" won't show these messages as false
positives any more.
fixes#1683
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
With tracing enabled, grpc health check generates a large number of
spans which creates too much data for tasks running longer than a few
minutes. To solve this, remove span creation from kata agent check() and
sendReq() where the majority of the spans come from. Leave contexts in
functions for subsequent calls that create spans.
Fixes#1395
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
The property name make newcomers confused when reading code.
Since in Kata Containers 2.0 there will only be one type of store,
so it's safe to replace it by `store` simply.
Fixes: #1660
Signed-off-by: bin <bin@hyper.sh>
DevAddrMatcher existed purely as a transitional step as we refined the
uevent matching logic for each of the different device types we care about.
We've now done that, so it can be removed along with several related
pieces.
fixes#1628
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Use the new uevent matching infrastructure to refine the matching for pmem
devices to something more pinned down to that device type. While we're
there, fix a few anciliary problems with get_pmem_device_name():
- The name is poor - the *input* to this function is the expected device
name, so the result isn't helpful, except that it needs to wait for the
device to be ready in the guest. Change it to wait_for_pmem_device() and
explicitly check that the returned device name matches the one expected.
- Remove an incorrect comment in nvdimm_storage_handler() (the only caller)
which appears to have been copied from the virtio-blk path, but then
become stale.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Current get_scsi_device_name() uses the legacy uevent matching which
isn't very precise. This refines it to use a specific matcher
implementation. While we're at it:
- No longer insist on the SCSI controller being under the PCI root.
It generally will be, but there's no particular reason to require
it.
The matcher still has a problem in that it won't work sensibly if
there are multiple SCSI busses in the guest. Fixing that requires
changes on the runtime side as well, though, so it's beyond scope for
this change.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
There are some problems with get_pci_device_name():
1) It's misnamed: in fact it is only used for handling virtio-blk PCI
devices. It's also only correct for virtio-blk devices, the event
matching doesn't locate the "raw" PCI device, but rather the block
device created by virtio-blk as a child of the PCI device itself.
2) The uevent matching is imprecise. As all things using the legacy
DevAddrMatcher, it matches on a bunch of conditions used across several
different device types, not all of which make sense for virtio-blk pci
devices specifically.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
first get the "HOME" env from "/etc/passwd", if
there's no corresponding uid entry in /etc/passwd,
then set "/" as the home env.
Fixes: #1643
Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
On commit 17e9a2cff5 it was introduced a guard for the case the mount point is already
mounted. Instead of log only the mount tag ("kataShared") with this change it will print
both tag and mount point path.
Fixes: #1398
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
get_device_name() contains logic to wait for a specific uevent, then
extract the /dev node name from it. In future we're going to want similar
logic to wait on uevents, but using different match criteria, or getting
different information out.
To simplify this, add a wait_for_uevent() helper in the uevent module,
which takes an explicit UeventMatcher object and returns the whole uevent
found.
To make testing easier, we also extract the cut down uevent watcher from
test_get_device_name() into a new spawn_test_watcher() helper. Its used
for both test_get_device_name() and a new test_wait_for_uevent() amd will
be useful for more tests in future.
fixes#1484
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
get_device_name() looks at kernel uevents to work out the device name for
a given PCI (usually) address. However, when we call it we can't know if
the uevent we're interested in has already happened (in which case it will
have been recorded in Sandbox::uevent_map) or yet to come, in which case
we need to register to watch it.
However, we currently match differently against past and future events.
For past events we simply look for a sysfs path including the address, but
for future events we use a complex bit of logic in the is_match() closure.
Change it to use the exact same matching logic in both cases.
fixes#1397
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently, Sandbox::uevent_watchers lists uevents to watch for by a
"device address" string. This is not very clearly defined, and is
matched against events with a rather complex closure created in
Uevent::process_add().
That closure makes a bunch of fragile assumptions about what sort of
events we could ever be interested in. In some ways it is too
restrictive (requires everything to be a block device), but in others
is not restrictive enough (allows things matching NVDIMM paths, even
if we're looking for a PCI block device).
To allow the clients more precise control over uevent matching, we
define a new UeventMatcher trait with a method to match uevents. We
then have the atchers list include UeventMatcher trait objects which
are used directly by Uevent::process_add(), instead of constructing
our match directly from dev_addr.
For now we don't actually change the matching function, or even use
multiple different trait implementations, but we'll refine that in
future.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The event matching logic in Uevent::process_add() is split into two parts.
The first checks if we care about the event at all, the second checks
whether the event is relevant to a particular watcher.
However, we're going to be adding more types of watchers in future, which
will make the global filter too restrictive. Fold the two bits of logic
together into a per-watcher filter function.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Uevent::process() is a bit oddly organized. It treats the onlining of
hotplugged memory as the "default" case, although that's quite specific,
while treating the handling of hotplugged block devices more like a special
case, although that's pretty close to being very general.
Furthermore splitting Uevent::is_block_add_event() from
Uevent::handle_block_add_event() doesn't make a lot of sense, since their
logic is intimately related to each other.
Alter the code to be a bit more sensible: first split on the "action" type
since that's the most fundamental difference, then handle the memory
onlining special case, then the block device add (which will become a lot
more general in future changes).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Sandbox:dev_watcher is a HashMap from a "device address" to a channel used
to notify get_device_name() that a suitable uevent has been found.
However, "device address" isn't well defined, having somewhat different
meanings for different device/event types. We never actually look up this
HashMap by key, except to remove entries.
Not looking up by key suggests that a map is not the appropriate data
structure here. Furthermore, HashMap imposes limitations on the types
which will prevent some future extensions we want.
So, replace the HashMap with a Vec<Option<>>. We need the Option<> so that
we can remove entries by index (removing them from the Vec completely would
hange the indices of other entries, possibly breaking concurrent work.
This does mean that the vector will keep growing as we watch for different
events during startup. However, we don't expect the number of device
events we watch for during a run to be very large, so that shouldn't be
a problem. We can optimize this later if it becomes a problem.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently, when Uevent::handle_block_add_event() receives an event matching
a registered watcher, it reports the /dev node name from the event back
to the watcher.
This changes it to report the entire uevent, not just the /dev node name.
This will allow various future extensions. It also makes the client side
of the uevent watching - get_device_name() - more consistent between its
two paths: finding a past uevent in Sandbox::uevent_map() or waiting for
a new uevent via a watcher.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Sandbox::pci_device_map contains a mapping from sysfs paths to /dev entries
which is used by get_device_name() to look up the right /dev node. But,
the map only supplies the answer if the uevent for the device has already
been received, otherwise get_device_name() has to wait for it.
However the matching for already-received and yet-to-come uevents isn't
quite the same which makes the whole system fragile.
In order to make sure the matching for both cases is identical, we need the
already-received side to store the whole uevent to match against, not just
the sysfs path and device name.
So, rename pci_device_map to uevent_map and store the whole uevent there
verbatim.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
In Kata 1.x, both the sysToDevMap and the deviceWatchers are in the sandbox
structure. For some reason in Kata 2.x, the device watchers have moved to
a separate global variable, GLOBAL_DEVICE_WATCHER.
This is a bad idea: apart from introducing an extra global variable
unnecessarily, it means that Sandbox::pci_device_map and
GLOBAL_DEVICE_WATCHER are protected by separate mutexes. Since the
information in these two structures has to be kept in sync with each other,
it makes much more sense to keep them both under the same single Sandbox
mutex.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
For the case of virtio-blk PCI devices, when matching uevents we create
a pci_p temporary. However, we build it incorrectly: the dev_addr values
we use for PCI devices are a relative sysfs paths from the PCI root to the
device in question *including an initial /*. But when we construct pci_p
we add an extra /, meaning the resulting path will *not* match properly.
AFAICT the only reason we got away with this is because in practice the
virtio-blk devices where discovered by the kernel before we looked for them
meaning the loosed matching in get_device_name() was used, rather than the
pci_p logic in handle_block_add_event().
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The current test_get_device_name(), ported from Kata 1.x doesn't really
reflect how the function is used in practice. The example path appears
to be for a virtio-blk device, but it's an s390 specific variant, not a
PCI device. The s390 form isn't actually supported by any of the existing
users of get_device_name().
Change it to a plausible virtio-blk-pci style path to better test how
get_device_name() will actually be used in practice.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The only right way to shutdown pipe is drop it
Otherwise PipeStream will conflict with its twins
Because they both have the same fd, and both registered.
Fixes: #1614
Signed-off-by: Tim Zhang <tim@hyper.sh>
Kata 1.x had a testcase for the equivalent getDeviceName function in Go,
this adapts it to Rust and adds it to Kata 2.x.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Rust 1.51 appears to have added a new warning in anticipation of Rust 2021,
which requires the format string for panic!()s (including via the various
assert!() macros) to be a string literal. This triggers quite a few times
in the agent code. This patch fixes them.
fixes#1626
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
On some setups, starting multiple kata pods (qemu) simultaneously on the same node
might cause kata VMs booting time to increase and the pods to fail with:
Failed to check if grpc server is working: rpc error: code = DeadlineExceeded desc = timed
out connecting to vsock 1358662990:1024: unknown
Increasing default dialing timeout to 30s should cover most cases.
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
Fixes: #1543
There are many requests to the agent that happen with relatively
high frequency when a workload is running (checkRequest, as an example).
Let's move from Debug to Trace to avoid bombarding journal.
Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
For k8s emptyDir volume, a specific fsGroup would
be set for it, thus guest should get this fsGroup
from runtime and set it properly on the emptyDir volume
in guest.
Fixes: #1580
Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
For k8s emptyDir volume, a specific fsGroup would
be set for it, thus runtime should pass this fsGroup
to guest and set it properly on the emptyDir volume
in guest.
Fixes: #1580
Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
Instead of having different VERSION files spread accross the project,
let's always use the one in the topsrcdir and remove all the others,
keeping only a synlink to the topsrcdir one.
Fixes: #1579
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
This patch extends the current process of generating client code for
cloud-hypervisor API with an additional step, `go-fmt`, which will remove
the generated `client/go.mod` file and format all auto-generated code.
Fixes: #1606
Signed-off-by: Bo Chen <chen.bo@intel.com>
Highlights for cloud-hypervisor version 0.14.0 include: 1) Structured
event monitoring; 2) MSHV improvements; 3) Improved aarch64 platform; 4)
Updated hotplug documentation; 6) PTY control for serial and
virtio-console; 7) Block device rate limiting; 8) Plan to deprecate the
support of "LinuxBoot" protocol and support PVH protocol only.
Highlights for cloud-hypervisor version 0.13.0 include: 1) Wider VFIO
device support; 2) Improve huge page support; 3) MACvTAP support; 4) VHD
disk image support; 5) Improved Virtio device threading; 6) Clean
shutdown support via synthetic power button.
Details can be found:
https://github.com/cloud-hypervisor/cloud-hypervisor/releases
Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by `openapi-generator` [1-2]. As the API changes do not
impact usages in Kata, no additional changes in kata's runtime are
needed to work with the latest version of cloud-hypervisor.
[1] https://github.com/OpenAPITools/openapi-generator
[2] https://github.com/kata-containers/kata-containers/blob/main/src/runtime/virtcontainers/pkg/cloud-hypervisor/README.mdFixes: #1591
Signed-off-by: Bo Chen <chen.bo@intel.com>
This patch upgrades Firecracker version from v0.21.1 to v0.23.1
* Generate swagger models for v0.23.1 (from firecracker.yaml)
* Change uint64 types in TokenBucket object according to rate-limiter
implementation (introduced in commit #cfeb966)
* Update Firecracker Logger/Metrics to support the new API
* Update payload in fc.vmRunning to support the new API
* Add Metrics type to fcConfig
Fixes: #1518
Signed-off-by: Orestis Lagkas Nikolos <olagkasn@nubificus.co.uk>
Shimv2 protocol CreateTaskRequest.Options has a type of *google_protobuf.Any.
If the call is from Docker, to decode the request,
the proto types(github.com/containerd/containerd/runtime/v2/runc/options)
should be imported.
Fixes: #1576
Signed-off-by: bin <bin@hyper.sh>
A wrong path was being used for container directory when
virtiofs is utilized. This resulted in a warning message in
logs when a container is killed, or completes:
level=warning msg="Could not remove container share dir"
Without proper removal, they'd later be cleaned up when the shared
path is removed as part of stopping the sandbox.
Fixes: #1559
Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
getExpectedHostDetails did not offload any work to
genericGetExpectedHostDetails on s390x. By using that function, much
redundant code can be saved. This also resolves 2 issues with the
previous version:
- The number of CPUs was not calculated.
- vcUtils.SupportsVsocks() still used the Kata v1 signature.
Fixes: #1564
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
#1389 has added a context for many signatures to improve trace spans.
Functions specific to s390x lack this. Add context where required. This
affects some common code signatures, since some functions that do not
require context on other architectures do require it on s390x.
Also remove an unnecessary import in test_qemu_s390x.go.
Fixes: #1562
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
statfs f_types are long on most architectures, but not on s390x, where
they are uint. Following the fix in rust-lang/libc at
https://github.com/rust-lang/libc/pull/1999, the custom defined
PROC_SUPER_MAGIC must be updated in a similar way.
Fixes: #1204
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
`make test` depends mock hook in virtcontainers directory,
before test, install it first.
And also run test as normal user and root in GitHub actions.
Fixes: #1554
Signed-off-by: bin <bin@hyper.sh>
Move the signal handling code into a new module and refactor into the
main handler and a new SIGCHLD handling function to make the code
simpler and easier to understand.
Also added a unit test for shutdown.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Make changes to logger thread to allow the logger to be replaced with
a NOP logger (required for agent shutdown).
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The agent doesn't normally shutdown: it doesn't need to be as it is
killed *after* the workload has finished. However, a clean and ordered
shutdown sequence is required to support agent tracing, since all trace
spans need to be completed to ensure a valid trace transaction.
Enable a controlled shutdown by allowing the main threads (tasks) to be
stopped.
To allow this to happen, each thread is now passed a shutdown channel
which it must listen to asynchronously, and shut down the thread if
activity is detected on that channel.
Since some threads are created for I/O and since the standard `io::copy`
cannot be stopped, added a new `interruptable_io_copier()` function
which shares the same semantics as `io::copy()`, but which is also
passed a shutdown channel to allow asynchronous I/O operations to be
stopped cleanly.
Fixes: #1531.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Encapsulate the logic for handling the task that displays logger output
into a new function to simplify the code and remove another anonymous
async block.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Maintain a list of tasks and wait on them all before main returns.
This is preparatory work for the agent shutdown: all tasks that are
started need to be added to the list. This aggregation makes it easier
to identify what needs to stop before the agent can exit cleanly.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Move most of the main logic into a separate async function. This makes
the code clearer and avoids the anonymous async block.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Return a guard variable from `create_logger()` which the caller can
implicitly drop to guarantee that all threads started by the async log
drain are stopped.
This fixes a long-standing bug [1] whereby the agent could panic with
the following error, generated by the `slog` logging crate:
```
slog::Fuse Drain: Custom { kind: Other, error: "serde serialization error: Bad file descriptor (os error 9)" }
```
[1] - See https://github.com/kata-containers/kata-containers/issues/171.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Fixed the `assert_error!()` test macro so that it correctly handles the
scenario where the test expects an error, but the actual result was `Ok`
(no error).
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
