Commit Graph

8779 Commits

Author SHA1 Message Date
Tim Zhang
4be7185aa4 runtime-rs: runtime part implement
Fixes: #3785
Signed-off-by: Tim Zhang <tim@hyper.sh>
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
2022-06-10 22:01:12 +08:00
Zhongtao Hu
10343b1f3d runtime-rs: enhance runtimes
1. support oom event
2. use ContainerProcess to store container_id and exec_id
3. support stats

Fixes: #3785
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 22:01:05 +08:00
Quanwei Zhou
9887272db9 libs: enhance kata-sys-util and kata-types
Fixes: #3785
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
2022-06-10 21:59:47 +08:00
Quanwei Zhou
3ff0db05a7 runtime-rs: support rootfs volume for resource
Fixes: #3785
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
2022-06-10 19:58:01 +08:00
Tim Zhang
234d7bca04 runtime-rs: support cgroup resource
Fixes: #3785
Signed-off-by: Tim Zhang <tim@hyper.sh>
2022-06-10 19:57:53 +08:00
Quanwei Zhou
75e282b4c1 runtime-rs: hypervisor base define
Responsible for VM manager, such as Qemu, Dragonball

Fixes: #3785
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
2022-06-10 19:57:45 +08:00
Quanwei Zhou
bdfee005fa runtime-rs: service and runtime framework
1. service: Responsible for processing services, such as task service, image service
2. Responsible for implementing different runtimes, such as Virt-container,
Linux-container, Wasm-container

Fixes: #3785
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
2022-06-10 19:57:36 +08:00
Quanwei Zhou
4296e3069f runtime-rs: agent implements
Responsible for communicating with the agent, such as kata-agent in the VM

Fixes: #3785
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
2022-06-10 19:57:29 +08:00
Jakob Naucke
d3da156eea runtime-rs: uint FsType for s390x
statfs type on s390x should be c_uint, not __fsword_t

Fixes: #3888
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2022-06-10 19:57:23 +08:00
quanwei.zqw
e705ee07c5 runtime-rs: update containerd-shim-protos to 0.2.0
Fixes: #3866
Signed-off-by: quanwei.zqw <quanwei.zqw@alibaba-inc.com>
2022-06-10 19:57:14 +08:00
quanwei.zqw
8c0a60e191 runtime-rs: modify the review suggestion
Fixes: #3876
Signed-off-by: quanwei.zqw <quanwei.zqw@alibaba-inc.com>
2022-06-10 19:57:07 +08:00
Zack
278f843f92 runtime-rs: shim implements for runtime-rs
Responsible for processing shim related commands: start, delete.

This patch is extracted from Alibaba Cloud's internal repository *runD*
Thanks to all contributors!

Fixes: #3785
Signed-off-by: acetang <aceapril@126.com>
Signed-off-by: Bin Liu <bin@hyper.sh>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
Signed-off-by: Fupan Li <lifupan@gmail.com>
Signed-off-by: gexuyang <gexuyang@linux.alibaba.com>
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
Signed-off-by: He Rongguang <herongguang@linux.alibaba.com>
Signed-off-by: Hui Zhu <teawater@gmail.com>
Signed-off-by: Issac Hai <hjwissac@linux.alibaba.com>
Signed-off-by: Jiahuan Chao <jhchao@linux.alibaba.com>
Signed-off-by: lichenglong9 <lichenglong9@163.com>
Signed-off-by: mengze <mengze@linux.alibaba.com>
Signed-off-by: Qingyuan Hou <qingyuan.hou@linux.alibaba.com>
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
Signed-off-by: shiqiangzhang <shiyu.zsq@linux.alibaba.com>
Signed-off-by: Simon Guo <wei.guo.simon@linux.alibaba.com>
Signed-off-by: Tim Zhang <tim@hyper.sh>
Signed-off-by: wanglei01 <wllenyj@linux.alibaba.com>
Signed-off-by: Wei Yang <wei.yang1@linux.alibaba.com>
Signed-off-by: yanlei <yl.on.the.way@gmail.com>
Signed-off-by: Yiqun Leng <yqleng@linux.alibaba.com>
Signed-off-by: yuchang.xu <yuchang.xu@linux.alibaba.com>
Signed-off-by: Yves Chan <lingfu@linux.alibaba.com>
Signed-off-by: Zack <zmlcc@linux.alibaba.com>
Signed-off-by: Zhiheng Tao <zhihengtao@linux.alibaba.com>
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Signed-off-by: Zizheng Bian <zizheng.bian@linux.alibaba.com>
2022-06-10 19:56:59 +08:00
Quanwei Zhou
641b736106 libs: enhance kata-sys-util
1. move verify_cid from agent to libs/kata-sys-util
2. enhance kata-sys-util/k8s

Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
2022-06-10 19:55:39 +08:00
Fupan Li
69ba1ae9e4 trans: fix the issue of wrong swapness type
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
2022-06-10 19:46:25 +08:00
Quanwei Zhou
d2a9bc6674 agent: agent-protocol support async
1. support async.
2. update ttrpc and protobuf
update ttrpc to 0.6.0
update protobuf to 2.23.0
3. support trans from oci

Fixes: #3746
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
2022-06-10 19:36:55 +08:00
Liu Jiang
aee9633ced libs/sys-util: provide functions to execute hooks
Provide functions to execute OCI hooks.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Bin Liu <bin@hyper.sh>
Signed-off-by: Huamin Tang <huamin.thm@alibaba-inc.com>
Signed-off-by: Lei Wang <wllenyj@linux.alibaba.com>
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
2022-06-10 19:24:30 +08:00
Liu Jiang
8509de0aea libs/sys-util: add function to detect and update K8s emptyDir volume
Add function to detect and update K8s emptyDir volume.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Qingyuan Hou <qingyuan.hou@linux.alibaba.com>
2022-06-10 19:15:59 +08:00
Liu Jiang
6d59e8e197 libs/sys-util: introduce function to get device id
Introduce get_devid() to get major/minor number of a block device.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
2022-06-10 19:15:28 +08:00
Liu Jiang
5300ea23ad libs/sys-util: implement reflink_copy()
Implement reflink_copy() to copy file by reflink, and fallback to normal
file copy.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
2022-06-10 19:15:20 +08:00
Liu Jiang
1d5c898d7f libs/sys-util: add utilities to parse NUMA information
Add utilities to parse NUMA information.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Qingyuan Hou <qingyuan.hou@linux.alibaba.com>
Signed-off-by: Simon Guo <wei.guo.simon@linux.alibaba.com>
2022-06-10 19:15:12 +08:00
Liu Jiang
87887026f6 libs/sys-util: add utilities to manipulate cgroup
Add utilities to manipulate cgroup, currently only v1 is supported.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: He Rongguang <herongguang@linux.alibaba.com>
Signed-off-by: Jiahuan Chao <jhchao@linux.alibaba.com>
Signed-off-by: Qingyuan Hou <qingyuan.hou@linux.alibaba.com>
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
Signed-off-by: Tim Zhang <tim@hyper.sh>
2022-06-10 19:14:59 +08:00
Liu Jiang
ccd03e2cae libs/sys-util: add wrappers for mount and fs
Add some wrappers for mount and fs syscall.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Bin Liu <bin@hyper.sh>
Signed-off-by: Fupan Li <lifupan@gmail.com>
Signed-off-by: Huamin Tang <huamin.thm@alibaba-inc.com>
Signed-off-by: Lei Wang <wllenyj@linux.alibaba.com>
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
2022-06-10 19:14:06 +08:00
Liu Jiang
45a00b4f02 libs/sys-util: add kata-sys-util crate under src/libs
The kata-sys-util crate is a collection of modules that provides helpers
and utilities used by multiple Kata Containers components.

Fixes: #3305

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 19:10:40 +08:00
Zhongtao Hu
48c201a1ac libs/types: make the variable name easier to understand
1. modify default values for hypervisor
2. change the variable name
3. check the min memory limit

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:01:31 +08:00
Zhongtao Hu
b9b6d70aae libs/types: modify implementation details
1. fix nit problems
2. use generic type when parsing different type

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:01:24 +08:00
Zhongtao Hu
05ad026fc0 libs/types: fix implementation details
use ok_or_else to handle get_mut(hypervisor) to substitue unwrap

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:01:17 +08:00
Zhongtao Hu
d96716b4d2 libs/types:fix styles and implementation details
1. Some Nit problems are fixed
2. Make the code more readable
3. Modify some implementation details

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:01:09 +08:00
Zhongtao Hu
6cffd943be libs/types:return Result to handle parse error
If there is a parse error when we are trying to get the annotations, we
will return Result<Option<type>> to handle that.

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:00:58 +08:00
Zhongtao Hu
6ae87d9d66 libs/types: use contains to make code more readable
use contains to when validate hypervisor block_device_driver

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:00:50 +08:00
Zhongtao Hu
45e5780e7c libs/types: fixed spelling and grammer error
fixed spelling and grammer error in some files

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:00:43 +08:00
Zhongtao Hu
2599a06a56 libs/types:use include_str! in test file
use include_str! to load toml file to string fmt

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 18:28:14 +08:00
Zhongtao Hu
8ffff40af4 libs/types:Option type to handle empty tomlconfig
loading from empty string is only used to identity that the config is
not initialized yet, so Option<TomlConfig> is a better option

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 18:28:05 +08:00
Zhongtao Hu
626828696d libs/types: add license for test-config.rs
add SPDX license identifier: Apache-2.0

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 18:27:57 +08:00
Zhongtao Hu
97d8c6c0fa docs: modify move-issues-to-in-progress.yaml
change issue backlog to runtime-rs

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 18:27:49 +08:00
Liu Jiang
8cdd70f6c2 libs/types: change method to update config by annotation
Some annotations are used to override hypervisor configurations, and you
know it's dangerous. We must be careful when overriding hypervisor configuration
by annotations, to avoid security flaws.
There are two existing mechanisms to prevent attacks by annotations:
1) config.hypervisor.enable_annotations defines the allowed annotation
keys for config.hypervisor.
2) config.hyperisor.xxxx_paths defines allowd values for specific keys.

The access methods for config.hypervisor.xxx enforces the permisstion
checks for above rules.

To update conifg, traverse the annotation hashmap,check if the key is enabled in hypervisor or not.
If it is enabled. For path related annotation, check whether it is valid or not
before updating conifg. For cpu and memory related annotation, check whether it
is more than or less than the limitation for DB and qemu beforing updating config.

If it is not enabled, there will be three possibilities, agent related
annotation, runtime related annotation and hypervisor related annotation
but not enabled. The function will handle agent and runtime annotation
first, then the option left will be the invlaid hypervisor, err message
will be returned.

add more edge cases tests for updating config

clean up unused functions, delete unused files and fix warnings

Fixes: #3523

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:27:36 +08:00
Liu Jiang
e19d04719f libs/types: implement KataConfig to wrap TomlConfig
The TomlConfig structure is a parsed form of Kata configuration file,
but it's a little inconveneient to access those configuration
information directly. So introduce a wrapper KataConfig to easily
access those configuration information.

Two singletons of KataConfig is provided:
- KATA_DEFAULT_CONFIG: the original version directly loaded from Kata
configuration file.
- KATA_ACTIVE_CONFIG: the active version is the KATA_DEFAULT_CONFIG
patched by annotations.

So the recommended to way to use these two singletons:
- Load TomlConfig from configuration file and set it as the default one.
- Clone the default one and patch it with values from annotations.
- Use the default one for permission checks, such as to check for
  allowed annotation keys/values.
- The patched version may be set as the active one or passed to clients.
- The clients directly accesses information from the active/passed one,
  and do not need to check annotation for override.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:26:48 +08:00
Liu Jiang
387ffa914e libs/types: support load Kata agent configuration from file
Add structures to load Kata agent configuration from configuration files.
Also define a mechanism for vendor to extend the Kata configuration
structure.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:26:37 +08:00
Liu Jiang
69f10afb71 libs/types: support load Kata hypervisor configuration from file
Add structures to load Kata hypevisor configuration from configuration
files. Also define a mechanisms to:
1) for hypervisors to handle the configuration info.
2) for vendor to extend the Kata configuration structure.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
21cc02d724 libs/types: support load Kata runtime configuration from file
Add structures to load Kata runtime configuration from configuration
files. Also define a mechanism for vendor to extend the Kata
configuration structure.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
5b89c1df2f libs/types: add kata-types crate under src/libs
Add kata-types crate to host constants and data types shared by multiple
Kata Containers components.

Fixes: #3305

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Fupan Li <lifupan@gmail.com>
Signed-off-by: Huamin Tang <huamin.thm@alibaba-inc.com>
Signed-off-by: Lei Wang <wllenyj@linux.alibaba.com>
Signed-off-by: yanlei <yl.on.the.way@gmail.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
4f62a7618c libs/logging: fix clippy warnings
Fix clippy warnings of libs/logging.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
6f8acb94c2 libs: refine Makefile rules
Refine Makefile rules to better support the KATA ci env.

Fixes: #3536

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
7cdee4980c libs/logging: introduce a wrapper writer for logging
Introduce a wrapper writer `LogWriter` which converts every line written
to it into a log record.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Wei Yang <wei.yang1@linux.alibaba.com>
Signed-off-by: yanlei <yl.on.the.way@gmail.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
426f38de94 libs/logging: implement rotator for log files
Add FileRotator to rotate log files.

The FileRotator structure may be used as writer for create_logger()
and limits the storage space occupied by log files.

Fixes: #3304

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Wei Yang <wei.yang1@linux.alibaba.com>
Signed-off-by: yanlei <yl.on.the.way@gmail.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
392f1ecdf5 libs: convert to a cargo workspace
Convert libs into a Cargo workspace, so all libraries could share the
build infrastructure.

Fixes #3282

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
575df4dc4d static-checks: Allow Merge commit to be >75 chars
Some generated merge commit messages are >75 chars
Allow these to not trigger the subject line length failure

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-06-10 18:25:24 +08:00
Bin Liu
ae911d0cd3
Merge pull request #4378 from cmaf/update-containerd-docs-critools
docs: Update source for cri-tools
2022-06-09 15:12:37 +08:00
Bin Liu
05022975c8
Merge pull request #4413 from jodh-intel/tools-full-err-output
tools: Enable extra detail on error
2022-06-09 13:52:08 +08:00
Chelsea Mafrica
aaa74e8a2b
Merge pull request #4415 from jodh-intel/agent-ctl-doc-examples
docs: Add agent-ctl examples section
2022-06-08 09:51:30 -07:00
snir911
a57515bdae
Merge pull request #4384 from snir911/2.5.0-alpha2-branch-bump
# Kata Containers 2.5.0-alpha2
2022-06-08 19:32:57 +03:00