Commit Graph

8757 Commits

Author SHA1 Message Date
Liu Jiang
45a00b4f02 libs/sys-util: add kata-sys-util crate under src/libs
The kata-sys-util crate is a collection of modules that provides helpers
and utilities used by multiple Kata Containers components.

Fixes: #3305

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 19:10:40 +08:00
Zhongtao Hu
48c201a1ac libs/types: make the variable name easier to understand
1. modify default values for hypervisor
2. change the variable name
3. check the min memory limit

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:01:31 +08:00
Zhongtao Hu
b9b6d70aae libs/types: modify implementation details
1. fix nit problems
2. use generic type when parsing different type

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:01:24 +08:00
Zhongtao Hu
05ad026fc0 libs/types: fix implementation details
use ok_or_else to handle get_mut(hypervisor) to substitue unwrap

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:01:17 +08:00
Zhongtao Hu
d96716b4d2 libs/types:fix styles and implementation details
1. Some Nit problems are fixed
2. Make the code more readable
3. Modify some implementation details

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:01:09 +08:00
Zhongtao Hu
6cffd943be libs/types:return Result to handle parse error
If there is a parse error when we are trying to get the annotations, we
will return Result<Option<type>> to handle that.

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:00:58 +08:00
Zhongtao Hu
6ae87d9d66 libs/types: use contains to make code more readable
use contains to when validate hypervisor block_device_driver

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:00:50 +08:00
Zhongtao Hu
45e5780e7c libs/types: fixed spelling and grammer error
fixed spelling and grammer error in some files

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 19:00:43 +08:00
Zhongtao Hu
2599a06a56 libs/types:use include_str! in test file
use include_str! to load toml file to string fmt

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 18:28:14 +08:00
Zhongtao Hu
8ffff40af4 libs/types:Option type to handle empty tomlconfig
loading from empty string is only used to identity that the config is
not initialized yet, so Option<TomlConfig> is a better option

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 18:28:05 +08:00
Zhongtao Hu
626828696d libs/types: add license for test-config.rs
add SPDX license identifier: Apache-2.0

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 18:27:57 +08:00
Zhongtao Hu
97d8c6c0fa docs: modify move-issues-to-in-progress.yaml
change issue backlog to runtime-rs

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 18:27:49 +08:00
Liu Jiang
8cdd70f6c2 libs/types: change method to update config by annotation
Some annotations are used to override hypervisor configurations, and you
know it's dangerous. We must be careful when overriding hypervisor configuration
by annotations, to avoid security flaws.
There are two existing mechanisms to prevent attacks by annotations:
1) config.hypervisor.enable_annotations defines the allowed annotation
keys for config.hypervisor.
2) config.hyperisor.xxxx_paths defines allowd values for specific keys.

The access methods for config.hypervisor.xxx enforces the permisstion
checks for above rules.

To update conifg, traverse the annotation hashmap,check if the key is enabled in hypervisor or not.
If it is enabled. For path related annotation, check whether it is valid or not
before updating conifg. For cpu and memory related annotation, check whether it
is more than or less than the limitation for DB and qemu beforing updating config.

If it is not enabled, there will be three possibilities, agent related
annotation, runtime related annotation and hypervisor related annotation
but not enabled. The function will handle agent and runtime annotation
first, then the option left will be the invlaid hypervisor, err message
will be returned.

add more edge cases tests for updating config

clean up unused functions, delete unused files and fix warnings

Fixes: #3523

Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:27:36 +08:00
Liu Jiang
e19d04719f libs/types: implement KataConfig to wrap TomlConfig
The TomlConfig structure is a parsed form of Kata configuration file,
but it's a little inconveneient to access those configuration
information directly. So introduce a wrapper KataConfig to easily
access those configuration information.

Two singletons of KataConfig is provided:
- KATA_DEFAULT_CONFIG: the original version directly loaded from Kata
configuration file.
- KATA_ACTIVE_CONFIG: the active version is the KATA_DEFAULT_CONFIG
patched by annotations.

So the recommended to way to use these two singletons:
- Load TomlConfig from configuration file and set it as the default one.
- Clone the default one and patch it with values from annotations.
- Use the default one for permission checks, such as to check for
  allowed annotation keys/values.
- The patched version may be set as the active one or passed to clients.
- The clients directly accesses information from the active/passed one,
  and do not need to check annotation for override.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:26:48 +08:00
Liu Jiang
387ffa914e libs/types: support load Kata agent configuration from file
Add structures to load Kata agent configuration from configuration files.
Also define a mechanism for vendor to extend the Kata configuration
structure.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:26:37 +08:00
Liu Jiang
69f10afb71 libs/types: support load Kata hypervisor configuration from file
Add structures to load Kata hypevisor configuration from configuration
files. Also define a mechanisms to:
1) for hypervisors to handle the configuration info.
2) for vendor to extend the Kata configuration structure.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
21cc02d724 libs/types: support load Kata runtime configuration from file
Add structures to load Kata runtime configuration from configuration
files. Also define a mechanism for vendor to extend the Kata
configuration structure.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
5b89c1df2f libs/types: add kata-types crate under src/libs
Add kata-types crate to host constants and data types shared by multiple
Kata Containers components.

Fixes: #3305

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Fupan Li <lifupan@gmail.com>
Signed-off-by: Huamin Tang <huamin.thm@alibaba-inc.com>
Signed-off-by: Lei Wang <wllenyj@linux.alibaba.com>
Signed-off-by: yanlei <yl.on.the.way@gmail.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
4f62a7618c libs/logging: fix clippy warnings
Fix clippy warnings of libs/logging.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
6f8acb94c2 libs: refine Makefile rules
Refine Makefile rules to better support the KATA ci env.

Fixes: #3536

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
7cdee4980c libs/logging: introduce a wrapper writer for logging
Introduce a wrapper writer `LogWriter` which converts every line written
to it into a log record.

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Wei Yang <wei.yang1@linux.alibaba.com>
Signed-off-by: yanlei <yl.on.the.way@gmail.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
426f38de94 libs/logging: implement rotator for log files
Add FileRotator to rotate log files.

The FileRotator structure may be used as writer for create_logger()
and limits the storage space occupied by log files.

Fixes: #3304

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Wei Yang <wei.yang1@linux.alibaba.com>
Signed-off-by: yanlei <yl.on.the.way@gmail.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
392f1ecdf5 libs: convert to a cargo workspace
Convert libs into a Cargo workspace, so all libraries could share the
build infrastructure.

Fixes #3282

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
2022-06-10 18:25:24 +08:00
Liu Jiang
575df4dc4d static-checks: Allow Merge commit to be >75 chars
Some generated merge commit messages are >75 chars
Allow these to not trigger the subject line length failure

Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2022-06-10 18:25:24 +08:00
Bin Liu
ae911d0cd3
Merge pull request #4378 from cmaf/update-containerd-docs-critools
docs: Update source for cri-tools
2022-06-09 15:12:37 +08:00
Bin Liu
05022975c8
Merge pull request #4413 from jodh-intel/tools-full-err-output
tools: Enable extra detail on error
2022-06-09 13:52:08 +08:00
Chelsea Mafrica
aaa74e8a2b
Merge pull request #4415 from jodh-intel/agent-ctl-doc-examples
docs: Add agent-ctl examples section
2022-06-08 09:51:30 -07:00
snir911
a57515bdae
Merge pull request #4384 from snir911/2.5.0-alpha2-branch-bump
# Kata Containers 2.5.0-alpha2
2022-06-08 19:32:57 +03:00
Eric Ernst
4ebf9d38b9
Merge pull request #4310 from egernst/core-sched
shim: add support for core scheduling
2022-06-08 17:42:45 +02:00
Snir Sheriber
eb24e97150 release: Kata Containers 2.5.0-alpha2
- docs: Update storage documentation link
- rustjail: get home dir using nix crate
- runk: Support `list` sub-command
- docs: Update vGPU use-case
- runtime: ignore ESRCH error from stop container
- docs: Update configuration reference for snap documentation
- workflows: add workflow_dispatch triggering to test-kata-deploy
- snap: Use helper script and cleanup
- feature: add ability to interact with IPTables within the guest
- agent: return mount file content if parse mountinfo failed
- docs: Update Intel QAT documentation links
- osbuilder: add iptables package
- runk: Return error when tty is used without console socket
- runk: Add Podman guide in README
- agent: Pass standard I/O to container launched by runk
- agent, runk: Enable test for the agent built with standard-oci-runtime feature
- runk: Handle rootfs path in config.json properly
- Update containerd docs
- clh: Update to v24.0
- snap: Build and package rust version of virtiofsd
- runk: merge oci-kata-agent into runk
- virtiofsd: static build virtiofsd from rust code for non-x86
- Fix issues with direct-volume stats feature
- runtime: fix incorrect Action function for direct-volume stats
- runtime: Adding the correct detection of mediated PCIe devices
- runtime: remove duplicate 'types' import
- runtime: sync docstrings with function names
- qemu: allow using legacy serial device for the console
- docs: Remove clear containers reference in README
- runtime: do not check for EOF error in console watcher
- kernel: Remove nemu.conf from packaging
- tools: delete unused param from get_from_kata_deps callers
- agent: Fix is_signal_handled failing parsing str to u64
- Improve Go unit test script
- packaging: Add kernel config option for SGX in Gramine
- ci: Don't run Docs URL Alive Check workflow on forks
- tools: Add QEMU patches for SGX numa support
- docs: Update runc containerd runtime
- Build and distribute the rust version of virtiofsd
- doc: Update log parser link
- Move the kata-log-parser from the tests repo
- versions: Upgrade to Cloud Hypervisor v23.1
- agent: Add a macro to skip a loop easier
- runk: use custom Kill command to support --all option
- agent: add test coverage for functions find_process and online_resources

fe3c1d9cd docs: Update storage documentation link
9d27c1fce agent: ignore ESRCH error when destroying containers
9726f56fd runtime: force stop container after the container process exits
168f325c4 docs: Update configuration reference for snap documentation
38a318820 runk: Support `list` sub-command
b9fc24ff3 docs: update release process github token instructions
c1476a174 docs: update release process with latest workflow triggering
002f2cd10 snap: Use helper script and cleanup
2e04833fb docs: Update Intel QAT documentation links
8b57bf97a workflows: add workflow_dispatch triggering to test-kata-deploy
6d0ff901a docs: Update vGPU use-case
9b108d993 docs: Improve snap formatting
894f661cc docs: Add warning to snap build
d759f6c3e snap: Fix CH architecture check
590381574 agent: Pass standard I/O to container launched by runk
af2ef3f7a agent-ctl: introduce handle for iptables get/set
65f0cef16 kata-runtime: add iptables CLI to test http endpoint
3201ad083 shim-client: ensure we check resp status for Put/Post
0706fb28a kata-runtime: shmgmt: make url usage consistent
2a09378dd shim-client: add support for DoPut
640173cfc shim-mgmt: Add endpoint handler for interacting with iptables
0136be22c virtcontainers: plumb iptable set/get from sandbox to agent
bd50d463b agent: iptables: get/set handling for iptables
7c4049aab osbuilder: add iptables package
03176a9e0 proto: update generated code based on proto update
38ebbc705 proto: update to add set/get iptables
78d45b434 agent: return mount file content if parse mountinfo failed
c7b3941c9 runk: Enable test for the agent built with standard-oci-runtime feature
6dbce7c3d agent: Remove unused import in console test
6ecea84bc rustjail: get home dir using nix crate
648b8d0ae runk: Return error when tty is used without console socket
5205efd9b runk: Add Podman guide in README
d862ca059 runk: Handle rootfs path in config.json properly
56591804b docs: Improve snap build instructions
cb2b30970 snap: Build using destructive mode
60823abb9 docs: Move snap README
fff832874 clh: Update to v24.0
49361749e snap: Build and package rust version of virtiofsd
27d903b76 snap: Put the yq binary in the staging bin directory
d7b4ce049 snap: Remove unused variable
43de5440e snap: Fix unbound variable error
c9b291509 snap: Fix whitespace
122a85e22 agent: remove bin oci-kata-agent
35619b45a runk: merge oci-kata-agent into runk
10c13d719 qemu: remove virtiofsd option in qemu config
d20bc5a4d virtiofsd: build rust based virtiofsd from source for non-x86_64
c95ba63c0 docs: Remove information related to Kata 1.x
34b80382b docs: Get rid of note related to networking.
dfad5728a docs: Mention --cni flag while invoking ctr
8e7c5975c agent: fix direct-assigned volume stats
4428ceae1 runtime: direct-volume stats use correct name
ffdc065b4 runtime: direct-volume stats update to use GET parameter
f29595318 runtime: fix incorrect Action function for direct-volume stats
7a5ccd126 runtime: sync docstrings with function names
ce2e521a0 runtime: remove duplicate 'types' import
834f93ce8 docs: fix annotations example
f4994e486 runtime: allow annotation configuration to use_legacy_serial
24a2b0f6a docs: Remove clear containers reference in README
abad33eba kernel: Remove nemu.conf from packaging
e87eb13c4 tools: delete unused param from get_from_kata_deps callers
8052fe62f runtime: do not check for EOF error in console watcher
c67b9d297 qemu: allow using legacy serial device for the console
44814dce1 qemu: treat console kernel params within appendConsole
4f586d2a9 packaging: Add kernel config option for SGX in Gramine
4b437d91f agent: Fix is_signal_handled failing parsing str to u64
88fb9b72e docs: Update runc containerd runtime
d1f2852d8 tools: Stop building virtiofsd with qemu (for x86_64)
c39852e83 runtime: Use ${LIBEXEC}/virtiofsd as the default virtiofsd path
b4b9068cb tools: Add QEMU patches for SGX numa support
a475956ab workflows: Add support for building virtiofsd
71f59f3a7 local-build: Add support for building virtiofsd
c7ac55b6d dockerbuild: Install unzip
8e2042d05 tools: add script to pull virtiofsd
dbedea508 versions: Add virtiofsd entry
e73b70baf runtime: Don't run unit tests verbose by default
f24a6e761 runtime: Consolidate flags setting in unit tests script
cf465feb0 runtime: Don't change test behaviour based on $CI or $KATA_DEV_MODE
34c4ac599 runtime: Remove redundant subcommands from go-test.sh
0aff5aaa3 runtime: Simplify package listing in go-test.sh
557c4cfd0 runtime: Don't chmod coverage files in Go tests
04c8b52e0 runtime: Remove HTML coverage option from go-test.sh
7f7691442 runtime: Add coverage.txt.tmp to gitignore
13c257700 runtime: Move go testing script locally
421064680 doc: Update log parser link
271933fec log-parser: fix some of the documentation
c7dacb121 log-parser: move the kata-log-parser from the tests repo
82ea01828 versions: Upgrade to Cloud Hypervisor v23.1
2a1d39414 runtime: Adding the correct detection of mediated PCIe devices
7bc4ab68c ci: Don't run Docs URL Alive Check workflow on forks
475e3bf38 agent: add test coverage for functions find_process and online_resources
383be2203 agent: Add a macro to skip a loop easier
97d7b1845 runk: use custom Kill command to support --all option

Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2022-06-08 11:56:30 +03:00
James O. D. Hunt
34bcef8846 docs: Add agent-ctl examples section
Add a new `Examples` section to the `agent-ctl` docs giving some
examples of how to use the tool with QEMU and stand-alone.

Fixes: #4414.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-06-08 08:39:38 +01:00
James O. D. Hunt
815157bf02 docs: Remove erroneous whitespace
Deleted an extra blank line.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-06-08 08:39:38 +01:00
GabyCT
5bd81ba232
Merge pull request #4399 from GabyCT/topic/updatestoragedoc
docs: Update storage documentation link
2022-06-07 09:13:45 -05:00
James O. D. Hunt
f5099620f1 tools: Enable extra detail on error
The `agent-ctl` and `trace-forwarder` tools make use of
`anyhow::Context` to provide additional call site information on error.

However, previously neither tool was using the "alternate debug" format
to display the error, meaning full error output was not displayed.

Fixes: #4411.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-06-07 14:00:29 +01:00
Gabriela Cervantes
fe3c1d9cdd docs: Update storage documentation link
This PR updates the storage documentation link for the devicemapper
snapshotter.

Fixes #4398

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-06-06 14:48:34 +00:00
Bin Liu
a238d8c6bd
Merge pull request #4300 from justxuewei/fix/rustjail/home-env
rustjail: get home dir using nix crate
2022-06-06 11:03:46 +08:00
Bin Liu
f981190621
Merge pull request #4383 from cyyzero/runk-list
runk: Support `list` sub-command
2022-06-06 10:25:33 +08:00
Bin Liu
f7b22eb777
Merge pull request #4344 from zvonkok/vgpu-documentation
docs: Update vGPU use-case
2022-06-06 10:25:05 +08:00
Eric Ernst
430da47215
Merge pull request #4360 from fengwang666/shim-leak
runtime: ignore ESRCH error from stop container
2022-06-02 12:42:19 -07:00
GabyCT
9c9e5984ba
Merge pull request #4342 from GabyCT/topic/updatesnapdoc
docs: Update configuration reference for snap documentation
2022-06-02 14:00:22 -05:00
Feng Wang
9d27c1fced agent: ignore ESRCH error when destroying containers
destroy() method should ignore the ESRCH error from signal::kill
and continue the operation as ESRCH is often considered harmless.

Fixes: #4359

Signed-off-by: Feng Wang <feng.wang@databricks.com>
2022-06-02 08:19:48 -07:00
Feng Wang
9726f56fdc runtime: force stop container after the container process exits
Set thestop container force flag to true so that the container state is always set to
“StateStopped” after the container wait goroutine is finished. This is necessary for
the following delete container step to succeed.

Fixes: #4359

Signed-off-by: Feng Wang <feng.wang@databricks.com>
2022-06-02 08:17:08 -07:00
Gabriela Cervantes
168f325c43 docs: Update configuration reference for snap documentation
This PR updates the url link for the kata containers configuration
for the general snap documentation.

Fixes #4341

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-06-02 14:55:06 +00:00
Chen Yiyang
38a3188206
runk: Support list sub-command
Support list sub-command. It will traverse the root directory, parse
status file and print basic information of containers. Behavior and
print format consistent with runc. To handle race with runk delete
or system user modify, the loop will continue to traverse when errors
are encountered.

Fixes: #4362

Signed-off-by: Chen Yiyang <cyyzero@qq.com>
2022-06-02 18:24:51 +08:00
snir911
a0805742d6
Merge pull request #4350 from snir911/fix_workflow
workflows: add workflow_dispatch triggering to test-kata-deploy
2022-06-02 13:19:13 +03:00
Fabiano Fidêncio
24182d72d9
Merge pull request #4322 from jodh-intel/snap-cleanup
snap: Use helper script and cleanup
2022-06-02 11:47:02 +02:00
Peng Tao
295a01f9b1
Merge pull request #4159 from egernst/topic/iptables
feature: add ability to interact with IPTables within the guest
2022-06-02 11:19:41 +08:00
Tim Zhang
b8e98b175c
Merge pull request #4355 from liubin/fix/add-debug-info-for-parse-mount-error
agent: return mount file content if parse mountinfo failed
2022-06-02 10:31:46 +08:00
GabyCT
e8d0be364f
Merge pull request #4375 from GabyCT/topic/updateqat
docs: Update Intel QAT documentation links
2022-06-01 15:52:02 -05:00
Chelsea Mafrica
7ae11cad67 docs: Update source for cri-tools
Kubernetes-incubator was previously deprecated in favor of
kubernetes-sigs.

Fixes #4377

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2022-06-01 12:48:48 -07:00