Commit Graph

10979 Commits

Author SHA1 Message Date
Ji-Xinyou
fa6dff9f70 feat(runtime-rs): support vcpu resizing on runtime side
Support vcpu resizing on runtime side:
1. Calculate vcpu numbers in resource_manager using all the containers'
   linux_resources in the spec.
2. Call the hypervisor(vmm) to do the vcpu resize.
3. Call the agent to online vcpus.

Fixes: #5030
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
2023-06-12 17:53:16 +08:00
James O. D. Hunt
8cb4238b46 packaging: Remove snap package
Nobody has volunteered to maintain the (currently broken) snap build, so
remove it.

Fixes: #6769.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-06-12 09:24:09 +01:00
Helin Guo
2137739987 runtime-rs: update Cargo.lock
After we support memory resize in Dragonball, we need to update
Cargo.lock in runtime-rs.

Fixes: #6719

Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
2023-06-12 11:25:59 +08:00
Chao Wu
2988553305
Merge pull request #6998 from HerlinCoder/herlincoder/vpa
Dragonball: support resize memory
2023-06-11 17:21:12 +08:00
Archana Shinde
56d2ea9b78 kata-ctl: Refactor kernel module check
Adding vhost and vhost-net to the kernel modules. These do not require
any kernel module parameters to be checked. Currently, kernel params is
a required field. Make this as optional. Could make this as <Option>,
but making this a slice instead, as a module could have multiple kernel
params. Refactor the function that checks are for kernel modules into
two with one specifically checking if the module is loaded and other
checking for module parameters.

Refactor some of the tests to take into account these changes.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2023-06-09 14:10:31 -07:00
Aurélien Bombo
9f7a45996c gha: Add rootfs-initrd-mariner build target
This adds the Mariner guest image build target to the list of assets
as preparation for #6839.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-06-09 11:36:42 -07:00
Aurélien Bombo
f28a62164a gha: Add cloud-hypervisor-glibc build target
This adds the glibc flavor of CLH to the list of assets as preparation
for #6839. Mariner Kata is only tested with glibc.

Fixes: #7026

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-06-09 11:35:50 -07:00
Fabiano Fidêncio
b50f62ce48
Merge pull request #6756 from arronwy/measured_rootfs
Port Measured rootfs feature from CCv0 branch to main
2023-06-09 12:35:05 +02:00
Helin Guo
8fb7ab7518 dragonball: introduce virtio-balloon device
We introduce virtio-balloon device to support memory resize.
virtio-balloon device could reclaim memory from guest to host.

Fixes: #6719

Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
2023-06-09 17:47:27 +08:00
Helin Guo
7ed9494973 dragonball: introduce virtio-mem device
We introduce virtio-mem device to support memory resize. virtio-mem
device could hot-plug more memory blocks to guest and could also
hot-unplug them from guest.

Fixes: #6719

Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
2023-06-09 17:47:21 +08:00
Chao Wu
c7c45626c9
Merge pull request #6973 from Apokleos/direct-vol
add support direct volume and refactor device manager
2023-06-09 11:29:00 +08:00
alex.lyn
776a15e092 runtime-rs: add support direct volume.
As block/direct volume use similar steps of device adding,
so making full use of block volume code is a better way to
handle direct volume.

the only different point is that direct volume will use
DirectVolume and get_volume_mount_info to parse mountinfo.json
from the direct volume path. That's to say, direct volume needs
the help of `kata-ctl direct-volume ...`.

Details seen at Advanced Topics:
[How to run Kata Containers with kinds of Block Volumes]
docs/how-to/how-to-run-kata-containers-with-kinds-of-Block-Volumes.md

Fixes: #5656

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-06-09 08:16:26 +08:00
Helin Guo
a8e0f51c52 dragonball: extend DeviceOpContext
In order to support virtio-mem and virtio-balloon devices, we need to
extend DeviceOpContext with VmConfigInfo and InstanceInfo.

Fixes: #6719

Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
2023-06-08 22:04:31 +08:00
alex.lyn
abae114046 runtime-rs: refactor device manager implementation
The key aspects of the DM implementation refactoring as below:

1. reduce duplicated code
 Many scenarios have similar steps when adding devices. so to reduce
 duplicated code, we should create a common method abstracted and use
 it in various scenarios.
do_handle_device:
(1) new_device with DeviceConfig and return device_id;
(2) try_add_device with device_id and do really add device;
(3) return device info of device's info;

2. return full info of Device Trait get_device_info
 replace the original type DeviceConfig with full info DeviceType.

3. refactor find_device method.

Fixes: #5656

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-06-08 08:47:08 +08:00
Fabiano Fidêncio
08d10d38be
Merge pull request #7048 from sprt/sprt/fix-gha
gha: Fix gha-run.sh and unbreak CI
2023-06-07 23:40:02 +02:00
James O. D. Hunt
452f286552
Merge pull request #6764 from byron-marohn/fix_5401
kata-ctl: Switch to slog logging; add --log-level and --json-logging arguments
2023-06-07 16:08:53 +01:00
Fuu
210a15794c dragonball: avoid obtaining lock twice in create_stdio_console
Fixes #7055

Signed-off-by: Fuu <fuu-open@linux.alibaba.com>
2023-06-07 16:12:22 +08:00
Aurélien Bombo
69668ce87f tests: gha-run: Use correct env variable for repo
s/DOCKER_IMAGE/DOCKER_REPO

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-06-06 11:54:43 -07:00
Aurélien Bombo
f487199edf gha: aks: Fix argument in call to gha-run.sh
Fixes: #7047

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-06-06 11:51:18 -07:00
GabyCT
5ad8aaf9df
Merge pull request #7035 from GabyCT/topic/logparserdoc
log-parser: Update log parser link at README
2023-06-06 12:02:25 -06:00
Fabiano Fidêncio
de2e507483
Merge pull request #6972 from sprt/sprt/gha-run-script
gha: aks: Extract `run` commands to a script
2023-06-06 14:54:03 +02:00
Wang, Arron
f6afae9c73 packaging: Add rootfs-image-tdx-tarball target
Add rootfs-image-tdx target:
./tools/packaging/kata-deploy/local-build/kata-deploy-binaries.sh --build=rootfs-image-tdx
./opt/kata/share/kata-containers/kata-containers-tdx.img
./opt/kata/share/kata-containers/kata-ubuntu-latest-tdx.image

Fixes: #6674

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2023-06-06 12:34:20 +02:00
Wang, Arron
f62b2670c0 config: Add root hash value and measure config to kernel params
After we have a guest kernel with builtin initramfs which
provide the rootfs measurement capability and Kata rootfs
image with hash device, we need set related root hash value
and measure config to the kernel params in kata configuration file.

Fixes: #6674

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2023-06-06 12:34:13 +02:00
Wang, Arron
0080588075 kernel: Integrate initramfs into Guest kernel
Integrate initramfs into guest kernel as one binary,
which will be measured by the firmware together.

Fixes: #6674

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2023-06-06 12:33:41 +02:00
Wang, Arron
28b2645624 initramfs: Add build script to generate initramfs
The init.sh in initramfs will parse the verity scheme,
roothash, root device and setup the root device accordingly.

Fixes: #6674

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2023-06-06 12:33:28 +02:00
Wang, Arron
5cb02a8067 image-build: generate root hash as an separate partition for rootfs
Generate rootfs hash data during creating the kata rootfs,
current kata image only have one partition, we add another
partition as hash device to save hash data of rootfs data blocks.

Fixes: #6674

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2023-06-06 12:31:14 +02:00
Arron Wang
31c0ad2076 packaging: Add cryptsetup support in Guest kernel and rootfs
Add required kernel config for dm-crypt/dm-integrity/dm-verity
and related crypto config.

Add userspace command line tools for disk encryption support
and ext4 file system utilities.

Fixes: #6674

Signed-off-by: Arron Wang <arron.wang@intel.com>
2023-06-06 12:30:07 +02:00
Fabiano Fidêncio
eb1bfa922b
Merge pull request #6980 from nubificus/feat_sharefs_files
runtime-rs: handle copy files when share_fs is not available
2023-06-06 12:26:55 +02:00
Chao Wu
b0c6cd05a2
Merge pull request #7033 from openanolis/fix-agent-ctl
agent-ctl: fix the compile error
2023-06-06 11:55:15 +08:00
Gabriela Cervantes
980d084f47 log-parser: Update log parser link at README
This PR updates the link to the correspondent Developer Guide at the
enabling full containerd debug that we have for kata 2.0 documentation.

Fixes #7034

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-06-05 15:59:52 +00:00
Yushuo
410bc18143 agent-ctl: fix the compile error
When the version of libc is upgraded to 0.2.145, older getrandom could not adapt
to new API, and this will make agent-ctl fail to compile.

We upgrade the version of `rand`, so the low version of getrandom will no longer
need.

Fixes: #7032

Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
2023-06-05 21:48:36 +08:00
Jayant Singh
77519fd120 kata-ctl: Switch to slog logging; add --log-level, --json-logging args
Fixes: #5401, #6654

- Switch kata-ctl from eprintln!()/println!() to structured logging via
  the logging library which uses slog.
- Adds a new create_term_logger() library call which enables printing
  log messages to the terminal via a less verbose / more human readable
  terminal format with colors.
- Adds --log-level argument to select the minimum log level of printed messages.
- Adds --json-logging argument to switch to logging in JSON format.

Co-authored-by: Byron Marohn <byron.marohn@intel.com>
Co-authored-by: Luke Phillips <lucas.phillips@intel.com>
Signed-off-by: Jayant Singh <jayant.singh@intel.com>
Signed-off-by: Byron Marohn <byron.marohn@intel.com>
Signed-off-by: Luke Phillips <lucas.phillips@intel.com>
Signed-off-by: Kelby Madal-Hellmuth <kelby.madal-hellmuth@intel.com>
Signed-off-by: Liz Lawrens <liz.lawrens@intel.com>
2023-06-02 20:13:22 +00:00
Aurélien Bombo
aab6030962 gha: aks: Extract run commands to a script
Github Actions reads and runs workflow files from the main branch,
rather than from the PR branch. This means that PRs that modify workflow
files aren't being tested with the updated workflows coming from the PR,
but rather with the old workflows from the main branch. AFAIK, this
behavior isn't avoidable for workflow files (but is for other scripts).

This makes it very hard to reliably test workflow changes before they're
actually merged into main and leads to issues that we have to hotifx
(see #6983, #6995).

This PR aims to mitigate that by extracting the commands used in
workflows to a separate script file. The way our CI is set up, those
script files are read from the PR branch and thus changes would be
reflected in the CI checks.

Fixes: #6971

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-06-02 10:22:35 -07:00
Fupan Li
465f5a5ced
Merge pull request #4748 from lifupan/main_fix
agent: fix the issue of exec hang with a backgroud process
2023-06-02 10:46:43 +08:00
Chao Wu
2128fa2b4e
Merge pull request #7013 from xuejun-xj/xuejun/bugfix
runtime-rs: bugfix: update Cargo.lock
2023-06-02 10:08:27 +08:00
Anastassios Nanos
e4eb664d27 runtime-rs: update rust to 1.69.0
We are probably hitting this:
https://github.com/rust-lang/rust/issues/63033

Seems like it is worth a try to upgrade to 1.69.0

Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
2023-06-01 21:40:56 +00:00
Anastassios Nanos
ed37715e05 runtime-rs: handle copy files when share_fs is not available
In hypervisors that do not support virtiofs we have to copy files in
the VM sandbox to properly setup the network (resolv.conf, hosts, and hostname).

To do that, we construct the volume as before, with the addition of an extra
variable that designates the path where the file will reside in the sandbox.

In this case, we issue a `copy_file` agent request *and* we patch the spec
to account for this change.

Fixes: #6978

Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
Signed-off-by: George Pyrros <gpyrros@nubificus.co.uk>
2023-06-01 21:40:56 +00:00
Fabiano Fidêncio
18b1a019d4
Merge pull request #7011 from jepio/fix-aks-cluster-name
gha: aks: Use short SHA in cluster name
2023-06-01 15:56:20 +02:00
Fabiano Fidêncio
5ab42d87fb
Merge pull request #7009 from fidencio/topic/display-badge-for-the-publish-artefacts-job
README: Display badge for the "Publish Artefacts" job and update the Kata Containers logo
2023-06-01 15:13:41 +02:00
Fabiano Fidêncio
eb1f44f111
Merge pull request #7007 from fidencio/topic/try-to-fix-ubuntu-k8s-key-not-available
kata-deploy: Change how we get the Ubuntu k8s key
2023-06-01 15:13:22 +02:00
xuejun-xj
5f6fc3ed76 runtime-rs: bugfix: update Cargo.lock
When dragonball update dbs-boot crate in commit
64c764c147, the Cargo.lock in runtime-rs
should also be updated.

Fixes: #6969

Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
2023-06-01 20:25:35 +08:00
Jeremi Piotrowski
1c6d22c803 gha: aks: Use short SHA in cluster name
Full SHA is 40 characters, while AKS cluster name has a limit of 63. Trim the
SHA to 12 characters, which is widely considered to be unique enough and is
short enough to be used in the cluster name

Fixes: #7010
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-06-01 14:03:53 +02:00
Fabiano Fidêncio
3c1f6d36dc readme: Update Kata Containers logo
Let's use the horizontal logo, as it occupies better the space the we
have.

The logo comes from:
https://openinfra.dev/brand/logos

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-06-01 12:25:13 +02:00
Fabiano Fidêncio
3886841131 readme: Add status badge for the "Publish Artefacts" job
Let's start adding the status of our jobs as part of our main page, so
folks monitoring those can easily check whether they're okay, or if
someone has to be pinged about those.

Fixes: #7008

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-06-01 12:25:01 +02:00
Fabiano Fidêncio
26f7520387 kata-deploy: Change how we get the Ubuntu k8s key
The current method has been failing every now and then, and was reported
on https://github.com/kubernetes/release/issues/2862.

Ding poked me and suggested to do this change here, so here we go. :-)

Fixes: #7006

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-06-01 12:10:30 +02:00
Fabiano Fidêncio
9ec2bca101
Merge pull request #7002 from fidencio/topic/follow-up-on-7000
gha: aks: Ensure host_os is used everywhere needed
2023-06-01 08:51:27 +02:00
Fabiano Fidêncio
8cbb80da66
Merge pull request #6929 from LindaYu17/dev
kubernetes: add agnhost command in pod yaml
2023-06-01 08:39:58 +02:00
Fabiano Fidêncio
aebd3b47d9 gha: aks: Ensure host_os is used everywhere needed
We added that to create the cluster name, but I forgot to add that to
the part we get the k8s config file, or to the part where we delete the
AKS cluster.

Fixes: #6999

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-31 20:50:55 +02:00
Fabiano Fidêncio
e01f75723a
Merge pull request #6997 from singhwang/main
main | release: Standardize kata static file name
2023-05-31 15:22:30 +02:00
Fabiano Fidêncio
1ed917a079
Merge pull request #6989 from BbolroC/configurable-build-registry
packaging: make BUILDER_REGISTRY configurable
2023-05-31 15:18:51 +02:00