Commit Graph

12379 Commits

Author SHA1 Message Date
ChengyuZhu6
dfad0e6622 .github: fix the failure without devicemapper for host sharing
fix error when running checks and tests:
error: failed to run custom build command for `devicemapper-sys v0.1.5`
fatal error: 'libdevmapper.h' file not found

thread 'main' panicked at 'Could not generate dm.h bindings:
ClangDiagnostic("dm.h:2:10: fatal error: 'libdevmapper.h' file not found\n")',
/home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/devicemapper-sys-0.1.5/build.rs:24:10
  stack backtrace:
     0: rust_begin_unwind
               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:593:5
     1: core::panicking::panic_fmt
               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/panicking.rs:67:14
     2: core::result::unwrap_failed
               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/result.rs:1651:5
     3: core::result::Result<T,E>::expect
     4: build_script_build::main
     5: core::ops::function::FnOnce::call_once
  note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
warning: build failed, waiting for other jobs to finish...
make: *** [../../utils.mk:177: standard_rust_check] Error 101

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-12-14 20:47:47 +08:00
ChengyuZhu6
983479748f .github: fix error when making checks for CoCo guest pull
Fix error when making checks:
```
error: failed to run custom build command for `image-rs v0.1.0
(https://github.com/confidential-containers/guest-components?tag=v0.8.0#e849dc89)`

Caused by:
  process didn't exit successfully: `/home/runner/work/kata-containers/kata-containers/src/
  agent/target/release/build/image-rs-fd932206d09362b7/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-changed=./protos/getresource.proto
  cargo:rerun-if-changed=./protos

  --- stderr
  thread 'main' panicked at 'Could not find `protoc` installation and this build crate cannot proceed without
  this knowledge. If `protoc` is installed and this crate had trouble finding
  it, you can set the `PROTOC` environment variable with the specific path to your
  installed `protoc` binary.If you're on debian, try `apt-get install protobuf-compiler`
  or download it from https://github.com/protocolbuffers/protobuf/releases
```

Fixes #8673

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2023-12-14 20:47:42 +08:00
Fabiano Fidêncio
a998e89bcf
Merge pull request #8639 from fidencio/topic/kata-deploy-use-tomlq-to-configure-containerd
kata-deploy: Use `tomlq` to configure containerd
2023-12-13 14:11:45 +01:00
Chao Wu
dfaf006fcc
Merge pull request #8564 from openanolis/chao/add_pci_root_bus_device
dragonball: add pci root bus and root device
2023-12-13 17:57:16 +08:00
Fabiano Fidêncio
7ad873cf29
kata-deploy: Simplify shim configuration
We never have to add a configuration for the "default" case, as we're
already creating the runtime class pointing to what should be the
"default" handler.

This helps to simplify the logic by quite a lot.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-12-13 10:52:54 +01:00
Fabiano Fidêncio
e618949937
kata-deploy: Remove useless comment from CRI-O drop-in
The comment adds absolutely nothing to the runtime handler added, and
it'd make our life slightly harder to properly say which VMM is being
used when setting the default `kata` handler.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-12-13 10:49:52 +01:00
Fabiano Fidêncio
dd9f5b07b9
kata-deploy: Use tomlq to configure containerd
This save us a lot of trouble on properly sed'ing content that may or
may not be in the containerd configuration file.

Fixes: #8638

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-12-13 10:49:49 +01:00
Fabiano Fidêncio
4f01f294bb
kata-deploy: Install tomlq to the base image
This will help us to have an easier time playing with the containerd
configuration, instead of having to sed the **** out of it, which is
super error prone.

`tomlq` is a tool that comes from https://github.com/kislyuk/yq, and
that depends on `jq` to do the toml parsing / editing.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-12-13 10:49:07 +01:00
James O. D. Hunt
d7c6219dfe
Merge pull request #8630 from jodh-intel/runtime-rs-ch-set-state-on-vm-stop
runtime-rs: ch: Change state when VM stopped
2023-12-13 09:26:30 +00:00
Xuewei Niu
855adbc63b
Merge pull request #8634 from justxuewei/disable-packed-vq
dragonball: Disable packed virtqueue for vhost-user devices
2023-12-13 17:03:05 +08:00
James O. D. Hunt
2a518f0898 runtime-rs: ch: Change state when VM stopped
Make the CH (Cloud Hypervisor) `stop_vm()` method check the VM state before
attempting to stop the VM, and update the state once the VM has stopped.

This avoids the method failing if called multiple times which will
happen if the workload exits before the container manager requests that
the container stop.

This change ensures the CH driver finishes cleanly.

Fixes: #8629.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-12 18:25:20 +00:00
Fabiano Fidêncio
39f5cea3b1
kata-deploy: Fix k0s cri notation comment
We can safely assume we're using the *newer* notation, not the *older*
one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-12-12 18:20:18 +01:00
James O. D. Hunt
1195692d3c runtime-rs: ch: Move state handling to top-level APIs
Move the state setting to the `Hypervisor` trait calls. This makes the
code clearer.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-12 15:25:27 +00:00
Xuewei Niu
86918e91b3 dragonball: Disable packed virtqueue for vhost-user devices
The layout of packed virtqueue isn't supported by `Endpoint::negotiate()`.
Communication between device and driver will be failed due to the failure
of parsing virtqueue if we don't disable the packed feature. This patch
fixes this issue.

Fixes: #8633

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2023-12-12 17:24:20 +08:00
Chao Wu
b079e1aabc dragonball: add pci root bus and root device
In order to follow up the PCI implementation in Dragonball, we need to
add PCI root device and root bus support.

root device is a pseudo PCI root device to manage accessing to PCI
configuration space.

root bus is mainly for emulating PCI root bridge and also create the PCI
root bus with the given bus ID with the PCI root bridge.

fixes: #8563

Signed-off-by: Gerry Liu <gerry@linux.alibaba.com>
Signed-off-by: Zizheng Bian <zizheng.bian@linux.alibaba.com>
Signed-off-by: Shifang Feng <fengshifang@linux.alibaba.com>
Signed-off-by: Yang Su <yang.su@linux.alibaba.com>
Signed-off-by: Zha Bin <zhabin@linux.alibaba.com>
Signed-off-by: Xin Lin <jingshan@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2023-12-12 11:43:14 +08:00
GabyCT
ee74fca92c
Merge pull request #8617 from GabyCT/topic/enabletestnerdctl
tests: nerdctl: Enable nerdctl tests for cloud hypervisor runtime-rs
2023-12-11 14:09:58 -06:00
David Esparza
584a26dab0
Merge pull request #8542 from dborquez/metrics_fix_deployment_cleaning
metrics: cleans k8s iperf deployment when the test finishes.
2023-12-11 13:14:39 -06:00
Chao Wu
198e4adcb1
Merge pull request #8599 from openanolis/chao/fix_cargo_fmt
dragonball: add --all for fmt ci
2023-12-12 00:20:21 +08:00
GabyCT
43410e1918
Merge pull request #8560 from GabyCT/topic/enablek8srs
gha: k8s: Add cloud-hypervisor (runtime-rs) support
2023-12-11 09:42:49 -06:00
Hyounggyu Choi
ea2a0dc69d
Merge pull request #7769 from BbolroC/opa-multiarch
rootfs: build OPA binary from source for ppc64le and s390x
2023-12-11 15:25:33 +01:00
Chao Wu
52f7a40e4e dragonball: add --all for fmt ci
Right now, cargo fmt check in Dragonball only test with the default
features but not all features. This will cause some code being untested
by the fmt tool.

This PR adds --all option for the Dragonball CI and also fix some code
that forgets to do cargo fmt --all.

fixes: #8598

Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2023-12-11 20:54:25 +08:00
Hyounggyu Choi
375c787e09 rootfs: build OPA binary from source for ppc64le and s390x
This PR is to build a binary for OPA from source code for ppc64le and s390x.

Fixes: #7616

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-12-11 12:59:48 +01:00
Hyounggyu Choi
16e2a50d17
Merge pull request #8624 from BbolroC/fix-runtime-class-check-qemu-se
GHA: Fix kata-deploy-runtime-classes-check for kata-qemu-se
2023-12-11 12:58:00 +01:00
James O. D. Hunt
2a35541af7
Merge pull request #8592 from jodh-intel/static-checks-try-multiple-user-agents
CI: static-checks: Try multiple user agents
2023-12-11 11:52:29 +00:00
Hyounggyu Choi
28c3e0e5f0 GHA: Fix kata-deploy-runtime-classes-check for kata-qemu-se
This is to fix an error on kata-deploy-runtime-classes-check for kata-qemu-se.

Fixes: #8623

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-12-11 10:30:00 +01:00
Hyounggyu Choi
b469dbf92f
Merge pull request #8622 from BbolroC/hotfix-k3s-kubectl-version
GHA: Use --client=true for k3s kubectl version
2023-12-11 10:00:16 +01:00
Hyounggyu Choi
40f0c8fbb7 GHA: Use --client=true for k3s kubectl version
This is to fix a broken usage for `k3s kubectl version` by switching
an option `--short` to `--client=true`.

Fixes: #8621

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-12-11 08:26:39 +01:00
Chao Wu
df7f416cb8
Merge pull request #8566 from liubogithub/liubo/dev/panic_fix
runtime-rs: fix panic when hypervisor mismatches with configuration
2023-12-10 21:33:59 +08:00
Gabriela Cervantes
1662a3e859 common: Add cloud hypervisor in enabling hypervisor function
This PR adds the cloud hypervisor in the enabling hypervisor function.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-12-08 21:32:00 +00:00
Chelsea Mafrica
1c42d94550
Merge pull request #6826 from gabevenberg/log-parser-rs
kata-ctl: Moved log-parser-rs into kata-ctl
2023-12-08 11:33:09 -08:00
James O. D. Hunt
5d085a3042 CI: static-checks: Try multiple user agents
Make the URL checker cycle through a list of user agent values until we
hit one the remote server is happy with.

This is required since, unfortunately, we really, really want to check
these URLs, but some sites block clients based on their `User-Agent`
(UA) request header value. And of course, each site is different and can
change its behaviour at any time.

Our strategy therefore is to try various UA's until we find one the
server accepts:

- No explicit UA (use `curl`'s default)
- Explicitly no UA.
- A blank UA.
- Partial UA values for various CLI tools.
- Partial UA values for various console web browsers.
- Partial UA for Emacs's built-in browser.
- The existing UA which is used as a "last ditch" attempt where the UA implies multiple platforms and browser.

> **Notes:**
>
> - The "partial UA" values specify specify the UA "product" but not the
>   UA "product version": we specify `foo` and not `foo/1.2.3`). We do
>   this since most sites tested appear to not care about the version.
>   This is as expected given that the version is strictly optional (see `[*]`).
>
> - We now log all errors and display an error summary if none of the UAs
>   worked, in addition to the simple list of the URLs we believe to be
>   invalid. This should make future debugging simpler.

`[*]` - https://www.rfc-editor.org/rfc/rfc9110#section-10.1.5

Fixes: #8553.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-08 18:02:41 +00:00
James O. D. Hunt
3174c18772 docs: Remove problematic URL
Removed the Azure Portal URL (https://portal.azure.com) since this
causes problems with our static checks script: that URL returns HTTP 403
("Forbidden") when queried using command-line tools like `curl(1)`,
which is used by the static check script.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-08 17:11:20 +00:00
James O. D. Hunt
3779261a99 docs: Fix whitespace
Remove some extraneous whitespace.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-08 17:11:20 +00:00
James O. D. Hunt
613def0328 CI: static-checks: Move curl to a separate function
Split the call to `curl` in the URL checker out into a new
`run_url_check_cmd()` function to make `check_url()` slightly clearer.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-08 17:11:20 +00:00
James O. D. Hunt
6d859f97ee CI: static-checks: Lint fixes
Declare and then define a couple of variables separately.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-08 17:11:20 +00:00
James O. D. Hunt
efa8e6547c CI: static-checks: Check params have a value
Check that the `check_url()` parameters have a value.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-08 17:11:20 +00:00
James O. D. Hunt
563ea020b0 CI: static-checks: Fold long line
Break up a long line as little to make it easier to read.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-08 17:11:20 +00:00
James O. D. Hunt
3ad43df946 CI: static-checks: Improve markdown checker test
Only attempt to build the markdown checker if it doesn't already exist.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-08 17:11:20 +00:00
Liu Bo
bf97051f11 runtime-rs: fix panic when hypervisor mismatches with configuration
If a wrong configuration.toml file is used by accidentally, runtime-rs
binary could run into panic because of unwrap().

This fixes the panic by returning errors instead of unwrap().

fixes: #8565

Signed-off-by: Liu Bo <liub.liubo@gmail.com>
2023-12-08 08:56:23 -08:00
Zvonko Kaiser
9d38f01c2f
Merge pull request #8612 from BbolroC/introduce-secret-inheritance-s390x
GHA: make secrets inherited for build-kata-static-tarball-s390x
2023-12-08 17:32:47 +01:00
Gabriela Cervantes
f3eeab10ab tests: nerdctl: Enable nerdctl tests for cloud hypervisor runtime-rs
This PR enables the nerdctl tests for cloud hypervisor runtime-rs.

Fixes #8616

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-12-08 16:12:36 +00:00
Hyounggyu Choi
636eef8907 GHA: make secrets inherited for build-kata-static-tarball-s390x
This is to make GHA secrets inherited for the workflow titled
`build-kata-static-tarball-s390x` to configure an environment
variable `CI_HKD_PATH` for a `build-asset-boot-image-se` step.

Fixes: #8611

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-12-08 13:55:45 +01:00
Chao Wu
5054e59ccb
Merge pull request #8429 from adamqqqplay/support-vhost-user-fs
dragonball: introduce vhost-user-fs device
2023-12-08 17:20:52 +08:00
Hyounggyu Choi
588f639a69
Merge pull request #6755 from BbolroC/add-se-artifacts-to-main
packaging: Add IBM Z SE artifacts to main
2023-12-08 05:17:38 +01:00
Gabe Venberg
69fdd05ce5 kata-ctl: Moved log-parser-rs into kata-ctl
Log-parser-rs was always intended to become a sub-functionality of
kata-ctl, but it was useful to develop it and initaly merge it as a
standalone program, and migrate it to a subcommand later.

Fixes #6797

Signed-off-by: Gabe Venberg <gabevenberg@gmail.com>
2023-12-07 21:35:28 -06:00
David Esparza
b2577000e7
metrics: Expose iperf3 pods over a k8s networks.
A prerequisite for measuring kata network bandwidth is
run Iperf3 tool at a the transport layer provided by a
k8s service for exposing a network where the clients
inside the cluster can use to contact Pods in the service.

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-12-07 18:07:05 -06:00
David Esparza
a062ba166b
metrics: cleans k8s iperf deployment when the test finishes.
This PR fixes small issues like:
1. Cleaning up the k8s environment by removing the iperf test
implementation even when the test fails.
2. Checks if the workload returned a result before generating
an empty results json file as it was bein done.
3. Removes the redundancy of calls to functions that process
subtests and should compose the results json file only when
all results are ready and not before.
4. The tcp service manifest was added to the server deployment
which targets TCP port 5201.

Fixes: #8534

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-12-07 18:02:39 -06:00
Archana Shinde
a5105b4227
Merge pull request #8582 from amshinde/runtime-rs-tryfrom-blkconfig
Implement and use try_from for DiskConfig
2023-12-07 15:02:00 -08:00
GabyCT
0e0a7d9410
Merge pull request #8604 from GabyCT/topic/enablenerdctlrs
gha: nerdctl: Enable cloud hypervisor runtime-rs for nerdctl CI
2023-12-07 14:35:26 -06:00
Hyounggyu Choi
3fab1690a4 local-build: make strip support for cross-compilation
This is to adjust a name of the binary `strip` to a target architecture for cross-compilation.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-12-07 20:05:40 +01:00