Commit Graph

12805 Commits

Author SHA1 Message Date
Hyounggyu Choi
b469dbf92f
Merge pull request #8622 from BbolroC/hotfix-k3s-kubectl-version
GHA: Use --client=true for k3s kubectl version
2023-12-11 10:00:16 +01:00
Hyounggyu Choi
40f0c8fbb7 GHA: Use --client=true for k3s kubectl version
This is to fix a broken usage for `k3s kubectl version` by switching
an option `--short` to `--client=true`.

Fixes: #8621

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-12-11 08:26:39 +01:00
Chao Wu
df7f416cb8
Merge pull request #8566 from liubogithub/liubo/dev/panic_fix
runtime-rs: fix panic when hypervisor mismatches with configuration
2023-12-10 21:33:59 +08:00
Gabriela Cervantes
1662a3e859 common: Add cloud hypervisor in enabling hypervisor function
This PR adds the cloud hypervisor in the enabling hypervisor function.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-12-08 21:32:00 +00:00
Chelsea Mafrica
1c42d94550
Merge pull request #6826 from gabevenberg/log-parser-rs
kata-ctl: Moved log-parser-rs into kata-ctl
2023-12-08 11:33:09 -08:00
James O. D. Hunt
5d085a3042 CI: static-checks: Try multiple user agents
Make the URL checker cycle through a list of user agent values until we
hit one the remote server is happy with.

This is required since, unfortunately, we really, really want to check
these URLs, but some sites block clients based on their `User-Agent`
(UA) request header value. And of course, each site is different and can
change its behaviour at any time.

Our strategy therefore is to try various UA's until we find one the
server accepts:

- No explicit UA (use `curl`'s default)
- Explicitly no UA.
- A blank UA.
- Partial UA values for various CLI tools.
- Partial UA values for various console web browsers.
- Partial UA for Emacs's built-in browser.
- The existing UA which is used as a "last ditch" attempt where the UA implies multiple platforms and browser.

> **Notes:**
>
> - The "partial UA" values specify specify the UA "product" but not the
>   UA "product version": we specify `foo` and not `foo/1.2.3`). We do
>   this since most sites tested appear to not care about the version.
>   This is as expected given that the version is strictly optional (see `[*]`).
>
> - We now log all errors and display an error summary if none of the UAs
>   worked, in addition to the simple list of the URLs we believe to be
>   invalid. This should make future debugging simpler.

`[*]` - https://www.rfc-editor.org/rfc/rfc9110#section-10.1.5

Fixes: #8553.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-08 18:02:41 +00:00
James O. D. Hunt
3174c18772 docs: Remove problematic URL
Removed the Azure Portal URL (https://portal.azure.com) since this
causes problems with our static checks script: that URL returns HTTP 403
("Forbidden") when queried using command-line tools like `curl(1)`,
which is used by the static check script.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-08 17:11:20 +00:00
James O. D. Hunt
3779261a99 docs: Fix whitespace
Remove some extraneous whitespace.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-08 17:11:20 +00:00
James O. D. Hunt
613def0328 CI: static-checks: Move curl to a separate function
Split the call to `curl` in the URL checker out into a new
`run_url_check_cmd()` function to make `check_url()` slightly clearer.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-08 17:11:20 +00:00
James O. D. Hunt
6d859f97ee CI: static-checks: Lint fixes
Declare and then define a couple of variables separately.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-08 17:11:20 +00:00
James O. D. Hunt
efa8e6547c CI: static-checks: Check params have a value
Check that the `check_url()` parameters have a value.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-08 17:11:20 +00:00
James O. D. Hunt
563ea020b0 CI: static-checks: Fold long line
Break up a long line as little to make it easier to read.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-08 17:11:20 +00:00
James O. D. Hunt
3ad43df946 CI: static-checks: Improve markdown checker test
Only attempt to build the markdown checker if it doesn't already exist.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-12-08 17:11:20 +00:00
Liu Bo
bf97051f11 runtime-rs: fix panic when hypervisor mismatches with configuration
If a wrong configuration.toml file is used by accidentally, runtime-rs
binary could run into panic because of unwrap().

This fixes the panic by returning errors instead of unwrap().

fixes: #8565

Signed-off-by: Liu Bo <liub.liubo@gmail.com>
2023-12-08 08:56:23 -08:00
Zvonko Kaiser
9d38f01c2f
Merge pull request #8612 from BbolroC/introduce-secret-inheritance-s390x
GHA: make secrets inherited for build-kata-static-tarball-s390x
2023-12-08 17:32:47 +01:00
Gabriela Cervantes
f3eeab10ab tests: nerdctl: Enable nerdctl tests for cloud hypervisor runtime-rs
This PR enables the nerdctl tests for cloud hypervisor runtime-rs.

Fixes #8616

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-12-08 16:12:36 +00:00
Hyounggyu Choi
636eef8907 GHA: make secrets inherited for build-kata-static-tarball-s390x
This is to make GHA secrets inherited for the workflow titled
`build-kata-static-tarball-s390x` to configure an environment
variable `CI_HKD_PATH` for a `build-asset-boot-image-se` step.

Fixes: #8611

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-12-08 13:55:45 +01:00
Chao Wu
5054e59ccb
Merge pull request #8429 from adamqqqplay/support-vhost-user-fs
dragonball: introduce vhost-user-fs device
2023-12-08 17:20:52 +08:00
Hyounggyu Choi
588f639a69
Merge pull request #6755 from BbolroC/add-se-artifacts-to-main
packaging: Add IBM Z SE artifacts to main
2023-12-08 05:17:38 +01:00
Gabe Venberg
69fdd05ce5 kata-ctl: Moved log-parser-rs into kata-ctl
Log-parser-rs was always intended to become a sub-functionality of
kata-ctl, but it was useful to develop it and initaly merge it as a
standalone program, and migrate it to a subcommand later.

Fixes #6797

Signed-off-by: Gabe Venberg <gabevenberg@gmail.com>
2023-12-07 21:35:28 -06:00
David Esparza
b2577000e7
metrics: Expose iperf3 pods over a k8s networks.
A prerequisite for measuring kata network bandwidth is
run Iperf3 tool at a the transport layer provided by a
k8s service for exposing a network where the clients
inside the cluster can use to contact Pods in the service.

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-12-07 18:07:05 -06:00
David Esparza
a062ba166b
metrics: cleans k8s iperf deployment when the test finishes.
This PR fixes small issues like:
1. Cleaning up the k8s environment by removing the iperf test
implementation even when the test fails.
2. Checks if the workload returned a result before generating
an empty results json file as it was bein done.
3. Removes the redundancy of calls to functions that process
subtests and should compose the results json file only when
all results are ready and not before.
4. The tcp service manifest was added to the server deployment
which targets TCP port 5201.

Fixes: #8534

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-12-07 18:02:39 -06:00
Archana Shinde
a5105b4227
Merge pull request #8582 from amshinde/runtime-rs-tryfrom-blkconfig
Implement and use try_from for DiskConfig
2023-12-07 15:02:00 -08:00
Archana Shinde
458e91b289 runtime-rs: Update readme to indicate cloud-hypervisor support
Since cloud-hypervisor is no longer built as an optional feature,
lets mention cloud-hypervisor in the list of hypervisors supported by
runtime-rs.

Fixes: #8587

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2023-12-07 14:59:43 -08:00
GabyCT
0e0a7d9410
Merge pull request #8604 from GabyCT/topic/enablenerdctlrs
gha: nerdctl: Enable cloud hypervisor runtime-rs for nerdctl CI
2023-12-07 14:35:26 -06:00
Hyounggyu Choi
3fab1690a4 local-build: make strip support for cross-compilation
This is to adjust a name of the binary `strip` to a target architecture for cross-compilation.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-12-07 20:05:40 +01:00
Hyounggyu Choi
f38c7f14c5 gha: remove build redundancy of kernel and rootfs-initrd
It is to remove the build redundancy of `kernel` and `rootfs-initrd` by making `boot-image-se` built based on them at the second build stage.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-12-07 20:05:40 +01:00
Hyounggyu Choi
31db56207b local-build: add support for key verification for IBM Secure Execution
This is to make `build_se_image.sh` incorporate the key verification originally supported by `genprotimg`.
It can be achieved by specifying two environment variables called `SIGNING_KEY_CERT_PATH` and `INTERMEDIATE_CA_CERT_PATH`.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-12-07 20:05:40 +01:00
Hyounggyu Choi
52bdc87fe9 local-build: make kernel parameters configurable
This is to make kernel parameters configurable during the secure image build by adding an environment variable SE_KERNEL_PARAMS.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-12-07 20:05:40 +01:00
Hyounggyu Choi
9ceb2c27e0 local-build: consider cross-compilation env
This is to make a base builder image build genprotimg without a package
manager under the cross-compilation environment.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-12-07 20:05:40 +01:00
David Esparza
298be4aa1c
Merge pull request #8594 from GabyCT/topic/updatedockerfilet
metrics: Update TensorFlow ResNet FP32 dockerfile
2023-12-07 11:14:48 -06:00
Gabriela Cervantes
ce694b905b tests: Fix indentation of gha-run script
This PR fixes the indentation of gha run script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-12-07 16:56:19 +00:00
Gabriela Cervantes
33b300431e tests: Enable but do not run k8s tests for cloud hypervisor
This PR enables but do not run k8s tests for cloud hypervisor
for runtime-rs.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-12-07 16:39:15 +00:00
Gabriela Cervantes
acee3d8438 gha: k8s: Add cloud-hypervisor (runtime-rs) support
This PR adds the Cloud Hypervisor driver, integrated with the runtime-rs,
as part of the kubernetes tests.

Fixes #8559

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-12-07 16:33:59 +00:00
Gabriela Cervantes
50a5fa9a65 tests: Enable but do not run the nerdctl tests for cloud hypervisor
This PR enables but do not run the nerdctl tests for cloud hypervisor
runtime-rs until we find out how stable they are.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-12-07 16:29:51 +00:00
Gabriela Cervantes
e70b2ea95d gha: nerdctl: Enable cloud hypervisor runtime-rs for nerdctl CI
This PR enables the cloud hypervisor runtime-rs for the nerdctl
gha CI.

Fixes #8603

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-12-07 16:24:36 +00:00
Hyounggyu Choi
ad6aab9918
Merge pull request #8601 from BbolroC/conflict-handling-for-self-hosted-runners
GHA: remove GITHUB_WORKSPACE when workflow fails due to merge conflict
2023-12-07 12:17:31 +01:00
Hyounggyu Choi
0d5a970e54 GHA: remove GITHUB_WORKSPACE when workflow fails due to merge conflict
It is to remove a GITHUB_WORKSPACE directory for self-hosted runners
when a workflow fails due to the merge conflict. This will prevent
the subsequent workflows from getting stuck in the same situation.

Fixes: #8600

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-12-07 10:25:57 +01:00
Greg Kurz
501910d743
Merge pull request #8509 from zvonkok/stable-overlay
deployment: Add stable overlay for kata-deploy.yaml
2023-12-07 09:43:41 +01:00
Huang Jianan
5629b7454f dragonball: support vhost-user-fs in device manager
This patch implements the virtio-fs device used for filesystem sharing
and heavily based on the vhost-user protocol.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
Signed-off-by: Huang Jianan <jnhuang@linux.alibaba.com>
Signed-off-by: Qinqi Qu <quqinqi@linux.alibaba.com>
2023-12-07 11:59:07 +08:00
Archana Shinde
a661ac3a0e runtime-rs: Implement and use try_from for DiskConfig
Implement try_from trait function to convert runtime-rs BlockConfig
to cloud-hypervisor DiskConfig. This can allow for code reuse in the
future.

Fixes: #8581

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2023-12-06 12:10:34 -08:00
Fabiano Fidêncio
c14e3096c8
Merge pull request #8580 from amshinde/runtime-rs-clh-network-hotplug
runtime-rs: add network hotplug for clh
2023-12-06 20:50:04 +01:00
Gabriela Cervantes
56dddab04f metrics: Update command to run tensorflow resnet fp32 benchmark
This PR updates the command needed to run the tensorflow benchmark.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-12-06 17:02:10 +00:00
Gabriela Cervantes
62fdebeeb5 metrics: Update TensorFlow ResNet FP32 dockerfile
This PR updates the python version for the TensorFlow ResNet FP32
dockerfile so the benchmark can run without issues.

Fixes #8593

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-12-06 16:53:21 +00:00
GabyCT
3d149d3455
Merge pull request #8578 from GabyCT/topic/fixlinkconfig
docs: Update config containerd url link
2023-12-06 10:40:29 -06:00
Zvonko Kaiser
16380558e0 deployment: Create a stable overaly for kata-deploy
Fixes: #8508

Create a stable overlay for kata-deploy.yaml so we do not have to maintain two files, only one.
Single source for both. This is also preparation for the helm-overlay

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2023-12-06 14:23:22 +00:00
Huang Jianan
2a1fc29e84 dragonball: add unit test for vhost-user-fs
Add some test cases for vhost-user-fs function.

Signed-off-by: Beiyue <beiyue@linux.alibaba.com>
Signed-off-by: Huang Jianan <jnhuang@linux.alibaba.com>
2023-12-06 10:43:24 +08:00
Huang Jianan
d6cfbe9436 dragonball: support vhost-user-fs
This patch implements the virtio-fs device used for filesystem sharing
and heavily based on the vhost-user protocol.

This vhost-user-fs device defines 5 parameters:
  - path: vhost-user socket path
  - tag: mount tag used from the guest to mount the filesystem
  - req_num_queues: number of request virtqueues
  - queue_size: depth of each virtqueue
  - cache_size: cache window size for dax

This device needs to be defined before the VM instance is started,
which can be done through the dbs-cli tool with --fs option:
--fs '{
    "sock_path":"/path/to/virtiofs.socket",
    "tag":"myfs",
    "num_queues":1,
    "queue_size":1024,
    "cache_size":0,
    "thread_pool_size":1,
    "cache_policy":"auto",
    "writeback_cache":true,
    "no_open":true,
    "xattr":true,
    "drop_sys_resource":false,
    "mode":"vhostuser",
    "fuse_killpriv_v2":true,
    "no_readdir":false,
}'

Fixes: #8428

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
Signed-off-by: Huang Jianan <jnhuang@linux.alibaba.com>
2023-12-06 10:43:17 +08:00
Archana Shinde
955dec06da runtime-rs: add network hotplug for clh
This is required for clh to work with nerdtcl and docker.
This fixes the issues seen with nerdctl while starting a container.
Hoewever, container exit with docker is still broken due to an unrelated
issue.

Fixes: #8579

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2023-12-05 15:29:53 -08:00
Fabiano Fidêncio
b056683b7a
Merge pull request #8436 from Lu-Biao/main
image-builder: bugfix incorrect partition location
2023-12-06 00:10:06 +01:00