Commit Graph

11118 Commits

Author SHA1 Message Date
Fabiano Fidêncio
db77c9a438 tests: Make install_kata take care of the links
It makes the kata-containers installation more complete.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
13715db1f8 tests: Do not call install_check_metrics when installing kata
The `install_kata` function was moved from the metrics' `gha-run.sh`
file to the `common.bash` in the commit 3ffd48bc16, but I didn't notice
that it brought with it a call to `install_check_metrics`, which is
totally unrelated to installing Kata Containers.

Let's remove the call so the function is a little bit less specific, and
move the call to install_check_metrics to the metrics `gha-run.sh` file.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
e149a3c783
Merge pull request #7404 from fidencio/topic/cache-consider-changes-in-the-scripts-used-to-build-the-kernel
cache: kernel: Consider changes in tools/packaging/kernel
2023-07-21 15:05:01 +02:00
Fabiano Fidêncio
630634c5df ci: k8s: Group logs to make them easier to read
Otherwise it becomes really hard to find the info you're looking for.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 14:05:30 +02:00
Fabiano Fidêncio
228b30f31c ci: k8s: Gather node info during the cleanup
This will make our lives easier to debug issues with the CI.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 14:05:30 +02:00
Fabiano Fidêncio
81f99543ec ci: k8s: Cleanup cluster before deleting it
This will help us to in two fronts:
* catching possible issues related to kata-deploy cleanup
* do more (like, in the future, collect logs) after the tests run

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 14:05:30 +02:00
Fabiano Fidêncio
38a7b5325f packaging/tools: Add kata-debug
kata-debug is a tool that is used as part of the Kata Containers CI to gather
information from the node, in order to help debugging issues with Kata
Containers.

As one can imagine, this can be expanded and used outside of the CI context,
and any contribution back to the script is very much welcome.

The resulting container is stored at the [Kata Containers quay.io
space](https://quay.io/repository/kata-containers/kata-debug) and can
be used as shown below:
```sh
kubectl debug $NODE_NAME -it --image=quay.io/kata-containers/kata-debug:latest
```

Fixes: #7397

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 14:05:30 +02:00
Fabiano Fidêncio
a0fd41fd37
Merge pull request #7406 from fidencio/topic/merge-tarball-fix-version-yaml-not-found
kata-deploy: Properly get the path of the versions.yaml file
2023-07-21 14:04:18 +02:00
Fabiano Fidêncio
ae6e8d2b38 kata-deploy: Properly get the path of the versions.yaml file
We need to correctly get the full path of the versions.yaml file as part
of the merge-builds.sh script, as we do a `pushd` there and that leads
to a fail merging the artefacts as the `versions.yaml` file does not
exists in that path.

Fixes: #7405

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 12:02:11 +02:00
Fabiano Fidêncio
309e232553 cache: kernel: Consider changes in tools/packaging/kernel
Any change in the script used to build the kernel should invalidate the
cache.

Fixes: #7403

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 11:48:29 +02:00
GabyCT
f95a7896b1
Merge pull request #7394 from fidencio/topic/ship-VERSIOB-and-versions.yaml-as-part-of-release-tarball
kata-deploy: Add VERSION and versions.yaml to the final tarball
2023-07-20 14:38:21 -06:00
GabyCT
14025baafe
Merge pull request #7376 from GabyCT/topic/addcray
metrics: Add C-Ray performance test
2023-07-20 14:37:53 -06:00
GabyCT
b629f6a822
Merge pull request #7363 from GabyCT/topic/enabletensorflow
metrics: enable TensorFlow benchmark to be run on gha
2023-07-20 13:36:55 -06:00
Fabiano Fidêncio
59fdd69b85 kata-deploy: Add VERSION and versions.yaml to the final tarball
Let's make things simpler to figure out which version of Kata
Containers has been deployed, and also which artefacts come with it.

This will help us immensely in the future, for the TEEs use case, so we
can easily know whether we can deploy a specific guest kernel for a
specific host kernel.

Fixes: #7394

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-20 18:33:14 +02:00
Fabiano Fidêncio
5dddd7c5d1 release: Upload versions.yaml as part of the release
Although this file is far away from being a SBOM, it'll help folks to
easily visualise which components are part of a release, and even have
SBOMs generated from that.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-20 18:31:21 +02:00
Gabriela Cervantes
bad3ac84b0 metrics: Rename C-Ray to cpu performance tests
This PR renames C-Ray tests to cpu category.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-20 15:56:02 +00:00
Fabiano Fidêncio
87d99a71ec versions: Remove "kernel-experimental"
We've not been using nor shipping this kernel for a very long time.

Regardless, we're leaving behind the logic in the kernel scripts to
build it, in case it becomes necessary in the future.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-20 17:14:22 +02:00
Zvonko Kaiser
545de5042a vfio: Fix tests
Now with more elaborate checking of cold|hot plug ports
we needed to update some of the tests.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2023-07-20 13:42:44 +00:00
Zvonko Kaiser
62aa6750ec vfio: Added better handling of VFIO Control Devices
Depending on the vfio_mode we need to mount the
VFIO control device additionally into the container.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2023-07-20 13:42:42 +00:00
Fabiano Fidêncio
fe07ac662d
Merge pull request #7387 from GabyCT/topic/fixmemoryinsidec
metrics: Add function to memory inside container script
2023-07-20 10:06:15 +02:00
Zvonko Kaiser
dd422ccb69 vfio: Remove obsolete HotplugVFIOonRootBus
Removing HotplugVFIOonRootBus which is obsolete with the latest PCI
topology changes, users can set cold_plug_vfio or hot_plug_vfio either
in the configuration.toml or via annotations.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2023-07-20 07:25:40 +00:00
Zvonko Kaiser
114542e2ba s390x: Fixing device.Bus assignment
The device.Bus was reset if a specific combination of
configuration parameters were not met. With the new
PCIe topology this should not happen anymore

Fixes: #7381

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2023-07-20 07:24:26 +00:00
Alakesh Haloi
371a118ad0 agent: exclude symlinks from recursive ownership change
currently when fsGroup is used with direct-assign, kata agent
recursively changes ownership and permission for each file including
symlinks. However the problem with symlinks is, the permission of
the symlink itself may not be same as the underlying file. So while
doing recursive ownership and permission changes we should skip
symlinks.

Fixes: #7364
Signed-off-by: Alakesh Haloi <a_haloi@apple.com>
2023-07-19 20:42:55 -07:00
Gabriela Cervantes
e64edf41e5 metrics: Add tensorflow function in gha-run script
This PR adds the tensorflow function in gha-run script in order to
be triggered in the gha.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-19 21:31:51 +00:00
Gabriela Cervantes
67a6fff4f7 metrics: Enable tensorflow benchmark on gha
This PR enables the TensorFlow benchmark on gha for the kata metrics CI.

Fixes #7362

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-19 21:31:51 +00:00
GabyCT
c3f21c36f3
Merge pull request #7388 from dborquez/revert-commit-broke-checkmetrics-baseline-values
Revert "metrics: Replace backslashes used to escape double quoted key in jq expr"
2023-07-19 14:36:16 -06:00
David Esparza
01450deb6a
Revert "metrics: Replace backslashes used to escape double quoted key in jq expr."
This reverts commit 468f017e21.

Fixes: #7385

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-07-19 10:07:11 -06:00
Gabriela Cervantes
8430068058 metrics: Add function to memory inside container script
This PR adds function before function of the variables at the memory
inside container script in order to have uniformity across the script.

Fixes #7386

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-19 16:00:53 +00:00
Chao Wu
bbd3c1b6ab Dragonball: migrate dragonball-sandbox crates to Kata
In order to make it easier for developers to contribute to Dragonball,
we decide to migrate all dragonball-sandbox crates to Kata.

fixes: #7262

Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2023-07-19 19:41:57 +08:00
Chao Wu
7153b51578
Merge pull request #7372 from fidencio/topic/bump-virtiofsd-to-v1.7.0
versions: Bump virtiofsd to v1.7.0
2023-07-19 10:51:49 +08:00
GabyCT
8c662916ab
Merge pull request #7377 from dborquez/add_verbosity_to_blogbench
metrics: stop hypervirsor and shim at init_env stage
2023-07-18 15:57:54 -06:00
Fabiano Fidêncio
5f7da301fd
Merge pull request #7378 from fidencio/topic/ci-k8s-fix-source-path
ci: k8s: Adapt "source ..." to the new location of gha-run.sh
2023-07-18 22:30:55 +02:00
Fabiano Fidêncio
fad801d0fb ci: k8s: Adapt "source ..." to the new location of gha-run.sh
This is a follow up of 2ee2cd307b, which
changed the location of gha-run.sh

Fixes: #7373

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-18 21:26:41 +02:00
David Esparza
55e2f0955b
metrics: stop hypervirsor and shim at init_env stage
This PR kills the hypervisor and the kata shim in the
init_env stage prior to launch any metric test.
Additionally this PR adds info messages in the main blocks
of the blogbench test to help in debugging.

Fixes: #7366

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-07-18 12:05:29 -06:00
Gabriela Cervantes
556e663fce metrics: Add disk link to general metrics README
This PR adds the disk link information to the general metrics README.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-18 16:42:35 +00:00
Gabriela Cervantes
98c1217093 metrics: Add C-Ray README
This PR adds the C-Ray documentation at the README file.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-18 16:35:54 +00:00
Gabriela Cervantes
8e7d9926e4 metrics: Add C-Ray Dockerfile
This PR adds the C-Ray Dockerfile for kata metrics.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-18 16:33:55 +00:00
Gabriela Cervantes
e2ee769783 metrics: Add C-Ray performance test
This PR adds C-Ray performance test in order to be part of the kata
metrics CI.

Fixes #7375

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-18 16:32:23 +00:00
Fabiano Fidêncio
2011e3d72a
Merge pull request #7374 from fidencio/topic/ci-tdx-adjust-kubeconfig-path
ci: Move `tests/integration/gha-run.sh`  to `tests/integration/kuberentes/` ... and also remove KUBECONFIG from the tdx envs
2023-07-18 17:32:57 +02:00
Fabiano Fidêncio
8e09e04f48
Merge pull request #6788 from jepio/kernel-update-6.1-lts
versions: Update kernel to version v6.1.x
2023-07-18 17:29:21 +02:00
Chao Wu
935432c36d
Merge pull request #7352 from justxuewei/exec-hang
agent: Fix exec hang issues with a backgroud process
2023-07-18 23:02:18 +08:00
Fabiano Fidêncio
2ee2cd307b ci: k8s: Move gha-run.sh to the kubernetes dir
The file belongs there, as it's only used for k8s related tests.

Fixes: #7373

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-18 15:45:06 +02:00
Fabiano Fidêncio
88eaff5330 ci: tdx: Adjust KUBECONFIG
We don't need to export KUBECONFIG there.  Let's just make sure we have
the server correctly setup and avoid doing that.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-18 15:39:52 +02:00
Jeremi Piotrowski
c09e268a1b versions: Downgrade SEV(-SNP) kernel back to v5.19.x
CC-GPU seems to have issues with v6.1, so downgrade the kernels used for
SEV-SNP to a known-working version. It is worth mentioning that TDX is also
still on 5.19.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-18 15:29:46 +02:00
Fabiano Fidêncio
25d80fcec2
Merge pull request #6993 from zvonkok/kata-agent-init-mount
agent: Ignore already mounted dev/fs/pseudo-fs
2023-07-18 14:11:44 +02:00
Fabiano Fidêncio
4687f2bf9d
Merge pull request #7369 from fidencio/topic/gha-ci-bring-tdx-back
ci: k8s: Bring TDX tests back
2023-07-18 13:28:33 +02:00
Fabiano Fidêncio
6a7a323656 versions: Bump virtiofsd to v1.7.0
https://gitlab.com/virtio-fs/virtiofsd/-/releases/v1.7.0 was released
Today.

Fixes: #7371

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-18 12:33:13 +02:00
Fabiano Fidêncio
ac5f5353ba ci: k8s: Bring TDX tests back
Now that we have a new TDX machine plugged into our CI, let's re-enable
the TDX tests.

Fixes: #7368

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-18 10:33:43 +02:00
Jeremi Piotrowski
950b89ffac versions: Update kernel to version v6.1.38
Kernel v6.1.38 is the current latest LTS version, switch to it.  No
patches should be necessary. Some CONFIG options have been removed:

- CONFIG_MEMCG_SWAP is covered by CONFIG_SWAP and CONFIG_MEMCG
- CONFIG_ARCH_RANDOM is unconditionally compiled in
- CONFIG_ARM64_CRYPTO is covered by CONFIG_CRYPTO and ARCH=arm64

Fixes: #6086
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-18 10:04:21 +02:00
GabyCT
7729d82e6e
Merge pull request #7360 from GabyCT/topic/updategraldoc
metrics: Update machine learning documentation
2023-07-17 15:30:13 -06:00