Commit Graph

12371 Commits

Author SHA1 Message Date
Fabiano Fidêncio
ce926439f0
Merge pull request #7443 from fidencio/topic/CCv0-converge-build-and-payload-scripts-follow-up-3
CC| cc: cache: Enable more cached components back and ensure shim-v2 can be properly cached
2023-07-25 20:32:02 +02:00
Eric Ernst
5385ddc560
Merge pull request #7365 from alakesh/symlink-fix
agent: exclude symlinks from recursive ownership change
2023-07-25 11:27:48 -07:00
Gabriela Cervantes
6177a0db3e metrics: Add env files for FIO
This PR adds the env files for FIO for kata metrics.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-25 17:48:45 +00:00
Hyounggyu Choi
1093e71cc5 CCv0: Remove cc from kernel for IBM SE image build
This is a quick fix for the error on IBM SE image build.

Fixes: #7444

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-07-25 19:46:31 +02:00
Gabriela Cervantes
a45900324d metrics: Add fio exec
This PR adds fio exec for the FIO benchmark.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-25 17:36:08 +00:00
Gabriela Cervantes
ea198fddcc metrics: Add FIO runner k8s
Add program to execute FIO workloads using k8s.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-25 17:34:29 +00:00
Gabriela Cervantes
8f7ef41c14 metrics: Add FIO vendor code
This PR adds the FIO vendor code.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-25 17:24:29 +00:00
Gabriela Cervantes
6293c17bde metrics: Add FIO benchmark for metrics tests
This PR adds the FIO benchmark scripts and resources for the metrics
tests section.

Fixes #7441

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-25 16:36:33 +00:00
Fabiano Fidêncio
ad8c96b6c0 cc: cache: Enable more cached components after rebuild was done
Let's re-enabled caching for the following components, as those were
rebuilt with the new prefix:
* cc-rootfs-image
* cc-rootfs-initrd
* cc-tdx-rootfs-image
* cc-tdx-td-shim
* cc-sev-rootfs-initrd

"cc-se-image" was part of the list, but we never had a target for it.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 18:33:27 +02:00
Fabiano Fidêncio
2213660bf3 cc: cache: shim-v2: Allow root_hashes to be downloaded
We should not return, in case cache is not used, before actually
downloading the root_hash_*.txt provided by the other components,
otherwise the job used to do the caching will always fail.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 18:33:27 +02:00
Fabiano Fidêncio
7f1226ae2b
Merge pull request #7440 from fidencio/topic/CCv0-converge-build-and-payload-scripts-follow-up-2
CC | Adjust cache URL for TDX components
2023-07-25 18:33:08 +02:00
Fabiano Fidêncio
cdf04e5018
Merge pull request #7437 from jepio/fix-sev-kernel-cache
cache: kernel: Fix kernel caching
2023-07-25 18:10:03 +02:00
GabyCT
7a3b55ce67
Merge pull request #7432 from ManaSugi/runk/doc-docker
runk: Add Docker guide to README
2023-07-25 09:56:02 -06:00
GabyCT
c1bd527163
Merge pull request #7430 from GabyCT/topic/fixjson
metrics: General improvements to json.bash script
2023-07-25 09:45:53 -06:00
Fabiano Fidêncio
6efd684a46
Merge pull request #7408 from fidencio/topic/kata-deploy-add-SHIMS-and-SHIM_DEFAULT-as-env
kata-deploy: Allow shim creation based on what's passed to the daemonset
2023-07-25 16:56:46 +02:00
Fabiano Fidêncio
5b82268d2c
Merge pull request #7436 from jepio/vfio-gha
gha: ci: Add skeleton of vfio job
2023-07-25 14:44:04 +02:00
Manabu Sugimoto
ff4cfcd8a2 runk: Add Docker guide to README
`runk` can launch containers using Docker, so add the guide
to it's README.

```sh
$ sudo dockerd --experimental --add-runtime="runk=/usr/local/bin/runk"
$ sudo docker run -it --rm --runtime runk busybox echo hello runk
hello runk
```

Fixes: #7431

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2023-07-25 20:10:49 +09:00
Fabiano Fidêncio
471e23cb12 cc: kernel-tdx: Ensure we try the cache for the specific CC version
Otherwise we'd have to build the component every single time as the main
version is different from the CC one.

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 13:06:43 +02:00
Fabiano Fidêncio
3f309fad01 cc: qemu-tdx: Ensure we try the cache for the specific CC version
Otherwise we'd have to build the component every single time as the main
version is different from the CC one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 13:01:15 +02:00
Fabiano Fidêncio
d03685004e cc: tdvf: Ensure we try the cache for the specific CC version
Otherwise we'd have to build the component every single time as the main
version is different from the CC one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 13:01:15 +02:00
Fabiano Fidêncio
0749022f8c cc: clh: Ensure we try the cache for the specific CC version
Otherwise we'd have to build the component every single time as the main
version is different from the CC one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 12:41:17 +02:00
Jeremi Piotrowski
c8ac56569a cache: kernel: Harmonize commit with fetching side
kata-deploy-binaries.sh uses the last commit in
tools/packaging/static-build/kernel for its version check, while the cache
generation uses tools/packaging/kernel. Use tools/packaging/static-build/kernel
as $kata_config_version is already part of the version string and covers any
changes to tools/packaging/kernel.

Fixes: #7403
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-25 12:23:05 +02:00
Jeremi Piotrowski
81775ab1b3 cache: kernel: Fix SEV kernel caching
The SEV kernel cache calls create_cache_asset() twice, once for the kernel and
once for modules. Both calls need to use the same version string, otherwise the
second call overwrites the "latest" file of the first one and the cache is not
used.

Fixes: #7403
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-25 11:58:19 +02:00
Jeremi Piotrowski
717f775f30 gha: ci: Add skeleton of vfio job
This job will run on a nested virt capable Azure VM (improving test
concurrency). This is just a placeholder while we adapt the test to GHA.

Fixes: #6555
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-25 11:13:04 +02:00
Manabu Sugimoto
b9f100b391 agent,libs: Remove unused 'mut' keywords
Remove unused `mut` because the agent compilation fails
when the rust compiler is >= 1.71. This is related to #7425

Fixes: #7438

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2023-07-25 17:41:08 +09:00
Fabiano Fidêncio
19a86aa072
Merge pull request #7434 from fidencio/topic/CCv0-converge-build-and-payload-scripts-follow-up-1
gha: Fix typo in the workflow name
2023-07-25 10:02:11 +02:00
Fabiano Fidêncio
a40ea43413 gha: Fix typo in the workflow name
Kudos to Magnus who has a better vision than I do.
exprimental -> experimental

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 09:56:18 +02:00
Fabiano Fidêncio
eb82e0d4b9
Merge pull request #7402 from fidencio/topic/CCv0-converge-build-and-payload-scripts
CCv0 | Ensure kata-deploy scripts from CCv0 are as close to main as possible
2023-07-25 08:45:16 +02:00
Fabiano Fidêncio
a56f96bb2b kata-deploy: Allow shim creation based on what's passed to the daemonset
Instead of hardcoding shims as part of the script, let's ensure we can
allow them to be created based on environment variables passed to the
daemonset.

This change brings no functionality change as the default values in the
daemonset are exactly what has been used as part of the scripts.

Fixes: #7407

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 08:30:00 +02:00
Fabiano Fidêncio
5ce0b4743f
Merge pull request #7382 from zvonkok/vfio-ap-debug
s390x: Fixing device.Bus assignment
2023-07-25 08:26:25 +02:00
Fabiano Fidêncio
068e535b9d runtime: tdx: Adjust QEMU TDX path
We need to use qemu-system-x86_64-tdx-experimental instead of
qemu-system-x86_64-tdx.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:39:52 +02:00
Fabiano Fidêncio
7204b991e7 kata-deploy-binaries: kernel_cache: Take module_dir into account
`module_dir` has been passed to the function but was never assigned to a
var, leading to errors when trying to use it.

Fixes: #7416

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit d4eba36980)
2023-07-25 00:19:26 +02:00
Fabiano Fidêncio
b8abd6bfee kata-deploy-binaries: Adjust TDVF edk2 tarball name
We must use "edk2-staging-tdx" instead of "edk2-tdx".  The reason for
that is versions diverging between main and CCv0.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:18:35 +02:00
Fabiano Fidêncio
0f022d5771 guest-image: Update kernel_module_dir to main sev kernel
As we're building SEV kernel from the main branch, we can stop relying
on the path produced by the one from the CCv0 branch (which is now
removed).

Fixes: #7422

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:18:35 +02:00
Fabiano Fidêncio
344921849c kata-deploy-binaries: Temporarily disable using cached components
We need to rebuild those with the appropriate path.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:21 +02:00
Fabiano Fidêncio
507a89bb32 gha: cc-payload: Adjust to using main componenets
Again, it'll make our lives easier in the near future.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:21 +02:00
Fabiano Fidêncio
ef6c0be984 kata-depkoy-binarues: Add tarballs from main to the cc target
Same as the others, it'll help us in the merges.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
20a523f81b kata-deloy-binaries: Get rid of cc_prefix
We'll be using prefix (/opt/kata) from now on, as it simplifies things
on our side.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
4d0b319a8b kata-deploy-binaries: Remove CC OVMF / TDVF
Let's just rely on whatever we have on main.  The big execption here is
TDVF, but we have a big note saying to not update the version n this
branch.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
8d1e1d4b0a kata-deploy-binaries: Remove CC kernel builds
We can simply rely on those coming from main.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
3fa936e492 kata-deploy-binaires: Remove CC virtiofsd build
We can simply ship the one from main.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
f62a88f179 kata-deploy-binaries: Remove CC hypervisor builds
We can just rely on the hypervisors builds from `main`, with the TDX one
being the only discrepancy here.

However, we have a big note in the versions.yaml to **not** update the
TDX hypervisor versions on this branch, so we should be good.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
6f552b010c kata-deploy: Make sure kata-deploy handles kata-deploy-cc content
This will also help us immensely on main -> CCv0 merges

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:20 +02:00
Fabiano Fidêncio
4e883fc5be versions: Converge to the same asset names used on main
This will make things easier in the future `main -> CCv0` merges.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-25 00:12:16 +02:00
David Esparza
b11d618a3f
Merge pull request #7413 from fidencio/topic/release-publish-builder-images
release: Mention the container images used to build the project
2023-07-24 15:46:31 -06:00
Fabiano Fidêncio
56fdeb1247
Merge pull request #7417 from fidencio/topic/kata-deploy-binaries-cached-kernel-fix
kata-deploy-binaries: kernel_cache: Take module_dir into account
2023-07-24 22:26:09 +02:00
Gabriela Cervantes
4a5ab38f16 metrics: General improvements to json.bash script
This PR adds general improvements like putting function before function
name and consistency in how we declare variables and so on to have
uniformity across the metrics scripts.

Fixes #7429

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-24 16:51:38 +00:00
Fabiano Fidêncio
d4eba36980 kata-deploy-binaries: kernel_cache: Take module_dir into account
`module_dir` has been passed to the function but was never assigned to a
var, leading to errors when trying to use it.

Fixes: #7416

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-24 18:19:13 +02:00
Fabiano Fidêncio
b7c9867d60 release: Mention the container images used to build the project
This is a small step towards build reproducibility.

Fixes: #7412

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-24 18:01:57 +02:00
Wainer Moschetta
2e9853c761
Merge pull request #7427 from fidencio/topic/gha-port-nydus-tests-follow-up-1
ci: nydus: Fix typo in "source"
2023-07-24 11:20:05 -03:00