Commit Graph

190 Commits

Author SHA1 Message Date
Fabiano Fidêncio
bd08d745f4 tests: metrics: Move metrics specific function to metrics gha-run.sh
`compress_metrics_results_dir()` is only used by the metrics GHA.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-12 20:56:55 +02:00
Fabiano Fidêncio
3ffd48bc16 tests: common: Move a few utility functions to common.bash
Those functions were originally introduced as part of the
`metrics/gha-run.sh` file, but those will be very hand at the time we
start adding more tests.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-12 20:55:05 +02:00
Gabriela Cervantes
7f961461bd tests: Add machine learning README
This PR adds machine learning README.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-12 16:37:15 +00:00
Fabiano Fidêncio
bb2ef4ca34 tests: Add function before each function
Let's just keep this standardised.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-12 18:36:09 +02:00
Gabriela Cervantes
063f7aa7cb tests: Add Pytorch Dockerfile
This PR adds Pytorch Dockerfile for kata metrics.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-12 16:34:17 +00:00
Fabiano Fidêncio
b6282f7053 Merge pull request #7255 from GabyCT/topic/memoryinsideenabled
metrics: Enable memory inside container metrics
2023-07-12 18:33:36 +02:00
Gabriela Cervantes
1af03b9b32 tests: Add Pytorch performance test
This PR adds Pytorch performance test for kata metrics.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-12 16:33:02 +00:00
Gabriela Cervantes
4cecd62370 tests: Add tensorflow Dockerfile
This PR adds the tensorflow Dockerfile.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-12 16:31:32 +00:00
Gabriela Cervantes
c4094f62c9 tests: Add metrics machine learning performance tests
This PR adds metrics machine learning performance tests like
Tensorflow and Pytorch.

Fixes #7313

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-12 16:28:25 +00:00
Jeremi Piotrowski
b9a63d66a4 Merge pull request #7297 from jepio/fix-mariner-cache
tools: Use a consistent target name when building mariner initrd
2023-07-12 13:43:47 +02:00
Fabiano Fidêncio
1ab99bd6bb Merge pull request #7276 from fidencio/topic/gha-debug-gha-tests-start
gha: ci: Gather info about the node / pods
2023-07-12 12:35:10 +02:00
Fabiano Fidêncio
8c9d08e872 gha: ci: Gather info about the node / pods
This is a very simple addition, that should be expanded by
https://github.com/kata-containers/kata-containers/pull/7185, and it's
targetting gathering more info that will help us to debug CI failures.

Fixes: #7296

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-12 08:04:37 +02:00
Gabriela Cervantes
ce54e43ebe metrics: Update memory usage script
This PR updates memory usage script by applying the clean_env_ctr at the main
in order to avoid failures of leaving certain processes not removed.

Fixes #7302

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-11 17:03:25 +00:00
Jeremi Piotrowski
307cfc8f7a tools: Use a consistent target name when building mariner initrd
Currently a mixture of cbl-mariner and mariner is used when creating the
mariner initrd. The kata-static tarball has mariner in the name, but the
jenkins url uses cbl-mariner. This breaks cache usage.

Use mariner as the target name throughout the build, so that caching works.

Fixes: #7292
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-11 14:17:14 +02:00
Gabriela Cervantes
310e069f73 checkmetrics: Enable checkmetrics for memory inside test
This PR enables the checkmetrics to include the memory inside
container test.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-10 17:05:13 +00:00
Yushuo
28c29b248d bugfix: plus default_memory when calculating mem size
We've noticed this caused regressions with the k8s-oom tests, and then
decided to take a step back and do this in the same way it was done
before 67972ec48a.

Moreover, this step back is also more reasonable in terms of the
controlling logic.

And by doing this we can re-enable the k8s-oom.bats tests, which is done
as part of this PR.

Fixes: #7271
Depends-on: github.com/kata-containers/tests#5705

Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
2023-07-10 15:53:04 +08:00
Fabiano Fidêncio
38f0aaa516 Revert "gha: k8s: dragonball: Skip k8s-number-cpus"
This reverts commit a79505b667.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-08 14:43:49 +02:00
Fabiano Fidêncio
828a721838 gha: k8s: dragonball: Skip k8s-oom
Let's skip the k8s-oom, as the test is currently failing.

We've an issue opened for that, and we'll be working on re-enabling it
as soon as possible.

Reference:
https://github.com/kata-containers/kata-containers/issues/7271

Fixes: #7253

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-08 14:27:49 +02:00
Fabiano Fidêncio
a79505b667 gha: k8s: dragonball: Skip k8s-number-cpus
Let's skip the k8s-number-cpus, as the test is currently failing.

We've an issue opened for that, and we'll be working on re-enabling it
as soon as possible.

Reference:
https://github.com/kata-containers/kata-containers/issues/7270

Fixes: #7253

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-08 14:27:42 +02:00
Gabriela Cervantes
2be342023b checkmetrics: Add memory usage inside container value for qemu
This PR adds the memory usage inside container value for qemu.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-07 16:28:28 +00:00
Gabriela Cervantes
6ca34f949e checkmetrics: Add memory inside container value for clh
Add memory inside container value for clh.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-07 16:28:28 +00:00
Gabriela Cervantes
6c68924230 metrics: Enable memory inside container metrics
This PR will enable the memory inside container metrics for the Kata CI.

Fixes #7254

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-07 16:28:28 +00:00
Fabiano Fidêncio
48c3cec1f4 Merge pull request #7243 from sprt/ensure-cluster-no-exist
gha: k8s: Ensure cluster doesn't exist before creating it
2023-07-07 14:03:41 +02:00
Fabiano Fidêncio
18bd2d6e4a Merge pull request #6839 from sprt/sprt/mariner-ci-tests
tests: Enable running k8s tests on Mariner
2023-07-07 13:36:28 +02:00
Aurélien Bombo
c45f646b9d gha: k8s: Ensure cluster doesn't exist before creating it
The cluster cleanup step will sometimes fail to run, meaning the next
run would fail in the cluster creation step. This PR addresses that.

Example: https://github.com/kata-containers/kata-containers/actions/runs/5349582743/jobs/9867845852

Fixes: #7242

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-07-06 15:06:30 -07:00
GabyCT
54da0d7c91 Merge pull request #7230 from GabyCT/topic/enabmemory
tests: Enable memory usage metrics tests
2023-07-06 14:30:56 -06:00
Gabriela Cervantes
6acce83e12 metrics: Fix the call to check_metrics function
This PR fixes the call to check_metrics function as KATA_HYPERVISOR
is not needed to be passed.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-06 17:22:49 +00:00
David Esparza
0bd21c173a Merge pull request #7240 from dborquez/storing_metrics_artifacts
metrics: storing metrics workflow artifacts
2023-07-06 09:49:45 -06:00
Fabiano Fidêncio
7c0de8703c gha: k8s: Ensure tests are running on a specific namespace
Let's make sure we run our tests in a specific namespace, as in case of
any kind of issue, we will just get rid of the namespace itself, which
will take care of cleaning up any leftover from failing tests.

One important thing to mention is why we can get rid of the `namespace:
${namespace}` on the tests that are already using it, and let's do it in
parts:
* namespace: default
  We can easily get rid of this as that's the default namespace where
  pods are created, so it was a no-op so far.
* namespace: test-quota-ns
  My understanding is that we'd need this in order to get a clean
  namespace where we'd be setting a quota for.  Doing this in the
  namespace that's only used for tests should **not** cause any
  side-effect on the tests, as we're running those in serial and there's
  no other pods running on the `kata-containers-k8s-tests` namespace

Last but not least, we're not dynamically creating namespaces as the
tests are not running in parallel, **never**, not in the case of having
2 tests being ran at same time, neither in the case of having 2 jobs
being scheduled to the same machine.

Fixes: #6864

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 14:14:50 +02:00
David Esparza
4e396e7285 metrics: Add function keyword to to helper metrics functions
Use the 'function' keyword to prevent bash aliases from colliding
with other function's name.

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-07-05 20:59:21 -06:00
David Esparza
1ca17c2f70 metrics: storing metrics workflow artifacts
This PR enables storing metrics workflow artifacts in two
separated flavours: clh and qemu.

Fixes: #7239

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-07-05 20:57:10 -06:00
Gabriela Cervantes
5a61065ab7 checkmetrics: Add checkmetrics value for memory usage in qemu
This PR adds the checkmetrics value for memory usage in qemu.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-05 19:22:12 +00:00
Gabriela Cervantes
78086ed1fe checkmetrics: Add memory usage value for clh
This PR adds the memory usage value for clh.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-05 19:19:04 +00:00
Gabriela Cervantes
1c3dbafbf0 metrics: Fix function of how to retrieve multiple values
This PR fixes the function of how to add multiple values of pss memory.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-05 18:19:36 +00:00
Gabriela Cervantes
18968f428f metrics: Add function to have uniformity
This PR adds the function name before the function to have uniformity
across all the test.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-05 18:15:31 +00:00
David Esparza
35d096b607 metrics: Adds blogbench and webtool metrics tests
This PR adds blogbench and webtooling metrics checks to this repo.
The function running the test intentionally returns zero, so
the test will be enabled in another PR once the workflow is
green.

Fixes: #7069

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-07-04 14:38:52 -06:00
Gabriela Cervantes
d8f90e89d5 metrics: Rename function at memory usage script
This PR renames the function name for the memory usage script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-04 19:58:09 +00:00
Gabriela Cervantes
b9d66e0d53 metrics: Fix double quotes variables in memory usage script
This PR usses double quotes in all the variables as well as general fixes
to the memory usage script in order to have uniformity.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-04 19:51:36 +00:00
Gabriela Cervantes
476a11194a tests: Enable memory usage metrics tests
This PR enables the memory usage metrics tests for kata CI.

Fixes #7229

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-04 16:11:54 +00:00
Jeremi Piotrowski
b568c7f7d8 tests/integration: Provide default value for KATA_HOST_OS
Non AKS k8s tests (SEV/SNP/TDX) don't currently set KATA_HOST_OS, so provide a
default empty value for the variable so that those tests can run.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-04 14:28:29 +02:00
Jeremi Piotrowski
d6e96ea06d tests/integration: Use AzureLinux instead of Mariner
as OSSKU value, to get rid of this warning when creating the AKS cluster:

WARNING: The osSKU "AzureLinux" should be used going forward instead of
"CBLMariner" or "Mariner". The osSKUs "CBLMariner" and "Mariner" will
eventually be deprecated.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-04 12:49:07 +02:00
Jeremi Piotrowski
40c46c75ed tests/integration: Perform yq install in run_tests()
We only need to install in run_tests() so that the yq install is picked up by
kubernets/setup.sh as well. We also need to either use (sudo &&
INSTALL_IN_GOPATH=false) || (INSTALL_IN_GOPATH=true).

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-04 12:49:07 +02:00
Gabriela Cervantes
d8b8f7e94d metrics: Enable launch tests time metrics
This PR enables the launch tests metrics for kata CI.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-03 22:38:04 +00:00
Gabriela Cervantes
0502354b42 checkmetrics: Add checkmetrics json for qemu
This PR adds checkmetrics json file for qemu metrics.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-03 16:47:03 +00:00
Gabriela Cervantes
b481ef1883 makefile: Add -buildvcs=false flag to go build
This PR adds the -buildvcs=false flag to the go build of checkmetrics.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-03 16:42:51 +00:00
Gabriela Cervantes
e94aaed3c7 ci_worker: Add checkmetrics ci worker for cloud hypervisor
This PR adds the checkmetrics ci worker file for cloud hypervisor in
order to check the boot times limit.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-03 16:42:51 +00:00
Gabriela Cervantes
917576e6fb metrics: Add double quotes in all variables
This PR adds double quotes in all variables to have uniformity across
all the gha-run.sh script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-03 16:42:50 +00:00
Gabriela Cervantes
cc8f0a24e4 metrics: Add checkmetrics to gha-run.sh for metrics CI
This PR adds checkmetrics installation for gha-run.sh in order to compare
results limits as part of the metrics CI.

Fixes #7198

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-03 16:41:31 +00:00
Aurélien Bombo
80c78eadce tests: Use baked-in kernel with Mariner
Mariner ships a bleeding-edge kernel that might be ahead of upstream, so
we use that to guarantee compatibility with the host.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-06-30 12:51:40 -07:00
Aurélien Bombo
532755ce31 tests: Build Mariner rootfs initrd
* Adds a new `rootfs-initrd-mariner` build target.
 * Sets the custom initrd path via annotation in `setup.sh` at test
   time.
 * Adapts versions.yaml to specify a `cbl-mariner` initrd variant.
 * Introduces env variable `HOST_OS` at deploy time to enable using a
   custom initrd.
 * Refactors the image builder so that its caller specifies the desired
   guest OS.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-06-30 12:51:40 -07:00