Commit Graph

11820 Commits

Author SHA1 Message Date
GabyCT
58e921eace
Merge pull request #7260 from fidencio/topic/add-nightly-jobs-follow-up-1
gha: ci: Follow up fixes for the nightly jobs
2023-07-06 15:45:13 -06:00
GabyCT
54da0d7c91
Merge pull request #7230 from GabyCT/topic/enabmemory
tests: Enable memory usage metrics tests
2023-07-06 14:30:56 -06:00
Fabiano Fidêncio
1a7bbcd398 gha: ci: Fix typo pull_requesst -> pull_request
Thanks David Esparza for pointing this one out.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 22:29:00 +02:00
Fabiano Fidêncio
ddf4afb961 gha: ci: Fix set-fake-pr-number job
It has to have steps declared, and we need to make it a dependency for
the nightly kata-containers-ci-on-push job.

Fixes: #7247

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 22:02:08 +02:00
Fabiano Fidêncio
8a0a66655d gha: ci: schedule expects a list, not a map
And because of that we need to declare '- cron', instead of 'cron'.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 22:02:08 +02:00
Fabiano Fidêncio
5c0269dc5a gha: ci: Add pr-number input to the correct job
It must have been an input for the AKS jobs, not the SNP one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 22:02:08 +02:00
Fabiano Fidêncio
de83cd9de7 gha: ci: Use $VAR instead of ${{ env.VAR }}
Otherwise we'll get the following error from the workflow:
```
The workflow is not valid. .github/workflows/ci-on-push.yaml (Line: 24,
Col: 20): Unrecognized named-value: 'env'. Located at position 1 within
expression: env.COMMIT_HASH .github/workflows/ci-on-push.yaml (Line: 25,
Col: 18): Unrecognized named-value: 'env'. Located at position 1 within
expression: env.PR_NUMBER
```

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 22:02:08 +02:00
Wainer Moschetta
1a4ae1ef47
Merge pull request #6953 from fidencio/topic/add-nightly-jobs
gha: Add nightly jobs
2023-07-06 14:50:10 -03:00
Gabriela Cervantes
6acce83e12 metrics: Fix the call to check_metrics function
This PR fixes the call to check_metrics function as KATA_HYPERVISOR
is not needed to be passed.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-06 17:22:49 +00:00
David Esparza
0bd21c173a
Merge pull request #7240 from dborquez/storing_metrics_artifacts
metrics: storing metrics workflow artifacts
2023-07-06 09:49:45 -06:00
Fabiano Fidêncio
152e2509ca
Merge pull request #7238 from fidencio/topic/gha-run-tests-on-specific-namespace
gha: k8s: Ensure tests are running on a specific namespace
2023-07-06 17:25:00 +02:00
Fabiano Fidêncio
e067d18333 gha: Add a nightly CI job
The idea is to mimic what's been done with Jenkins and the "Green CI"
effort, but now using our GHA and the GHA infrastructure.

Fixes: #7247

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 14:39:49 +02:00
Fabiano Fidêncio
7c0de8703c gha: k8s: Ensure tests are running on a specific namespace
Let's make sure we run our tests in a specific namespace, as in case of
any kind of issue, we will just get rid of the namespace itself, which
will take care of cleaning up any leftover from failing tests.

One important thing to mention is why we can get rid of the `namespace:
${namespace}` on the tests that are already using it, and let's do it in
parts:
* namespace: default
  We can easily get rid of this as that's the default namespace where
  pods are created, so it was a no-op so far.
* namespace: test-quota-ns
  My understanding is that we'd need this in order to get a clean
  namespace where we'd be setting a quota for.  Doing this in the
  namespace that's only used for tests should **not** cause any
  side-effect on the tests, as we're running those in serial and there's
  no other pods running on the `kata-containers-k8s-tests` namespace

Last but not least, we're not dynamically creating namespaces as the
tests are not running in parallel, **never**, not in the case of having
2 tests being ran at same time, neither in the case of having 2 jobs
being scheduled to the same machine.

Fixes: #6864

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 14:14:50 +02:00
Fabiano Fidêncio
80df87321f
Merge pull request #7246 from fidencio/topic/cc-gha-fix-measured-rootfs-builds-follow-up-again-follow-up
cc: gha: Avoid building twice the components set as "include"
2023-07-06 14:12:04 +02:00
Fabiano Fidêncio
106e305717 gha: Create a re-usable ci.yaml file
This is based on the `ci-on-push.yaml` file, and it's called from ther
The reason to split on a new file is that we can easily introduce a
`ci-nightly.yaml` file and re-use the `ci.yaml` file there as well.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 13:07:59 +02:00
Fabiano Fidêncio
0425c4fe7c cc: gha: Avoid building twice the components set as "include"
Those components are being built twice, one as part of the normal matrix
assets, and the second one as part of the include.

Fixes: #7235

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 12:55:11 +02:00
Fabiano Fidêncio
a2218f0a99
Merge pull request #7244 from fidencio/topic/cc-gha-fix-measured-rootfs-builds-follow-up-again
cc: gha: Export MEASURE_ROOTFS=yes for rootfs-image builds
2023-07-06 12:06:02 +02:00
Fabiano Fidêncio
cc3993d860 gha: Pass event specific info from the caller workflow
Let's ensure we're not relying, on any of the called workflows, on event
specific information.

Right now, the two information we've been relying on are:
* PR number, coming from github.event.pull_request.number
* Commit hash, coming from github.event.pull_request.head.sha

As we want to, in the future, add nightly jobs, which will be triggered
by a different event (thus, having different fields populated), we
should ensure that those are not used unless it's in the "top action"
that's trigerred by the event.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 11:23:17 +02:00
Fabiano Fidêncio
8d47e34558 cc: gha: Export MEASURE_ROOTFS=yes for rootfs-image builds
We need to export MEASURED_ROOTFS=yes for the rootfs-image builds, as
shown here[0], otherwise the root_hash.txt file won't be generated.

A huge thanks to Choi for quickly finding this out.

Fixes: #7235

[0]:
https://github.com/kata-containers/kata-containers/blob/CCv0/tools/osbuilder/image-builder/image_builder.sh#L507,

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 10:50:17 +02:00
David Esparza
4e396e7285
metrics: Add function keyword to to helper metrics functions
Use the 'function' keyword to prevent bash aliases from colliding
with other function's name.

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-07-05 20:59:21 -06:00
David Esparza
1ca17c2f70
metrics: storing metrics workflow artifacts
This PR enables storing metrics workflow artifacts in two
separated flavours: clh and qemu.

Fixes: #7239

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-07-05 20:57:10 -06:00
David Esparza
a3fc673121
Merge pull request #7181 from dborquez/add_blogbench_and_webtooling
metrics: Adds blogbench and webtool metrics tests
2023-07-05 20:37:33 -06:00
Fabiano Fidêncio
a40cd94a66
Merge pull request #7241 from fidencio/topic/cc-gha-fix-measured-rootfs-builds-follow-up
cc: gha: Matrix "include" cannot be a list
2023-07-05 23:23:11 +02:00
Fabiano Fidêncio
5751781879 cc: gha: Matrix "include" cannot be a list
TIL that when using "include" for a matrix we must duplicate the value
we're overriding for each element we want that to happen.

Fixes: #7235

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-05 21:46:51 +02:00
Gabriela Cervantes
5a61065ab7 checkmetrics: Add checkmetrics value for memory usage in qemu
This PR adds the checkmetrics value for memory usage in qemu.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-05 19:22:12 +00:00
Gabriela Cervantes
78086ed1fe checkmetrics: Add memory usage value for clh
This PR adds the memory usage value for clh.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-05 19:19:04 +00:00
Fabiano Fidêncio
2ae98093f6
Merge pull request #7237 from fidencio/topic/cc-gha-fix-measured-rootfs-builds
cc: gha: Pass MEASURED_ROOTFS to the artefacts build
2023-07-05 20:27:44 +02:00
Gabriela Cervantes
1c3dbafbf0 metrics: Fix function of how to retrieve multiple values
This PR fixes the function of how to add multiple values of pss memory.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-05 18:19:36 +00:00
Gabriela Cervantes
18968f428f metrics: Add function to have uniformity
This PR adds the function name before the function to have uniformity
across all the test.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-05 18:15:31 +00:00
Fabiano Fidêncio
8bc30b8a24 cc: gha: Pass MEASURED_ROOTFS to the artefacts build
Since the measured rootfs work has been merged to main, and then
brought in to the CCv0 via the weekly merge, we have introduced a few
regressions related to how we build it / use it.

This PR attempts to make sure the artefacts are properly built, using
GitHub Actions, so the feature can be used with the operator.

Fixes: #7235

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-05 17:10:39 +02:00
Fabiano Fidêncio
2cf48e4a55
Merge pull request #7234 from fidencio/topic/fix-cc-operator-uninstall-issue
cc: kata-deploy: Adapt the SNP's QEMU name
2023-07-05 13:39:35 +02:00
Fabiano Fidêncio
701a0defb6 cc: kata-deploy: Adapt the SNP's QEMU name
SNP's QEMU has changed its name some time ago and, due to that, we have
been leaving the new binary behind during the uninstall process, which
lead to the Operator hanging when uninstalling.

Fixes: #7233

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-05 12:04:14 +02:00
Fabiano Fidêncio
191071b10b
Merge pull request #7228 from stevenhorsman/update-dependencies
Update dependencies after guest-components repo updates
2023-07-05 08:55:39 +02:00
David Esparza
35d096b607
metrics: Adds blogbench and webtool metrics tests
This PR adds blogbench and webtooling metrics checks to this repo.
The function running the test intentionally returns zero, so
the test will be enabled in another PR once the workflow is
green.

Fixes: #7069

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-07-04 14:38:52 -06:00
Gabriela Cervantes
d8f90e89d5 metrics: Rename function at memory usage script
This PR renames the function name for the memory usage script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-04 19:58:09 +00:00
Gabriela Cervantes
b9d66e0d53 metrics: Fix double quotes variables in memory usage script
This PR usses double quotes in all the variables as well as general fixes
to the memory usage script in order to have uniformity.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-04 19:51:36 +00:00
Gabriela Cervantes
476a11194a tests: Enable memory usage metrics tests
This PR enables the memory usage metrics tests for kata CI.

Fixes #7229

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-04 16:11:54 +00:00
stevenhorsman
aadc68633e agent: Update image-rs
- Update location and version of image-rs after
the repo merge

Fixes: #7152
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-07-04 17:07:00 +01:00
stevenhorsman
6e67d52242 version: Update attestation-agent
- Update attestation-agent to reflect the new URL and version

Fixes: #7152
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-07-04 16:56:27 +01:00
stevenhorsman
1fc8f7040f rootfs: Update attestation-agent install logic
- Update attestation-agent install now the create
is in the guest-components repo

Fixes: #7152
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-07-04 16:55:54 +01:00
Fabiano Fidêncio
a25d5b9807
Merge pull request #7222 from jepio/fix-dragonball-check
gha: dragonball: Correctly propagate PATH update
2023-07-04 15:59:13 +02:00
Jeremi Piotrowski
b568c7f7d8 tests/integration: Provide default value for KATA_HOST_OS
Non AKS k8s tests (SEV/SNP/TDX) don't currently set KATA_HOST_OS, so provide a
default empty value for the variable so that those tests can run.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-04 14:28:29 +02:00
Fabiano Fidêncio
6d2e6ed7b6
Merge pull request #7217 from likebreath/0630/clh_v33.0
versions: Upgrade to Cloud Hypervisor v33.0
2023-07-04 12:52:26 +02:00
Jeremi Piotrowski
d6e96ea06d tests/integration: Use AzureLinux instead of Mariner
as OSSKU value, to get rid of this warning when creating the AKS cluster:

WARNING: The osSKU "AzureLinux" should be used going forward instead of
"CBLMariner" or "Mariner". The osSKUs "CBLMariner" and "Mariner" will
eventually be deprecated.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-04 12:49:07 +02:00
Jeremi Piotrowski
40c46c75ed tests/integration: Perform yq install in run_tests()
We only need to install in run_tests() so that the yq install is picked up by
kubernets/setup.sh as well. We also need to either use (sudo &&
INSTALL_IN_GOPATH=false) || (INSTALL_IN_GOPATH=true).

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-04 12:49:07 +02:00
Bin Liu
f214058b07
Merge pull request #7202 from wedsonaf/macros
Convert `is_allowed`, `ttrpc_error` and `sl` to functions
2023-07-04 14:23:08 +08:00
Peng Tao
f5658c7833
Merge pull request #7224 from fidencio/topic/gha-release-fix-hub-download
gha: release: Use a specific release of hub
2023-07-04 10:21:17 +08:00
GabyCT
5950df7d95
Merge pull request #7199 from GabyCT/topic/installchem
metrics: Add checkmetrics to gha-run.sh for metrics CI
2023-07-03 17:49:18 -06:00
Gabriela Cervantes
d8b8f7e94d metrics: Enable launch tests time metrics
This PR enables the launch tests metrics for kata CI.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-03 22:38:04 +00:00
Fabiano Fidêncio
72fd562bd6 gha: release: Use a specific release of hub
ideally we should never ever use hub again, and switch to a supported /
release tool instead.  However, in order to get v3.1.3 released, let's
just stick to the last released version of hub, as trying to get its
release is leading to:
```
curl -s "https://api.github.com/repos/github/hub/releases/latest"
{
  "message": "Moved Permanently",
  "url": "https://api.github.com/repositories/401025/releases/latest",
  "documentation_url": "https://docs.github.com/v3/#http-redirects"
}
```

And that breaks the release process. :-/

Fixes: #7223

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-03 22:00:55 +02:00