Commit Graph

11118 Commits

Author SHA1 Message Date
Fabiano Fidêncio
96e9374d4b dragonball: Don't fail if a request asks for more CPUs than allowed
Let's take the same approach of the go runtime, instead, and allocate
the maximum allowed number of vcpus instead.

Fixes: #7270

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-08 15:50:23 +02:00
Fabiano Fidêncio
38f0aaa516 Revert "gha: k8s: dragonball: Skip k8s-number-cpus"
This reverts commit a79505b667.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-08 14:43:49 +02:00
Fabiano Fidêncio
828a721838 gha: k8s: dragonball: Skip k8s-oom
Let's skip the k8s-oom, as the test is currently failing.

We've an issue opened for that, and we'll be working on re-enabling it
as soon as possible.

Reference:
https://github.com/kata-containers/kata-containers/issues/7271

Fixes: #7253

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-08 14:27:49 +02:00
Fabiano Fidêncio
a79505b667 gha: k8s: dragonball: Skip k8s-number-cpus
Let's skip the k8s-number-cpus, as the test is currently failing.

We've an issue opened for that, and we'll be working on re-enabling it
as soon as possible.

Reference:
https://github.com/kata-containers/kata-containers/issues/7270

Fixes: #7253

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-08 14:27:42 +02:00
Fabiano Fidêncio
275c84e7b5 Revert "agent: fix the issue of exec hang with a backgroud process"
This reverts commit 25d2fb0fde.

The reason we're reverting the commit is because it to check whether
it's the cause for the regression on devmapper tests.

Fixes: #7253
Depends-on: github.com/kata-containers/tests#5705

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-08 14:27:40 +02:00
Gabriela Cervantes
2be342023b checkmetrics: Add memory usage inside container value for qemu
This PR adds the memory usage inside container value for qemu.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-07 16:28:28 +00:00
Gabriela Cervantes
6ca34f949e checkmetrics: Add memory inside container value for clh
Add memory inside container value for clh.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-07 16:28:28 +00:00
Gabriela Cervantes
6c68924230 metrics: Enable memory inside container metrics
This PR will enable the memory inside container metrics for the Kata CI.

Fixes #7254

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-07 16:28:28 +00:00
Fabiano Fidêncio
b7c58320a5
Merge pull request #7267 from fidencio/topic/add-nightly-jobs-follow-up-5
gha: ci: Fix refernce passed to checkout@v3
2023-07-07 18:26:44 +02:00
Fabiano Fidêncio
0ad298895e gha: ci: Fix refernce passed to checkout@v3
On cc3993d860 we introduced a regression,
where we started passing inputs.commit-hash, instead of
github.event.pull_request.head.sha. However, we have been setting
commit-hash to github.event.pull_request.sha, meaning that we're mssing
a `.head.` there.

github.event.pull_request.sha is empty for the pull_request_target
event, leading the CI to pull the content from `main` instead of the
content from the PR.

Fixes: #7247

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-07 17:55:11 +02:00
Fabiano Fidêncio
48d9f8769e
Merge pull request #7264 from fidencio/topic/add-nightly-jobs-follow-up-4
gha: ci: Avoid using env also in the ci-nightly and payload-after-push
2023-07-07 17:10:43 +02:00
Fabiano Fidêncio
86904909aa gha: ci: Avoid using env also in the ci-nightly and payload-after-push
The latter workflow is breaking as it doesn't recognise ${GITHUB_REF},
the former would most likely break as well, but it didn't get triggered
yet.

The error we're facing is:
```
Determining the checkout info
  /usr/bin/git branch --list --remote origin/${GITHUB_REF}
  /usr/bin/git tag --list ${GITHUB_REF}
  Error: A branch or tag with the name '${GITHUB_REF}' could not be found
```

Fixes: #7247

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-07 14:46:30 +02:00
Fabiano Fidêncio
48c3cec1f4
Merge pull request #7243 from sprt/ensure-cluster-no-exist
gha: k8s: Ensure cluster doesn't exist before creating it
2023-07-07 14:03:41 +02:00
Fabiano Fidêncio
3e2b723487
Merge pull request #7263 from fidencio/topic/add-nightly-jobs-follow-up-3
gha: ci: More follow up fixes after adding a nightly CI
2023-07-07 13:58:26 +02:00
Fabiano Fidêncio
18bd2d6e4a
Merge pull request #6839 from sprt/sprt/mariner-ci-tests
tests: Enable running k8s tests on Mariner
2023-07-07 13:36:28 +02:00
Zvonko Kaiser
f72cb2fc12 agent: Remove shadowed function, add slog-term
Remove shadowed get_mounts(), added slog-term as a new crate,
slog can directly log to stdout and we can capture output
in the test-cases that are created in the function to be tested.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2023-07-07 11:28:14 +00:00
Fabiano Fidêncio
1d05b9cc71 gha: ci: Pass down secrets to ci-on-push / ci-nightly
We have to do this, otherwise we cannot log into azure.

This is a regression introduced by
106e305717.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-07 12:00:33 +02:00
Fabiano Fidêncio
c5b4164cb1 gha: ci: Fix tarball-suffix passed to the metrics tests
Instead of passing "-${{ inputs.tag }}-amd64", we must only pass
"-${{ inputs.tag }}".

This is a regression introduced by
106e305717.

Fixes: #7247

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-07 12:00:24 +02:00
Fabiano Fidêncio
fa0f9954a1
Merge pull request #7261 from fidencio/topic/add-nightly-jobs-follow-up-2
gha: ci: Avoid using env unless it's really needed
2023-07-07 10:13:25 +02:00
Zvonko Kaiser
07810bf71f agent: Ignore already mounted dev/fs/pseudo-fs
Using an initrd and setting KATA_INIT=yes meaning we're using the kata-agent
as the init process we need to make sure that the agent is not segfaulting
if mounts are already happened. Some workloads need to configure several
things in the initrd before the kata-agent starts which involves having
/proc or /sys already mounted.

Fixes: #6992

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2023-07-07 07:36:04 +00:00
Fabiano Fidêncio
11e3ccfa4d gha: ci: Avoid using env unless it's really needed
de83cd9de7 tried to solve an issue, but it
clearly seems that I'm using env wrongly, as what ended up being passed
as input was "$VAR", instead of the content of the VAR variable.

As we can simply avoid using those here, let's do it and save us a
headache.

Fixes: #7247

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-07 07:31:10 +02:00
Aurélien Bombo
c45f646b9d gha: k8s: Ensure cluster doesn't exist before creating it
The cluster cleanup step will sometimes fail to run, meaning the next
run would fail in the cluster creation step. This PR addresses that.

Example: https://github.com/kata-containers/kata-containers/actions/runs/5349582743/jobs/9867845852

Fixes: #7242

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-07-06 15:06:30 -07:00
GabyCT
58e921eace
Merge pull request #7260 from fidencio/topic/add-nightly-jobs-follow-up-1
gha: ci: Follow up fixes for the nightly jobs
2023-07-06 15:45:13 -06:00
GabyCT
54da0d7c91
Merge pull request #7230 from GabyCT/topic/enabmemory
tests: Enable memory usage metrics tests
2023-07-06 14:30:56 -06:00
Fabiano Fidêncio
1a7bbcd398 gha: ci: Fix typo pull_requesst -> pull_request
Thanks David Esparza for pointing this one out.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 22:29:00 +02:00
Fabiano Fidêncio
ddf4afb961 gha: ci: Fix set-fake-pr-number job
It has to have steps declared, and we need to make it a dependency for
the nightly kata-containers-ci-on-push job.

Fixes: #7247

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 22:02:08 +02:00
Fabiano Fidêncio
8a0a66655d gha: ci: schedule expects a list, not a map
And because of that we need to declare '- cron', instead of 'cron'.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 22:02:08 +02:00
Fabiano Fidêncio
5c0269dc5a gha: ci: Add pr-number input to the correct job
It must have been an input for the AKS jobs, not the SNP one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 22:02:08 +02:00
Fabiano Fidêncio
de83cd9de7 gha: ci: Use $VAR instead of ${{ env.VAR }}
Otherwise we'll get the following error from the workflow:
```
The workflow is not valid. .github/workflows/ci-on-push.yaml (Line: 24,
Col: 20): Unrecognized named-value: 'env'. Located at position 1 within
expression: env.COMMIT_HASH .github/workflows/ci-on-push.yaml (Line: 25,
Col: 18): Unrecognized named-value: 'env'. Located at position 1 within
expression: env.PR_NUMBER
```

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 22:02:08 +02:00
Wainer Moschetta
1a4ae1ef47
Merge pull request #6953 from fidencio/topic/add-nightly-jobs
gha: Add nightly jobs
2023-07-06 14:50:10 -03:00
Gabriela Cervantes
6acce83e12 metrics: Fix the call to check_metrics function
This PR fixes the call to check_metrics function as KATA_HYPERVISOR
is not needed to be passed.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-06 17:22:49 +00:00
David Esparza
0bd21c173a
Merge pull request #7240 from dborquez/storing_metrics_artifacts
metrics: storing metrics workflow artifacts
2023-07-06 09:49:45 -06:00
Fabiano Fidêncio
152e2509ca
Merge pull request #7238 from fidencio/topic/gha-run-tests-on-specific-namespace
gha: k8s: Ensure tests are running on a specific namespace
2023-07-06 17:25:00 +02:00
Fabiano Fidêncio
e067d18333 gha: Add a nightly CI job
The idea is to mimic what's been done with Jenkins and the "Green CI"
effort, but now using our GHA and the GHA infrastructure.

Fixes: #7247

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 14:39:49 +02:00
Fabiano Fidêncio
7c0de8703c gha: k8s: Ensure tests are running on a specific namespace
Let's make sure we run our tests in a specific namespace, as in case of
any kind of issue, we will just get rid of the namespace itself, which
will take care of cleaning up any leftover from failing tests.

One important thing to mention is why we can get rid of the `namespace:
${namespace}` on the tests that are already using it, and let's do it in
parts:
* namespace: default
  We can easily get rid of this as that's the default namespace where
  pods are created, so it was a no-op so far.
* namespace: test-quota-ns
  My understanding is that we'd need this in order to get a clean
  namespace where we'd be setting a quota for.  Doing this in the
  namespace that's only used for tests should **not** cause any
  side-effect on the tests, as we're running those in serial and there's
  no other pods running on the `kata-containers-k8s-tests` namespace

Last but not least, we're not dynamically creating namespaces as the
tests are not running in parallel, **never**, not in the case of having
2 tests being ran at same time, neither in the case of having 2 jobs
being scheduled to the same machine.

Fixes: #6864

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 14:14:50 +02:00
Fabiano Fidêncio
106e305717 gha: Create a re-usable ci.yaml file
This is based on the `ci-on-push.yaml` file, and it's called from ther
The reason to split on a new file is that we can easily introduce a
`ci-nightly.yaml` file and re-use the `ci.yaml` file there as well.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 13:07:59 +02:00
Fabiano Fidêncio
cc3993d860 gha: Pass event specific info from the caller workflow
Let's ensure we're not relying, on any of the called workflows, on event
specific information.

Right now, the two information we've been relying on are:
* PR number, coming from github.event.pull_request.number
* Commit hash, coming from github.event.pull_request.head.sha

As we want to, in the future, add nightly jobs, which will be triggered
by a different event (thus, having different fields populated), we
should ensure that those are not used unless it's in the "top action"
that's trigerred by the event.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 11:23:17 +02:00
David Esparza
4e396e7285
metrics: Add function keyword to to helper metrics functions
Use the 'function' keyword to prevent bash aliases from colliding
with other function's name.

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-07-05 20:59:21 -06:00
David Esparza
1ca17c2f70
metrics: storing metrics workflow artifacts
This PR enables storing metrics workflow artifacts in two
separated flavours: clh and qemu.

Fixes: #7239

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-07-05 20:57:10 -06:00
David Esparza
a3fc673121
Merge pull request #7181 from dborquez/add_blogbench_and_webtooling
metrics: Adds blogbench and webtool metrics tests
2023-07-05 20:37:33 -06:00
Gabriela Cervantes
5a61065ab7 checkmetrics: Add checkmetrics value for memory usage in qemu
This PR adds the checkmetrics value for memory usage in qemu.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-05 19:22:12 +00:00
Gabriela Cervantes
78086ed1fe checkmetrics: Add memory usage value for clh
This PR adds the memory usage value for clh.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-05 19:19:04 +00:00
Gabriela Cervantes
1c3dbafbf0 metrics: Fix function of how to retrieve multiple values
This PR fixes the function of how to add multiple values of pss memory.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-05 18:19:36 +00:00
Gabriela Cervantes
18968f428f metrics: Add function to have uniformity
This PR adds the function name before the function to have uniformity
across all the test.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-05 18:15:31 +00:00
David Esparza
35d096b607
metrics: Adds blogbench and webtool metrics tests
This PR adds blogbench and webtooling metrics checks to this repo.
The function running the test intentionally returns zero, so
the test will be enabled in another PR once the workflow is
green.

Fixes: #7069

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-07-04 14:38:52 -06:00
Gabriela Cervantes
d8f90e89d5 metrics: Rename function at memory usage script
This PR renames the function name for the memory usage script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-04 19:58:09 +00:00
Gabriela Cervantes
b9d66e0d53 metrics: Fix double quotes variables in memory usage script
This PR usses double quotes in all the variables as well as general fixes
to the memory usage script in order to have uniformity.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-04 19:51:36 +00:00
Gabriela Cervantes
476a11194a tests: Enable memory usage metrics tests
This PR enables the memory usage metrics tests for kata CI.

Fixes #7229

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-07-04 16:11:54 +00:00
Fabiano Fidêncio
a25d5b9807
Merge pull request #7222 from jepio/fix-dragonball-check
gha: dragonball: Correctly propagate PATH update
2023-07-04 15:59:13 +02:00
Jeremi Piotrowski
b568c7f7d8 tests/integration: Provide default value for KATA_HOST_OS
Non AKS k8s tests (SEV/SNP/TDX) don't currently set KATA_HOST_OS, so provide a
default empty value for the variable so that those tests can run.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-04 14:28:29 +02:00