Commit Graph

298 Commits

Author SHA1 Message Date
Fabiano Fidêncio
fbc2a91ab5 gha: Cancel previous jobs if a PR is updated
Let's make sure we cancel previous runs, mainly as we have some of those
that take a lot of time to run, whenever the PR is updated.

This is based on the following stack overflow suggestion:
https://stackoverflow.com/questions/66335225/how-to-cancel-previous-runs-in-the-pr-when-you-push-new-commitsupdate-the-curre

This is very much needed as we don't want to wait for a long time to
have access to a runner because of other runners are still being used
performing a task that's meaningless due to the PR update.

Fixes: #7298

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-11 14:37:10 +02:00
Fabiano Fidêncio
b61b15aab6 Merge pull request #7259 from fidencio/topic/gha-restrict-job-run-according-to-files-touched
gha: Do not run all the tests if only docs are updated
2023-07-10 18:12:29 +02:00
Fabiano Fidêncio
1776b18fa0 gha: Do not run all the tests if only docs are updated
We should not go through the trouble of running all our tests on AKS /
Azure / baremetal machines in case a PR only changes our documentation.

Fixes: #7258

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-10 10:30:46 +02:00
Fabiano Fidêncio
0c1cbd01d8 gha: ci: after-push: Use github.sha to get the last commit reference
As we need to pass down the commit sha to the jobs that will be
triggered from the `push` event, we must be careful on what exactly
we're using there.

At first we were using ${{ github.ref }}, but this turns out to be the
**branch name**, rather than the commit hash.  In order to actually get
the commit hash, Let's use ${{ github.sha }} instead.

Fixes: #7247

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-10 09:39:33 +02:00
Fabiano Fidêncio
37a9556789 gha: ci: nightly: Use github.sha to get the last commit reference
As we need to pass down the commit sha to the jobs that will be
triggered from the `schedule` event, we must be careful on what exactly
we're using there.

At first we were using ${{ github.ref }}, but this turns out to be the
**branch name**, rather than the commit hash.  In order to actually get
the commit hash, Let's use ${{ github.sha }} instead, as described by
https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#

Fixes: #7247

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-10 09:39:26 +02:00
Fabiano Fidêncio
0ad298895e gha: ci: Fix refernce passed to checkout@v3
On cc3993d860 we introduced a regression,
where we started passing inputs.commit-hash, instead of
github.event.pull_request.head.sha. However, we have been setting
commit-hash to github.event.pull_request.sha, meaning that we're mssing
a `.head.` there.

github.event.pull_request.sha is empty for the pull_request_target
event, leading the CI to pull the content from `main` instead of the
content from the PR.

Fixes: #7247

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-07 17:55:11 +02:00
Fabiano Fidêncio
86904909aa gha: ci: Avoid using env also in the ci-nightly and payload-after-push
The latter workflow is breaking as it doesn't recognise ${GITHUB_REF},
the former would most likely break as well, but it didn't get triggered
yet.

The error we're facing is:
```
Determining the checkout info
  /usr/bin/git branch --list --remote origin/${GITHUB_REF}
  /usr/bin/git tag --list ${GITHUB_REF}
  Error: A branch or tag with the name '${GITHUB_REF}' could not be found
```

Fixes: #7247

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-07 14:46:30 +02:00
Fabiano Fidêncio
3e2b723487 Merge pull request #7263 from fidencio/topic/add-nightly-jobs-follow-up-3
gha: ci: More follow up fixes after adding a nightly CI
2023-07-07 13:58:26 +02:00
Fabiano Fidêncio
18bd2d6e4a Merge pull request #6839 from sprt/sprt/mariner-ci-tests
tests: Enable running k8s tests on Mariner
2023-07-07 13:36:28 +02:00
Fabiano Fidêncio
1d05b9cc71 gha: ci: Pass down secrets to ci-on-push / ci-nightly
We have to do this, otherwise we cannot log into azure.

This is a regression introduced by
106e305717.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-07 12:00:33 +02:00
Fabiano Fidêncio
c5b4164cb1 gha: ci: Fix tarball-suffix passed to the metrics tests
Instead of passing "-${{ inputs.tag }}-amd64", we must only pass
"-${{ inputs.tag }}".

This is a regression introduced by
106e305717.

Fixes: #7247

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-07 12:00:24 +02:00
Fabiano Fidêncio
11e3ccfa4d gha: ci: Avoid using env unless it's really needed
de83cd9de7 tried to solve an issue, but it
clearly seems that I'm using env wrongly, as what ended up being passed
as input was "$VAR", instead of the content of the VAR variable.

As we can simply avoid using those here, let's do it and save us a
headache.

Fixes: #7247

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-07 07:31:10 +02:00
Fabiano Fidêncio
1a7bbcd398 gha: ci: Fix typo pull_requesst -> pull_request
Thanks David Esparza for pointing this one out.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 22:29:00 +02:00
Fabiano Fidêncio
ddf4afb961 gha: ci: Fix set-fake-pr-number job
It has to have steps declared, and we need to make it a dependency for
the nightly kata-containers-ci-on-push job.

Fixes: #7247

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 22:02:08 +02:00
Fabiano Fidêncio
8a0a66655d gha: ci: schedule expects a list, not a map
And because of that we need to declare '- cron', instead of 'cron'.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 22:02:08 +02:00
Fabiano Fidêncio
5c0269dc5a gha: ci: Add pr-number input to the correct job
It must have been an input for the AKS jobs, not the SNP one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 22:02:08 +02:00
Fabiano Fidêncio
de83cd9de7 gha: ci: Use $VAR instead of ${{ env.VAR }}
Otherwise we'll get the following error from the workflow:
```
The workflow is not valid. .github/workflows/ci-on-push.yaml (Line: 24,
Col: 20): Unrecognized named-value: 'env'. Located at position 1 within
expression: env.COMMIT_HASH .github/workflows/ci-on-push.yaml (Line: 25,
Col: 18): Unrecognized named-value: 'env'. Located at position 1 within
expression: env.PR_NUMBER
```

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 22:02:08 +02:00
Wainer Moschetta
1a4ae1ef47 Merge pull request #6953 from fidencio/topic/add-nightly-jobs
gha: Add nightly jobs
2023-07-06 14:50:10 -03:00
Fabiano Fidêncio
e067d18333 gha: Add a nightly CI job
The idea is to mimic what's been done with Jenkins and the "Green CI"
effort, but now using our GHA and the GHA infrastructure.

Fixes: #7247

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 14:39:49 +02:00
Fabiano Fidêncio
106e305717 gha: Create a re-usable ci.yaml file
This is based on the `ci-on-push.yaml` file, and it's called from ther
The reason to split on a new file is that we can easily introduce a
`ci-nightly.yaml` file and re-use the `ci.yaml` file there as well.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 13:07:59 +02:00
Fabiano Fidêncio
cc3993d860 gha: Pass event specific info from the caller workflow
Let's ensure we're not relying, on any of the called workflows, on event
specific information.

Right now, the two information we've been relying on are:
* PR number, coming from github.event.pull_request.number
* Commit hash, coming from github.event.pull_request.head.sha

As we want to, in the future, add nightly jobs, which will be triggered
by a different event (thus, having different fields populated), we
should ensure that those are not used unless it's in the "top action"
that's trigerred by the event.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-06 11:23:17 +02:00
David Esparza
1ca17c2f70 metrics: storing metrics workflow artifacts
This PR enables storing metrics workflow artifacts in two
separated flavours: clh and qemu.

Fixes: #7239

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-07-05 20:57:10 -06:00
David Esparza
35d096b607 metrics: Adds blogbench and webtool metrics tests
This PR adds blogbench and webtooling metrics checks to this repo.
The function running the test intentionally returns zero, so
the test will be enabled in another PR once the workflow is
green.

Fixes: #7069

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-07-04 14:38:52 -06:00
Fabiano Fidêncio
a25d5b9807 Merge pull request #7222 from jepio/fix-dragonball-check
gha: dragonball: Correctly propagate PATH update
2023-07-04 15:59:13 +02:00
Fabiano Fidêncio
72fd562bd6 gha: release: Use a specific release of hub
ideally we should never ever use hub again, and switch to a supported /
release tool instead.  However, in order to get v3.1.3 released, let's
just stick to the last released version of hub, as trying to get its
release is leading to:
```
curl -s "https://api.github.com/repos/github/hub/releases/latest"
{
  "message": "Moved Permanently",
  "url": "https://api.github.com/repositories/401025/releases/latest",
  "documentation_url": "https://docs.github.com/v3/#http-redirects"
}
```

And that breaks the release process. :-/

Fixes: #7223

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-03 22:00:55 +02:00
Jeremi Piotrowski
477856c1e3 gha: dragonball: Correctly propagate PATH update
cargo/rust is installed in one step, we need to write the PATH update to
GITHUBENV so that it becomes visible in the next steps.

Fixes: #7221
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-03 17:05:12 +02:00
Aurélien Bombo
1c211cd730 gha: Swap asset/release in build matrix
This simply displays the asset name first in GH's UI, so that the
release name (always "test") is truncated rather than the asset name.
Makes things slightly easier to read.

e.g.

    build-asset (cloud-hypervisor-glibc, te...

instead of

    build-asset (test, cloud-hypervisor-gli...

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-06-30 12:51:40 -07:00
David Esparza
b2ce8b4d61 metrics: Add memory footprint tests to the CI
This PR adds memory foot print metrics to tests/metrics/density
folder.

Intentionally, each test exits w/ zero in all test cases to ensure
that tests would be green when added, and will be enabled in a
subsequent PR.

A workflow matrix was added to define hypervisor variation on
each job, in order to run them sequentially.

The launch-times test was updated to make use of the matrix
environment variables.

Fixes: #7066

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-06-30 09:52:27 -06:00
Aurélien Bombo
e0d6475b49 gha: Don't automatically trigger CI
We have GH configured so that manual approval is required for CI runs
triggered by outside contributors. However, because CI is triggered by
the `pull_request_target` event, this setting isn't being honored
(see [1]). This means that an attacker could trivially extracts secrets
by submitting a PR.

This change aims to mititgate this issue by preventing PRs from
triggering CI unless the `ok-to-test` label is set.

Note: For further context, we use the `pull_request_target` event and
manually check out the PR branch because it is the only way to both
access secrets and test incoming code changes.

Fixes: #7163

 [1]: https://docs.github.com/en/actions/managing-workflow-runs/approving-workflow-runs-from-public-forks

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-06-22 11:05:53 -07:00
Gabriela Cervantes
7185afc50e gha: Fix gha actions
This PR removes an unrecognized value located in one of the yamls for the
gha in order to make it work the CI again.

Fixes #7149

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-06-20 23:13:25 +00:00
David Esparza
fad3ac9f58 metrics: install kata and launch-times test
This PR installs kata static tarball on metrics runner
and run launch-times tests.

Fixes: #7049

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-06-20 13:58:09 -06:00
Gabriela Cervantes
6b08489301 gha: Fix format for run launchtimes metrics yaml
This PR fixes the format for the run launchtimes metrics yaml which
is causing to the workflow to fail.

Fixes #7130

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-06-16 22:00:36 +00:00
David Esparza
b16e0de734 gha: Add base branch on SHA on pull requst
The run-launchtimes-metrics workflow needs to get the commit ID
for the last commit to the head branch of the PR.

Fixes: #7116

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-06-15 13:11:33 -06:00
David Esparza
bc152b1141 gha: ci-on-push: Run metrics tests
This gh-workflow prints a simple msg, but is the base for future
PRs that will gradually add the jobs corresponding to the kata
metrics test.

Fixes: #7100

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2023-06-14 15:15:08 -06:00
Fabiano Fidêncio
ff38937246 Merge pull request #7087 from sprt/fix-gha-stage
gha: Fix `stage` definition in matrix
2023-06-13 12:17:25 +02:00
Chelsea Mafrica
1763b1f69f Merge pull request #7082 from jodh-intel/remove-snap
packaging: Remove snap package
2023-06-12 17:05:00 -07:00
Aurélien Bombo
0e2379909b gha: Fix stage definition in matrix
This defines `stage` as a list instead of a literal to fix the GHA CI.

Fixes: #7086

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-06-12 11:24:45 -07:00
James O. D. Hunt
8cb4238b46 packaging: Remove snap package
Nobody has volunteered to maintain the (currently broken) snap build, so
remove it.

Fixes: #6769.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2023-06-12 09:24:09 +01:00
Aurélien Bombo
9f7a45996c gha: Add rootfs-initrd-mariner build target
This adds the Mariner guest image build target to the list of assets
as preparation for #6839.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-06-09 11:36:42 -07:00
Aurélien Bombo
f28a62164a gha: Add cloud-hypervisor-glibc build target
This adds the glibc flavor of CLH to the list of assets as preparation
for #6839. Mariner Kata is only tested with glibc.

Fixes: #7026

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-06-09 11:35:50 -07:00
Fabiano Fidêncio
b50f62ce48 Merge pull request #6756 from arronwy/measured_rootfs
Port Measured rootfs feature from CCv0 branch to main
2023-06-09 12:35:05 +02:00
Aurélien Bombo
f487199edf gha: aks: Fix argument in call to gha-run.sh
Fixes: #7047

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-06-06 11:51:18 -07:00
Wang, Arron
f6afae9c73 packaging: Add rootfs-image-tdx-tarball target
Add rootfs-image-tdx target:
./tools/packaging/kata-deploy/local-build/kata-deploy-binaries.sh --build=rootfs-image-tdx
./opt/kata/share/kata-containers/kata-containers-tdx.img
./opt/kata/share/kata-containers/kata-ubuntu-latest-tdx.image

Fixes: #6674

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2023-06-06 12:34:20 +02:00
Aurélien Bombo
aab6030962 gha: aks: Extract run commands to a script
Github Actions reads and runs workflow files from the main branch,
rather than from the PR branch. This means that PRs that modify workflow
files aren't being tested with the updated workflows coming from the PR,
but rather with the old workflows from the main branch. AFAIK, this
behavior isn't avoidable for workflow files (but is for other scripts).

This makes it very hard to reliably test workflow changes before they're
actually merged into main and leads to issues that we have to hotifx
(see #6983, #6995).

This PR aims to mitigate that by extracting the commands used in
workflows to a separate script file. The way our CI is set up, those
script files are read from the PR branch and thus changes would be
reflected in the CI checks.

Fixes: #6971

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-06-02 10:22:35 -07:00
Jeremi Piotrowski
1c6d22c803 gha: aks: Use short SHA in cluster name
Full SHA is 40 characters, while AKS cluster name has a limit of 63. Trim the
SHA to 12 characters, which is widely considered to be unique enough and is
short enough to be used in the cluster name

Fixes: #7010
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-06-01 14:03:53 +02:00
Fabiano Fidêncio
aebd3b47d9 gha: aks: Ensure host_os is used everywhere needed
We added that to create the cluster name, but I forgot to add that to
the part we get the k8s config file, or to the part where we delete the
AKS cluster.

Fixes: #6999

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-31 20:50:55 +02:00
Fabiano Fidêncio
e01f75723a Merge pull request #6997 from singhwang/main
main | release: Standardize kata static file name
2023-05-31 15:22:30 +02:00
Fabiano Fidêncio
0c8282c224 gha: aks: Add the host_os as part of the aks cluster's name
We need to do so, otherwise we'll create two clusters for testing Cloud
Hypervisor with exactly the same name, one using Ubuntu, and one using
Mariner.

Fixes: #6999

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-31 05:20:04 +02:00
SinghWang
4b89a6bdac release: Standardize kata static file name
The string representing the architecture aarch64 and x86_64 need to be changed to arm64 and amd64 for the release.

Fixes: #6986
Signed-off-by: SinghWang <wangxin_0611@126.com>
2023-05-31 10:24:45 +08:00
Aurélien Bombo
03027a7399 gha: Fix Mariner cluster creation
While the Mariner Kata host is in preview, we need the `aks-preview`
extension to enable the `--workload-runtime KataMshvVmIsolation` flag.

Fixes: #6994

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2023-05-30 13:26:49 -07:00