Commit Graph

11051 Commits

Author SHA1 Message Date
Fabiano Fidêncio
e1a4040a6c
Merge pull request #7326 from fidencio/topic/gha-ci-add-cri-containerd-tests
ci: gha: Add cri-containerd tests (but still do not enable them)
2023-07-21 19:29:38 +02:00
Fabiano Fidêncio
6a59e227b6
Merge pull request #7399 from fidencio/topic/add-kata-debug
packaging/tools: Add kata-debug and use it as part of our CI
2023-07-21 17:05:27 +02:00
Fabiano Fidêncio
e91f5edba0 ci: cri-containerd: Fix default typo for testContainerStart()
It must but {1:-0}, instead of {1-0}.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
8b8aef09af ci: cri-containerd: Temporarily disable TestContainerSwap
The test is currently failing with GHA, and I don't think it makes sense
to block all the other tests to get merged while it's happening.

For now, let's disable it and re-enable it as soon as we have it
passing.

Reference: https://github.com/kata-containers/kata-containers/issues/7410

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
56767001cb ci: cri-containerd: Add namespace / uid to the pods
Otherwise crictl will fail to remove them with:
```
getting sandbox status of pod "$pod": metadata.Name, metadata.Namespace
or metadata.Uid is not in metadata "..."
```

A huge shout out to Steven Horsman for helping to debug this one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
a84773652c ci: cri-containerd: Always use sudo to call crictl
Otherwise we may get the following error:
```
time="2023-07-15T21:12:13Z" level=fatal msg="validate service connection: validate CRI v1 runtime API for endpoint \"unix:///run/containerd/containerd.sock\": rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial unix /run/containerd/containerd.sock: connect: permission denied\""
```

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
99ba86a1b2 ci: cri-containerd: Add /usr/local/go/bin to the PATH
Otherwise go is not picked up.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
7f3b309997 ci: cri-containerd: Add function before each function
We've been doing this for all files moved to this repo.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
fde22d6bce ci: cri-containerd: Assume podman is always used
For this set of tests, we'll always be using podman in order to avoid
having containerd pulled in by docker.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
9465a04963 ci: cri-containerd: Adapt "source ..." to this repo
Let's adapt what we "source" to the kata-containers repo.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
df8d144119 ci: cri-containerd: Remove CI variable
We always want to run the tests using as much debug as possible.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
f90570aef0 ci: cri-containerd: Remove unused runc_runtime_bin
The variable is not used anywhere in our tests.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
c3637039f4 ci: cri-containerd: Remove KILL_VMM_TEST env var
We don't need the env var, we just need to restrict the test according
to the KATA_HYPERVISOR used, as right now it's very specifict to QEMU.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
bc4919f9b2 ci: cri-containerd: Always run shim-v2 tests
We only have shim-v2 as the runtime type, so we always need to run tests
using it. :-)

We had to adjust the script in order to properly run the tests with the
current logic.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
f9e332c6db ci: cri-containerd: Stop cloning containerd
It's already done as part of the install_dependencies()

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
cfd662fee9 ci: cri-containerd: Remove ununsed SNAP_CI var
We don't support SNAP anymore, thus we can remove the var.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
d36c3395c0 ci: cri-containerd: Update copyright
As we're touching the file already, let's update its Copyright info.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
b5be8a4a8f ci: cri-containerd: Move integration-tests.sh as it was
Let's move the `integration/containerd/cri/integration-tests.sh` file
from the tests repo to this one.

The file has been moved as it is, it's not used, and in the following
commits we'll clean it up before actually using it.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
f2e00c95c0 ci: cri-containerd: Populate install_dependencies()
Let's install all the dependencies needed for running the
`cri-containerd` tests.

The list of dependencies we have are:
* From the system
  - build-essential
  - jq
  - podman-docker
* From our own repo
  - yq
  - go
* From GitHub projects
  - containerd
  - cri-tools

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
8979552527 versions: Add "latest" field for cri-tools
As we don't want to disrupt what we have on the `tests` repo, let's
create a "latest" entry and use that for the GitHub actions tests.

Once we deprecate the `tests` repo we can decide whether we want to
stick to using "latest" or switch back to "version".

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
1bbcbafa67 ci: Add clone_cri_container()
This function will simply clone containerd repo, specifically on a tag
we want to use to test.

This can be expanded for different projects, and it will be the case as
soon as we grow the tests.  But, for now, let's keep it simple.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
f66c68a2bf ci: Add install_cri_tools()
This function will install cri-tools in the host, and soon enough (as
part of this PR) we'll be using it to install cri-tools as part of the
cri-containerd tests.

I've decided to have this as part of the `common.bash` as other tests
that will be added in the future will require cri-tools to be installed
as well.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
4dd828414f ci: Add install_cri_containerd()
This function will install cri-containerd in the host, and soon enough
(as part of this PR) we'll be using it to install cri-containerd as part
of the cri-containerd tests.

I've decided to have this as part of the `common.bash` as other tests
that will be added in the future will require cri-containerd to be
installed as well.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
ad47d1b9f8 ci: Add download_github_project_tarball()
This function will hel us to get the tarball, from a github project,
that we're going to use as part of our tests.

Right now this is not used anywhere, but it'll soon enough (as part of
this series) be used to download the cri-containerd / cri-tools / cni
tarballs.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
788c562a95 ci: Add get_latest_patch_release_from_a_github_project()
This function will help us to get the latest patch release from a
GitHub project.

The idea behind this function is that we don't have to keep updating
versions.yaml that frequently (or worse, have it outdated as it
currently is), and always test against the latest patch release of a
given project's version that we care about.

Although right now this is not used anywhere, this will be used with the
coming cri-containerd tests, which will be part of this series.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
6742f3a898 ci: Use function before each install_go.sh function
We've been doing this for all files moved to this repo.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
5eacecffc3 ci: Adjust paths for install_go.sh
Let's adjust paths for what we source and the scripts we call, after
moving from the tests repo to this one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
8ed1595f96 ci: Update copyright for install_go.sh
As we're touching the file already, let's update its Copyright info.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
6123d0db2c ci: Move install_go.sh as it was
Let's move `.ci/install_go.sh` file from the tests repo to this one.

The file has been moved as it is, it's not used, and in the following
commits we'll clean it up before actually using it.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
8653be71b2 ci: Do not take cross-build into consideration for kata-arch.sh
Right now we'd need to import lib.sh just in order to get cross-build
information for rust, and it seems a little bit premature to do so at
this stage and only for rust.

Let's skip it and keep this transition simple.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
6a76bf92cb ci: Fix style / identation if kata-arch.sh
We've been using:
```
function foo() {
}
```

instead of
```
function foo()
{
}
```

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
72743851c1 ci: Add function before each kata-arch.sh function
We've been doing this for all files moved to this repo.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
9f6d4892c8 ci: Update copyright for kata-arch.sh
As we're touching the file already, let's update its Copyright info.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
6f73a72839 ci: Move kata-arch.sh as it was
Let's move `.ci/kata-arch.sh` file from the tests repo to this one.

The file has been moved as it is, it's not used, and in the following
commits we'll clean it up before actually using it.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
3615d73433 ci: Add get_from_kata_deps()
First of all, I'm 100% aware that I'm duplicating this function here as
I've copied it from the packaging stuff, and I'm not exactly proud of
that.

However, right now it seems a little bit premature to combine that set
of scripts with this set of scripts in a single one and make them used
by both pieces of our project.

Anyways, this functions helps to get information from the
`versions.yaml` file, and it'll be used as part of the cri-containerd
tests and a few others in the future.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
34779491e0 gha: kubernetes: Avoid declaring repo_root_dir
This is already declared as part of the `common.bash` file, so let's
just make sure we use it from there.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
f3738beaca tests: Use $HOME/go as fallback for $GOPATH
Considering that someone may want to run the tests locally, we shouldn't
rely on having GITHUB_WORKSPACE exported, and fallback to $HOME/go if
needed.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
b87ed27416 tests: Move ensure_yq to common.bash
As this function will be used by different scripts, let's move it to a
common place.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Jeremi Piotrowski
124e390333 tests: common: Fix quoting when globbing
When the glob star is inside quotes, there is only one iteration of the loop
and b holds all matches at once. Move the glob out of the quotes so that we
actually iterate over matched paths.

Fixes: #6543
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
db77c9a438 tests: Make install_kata take care of the links
It makes the kata-containers installation more complete.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
13715db1f8 tests: Do not call install_check_metrics when installing kata
The `install_kata` function was moved from the metrics' `gha-run.sh`
file to the `common.bash` in the commit 3ffd48bc16, but I didn't notice
that it brought with it a call to `install_check_metrics`, which is
totally unrelated to installing Kata Containers.

Let's remove the call so the function is a little bit less specific, and
move the call to install_check_metrics to the metrics `gha-run.sh` file.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 16:54:27 +02:00
Fabiano Fidêncio
e149a3c783
Merge pull request #7404 from fidencio/topic/cache-consider-changes-in-the-scripts-used-to-build-the-kernel
cache: kernel: Consider changes in tools/packaging/kernel
2023-07-21 15:05:01 +02:00
Fabiano Fidêncio
630634c5df ci: k8s: Group logs to make them easier to read
Otherwise it becomes really hard to find the info you're looking for.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 14:05:30 +02:00
Fabiano Fidêncio
228b30f31c ci: k8s: Gather node info during the cleanup
This will make our lives easier to debug issues with the CI.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 14:05:30 +02:00
Fabiano Fidêncio
81f99543ec ci: k8s: Cleanup cluster before deleting it
This will help us to in two fronts:
* catching possible issues related to kata-deploy cleanup
* do more (like, in the future, collect logs) after the tests run

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 14:05:30 +02:00
Fabiano Fidêncio
38a7b5325f packaging/tools: Add kata-debug
kata-debug is a tool that is used as part of the Kata Containers CI to gather
information from the node, in order to help debugging issues with Kata
Containers.

As one can imagine, this can be expanded and used outside of the CI context,
and any contribution back to the script is very much welcome.

The resulting container is stored at the [Kata Containers quay.io
space](https://quay.io/repository/kata-containers/kata-debug) and can
be used as shown below:
```sh
kubectl debug $NODE_NAME -it --image=quay.io/kata-containers/kata-debug:latest
```

Fixes: #7397

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 14:05:30 +02:00
Fabiano Fidêncio
a0fd41fd37
Merge pull request #7406 from fidencio/topic/merge-tarball-fix-version-yaml-not-found
kata-deploy: Properly get the path of the versions.yaml file
2023-07-21 14:04:18 +02:00
Fabiano Fidêncio
ae6e8d2b38 kata-deploy: Properly get the path of the versions.yaml file
We need to correctly get the full path of the versions.yaml file as part
of the merge-builds.sh script, as we do a `pushd` there and that leads
to a fail merging the artefacts as the `versions.yaml` file does not
exists in that path.

Fixes: #7405

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 12:02:11 +02:00
Fabiano Fidêncio
309e232553 cache: kernel: Consider changes in tools/packaging/kernel
Any change in the script used to build the kernel should invalidate the
cache.

Fixes: #7403

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-07-21 11:48:29 +02:00
GabyCT
f95a7896b1
Merge pull request #7394 from fidencio/topic/ship-VERSIOB-and-versions.yaml-as-part-of-release-tarball
kata-deploy: Add VERSION and versions.yaml to the final tarball
2023-07-20 14:38:21 -06:00