Commit Graph

60 Commits

Author SHA1 Message Date
stevenhorsman
d06dadd8ef docs: Spelling updates
Either fixing typos, or including program/repo name in
backticks

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2026-03-19 10:22:54 +00:00
Lukáš Doktor
5250d4bacd ci.ocp: Use 0.0.0-dev tagged helm chart
in CI we are testing the latest kata-deploy, which requires the latest
helm chart. The previous query doesn't work anymore, but these days we
should be able to rely on the "0.0.0-dev" tag and on helm to print the
to-be-installed version into console.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2026-01-27 14:58:46 +01:00
Lukáš Doktor
971b096a1f ci.ocp: Update cleanup.sh to cope with helm deployment
replaces the old kata-deploy and uses "helm uninstall" instead.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2026-01-27 07:59:13 +01:00
Lukáš Doktor
272ff9c568 ci.ocp: Add notes about where to get other podvm images
I keep struggling finding the debug images, let's include them in the
peer-pods-azure.sh script so people can find them easier.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2026-01-27 07:59:12 +01:00
Fabiano Fidêncio
94adc58342 tests: Ensure helm secret for kata-deploy installation is cleaned up
Every now and then, in case a failure happens, helm leaves the secret
behind without cleaning it up, leading to issues in the consecutive
runs.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2025-10-23 11:15:13 +02:00
Lukáš Doktor
5038578fba ci.ocp: Install helm in local dir
in CI helm is not yet installed and we don't have root access. Let's use
the current dir, which should be writable, and --no-sudo option to
install it.

Note when helm is installed it should not change anything and simply use
the syste-wide installation.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-10-21 06:28:36 +02:00
Lukáš Doktor
bdb0afc4e0 ci.ocp: Fix incorrectly quoted argument
with the shellcheck fixes we accidentally quoted the "-n NAMESPACE"
argument where we should have used array instead, which lead to oc
considering this as a pod name and returning error.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-10-14 17:59:33 +02:00
Lukáš Doktor
f891f340bc ci.ocp: Use helm to install kata
which is the current supported way to deploy kata-containers directly.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-10-14 17:59:33 +02:00
Lukáš Doktor
5c14d2956a ci.ocp: Avoid unsupported "git --revision"
the git version in CI doesn't support "git clone --revision", workaround
it by using fetch directly.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-09-23 09:29:06 +02:00
Lukáš Doktor
346ebd0ff9 ci.ocp: Allow to set CAA_IMAGE
we might want to provide different CAA_IMAGE (repo) to reproduce issues.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-09-20 06:57:54 +02:00
Lukáš Doktor
bf90ccaf75 ci.ocp: Allow to set/provide PP_IMAGE_ID
to be able to test with older or custom peer-pod image.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-09-20 06:57:54 +02:00
Lukáš Doktor
b7143488d9 ci.ocp: Allow to set CAA TAG
to allow re-running with older CAA tag for bisection/reproduction.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-09-20 06:57:54 +02:00
Lukáš Doktor
12c5e0f33f ci.ocp: Log more details on failure
recently we got ErrImagePull, having more details should help analyzing
issues.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-09-20 06:57:54 +02:00
Lukáš Doktor
7565c881e6 ci.ocp: Log variables in bash-friendly format
this should simplify copy&paste of the values from logs.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-09-20 06:57:54 +02:00
Lukáš Doktor
a300b6b9a9 ci.ocp: Allow to set operator/caa commits
this can help reproducing or bisecting issues related to operator/caa
versions.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-09-20 06:57:53 +02:00
Lukáš Doktor
67ee9f3425 ci.ocp: Improve logging of extra new resources
this script relies on temporary subscriptions and won't cleanup any
resources. Let's improve the logging to better describe what resources
were created and how to clean them, if the user needs to do so.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-05-21 11:02:36 +02:00
Lukáš Doktor
32dbc5d2a9 ci.ocp: Use SCRIPT_DIR to allow execution from any folder
We used hardcoded "ci/openshift-ci/cluster" location which expects this
script to be only executed from the root. Let's use SCRIPT_DIR instead
to allow execution from elsewhere eg. by user bisecting a failed CI run.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-05-21 10:30:03 +02:00
Lukáš Doktor
0e4fb62bb4 ci.ocp: Retry first az command as login takes time to propagate
In CI we hit problem where just after `az login` the first `az
network vnet list` command fails due to permission. We see
"insufficient permissions" or "pending permissions", suggesting we should
retry later. Manual tests and successful runs indicate we do have the
permissions, but not immediately after login.

Azure docs suggest using extra `az account set` but still the
propagation might take some time. Add a loop retrying
the first command a few times before declaring failure.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-05-21 10:28:01 +02:00
Lukáš Doktor
c203d7eba6 ci.ocp: Set peer-pods-azure license
We forgot to add the license header when introducing this test.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-05-20 17:03:48 +02:00
Lukáš Doktor
b97b20295b ci.ocp: Make peer-pods setup executable
set permissions of the peer-pods-azure.sh script to executable

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-05-20 17:03:48 +02:00
Steve Horsman
a6d1dc7df3 Merge pull request #10940 from ldoktor/peer-pods
ci.ocp: Add peer-pods setup script
2025-04-29 15:57:30 +01:00
Lukáš Doktor
bfdf4e7a6a ci.ocp: Add peer-pods setup script
this script will be used in a new OCP integration pipeline to monitor
basic workflows of OCP+peer-pods.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-04-15 12:13:22 +02:00
Lukáš Doktor
009aa6257b ci.ocp: Override default runtimeclass CPU resources
some of the e2e tests spawn a lot of workers which are mainly idle, but
the scheduler fails to schedule them due to cpu resource overcommit. For
our testing we are more focused on having actual pods running than the
speed of the scheduled pods so let's increase the amount of schedulable
pods by decreasing the default cpu requests.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-04-02 10:30:40 +02:00
Lukáš Doktor
d708866b2a ci.ocp: shellcheck various fixes
various manual fixes.

Related to: #10951

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-03-19 12:26:28 +01:00
Lukáš Doktor
02deb1d782 ci: shellcheck SC2248
SC2248 (style): Prefer double quoting even when variables don't contain
special characters, might result in arguments difference, shouldn't in
our cases.

Related to: #10951

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-03-19 12:26:16 +01:00
Lukáš Doktor
6552ac41e0 ci: shellcheck SC2086
SC2086 Double quote to prevent globbing and word splitting, might break
places where we deliberately use word splitting, but we are not using it
here.

Related to: #10951

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-03-19 12:26:08 +01:00
Lukáš Doktor
154a4ddc00 ci: shellcheck SC2292
SC2292 (style): Prefer [[ ]] over [ ] for tests in Bash/Ksh. This might
result in different handling of globs and some ops which we don't use.

Related to: #10951

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-03-19 12:26:03 +01:00
Lukáš Doktor
667e26036c ci: shellcheck SC2250
Treat the SC2250 require-variable-braces in CI. There are no functional
changes.

Related to: #10951

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-03-19 12:25:44 +01:00
Wainer Moschetta
8c2d1b374c Merge pull request #10892 from ldoktor/webhook
ci: Change the way we modify runtimeclass in webhook
2025-03-12 12:32:45 -03:00
stevenhorsman
2df3e5937a ci/openshift-ci: Fix script error
The space was missing before `]`, so fix this and also
swtich to double square brackets and variable braces

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-03-04 09:39:10 +00:00
stevenhorsman
67bfd4793e shellcheck: Fix shellcheck SC2242
> Can only exit with status 0-255. Other data should be written to stdout/stderr.

Switch exit -1 to exit 1

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-03-04 09:39:01 +00:00
stevenhorsman
c5ff513e0b shellcheck: Fix shellcheck SC2068
> Double quote array expansions to avoid re-splitting elements

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-03-04 09:35:46 +00:00
stevenhorsman
58672068ff shellcheck: Fix shellcheck SC2145
> Argument mixes string and array. Use * or separate argument.

- Swap echos for printfs and improve formatting
- Replace $@ with $*
- Split arrays into separate arguments

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-03-04 09:35:46 +00:00
Lukáš Doktor
d0ef78d3a4 ci: Change the way we modify runtimeclass in webhook
previously we used to deploy the webhook and then modified the cm from
our ci/openshift-ci/ script to the desired value, but sometimes it
happens that the webhook pod starts before we modify the cm and keeps
using the default value.

Let's change the approach and modify the deployments in-place. The only
cons is it leaves the git dirty, but since this script is only supposed
to be used in ci it should be safe.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2025-02-18 11:39:22 +01:00
Lukáš Doktor
2f7d34417a ci.ocp: Use the official python:3 container for sanity
Fedora F40 removed python3 from the base container, to avoid such issues
let's rely on the latest and greates official python container.

Fixes: #10497

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-11-08 07:16:30 +01:00
Lukáš Doktor
820e000f1c ci.ocp: Sort images according to git
The quay.io registry returns the tags sorted alphabetically and doesn't
seem to provide a way to sort it by age. Let's use "git log" to get all
changes between the commits and print all tags that were actually
pushed.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-10-01 16:08:00 +02:00
Lukáš Doktor
8355eee9f5 ci: Reorder webhook deployment
in b9d88f74ed the `runtime_class` CM was
added which overrides the one we previously set. Let's reorder our logic
to first deploy webhook and then override the default CM in order to use
the one we really want.

Since we need to change dirs we also have to use realpath to ensure the
files are located well.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-09-24 17:01:28 +02:00
Lukáš Doktor
b08c019003 ci.ocp: Ensure we smoke-test with the right runtime class
we do encourage people to set the KATA_RUNTIME, but it is only used by
the webhook. Let's define it in the main `test.sh` and use it in the
smoke test to ensure the user-defined runtime is smoke-tested rather
than hard-coded kata-qemu one.

Related to: #9804

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-06-20 11:15:02 +02:00
Lukáš Doktor
699376c535 ci.ocp: Switch base to centos-9
Centos8 is EOL and repos are not available anymore. Centos9 contains the
same packages and should do well as a base for testing.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-06-06 09:03:17 +02:00
Lukáš Doktor
f994f79078 ci.ocp: Add steps to reproduce/bisect CI runs
in case the upstream CI fails it's useful to pin-point the PR that
caused the regression. Currently openshift-ci does not allow doing that
from their setup but we can mimic the setup on our infrastructure and
use the available kata-deploy-ci images to find the first failing one.
To help with that add a few helper scripts and a howto.

Fixes: #9228

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-05-16 20:20:05 +02:00
Lukáš Doktor
a556ad7e01 ci.ocp: Document how to run openshift-tests with kata
document the ocp pipeline.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-05-16 20:15:32 +02:00
Lukáš Doktor
ea081bd882 ci.ocp: Add webhook cleanup
cleanup the webhook resources as well.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-05-16 20:15:31 +02:00
Lukáš Doktor
b8382cea88 ci.ocp: Increase the MCP update time
updating the machine config takes even longer than 1200s, use 60m to be
sure everything is updated.

Fixes: #9338

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-04-03 15:01:29 +02:00
Lukáš Doktor
46e62eecb1 ci.ocp: Log the full grepped line rather than the expected msg
we are grepping for an expected message but it might contain extra bits
of information fruitful for later debugging. Let's include it in the
output and the full log in case of an error.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-03-12 17:03:46 +01:00
Lukáš Doktor
7ff2eb508e ci.ocp: Increase the mcp update timeout
we're hitting this timeout quite often, looks like newer OCP takes
longer to reconfigure. Increase the timeout to 1200.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-03-12 16:38:04 +01:00
Lukáš Doktor
cc02329fd1 ci.ocp: Add a cleanup script
This script doesn't serve as a complete cleanup, but it can be used as a
best-effort cleaner between deploying different versions of
kata-containers on the same OCP cluster.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-03-12 16:38:04 +01:00
Lukáš Doktor
b811ee0650 ci.ocp: Allow to override the kata-deploy image
sometimes we want to test a different than the latest image (eg. when
verifying a PR via ghcr images or when bisecting a failure over older
builds). Let's add a KATA_DEPLOY_IMAGE variable for that while keeping
the latest image by default.

Fixes: #9228

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-03-12 16:38:04 +01:00
Lukáš Doktor
2936503b24 ci.ocp: Always replace the kata-deploy image in OCP pipeline
previously we only replaced the image when the previously defined one
matched the "old_img". This is good to avoid modifying developers custom
changes, but it might lead to hard-to-debug issues when the image stays
different. Let's ensure we always replace the image with the one we
asked for.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-03-12 16:38:04 +01:00
Lukáš Doktor
6525c94065 ci.ocp: Add a workaround to optionally enable skip_mount_home
the latest upstream kata-containers requires the skip_mount_home to be
enabled, which is default on OCP 4.14+ but disabled on OCP 4.13-. Let's
use a "WORKAROUND_9206_CRIO" (called by kata-containers GH issue)
variable to allow users to enable this treatement when needed.

Related to: #9206

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-03-12 16:38:04 +01:00
Lukáš Doktor
739d627b4e ci.ocp: Turn selinux relabel failures into warnings
Instead of failing the pipeline let's proceed with an error message that
selinux setup failed so, in case of a later failure, we know what might
have caused it while keeping the coverage in case of a false setup
issue.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-03-12 16:38:04 +01:00