Commit Graph

14124 Commits

Author SHA1 Message Date
Lukáš Doktor
492ee1513f
ci: Ensure we check the latest workflow run in gatekeeper
with multiple iterations/reruns we need to use the latest run of each
workflow. For that we can use the "run_id" and only update results of
the same or newer run_ids.

To do that we need to store the "run_id". To avoid adding individual
attributes this commit stores the full job object that contains the
status, conclussion as well as other attributes of the individual jobs,
which might come handy in the future in exchange for slightly bigger
memory overhead (still we only store the latest run of required jobs
only).

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-07-25 10:23:55 +02:00
Lukáš Doktor
30d429ec6a
ci: Add extra gatekeeper debug output to stderr
which might be useful to assess the amount of querries.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-07-25 10:23:55 +02:00
Lukáš Doktor
6348917f87
ci: Check required lables before checking tests in gatekeeper
some tests require certain labels before they are executed. When our PR
is not labeled appropriately the gatekeeper detects skipped required
tests and reports a failure. With this change we add "required-labeles"
to the tests mapping and check the expected labels first informing the
user about the missing labeles before even checking the test statuses.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-07-25 10:23:55 +02:00
Lukáš Doktor
379334c166
ci: Unify character for separating items
the test names are using `;` and regexps were designed to use `,` but
during development simply joined the expressions by `|`. This should
work but might be confusing so let's go with the semi-colon separator
everywhere.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-07-25 10:23:55 +02:00
Wainer dos Santos Moschetta
9ab325c430
workflows/gatekeeper: export COMMIT_HASH variable
The Github SHA of triggering PR should be exported in the environment
so that gatekeeper can fetch the right workflows/jobs.

Note: by default github will export GITHUB_SHA in the job's environment
but that value cannot be used if the gatekeeper was triggered from a
pull_request_target event, because the SHA correspond to the push
branch.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-07-25 10:23:55 +02:00
Wainer dos Santos Moschetta
c176896b2d
workflows/gatekeeper: configure concurrency properly
This will allow to cancel-in-progress the gatekeeper jobs.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-07-25 10:23:53 +02:00
Lukáš Doktor
62ffd83573
ci: Reshuffle run-k8s-tests to avoid "too many workflows"
github action has a limit of 20 referenced workflows. Let's combine all
the amd64 related run-k8s-tests into a single job file to workaround
this limitation. They all share the same prerequisits so the result
should be the same.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-07-25 07:04:48 +02:00
Lukáš Doktor
84e5133cd8
ci: Select jobs by touched code
to allow selective testing as well as selective list of required tests
let's add a mapping of required jobs/tests in "skips.py" and a
"gatekeaper" workflow that will ensure the expected required jobs were
successful. Then we can only mark the "gatekeaper" as the required job
and modify the logic to suit our needs.

Fixes: #9237

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
2024-07-18 13:58:52 +02:00
GabyCT
6aff5f300a
Merge pull request #10021 from GabyCT/topic/fixarchdoc
docs: Update devmapper docs
2024-07-17 14:56:40 -06:00
Steve Horsman
e5d5284761
Merge pull request #10026 from wainersm/release_370
release: Bump VERSION to 3.7.0
2024-07-17 18:43:51 +01:00
Wainer dos Santos Moschetta
6f7ab31860 release: Bump VERSION to 3.7.0
On preparation for the 3.7.0 release, bumped the version in VERSION file.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-07-17 14:19:44 -03:00
Dan Mihai
f31c1b121e
Merge pull request #9812 from microsoft/saulparedes/test_policy_on_tdx
gha: enable policy testing on TDX
2024-07-17 08:47:44 -07:00
Dan Mihai
449103c7bf
Merge pull request #10020 from microsoft/danmihai1/pod-security-context
tests: fix ps command in k8s-security-context
2024-07-17 08:12:57 -07:00
Fabiano Fidêncio
b7051890af
Merge pull request #9722 from zvonkok/busybox-build
deploy: Add busybox target
2024-07-17 13:47:15 +02:00
Steve Horsman
5ce2c1010a
Merge pull request #9904 from stevenhorsman/registry-authentication
Support for registry authentication in guest pull
2024-07-17 10:48:38 +01:00
Fupan Li
65f2bfb8c4
Merge pull request #9967 from zvonkok/kernel-dragonball-6.1.x
dragonball: kernel dragonball 6.1.x
2024-07-17 14:38:06 +08:00
Dan Mihai
0e86a96157 tests: fix ps command in k8s-security-context
1. Use a container image that supports "ps --user 1000 -f".
2. Execute that command using:

sh -c "ps --user 1000 -f"

instead of passing additional arguments to sh:

sh -c ps --user 1000 -f

Fixes: #10019

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-07-17 01:33:31 +00:00
stevenhorsman
567b4d5788 test/k8s: Fix up node logging typo
We had a typo in the attestation tests that we've copied around a
lot and Wainer spotted it in the authenticated registry tests, so let's fix it up now

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-07-16 21:39:31 -03:00
stevenhorsman
0015c8ef51 tests: Add guest-pull auth registry tests
Add three new test cases for guest pull from an authenticated registry for
the following scenarios:

_**Scenario**: Creating a container from an authenticated image, with correct credentials via KBC works_
**Given** An authenticated container registry *quay.io/kata-containers/confidential-containers-auth*
  **And** a version of kata deployed with a guest image that has an agent with `guest_pull`
feature enabled and nydus-snapshotter installed and configured for
[guest-pulling](https://github.com/containerd/nydus-snapshotter/blob/main/misc/snapshotter/config-coco-guest-pulling.toml)
  **And** a KBS set up to have the correct auth.json for
registry *quay.io/kata-containers/confidential-containers-auth* embedded in the `"Credential"` section of `its resources file`
**When** I create a pod from the container image *quay.io/kata-containers/confidential-containers-auth:test*
**Then** The pull image works and the pod can start

_**Scenario**: Creating a container from an authenticated image, with incorrect credentials via KBC fails_
**Given**  An authenticated container registry *quay.io/kata-containers/confidential-containers-auth*
  **And** a version of kata deployed with a guest image that has an agent with `guest_pull`
feature enabled and nydus-snapshotter installed and configured for
[guest-pulling](https://github.com/containerd/nydus-snapshotter/blob/main/misc/snapshotter/config-coco-guest-pulling.toml)
  **And** An installed kata CC with the sample_kbs set up to have the auth.json for registry
*quay.io/kata-containers/confidential-containers-auth* embedded in the `"Credential"` resource, but with a dummy user name and password
**When** I create a pod from the container image *quay.io/kata-containers/confidential-containers-auth:test*
**Then** The pull image fails with a message that reflects that the authorisation failed

_**Scenario**: Creating a container from an authenticated image, with no credentials fails_
**Given**  An authenticated container registry *quay.io/kata-containers/confidential-containers-auth*
  **And** a version of kata deployed with a guest image that has an agent with `guest_pull`
feature enabled and nydus-snapshotter installed and configured for
[guest-pulling](https://github.com/containerd/nydus-snapshotter/blob/main/misc/snapshotter/config-coco-guest-pulling.toml)
  **And** An installed kata CC with no credentials section
**When** I create a pod from the container image *quay.io/kata-containers/confidential-containers-auth:test*
**Then** The pull image fails with a message that reflects that the authorisation failed

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-07-16 21:39:31 -03:00
stevenhorsman
eb07f5ef5e agent: doc: Fix ordering of options
- Fix the config options to be back in alphabetical order to be
easier to find

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-07-16 21:39:31 -03:00
stevenhorsman
7cc81ce867 agent: image: Set image-rs auth config
If the agent-config has a value for `image_registry_auth`,
Then pass this to the image-rs client and enable auth mode too

Fixes: #8122

Signed-off-by: Magnus Kulke <magnuskulke@microsoft.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-07-16 21:39:31 -03:00
stevenhorsman
265322990a agent: config: Add config option to provide auth for guest-pull
Add optional config for agent.image_registry_auth, to specify
the uri of credentials to be used when pulling images in the guest
from an authenticated registry

Fixes: #8122

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
Signed-off-by: Magnus Kulke <magnuskulke@microsoft.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-07-16 21:39:31 -03:00
Steve Horsman
064b45a2fa
Merge pull request #10016 from wainersm/ibm-se-auth-reg
workflows: setup environment to run auth registry tests on s390x
2024-07-16 22:24:39 +01:00
Gabriela Cervantes
d2866081d2 docs: Update devmapper docs
This PR updates the devmapper docs by updating the url link
for the current containerd devmapper information.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-07-16 21:07:51 +00:00
GabyCT
2206e2dd5c
Merge pull request #10013 from GabyCT/topic/updatecontdoc
docs: Update cri installion guide url in containerd documentation
2024-07-16 14:32:59 -06:00
Wainer dos Santos Moschetta
66c600f8d8 gha: delint the s390x workflow
Made run-k8s-tests-on-zvsi.yaml free of warnings by removing:

SC2086:info:1:1: Double quote to prevent globbing and word splitting ...
SC2086:info:2:1: Double quote to prevent globbing and word splitting ...

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-07-16 15:20:46 -03:00
Wainer dos Santos Moschetta
a98985fab8 gha: export user/password for auth registry tests on s390x
Counterpart of commit d8961cbd4a for run-k8s-tests-on-zvsi workflow

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-07-16 15:18:40 -03:00
Saul Paredes
af49252c69 gha: enable policy testing on TDX
Enable policy testing on TDX

Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
2024-07-15 14:09:49 -07:00
Saul Paredes
0b3d193730 genpolicy: Support cpath for mount sources
Add setting to allow specifying the cpath for a mount source.

cpath is the root path for most files used by a container. For example,
the container rootfs and various files copied from the Host to the
Guest when shared_fs=none are hosted under cpath.

mount_source_cpath is the root of the paths used a storage mount
sources. Depending on Kata settings, mount_source_cpath might have the
same value as cpath - but on TDX for example these two paths are
different: TDX uses "/run/kata-containers" as cpath,
but "/run/kata-containers/shared/containers" as mount_source_cpath.

Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
2024-07-15 14:09:49 -07:00
Gabriela Cervantes
e4045ff29a docs: Update runtime v2 containerd url information
This PR updates the runtime v2 containerd url information at containerd
documentation.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-07-15 20:36:17 +00:00
Dan Mihai
bcaf7fc3b4
Merge pull request #10008 from microsoft/danmihai1/runAsUser
genpolicy: add support for runAsUser fields
2024-07-15 12:08:50 -07:00
Gabriela Cervantes
9f738f0d05 docs: Update cri installion guide url in containerd documentation
This PR updates the cri installation guide url link in the containerd
documentation guide as the previous url link does not exists.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-07-15 16:58:18 +00:00
Dan Mihai
648265d80e
Merge pull request #9998 from microsoft/danmihai1/GENPOLICY_PULL_METHOD
tests: k8s: GENPOLICY_PULL_METHOD clean-up
2024-07-15 09:32:29 -07:00
Steve Horsman
02b9fd6e95
Merge pull request #9382 from Xynnn007/feat-encrypt-image
Merge to main: supporting pull encrypted images
2024-07-15 15:58:42 +01:00
stevenhorsman
b060fb5b31 tests/k8s: Skip measured rootfs test
The only kernel built for measured rootfs was the kernel-tdx-experimental,
so this test only ran in the qemu-tdx job runs the test.
In commit 6cbdba7 we switched all TEE configurations to use the same kernel-confidential,
so rootfs measured is disabled for qemu-tdx too now.
The VM still fails to boot (because of a different reason...) but the bug
in the assert_logs_contain, fixed in this PR was masking the checks on the logs.
We still have a few open issues related to measured rootfs and generating
the root hash, so let's skip this test that doesn't work until they are looked at

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-07-15 12:00:50 +01:00
stevenhorsman
2cf94ae717 tests: Add guest-pull encrypted image tests
Add three new tests cases for guest-pull of an encrypted image
for the following scenarios:

_**Scenario: Pull encrypted image on guest with correct key works**_
**Given** I have a version of kata deployed with a guest image that has
an agent with `guest_pull` feature enabled and nydus-snapshotter installed
and configured for guest-pulling
  **And** A public encrypted container image *i* with a decryption key *k*
that is configured as a resource the KBS, so that image-rs on the guest can
connect to it
**When** I try and create a pod from *i*
**Then** The pod is successfully created and runs

_**Scenario: Cannot pull encrypted image with no decryption key**_
**Given** I have a version of kata deployed with a guest image that has
an agent with `guest_pull` feature enabled and nydus-snapshotter installed
and configured for guest-pulling
  **And** A public encrypted container image *i* with a decryption key *k*,
that is **not** configured in a KBS that image-rs on the guest can connect to
**When** I try and create a pod from *i*
**Then** The pod is not created with an error message that reflects why

_**Scenario: Cannot pull encrypted image with wrong decryption key**_
**Given** I have a version of kata deployed with a guest image that has
an agent with `guest_pull` feature enabled and nydus-snapshotter installed
and configured for guest-pulling
  **And** A public encrypted container image *i* with a decryption key *k*
and a different key *k'* that is set as a resource in a KBS, that image-rs
on the guest can connect to
**When** I try and create a pod from *i*
**Then** The pod is not created with an error message that reflects why

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-07-15 12:00:50 +01:00
Xynnn007
a56b15112a agent: add ocicrypt config
ocicrypt config is for kata-agent to connect to CDH to request for image
decryption key. This value is specified by an env. We use this
workaround the same as CCv0 branch.

In future, we will consider better ways instead of writting files and
setting envs inside inner logic of kata-agent.

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
2024-07-15 12:00:50 +01:00
Xynnn007
1072658219 agent: Enable kata-cc-rustls-tls in image-rs
- Enable the kata-cc-rustls-tls feature in image-rs, so that it
can get resources from the KBS in order to retrieve the registry
credentials.
- Also bump to the latest image-rs to pick up protobuf fixes
- Add libprotobuf-dev dependency to the agent packaging
as it is needed by the new image-rs feature
- Add extra env in the agent make test as the
new version of the anyhow crate has changed the backtrace capture thus unit
tests of kata-agent that compares a raised error with an expected one
would fail. To fix this, we need only panics to have backtraces, thus
set RUST_BACKTRACE=0 for tests due to document
https://docs.rs/anyhow/latest/anyhow/

Fixes #9538

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-07-15 12:00:50 +01:00
stevenhorsman
3b72e9ffab tests/k8s: Fix assert_logs_contain
The pipe needs adding to the grep, otherwise the grep
gets consumed as an argument to `print_node_journal` and
run in the debug pod.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-07-15 12:00:50 +01:00
Hyounggyu Choi
83b3a681f4
Merge pull request #10010 from BbolroC/osbuilder-bump-fedora-to-40
osbuilder: Bump Fedora to 40
2024-07-15 13:00:28 +02:00
Greg Kurz
203d9e7803
Merge pull request #10000 from littlejawa/kata_deploy_add_storage_config_for_crio
kata-deploy: add storage configuration for cri-o
2024-07-15 12:29:21 +02:00
Hyounggyu Choi
08d2f6bfe4 osbuilder: Bump Fedora to 40
As Fedora 38 has reached EOL, we are encountering 404 errors for s390x, such as:

```
Status code: 404 for https://dl.fedoraproject.org/pub/fedora-secondary/updates/38/Everything/s390x/repodata/repomd.xml
```

Let's bump the OS to the latest version.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-07-15 09:58:54 +02:00
Fupan Li
a7179be31d
Merge pull request #9534 from Tim-Zhang/fix-stdin-stuck
Fix ctr exec stuck problem
2024-07-15 13:19:19 +08:00
Dan Mihai
dded329d26 tests: k8s: SecurityContext.runAsUser policy test
Add test for auto-generating policy for a pod spec that includes the
SecurityContext.runAsUser field.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-07-13 01:37:58 +00:00
Dan Mihai
7040fb8c50 tests: k8s-security-context auto-generated policy
Auto-generate the policy in k8s-security-context.bats - previously
blocked by lacking support for PodSecurityContext.runAsUser.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-07-13 01:23:54 +00:00
Dan Mihai
f087044ecb genpolicy: add support for runAsUser
Add ability to auto-generate policy for SecurityContext.runAsUser and
PodSecurityContext.runAsUser.

Fixes: #8879

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-07-13 01:10:43 +00:00
Dan Mihai
5282701b5b genpolicy: add link to allow_user() active issue
Improve comment to workaround in rules.rego, to explain better the
reason for that workaround.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-07-13 01:05:58 +00:00
GabyCT
3c0171df3d
Merge pull request #10005 from GabyCT/topic/katadragonball
common: Add share fs information for dragonball
2024-07-12 16:10:29 -06:00
Wainer Moschetta
646d7ea4fb
Merge pull request #9951 from BbolroC/enable-attestation-for-ibm-se
tests: Enable attestation e2e tests for IBM SE
2024-07-11 16:02:59 -03:00
Hyounggyu Choi
ca80301b4b
Merge pull request #10003 from BbolroC/skip-pod-shared-volume-for-ibm-se
k8s: Skip shared-volume relevant tests for IBM SE
2024-07-11 19:29:13 +02:00