Compare commits

..

19 Commits
3.27.0 ... main

Author SHA1 Message Date
Fabiano Fidêncio
96c20f8baa tests: k8s: set CreateContainerRequest (on free runners) timeout to 600s
Set KubeletConfiguration runtimeRequestTimeout to 600s mainly for CoCo
(Confidential Containers) tests, so container creation (attestation,
policy, image pull, VM start) does not hit the default CRI timeout.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-21 08:44:47 +01:00
Fabiano Fidêncio
9634dfa859 gatekeeper: Update tests name
We need to do so after moving some of the tests to the free runners.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-21 08:44:47 +01:00
Fabiano Fidêncio
a6b7a2d8a4 tests: assert_pod_fail accept RunContainerError and StartError
Treat waiting.reason RunContainerError and terminated.reason StartError/Error
as container failure, so tests that expect guest image-pull failure (e.g.
wrong credentials) pass when the container fails with those states instead
of only BackOff.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-21 08:44:47 +01:00
Fabiano Fidêncio
42d980815a tests: skip k8s-policy-pvc on non-AKS
Otherwise it'll fail as we cannot bind the device.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-21 08:44:47 +01:00
Fabiano Fidêncio
1523c48a2b tests: k8s: Align coco / erofs job declaration
Later on we may even think about merging those, but for now let's at
least make sure the envs used are the same / declared in a similar place
for each job.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-21 08:44:47 +01:00
Fabiano Fidêncio
1b9b53248e tests: k8s: coco: rely more on free runners
Run all CoCo non-TEE variants in a single job on the free runner with an
explicit environment matrix (vmm, snapshotter, pull_type, kbs,
containerd_version).

Here we're testing CoCo only with the "active" version of containerd.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-21 08:44:47 +01:00
Fabiano Fidêncio
1fa3475e36 tests: k8s: rely more on free runners
We were running most of the k8s integration tests on AKS. The ones that
don't actually depend on AKS's environment now run on normal
ubuntu-24.04 GitHub runners instead: we bring up a kubeadm cluster
there, test with both containerd lts and active, and skip attestation
tests since those runtimes don't need them. AKS is left only for the
jobs that do depend on it.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-21 08:44:47 +01:00
Fabiano Fidêncio
2f056484f3 versions: Bump containerd active to 2.2
SSIA

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-21 08:44:47 +01:00
Zvonko Kaiser
6d1eaa1065 Merge pull request #12461 from manuelh-dev/mahuber/guest-pull-bats
tests: enable more scenarios for k8s-guest-pull-image.bats
2026-02-20 08:48:54 -05:00
Zvonko Kaiser
1de7dd58f5 gpu: Add NVLSM daemon
We need to chissel the NVLSM daemon for NVL5 systems

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2026-02-20 11:39:59 +01:00
Zvonko Kaiser
67d154fe47 gpu: Enable NVL5 based platform
NVL5 based HGX systems need ib_umad and
fabricmanager and nvlsm installed.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2026-02-20 11:39:59 +01:00
Dan Mihai
ea53779b90 ci: k8s: temporarily disable mariner host
Disable mariner host testing in CI, and auto-generated policy testing
for the temporary replacements of these hosts (based on ubuntu), to work
around missing:

1. cloud-hypervisor/cloud-hypervisor@0a5e79a, that will allow Kata
   in the future to disable the nested property of guest VPs. Nested
   is enabled by default and doesn't work yet with mariner's MSHV.
2. cloud-hypervisor/cloud-hypervisor@bf6f0f8, exposed by the large
   ttrpc replies intentionally produced by the Kata CI Policy tests.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2026-02-19 20:42:50 +01:00
Dan Mihai
3e2153bbae ci: k8s: easier to modify az aks create command
Make `az aks create` command easier to change when needed, by moving the
arguments specific to mariner nodes onto a separate line of this script.
This change also removes the need for `shellcheck disable=SC2046` here.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2026-02-19 20:42:50 +01:00
Fabiano Fidêncio
cadbf51015 versions: Update Cloud Hypervisor to v50.0
```
This release has been tracked in v50.0 group of our roadmap project.

Configurable Nested Virtualization Option on x86_64
The nested=on|off option has been added to --cpu to allow users
to configure nested virtualization support in the guest on x86_64
hosts (for both KVM and MSHV). The default value is on to maintain
consistency with existing behavior. (#7408)

Compression Support for QCOW2
QCOW2 support has been extended to handle compression clusters based on
zlib and zstd. (#7462)

Notable Performance Improvements
Performance of live migration has been improved via an optimized
implementation of dirty bitmap maintenance. (#7468)

Live Disk Resizing Support for Raw Images
The /vm.resize-disk API has been introduced to allow users to resize block
devices backed by raw images while a guest is running. (#7476)

Developer Experience Improvements
Significant improvements have been made to developer experience and
productivity. These include a simplified root manifest, codified and
tightened Clippy lints, and streamlined workflows for cargo clippy and
cargo test. (#7489)

Improved File-level Locking Support
Block devices now use byte-range advisory locks instead of whole-file
locks. While both approaches prevent multiple Cloud Hypervisor instances
from simultaneously accessing the same disk image with write
permissions, byte-range locks provide better compatibility with network
storage backends. (#7494)

Logging Improvements
Logs now include event information generated by the event-monitor
module. (#7512)

Notable Bug Fixes
* Fix several issues around CPUID in the guest (#7485, #7495, #7508)
* Fix snapshot/restore for Windows Guest (#7492)
* Respect queue size in block performance tests (#7515)
* Fix several Serial Manager issues (#7502)
* Fix several seccomp violation issues (#7477, #7497, #7518)
* Fix various issues around block and qcow (#7526, #7528, #7537, #7546,
  #7549)
* Retrieve MSRs list correctly on MSHV (#7543)
* Fix live migration (and snapshot/restore) with AMX state (#7534)
```

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-02-19 20:42:50 +01:00
Dan Mihai
d8b403437f static-build: delete cloud-hypervisor directory
This cloud-hypervisor is a directory, so it needs "rm -rf" instead of
"rm -f".

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2026-02-19 20:42:50 +01:00
Manuel Huber
fd340ac91c tests: remove skips for some guest-pull scenarios
Issue 10838 is resolved by the prior commit, enabling the -m
option of the kernel build for confidential guests which are
not users of the measured rootfs, and by commit
976df22119, which ensures
relevant user space packages are present.
Not every confidential guest has the measured rootfs option
enabled. Every confidential guest is assumed to support CDH's
secure storage features, in contrast.

We also adjust test timeouts to account for occasional spikes on
our bare metal runners (e.g., SNP, TDX, s390x).

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2026-02-19 10:10:55 -08:00
Harshitha Gowda
728d8656ee tests: Set sev-snp, qemu-snp CIs as required
run-k8s-tests-on-tee (sev-snp, qemu-snp)

Signed-off-by: Harshitha Gowda <hgowda@amd.com>
2026-02-19 16:41:29 +01:00
Manuel Huber
4c760fd031 build: add CONFIDENTIAL_GUEST variable for kernel
This change adds the CONFIDENTIAL_GUEST variable to the kernel
build logic. Similar to commit
976df22119, we would like to enable
the cryptsetup functionalities not only when building a measured
root file system, but also when building for a confidential guest.
The current state is that not all confidential guests use a
measured root filesystem, and as a matter of fact, we should
indeed decouple these aspects.

With the current convention, a confidential guest is a user of CDH
with its storage features. A better naming of the
CONFIDENTIAL_GUEST variable could have been a naming related to CDH
storage functionality. Further, the kernel build script's -m
parameter could be improved too - as indicated by this change, not
only measured rootfs builds will need the cryptsetup.conf file.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2026-02-17 12:44:50 -08:00
Manuel Huber
d3742ca877 tests: enable guest pull bats for force guest pull
Similar to k8s-guest-pull-image-authenticated and to
k8s-guest-pull-image-signature, enabling k8s-guest-pull-image to
run against the experimental force guest pull method.
Only k8s-guest-pull-image-encrypted requires nydus.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2026-02-17 12:44:50 -08:00
34 changed files with 1153 additions and 249 deletions

View File

@@ -297,6 +297,21 @@ jobs:
AZ_TENANT_ID: ${{ secrets.AZ_TENANT_ID }}
AZ_SUBSCRIPTION_ID: ${{ secrets.AZ_SUBSCRIPTION_ID }}
run-k8s-tests-on-free-runner:
if: ${{ inputs.skip-test != 'yes' }}
needs: publish-kata-deploy-payload-amd64
permissions:
contents: read
uses: ./.github/workflows/run-k8s-tests-on-free-runner.yaml
with:
tarball-suffix: -${{ inputs.tag }}
registry: ghcr.io
repo: ${{ github.repository_owner }}/kata-deploy-ci
tag: ${{ inputs.tag }}-amd64
commit-hash: ${{ inputs.commit-hash }}
pr-number: ${{ inputs.pr-number }}
target-branch: ${{ inputs.target-branch }}
run-k8s-tests-on-arm64:
if: ${{ inputs.skip-test != 'yes' }}
needs: publish-kata-deploy-payload-arm64

View File

@@ -42,17 +42,6 @@ jobs:
strategy:
fail-fast: false
matrix:
host_os:
- ubuntu
vmm:
- clh
- dragonball
- qemu
- qemu-runtime-rs
- cloud-hypervisor
instance-type:
- small
- normal
include:
- host_os: cbl-mariner
vmm: clh
@@ -80,6 +69,7 @@ jobs:
KUBERNETES: "vanilla"
K8S_TEST_HOST_TYPE: ${{ matrix.instance-type }}
GENPOLICY_PULL_METHOD: ${{ matrix.genpolicy-pull-method }}
RUNS_ON_AKS: "true"
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:

View File

@@ -0,0 +1,127 @@
# Run Kubernetes integration tests on free GitHub runners with a locally
# deployed cluster (kubeadm).
name: CI | Run kubernetes tests on free runner
on:
workflow_call:
inputs:
tarball-suffix:
required: false
type: string
registry:
required: true
type: string
repo:
required: true
type: string
tag:
required: true
type: string
pr-number:
required: true
type: string
commit-hash:
required: false
type: string
target-branch:
required: false
type: string
default: ""
permissions: {}
jobs:
run-k8s-tests:
name: run-k8s-tests
strategy:
fail-fast: false
matrix:
environment: [
{ vmm: clh, containerd_version: lts },
{ vmm: clh, containerd_version: active },
{ vmm: dragonball, containerd_version: lts },
{ vmm: dragonball, containerd_version: active },
{ vmm: qemu, containerd_version: lts },
{ vmm: qemu, containerd_version: active },
{ vmm: qemu-runtime-rs, containerd_version: lts },
{ vmm: qemu-runtime-rs, containerd_version: active },
{ vmm: cloud-hypervisor, containerd_version: lts },
{ vmm: cloud-hypervisor, containerd_version: active },
]
runs-on: ubuntu-24.04
permissions:
contents: read
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
GH_PR_NUMBER: ${{ inputs.pr-number }}
KATA_HOST_OS: ubuntu
KATA_HYPERVISOR: ${{ matrix.environment.vmm }}
KUBERNETES: vanilla
K8S_TEST_HOST_TYPE: baremetal-no-attestation
CONTAINER_ENGINE: containerd
CONTAINER_ENGINE_VERSION: ${{ matrix.environment.containerd_version }}
GH_TOKEN: ${{ github.token }}
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
persist-credentials: false
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: get-kata-tools-tarball
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: kata-tools-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-tools-artifacts
- name: Install kata-tools
run: bash tests/integration/kubernetes/gha-run.sh install-kata-tools kata-tools-artifacts
- name: Remove unnecessary directories to free up space
run: |
sudo rm -rf /usr/local/.ghcup
sudo rm -rf /opt/hostedtoolcache/CodeQL
sudo rm -rf /usr/local/lib/android
sudo rm -rf /usr/share/dotnet
sudo rm -rf /opt/ghc
sudo rm -rf /usr/local/share/boost
sudo rm -rf /usr/lib/jvm
sudo rm -rf /usr/share/swift
sudo rm -rf /usr/local/share/powershell
sudo rm -rf /usr/local/julia*
sudo rm -rf /opt/az
sudo rm -rf /usr/local/share/chromium
sudo rm -rf /opt/microsoft
sudo rm -rf /opt/google
sudo rm -rf /usr/lib/firefox
- name: Deploy k8s (kubeadm)
run: bash tests/integration/kubernetes/gha-run.sh deploy-k8s
- name: Install `bats`
run: bash tests/integration/kubernetes/gha-run.sh install-bats
- name: Deploy Kata
timeout-minutes: 20
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata
- name: Run tests
timeout-minutes: 60
run: bash tests/integration/kubernetes/gha-run.sh run-tests
- name: Report tests
if: always()
run: bash tests/integration/kubernetes/gha-run.sh report-tests
- name: Delete kata-deploy
if: always()
timeout-minutes: 15
run: bash tests/integration/kubernetes/gha-run.sh cleanup

View File

@@ -140,165 +140,35 @@ jobs:
strategy:
fail-fast: false
matrix:
vmm:
- qemu-coco-dev
- qemu-coco-dev-runtime-rs
snapshotter:
- nydus
pull-type:
- guest-pull
include:
- pull-type: experimental-force-guest-pull
vmm: qemu-coco-dev
snapshotter: ""
runs-on: ubuntu-22.04
environment: [
{ vmm: qemu-coco-dev, snapshotter: nydus, pull_type: guest-pull },
{ vmm: qemu-coco-dev-runtime-rs, snapshotter: nydus, pull_type: guest-pull },
{ vmm: qemu-coco-dev, snapshotter: "", pull_type: experimental-force-guest-pull },
]
runs-on: ubuntu-24.04
permissions:
id-token: write # Used for OIDC access to log into Azure
contents: read
environment: ci
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
GH_PR_NUMBER: ${{ inputs.pr-number }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
KATA_HYPERVISOR: ${{ matrix.environment.vmm }}
# Some tests rely on that variable to run (or not)
KBS: "true"
# Set the KBS ingress handler (empty string disables handling)
KBS_INGRESS: "aks"
KBS_INGRESS: "nodeport"
KUBERNETES: "vanilla"
PULL_TYPE: ${{ matrix.pull-type }}
PULL_TYPE: ${{ matrix.environment.pull_type }}
AUTHENTICATED_IMAGE_USER: ${{ vars.AUTHENTICATED_IMAGE_USER }}
AUTHENTICATED_IMAGE_PASSWORD: ${{ secrets.AUTHENTICATED_IMAGE_PASSWORD }}
SNAPSHOTTER: ${{ matrix.snapshotter }}
EXPERIMENTAL_FORCE_GUEST_PULL: ${{ matrix.pull-type == 'experimental-force-guest-pull' && matrix.vmm || '' }}
# Caution: current ingress controller used to expose the KBS service
# requires much vCPUs, lefting only a few for the tests. Depending on the
# host type chose it will result on the creation of a cluster with
# insufficient resources.
SNAPSHOTTER: ${{ matrix.environment.snapshotter }}
EXPERIMENTAL_FORCE_GUEST_PULL: ${{ matrix.environment.pull_type == 'experimental-force-guest-pull' && matrix.environment.vmm || '' }}
K8S_TEST_HOST_TYPE: "all"
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
persist-credentials: false
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: get-kata-tools-tarball
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: kata-tools-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-tools-artifacts
- name: Install kata-tools
run: bash tests/integration/kubernetes/gha-run.sh install-kata-tools kata-tools-artifacts
- name: Log into the Azure account
uses: azure/login@a457da9ea143d694b1b9c7c869ebb04ebe844ef5 # v2.3.0
with:
client-id: ${{ secrets.AZ_APPID }}
tenant-id: ${{ secrets.AZ_TENANT_ID }}
subscription-id: ${{ secrets.AZ_SUBSCRIPTION_ID }}
- name: Create AKS cluster
uses: nick-fields/retry@ce71cc2ab81d554ebbe88c79ab5975992d79ba08 # v3.0.2
with:
timeout_minutes: 15
max_attempts: 20
retry_on: error
retry_wait_seconds: 10
command: bash tests/integration/kubernetes/gha-run.sh create-cluster
- name: Install `bats`
run: bash tests/integration/kubernetes/gha-run.sh install-bats
- name: Install `kubectl`
uses: azure/setup-kubectl@776406bce94f63e41d621b960d78ee25c8b76ede # v4.0.1
with:
version: 'latest'
- name: Download credentials for the Kubernetes CLI to use them
run: bash tests/integration/kubernetes/gha-run.sh get-cluster-credentials
- name: Deploy Kata
timeout-minutes: 20
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata-aks
env:
USE_EXPERIMENTAL_SETUP_SNAPSHOTTER: ${{ env.SNAPSHOTTER == 'nydus' }}
AUTO_GENERATE_POLICY: ${{ env.PULL_TYPE == 'experimental-force-guest-pull' && 'no' || 'yes' }}
- name: Deploy CoCo KBS
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-coco-kbs
- name: Install `kbs-client`
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh install-kbs-client
- name: Deploy CSI driver
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh deploy-csi-driver
- name: Run tests
timeout-minutes: 80
run: bash tests/integration/kubernetes/gha-run.sh run-tests
- name: Report tests
if: always()
run: bash tests/integration/kubernetes/gha-run.sh report-tests
- name: Refresh OIDC token in case access token expired
if: always()
uses: azure/login@a457da9ea143d694b1b9c7c869ebb04ebe844ef5 # v2.3.0
with:
client-id: ${{ secrets.AZ_APPID }}
tenant-id: ${{ secrets.AZ_TENANT_ID }}
subscription-id: ${{ secrets.AZ_SUBSCRIPTION_ID }}
- name: Delete AKS cluster
if: always()
timeout-minutes: 15
run: bash tests/integration/kubernetes/gha-run.sh delete-cluster
# Generate jobs for testing CoCo on non-TEE environments with erofs-snapshotter
run-k8s-tests-coco-nontee-with-erofs-snapshotter:
name: run-k8s-tests-coco-nontee-with-erofs-snapshotter
strategy:
fail-fast: false
matrix:
vmm:
- qemu-coco-dev
snapshotter:
- erofs
pull-type:
- default
runs-on: ubuntu-24.04
environment: ci
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
GH_PR_NUMBER: ${{ inputs.pr-number }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
# Some tests rely on that variable to run (or not)
KBS: "false"
# Set the KBS ingress handler (empty string disables handling)
KBS_INGRESS: ""
KUBERNETES: "vanilla"
CONTAINER_ENGINE: "containerd"
CONTAINER_ENGINE_VERSION: "v2.2"
PULL_TYPE: ${{ matrix.pull-type }}
SNAPSHOTTER: ${{ matrix.snapshotter }}
USE_EXPERIMENTAL_SETUP_SNAPSHOTTER: "true"
K8S_TEST_HOST_TYPE: "all"
# We are skipping the auto generated policy tests for now,
# but those should be enabled as soon as we work on that.
AUTO_GENERATE_POLICY: "no"
CONTAINER_ENGINE_VERSION: "active"
GH_TOKEN: ${{ github.token }}
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
@@ -342,8 +212,130 @@ jobs:
- name: Deploy kubernetes
timeout-minutes: 15
run: bash tests/integration/kubernetes/gha-run.sh deploy-k8s
- name: Install `bats`
run: bash tests/integration/kubernetes/gha-run.sh install-bats
- name: Deploy Kata
timeout-minutes: 20
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata
env:
GH_TOKEN: ${{ github.token }}
USE_EXPERIMENTAL_SETUP_SNAPSHOTTER: ${{ matrix.environment.snapshotter == 'nydus' }}
AUTO_GENERATE_POLICY: ${{ matrix.environment.pull_type == 'experimental-force-guest-pull' && 'no' || 'yes' }}
- name: Deploy CoCo KBS
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh deploy-coco-kbs
- name: Install `kbs-client`
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh install-kbs-client
- name: Deploy CSI driver
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh deploy-csi-driver
- name: Run tests
timeout-minutes: 80
run: bash tests/integration/kubernetes/gha-run.sh run-tests
- name: Report tests
if: always()
run: bash tests/integration/kubernetes/gha-run.sh report-tests
- name: Delete kata-deploy
if: always()
timeout-minutes: 15
run: bash tests/integration/kubernetes/gha-run.sh cleanup
- name: Delete CoCo KBS
if: always()
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh delete-coco-kbs
- name: Delete CSI driver
if: always()
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh delete-csi-driver
# Generate jobs for testing CoCo on non-TEE environments with erofs-snapshotter
run-k8s-tests-coco-nontee-with-erofs-snapshotter:
name: run-k8s-tests-coco-nontee-with-erofs-snapshotter
strategy:
fail-fast: false
matrix:
vmm:
- qemu-coco-dev
snapshotter:
- erofs
pull-type:
- default
runs-on: ubuntu-24.04
environment: ci
env:
DOCKER_REGISTRY: ${{ inputs.registry }}
DOCKER_REPO: ${{ inputs.repo }}
DOCKER_TAG: ${{ inputs.tag }}
GH_PR_NUMBER: ${{ inputs.pr-number }}
KATA_HYPERVISOR: ${{ matrix.vmm }}
# Some tests rely on that variable to run (or not)
KBS: "false"
# Set the KBS ingress handler (empty string disables handling)
KBS_INGRESS: ""
KUBERNETES: "vanilla"
CONTAINER_ENGINE: "containerd"
CONTAINER_ENGINE_VERSION: "active"
PULL_TYPE: ${{ matrix.pull-type }}
SNAPSHOTTER: ${{ matrix.snapshotter }}
USE_EXPERIMENTAL_SETUP_SNAPSHOTTER: "true"
K8S_TEST_HOST_TYPE: "all"
# We are skipping the auto generated policy tests for now,
# but those should be enabled as soon as we work on that.
AUTO_GENERATE_POLICY: "no"
GH_TOKEN: ${{ github.token }}
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ inputs.commit-hash }}
fetch-depth: 0
persist-credentials: false
- name: Rebase atop of the latest target branch
run: |
./tests/git-helper.sh "rebase-atop-of-the-latest-target-branch"
env:
TARGET_BRANCH: ${{ inputs.target-branch }}
- name: get-kata-tools-tarball
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: kata-tools-static-tarball-amd64${{ inputs.tarball-suffix }}
path: kata-tools-artifacts
- name: Install kata-tools
run: bash tests/integration/kubernetes/gha-run.sh install-kata-tools kata-tools-artifacts
- name: Remove unnecessary directories to free up space
run: |
sudo rm -rf /usr/local/.ghcup
sudo rm -rf /opt/hostedtoolcache/CodeQL
sudo rm -rf /usr/local/lib/android
sudo rm -rf /usr/share/dotnet
sudo rm -rf /opt/ghc
sudo rm -rf /usr/local/share/boost
sudo rm -rf /usr/lib/jvm
sudo rm -rf /usr/share/swift
sudo rm -rf /usr/local/share/powershell
sudo rm -rf /usr/local/julia*
sudo rm -rf /opt/az
sudo rm -rf /usr/local/share/chromium
sudo rm -rf /opt/microsoft
sudo rm -rf /opt/google
sudo rm -rf /usr/lib/firefox
- name: Deploy kubernetes
timeout-minutes: 15
run: bash tests/integration/kubernetes/gha-run.sh deploy-k8s
- name: Install `bats`
run: bash tests/integration/kubernetes/gha-run.sh install-bats
@@ -363,3 +355,13 @@ jobs:
- name: Report tests
if: always()
run: bash tests/integration/kubernetes/gha-run.sh report-tests
- name: Delete kata-deploy
if: always()
timeout-minutes: 15
run: bash tests/integration/kubernetes/gha-run.sh cleanup
- name: Delete CSI driver
if: always()
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh delete-csi-driver

View File

@@ -49,6 +49,8 @@ In order to allow Kubelet to use containerd (using the CRI interface), configure
EOF
```
For Kata Containers (and especially CoCo / Confidential Containers tests), use at least `--runtime-request-timeout=600s` (10m) so CRI CreateContainerRequest does not time out.
- Inform systemd about the new configuration
```bash

View File

@@ -45,6 +45,7 @@ docs/VmCoredumpData.md
docs/VmInfo.md
docs/VmRemoveDevice.md
docs/VmResize.md
docs/VmResizeDisk.md
docs/VmResizeZone.md
docs/VmSnapshotConfig.md
docs/VmmPingResponse.md
@@ -90,6 +91,7 @@ model_vm_coredump_data.go
model_vm_info.go
model_vm_remove_device.go
model_vm_resize.go
model_vm_resize_disk.go
model_vm_resize_zone.go
model_vm_snapshot_config.go
model_vmm_ping_response.go

View File

@@ -99,6 +99,7 @@ Class | Method | HTTP request | Description
*DefaultApi* | [**VmInfoGet**](docs/DefaultApi.md#vminfoget) | **Get** /vm.info | Returns general information about the cloud-hypervisor Virtual Machine (VM) instance.
*DefaultApi* | [**VmReceiveMigrationPut**](docs/DefaultApi.md#vmreceivemigrationput) | **Put** /vm.receive-migration | Receive a VM migration from URL
*DefaultApi* | [**VmRemoveDevicePut**](docs/DefaultApi.md#vmremovedeviceput) | **Put** /vm.remove-device | Remove a device from the VM
*DefaultApi* | [**VmResizeDiskPut**](docs/DefaultApi.md#vmresizediskput) | **Put** /vm.resize-disk | Resize a disk
*DefaultApi* | [**VmResizePut**](docs/DefaultApi.md#vmresizeput) | **Put** /vm.resize | Resize the VM
*DefaultApi* | [**VmResizeZonePut**](docs/DefaultApi.md#vmresizezoneput) | **Put** /vm.resize-zone | Resize a memory zone
*DefaultApi* | [**VmRestorePut**](docs/DefaultApi.md#vmrestoreput) | **Put** /vm.restore | Restore a VM from a snapshot.
@@ -148,6 +149,7 @@ Class | Method | HTTP request | Description
- [VmInfo](docs/VmInfo.md)
- [VmRemoveDevice](docs/VmRemoveDevice.md)
- [VmResize](docs/VmResize.md)
- [VmResizeDisk](docs/VmResizeDisk.md)
- [VmResizeZone](docs/VmResizeZone.md)
- [VmSnapshotConfig](docs/VmSnapshotConfig.md)
- [VmmPingResponse](docs/VmmPingResponse.md)

View File

@@ -153,6 +153,21 @@ paths:
description: The VM instance could not be resized because a cpu removal
is still pending.
summary: Resize the VM
/vm.resize-disk:
put:
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/VmResizeDisk'
description: Resizes a disk attached to the VM
required: true
responses:
"204":
description: The disk was successfully resized.
"500":
description: The disk could not be resized.
summary: Resize a disk
/vm.resize-zone:
put:
requestBody:
@@ -649,7 +664,9 @@ components:
- tap: tap
host_mac: host_mac
num_queues: 6
offload_ufo: true
queue_size: 1
offload_csum: true
ip: 192.168.249.1
rate_limiter_config:
ops:
@@ -663,6 +680,7 @@ components:
mac: mac
mtu: 3
pci_segment: 2
offload_tso: true
vhost_mode: Client
iommu: false
vhost_socket: vhost_socket
@@ -672,7 +690,9 @@ components:
- tap: tap
host_mac: host_mac
num_queues: 6
offload_ufo: true
queue_size: 1
offload_csum: true
ip: 192.168.249.1
rate_limiter_config:
ops:
@@ -686,6 +706,7 @@ components:
mac: mac
mtu: 3
pci_segment: 2
offload_tso: true
vhost_mode: Client
iommu: false
vhost_socket: vhost_socket
@@ -1079,7 +1100,9 @@ components:
- tap: tap
host_mac: host_mac
num_queues: 6
offload_ufo: true
queue_size: 1
offload_csum: true
ip: 192.168.249.1
rate_limiter_config:
ops:
@@ -1093,6 +1116,7 @@ components:
mac: mac
mtu: 3
pci_segment: 2
offload_tso: true
vhost_mode: Client
iommu: false
vhost_socket: vhost_socket
@@ -1102,7 +1126,9 @@ components:
- tap: tap
host_mac: host_mac
num_queues: 6
offload_ufo: true
queue_size: 1
offload_csum: true
ip: 192.168.249.1
rate_limiter_config:
ops:
@@ -1116,6 +1142,7 @@ components:
mac: mac
mtu: 3
pci_segment: 2
offload_tso: true
vhost_mode: Client
iommu: false
vhost_socket: vhost_socket
@@ -1741,7 +1768,9 @@ components:
tap: tap
host_mac: host_mac
num_queues: 6
offload_ufo: true
queue_size: 1
offload_csum: true
ip: 192.168.249.1
rate_limiter_config:
ops:
@@ -1755,6 +1784,7 @@ components:
mac: mac
mtu: 3
pci_segment: 2
offload_tso: true
vhost_mode: Client
iommu: false
vhost_socket: vhost_socket
@@ -1803,6 +1833,15 @@ components:
type: integer
rate_limiter_config:
$ref: '#/components/schemas/RateLimiterConfig'
offload_tso:
default: true
type: boolean
offload_ufo:
default: true
type: boolean
offload_csum:
default: true
type: boolean
type: object
RngConfig:
example:
@@ -2103,6 +2142,19 @@ components:
format: int64
type: integer
type: object
VmResizeDisk:
example:
desired_size: 0
id: id
properties:
id:
description: disk identifier
type: string
desired_size:
description: desired disk size in bytes
format: int64
type: integer
type: object
VmResizeZone:
example:
id: id

View File

@@ -2226,6 +2226,106 @@ func (a *DefaultApiService) VmRemoveDevicePutExecute(r ApiVmRemoveDevicePutReque
return localVarHTTPResponse, nil
}
type ApiVmResizeDiskPutRequest struct {
ctx _context.Context
ApiService *DefaultApiService
vmResizeDisk *VmResizeDisk
}
// Resizes a disk attached to the VM
func (r ApiVmResizeDiskPutRequest) VmResizeDisk(vmResizeDisk VmResizeDisk) ApiVmResizeDiskPutRequest {
r.vmResizeDisk = &vmResizeDisk
return r
}
func (r ApiVmResizeDiskPutRequest) Execute() (*_nethttp.Response, error) {
return r.ApiService.VmResizeDiskPutExecute(r)
}
/*
VmResizeDiskPut Resize a disk
@param ctx _context.Context - for authentication, logging, cancellation, deadlines, tracing, etc. Passed from http.Request or context.Background().
@return ApiVmResizeDiskPutRequest
*/
func (a *DefaultApiService) VmResizeDiskPut(ctx _context.Context) ApiVmResizeDiskPutRequest {
return ApiVmResizeDiskPutRequest{
ApiService: a,
ctx: ctx,
}
}
// Execute executes the request
func (a *DefaultApiService) VmResizeDiskPutExecute(r ApiVmResizeDiskPutRequest) (*_nethttp.Response, error) {
var (
localVarHTTPMethod = _nethttp.MethodPut
localVarPostBody interface{}
localVarFormFileName string
localVarFileName string
localVarFileBytes []byte
)
localBasePath, err := a.client.cfg.ServerURLWithContext(r.ctx, "DefaultApiService.VmResizeDiskPut")
if err != nil {
return nil, GenericOpenAPIError{error: err.Error()}
}
localVarPath := localBasePath + "/vm.resize-disk"
localVarHeaderParams := make(map[string]string)
localVarQueryParams := _neturl.Values{}
localVarFormParams := _neturl.Values{}
if r.vmResizeDisk == nil {
return nil, reportError("vmResizeDisk is required and must be specified")
}
// to determine the Content-Type header
localVarHTTPContentTypes := []string{"application/json"}
// set Content-Type header
localVarHTTPContentType := selectHeaderContentType(localVarHTTPContentTypes)
if localVarHTTPContentType != "" {
localVarHeaderParams["Content-Type"] = localVarHTTPContentType
}
// to determine the Accept header
localVarHTTPHeaderAccepts := []string{}
// set Accept header
localVarHTTPHeaderAccept := selectHeaderAccept(localVarHTTPHeaderAccepts)
if localVarHTTPHeaderAccept != "" {
localVarHeaderParams["Accept"] = localVarHTTPHeaderAccept
}
// body params
localVarPostBody = r.vmResizeDisk
req, err := a.client.prepareRequest(r.ctx, localVarPath, localVarHTTPMethod, localVarPostBody, localVarHeaderParams, localVarQueryParams, localVarFormParams, localVarFormFileName, localVarFileName, localVarFileBytes)
if err != nil {
return nil, err
}
localVarHTTPResponse, err := a.client.callAPI(req)
if err != nil || localVarHTTPResponse == nil {
return localVarHTTPResponse, err
}
localVarBody, err := _ioutil.ReadAll(localVarHTTPResponse.Body)
localVarHTTPResponse.Body.Close()
localVarHTTPResponse.Body = _ioutil.NopCloser(bytes.NewBuffer(localVarBody))
if err != nil {
return localVarHTTPResponse, err
}
if localVarHTTPResponse.StatusCode >= 300 {
newErr := GenericOpenAPIError{
body: localVarBody,
error: localVarHTTPResponse.Status,
}
return localVarHTTPResponse, newErr
}
return localVarHTTPResponse, nil
}
type ApiVmResizePutRequest struct {
ctx _context.Context
ApiService *DefaultApiService

View File

@@ -26,6 +26,7 @@ Method | HTTP request | Description
[**VmInfoGet**](DefaultApi.md#VmInfoGet) | **Get** /vm.info | Returns general information about the cloud-hypervisor Virtual Machine (VM) instance.
[**VmReceiveMigrationPut**](DefaultApi.md#VmReceiveMigrationPut) | **Put** /vm.receive-migration | Receive a VM migration from URL
[**VmRemoveDevicePut**](DefaultApi.md#VmRemoveDevicePut) | **Put** /vm.remove-device | Remove a device from the VM
[**VmResizeDiskPut**](DefaultApi.md#VmResizeDiskPut) | **Put** /vm.resize-disk | Resize a disk
[**VmResizePut**](DefaultApi.md#VmResizePut) | **Put** /vm.resize | Resize the VM
[**VmResizeZonePut**](DefaultApi.md#VmResizeZonePut) | **Put** /vm.resize-zone | Resize a memory zone
[**VmRestorePut**](DefaultApi.md#VmRestorePut) | **Put** /vm.restore | Restore a VM from a snapshot.
@@ -1370,6 +1371,68 @@ No authorization required
[[Back to README]](../README.md)
## VmResizeDiskPut
> VmResizeDiskPut(ctx).VmResizeDisk(vmResizeDisk).Execute()
Resize a disk
### Example
```go
package main
import (
"context"
"fmt"
"os"
openapiclient "./openapi"
)
func main() {
vmResizeDisk := *openapiclient.NewVmResizeDisk() // VmResizeDisk | Resizes a disk attached to the VM
configuration := openapiclient.NewConfiguration()
api_client := openapiclient.NewAPIClient(configuration)
resp, r, err := api_client.DefaultApi.VmResizeDiskPut(context.Background()).VmResizeDisk(vmResizeDisk).Execute()
if err != nil {
fmt.Fprintf(os.Stderr, "Error when calling `DefaultApi.VmResizeDiskPut``: %v\n", err)
fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
}
}
```
### Path Parameters
### Other Parameters
Other parameters are passed through a pointer to a apiVmResizeDiskPutRequest struct via the builder pattern
Name | Type | Description | Notes
------------- | ------------- | ------------- | -------------
**vmResizeDisk** | [**VmResizeDisk**](VmResizeDisk.md) | Resizes a disk attached to the VM |
### Return type
(empty response body)
### Authorization
No authorization required
### HTTP request headers
- **Content-Type**: application/json
- **Accept**: Not defined
[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints)
[[Back to Model list]](../README.md#documentation-for-models)
[[Back to README]](../README.md)
## VmResizePut
> VmResizePut(ctx).VmResize(vmResize).Execute()

View File

@@ -19,6 +19,9 @@ Name | Type | Description | Notes
**Id** | Pointer to **string** | | [optional]
**PciSegment** | Pointer to **int32** | | [optional]
**RateLimiterConfig** | Pointer to [**RateLimiterConfig**](RateLimiterConfig.md) | | [optional]
**OffloadTso** | Pointer to **bool** | | [optional] [default to true]
**OffloadUfo** | Pointer to **bool** | | [optional] [default to true]
**OffloadCsum** | Pointer to **bool** | | [optional] [default to true]
## Methods
@@ -414,6 +417,81 @@ SetRateLimiterConfig sets RateLimiterConfig field to given value.
HasRateLimiterConfig returns a boolean if a field has been set.
### GetOffloadTso
`func (o *NetConfig) GetOffloadTso() bool`
GetOffloadTso returns the OffloadTso field if non-nil, zero value otherwise.
### GetOffloadTsoOk
`func (o *NetConfig) GetOffloadTsoOk() (*bool, bool)`
GetOffloadTsoOk returns a tuple with the OffloadTso field if it's non-nil, zero value otherwise
and a boolean to check if the value has been set.
### SetOffloadTso
`func (o *NetConfig) SetOffloadTso(v bool)`
SetOffloadTso sets OffloadTso field to given value.
### HasOffloadTso
`func (o *NetConfig) HasOffloadTso() bool`
HasOffloadTso returns a boolean if a field has been set.
### GetOffloadUfo
`func (o *NetConfig) GetOffloadUfo() bool`
GetOffloadUfo returns the OffloadUfo field if non-nil, zero value otherwise.
### GetOffloadUfoOk
`func (o *NetConfig) GetOffloadUfoOk() (*bool, bool)`
GetOffloadUfoOk returns a tuple with the OffloadUfo field if it's non-nil, zero value otherwise
and a boolean to check if the value has been set.
### SetOffloadUfo
`func (o *NetConfig) SetOffloadUfo(v bool)`
SetOffloadUfo sets OffloadUfo field to given value.
### HasOffloadUfo
`func (o *NetConfig) HasOffloadUfo() bool`
HasOffloadUfo returns a boolean if a field has been set.
### GetOffloadCsum
`func (o *NetConfig) GetOffloadCsum() bool`
GetOffloadCsum returns the OffloadCsum field if non-nil, zero value otherwise.
### GetOffloadCsumOk
`func (o *NetConfig) GetOffloadCsumOk() (*bool, bool)`
GetOffloadCsumOk returns a tuple with the OffloadCsum field if it's non-nil, zero value otherwise
and a boolean to check if the value has been set.
### SetOffloadCsum
`func (o *NetConfig) SetOffloadCsum(v bool)`
SetOffloadCsum sets OffloadCsum field to given value.
### HasOffloadCsum
`func (o *NetConfig) HasOffloadCsum() bool`
HasOffloadCsum returns a boolean if a field has been set.
[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)

View File

@@ -0,0 +1,82 @@
# VmResizeDisk
## Properties
Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
**Id** | Pointer to **string** | disk identifier | [optional]
**DesiredSize** | Pointer to **int64** | desired disk size in bytes | [optional]
## Methods
### NewVmResizeDisk
`func NewVmResizeDisk() *VmResizeDisk`
NewVmResizeDisk instantiates a new VmResizeDisk object
This constructor will assign default values to properties that have it defined,
and makes sure properties required by API are set, but the set of arguments
will change when the set of required properties is changed
### NewVmResizeDiskWithDefaults
`func NewVmResizeDiskWithDefaults() *VmResizeDisk`
NewVmResizeDiskWithDefaults instantiates a new VmResizeDisk object
This constructor will only assign default values to properties that have it defined,
but it doesn't guarantee that properties required by API are set
### GetId
`func (o *VmResizeDisk) GetId() string`
GetId returns the Id field if non-nil, zero value otherwise.
### GetIdOk
`func (o *VmResizeDisk) GetIdOk() (*string, bool)`
GetIdOk returns a tuple with the Id field if it's non-nil, zero value otherwise
and a boolean to check if the value has been set.
### SetId
`func (o *VmResizeDisk) SetId(v string)`
SetId sets Id field to given value.
### HasId
`func (o *VmResizeDisk) HasId() bool`
HasId returns a boolean if a field has been set.
### GetDesiredSize
`func (o *VmResizeDisk) GetDesiredSize() int64`
GetDesiredSize returns the DesiredSize field if non-nil, zero value otherwise.
### GetDesiredSizeOk
`func (o *VmResizeDisk) GetDesiredSizeOk() (*int64, bool)`
GetDesiredSizeOk returns a tuple with the DesiredSize field if it's non-nil, zero value otherwise
and a boolean to check if the value has been set.
### SetDesiredSize
`func (o *VmResizeDisk) SetDesiredSize(v int64)`
SetDesiredSize sets DesiredSize field to given value.
### HasDesiredSize
`func (o *VmResizeDisk) HasDesiredSize() bool`
HasDesiredSize returns a boolean if a field has been set.
[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)

View File

@@ -33,6 +33,9 @@ type NetConfig struct {
Id *string `json:"id,omitempty"`
PciSegment *int32 `json:"pci_segment,omitempty"`
RateLimiterConfig *RateLimiterConfig `json:"rate_limiter_config,omitempty"`
OffloadTso *bool `json:"offload_tso,omitempty"`
OffloadUfo *bool `json:"offload_ufo,omitempty"`
OffloadCsum *bool `json:"offload_csum,omitempty"`
}
// NewNetConfig instantiates a new NetConfig object
@@ -55,6 +58,12 @@ func NewNetConfig() *NetConfig {
this.VhostUser = &vhostUser
var vhostMode string = "Client"
this.VhostMode = &vhostMode
var offloadTso bool = true
this.OffloadTso = &offloadTso
var offloadUfo bool = true
this.OffloadUfo = &offloadUfo
var offloadCsum bool = true
this.OffloadCsum = &offloadCsum
return &this
}
@@ -77,6 +86,12 @@ func NewNetConfigWithDefaults() *NetConfig {
this.VhostUser = &vhostUser
var vhostMode string = "Client"
this.VhostMode = &vhostMode
var offloadTso bool = true
this.OffloadTso = &offloadTso
var offloadUfo bool = true
this.OffloadUfo = &offloadUfo
var offloadCsum bool = true
this.OffloadCsum = &offloadCsum
return &this
}
@@ -560,6 +575,102 @@ func (o *NetConfig) SetRateLimiterConfig(v RateLimiterConfig) {
o.RateLimiterConfig = &v
}
// GetOffloadTso returns the OffloadTso field value if set, zero value otherwise.
func (o *NetConfig) GetOffloadTso() bool {
if o == nil || o.OffloadTso == nil {
var ret bool
return ret
}
return *o.OffloadTso
}
// GetOffloadTsoOk returns a tuple with the OffloadTso field value if set, nil otherwise
// and a boolean to check if the value has been set.
func (o *NetConfig) GetOffloadTsoOk() (*bool, bool) {
if o == nil || o.OffloadTso == nil {
return nil, false
}
return o.OffloadTso, true
}
// HasOffloadTso returns a boolean if a field has been set.
func (o *NetConfig) HasOffloadTso() bool {
if o != nil && o.OffloadTso != nil {
return true
}
return false
}
// SetOffloadTso gets a reference to the given bool and assigns it to the OffloadTso field.
func (o *NetConfig) SetOffloadTso(v bool) {
o.OffloadTso = &v
}
// GetOffloadUfo returns the OffloadUfo field value if set, zero value otherwise.
func (o *NetConfig) GetOffloadUfo() bool {
if o == nil || o.OffloadUfo == nil {
var ret bool
return ret
}
return *o.OffloadUfo
}
// GetOffloadUfoOk returns a tuple with the OffloadUfo field value if set, nil otherwise
// and a boolean to check if the value has been set.
func (o *NetConfig) GetOffloadUfoOk() (*bool, bool) {
if o == nil || o.OffloadUfo == nil {
return nil, false
}
return o.OffloadUfo, true
}
// HasOffloadUfo returns a boolean if a field has been set.
func (o *NetConfig) HasOffloadUfo() bool {
if o != nil && o.OffloadUfo != nil {
return true
}
return false
}
// SetOffloadUfo gets a reference to the given bool and assigns it to the OffloadUfo field.
func (o *NetConfig) SetOffloadUfo(v bool) {
o.OffloadUfo = &v
}
// GetOffloadCsum returns the OffloadCsum field value if set, zero value otherwise.
func (o *NetConfig) GetOffloadCsum() bool {
if o == nil || o.OffloadCsum == nil {
var ret bool
return ret
}
return *o.OffloadCsum
}
// GetOffloadCsumOk returns a tuple with the OffloadCsum field value if set, nil otherwise
// and a boolean to check if the value has been set.
func (o *NetConfig) GetOffloadCsumOk() (*bool, bool) {
if o == nil || o.OffloadCsum == nil {
return nil, false
}
return o.OffloadCsum, true
}
// HasOffloadCsum returns a boolean if a field has been set.
func (o *NetConfig) HasOffloadCsum() bool {
if o != nil && o.OffloadCsum != nil {
return true
}
return false
}
// SetOffloadCsum gets a reference to the given bool and assigns it to the OffloadCsum field.
func (o *NetConfig) SetOffloadCsum(v bool) {
o.OffloadCsum = &v
}
func (o NetConfig) MarshalJSON() ([]byte, error) {
toSerialize := map[string]interface{}{}
if o.Tap != nil {
@@ -607,6 +718,15 @@ func (o NetConfig) MarshalJSON() ([]byte, error) {
if o.RateLimiterConfig != nil {
toSerialize["rate_limiter_config"] = o.RateLimiterConfig
}
if o.OffloadTso != nil {
toSerialize["offload_tso"] = o.OffloadTso
}
if o.OffloadUfo != nil {
toSerialize["offload_ufo"] = o.OffloadUfo
}
if o.OffloadCsum != nil {
toSerialize["offload_csum"] = o.OffloadCsum
}
return json.Marshal(toSerialize)
}

View File

@@ -0,0 +1,151 @@
/*
Cloud Hypervisor API
Local HTTP based API for managing and inspecting a cloud-hypervisor virtual machine.
API version: 0.3.0
*/
// Code generated by OpenAPI Generator (https://openapi-generator.tech); DO NOT EDIT.
package openapi
import (
"encoding/json"
)
// VmResizeDisk struct for VmResizeDisk
type VmResizeDisk struct {
// disk identifier
Id *string `json:"id,omitempty"`
// desired disk size in bytes
DesiredSize *int64 `json:"desired_size,omitempty"`
}
// NewVmResizeDisk instantiates a new VmResizeDisk object
// This constructor will assign default values to properties that have it defined,
// and makes sure properties required by API are set, but the set of arguments
// will change when the set of required properties is changed
func NewVmResizeDisk() *VmResizeDisk {
this := VmResizeDisk{}
return &this
}
// NewVmResizeDiskWithDefaults instantiates a new VmResizeDisk object
// This constructor will only assign default values to properties that have it defined,
// but it doesn't guarantee that properties required by API are set
func NewVmResizeDiskWithDefaults() *VmResizeDisk {
this := VmResizeDisk{}
return &this
}
// GetId returns the Id field value if set, zero value otherwise.
func (o *VmResizeDisk) GetId() string {
if o == nil || o.Id == nil {
var ret string
return ret
}
return *o.Id
}
// GetIdOk returns a tuple with the Id field value if set, nil otherwise
// and a boolean to check if the value has been set.
func (o *VmResizeDisk) GetIdOk() (*string, bool) {
if o == nil || o.Id == nil {
return nil, false
}
return o.Id, true
}
// HasId returns a boolean if a field has been set.
func (o *VmResizeDisk) HasId() bool {
if o != nil && o.Id != nil {
return true
}
return false
}
// SetId gets a reference to the given string and assigns it to the Id field.
func (o *VmResizeDisk) SetId(v string) {
o.Id = &v
}
// GetDesiredSize returns the DesiredSize field value if set, zero value otherwise.
func (o *VmResizeDisk) GetDesiredSize() int64 {
if o == nil || o.DesiredSize == nil {
var ret int64
return ret
}
return *o.DesiredSize
}
// GetDesiredSizeOk returns a tuple with the DesiredSize field value if set, nil otherwise
// and a boolean to check if the value has been set.
func (o *VmResizeDisk) GetDesiredSizeOk() (*int64, bool) {
if o == nil || o.DesiredSize == nil {
return nil, false
}
return o.DesiredSize, true
}
// HasDesiredSize returns a boolean if a field has been set.
func (o *VmResizeDisk) HasDesiredSize() bool {
if o != nil && o.DesiredSize != nil {
return true
}
return false
}
// SetDesiredSize gets a reference to the given int64 and assigns it to the DesiredSize field.
func (o *VmResizeDisk) SetDesiredSize(v int64) {
o.DesiredSize = &v
}
func (o VmResizeDisk) MarshalJSON() ([]byte, error) {
toSerialize := map[string]interface{}{}
if o.Id != nil {
toSerialize["id"] = o.Id
}
if o.DesiredSize != nil {
toSerialize["desired_size"] = o.DesiredSize
}
return json.Marshal(toSerialize)
}
type NullableVmResizeDisk struct {
value *VmResizeDisk
isSet bool
}
func (v NullableVmResizeDisk) Get() *VmResizeDisk {
return v.value
}
func (v *NullableVmResizeDisk) Set(val *VmResizeDisk) {
v.value = val
v.isSet = true
}
func (v NullableVmResizeDisk) IsSet() bool {
return v.isSet
}
func (v *NullableVmResizeDisk) Unset() {
v.value = nil
v.isSet = false
}
func NewNullableVmResizeDisk(val *VmResizeDisk) *NullableVmResizeDisk {
return &NullableVmResizeDisk{value: val, isSet: true}
}
func (v NullableVmResizeDisk) MarshalJSON() ([]byte, error) {
return json.Marshal(v.value)
}
func (v *NullableVmResizeDisk) UnmarshalJSON(src []byte) error {
v.isSet = true
return json.Unmarshal(src, &v.value)
}

View File

@@ -163,6 +163,22 @@ paths:
429:
description: The VM instance could not be resized because a cpu removal is still pending.
/vm.resize-disk:
put:
summary: Resize a disk
requestBody:
description: Resizes a disk attached to the VM
content:
application/json:
schema:
$ref: "#/components/schemas/VmResizeDisk"
required: true
responses:
204:
description: The disk was successfully resized.
500:
description: The disk could not be resized.
/vm.resize-zone:
put:
summary: Resize a memory zone
@@ -966,6 +982,15 @@ components:
format: int16
rate_limiter_config:
$ref: "#/components/schemas/RateLimiterConfig"
offload_tso:
type: boolean
default: true
offload_ufo:
type: boolean
default: true
offload_csum:
type: boolean
default: true
RngConfig:
required:
@@ -1194,6 +1219,17 @@ components:
type: integer
format: int64
VmResizeDisk:
type: object
properties:
id:
description: disk identifier
type: string
desired_size:
description: desired disk size in bytes
type: integer
format: int64
VmResizeZone:
type: object
properties:

View File

@@ -95,6 +95,7 @@ function create_cluster() {
local short_sha
local tags
local rg
local aks_create
# First ensure it didn't fail to get cleaned up from a previous run.
delete_cluster "${test_type}" || true
@@ -117,19 +118,16 @@ function create_cluster() {
# Required by e.g. AKS App Routing for KBS installation.
az extension add --name aks-preview
# Adding a double quote on the last line ends up causing issues
# ine the cbl-mariner installation. Because of that, let's just
# disable the warning for this specific case.
# shellcheck disable=SC2046
az aks create \
-g "${rg}" \
--node-resource-group "node-${rg}" \
-n "$(_print_cluster_name "${test_type}")" \
-s "$(_print_instance_type)" \
--node-count 1 \
--generate-ssh-keys \
--tags "${tags[@]}" \
$([[ "${KATA_HOST_OS}" = "cbl-mariner" ]] && echo "--os-sku AzureLinux --workload-runtime KataVmIsolation")
# Create the cluster.
aks_create=(az aks create
-g "${rg}"
--node-resource-group "node-${rg}"
-n "$(_print_cluster_name "${test_type}")"
-s "$(_print_instance_type)"
--node-count 1
--generate-ssh-keys
--tags "${tags[@]}")
"${aks_create[@]}"
}
function install_bats() {
@@ -397,8 +395,27 @@ EOF
sudo apt-get -y install kubeadm kubelet kubectl --allow-downgrades
sudo apt-mark hold kubeadm kubelet kubectl
# Deploy k8s using kubeadm
sudo kubeadm init --pod-network-cidr=10.244.0.0/16
# Deploy k8s using kubeadm with CreateContainerRequest (CRI) timeout set to 600s,
# mainly for CoCo (Confidential Containers) tests (attestation, policy, image pull, VM start).
local kubeadm_config
kubeadm_config="$(mktemp --tmpdir kubeadm-config.XXXXXX.yaml)"
cat <<EOF | tee "${kubeadm_config}"
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
nodeRegistration:
criSocket: "/run/containerd/containerd.sock"
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
networking:
podSubnet: "10.244.0.0/16"
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
runtimeRequestTimeout: "600s"
EOF
sudo kubeadm init --config "${kubeadm_config}"
rm -f "${kubeadm_config}"
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
@@ -419,6 +436,14 @@ function deploy_vanilla_k8s() {
[[ -z "${container_engine}" ]] && die "container_engine is required"
[[ -z "${container_engine_version}" ]] && die "container_engine_version is required"
# Resolve lts/active to the actual version from versions.yaml (e.g. v1.7, v2.1)
case "${container_engine_version}" in
lts|active)
container_engine_version=$(get_from_kata_deps ".externals.containerd.${container_engine_version}")
;;
*) ;;
esac
install_system_dependencies "runc"
load_k8s_needed_modules
set_k8s_network_parameters

View File

@@ -36,6 +36,7 @@ export PULL_TYPE="${PULL_TYPE:-default}"
export TEST_CLUSTER_NAMESPACE="${TEST_CLUSTER_NAMESPACE:-kata-containers-k8s-tests}"
export GENPOLICY_PULL_METHOD="${GENPOLICY_PULL_METHOD:-oci-distribution}"
export TARGET_ARCH="${TARGET_ARCH:-x86_64}"
export RUNS_ON_AKS="${RUNS_ON_AKS:-false}"
function configure_devmapper() {
sudo mkdir -p /var/lib/containerd/devmapper
@@ -555,18 +556,22 @@ function main() {
export KATA_HOST_OS="${KATA_HOST_OS:-}"
export K8S_TEST_HOST_TYPE="${K8S_TEST_HOST_TYPE:-}"
AUTO_GENERATE_POLICY="${AUTO_GENERATE_POLICY:-}"
if [[ "${KATA_HOST_OS}" = "cbl-mariner" ]]; then
# Temporary workaround for missing cloud-hypervisor/cloud-hypervisor@bf6f0f8, the fix for a bug
# exposed by the large ttrpc replies intentionally produced by the Kata CI Policy tests.
AUTO_GENERATE_POLICY="no"
else
AUTO_GENERATE_POLICY="${AUTO_GENERATE_POLICY:-}"
# Auto-generate policy on some Host types, if the caller didn't specify an AUTO_GENERATE_POLICY value.
if [[ -z "${AUTO_GENERATE_POLICY}" ]]; then
if [[ "${KATA_HOST_OS}" = "cbl-mariner" ]]; then
AUTO_GENERATE_POLICY="yes"
elif [[ "${KATA_HYPERVISOR}" = qemu-coco-dev* && \
"${TARGET_ARCH}" = "x86_64" && \
"${PULL_TYPE}" != "experimental-force-guest-pull" ]]; then
AUTO_GENERATE_POLICY="yes"
elif [[ "${KATA_HYPERVISOR}" = qemu-nvidia-gpu-* ]]; then
AUTO_GENERATE_POLICY="yes"
# Auto-generate policy on some Host types, if the caller didn't specify an AUTO_GENERATE_POLICY value.
if [[ -z "${AUTO_GENERATE_POLICY}" ]]; then
if [[ "${KATA_HYPERVISOR}" = qemu-coco-dev* && \
"${TARGET_ARCH}" = "x86_64" && \
"${PULL_TYPE}" != "experimental-force-guest-pull" ]]; then
AUTO_GENERATE_POLICY="yes"
elif [[ "${KATA_HYPERVISOR}" = qemu-nvidia-gpu-* ]]; then
AUTO_GENERATE_POLICY="yes"
fi
fi
fi

View File

@@ -8,12 +8,17 @@
load "${BATS_TEST_DIRNAME}/lib.sh"
load "${BATS_TEST_DIRNAME}/confidential_common.sh"
export SNAPSHOTTER="${SNAPSHOTTER:-}"
export EXPERIMENTAL_FORCE_GUEST_PULL="${EXPERIMENTAL_FORCE_GUEST_PULL:-}"
setup() {
if ! is_confidential_runtime_class; then
skip "Test not supported for ${KATA_HYPERVISOR}."
fi
[ "${SNAPSHOTTER:-}" = "nydus" ] || skip "None snapshotter was found but this test requires one"
if [ "${SNAPSHOTTER}" != "nydus" ] && [ -z "${EXPERIMENTAL_FORCE_GUEST_PULL}" ]; then
skip "Either SNAPSHOTTER=nydus or EXPERIMENTAL_FORCE_GUEST_PULL must be set for this test"
fi
setup_common || die "setup_common failed"
unencrypted_image="quay.io/prometheus/busybox:latest"
@@ -87,9 +92,6 @@ setup() {
}
@test "Test we can pull an image inside the guest using trusted storage" {
[ "$(uname -m)" == "s390x" ] && skip "See: https://github.com/kata-containers/kata-containers/issues/10838"
[ "${KATA_HYPERVISOR}" == "qemu-snp" ] && skip "See: https://github.com/kata-containers/kata-containers/issues/10838"
[ "${KATA_HYPERVISOR}" == "qemu-tdx" ] && skip "See: https://github.com/kata-containers/kata-containers/issues/10838"
# The image pulled in the guest will be downloaded and unpacked in the `/run/kata-containers/image` directory.
# The tests will use `cryptsetup` to encrypt a block device and mount it at `/run/kata-containers/image`.
@@ -107,14 +109,18 @@ setup() {
pod_config=$(mktemp "${BATS_FILE_TMPDIR}/$(basename "${pod_config_template}").XXX")
IMAGE="$image_pulled_time_less_than_default_time" NODE_NAME="$node" envsubst < "$pod_config_template" > "$pod_config"
# Set CreateContainerRequest timeout for qemu-coco-dev
if [[ "${KATA_HYPERVISOR}" == qemu-coco-dev* ]]; then
create_container_timeout=300
set_metadata_annotation "$pod_config" \
"io.katacontainers.config.runtime.create_container_timeout" \
"${create_container_timeout}"
# Set CreateContainerRequest timeout in the annotation to allow for enough time for guest-pull where
# the container remains in 'creating' state until the pull completes. Usually pulling this and the large image in
# below test takes 30-60 seconds, but we occasionally observe spikes on all our bare-metal runners.
create_container_timeout=300
# On AKS, so far, these spikes have not been observed. Issue 10299, as referenced in other parts of this test, tells us
# that we cannot modify the runtimeRequestTimeout on AKS. We hence set the timeout to the 120s default value.
if [[ "${KATA_HYPERVISOR}" == qemu-coco-dev* ]] && [ "${KBS_INGRESS}" = "aks" ]; then
create_container_timeout=120
fi
set_metadata_annotation "$pod_config" \
"io.katacontainers.config.runtime.create_container_timeout" \
"${create_container_timeout}"
# Set annotation to pull image in guest
set_metadata_annotation "${pod_config}" \
@@ -126,16 +132,14 @@ setup() {
cat $pod_config
add_allow_all_policy_to_yaml "$pod_config"
local wait_time=120
[[ "${KATA_HYPERVISOR}" == qemu-coco-dev* ]] && wait_time=300
local wait_time=300
if [[ "${KATA_HYPERVISOR}" == qemu-coco-dev* ]] && [ "${KBS_INGRESS}" = "aks" ]; then
wait_time=120
fi
k8s_create_pod "$pod_config" "$wait_time"
}
@test "Test we cannot pull a large image that pull time exceeds createcontainer timeout inside the guest" {
[ "$(uname -m)" == "s390x" ] && skip "See: https://github.com/kata-containers/kata-containers/issues/10838"
[ "${KATA_HYPERVISOR}" == "qemu-snp" ] && skip "See: https://github.com/kata-containers/kata-containers/issues/10838"
[ "${KATA_HYPERVISOR}" == "qemu-tdx" ] && skip "See: https://github.com/kata-containers/kata-containers/issues/10838"
storage_config=$(mktemp "${BATS_FILE_TMPDIR}/$(basename "${storage_config_template}").XXX")
local_device=$(create_loop_device)
LOCAL_DEVICE="$local_device" NODE_NAME="$node" envsubst < "$storage_config_template" > "$storage_config"
@@ -181,10 +185,6 @@ setup() {
}
@test "Test we can pull a large image inside the guest with large createcontainer timeout" {
[ "$(uname -m)" == "s390x" ] && skip "See: https://github.com/kata-containers/kata-containers/issues/10838"
[ "${KATA_HYPERVISOR}" == "qemu-snp" ] && skip "See: https://github.com/kata-containers/kata-containers/issues/10838"
[ "${KATA_HYPERVISOR}" == "qemu-tdx" ] && skip "See: https://github.com/kata-containers/kata-containers/issues/10838"
if [[ "${KATA_HYPERVISOR}" == qemu-coco-dev* ]] && [ "${KBS_INGRESS}" = "aks" ]; then
skip "skip this specific one due to issue https://github.com/kata-containers/kata-containers/issues/10299"
fi
@@ -203,8 +203,8 @@ setup() {
IMAGE="$large_image" NODE_NAME="$node" envsubst < "$pod_config_template" > "$pod_config"
# Set CreateContainerRequest timeout in the annotation to pull large image in guest
create_container_timeout=120
[[ "${KATA_HYPERVISOR}" == qemu-coco-dev* ]] && create_container_timeout=600
# Bare-metal CI runners' kubelets are configured with an equivalent runtimeRequestTimeout of 600s
create_container_timeout=600
set_metadata_annotation "$pod_config" \
"io.katacontainers.config.runtime.create_container_timeout" \
"${create_container_timeout}"
@@ -219,8 +219,7 @@ setup() {
cat $pod_config
add_allow_all_policy_to_yaml "$pod_config"
local wait_time=120
[[ "${KATA_HYPERVISOR}" == qemu-coco-dev* ]] && wait_time=600
local wait_time=600
k8s_create_pod "$pod_config" "$wait_time"
}
@@ -229,7 +228,9 @@ teardown() {
skip "Test not supported for ${KATA_HYPERVISOR}."
fi
[ "${SNAPSHOTTER:-}" = "nydus" ] || skip "None snapshotter was found but this test requires one"
if [ "${SNAPSHOTTER}" != "nydus" ] && [ -z "${EXPERIMENTAL_FORCE_GUEST_PULL}" ]; then
skip "Either SNAPSHOTTER=nydus or EXPERIMENTAL_FORCE_GUEST_PULL must be set for this test"
fi
teardown_common "${node}" "${node_start_time:-}"
kubectl delete --ignore-not-found pvc trusted-pvc

View File

@@ -11,7 +11,7 @@ load "${BATS_TEST_DIRNAME}/tests_common.sh"
setup() {
auto_generate_policy_enabled || skip "Auto-generated policy tests are disabled."
( [ "${KATA_HYPERVISOR}" == "qemu-tdx" ] || [ "${KATA_HYPERVISOR}" == "qemu-snp" ] ) && skip "https://github.com/kata-containers/kata-containers/issues/9846"
[[ "${RUNS_ON_AKS}" == "true" ]] || skip "https://github.com/kata-containers/kata-containers/issues/9846"
setup_common || die "setup_common failed"
pod_name="policy-pod-pvc"
pvc_name="policy-dev"
@@ -58,7 +58,7 @@ test_pod_policy_error() {
teardown() {
auto_generate_policy_enabled || skip "Auto-generated policy tests are disabled."
( [ "${KATA_HYPERVISOR}" == "qemu-tdx" ] || [ "${KATA_HYPERVISOR}" == "qemu-snp" ] ) && skip "https://github.com/kata-containers/kata-containers/issues/9846"
[[ "${RUNS_ON_AKS}" == "true" ]] || skip "https://github.com/kata-containers/kata-containers/issues/9846"
# Debugging information. Don't print the "Message:" line because it contains a truncated policy log.
kubectl describe pod "${pod_name}" | grep -v "Message:"

View File

@@ -194,8 +194,15 @@ assert_pod_fail() {
echo "Waiting for a container to fail"
sleep "${sleep_time}"
elapsed_time=$((elapsed_time+sleep_time))
if [[ $(kubectl get pod "${pod_name}" \
-o jsonpath='{.status.containerStatuses[0].state.waiting.reason}') = *BackOff* ]]; then
waiting_reason=$(kubectl get pod "${pod_name}" \
-o jsonpath='{.status.containerStatuses[0].state.waiting.reason}' 2>/dev/null || true)
terminated_reason=$(kubectl get pod "${pod_name}" \
-o jsonpath='{.status.containerStatuses[0].state.terminated.reason}' 2>/dev/null || true)
# BackOff/CrashLoopBackOff = container repeatedly failed; RunContainerError = e.g. image pull in guest failed
if [[ "${waiting_reason}" == *BackOff* ]] || [[ "${waiting_reason}" == *RunContainerError* ]]; then
return 0
fi
if [[ "${terminated_reason}" == "StartError" ]] || [[ "${terminated_reason}" == "Error" ]]; then
return 0
fi
if [[ "${elapsed_time}" -gt "${duration}" ]]; then

View File

@@ -14,6 +14,7 @@ export AUTO_GENERATE_POLICY="${AUTO_GENERATE_POLICY:-no}"
export KATA_HOST_OS="${KATA_HOST_OS:-}"
export KATA_HYPERVISOR="${KATA_HYPERVISOR:-}"
export PULL_TYPE="${PULL_TYPE:-default}"
export RUNS_ON_AKS="${RUNS_ON_AKS:-false}"
declare -r kubernetes_dir=$(dirname "$(readlink -f "$0")")
declare -r runtimeclass_workloads_work_dir="${kubernetes_dir}/runtimeclass_workloads_work"
@@ -102,13 +103,8 @@ add_annotations_to_yaml() {
add_cbl_mariner_annotation_to_yaml() {
local -r yaml_file="$1"
local -r mariner_annotation_kernel="io.katacontainers.config.hypervisor.kernel"
local -r mariner_kernel_path="/usr/share/cloud-hypervisor/vmlinux.bin"
local -r mariner_annotation_image="io.katacontainers.config.hypervisor.image"
local -r mariner_image_path="/opt/kata/share/kata-containers/kata-containers-mariner.img"
add_annotations_to_yaml "${yaml_file}" "${mariner_annotation_kernel}" "${mariner_kernel_path}"
add_annotations_to_yaml "${yaml_file}" "${mariner_annotation_image}" "${mariner_image_path}"
}

View File

@@ -39,6 +39,7 @@ AUTO_GENERATE_POLICY="${AUTO_GENERATE_POLICY:-}"
GENPOLICY_PULL_METHOD="${GENPOLICY_PULL_METHOD:-}"
KATA_HYPERVISOR="${KATA_HYPERVISOR:-}"
KATA_HOST_OS="${KATA_HOST_OS:-}"
RUNS_ON_AKS="${RUNS_ON_AKS:-false}"
# Common setup for tests.
#
@@ -98,13 +99,11 @@ is_nvidia_gpu_platform() {
}
is_aks_cluster() {
case "${KATA_HYPERVISOR}" in
"qemu-tdx"|"qemu-snp"|qemu-nvidia-gpu*)
return 1
;;
*)
return 0
esac
if [[ "${RUNS_ON_AKS}" = "true" ]]; then
return 0
fi
return 1
}
adapt_common_policy_settings_for_non_coco() {
@@ -172,6 +171,15 @@ adapt_common_policy_settings_for_nvidia_gpu() {
jq '.kata_config.oci_version = "1.2.1"' "${settings_dir}/genpolicy-settings.json" > temp.json && mv temp.json "${settings_dir}/genpolicy-settings.json"
}
# Adapt OCI version in policy settings to match containerd version.
# containerd 2.2.x (active) vendors v1.3.0.
adapt_common_policy_settings_for_containerd_version() {
local settings_dir=${1}
info "Adapting common policy settings for containerd's latest release"
jq '.kata_config.oci_version = "1.3.0"' "${settings_dir}/genpolicy-settings.json" > temp.json && mv temp.json "${settings_dir}/genpolicy-settings.json"
}
# adapt common policy settings for various platforms
adapt_common_policy_settings() {
local settings_dir=$1
@@ -179,6 +187,7 @@ adapt_common_policy_settings() {
is_coco_platform || adapt_common_policy_settings_for_non_coco "${settings_dir}"
is_aks_cluster && adapt_common_policy_settings_for_aks "${settings_dir}"
is_nvidia_gpu_platform && adapt_common_policy_settings_for_nvidia_gpu "${settings_dir}"
[[ -n "${CONTAINER_ENGINE_VERSION:-}" ]] && adapt_common_policy_settings_for_containerd_version "${settings_dir}"
case "${KATA_HOST_OS}" in
"cbl-mariner")

View File

@@ -45,8 +45,8 @@ install_nvidia_fabricmanager() {
return
}
echo "chroot: Install NVIDIA fabricmanager"
eval "${APT_INSTALL}" nvidia-fabricmanager libnvidia-nscq
apt-mark hold nvidia-fabricmanager libnvidia-nscq
eval "${APT_INSTALL}" nvidia-fabricmanager libnvidia-nscq nvlsm
apt-mark hold nvidia-fabricmanager libnvidia-nscq nvlsm
}
install_userspace_components() {

View File

@@ -145,8 +145,8 @@ chisseled_nvswitch() {
mkdir -p usr/share/nvidia/nvswitch
cp -a "${stage_one}"/usr/bin/nv-fabricmanager bin/.
cp -a "${stage_one}"/usr/share/nvidia/nvswitch usr/share/nvidia/.
cp -a "${stage_one}"/usr/bin/nv-fabricmanager bin/.
cp -a "${stage_one}"/usr/share/nvidia/nvswitch usr/share/nvidia/.
libdir=usr/lib/"${machine_arch}"-linux-gnu
@@ -156,6 +156,14 @@ chisseled_nvswitch() {
# if the specified log file can't be opened or the path is empty.
# LOG_FILE_NAME=/var/log/fabricmanager.log -> setting to empty for stderr -> kmsg
sed -i 's|^LOG_FILE_NAME=.*|LOG_FILE_NAME=|' usr/share/nvidia/nvswitch/fabricmanager.cfg
# NVLINK SubnetManager dependencies
local nvlsm=usr/share/nvidia/nvlsm
mkdir -p "${nvlsm}"
cp -a "${stage_one}"/opt/nvidia/nvlsm/lib/libgrpc_mgr.so lib/.
cp -a "${stage_one}"/opt/nvidia/nvlsm/sbin/nvlsm sbin/.
cp -a "${stage_one}/${nvlsm}"/*.conf "${nvlsm}"/.
}
chisseled_dcgm() {

View File

@@ -100,6 +100,7 @@ TOOLS_CONTAINER_BUILDER="${TOOLS_CONTAINER_BUILDER:-}"
VIRTIOFSD_CONTAINER_BUILDER="${VIRTIOFSD_CONTAINER_BUILDER:-}"
AGENT_INIT="${AGENT_INIT:-no}"
MEASURED_ROOTFS="${MEASURED_ROOTFS:-no}"
CONFIDENTIAL_GUEST="${CONFIDENTIAL_GUEST:-no}"
USE_CACHE="${USE_CACHE:-}"
BUSYBOX_CONF_FILE=${BUSYBOX_CONF_FILE:-}
NVIDIA_GPU_STACK="${NVIDIA_GPU_STACK:-}"
@@ -141,6 +142,7 @@ docker run \
--env VIRTIOFSD_CONTAINER_BUILDER="${VIRTIOFSD_CONTAINER_BUILDER}" \
--env AGENT_INIT="${AGENT_INIT}" \
--env MEASURED_ROOTFS="${MEASURED_ROOTFS}" \
--env CONFIDENTIAL_GUEST="${CONFIDENTIAL_GUEST}" \
--env USE_CACHE="${USE_CACHE}" \
--env BUSYBOX_CONF_FILE="${BUSYBOX_CONF_FILE}" \
--env NVIDIA_GPU_STACK="${NVIDIA_GPU_STACK}" \

View File

@@ -43,6 +43,7 @@ readonly se_image_builder="${repo_root_dir}/tools/packaging/guest-image/build_se
ARCH=${ARCH:-$(uname -m)}
BUSYBOX_CONF_FILE="${BUSYBOX_CONF_FILE:-}"
MEASURED_ROOTFS=${MEASURED_ROOTFS:-no}
CONFIDENTIAL_GUEST=${CONFIDENTIAL_GUEST:-no}
USE_CACHE="${USE_CACHE:-"yes"}"
ARTEFACT_REGISTRY="${ARTEFACT_REGISTRY:-ghcr.io}"
ARTEFACT_REPOSITORY="${ARTEFACT_REPOSITORY:-kata-containers}"
@@ -452,6 +453,7 @@ install_image() {
#Install guest image for confidential guests
install_image_confidential() {
export CONFIDENTIAL_GUEST="yes"
if [ "${ARCH}" == "s390x" ]; then
export MEASURED_ROOTFS="no"
else
@@ -563,6 +565,7 @@ install_initrd() {
#Install guest initrd for confidential guests
install_initrd_confidential() {
export CONFIDENTIAL_GUEST="yes"
export MEASURED_ROOTFS="no"
install_initrd "confidential"
}
@@ -593,7 +596,7 @@ install_image_nvidia_gpu() {
export MEASURED_ROOTFS="yes"
local version=$(get_from_kata_deps .externals.nvidia.driver.version)
EXTRA_PKGS="apt curl ${EXTRA_PKGS}"
NVIDIA_GPU_STACK=${NVIDIA_GPU_STACK:-"driver=${version},compute,dcgm"}
NVIDIA_GPU_STACK=${NVIDIA_GPU_STACK:-"driver=${version},compute,dcgm,nvswitch"}
install_image "nvidia-gpu"
}
@@ -603,27 +606,29 @@ install_initrd_nvidia_gpu() {
export MEASURED_ROOTFS="no"
local version=$(get_from_kata_deps .externals.nvidia.driver.version)
EXTRA_PKGS="apt curl ${EXTRA_PKGS}"
NVIDIA_GPU_STACK=${NVIDIA_GPU_STACK:-"driver=${version},compute,dcgm"}
NVIDIA_GPU_STACK=${NVIDIA_GPU_STACK:-"driver=${version},compute,dcgm,nvswitch"}
install_initrd "nvidia-gpu"
}
# Instal NVIDIA GPU confidential image
install_image_nvidia_gpu_confidential() {
export CONFIDENTIAL_GUEST="yes"
export AGENT_POLICY
export MEASURED_ROOTFS="yes"
local version=$(get_from_kata_deps .externals.nvidia.driver.version)
EXTRA_PKGS="apt curl ${EXTRA_PKGS}"
NVIDIA_GPU_STACK=${NVIDIA_GPU_STACK:-"driver=${version},compute,dcgm"}
NVIDIA_GPU_STACK=${NVIDIA_GPU_STACK:-"driver=${version},compute,dcgm,nvswitch"}
install_image "nvidia-gpu-confidential"
}
# Install NVIDIA GPU confidential initrd
install_initrd_nvidia_gpu_confidential() {
export CONFIDENTIAL_GUEST="yes"
export AGENT_POLICY
export MEASURED_ROOTFS="no"
local version=$(get_from_kata_deps .externals.nvidia.driver.version)
EXTRA_PKGS="apt curl ${EXTRA_PKGS}"
NVIDIA_GPU_STACK=${NVIDIA_GPU_STACK:-"driver=${version},compute,dcgm"}
NVIDIA_GPU_STACK=${NVIDIA_GPU_STACK:-"driver=${version},compute,dcgm,nvswitch"}
install_initrd "nvidia-gpu-confidential"
}
@@ -726,10 +731,12 @@ install_kernel() {
local extra_cmd=""
case "${ARCH}" in
s390x)
export CONFIDENTIAL_GUEST="yes"
export MEASURED_ROOTFS="no"
extra_cmd="-x"
;;
x86_64)
export CONFIDENTIAL_GUEST="yes"
export MEASURED_ROOTFS="yes"
extra_cmd="-x"
;;
@@ -741,6 +748,7 @@ install_kernel() {
}
install_kernel_cca_confidential() {
export CONFIDENTIAL_GUEST="yes"
export MEASURED_ROOTFS="yes"
install_kernel_helper \
@@ -765,6 +773,7 @@ install_kernel_nvidia_gpu_dragonball_experimental() {
#Install GPU enabled kernel asset
install_kernel_nvidia_gpu() {
export CONFIDENTIAL_GUEST="yes"
export MEASURED_ROOTFS="yes"
install_kernel_helper \
"assets.kernel.nvidia" \

View File

@@ -520,9 +520,12 @@ build_kernel() {
popd >>/dev/null
if [[ "${gpu_vendor}" == "${VENDOR_NVIDIA}" ]]; then
# We need in-tree modules as well as out-of-tree ones for NVIDIA GPU
make -C "${kernel_path}" -j "$(nproc)" INSTALL_MOD_STRIP=1 INSTALL_MOD_PATH="${kernel_path}" modules_install
pushd open-gpu-kernel-modules
make -j "$(nproc)" CC=gcc SYSSRC="${kernel_path}" > /dev/null
make INSTALL_MOD_STRIP=1 INSTALL_MOD_PATH=${kernel_path} -j "$(nproc)" CC=gcc SYSSRC="${kernel_path}" modules_install
make INSTALL_MOD_STRIP=1 INSTALL_MOD_PATH="${kernel_path}" -j "$(nproc)" CC=gcc SYSSRC="${kernel_path}" modules_install
make -j "$(nproc)" CC=gcc SYSSRC="${kernel_path}" clean > /dev/null
fi
}

View File

@@ -27,3 +27,11 @@ CONFIG_ARM_SMMU_V3_SVA=y
CONFIG_CRYPTO_ECC=y
CONFIG_CRYPTO_ECDH=y
CONFIG_CRYPTO_ECDSA=y
# HGX/DGX platform
CONFIG_INFINIBAND=m
CONFIG_INFINIBAND_USER_MAD=m
CONFIG_NET_VENDOR_MELLANOX=y
CONFIG_MLX5_CORE=m
CONFIG_MLX5_INFINIBAND=m

View File

@@ -26,3 +26,11 @@ CONFIG_CRYPTO_ECDSA=y
# Dependency of _CRYPTO_
CONFIG_MODULE_SIG=y
# HGX/DGX platform
CONFIG_INFINIBAND=m
CONFIG_INFINIBAND_USER_MAD=m
CONFIG_NET_VENDOR_MELLANOX=y
CONFIG_MLX5_CORE=m
CONFIG_MLX5_INFINIBAND=m

View File

@@ -1 +1 @@
181
182

View File

@@ -79,7 +79,7 @@ build_clh_from_source() {
else
./scripts/dev_cli.sh build --release --libc "${libc}"
fi
rm -f cloud-hypervisor
rm -rf cloud-hypervisor
cp build/cargo_target/$(uname -m)-unknown-linux-${libc}/release/cloud-hypervisor .
popd
}

View File

@@ -26,11 +26,12 @@ DESTDIR=${DESTDIR:-${PWD}}
PREFIX=${PREFIX:-/opt/kata}
container_image="${KERNEL_CONTAINER_BUILDER:-$(get_kernel_image_name)}"
MEASURED_ROOTFS=${MEASURED_ROOTFS:-no}
CONFIDENTIAL_GUEST=${CONFIDENTIAL_GUEST:-no}
KBUILD_SIGN_PIN="${KBUILD_SIGN_PIN:-}"
kernel_builder_args="-a ${ARCH:-} $*"
KERNEL_DEBUG_ENABLED=${KERNEL_DEBUG_ENABLED:-"no"}
if [[ "${MEASURED_ROOTFS}" == "yes" ]]; then
if [[ "${MEASURED_ROOTFS}" == "yes" ]] || [[ "${CONFIDENTIAL_GUEST}" == "yes" ]]; then
kernel_builder_args+=" -m"
fi

View File

@@ -78,19 +78,19 @@ mapping:
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-aks / run-k8s-tests (cbl-mariner, clh, normal)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-aks / run-k8s-tests (cbl-mariner, clh, small, containerd)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-aks / run-k8s-tests (cbl-mariner, clh, small, oci-distribution)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-aks / run-k8s-tests (ubuntu, clh, normal)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-aks / run-k8s-tests (ubuntu, clh, small)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-aks / run-k8s-tests (ubuntu, cloud-hypervisor, normal)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-aks / run-k8s-tests (ubuntu, cloud-hypervisor, small)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-aks / run-k8s-tests (ubuntu, dragonball, normal)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-aks / run-k8s-tests (ubuntu, dragonball, small)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-aks / run-k8s-tests (ubuntu, qemu, normal)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-aks / run-k8s-tests (ubuntu, qemu, small)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-aks / run-k8s-tests (ubuntu, qemu-runtime-rs, small)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-aks / run-k8s-tests (ubuntu, qemu-runtime-rs, normal)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-free-runner / run-k8s-tests (clh, lts)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-free-runner / run-k8s-tests (clh, active)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-free-runner / run-k8s-tests (dragonball, lts)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-free-runner / run-k8s-tests (dragonball, active)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-free-runner / run-k8s-tests (qemu, lts)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-free-runner / run-k8s-tests (qemu, active)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-free-runner / run-k8s-tests (qemu-runtime-rs, lts)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-free-runner / run-k8s-tests (qemu-runtime-rs, active)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-free-runner / run-k8s-tests (cloud-hypervisor, lts)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-free-runner / run-k8s-tests (cloud-hypervisor, active)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-zvsi / run-k8s-tests (devmapper, qemu, kubeadm)
- Kata Containers CI / kata-containers-ci-on-push / run-k8s-tests-on-zvsi / run-k8s-tests (nydus, qemu-coco-dev, kubeadm)
# - Kata Containers CI / kata-containers-ci-on-push / run-kata-coco-tests / run-k8s-tests-on-tee (sev-snp, qemu-snp)
- Kata Containers CI / kata-containers-ci-on-push / run-kata-coco-tests / run-k8s-tests-on-tee (sev-snp, qemu-snp)
- Kata Containers CI / kata-containers-ci-on-push / run-kata-coco-tests / run-k8s-tests-coco-nontee (qemu-coco-dev, nydus, guest-pull)
- Kata Containers CI / kata-containers-ci-on-push / run-kata-coco-tests / run-k8s-tests-coco-nontee (qemu-coco-dev-runtime-rs, nydus, guest-pull)
- Kata Containers CI / kata-containers-ci-on-push / run-kata-deploy-tests / run-kata-deploy-tests (qemu, k0s)

View File

@@ -75,7 +75,7 @@ assets:
url: "https://github.com/cloud-hypervisor/cloud-hypervisor"
uscan-url: >-
https://github.com/cloud-hypervisor/cloud-hypervisor/tags.*/v?(\d\S+)\.tar\.gz
version: "v48.0"
version: "v50.0"
firecracker:
description: "Firecracker micro-VMM"
@@ -309,7 +309,7 @@ externals:
# version older than them.
version: "v1.7.25"
lts: "v1.7"
active: "v2.1"
active: "v2.2"
critools:
description: "CLI tool for Container Runtime Interface (CRI)"