For kata tarballs we eventually release to GitHub, check their
size against the GitHub size limit. With this, we fail in case of
an ongoing release process in 'CI | Publish Kata Containers payload'
instead of only later on in the 'Release Kata Containers' action,
and we fail during PR builds, avoiding this situation at all.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
It's a dev-dependency that doesn't seem to be used, so
remove it and resolve RUSTSEC-2025-0052
Assisted-By: Bob
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Bump tracing-subscriber to 0.3.20 to resolve RUSTSEC-2025-0055
- Switch deprecated `slog_info!` for `slog::info!`
Generated-By: Bob
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The old hunspell based spell-check was causing contributors
challenges and proving a barrier to doc updates. We've replaced
it with a cspell based-solution, so clean up the old approach.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Add a separate spellcheck workflow, so we can replace
the complex hunspell approach embedded in static-checks
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
`allow(unused_assignments)` isn't working as it's
in macro generated code, so referencing the command
in the error, to use it
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This test was failing clippy by calling .unwrap() after
an .is_ok(), but after I looked at it, it seemed a bit messy,
so I split it up and tried rewriting it to make it more readable
IMHO.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Calling .unwrap() after an .is_some() check is considered non-idiomatic in
as it performs redundant work and makes the code more verbose.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
As the NVIDIA stack has shifted to using an image for both the
confidential and non-confidential variants, we retire the initrd
build.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
With the new CDH version, the LUKS header is moved off of the disk
into guest memory. We hence adapt the test's filesystem type checks.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
With signature support for sealed secret, use pre-created signed
sealed secrets and provision the signing public key to the KBS.
Add instructions for re-creating these signed secrets.
Improve k8s-sealed-secrets.bats by reducing repeated kubectl logs
calls. A test run showed a SIGPIPE error one one of the grep-logs
while the printouts of the initial kubectl logs invocation showed
that the expected values were actually in the logs.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
With CDH requiring libcryptsetup, mkfs.ext4, dd, and their
dependencies, we will need to update the chiseled NVIDIA rootfs
accordingly.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
With the new CDH version, the secure_mount API changes.
Further, the new CDH version no longer uses the luks-encrypt-storage
script but utilizes libcryptsetup as well as mkfs.ext4 and dd. Hence, adapt
some of the CDH and Kata components build steps
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Replace hardcoded NVIDIA vendor ID (0x10de) and class (0x030) checks
with a vendor-agnostic lookup table (cdiDeviceKind) that maps PCI
vendor/class pairs to CDI device kinds. This makes it straightforward
to add support for new device types by adding entries to the table.
Refactor siblingAnnotation to resolve device BDFs once upfront and
reuse them for both CDI type detection and sibling matching, eliminating
redundant sysfs reads. Devices not in the lookup table (e.g. NVSwitches)
are skipped with errNoSiblingFound, while known device types that fail
to match a sibling produce a hard error.
Consolidate the hot-plug and cold-plug device loops into a single loop
over extracted container paths, removing duplicated filtering logic.
Export GetPCIDeviceProperty from the device drivers package to allow
vendor/class lookup from sysfs in the container annotation path.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
For cold-plug when running with nerdctl the timeouts in the config
are being used, increase the dial_timeout (e.g. for CreateSandbox) to match
create_container_timeout.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Call the setup_genpolicy_registry_auth in run_kubernetes_nv_tests.sh.
Authenticate before exercising any tests.
Recently, we have seen UnauthorizedError messages for the CUDA
vectorAdd image. While this image is not gated behind authentication,
rate limiting may be a possible issue.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
This URL check performed a CURL command to see if it was real. This will
not work in the mkdocs world because the docs might reference a link that
is not yet built on the main page. This is a chicken-and-egg problem.
For reference:
```
ERROR: Invalid URL 'https://kata-containers.github.io/kata-containers/installation/#helm-chart' found in the following files:
tools/packaging/kata-deploy/helm-chart/README.md
```
The markdown reference requirement was put in place for the old docs system, but this
will not apply anymore in the new mkdocs system. I'm removing this
entirely because it will only get in the way and cause confusion.
Signed-off-by: LandonTClipp <11232769+LandonTClipp@users.noreply.github.com>
We disable the Kata artifact builds and testing if the PR is only
related to documentation. Regular static checks will remain.
Signed-off-by: LandonTClipp <11232769+LandonTClipp@users.noreply.github.com>
Moving the genpolicy crate into the root workspace causes the build
outputs to go into the root workspace's target directory, instead of
src/tools/genpolicy/target, invalidating assumptions made by the
kata-deploy-binaries script.
This commit adds a special case for the lookup path of the genpolicy
binary, and fixes two bugs that made identifying this problem harder.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
The yaml-rust dependency is unmaintained, but no suitable alternatives
exist. We log an exception for this now and will revisit the topic after
some time.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
This commit adds the genpolicy utility to the root workspace. For now,
only dependencies that are already in the root workspace are consumed
from there, the genpolicy-specific ones should be added later.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
Before this change, `make test` for runtime-rs used to test all crates
in the root workspace (due to the `--all` flag). This was not intended
but happened to be mostly working. However, genpolicy needs additional
steps before it can build, so this behavior blocks adding genpolicy to
the root workspace.
The solution here is to only build the inteded packages. For the build
and run commands, this is the runtime-rs crate itself. For testing, we
need to include the sub-crates, too, which needs a bit of cargo metadata
scraping.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
These are two changes following a Copilot review on #10559:
1. Restore the p_storage.driver != "blk" check in allow_storage_options():
- An early version of #10599 hardcoded p_storage.driver to "blk".
- Hence that check needed to be removed to validate "blk" storage options.
- The final version of #10599 hardcodes p_storage.driver to "" to
account for both "blk" and "scsi", and checks storage options in
allow_block_storage().
- Hence that check should be restored to preserve the original behavior.
https://github.com/kata-containers/kata-containers/pull/10559#discussion_r2907646552
2. Don't use a regex to validate emptyDir storage mount points:
- It's risky to use a regex to validate a path that has base64-encoded
components.
- We can infer the exact path anyway so the regex is redundant.
https://github.com/kata-containers/kata-containers/pull/10559#discussion_r2907646582
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
1. Reduce the complexity of the new allow_mount rules for emptyDir.
2. Reverse the order of the two allow_mount versions, as a hint to the
rego engine that the first version is more often matching the input.
3. Remove `p_mount.source != ""` from mount_source_allows, because:
- Policy rules typically test the values from input, not values read
from Policy.
- mount_source_allows is no longer called for emptyDir mounts after
these changes, so p_mount.source is not empty.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
k0s uses /var/lib/k0s/kubelet instead of /var/lib/kubelet as its
kubelet data directory. Introduce get_kubelet_data_dir() in
tests_common.sh and use it in k8s-trusted-ephemeral-data-storage.bats
instead of hardcoding /var/lib/kubelet.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
It can be useful to set these variables during local testing:
* AZ_REGION: Region for the cluster.
* AZ_NODEPOOL_TAGS: Node pool tags for the cluster.
* GENPOLICY_BINARY: Path to the genpolicy binary.
* GENPOLICY_SETTINGS_DIR: Directory holding the genpolicy settings.
I've also made it so that tests_common.sh modifies the duplicated
genpolicy-settings.json (used for testing) instead of the original git-tracked
one.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
* Introduces a new cluster_config setting encrypted_emptydir defaulting to true.
* Adapts genpolicy for encrypted emptyDirs.
Crucially, the rules.rego change checks that the mount and the storage are
well-formed together:
* i_storage.source matches a known regex.
* i_storage.mount_point == $(spath)/BASE64(i_storage.source)
* i_storage.mount_point == p_storage.mount_point
* i_storage.mount_point == i_mount.source
Note that policy enforcement is necessary to prevent rogue device injection.
E.g. the agent could not blindly encrypt all block devices as some use cases
only need dm-verity.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Handles block-based emptyDirs plugged via virtio-blk and virtio-scsi by
encrypting and formatting them.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
* Introduces the `emptydir_mode` config flag to allow instructing the runtime
to create a block device for emptyDir volumes.
* The block device is created in the original emptyDir folder on the host
so that Kubelet can monitors its disk usage and evict the pod if it exceeds
its sizeLimit. This matches runc and virtio-fs.
* The block device's disk image file is sparse to minimize host disk
footprint.
Fixes: #10560
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Some VMMs support plugging a disk as an image file instead of a block device,
so we adapt the runtime to support that.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Co-authored-by: Aurélien Bombo <abombo@microsoft.com>
The vhost-kern net unit test used a fixed TAP interface name
("test_vhosttap"). When tests run in parallel or a previous run
leaves the interface behind, TAP creation can fail with
EBUSY ("Resource busy"), making CI flaky.
Introduce a unique_tap_name() helper in the tests and use it to
generate a per-test TAP name (based on pid/thread/counter),
avoiding name collisions and stabilizing CI.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The signal_handler test was intermittently failing because it used
kill(pid, sig), which sends signals asynchronously to the process.
This created a race condition where the child thread could exit and
be joined before the signal was delivered or processed.
This fix including:
1. Replaces `kill` with `libc::raise` to ensure signals are delivered
synchronously to the calling thread.
2. Reorders triggers to verify standard signals before installing
seccomp filters.
3. Guarantees that metrics are incremented before the child thread
terminates and is joined by the main thread.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
warning: unnecessary parentheses around type
--> src/dragonball/dbs_legacy_devices/src/serial.rs:245:39
|
245 | let out: Arc<Mutex<Option<Box<(dyn std::io::Write + Send +
'static)>>>> =
| ^
^
|
= note: `#[warn(unused_parens)]` (part of `#[warn(unused)]`) on by
default
help: remove these parentheses
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The vhost-user-net tests could hang in CI because
VhostUserNet::new_server() blocks indefinitely on listener.accept()
when the slave fails to connect in time
(e.g. due to scheduler delays or flaky socket paths). This also caused
panics when connect_slave() returned None and the test unwrapped it.
Fix the tests by:
- using a `/tmp`, absolute, unique unix socket path per test run
retrying slave connect with a deadline
- running new_server() in a separate thread and waiting via
recv_timeout() to ensure the test never blocks indefinitely
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
It aims to fix flaky test hang by implementing thread timeouts.
The `test_vhost_user_net_virtio_device_normal` was hanging in CI
when master/slave threads drifted.
This commit stabilizes the test by:
- Using `tempfile` and unique paths to ensure socket isolation.
- Adding a 5s deadline for slave connections to handle CI jitter.
- Running `new_server` in a separate thread with a `recv_timeout`
to prevent the CI pipeline from deadlocking.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The `test_inner_stream_timeout` test case was prone to failure due to a
race condition between the main thread and the background handler. The
test relied on hardcoded `thread::sleep` durations, which could cause
the second read operation to time out (150ms window) before the main
thread performed its write (after a 300ms sleep) under high system load.
This commit stabilizes the test by:
1. Replacing fixed sleep durations with a `Condvar` and a `stage`
variable to implement a deterministic state machine.
2. Synchronizing the threads so that the main thread only writes data
after the background handler has confirmed it is ready or has
completed its previous phase.
3. Ensuring the read timeout is explicitly managed between different
validation stages to prevent accidental `TimedOut` errors.
This change eliminates the flakiness and ensures the test passes
consistently across different CIenvironments.
Fixes#12618
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit aims to address issues of "Infinite loop in epoll_manager
tests" and improve stablity.
Root causes as below:
1. Using `handle_events(-1)` caused the worker thread to block forever
if an event was missed or if the internal `kick()` signal was not
accounted for correctly.
2. Relying on event counts was unreliable because internal signals could
fluctuate the total count, causing the it to enter an infinite loop.
3. Using `EventSet::OUT` on an EventFd is often continuously ready,
leading to non-deterministic trigger behavior.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
In some CI runs, `mktemp` generates random characters that accidentally
form file extensions like `.cSV` or `.Xml`. This triggers downstream
parsing errors because the YAML content is misidentified as CSV/XML.
The issues look like as below:
```
'/tmp/bats-run-KodZEA/.../pod-guest-pull-in-trusted-storage.yaml.in.cSV':
...
```
This commit fixes the issue by:
1. Moving the `XXXXXX` placeholder before the `.yaml` extension.
2. Ensuring the generated file always ends in `.yaml`.
This prevents format misidentification while maintaining filename
uniqueness and security.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Let's update the nvidia-container-toolkit to 1.18.1 (from 1.17.6).
We're, from now on, relying on the version set in the versions.yaml
file.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
aws-lc has mutliple high severity CVEs:
- GHSA-vw5v-4f2q-w9xf
- GHSA-65p9-r9h6-22vj
- GHSA-hfpc-8r3f-gw53
so try and bump to the latest `aws-lc-rs` crate to pull in the available fixed versions
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Allow users to override the default RuntimeClass pod overhead for
any shim via shims.<name>.runtimeClass.overhead.{memory,cpu}.
When the field is absent the existing hardcoded defaults from the
dict are used, so this is fully
backward compatible.
Signed-off-by: Zachary Spar <zspar@coreweave.com>
Create local block storage (loop device, StorageClass, PV) in the test
only when the cluster has no default StorageClass, matching the approach
used in k8s-volume.bats. Set our StorageClass as default so the PVC
binds to our PV; tear it down after the test.
When a default already exists (e.g. AKS), skip creation and cleanup so
we do not change the cluster's default storage class.
Fixes: #9846
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Virtio-mmio transport is not hardened for confidential computing (unlike
virtio-pci). Reject config that would use virtio-blk-mmio for rootfs/block
when confidential_guest is set, so CoCo guests only use virtio-blk-pci.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Remove # !confidential from mmio.conf so CONFIG_VIRTIO_MMIO and
CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES are included when building the
unified x86_64/s390x kernel with -x
Firecracker requires virtio-mmio for block devices; without it the
guest kernel panics (no /dev/vda).
Fixes: #12581
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Allow genpolicy -j to accept a directory instead of a single file.
When given a directory, genpolicy loads genpolicy-settings.json from it
and applies all genpolicy-settings.d/*.json files (sorted by name) as
RFC 6902 JSON Patches. This gives precise control over settings with
explicit operations (add, remove, replace, move, copy, test), including
array index manipulation and assertions.
Ship composable drop-in examples in drop-in-examples/:
- 10-* files set platform base settings (non-CoCo, AKS, CBL-Mariner)
- 20-* files overlay specific adjustments (OCI version, guest pull)
Users copy the combination they need into genpolicy-settings.d/.
Replace the old adapt_common_policy_settings_* jq-patching functions
in tests_common.sh with install_genpolicy_drop_ins(), which copies the
right combination of 10-* and 20-* drop-ins for the CI scenario.
Tests still generate 99-test-overrides.json on the fly for per-test
request/exec overrides.
Packaging installs 10-* and 20-* drop-ins from drop-in-examples/ into
the tarball; the default genpolicy-settings.d/ is left empty.
Made-with: Cursor
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Add simple -------- text lines separator to the beginnning of the
allow_mount log output, to help log readers easier separate the ~30
lines of text generated while verifying each mount.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The SNP tests have been unstable on nightlies, but even when these
it seems to be manually cleaned up or something as PR tests are consistently
failing, so we should skip this from the required list until it is reliable.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
kata-deploy's SIGTERM cleanup restarts the CRI runtime, which on
k3s/rke2 takes down the API server temporarily. The helm uninstall
may complete with errors, and the next test suite would start with
a dead API. Add a wait loop after uninstall to ensure the API is
available before proceeding.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
kata-deploy restarts the CRI runtime during install, which can cause
the kata-deploy pod to be killed and recreated by the DaemonSet
controller. On k3s and rke2 in particular, the restart can take
several minutes. Increase the default timeout from 600s (10m) to
900s (15m) to accommodate this.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
kata-deploy restarts the CRI runtime (k3s/containerd) during install,
which can kill the verification job pod or cause transient API server
errors. Bump backoffLimit from 0 to 3 so the job can retry after being
killed, and add a retry loop around kubectl rollout status to handle
transient connection failures.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Move the cleanup logic from a preStop lifecycle hook (separate exec)
into the main process's SIGTERM handler. This simplifies the
architecture: the install process now handles its own teardown when
the pod is terminated.
The SIGTERM handler is registered before install begins, and
tokio::select! races install against SIGTERM so cleanup always runs
even if SIGTERM arrives mid-install (e.g. helm uninstall while the
container is restarting after a failed install attempt).
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
This reverts commit ab25592533, as now
we're deploying k3s/rke2 in a way that we properly test them.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
If the rendered config-v3.toml does not import the drop-in dir, write
the full k3s ContainerdConfigTemplateV3 (with hardcoded import path) so
kata-deploy can use drop-in.
This allows us to test with K3s/RKE2 before my patch there gets
released.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Check the rendered containerd config for the versioned drop-in dir import
(config.toml.d or config-v3.toml.d) and bail with a clear error if it is
missing.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
When I added this I had in mind the period that we still relied on the
SEV module being generated, which we don't do for quite a long time.
This wrong assumption caused the cache to **ALWAYS** fail, increasing
our build time considerably for no reason.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
The remote hypervisor delegates VM creation to a remote service.
The VM runs on cloud infrastructure, not the local host kernel.
So requiring a KVM/MSHV device is semantically wrong and would
cause a hard failure on any host where these devices are absent
(e.g., a VM that doesn't expose nested virtualization).
Skip sandboxDevices() entirely when the configured hypervisor type
is remoteHypervisor{}.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
We depend on a k3s commit so we can properly test it, or we need to
change our CI quite a bit to deploy a full template with that imports
in. For now, let's just skip the testing in k3s/rke2 and we'll address
it in a different PR.
ref:
b51167a996
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
These tests run only on nightly and when triggering the dev CI manually.
They cover both nydus snapshotter with guest-pull and experimental-force-guest-pull,
using qemu-coco-dev and qemu-coco-dev-runtime-rs, and are included in the
run-kata-coco-tests workflow behind the extensive-matrix-autogenerated-policy flag.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
As the default is `info` and that actually overrides whatever is set in
the drop-in file used by k0s.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Align with kubeadm and bare metal by setting the kubelet CRI
runtime-request-timeout to 600s in deploy functions for k0s (worker
profile), k3s (--kubelet-arg), rke2 (config.yaml), and microk8s
(args/kubelet + restart).
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
k3s and rke2 use containerd that expects OCI bundle 1.2.1; otherwise
autogenerated policy tests fail. Add adapt_common_policy_settings_for_k3s_rke2
and call it from adapt_common_policy_settings when KUBERNETES is k3s or rke2.
Tested with k3s v1.34.4+k3s1, rke2 v1.34.4+rke2r1.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
By default, setup-go installs ppc64 binary instead of ppc64le,
resulting in an exec format error. Pass the arch explicitly to fix this.
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
get_kernel() thinks it knows when it needs to skip sha256sum validation for
RC kernels since sha256sums.asc is not available:
INFO: Config version: 176
INFO: Kernel version: 6.18-rc5
INFO: kernel path does not exist, will download kernel
INFO: Release candidate kernels are not part of the official sha256sums.asc -- skipping sha256sum validation
But continues to check it anyway since ${rc} matches
with -n. sha256sum should only be checked when ${rc} is NOT
set.
Fixes a problem where downloaded RC kernels are always removed
and downloaded again.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Sometimes OVFM provides incorrect values to the kernel
we override it by telling the kernel to handle the PCI space setup
like allocating the proper window sizes and assigning the proper busses
to each device.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Stop computing and setting mem-reserve and pref64-reserve on PCIe root
ports and switch ports. Remove getBARsMaxAddressableMemory() which
scanned host GPU BARs to pre-calculate these values.
The previous approach only considered GPU devices (IsGPU(), class
0x0302) when scanning for BAR sizes, so devices like NVSwitches (class
0x0680) with their 32MB non-prefetchable BAR0 were not accounted for
and received the 4MB default. Additionally, GetTotalAddressableMemory()
classifies BARs by 32/64-bit address width rather than by the
prefetchable flag that QEMU's mem-reserve vs pref64-reserve maps to.
Modern QEMU introspects VFIO device BARs when they are attached to
root ports and sizes the MMIO windows accordingly. Modern OVMF
(edk2-stable202502+) automatically calculates the 64-bit PCI MMIO
aperture based on the BARs of actually present devices during PCI
enumeration. Omitting the reserve parameters lets QEMU and OVMF
handle MMIO window sizing correctly for all device types including
GPUs, NVSwitches, and NICs without requiring host-side BAR scanning.
This also removes the nvpci dependency from qemu_arch_base.go.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Modern OVMF (edk2-stable202502 and later) automatically sizes the
64-bit PCI MMIO aperture based on the BARs of actually attached
devices during PCI enumeration. The opt/ovmf/X-PciMmio64Mb fw_cfg
hint is no longer needed to ensure large-BAR devices like NVIDIA
GPUs receive adequate MMIO space.
The previous approach was fragile: the runtime scanned host PCI
devices to estimate the required aperture size, but only considered
GPU devices (class 0x0302), missing NVSwitches and other devices
with large BARs. Removing this code avoids confusion about MMIO
sizing responsibility.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
On containerd v3 config, disable_snapshot_annotations must be set under the
images plugin, not the runtime plugin.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Different kubernetes distributions, such as k0s, use a different kubelet
root dir location instead of the default /var/lib/kubelet, so ConfigMap
and Secret volume propagation were failing.
This adds a kubelet_root_dir config option that the go runtime uses when
matching volume paths and kata-deploy now sets it automatically for k0s
via a drop-in file.
runtime-rs does not need this option: it identifies ConfigMap/Secret,
projected, and downward-api volumes by volume-type path segment
(kubernetes.io~configmap, etc.), not by kubelet root prefix.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Having a script to install go is legacy from Jenkins, so
delete it, so there is less code in our repo.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The host path of bundles is not portable and could be literally anything
depending on containerd configuration, so we can't rely on a specific
prefix when deriving the bundle-id. Instead, we derive the bundle-id
from the target root path in the guest.
NOTE: fixes https://github.com/kata-containers/kata-containers/issues/10065
Signed-off-by: Markus Rudy <mr@edgeless.systems>
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
This commit adds logic to properly handle memory hotplug
for QemuCCWVirtio in the ExecMemdevAdd() path.
The new logic is triggered only when virtio-mem is enabled.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
ResizeMemory() already contains the virtio-mem resize logic.
However, hotplugAddMemory(), which is invoked via a different
path, lacked this handling and always fell back to the pc-dimm
path, even when virtio-mem was configured.
This commit adds virtio-mem resize handling to hotplugAddMemory().
It also adds corresponding unit tests.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
When image.reference or kubectlImage.reference already contains a digest
(e.g. quay.io/...@sha256:...), use the reference as-is instead of
appending :tag. This avoids invalid image strings like 'image@sha256🔤'
when tag is empty and allows users to pin by digest.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
The actionlint gh extension is outdated and the wrapping seems
unnecessary when there is a github action that seems to be maintained,
so let's update to use that
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
cargo machete has identified the follow crates as unused:
- containerd-shim-protos
- safe-path
- strum
- ttrpc
strum is neded (and maybe isn't picked up due to it being
used by macros?), so add it to the ignore list and remove
the rest
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
`cargo machete` has identified `openssl` and `serde-transcode`
as being un-used. openssl is required, so add it to the ignore
list and just remove serde-transcode
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
`log` and `rustjail` are flagged by cargo machete as unused,
so lets remove them to reduce the footprint of crates in this tool
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Remove unused crates to reduce our size and the work needed
to do updates
- Also update package.metadata.cargo-machete with some crates
that are incorrectly coming up as unused
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
cargo machete can't understand `host-device = ["dep:vfio-bindings"`,
so tell it to ignore `vfio-bindings` and not suggest it's unused
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
I ran cargo machete in trace-forwarder and it suggested that some
of the packages were not used, including a chain with a vulnerability,
so try and remove them to resolve RUSTSEC-2021-0139
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Reinstate mariner host testing - including the Agent Policy tests on
these hosts - now that a new CLH version brought in the required fixes.
This reverts commit ea53779b90.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Specify raw image format for all guest block devices.
- Attempting to auto-detect the image format from CLH would be riskier
for the Host.
- Creating a new raw image file, auto-detecting its format, and then
creating a filesystem from the Guest onto the block device is no
longer supported by CLH v51. Therefore, Kata CI's k8s-block-volume.bats
would fail without specifying the raw format when hot plugging its block
device.
- See cloud-hypervisor/cloud-hypervisor@b3e8e2a for additional information.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Specify raw image format for all guest block devices.
- Attempting to auto-detect the image format from CLH would be riskier
for the Host.
- Creating a new raw image file, auto-detecting its format, and then
creating a filesystem from the Guest onto the block device is no
longer supported by CLH v51. Therefore, Kata CI's k8s-block-volume.bats
would fail without specifying the raw format when hot plugging its block
device.
- See cloud-hypervisor/cloud-hypervisor@b3e8e2a for additional information.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
```
v51.1
=====
This is a bug fix release. The following issues have been addressed:
* Fix image_type in OpenAPI definition (#7734)
v51.0
=====
This release has been tracked in v51.0 group of our roadmap project.
Security Fixes
This release fixes a security vulnerability in disk image handling.
Details can be found in GHSA-jmr4-g2hv-mjj6.
* A new `backing_files=on|off` option has been added to `--disk` to
explicitly control whether QCOW2 backing files are permitted. This
defaults to `off` to prevent the loading of backing files entirely.
(#7685)
* Explicit image type specification via the user interface, removing
reliance on format autodetection (#7728).
* Prevent sector-zero writes for autodetected raw images (#7728).
Significant QCOW2 v3 Improvements
A large number of QCOW2 v3 specification features have been implemented:
* RAW backing file support for QCOW2 overlays (#7570)
* Zero bit in L2 entries (#7627)
* Incompatible feature bit validation (#7612)
* Dirty bit support (#7636)
* Variable refcount widths (1 to 64-bit) (#7633)
* Corrupt bit detection and marking (#7639)
* Autoclear feature bits handling (#7648)
* Thread safety fix for multiple virtio queues (`num_queues > 1`)
(#7661)
* Correct zero-fill for reads beyond backing file size (#7678)
* Live disk resize support (#7687)
ACPI Generic Initiator Support
ACPI Generic Initiator Affinity (SRAT Type 5) support has been added
to associate VFIO-PCI devices with dedicated memory/CPU-less NUMA
nodes. This enables the guest OS to make NUMA-aware memory allocation
decisions for device workloads. A new `device_id` parameter has been
added to `--numa` for specifying VFIO devices. (#7626)
Block Device DISCARD and WRITE_ZEROES Support
The `virtio-blk` device now supports `DISCARD` and `WRITE_ZEROES`
operations for QCOW2 and RAW image formats. This enables thin
provisioning and efficient space reclamation when guests trim
filesystems. A new `sparse=on|off` option has been added to `--disk` to
control disk space management: `sparse=on` (default) enables thin
provisioning with space reclamation, while `sparse=off` provides thick
provisioning with consistent I/O latency. (#7666)
Notable Performance Improvements
* Transparent Huge Pages (THP) support has been extended to cover
anonymous shared memory (`shared=on`) via `madvise`. Previously, THP
was only used for non-shared memory. (#7646)
* The `vhost-user-net` device now uses the default set of vhost-user
virtio features, including `VIRTIO_F_RING_INDIRECT_DESC`, which
provides a performance improvement. (#7653)
MSHV Support Improvements
* Optimize CPU state update after emulation by only updating special
registers when changed (#7603)
* Enable SMT for guests with `threads_per_core > 1` (#7668)
* Stub `save_data_tables()` to unblock VM pause/resume (#7692)
* Handle `GHCB_INFO_SPECIAL_DBGPRINT` VMG exit in SEV-SNP guest exit
handler (#7703)
* Fix CVM boot failure on MSHV (#7548)
* Fix CPU topology detection for multithreaded configurations (#7576)
Notable Bug Fixes
* Fix VFIO device hot-remove leaving group and container file
descriptors open, preventing re-add (#7676)
* Fix snapshot restore when backing file is on read-only storage with
`shared=false` (#7674)
* Enforce `VIRTIO_BLK_F_RO` even if guest does not negotiate it
(#7705)
* Fix read-only block device FLUSH requests from OVMF preventing VMs
from booting (#7706)
* Fix vhost-user device not properly dropping unowned file descriptors
(#7679)
* Fix `vhost-user-block` `get_config` interoperability (#7617)
* Fix vsock TOCTOU race condition by copying packet header from guest
memory before processing (#7530)
* Fix vsock handling of large TX packets spanning multiple data
descriptors (#7680)
* Add `gettid()` to all seccomp filters (#7596)
* Fix MAC address parsing that wrongly allowed `+` instead of hex
characters (#7579)
* Improve UUID parse error message and `--net` fd help text (#7702)
* Fix various inconsistencies in our OpenAPI specification file
(#7716, #7726)
* Various documentation fixes (#7602, #7606)
```
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Trustee now returns the binary SNP TCB claims as hex rather than base64
(for consistency with other platforms). Fortunately, the sev-snp-measure
tool has a flag for setting the output type of the launch digest.
I think hex is the default, but let's keep the flag here to be explicit.
Signed-off-by: Tobin Feldman-Fitzthum <tfeldmanfitz@nvidia.com>
Unfortunately, due to golang/go#75031, there is an issue
that results in `go: no such tool "covdata"`
with a automatically installed 1.25 toolchain, so
the approach to skip the install_go.sh script (which causes
double install problems) didn't work. Try the alternative approach
of using setup-go action, which should do a more comprehensive job
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This commit enables the SEV-SNP guest policy to be explicitly
configured via the runtime configuration in runtime-rs.
To provide both ease of use and maximum flexibility, the following
logic is implemented:
1. If the user provides a custom `snp_guest_policy` in the
configuration, this value is passed directly to the QEMU SEV-SNP
guest object.
2. If the user does not specify a policy, the driver defaults to
`0x30000`, matching QEMU's standard default for SEV-SNP guests.
This enhancement allows users to fine-tune security constraints through
the policy bitmask, while ensuring a sensible and functional default
for standard SNP deployments.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit introduces three new fields to the `SecurityInfo` struct
to support SEV-SNP (Secure Nested Paging) attestation and measurement
capabilities:
(1) `snp_id_block`: A 96-byte Base64-encoded ID block for the
SNP_LAUNCH_FINISH command.
(2) `snp_id_auth`: A 4096-byte Base64-encoded authentication structure
accompanying the ID block.
(3) `snp_guest_policy`: A bitmask for the SNP guest policy, passed to
the SNP_LAUNCH_START command.
These fields enable users to provide identity information to the SNP
firmware, allowing for remote attestation and verified guest launches.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
A bitmask for the SNP guest policy is introduced in ObjectSevSnpGuest
to help pass to Qemu cmdline.
And defaults to 0x30000 (QEMU's default) to maintain standard behavior
it just looks like as: "policy=0x30000"
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Introduce host_memory_mib() with OS-specific implementations
(Linux/Android via nix::sysinfo,
macOS via sysctl) selected at compile time. This improves
portability and allows consistent host memory sizing/validation
across different platforms.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
As the memory related information has been serialized at the sandbox
initalization specially at the moment of parsing configuration toml.
This commit aims to refactor MemoryInfo initialization logics:
(1) Remove memory sizing/host-memory adjustment logic from QEMU cmdline
Memory::new()
(2) Initialize/adjust memory values via kata-types MemoryInfo (single
source of truth)
(3) Replace sysinfo::System::new_with_specifics with
nix::sys::sysinfo::sysinfo() to get host RAM
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Since QEMU v10.0.0 and Linux v6.13, virtio-mem-ccw is supported.
Let's enable the required kernel configs for s390x.
This commit enables `CONFIG_VIRTIO_MEM` and `CONFIG_MEMORY_HOTREMOVE`
to support memory hotplug in the VM guest.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Update Trustee to pickup a few recent features, such as improvements to
TDX attestation configuration, and fixes to our vault/OpenBao backend.
This will also pickup our bump of Trustee to Rust 1.90.0.
We should be able to use this version of Trustee with the current
version of guest-components, which cannot be bumped at the moment due to
development dependencies.
Signed-off-by: Tobin Feldman-Fitzthum <tfeldmanfitz@nvidia.com>
I think that c727332b0e
broke the arm unit test by removing the arm specific overrides,
so update the expected output
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Some of our static checks are hitting issues with duplicate
go versions installed. Given that we in go.mod we set the
version to match our required toolchain, if go is already installed
we can let go handle the toolchain version management instead
of installing a second version
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Fix `T1005: error strings should not be capitalized (staticcheck)`
This is to comply with go conventitions as errors are normally appended,
so there would be a spurious captialisation in the middle of the message
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
strings.ReplaceAll was introduced in Go 1.12 as a more readable and self-documenting way to say "replace everything".
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
strings.ReplaceAll was introduced in Go 1.12 as a more readable and self-documenting way to say "replace everything".
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
QF1001: Distributing negation across terms and flipping operators, makes it
easy for humans to process expressions at a time, vs evaluating a whole block
and then flipping it and can allow for earlier exit
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
QF1001: Distributing negation across terms and flipping operators, makes it
easy for humans to process expressions at a time, vs evaluating a whole block
and then flipping it and can allow for earlier exit
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
fixup: demorgans
strings.ReplaceAll was introduced in Go 1.12 as a more readable and self-documenting way to say "replace everything".
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Update from `this` to fix:
```
ST1006: receiver name should be a reflection of its identity; don't use generic names such as "this" or "self" (staticcheck)
```
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
strings.SplitN(s, sep, -1) is functionally identical to strings.Split(s, sep)
as -1 says to return all substrings, so choose the more concise version
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
GenericDevice is an embedded (anonymous) field in the device struct, so its fields
and methods are "promoted" to the outer struct, so we go straight to it.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Add a setting to skip the
`T1005: error strings should not be capitalized (staticcheck)`
rule to avoid impact to our error strings
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Since runtime-rs added support for virtio-blk-ccw on s390x in #12531,
the assertion in k8s-guest-pull-image.bats should be generalized
to apply to all hypervisors ending with `-runtime-rs`.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Specify runAsUser, runAsGroup, supplementalGroups values embedded
in the image's /etc/group file explicitly in the security context.
With this, both genpolicy and containerd, which in case of using
nydus guest-pull, lack image introspection capabilities, use the
same values for user/group/additionalG IDs at policy generation
time and at runtime when the OCI spec is passed.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Extend the timeout for the assert_pod_fail function call for the
test case "Test we cannot pull a large image that pull time exceeds
createcontainer timeout inside the guest" when the experimental
force guest-pull method is being used. In this method, the image is
first pulled on the host before creating the pod sandbox. While
image pull times can suddenly spike, we already time out in the
assert_pod_fail function before the image is even pulled on the
host.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Some constants are duplicated in runtime-rs even though they
are already defined in kata-types. Use the definitions from
kata-types as the single source of truth to avoid inconsistencies
between components (e.g. agent and runtime).
This change makes runtime-rs use the constants defined in
kata-types.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
`DRIVER_BLK_CCW_TYPE` is defined as `blk-ccw`
in src/libs/kata-types/src/device.rs, so set
the variable in runtime-rs accordingly.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
getDefaultShimForArch previously returned whatever string was set in
defaultShim.<arch> without any validation. A typo, a non-existent shim,
or a shim that is disabled via disableAll would all silently produce a
bogus DEFAULT_SHIM_* env var, causing kata-deploy to fail at runtime.
Guard the return value by checking whether the configured shim is
present in the list of shims that are both enabled and support the
requested architecture. If not, return empty string so the env var is
simply omitted.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Using `$runtime.containerd.snapshotter` and `$runtime.crio.pullType`
panics with a nil pointer error when the containerd or crio block is
absent from the custom runtime definition.
Let's use the `dig` function which safely traverses nested keys and
returns an empty string as the default when any key in the path is
missing.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Set KubeletConfiguration runtimeRequestTimeout to 600s mainly for CoCo
(Confidential Containers) tests, so container creation (attestation,
policy, image pull, VM start) does not hit the default CRI timeout.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Treat waiting.reason RunContainerError and terminated.reason StartError/Error
as container failure, so tests that expect guest image-pull failure (e.g.
wrong credentials) pass when the container fails with those states instead
of only BackOff.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Later on we may even think about merging those, but for now let's at
least make sure the envs used are the same / declared in a similar place
for each job.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Run all CoCo non-TEE variants in a single job on the free runner with an
explicit environment matrix (vmm, snapshotter, pull_type, kbs,
containerd_version).
Here we're testing CoCo only with the "active" version of containerd.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
We were running most of the k8s integration tests on AKS. The ones that
don't actually depend on AKS's environment now run on normal
ubuntu-24.04 GitHub runners instead: we bring up a kubeadm cluster
there, test with both containerd lts and active, and skip attestation
tests since those runtimes don't need them. AKS is left only for the
jobs that do depend on it.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Disable mariner host testing in CI, and auto-generated policy testing
for the temporary replacements of these hosts (based on ubuntu), to work
around missing:
1. cloud-hypervisor/cloud-hypervisor@0a5e79a, that will allow Kata
in the future to disable the nested property of guest VPs. Nested
is enabled by default and doesn't work yet with mariner's MSHV.
2. cloud-hypervisor/cloud-hypervisor@bf6f0f8, exposed by the large
ttrpc replies intentionally produced by the Kata CI Policy tests.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Make `az aks create` command easier to change when needed, by moving the
arguments specific to mariner nodes onto a separate line of this script.
This change also removes the need for `shellcheck disable=SC2046` here.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
```
This release has been tracked in v50.0 group of our roadmap project.
Configurable Nested Virtualization Option on x86_64
The nested=on|off option has been added to --cpu to allow users
to configure nested virtualization support in the guest on x86_64
hosts (for both KVM and MSHV). The default value is on to maintain
consistency with existing behavior. (#7408)
Compression Support for QCOW2
QCOW2 support has been extended to handle compression clusters based on
zlib and zstd. (#7462)
Notable Performance Improvements
Performance of live migration has been improved via an optimized
implementation of dirty bitmap maintenance. (#7468)
Live Disk Resizing Support for Raw Images
The /vm.resize-disk API has been introduced to allow users to resize block
devices backed by raw images while a guest is running. (#7476)
Developer Experience Improvements
Significant improvements have been made to developer experience and
productivity. These include a simplified root manifest, codified and
tightened Clippy lints, and streamlined workflows for cargo clippy and
cargo test. (#7489)
Improved File-level Locking Support
Block devices now use byte-range advisory locks instead of whole-file
locks. While both approaches prevent multiple Cloud Hypervisor instances
from simultaneously accessing the same disk image with write
permissions, byte-range locks provide better compatibility with network
storage backends. (#7494)
Logging Improvements
Logs now include event information generated by the event-monitor
module. (#7512)
Notable Bug Fixes
* Fix several issues around CPUID in the guest (#7485, #7495, #7508)
* Fix snapshot/restore for Windows Guest (#7492)
* Respect queue size in block performance tests (#7515)
* Fix several Serial Manager issues (#7502)
* Fix several seccomp violation issues (#7477, #7497, #7518)
* Fix various issues around block and qcow (#7526, #7528, #7537, #7546,
#7549)
* Retrieve MSRs list correctly on MSHV (#7543)
* Fix live migration (and snapshot/restore) with AMX state (#7534)
```
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Issue 10838 is resolved by the prior commit, enabling the -m
option of the kernel build for confidential guests which are
not users of the measured rootfs, and by commit
976df22119, which ensures
relevant user space packages are present.
Not every confidential guest has the measured rootfs option
enabled. Every confidential guest is assumed to support CDH's
secure storage features, in contrast.
We also adjust test timeouts to account for occasional spikes on
our bare metal runners (e.g., SNP, TDX, s390x).
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
After containerd 2.0.4, privileged containers handle sysfs mounts a bit
differently, so we can end up with the policy expecting RO and the input
having RW.
The sandbox needs to get privileged mounts when any container in the pod
is privileged, not only when the pause container itself is marked
privileged. So we now compute that and pass it into get_mounts.
One downside: we’re relaxing policy checks (accepting RO/RW mismatch for
sysfs) and giving the pause container privileged mounts whenever the pod
has any privileged workload. For Kata, that means a slightly broader
attack surface for privileged pods—the pause container sees more than it
strictly needs, and we’re being more permissive on sysfs.
It’s a trade-off for compatibility with newer containerd; if you need
maximum isolation, you may want to avoid privileged pods or tighten
policy elsewhere.
Fixes: #12532
Signed-off-by: Markus Rudy <mr@edgeless.systems>
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
When kata-deploy is deployed with cloud-api-adaptor, it
defaults to qemu instead of configuring the remote shim.
Support ppc64le to enable it correctly when shims.remote.enabled=true
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
The VcpuThreadIds struct expects a mapping from vcpu_id to thread_id,
but get_ch_vcpu_tids() was inserting (tid, vcpu_id) instead of
(vcpu_id, tid).
This caused move_vcpus_to_sandbox_cgroup() to interpret vcpu IDs
(0, 1, 2...) as process IDs when sandbox_cgroup_only=false, leading
to failed attempts to read /proc/0/status.
Fixes: #12479
Signed-off-by: Chiranjeevi Uddanti <244287281+chiranjeevi-max@users.noreply.github.com>
Nowadays on arm64 we use a modern QEMU version which supports the features we
require for NVDIMM, so we remove the arm64-specific code and use the generic
implementation.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This disables virtio-pmem support for Cloud Hypervisor by changing
Kata config defaults and removing the relevant code paths.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Use rstest parameterized tests for QEMU variants, other hypervisors,
and unknown/empty shim cases.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
When copying artifacts from the container to the host, detect source
entries that are symlinks and recreate them as symlinks at the
destination instead of copying the target file.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
This test uses YAML files from a different directory than the other
k8s CI tests, so annotations have to be added into these separate
files.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This change adds the CONFIDENTIAL_GUEST variable to the kernel
build logic. Similar to commit
976df22119, we would like to enable
the cryptsetup functionalities not only when building a measured
root file system, but also when building for a confidential guest.
The current state is that not all confidential guests use a
measured root filesystem, and as a matter of fact, we should
indeed decouple these aspects.
With the current convention, a confidential guest is a user of CDH
with its storage features. A better naming of the
CONFIDENTIAL_GUEST variable could have been a naming related to CDH
storage functionality. Further, the kernel build script's -m
parameter could be improved too - as indicated by this change, not
only measured rootfs builds will need the cryptsetup.conf file.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Similar to k8s-guest-pull-image-authenticated and to
k8s-guest-pull-image-signature, enabling k8s-guest-pull-image to
run against the experimental force guest pull method.
Only k8s-guest-pull-image-encrypted requires nydus.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
When qemu exits prematurely, we usually see a message like
msg="Cannot start VM" error="exiting QMP loop, command cancelled"
This is an indirect hint, caused by the QMP server shutting down. It
takes experience to understand what it even means, and it still does not
show what's actually the problem.
With this commit, we're taking the error return from the qemu
subprocess and surface it in the logs, if it's not nil. This means we
automatically capture any non-zero exit codes in the logs.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
Turn test_toml_value_types into a parameterized test with one case per type
(string, bool, int). Merge the two invalid-TOML tests (get and set) into one
rstest with two cases, and the two "not an array" tests into one rstest
with two cases.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
When writing containerd drop-in or other TOML (e.g. initially empty file),
the serialized document could start with many newlines.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Replace multiple #[test] functions for snapshotter and erofs version
checks with parameterized #[rstest] #[case] tests for consistency and
easier extension.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
With some older kernels some fs implementations don't handle empty
options strings well. This leads to failures in "setup rootfs" step.
E.g. `cgroup: cgroup2: unknown option ""`.
This is fixed by mapping empty string to `None` before passing to
`nix::mount`.
Signed-off-by: Jacek Tomasiak <jtomasiak@arista.com>
Signed-off-by: Jacek Tomasiak <jacek.tomasiak@gmail.com>
Disable provenance and SBOM when building per-arch kata-deploy images so
each tag is a single image manifest. quay.io rejects pushing multi-arch
manifest lists that include attestation manifests (400 manifest invalid).
Add a note in the release script documenting this.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
The default RuntimeClass (e.g. kata) is meant to point at the default shim
handler (e.g. kata-qemu-$tee). We were building it in a separate block and
only sometimes adding the same TEE nodeSelectors as the shim-specific
RuntimeClass, leading to kata ending up without the SE/SNP/TDX
nodeSelector while kata-qemu-$tee had it.
The fix is to stop duplicating the RuntimeClass definition, having a
single template that renders one RuntimeClass (name, handler, overhead,
nodeSelectors).
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
When NFD is detected (deployed by the chart or existing in the cluster),
apply shim-specific nodeSelectors only for TEE runtime classes (snp,
tdx, and se).
Non-TEE shims keep existing behavior (e.g. runtimeClass.nodeSelector for
nvidia GPU from f3bba0885 is unchanged).
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Per-arch images were failing publish-multiarch-manifest with 'X is a manifest
list' because Buildx now enables attestations by default, so each arch tag
became an image index. Use 'docker buildx imagetools create' instead of
'docker manifest create' so we can merge those indexes into the final
multi-arch manifest while keeping provenance and SBOM on per-arch images.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
We depend on GPU Operator v26.3 release, which is not out yet.
Although we have been testing with it, it's not yet publicly available,
which would break anyone actually trying to use the GPU runtime classes.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
According to shimv2 proto, it should send task delete event to
containerd once a container task delete succesfully.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
According to shimv2 proto, it should send task create event to
containerd once a container task create succesfully.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
According to shimv2 proto, it should send task start event to
containerd once a container task start succesfully.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Enhance k8s-configmap.bats and k8s-credentials-secrets.bats to test that ConfigMap and Secret updates propagate to volume-mounted pods.
- Enhanced k8s-configmap.bats to test ConfigMap propagation
* Added volume mount test for ConfigMap consumption
* Added verification that ConfigMap updates propagate to volume-mounted pods
- Enhanced k8s-credentials-secrets.bats to test Secret propagation
* Added verification that Secret updates propagate to volume-mounted pods
Fixes#8015
Signed-off-by: Ajay Victor <ajvictor@in.ibm.com>
In containerd config v3 the CRI plugin is split into runtime and images,
and setting the snapshotter only on the runtime plugin is not enough for image
pull/prepare.
The images plugin must have runtime_platform.<runtime>.snapshotter so it
uses the correct snapshotter per runtime (e.g. nydus, erofs).
A PR on the containerd side is open so we can rely on the runtime plugin
snapshotter alone: https://github.com/containerd/containerd/pull/12836
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Separatly added hypervisor devices to cgroup to
omit not relevant warnings and fail if none of them
are available.
Also fix a testcase reload removed kernel modules to later testcases
and skip some tests on ARM because lack of virtualization support
Fixes#6656
Signed-off-by: Balint Tobik <btobik@redhat.com>
Previously zizmor only mandated pinning of third-party actions,
but has recommended rolling this out to all actions now.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
In a refactor we've remove the `matrix` section of this strategy, so
the whole section isn't needed any more, so clean this up.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
@@ -8,7 +8,7 @@ The following benchmarking result shows the performance improvement compared wit
## Proposal - Bring `lazyload` ability to Kata Containers
`Nydusd` is a fuse/`virtiofs` daemon which is provided by `nydus` project and it supports `PassthroughFS` and [RAFS](https://github.com/dragonflyoss/image-service/blob/master/docs/nydus-design.md) (Registry Acceleration File System) natively, so in Kata Containers, we can use `nydusd` in place of `virtiofsd` and mount `nydus` image to guest in the meanwhile.
`Nydusd` is a fuse/`virtiofs` daemon which is provided by `nydus` project and it supports `PassthroughFS` and [`rafs`](https://github.com/dragonflyoss/image-service/blob/master/docs/nydus-design.md) (Registry Acceleration File System) natively, so in Kata Containers, we can use `nydusd` in place of `virtiofsd` and mount `nydus` image to guest in the meanwhile.
The process of creating/starting Kata Containers with `virtiofsd`,
@@ -18,7 +18,7 @@ The host kernel must be equal to or later than upstream version [6.11](https://c
[`sev-utils`](https://github.com/amd/sev-utils/blob/coco-202501150000/docs/snp.md) is an easy way to install the required host kernel with the `setup-host` command. However, it will also build compatible guest kernel, OVMF, and QEMU components which are not necessary as these components are packaged with kata. The `sev-utils` script utility can be used with these additional components to test the memory encrypted launch and attestation of a base QEMU SNP guest.
For a simplified way to build just the upstream compatible host kernel, use the Confidential Containers fork of [AMDESE AMDSEV](https://github.com/confidential-containers/amdese-amdsev/tree/amd-snp-202501150000). Individual components can be built by running the following command:
For a simplified way to build just the upstream compatible host kernel, use the Confidential Containers fork of [`amdese-amdsev`](https://github.com/confidential-containers/amdese-amdsev/tree/amd-snp-202501150000). Individual components can be built by running the following command:
@@ -49,6 +49,8 @@ In order to allow Kubelet to use containerd (using the CRI interface), configure
EOF
```
For Kata Containers (and especially CoCo / Confidential Containers tests), use at least `--runtime-request-timeout=600s` (10m) so CRI CreateContainerRequest does not time out.
Control the mem-agent compaction function compact threshold.<br>
compact_threshold is the pages number.<br>
When examining the /proc/pagetypeinfo, if there's an increase in the number of movable pages of orders smaller than the compact_order compared to the amount following the previous compaction period, and this increase surpasses a certain threshold specifically, more than compact_threshold number of pages, or the number of free pages has decreased by compact_threshold since the previous compaction. Current compact run period will not do compaction because there is no enough fragmented pages to be compaction.<br>
When examining the `/proc/pagetypeinfo`, if there's an increase in the number of movable pages of orders smaller than the compact_order compared to the amount following the previous compaction period, and this increase surpasses a certain threshold specifically, more than compact_threshold number of pages, or the number of free pages has decreased by compact_threshold since the previous compaction. Current compact run period will not do compaction because there is no enough fragmented pages to be compaction.<br>
This design aims to minimize the impact of unnecessary compaction calls on system performance.<br>
@@ -99,6 +99,9 @@ The [`genpolicy`](../../src/tools/genpolicy/) application can be used to generat
**Warning** Users should review carefully the automatically-generated Policy, and modify the Policy file if needed to match better their use case, before using this Policy.
**Important — User / Group / Supplemental groups for Policy and genpolicy**
When using features like **nydus guest-pull**, set user/group IDs explicitly in the pod spec, as described in [Limitations](../Limitations.md#guest-pulled-container-images).
See the [`genpolicy` documentation](../../src/tools/genpolicy/README.md) and the [Policy contents examples](#policy-contents) for additional information.
> **Note:** `cri-tools` is only used for debugging and validation purpose, and don't use it to run production workloads.
> **Note:** For how to install and configure `cri-tools` with CRI runtimes like `containerd` or CRI-O, please also refer to other [howtos](./README.md).
> **Note:** For how to install and configure `cri-tools` with CRI runtimes like `containerd` or CRI-O, please also refer to other [how-tos](./README.md).
// we need ensure opening network device happens in netns.
letnetns=self.netns.clone().unwrap_or_default();
@@ -550,6 +554,7 @@ impl TryFrom<BlockConfig> for DiskConfig {
readonly: blkcfg.is_readonly,
num_queues: blkcfg.num_queues,
queue_size: blkcfg.queue_sizeasu16,
image_type: ImageType::Raw,
..Default::default()
};
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
