1. Reduce the complexity of the new allow_mount rules for emptyDir.
2. Reverse the order of the two allow_mount versions, as a hint to the
rego engine that the first version is more often matching the input.
3. Remove `p_mount.source != ""` from mount_source_allows, because:
- Policy rules typically test the values from input, not values read
from Policy.
- mount_source_allows is no longer called for emptyDir mounts after
these changes, so p_mount.source is not empty.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
k0s uses /var/lib/k0s/kubelet instead of /var/lib/kubelet as its
kubelet data directory. Introduce get_kubelet_data_dir() in
tests_common.sh and use it in k8s-trusted-ephemeral-data-storage.bats
instead of hardcoding /var/lib/kubelet.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
It can be useful to set these variables during local testing:
* AZ_REGION: Region for the cluster.
* AZ_NODEPOOL_TAGS: Node pool tags for the cluster.
* GENPOLICY_BINARY: Path to the genpolicy binary.
* GENPOLICY_SETTINGS_DIR: Directory holding the genpolicy settings.
I've also made it so that tests_common.sh modifies the duplicated
genpolicy-settings.json (used for testing) instead of the original git-tracked
one.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
* Introduces a new cluster_config setting encrypted_emptydir defaulting to true.
* Adapts genpolicy for encrypted emptyDirs.
Crucially, the rules.rego change checks that the mount and the storage are
well-formed together:
* i_storage.source matches a known regex.
* i_storage.mount_point == $(spath)/BASE64(i_storage.source)
* i_storage.mount_point == p_storage.mount_point
* i_storage.mount_point == i_mount.source
Note that policy enforcement is necessary to prevent rogue device injection.
E.g. the agent could not blindly encrypt all block devices as some use cases
only need dm-verity.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Handles block-based emptyDirs plugged via virtio-blk and virtio-scsi by
encrypting and formatting them.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
* Introduces the `emptydir_mode` config flag to allow instructing the runtime
to create a block device for emptyDir volumes.
* The block device is created in the original emptyDir folder on the host
so that Kubelet can monitors its disk usage and evict the pod if it exceeds
its sizeLimit. This matches runc and virtio-fs.
* The block device's disk image file is sparse to minimize host disk
footprint.
Fixes: #10560
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Some VMMs support plugging a disk as an image file instead of a block device,
so we adapt the runtime to support that.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Co-authored-by: Aurélien Bombo <abombo@microsoft.com>
The vhost-kern net unit test used a fixed TAP interface name
("test_vhosttap"). When tests run in parallel or a previous run
leaves the interface behind, TAP creation can fail with
EBUSY ("Resource busy"), making CI flaky.
Introduce a unique_tap_name() helper in the tests and use it to
generate a per-test TAP name (based on pid/thread/counter),
avoiding name collisions and stabilizing CI.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The signal_handler test was intermittently failing because it used
kill(pid, sig), which sends signals asynchronously to the process.
This created a race condition where the child thread could exit and
be joined before the signal was delivered or processed.
This fix including:
1. Replaces `kill` with `libc::raise` to ensure signals are delivered
synchronously to the calling thread.
2. Reorders triggers to verify standard signals before installing
seccomp filters.
3. Guarantees that metrics are incremented before the child thread
terminates and is joined by the main thread.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
warning: unnecessary parentheses around type
--> src/dragonball/dbs_legacy_devices/src/serial.rs:245:39
|
245 | let out: Arc<Mutex<Option<Box<(dyn std::io::Write + Send +
'static)>>>> =
| ^
^
|
= note: `#[warn(unused_parens)]` (part of `#[warn(unused)]`) on by
default
help: remove these parentheses
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The vhost-user-net tests could hang in CI because
VhostUserNet::new_server() blocks indefinitely on listener.accept()
when the slave fails to connect in time
(e.g. due to scheduler delays or flaky socket paths). This also caused
panics when connect_slave() returned None and the test unwrapped it.
Fix the tests by:
- using a `/tmp`, absolute, unique unix socket path per test run
retrying slave connect with a deadline
- running new_server() in a separate thread and waiting via
recv_timeout() to ensure the test never blocks indefinitely
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
It aims to fix flaky test hang by implementing thread timeouts.
The `test_vhost_user_net_virtio_device_normal` was hanging in CI
when master/slave threads drifted.
This commit stabilizes the test by:
- Using `tempfile` and unique paths to ensure socket isolation.
- Adding a 5s deadline for slave connections to handle CI jitter.
- Running `new_server` in a separate thread with a `recv_timeout`
to prevent the CI pipeline from deadlocking.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The `test_inner_stream_timeout` test case was prone to failure due to a
race condition between the main thread and the background handler. The
test relied on hardcoded `thread::sleep` durations, which could cause
the second read operation to time out (150ms window) before the main
thread performed its write (after a 300ms sleep) under high system load.
This commit stabilizes the test by:
1. Replacing fixed sleep durations with a `Condvar` and a `stage`
variable to implement a deterministic state machine.
2. Synchronizing the threads so that the main thread only writes data
after the background handler has confirmed it is ready or has
completed its previous phase.
3. Ensuring the read timeout is explicitly managed between different
validation stages to prevent accidental `TimedOut` errors.
This change eliminates the flakiness and ensures the test passes
consistently across different CIenvironments.
Fixes#12618
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit aims to address issues of "Infinite loop in epoll_manager
tests" and improve stablity.
Root causes as below:
1. Using `handle_events(-1)` caused the worker thread to block forever
if an event was missed or if the internal `kick()` signal was not
accounted for correctly.
2. Relying on event counts was unreliable because internal signals could
fluctuate the total count, causing the it to enter an infinite loop.
3. Using `EventSet::OUT` on an EventFd is often continuously ready,
leading to non-deterministic trigger behavior.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
In some CI runs, `mktemp` generates random characters that accidentally
form file extensions like `.cSV` or `.Xml`. This triggers downstream
parsing errors because the YAML content is misidentified as CSV/XML.
The issues look like as below:
```
'/tmp/bats-run-KodZEA/.../pod-guest-pull-in-trusted-storage.yaml.in.cSV':
...
```
This commit fixes the issue by:
1. Moving the `XXXXXX` placeholder before the `.yaml` extension.
2. Ensuring the generated file always ends in `.yaml`.
This prevents format misidentification while maintaining filename
uniqueness and security.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Let's update the nvidia-container-toolkit to 1.18.1 (from 1.17.6).
We're, from now on, relying on the version set in the versions.yaml
file.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
aws-lc has mutliple high severity CVEs:
- GHSA-vw5v-4f2q-w9xf
- GHSA-65p9-r9h6-22vj
- GHSA-hfpc-8r3f-gw53
so try and bump to the latest `aws-lc-rs` crate to pull in the available fixed versions
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Allow users to override the default RuntimeClass pod overhead for
any shim via shims.<name>.runtimeClass.overhead.{memory,cpu}.
When the field is absent the existing hardcoded defaults from the
dict are used, so this is fully
backward compatible.
Signed-off-by: Zachary Spar <zspar@coreweave.com>
Create local block storage (loop device, StorageClass, PV) in the test
only when the cluster has no default StorageClass, matching the approach
used in k8s-volume.bats. Set our StorageClass as default so the PVC
binds to our PV; tear it down after the test.
When a default already exists (e.g. AKS), skip creation and cleanup so
we do not change the cluster's default storage class.
Fixes: #9846
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Virtio-mmio transport is not hardened for confidential computing (unlike
virtio-pci). Reject config that would use virtio-blk-mmio for rootfs/block
when confidential_guest is set, so CoCo guests only use virtio-blk-pci.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Remove # !confidential from mmio.conf so CONFIG_VIRTIO_MMIO and
CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES are included when building the
unified x86_64/s390x kernel with -x
Firecracker requires virtio-mmio for block devices; without it the
guest kernel panics (no /dev/vda).
Fixes: #12581
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Allow genpolicy -j to accept a directory instead of a single file.
When given a directory, genpolicy loads genpolicy-settings.json from it
and applies all genpolicy-settings.d/*.json files (sorted by name) as
RFC 6902 JSON Patches. This gives precise control over settings with
explicit operations (add, remove, replace, move, copy, test), including
array index manipulation and assertions.
Ship composable drop-in examples in drop-in-examples/:
- 10-* files set platform base settings (non-CoCo, AKS, CBL-Mariner)
- 20-* files overlay specific adjustments (OCI version, guest pull)
Users copy the combination they need into genpolicy-settings.d/.
Replace the old adapt_common_policy_settings_* jq-patching functions
in tests_common.sh with install_genpolicy_drop_ins(), which copies the
right combination of 10-* and 20-* drop-ins for the CI scenario.
Tests still generate 99-test-overrides.json on the fly for per-test
request/exec overrides.
Packaging installs 10-* and 20-* drop-ins from drop-in-examples/ into
the tarball; the default genpolicy-settings.d/ is left empty.
Made-with: Cursor
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Add simple -------- text lines separator to the beginnning of the
allow_mount log output, to help log readers easier separate the ~30
lines of text generated while verifying each mount.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The SNP tests have been unstable on nightlies, but even when these
it seems to be manually cleaned up or something as PR tests are consistently
failing, so we should skip this from the required list until it is reliable.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
kata-deploy's SIGTERM cleanup restarts the CRI runtime, which on
k3s/rke2 takes down the API server temporarily. The helm uninstall
may complete with errors, and the next test suite would start with
a dead API. Add a wait loop after uninstall to ensure the API is
available before proceeding.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
kata-deploy restarts the CRI runtime during install, which can cause
the kata-deploy pod to be killed and recreated by the DaemonSet
controller. On k3s and rke2 in particular, the restart can take
several minutes. Increase the default timeout from 600s (10m) to
900s (15m) to accommodate this.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
kata-deploy restarts the CRI runtime (k3s/containerd) during install,
which can kill the verification job pod or cause transient API server
errors. Bump backoffLimit from 0 to 3 so the job can retry after being
killed, and add a retry loop around kubectl rollout status to handle
transient connection failures.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Move the cleanup logic from a preStop lifecycle hook (separate exec)
into the main process's SIGTERM handler. This simplifies the
architecture: the install process now handles its own teardown when
the pod is terminated.
The SIGTERM handler is registered before install begins, and
tokio::select! races install against SIGTERM so cleanup always runs
even if SIGTERM arrives mid-install (e.g. helm uninstall while the
container is restarting after a failed install attempt).
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
