* Introduces a new cluster_config setting encrypted_emptydir defaulting to true.
* Adapts genpolicy for encrypted emptyDirs.
Crucially, the rules.rego change checks that the mount and the storage are
well-formed together:
* i_storage.source matches a known regex.
* i_storage.mount_point == $(spath)/BASE64(i_storage.source)
* i_storage.mount_point == p_storage.mount_point
* i_storage.mount_point == i_mount.source
Note that policy enforcement is necessary to prevent rogue device injection.
E.g. the agent could not blindly encrypt all block devices as some use cases
only need dm-verity.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Handles block-based emptyDirs plugged via virtio-blk and virtio-scsi by
encrypting and formatting them.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Introduce host_memory_mib() with OS-specific implementations
(Linux/Android via nix::sysinfo,
macOS via sysctl) selected at compile time. This improves
portability and allows consistent host memory sizing/validation
across different platforms.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
As the memory related information has been serialized at the sandbox
initalization specially at the moment of parsing configuration toml.
This commit aims to refactor MemoryInfo initialization logics:
(1) Remove memory sizing/host-memory adjustment logic from QEMU cmdline
Memory::new()
(2) Initialize/adjust memory values via kata-types MemoryInfo (single
source of truth)
(3) Replace sysinfo::System::new_with_specifics with
nix::sys::sysinfo::sysinfo() to get host RAM
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
With some older kernels some fs implementations don't handle empty
options strings well. This leads to failures in "setup rootfs" step.
E.g. `cgroup: cgroup2: unknown option ""`.
This is fixed by mapping empty string to `None` before passing to
`nix::mount`.
Signed-off-by: Jacek Tomasiak <jtomasiak@arista.com>
Signed-off-by: Jacek Tomasiak <jacek.tomasiak@gmail.com>
Update time to resolve CVE-2026-25727.
Note: this involved bumping the versions of slog-term and slog-json
and bumping the MSRV to 1.88.0 which time 0.3.47 requires.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The previous comment incorrectly implied that `biased` prevents data
loss and the exit notifier would never be polled before all buffered
data is read. And the detailed info can be seen from the document:
https://docs.rs/tokio/latest/src/tokio/macros/select.rs.html#67
Tokio's `biased` only makes polling order deterministic(top-to-bottom)
when multiple branches are ready in the same poll, and it makes fairness
the caller's responsibility. Output can still be truncated if the exit
notification becomes ready while `read_stream` is pending.
This change updates the comment to reflect the actual semantics and
caveats. No functional behavior change.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Short-lived processes (e.g., `kubectl exec echo`) in legacy-io mode
occasionally lose the last segments of their output.
The root cause is a race condition where the `term_exit_notifier`
triggers before the pipe buffers are fully drained. In the previous
implementation, once the exit notification was received, the agent
immediately returned an EOF, causing the runtime's `run_io_copy` to
terminate and drop any residual data in the pipe.
This patch introduces a "drain after exit" mechanism:
- Upon receiving an exit notification, the agent enters a 500ms window
for polling `read_streaim` to flush remaining data from the buffer.
- A true EOF is only returned if the stream is confirmed empty or the
timeout is reached.
This ensures reliable output delivery for transient exec tasks under
high concurrency.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Legacy IO uses shim polling via read_stdout/read_stderr. The agent
previously mapped pipe EOF (read() == 0) and term_exit_notifier to
errors ("read meet eof"/"eof"), which became ttrpc INTERNAL failures.
This caused runtime IO copy to abort early, leading to lost
stdout/stderr for short-lived exec (e.g."echo") and spurious failures.
Normalize EOF semantics: read_stream now returns Ok(empty) on EOF
instead of Err("read meet eof").
This makes legacy IO behave like a proper stream: data until EOF, no
INTERNAL errors for normal termination.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
We're introducing a root_complex to assign each
and every device to a NUMA node or to the default
root_complex="00" aka pcie.0. This patch introduces
the proper handling of the current qom path being
bus/device == "00/02" with NUMAA we need to extend it
with the root_complex/bus/device == "10/00/02".
We're defaulting to root_complex="00".
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This comment was first introduced in e111093 with secure_join()
but then we forgot to remove it when we switched to the safe-path
lib in c0ceaf6
Signed-off-by: Qingyuan Hou <lenohou@gmail.com>
Downstream builders at Red Hat complain that `Cargo.lock` doesn't match
`Cargo.toml`.
Run `cargo check` to refresh `Cargo.lock`.
`git bisect` shows that 7cfb97d41b is the first commit where
`cargo check` has an effect in `src/agent`.
Signed-off-by: Greg Kurz <groug@kaod.org>
Change the secure_storage_integrity option's default value to true.
With this, integrity protection for encrypted block device contents
will be requested from the confidential data hub by default, see the
agent's cdh_handler_trusted_storage function in rpc.rs.
This behavior can be disabled by explicitly setting the
agent.secure_storage_integrity parameter to 0 or false via kernel
command line parameters.
This will affect the trusted storage implementation for the guest-pull
mechanism, and it will affect future implementations using this code
path, such as implementations for ephemeral secure storage.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Clippy is recommending that format args are inlined for
better clarity, so update our code to remove these warnings
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
In #12151 the version was bumped in cargo.toml, but the update not
done, so run `cargo update -p container-device-interface` to apply it
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
VirtioBlkCcwDeviceHandler and VirtioBlkCcwHandler
are only constructed on s390x, so add #[cfg(target_arch = "s390x")]
to all the code
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
When updating ephemeral storages, MS_REMOUNT is explicitly passed as,
for instance, `/dev/shm` should be remounted after memory is hotplugged.
Till now Kata Containers has been explicitly ignoring such updates,
leading to the containers' `/dev/shm` having the size of "half of the
memory allocated, during the startup time", which goes against the
expected behaviour.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
- Replace generic errors in sandbox operations with typed SandboxError variants (InvalidContainerId, InitProcessNotFound, InvalidExecId).
- This enables the kata shim to handle specific failure cases differently.
Fixes#12120
Signed-off-by: Adeet Phanse <adeet.phanse@mongodb.com>
Allow users to build the Kata Agent using INIT_DATA=no to disable the
detect_initdata_device() code loop and associated debug log output.
Future additional improvements related to Init Data are tracked by #11532.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The current format of genpolicy request logs looks a bit like JSON, but
it does not parse out of the box and needs post-processing with sed, for
example.
This commit changes the log format to jsonlines[1], which is basically
newline-delimited compact JSON values. Compared to standard JSON, this
allows streaming output. The resulting file can be converted and
processed programmatically, for example with `jq -s`.
The fields are also adjusted to match the field names of TestRequest, so
that the logged requests can be used immediately in tests.
[1]: https://jsonlines.org/
Signed-off-by: Markus Rudy <mr@edgeless.systems>
Currently, the initdata module only detects virtio-blk devices
(/dev/vd*) when searching for the initdata block device. However,
when using virtio-scsi, the devices appear as /dev/sd* in the
guest, causing the initdata detection to fail.
This commit extends the device detection logic to support both
device types:
- virtio-blk devices: /dev/vda, /dev/vdb, etc.
- virtio-scsi devices: /dev/sda, /dev/sdb, etc.
This commits aims to address issue of theinitdata device not being
found when using virtio-scsi
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
- version.rs gets generated from version.rs.in
- version.rs.in contains values read from VERSION
- so version.rs (and maybe other Agent files too) must be
re-generated when the VERSION file changes
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
On IBM actionspz Z runners, the following error occurs when running
`modprobe`:
```
modprobe: FATAL: Module bridge not found in directory /lib/modules/6.8.0-85-generic
```
Additionally, there are no files under `/lib/modules`, for example:
```
total 0
drwxr-xr-x 1 root root 0 Aug 5 13:09 .
drwxr-xr-x 1 root root 2.0K Oct 1 22:59 ..
```
This commit skips the `test_load_kernel_module` test if the module is
not found or if running `modprobe` is not permitted.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
On IBM actionspz Z runners, write operations on network interfaces
are not allowed, even for the root user.
This commit skips the `add_update_addresses` test if the operation
fails with EACCES (-13, permission denied).
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
On IBM actionspz Z runners, the ioctl system call is not allowed even
for the root user. There is likely an additional security mechanism
(such as AppArmor or seccomp) in place on Ubuntu runners.
This commit introduces a new helper, `is_permission_error()`,
which skips the test if ioctl operations in `reseed_rng()` are not
permitted.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The IBM actionspz Z runners mount /dev as tmpfs, while other systems
use devtmpfs. This difference causes an assertion failure for
test_already_baremounted.
This commit sets the detected filesystem for bare-mounted points
as the expected value.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The root filesystem for IBM actionspz Z runners is `btrfs` instead of `ext4`.
The error message differs when an unprivileged user tries to perform a bind mount.
This commit adjusts the handling of error messages based on the detected root
filesystem type.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
`mem-agent` now does not ship example binaries and serves as a library
for `agent` to reference, so we move it into `libs` to better manage it.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
We fix the following error:
```
thread 'sandbox::tests::add_and_get_container' panicked at src/sandbox.rs:901:10:
called `Result::unwrap()` on an `Err` value: Create cgroupfs manager
Caused by:
0: fs error caused by: Os { code: 17, kind: AlreadyExists, message: "File exists" }
1: File exists (os error 17)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
by ensuring that the cgroup path is unique for tests run in the same millisecond.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Use CDI exclusively from crates.io and not from a GH repository.
Cargo can easily check if a new version is available and we can
far more easier bump it if needed.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
test_add_one_arp_neighbor modifies the root network namespace, so we
should ensure that it does not interfere with normal network setup.
Adding an IP to a device results in automatic routes, which may affect
routing to non-test endpoints. Thus, we change the addresses used in the
test to come from TEST-NET-1, which is designated for tests and usually
not routable.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
list_routes and test_add_one_arp_neighbor have been flaky in the past
(#10856), but it's been hard to tell what exactly is going wrong.
This commit adds debug information for the most likely problem in
list_routes: devices being added/removed/modified concurrently.
Furthermore, it adds the exit code and stderr of the ip command, in case
it failed to list the ARP neighborhood.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
The previous code only checked the result of with_nix_path(), not statfs(),
thus leading to an uninitialized memory read if statfs() failed.
No functional change otherwise.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Most of the kata-types code is reusable across platforms. However, some
functions in the mount module require safe-path, which is Linux-specific
and can't be used on other platforms, notably darwin.
This commit adds a new feature `safe-path` to kata-types, which enables
the functions that use safe-path. The Linux-only callers kata-ctl and
runtime-rs enable this feature, whereas genpolicy only needs initdata
and does not need the functions from the mount module. Using a feature
instead of a target_os restriction ensures that the developer experience
for genpolicy remains the same.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
The repro below would show this error in the logs (in debug mode only):
fatal runtime error: IO Safety violation: owned file descriptor already closed
The issue was that the `pseudo.slave` file descriptor was being owned by
multiple variables simultaneously. When any of those variables would go out
of scope, they would close the same file descriptor, which is undefined
behavior.
To fix this, we clone: we create a new file descriptOR that refers to the same
file descriptION as the original. When the cloned descriptor is closed, this
affect neither the original descriptor nor the description. Only when the last
descriptor is closed does the kernel cleans up the description.
Note that we purposely consume (not clone) the original descriptor with
`child_stdin` as `pseudo` is NOT dropped automatically.
Repro
-----
Prerequisites:
- Use Rust 1.80+.
- Build the agent in debug mode.
$ cat busybox.yaml
apiVersion: v1
kind: Pod
metadata:
name: busybox
spec:
containers:
- image: busybox:latest
name: busybox
runtimeClassName: kata
$ kubectl apply -f busyboox.yaml
pod/busybox created
$ kubectl exec -it busybox -- sh
error: Internal error occurred: Internal error occurred: error executing
command in container: failed to exec in container: failed to start exec
"e6c602352849647201860c1e1888d99ea3166512f1cc548b9d7f2533129508a9":
cannot enter container 76a499cbf747b9806689e51f6ba35e46d735064a3f176f9be034777e93a242d5,
with err ttrpc: closed
Fixes: #11054
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
In the previous commit we've added some code that broke `cargo fmt --
--check` without even noticing, as the code didn't go through the CI
process (due to it being a security advisory).
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
