Compare commits

..

10 Commits

Author SHA1 Message Date
Fabiano Fidêncio
81e03aee20 tests: Fix kubelet data dir for k0s in trusted ephemeral storage test
k0s uses /var/lib/k0s/kubelet instead of /var/lib/kubelet as its
kubelet data directory. Introduce get_kubelet_data_dir() in
tests_common.sh and use it in k8s-trusted-ephemeral-data-storage.bats
instead of hardcoding /var/lib/kubelet.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-03-05 19:24:41 -06:00
Aurélien Bombo
8b176e8a52 build: Add artifacts to .gitignore
This adds various files that are generated during development.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2026-03-05 19:24:41 -06:00
Aurélien Bombo
94fff98ad1 tests: Improve logging for some tests
Use modern test semantics to ease debugging.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2026-03-05 19:24:41 -06:00
Aurélien Bombo
d15016454a tests: Introduce new env variables to ease development
It can be useful to set these variables during local testing:

 * AZ_REGION: Region for the cluster.
 * AZ_NODEPOOL_TAGS: Node pool tags for the cluster.
 * GENPOLICY_BINARY: Path to the genpolicy binary.
 * GENPOLICY_SETTINGS_DIR: Directory holding the genpolicy settings.

I've also made it so that tests_common.sh modifies the duplicated
genpolicy-settings.json (used for testing) instead of the original git-tracked
one.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2026-03-05 19:24:41 -06:00
Aurélien Bombo
59719f64d2 ci: Stop deploying the CSI driver
The design moved away from CSI driver so stop deploying that.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2026-03-05 19:24:41 -06:00
Aurélien Bombo
02aa27c7b5 tests: Add test for trusted ephemeral data storage
This tests the feature on CoCo machines. We use virtio-blk as the basic
virtio-scsi flow is already tested in k8s-empty-dirs.bats.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2026-03-05 19:24:41 -06:00
Aurélien Bombo
b655494dab genpolicy: Support trusted ephemeral data storage
* Introduces a new cluster_config setting encrypted_emptydir defaulting to true.
 * Adapts genpolicy for encrypted emptyDirs.

Crucially, the rules.rego change checks that the mount and the storage are
well-formed together:

 * i_storage.source matches a known regex.
 * i_storage.mount_point == $(spath)/BASE64(i_storage.source)
 * i_storage.mount_point == p_storage.mount_point
 * i_storage.mount_point == i_mount.source

Note that policy enforcement is necessary to prevent rogue device injection.
E.g. the agent could not blindly encrypt all block devices as some use cases
only need dm-verity.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2026-03-05 19:24:34 -06:00
Aurélien Bombo
6cc2365a9f agent: Support trusted ephemeral data storage
Handles block-based emptyDirs plugged via virtio-blk and virtio-scsi by
encrypting and formatting them.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2026-03-05 09:32:11 +01:00
Aurélien Bombo
e863558285 runtime: Support trusted ephemeral data storage
* Introduces the `emptydir_mode` config flag to allow instructing the runtime
   to create a block device for emptyDir volumes.
 * The block device is created in the original emptyDir folder on the host
   so that Kubelet can monitors its disk usage and evict the pod if it exceeds
   its sizeLimit. This matches runc and virtio-fs.
 * The block device's disk image file is sparse to minimize host disk
   footprint.

Fixes: #10560

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2026-03-05 09:32:11 +01:00
Alex Lyn
b45fd3bbc2 runtime: Support plugging a disk as an image file
Some VMMs support plugging a disk as an image file instead of a block device,
so we adapt the runtime to support that.

Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Co-authored-by: Aurélien Bombo <abombo@microsoft.com>
2026-03-05 09:32:11 +01:00
69 changed files with 4974 additions and 2446 deletions

View File

@@ -168,6 +168,8 @@ jobs:
- rootfs-image-nvidia-gpu-confidential
- rootfs-initrd
- rootfs-initrd-confidential
- rootfs-initrd-nvidia-gpu
- rootfs-initrd-nvidia-gpu-confidential
steps:
- name: Login to Kata Containers quay.io
if: ${{ inputs.push-to-registry == 'yes' }}
@@ -365,6 +367,7 @@ jobs:
matrix:
asset:
- agent-ctl
- csi-kata-directvolume
- genpolicy
- kata-ctl
- kata-manager

View File

@@ -152,6 +152,7 @@ jobs:
- rootfs-image
- rootfs-image-nvidia-gpu
- rootfs-initrd
- rootfs-initrd-nvidia-gpu
steps:
- name: Login to Kata Containers quay.io
if: ${{ inputs.push-to-registry == 'yes' }}

View File

@@ -19,25 +19,23 @@ permissions: {}
jobs:
scan-scheduled:
name: Scan of whole repo
permissions:
actions: read # # Required to upload SARIF file to CodeQL
contents: read # Read commit contents
security-events: write # Require writing security events to upload SARIF file to security tab
if: ${{ github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' }}
uses: "google/osv-scanner-action/.github/workflows/osv-scanner-reusable.yml@8ae4be80636b94886b3c271caad730985ce0611c" # v2.3.3
uses: "google/osv-scanner-action/.github/workflows/osv-scanner-reusable.yml@b00f71e051ddddc6e46a193c31c8c0bf283bf9e6" # v2.1.0
with:
scan-args: |-
-r
./
scan-pr:
name: Scan of just PR code
permissions:
actions: read # Required to upload SARIF file to CodeQL
contents: read # Read commit contents
security-events: write # Require writing security events to upload SARIF file to security tab
if: ${{ github.event_name == 'pull_request' }}
uses: "google/osv-scanner-action/.github/workflows/osv-scanner-reusable-pr.yml@8ae4be80636b94886b3c271caad730985ce0611c" # v2.3.3
uses: "google/osv-scanner-action/.github/workflows/osv-scanner-reusable-pr.yml@b00f71e051ddddc6e46a193c31c8c0bf283bf9e6" # v2.1.0
with:
# Example of specifying custom arguments
scan-args: |-

View File

@@ -110,6 +110,10 @@ jobs:
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh install-kbs-client
- name: Deploy CSI driver
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh deploy-csi-driver
- name: Run tests
timeout-minutes: 100
run: bash tests/integration/kubernetes/gha-run.sh run-tests
@@ -130,6 +134,10 @@ jobs:
[[ "${KATA_HYPERVISOR}" == "qemu-tdx" ]] && echo "ITA_KEY=${GH_ITA_KEY}" >> "${GITHUB_ENV}"
bash tests/integration/kubernetes/gha-run.sh delete-coco-kbs
- name: Delete CSI driver
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh delete-csi-driver
# Generate jobs for testing CoCo on non-TEE environments
run-k8s-tests-coco-nontee:
name: run-k8s-tests-coco-nontee
@@ -227,6 +235,10 @@ jobs:
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh install-kbs-client
- name: Deploy CSI driver
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh deploy-csi-driver
- name: Run tests
timeout-minutes: 80
run: bash tests/integration/kubernetes/gha-run.sh run-tests
@@ -245,6 +257,11 @@ jobs:
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh delete-coco-kbs
- name: Delete CSI driver
if: always()
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh delete-csi-driver
# Extensive matrix: autogenerated policy tests (nydus + experimental-force-guest-pull) on k0s, k3s, rke2, microk8s with qemu-coco-dev / qemu-coco-dev-runtime-rs
run-k8s-tests-coco-nontee-extensive-matrix:
if: ${{ inputs.extensive-matrix-autogenerated-policy == 'yes' }}
@@ -348,6 +365,10 @@ jobs:
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh install-kbs-client
- name: Deploy CSI driver
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh deploy-csi-driver
- name: Run tests
timeout-minutes: 80
run: bash tests/integration/kubernetes/gha-run.sh run-tests
@@ -366,6 +387,11 @@ jobs:
timeout-minutes: 10
run: bash tests/integration/kubernetes/gha-run.sh delete-coco-kbs
- name: Delete CSI driver
if: always()
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh delete-csi-driver
# Generate jobs for testing CoCo on non-TEE environments with erofs-snapshotter
run-k8s-tests-coco-nontee-with-erofs-snapshotter:
name: run-k8s-tests-coco-nontee-with-erofs-snapshotter
@@ -452,6 +478,10 @@ jobs:
timeout-minutes: 20
run: bash tests/integration/kubernetes/gha-run.sh deploy-kata
- name: Deploy CSI driver
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh deploy-csi-driver
- name: Run tests
timeout-minutes: 80
run: bash tests/integration/kubernetes/gha-run.sh run-tests
@@ -464,3 +494,8 @@ jobs:
if: always()
timeout-minutes: 15
run: bash tests/integration/kubernetes/gha-run.sh cleanup
- name: Delete CSI driver
if: always()
timeout-minutes: 5
run: bash tests/integration/kubernetes/gha-run.sh delete-csi-driver

4
.gitignore vendored
View File

@@ -23,3 +23,7 @@ site/
opt/
tools/packaging/kernel/configs/**/.config
root_hash.txt
# Generated by genpolicy and stored in the working directory, so only
# the basename is ignored.
layers-cache.json

1508
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -22,9 +22,6 @@ members = [
"src/dragonball/dbs_utils",
"src/dragonball/dbs_virtio_devices",
# genpolicy
"src/tools/genpolicy",
# runtime-rs
"src/runtime-rs",
"src/runtime-rs/crates/agent",
@@ -110,9 +107,6 @@ safe-path = { path = "src/libs/safe-path" }
shim-interface = { path = "src/libs/shim-interface" }
test-utils = { path = "src/libs/test-utils" }
# Local dependencies from `src/agent`
kata-agent-policy = { path = "src/agent/policy" }
# Outside dependencies
actix-rt = "2.7.0"
anyhow = "1.0"

View File

@@ -1 +1 @@
3.28.0
3.27.0

View File

@@ -1,64 +1,57 @@
# How to do a Kata Containers Release
This document lists the tasks required to create a Kata Release.
## Requirements
- GitHub permissions to run workflows.
## Release Model
## Versioning
Kata Containers follows a rolling release model with monthly snapshots.
New features, bug fixes, and improvements are continuously integrated into
`main`. Each month, a snapshot is tagged as a new `MINOR` release.
The Kata Containers project uses [semantic versioning](http://semver.org/) for all releases.
Semantic versions are comprised of three fields in the form:
### Versioning
```
MAJOR.MINOR.PATCH
```
Releases use the `MAJOR.MINOR.PATCH` scheme. Monthly snapshots increment
`MINOR`; `PATCH` is typically `0`. Major releases are rare (years apart) and
signal significant architectural changes that may require updates to container
managers (Containerd, CRI-O) or other infrastructure. Breaking changes in
`MINOR` releases are avoided where possible, but may occasionally occur as
features are deprecated or removed.
When `MINOR` increases, the new release adds **new features** but *without changing the existing behavior*.
### No Stable Branches
When `MAJOR` increases, the new release adds **new features, bug fixes, or
both** and which **changes the behavior from the previous release** (incompatible with previous releases).
The Kata Containers project does not maintain stable branches (see
[#9064](https://github.com/kata-containers/kata-containers/issues/9064)).
Bug fixes land on `main` and ship in the next monthly snapshot rather than
being backported. Downstream projects that need extended support or compliance
certifications should select a monthly snapshot as their stable base and manage
their own validation and patch backporting from there.
A major release will also likely require a change of the container manager version used,
-for example Containerd or CRI-O. Please refer to the release notes for further details.
**Important** : the Kata Containers project doesn't have stable branches (see
[this issue](https://github.com/kata-containers/kata-containers/issues/9064) for details).
Bug fixes are released as part of `MINOR` or `MAJOR` releases only. `PATCH` is always `0`.
## Release Process
### Bump the `VERSION` and `Chart.yaml` file
When the `kata-containers/kata-containers` repository is ready for a new
release, first create a PR to set the release in the [`VERSION`](./../VERSION)
file and update the `version` and `appVersion` in the
[`Chart.yaml`](./../tools/packaging/kata-deploy/helm-chart/kata-deploy/Chart.yaml)
file and have it merged.
When the `kata-containers/kata-containers` repository is ready for a new release,
first create a PR to set the release in the [`VERSION`](./../VERSION) file and update the
`version` and `appVersion` in the
[`Chart.yaml`](./../tools/packaging/kata-deploy/helm-chart/kata-deploy/Chart.yaml) file and
have it merged.
### Lock the `main` branch
In order to prevent any PRs getting merged during the release process, and
slowing the release process down, by impacting the payload caches, we have
recently trialed setting the `main` branch to read only whilst the release
action runs.
In order to prevent any PRs getting merged during the release process, and slowing the release
process down, by impacting the payload caches, we have recently trailed setting the `main`
branch to read only whilst the release action runs.
> [!NOTE]
> Admin permission is needed to complete this task.
### Wait for the `VERSION` bump PR payload publish to complete
To reduce the chance of need to re-run the release workflow, check the [CI |
Publish Kata Containers
payload](https://github.com/kata-containers/kata-containers/actions/workflows/payload-after-push.yaml)
To reduce the chance of need to re-run the release workflow, check the
[CI | Publish Kata Containers payload](https://github.com/kata-containers/kata-containers/actions/workflows/payload-after-push.yaml)
once the `VERSION` PR bump has merged to check that the assets build correctly
and are cached, so that the release process can just download these artifacts
rather than needing to build them all, which takes time and can reveal errors in
infra.
rather than needing to build them all, which takes time and can reveal errors in infra.
### Check GitHub Actions
@@ -70,10 +63,11 @@ release artifacts.
> [!NOTE]
> Write permissions to trigger the action.
The action is manually triggered and is responsible for generating a new release
(including a new tag), pushing those to the `kata-containers/kata-containers`
repository. The new release is initially created as a draft. It is promoted to
an official release when the whole workflow has completed successfully.
The action is manually triggered and is responsible for generating a new
release (including a new tag), pushing those to the
`kata-containers/kata-containers` repository. The new release is initially
created as a draft. It is promoted to an official release when the whole
workflow has completed successfully.
Check the [actions status
page](https://github.com/kata-containers/kata-containers/actions) to verify all
@@ -81,13 +75,12 @@ steps in the actions workflow have completed successfully. On success, a static
tarball containing Kata release artifacts will be uploaded to the [Release
page](https://github.com/kata-containers/kata-containers/releases).
If the workflow fails because of some external environmental causes, e.g.
network timeout, simply re-run the failed jobs until they eventually succeed.
If the workflow fails because of some external environmental causes, e.g. network
timeout, simply re-run the failed jobs until they eventually succeed.
If for some reason you need to cancel the workflow or re-run it entirely, go
first to the [Release
page](https://github.com/kata-containers/kata-containers/releases) and delete
the draft release from the previous run.
If for some reason you need to cancel the workflow or re-run it entirely, go first
to the [Release page](https://github.com/kata-containers/kata-containers/releases) and
delete the draft release from the previous run.
### Unlock the `main` branch
@@ -97,8 +90,9 @@ an admin to do it.
### Improve the release notes
Release notes are auto-generated by the GitHub CLI tool used as part of our
release workflow. However, some manual tweaking may still be necessary in order
to highlight the most important features and bug fixes in a specific release.
release workflow. However, some manual tweaking may still be necessary in
order to highlight the most important features and bug fixes in a specific
release.
With this in mind, please, poke @channel on #kata-dev and people who worked on
the release will be able to contribute to that.

View File

@@ -1,8 +0,0 @@
[[IgnoredVulns]]
# yaml-rust is unmaintained.
# We tried the most promising alternative in https://github.com/kata-containers/kata-containers/pull/12509,
# but its literal quoting is not conformant.
id = "RUSTSEC-2024-0320"
ignoreUntil = 2026-10-01 # TODO(burgerdev): revisit yml library ecosystem
reason = "No alternative currently supports 'yes' strings correctly; genpolicy processes only trusted input."

View File

@@ -2317,13 +2317,8 @@ async fn cdh_handler_trusted_storage(oci: &mut Spec) -> Result<()> {
for specdev in devices.iter() {
if specdev.path().as_path().to_str() == Some(TRUSTED_IMAGE_STORAGE_DEVICE) {
let dev_major_minor = format!("{}:{}", specdev.major(), specdev.minor());
cdh_secure_mount(
"block-device",
&dev_major_minor,
"luks2",
KATA_IMAGE_WORK_DIR,
)
.await?;
cdh_secure_mount("BlockDevice", &dev_major_minor, "LUKS", KATA_IMAGE_WORK_DIR)
.await?;
break;
}
}
@@ -2354,21 +2349,10 @@ pub(crate) async fn cdh_secure_mount(
let options = std::collections::HashMap::from([
("deviceId".to_string(), device_id.to_string()),
("sourceType".to_string(), "empty".to_string()),
("targetType".to_string(), "fileSystem".to_string()),
("filesystemType".to_string(), "ext4".to_string()),
("mkfsOpts".to_string(), "-E lazy_journal_init".to_string()),
("encryptionType".to_string(), encrypt_type.to_string()),
("encryptType".to_string(), encrypt_type.to_string()),
("dataIntegrity".to_string(), integrity),
]);
std::fs::create_dir_all(mount_point).inspect_err(|e| {
error!(
sl(),
"Failed to create mount point directory {}: {:?}", mount_point, e
);
})?;
confidential_data_hub::secure_mount(device_type, &options, vec![], mount_point).await?;
Ok(())

View File

@@ -59,8 +59,7 @@ async fn handle_block_storage(
.contains(&"encryption_key=ephemeral".to_string());
if has_ephemeral_encryption {
crate::rpc::cdh_secure_mount("block-device", dev_num, "luks2", &storage.mount_point)
.await?;
crate::rpc::cdh_secure_mount("BlockDevice", dev_num, "LUKS", &storage.mount_point).await?;
set_ownership(logger, storage)?;
new_device(storage.mount_point.clone())
} else {

View File

@@ -242,7 +242,7 @@ mod tests {
let metrics = Arc::new(SerialDeviceMetrics::default());
let out: Arc<Mutex<Option<Box<dyn std::io::Write + Send + 'static>>>> =
let out: Arc<Mutex<Option<Box<(dyn std::io::Write + Send + 'static)>>>> =
Arc::new(Mutex::new(Some(Box::new(std::io::sink()))));
let mut serial = SerialDevice {
serial: Serial::with_events(

View File

@@ -1174,6 +1174,7 @@ pub(crate) mod tests {
use dbs_virtio_devices::Result as VirtIoResult;
use dbs_virtio_devices::{
ActivateResult, VirtioDeviceConfig, VirtioDeviceInfo, VirtioSharedMemory,
DEVICE_ACKNOWLEDGE, DEVICE_DRIVER, DEVICE_DRIVER_OK, DEVICE_FEATURES_OK, DEVICE_INIT,
};
use dbs_address_space::{AddressSpaceLayout, AddressSpaceRegion, AddressSpaceRegionType};

View File

@@ -99,61 +99,76 @@ impl Default for EpollManager {
#[cfg(test)]
mod tests {
use super::*;
use std::os::fd::AsRawFd;
use std::sync::mpsc::channel;
use std::time::Duration;
use std::os::unix::io::AsRawFd;
use vmm_sys_util::{epoll::EventSet, eventfd::EventFd};
struct DummySubscriber {
pub event: Arc<EventFd>,
pub notify: std::sync::mpsc::Sender<()>,
pub event: EventFd,
}
impl DummySubscriber {
fn new(event: Arc<EventFd>, notify: std::sync::mpsc::Sender<()>) -> Self {
Self { event, notify }
fn new() -> Self {
Self {
event: EventFd::new(0).unwrap(),
}
}
}
impl MutEventSubscriber for DummySubscriber {
fn init(&mut self, ops: &mut EventOps) {
ops.add(Events::new(self.event.as_ref(), EventSet::IN))
.unwrap();
}
fn process(&mut self, events: Events, _ops: &mut EventOps) {
if events.fd() == self.event.as_raw_fd() && events.event_set().contains(EventSet::IN) {
let _ = self.event.read();
let _ = self.notify.send(());
let source = events.fd();
let event_set = events.event_set();
assert_ne!(source, self.event.as_raw_fd());
match event_set {
EventSet::IN => {
unreachable!()
}
EventSet::OUT => {
self.event.read().unwrap();
}
_ => {
unreachable!()
}
}
}
fn init(&mut self, _ops: &mut EventOps) {}
}
#[test]
fn test_epoll_manager() {
let epoll_manager = EpollManager::default();
let (stop_tx, stop_rx) = channel::<()>();
let worker_mgr = epoll_manager.clone();
let worker = std::thread::spawn(move || {
while stop_rx.try_recv().is_err() {
let _ = worker_mgr.handle_events(50);
let mut epoll_manager = EpollManager::default();
let epoll_manager_clone = epoll_manager.clone();
let thread = std::thread::spawn(move || loop {
let count = epoll_manager_clone.handle_events(-1).unwrap();
if count == 0 {
continue;
}
assert_eq!(count, 1);
break;
});
let (notify_tx, notify_rx) = channel::<()>();
let event = Arc::new(EventFd::new(0).unwrap());
let handler = DummySubscriber::new(event.clone(), notify_tx);
let handler = DummySubscriber::new();
let event = handler.event.try_clone().unwrap();
let id = epoll_manager.add_subscriber(Box::new(handler));
thread.join().unwrap();
epoll_manager
.add_event(id, Events::new(&event, EventSet::OUT))
.unwrap();
event.write(1).unwrap();
notify_rx
.recv_timeout(Duration::from_secs(2))
.expect("timeout waiting for subscriber to be processed");
let epoll_manager_clone = epoll_manager.clone();
let thread = std::thread::spawn(move || loop {
let count = epoll_manager_clone.handle_events(-1).unwrap();
if count == 0 {
continue;
}
assert_eq!(count, 2);
break;
});
epoll_manager.clone().remove_subscriber(id).unwrap();
let _ = stop_tx.send(());
worker.join().unwrap();
thread.join().unwrap();
epoll_manager.remove_subscriber(id).unwrap();
}
}

View File

@@ -690,15 +690,6 @@ mod tests {
use crate::tests::{create_address_space, create_vm_and_irq_manager};
use crate::{create_queue_notifier, VirtioQueueConfig};
fn unique_tap_name(prefix: &str) -> String {
use std::sync::atomic::{AtomicUsize, Ordering};
static CNT: AtomicUsize = AtomicUsize::new(0);
let n = CNT.fetch_add(1, Ordering::Relaxed);
// "vtap" + pid(<=5) + n(<=3) => max len <= 15
format!("{}{:x}{:x}", prefix, std::process::id() & 0xfff, n & 0xfff)
}
fn create_vhost_kern_net_epoll_handler(
id: String,
) -> NetEpollHandler<Arc<GuestMemoryMmap>, QueueSync, GuestRegionMmap> {
@@ -732,16 +723,13 @@ mod tests {
let guest_mac = MacAddr::parse_str(guest_mac_str).unwrap();
let queue_sizes = Arc::new(vec![128]);
let epoll_mgr = EpollManager::default();
let tap_name = unique_tap_name("vtap");
let dev_result: VirtioResult<Net<Arc<GuestMemoryMmap>, QueueSync, GuestRegionMmap>> =
Net::new(tap_name.clone(), Some(&guest_mac), queue_sizes, epoll_mgr);
let mut dev: Net<Arc<GuestMemoryMmap>, QueueSync, GuestRegionMmap> = match dev_result {
Ok(d) => d,
Err(e) => {
eprintln!("skip test: failed to create tap {}: {:?}", tap_name, e);
return;
}
};
let mut dev: Net<Arc<GuestMemoryMmap>, QueueSync, GuestRegionMmap> = Net::new(
String::from("test_vhosttap"),
Some(&guest_mac),
queue_sizes,
epoll_mgr,
)
.unwrap();
assert_eq!(dev.device_type(), TYPE_NET);
@@ -777,16 +765,14 @@ mod tests {
{
let queue_sizes = Arc::new(vec![128]);
let epoll_mgr = EpollManager::default();
let tap_name = unique_tap_name("vtap");
let dev_result: VirtioResult<Net<Arc<GuestMemoryMmap>, QueueSync, GuestRegionMmap>> =
Net::new(tap_name.clone(), Some(&guest_mac), queue_sizes, epoll_mgr);
let mut dev: Net<Arc<GuestMemoryMmap>, QueueSync, GuestRegionMmap> = match dev_result {
Ok(d) => d,
Err(e) => {
eprintln!("skip test: failed to create tap {}: {:?}", tap_name, e);
return;
}
};
let mut dev: Net<Arc<GuestMemoryMmap>, QueueSync, GuestRegionMmap> = Net::new(
String::from("test_vhosttap"),
Some(&guest_mac),
queue_sizes,
epoll_mgr,
)
.unwrap();
let queues = vec![
VirtioQueueConfig::create(128, 0).unwrap(),
VirtioQueueConfig::create(128, 0).unwrap(),
@@ -823,17 +809,13 @@ mod tests {
let queue_eventfd2 = Arc::new(EventFd::new(0).unwrap());
let queue_sizes = Arc::new(vec![128, 128]);
let epoll_mgr = EpollManager::default();
let tap_name = unique_tap_name("vtap");
let dev_result: VirtioResult<Net<Arc<GuestMemoryMmap>, Queue, GuestRegionMmap>> =
Net::new(tap_name.clone(), Some(&guest_mac), queue_sizes, epoll_mgr);
let mut dev: Net<Arc<GuestMemoryMmap>, Queue, GuestRegionMmap> = match dev_result {
Ok(d) => d,
Err(e) => {
eprintln!("skip test: failed to create tap {}: {:?}", tap_name, e);
return;
}
};
let mut dev: Net<Arc<GuestMemoryMmap>, Queue, GuestRegionMmap> = Net::new(
String::from("test_vhosttap"),
Some(&guest_mac),
queue_sizes,
epoll_mgr,
)
.unwrap();
let queues = vec![
VirtioQueueConfig::new(queue, queue_eventfd, notifier.clone(), 1),

View File

@@ -590,7 +590,6 @@ where
mod tests {
use std::sync::Arc;
use std::thread;
use std::time::{Duration, Instant};
use dbs_device::resources::DeviceResources;
use dbs_interrupt::{InterruptManager, InterruptSourceType, MsiNotifier, NoopNotifier};
@@ -610,16 +609,19 @@ mod tests {
};
use crate::{VirtioDevice, VirtioDeviceConfig, VirtioQueueConfig, TYPE_NET};
fn connect_slave(path: &str, timeout: Duration) -> Option<Endpoint<MasterReq>> {
let deadline = Instant::now() + timeout;
fn connect_slave(path: &str) -> Option<Endpoint<MasterReq>> {
let mut retry_count = 5;
loop {
match Endpoint::<MasterReq>::connect(path) {
Ok(ep) => return Some(ep),
Ok(endpoint) => return Some(endpoint),
Err(_) => {
if Instant::now() >= deadline {
if retry_count > 0 {
std::thread::sleep(std::time::Duration::from_millis(100));
retry_count -= 1;
continue;
} else {
return None;
}
thread::sleep(Duration::from_millis(20));
}
}
}
@@ -637,88 +639,62 @@ mod tests {
#[test]
fn test_vhost_user_net_virtio_device_normal() {
let dir_path = std::path::Path::new("/tmp");
let socket_path = dir_path.join(format!(
"vhost-user-net-{}-{:?}.sock",
std::process::id(),
thread::current().id()
));
let socket_str = socket_path.to_str().unwrap().to_string();
let _ = std::fs::remove_file(&socket_path);
let queue_sizes = Arc::new(vec![128u16]);
let device_socket = concat!("vhost.", line!());
let queue_sizes = Arc::new(vec![128]);
let epoll_mgr = EpollManager::default();
let socket_for_slave = socket_str.clone();
let slave_th = thread::spawn(move || {
let mut slave = connect_slave(&socket_for_slave, Duration::from_secs(5))
.unwrap_or_else(|| panic!("slave connect timeout: {}", socket_for_slave));
let handler = thread::spawn(move || {
let mut slave = connect_slave(device_socket).unwrap();
create_vhost_user_net_slave(&mut slave);
});
let (tx, rx) = std::sync::mpsc::channel();
let socket_for_master = socket_str.clone();
let queue_sizes_for_master = queue_sizes.clone();
let epoll_mgr_for_master = epoll_mgr.clone();
thread::spawn(move || {
let res = VhostUserNet::<Arc<GuestMemoryMmap>>::new_server(
&socket_for_master,
None,
queue_sizes_for_master,
epoll_mgr_for_master,
);
let _ = tx.send(res);
});
let dev_res = rx
.recv_timeout(Duration::from_secs(5))
.unwrap_or_else(|_| panic!("new_server() stuck/timeout: {}", socket_str));
let dev: VhostUserNet<Arc<GuestMemoryMmap>> = dev_res.unwrap_or_else(|e| {
panic!(
"new_server() returned error: {:?}, socket={}",
e, socket_str
)
});
let mut dev: VhostUserNet<Arc<GuestMemoryMmap>> =
VhostUserNet::new_server(device_socket, None, queue_sizes, epoll_mgr).unwrap();
assert_eq!(
VirtioDevice::<Arc<GuestMemoryMmap<()>>, QueueSync, GuestRegionMmap>::device_type(&dev),
TYPE_NET
);
let queue_size = [128u16];
let queue_size = [128];
assert_eq!(
VirtioDevice::<Arc<GuestMemoryMmap<()>>, QueueSync, GuestRegionMmap>::queue_max_sizes(
&dev
),
&queue_size[..]
);
slave_th.join().unwrap();
let _ = std::fs::remove_file(&socket_path);
drop(dev);
assert_eq!(
VirtioDevice::<Arc<GuestMemoryMmap<()>>, QueueSync, GuestRegionMmap>::get_avail_features(&dev, 0),
dev.device().device_info.get_avail_features(0)
);
assert_eq!(
VirtioDevice::<Arc<GuestMemoryMmap<()>>, QueueSync, GuestRegionMmap>::get_avail_features(&dev, 1),
dev.device().device_info.get_avail_features(1)
);
assert_eq!(
VirtioDevice::<Arc<GuestMemoryMmap<()>>, QueueSync, GuestRegionMmap>::get_avail_features(&dev, 2),
dev.device().device_info.get_avail_features(2)
);
VirtioDevice::<Arc<GuestMemoryMmap<()>>, QueueSync, GuestRegionMmap>::set_acked_features(
&mut dev, 2, 0,
);
assert_eq!(VirtioDevice::<Arc<GuestMemoryMmap<()>>, QueueSync, GuestRegionMmap>::get_avail_features(&dev, 2), 0);
let config: [u8; 8] = [0; 8];
let _ = VirtioDevice::<Arc<GuestMemoryMmap<()>>, QueueSync, GuestRegionMmap>::write_config(
&mut dev, 0, &config,
);
let mut data: [u8; 8] = [1; 8];
let _ = VirtioDevice::<Arc<GuestMemoryMmap<()>>, QueueSync, GuestRegionMmap>::read_config(
&mut dev, 0, &mut data,
);
assert_eq!(config, data);
handler.join().unwrap();
}
#[test]
fn test_vhost_user_net_virtio_device_activate() {
skip_if_kvm_unaccessable!();
let dir_path = std::path::Path::new("/tmp");
let socket_path = dir_path.join(format!(
"vhost-user-net-{}-{:?}.sock",
std::process::id(),
thread::current().id()
));
let socket_str = socket_path.to_str().unwrap().to_string();
let _ = std::fs::remove_file(&socket_path);
let queue_sizes = Arc::new(vec![128u16]);
let device_socket = concat!("vhost.", line!());
let queue_sizes = Arc::new(vec![128]);
let epoll_mgr = EpollManager::default();
let socket_for_slave = socket_str.clone();
let slave_th = thread::spawn(move || {
let mut slave = connect_slave(&socket_for_slave, Duration::from_secs(10))
.unwrap_or_else(|| panic!("slave connect timeout: {}", socket_for_slave));
let handler = thread::spawn(move || {
let mut slave = connect_slave(device_socket).unwrap();
create_vhost_user_net_slave(&mut slave);
let mut pfeatures = VhostUserProtocolFeatures::all();
// A workaround for no support for `INFLIGHT_SHMFD`. File an issue to track
@@ -726,30 +702,8 @@ mod tests {
pfeatures -= VhostUserProtocolFeatures::INFLIGHT_SHMFD;
negotiate_slave(&mut slave, pfeatures, true, 1);
});
let (tx, rx) = std::sync::mpsc::channel();
let socket_for_master = socket_str.clone();
let queue_sizes_for_master = queue_sizes.clone();
let epoll_mgr_for_master = epoll_mgr.clone();
thread::spawn(move || {
let res = VhostUserNet::<Arc<GuestMemoryMmap>>::new_server(
&socket_for_master,
None,
queue_sizes_for_master,
epoll_mgr_for_master,
);
let _ = tx.send(res);
});
let mut dev: VhostUserNet<Arc<GuestMemoryMmap>> = rx
.recv_timeout(Duration::from_secs(10))
.unwrap_or_else(|_| panic!("new_server() stuck/timeout: {}", socket_str))
.unwrap_or_else(|e| {
panic!(
"new_server() returned error: {:?}, socket={}",
e, socket_str
)
});
let mut dev: VhostUserNet<Arc<GuestMemoryMmap>> =
VhostUserNet::new_server(device_socket, None, queue_sizes, epoll_mgr).unwrap();
// invalid queue size
{
let kvm = Kvm::new().unwrap();
@@ -806,9 +760,6 @@ mod tests {
);
dev.activate(config).unwrap();
}
slave_th.join().unwrap();
let _ = std::fs::remove_file(&socket_path);
drop(dev);
handler.join().unwrap();
}
}

View File

@@ -867,96 +867,56 @@ mod tests {
.set_read_timeout(Some(Duration::from_millis(150)))
.is_ok());
// stage:
// 0 = handler started
// 1 = first read timed out (main can do first write now)
// 2 = timeout cancelled, handler is about to do 3rd blocking read
let stage = Arc::new((Mutex::new(0u32), Condvar::new()));
let stage2 = Arc::clone(&stage);
let handler = thread::spawn(move || {
// notify started
{
let (lock, cvar) = &*stage2;
let mut s = lock.lock().unwrap();
*s = 0;
let cond_pair = Arc::new((Mutex::new(false), Condvar::new()));
let cond_pair_2 = Arc::clone(&cond_pair);
let handler = thread::Builder::new()
.spawn(move || {
// notify handler thread start
let (lock, cvar) = &*cond_pair_2;
let mut started = lock.lock().unwrap();
*started = true;
cvar.notify_one();
}
drop(started);
let mut reader_buf = [0u8; 5];
let start_time1 = Instant::now();
let mut reader_buf = [0; 5];
// first read would timed out
assert_eq!(
outer_stream.read_exact(&mut reader_buf).unwrap_err().kind(),
ErrorKind::TimedOut
);
let end_time1 = Instant::now().duration_since(start_time1).as_millis();
assert!((150..250).contains(&end_time1));
// 1) first read should timed out
let start_time1 = Instant::now();
assert_eq!(
outer_stream.read_exact(&mut reader_buf).unwrap_err().kind(),
ErrorKind::TimedOut
);
let end_time1 = start_time1.elapsed().as_millis();
assert!((150..300).contains(&end_time1));
// second read would ok
assert!(outer_stream.read_exact(&mut reader_buf).is_ok());
assert_eq!(reader_buf, [1, 2, 3, 4, 5]);
outer_stream
.set_read_timeout(Some(Duration::from_secs(10)))
.unwrap();
// cancel the read timeout
let start_time2 = Instant::now();
outer_stream.set_read_timeout(None).unwrap();
assert!(outer_stream.read_exact(&mut reader_buf).is_ok());
let end_time2 = Instant::now().duration_since(start_time2).as_millis();
assert!(end_time2 >= 500);
})
.unwrap();
// notify main: timeout observed, now do first write
{
let (lock, cvar) = &*stage2;
let mut s = lock.lock().unwrap();
*s = 1;
cvar.notify_one();
}
// 2) second read should ok (main will write after stage==1)
outer_stream.read_exact(&mut reader_buf).unwrap();
assert_eq!(reader_buf, [1, 2, 3, 4, 5]);
// 3) cancel timeout, then do a blocking read; notify main before blocking
outer_stream.set_read_timeout(None).unwrap();
{
let (lock, cvar) = &*stage2;
let mut s = lock.lock().unwrap();
*s = 2;
cvar.notify_one();
}
let start_time2 = Instant::now();
outer_stream.read_exact(&mut reader_buf).unwrap();
let end_time2 = start_time2.elapsed().as_millis();
assert!(end_time2 >= 500);
assert_eq!(reader_buf, [1, 2, 3, 4, 5]);
});
// wait handler started (stage==0)
{
let (lock, cvar) = &*stage;
let mut s = lock.lock().unwrap();
while *s != 0 {
s = cvar.wait(s).unwrap();
}
// wait handler thread started
let (lock, cvar) = &*cond_pair;
let mut started = lock.lock().unwrap();
while !*started {
started = cvar.wait(started).unwrap();
}
// wait first timeout done (stage==1), then do first write
{
let (lock, cvar) = &*stage;
let mut s = lock.lock().unwrap();
while *s < 1 {
s = cvar.wait(s).unwrap();
}
}
inner_stream.write_all(&[1, 2, 3, 4, 5]).unwrap();
// wait handler cancelled timeout and is about to block-read (stage==2)
{
let (lock, cvar) = &*stage;
let mut s = lock.lock().unwrap();
while *s < 2 {
s = cvar.wait(s).unwrap();
}
}
// sleep 300ms, test timeout
thread::sleep(Duration::from_millis(300));
let writer_buf = [1, 2, 3, 4, 5];
inner_stream.write_all(&writer_buf).unwrap();
// sleep 500ms again, test cancel timeout
thread::sleep(Duration::from_millis(500));
inner_stream.write_all(&[1, 2, 3, 4, 5]).unwrap();
let writer_buf = [1, 2, 3, 4, 5];
inner_stream.write_all(&writer_buf).unwrap();
handler.join().unwrap();
}

View File

@@ -120,7 +120,7 @@ mod tests {
use libc::{cpu_set_t, syscall};
use std::convert::TryInto;
use std::{mem, thread};
use std::{mem, process, thread};
use seccompiler::{apply_filter, BpfProgram, SeccompAction, SeccompFilter};
@@ -157,16 +157,6 @@ mod tests {
let child = thread::spawn(move || {
assert!(register_signal_handlers().is_ok());
// Trigger SIGBUS/SIGSEGV *before* installing the seccomp filter.
// Call SIGBUS signal handler.
assert_eq!(METRICS.read().unwrap().signals.sigbus.count(), 0);
unsafe { libc::raise(SIGBUS) };
// Call SIGSEGV signal handler.
assert_eq!(METRICS.read().unwrap().signals.sigsegv.count(), 0);
unsafe { libc::raise(SIGSEGV) };
// Install a seccomp filter that traps a known syscall so that we can verify SIGSYS handling.
let filter = SeccompFilter::new(
vec![(libc::SYS_mkdirat, vec![])].into_iter().collect(),
SeccompAction::Allow,
@@ -178,8 +168,20 @@ mod tests {
assert!(apply_filter(&TryInto::<BpfProgram>::try_into(filter).unwrap()).is_ok());
assert_eq!(METRICS.read().unwrap().seccomp.num_faults.count(), 0);
// Invoke the blacklisted syscall to trigger SIGSYS and exercise the SIGSYS handler.
// Call the blacklisted `SYS_mkdirat`.
unsafe { syscall(libc::SYS_mkdirat, "/foo/bar\0") };
// Call SIGBUS signal handler.
assert_eq!(METRICS.read().unwrap().signals.sigbus.count(), 0);
unsafe {
syscall(libc::SYS_kill, process::id(), SIGBUS);
}
// Call SIGSEGV signal handler.
assert_eq!(METRICS.read().unwrap().signals.sigsegv.count(), 0);
unsafe {
syscall(libc::SYS_kill, process::id(), SIGSEGV);
}
});
assert!(child.join().is_ok());

View File

@@ -13,7 +13,6 @@ use super::{default, register_hypervisor_plugin};
use crate::config::default::MAX_CH_VCPUS;
use crate::config::default::MIN_CH_MEMORY_SIZE_MB;
use crate::config::hypervisor::VIRTIO_BLK_MMIO;
use crate::config::{ConfigPlugin, TomlConfig};
use crate::{resolve_path, validate_path};
@@ -105,16 +104,6 @@ impl ConfigPlugin for CloudHypervisorConfig {
));
}
// CoCo guest hardening: virtio-mmio is not hardened for confidential computing.
if ch.security_info.confidential_guest
&& ch.boot_info.vm_rootfs_driver == VIRTIO_BLK_MMIO
{
return Err(std::io::Error::other(
"Confidential guests must not use virtio-blk-mmio (use virtio-blk-pci); \
virtio-mmio is not hardened for CoCo",
));
}
if ch.boot_info.kernel.is_empty() {
return Err(std::io::Error::other("Guest kernel image for CH is empty"));
}

View File

@@ -124,17 +124,6 @@ impl ConfigPlugin for QemuConfig {
));
}
// CoCo guest hardening: virtio-mmio transport is not hardened for confidential
// computing; only virtio-pci is. Ensure we never use virtio-blk-mmio for rootfs.
if qemu.security_info.confidential_guest
&& qemu.boot_info.vm_rootfs_driver == VIRTIO_BLK_MMIO
{
return Err(std::io::Error::other(
"Confidential guests must not use virtio-blk-mmio (use virtio-blk-pci); \
virtio-mmio is not hardened for CoCo",
));
}
if qemu.boot_info.kernel.is_empty() {
return Err(std::io::Error::other(
"Guest kernel image for qemu is empty",

View File

@@ -24,7 +24,9 @@ message SecureMountRequest {
string mount_point = 4;
}
message SecureMountResponse {}
message SecureMountResponse {
string mount_path = 1;
}
message ImagePullRequest {
// - `image_url`: The reference of the image to pull

View File

@@ -15,11 +15,6 @@ PROJECT_URL = https://github.com/kata-containers
PROJECT_COMPONENT = containerd-shim-kata-v2
CONTAINERD_RUNTIME_NAME = io.containerd.kata.v2
# This snippet finds all packages inside runtime-rs. Used for tessting.
PACKAGES := $(shell cargo metadata --no-deps --format-version 1 | \
jq -r '.packages[] | select(.manifest_path | contains("runtime-rs")) | .name')
PACKAGE_FLAGS := $(patsubst %,-p %,$(PACKAGES))
include ../../utils.mk
ARCH_DIR = arch
@@ -50,9 +45,9 @@ test:
else
##TARGET default: build code
default: runtime show-header
##TARGET test: run cargo tests for runtime-rs and all its sub-crates.
##TARGET test: run cargo tests
test: static-checks-build
@cargo test $(PACKAGE_FLAGS) --target $(TRIPLE) $(EXTRA_RUSTFEATURES) -- --nocapture --skip bindgen
@cargo test --all --target $(TRIPLE) $(EXTRA_RUSTFEATURES) -- --nocapture --skip bindgen
install: install-runtime install-configs
endif
@@ -738,7 +733,7 @@ static-checks-build: $(GENERATED_FILES)
$(TARGET): $(GENERATED_FILES) $(TARGET_PATH)
$(TARGET_PATH): $(SOURCES) | show-summary
@RUSTFLAGS="$(EXTRA_RUSTFLAGS) --deny warnings" cargo build -p runtime-rs --target $(TRIPLE) $(if $(findstring release,$(BUILD_TYPE)),--release) $(EXTRA_RUSTFEATURES)
@RUSTFLAGS="$(EXTRA_RUSTFLAGS) --deny warnings" cargo build --target $(TRIPLE) $(if $(findstring release,$(BUILD_TYPE)),--release) $(EXTRA_RUSTFEATURES)
$(GENERATED_FILES): %: %.in
@sed \
@@ -774,7 +769,7 @@ endif
##TARGET run: build and run agent
run:
@cargo run -p runtime-rs --target $(TRIPLE)
@cargo run --target $(TRIPLE)
show-header:
@printf "%s - version %s (commit %s)\n\n" "$(TARGET)" "$(VERSION)" "$(COMMIT_MSG)"

View File

@@ -470,10 +470,7 @@ impl CloudHypervisorInner {
net_config.id = None;
net_config.num_queues = network_queues_pairs * 2;
info!(
sl!(),
"network device queue pairs {:?}", network_queues_pairs
);
info!(sl!(), "network device queue pairs {:?}", network_queues_pairs);
// we need ensure opening network device happens in netns.
let netns = self.netns.clone().unwrap_or_default();

View File

@@ -9,8 +9,8 @@ use crate::device::topology::PCIePort;
use crate::qemu::qmp::get_qmp_socket_path;
use crate::{
device::driver::ProtectionDeviceConfig, hypervisor_persist::HypervisorState, selinux,
HypervisorConfig, MemoryConfig, VcpuThreadIds, VsockDevice, HYPERVISOR_QEMU, KATA_BLK_DEV_TYPE,
KATA_CCW_DEV_TYPE, KATA_NVDIMM_DEV_TYPE, KATA_SCSI_DEV_TYPE,
HypervisorConfig, MemoryConfig, VcpuThreadIds, VsockDevice, HYPERVISOR_QEMU,
KATA_BLK_DEV_TYPE, KATA_CCW_DEV_TYPE, KATA_NVDIMM_DEV_TYPE, KATA_SCSI_DEV_TYPE,
};
use crate::utils::{
@@ -138,16 +138,15 @@ impl QemuInner {
&block_dev.config.path_on_host,
block_dev.config.is_readonly,
)?,
KATA_CCW_DEV_TYPE | KATA_BLK_DEV_TYPE | KATA_SCSI_DEV_TYPE => cmdline
.add_block_device(
block_dev.device_id.as_str(),
&block_dev.config.path_on_host,
block_dev
.config
.is_direct
.unwrap_or(self.config.blockdev_info.block_device_cache_direct),
block_dev.config.driver_option.as_str() == KATA_SCSI_DEV_TYPE,
)?,
KATA_CCW_DEV_TYPE | KATA_BLK_DEV_TYPE | KATA_SCSI_DEV_TYPE => cmdline.add_block_device(
block_dev.device_id.as_str(),
&block_dev.config.path_on_host,
block_dev
.config
.is_direct
.unwrap_or(self.config.blockdev_info.block_device_cache_direct),
block_dev.config.driver_option.as_str() == KATA_SCSI_DEV_TYPE,
)?,
unsupported => {
info!(sl!(), "unsupported block device driver: {}", unsupported)
}

View File

@@ -187,21 +187,11 @@ impl Qmp {
continue;
}
(None, _) => {
warn!(
sl!(),
"hotpluggable vcpu {} has no socket_id for driver {}, skipping",
core_id,
driver
);
warn!(sl!(), "hotpluggable vcpu {} has no socket_id for driver {}, skipping", core_id, driver);
continue;
}
(_, None) => {
warn!(
sl!(),
"hotpluggable vcpu {} has no thread_id for driver {}, skipping",
core_id,
driver
);
warn!(sl!(), "hotpluggable vcpu {} has no thread_id for driver {}, skipping", core_id, driver);
continue;
}
}
@@ -763,9 +753,10 @@ impl Qmp {
Ok((None, Some(scsi_addr)))
} else if block_driver == VIRTIO_BLK_CCW {
let subchannel = self.ccw_subchannel.as_mut().ok_or_else(|| {
anyhow!("CCW subchannel not available for virtio-blk-ccw hotplug")
})?;
let subchannel = self
.ccw_subchannel
.as_mut()
.ok_or_else(|| anyhow!("CCW subchannel not available for virtio-blk-ccw hotplug"))?;
let slot = subchannel
.add_device(&node_name)

View File

@@ -65,6 +65,8 @@ INITRDCONFIDENTIALNAME = $(PROJECT_TAG)-initrd-confidential.img
IMAGENAME_NV = $(PROJECT_TAG)-nvidia-gpu.img
IMAGENAME_CONFIDENTIAL_NV = $(PROJECT_TAG)-nvidia-gpu-confidential.img
INITRDNAME_NV = $(PROJECT_TAG)-initrd-nvidia-gpu.img
INITRDNAME_CONFIDENTIAL_NV = $(PROJECT_TAG)-initrd-nvidia-gpu-confidential.img
TARGET = $(BIN_PREFIX)-runtime
RUNTIME_OUTPUT = $(CURDIR)/$(TARGET)
@@ -134,6 +136,8 @@ INITRDCONFIDENTIALPATH := $(PKGDATADIR)/$(INITRDCONFIDENTIALNAME)
IMAGEPATH_NV := $(PKGDATADIR)/$(IMAGENAME_NV)
IMAGEPATH_CONFIDENTIAL_NV := $(PKGDATADIR)/$(IMAGENAME_CONFIDENTIAL_NV)
INITRDPATH_NV := $(PKGDATADIR)/$(INITRDNAME_NV)
INITRDPATH_CONFIDENTIAL_NV := $(PKGDATADIR)/$(INITRDNAME_CONFIDENTIAL_NV)
ROOTFSTYPE_EXT4 := \"ext4\"
ROOTFSTYPE_XFS := \"xfs\"
@@ -479,12 +483,16 @@ ifneq (,$(QEMUCMD))
KERNELPATH_CONFIDENTIAL_NV = $(KERNELDIR)/$(KERNELNAME_CONFIDENTIAL_NV)
DEFAULTVCPUS_NV = 1
DEFAULTMEMORY_NV = 8192
DEFAULTMEMORY_NV = 2048
DEFAULTTIMEOUT_NV = 1200
DEFAULTVFIOPORT_NV = root-port
DEFAULTPCIEROOTPORT_NV = 8
# Disable the devtmpfs mount in guest. NVRC does this, and later kata-agent
# attempts this as well in a non-failing manner. Otherwise, NVRC fails when
# using an image and /dev is already mounted.
KERNELPARAMS_NV = "cgroup_no_v1=all"
KERNELPARAMS_NV += "devtmpfs.mount=0"
KERNELPARAMS_NV += "pci=realloc"
KERNELPARAMS_NV += "pci=nocrs"
KERNELPARAMS_NV += "pci=assign-busses"
@@ -652,6 +660,10 @@ USER_VARS += IMAGENAME_NV
USER_VARS += IMAGENAME_CONFIDENTIAL_NV
USER_VARS += IMAGEPATH_NV
USER_VARS += IMAGEPATH_CONFIDENTIAL_NV
USER_VARS += INITRDNAME_NV
USER_VARS += INITRDNAME_CONFIDENTIAL_NV
USER_VARS += INITRDPATH_NV
USER_VARS += INITRDPATH_CONFIDENTIAL_NV
USER_VARS += KERNELNAME_NV
USER_VARS += KERNELPATH_NV
USER_VARS += KERNELNAME_CONFIDENTIAL_NV

View File

@@ -599,7 +599,7 @@ debug_console_enabled = false
# Agent connection dialing timeout value in seconds
# (default: 90)
dial_timeout = @DEFAULTTIMEOUT_NV@
dial_timeout = 90
[runtime]
# If enabled, the runtime will log additional debug messages to the

View File

@@ -576,7 +576,7 @@ debug_console_enabled = false
# Agent connection dialing timeout value in seconds
# (default: 90)
dial_timeout = @DEFAULTTIMEOUT_NV@
dial_timeout = 90
[runtime]
# If enabled, the runtime will log additional debug messages to the

View File

@@ -578,7 +578,7 @@ debug_console_enabled = false
# Agent connection dialing timeout value in seconds
# (default: 90)
dial_timeout = @DEFAULTTIMEOUT_NV@
dial_timeout = 90
[runtime]
# If enabled, the runtime will log additional debug messages to the

View File

@@ -1,7 +1,7 @@
module github.com/kata-containers/kata-containers/src/runtime
// Keep in sync with version in versions.yaml
go 1.25.8
go 1.25.7
// WARNING: Do NOT use `replace` directives as those break dependabot:
// https://github.com/kata-containers/kata-containers/issues/11020

View File

@@ -72,7 +72,7 @@ func IsPCIeDevice(bdf string) bool {
}
// read from /sys/bus/pci/devices/xxx/property
func GetPCIDeviceProperty(bdf string, property PCISysFsProperty) string {
func getPCIDeviceProperty(bdf string, property PCISysFsProperty) string {
if len(strings.Split(bdf, ":")) == 2 {
bdf = PCIDomain + ":" + bdf
}
@@ -220,9 +220,9 @@ func GetDeviceFromVFIODev(device config.DeviceInfo) ([]*config.VFIODev, error) {
return nil, err
}
vendorID := GetPCIDeviceProperty(deviceBDF, PCISysFsDevicesVendor)
deviceID := GetPCIDeviceProperty(deviceBDF, PCISysFsDevicesDevice)
pciClass := GetPCIDeviceProperty(deviceBDF, PCISysFsDevicesClass)
vendorID := getPCIDeviceProperty(deviceBDF, PCISysFsDevicesVendor)
deviceID := getPCIDeviceProperty(deviceBDF, PCISysFsDevicesDevice)
pciClass := getPCIDeviceProperty(deviceBDF, PCISysFsDevicesClass)
i, err := extractIndex(device.HostPath)
if err != nil {
@@ -276,7 +276,7 @@ func GetAllVFIODevicesFromIOMMUGroup(device config.DeviceInfo) ([]*config.VFIODe
switch vfioDeviceType {
case config.VFIOPCIDeviceNormalType, config.VFIOPCIDeviceMediatedType:
// This is vfio-pci and vfio-mdev specific
pciClass := GetPCIDeviceProperty(deviceBDF, PCISysFsDevicesClass)
pciClass := getPCIDeviceProperty(deviceBDF, PCISysFsDevicesClass)
// We need to ignore Host or PCI Bridges that are in the same IOMMU group as the
// passed-through devices. One CANNOT pass-through a PCI bridge or Host bridge.
// Class 0x0604 is PCI bridge, 0x0600 is Host bridge
@@ -288,8 +288,8 @@ func GetAllVFIODevicesFromIOMMUGroup(device config.DeviceInfo) ([]*config.VFIODe
continue
}
// Fetch the PCI Vendor ID and Device ID
vendorID := GetPCIDeviceProperty(deviceBDF, PCISysFsDevicesVendor)
deviceID := GetPCIDeviceProperty(deviceBDF, PCISysFsDevicesDevice)
vendorID := getPCIDeviceProperty(deviceBDF, PCISysFsDevicesVendor)
deviceID := getPCIDeviceProperty(deviceBDF, PCISysFsDevicesDevice)
// Do not directly assign to `vfio` -- need to access field still
vfio = config.VFIODev{

View File

@@ -7,7 +7,6 @@ package virtcontainers
import (
"context"
"errors"
"fmt"
"io"
"os"
@@ -1136,9 +1135,7 @@ func (c *Container) createDevices(ctx context.Context, contConfig *ContainerConf
// If we're hot-plugging this will be a no-op because at this stage
// no devices are attached to the root-port or switch-port
if err := c.annotateContainerWithVFIOMetadata(vfioColdPlugDevices); err != nil {
return err
}
c.annotateContainerWithVFIOMetadata(vfioColdPlugDevices)
return nil
}
@@ -1197,40 +1194,11 @@ func sortContainerVFIODevices(devices []config.DeviceInfo) []config.DeviceInfo {
return vfioDevices
}
// errNoSiblingFound is returned by siblingAnnotation when the VFIO device is
// not of a supported CDI device type, i.e. it has no entry in the cdiDeviceKind
// table (e.g. NVSwitches). Callers should treat this as a non-fatal "device not
// applicable" condition rather than a sibling-matching failure.
var errNoSiblingFound = fmt.Errorf("no suitable sibling found")
// cdiDeviceKey identifies a device type by vendor ID and PCI class prefix.
type cdiDeviceKey struct {
VendorID string
ClassPrefix string
}
// cdiDeviceKind maps known device types to their CDI annotation kind.
var cdiDeviceKind = map[cdiDeviceKey]string{
{VendorID: "0x10de", ClassPrefix: "0x030"}: "nvidia.com/gpu",
}
// cdiKindForDevice returns the CDI kind for a given vendor ID and PCI class,
// or empty string and false if the device is not recognized.
func cdiKindForDevice(vendorID, class string) (string, bool) {
for key, kind := range cdiDeviceKind {
if vendorID == key.VendorID && strings.Contains(class, key.ClassPrefix) {
return kind, true
}
}
return "", false
}
type DeviceRelation struct {
Bus string
Path string
Index int
BDF string
CDIKind string
Bus string
Path string
Index int
BDF string
}
// Depending on the HW we might need to inject metadata into the container
@@ -1255,13 +1223,15 @@ func (c *Container) annotateContainerWithVFIOMetadata(devices interface{}) error
// so lets first iterate over all root-port devices and then
// switch-port devices no special handling for bridge-port (PCI)
for _, dev := range config.PCIeDevicesPerPort["root-port"] {
if kind, ok := cdiKindForDevice(dev.VendorID, dev.Class); ok {
siblings = append(siblings, DeviceRelation{Bus: dev.Bus, Path: dev.HostPath, BDF: dev.BDF, CDIKind: kind})
// For the NV GPU we need special handling let's use only those
if dev.VendorID == "0x10de" && strings.Contains(dev.Class, "0x030") {
siblings = append(siblings, DeviceRelation{Bus: dev.Bus, Path: dev.HostPath, BDF: dev.BDF})
}
}
for _, dev := range config.PCIeDevicesPerPort["switch-port"] {
if kind, ok := cdiKindForDevice(dev.VendorID, dev.Class); ok {
siblings = append(siblings, DeviceRelation{Bus: dev.Bus, Path: dev.HostPath, BDF: dev.BDF, CDIKind: kind})
// For the NV GPU we need special handling let's use only those
if dev.VendorID == "0x10de" && strings.Contains(dev.Class, "0x030") {
siblings = append(siblings, DeviceRelation{Bus: dev.Bus, Path: dev.HostPath, BDF: dev.BDF})
}
}
// We need to sort the VFIO devices by bus to get the correct
@@ -1274,53 +1244,48 @@ func (c *Container) annotateContainerWithVFIOMetadata(devices interface{}) error
siblings[i].Index = i
}
// Collect container paths from either hot-plug or cold-plug devices
var containerPaths []string
if devs, ok := devices.([]ContainerDevice); ok {
for _, dev := range devs {
containerPaths = append(containerPaths, dev.ContainerPath)
}
}
if devs, ok := devices.([]config.DeviceInfo); ok {
for _, dev := range devs {
containerPaths = append(containerPaths, dev.ContainerPath)
// Now that we have the index lets connect the /dev/vfio/<num>
// to the correct index
if devices, ok := devices.([]ContainerDevice); ok {
for _, dev := range devices {
if dev.ContainerPath == "/dev/vfio/vfio" {
c.Logger().Infof("skipping /dev/vfio/vfio for vfio_mode=guest-kernel")
continue
}
err := c.siblingAnnotation(dev.ContainerPath, siblings)
if err != nil {
return err
}
}
}
// Now that we have the index lets connect the /dev/vfio/<num>
// to the correct index
for _, devPath := range containerPaths {
if !strings.HasPrefix(devPath, "/dev/vfio") {
c.Logger().Infof("skipping guest annotations for non-VFIO device %q", devPath)
continue
}
if devPath == "/dev/vfio/vfio" {
c.Logger().Infof("skipping /dev/vfio/vfio for vfio_mode=guest-kernel")
continue
}
if err := c.siblingAnnotation(devPath, siblings); err != nil {
if errors.Is(err, errNoSiblingFound) {
c.Logger().Infof("no CDI annotation for device %s (not a known CDI device type)", devPath)
if devices, ok := devices.([]config.DeviceInfo); ok {
for _, dev := range devices {
if dev.ContainerPath == "/dev/vfio/vfio" {
c.Logger().Infof("skipping /dev/vfio/vfio for vfio_mode=guest-kernel")
continue
}
return err
err := c.siblingAnnotation(dev.ContainerPath, siblings)
if err != nil {
return err
}
}
}
}
return nil
}
// createCDIAnnotation adds a container annotation mapping a VFIO device to a device index.
// createCDIAnnotation adds a container annotation mapping a VFIO device to a GPU index.
//
// devPath is the path to the VFIO device, which can be in the format
// "/dev/vfio/<num>" or "/dev/vfio/devices/vfio<num>". The function extracts
// the device number from the path and creates an annotation with the key
// "cdi.k8s.io/vfio<num>" and the value "<cdiKind>=<index>", where
// <cdiKind> is the CDI device kind (e.g. "nvidia.com/gpu"),
// <num> is the device number and <index> is the provided device index.
// "cdi.k8s.io/vfio<num>" and the value "nvidia.com/gpu=<index>", where
// <num> is the device number and <index> is the provided GPU index.
// The annotation is stored in c.config.CustomSpec.Annotations.
func (c *Container) createCDIAnnotation(devPath string, index int, cdiKind string) {
func (c *Container) createCDIAnnotation(devPath string, index int) {
// We have here either /dev/vfio/<num> or /dev/vfio/devices/vfio<num>
baseName := filepath.Base(devPath)
vfioNum := baseName
@@ -1329,68 +1294,66 @@ func (c *Container) createCDIAnnotation(devPath string, index int, cdiKind strin
vfioNum = strings.TrimPrefix(baseName, "vfio")
}
annoKey := fmt.Sprintf("cdi.k8s.io/vfio%s", vfioNum)
annoValue := fmt.Sprintf("%s=%d", cdiKind, index)
annoValue := fmt.Sprintf("nvidia.com/gpu=%d", index)
if c.config.CustomSpec.Annotations == nil {
c.config.CustomSpec.Annotations = make(map[string]string)
}
c.config.CustomSpec.Annotations[annoKey] = annoValue
c.Logger().Infof("annotated container with %s: %s", annoKey, annoValue)
}
func (c *Container) siblingAnnotation(devPath string, siblings []DeviceRelation) error {
// Resolve the device's BDFs once upfront. This serves two purposes:
// 1. Determine if the device is a known CDI type (if not, skip it)
// 2. Reuse the BDFs for sibling matching without redundant sysfs reads
isKnownCDIDevice := false
var devBDFs []string
if strings.HasPrefix(filepath.Base(devPath), "vfio") {
// IOMMUFD device (/dev/vfio/devices/vfio<NUM>): single device per char dev
major, minor, err := deviceUtils.GetMajorMinorFromDevPath(devPath)
if err != nil {
return err
for _, sibling := range siblings {
if sibling.Path == devPath {
c.createCDIAnnotation(devPath, sibling.Index)
return nil
}
bdf, err := deviceUtils.GetBDFFromVFIODev(major, minor)
if err != nil {
return err
// If the sandbox has cold-plugged an IOMMUFD device and if the
// device-plugins sends us a /dev/vfio/<NUM> device we need to
// check if the IOMMUFD device and the VFIO device are the same
// We have the sibling.BDF we now need to extract the BDF of the
// devPath that is either /dev/vfio/<NUM> or
// /dev/vfio/devices/vfio<NUM>
if strings.HasPrefix(filepath.Base(devPath), "vfio") {
// IOMMUFD device format (/dev/vfio/devices/vfio<NUM>), extract BDF from sysfs
major, minor, err := deviceUtils.GetMajorMinorFromDevPath(devPath)
if err != nil {
return err
}
iommufdBDF, err := deviceUtils.GetBDFFromVFIODev(major, minor)
if err != nil {
return err
}
if sibling.BDF == iommufdBDF {
c.createCDIAnnotation(devPath, sibling.Index)
// exit handling IOMMUFD device
return nil
}
}
devBDFs = []string{bdf}
vendorID := deviceUtils.GetPCIDeviceProperty(bdf, deviceUtils.PCISysFsDevicesVendor)
class := deviceUtils.GetPCIDeviceProperty(bdf, deviceUtils.PCISysFsDevicesClass)
_, isKnownCDIDevice = cdiKindForDevice(vendorID, class)
} else {
// Legacy VFIO group (/dev/vfio/<GROUP>): may contain multiple devices
// Legacy VFIO group device (/dev/vfio/<GROUP_NUM>), extract BDF from sysfs
vfioGroup := filepath.Base(devPath)
iommuDevicesPath := filepath.Join(config.SysIOMMUGroupPath, vfioGroup, "devices")
deviceFiles, err := os.ReadDir(iommuDevicesPath)
if err != nil {
return err
}
vfioBDFs := make([]string, 0)
for _, deviceFile := range deviceFiles {
// Get bdf of device eg 0000:00:1c.0
deviceBDF, _, _, err := deviceUtils.GetVFIODetails(deviceFile.Name(), iommuDevicesPath)
if err != nil {
return err
}
devBDFs = append(devBDFs, deviceBDF)
if !isKnownCDIDevice {
vendorID := deviceUtils.GetPCIDeviceProperty(deviceBDF, deviceUtils.PCISysFsDevicesVendor)
class := deviceUtils.GetPCIDeviceProperty(deviceBDF, deviceUtils.PCISysFsDevicesClass)
if _, ok := cdiKindForDevice(vendorID, class); ok {
isKnownCDIDevice = true
}
}
vfioBDFs = append(vfioBDFs, deviceBDF)
}
}
if !isKnownCDIDevice {
return fmt.Errorf("device %s: %w", devPath, errNoSiblingFound)
}
for _, sibling := range siblings {
if sibling.Path == devPath || slices.Contains(devBDFs, sibling.BDF) {
c.createCDIAnnotation(devPath, sibling.Index, sibling.CDIKind)
if slices.Contains(vfioBDFs, sibling.BDF) {
c.createCDIAnnotation(devPath, sibling.Index)
// exit handling legacy VFIO device
return nil
}
}
return fmt.Errorf("device %s is a known CDI device type but failed to match any sibling by path or BDF", devPath)
return fmt.Errorf("failed to match device %s with any cold-plugged GPU device by path or BDF; no suitable sibling found", devPath)
}
// create creates and starts a container inside a Sandbox. It has to be
@@ -1419,9 +1382,7 @@ func (c *Container) create(ctx context.Context) (err error) {
return
}
if err := c.annotateContainerWithVFIOMetadata(c.devices); err != nil {
return fmt.Errorf("annotating VFIO devices: %w", err)
}
c.annotateContainerWithVFIOMetadata(c.devices)
// Deduce additional system mount info that should be handled by the agent
// inside the VM

View File

@@ -54,11 +54,6 @@ func validateHypervisorConfig(conf *HypervisorConfig) error {
conf.BlockDeviceDriver = config.VirtioBlockCCW
}
// CoCo guest hardening: virtio-mmio is not hardened for confidential computing.
if conf.ConfidentialGuest && conf.BlockDeviceDriver == config.VirtioMmio {
return fmt.Errorf("confidential guests must not use virtio-mmio (use virtio-blk-pci); virtio-mmio is not hardened for CoCo")
}
if conf.DefaultMaxVCPUs == 0 || conf.DefaultMaxVCPUs > defaultMaxVCPUs {
conf.DefaultMaxVCPUs = defaultMaxVCPUs
}

View File

@@ -841,6 +841,7 @@ func (q *qemu) createPCIeTopology(qemuConfig *govmmQemu.Config, hypervisorConfig
// /dev/vfio/devices/vfio0
// (1) Check if we have the new IOMMUFD or old container based VFIO
if strings.HasPrefix(dev.HostPath, pkgDevice.IommufdDevPath) {
q.Logger().Infof("### IOMMUFD Path: %s", dev.HostPath)
vfioDevices, err = drivers.GetDeviceFromVFIODev(dev)
if err != nil {
return fmt.Errorf("Cannot get VFIO device from IOMMUFD with device: %v err: %v", dev, err)

View File

@@ -80,7 +80,7 @@ version = "0.7.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a824f2aa7e75a0c98c5a504fceb80649e9c35265d44525b5f94de4771a395cd"
dependencies = [
"getrandom 0.2.15",
"getrandom",
"once_cell",
"version_check",
]
@@ -260,11 +260,11 @@ checksum = "d468802bab17cbc0cc575e9b053f41e72aa36bfa6b7f55e3529ffa43161b97fa"
[[package]]
name = "aws-lc-fips-sys"
version = "0.13.12"
version = "0.13.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5ed8cd42adddefbdb8507fb7443fa9b666631078616b78f70ed22117b5c27d90"
checksum = "2608e5a7965cc9d58c56234d346c9c89b824c4c8652b6f047b3bd0a777c0644f"
dependencies = [
"bindgen 0.72.1",
"bindgen 0.69.5",
"cc",
"cmake",
"dunce",
@@ -274,9 +274,9 @@ dependencies = [
[[package]]
name = "aws-lc-rs"
version = "1.16.1"
version = "1.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "94bffc006df10ac2a68c83692d734a465f8ee6c5b384d8545a636f81d858f4bf"
checksum = "93fcc8f365936c834db5514fc45aee5b1202d677e6b40e48468aaaa8183ca8c7"
dependencies = [
"aws-lc-fips-sys",
"aws-lc-sys",
@@ -286,11 +286,11 @@ dependencies = [
[[package]]
name = "aws-lc-sys"
version = "0.38.0"
version = "0.29.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4321e568ed89bb5a7d291a7f37997c2c0df89809d7b6d12062c81ddb54aa782e"
checksum = "61b1d86e7705efe1be1b569bab41d4fa1e14e220b60a160f78de2db687add079"
dependencies = [
"bindgen 0.72.1",
"bindgen 0.69.5",
"cc",
"cmake",
"dunce",
@@ -407,22 +407,25 @@ dependencies = [
[[package]]
name = "bindgen"
version = "0.72.1"
version = "0.69.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "993776b509cfb49c750f11b8f07a46fa23e0a1386ffc01fb1e7d343efc387895"
checksum = "271383c67ccabffb7381723dea0672a673f292304fcb45c01cc648c7a8d58088"
dependencies = [
"bitflags 2.6.0",
"cexpr",
"clang-sys",
"itertools 0.11.0",
"lazy_static",
"lazycell",
"log",
"prettyplease",
"proc-macro2",
"quote",
"regex",
"rustc-hash 2.1.1",
"rustc-hash 1.1.0",
"shlex",
"syn 2.0.87",
"which",
]
[[package]]
@@ -654,11 +657,10 @@ dependencies = [
[[package]]
name = "cc"
version = "1.2.56"
version = "1.2.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aebf35691d1bfb0ac386a69bac2fde4dd276fb618cf8bf4f5318fe285e821bb2"
checksum = "8691782945451c1c383942c4874dbe63814f61cb57ef773cda2972682b7bb3c0"
dependencies = [
"find-msvc-tools",
"jobserver",
"libc",
"shlex",
@@ -974,7 +976,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0dc92fb57ca44df6db8059111ab3af99a63d5d0f8375d9972e319a379c6bab76"
dependencies = [
"generic-array",
"rand_core 0.6.4",
"rand_core",
"subtle",
"zeroize",
]
@@ -986,7 +988,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1bfb12502f3fc46cca1bb51ac28df9d618d813cdc3d2f25b9fe775a34af26bb3"
dependencies = [
"generic-array",
"rand_core 0.6.4",
"rand_core",
"typenum",
]
@@ -1375,7 +1377,7 @@ checksum = "4a3daa8e81a3963a60642bcc1f90a670680bd4a77535faa384e9d1c79d620871"
dependencies = [
"curve25519-dalek",
"ed25519",
"rand_core 0.6.4",
"rand_core",
"serde",
"sha2 0.10.9",
"subtle",
@@ -1403,7 +1405,7 @@ dependencies = [
"hkdf",
"pem-rfc7468",
"pkcs8",
"rand_core 0.6.4",
"rand_core",
"sec1",
"subtle",
"zeroize",
@@ -1485,7 +1487,7 @@ checksum = "fe5e43d0f78a42ad591453aedb1d7ae631ce7ee445c7643691055a9ed8d3b01c"
dependencies = [
"log",
"once_cell",
"rand 0.8.5",
"rand",
]
[[package]]
@@ -1503,7 +1505,7 @@ version = "0.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ded41244b729663b1e574f1b4fb731469f69f79c17667b5d776b16cda0479449"
dependencies = [
"rand_core 0.6.4",
"rand_core",
"subtle",
]
@@ -1525,12 +1527,6 @@ dependencies = [
"windows-sys 0.59.0",
]
[[package]]
name = "find-msvc-tools"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582"
[[package]]
name = "fixedbitset"
version = "0.2.0"
@@ -1705,20 +1701,6 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "getrandom"
version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd"
dependencies = [
"cfg-if 1.0.4",
"js-sys",
"libc",
"r-efi",
"wasip2",
"wasm-bindgen",
]
[[package]]
name = "getset"
version = "0.1.6"
@@ -1769,7 +1751,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0f9ef7462f7c099f518d754361858f86d8a07af53ba9af0fe635bbccb151a63"
dependencies = [
"ff",
"rand_core 0.6.4",
"rand_core",
"subtle",
]
@@ -2105,7 +2087,7 @@ dependencies = [
"qapi",
"qapi-qmp",
"qapi-spec",
"rand 0.8.5",
"rand",
"rust-ini",
"safe-path 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
"seccompiler",
@@ -2432,11 +2414,10 @@ dependencies = [
[[package]]
name = "js-sys"
version = "0.3.91"
version = "0.3.70"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b49715b7073f385ba4bc528e5747d02e66cb39c6146efb66b781f131f0fb399c"
checksum = "1868808506b929d7b0cfa8f75951347aa71bb21144b7791bae35d9bccfcfe37a"
dependencies = [
"once_cell",
"wasm-bindgen",
]
@@ -2475,7 +2456,7 @@ dependencies = [
"oci-spec",
"protobuf",
"protocols",
"rand 0.8.5",
"rand",
"safe-path 0.1.0",
"serde",
"serde_json",
@@ -2500,7 +2481,7 @@ dependencies = [
"nix 0.26.4",
"oci-spec",
"pci-ids",
"rand 0.8.5",
"rand",
"runtime-spec",
"serde",
"serde_json",
@@ -2701,12 +2682,6 @@ dependencies = [
"libc",
]
[[package]]
name = "lru-slab"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154"
[[package]]
name = "matchit"
version = "0.8.4"
@@ -2916,7 +2891,7 @@ dependencies = [
"num-integer",
"num-iter",
"num-traits",
"rand 0.8.5",
"rand",
"smallvec",
"zeroize",
]
@@ -2975,9 +2950,9 @@ checksum = "51e219e79014df21a225b1860a479e2dcd7cbd9130f4defd4bd0e191ea31d67d"
dependencies = [
"base64 0.22.1",
"chrono",
"getrandom 0.2.15",
"getrandom",
"http 1.1.0",
"rand 0.8.5",
"rand",
"reqwest",
"serde",
"serde_json",
@@ -3121,7 +3096,7 @@ dependencies = [
"oauth2",
"p256",
"p384",
"rand 0.8.5",
"rand",
"rsa",
"serde",
"serde-value",
@@ -3194,7 +3169,7 @@ dependencies = [
"ecdsa",
"elliptic-curve",
"primeorder",
"rand_core 0.6.4",
"rand_core",
"sha2 0.10.9",
]
@@ -3228,7 +3203,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "346f04948ba92c43e8469c1ee6736c7563d71012b17d40745260fe106aac2166"
dependencies = [
"base64ct",
"rand_core 0.6.4",
"rand_core",
"subtle",
]
@@ -3351,7 +3326,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3c80231409c20246a13fddb31776fb942c38553c51e871f8cbd687a4cfb5843d"
dependencies = [
"phf_shared",
"rand 0.8.5",
"rand",
]
[[package]]
@@ -3429,7 +3404,7 @@ checksum = "f950b2377845cebe5cf8b5165cb3cc1a5e0fa5cfa3e1f7f55707d8fd82e0a7b7"
dependencies = [
"der",
"pkcs5",
"rand_core 0.6.4",
"rand_core",
"spki",
]
@@ -3827,23 +3802,19 @@ dependencies = [
[[package]]
name = "quinn-proto"
version = "0.11.14"
version = "0.11.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "434b42fec591c96ef50e21e886936e66d3cc3f737104fdb9b737c40ffb94c098"
checksum = "fadfaed2cd7f389d0161bb73eeb07b7b78f8691047a6f3e73caaeae55310a4a6"
dependencies = [
"bytes",
"getrandom 0.3.4",
"lru-slab",
"rand 0.9.2",
"rand",
"ring",
"rustc-hash 2.1.1",
"rustls",
"rustls-pki-types",
"slab",
"thiserror 2.0.12",
"thiserror 1.0.40",
"tinyvec",
"tracing",
"web-time",
]
[[package]]
@@ -3868,12 +3839,6 @@ dependencies = [
"proc-macro2",
]
[[package]]
name = "r-efi"
version = "5.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f"
[[package]]
name = "radium"
version = "0.7.0"
@@ -3887,18 +3852,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34af8d1a0e25924bc5b7c43c079c942339d8f0a8b57c39049bef581b46327404"
dependencies = [
"libc",
"rand_chacha 0.3.1",
"rand_core 0.6.4",
]
[[package]]
name = "rand"
version = "0.9.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6db2770f06117d490610c7488547d543617b21bfa07796d7a12f6f1bd53850d1"
dependencies = [
"rand_chacha 0.9.0",
"rand_core 0.9.5",
"rand_chacha",
"rand_core",
]
[[package]]
@@ -3908,17 +3863,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88"
dependencies = [
"ppv-lite86",
"rand_core 0.6.4",
]
[[package]]
name = "rand_chacha"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3022b5f1df60f26e1ffddd6c66e8aa15de382ae63b3a0c1bfc0e4d3e3f325cb"
dependencies = [
"ppv-lite86",
"rand_core 0.9.5",
"rand_core",
]
[[package]]
@@ -3927,16 +3872,7 @@ version = "0.6.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c"
dependencies = [
"getrandom 0.2.15",
]
[[package]]
name = "rand_core"
version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "76afc826de14238e6e8c374ddcc1fa19e374fd8dd986b0d2af0d02377261d83c"
dependencies = [
"getrandom 0.3.4",
"getrandom",
]
[[package]]
@@ -3972,7 +3908,7 @@ version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b033d837a7cf162d7993aded9304e30a83213c648b6e389db233191f891e5c2b"
dependencies = [
"getrandom 0.2.15",
"getrandom",
"redox_syscall 0.2.16",
"thiserror 1.0.40",
]
@@ -3999,9 +3935,9 @@ dependencies = [
[[package]]
name = "regex"
version = "1.12.3"
version = "1.10.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e10754a14b9137dd7b1e3e5b0493cc9171fdd105e0ab477f51b72e7f3ac0e276"
checksum = "b91213439dad192326a0d7c6ee3955910425f441d7038e0d6933b0aec5c4517f"
dependencies = [
"aho-corasick",
"memchr",
@@ -4011,9 +3947,9 @@ dependencies = [
[[package]]
name = "regex-automata"
version = "0.4.14"
version = "0.4.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e1dd4122fc1595e8162618945476892eefca7b88c52820e74af6262213cae8f"
checksum = "38caf58cc5ef2fed281f89292ef23f6365465ed9a41b7a7754eb4e26496c92df"
dependencies = [
"aho-corasick",
"memchr",
@@ -4022,9 +3958,9 @@ dependencies = [
[[package]]
name = "regex-syntax"
version = "0.8.10"
version = "0.8.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc897dd8d9e8bd1ed8cdad82b5966c3e0ecae09fb1907d58efaa013543185d0a"
checksum = "7a66a03ae7c801facd77a29370b4faec201768915ac14a721ba36f20bc9c209b"
[[package]]
name = "rend"
@@ -4100,7 +4036,7 @@ checksum = "a4689e6c2294d81e88dc6261c768b63bc4fcdb852be6d1352498b114f61383b7"
dependencies = [
"cc",
"cfg-if 1.0.4",
"getrandom 0.2.15",
"getrandom",
"libc",
"untrusted 0.9.0",
"windows-sys 0.52.0",
@@ -4157,7 +4093,7 @@ dependencies = [
"num-traits",
"pkcs1",
"pkcs8",
"rand_core 0.6.4",
"rand_core",
"signature",
"spki",
"subtle",
@@ -4193,7 +4129,7 @@ dependencies = [
"borsh",
"bytes",
"num-traits",
"rand 0.8.5",
"rand",
"rkyv",
"serde",
"serde_json",
@@ -4294,7 +4230,6 @@ version = "1.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "229a4a4c221013e7e1f1a043678c5cc39fe5171437c88fb47151a21e6f5b5c79"
dependencies = [
"web-time",
"zeroize",
]
@@ -4484,7 +4419,7 @@ dependencies = [
"ed25519",
"ed25519-dalek",
"flate2",
"getrandom 0.2.15",
"getrandom",
"hkdf",
"idea",
"idna",
@@ -4498,8 +4433,8 @@ dependencies = [
"p256",
"p384",
"p521",
"rand 0.8.5",
"rand_core 0.6.4",
"rand",
"rand_core",
"regex",
"regex-syntax",
"ripemd",
@@ -4758,7 +4693,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "77549399552de45a898a580c1b41d445bf730df867cc44e6c0233bbc4b8329de"
dependencies = [
"digest 0.10.7",
"rand_core 0.6.4",
"rand_core",
]
[[package]]
@@ -4787,7 +4722,7 @@ dependencies = [
"pem",
"pkcs1",
"pkcs8",
"rand 0.8.5",
"rand",
"regex",
"reqwest",
"rsa",
@@ -5117,7 +5052,7 @@ version = "0.1.0"
dependencies = [
"anyhow",
"kata-types",
"rand 0.8.5",
"rand",
]
[[package]]
@@ -5762,25 +5697,29 @@ version = "0.11.0+wasi-snapshot-preview1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9c8d87e72b64a3b4db28d11ce29237c246188f4f51057d65a7eab63b7987e423"
[[package]]
name = "wasip2"
version = "1.0.2+wasi-0.2.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9517f9239f02c069db75e65f174b3da828fe5f5b945c4dd26bd25d89c03ebcf5"
dependencies = [
"wit-bindgen",
]
[[package]]
name = "wasm-bindgen"
version = "0.2.114"
version = "0.2.93"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6532f9a5c1ece3798cb1c2cfdba640b9b3ba884f5db45973a6f442510a87d38e"
checksum = "a82edfc16a6c469f5f44dc7b571814045d60404b55a0ee849f9bcfa2e63dd9b5"
dependencies = [
"cfg-if 1.0.4",
"once_cell",
"rustversion",
"wasm-bindgen-macro",
]
[[package]]
name = "wasm-bindgen-backend"
version = "0.2.93"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9de396da306523044d3302746f1208fa71d7532227f15e347e2d93e4145dd77b"
dependencies = [
"bumpalo",
"log",
"once_cell",
"proc-macro2",
"quote",
"syn 2.0.87",
"wasm-bindgen-shared",
]
@@ -5798,9 +5737,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.114"
version = "0.2.93"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "18a2d50fcf105fb33bb15f00e7a77b772945a2ee45dcf454961fd843e74c18e6"
checksum = "585c4c91a46b072c92e908d99cb1dcdf95c5218eeb6f3bf1efa991ee7a68cccf"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
@@ -5808,25 +5747,22 @@ dependencies = [
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.114"
version = "0.2.93"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "03ce4caeaac547cdf713d280eda22a730824dd11e6b8c3ca9e42247b25c631e3"
checksum = "afc340c74d9005395cf9dd098506f7f44e38f2b4a21c6aaacf9a105ea5e1e836"
dependencies = [
"bumpalo",
"proc-macro2",
"quote",
"syn 2.0.87",
"wasm-bindgen-backend",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.114"
version = "0.2.93"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75a326b8c223ee17883a4251907455a2431acc2791c98c26279376490c378c16"
dependencies = [
"unicode-ident",
]
checksum = "c62a0a307cb4a311d3a07867860911ca130c3494e8c2719593806c08bc5d0484"
[[package]]
name = "wasm-streams"
@@ -5851,16 +5787,6 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "web-time"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a6580f308b1fad9207618087a65c04e7a10bc77e02c8e84e9b00dd4b12fa0bb"
dependencies = [
"js-sys",
"wasm-bindgen",
]
[[package]]
name = "webpki-roots"
version = "0.26.6"
@@ -6233,12 +6159,6 @@ dependencies = [
"memchr",
]
[[package]]
name = "wit-bindgen"
version = "0.51.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d7249219f66ced02969388cf2bb044a09756a083d0fab1e566056b04d9fbcaa5"
[[package]]
name = "writeable"
version = "0.6.1"
@@ -6261,7 +6181,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c7e468321c81fb07fa7f4c636c3972b9100f0346e5b6a9f2bd0603a52f7ed277"
dependencies = [
"curve25519-dalek",
"rand_core 0.6.4",
"rand_core",
"zeroize",
]

View File

@@ -1,7 +1,7 @@
module kata-containers/csi-kata-directvolume
// Keep in sync with version in versions.yaml
go 1.25.8
go 1.25.7
// WARNING: Do NOT use `replace` directives as those break dependabot:
// https://github.com/kata-containers/kata-containers/issues/11020

4065
src/tools/genpolicy/Cargo.lock generated Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -6,21 +6,21 @@
[package]
name = "genpolicy"
version = "0.1.0"
authors.workspace = true
authors = ["The Kata Containers community <kata-dev@lists.katacontainers.io>"]
edition = "2021"
license.workspace = true
license = "Apache-2.0"
[dependencies]
# Logging.
env_logger = "0.10.0"
log.workspace = true
log = "0.4.17"
# Command line parsing.
clap = { version = "4.5.40", features = ["derive"] }
# YAML file serialization/deserialization.
base64 = "0.21.0"
serde.workspace = true
serde = { version = "1.0.159", features = ["derive"] }
regex = "1.10.5"
# Newer serde_yaml versions are using unsafe-libyaml instead of yaml-rust,
@@ -37,25 +37,25 @@ regex = "1.10.5"
serde_yaml = "0.8"
# Container repository.
anyhow.workspace = true
async-trait.workspace = true
anyhow = "1.0.32"
async-trait = "0.1.68"
docker_credential = "1.3.1"
flate2 = "1.1.5"
fs2 = "0.4.3"
oci-client = { version = "0.16" }
oci-client = { version = "0.12.0" }
openssl = { version = "0.10.73", features = ["vendored"] }
serde_ignored = "0.1.7"
serde_json.workspace = true
serde_json = "1.0.39"
json-patch = "4.1"
tempfile.workspace = true
tokio = { workspace = true, features = ["rt-multi-thread"] }
tempfile = "3.19.1"
tokio = { version = "1.38.0", features = ["rt-multi-thread"] }
# OCI container specs.
oci-spec.workspace = true
oci-spec = { version = "0.8.1", features = ["runtime"] }
# Kata Agent protocol.
protocols = { workspace = true, features = ["with-serde"] }
protobuf.workspace = true
protocols = { path = "../../libs/protocols", features = ["with-serde"] }
protobuf = "3.2.0"
# containerd image pull support
k8s-cri = "0.7.0"
@@ -67,11 +67,11 @@ containerd-client = "0.4.0"
tar = "0.4.41"
# init data support
kata-types = { path = "../../libs/kata-types" } # TODO(burgerdev): figure out how to use this from workspace without including safe-path.
kata-types = { path = "../../libs/kata-types" }
[dev-dependencies]
kata-agent-policy.workspace = true
slog.workspace = true
kata-agent-policy = { path = "../../agent/policy" }
slog = "2.5.2"
assert_cmd = "2.0.14"
[package.metadata.cargo-machete]

View File

@@ -37,7 +37,7 @@ vendor:
cargo vendor
test: $(GENERATED_FILES)
@RUSTFLAGS="$(EXTRA_RUSTFLAGS) --deny warnings" cargo test -p genpolicy --all-targets --all-features --target $(TRIPLE)
@RUSTFLAGS="$(EXTRA_RUSTFLAGS) --deny warnings" cargo test --all-targets --all-features --target $(TRIPLE)
install: $(GENERATED_FILES)
@RUSTFLAGS="$(EXTRA_RUSTFLAGS) --deny warnings" cargo install --locked --target $(TRIPLE) --path .

View File

@@ -173,7 +173,7 @@
"encryption_key=ephemeral"
],
"source": "",
"mount_point": "$(spath)/$(b64_device_id)",
"mount_point": "^$(spath)/$(b64_device_id)$",
"fstype": "ext4",
"options": [],
"shared": true

View File

@@ -801,6 +801,8 @@ allow_by_bundle_or_sandbox_id(p_oci, i_oci, p_storages, i_storages) if {
# Match each input mount with a Policy mount.
# Reject possible attempts to match multiple input mounts with a single Policy mount.
print("allow_by_bundle_or_sandbox_id: p_oci.Mounts =", p_oci.Mounts)
print("allow_by_bundle_or_sandbox_id: i_oci.Mounts =", i_oci.Mounts)
p_matches := { p_index | some i_index; p_index = allow_mount(p_oci, i_oci.Mounts[i_index], i_storages, bundle_id, sandbox_id) }
print("allow_by_bundle_or_sandbox_id: p_matches =", p_matches)
@@ -1096,48 +1098,47 @@ is_ip_other_byte(component) if {
number <= 255
}
# device mounts
# allow_mount returns the policy index (p_index) if a given input mount matches a policy mount.
allow_mount(p_oci, i_mount, i_storages, bundle_id, sandbox_id):= p_index if {
print("-------- allow_mount 1: i_mount =", i_mount)
some p_index, p_mount in p_oci.Mounts
some i_storage in i_storages
print("allow_mount 1: p_mount =", p_mount)
check_mount(p_mount, i_mount, bundle_id, sandbox_id)
print("allow_mount 1: p_index =", p_index, "p_mount =", p_mount)
check_mount(p_mount, i_mount, i_storage, bundle_id, sandbox_id)
print("allow_mount 1: true, p_index =", p_index)
}
# This rule is identical to the first one, minus the i_storage requirement:
# - Rule 1 accepts mounts that rule 2 rejects (e.g. when i_storage.mount_point == i_mount.source).
# - Rule 2 does NOT accept all the mounts that rule 1 rejects (because it's missing i_storage).
allow_mount(p_oci, i_mount, i_storages, bundle_id, sandbox_id):= p_index if {
print("-------- allow_mount 2: i_mount =", i_mount)
print("allow_mount 2: i_mount =", i_mount)
some p_index, p_mount in p_oci.Mounts
print("allow_mount 2: p_mount =", p_mount)
p_mount.destination == i_mount.destination
p_mount.type_ == i_mount.type_
p_mount.options == i_mount.options
some i_storage in i_storages
print("allow_mount 2: i_storage =", i_storage)
i_storage.mount_point == i_mount.source
print("allow_mount 2: p_index =", p_index, "p_mount =", p_mount)
check_mount(p_mount, i_mount, null, bundle_id, sandbox_id)
print("allow_mount 2: true, p_index =", p_index)
}
check_mount(p_mount, i_mount, bundle_id, sandbox_id) if {
check_mount(p_mount, i_mount, i_storage, bundle_id, sandbox_id) if {
p_mount == i_mount
print("check_mount 1: true")
}
check_mount(p_mount, i_mount, bundle_id, sandbox_id) if {
check_mount(p_mount, i_mount, i_storage, bundle_id, sandbox_id) if {
p_mount.destination == i_mount.destination
p_mount.type_ == i_mount.type_
p_mount.options == i_mount.options
mount_source_allows(p_mount, i_mount, bundle_id, sandbox_id)
mount_source_allows(p_mount, i_mount, i_storage, bundle_id, sandbox_id)
print("check_mount 2: true")
}
check_mount(p_mount, i_mount, bundle_id, sandbox_id) if {
check_mount(p_mount, i_mount, i_storage, bundle_id, sandbox_id) if {
# This check passes if the policy container has RW, the input container has
# RO and the volume type is sysfs, working around different handling of
# privileged containers after containerd 2.0.4.
@@ -1149,11 +1150,12 @@ check_mount(p_mount, i_mount, bundle_id, sandbox_id) if {
i_options := {x | x = i_mount.options[_]} | {"rw"}
p_options := {x | x = p_mount.options[_]} | {"ro"}
p_options == i_options
print("check_mount 3: true")
}
mount_source_allows(p_mount, i_mount, bundle_id, sandbox_id) if {
mount_source_allows(p_mount, i_mount, i_storage, bundle_id, sandbox_id) if {
p_mount.source != ""
regex1 := p_mount.source
print("mount_source_allows 1: regex1 =", regex1)
@@ -1169,7 +1171,9 @@ mount_source_allows(p_mount, i_mount, bundle_id, sandbox_id) if {
print("mount_source_allows 1: true")
}
mount_source_allows(p_mount, i_mount, bundle_id, sandbox_id) if {
mount_source_allows(p_mount, i_mount, i_storage, bundle_id, sandbox_id) if {
p_mount.source != ""
regex1 := p_mount.source
print("mount_source_allows 2: regex1 =", regex1)
@@ -1185,6 +1189,10 @@ mount_source_allows(p_mount, i_mount, bundle_id, sandbox_id) if {
print("mount_source_allows 2: true")
}
mount_source_allows(p_mount, i_mount, i_storage, bundle_id, sandbox_id) if {
i_storage.mount_point == i_mount.source
print("mount_source_allows 3: true")
}
######################################################################
# Create container Storages
@@ -1269,6 +1277,10 @@ allow_block_storage(p_storages, i_storage, bundle_id, sandbox_id) if {
some p_storage in p_storages
# This matches the `volumes.emptyDir_encrypted` setting.
# `i_storage.driver` is validated by the caller.
p_storage.driver == ""
allow_storage_base(p_storage, i_storage, bundle_id, sandbox_id)
print("allow_block_storage: true")
@@ -1306,7 +1318,6 @@ allow_storage_source(p_storage, i_storage, bundle_id) if {
allow_storage_options(p_storage, i_storage) if {
print("allow_storage_options 1: start")
p_storage.driver != "blk"
p_storage.driver != "overlayfs"
p_storage.options == i_storage.options
@@ -1390,7 +1401,7 @@ allow_mount_point_by_device_id(p_storage, i_storage) if {
mount3 := replace(mount2, "$(b64_device_id)", base64url.encode(i_storage.source))
print("allow_mount_point_by_device_id: mount3 =", mount3)
mount3 == i_storage.mount_point
regex.match(mount3, i_storage.mount_point)
print("allow_mount_point_by_device_id: true")
}

View File

@@ -168,10 +168,10 @@ fn get_empty_dir_mount_and_storage(
source: settings_empty_dir.source.clone(),
fstype: settings_empty_dir.fstype.clone(),
options,
mount_point: if settings_empty_dir.mount_point.ends_with('/') {
format!("{}{}$", &settings_empty_dir.mount_point, &yaml_mount.name)
} else {
mount_point: if settings_empty_dir.mount_point.ends_with('$') {
settings_empty_dir.mount_point.clone()
} else {
format!("{}{}$", &settings_empty_dir.mount_point, &yaml_mount.name)
},
fs_group: protobuf::MessageField::none(),
shared: settings_empty_dir.shared,

View File

@@ -1,7 +1,7 @@
module github.com/kata-containers/kata-containers/src/tools/log-parser
// Keep in sync with version in versions.yaml
go 1.25.8
go 1.25.7
require (
github.com/BurntSushi/toml v1.1.0

View File

@@ -1,7 +1,7 @@
module github.com/kata-containers/tests
// Keep in sync with version in versions.yaml
go 1.25.8
go 1.25.7
// WARNING: Do NOT use `replace` directives as those break dependabot:
// https://github.com/kata-containers/kata-containers/issues/11020

View File

@@ -237,60 +237,6 @@ function create_coco_pod_yaml_with_annotations() {
fi
}
# Sealed secrets (signed JWS ES256). Pre-created with guest-components secret CLI; see
# https://github.com/confidential-containers/guest-components/blob/main/confidential-data-hub/docs/SEALED_SECRET.md
# Tests provision the signing public key to KBS and use these pre-created sealed secret strings.
#
# To regenerate the signing key and sealed secrets:
# Install required dependencies, clone guest-components repository and change to guest-components/confidential-data-hub
# Create private and public JWK, for example:
# python3 -c "
# from jwcrypto import jwk
# k = jwk.JWK.generate(kty='EC', crv='P-256', alg='ES256', use='sig', kid='sealed-secret-test-key')
# with open('signing-key-private.jwk', 'w') as f:
# f.write(k.export_private())
# with open('signing-key-public.jwk', 'w') as f:
# f.write(k.export_public())
# print('Created signing-key-private.jwk and signing-key-public.jwk')
# "
#
# Build the secret CLI:
# cargo build -p confidential-data-hub --bin secret
#
# Create the sealed secret test secret:
# cargo run -p confidential-data-hub --bin secret -q -- seal \
# --signing-kid "kbs:///default/signing-key/sealed-secret" \
# --signing-jwk-path ./signing-key-private.jwk \
# vault --resource-uri "kbs:///default/sealed-secret/test" --provider kbs
#
# Create the NIM test instruct secret:
# cargo run -p confidential-data-hub --bin secret -q -- seal \
# --signing-kid "kbs:///default/signing-key/sealed-secret" \
# --signing-jwk-path ./signing-key-private.jwk \
# vault --resource-uri "kbs:///default/ngc-api-key/instruct" --provider kbs
#
# Create the NIM test embedqa secret:
# cargo run -p confidential-data-hub --bin secret -q -- seal \
# --signing-kid "kbs:///default/signing-key/sealed-secret" \
# --signing-jwk-path ./signing-key-private.jwk \
# vault --resource-uri "kbs:///default/ngc-api-key/embedqa" --provider kbs
#
# Public JWK (no private key) used to verify the pre-created sealed secrets. Must match the key pair
# that was used to sign SEALED_SECRET_PRECREATED_*.
SEALED_SECRET_SIGNING_PUBLIC_JWK='{"alg":"ES256","crv":"P-256","kid":"sealed-secret-test-key","kty":"EC","use":"sig","x":"4jH376AuwTUCIx65AJ_56D7SZzWf7sGcEA7_Csq21UM","y":"rjdceysnSa5ZfzWOPGCURMUuHndxBAGUu4ISTIVN0yA"}'
# Pre-created sealed secret for k8s-sealed-secret.bats (points to kbs:///default/sealed-secret/test)
export SEALED_SECRET_PRECREATED_TEST="sealed.eyJiNjQiOnRydWUsImFsZyI6IkVTMjU2Iiwia2lkIjoia2JzOi8vL2RlZmF1bHQvc2lnbmluZy1rZXkvc2VhbGVkLXNlY3JldCJ9.eyJ2ZXJzaW9uIjoiMC4xLjAiLCJ0eXBlIjoidmF1bHQiLCJuYW1lIjoia2JzOi8vL2RlZmF1bHQvc2VhbGVkLXNlY3JldC90ZXN0IiwicHJvdmlkZXIiOiJrYnMiLCJwcm92aWRlcl9zZXR0aW5ncyI6e30sImFubm90YXRpb25zIjp7fX0.ZI2fTv5ramHqHQa9DKBFD5hlJ_Mjf6cEIcpsNGshpyhEiKklML0abfH600TD7LAFHf53oDIJmEcVsDtJ20UafQ"
# Pre-created sealed secrets for k8s-nvidia-nim.bats (point to kbs:///default/ngc-api-key/instruct and embedqa)
export SEALED_SECRET_PRECREATED_NIM_INSTRUCT="sealed.eyJiNjQiOnRydWUsImFsZyI6IkVTMjU2Iiwia2lkIjoia2JzOi8vL2RlZmF1bHQvc2lnbmluZy1rZXkvc2VhbGVkLXNlY3JldCJ9.eyJ2ZXJzaW9uIjoiMC4xLjAiLCJ0eXBlIjoidmF1bHQiLCJuYW1lIjoia2JzOi8vL2RlZmF1bHQvbmdjLWFwaS1rZXkvaW5zdHJ1Y3QiLCJwcm92aWRlciI6ImticyIsInByb3ZpZGVyX3NldHRpbmdzIjp7fSwiYW5ub3RhdGlvbnMiOnt9fQ.wpqvVFUaQymqgf54h70shZWDpk2NLW305wALz09YF0GKFBKBQiQB2sRwvn9Jk_rSju3YGLYxPO2Ub8qUbiMCuA"
export SEALED_SECRET_PRECREATED_NIM_EMBEDQA="sealed.eyJiNjQiOnRydWUsImFsZyI6IkVTMjU2Iiwia2lkIjoia2JzOi8vL2RlZmF1bHQvc2lnbmluZy1rZXkvc2VhbGVkLXNlY3JldCJ9.eyJ2ZXJzaW9uIjoiMC4xLjAiLCJ0eXBlIjoidmF1bHQiLCJuYW1lIjoia2JzOi8vL2RlZmF1bHQvbmdjLWFwaS1rZXkvZW1iZWRxYSIsInByb3ZpZGVyIjoia2JzIiwicHJvdmlkZXJfc2V0dGluZ3MiOnt9LCJhbm5vdGF0aW9ucyI6e319.4C1uqtVXi_qZT8vh_yZ4KpsRdgr2s4hU6ElKj18Hq1DJi_Iji61yuKsS6S1jWdb7drdoKKACvMD6RmCd85SJOQ"
# Provision the signing public key to KBS so CDH can verify the pre-created sealed secrets.
function setup_sealed_secret_signing_public_key() {
kbs_set_resource "default" "signing-key" "sealed-secret" "${SEALED_SECRET_SIGNING_PUBLIC_JWK}"
}
function get_initdata_with_cdh_image_section() {
CDH_IMAGE_SECTION=${1:-""}

View File

@@ -588,6 +588,7 @@ function main() {
install-kata-tools) install_kata_tools "${2:-}" ;;
install-kbs-client) install_kbs_client ;;
get-cluster-credentials) get_cluster_credentials ;;
deploy-csi-driver) return 0 ;;
deploy-kata) deploy_kata ;;
deploy-kata-aks) deploy_kata "aks" ;;
deploy-kata-kcli) deploy_kata "kcli" ;;
@@ -612,6 +613,7 @@ function main() {
cleanup-garm) cleanup "garm" ;;
cleanup-zvsi) cleanup "zvsi" ;;
cleanup-snapshotter) cleanup_snapshotter ;;
delete-csi-driver) return 0 ;;
delete-coco-kbs) delete_coco_kbs ;;
delete-cluster) cleanup "aks" ;;
delete-cluster-kcli) delete_cluster_kcli ;;

View File

@@ -95,7 +95,7 @@ setup() {
# The image pulled in the guest will be downloaded and unpacked in the `/run/kata-containers/image` directory.
# The tests will use `cryptsetup` to encrypt a block device and mount it at `/run/kata-containers/image`.
storage_config=$(mktemp "${BATS_FILE_TMPDIR}/$(basename "${storage_config_template}").XXXXXX.yaml")
storage_config=$(mktemp "${BATS_FILE_TMPDIR}/$(basename "${storage_config_template}").XXX")
local_device=$(create_loop_device)
LOCAL_DEVICE="$local_device" NODE_NAME="$node" envsubst < "$storage_config_template" > "$storage_config"
@@ -106,7 +106,7 @@ setup() {
# Create persistent volume and persistent volume claim
retry_kubectl_apply $storage_config
pod_config=$(mktemp "${BATS_FILE_TMPDIR}/$(basename "${pod_config_template}").XXXXXX.yaml")
pod_config=$(mktemp "${BATS_FILE_TMPDIR}/$(basename "${pod_config_template}").XXX")
IMAGE="$image_pulled_time_less_than_default_time" NODE_NAME="$node" envsubst < "$pod_config_template" > "$pod_config"
# Set CreateContainerRequest timeout in the annotation to allow for enough time for guest-pull where
@@ -140,7 +140,7 @@ setup() {
}
@test "Test we cannot pull a large image that pull time exceeds createcontainer timeout inside the guest" {
storage_config=$(mktemp "${BATS_FILE_TMPDIR}/$(basename "${storage_config_template}").XXXXXX.yaml")
storage_config=$(mktemp "${BATS_FILE_TMPDIR}/$(basename "${storage_config_template}").XXX")
local_device=$(create_loop_device)
LOCAL_DEVICE="$local_device" NODE_NAME="$node" envsubst < "$storage_config_template" > "$storage_config"
@@ -151,7 +151,7 @@ setup() {
# Create persistent volume and persistent volume claim
retry_kubectl_apply $storage_config
pod_config=$(mktemp "${BATS_FILE_TMPDIR}/$(basename "${pod_config_template}").XXXXXX.yaml")
pod_config=$(mktemp "${BATS_FILE_TMPDIR}/$(basename "${pod_config_template}").XXX")
IMAGE="$large_image" NODE_NAME="$node" envsubst < "$pod_config_template" > "$pod_config"
# Set a short CreateContainerRequest timeout in the annotation to fail to pull image in guest
@@ -191,7 +191,7 @@ setup() {
if [[ "${KATA_HYPERVISOR}" == qemu-coco-dev* ]] && [ "${KBS_INGRESS}" = "aks" ]; then
skip "skip this specific one due to issue https://github.com/kata-containers/kata-containers/issues/10299"
fi
storage_config=$(mktemp "${BATS_FILE_TMPDIR}/$(basename "${storage_config_template}").XXXXXX.yaml")
storage_config=$(mktemp "${BATS_FILE_TMPDIR}/$(basename "${storage_config_template}").XXX")
local_device=$(create_loop_device)
LOCAL_DEVICE="$local_device" NODE_NAME="$node" envsubst < "$storage_config_template" > "$storage_config"
@@ -202,7 +202,7 @@ setup() {
# Create persistent volume and persistent volume claim
retry_kubectl_apply $storage_config
pod_config=$(mktemp "${BATS_FILE_TMPDIR}/$(basename "${pod_config_template}").XXXXXX.yaml")
pod_config=$(mktemp "${BATS_FILE_TMPDIR}/$(basename "${pod_config_template}").XXX")
IMAGE="$large_image" NODE_NAME="$node" envsubst < "$pod_config_template" > "$pod_config"
# Set CreateContainerRequest timeout in the annotation to pull large image in guest

View File

@@ -54,8 +54,27 @@ NGC_API_KEY_BASE64=$(
)
export NGC_API_KEY_BASE64
# pre-created signed sealed secrets for TEE pods (from confidential_common.sh)
NGC_API_KEY_SEALED_SECRET_INSTRUCT="${SEALED_SECRET_PRECREATED_NIM_INSTRUCT}"
# Sealed secret format for TEE pods (vault type pointing to KBS resource)
# Format: sealed.<base64url JWS header>.<base64url payload>.<base64url signature>
# IMPORTANT: JWS uses base64url encoding WITHOUT padding (no trailing '=')
# We use tr to convert standard base64 (+/) to base64url (-_) and remove padding (=)
# For vault type, header and signature can be placeholders since the payload
# contains the KBS resource path where the actual secret is stored.
#
# Vault type sealed secret payload for instruct pod:
# {
# "version": "0.1.0",
# "type": "vault",
# "name": "kbs:///default/ngc-api-key/instruct",
# "provider": "kbs",
# "provider_settings": {},
# "annotations": {}
# }
NGC_API_KEY_SEALED_SECRET_INSTRUCT_PAYLOAD=$(
echo -n '{"version":"0.1.0","type":"vault","name":"kbs:///default/ngc-api-key/instruct","provider":"kbs","provider_settings":{},"annotations":{}}' |
base64 -w0 | tr '+/' '-_' | tr -d '='
)
NGC_API_KEY_SEALED_SECRET_INSTRUCT="sealed.fakejwsheader.${NGC_API_KEY_SEALED_SECRET_INSTRUCT_PAYLOAD}.fakesignature"
export NGC_API_KEY_SEALED_SECRET_INSTRUCT
# Base64 encode the sealed secret for use in Kubernetes Secret data field
@@ -63,7 +82,20 @@ export NGC_API_KEY_SEALED_SECRET_INSTRUCT
NGC_API_KEY_SEALED_SECRET_INSTRUCT_BASE64=$(echo -n "${NGC_API_KEY_SEALED_SECRET_INSTRUCT}" | base64 -w0)
export NGC_API_KEY_SEALED_SECRET_INSTRUCT_BASE64
NGC_API_KEY_SEALED_SECRET_EMBEDQA="${SEALED_SECRET_PRECREATED_NIM_EMBEDQA}"
# Vault type sealed secret payload for embedqa pod:
# {
# "version": "0.1.0",
# "type": "vault",
# "name": "kbs:///default/ngc-api-key/embedqa",
# "provider": "kbs",
# "provider_settings": {},
# "annotations": {}
# }
NGC_API_KEY_SEALED_SECRET_EMBEDQA_PAYLOAD=$(
echo -n '{"version":"0.1.0","type":"vault","name":"kbs:///default/ngc-api-key/embedqa","provider":"kbs","provider_settings":{},"annotations":{}}' |
base64 -w0 | tr '+/' '-_' | tr -d '='
)
NGC_API_KEY_SEALED_SECRET_EMBEDQA="sealed.fakejwsheader.${NGC_API_KEY_SEALED_SECRET_EMBEDQA_PAYLOAD}.fakesignature"
export NGC_API_KEY_SEALED_SECRET_EMBEDQA
NGC_API_KEY_SEALED_SECRET_EMBEDQA_BASE64=$(echo -n "${NGC_API_KEY_SEALED_SECRET_EMBEDQA}" | base64 -w0)
@@ -81,6 +113,27 @@ setup_langchain_flow() {
[[ "$(pip show beautifulsoup4 2>/dev/null | awk '/^Version:/{print $2}')" = "4.13.4" ]] || pip install beautifulsoup4==4.13.4
}
# Create Docker config for genpolicy so it can authenticate to nvcr.io when
# pulling image manifests (avoids "UnauthorizedError" from genpolicy's registry pull).
# Genpolicy (src/tools/genpolicy) uses docker_credential::get_credential() in
# src/tools/genpolicy/src/registry.rs build_auth(). The docker_credential crate
# reads config from DOCKER_CONFIG (directory) + "/config.json", so we set
# DOCKER_CONFIG to a directory containing config.json with nvcr.io auth.
setup_genpolicy_registry_auth() {
if [[ -z "${NGC_API_KEY:-}" ]]; then
return
fi
local auth_dir
auth_dir="${BATS_SUITE_TMPDIR}/.docker-genpolicy"
mkdir -p "${auth_dir}"
# Docker config format: auths -> registry -> auth (base64 of "user:password")
echo -n "{\"auths\":{\"nvcr.io\":{\"username\":\"\$oauthtoken\",\"password\":\"${NGC_API_KEY}\",\"auth\":\"$(echo -n "\$oauthtoken:${NGC_API_KEY}" | base64 -w0)\"}}}" \
> "${auth_dir}/config.json"
export DOCKER_CONFIG="${auth_dir}"
# REGISTRY_AUTH_FILE (containers-auth.json format) is the same structure for auths
export REGISTRY_AUTH_FILE="${auth_dir}/config.json"
}
# Create initdata TOML file for genpolicy with CDH configuration.
# This file is used by genpolicy via --initdata-path. Genpolicy will add the
# generated policy.rego to it and set it as the cc_init_data annotation.
@@ -190,9 +243,10 @@ setup_file() {
add_requests_to_policy_settings "${policy_settings_dir}" "ReadStreamRequest"
if [ "${TEE}" = "true" ]; then
# So genpolicy can pull nvcr.io image manifests when generating policy (avoids UnauthorizedError).
setup_genpolicy_registry_auth
setup_kbs_credentials
# provision signing public key to KBS so that CDH can verify pre-created, signed secret.
setup_sealed_secret_signing_public_key
# Overwrite the empty default-initdata.toml with our CDH configuration.
# This must happen AFTER create_tmp_policy_settings_dir() copies the empty
# file and BEFORE auto_generate_policy() runs.

View File

@@ -48,13 +48,25 @@ setup() {
"${kernel_params_annotation}" \
"${kernel_params_value}"
# provision signing public key to KBS so that CDH can verify pre-created, signed secret.
setup_sealed_secret_signing_public_key
# Setup k8s secret
kubectl delete secret sealed-secret --ignore-not-found
kubectl delete secret not-sealed-secret --ignore-not-found
kubectl create secret generic sealed-secret --from-literal="secret=${SEALED_SECRET_PRECREATED_TEST}"
# Sealed secret format is defined at: https://github.com/confidential-containers/guest-components/blob/main/confidential-data-hub/docs/SEALED_SECRET.md#vault
# sealed.BASE64URL(UTF8(JWS Protected Header)) || '.
# || BASE64URL(JWS Payload) || '.'
# || BASE64URL(JWS Signature)
# test payload:
# {
# "version": "0.1.0",
# "type": "vault",
# "name": "kbs:///default/sealed-secret/test",
# "provider": "kbs",
# "provider_settings": {},
# "annotations": {}
# }
kubectl create secret generic sealed-secret --from-literal='secret=sealed.fakejwsheader.eyJ2ZXJzaW9uIjoiMC4xLjAiLCJ0eXBlIjoidmF1bHQiLCJuYW1lIjoia2JzOi8vL2RlZmF1bHQvc2VhbGVkLXNlY3JldC90ZXN0IiwicHJvdmlkZXIiOiJrYnMiLCJwcm92aWRlcl9zZXR0aW5ncyI6e30sImFubm90YXRpb25zIjp7fX0.fakesignature'
kubectl create secret generic not-sealed-secret --from-literal='secret=not_sealed_secret'
if ! is_confidential_hardware; then
@@ -67,10 +79,10 @@ setup() {
@test "Cannot Unseal Env Secrets with CDH without key" {
k8s_create_pod "${K8S_TEST_ENV_YAML}"
logs=$(kubectl logs secret-test-pod-cc)
echo "$logs"
grep -q "UNPROTECTED_SECRET = not_sealed_secret" <<< "$logs"
run grep -q "PROTECTED_SECRET = unsealed_secret" <<< "$logs"
kubectl logs secret-test-pod-cc
kubectl logs secret-test-pod-cc | grep -q "UNPROTECTED_SECRET = not_sealed_secret"
cmd="kubectl logs secret-test-pod-cc | grep -q \"PROTECTED_SECRET = unsealed_secret\""
run $cmd
[ "$status" -eq 1 ]
}
@@ -79,20 +91,18 @@ setup() {
kbs_set_resource "default" "sealed-secret" "test" "unsealed_secret"
k8s_create_pod "${K8S_TEST_ENV_YAML}"
logs=$(kubectl logs secret-test-pod-cc)
echo "$logs"
grep -q "UNPROTECTED_SECRET = not_sealed_secret" <<< "$logs"
grep -q "PROTECTED_SECRET = unsealed_secret" <<< "$logs"
kubectl logs secret-test-pod-cc
kubectl logs secret-test-pod-cc | grep -q "UNPROTECTED_SECRET = not_sealed_secret"
kubectl logs secret-test-pod-cc | grep -q "PROTECTED_SECRET = unsealed_secret"
}
@test "Unseal File Secrets with CDH" {
kbs_set_resource "default" "sealed-secret" "test" "unsealed_secret"
k8s_create_pod "${K8S_TEST_FILE_YAML}"
logs=$(kubectl logs secret-test-pod-cc)
echo "$logs"
grep -q "UNPROTECTED_SECRET = not_sealed_secret" <<< "$logs"
grep -q "PROTECTED_SECRET = unsealed_secret" <<< "$logs"
kubectl logs secret-test-pod-cc
kubectl logs secret-test-pod-cc | grep -q "UNPROTECTED_SECRET = not_sealed_secret"
kubectl logs secret-test-pod-cc | grep -q "PROTECTED_SECRET = unsealed_secret"
}
teardown() {

View File

@@ -18,10 +18,8 @@ setup() {
pod_name="trusted-ephemeral-data-storage"
mountpoint="/mnt/temp-encrypted"
host_df="$(exec_host "${node}" df -PT -B1 "$(get_kubelet_data_dir)" | tail -n +2)"
info "host_df output:"
info "${host_df}"
host_cap_bytes="$(echo "${host_df}" | awk '{print $3}')"
host_df="$(exec_host "${node}" df -P -B1 "$(get_kubelet_data_dir)" | tail -n +2)"
host_capacity_bytes="$(echo "${host_df}" | awk '{print $3}')"
yaml_file="${pod_config_dir}/pod-trusted-ephemeral-data-storage.yaml"
policy_settings_dir="$(create_tmp_policy_settings_dir "${pod_config_dir}")"
@@ -37,29 +35,28 @@ setup() {
kubectl wait --for=condition=Ready --timeout="${timeout}" pod "${pod_name}"
# With long device names, df adds line breaks by default, so we pass -P to prevent that.
emptydir_df="$(kubectl exec "${pod_name}" -- df -PT -B1 "${mountpoint}" | tail -n +2)"
info "emptydir_df output:"
info "${emptydir_df}"
df="$(kubectl exec "${pod_name}" -- df -PT -B1 "${mountpoint}" | tail -n +2)"
info "df output:"
info "${df}"
dm_device="$(echo "${emptydir_df}" | awk '{print $1}')"
fs_type="$(echo "${emptydir_df}" | awk '{print $2}')"
emptydir_cap_bytes="$(echo "${emptydir_df}" | awk '{print $3}')"
emptydir_avail_bytes="$(echo "${emptydir_df}" | awk '{print $5}')"
dm_device="$(echo "${df}" | awk '{print $1}')"
fs_type="$(echo "${df}" | awk '{print $2}')"
guest_capacity_bytes="$(echo "${df}" | awk '{print $3}')"
guest_available_bytes="$(echo "${df}" | awk '{print $5}')"
# The output of the cryptsetup command will contain something like this:
#
# /dev/mapper/741ed4bf-3073-49ed-9b7a-d6fa7cce0db1 is active and is in use.
# type: n/a
# cipher: aes-xts-plain
# /dev/mapper/encrypted_disk_N6PxO is active and is in use.
# type: LUKS2
# cipher: aes-xts-plain64
# keysize: 768 bits
# key location: keyring
# integrity: hmac(sha256)
# integrity keysize: 256 bits
# integrity tag size: 32 bytes
# device: /dev/sdd
# device: /dev/vda
# sector size: 4096
# offset: 0 sectors
# size: 300052568 sectors
# size: 2031880 sectors
# mode: read/write
crypt_status="$(kubectl exec "${pod_name}" -- cryptsetup status "${dm_device}")"
info "cryptsetup status output:"
@@ -68,15 +65,16 @@ setup() {
# Check filesystem type and capacity.
[[ "${fs_type}" == "ext4" ]]
# Allow up to 4% metadata overhead.
(( emptydir_cap_bytes >= host_cap_bytes * 96 / 100 ))
# Allow up to 10% metadata overhead.
(( emptydir_avail_bytes >= host_cap_bytes * 90 / 100 ))
# Allow up to 7% LUKS metadata overhead.
(( guest_capacity_bytes >= host_capacity_bytes * 93 / 100 ))
# Allow up to 15% LUKS + ext4 metadata overhead.
(( guest_available_bytes >= host_capacity_bytes * 85 / 100 ))
# Check encryption settings.
grep -q "${dm_device} is active and is in use" <<< "${crypt_status}"
grep -Eq "type: +n/a" <<< "${crypt_status}" # The LUKS header is detached.
grep -Eq "cipher: +aes-xts-plain" <<< "${crypt_status}"
grep -Eq "type: +LUKS2" <<< "${crypt_status}"
grep -Eq "cipher: +aes-xts-plain64" <<< "${crypt_status}"
grep -Eq "integrity: +hmac\(sha256\)" <<< "${crypt_status}"
# Check I/O.

View File

@@ -51,27 +51,6 @@ kernel_params = "${new_params}"
EOF
}
# Create Docker config for genpolicy so it can authenticate to nvcr.io when
# pulling image manifests (avoids "UnauthorizedError" from genpolicy's registry pull).
# Genpolicy (src/tools/genpolicy) uses docker_credential::get_credential() in
# src/tools/genpolicy/src/registry.rs build_auth(). The docker_credential crate
# reads config from DOCKER_CONFIG (directory) + "/config.json", so we set
# DOCKER_CONFIG to a directory containing config.json with nvcr.io auth.
setup_genpolicy_registry_auth() {
if [[ -z "${NGC_API_KEY:-}" ]]; then
return
fi
local auth_dir
auth_dir="${kubernetes_dir}/.docker-genpolicy"
mkdir -p "${auth_dir}"
# Docker config format: auths -> registry -> auth (base64 of "user:password")
echo -n "{\"auths\":{\"nvcr.io\":{\"username\":\"\$oauthtoken\",\"password\":\"${NGC_API_KEY}\",\"auth\":\"$(echo -n "\$oauthtoken:${NGC_API_KEY}" | base64 -w0)\"}}}" \
> "${auth_dir}/config.json"
export DOCKER_CONFIG="${auth_dir}"
# REGISTRY_AUTH_FILE (containers-auth.json format) is the same structure for auths
export REGISTRY_AUTH_FILE="${auth_dir}/config.json"
}
cleanup() {
true
}
@@ -105,9 +84,6 @@ if [[ "${ENABLE_NVRC_TRACE:-true}" == "true" ]]; then
enable_nvrc_trace
fi
# So genpolicy can pull nvcr.io image manifests when generating policy (avoids UnauthorizedError).
setup_genpolicy_registry_auth
# Use common bats test runner with proper reporting
export BATS_TEST_FAIL_FAST="${K8S_TEST_FAIL_FAST}"
run_bats_tests "${kubernetes_dir}" K8S_TEST_NV

View File

@@ -1,7 +1,7 @@
module example.com/m
// Keep in sync with version in versions.yaml
go 1.25.8
go 1.25.7
require (
github.com/BurntSushi/toml v1.3.2

View File

@@ -103,7 +103,6 @@ long_options=(
[no-arch]="Run/list all tests except architecture-specific ones"
[only-arch]="Only run/list architecture-specific tests"
[repo:]="Specify GitHub URL of repo to use (github.com/user/repo)"
[repo-path:]="Specify path to repository to check (default: \$GOPATH/src/\$repo)"
[scripts]="Check script files"
[vendor]="Check vendor files"
[versions]="Check versions files"
@@ -597,6 +596,19 @@ check_url()
local curl_ua_args
[ -n "$user_agent" ] && curl_ua_args="-A '$user_agent'"
{ run_url_check_cmd "$url" "$curl_out" "$curl_ua_args"; ret=$?; } || true
# A transitory error, or the URL is incorrect,
# but capture either way.
if [ "$ret" -ne 0 ]; then
errors+=("Failed to check URL '$url' (user agent: '$user_agent', return code $ret)")
# Try again with another UA since it appears that some return codes
# indicate the server was unhappy with the details
# presented by the client.
continue
fi
local http_statuses
http_statuses=$(grep -E "^HTTP" "$curl_out" |\
@@ -786,13 +798,111 @@ static_check_docs()
# Convert the list of files into an grep(1) alternation pattern.
exclude_pattern=$(echo "${exclude_doc_regexs[@]}"|sed 's, ,|,g')
# Every document in the repo (except a small handful of exceptions)
# should be referenced by another document.
for doc in $md_docs_to_check
do
# Check the ignore list for markdown files that do not need to
# be referenced by others.
echo "$doc"|grep -q -E "(${exclude_pattern})" && continue
grep -q "$doc" "$md_links" || die "Document $doc is not referenced"
done
info "Checking document code blocks"
local doc_to_script_cmd="${cidir}/kata-doc-to-script.sh"
for doc in $docs
do
bash "${doc_to_script_cmd}" -csv "$doc"
# Look for URLs in the document
urls=$("${doc_to_script_cmd}" -i "$doc" - | "$cmd")
# Gather URLs
for url in $urls
do
printf "%s\t%s\n" "${url}" "${doc}" >> "$url_map"
done
done
# Get unique list of URLs
urls=$(awk '{print $1}' "$url_map" | sort -u)
info "Checking all document URLs"
local invalid_urls_dir=$(mktemp -d)
files_to_remove+=("${invalid_urls_dir}")
for url in $urls
do
if [ "$specific_branch" != "true" ]
then
# If the URL is new on this PR, it cannot be checked.
echo "$new_urls" | grep -q -E "\<${url}\>" && \
info "ignoring new (but correct) URL: $url" && continue
fi
# Ignore local URLs. The only time these are used is in
# examples (meaning these URLs won't exist).
echo "$url" | grep -q "^file://" && continue
echo "$url" | grep -q "^http://localhost" && continue
# Ignore the install guide URLs that contain a shell variable
echo "$url" | grep -q "\\$" && continue
# This prefix requires the client to be logged in to github, so ignore
echo "$url" | grep -q 'https://github.com/pulls' && continue
# Sigh.
echo "$url"|grep -q 'https://example.com' && continue
# Google APIs typically require an auth token.
echo "$url"|grep -q 'https://www.googleapis.com' && continue
# Git repo URL check
if echo "$url"|grep -q '^https.*git'
then
timeout "${KATA_NET_TIMEOUT}" git ls-remote "$url" > /dev/null 2>&1 && continue
fi
# Check the URL, saving it if invalid
#
# Each URL is checked in a separate process as each unique URL
# requires us to hit the network.
check_url "$url" "$invalid_urls_dir" &
done
# Synchronisation point
wait
# Combine all the separate invalid URL files into one
local invalid_files=$(ls "$invalid_urls_dir")
if [ -n "$invalid_files" ]; then
pushd "$invalid_urls_dir" &>/dev/null
cat $(echo "$invalid_files"|tr '\n' ' ') > "$invalid_urls"
popd &>/dev/null
fi
if [ -s "$invalid_urls" ]
then
local files
cat "$invalid_urls" | while read url
do
files=$(grep "^${url}" "$url_map" | awk '{print $2}' | sort -u)
echo >&2 -e "ERROR: Invalid URL '$url' found in the following files:\n"
for file in $files
do
echo >&2 "$file"
done
done
exit 1
fi
# Now, spell check the docs
cmd="${test_dir}/cmd/check-spelling/kata-spell-check.sh"
@@ -1406,8 +1516,6 @@ main()
local func=
repo_path=""
while [ $# -gt 1 ]
do
case "$1" in
@@ -1428,7 +1536,6 @@ main()
--only-arch) handle_funcs="arch-specific" ;;
--rego) func=static_check_rego ;;
--repo) repo="$2"; shift ;;
--repo-path) repo_path="$2"; shift ;;
--scripts) func=static_check_shell ;;
--vendor) func=static_check_vendor;;
--versions) func=static_check_versions ;;
@@ -1461,10 +1568,7 @@ main()
test_path="${test_path:-"${repo}/tests"}"
test_dir="${GOPATH}/src/${test_path}"
if [ -z "$repo_path" ]
then
repo_path=$GOPATH/src/$repo
fi
repo_path=$GOPATH/src/$repo
announce

View File

@@ -24,7 +24,6 @@ cuda_repo_url="${4:?cuda_repo_url not specified}"
cuda_repo_pkg="${5:?cuda_repo_pkg not specified}"
tools_repo_url="${6:?tools_repo_url not specified}"
tools_repo_pkg="${7:?tools_repo_pkg not specified}"
ctk_version="${8:?ctk_version not specified}"
APT_INSTALL="apt -o Dpkg::Options::='--force-confdef' -o Dpkg::Options::='--force-confold' -yqq --no-install-recommends install"
export DEBIAN_FRONTEND=noninteractive
@@ -37,7 +36,7 @@ is_feature_enabled() {
install_nvidia_ctk() {
echo "chroot: Installing NVIDIA GPU container runtime"
# Base gives a nvidia-ctk and the nvidia-container-runtime
eval "${APT_INSTALL}" nvidia-container-toolkit-base="${ctk_version}"
eval "${APT_INSTALL}" nvidia-container-toolkit-base=1.17.6-1
}
install_nvidia_fabricmanager() {
@@ -61,19 +60,12 @@ install_userspace_components() {
eval "${APT_INSTALL}" nvidia-imex nvidia-firmware \
libnvidia-cfg1 libnvidia-gl libnvidia-extra \
libnvidia-decode libnvidia-fbc1 libnvidia-encode \
libnvidia-nscq libnvidia-compute nvidia-settings
libnvidia-nscq
apt-mark hold nvidia-imex nvidia-firmware \
libnvidia-cfg1 libnvidia-gl libnvidia-extra \
libnvidia-decode libnvidia-fbc1 libnvidia-encode \
libnvidia-nscq libnvidia-compute nvidia-settings
# Needed for confidential-data-hub runtime dependencies
eval "${APT_INSTALL}" cryptsetup-bin dmsetup \
libargon2-1 e2fsprogs
apt-mark hold cryptsetup-bin dmsetup libargon2-1 \
e2fsprogs
libnvidia-nscq
}
setup_apt_repositories() {

View File

@@ -97,7 +97,7 @@ setup_nvidia_gpu_rootfs_stage_one() {
mount --make-rslave ./dev
mount -t proc /proc ./proc
local cuda_repo_url cuda_repo_pkg gpu_base_os_version ctk_version
local cuda_repo_url cuda_repo_pkg gpu_base_os_version
cuda_repo_url=$(get_package_version_from_kata_yaml "externals.nvidia.cuda.repo.${machine_arch}.url")
cuda_repo_pkg=$(get_package_version_from_kata_yaml "externals.nvidia.cuda.repo.${machine_arch}.pkg")
gpu_base_os_version=$(get_package_version_from_kata_yaml "assets.image.architecture.x86_64.nvidia-gpu.version")
@@ -105,10 +105,8 @@ setup_nvidia_gpu_rootfs_stage_one() {
tools_repo_url=$(get_package_version_from_kata_yaml "externals.nvidia.tools.repo.${machine_arch}.url")
tools_repo_pkg=$(get_package_version_from_kata_yaml "externals.nvidia.tools.repo.${machine_arch}.pkg")
ctk_version=$(get_package_version_from_kata_yaml "externals.nvidia.ctk.version")
chroot . /bin/bash -c "/nvidia_chroot.sh ${machine_arch} ${NVIDIA_GPU_STACK} \
${gpu_base_os_version} ${cuda_repo_url} ${cuda_repo_pkg} ${tools_repo_url} ${tools_repo_pkg} ${ctk_version}"
${gpu_base_os_version} ${cuda_repo_url} ${cuda_repo_pkg} ${tools_repo_url} ${tools_repo_pkg}"
umount -R ./dev
umount ./proc
@@ -151,8 +149,14 @@ chisseled_nvswitch() {
cp -a "${stage_one}"/usr/share/nvidia/nvswitch usr/share/nvidia/.
libdir=usr/lib/"${machine_arch}"-linux-gnu
cp -a "${stage_one}/${libdir}"/libnvidia-nscq.so.* lib/"${machine_arch}"-linux-gnu/.
# Logs will be redirected to console(stderr)
# if the specified log file can't be opened or the path is empty.
# LOG_FILE_NAME=/var/log/fabricmanager.log -> setting to empty for stderr -> kmsg
sed -i 's|^LOG_FILE_NAME=.*|LOG_FILE_NAME=|' usr/share/nvidia/nvswitch/fabricmanager.cfg
# NVLINK SubnetManager dependencies
local nvlsm=usr/share/nvidia/nvlsm
mkdir -p "${nvlsm}"
@@ -160,8 +164,6 @@ chisseled_nvswitch() {
cp -a "${stage_one}"/opt/nvidia/nvlsm/lib/libgrpc_mgr.so lib/.
cp -a "${stage_one}"/opt/nvidia/nvlsm/sbin/nvlsm sbin/.
cp -a "${stage_one}/${nvlsm}"/*.conf "${nvlsm}"/.
# Redirect all the logs to syslog instead of logging to file
sed -i 's|^LOG_USE_SYSLOG=.*|LOG_USE_SYSLOG=1|' usr/share/nvidia/nvswitch/fabricmanager.cfg
}
chisseled_dcgm() {
@@ -207,8 +209,9 @@ chisseled_compute() {
cp -aL "${stage_one}/${libdir}"/ld-linux-* "${libdir}"/.
libdir=usr/lib/"${machine_arch}"-linux-gnu
cp -a "${stage_one}/${libdir}"/libnv* lib/"${machine_arch}"-linux-gnu/.
cp -a "${stage_one}/${libdir}"/libnvidia-ml.so.* lib/"${machine_arch}"-linux-gnu/.
cp -a "${stage_one}/${libdir}"/libcuda.so.* lib/"${machine_arch}"-linux-gnu/.
cp -a "${stage_one}/${libdir}"/libnvidia-cfg.so.* lib/"${machine_arch}"-linux-gnu/.
# basic GPU admin tools
cp -a "${stage_one}"/usr/bin/nvidia-persistenced bin/.
@@ -240,8 +243,6 @@ chisseled_init() {
usr/bin etc/modprobe.d etc/ssl/certs
ln -sf ../run var/run
ln -sf ../run var/log
ln -sf ../run var/cache
# Needed for various RUST static builds with LIBC=gnu
libdir=lib/"${machine_arch}"-linux-gnu
@@ -308,44 +309,6 @@ compress_rootfs() {
chmod +x "${libdir}"/ld-linux-*
}
copy_cdh_runtime_deps() {
local libdir="lib/${machine_arch}-linux-gnu"
# Shared libraries required by /usr/local/bin/confidential-data-hub.
# Note: libcryptsetup loads some optional helpers (e.g. libpopt/libssh) only
# when specific features are used. The current CDH path (LUKS2 + mkfs.ext4)
# does not require those optional libs.
cp -a "${stage_one}/${libdir}"/libcryptsetup.so.12* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libuuid.so.1* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libdevmapper.so.1.02.1* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libselinux.so.1* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libpcre2-8.so.0* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libudev.so.1* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libcap.so.2* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libcrypto.so.3* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libz.so.1* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libzstd.so.1* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libjson-c.so.5* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libblkid.so.1* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libargon2.so.1* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libgcc_s.so.1* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libm.so.6* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libc.so.6* "${libdir}/."
# e2fsprogs (mkfs.ext4) runtime libs
cp -a "${stage_one}/${libdir}"/libext2fs.so.2* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libe2p.so.2* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libss.so.2* "${libdir}/."
cp -a "${stage_one}/${libdir}"/libcom_err.so.2* "${libdir}/."
# mkfs.ext4 and dd are used by CDH secure_mount
mkdir -p sbin etc usr/bin bin
cp -a "${stage_one}/sbin/mke2fs" sbin/.
cp -a "${stage_one}/sbin/mkfs.ext4" sbin/.
cp -a "${stage_one}/etc/mke2fs.conf" etc/.
cp -a "${stage_one}/usr/bin/dd" bin/.
}
coco_guest_components() {
if [[ "${type}" != "confidential" ]]; then
return
@@ -368,7 +331,7 @@ coco_guest_components() {
cp -a "${stage_one}/${pause_dir}"/config.json "${pause_dir}/."
cp -a "${stage_one}/${pause_dir}"/rootfs/pause "${pause_dir}/rootfs/."
copy_cdh_runtime_deps
info "TODO: nvidia: luks-encrypt-storage is a bash script, we do not have a shell!"
}
setup_nvidia_gpu_rootfs_stage_two() {

View File

@@ -15,13 +15,13 @@ type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: "3.28.0"
version: "3.27.0"
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "3.28.0"
appVersion: "3.27.0"
dependencies:
- name: node-feature-discovery

View File

@@ -96,9 +96,9 @@ scheduling:
"qemu-snp-runtime-rs" (dict "memory" "2048Mi" "cpu" "1.0")
"qemu-tdx" (dict "memory" "2048Mi" "cpu" "1.0")
"qemu-tdx-runtime-rs" (dict "memory" "2048Mi" "cpu" "1.0")
"qemu-nvidia-gpu" (dict "memory" "10240Mi" "cpu" "1.0")
"qemu-nvidia-gpu-snp" (dict "memory" "10240Mi" "cpu" "1.0")
"qemu-nvidia-gpu-tdx" (dict "memory" "10240Mi" "cpu" "1.0")
"qemu-nvidia-gpu" (dict "memory" "4096Mi" "cpu" "1.0")
"qemu-nvidia-gpu-snp" (dict "memory" "20480Mi" "cpu" "1.0")
"qemu-nvidia-gpu-tdx" (dict "memory" "20480Mi" "cpu" "1.0")
"qemu-cca" (dict "memory" "2048Mi" "cpu" "1.0")
"stratovirt" (dict "memory" "130Mi" "cpu" "250m")
"remote" (dict "memory" "120Mi" "cpu" "250m")
@@ -110,14 +110,9 @@ scheduling:
{{- $config := index $runtimeClassConfigs $shim }}
{{- $shimConfig := index $.Values.shims $shim }}
{{- if $config }}
{{- /* Allow per-shim overrides of pod overhead via .shims.<name>.runtimeClass.overhead */ -}}
{{- $effectiveConfig := deepCopy $config }}
{{- if and $shimConfig.runtimeClass $shimConfig.runtimeClass.overhead }}
{{- $effectiveConfig = mergeOverwrite $effectiveConfig $shimConfig.runtimeClass.overhead }}
{{- end }}
{{ include "kata-deploy.runtimeclass" (dict "root" $ "shim" $shim "config" $effectiveConfig "shimConfig" $shimConfig "nameOverride" "" "useShimNodeSelectors" $useShimNodeSelectors) }}
{{ include "kata-deploy.runtimeclass" (dict "root" $ "shim" $shim "config" $config "shimConfig" $shimConfig "nameOverride" "" "useShimNodeSelectors" $useShimNodeSelectors) }}
{{- if and $createDefaultRC (not $multiInstallSuffix) (eq $shim $defaultShim) }}
{{ include "kata-deploy.runtimeclass" (dict "root" $ "shim" $shim "config" $effectiveConfig "shimConfig" $shimConfig "nameOverride" $defaultRCName "useShimNodeSelectors" $useShimNodeSelectors) }}
{{ include "kata-deploy.runtimeclass" (dict "root" $ "shim" $shim "config" $config "shimConfig" $shimConfig "nameOverride" $defaultRCName "useShimNodeSelectors" $useShimNodeSelectors) }}
{{- end }}
{{- end }}
{{- end }}

View File

@@ -50,28 +50,6 @@ snapshotter:
# disableAll: true
# qemu:
# enabled: true # Only qemu is enabled
#
# All configurable keys per shim (all optional unless noted):
# shims:
# <shim-name>:
# enabled: ~ # true | false | ~ (null = follow disableAll)
# supportedArches: # list of supported architectures
# - amd64
# allowedHypervisorAnnotations: [] # hypervisor annotations to pass through
# containerd:
# snapshotter: "" # e.g. nydus, erofs, devmapper, or "" for default
# forceGuestPull: false # force guest-side image pull in containerd
# crio:
# guestPull: false # enable guest-pull in CRI-O
# agent:
# httpsProxy: "" # HTTPS proxy for the Kata agent
# noProxy: "" # no-proxy list for the Kata agent
# runtimeClass:
# nodeSelector: # extra node selectors added to the RuntimeClass
# example.io/feature: "true"
# overhead: # override pod overhead (falls back to built-in defaults)
# memory: "160Mi"
# cpu: "250m"
shims:
disableAll: false

View File

@@ -110,6 +110,9 @@ cloud-hypervisor-tarball:
cloud-hypervisor-glibc-tarball:
${MAKE} $@-build
csi-kata-directvolume-tarball: copy-scripts-for-the-tools-build
${MAKE} $@-build
firecracker-tarball:
${MAKE} $@-build
@@ -191,9 +194,15 @@ rootfs-initrd-tarball: agent-tarball
rootfs-image-nvidia-gpu-tarball: agent-tarball busybox-tarball kernel-nvidia-gpu-tarball
${MAKE} $@-build
rootfs-initrd-nvidia-gpu-tarball: agent-tarball busybox-tarball kernel-nvidia-gpu-tarball
${MAKE} $@-build
rootfs-image-nvidia-gpu-confidential-tarball: agent-tarball busybox-tarball pause-image-tarball coco-guest-components-tarball kernel-nvidia-gpu-tarball
${MAKE} $@-build
rootfs-initrd-nvidia-gpu-confidential-tarball: agent-tarball busybox-tarball pause-image-tarball coco-guest-components-tarball kernel-nvidia-gpu-tarball
${MAKE} $@-build
rootfs-cca-confidential-image-tarball: agent-tarball pause-image-tarball coco-guest-components-tarball kernel-cca-confidential-tarball
${MAKE} $@-build

View File

@@ -108,6 +108,7 @@ options:
coco-guest-components
cloud-hypervisor
cloud-hypervisor-glibc
csi-kata-directvolume
firecracker
genpolicy
kata-ctl
@@ -360,10 +361,6 @@ get_latest_kernel_nvidia_artefact_and_builder_image_version() {
echo "${latest_kernel_artefact}-${latest_kernel_builder_image}"
}
get_latest_ctk_version() {
echo $(get_from_kata_deps ".externals.nvidia.ctk.version")
}
#Install guest image
install_image() {
local variant="${1:-}"
@@ -396,7 +393,6 @@ install_image() {
# measured boot is used
if [[ "${variant}" == "nvidia-gpu-confidential" ]]; then
latest_artefact+="-$(get_latest_kernel_nvidia_artefact_and_builder_image_version)"
latest_artefact+="-$(get_latest_ctk_version)"
else
latest_artefact+="-$(get_latest_kernel_artefact_and_builder_image_version)"
fi
@@ -408,7 +404,6 @@ install_image() {
if [[ "${variant}" == "nvidia-gpu" ]]; then
# If we bump the kernel we need to rebuild the image
latest_artefact+="-$(get_latest_kernel_nvidia_artefact_and_builder_image_version)"
latest_artefact+="-$(get_latest_ctk_version)"
fi
latest_builder_image=""
@@ -504,7 +499,6 @@ install_initrd() {
# measured boot is used
if [[ "${variant}" == "nvidia-gpu-confidential" ]]; then
latest_artefact+="-$(get_latest_kernel_nvidia_artefact_and_builder_image_version)"
latest_artefact+="-$(get_latest_ctk_version)"
else
latest_artefact+="-$(get_latest_kernel_artefact_and_builder_image_version)"
fi
@@ -515,7 +509,6 @@ install_initrd() {
if [[ "${variant}" == "nvidia-gpu" ]]; then
# If we bump the kernel we need to rebuild the initrd as well
latest_artefact+="-$(get_latest_kernel_nvidia_artefact_and_builder_image_version)"
latest_artefact+="-$(get_latest_ctk_version)"
fi
latest_builder_image=""
@@ -607,6 +600,16 @@ install_image_nvidia_gpu() {
install_image "nvidia-gpu"
}
# Install NVIDIA GPU initrd
install_initrd_nvidia_gpu() {
export AGENT_POLICY
export MEASURED_ROOTFS="no"
local version=$(get_from_kata_deps .externals.nvidia.driver.version)
EXTRA_PKGS="apt curl ${EXTRA_PKGS}"
NVIDIA_GPU_STACK=${NVIDIA_GPU_STACK:-"driver=${version},compute,dcgm,nvswitch"}
install_initrd "nvidia-gpu"
}
# Instal NVIDIA GPU confidential image
install_image_nvidia_gpu_confidential() {
export CONFIDENTIAL_GUEST="yes"
@@ -618,6 +621,18 @@ install_image_nvidia_gpu_confidential() {
install_image "nvidia-gpu-confidential"
}
# Install NVIDIA GPU confidential initrd
install_initrd_nvidia_gpu_confidential() {
export CONFIDENTIAL_GUEST="yes"
export AGENT_POLICY
export MEASURED_ROOTFS="no"
local version=$(get_from_kata_deps .externals.nvidia.driver.version)
EXTRA_PKGS="apt curl ${EXTRA_PKGS}"
NVIDIA_GPU_STACK=${NVIDIA_GPU_STACK:-"driver=${version},compute,dcgm,nvswitch"}
install_initrd "nvidia-gpu-confidential"
}
install_se_image() {
info "Create IBM SE image configured with AA_KBC=${AA_KBC}"
"${se_image_builder}" --destdir="${destdir}"
@@ -1182,15 +1197,13 @@ install_tools_helper() {
tool_binary=${tool}
[ ${tool} = "agent-ctl" ] && tool_binary="kata-agent-ctl"
[ ${tool} = "csi-kata-directvolume" ] && tool_binary="directvolplugin"
[ ${tool} = "trace-forwarder" ] && tool_binary="kata-trace-forwarder"
local tool_build_dir="src/tools/${tool}"
[ ${tool} = "genpolicy" ] && tool_build_dir=target
binary=$(find "${repo_root_dir}/${tool_build_dir}" -type f -name "${tool_binary}")
binary=$(find ${repo_root_dir}/src/tools/${tool}/ -type f -name ${tool_binary})
binary_count=$(echo "${binary}" | grep -c '^' || echo "0")
if [[ "${binary}" = "" ]]; then
die "No binary found for ${tool} in ${repo_root_dir}/${tool_build_dir} (expected: ${tool_binary})."
if [[ "${binary_count}" -eq 0 ]]; then
die "No binary found for ${tool} (expected: ${tool_binary})."
elif [[ "${binary_count}" -gt 1 ]]; then
die "Multiple binaries found for ${tool} (expected single ${tool_binary}). Found:"$'\n'"${binary}"
fi
@@ -1224,7 +1237,8 @@ install_tools_helper() {
info "Install static ${tool_binary}"
mkdir -p "${destdir}/opt/kata/bin/"
install -D --mode "${binary_permissions}" "${binary}" "${destdir}/opt/kata/bin/${tool_binary}"
[ ${tool} = "csi-kata-directvolume" ] && tool_binary="csi-kata-directvolume"
install -D --mode ${binary_permissions} ${binary} "${destdir}/opt/kata/bin/${tool_binary}"
}
install_agent_ctl() {
@@ -1235,6 +1249,10 @@ install_genpolicy() {
install_tools_helper "genpolicy"
}
install_csi_kata_directvolume() {
install_tools_helper "csi-kata-directvolume"
}
install_kata_ctl() {
install_tools_helper "kata-ctl"
}
@@ -1309,6 +1327,8 @@ handle_build() {
cloud-hypervisor-glibc) install_clh_glibc ;;
csi-kata-directvolume) install_csi_kata_directvolume ;;
firecracker) install_firecracker ;;
genpolicy) install_genpolicy ;;
@@ -1361,8 +1381,12 @@ handle_build() {
rootfs-image-nvidia-gpu) install_image_nvidia_gpu ;;
rootfs-initrd-nvidia-gpu) install_initrd_nvidia_gpu ;;
rootfs-image-nvidia-gpu-confidential) install_image_nvidia_gpu_confidential ;;
rootfs-initrd-nvidia-gpu-confidential) install_initrd_nvidia_gpu_confidential ;;
rootfs-cca-confidential-image) install_image_confidential ;;
rootfs-cca-confidential-initrd) install_initrd_confidential ;;
@@ -1521,6 +1545,7 @@ main() {
agent-ctl
cloud-hypervisor
coco-guest-components
csi-kata-directvolume
firecracker
genpolicy
kata-ctl

View File

@@ -611,7 +611,6 @@ install_kata() {
fi
install --mode 0644 -D ./.config "${install_path}/config-${kernel_version}-${config_version}${suffix}"
install --mode 0644 -D ./System.map "${install_path}/System.map-${kernel_version}-${config_version}${suffix}"
ln -sf "${vmlinuz}" "${install_path}/vmlinuz${suffix}.container"
ln -sf "${vmlinux}" "${install_path}/vmlinux${suffix}.container"

View File

@@ -1,3 +1,4 @@
# mmio devices are required for firecracker (and the unified x86_64/s390x kernel)
# !confidential
# mmio devices are required for firecracker
CONFIG_VIRTIO_MMIO=y
CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y

View File

@@ -1 +1 @@
186
184

View File

@@ -25,7 +25,6 @@ RUN apt-get update && \
g++ \
gcc \
git \
libcryptsetup-dev \
libssl-dev \
libtss2-dev \
make \

View File

@@ -34,6 +34,7 @@ build_coco_guest_components_from_source() {
strip "target/${RUST_ARCH}-unknown-linux-${LIBC}/release/api-server-rest"
DESTDIR="${DESTDIR}/usr/local/bin" TEE_PLATFORM=${TEE_PLATFORM} make install
install -D -m0755 "confidential-data-hub/hub/src/storage/scripts/luks-encrypt-storage" "${DESTDIR}/usr/local/bin/luks-encrypt-storage"
install -D -m0644 "confidential-data-hub/hub/src/image/ocicrypt_config.json" "${DESTDIR}/etc/ocicrypt_config.json"
popd
}

View File

@@ -19,8 +19,6 @@ paths:
- "^ci/openshift-ci/": []
- "^\\.github/workflows/static-checks": ["static"]
- "^\\.github/workflows/": []
- "^docs/": ["static"]
- "^mkdocs\\.yaml$": ["static"]
- "\\.md$": ["static"]
# TODO: Expand filters
# Sources
@@ -125,6 +123,7 @@ mapping:
- Kata Containers CI / kata-containers-ci-on-push / build-kata-static-tarball-amd64 / build-asset (virtiofsd, test)
- Kata Containers CI / kata-containers-ci-on-push / build-kata-static-tarball-amd64 / create-kata-tarball
- Kata Containers CI / kata-containers-ci-on-push / build-kata-static-tarball-amd64 / build-tools-asset (agent-ctl, test)
- Kata Containers CI / kata-containers-ci-on-push / build-kata-static-tarball-amd64 / build-tools-asset (csi-kata-directvolume, test)
- Kata Containers CI / kata-containers-ci-on-push / build-kata-static-tarball-amd64 / build-tools-asset (genpolicy, test)
- Kata Containers CI / kata-containers-ci-on-push / build-kata-static-tarball-amd64 / build-tools-asset (kata-ctl, test)
- Kata Containers CI / kata-containers-ci-on-push / build-kata-static-tarball-amd64 / build-tools-asset (kata-manager, test)

View File

@@ -1,7 +1,7 @@
module module-path
// Keep in sync with version in versions.yaml
go 1.25.8
go 1.25.7
require (
github.com/sirupsen/logrus v1.9.3

View File

@@ -234,7 +234,7 @@ externals:
nvrc:
# yamllint disable-line rule:line-length
desc: "The NVRC project provides a Rust binary that implements a simple init system for microVMs"
version: "v0.1.3"
version: "v0.1.1"
url: "https://github.com/NVIDIA/nvrc/releases/download/"
nvidia:
@@ -266,9 +266,6 @@ externals:
# yamllint disable-line rule:line-length
url: "https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/sbsa/"
pkg: "cuda-keyring_1.1-1_all.deb"
ctk:
version: "1.18.1-1"
url: "https://github.com/NVIDIA/nvidia-container-toolkit"
busybox:
desc: "The Swiss Army Knife of Embedded Linux"
@@ -288,18 +285,18 @@ externals:
coco-guest-components:
description: "Provides attested key unwrapping for image decryption"
url: "https://github.com/confidential-containers/guest-components/"
version: "ab95914ac84c32a43102463cc0ae330710af47be"
version: "9aae2eae6a03ab97d6561bbe74f8b99843836bba"
toolchain: "1.90.0"
coco-trustee:
description: "Provides attestation and secret delivery components"
url: "https://github.com/confidential-containers/trustee"
version: "f5cb8fc1b51b652fc24e2d6b8742cf417805352e"
version: "3b2356a52e0d8a58730a1977e235a7e7f2007b5e"
# image / ita_image and image_tag / ita_image_tag must be in sync
image: "ghcr.io/confidential-containers/staged-images/kbs"
image_tag: "f5cb8fc1b51b652fc24e2d6b8742cf417805352e"
image_tag: "3b2356a52e0d8a58730a1977e235a7e7f2007b5e"
ita_image: "ghcr.io/confidential-containers/staged-images/kbs-ita-as"
ita_image_tag: "f5cb8fc1b51b652fc24e2d6b8742cf417805352e-x86_64"
ita_image_tag: "3b2356a52e0d8a58730a1977e235a7e7f2007b5e-x86_64"
toolchain: "1.90.0"
containerd:
@@ -468,12 +465,12 @@ languages:
description: "Google's 'go' language"
notes: "'version' is the default minimum version used by this project."
# When updating this, also update in go.mod files.
version: "1.25.8"
version: "1.25.7"
meta:
description: |
'newest-version' is the latest version known to work when
building Kata
newest-version: "1.25.8"
newest-version: "1.25.7"
rust:
description: "Rust language"