Merge pull request #11197 from Xynnn007/move-image-pull

Move image pull abilities to CDH
This commit is contained in:
Alex Lyn 2025-06-16 16:43:59 +08:00 committed by GitHub
commit a966d1be50
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
31 changed files with 448 additions and 3136 deletions

Binary file not shown.

Before

Width:  |  Height:  |  Size: 61 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 122 KiB

View File

@ -49,13 +49,58 @@ Pull the container image directly from the guest VM using `nydus snapshotter` ba
#### Architecture #### Architecture
The following diagram provides an overview of the architecture for pulling image in the guest with key components. The following diagram provides an overview of the architecture for pulling image in the guest with key components.
![guest-image-management-architecture](arch-images/guest-image-management-architecture.png) ```mermaid
flowchart LR
Kubelet[kubelet]--> |1\. Pull image request & metadata|Containerd
Containerd-->|2\. Pull image metadata| E
Containerd-->Snapshotter[Nydus Snapshotter]
Snapshotter-->|3\. Pack image info| Containerd
Containerd-->Runtime[Kata Runtime]
Runtime-->Hypervisor
Hypervisor-->TEE
Runtime-->|4\. Pass image info to VM| Agent
CDH1-->|6\. Pull image with image info|E[Container Images Registry]
subgraph TEE [Virtual Machine]
Images[Container Images]-->|7\. Prepare container rootfs|H[Container]
subgraph CDH [Confidential Data Hub]
CDH1[Image Mgmt]
end
CDH-->Images
Agent[Kata Agent]-->|5\. Call image pull RPC|CDH
end
```
#### Sequence diagrams #### Sequence diagrams
The following sequence diagram depicted below offers a detailed overview of the messages/calls exchanged to pull an unencrypted unsigned image from an unauthenticated container registry. This involves the kata-runtime, kata-agent, and the guest-components image-rs to use the guest pull mechanism. The following sequence diagram depicted below offers a detailed overview of the messages/calls exchanged to pull an unencrypted unsigned image from an unauthenticated container registry. This involves the kata-runtime, kata-agent, and the guest-components image-rs to use the guest pull mechanism.
![guest-image-management-details](arch-images/guest-image-management-details.png) ```mermaid
sequenceDiagram
par Hosts Side
Containerd/Kubelet->>runtime.kata_agent: createContainer(ctx,sandbox,c)
runtime.kata_agent->>runtime.fs_share_linux: ShareRootFilesystem(ctx,c)
runtime.fs_share_linux->>runtime.kata_agent: handleVirtualVolumeStorageObject(c,...,KataVolumeType)
runtime.kata_agent->>runtime.kata_agent: handleImageGuestPullBlockVolume(c,virtVolume,vol)
runtime.kata_agent->>runtime.fs_share_linux: ret:storage
runtime.fs_share_linux->>runtime.kata_agent: ret:sharedFile
and Guest Side
runtime.kata_agent->>agent.rpc: CreateContainerRequest(cid,...,storages,...,oci,...)
agent.rpc->>agent.storage: add_storage(storages...)
agent.storage->>agent.storage: StorageHandler.handler(driver)
agent.storage->>agent.storage.StorageHandler.ImagePullHandler: create_device(storage)
agent.storage.StorageHandler.ImagePullHandler->>agent.confidential_data_hub: pull_image(img,cid,img_metadata)
agent.confidential_data_hub->>Confidential Data Hub: pull_image(img,bundle_path)
Confidential Data Hub->>agent.confidential_data_hub: ret
agent.confidential_data_hub->>agent.storage.StorageHandler.ImagePullHandler: ret: bundle_path
agent.storage.StorageHandler.ImagePullHandler->>agent.storage: ret: device
agent.storage->>agent.rpc: ret: mount_list
and Return
agent.rpc->>runtime.kata_agent: ret: ok
runtime.kata_agent->>Containerd/Kubelet: ret: ok
end
```
First and foremost, the guest pull code path is only activated when `nydus snapshotter` requires the handling of a volume which type is `image_guest_pull`, as can be seen on the message below: First and foremost, the guest pull code path is only activated when `nydus snapshotter` requires the handling of a volume which type is `image_guest_pull`, as can be seen on the message below:
```json ```json
@ -108,10 +153,10 @@ Below is an example of storage information packaged in the message sent to the k
``` ```
Next, the kata-agent's RPC module will handle the create container request which, among other things, involves adding storages to the sandbox. The storage module contains implementations of `StorageHandler` interface for various storage types, being the `ImagePullHandler` in charge of handling the storage object for the container image (the storage manager instantiates the handler based on the value of the "driver"). Next, the kata-agent's RPC module will handle the create container request which, among other things, involves adding storages to the sandbox. The storage module contains implementations of `StorageHandler` interface for various storage types, being the `ImagePullHandler` in charge of handling the storage object for the container image (the storage manager instantiates the handler based on the value of the "driver").
`ImagePullHandler` delegates the image pulling operation to the `ImageService.pull_image()` that is going to create the image's bundle directory on the guest filesystem and, in turn, class the image-rs to in fact fetch and uncompress the image's bundle. `ImagePullHandler` delegates the image pulling operation to the `confidential_data_hub.pull_image()` that is going to create the image's bundle directory on the guest filesystem and, in turn, the `ImagePullService` of Confidential Data Hub to fetch, uncompress and mount the image's rootfs.
> **Notes:** > **Notes:**
> In this flow, `ImageService.pull_image()` parses the image metadata, looking for either the `io.kubernetes.cri.container-type: sandbox` or `io.kubernetes.cri-o.ContainerType: sandbox` (CRI-IO case) annotation, then it never calls the `image-rs.pull_image()` because the pause image is expected to already be inside the guest's filesystem, so instead `ImageService.unpack_pause_image()` is called. > In this flow, `confidential_data_hub.pull_image()` parses the image metadata, looking for either the `io.kubernetes.cri.container-type: sandbox` or `io.kubernetes.cri-o.ContainerType: sandbox` (CRI-IO case) annotation, then it never calls the `pull_image()` RPC of Confidential Data Hub because the pause image is expected to already be inside the guest's filesystem, so instead `confidential_data_hub.unpack_pause_image()` is called.
## Using guest image pull with `nerdctl` ## Using guest image pull with `nerdctl`
@ -121,6 +166,6 @@ nerdctl run --runtime io.containerd.kata.v2 --snapshotter nydus --label io.kuber
``` ```
References: References:
[1] [[RFC] Image management proposal for hosting sharing and peer pods](https://github.com/confidential-containers/confidential-containers/issues/137) 1. [[RFC] Image management proposal for hosting sharing and peer pods](https://github.com/confidential-containers/confidential-containers/issues/137)
[2] https://github.com/containerd/containerd/blob/main/docs/content-flow.md 2. https://github.com/containerd/containerd/blob/main/docs/content-flow.md
3. [Move guest pull ability to a configurable component](https://github.com/kata-containers/kata-containers/issues/9266)

2624
src/agent/Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@ -162,9 +162,6 @@ clap.workspace = true
strum.workspace = true strum.workspace = true
strum_macros.workspace = true strum_macros.workspace = true
# Image pull/decrypt
image-rs = { git = "https://github.com/confidential-containers/guest-components", rev = "0a06ef241190780840fbb0542e51b198f1f72b0b", default-features = false, optional = true }
# Agent Policy # Agent Policy
cdi = { git = "https://github.com/cncf-tags/container-device-interface-rs", rev = "fba5677a8e7cc962fc6e495fcec98d7d765e332a" } cdi = { git = "https://github.com/cncf-tags/container-device-interface-rs", rev = "fba5677a8e7cc962fc6e495fcec98d7d765e332a" }
@ -202,12 +199,9 @@ test-utils.workspace = true
lto = true lto = true
[features] [features]
# The default-pull feature supports all sharing images by virtio-fs, for guest-pull build with the guest-pull feature
default-pull = []
seccomp = ["rustjail/seccomp"] seccomp = ["rustjail/seccomp"]
standard-oci-runtime = ["rustjail/standard-oci-runtime"] standard-oci-runtime = ["rustjail/standard-oci-runtime"]
agent-policy = ["kata-agent-policy"] agent-policy = ["kata-agent-policy"]
guest-pull = ["image-rs/kata-cc-rustls-tls"]
[[bin]] [[bin]]
name = "kata-agent" name = "kata-agent"

View File

@ -41,16 +41,6 @@ ifeq ($(AGENT_POLICY),yes)
override EXTRA_RUSTFEATURES += agent-policy override EXTRA_RUSTFEATURES += agent-policy
endif endif
##VAR PULL_TYPE=default|guest-pull define if agent enables the guest pull image feature
PULL_TYPE ?= default
ifeq ($(PULL_TYPE),default)
override EXTRA_RUSTFEATURES += default-pull
# Enable guest pull image feature of rust build
else ifeq ($(PULL_TYPE),guest-pull)
override EXTRA_RUSTFEATURES += guest-pull
endif
include ../../utils.mk include ../../utils.mk
##VAR STANDARD_OCI_RUNTIME=yes|no define if agent enables standard oci runtime feature ##VAR STANDARD_OCI_RUNTIME=yes|no define if agent enables standard oci runtime feature

View File

@ -129,6 +129,7 @@ The kata agent has the ability to configure agent options in guest kernel comman
| `agent.guest_components_procs` | guest-components processes | Attestation-related processes that should be spawned as children of the guest. Valid values are `none`, `attestation-agent`, `confidential-data-hub` (implies `attestation-agent`), `api-server-rest` (implies `attestation-agent` and `confidential-data-hub`) | string | `api-server-rest` | | `agent.guest_components_procs` | guest-components processes | Attestation-related processes that should be spawned as children of the guest. Valid values are `none`, `attestation-agent`, `confidential-data-hub` (implies `attestation-agent`), `api-server-rest` (implies `attestation-agent` and `confidential-data-hub`) | string | `api-server-rest` |
| `agent.hotplug_timeout` | Hotplug timeout | Allow to configure hotplug timeout(seconds) of block devices | integer | `3` | | `agent.hotplug_timeout` | Hotplug timeout | Allow to configure hotplug timeout(seconds) of block devices | integer | `3` |
| `agent.cdh_api_timeout` | Confidential Data Hub (CDH) API timeout | Allow to configure CDH API timeout(seconds) | integer | `50` | | `agent.cdh_api_timeout` | Confidential Data Hub (CDH) API timeout | Allow to configure CDH API timeout(seconds) | integer | `50` |
| `agent.image_pull_timeout` | Confidential Data Hub (CDH) Image Pull API timeout | Allow to configure CDH API image pull timeout(seconds) | integer | `1200` |
| `agent.https_proxy` | HTTPS proxy | Allow to configure `https_proxy` in the guest | string | `""` | | `agent.https_proxy` | HTTPS proxy | Allow to configure `https_proxy` in the guest | string | `""` |
| `agent.image_registry_auth` | Image registry credential URI | The URI to where image-rs can find the credentials for pulling images from private registries e.g. `file:///root/.docker/config.json` to read from a file in the guest image, or `kbs:///default/credentials/test` to get the file from the KBS| string | `""` | | `agent.image_registry_auth` | Image registry credential URI | The URI to where image-rs can find the credentials for pulling images from private registries e.g. `file:///root/.docker/config.json` to read from a file in the guest image, or `kbs:///default/credentials/test` to get the file from the KBS| string | `""` |
| `agent.enable_signature_verification` | Image security policy flag | Whether enable image security policy enforcement. If `true`, the resource indexed by URI `agent.image_policy_file` will be got to work as image pulling policy. | string | `""` | | `agent.enable_signature_verification` | Image security policy flag | Whether enable image security policy enforcement. If `true`, the resource indexed by URI `agent.image_policy_file` will be got to work as image pulling policy. | string | `""` |
@ -148,7 +149,7 @@ The kata agent has the ability to configure agent options in guest kernel comman
> The agent will fail to start if the configuration file is not present, > The agent will fail to start if the configuration file is not present,
> or if it can't be parsed properly. > or if it can't be parsed properly.
> - `agent.devmode`: true | false > - `agent.devmode`: true | false
> - `agent.hotplug_timeout` and `agent.cdh_api_timeout`: a whole number of seconds > - `agent.hotplug_timeout`, `agent.image_pull_timeout` and `agent.cdh_api_timeout`: a whole number of seconds
> - `agent.log`: "critical"("fatal" | "panic") | "error" | "warn"("warning") | "info" | "debug" > - `agent.log`: "critical"("fatal" | "panic") | "error" | "warn"("warning") | "info" | "debug"
> - `agent.server_addr`: "{VSOCK_ADDR}:{VSOCK_PORT}" > - `agent.server_addr`: "{VSOCK_ADDR}:{VSOCK_PORT}"
> - `agent.trace`: true | false > - `agent.trace`: true | false

View File

@ -0,0 +1,156 @@
// Copyright (c) 2021 Alibaba Cloud
// Copyright (c) 2021, 2023 IBM Corporation
// Copyright (c) 2022 Intel Corporation
//
// SPDX-License-Identifier: Apache-2.0
//
use safe_path::scoped_join;
use std::collections::HashMap;
use std::fs;
use std::path::Path;
use anyhow::{anyhow, bail, Context, Result};
use kata_sys_util::validate::verify_id;
use oci_spec::runtime as oci;
use crate::rpc::CONTAINER_BASE;
use kata_types::mount::KATA_VIRTUAL_VOLUME_IMAGE_GUEST_PULL;
use protocols::agent::Storage;
pub const KATA_IMAGE_WORK_DIR: &str = "/run/kata-containers/image/";
const CONFIG_JSON: &str = "config.json";
const KATA_PAUSE_BUNDLE: &str = "/pause_bundle";
const K8S_CONTAINER_TYPE_KEYS: [&str; 2] = [
"io.kubernetes.cri.container-type",
"io.kubernetes.cri-o.ContainerType",
];
// Convenience function to obtain the scope logger.
fn sl() -> slog::Logger {
slog_scope::logger().new(o!("subsystem" => "image"))
}
// Function to copy a file if it does not exist at the destination
// This function creates a dir, writes a file and if necessary,
// overwrites an existing file.
fn copy_if_not_exists(src: &Path, dst: &Path) -> Result<()> {
if let Some(dst_dir) = dst.parent() {
fs::create_dir_all(dst_dir)?;
}
fs::copy(src, dst)?;
Ok(())
}
/// get guest pause image process specification
fn get_pause_image_process() -> Result<oci::Process> {
let guest_pause_bundle = Path::new(KATA_PAUSE_BUNDLE);
if !guest_pause_bundle.exists() {
bail!("Pause image not present in rootfs");
}
let guest_pause_config = scoped_join(guest_pause_bundle, CONFIG_JSON)?;
let image_oci = oci::Spec::load(guest_pause_config.to_str().ok_or_else(|| {
anyhow!(
"Failed to load the guest pause image config from {:?}",
guest_pause_config
)
})?)
.context("load image config file")?;
let image_oci_process = image_oci.process().as_ref().ok_or_else(|| {
anyhow!("The guest pause image config does not contain a process specification. Please check the pause image.")
})?;
Ok(image_oci_process.clone())
}
/// pause image is packaged in rootfs
pub fn unpack_pause_image(cid: &str) -> Result<String> {
verify_id(cid).context("The guest pause image cid contains invalid characters.")?;
let guest_pause_bundle = Path::new(KATA_PAUSE_BUNDLE);
if !guest_pause_bundle.exists() {
bail!("Pause image not present in rootfs");
}
let guest_pause_config = scoped_join(guest_pause_bundle, CONFIG_JSON)?;
info!(sl(), "use guest pause image cid {:?}", cid);
let image_oci = oci::Spec::load(guest_pause_config.to_str().ok_or_else(|| {
anyhow!(
"Failed to load the guest pause image config from {:?}",
guest_pause_config
)
})?)
.context("load image config file")?;
let image_oci_process = image_oci.process().as_ref().ok_or_else(|| {
anyhow!("The guest pause image config does not contain a process specification. Please check the pause image.")
})?;
info!(
sl(),
"pause image oci process {:?}",
image_oci_process.clone()
);
// Ensure that the args vector is not empty before accessing its elements.
// Check the number of arguments.
let args = if let Some(args_vec) = image_oci_process.args() {
args_vec
} else {
bail!("The number of args should be greater than or equal to one! Please check the pause image.");
};
let pause_bundle = scoped_join(CONTAINER_BASE, cid)?;
fs::create_dir_all(&pause_bundle)?;
let pause_rootfs = scoped_join(&pause_bundle, "rootfs")?;
fs::create_dir_all(&pause_rootfs)?;
info!(sl(), "pause_rootfs {:?}", pause_rootfs);
copy_if_not_exists(&guest_pause_config, &pause_bundle.join(CONFIG_JSON))?;
let arg_path = Path::new(&args[0]).strip_prefix("/")?;
copy_if_not_exists(
&guest_pause_bundle.join("rootfs").join(arg_path),
&pause_rootfs.join(arg_path),
)?;
Ok(pause_rootfs.display().to_string())
}
/// check whether the image is for sandbox or for container.
pub fn is_sandbox(image_metadata: &HashMap<String, String>) -> bool {
let mut is_sandbox = false;
for key in K8S_CONTAINER_TYPE_KEYS.iter() {
if let Some(value) = image_metadata.get(key as &str) {
if value == "sandbox" {
is_sandbox = true;
break;
}
}
}
is_sandbox
}
/// get_process overrides the OCI process spec with pause image process spec if needed
pub fn get_process(
ocip: &oci::Process,
oci: &oci::Spec,
storages: Vec<Storage>,
) -> Result<oci::Process> {
let mut guest_pull = false;
for storage in storages {
if storage.driver == KATA_VIRTUAL_VOLUME_IMAGE_GUEST_PULL {
guest_pull = true;
break;
}
}
if guest_pull {
if let Some(a) = oci.annotations() {
if is_sandbox(a) {
return get_pause_image_process();
}
}
}
Ok(ocip.clone())
}

View File

@ -1,4 +1,5 @@
// Copyright (c) 2023 Intel Corporation // Copyright (c) 2023 Intel Corporation
// Copyright (c) 2025 Alibaba Cloud
// //
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
// //
@ -15,19 +16,19 @@ use protocols::{
confidential_data_hub::GetResourceRequest, confidential_data_hub::GetResourceRequest,
confidential_data_hub_ttrpc_async, confidential_data_hub_ttrpc_async,
confidential_data_hub_ttrpc_async::{ confidential_data_hub_ttrpc_async::{
GetResourceServiceClient, SealedSecretServiceClient, SecureMountServiceClient, GetResourceServiceClient, ImagePullServiceClient, SealedSecretServiceClient,
SecureMountServiceClient,
}, },
}; };
use safe_path::scoped_join;
use std::fs; use std::fs;
use std::os::unix::fs::symlink;
use std::path::Path; use std::path::Path;
use std::{os::unix::fs::symlink, path::PathBuf};
use tokio::sync::OnceCell; use tokio::sync::OnceCell;
// Nanoseconds pub mod image;
lazy_static! {
static ref CDH_API_TIMEOUT: i64 = AGENT_CONFIG.cdh_api_timeout.as_nanos() as i64; pub static CDH_CLIENT: OnceCell<CDHClient> = OnceCell::const_new();
pub static ref CDH_CLIENT: OnceCell<CDHClient> = OnceCell::new();
}
const SEALED_SECRET_PREFIX: &str = "sealed."; const SEALED_SECRET_PREFIX: &str = "sealed.";
@ -45,6 +46,8 @@ pub struct CDHClient {
secure_mount_client: SecureMountServiceClient, secure_mount_client: SecureMountServiceClient,
#[derivative(Debug = "ignore")] #[derivative(Debug = "ignore")]
get_resource_client: GetResourceServiceClient, get_resource_client: GetResourceServiceClient,
#[derivative(Debug = "ignore")]
image_pull_client: ImagePullServiceClient,
} }
impl CDHClient { impl CDHClient {
@ -52,6 +55,8 @@ impl CDHClient {
let client = ttrpc::asynchronous::Client::connect(cdh_socket_uri)?; let client = ttrpc::asynchronous::Client::connect(cdh_socket_uri)?;
let sealed_secret_client = let sealed_secret_client =
confidential_data_hub_ttrpc_async::SealedSecretServiceClient::new(client.clone()); confidential_data_hub_ttrpc_async::SealedSecretServiceClient::new(client.clone());
let image_pull_client =
confidential_data_hub_ttrpc_async::ImagePullServiceClient::new(client.clone());
let secure_mount_client = let secure_mount_client =
confidential_data_hub_ttrpc_async::SecureMountServiceClient::new(client.clone()); confidential_data_hub_ttrpc_async::SecureMountServiceClient::new(client.clone());
let get_resource_client = let get_resource_client =
@ -60,6 +65,7 @@ impl CDHClient {
sealed_secret_client, sealed_secret_client,
secure_mount_client, secure_mount_client,
get_resource_client, get_resource_client,
image_pull_client,
}) })
} }
@ -69,12 +75,14 @@ impl CDHClient {
let unsealed_secret = self let unsealed_secret = self
.sealed_secret_client .sealed_secret_client
.unseal_secret(ttrpc::context::with_timeout(*CDH_API_TIMEOUT), &input) .unseal_secret(
ttrpc::context::with_timeout(AGENT_CONFIG.cdh_api_timeout.as_nanos() as i64),
&input,
)
.await?; .await?;
Ok(unsealed_secret.plaintext) Ok(unsealed_secret.plaintext)
} }
#[cfg(feature = "guest-pull")]
pub async fn secure_mount( pub async fn secure_mount(
&self, &self,
volume_type: &str, volume_type: &str,
@ -90,7 +98,10 @@ impl CDHClient {
..Default::default() ..Default::default()
}; };
self.secure_mount_client self.secure_mount_client
.secure_mount(ttrpc::context::with_timeout(*CDH_API_TIMEOUT), &req) .secure_mount(
ttrpc::context::with_timeout(AGENT_CONFIG.cdh_api_timeout.as_nanos() as i64),
&req,
)
.await?; .await?;
Ok(()) Ok(())
} }
@ -102,10 +113,31 @@ impl CDHClient {
}; };
let res = self let res = self
.get_resource_client .get_resource_client
.get_resource(ttrpc::context::with_timeout(*CDH_API_TIMEOUT), &req) .get_resource(
ttrpc::context::with_timeout(AGENT_CONFIG.cdh_api_timeout.as_nanos() as i64),
&req,
)
.await?; .await?;
Ok(res.Resource) Ok(res.Resource)
} }
pub async fn pull_image(&self, image: &str, bundle_path: &str) -> Result<()> {
let req = confidential_data_hub::ImagePullRequest {
image_url: image.to_string(),
bundle_path: bundle_path.to_string(),
..Default::default()
};
let _ = self
.image_pull_client
.pull_image(
ttrpc::context::with_timeout(AGENT_CONFIG.image_pull_timeout.as_nanos() as i64),
&req,
)
.await?;
Ok(())
}
} }
pub async fn init_cdh_client(cdh_socket_uri: &str) -> Result<()> { pub async fn init_cdh_client(cdh_socket_uri: &str) -> Result<()> {
@ -114,11 +146,12 @@ pub async fn init_cdh_client(cdh_socket_uri: &str) -> Result<()> {
CDHClient::new(cdh_socket_uri).context("Failed to create CDH Client") CDHClient::new(cdh_socket_uri).context("Failed to create CDH Client")
}) })
.await?; .await?;
Ok(()) Ok(())
} }
/// Check if the CDH client is initialized /// Check if the CDH client is initialized
pub async fn is_cdh_client_initialized() -> bool { pub fn is_cdh_client_initialized() -> bool {
CDH_CLIENT.get().is_some() // Returns true if CDH_CLIENT is initialized, false otherwise CDH_CLIENT.get().is_some() // Returns true if CDH_CLIENT is initialized, false otherwise
} }
@ -138,6 +171,29 @@ pub async fn unseal_env(env: &str) -> Result<String> {
Ok((*env.to_owned()).to_string()) Ok((*env.to_owned()).to_string())
} }
/// pull_image is used for call confidential data hub to pull image in the guest.
/// Image layers will store at [`image::KATA_IMAGE_WORK_DIR`]`,
/// rootfs and config.json will store under given `bundle_path`.
///
/// # Parameters
/// - `image`: Image name (exp: quay.io/prometheus/busybox:latest)
/// - `bundle_path`: The path to store the image bundle (exp. /run/kata-containers/cb0b47276ea66ee9f44cc53afa94d7980b57a52c3f306f68cb034e58d9fbd3c6/rootfs)
pub async fn pull_image(image: &str, bundle_path: PathBuf) -> Result<String> {
fs::create_dir_all(&bundle_path)?;
info!(sl(), "pull image {image:?}, bundle path {bundle_path:?}");
let cdh_client = CDH_CLIENT
.get()
.expect("Confidential Data Hub not initialized");
cdh_client
.pull_image(image, bundle_path.to_string_lossy().as_ref())
.await?;
let image_bundle_path = scoped_join(&bundle_path, "rootfs")?;
Ok(image_bundle_path.as_path().display().to_string())
}
pub async fn unseal_file(path: &str) -> Result<()> { pub async fn unseal_file(path: &str) -> Result<()> {
let cdh_client = CDH_CLIENT let cdh_client = CDH_CLIENT
.get() .get()
@ -206,7 +262,6 @@ pub async fn unseal_file(path: &str) -> Result<()> {
Ok(()) Ok(())
} }
#[cfg(feature = "guest-pull")]
pub async fn secure_mount( pub async fn secure_mount(
volume_type: &str, volume_type: &str,
options: &std::collections::HashMap<String, String>, options: &std::collections::HashMap<String, String>,
@ -257,6 +312,18 @@ mod tests {
} }
} }
#[async_trait]
impl confidential_data_hub_ttrpc_async::ImagePullService for TestService {
async fn pull_image(
&self,
_ctx: &::ttrpc::asynchronous::TtrpcContext,
_req: confidential_data_hub::ImagePullRequest,
) -> ttrpc::error::Result<confidential_data_hub::ImagePullResponse> {
let output = confidential_data_hub::ImagePullResponse::new();
Ok(output)
}
}
fn remove_if_sock_exist(sock_addr: &str) -> std::io::Result<()> { fn remove_if_sock_exist(sock_addr: &str) -> std::io::Result<()> {
let path = sock_addr let path = sock_addr
.strip_prefix("unix://") .strip_prefix("unix://")

View File

@ -23,6 +23,7 @@ const SERVER_ADDR_OPTION: &str = "agent.server_addr";
const PASSFD_LISTENER_PORT: &str = "agent.passfd_listener_port"; const PASSFD_LISTENER_PORT: &str = "agent.passfd_listener_port";
const HOTPLUG_TIMOUT_OPTION: &str = "agent.hotplug_timeout"; const HOTPLUG_TIMOUT_OPTION: &str = "agent.hotplug_timeout";
const CDH_API_TIMOUT_OPTION: &str = "agent.cdh_api_timeout"; const CDH_API_TIMOUT_OPTION: &str = "agent.cdh_api_timeout";
const CDH_IMAGE_PULL_TIMEOUT_OPTION: &str = "agent.image_pull_timeout";
const CDI_TIMEOUT_OPTION: &str = "agent.cdi_timeout"; const CDI_TIMEOUT_OPTION: &str = "agent.cdi_timeout";
const DEBUG_CONSOLE_VPORT_OPTION: &str = "agent.debug_console_vport"; const DEBUG_CONSOLE_VPORT_OPTION: &str = "agent.debug_console_vport";
const LOG_VPORT_OPTION: &str = "agent.log_vport"; const LOG_VPORT_OPTION: &str = "agent.log_vport";
@ -32,16 +33,8 @@ const UNIFIED_CGROUP_HIERARCHY_OPTION: &str = "systemd.unified_cgroup_hierarchy"
const CONFIG_FILE: &str = "agent.config_file"; const CONFIG_FILE: &str = "agent.config_file";
const GUEST_COMPONENTS_REST_API_OPTION: &str = "agent.guest_components_rest_api"; const GUEST_COMPONENTS_REST_API_OPTION: &str = "agent.guest_components_rest_api";
const GUEST_COMPONENTS_PROCS_OPTION: &str = "agent.guest_components_procs"; const GUEST_COMPONENTS_PROCS_OPTION: &str = "agent.guest_components_procs";
#[cfg(feature = "guest-pull")]
const IMAGE_REGISTRY_AUTH_OPTION: &str = "agent.image_registry_auth";
const SECURE_STORAGE_INTEGRITY_OPTION: &str = "agent.secure_storage_integrity"; const SECURE_STORAGE_INTEGRITY_OPTION: &str = "agent.secure_storage_integrity";
#[cfg(feature = "guest-pull")]
const ENABLE_SIGNATURE_VERIFICATION: &str = "agent.enable_signature_verification";
#[cfg(feature = "guest-pull")]
const IMAGE_POLICY_FILE: &str = "agent.image_policy_file";
// Configure the proxy settings for HTTPS requests in the guest, // Configure the proxy settings for HTTPS requests in the guest,
// to solve the problem of not being able to access the specified image in some cases. // to solve the problem of not being able to access the specified image in some cases.
const HTTPS_PROXY: &str = "agent.https_proxy"; const HTTPS_PROXY: &str = "agent.https_proxy";
@ -71,6 +64,7 @@ const MEM_AGENT_COMPACT_FORCE_TIMES: &str = "agent.mem_agent_compact_force_times
const DEFAULT_LOG_LEVEL: slog::Level = slog::Level::Info; const DEFAULT_LOG_LEVEL: slog::Level = slog::Level::Info;
const DEFAULT_HOTPLUG_TIMEOUT: time::Duration = time::Duration::from_secs(3); const DEFAULT_HOTPLUG_TIMEOUT: time::Duration = time::Duration::from_secs(3);
const DEFAULT_CDH_API_TIMEOUT: time::Duration = time::Duration::from_secs(50); const DEFAULT_CDH_API_TIMEOUT: time::Duration = time::Duration::from_secs(50);
const DEFAULT_IMAGE_PULL_TIMEOUT: time::Duration = time::Duration::from_secs(1200);
const DEFAULT_CDI_TIMEOUT: time::Duration = time::Duration::from_secs(100); const DEFAULT_CDI_TIMEOUT: time::Duration = time::Duration::from_secs(100);
const DEFAULT_CONTAINER_PIPE_SIZE: i32 = 0; const DEFAULT_CONTAINER_PIPE_SIZE: i32 = 0;
const VSOCK_ADDR: &str = "vsock://-1"; const VSOCK_ADDR: &str = "vsock://-1";
@ -134,6 +128,7 @@ pub struct AgentConfig {
pub log_level: slog::Level, pub log_level: slog::Level,
pub hotplug_timeout: time::Duration, pub hotplug_timeout: time::Duration,
pub cdh_api_timeout: time::Duration, pub cdh_api_timeout: time::Duration,
pub image_pull_timeout: time::Duration,
pub cdi_timeout: time::Duration, pub cdi_timeout: time::Duration,
pub debug_console_vport: i32, pub debug_console_vport: i32,
pub log_vport: i32, pub log_vport: i32,
@ -147,13 +142,7 @@ pub struct AgentConfig {
pub no_proxy: String, pub no_proxy: String,
pub guest_components_rest_api: GuestComponentsFeatures, pub guest_components_rest_api: GuestComponentsFeatures,
pub guest_components_procs: GuestComponentsProcs, pub guest_components_procs: GuestComponentsProcs,
#[cfg(feature = "guest-pull")]
pub image_registry_auth: String,
pub secure_storage_integrity: bool, pub secure_storage_integrity: bool,
#[cfg(feature = "guest-pull")]
pub enable_signature_verification: bool,
#[cfg(feature = "guest-pull")]
pub image_policy_file: String,
#[cfg(feature = "agent-policy")] #[cfg(feature = "agent-policy")]
pub policy_file: String, pub policy_file: String,
pub mem_agent: Option<MemAgentConfig>, pub mem_agent: Option<MemAgentConfig>,
@ -172,6 +161,7 @@ pub struct AgentConfigBuilder {
pub log_level: Option<String>, pub log_level: Option<String>,
pub hotplug_timeout: Option<time::Duration>, pub hotplug_timeout: Option<time::Duration>,
pub cdh_api_timeout: Option<time::Duration>, pub cdh_api_timeout: Option<time::Duration>,
pub image_pull_timeout: Option<time::Duration>,
pub cdi_timeout: Option<time::Duration>, pub cdi_timeout: Option<time::Duration>,
pub debug_console_vport: Option<i32>, pub debug_console_vport: Option<i32>,
pub log_vport: Option<i32>, pub log_vport: Option<i32>,
@ -184,13 +174,7 @@ pub struct AgentConfigBuilder {
pub no_proxy: Option<String>, pub no_proxy: Option<String>,
pub guest_components_rest_api: Option<GuestComponentsFeatures>, pub guest_components_rest_api: Option<GuestComponentsFeatures>,
pub guest_components_procs: Option<GuestComponentsProcs>, pub guest_components_procs: Option<GuestComponentsProcs>,
#[cfg(feature = "guest-pull")]
pub image_registry_auth: Option<String>,
pub secure_storage_integrity: Option<bool>, pub secure_storage_integrity: Option<bool>,
#[cfg(feature = "guest-pull")]
pub enable_signature_verification: Option<bool>,
#[cfg(feature = "guest-pull")]
pub image_policy_file: Option<String>,
#[cfg(feature = "agent-policy")] #[cfg(feature = "agent-policy")]
pub policy_file: Option<String>, pub policy_file: Option<String>,
pub mem_agent_enable: Option<bool>, pub mem_agent_enable: Option<bool>,
@ -271,6 +255,7 @@ impl Default for AgentConfig {
log_level: DEFAULT_LOG_LEVEL, log_level: DEFAULT_LOG_LEVEL,
hotplug_timeout: DEFAULT_HOTPLUG_TIMEOUT, hotplug_timeout: DEFAULT_HOTPLUG_TIMEOUT,
cdh_api_timeout: DEFAULT_CDH_API_TIMEOUT, cdh_api_timeout: DEFAULT_CDH_API_TIMEOUT,
image_pull_timeout: DEFAULT_IMAGE_PULL_TIMEOUT,
cdi_timeout: DEFAULT_CDI_TIMEOUT, cdi_timeout: DEFAULT_CDI_TIMEOUT,
debug_console_vport: 0, debug_console_vport: 0,
log_vport: 0, log_vport: 0,
@ -284,13 +269,7 @@ impl Default for AgentConfig {
no_proxy: String::from(""), no_proxy: String::from(""),
guest_components_rest_api: GuestComponentsFeatures::default(), guest_components_rest_api: GuestComponentsFeatures::default(),
guest_components_procs: GuestComponentsProcs::default(), guest_components_procs: GuestComponentsProcs::default(),
#[cfg(feature = "guest-pull")]
image_registry_auth: String::from(""),
secure_storage_integrity: false, secure_storage_integrity: false,
#[cfg(feature = "guest-pull")]
enable_signature_verification: false,
#[cfg(feature = "guest-pull")]
image_policy_file: String::from(""),
#[cfg(feature = "agent-policy")] #[cfg(feature = "agent-policy")]
policy_file: String::from(""), policy_file: String::from(""),
mem_agent: None, mem_agent: None,
@ -317,6 +296,7 @@ impl FromStr for AgentConfig {
); );
config_override!(agent_config_builder, agent_config, hotplug_timeout); config_override!(agent_config_builder, agent_config, hotplug_timeout);
config_override!(agent_config_builder, agent_config, cdh_api_timeout); config_override!(agent_config_builder, agent_config, cdh_api_timeout);
config_override!(agent_config_builder, agent_config, image_pull_timeout);
config_override!(agent_config_builder, agent_config, cdi_timeout); config_override!(agent_config_builder, agent_config, cdi_timeout);
config_override!(agent_config_builder, agent_config, debug_console_vport); config_override!(agent_config_builder, agent_config, debug_console_vport);
config_override!(agent_config_builder, agent_config, log_vport); config_override!(agent_config_builder, agent_config, log_vport);
@ -333,16 +313,6 @@ impl FromStr for AgentConfig {
guest_components_rest_api guest_components_rest_api
); );
config_override!(agent_config_builder, agent_config, guest_components_procs); config_override!(agent_config_builder, agent_config, guest_components_procs);
#[cfg(feature = "guest-pull")]
{
config_override!(agent_config_builder, agent_config, image_registry_auth);
config_override!(
agent_config_builder,
agent_config,
enable_signature_verification
);
config_override!(agent_config_builder, agent_config, image_policy_file);
}
config_override!(agent_config_builder, agent_config, secure_storage_integrity); config_override!(agent_config_builder, agent_config, secure_storage_integrity);
#[cfg(feature = "agent-policy")] #[cfg(feature = "agent-policy")]
@ -493,6 +463,15 @@ impl AgentConfig {
|cdh_api_timeout: &time::Duration| cdh_api_timeout.as_secs() > 0 |cdh_api_timeout: &time::Duration| cdh_api_timeout.as_secs() > 0
); );
// ensure the timeout is a positive value
parse_cmdline_param!(
param,
CDH_IMAGE_PULL_TIMEOUT_OPTION,
config.image_pull_timeout,
get_timeout,
|image_pull_timeout: &time::Duration| image_pull_timeout.as_secs() > 0
);
// ensure the timeout is a positive value // ensure the timeout is a positive value
parse_cmdline_param!( parse_cmdline_param!(
param, param,
@ -557,27 +536,6 @@ impl AgentConfig {
config.guest_components_procs, config.guest_components_procs,
get_guest_components_procs_value get_guest_components_procs_value
); );
#[cfg(feature = "guest-pull")]
{
parse_cmdline_param!(
param,
IMAGE_REGISTRY_AUTH_OPTION,
config.image_registry_auth,
get_string_value
);
parse_cmdline_param!(
param,
ENABLE_SIGNATURE_VERIFICATION,
config.enable_signature_verification,
get_bool_value
);
parse_cmdline_param!(
param,
IMAGE_POLICY_FILE,
config.image_policy_file,
get_string_value
);
}
parse_cmdline_param!( parse_cmdline_param!(
param, param,
SECURE_STORAGE_INTEGRITY_OPTION, SECURE_STORAGE_INTEGRITY_OPTION,
@ -780,7 +738,10 @@ fn get_timeout(param: &str) -> Result<time::Duration> {
ensure!( ensure!(
matches!( matches!(
fields[0], fields[0],
HOTPLUG_TIMOUT_OPTION | CDH_API_TIMOUT_OPTION | CDI_TIMEOUT_OPTION HOTPLUG_TIMOUT_OPTION
| CDH_API_TIMOUT_OPTION
| CDH_IMAGE_PULL_TIMEOUT_OPTION
| CDI_TIMEOUT_OPTION
), ),
ERR_INVALID_TIMEOUT_KEY ERR_INVALID_TIMEOUT_KEY
); );
@ -901,11 +862,6 @@ mod tests {
assert!(!config.dev_mode); assert!(!config.dev_mode);
assert_eq!(config.log_level, DEFAULT_LOG_LEVEL); assert_eq!(config.log_level, DEFAULT_LOG_LEVEL);
assert_eq!(config.hotplug_timeout, DEFAULT_HOTPLUG_TIMEOUT); assert_eq!(config.hotplug_timeout, DEFAULT_HOTPLUG_TIMEOUT);
#[cfg(feature = "guest-pull")]
{
assert!(!config.enable_signature_verification);
assert_eq!(config.image_policy_file, "");
}
} }
#[test] #[test]
@ -931,13 +887,7 @@ mod tests {
no_proxy: &'a str, no_proxy: &'a str,
guest_components_rest_api: GuestComponentsFeatures, guest_components_rest_api: GuestComponentsFeatures,
guest_components_procs: GuestComponentsProcs, guest_components_procs: GuestComponentsProcs,
#[cfg(feature = "guest-pull")]
image_registry_auth: &'a str,
secure_storage_integrity: bool, secure_storage_integrity: bool,
#[cfg(feature = "guest-pull")]
enable_signature_verification: bool,
#[cfg(feature = "guest-pull")]
image_policy_file: &'a str,
#[cfg(feature = "agent-policy")] #[cfg(feature = "agent-policy")]
policy_file: &'a str, policy_file: &'a str,
mem_agent: Option<MemAgentConfig>, mem_agent: Option<MemAgentConfig>,
@ -961,13 +911,7 @@ mod tests {
no_proxy: "", no_proxy: "",
guest_components_rest_api: GuestComponentsFeatures::default(), guest_components_rest_api: GuestComponentsFeatures::default(),
guest_components_procs: GuestComponentsProcs::default(), guest_components_procs: GuestComponentsProcs::default(),
#[cfg(feature = "guest-pull")]
image_registry_auth: "",
secure_storage_integrity: false, secure_storage_integrity: false,
#[cfg(feature = "guest-pull")]
enable_signature_verification: false,
#[cfg(feature = "guest-pull")]
image_policy_file: "",
#[cfg(feature = "agent-policy")] #[cfg(feature = "agent-policy")]
policy_file: "", policy_file: "",
mem_agent: None, mem_agent: None,
@ -1418,18 +1362,6 @@ mod tests {
guest_components_procs: GuestComponentsProcs::None, guest_components_procs: GuestComponentsProcs::None,
..Default::default() ..Default::default()
}, },
#[cfg(feature = "guest-pull")]
TestData {
contents: "agent.image_registry_auth=file:///root/.docker/config.json",
image_registry_auth: "file:///root/.docker/config.json",
..Default::default()
},
#[cfg(feature = "guest-pull")]
TestData {
contents: "agent.image_registry_auth=kbs:///default/credentials/test",
image_registry_auth: "kbs:///default/credentials/test",
..Default::default()
},
TestData { TestData {
contents: "", contents: "",
secure_storage_integrity: false, secure_storage_integrity: false,
@ -1455,24 +1387,6 @@ mod tests {
secure_storage_integrity: false, secure_storage_integrity: false,
..Default::default() ..Default::default()
}, },
#[cfg(feature = "guest-pull")]
TestData {
contents: "agent.enable_signature_verification=true",
enable_signature_verification: true,
..Default::default()
},
#[cfg(feature = "guest-pull")]
TestData {
contents: "agent.image_policy_file=kbs:///default/image-policy/test",
image_policy_file: "kbs:///default/image-policy/test",
..Default::default()
},
#[cfg(feature = "guest-pull")]
TestData {
contents: "agent.image_policy_file=file:///etc/image-policy.json",
image_policy_file: "file:///etc/image-policy.json",
..Default::default()
},
#[cfg(feature = "agent-policy")] #[cfg(feature = "agent-policy")]
// Test environment // Test environment
TestData { TestData {
@ -1575,16 +1489,6 @@ mod tests {
"{}", "{}",
msg msg
); );
#[cfg(feature = "guest-pull")]
{
assert_eq!(d.image_registry_auth, config.image_registry_auth, "{}", msg);
assert_eq!(
d.enable_signature_verification, config.enable_signature_verification,
"{}",
msg
);
assert_eq!(d.image_policy_file, config.image_policy_file, "{}", msg);
}
assert_eq!( assert_eq!(
d.secure_storage_integrity, config.secure_storage_integrity, d.secure_storage_integrity, config.secure_storage_integrity,
"{}", "{}",
@ -1722,6 +1626,7 @@ Caused by:
)))] )))]
#[case("agent.chd_api_timeout=1", Err(anyhow!(ERR_INVALID_TIMEOUT_KEY)))] #[case("agent.chd_api_timeout=1", Err(anyhow!(ERR_INVALID_TIMEOUT_KEY)))]
#[case("agent.cdh_api_timeout=600", Ok(time::Duration::from_secs(600)))] #[case("agent.cdh_api_timeout=600", Ok(time::Duration::from_secs(600)))]
#[case("agent.image_pull_timeout=1200", Ok(time::Duration::from_secs(1200)))]
#[case("agent.cdi_timeout=320", Ok(time::Duration::from_secs(320)))] #[case("agent.cdi_timeout=320", Ok(time::Duration::from_secs(320)))]
fn test_timeout(#[case] param: &str, #[case] expected: Result<time::Duration>) { fn test_timeout(#[case] param: &str, #[case] expected: Result<time::Duration>) {
let result = get_timeout(param); let result = get_timeout(param);

View File

@ -29,7 +29,7 @@ use tracing::instrument;
cfg_if! { cfg_if! {
if #[cfg(target_arch = "s390x")] { if #[cfg(target_arch = "s390x")] {
use crate::ap; use crate::ap;
use crate::cdh::get_cdh_resource; use crate::confidential_data_hub::get_cdh_resource;
use std::convert::TryFrom; use std::convert::TryFrom;
use pv_core::ap::{ use pv_core::ap::{
Apqn, Apqn,

View File

@ -8,8 +8,6 @@ pub fn get_build_features() -> Vec<String> {
let features: Vec<&str> = vec![ let features: Vec<&str> = vec![
#[cfg(feature = "agent-policy")] #[cfg(feature = "agent-policy")]
"agent-policy", "agent-policy",
#[cfg(feature = "guest-pull")]
"guest-pull",
#[cfg(feature = "seccomp")] #[cfg(feature = "seccomp")]
"seccomp", "seccomp",
#[cfg(feature = "standard-oci-runtime")] #[cfg(feature = "standard-oci-runtime")]

View File

@ -1,296 +0,0 @@
// Copyright (c) 2021 Alibaba Cloud
// Copyright (c) 2021, 2023 IBM Corporation
// Copyright (c) 2022 Intel Corporation
//
// SPDX-License-Identifier: Apache-2.0
//
use safe_path::scoped_join;
use std::collections::HashMap;
use std::env;
use std::fs;
use std::path::Path;
use std::sync::Arc;
use anyhow::{anyhow, bail, Context, Result};
use image_rs::builder::ClientBuilder;
use image_rs::image::ImageClient;
use kata_sys_util::validate::verify_id;
use oci_spec::runtime as oci;
use tokio::sync::Mutex;
use crate::rpc::CONTAINER_BASE;
use crate::AGENT_CONFIG;
use kata_types::mount::KATA_VIRTUAL_VOLUME_IMAGE_GUEST_PULL;
use protocols::agent::Storage;
pub const KATA_IMAGE_WORK_DIR: &str = "/run/kata-containers/image/";
const CONFIG_JSON: &str = "config.json";
const KATA_PAUSE_BUNDLE: &str = "/pause_bundle";
const K8S_CONTAINER_TYPE_KEYS: [&str; 2] = [
"io.kubernetes.cri.container-type",
"io.kubernetes.cri-o.ContainerType",
];
#[rustfmt::skip]
lazy_static! {
pub static ref IMAGE_SERVICE: Arc<Mutex<Option<ImageService>>> = Arc::new(Mutex::new(None));
}
// Convenience function to obtain the scope logger.
fn sl() -> slog::Logger {
slog_scope::logger().new(o!("subsystem" => "image"))
}
// Function to copy a file if it does not exist at the destination
fn copy_if_not_exists(src: &Path, dst: &Path) -> Result<()> {
if let Some(dst_dir) = dst.parent() {
fs::create_dir_all(dst_dir)?;
}
fs::copy(src, dst)?;
Ok(())
}
pub struct ImageService {
image_client: ImageClient,
}
impl ImageService {
pub async fn new() -> Result<Self> {
let mut image_client_builder =
ClientBuilder::default().work_dir(KATA_IMAGE_WORK_DIR.into());
#[cfg(feature = "guest-pull")]
{
if !AGENT_CONFIG.image_registry_auth.is_empty() {
let registry_auth = &AGENT_CONFIG.image_registry_auth;
debug!(sl(), "Set registry auth file {:?}", registry_auth);
image_client_builder = image_client_builder
.authenticated_registry_credentials_uri(registry_auth.into());
}
let enable_signature_verification = &AGENT_CONFIG.enable_signature_verification;
debug!(
sl(),
"Enable image signature verification: {:?}", enable_signature_verification
);
if !AGENT_CONFIG.image_policy_file.is_empty() && *enable_signature_verification {
let image_policy_file = &AGENT_CONFIG.image_policy_file;
debug!(sl(), "Use image policy file {:?}", image_policy_file);
image_client_builder =
image_client_builder.image_security_policy_uri(image_policy_file.into());
}
}
let image_client = image_client_builder.build().await?;
Ok(Self { image_client })
}
/// get guest pause image process specification
fn get_pause_image_process() -> Result<oci::Process> {
let guest_pause_bundle = Path::new(KATA_PAUSE_BUNDLE);
if !guest_pause_bundle.exists() {
bail!("Pause image not present in rootfs");
}
let guest_pause_config = scoped_join(guest_pause_bundle, CONFIG_JSON)?;
let image_oci = oci::Spec::load(guest_pause_config.to_str().ok_or_else(|| {
anyhow!(
"Failed to load the guest pause image config from {:?}",
guest_pause_config
)
})?)
.context("load image config file")?;
let image_oci_process = image_oci.process().as_ref().ok_or_else(|| {
anyhow!("The guest pause image config does not contain a process specification. Please check the pause image.")
})?;
Ok(image_oci_process.clone())
}
/// pause image is packaged in rootfs
fn unpack_pause_image(cid: &str) -> Result<String> {
verify_id(cid).context("The guest pause image cid contains invalid characters.")?;
let guest_pause_bundle = Path::new(KATA_PAUSE_BUNDLE);
if !guest_pause_bundle.exists() {
bail!("Pause image not present in rootfs");
}
let guest_pause_config = scoped_join(guest_pause_bundle, CONFIG_JSON)?;
info!(sl(), "use guest pause image cid {:?}", cid);
let image_oci = oci::Spec::load(guest_pause_config.to_str().ok_or_else(|| {
anyhow!(
"Failed to load the guest pause image config from {:?}",
guest_pause_config
)
})?)
.context("load image config file")?;
let image_oci_process = image_oci.process().as_ref().ok_or_else(|| {
anyhow!("The guest pause image config does not contain a process specification. Please check the pause image.")
})?;
info!(
sl(),
"pause image oci process {:?}",
image_oci_process.clone()
);
// Ensure that the args vector is not empty before accessing its elements.
// Check the number of arguments.
let args = if let Some(args_vec) = image_oci_process.args() {
args_vec
} else {
bail!("The number of args should be greater than or equal to one! Please check the pause image.");
};
let pause_bundle = scoped_join(CONTAINER_BASE, cid)?;
fs::create_dir_all(&pause_bundle)?;
let pause_rootfs = scoped_join(&pause_bundle, "rootfs")?;
fs::create_dir_all(&pause_rootfs)?;
info!(sl(), "pause_rootfs {:?}", pause_rootfs);
copy_if_not_exists(&guest_pause_config, &pause_bundle.join(CONFIG_JSON))?;
let arg_path = Path::new(&args[0]).strip_prefix("/")?;
copy_if_not_exists(
&guest_pause_bundle.join("rootfs").join(arg_path),
&pause_rootfs.join(arg_path),
)?;
Ok(pause_rootfs.display().to_string())
}
/// check whether the image is for sandbox or for container.
fn is_sandbox(image_metadata: &HashMap<String, String>) -> bool {
let mut is_sandbox = false;
for key in K8S_CONTAINER_TYPE_KEYS.iter() {
if let Some(value) = image_metadata.get(key as &str) {
if value == "sandbox" {
is_sandbox = true;
break;
}
}
}
is_sandbox
}
/// pull_image is used for call image-rs to pull image in the guest.
/// # Parameters
/// - `image`: Image name (exp: quay.io/prometheus/busybox:latest)
/// - `cid`: Container id
/// - `image_metadata`: Annotations about the image (exp: "containerd.io/snapshot/cri.layer-digest": "sha256:24fb2886d6f6c5d16481dd7608b47e78a8e92a13d6e64d87d57cb16d5f766d63")
/// # Returns
/// - The image rootfs bundle path. (exp. /run/kata-containers/cb0b47276ea66ee9f44cc53afa94d7980b57a52c3f306f68cb034e58d9fbd3c6/rootfs)
pub async fn pull_image(
&mut self,
image: &str,
cid: &str,
image_metadata: &HashMap<String, String>,
) -> Result<String> {
info!(sl(), "image metadata: {image_metadata:?}");
if Self::is_sandbox(image_metadata) {
let mount_path = Self::unpack_pause_image(cid)?;
return Ok(mount_path);
}
// Image layers will store at KATA_IMAGE_WORK_DIR, generated bundles
// with rootfs and config.json will store under CONTAINER_BASE/cid/images.
let bundle_path = scoped_join(CONTAINER_BASE, cid)?;
fs::create_dir_all(&bundle_path)?;
info!(sl(), "pull image {image:?}, bundle path {bundle_path:?}");
let res = self
.image_client
.pull_image(image, &bundle_path, &None, &None)
.await;
match res {
Ok(image) => {
info!(
sl(),
"pull and unpack image {image:?}, cid: {cid:?} succeeded."
);
}
Err(e) => {
error!(
sl(),
"pull and unpack image {image:?}, cid: {cid:?} failed with {:?}.",
e.to_string()
);
return Err(e);
}
};
let image_bundle_path = scoped_join(&bundle_path, "rootfs")?;
Ok(image_bundle_path.as_path().display().to_string())
}
}
/// get_process overrides the OCI process spec with pause image process spec if needed
pub fn get_process(
ocip: &oci::Process,
oci: &oci::Spec,
storages: Vec<Storage>,
) -> Result<oci::Process> {
let mut guest_pull = false;
for storage in storages {
if storage.driver == KATA_VIRTUAL_VOLUME_IMAGE_GUEST_PULL {
guest_pull = true;
break;
}
}
if guest_pull {
if let Some(a) = oci.annotations() {
if ImageService::is_sandbox(a) {
return ImageService::get_pause_image_process();
}
}
}
Ok(ocip.clone())
}
/// Set proxy environment from AGENT_CONFIG
pub async fn set_proxy_env_vars() {
if env::var("HTTPS_PROXY").is_err() {
let https_proxy = &AGENT_CONFIG.https_proxy;
if !https_proxy.is_empty() {
env::set_var("HTTPS_PROXY", https_proxy);
}
}
match env::var("HTTPS_PROXY") {
Ok(val) => info!(sl(), "https_proxy is set to: {}", val),
Err(e) => info!(sl(), "https_proxy is not set ({})", e),
};
if env::var("NO_PROXY").is_err() {
let no_proxy = &AGENT_CONFIG.no_proxy;
if !no_proxy.is_empty() {
env::set_var("NO_PROXY", no_proxy);
}
}
match env::var("NO_PROXY") {
Ok(val) => info!(sl(), "no_proxy is set to: {}", val),
Err(e) => info!(sl(), "no_proxy is not set ({})", e),
};
}
/// Init the image service
pub async fn init_image_service() -> Result<()> {
let image_service = ImageService::new().await?;
*IMAGE_SERVICE.lock().await = Some(image_service);
Ok(())
}
pub async fn pull_image(
image: &str,
cid: &str,
image_metadata: &HashMap<String, String>,
) -> Result<String> {
let image_service = IMAGE_SERVICE.clone();
let mut image_service = image_service.lock().await;
let image_service = image_service
.as_mut()
.expect("Image Service not initialized");
image_service.pull_image(image, cid, image_metadata).await
}

View File

@ -22,7 +22,7 @@ use anyhow::{anyhow, bail, Context, Result};
use base64::Engine; use base64::Engine;
use cfg_if::cfg_if; use cfg_if::cfg_if;
use clap::{AppSettings, Parser}; use clap::{AppSettings, Parser};
use const_format::{concatcp, formatcp}; use const_format::concatcp;
use initdata::{InitdataReturnValue, AA_CONFIG_PATH, CDH_CONFIG_PATH}; use initdata::{InitdataReturnValue, AA_CONFIG_PATH, CDH_CONFIG_PATH};
use nix::fcntl::OFlag; use nix::fcntl::OFlag;
use nix::sys::reboot::{reboot, RebootMode}; use nix::sys::reboot::{reboot, RebootMode};
@ -38,7 +38,7 @@ use std::process::exit;
use std::sync::Arc; use std::sync::Arc;
use tracing::{instrument, span}; use tracing::{instrument, span};
mod cdh; mod confidential_data_hub;
mod config; mod config;
mod console; mod console;
mod device; mod device;
@ -79,9 +79,6 @@ use tokio::{
task::JoinHandle, task::JoinHandle,
}; };
#[cfg(feature = "guest-pull")]
mod image;
mod rpc; mod rpc;
mod tracer; mod tracer;
@ -110,19 +107,9 @@ const CDH_SOCKET_URI: &str = concatcp!(UNIX_SOCKET_PREFIX, CDH_SOCKET);
const API_SERVER_PATH: &str = "/usr/local/bin/api-server-rest"; const API_SERVER_PATH: &str = "/usr/local/bin/api-server-rest";
/// Path of ocicrypt config file. This is used by image-rs when decrypting image. /// Path of ocicrypt config file. This is used by CDH when decrypting image.
const OCICRYPT_CONFIG_PATH: &str = "/run/confidential-containers/ocicrypt_config.json"; /// TODO: remove this when we move the launch of CDH out of the kata-agent.
const OCICRYPT_CONFIG_PATH: &str = "/etc/ocicrypt_config.json";
const OCICRYPT_CONFIG: &str = formatcp!(
r#"{{
"key-providers": {{
"attestation-agent": {{
"ttrpc": "{}"
}}
}}
}}"#,
CDH_SOCKET_URI
);
const DEFAULT_LAUNCH_PROCESS_TIMEOUT: i32 = 6; const DEFAULT_LAUNCH_PROCESS_TIMEOUT: i32 = 6;
@ -394,9 +381,6 @@ async fn start_sandbox(
s.rtnl.handle_localhost().await?; s.rtnl.handle_localhost().await?;
} }
#[cfg(feature = "guest-pull")]
image::set_proxy_env_vars().await;
#[cfg(feature = "agent-policy")] #[cfg(feature = "agent-policy")]
if let Err(e) = initialize_policy().await { if let Err(e) = initialize_policy().await {
error!(logger, "Failed to initialize agent policy: {:?}", e); error!(logger, "Failed to initialize agent policy: {:?}", e);
@ -516,6 +500,7 @@ async fn launch_guest_component_procs(
Some(AA_CONFIG_PATH), Some(AA_CONFIG_PATH),
AA_ATTESTATION_SOCKET, AA_ATTESTATION_SOCKET,
DEFAULT_LAUNCH_PROCESS_TIMEOUT, DEFAULT_LAUNCH_PROCESS_TIMEOUT,
&[],
) )
.await .await
.map_err(|e| anyhow!("launch_process {} failed: {:?}", AA_PATH, e))?; .map_err(|e| anyhow!("launch_process {} failed: {:?}", AA_PATH, e))?;
@ -537,6 +522,7 @@ async fn launch_guest_component_procs(
Some(CDH_CONFIG_PATH), Some(CDH_CONFIG_PATH),
CDH_SOCKET, CDH_SOCKET,
DEFAULT_LAUNCH_PROCESS_TIMEOUT, DEFAULT_LAUNCH_PROCESS_TIMEOUT,
&[("OCICRYPT_KEYPROVIDER_CONFIG", OCICRYPT_CONFIG_PATH)],
) )
.await .await
.map_err(|e| anyhow!("launch_process {} failed: {:?}", CDH_PATH, e))?; .map_err(|e| anyhow!("launch_process {} failed: {:?}", CDH_PATH, e))?;
@ -558,6 +544,7 @@ async fn launch_guest_component_procs(
None, None,
"", "",
0, 0,
&[],
) )
.await .await
.map_err(|e| anyhow!("launch_process {} failed: {:?}", API_SERVER_PATH, e))?; .map_err(|e| anyhow!("launch_process {} failed: {:?}", API_SERVER_PATH, e))?;
@ -580,9 +567,7 @@ async fn init_attestation_components(
match tokio::fs::metadata(CDH_SOCKET).await { match tokio::fs::metadata(CDH_SOCKET).await {
Ok(md) => { Ok(md) => {
if md.file_type().is_socket() { if md.file_type().is_socket() {
cdh::init_cdh_client(CDH_SOCKET_URI).await?; confidential_data_hub::init_cdh_client(CDH_SOCKET_URI).await?;
fs::write(OCICRYPT_CONFIG_PATH, OCICRYPT_CONFIG.as_bytes())?;
env::set_var("OCICRYPT_KEYPROVIDER_CONFIG", OCICRYPT_CONFIG_PATH);
} else { } else {
debug!(logger, "File {} is not a socket", CDH_SOCKET); debug!(logger, "File {} is not a socket", CDH_SOCKET);
} }
@ -624,6 +609,7 @@ async fn launch_process(
config: Option<&str>, config: Option<&str>,
unix_socket_path: &str, unix_socket_path: &str,
timeout_secs: i32, timeout_secs: i32,
envs: &[(&str, &str)],
) -> Result<()> { ) -> Result<()> {
if !Path::new(path).exists() { if !Path::new(path).exists() {
bail!("path {} does not exist.", path); bail!("path {} does not exist.", path);
@ -640,7 +626,12 @@ async fn launch_process(
tokio::fs::remove_file(unix_socket_path).await?; tokio::fs::remove_file(unix_socket_path).await?;
} }
tokio::process::Command::new(path).args(args).spawn()?; let mut process = tokio::process::Command::new(path);
process.args(args);
for (k, v) in envs {
process.env(k, v);
}
process.spawn()?;
if !unix_socket_path.is_empty() && timeout_secs > 0 { if !unix_socket_path.is_empty() && timeout_secs > 0 {
wait_for_path_to_exist(logger, unix_socket_path, timeout_secs).await?; wait_for_path_to_exist(logger, unix_socket_path, timeout_secs).await?;
} }

View File

@ -57,7 +57,7 @@ use rustjail::process::ProcessOperations;
#[cfg(target_arch = "s390x")] #[cfg(target_arch = "s390x")]
use crate::ccw; use crate::ccw;
use crate::cdh; use crate::confidential_data_hub::image::KATA_IMAGE_WORK_DIR;
use crate::device::block_device_handler::get_virtio_blk_pci_device_name; use crate::device::block_device_handler::get_virtio_blk_pci_device_name;
#[cfg(target_arch = "s390x")] #[cfg(target_arch = "s390x")]
use crate::device::network_device_handler::wait_for_ccw_net_interface; use crate::device::network_device_handler::wait_for_ccw_net_interface;
@ -65,9 +65,6 @@ use crate::device::network_device_handler::wait_for_ccw_net_interface;
use crate::device::network_device_handler::wait_for_pci_net_interface; use crate::device::network_device_handler::wait_for_pci_net_interface;
use crate::device::{add_devices, handle_cdi_devices, update_env_pci}; use crate::device::{add_devices, handle_cdi_devices, update_env_pci};
use crate::features::get_build_features; use crate::features::get_build_features;
#[cfg(feature = "guest-pull")]
use crate::image::KATA_IMAGE_WORK_DIR;
use crate::linux_abi::*;
use crate::metrics::get_metrics; use crate::metrics::get_metrics;
use crate::mount::baremount; use crate::mount::baremount;
use crate::namespace::{NSTYPEIPC, NSTYPEPID, NSTYPEUTS}; use crate::namespace::{NSTYPEIPC, NSTYPEPID, NSTYPEUTS};
@ -80,6 +77,7 @@ use crate::storage::{add_storages, update_ephemeral_mounts, STORAGE_HANDLERS};
use crate::util; use crate::util;
use crate::version::{AGENT_VERSION, API_VERSION}; use crate::version::{AGENT_VERSION, API_VERSION};
use crate::AGENT_CONFIG; use crate::AGENT_CONFIG;
use crate::{confidential_data_hub, linux_abi::*};
use crate::trace_rpc_call; use crate::trace_rpc_call;
use crate::tracer::extract_carrier_from_ttrpc; use crate::tracer::extract_carrier_from_ttrpc;
@ -87,9 +85,6 @@ use crate::tracer::extract_carrier_from_ttrpc;
#[cfg(feature = "agent-policy")] #[cfg(feature = "agent-policy")]
use crate::policy::{do_set_policy, is_allowed}; use crate::policy::{do_set_policy, is_allowed};
#[cfg(feature = "guest-pull")]
use crate::image;
use opentelemetry::global; use opentelemetry::global;
use tracing::span; use tracing::span;
use tracing_opentelemetry::OpenTelemetrySpanExt; use tracing_opentelemetry::OpenTelemetrySpanExt;
@ -112,7 +107,6 @@ use kata_types::k8s;
pub const CONTAINER_BASE: &str = "/run/kata-containers"; pub const CONTAINER_BASE: &str = "/run/kata-containers";
const MODPROBE_PATH: &str = "/sbin/modprobe"; const MODPROBE_PATH: &str = "/sbin/modprobe";
#[cfg(feature = "guest-pull")]
const TRUSTED_IMAGE_STORAGE_DEVICE: &str = "/dev/trusted_store"; const TRUSTED_IMAGE_STORAGE_DEVICE: &str = "/dev/trusted_store";
/// the iptables seriers binaries could appear either in /sbin /// the iptables seriers binaries could appear either in /sbin
/// or /usr/sbin, we need to check both of them /// or /usr/sbin, we need to check both of them
@ -242,7 +236,6 @@ impl AgentService {
handle_cdi_devices(&sl(), &mut oci, "/var/run/cdi", AGENT_CONFIG.cdi_timeout).await?; handle_cdi_devices(&sl(), &mut oci, "/var/run/cdi", AGENT_CONFIG.cdi_timeout).await?;
// Handle trusted storage configuration before mounting any storage // Handle trusted storage configuration before mounting any storage
#[cfg(feature = "guest-pull")]
cdh_handler_trusted_storage(&mut oci) cdh_handler_trusted_storage(&mut oci)
.await .await
.map_err(|e| anyhow!("failed to handle trusted storage: {}", e))?; .map_err(|e| anyhow!("failed to handle trusted storage: {}", e))?;
@ -319,20 +312,14 @@ impl AgentService {
let pipe_size = AGENT_CONFIG.container_pipe_size; let pipe_size = AGENT_CONFIG.container_pipe_size;
let p = if let Some(p) = oci.process() { let Some(p) = oci.process() else {
#[cfg(feature = "guest-pull")]
{
let new_p = image::get_process(p, &oci, req.storages.clone())?;
Process::new(&sl(), &new_p, cid.as_str(), true, pipe_size, proc_io)?
}
#[cfg(not(feature = "guest-pull"))]
Process::new(&sl(), p, cid.as_str(), true, pipe_size, proc_io)?
} else {
info!(sl(), "no process configurations!"); info!(sl(), "no process configurations!");
return Err(anyhow!(nix::Error::EINVAL)); return Err(anyhow!(nix::Error::EINVAL));
}; };
let new_p = confidential_data_hub::image::get_process(p, &oci, req.storages.clone())?;
let p = Process::new(&sl(), &new_p, cid.as_str(), true, pipe_size, proc_io)?;
// if starting container failed, we will do some rollback work // if starting container failed, we will do some rollback work
// to ensure no resources are leaked. // to ensure no resources are leaked.
if let Err(err) = ctr.start(p).await { if let Err(err) = ctr.start(p).await {
@ -1332,9 +1319,6 @@ impl agent_ttrpc::AgentService for AgentService {
} }
} }
#[cfg(feature = "guest-pull")]
image::init_image_service().await.map_ttrpc_err(same)?;
Ok(Empty::new()) Ok(Empty::new())
} }
@ -2274,9 +2258,8 @@ fn is_sealed_secret_path(source_path: &str) -> bool {
.any(|suffix| source_path.ends_with(suffix)) .any(|suffix| source_path.ends_with(suffix))
} }
#[cfg(feature = "guest-pull")]
async fn cdh_handler_trusted_storage(oci: &mut Spec) -> Result<()> { async fn cdh_handler_trusted_storage(oci: &mut Spec) -> Result<()> {
if !cdh::is_cdh_client_initialized().await { if !confidential_data_hub::is_cdh_client_initialized() {
return Ok(()); return Ok(());
} }
let linux = oci let linux = oci
@ -2301,7 +2284,13 @@ async fn cdh_handler_trusted_storage(oci: &mut Spec) -> Result<()> {
("encryptType".to_string(), "LUKS".to_string()), ("encryptType".to_string(), "LUKS".to_string()),
("dataIntegrity".to_string(), secure_storage_integrity), ("dataIntegrity".to_string(), secure_storage_integrity),
]); ]);
cdh::secure_mount("BlockDevice", &options, vec![], KATA_IMAGE_WORK_DIR).await?; confidential_data_hub::secure_mount(
"BlockDevice",
&options,
vec![],
KATA_IMAGE_WORK_DIR,
)
.await?;
break; break;
} }
} }
@ -2310,7 +2299,7 @@ async fn cdh_handler_trusted_storage(oci: &mut Spec) -> Result<()> {
} }
async fn cdh_handler_sealed_secrets(oci: &mut Spec) -> Result<()> { async fn cdh_handler_sealed_secrets(oci: &mut Spec) -> Result<()> {
if !cdh::is_cdh_client_initialized().await { if !confidential_data_hub::is_cdh_client_initialized() {
return Ok(()); return Ok(());
} }
let process = oci let process = oci
@ -2319,7 +2308,7 @@ async fn cdh_handler_sealed_secrets(oci: &mut Spec) -> Result<()> {
.ok_or_else(|| anyhow!("Spec didn't contain process field"))?; .ok_or_else(|| anyhow!("Spec didn't contain process field"))?;
if let Some(envs) = process.env_mut().as_mut() { if let Some(envs) = process.env_mut().as_mut() {
for env in envs.iter_mut() { for env in envs.iter_mut() {
match cdh::unseal_env(env).await { match confidential_data_hub::unseal_env(env).await {
Ok(unsealed_env) => *env = unsealed_env.to_string(), Ok(unsealed_env) => *env = unsealed_env.to_string(),
Err(e) => { Err(e) => {
warn!(sl(), "Failed to unseal secret: {}", e) warn!(sl(), "Failed to unseal secret: {}", e)
@ -2357,7 +2346,7 @@ async fn cdh_handler_sealed_secrets(oci: &mut Spec) -> Result<()> {
// But currently there is no quick way to determine which volume-mount is referring // But currently there is no quick way to determine which volume-mount is referring
// to a sealed secret without reading the file. // to a sealed secret without reading the file.
// And relying on file naming heuristic is inflexible. So we are going with this approach. // And relying on file naming heuristic is inflexible. So we are going with this approach.
if let Err(e) = cdh::unseal_file(source_path).await { if let Err(e) = confidential_data_hub::unseal_file(source_path).await {
warn!( warn!(
sl(), sl(),
"Failed to unseal file: {:?}, Error: {:?}", source_path, e "Failed to unseal file: {:?}, Error: {:?}", source_path, e

View File

@ -4,12 +4,15 @@
// //
use super::new_device; use super::new_device;
use crate::image; use crate::confidential_data_hub;
use crate::confidential_data_hub::image::{is_sandbox, unpack_pause_image};
use crate::rpc::CONTAINER_BASE;
use crate::storage::{StorageContext, StorageHandler}; use crate::storage::{StorageContext, StorageHandler};
use anyhow::{anyhow, Result}; use anyhow::{anyhow, Result};
use kata_types::mount::KATA_VIRTUAL_VOLUME_IMAGE_GUEST_PULL; use kata_types::mount::KATA_VIRTUAL_VOLUME_IMAGE_GUEST_PULL;
use kata_types::mount::{ImagePullVolume, StorageDevice}; use kata_types::mount::{ImagePullVolume, StorageDevice};
use protocols::agent::Storage; use protocols::agent::Storage;
use safe_path::scoped_join;
use std::sync::Arc; use std::sync::Arc;
use tracing::instrument; use tracing::instrument;
@ -53,7 +56,35 @@ impl StorageHandler for ImagePullHandler {
.cid .cid
.clone() .clone()
.ok_or_else(|| anyhow!("failed to get container id"))?; .ok_or_else(|| anyhow!("failed to get container id"))?;
let bundle_path = image::pull_image(image_name, &cid, &image_pull_volume.metadata).await?;
info!(
ctx.logger,
"image metadata: {:?}", image_pull_volume.metadata
);
if is_sandbox(&image_pull_volume.metadata) {
let mount_path = unpack_pause_image(&cid)?;
return new_device(mount_path);
}
// generated bundles with rootfs and config.json will store under CONTAINER_BASE/cid/images.
let bundle_path = scoped_join(CONTAINER_BASE, &cid)?;
let bundle_path = match confidential_data_hub::pull_image(image_name, bundle_path).await {
Ok(path) => {
info!(
ctx.logger,
"pull and unpack image {image_name}, cid: {cid} succeeded."
);
path
}
Err(e) => {
error!(
ctx.logger,
"pull and unpack image {image_name}, cid: {cid} failed with {:?}.",
e.to_string()
);
return Err(e);
}
};
new_device(bundle_path) new_device(bundle_path)
} }

View File

@ -24,7 +24,6 @@ use self::bind_watcher_handler::BindWatcherHandler;
use self::block_handler::{PmemHandler, ScsiHandler, VirtioBlkMmioHandler, VirtioBlkPciHandler}; use self::block_handler::{PmemHandler, ScsiHandler, VirtioBlkMmioHandler, VirtioBlkPciHandler};
use self::ephemeral_handler::EphemeralHandler; use self::ephemeral_handler::EphemeralHandler;
use self::fs_handler::{OverlayfsHandler, Virtio9pHandler, VirtioFsHandler}; use self::fs_handler::{OverlayfsHandler, Virtio9pHandler, VirtioFsHandler};
#[cfg(feature = "guest-pull")]
use self::image_pull_handler::ImagePullHandler; use self::image_pull_handler::ImagePullHandler;
use self::local_handler::LocalHandler; use self::local_handler::LocalHandler;
use crate::mount::{baremount, is_mounted, remove_mounts}; use crate::mount::{baremount, is_mounted, remove_mounts};
@ -36,7 +35,6 @@ mod bind_watcher_handler;
mod block_handler; mod block_handler;
mod ephemeral_handler; mod ephemeral_handler;
mod fs_handler; mod fs_handler;
#[cfg(feature = "guest-pull")]
mod image_pull_handler; mod image_pull_handler;
mod local_handler; mod local_handler;
@ -148,7 +146,6 @@ lazy_static! {
Arc::new(BindWatcherHandler {}), Arc::new(BindWatcherHandler {}),
#[cfg(target_arch = "s390x")] #[cfg(target_arch = "s390x")]
Arc::new(self::block_handler::VirtioBlkCcwHandler {}), Arc::new(self::block_handler::VirtioBlkCcwHandler {}),
#[cfg(feature = "guest-pull")]
Arc::new(ImagePullHandler {}), Arc::new(ImagePullHandler {}),
]; ];

View File

@ -28,6 +28,21 @@ message SecureMountResponse {
string mount_path = 1; string mount_path = 1;
} }
message ImagePullRequest {
// - `image_url`: The reference of the image to pull
string image_url = 1;
// - `bundle_path`: The path to store the OCI bundle. This path
// should be created by client, and initially empty. After the RPC is
// called, a mounted `rootfs` directory under the this path. Note
// that this path is CDH's root filesystem, not the caller's root filesystem.
// However, usually the caller (kata-agent) and the server (CDH) runs on the same
// root, so it's safe to use an absolute path of kata-agent.
string bundle_path = 2;
}
message ImagePullResponse {}
service SealedSecretService { service SealedSecretService {
rpc UnsealSecret(UnsealSecretInput) returns (UnsealSecretOutput) {}; rpc UnsealSecret(UnsealSecretInput) returns (UnsealSecretOutput) {};
} }
@ -47,3 +62,9 @@ message GetResourceResponse {
service GetResourceService { service GetResourceService {
rpc GetResource(GetResourceRequest) returns (GetResourceResponse) {}; rpc GetResource(GetResourceRequest) returns (GetResourceResponse) {};
} }
// ImagePullService is used to pull images from a remote registry
// and mount the resulting root filesystems.
service ImagePullService {
rpc PullImage(ImagePullRequest) returns (ImagePullResponse) {};
}

View File

@ -87,7 +87,7 @@ function setup_kbs_credentials() {
echo "Pod ${kata_pod}: $(cat ${kata_pod})" echo "Pod ${kata_pod}: $(cat ${kata_pod})"
assert_pod_fail "${kata_pod}" assert_pod_fail "${kata_pod}"
assert_logs_contain "${node}" kata "${node_start_time}" "failed to pull image" assert_logs_contain "${node}" kata "${node_start_time}" "Not authorized"
} }
@test "Test that creating a container from an authenticated image, with no credentials fails" { @test "Test that creating a container from an authenticated image, with no credentials fails" {
@ -100,7 +100,7 @@ function setup_kbs_credentials() {
echo "Pod ${kata_pod}: $(cat ${kata_pod})" echo "Pod ${kata_pod}: $(cat ${kata_pod})"
assert_pod_fail "${kata_pod}" assert_pod_fail "${kata_pod}"
assert_logs_contain "${node}" kata "${node_start_time}" "failed to pull image" assert_logs_contain "${node}" kata "${node_start_time}" "Not authorized"
} }
teardown() { teardown() {

View File

@ -52,8 +52,7 @@ function setup_kbs_decryption_key() {
echo "Pod ${kata_pod}: $(cat ${kata_pod})" echo "Pod ${kata_pod}: $(cat ${kata_pod})"
assert_pod_fail "${kata_pod}" assert_pod_fail "${kata_pod}"
assert_logs_contain "${node}" kata "${node_start_time}" 'decrypt image (unwrap key) failed' assert_logs_contain "${node}" kata "${node_start_time}" 'Failed to decrypt the image layer, please ensure that the decryption key is placed and correct'
assert_logs_contain "${node}" kata "${node_start_time}" 'kms interface when get KEK failed'
} }
@ -80,7 +79,7 @@ function setup_kbs_decryption_key() {
echo "Pod ${kata_pod}: $(cat ${kata_pod})" echo "Pod ${kata_pod}: $(cat ${kata_pod})"
assert_pod_fail "${kata_pod}" assert_pod_fail "${kata_pod}"
assert_logs_contain "${node}" kata "${node_start_time}" 'decrypt image (unwrap key) failed' assert_logs_contain "${node}" kata "${node_start_time}" 'Failed to decrypt the image layer, please ensure that the decryption key is placed and correct'
} }
teardown() { teardown() {

View File

@ -97,7 +97,7 @@ EOF
echo "Pod ${kata_pod}: $(cat ${kata_pod})" echo "Pod ${kata_pod}: $(cat ${kata_pod})"
assert_pod_fail "${kata_pod}" assert_pod_fail "${kata_pod}"
assert_logs_contain "${node}" kata "${node_start_time}" "failed to pull image" assert_logs_contain "${node}" kata "${node_start_time}" "Image policy rejected: Denied by policy"
} }
@test "Create a pod from a signed image, on a 'restricted registry' is successful" { @test "Create a pod from a signed image, on a 'restricted registry' is successful" {
@ -123,7 +123,7 @@ EOF
echo "Pod ${kata_pod}: $(cat ${kata_pod})" echo "Pod ${kata_pod}: $(cat ${kata_pod})"
assert_pod_fail "${kata_pod}" assert_pod_fail "${kata_pod}"
assert_logs_contain "${node}" kata "${node_start_time}" "failed to pull image" assert_logs_contain "${node}" kata "${node_start_time}" "Image policy rejected: Denied by policy"
} }
@test "Create a pod from an unsigned image, on a 'restricted registry' works if policy files isn't set" { @test "Create a pod from an unsigned image, on a 'restricted registry' works if policy files isn't set" {

View File

@ -98,10 +98,6 @@ algorithm = "sha256"
[data] [data]
"aa.toml" = ''' "aa.toml" = '''
[token_configs] [token_configs]
[token_configs.coco_as]
# TODO: we should fix this on AA side to set this a default value if not set.
url = "${CC_KBS_ADDRESS}"
[token_configs.kbs] [token_configs.kbs]
url = "${CC_KBS_ADDRESS}" url = "${CC_KBS_ADDRESS}"
''' '''
@ -165,7 +161,7 @@ EOF
echo "Pod ${kata_pod}: $(cat ${kata_pod})" echo "Pod ${kata_pod}: $(cat ${kata_pod})"
assert_pod_fail "${kata_pod}" assert_pod_fail "${kata_pod}"
assert_logs_contain "${node}" kata "${node_start_time}" "failed to pull image" assert_logs_contain "${node}" kata "${node_start_time}" "Image policy rejected: Denied by policy"
} }
@test "Test that creating a container from an rejected image not configured by initdata, fails according to CDH error" { @test "Test that creating a container from an rejected image not configured by initdata, fails according to CDH error" {

View File

@ -17,8 +17,6 @@ RUST_VERSION="null"
AGENT_BIN=${AGENT_BIN:-kata-agent} AGENT_BIN=${AGENT_BIN:-kata-agent}
AGENT_INIT=${AGENT_INIT:-no} AGENT_INIT=${AGENT_INIT:-no}
MEASURED_ROOTFS=${MEASURED_ROOTFS:-no} MEASURED_ROOTFS=${MEASURED_ROOTFS:-no}
# The kata agent enables guest-pull feature.
PULL_TYPE=${PULL_TYPE:-default}
KERNEL_MODULES_DIR=${KERNEL_MODULES_DIR:-""} KERNEL_MODULES_DIR=${KERNEL_MODULES_DIR:-""}
OSBUILDER_VERSION="unknown" OSBUILDER_VERSION="unknown"
DOCKER_RUNTIME=${DOCKER_RUNTIME:-runc} DOCKER_RUNTIME=${DOCKER_RUNTIME:-runc}
@ -747,7 +745,7 @@ EOF
git checkout "${AGENT_VERSION}" && OK "git checkout successful" || die "checkout agent ${AGENT_VERSION} failed!" git checkout "${AGENT_VERSION}" && OK "git checkout successful" || die "checkout agent ${AGENT_VERSION} failed!"
fi fi
make clean make clean
make LIBC=${LIBC} INIT=${AGENT_INIT} SECCOMP=${SECCOMP} AGENT_POLICY=${AGENT_POLICY} PULL_TYPE=${PULL_TYPE} make LIBC=${LIBC} INIT=${AGENT_INIT} SECCOMP=${SECCOMP} AGENT_POLICY=${AGENT_POLICY}
make install DESTDIR="${ROOTFS_DIR}" LIBC=${LIBC} INIT=${AGENT_INIT} make install DESTDIR="${ROOTFS_DIR}" LIBC=${LIBC} INIT=${AGENT_INIT}
if [ "${SECCOMP}" == "yes" ]; then if [ "${SECCOMP}" == "yes" ]; then
rm -rf "${libseccomp_install_dir}" "${gperf_install_dir}" rm -rf "${libseccomp_install_dir}" "${gperf_install_dir}"

View File

@ -47,7 +47,6 @@ build_initrd() {
AGENT_TARBALL="${AGENT_TARBALL}" \ AGENT_TARBALL="${AGENT_TARBALL}" \
AGENT_INIT="${AGENT_INIT:-no}" \ AGENT_INIT="${AGENT_INIT:-no}" \
AGENT_POLICY="${AGENT_POLICY:-}" \ AGENT_POLICY="${AGENT_POLICY:-}" \
PULL_TYPE="${PULL_TYPE:-default}" \
COCO_GUEST_COMPONENTS_TARBALL="${COCO_GUEST_COMPONENTS_TARBALL:-}" \ COCO_GUEST_COMPONENTS_TARBALL="${COCO_GUEST_COMPONENTS_TARBALL:-}" \
PAUSE_IMAGE_TARBALL="${PAUSE_IMAGE_TARBALL:-}" \ PAUSE_IMAGE_TARBALL="${PAUSE_IMAGE_TARBALL:-}" \
GUEST_HOOKS_TARBALL="${GUEST_HOOKS_TARBALL}" GUEST_HOOKS_TARBALL="${GUEST_HOOKS_TARBALL}"
@ -77,7 +76,6 @@ build_image() {
ROOTFS_BUILD_DEST="${builddir}/rootfs-image" \ ROOTFS_BUILD_DEST="${builddir}/rootfs-image" \
AGENT_TARBALL="${AGENT_TARBALL}" \ AGENT_TARBALL="${AGENT_TARBALL}" \
AGENT_POLICY="${AGENT_POLICY:-}" \ AGENT_POLICY="${AGENT_POLICY:-}" \
PULL_TYPE="${PULL_TYPE:-default}" \
COCO_GUEST_COMPONENTS_TARBALL="${COCO_GUEST_COMPONENTS_TARBALL:-}" \ COCO_GUEST_COMPONENTS_TARBALL="${COCO_GUEST_COMPONENTS_TARBALL:-}" \
PAUSE_IMAGE_TARBALL="${PAUSE_IMAGE_TARBALL:-}" \ PAUSE_IMAGE_TARBALL="${PAUSE_IMAGE_TARBALL:-}" \
GUEST_HOOKS_TARBALL="${GUEST_HOOKS_TARBALL}" GUEST_HOOKS_TARBALL="${GUEST_HOOKS_TARBALL}"

View File

@ -100,7 +100,6 @@ TOOLS_CONTAINER_BUILDER="${TOOLS_CONTAINER_BUILDER:-}"
VIRTIOFSD_CONTAINER_BUILDER="${VIRTIOFSD_CONTAINER_BUILDER:-}" VIRTIOFSD_CONTAINER_BUILDER="${VIRTIOFSD_CONTAINER_BUILDER:-}"
AGENT_INIT="${AGENT_INIT:-no}" AGENT_INIT="${AGENT_INIT:-no}"
MEASURED_ROOTFS="${MEASURED_ROOTFS:-}" MEASURED_ROOTFS="${MEASURED_ROOTFS:-}"
PULL_TYPE="${PULL_TYPE:-guest-pull}"
USE_CACHE="${USE_CACHE:-}" USE_CACHE="${USE_CACHE:-}"
BUSYBOX_CONF_FILE=${BUSYBOX_CONF_FILE:-} BUSYBOX_CONF_FILE=${BUSYBOX_CONF_FILE:-}
NVIDIA_GPU_STACK="${NVIDIA_GPU_STACK:-}" NVIDIA_GPU_STACK="${NVIDIA_GPU_STACK:-}"
@ -140,7 +139,6 @@ docker run \
--env VIRTIOFSD_CONTAINER_BUILDER="${VIRTIOFSD_CONTAINER_BUILDER}" \ --env VIRTIOFSD_CONTAINER_BUILDER="${VIRTIOFSD_CONTAINER_BUILDER}" \
--env AGENT_INIT="${AGENT_INIT}" \ --env AGENT_INIT="${AGENT_INIT}" \
--env MEASURED_ROOTFS="${MEASURED_ROOTFS}" \ --env MEASURED_ROOTFS="${MEASURED_ROOTFS}" \
--env PULL_TYPE="${PULL_TYPE}" \
--env USE_CACHE="${USE_CACHE}" \ --env USE_CACHE="${USE_CACHE}" \
--env BUSYBOX_CONF_FILE="${BUSYBOX_CONF_FILE}" \ --env BUSYBOX_CONF_FILE="${BUSYBOX_CONF_FILE}" \
--env NVIDIA_GPU_STACK="${NVIDIA_GPU_STACK}" \ --env NVIDIA_GPU_STACK="${NVIDIA_GPU_STACK}" \

View File

@ -43,7 +43,6 @@ readonly se_image_builder="${repo_root_dir}/tools/packaging/guest-image/build_se
ARCH=${ARCH:-$(uname -m)} ARCH=${ARCH:-$(uname -m)}
BUSYBOX_CONF_FILE="${BUSYBOX_CONF_FILE:-}" BUSYBOX_CONF_FILE="${BUSYBOX_CONF_FILE:-}"
MEASURED_ROOTFS=${MEASURED_ROOTFS:-no} MEASURED_ROOTFS=${MEASURED_ROOTFS:-no}
PULL_TYPE=${PULL_TYPE:-guest-pull}
USE_CACHE="${USE_CACHE:-"yes"}" USE_CACHE="${USE_CACHE:-"yes"}"
ARTEFACT_REGISTRY="${ARTEFACT_REGISTRY:-ghcr.io}" ARTEFACT_REGISTRY="${ARTEFACT_REGISTRY:-ghcr.io}"
ARTEFACT_REPOSITORY="${ARTEFACT_REPOSITORY:-kata-containers}" ARTEFACT_REPOSITORY="${ARTEFACT_REPOSITORY:-kata-containers}"
@ -430,7 +429,6 @@ install_image_confidential() {
else else
export MEASURED_ROOTFS=yes export MEASURED_ROOTFS=yes
fi fi
export PULL_TYPE=default
install_image "confidential" install_image "confidential"
} }
@ -529,7 +527,6 @@ install_initrd() {
#Install guest initrd for confidential guests #Install guest initrd for confidential guests
install_initrd_confidential() { install_initrd_confidential() {
export MEASURED_ROOTFS=no export MEASURED_ROOTFS=no
export PULL_TYPE=default
install_initrd "confidential" install_initrd "confidential"
} }
@ -997,7 +994,7 @@ install_agent() {
export GPERF_URL="$(get_from_kata_deps ".externals.gperf.url")" export GPERF_URL="$(get_from_kata_deps ".externals.gperf.url")"
info "build static agent" info "build static agent"
DESTDIR="${destdir}" AGENT_POLICY="${AGENT_POLICY}" PULL_TYPE=${PULL_TYPE} "${agent_builder}" DESTDIR="${destdir}" AGENT_POLICY="${AGENT_POLICY}" "${agent_builder}"
} }
install_coco_guest_components() { install_coco_guest_components() {

View File

@ -18,8 +18,8 @@ build_agent_from_source() {
/usr/bin/install_libseccomp.sh /opt /opt /usr/bin/install_libseccomp.sh /opt /opt
cd src/agent cd src/agent
DESTDIR=${DESTDIR} AGENT_POLICY=${AGENT_POLICY} PULL_TYPE=${PULL_TYPE} make DESTDIR=${DESTDIR} AGENT_POLICY=${AGENT_POLICY} make
DESTDIR=${DESTDIR} AGENT_POLICY=${AGENT_POLICY} PULL_TYPE=${PULL_TYPE} make install DESTDIR=${DESTDIR} AGENT_POLICY=${AGENT_POLICY} make install
} }
build_agent_from_source "$@" build_agent_from_source "$@"

View File

@ -26,7 +26,6 @@ docker pull ${container_image} || \
docker run --rm -i -v "${repo_root_dir}:${repo_root_dir}" \ docker run --rm -i -v "${repo_root_dir}:${repo_root_dir}" \
--env DESTDIR=${DESTDIR} \ --env DESTDIR=${DESTDIR} \
--env AGENT_POLICY=${AGENT_POLICY:-no} \ --env AGENT_POLICY=${AGENT_POLICY:-no} \
--env PULL_TYPE=${PULL_TYPE:-default} \
--env LIBSECCOMP_VERSION=${LIBSECCOMP_VERSION} \ --env LIBSECCOMP_VERSION=${LIBSECCOMP_VERSION} \
--env LIBSECCOMP_URL=${LIBSECCOMP_URL} \ --env LIBSECCOMP_URL=${LIBSECCOMP_URL} \
--env GPERF_VERSION=${GPERF_VERSION} \ --env GPERF_VERSION=${GPERF_VERSION} \

View File

@ -3,7 +3,7 @@
# #
# SPDX-License-Identifier: Apache-2.0 # SPDX-License-Identifier: Apache-2.0
FROM ubuntu:22.04 FROM ubuntu:24.04
ARG RUST_TOOLCHAIN ARG RUST_TOOLCHAIN
ENV DEBIAN_FRONTEND=noninteractive ENV DEBIAN_FRONTEND=noninteractive
@ -28,6 +28,7 @@ RUN apt-get update && \
libssl-dev \ libssl-dev \
libtss2-dev \ libtss2-dev \
make \ make \
cmake \
musl-tools \ musl-tools \
openssl \ openssl \
perl \ perl \

View File

@ -35,6 +35,7 @@ build_coco_guest_components_from_source() {
DESTDIR="${DESTDIR}/usr/local/bin" TEE_PLATFORM=${TEE_PLATFORM} make install DESTDIR="${DESTDIR}/usr/local/bin" TEE_PLATFORM=${TEE_PLATFORM} make install
install -D -m0755 "confidential-data-hub/hub/src/storage/scripts/luks-encrypt-storage" "${DESTDIR}/usr/local/bin/luks-encrypt-storage" install -D -m0755 "confidential-data-hub/hub/src/storage/scripts/luks-encrypt-storage" "${DESTDIR}/usr/local/bin/luks-encrypt-storage"
install -D -m0644 "confidential-data-hub/hub/src/image/ocicrypt_config.json" "${DESTDIR}/etc/ocicrypt_config.json"
popd popd
} }

View File

@ -143,7 +143,7 @@ assets:
version: "jammy" # 22.04 lTS version: "jammy" # 22.04 lTS
confidential: confidential:
name: "ubuntu" name: "ubuntu"
version: "oracular" # 24.10 version: "noble" # 24.04 LTS
mariner: mariner:
name: "cbl-mariner" name: "cbl-mariner"
version: "3.0" version: "3.0"
@ -185,7 +185,7 @@ assets:
version: "3.18" version: "3.18"
confidential: confidential:
name: "ubuntu" name: "ubuntu"
version: "jammy" # 22.04 LTS version: "noble" # 24.04 LTS
nvidia-gpu: nvidia-gpu:
name: "ubuntu" name: "ubuntu"
version: "jammy" # 22.04 LTS version: "jammy" # 22.04 LTS
@ -233,18 +233,18 @@ externals:
coco-guest-components: coco-guest-components:
description: "Provides attested key unwrapping for image decryption" description: "Provides attested key unwrapping for image decryption"
url: "https://github.com/confidential-containers/guest-components/" url: "https://github.com/confidential-containers/guest-components/"
version: "0a06ef241190780840fbb0542e51b198f1f72b0b" version: "028978dbaef2510ea92bc5038928f1d70c8aaad6"
toolchain: "1.80.0" toolchain: "1.80.0"
coco-trustee: coco-trustee:
description: "Provides attestation and secret delivery components" description: "Provides attestation and secret delivery components"
url: "https://github.com/confidential-containers/trustee" url: "https://github.com/confidential-containers/trustee"
version: "d9eb5e0cb0aca97abe35b58908e061850ff60a51" version: "a333fa27a7ce538413bad3b537ffbeacf4a349d1"
# image / ita_image and image_tag / ita_image_tag must be in sync # image / ita_image and image_tag / ita_image_tag must be in sync
image: "ghcr.io/confidential-containers/staged-images/kbs" image: "ghcr.io/confidential-containers/staged-images/kbs"
image_tag: "d9eb5e0cb0aca97abe35b58908e061850ff60a51" image_tag: "a333fa27a7ce538413bad3b537ffbeacf4a349d1"
ita_image: "ghcr.io/confidential-containers/staged-images/kbs-ita-as" ita_image: "ghcr.io/confidential-containers/staged-images/kbs-ita-as"
ita_image_tag: "d9eb5e0cb0aca97abe35b58908e061850ff60a51-x86_64" ita_image_tag: "a333fa27a7ce538413bad3b537ffbeacf4a349d1-x86_64"
toolchain: "1.80.0" toolchain: "1.80.0"
crio: crio: