mirror of
https://github.com/kata-containers/kata-containers.git
synced 2025-07-05 11:36:56 +00:00
Merge pull request #11197 from Xynnn007/move-image-pull
Move image pull abilities to CDH
This commit is contained in:
Alex Lyn
GitHub
commit
a966d1be50
Binary file not shown.
|
Before Width: | Height: | Size: 61 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 122 KiB |
@ -49,13 +49,58 @@ Pull the container image directly from the guest VM using `nydus snapshotter` ba
|
||||
#### Architecture
|
||||
|
||||
The following diagram provides an overview of the architecture for pulling image in the guest with key components.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Kubelet[kubelet]--> |1\. Pull image request & metadata|Containerd
|
||||
Containerd-->|2\. Pull image metadata| E
|
||||
Containerd-->Snapshotter[Nydus Snapshotter]
|
||||
Snapshotter-->|3\. Pack image info| Containerd
|
||||
Containerd-->Runtime[Kata Runtime]
|
||||
Runtime-->Hypervisor
|
||||
Hypervisor-->TEE
|
||||
Runtime-->|4\. Pass image info to VM| Agent
|
||||
CDH1-->|6\. Pull image with image info|E[Container Images Registry]
|
||||
subgraph TEE [Virtual Machine]
|
||||
Images[Container Images]-->|7\. Prepare container rootfs|H[Container]
|
||||
|
||||
subgraph CDH [Confidential Data Hub]
|
||||
CDH1[Image Mgmt]
|
||||
end
|
||||
|
||||
CDH-->Images
|
||||
Agent[Kata Agent]-->|5\. Call image pull RPC|CDH
|
||||
end
|
||||
```
|
||||
|
||||
#### Sequence diagrams
|
||||
|
||||
The following sequence diagram depicted below offers a detailed overview of the messages/calls exchanged to pull an unencrypted unsigned image from an unauthenticated container registry. This involves the kata-runtime, kata-agent, and the guest-components’ image-rs to use the guest pull mechanism.
|
||||
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
par Hosts Side
|
||||
Containerd/Kubelet->>runtime.kata_agent: createContainer(ctx,sandbox,c)
|
||||
runtime.kata_agent->>runtime.fs_share_linux: ShareRootFilesystem(ctx,c)
|
||||
runtime.fs_share_linux->>runtime.kata_agent: handleVirtualVolumeStorageObject(c,...,KataVolumeType)
|
||||
runtime.kata_agent->>runtime.kata_agent: handleImageGuestPullBlockVolume(c,virtVolume,vol)
|
||||
runtime.kata_agent->>runtime.fs_share_linux: ret:storage
|
||||
runtime.fs_share_linux->>runtime.kata_agent: ret:sharedFile
|
||||
and Guest Side
|
||||
runtime.kata_agent->>agent.rpc: CreateContainerRequest(cid,...,storages,...,oci,...)
|
||||
agent.rpc->>agent.storage: add_storage(storages...)
|
||||
agent.storage->>agent.storage: StorageHandler.handler(driver)
|
||||
agent.storage->>agent.storage.StorageHandler.ImagePullHandler: create_device(storage)
|
||||
agent.storage.StorageHandler.ImagePullHandler->>agent.confidential_data_hub: pull_image(img,cid,img_metadata)
|
||||
agent.confidential_data_hub->>Confidential Data Hub: pull_image(img,bundle_path)
|
||||
Confidential Data Hub->>agent.confidential_data_hub: ret
|
||||
agent.confidential_data_hub->>agent.storage.StorageHandler.ImagePullHandler: ret: bundle_path
|
||||
agent.storage.StorageHandler.ImagePullHandler->>agent.storage: ret: device
|
||||
agent.storage->>agent.rpc: ret: mount_list
|
||||
and Return
|
||||
agent.rpc->>runtime.kata_agent: ret: ok
|
||||
runtime.kata_agent->>Containerd/Kubelet: ret: ok
|
||||
end
|
||||
```
|
||||
|
||||
First and foremost, the guest pull code path is only activated when `nydus snapshotter` requires the handling of a volume which type is `image_guest_pull`, as can be seen on the message below:
|
||||
```json
|
||||
@ -108,10 +153,10 @@ Below is an example of storage information packaged in the message sent to the k
|
||||
```
|
||||
Next, the kata-agent's RPC module will handle the create container request which, among other things, involves adding storages to the sandbox. The storage module contains implementations of `StorageHandler` interface for various storage types, being the `ImagePullHandler` in charge of handling the storage object for the container image (the storage manager instantiates the handler based on the value of the "driver").
|
||||
|
||||
`ImagePullHandler` delegates the image pulling operation to the `ImageService.pull_image()` that is going to create the image's bundle directory on the guest filesystem and, in turn, class the image-rs to in fact fetch and uncompress the image's bundle.
|
||||
`ImagePullHandler` delegates the image pulling operation to the `confidential_data_hub.pull_image()` that is going to create the image's bundle directory on the guest filesystem and, in turn, the `ImagePullService` of Confidential Data Hub to fetch, uncompress and mount the image's rootfs.
|
||||
|
||||
> **Notes:**
|
||||
> In this flow, `ImageService.pull_image()` parses the image metadata, looking for either the `io.kubernetes.cri.container-type: sandbox` or `io.kubernetes.cri-o.ContainerType: sandbox` (CRI-IO case) annotation, then it never calls the `image-rs.pull_image()` because the pause image is expected to already be inside the guest's filesystem, so instead `ImageService.unpack_pause_image()` is called.
|
||||
> In this flow, `confidential_data_hub.pull_image()` parses the image metadata, looking for either the `io.kubernetes.cri.container-type: sandbox` or `io.kubernetes.cri-o.ContainerType: sandbox` (CRI-IO case) annotation, then it never calls the `pull_image()` RPC of Confidential Data Hub because the pause image is expected to already be inside the guest's filesystem, so instead `confidential_data_hub.unpack_pause_image()` is called.
|
||||
|
||||
## Using guest image pull with `nerdctl`
|
||||
|
||||
@ -121,6 +166,6 @@ nerdctl run --runtime io.containerd.kata.v2 --snapshotter nydus --label io.kuber
|
||||
```
|
||||
|
||||
References:
|
||||
[1] [[RFC] Image management proposal for hosting sharing and peer pods](https://github.com/confidential-containers/confidential-containers/issues/137)
|
||||
[2] https://github.com/containerd/containerd/blob/main/docs/content-flow.md
|
||||
|
||||
1. [[RFC] Image management proposal for hosting sharing and peer pods](https://github.com/confidential-containers/confidential-containers/issues/137)
|
||||
2. https://github.com/containerd/containerd/blob/main/docs/content-flow.md
|
||||
3. [Move guest pull ability to a configurable component](https://github.com/kata-containers/kata-containers/issues/9266)
|
||||
|
||||
2624
src/agent/Cargo.lock
generated
2624
src/agent/Cargo.lock
generated
File diff suppressed because it is too large
Load Diff
@ -162,9 +162,6 @@ clap.workspace = true
|
||||
strum.workspace = true
|
||||
strum_macros.workspace = true
|
||||
|
||||
# Image pull/decrypt
|
||||
image-rs = { git = "https://github.com/confidential-containers/guest-components", rev = "0a06ef241190780840fbb0542e51b198f1f72b0b", default-features = false, optional = true }
|
||||
|
||||
# Agent Policy
|
||||
cdi = { git = "https://github.com/cncf-tags/container-device-interface-rs", rev = "fba5677a8e7cc962fc6e495fcec98d7d765e332a" }
|
||||
|
||||
@ -202,12 +199,9 @@ test-utils.workspace = true
|
||||
lto = true
|
||||
|
||||
[features]
|
||||
# The default-pull feature supports all sharing images by virtio-fs, for guest-pull build with the guest-pull feature
|
||||
default-pull = []
|
||||
seccomp = ["rustjail/seccomp"]
|
||||
standard-oci-runtime = ["rustjail/standard-oci-runtime"]
|
||||
agent-policy = ["kata-agent-policy"]
|
||||
guest-pull = ["image-rs/kata-cc-rustls-tls"]
|
||||
|
||||
[[bin]]
|
||||
name = "kata-agent"
|
||||
|
||||
@ -41,16 +41,6 @@ ifeq ($(AGENT_POLICY),yes)
|
||||
override EXTRA_RUSTFEATURES += agent-policy
|
||||
endif
|
||||
|
||||
##VAR PULL_TYPE=default|guest-pull define if agent enables the guest pull image feature
|
||||
PULL_TYPE ?= default
|
||||
ifeq ($(PULL_TYPE),default)
|
||||
override EXTRA_RUSTFEATURES += default-pull
|
||||
# Enable guest pull image feature of rust build
|
||||
else ifeq ($(PULL_TYPE),guest-pull)
|
||||
override EXTRA_RUSTFEATURES += guest-pull
|
||||
endif
|
||||
|
||||
|
||||
include ../../utils.mk
|
||||
|
||||
##VAR STANDARD_OCI_RUNTIME=yes|no define if agent enables standard oci runtime feature
|
||||
|
||||
@ -129,6 +129,7 @@ The kata agent has the ability to configure agent options in guest kernel comman
|
||||
| `agent.guest_components_procs` | guest-components processes | Attestation-related processes that should be spawned as children of the guest. Valid values are `none`, `attestation-agent`, `confidential-data-hub` (implies `attestation-agent`), `api-server-rest` (implies `attestation-agent` and `confidential-data-hub`) | string | `api-server-rest` |
|
||||
| `agent.hotplug_timeout` | Hotplug timeout | Allow to configure hotplug timeout(seconds) of block devices | integer | `3` |
|
||||
| `agent.cdh_api_timeout` | Confidential Data Hub (CDH) API timeout | Allow to configure CDH API timeout(seconds) | integer | `50` |
|
||||
| `agent.image_pull_timeout` | Confidential Data Hub (CDH) Image Pull API timeout | Allow to configure CDH API image pull timeout(seconds) | integer | `1200` |
|
||||
| `agent.https_proxy` | HTTPS proxy | Allow to configure `https_proxy` in the guest | string | `""` |
|
||||
| `agent.image_registry_auth` | Image registry credential URI | The URI to where image-rs can find the credentials for pulling images from private registries e.g. `file:///root/.docker/config.json` to read from a file in the guest image, or `kbs:///default/credentials/test` to get the file from the KBS| string | `""` |
|
||||
| `agent.enable_signature_verification` | Image security policy flag | Whether enable image security policy enforcement. If `true`, the resource indexed by URI `agent.image_policy_file` will be got to work as image pulling policy. | string | `""` |
|
||||
@ -148,7 +149,7 @@ The kata agent has the ability to configure agent options in guest kernel comman
|
||||
> The agent will fail to start if the configuration file is not present,
|
||||
> or if it can't be parsed properly.
|
||||
> - `agent.devmode`: true | false
|
||||
> - `agent.hotplug_timeout` and `agent.cdh_api_timeout`: a whole number of seconds
|
||||
> - `agent.hotplug_timeout`, `agent.image_pull_timeout` and `agent.cdh_api_timeout`: a whole number of seconds
|
||||
> - `agent.log`: "critical"("fatal" | "panic") | "error" | "warn"("warning") | "info" | "debug"
|
||||
> - `agent.server_addr`: "{VSOCK_ADDR}:{VSOCK_PORT}"
|
||||
> - `agent.trace`: true | false
|
||||
|
||||
156
src/agent/src/confidential_data_hub/image.rs
Normal file
156
src/agent/src/confidential_data_hub/image.rs
Normal file
@ -0,0 +1,156 @@
|
||||
// Copyright (c) 2021 Alibaba Cloud
|
||||
// Copyright (c) 2021, 2023 IBM Corporation
|
||||
// Copyright (c) 2022 Intel Corporation
|
||||
//
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
//
|
||||
|
||||
use safe_path::scoped_join;
|
||||
use std::collections::HashMap;
|
||||
use std::fs;
|
||||
use std::path::Path;
|
||||
|
||||
use anyhow::{anyhow, bail, Context, Result};
|
||||
use kata_sys_util::validate::verify_id;
|
||||
use oci_spec::runtime as oci;
|
||||
|
||||
use crate::rpc::CONTAINER_BASE;
|
||||
|
||||
use kata_types::mount::KATA_VIRTUAL_VOLUME_IMAGE_GUEST_PULL;
|
||||
use protocols::agent::Storage;
|
||||
|
||||
pub const KATA_IMAGE_WORK_DIR: &str = "/run/kata-containers/image/";
|
||||
const CONFIG_JSON: &str = "config.json";
|
||||
const KATA_PAUSE_BUNDLE: &str = "/pause_bundle";
|
||||
|
||||
const K8S_CONTAINER_TYPE_KEYS: [&str; 2] = [
|
||||
"io.kubernetes.cri.container-type",
|
||||
"io.kubernetes.cri-o.ContainerType",
|
||||
];
|
||||
|
||||
// Convenience function to obtain the scope logger.
|
||||
fn sl() -> slog::Logger {
|
||||
slog_scope::logger().new(o!("subsystem" => "image"))
|
||||
}
|
||||
|
||||
// Function to copy a file if it does not exist at the destination
|
||||
// This function creates a dir, writes a file and if necessary,
|
||||
// overwrites an existing file.
|
||||
fn copy_if_not_exists(src: &Path, dst: &Path) -> Result<()> {
|
||||
if let Some(dst_dir) = dst.parent() {
|
||||
fs::create_dir_all(dst_dir)?;
|
||||
}
|
||||
fs::copy(src, dst)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// get guest pause image process specification
|
||||
fn get_pause_image_process() -> Result<oci::Process> {
|
||||
let guest_pause_bundle = Path::new(KATA_PAUSE_BUNDLE);
|
||||
if !guest_pause_bundle.exists() {
|
||||
bail!("Pause image not present in rootfs");
|
||||
}
|
||||
let guest_pause_config = scoped_join(guest_pause_bundle, CONFIG_JSON)?;
|
||||
|
||||
let image_oci = oci::Spec::load(guest_pause_config.to_str().ok_or_else(|| {
|
||||
anyhow!(
|
||||
"Failed to load the guest pause image config from {:?}",
|
||||
guest_pause_config
|
||||
)
|
||||
})?)
|
||||
.context("load image config file")?;
|
||||
|
||||
let image_oci_process = image_oci.process().as_ref().ok_or_else(|| {
|
||||
anyhow!("The guest pause image config does not contain a process specification. Please check the pause image.")
|
||||
})?;
|
||||
Ok(image_oci_process.clone())
|
||||
}
|
||||
|
||||
/// pause image is packaged in rootfs
|
||||
pub fn unpack_pause_image(cid: &str) -> Result<String> {
|
||||
verify_id(cid).context("The guest pause image cid contains invalid characters.")?;
|
||||
|
||||
let guest_pause_bundle = Path::new(KATA_PAUSE_BUNDLE);
|
||||
if !guest_pause_bundle.exists() {
|
||||
bail!("Pause image not present in rootfs");
|
||||
}
|
||||
let guest_pause_config = scoped_join(guest_pause_bundle, CONFIG_JSON)?;
|
||||
info!(sl(), "use guest pause image cid {:?}", cid);
|
||||
|
||||
let image_oci = oci::Spec::load(guest_pause_config.to_str().ok_or_else(|| {
|
||||
anyhow!(
|
||||
"Failed to load the guest pause image config from {:?}",
|
||||
guest_pause_config
|
||||
)
|
||||
})?)
|
||||
.context("load image config file")?;
|
||||
|
||||
let image_oci_process = image_oci.process().as_ref().ok_or_else(|| {
|
||||
anyhow!("The guest pause image config does not contain a process specification. Please check the pause image.")
|
||||
})?;
|
||||
info!(
|
||||
sl(),
|
||||
"pause image oci process {:?}",
|
||||
image_oci_process.clone()
|
||||
);
|
||||
|
||||
// Ensure that the args vector is not empty before accessing its elements.
|
||||
// Check the number of arguments.
|
||||
let args = if let Some(args_vec) = image_oci_process.args() {
|
||||
args_vec
|
||||
} else {
|
||||
bail!("The number of args should be greater than or equal to one! Please check the pause image.");
|
||||
};
|
||||
|
||||
let pause_bundle = scoped_join(CONTAINER_BASE, cid)?;
|
||||
fs::create_dir_all(&pause_bundle)?;
|
||||
let pause_rootfs = scoped_join(&pause_bundle, "rootfs")?;
|
||||
fs::create_dir_all(&pause_rootfs)?;
|
||||
info!(sl(), "pause_rootfs {:?}", pause_rootfs);
|
||||
|
||||
copy_if_not_exists(&guest_pause_config, &pause_bundle.join(CONFIG_JSON))?;
|
||||
let arg_path = Path::new(&args[0]).strip_prefix("/")?;
|
||||
copy_if_not_exists(
|
||||
&guest_pause_bundle.join("rootfs").join(arg_path),
|
||||
&pause_rootfs.join(arg_path),
|
||||
)?;
|
||||
Ok(pause_rootfs.display().to_string())
|
||||
}
|
||||
|
||||
/// check whether the image is for sandbox or for container.
|
||||
pub fn is_sandbox(image_metadata: &HashMap<String, String>) -> bool {
|
||||
let mut is_sandbox = false;
|
||||
for key in K8S_CONTAINER_TYPE_KEYS.iter() {
|
||||
if let Some(value) = image_metadata.get(key as &str) {
|
||||
if value == "sandbox" {
|
||||
is_sandbox = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
is_sandbox
|
||||
}
|
||||
|
||||
/// get_process overrides the OCI process spec with pause image process spec if needed
|
||||
pub fn get_process(
|
||||
ocip: &oci::Process,
|
||||
oci: &oci::Spec,
|
||||
storages: Vec<Storage>,
|
||||
) -> Result<oci::Process> {
|
||||
let mut guest_pull = false;
|
||||
for storage in storages {
|
||||
if storage.driver == KATA_VIRTUAL_VOLUME_IMAGE_GUEST_PULL {
|
||||
guest_pull = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if guest_pull {
|
||||
if let Some(a) = oci.annotations() {
|
||||
if is_sandbox(a) {
|
||||
return get_pause_image_process();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Ok(ocip.clone())
|
||||
}
|
||||
@ -1,4 +1,5 @@
|
||||
// Copyright (c) 2023 Intel Corporation
|
||||
// Copyright (c) 2025 Alibaba Cloud
|
||||
//
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
//
|
||||
@ -15,19 +16,19 @@ use protocols::{
|
||||
confidential_data_hub::GetResourceRequest,
|
||||
confidential_data_hub_ttrpc_async,
|
||||
confidential_data_hub_ttrpc_async::{
|
||||
GetResourceServiceClient, SealedSecretServiceClient, SecureMountServiceClient,
|
||||
GetResourceServiceClient, ImagePullServiceClient, SealedSecretServiceClient,
|
||||
SecureMountServiceClient,
|
||||
},
|
||||
};
|
||||
use safe_path::scoped_join;
|
||||
use std::fs;
|
||||
use std::os::unix::fs::symlink;
|
||||
use std::path::Path;
|
||||
use std::{os::unix::fs::symlink, path::PathBuf};
|
||||
use tokio::sync::OnceCell;
|
||||
|
||||
// Nanoseconds
|
||||
lazy_static! {
|
||||
static ref CDH_API_TIMEOUT: i64 = AGENT_CONFIG.cdh_api_timeout.as_nanos() as i64;
|
||||
pub static ref CDH_CLIENT: OnceCell<CDHClient> = OnceCell::new();
|
||||
}
|
||||
pub mod image;
|
||||
|
||||
pub static CDH_CLIENT: OnceCell<CDHClient> = OnceCell::const_new();
|
||||
|
||||
const SEALED_SECRET_PREFIX: &str = "sealed.";
|
||||
|
||||
@ -45,6 +46,8 @@ pub struct CDHClient {
|
||||
secure_mount_client: SecureMountServiceClient,
|
||||
#[derivative(Debug = "ignore")]
|
||||
get_resource_client: GetResourceServiceClient,
|
||||
#[derivative(Debug = "ignore")]
|
||||
image_pull_client: ImagePullServiceClient,
|
||||
}
|
||||
|
||||
impl CDHClient {
|
||||
@ -52,6 +55,8 @@ impl CDHClient {
|
||||
let client = ttrpc::asynchronous::Client::connect(cdh_socket_uri)?;
|
||||
let sealed_secret_client =
|
||||
confidential_data_hub_ttrpc_async::SealedSecretServiceClient::new(client.clone());
|
||||
let image_pull_client =
|
||||
confidential_data_hub_ttrpc_async::ImagePullServiceClient::new(client.clone());
|
||||
let secure_mount_client =
|
||||
confidential_data_hub_ttrpc_async::SecureMountServiceClient::new(client.clone());
|
||||
let get_resource_client =
|
||||
@ -60,6 +65,7 @@ impl CDHClient {
|
||||
sealed_secret_client,
|
||||
secure_mount_client,
|
||||
get_resource_client,
|
||||
image_pull_client,
|
||||
})
|
||||
}
|
||||
|
||||
@ -69,12 +75,14 @@ impl CDHClient {
|
||||
|
||||
let unsealed_secret = self
|
||||
.sealed_secret_client
|
||||
.unseal_secret(ttrpc::context::with_timeout(*CDH_API_TIMEOUT), &input)
|
||||
.unseal_secret(
|
||||
ttrpc::context::with_timeout(AGENT_CONFIG.cdh_api_timeout.as_nanos() as i64),
|
||||
&input,
|
||||
)
|
||||
.await?;
|
||||
Ok(unsealed_secret.plaintext)
|
||||
}
|
||||
|
||||
#[cfg(feature = "guest-pull")]
|
||||
pub async fn secure_mount(
|
||||
&self,
|
||||
volume_type: &str,
|
||||
@ -90,7 +98,10 @@ impl CDHClient {
|
||||
..Default::default()
|
||||
};
|
||||
self.secure_mount_client
|
||||
.secure_mount(ttrpc::context::with_timeout(*CDH_API_TIMEOUT), &req)
|
||||
.secure_mount(
|
||||
ttrpc::context::with_timeout(AGENT_CONFIG.cdh_api_timeout.as_nanos() as i64),
|
||||
&req,
|
||||
)
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
@ -102,10 +113,31 @@ impl CDHClient {
|
||||
};
|
||||
let res = self
|
||||
.get_resource_client
|
||||
.get_resource(ttrpc::context::with_timeout(*CDH_API_TIMEOUT), &req)
|
||||
.get_resource(
|
||||
ttrpc::context::with_timeout(AGENT_CONFIG.cdh_api_timeout.as_nanos() as i64),
|
||||
&req,
|
||||
)
|
||||
.await?;
|
||||
Ok(res.Resource)
|
||||
}
|
||||
|
||||
pub async fn pull_image(&self, image: &str, bundle_path: &str) -> Result<()> {
|
||||
let req = confidential_data_hub::ImagePullRequest {
|
||||
image_url: image.to_string(),
|
||||
bundle_path: bundle_path.to_string(),
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
let _ = self
|
||||
.image_pull_client
|
||||
.pull_image(
|
||||
ttrpc::context::with_timeout(AGENT_CONFIG.image_pull_timeout.as_nanos() as i64),
|
||||
&req,
|
||||
)
|
||||
.await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn init_cdh_client(cdh_socket_uri: &str) -> Result<()> {
|
||||
@ -114,11 +146,12 @@ pub async fn init_cdh_client(cdh_socket_uri: &str) -> Result<()> {
|
||||
CDHClient::new(cdh_socket_uri).context("Failed to create CDH Client")
|
||||
})
|
||||
.await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Check if the CDH client is initialized
|
||||
pub async fn is_cdh_client_initialized() -> bool {
|
||||
pub fn is_cdh_client_initialized() -> bool {
|
||||
CDH_CLIENT.get().is_some() // Returns true if CDH_CLIENT is initialized, false otherwise
|
||||
}
|
||||
|
||||
@ -138,6 +171,29 @@ pub async fn unseal_env(env: &str) -> Result<String> {
|
||||
Ok((*env.to_owned()).to_string())
|
||||
}
|
||||
|
||||
/// pull_image is used for call confidential data hub to pull image in the guest.
|
||||
/// Image layers will store at [`image::KATA_IMAGE_WORK_DIR`]`,
|
||||
/// rootfs and config.json will store under given `bundle_path`.
|
||||
///
|
||||
/// # Parameters
|
||||
/// - `image`: Image name (exp: quay.io/prometheus/busybox:latest)
|
||||
/// - `bundle_path`: The path to store the image bundle (exp. /run/kata-containers/cb0b47276ea66ee9f44cc53afa94d7980b57a52c3f306f68cb034e58d9fbd3c6/rootfs)
|
||||
pub async fn pull_image(image: &str, bundle_path: PathBuf) -> Result<String> {
|
||||
fs::create_dir_all(&bundle_path)?;
|
||||
info!(sl(), "pull image {image:?}, bundle path {bundle_path:?}");
|
||||
|
||||
let cdh_client = CDH_CLIENT
|
||||
.get()
|
||||
.expect("Confidential Data Hub not initialized");
|
||||
|
||||
cdh_client
|
||||
.pull_image(image, bundle_path.to_string_lossy().as_ref())
|
||||
.await?;
|
||||
|
||||
let image_bundle_path = scoped_join(&bundle_path, "rootfs")?;
|
||||
Ok(image_bundle_path.as_path().display().to_string())
|
||||
}
|
||||
|
||||
pub async fn unseal_file(path: &str) -> Result<()> {
|
||||
let cdh_client = CDH_CLIENT
|
||||
.get()
|
||||
@ -206,7 +262,6 @@ pub async fn unseal_file(path: &str) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(feature = "guest-pull")]
|
||||
pub async fn secure_mount(
|
||||
volume_type: &str,
|
||||
options: &std::collections::HashMap<String, String>,
|
||||
@ -257,6 +312,18 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl confidential_data_hub_ttrpc_async::ImagePullService for TestService {
|
||||
async fn pull_image(
|
||||
&self,
|
||||
_ctx: &::ttrpc::asynchronous::TtrpcContext,
|
||||
_req: confidential_data_hub::ImagePullRequest,
|
||||
) -> ttrpc::error::Result<confidential_data_hub::ImagePullResponse> {
|
||||
let output = confidential_data_hub::ImagePullResponse::new();
|
||||
Ok(output)
|
||||
}
|
||||
}
|
||||
|
||||
fn remove_if_sock_exist(sock_addr: &str) -> std::io::Result<()> {
|
||||
let path = sock_addr
|
||||
.strip_prefix("unix://")
|
||||
@ -23,6 +23,7 @@ const SERVER_ADDR_OPTION: &str = "agent.server_addr";
|
||||
const PASSFD_LISTENER_PORT: &str = "agent.passfd_listener_port";
|
||||
const HOTPLUG_TIMOUT_OPTION: &str = "agent.hotplug_timeout";
|
||||
const CDH_API_TIMOUT_OPTION: &str = "agent.cdh_api_timeout";
|
||||
const CDH_IMAGE_PULL_TIMEOUT_OPTION: &str = "agent.image_pull_timeout";
|
||||
const CDI_TIMEOUT_OPTION: &str = "agent.cdi_timeout";
|
||||
const DEBUG_CONSOLE_VPORT_OPTION: &str = "agent.debug_console_vport";
|
||||
const LOG_VPORT_OPTION: &str = "agent.log_vport";
|
||||
@ -32,16 +33,8 @@ const UNIFIED_CGROUP_HIERARCHY_OPTION: &str = "systemd.unified_cgroup_hierarchy"
|
||||
const CONFIG_FILE: &str = "agent.config_file";
|
||||
const GUEST_COMPONENTS_REST_API_OPTION: &str = "agent.guest_components_rest_api";
|
||||
const GUEST_COMPONENTS_PROCS_OPTION: &str = "agent.guest_components_procs";
|
||||
#[cfg(feature = "guest-pull")]
|
||||
const IMAGE_REGISTRY_AUTH_OPTION: &str = "agent.image_registry_auth";
|
||||
const SECURE_STORAGE_INTEGRITY_OPTION: &str = "agent.secure_storage_integrity";
|
||||
|
||||
#[cfg(feature = "guest-pull")]
|
||||
const ENABLE_SIGNATURE_VERIFICATION: &str = "agent.enable_signature_verification";
|
||||
|
||||
#[cfg(feature = "guest-pull")]
|
||||
const IMAGE_POLICY_FILE: &str = "agent.image_policy_file";
|
||||
|
||||
// Configure the proxy settings for HTTPS requests in the guest,
|
||||
// to solve the problem of not being able to access the specified image in some cases.
|
||||
const HTTPS_PROXY: &str = "agent.https_proxy";
|
||||
@ -71,6 +64,7 @@ const MEM_AGENT_COMPACT_FORCE_TIMES: &str = "agent.mem_agent_compact_force_times
|
||||
const DEFAULT_LOG_LEVEL: slog::Level = slog::Level::Info;
|
||||
const DEFAULT_HOTPLUG_TIMEOUT: time::Duration = time::Duration::from_secs(3);
|
||||
const DEFAULT_CDH_API_TIMEOUT: time::Duration = time::Duration::from_secs(50);
|
||||
const DEFAULT_IMAGE_PULL_TIMEOUT: time::Duration = time::Duration::from_secs(1200);
|
||||
const DEFAULT_CDI_TIMEOUT: time::Duration = time::Duration::from_secs(100);
|
||||
const DEFAULT_CONTAINER_PIPE_SIZE: i32 = 0;
|
||||
const VSOCK_ADDR: &str = "vsock://-1";
|
||||
@ -134,6 +128,7 @@ pub struct AgentConfig {
|
||||
pub log_level: slog::Level,
|
||||
pub hotplug_timeout: time::Duration,
|
||||
pub cdh_api_timeout: time::Duration,
|
||||
pub image_pull_timeout: time::Duration,
|
||||
pub cdi_timeout: time::Duration,
|
||||
pub debug_console_vport: i32,
|
||||
pub log_vport: i32,
|
||||
@ -147,13 +142,7 @@ pub struct AgentConfig {
|
||||
pub no_proxy: String,
|
||||
pub guest_components_rest_api: GuestComponentsFeatures,
|
||||
pub guest_components_procs: GuestComponentsProcs,
|
||||
#[cfg(feature = "guest-pull")]
|
||||
pub image_registry_auth: String,
|
||||
pub secure_storage_integrity: bool,
|
||||
#[cfg(feature = "guest-pull")]
|
||||
pub enable_signature_verification: bool,
|
||||
#[cfg(feature = "guest-pull")]
|
||||
pub image_policy_file: String,
|
||||
#[cfg(feature = "agent-policy")]
|
||||
pub policy_file: String,
|
||||
pub mem_agent: Option<MemAgentConfig>,
|
||||
@ -172,6 +161,7 @@ pub struct AgentConfigBuilder {
|
||||
pub log_level: Option<String>,
|
||||
pub hotplug_timeout: Option<time::Duration>,
|
||||
pub cdh_api_timeout: Option<time::Duration>,
|
||||
pub image_pull_timeout: Option<time::Duration>,
|
||||
pub cdi_timeout: Option<time::Duration>,
|
||||
pub debug_console_vport: Option<i32>,
|
||||
pub log_vport: Option<i32>,
|
||||
@ -184,13 +174,7 @@ pub struct AgentConfigBuilder {
|
||||
pub no_proxy: Option<String>,
|
||||
pub guest_components_rest_api: Option<GuestComponentsFeatures>,
|
||||
pub guest_components_procs: Option<GuestComponentsProcs>,
|
||||
#[cfg(feature = "guest-pull")]
|
||||
pub image_registry_auth: Option<String>,
|
||||
pub secure_storage_integrity: Option<bool>,
|
||||
#[cfg(feature = "guest-pull")]
|
||||
pub enable_signature_verification: Option<bool>,
|
||||
#[cfg(feature = "guest-pull")]
|
||||
pub image_policy_file: Option<String>,
|
||||
#[cfg(feature = "agent-policy")]
|
||||
pub policy_file: Option<String>,
|
||||
pub mem_agent_enable: Option<bool>,
|
||||
@ -271,6 +255,7 @@ impl Default for AgentConfig {
|
||||
log_level: DEFAULT_LOG_LEVEL,
|
||||
hotplug_timeout: DEFAULT_HOTPLUG_TIMEOUT,
|
||||
cdh_api_timeout: DEFAULT_CDH_API_TIMEOUT,
|
||||
image_pull_timeout: DEFAULT_IMAGE_PULL_TIMEOUT,
|
||||
cdi_timeout: DEFAULT_CDI_TIMEOUT,
|
||||
debug_console_vport: 0,
|
||||
log_vport: 0,
|
||||
@ -284,13 +269,7 @@ impl Default for AgentConfig {
|
||||
no_proxy: String::from(""),
|
||||
guest_components_rest_api: GuestComponentsFeatures::default(),
|
||||
guest_components_procs: GuestComponentsProcs::default(),
|
||||
#[cfg(feature = "guest-pull")]
|
||||
image_registry_auth: String::from(""),
|
||||
secure_storage_integrity: false,
|
||||
#[cfg(feature = "guest-pull")]
|
||||
enable_signature_verification: false,
|
||||
#[cfg(feature = "guest-pull")]
|
||||
image_policy_file: String::from(""),
|
||||
#[cfg(feature = "agent-policy")]
|
||||
policy_file: String::from(""),
|
||||
mem_agent: None,
|
||||
@ -317,6 +296,7 @@ impl FromStr for AgentConfig {
|
||||
);
|
||||
config_override!(agent_config_builder, agent_config, hotplug_timeout);
|
||||
config_override!(agent_config_builder, agent_config, cdh_api_timeout);
|
||||
config_override!(agent_config_builder, agent_config, image_pull_timeout);
|
||||
config_override!(agent_config_builder, agent_config, cdi_timeout);
|
||||
config_override!(agent_config_builder, agent_config, debug_console_vport);
|
||||
config_override!(agent_config_builder, agent_config, log_vport);
|
||||
@ -333,16 +313,6 @@ impl FromStr for AgentConfig {
|
||||
guest_components_rest_api
|
||||
);
|
||||
config_override!(agent_config_builder, agent_config, guest_components_procs);
|
||||
#[cfg(feature = "guest-pull")]
|
||||
{
|
||||
config_override!(agent_config_builder, agent_config, image_registry_auth);
|
||||
config_override!(
|
||||
agent_config_builder,
|
||||
agent_config,
|
||||
enable_signature_verification
|
||||
);
|
||||
config_override!(agent_config_builder, agent_config, image_policy_file);
|
||||
}
|
||||
config_override!(agent_config_builder, agent_config, secure_storage_integrity);
|
||||
|
||||
#[cfg(feature = "agent-policy")]
|
||||
@ -493,6 +463,15 @@ impl AgentConfig {
|
||||
|cdh_api_timeout: &time::Duration| cdh_api_timeout.as_secs() > 0
|
||||
);
|
||||
|
||||
// ensure the timeout is a positive value
|
||||
parse_cmdline_param!(
|
||||
param,
|
||||
CDH_IMAGE_PULL_TIMEOUT_OPTION,
|
||||
config.image_pull_timeout,
|
||||
get_timeout,
|
||||
|image_pull_timeout: &time::Duration| image_pull_timeout.as_secs() > 0
|
||||
);
|
||||
|
||||
// ensure the timeout is a positive value
|
||||
parse_cmdline_param!(
|
||||
param,
|
||||
@ -557,27 +536,6 @@ impl AgentConfig {
|
||||
config.guest_components_procs,
|
||||
get_guest_components_procs_value
|
||||
);
|
||||
#[cfg(feature = "guest-pull")]
|
||||
{
|
||||
parse_cmdline_param!(
|
||||
param,
|
||||
IMAGE_REGISTRY_AUTH_OPTION,
|
||||
config.image_registry_auth,
|
||||
get_string_value
|
||||
);
|
||||
parse_cmdline_param!(
|
||||
param,
|
||||
ENABLE_SIGNATURE_VERIFICATION,
|
||||
config.enable_signature_verification,
|
||||
get_bool_value
|
||||
);
|
||||
parse_cmdline_param!(
|
||||
param,
|
||||
IMAGE_POLICY_FILE,
|
||||
config.image_policy_file,
|
||||
get_string_value
|
||||
);
|
||||
}
|
||||
parse_cmdline_param!(
|
||||
param,
|
||||
SECURE_STORAGE_INTEGRITY_OPTION,
|
||||
@ -780,7 +738,10 @@ fn get_timeout(param: &str) -> Result<time::Duration> {
|
||||
ensure!(
|
||||
matches!(
|
||||
fields[0],
|
||||
HOTPLUG_TIMOUT_OPTION | CDH_API_TIMOUT_OPTION | CDI_TIMEOUT_OPTION
|
||||
HOTPLUG_TIMOUT_OPTION
|
||||
| CDH_API_TIMOUT_OPTION
|
||||
| CDH_IMAGE_PULL_TIMEOUT_OPTION
|
||||
| CDI_TIMEOUT_OPTION
|
||||
),
|
||||
ERR_INVALID_TIMEOUT_KEY
|
||||
);
|
||||
@ -901,11 +862,6 @@ mod tests {
|
||||
assert!(!config.dev_mode);
|
||||
assert_eq!(config.log_level, DEFAULT_LOG_LEVEL);
|
||||
assert_eq!(config.hotplug_timeout, DEFAULT_HOTPLUG_TIMEOUT);
|
||||
#[cfg(feature = "guest-pull")]
|
||||
{
|
||||
assert!(!config.enable_signature_verification);
|
||||
assert_eq!(config.image_policy_file, "");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
@ -931,13 +887,7 @@ mod tests {
|
||||
no_proxy: &'a str,
|
||||
guest_components_rest_api: GuestComponentsFeatures,
|
||||
guest_components_procs: GuestComponentsProcs,
|
||||
#[cfg(feature = "guest-pull")]
|
||||
image_registry_auth: &'a str,
|
||||
secure_storage_integrity: bool,
|
||||
#[cfg(feature = "guest-pull")]
|
||||
enable_signature_verification: bool,
|
||||
#[cfg(feature = "guest-pull")]
|
||||
image_policy_file: &'a str,
|
||||
#[cfg(feature = "agent-policy")]
|
||||
policy_file: &'a str,
|
||||
mem_agent: Option<MemAgentConfig>,
|
||||
@ -961,13 +911,7 @@ mod tests {
|
||||
no_proxy: "",
|
||||
guest_components_rest_api: GuestComponentsFeatures::default(),
|
||||
guest_components_procs: GuestComponentsProcs::default(),
|
||||
#[cfg(feature = "guest-pull")]
|
||||
image_registry_auth: "",
|
||||
secure_storage_integrity: false,
|
||||
#[cfg(feature = "guest-pull")]
|
||||
enable_signature_verification: false,
|
||||
#[cfg(feature = "guest-pull")]
|
||||
image_policy_file: "",
|
||||
#[cfg(feature = "agent-policy")]
|
||||
policy_file: "",
|
||||
mem_agent: None,
|
||||
@ -1418,18 +1362,6 @@ mod tests {
|
||||
guest_components_procs: GuestComponentsProcs::None,
|
||||
..Default::default()
|
||||
},
|
||||
#[cfg(feature = "guest-pull")]
|
||||
TestData {
|
||||
contents: "agent.image_registry_auth=file:///root/.docker/config.json",
|
||||
image_registry_auth: "file:///root/.docker/config.json",
|
||||
..Default::default()
|
||||
},
|
||||
#[cfg(feature = "guest-pull")]
|
||||
TestData {
|
||||
contents: "agent.image_registry_auth=kbs:///default/credentials/test",
|
||||
image_registry_auth: "kbs:///default/credentials/test",
|
||||
..Default::default()
|
||||
},
|
||||
TestData {
|
||||
contents: "",
|
||||
secure_storage_integrity: false,
|
||||
@ -1455,24 +1387,6 @@ mod tests {
|
||||
secure_storage_integrity: false,
|
||||
..Default::default()
|
||||
},
|
||||
#[cfg(feature = "guest-pull")]
|
||||
TestData {
|
||||
contents: "agent.enable_signature_verification=true",
|
||||
enable_signature_verification: true,
|
||||
..Default::default()
|
||||
},
|
||||
#[cfg(feature = "guest-pull")]
|
||||
TestData {
|
||||
contents: "agent.image_policy_file=kbs:///default/image-policy/test",
|
||||
image_policy_file: "kbs:///default/image-policy/test",
|
||||
..Default::default()
|
||||
},
|
||||
#[cfg(feature = "guest-pull")]
|
||||
TestData {
|
||||
contents: "agent.image_policy_file=file:///etc/image-policy.json",
|
||||
image_policy_file: "file:///etc/image-policy.json",
|
||||
..Default::default()
|
||||
},
|
||||
#[cfg(feature = "agent-policy")]
|
||||
// Test environment
|
||||
TestData {
|
||||
@ -1575,16 +1489,6 @@ mod tests {
|
||||
"{}",
|
||||
msg
|
||||
);
|
||||
#[cfg(feature = "guest-pull")]
|
||||
{
|
||||
assert_eq!(d.image_registry_auth, config.image_registry_auth, "{}", msg);
|
||||
assert_eq!(
|
||||
d.enable_signature_verification, config.enable_signature_verification,
|
||||
"{}",
|
||||
msg
|
||||
);
|
||||
assert_eq!(d.image_policy_file, config.image_policy_file, "{}", msg);
|
||||
}
|
||||
assert_eq!(
|
||||
d.secure_storage_integrity, config.secure_storage_integrity,
|
||||
"{}",
|
||||
@ -1722,6 +1626,7 @@ Caused by:
|
||||
)))]
|
||||
#[case("agent.chd_api_timeout=1", Err(anyhow!(ERR_INVALID_TIMEOUT_KEY)))]
|
||||
#[case("agent.cdh_api_timeout=600", Ok(time::Duration::from_secs(600)))]
|
||||
#[case("agent.image_pull_timeout=1200", Ok(time::Duration::from_secs(1200)))]
|
||||
#[case("agent.cdi_timeout=320", Ok(time::Duration::from_secs(320)))]
|
||||
fn test_timeout(#[case] param: &str, #[case] expected: Result<time::Duration>) {
|
||||
let result = get_timeout(param);
|
||||
|
||||
@ -29,7 +29,7 @@ use tracing::instrument;
|
||||
cfg_if! {
|
||||
if #[cfg(target_arch = "s390x")] {
|
||||
use crate::ap;
|
||||
use crate::cdh::get_cdh_resource;
|
||||
use crate::confidential_data_hub::get_cdh_resource;
|
||||
use std::convert::TryFrom;
|
||||
use pv_core::ap::{
|
||||
Apqn,
|
||||
|
||||
@ -8,8 +8,6 @@ pub fn get_build_features() -> Vec<String> {
|
||||
let features: Vec<&str> = vec![
|
||||
#[cfg(feature = "agent-policy")]
|
||||
"agent-policy",
|
||||
#[cfg(feature = "guest-pull")]
|
||||
"guest-pull",
|
||||
#[cfg(feature = "seccomp")]
|
||||
"seccomp",
|
||||
#[cfg(feature = "standard-oci-runtime")]
|
||||
|
||||
@ -1,296 +0,0 @@
|
||||
// Copyright (c) 2021 Alibaba Cloud
|
||||
// Copyright (c) 2021, 2023 IBM Corporation
|
||||
// Copyright (c) 2022 Intel Corporation
|
||||
//
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
//
|
||||
|
||||
use safe_path::scoped_join;
|
||||
use std::collections::HashMap;
|
||||
use std::env;
|
||||
use std::fs;
|
||||
use std::path::Path;
|
||||
use std::sync::Arc;
|
||||
|
||||
use anyhow::{anyhow, bail, Context, Result};
|
||||
use image_rs::builder::ClientBuilder;
|
||||
use image_rs::image::ImageClient;
|
||||
use kata_sys_util::validate::verify_id;
|
||||
use oci_spec::runtime as oci;
|
||||
use tokio::sync::Mutex;
|
||||
|
||||
use crate::rpc::CONTAINER_BASE;
|
||||
use crate::AGENT_CONFIG;
|
||||
|
||||
use kata_types::mount::KATA_VIRTUAL_VOLUME_IMAGE_GUEST_PULL;
|
||||
use protocols::agent::Storage;
|
||||
|
||||
pub const KATA_IMAGE_WORK_DIR: &str = "/run/kata-containers/image/";
|
||||
const CONFIG_JSON: &str = "config.json";
|
||||
const KATA_PAUSE_BUNDLE: &str = "/pause_bundle";
|
||||
|
||||
const K8S_CONTAINER_TYPE_KEYS: [&str; 2] = [
|
||||
"io.kubernetes.cri.container-type",
|
||||
"io.kubernetes.cri-o.ContainerType",
|
||||
];
|
||||
|
||||
#[rustfmt::skip]
|
||||
lazy_static! {
|
||||
pub static ref IMAGE_SERVICE: Arc<Mutex<Option<ImageService>>> = Arc::new(Mutex::new(None));
|
||||
}
|
||||
|
||||
// Convenience function to obtain the scope logger.
|
||||
fn sl() -> slog::Logger {
|
||||
slog_scope::logger().new(o!("subsystem" => "image"))
|
||||
}
|
||||
|
||||
// Function to copy a file if it does not exist at the destination
|
||||
fn copy_if_not_exists(src: &Path, dst: &Path) -> Result<()> {
|
||||
if let Some(dst_dir) = dst.parent() {
|
||||
fs::create_dir_all(dst_dir)?;
|
||||
}
|
||||
fs::copy(src, dst)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub struct ImageService {
|
||||
image_client: ImageClient,
|
||||
}
|
||||
|
||||
impl ImageService {
|
||||
pub async fn new() -> Result<Self> {
|
||||
let mut image_client_builder =
|
||||
ClientBuilder::default().work_dir(KATA_IMAGE_WORK_DIR.into());
|
||||
#[cfg(feature = "guest-pull")]
|
||||
{
|
||||
if !AGENT_CONFIG.image_registry_auth.is_empty() {
|
||||
let registry_auth = &AGENT_CONFIG.image_registry_auth;
|
||||
debug!(sl(), "Set registry auth file {:?}", registry_auth);
|
||||
image_client_builder = image_client_builder
|
||||
.authenticated_registry_credentials_uri(registry_auth.into());
|
||||
}
|
||||
|
||||
let enable_signature_verification = &AGENT_CONFIG.enable_signature_verification;
|
||||
debug!(
|
||||
sl(),
|
||||
"Enable image signature verification: {:?}", enable_signature_verification
|
||||
);
|
||||
if !AGENT_CONFIG.image_policy_file.is_empty() && *enable_signature_verification {
|
||||
let image_policy_file = &AGENT_CONFIG.image_policy_file;
|
||||
debug!(sl(), "Use image policy file {:?}", image_policy_file);
|
||||
image_client_builder =
|
||||
image_client_builder.image_security_policy_uri(image_policy_file.into());
|
||||
}
|
||||
}
|
||||
let image_client = image_client_builder.build().await?;
|
||||
Ok(Self { image_client })
|
||||
}
|
||||
|
||||
/// get guest pause image process specification
|
||||
fn get_pause_image_process() -> Result<oci::Process> {
|
||||
let guest_pause_bundle = Path::new(KATA_PAUSE_BUNDLE);
|
||||
if !guest_pause_bundle.exists() {
|
||||
bail!("Pause image not present in rootfs");
|
||||
}
|
||||
let guest_pause_config = scoped_join(guest_pause_bundle, CONFIG_JSON)?;
|
||||
|
||||
let image_oci = oci::Spec::load(guest_pause_config.to_str().ok_or_else(|| {
|
||||
anyhow!(
|
||||
"Failed to load the guest pause image config from {:?}",
|
||||
guest_pause_config
|
||||
)
|
||||
})?)
|
||||
.context("load image config file")?;
|
||||
|
||||
let image_oci_process = image_oci.process().as_ref().ok_or_else(|| {
|
||||
anyhow!("The guest pause image config does not contain a process specification. Please check the pause image.")
|
||||
})?;
|
||||
Ok(image_oci_process.clone())
|
||||
}
|
||||
|
||||
/// pause image is packaged in rootfs
|
||||
fn unpack_pause_image(cid: &str) -> Result<String> {
|
||||
verify_id(cid).context("The guest pause image cid contains invalid characters.")?;
|
||||
|
||||
let guest_pause_bundle = Path::new(KATA_PAUSE_BUNDLE);
|
||||
if !guest_pause_bundle.exists() {
|
||||
bail!("Pause image not present in rootfs");
|
||||
}
|
||||
let guest_pause_config = scoped_join(guest_pause_bundle, CONFIG_JSON)?;
|
||||
info!(sl(), "use guest pause image cid {:?}", cid);
|
||||
|
||||
let image_oci = oci::Spec::load(guest_pause_config.to_str().ok_or_else(|| {
|
||||
anyhow!(
|
||||
"Failed to load the guest pause image config from {:?}",
|
||||
guest_pause_config
|
||||
)
|
||||
})?)
|
||||
.context("load image config file")?;
|
||||
|
||||
let image_oci_process = image_oci.process().as_ref().ok_or_else(|| {
|
||||
anyhow!("The guest pause image config does not contain a process specification. Please check the pause image.")
|
||||
})?;
|
||||
info!(
|
||||
sl(),
|
||||
"pause image oci process {:?}",
|
||||
image_oci_process.clone()
|
||||
);
|
||||
|
||||
// Ensure that the args vector is not empty before accessing its elements.
|
||||
// Check the number of arguments.
|
||||
let args = if let Some(args_vec) = image_oci_process.args() {
|
||||
args_vec
|
||||
} else {
|
||||
bail!("The number of args should be greater than or equal to one! Please check the pause image.");
|
||||
};
|
||||
|
||||
let pause_bundle = scoped_join(CONTAINER_BASE, cid)?;
|
||||
fs::create_dir_all(&pause_bundle)?;
|
||||
let pause_rootfs = scoped_join(&pause_bundle, "rootfs")?;
|
||||
fs::create_dir_all(&pause_rootfs)?;
|
||||
info!(sl(), "pause_rootfs {:?}", pause_rootfs);
|
||||
|
||||
copy_if_not_exists(&guest_pause_config, &pause_bundle.join(CONFIG_JSON))?;
|
||||
let arg_path = Path::new(&args[0]).strip_prefix("/")?;
|
||||
copy_if_not_exists(
|
||||
&guest_pause_bundle.join("rootfs").join(arg_path),
|
||||
&pause_rootfs.join(arg_path),
|
||||
)?;
|
||||
Ok(pause_rootfs.display().to_string())
|
||||
}
|
||||
|
||||
/// check whether the image is for sandbox or for container.
|
||||
fn is_sandbox(image_metadata: &HashMap<String, String>) -> bool {
|
||||
let mut is_sandbox = false;
|
||||
for key in K8S_CONTAINER_TYPE_KEYS.iter() {
|
||||
if let Some(value) = image_metadata.get(key as &str) {
|
||||
if value == "sandbox" {
|
||||
is_sandbox = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
is_sandbox
|
||||
}
|
||||
|
||||
/// pull_image is used for call image-rs to pull image in the guest.
|
||||
/// # Parameters
|
||||
/// - `image`: Image name (exp: quay.io/prometheus/busybox:latest)
|
||||
/// - `cid`: Container id
|
||||
/// - `image_metadata`: Annotations about the image (exp: "containerd.io/snapshot/cri.layer-digest": "sha256:24fb2886d6f6c5d16481dd7608b47e78a8e92a13d6e64d87d57cb16d5f766d63")
|
||||
/// # Returns
|
||||
/// - The image rootfs bundle path. (exp. /run/kata-containers/cb0b47276ea66ee9f44cc53afa94d7980b57a52c3f306f68cb034e58d9fbd3c6/rootfs)
|
||||
pub async fn pull_image(
|
||||
&mut self,
|
||||
image: &str,
|
||||
cid: &str,
|
||||
image_metadata: &HashMap<String, String>,
|
||||
) -> Result<String> {
|
||||
info!(sl(), "image metadata: {image_metadata:?}");
|
||||
|
||||
if Self::is_sandbox(image_metadata) {
|
||||
let mount_path = Self::unpack_pause_image(cid)?;
|
||||
return Ok(mount_path);
|
||||
}
|
||||
|
||||
// Image layers will store at KATA_IMAGE_WORK_DIR, generated bundles
|
||||
// with rootfs and config.json will store under CONTAINER_BASE/cid/images.
|
||||
let bundle_path = scoped_join(CONTAINER_BASE, cid)?;
|
||||
fs::create_dir_all(&bundle_path)?;
|
||||
info!(sl(), "pull image {image:?}, bundle path {bundle_path:?}");
|
||||
|
||||
let res = self
|
||||
.image_client
|
||||
.pull_image(image, &bundle_path, &None, &None)
|
||||
.await;
|
||||
match res {
|
||||
Ok(image) => {
|
||||
info!(
|
||||
sl(),
|
||||
"pull and unpack image {image:?}, cid: {cid:?} succeeded."
|
||||
);
|
||||
}
|
||||
Err(e) => {
|
||||
error!(
|
||||
sl(),
|
||||
"pull and unpack image {image:?}, cid: {cid:?} failed with {:?}.",
|
||||
e.to_string()
|
||||
);
|
||||
return Err(e);
|
||||
}
|
||||
};
|
||||
let image_bundle_path = scoped_join(&bundle_path, "rootfs")?;
|
||||
Ok(image_bundle_path.as_path().display().to_string())
|
||||
}
|
||||
}
|
||||
|
||||
/// get_process overrides the OCI process spec with pause image process spec if needed
|
||||
pub fn get_process(
|
||||
ocip: &oci::Process,
|
||||
oci: &oci::Spec,
|
||||
storages: Vec<Storage>,
|
||||
) -> Result<oci::Process> {
|
||||
let mut guest_pull = false;
|
||||
for storage in storages {
|
||||
if storage.driver == KATA_VIRTUAL_VOLUME_IMAGE_GUEST_PULL {
|
||||
guest_pull = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if guest_pull {
|
||||
if let Some(a) = oci.annotations() {
|
||||
if ImageService::is_sandbox(a) {
|
||||
return ImageService::get_pause_image_process();
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(ocip.clone())
|
||||
}
|
||||
|
||||
/// Set proxy environment from AGENT_CONFIG
|
||||
pub async fn set_proxy_env_vars() {
|
||||
if env::var("HTTPS_PROXY").is_err() {
|
||||
let https_proxy = &AGENT_CONFIG.https_proxy;
|
||||
if !https_proxy.is_empty() {
|
||||
env::set_var("HTTPS_PROXY", https_proxy);
|
||||
}
|
||||
}
|
||||
|
||||
match env::var("HTTPS_PROXY") {
|
||||
Ok(val) => info!(sl(), "https_proxy is set to: {}", val),
|
||||
Err(e) => info!(sl(), "https_proxy is not set ({})", e),
|
||||
};
|
||||
|
||||
if env::var("NO_PROXY").is_err() {
|
||||
let no_proxy = &AGENT_CONFIG.no_proxy;
|
||||
if !no_proxy.is_empty() {
|
||||
env::set_var("NO_PROXY", no_proxy);
|
||||
}
|
||||
}
|
||||
|
||||
match env::var("NO_PROXY") {
|
||||
Ok(val) => info!(sl(), "no_proxy is set to: {}", val),
|
||||
Err(e) => info!(sl(), "no_proxy is not set ({})", e),
|
||||
};
|
||||
}
|
||||
|
||||
/// Init the image service
|
||||
pub async fn init_image_service() -> Result<()> {
|
||||
let image_service = ImageService::new().await?;
|
||||
*IMAGE_SERVICE.lock().await = Some(image_service);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub async fn pull_image(
|
||||
image: &str,
|
||||
cid: &str,
|
||||
image_metadata: &HashMap<String, String>,
|
||||
) -> Result<String> {
|
||||
let image_service = IMAGE_SERVICE.clone();
|
||||
let mut image_service = image_service.lock().await;
|
||||
let image_service = image_service
|
||||
.as_mut()
|
||||
.expect("Image Service not initialized");
|
||||
|
||||
image_service.pull_image(image, cid, image_metadata).await
|
||||
}
|
||||
@ -22,7 +22,7 @@ use anyhow::{anyhow, bail, Context, Result};
|
||||
use base64::Engine;
|
||||
use cfg_if::cfg_if;
|
||||
use clap::{AppSettings, Parser};
|
||||
use const_format::{concatcp, formatcp};
|
||||
use const_format::concatcp;
|
||||
use initdata::{InitdataReturnValue, AA_CONFIG_PATH, CDH_CONFIG_PATH};
|
||||
use nix::fcntl::OFlag;
|
||||
use nix::sys::reboot::{reboot, RebootMode};
|
||||
@ -38,7 +38,7 @@ use std::process::exit;
|
||||
use std::sync::Arc;
|
||||
use tracing::{instrument, span};
|
||||
|
||||
mod cdh;
|
||||
mod confidential_data_hub;
|
||||
mod config;
|
||||
mod console;
|
||||
mod device;
|
||||
@ -79,9 +79,6 @@ use tokio::{
|
||||
task::JoinHandle,
|
||||
};
|
||||
|
||||
#[cfg(feature = "guest-pull")]
|
||||
mod image;
|
||||
|
||||
mod rpc;
|
||||
mod tracer;
|
||||
|
||||
@ -110,19 +107,9 @@ const CDH_SOCKET_URI: &str = concatcp!(UNIX_SOCKET_PREFIX, CDH_SOCKET);
|
||||
|
||||
const API_SERVER_PATH: &str = "/usr/local/bin/api-server-rest";
|
||||
|
||||
/// Path of ocicrypt config file. This is used by image-rs when decrypting image.
|
||||
const OCICRYPT_CONFIG_PATH: &str = "/run/confidential-containers/ocicrypt_config.json";
|
||||
|
||||
const OCICRYPT_CONFIG: &str = formatcp!(
|
||||
r#"{{
|
||||
"key-providers": {{
|
||||
"attestation-agent": {{
|
||||
"ttrpc": "{}"
|
||||
}}
|
||||
}}
|
||||
}}"#,
|
||||
CDH_SOCKET_URI
|
||||
);
|
||||
/// Path of ocicrypt config file. This is used by CDH when decrypting image.
|
||||
/// TODO: remove this when we move the launch of CDH out of the kata-agent.
|
||||
const OCICRYPT_CONFIG_PATH: &str = "/etc/ocicrypt_config.json";
|
||||
|
||||
const DEFAULT_LAUNCH_PROCESS_TIMEOUT: i32 = 6;
|
||||
|
||||
@ -394,9 +381,6 @@ async fn start_sandbox(
|
||||
s.rtnl.handle_localhost().await?;
|
||||
}
|
||||
|
||||
#[cfg(feature = "guest-pull")]
|
||||
image::set_proxy_env_vars().await;
|
||||
|
||||
#[cfg(feature = "agent-policy")]
|
||||
if let Err(e) = initialize_policy().await {
|
||||
error!(logger, "Failed to initialize agent policy: {:?}", e);
|
||||
@ -516,6 +500,7 @@ async fn launch_guest_component_procs(
|
||||
Some(AA_CONFIG_PATH),
|
||||
AA_ATTESTATION_SOCKET,
|
||||
DEFAULT_LAUNCH_PROCESS_TIMEOUT,
|
||||
&[],
|
||||
)
|
||||
.await
|
||||
.map_err(|e| anyhow!("launch_process {} failed: {:?}", AA_PATH, e))?;
|
||||
@ -537,6 +522,7 @@ async fn launch_guest_component_procs(
|
||||
Some(CDH_CONFIG_PATH),
|
||||
CDH_SOCKET,
|
||||
DEFAULT_LAUNCH_PROCESS_TIMEOUT,
|
||||
&[("OCICRYPT_KEYPROVIDER_CONFIG", OCICRYPT_CONFIG_PATH)],
|
||||
)
|
||||
.await
|
||||
.map_err(|e| anyhow!("launch_process {} failed: {:?}", CDH_PATH, e))?;
|
||||
@ -558,6 +544,7 @@ async fn launch_guest_component_procs(
|
||||
None,
|
||||
"",
|
||||
0,
|
||||
&[],
|
||||
)
|
||||
.await
|
||||
.map_err(|e| anyhow!("launch_process {} failed: {:?}", API_SERVER_PATH, e))?;
|
||||
@ -580,9 +567,7 @@ async fn init_attestation_components(
|
||||
match tokio::fs::metadata(CDH_SOCKET).await {
|
||||
Ok(md) => {
|
||||
if md.file_type().is_socket() {
|
||||
cdh::init_cdh_client(CDH_SOCKET_URI).await?;
|
||||
fs::write(OCICRYPT_CONFIG_PATH, OCICRYPT_CONFIG.as_bytes())?;
|
||||
env::set_var("OCICRYPT_KEYPROVIDER_CONFIG", OCICRYPT_CONFIG_PATH);
|
||||
confidential_data_hub::init_cdh_client(CDH_SOCKET_URI).await?;
|
||||
} else {
|
||||
debug!(logger, "File {} is not a socket", CDH_SOCKET);
|
||||
}
|
||||
@ -624,6 +609,7 @@ async fn launch_process(
|
||||
config: Option<&str>,
|
||||
unix_socket_path: &str,
|
||||
timeout_secs: i32,
|
||||
envs: &[(&str, &str)],
|
||||
) -> Result<()> {
|
||||
if !Path::new(path).exists() {
|
||||
bail!("path {} does not exist.", path);
|
||||
@ -640,7 +626,12 @@ async fn launch_process(
|
||||
tokio::fs::remove_file(unix_socket_path).await?;
|
||||
}
|
||||
|
||||
tokio::process::Command::new(path).args(args).spawn()?;
|
||||
let mut process = tokio::process::Command::new(path);
|
||||
process.args(args);
|
||||
for (k, v) in envs {
|
||||
process.env(k, v);
|
||||
}
|
||||
process.spawn()?;
|
||||
if !unix_socket_path.is_empty() && timeout_secs > 0 {
|
||||
wait_for_path_to_exist(logger, unix_socket_path, timeout_secs).await?;
|
||||
}
|
||||
|
||||
@ -57,7 +57,7 @@ use rustjail::process::ProcessOperations;
|
||||
|
||||
#[cfg(target_arch = "s390x")]
|
||||
use crate::ccw;
|
||||
use crate::cdh;
|
||||
use crate::confidential_data_hub::image::KATA_IMAGE_WORK_DIR;
|
||||
use crate::device::block_device_handler::get_virtio_blk_pci_device_name;
|
||||
#[cfg(target_arch = "s390x")]
|
||||
use crate::device::network_device_handler::wait_for_ccw_net_interface;
|
||||
@ -65,9 +65,6 @@ use crate::device::network_device_handler::wait_for_ccw_net_interface;
|
||||
use crate::device::network_device_handler::wait_for_pci_net_interface;
|
||||
use crate::device::{add_devices, handle_cdi_devices, update_env_pci};
|
||||
use crate::features::get_build_features;
|
||||
#[cfg(feature = "guest-pull")]
|
||||
use crate::image::KATA_IMAGE_WORK_DIR;
|
||||
use crate::linux_abi::*;
|
||||
use crate::metrics::get_metrics;
|
||||
use crate::mount::baremount;
|
||||
use crate::namespace::{NSTYPEIPC, NSTYPEPID, NSTYPEUTS};
|
||||
@ -80,6 +77,7 @@ use crate::storage::{add_storages, update_ephemeral_mounts, STORAGE_HANDLERS};
|
||||
use crate::util;
|
||||
use crate::version::{AGENT_VERSION, API_VERSION};
|
||||
use crate::AGENT_CONFIG;
|
||||
use crate::{confidential_data_hub, linux_abi::*};
|
||||
|
||||
use crate::trace_rpc_call;
|
||||
use crate::tracer::extract_carrier_from_ttrpc;
|
||||
@ -87,9 +85,6 @@ use crate::tracer::extract_carrier_from_ttrpc;
|
||||
#[cfg(feature = "agent-policy")]
|
||||
use crate::policy::{do_set_policy, is_allowed};
|
||||
|
||||
#[cfg(feature = "guest-pull")]
|
||||
use crate::image;
|
||||
|
||||
use opentelemetry::global;
|
||||
use tracing::span;
|
||||
use tracing_opentelemetry::OpenTelemetrySpanExt;
|
||||
@ -112,7 +107,6 @@ use kata_types::k8s;
|
||||
|
||||
pub const CONTAINER_BASE: &str = "/run/kata-containers";
|
||||
const MODPROBE_PATH: &str = "/sbin/modprobe";
|
||||
#[cfg(feature = "guest-pull")]
|
||||
const TRUSTED_IMAGE_STORAGE_DEVICE: &str = "/dev/trusted_store";
|
||||
/// the iptables seriers binaries could appear either in /sbin
|
||||
/// or /usr/sbin, we need to check both of them
|
||||
@ -242,7 +236,6 @@ impl AgentService {
|
||||
handle_cdi_devices(&sl(), &mut oci, "/var/run/cdi", AGENT_CONFIG.cdi_timeout).await?;
|
||||
|
||||
// Handle trusted storage configuration before mounting any storage
|
||||
#[cfg(feature = "guest-pull")]
|
||||
cdh_handler_trusted_storage(&mut oci)
|
||||
.await
|
||||
.map_err(|e| anyhow!("failed to handle trusted storage: {}", e))?;
|
||||
@ -319,20 +312,14 @@ impl AgentService {
|
||||
|
||||
let pipe_size = AGENT_CONFIG.container_pipe_size;
|
||||
|
||||
let p = if let Some(p) = oci.process() {
|
||||
#[cfg(feature = "guest-pull")]
|
||||
{
|
||||
let new_p = image::get_process(p, &oci, req.storages.clone())?;
|
||||
Process::new(&sl(), &new_p, cid.as_str(), true, pipe_size, proc_io)?
|
||||
}
|
||||
|
||||
#[cfg(not(feature = "guest-pull"))]
|
||||
Process::new(&sl(), p, cid.as_str(), true, pipe_size, proc_io)?
|
||||
} else {
|
||||
let Some(p) = oci.process() else {
|
||||
info!(sl(), "no process configurations!");
|
||||
return Err(anyhow!(nix::Error::EINVAL));
|
||||
};
|
||||
|
||||
let new_p = confidential_data_hub::image::get_process(p, &oci, req.storages.clone())?;
|
||||
let p = Process::new(&sl(), &new_p, cid.as_str(), true, pipe_size, proc_io)?;
|
||||
|
||||
// if starting container failed, we will do some rollback work
|
||||
// to ensure no resources are leaked.
|
||||
if let Err(err) = ctr.start(p).await {
|
||||
@ -1332,9 +1319,6 @@ impl agent_ttrpc::AgentService for AgentService {
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "guest-pull")]
|
||||
image::init_image_service().await.map_ttrpc_err(same)?;
|
||||
|
||||
Ok(Empty::new())
|
||||
}
|
||||
|
||||
@ -2274,9 +2258,8 @@ fn is_sealed_secret_path(source_path: &str) -> bool {
|
||||
.any(|suffix| source_path.ends_with(suffix))
|
||||
}
|
||||
|
||||
#[cfg(feature = "guest-pull")]
|
||||
async fn cdh_handler_trusted_storage(oci: &mut Spec) -> Result<()> {
|
||||
if !cdh::is_cdh_client_initialized().await {
|
||||
if !confidential_data_hub::is_cdh_client_initialized() {
|
||||
return Ok(());
|
||||
}
|
||||
let linux = oci
|
||||
@ -2301,7 +2284,13 @@ async fn cdh_handler_trusted_storage(oci: &mut Spec) -> Result<()> {
|
||||
("encryptType".to_string(), "LUKS".to_string()),
|
||||
("dataIntegrity".to_string(), secure_storage_integrity),
|
||||
]);
|
||||
cdh::secure_mount("BlockDevice", &options, vec![], KATA_IMAGE_WORK_DIR).await?;
|
||||
confidential_data_hub::secure_mount(
|
||||
"BlockDevice",
|
||||
&options,
|
||||
vec![],
|
||||
KATA_IMAGE_WORK_DIR,
|
||||
)
|
||||
.await?;
|
||||
break;
|
||||
}
|
||||
}
|
||||
@ -2310,7 +2299,7 @@ async fn cdh_handler_trusted_storage(oci: &mut Spec) -> Result<()> {
|
||||
}
|
||||
|
||||
async fn cdh_handler_sealed_secrets(oci: &mut Spec) -> Result<()> {
|
||||
if !cdh::is_cdh_client_initialized().await {
|
||||
if !confidential_data_hub::is_cdh_client_initialized() {
|
||||
return Ok(());
|
||||
}
|
||||
let process = oci
|
||||
@ -2319,7 +2308,7 @@ async fn cdh_handler_sealed_secrets(oci: &mut Spec) -> Result<()> {
|
||||
.ok_or_else(|| anyhow!("Spec didn't contain process field"))?;
|
||||
if let Some(envs) = process.env_mut().as_mut() {
|
||||
for env in envs.iter_mut() {
|
||||
match cdh::unseal_env(env).await {
|
||||
match confidential_data_hub::unseal_env(env).await {
|
||||
Ok(unsealed_env) => *env = unsealed_env.to_string(),
|
||||
Err(e) => {
|
||||
warn!(sl(), "Failed to unseal secret: {}", e)
|
||||
@ -2357,7 +2346,7 @@ async fn cdh_handler_sealed_secrets(oci: &mut Spec) -> Result<()> {
|
||||
// But currently there is no quick way to determine which volume-mount is referring
|
||||
// to a sealed secret without reading the file.
|
||||
// And relying on file naming heuristic is inflexible. So we are going with this approach.
|
||||
if let Err(e) = cdh::unseal_file(source_path).await {
|
||||
if let Err(e) = confidential_data_hub::unseal_file(source_path).await {
|
||||
warn!(
|
||||
sl(),
|
||||
"Failed to unseal file: {:?}, Error: {:?}", source_path, e
|
||||
|
||||
@ -4,12 +4,15 @@
|
||||
//
|
||||
|
||||
use super::new_device;
|
||||
use crate::image;
|
||||
use crate::confidential_data_hub;
|
||||
use crate::confidential_data_hub::image::{is_sandbox, unpack_pause_image};
|
||||
use crate::rpc::CONTAINER_BASE;
|
||||
use crate::storage::{StorageContext, StorageHandler};
|
||||
use anyhow::{anyhow, Result};
|
||||
use kata_types::mount::KATA_VIRTUAL_VOLUME_IMAGE_GUEST_PULL;
|
||||
use kata_types::mount::{ImagePullVolume, StorageDevice};
|
||||
use protocols::agent::Storage;
|
||||
use safe_path::scoped_join;
|
||||
use std::sync::Arc;
|
||||
use tracing::instrument;
|
||||
|
||||
@ -53,7 +56,35 @@ impl StorageHandler for ImagePullHandler {
|
||||
.cid
|
||||
.clone()
|
||||
.ok_or_else(|| anyhow!("failed to get container id"))?;
|
||||
let bundle_path = image::pull_image(image_name, &cid, &image_pull_volume.metadata).await?;
|
||||
|
||||
info!(
|
||||
ctx.logger,
|
||||
"image metadata: {:?}", image_pull_volume.metadata
|
||||
);
|
||||
if is_sandbox(&image_pull_volume.metadata) {
|
||||
let mount_path = unpack_pause_image(&cid)?;
|
||||
return new_device(mount_path);
|
||||
}
|
||||
|
||||
// generated bundles with rootfs and config.json will store under CONTAINER_BASE/cid/images.
|
||||
let bundle_path = scoped_join(CONTAINER_BASE, &cid)?;
|
||||
let bundle_path = match confidential_data_hub::pull_image(image_name, bundle_path).await {
|
||||
Ok(path) => {
|
||||
info!(
|
||||
ctx.logger,
|
||||
"pull and unpack image {image_name}, cid: {cid} succeeded."
|
||||
);
|
||||
path
|
||||
}
|
||||
Err(e) => {
|
||||
error!(
|
||||
ctx.logger,
|
||||
"pull and unpack image {image_name}, cid: {cid} failed with {:?}.",
|
||||
e.to_string()
|
||||
);
|
||||
return Err(e);
|
||||
}
|
||||
};
|
||||
|
||||
new_device(bundle_path)
|
||||
}
|
||||
|
||||
@ -24,7 +24,6 @@ use self::bind_watcher_handler::BindWatcherHandler;
|
||||
use self::block_handler::{PmemHandler, ScsiHandler, VirtioBlkMmioHandler, VirtioBlkPciHandler};
|
||||
use self::ephemeral_handler::EphemeralHandler;
|
||||
use self::fs_handler::{OverlayfsHandler, Virtio9pHandler, VirtioFsHandler};
|
||||
#[cfg(feature = "guest-pull")]
|
||||
use self::image_pull_handler::ImagePullHandler;
|
||||
use self::local_handler::LocalHandler;
|
||||
use crate::mount::{baremount, is_mounted, remove_mounts};
|
||||
@ -36,7 +35,6 @@ mod bind_watcher_handler;
|
||||
mod block_handler;
|
||||
mod ephemeral_handler;
|
||||
mod fs_handler;
|
||||
#[cfg(feature = "guest-pull")]
|
||||
mod image_pull_handler;
|
||||
mod local_handler;
|
||||
|
||||
@ -148,7 +146,6 @@ lazy_static! {
|
||||
Arc::new(BindWatcherHandler {}),
|
||||
#[cfg(target_arch = "s390x")]
|
||||
Arc::new(self::block_handler::VirtioBlkCcwHandler {}),
|
||||
#[cfg(feature = "guest-pull")]
|
||||
Arc::new(ImagePullHandler {}),
|
||||
];
|
||||
|
||||
|
||||
@ -28,6 +28,21 @@ message SecureMountResponse {
|
||||
string mount_path = 1;
|
||||
}
|
||||
|
||||
message ImagePullRequest {
|
||||
// - `image_url`: The reference of the image to pull
|
||||
string image_url = 1;
|
||||
|
||||
// - `bundle_path`: The path to store the OCI bundle. This path
|
||||
// should be created by client, and initially empty. After the RPC is
|
||||
// called, a mounted `rootfs` directory under the this path. Note
|
||||
// that this path is CDH's root filesystem, not the caller's root filesystem.
|
||||
// However, usually the caller (kata-agent) and the server (CDH) runs on the same
|
||||
// root, so it's safe to use an absolute path of kata-agent.
|
||||
string bundle_path = 2;
|
||||
}
|
||||
|
||||
message ImagePullResponse {}
|
||||
|
||||
service SealedSecretService {
|
||||
rpc UnsealSecret(UnsealSecretInput) returns (UnsealSecretOutput) {};
|
||||
}
|
||||
@ -46,4 +61,10 @@ message GetResourceResponse {
|
||||
|
||||
service GetResourceService {
|
||||
rpc GetResource(GetResourceRequest) returns (GetResourceResponse) {};
|
||||
}
|
||||
|
||||
// ImagePullService is used to pull images from a remote registry
|
||||
// and mount the resulting root filesystems.
|
||||
service ImagePullService {
|
||||
rpc PullImage(ImagePullRequest) returns (ImagePullResponse) {};
|
||||
}
|
||||
@ -87,7 +87,7 @@ function setup_kbs_credentials() {
|
||||
echo "Pod ${kata_pod}: $(cat ${kata_pod})"
|
||||
|
||||
assert_pod_fail "${kata_pod}"
|
||||
assert_logs_contain "${node}" kata "${node_start_time}" "failed to pull image"
|
||||
assert_logs_contain "${node}" kata "${node_start_time}" "Not authorized"
|
||||
}
|
||||
|
||||
@test "Test that creating a container from an authenticated image, with no credentials fails" {
|
||||
@ -100,7 +100,7 @@ function setup_kbs_credentials() {
|
||||
echo "Pod ${kata_pod}: $(cat ${kata_pod})"
|
||||
|
||||
assert_pod_fail "${kata_pod}"
|
||||
assert_logs_contain "${node}" kata "${node_start_time}" "failed to pull image"
|
||||
assert_logs_contain "${node}" kata "${node_start_time}" "Not authorized"
|
||||
}
|
||||
|
||||
teardown() {
|
||||
|
||||
@ -52,8 +52,7 @@ function setup_kbs_decryption_key() {
|
||||
echo "Pod ${kata_pod}: $(cat ${kata_pod})"
|
||||
|
||||
assert_pod_fail "${kata_pod}"
|
||||
assert_logs_contain "${node}" kata "${node_start_time}" 'decrypt image (unwrap key) failed'
|
||||
assert_logs_contain "${node}" kata "${node_start_time}" 'kms interface when get KEK failed'
|
||||
assert_logs_contain "${node}" kata "${node_start_time}" 'Failed to decrypt the image layer, please ensure that the decryption key is placed and correct'
|
||||
}
|
||||
|
||||
|
||||
@ -80,7 +79,7 @@ function setup_kbs_decryption_key() {
|
||||
echo "Pod ${kata_pod}: $(cat ${kata_pod})"
|
||||
|
||||
assert_pod_fail "${kata_pod}"
|
||||
assert_logs_contain "${node}" kata "${node_start_time}" 'decrypt image (unwrap key) failed'
|
||||
assert_logs_contain "${node}" kata "${node_start_time}" 'Failed to decrypt the image layer, please ensure that the decryption key is placed and correct'
|
||||
}
|
||||
|
||||
teardown() {
|
||||
|
||||
@ -97,7 +97,7 @@ EOF
|
||||
echo "Pod ${kata_pod}: $(cat ${kata_pod})"
|
||||
|
||||
assert_pod_fail "${kata_pod}"
|
||||
assert_logs_contain "${node}" kata "${node_start_time}" "failed to pull image"
|
||||
assert_logs_contain "${node}" kata "${node_start_time}" "Image policy rejected: Denied by policy"
|
||||
}
|
||||
|
||||
@test "Create a pod from a signed image, on a 'restricted registry' is successful" {
|
||||
@ -123,7 +123,7 @@ EOF
|
||||
echo "Pod ${kata_pod}: $(cat ${kata_pod})"
|
||||
|
||||
assert_pod_fail "${kata_pod}"
|
||||
assert_logs_contain "${node}" kata "${node_start_time}" "failed to pull image"
|
||||
assert_logs_contain "${node}" kata "${node_start_time}" "Image policy rejected: Denied by policy"
|
||||
}
|
||||
|
||||
@test "Create a pod from an unsigned image, on a 'restricted registry' works if policy files isn't set" {
|
||||
|
||||
@ -98,10 +98,6 @@ algorithm = "sha256"
|
||||
[data]
|
||||
"aa.toml" = '''
|
||||
[token_configs]
|
||||
[token_configs.coco_as]
|
||||
# TODO: we should fix this on AA side to set this a default value if not set.
|
||||
url = "${CC_KBS_ADDRESS}"
|
||||
|
||||
[token_configs.kbs]
|
||||
url = "${CC_KBS_ADDRESS}"
|
||||
'''
|
||||
@ -165,7 +161,7 @@ EOF
|
||||
echo "Pod ${kata_pod}: $(cat ${kata_pod})"
|
||||
|
||||
assert_pod_fail "${kata_pod}"
|
||||
assert_logs_contain "${node}" kata "${node_start_time}" "failed to pull image"
|
||||
assert_logs_contain "${node}" kata "${node_start_time}" "Image policy rejected: Denied by policy"
|
||||
}
|
||||
|
||||
@test "Test that creating a container from an rejected image not configured by initdata, fails according to CDH error" {
|
||||
|
||||
@ -17,8 +17,6 @@ RUST_VERSION="null"
|
||||
AGENT_BIN=${AGENT_BIN:-kata-agent}
|
||||
AGENT_INIT=${AGENT_INIT:-no}
|
||||
MEASURED_ROOTFS=${MEASURED_ROOTFS:-no}
|
||||
# The kata agent enables guest-pull feature.
|
||||
PULL_TYPE=${PULL_TYPE:-default}
|
||||
KERNEL_MODULES_DIR=${KERNEL_MODULES_DIR:-""}
|
||||
OSBUILDER_VERSION="unknown"
|
||||
DOCKER_RUNTIME=${DOCKER_RUNTIME:-runc}
|
||||
@ -747,7 +745,7 @@ EOF
|
||||
git checkout "${AGENT_VERSION}" && OK "git checkout successful" || die "checkout agent ${AGENT_VERSION} failed!"
|
||||
fi
|
||||
make clean
|
||||
make LIBC=${LIBC} INIT=${AGENT_INIT} SECCOMP=${SECCOMP} AGENT_POLICY=${AGENT_POLICY} PULL_TYPE=${PULL_TYPE}
|
||||
make LIBC=${LIBC} INIT=${AGENT_INIT} SECCOMP=${SECCOMP} AGENT_POLICY=${AGENT_POLICY}
|
||||
make install DESTDIR="${ROOTFS_DIR}" LIBC=${LIBC} INIT=${AGENT_INIT}
|
||||
if [ "${SECCOMP}" == "yes" ]; then
|
||||
rm -rf "${libseccomp_install_dir}" "${gperf_install_dir}"
|
||||
|
||||
@ -47,7 +47,6 @@ build_initrd() {
|
||||
AGENT_TARBALL="${AGENT_TARBALL}" \
|
||||
AGENT_INIT="${AGENT_INIT:-no}" \
|
||||
AGENT_POLICY="${AGENT_POLICY:-}" \
|
||||
PULL_TYPE="${PULL_TYPE:-default}" \
|
||||
COCO_GUEST_COMPONENTS_TARBALL="${COCO_GUEST_COMPONENTS_TARBALL:-}" \
|
||||
PAUSE_IMAGE_TARBALL="${PAUSE_IMAGE_TARBALL:-}" \
|
||||
GUEST_HOOKS_TARBALL="${GUEST_HOOKS_TARBALL}"
|
||||
@ -77,7 +76,6 @@ build_image() {
|
||||
ROOTFS_BUILD_DEST="${builddir}/rootfs-image" \
|
||||
AGENT_TARBALL="${AGENT_TARBALL}" \
|
||||
AGENT_POLICY="${AGENT_POLICY:-}" \
|
||||
PULL_TYPE="${PULL_TYPE:-default}" \
|
||||
COCO_GUEST_COMPONENTS_TARBALL="${COCO_GUEST_COMPONENTS_TARBALL:-}" \
|
||||
PAUSE_IMAGE_TARBALL="${PAUSE_IMAGE_TARBALL:-}" \
|
||||
GUEST_HOOKS_TARBALL="${GUEST_HOOKS_TARBALL}"
|
||||
|
||||
@ -100,7 +100,6 @@ TOOLS_CONTAINER_BUILDER="${TOOLS_CONTAINER_BUILDER:-}"
|
||||
VIRTIOFSD_CONTAINER_BUILDER="${VIRTIOFSD_CONTAINER_BUILDER:-}"
|
||||
AGENT_INIT="${AGENT_INIT:-no}"
|
||||
MEASURED_ROOTFS="${MEASURED_ROOTFS:-}"
|
||||
PULL_TYPE="${PULL_TYPE:-guest-pull}"
|
||||
USE_CACHE="${USE_CACHE:-}"
|
||||
BUSYBOX_CONF_FILE=${BUSYBOX_CONF_FILE:-}
|
||||
NVIDIA_GPU_STACK="${NVIDIA_GPU_STACK:-}"
|
||||
@ -140,7 +139,6 @@ docker run \
|
||||
--env VIRTIOFSD_CONTAINER_BUILDER="${VIRTIOFSD_CONTAINER_BUILDER}" \
|
||||
--env AGENT_INIT="${AGENT_INIT}" \
|
||||
--env MEASURED_ROOTFS="${MEASURED_ROOTFS}" \
|
||||
--env PULL_TYPE="${PULL_TYPE}" \
|
||||
--env USE_CACHE="${USE_CACHE}" \
|
||||
--env BUSYBOX_CONF_FILE="${BUSYBOX_CONF_FILE}" \
|
||||
--env NVIDIA_GPU_STACK="${NVIDIA_GPU_STACK}" \
|
||||
|
||||
@ -43,7 +43,6 @@ readonly se_image_builder="${repo_root_dir}/tools/packaging/guest-image/build_se
|
||||
ARCH=${ARCH:-$(uname -m)}
|
||||
BUSYBOX_CONF_FILE="${BUSYBOX_CONF_FILE:-}"
|
||||
MEASURED_ROOTFS=${MEASURED_ROOTFS:-no}
|
||||
PULL_TYPE=${PULL_TYPE:-guest-pull}
|
||||
USE_CACHE="${USE_CACHE:-"yes"}"
|
||||
ARTEFACT_REGISTRY="${ARTEFACT_REGISTRY:-ghcr.io}"
|
||||
ARTEFACT_REPOSITORY="${ARTEFACT_REPOSITORY:-kata-containers}"
|
||||
@ -430,7 +429,6 @@ install_image_confidential() {
|
||||
else
|
||||
export MEASURED_ROOTFS=yes
|
||||
fi
|
||||
export PULL_TYPE=default
|
||||
install_image "confidential"
|
||||
}
|
||||
|
||||
@ -529,7 +527,6 @@ install_initrd() {
|
||||
#Install guest initrd for confidential guests
|
||||
install_initrd_confidential() {
|
||||
export MEASURED_ROOTFS=no
|
||||
export PULL_TYPE=default
|
||||
install_initrd "confidential"
|
||||
}
|
||||
|
||||
@ -997,7 +994,7 @@ install_agent() {
|
||||
export GPERF_URL="$(get_from_kata_deps ".externals.gperf.url")"
|
||||
|
||||
info "build static agent"
|
||||
DESTDIR="${destdir}" AGENT_POLICY="${AGENT_POLICY}" PULL_TYPE=${PULL_TYPE} "${agent_builder}"
|
||||
DESTDIR="${destdir}" AGENT_POLICY="${AGENT_POLICY}" "${agent_builder}"
|
||||
}
|
||||
|
||||
install_coco_guest_components() {
|
||||
|
||||
@ -18,8 +18,8 @@ build_agent_from_source() {
|
||||
/usr/bin/install_libseccomp.sh /opt /opt
|
||||
|
||||
cd src/agent
|
||||
DESTDIR=${DESTDIR} AGENT_POLICY=${AGENT_POLICY} PULL_TYPE=${PULL_TYPE} make
|
||||
DESTDIR=${DESTDIR} AGENT_POLICY=${AGENT_POLICY} PULL_TYPE=${PULL_TYPE} make install
|
||||
DESTDIR=${DESTDIR} AGENT_POLICY=${AGENT_POLICY} make
|
||||
DESTDIR=${DESTDIR} AGENT_POLICY=${AGENT_POLICY} make install
|
||||
}
|
||||
|
||||
build_agent_from_source "$@"
|
||||
|
||||
@ -26,7 +26,6 @@ docker pull ${container_image} || \
|
||||
docker run --rm -i -v "${repo_root_dir}:${repo_root_dir}" \
|
||||
--env DESTDIR=${DESTDIR} \
|
||||
--env AGENT_POLICY=${AGENT_POLICY:-no} \
|
||||
--env PULL_TYPE=${PULL_TYPE:-default} \
|
||||
--env LIBSECCOMP_VERSION=${LIBSECCOMP_VERSION} \
|
||||
--env LIBSECCOMP_URL=${LIBSECCOMP_URL} \
|
||||
--env GPERF_VERSION=${GPERF_VERSION} \
|
||||
|
||||
@ -3,7 +3,7 @@
|
||||
#
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
FROM ubuntu:22.04
|
||||
FROM ubuntu:24.04
|
||||
ARG RUST_TOOLCHAIN
|
||||
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
@ -28,6 +28,7 @@ RUN apt-get update && \
|
||||
libssl-dev \
|
||||
libtss2-dev \
|
||||
make \
|
||||
cmake \
|
||||
musl-tools \
|
||||
openssl \
|
||||
perl \
|
||||
|
||||
@ -35,6 +35,7 @@ build_coco_guest_components_from_source() {
|
||||
DESTDIR="${DESTDIR}/usr/local/bin" TEE_PLATFORM=${TEE_PLATFORM} make install
|
||||
|
||||
install -D -m0755 "confidential-data-hub/hub/src/storage/scripts/luks-encrypt-storage" "${DESTDIR}/usr/local/bin/luks-encrypt-storage"
|
||||
install -D -m0644 "confidential-data-hub/hub/src/image/ocicrypt_config.json" "${DESTDIR}/etc/ocicrypt_config.json"
|
||||
popd
|
||||
}
|
||||
|
||||
|
||||
@ -143,7 +143,7 @@ assets:
|
||||
version: "jammy" # 22.04 lTS
|
||||
confidential:
|
||||
name: "ubuntu"
|
||||
version: "oracular" # 24.10
|
||||
version: "noble" # 24.04 LTS
|
||||
mariner:
|
||||
name: "cbl-mariner"
|
||||
version: "3.0"
|
||||
@ -185,7 +185,7 @@ assets:
|
||||
version: "3.18"
|
||||
confidential:
|
||||
name: "ubuntu"
|
||||
version: "jammy" # 22.04 LTS
|
||||
version: "noble" # 24.04 LTS
|
||||
nvidia-gpu:
|
||||
name: "ubuntu"
|
||||
version: "jammy" # 22.04 LTS
|
||||
@ -233,18 +233,18 @@ externals:
|
||||
coco-guest-components:
|
||||
description: "Provides attested key unwrapping for image decryption"
|
||||
url: "https://github.com/confidential-containers/guest-components/"
|
||||
version: "0a06ef241190780840fbb0542e51b198f1f72b0b"
|
||||
version: "028978dbaef2510ea92bc5038928f1d70c8aaad6"
|
||||
toolchain: "1.80.0"
|
||||
|
||||
coco-trustee:
|
||||
description: "Provides attestation and secret delivery components"
|
||||
url: "https://github.com/confidential-containers/trustee"
|
||||
version: "d9eb5e0cb0aca97abe35b58908e061850ff60a51"
|
||||
version: "a333fa27a7ce538413bad3b537ffbeacf4a349d1"
|
||||
# image / ita_image and image_tag / ita_image_tag must be in sync
|
||||
image: "ghcr.io/confidential-containers/staged-images/kbs"
|
||||
image_tag: "d9eb5e0cb0aca97abe35b58908e061850ff60a51"
|
||||
image_tag: "a333fa27a7ce538413bad3b537ffbeacf4a349d1"
|
||||
ita_image: "ghcr.io/confidential-containers/staged-images/kbs-ita-as"
|
||||
ita_image_tag: "d9eb5e0cb0aca97abe35b58908e061850ff60a51-x86_64"
|
||||
ita_image_tag: "a333fa27a7ce538413bad3b537ffbeacf4a349d1-x86_64"
|
||||
toolchain: "1.80.0"
|
||||
|
||||
crio:
|
||||
|
||||
Loading…
Reference in New Issue
Block a user