documentation: Confidential Containers architecture

Describe and explain the project architecture.

Signed-off-by: Ariel Adam <aadam@redhat.com>
Co-authored-by: Dan Middleton <dan.middleton@intel.com>
Co-authored-by: Samuel Ortiz <sameo@rivosinc.com>

README.md

@@ -20,6 +20,7 @@ delivering Confidential Computing for guest applications or data inside the TEE
 [![asciicast](https://asciinema.org/a/eGHhZdQY3uYnDalFAfuB7VYqF.svg)](https://asciinema.org/a/eGHhZdQY3uYnDalFAfuB7VYqF)
 
 - [Project Overview](./overview.md)
+- [Project Architecture](./architecture.md)
 - [Our Roadmap](./roadmap.md)
 - [Alignment with other Projects](alignment.md)
 

architecture.md

@@ -0,0 +1,203 @@
+# Confidential Computing Overview
+
+> Confidential Computing protects data in use by performing computation in a
+> hardware-based Trusted Execution Environment.
+> These secure and isolated environments prevent unauthorized access or
+> modification of applications and data while they are in use, thereby increasing
+> the security level of organizations that manage sensitive and regulated data.
+>
+> A Trusted Execution Environment (TEE) is commonly defined as an environment that
+> provides a level of assurance of data integrity, data confidentiality, and code
+> integrity. A hardware-based TEE uses hardware-backed techniques to provide
+> increased security guarantees for the execution of code and protection of data
+> within that environment.
+
+Key Characteristics of Confidential Computing, as defined by the
+[Confidential Computing Consortium](https://confidentialcomputing.io/wp-content/uploads/sites/85/2021/03/confidentialcomputing_outreach_whitepaper-8-5x11-1.pdf):
+
+- **Data confidentiality:** Unauthorized entities cannot view data while it is
+  in use within the TEE
+- **Data integrity:** Unauthorized entities cannot add, remove, or alter data
+  while it is in use within the TEE
+- **Code integrity:** Unauthorized entities cannot add, remove, or alter code
+  executing in the TEE
+
+# Confidential Computing Building Blocks
+
+## Key Components
+
+The following diagram shows how the different building blocks for Confidential
+Computing (CC) come together:
+
+![CC_all_blocks](https://github.com/confidential-containers/documentation/blob/main/images/CC_all_blocks.jpg)
+
+Let’s start by describing the lower level blocks of the CC solution
+(**colored blue**):
+
+- **Infra layer** - be it on-premises (bare metal, VMware etc.) or public clouds
+  (AWS, Azure, GCP, etc.)
+  - **Hardware with CC support** - this includes 2 distinct models:
+    - **VM-based TEEs**  - In this model, memory is encrypted along a
+      traditional VM boundary running on top of a VMM. AMD SEV-SNP, Intel TDX,
+      IBM Secure Execution and Protected Execution Functionality (PEF) are
+      examples of VM-based TEEs.
+    - **Process-based TEEs** - In this model, a process that needs to run
+      securely is divided into two components: trusted and untrusted.
+      The trusted component resides in encrypted memory and handles confidential
+      computing, while the untrusted component interfaces with the operating
+      system and propagates I/O from encrypted memory to the rest of the system.
+      Intel SGX is an example of a process-based TEE.
+- **Hypervisor** - this includes hypervisors such as QEMU/KVM, cloud hypervisor,
+  public cloud provider hypervisors etc...
+
+The **Confidential Computing services block** contains a number of services
+(**colored pink**) which are required for creating a holistic CC platform which
+the customer can then use to build their solution. This currently includes the
+following services:
+
+- **Attestation service** -  The primary purpose of the attestation service is
+  to validate the evidence provided by the hardware TEE. This is the *Verifier*,
+  as defined in the [RATS architecture](https://datatracker.ietf.org/doc/html/draft-ietf-rats-architecture-21).
+- **Key Broker Service (KBS)** -  The KBS is the *Relying Party*, as defined by
+  the [RATS architecture](https://datatracker.ietf.org/doc/html/draft-ietf-rats-architecture-21).
+  Following are its primary functions:
+  - Receive evidence from the *Attester* (confidential VM or container) via a
+    challenge-response protocol.
+  - Relay the evidence to the Attestation Service for verification.
+  - Apply appraisal policy for the returned Attestation Results to assess the
+    trustworthiness of the *Attester*.
+  - Interact with the Key Management Service to retrieve the keys and then
+    send them back to the *Attester*.
+- **Key management service** - A service for securely storing, managing and
+  backing up of cryptographic keys used by applications and users.
+- **Image build service** - Services used to build confidential containers or VM
+  images for end users.
+- **Image registry** - A service that is used to store encrypted and/or signed
+  container and VM images required for CC workloads. Examples of such registries
+  include [Quay.io](https://quay.io/), [Docker Hub](https://hub.docker.com/),
+  CSPs provided registries, etc.
+
+Both the **Confidential VM** and the ** Confidential Containers** block contain
+the **Enclave Software Stack**. This stack is made of the trusted CC components
+responsible for collectively running the whole attestation flows, and for
+example:
+- Unblocking a confidential VM boot process after injecting a guest owned secret
+  released by the relying party.
+- Decrypting and unpacking a protected container image, and running its
+  associated  workload.
+
+## Confidential Container Stack
+
+The CC components can be arranged together and configured to provide
+containerized worklooads with the Confidential Computing security guarantees.
+
+The two following sections describe a generic software architecture for
+Kubernetes pods to run in either VM-based or process-based TEEs.
+
+Although internal implementations for the two approaches differ, they share the
+same goals and attributes:
+
+- Remove cloud and infrastructure providers from the guest application Trusted
+  Computing Base (TCB).
+- Integrate natively with the Kubernetes control plane.
+- Provide an unmodified Kubernetes user and developer experience.
+- Deploy unmodified workloads.
+
+### VM-based TEE
+
+VM-based TEEs (e.g. AMD SEV, IBM SE or Intel TDX) can be used to build a
+confidential containers software architecture:
+
+![CC_TEE_container](https://github.com/confidential-containers/documentation/blob/main/images/CC_TEE_container.jpg)
+
+Following is the workflow when deploying a Kubernetes pod with VM-based TEEs:
+
+- CC workload preparation
+  - User builds the container image(s) (e.g. with tools like `podman`).
+  - User signs/encrypts the container image(s).
+  - User pushes the container image(s) to the image registry.
+- Deploying the CC workload in k8s
+  - User deploys the workload (`kubectl apply -f cc_workload.yaml`).
+  - Kubernetes schedules the workload to target host having the required
+    capability to run confidential containers.
+- CC workload execution flow (**red connector** in the diagram)
+  - Confidential containers runtime on the host starts the VM TEE (The enclave).
+  - Enclave (agent) performs remote attestation: **steps 1-2 in the diagram**.
+  - Enclave (agent) gets the keys required to verify/decrypt the containers
+    image(s):  **steps 3-4 in the diagram**.
+  - Enclave (image management) downloads the container image(s) : **step 5 in
+    the diagram**.
+  - Enclave verifies/decrypts the container image(s) : **step 6 in the diagram**.
+  - Enclave starts the container workload.
+
+### Process-based TEE
+
+The confidential containers software architecture can also be built on top of
+process-based TEEs like e.g. Intel SGX:
+
+![CC_SGX_container](https://github.com/confidential-containers/documentation/blob/main/images/CC_SGX_container.jpg)
+
+Following is the workflow when deploying a Kubernetes pod with a process-based
+TEEs. The main differences from the VM-based TEE approach are the last 3 steps
+involving interaction between 2 enclave processes:
+
+- CC workload preparation
+  - User builds the container image(s) (e.g. with tools like `podman`).
+  - User signs/encrypts the container image(s).
+  - User pushes the container image(s) to the image registry.
+- Deploying the CC workload in k8s
+  - User deploys the workload (`kubectl apply -f cc_workload.yaml`).
+  - Kubernetes schedules the workload to target host having the required
+    capability to run confidential containers.
+- CC workload execution flow (**red connector** in the diagram)
+  - Confidential containers runtime on the host starts the enclave agent.
+  - Enclave (agent) performs remote attestation: **steps 1-2 in the diagram**.
+  - Enclave (agent) gets the keys required to verify/decrypt the containers
+    image(s): **steps 3-4 in the diagram**.
+  - Enclave (image management) downloads the container image(s) : **step 5 in
+    the diagram**.
+  - Enclave verifies, decrypts and writes the container image(s) to a local
+    encrypted filesystem: **step 6 in the diagram**.
+  - The runtime starts the app enclave which reads the container bundle from the
+    encrypted filesystem.
+  - Secure use of the encrypted filesystem is facilitated by a key exchange
+    between the agent and app enclaves using either sealing or local attestation.
+
+As can be seen, the flows for process and VM-based TEEs are almost identical. It
+should be noted that process-based TEE requires a few additional software
+components like `libOS` without which the application requires re-architecting.
+This is not the case for VM-based TEEs. Conversely, Process based TEEs do not
+require a separate VM and CC-aware hypervisor.
+
+The up-to-date design for the Confidential Containers process-based architecture
+is maintained in the [enclave-cc documentation](https://github.com/confidential-containers/enclave-cc/blob/main/docs/design.md).
+
+# CNCF Confidential Containers
+
+The CNCF Confidential Containers project is an implementation of the
+confidential containers architecture described in the previous section.
+
+It relies on several major cloud native components like `containerd`, both the
+`Kata Containers` and the `Inclavare` runtimes, or the `ocicrypt` container
+image encryption APIs.
+It also depends on the standard Linux virtualization stack, including the `KVM`
+hypervisor and open source VMMs like `QEMU` or `cloud-hypervisor`.
+
+The project supports both VM-based and process-based TEEs, with the key
+objective to reuse the generic components around enclave agent and attestation
+services across both TEE models.
+
+It is a [Cloud Native Computing Foundation Sandbox](https://www.cncf.io/projects/confidential-containers/)
+project.
+
+The following diagram shows the upcoming v1 architecture to run Confidential
+Containers using VM-based TEEs and the Kata Containers runtime:
+
+![COCO_ccv1_TEE](https://github.com/confidential-containers/documentation/blob/main/images/COCO_ccv1_TEE.jpg)
+
+The following diagram shows the upcoming v1 architecture to run Confidential
+Containers using the Intel SGX process-based TEE. It relies on the
+[Inclavare Containers](https://github.com/inclavare-containers/inclavare-containers)
+runtime:
+
+![COCO_ccv1_enclave](https://github.com/confidential-containers/documentation/blob/main/images/COCO_ccv1_enclave.jpg)