sc: add Tobin to SC for NVIDIA

NVIDIA has been a major contributor to Confidential Containers and more
contributions are coming.

As such, let's expand the NVIDIA representation on the SC to two seats.

Signed-off-by: Tobin Feldman-Fitzthum <tfeldmanfitz@nvidia.com>

.github/ISSUE_TEMPLATE/release-check-list.md

@@ -20,6 +20,7 @@ flowchart LR
     Guest-Components .-> Client-tool
     Guest-Components --> enclave-agent
     enclave-cc --> kustomization.yaml
+    Operator --> versions.yaml
     Guest-Components --> versions.yaml
     Trustee --> versions.yaml
     Kata --> versions.yaml
@@ -47,7 +48,8 @@ flowchart LR
 Starting with v0.9.0 the release process no longer involves centralized dependency management.
 In other words, when doing a CoCo release, we don't push the most recent versions of the subprojects
 into Kata and enclave-cc. Instead, dependencies should be updated during the normal process of development.
-Releases of most subprojects are now decoupled from releases of the CoCo project.
+After the release, we typically cut a release of the subprojects that reflects whatever commit was used
+in the Kata release.
 
 ## The Steps
 
@@ -72,13 +74,9 @@ Identify/create the bundles that we will release for Kata and enclave-cc.
     If you absolutely cannot use a Kata release,
     you can consider releasing one of these bundles.
 
-- [ ] 3. :eyes: **Create a peer pods release**
+### Update the Operator
 
-    Create a peer pods release based on the Kata release, by following the [documented flow](https://github.com/confidential-containers/cloud-api-adaptor/blob/main/docs/Release-Process.md).
-
-### Test Release with Operator
-
-- [ ] 4. :eyes: **Check operator pre-installation and open PR if needed**
+- [ ] 3. :eyes: **Check operator pre-installation and open PR if needed**
     
     The operator uses a pre-install container to setup the node.
     Check that the container matches the dependencies used in Kata
@@ -88,7 +86,7 @@ Identify/create the bundles that we will release for Kata and enclave-cc.
         * Compare the `nydus-snapshotter` version in Kata [versions.yaml](https://github.com/kata-containers/kata-containers/blob/main/versions.yaml) (search for `nydus-snapshotter` and check its `version` field) with the [Makefile](https://github.com/confidential-containers/operator/blob/main/install/pre-install-payload/Makefile) (check the `NYDUS_SNAPSHOTTER_VERSION` value) for the operator pre-install container.
             * **If they do not match, stop and open a PR now. In the PR, update the operator's Makefile to match the version used in kata. After the PR is merged, continue.**
 
-- [ ] 5. :wrench: **Open a PR to the operator to update the release artifacts**
+- [ ] 4. :wrench: **Open a PR to the operator to update the release artifacts**
 
     Update the operator to use the payloads identified in steps 1, 2, 3, and 4.
 
@@ -114,13 +112,39 @@ Identify/create the bundles that we will release for Kata and enclave-cc.
 
 ### Final Touches
 
-- [ ] 6. :trophy: **Cut an operator release using the GitHub release tool**
+- [ ] 5. :trophy: **Cut an operator release using the GitHub release tool**
+
+- [ ] 6. :wrench: **Create a peer pods release**
+
+    Create a peer pods release based on the Kata release, by following the [documented flow](https://github.com/confidential-containers/cloud-api-adaptor/blob/main/docs/Release-Process.md).
 
 - [ ] 7. :green_book: **Make sure to update the [release notes](https://github.com/confidential-containers/confidential-containers/tree/main/releases) and tag/release the confidential-containers repo using the GitHub release tool.**
 
-- [ ] 8. :hammer: **Poke Wainer Moschetta (@wainersm) to update the release to the OperatorHub. Find the documented flow [here](https://github.com/confidential-containers/operator/blob/main/docs/OPERATOR_HUB.md).**
-
 ### Post-release
 
-- [ ] 9. :wrench: **Open a PR to the operator to go back to latest payloads after release**
+- [ ] 8. :wrench: **Open a PR to the operator to go back to latest payloads after release**
     After the release, the operator's payloads need to go back to what they were (e.g. using "latest" instead of a specific commit sha). As an example, the v0.9.0-alpha1 release applied [these changes](https://github.com/confidential-containers/operator/pull/389/files). You should use `git revert -s` for this.
+
+- [ ] 9. :pushpin: **Tag the version of guest-components used in the release**.
+
+    Go look at [versions.yaml](https://github.com/kata-containers/kata-containers/blob/main/versions.yaml)
+    in Kata Containers and find the version of the guest-components that was used in the Kata release.
+    Tag this commit in guest-components with the latest version of guest components.
+    Note that the version of guest-components might not be the same as the version of CoCo.
+
+- [ ] 10. :scissors: **Cut a release of guest-components using GitHub release tool**
+
+- [ ] 11. :pushpin: **Tag the version of Trustee used in the release**
+
+    Follow the same process as step 9 but for Trustee.
+
+- [ ] 12. :scissors: **Cut a release of Trustee using GitHub release tool**
+
+- [ ] 13. :wrench: **Tag the Trustee release images**
+
+    Use the Trustee release helper script to push the CI images corresponding to the released hash
+    as the release images.
+
+- [ ] 14. :pushpin: **Tag the latest version of the website for the release**
+
+    Make sure the website is up-to-date for the latest release, and then tag the repo.

CONTRIBUTING.md

@@ -78,6 +78,8 @@ with the community as early as possible. Consider making an `RFC` issue
 that explains the changes. You might also try to break large contributions
 into smaller steps.
 
+Any new feature must be accompanied by new unit tests.
+
 ### Making a Pull Request
 
 If you aren't familiar with Git or the GitHub PR workflow, take a look at [this section](https://github.com/kata-containers/community/blob/main/CONTRIBUTING.md#github-workflow)

MAINTAINERS

@@ -7,12 +7,11 @@ bpradipt, Pradipta Banerjee, Redhat
 peterzcst, Peter Zhu, Intel
 mythi, Mikko Ylinen, Intel
 magowan, James Magowan, IBM
-fitzthum, Tobin Feldman-Fitzthum, IBM
 jiazhang0, Zhang Jia, Alibaba
 jiangliu, Jiang Liu, Alibaba
-larrydewey, Larry Dewey, AMD
 ryansavino, Ryan Savino, AMD
 sameo, Samuel Ortiz, Rivos
 zvonkok, Zvonko Kaiser, NVIDIA
+fitzthum, Tobin Feldman-Fitzthum, NVIDIA
 vbatts, Vincent Batts, Microsoft
 danmihai1, Dan Mihai, Microsoft

governance.md

@@ -85,18 +85,19 @@ Further, as leaders in the community, the SC members will make themselves famili
 
 The current members of the SC are:
 
-* Larry Dewey (@larrydewey) and Ryan Savino (@ryansavino) - AMD
+* Ryan Savino (@ryansavino) - AMD
 * Jiang Liu (@jiangliu) and Jia Zhang (@jiazhang0) - Alibaba
-* James Magowan (@magowan)  and Tobin Feldman-Fitzthum (@fitzthum) - IBM
+* James Magowan (@magowan) and Nina Goradia (@ninag) - IBM
 * Peter Zhu (@peterzcst) and Mikko Ylinen (@mythi) - Intel
 * Pradipta Banerjee (@bpradipt)  and Ariel Adam (@ariel-adam) - Red Hat
 * Samuel Ortiz (@sameo) - Rivos
-* Zvonko Kaiser (@zvonkok) - NVIDIA
+* Zvonko Kaiser (@zvonkok) and Tobin Feldman-Fitzthum (@fitzthum) - NVIDIA
 * Vincent Batts (@vbatts) and Dan Mihai (@danmihai1) - Microsoft
 
 ### Emeritus Members
 
 * Dan Middleton [dcmiddle](https://github.com/dcmiddle) (he/him)
+* Larry Dewey (@larrydewey) - AMD
 
 #### Selection
 

guides/coco-dev.md

@@ -77,16 +77,13 @@ and confidential hardware.
 
 CoCo has a modular attestation interface and there are a few options for attestation.
 CoCo provides a generic Key Broker Service (KBS) that the rest of this guide will be focused on.
-The SEV runtime class uses `simple-kbs`, which is described in the [SEV guide](../guides/sev.md).
 
 ### Select Runtime Class
 
 To use CoCo with confidential hardware, first switch to the appropriate runtime class.
 TDX has one runtime class, `kata-qemu-tdx`. 
 
-For SEV(-ES) use the `kata-qemu-sev` runtime class and follow the [SEV guide](../guides/sev.md). 
-
-For SNP, use the `kata-qemu-snp` runtime class and follow the [SNP guide](../guides/snp.md).
+For SNP, use the `kata-qemu-snp` runtime class and follow the [SNP guide](https://confidentialcontainers.org/docs/examples/snp-container-launch/).
 
 For `enclave-cc` follow the [enclave-cc guide](../guides/enclave-cc.md).
 

guides/enclave-cc.md

@@ -19,7 +19,7 @@ https://github.com/intel/SGXDataCenterAttestationPrimitives) running on every SG
 ## Configuring enclave-cc custom resource to use a different KBC
 
 **Note** Before configuring KBC, please refer to the
-[guide](../quickstart.md#deploy-and-configure-tenant-side-coco-key-broker-system-cluster) to deploy KBS cluster.
+[guide](coco-dev.md#deploy-and-configure-tenant-side-coco-key-broker-system-cluster) to deploy KBS cluster.
 
 **Note** The KBC configuration changes to the enclave-cc custom resource yaml 
 must be made **before** deploying it. 

guides/ephemeral_storage.md

@@ -47,7 +47,6 @@ spec:
       storage: 1Gi
   storageClassName: open-local-lvm
 ```
-Before deploy the workload, we can follow this [documentation](https://github.com/kata-containers/kata-containers/blob/CCv0/docs/how-to/how-to-build-and-test-ccv0.md) and use [ccv0.sh](https://github.com/kata-containers/kata-containers/blob/CCv0/docs/how-to/ccv0.sh) to enable CoCo console debug(optional, check whether working as expected).
 
 Create the workload:
 ```sh

guides/sev.md

@@ -1,286 +0,0 @@
-# SEV-ES Guide
-
-## Platform Setup
-
-
-> [!WARNING]
->
-> In order to launch SEV or SNP memory encrypted guests, the host must be prepared with a compatible kernel, `6.8.0-rc5-next-20240221-snp-host-cc2568386`. AMD custom changes and required components and repositories will eventually be upstreamed. 
-
-> [Sev-utils](https://github.com/amd/sev-utils/blob/coco-202402240000/docs/snp.md) is an easy way to install the required host kernel, but it will unnecessarily build AMD compatible guest kernel, OVMF, and QEMU components. The additional components can be used with the script utility to test launch and attest a base QEMU SNP guest. However, for the CoCo use case, they are already packaged and delivered with Kata.
-Alternatively, refer to the [AMDESE guide](https://github.com/confidential-containers/amdese-amdsev/tree/amd-snp-202402240000?tab=readme-ov-file#prepare-host) to manually build the host kernel and other components.
-
-## Getting Started
-
-This guide covers platform-specific setup for SEV and walks through the complete flows for the different CoCo use cases:
-
-- [Container Launch with Memory Encryption](#container-launch-with-memory-encryption)
-- [Pre-Attestation Utilizing Signed and Encrypted Images](#pre-attestation-utilizing-signed-and-encrypted-images)
-
-## Container Launch With Memory Encryption
-
-### Launch a Confidential Service 
-
-To launch a container with SEV memory encryption, the SEV runtime class (`kata-qemu-sev`) must be specified as an annotation in the yaml. A base alpine docker container ([Dockerfile](https://github.com/kata-containers/kata-containers/blob/main/tests/integration/kubernetes/runtimeclass_workloads/confidential/unencrypted/Dockerfile)) has been previously built for testing purposes. This image has also been prepared with SSH access and provisioned with a [SSH public key](https://github.com/kata-containers/kata-containers/blob/main/tests/integration/kubernetes/runtimeclass_workloads/confidential/unencrypted/ssh/unencrypted.pub) for validation purposes.
-
-Here is a sample service yaml specifying the SEV runtime class: 
-
-```yaml
-kind: Service
-apiVersion: v1
-metadata:
-  name: "confidential-unencrypted"
-spec:
-  selector:
-    app: "confidential-unencrypted"
-  ports:
-  - port: 22
----
-kind: Deployment
-apiVersion: apps/v1
-metadata:
-  name: "confidential-unencrypted"
-spec:
-  selector:
-    matchLabels:
-      app: "confidential-unencrypted"
-  template:
-    metadata:
-      labels:
-        app: "confidential-unencrypted"
-      annotations:
-        io.containerd.cri.runtime-handler: kata-qemu-sev
-    spec:
-      runtimeClassName: kata-qemu-sev
-      containers:
-      - name: "confidential-unencrypted"
-        image: ghcr.io/kata-containers/test-images:unencrypted-nightly
-        imagePullPolicy: Always
-```
-
-Save the contents of this yaml to a file called `confidential-unencrypted.yaml`.
-
-Start the service:
-
-```shell 
-kubectl apply -f confidential-unencrypted.yaml
-```
-
-Check for errors:
-
-```shell
-kubectl describe pod confidential-unencrypted
-```
-
-If there are no errors in the Events section, then the container has been successfully created with SEV memory encryption.
-
-### Validate SEV Memory Encryption
-
-The container dmesg log can be parsed to indicate that SEV memory encryption is enabled and active. The container image defined in the yaml sample above was built with a predefined key that is authorized for SSH access.
-
-Get the pod IP:
-
-```shell
-pod_ip=$(kubectl get pod -o wide | grep confidential-unencrypted | awk '{print $6;}')
-```
-
-Download and save the [SSH private key](https://github.com/kata-containers/kata-containers/raw/main/tests/integration/kubernetes/runtimeclass_workloads/confidential/unencrypted/ssh/unencrypted) and set the permissions.
-
-```shell
-wget https://github.com/kata-containers/kata-containers/raw/main/tests/integration/kubernetes/runtimeclass_workloads/confidential/unencrypted/ssh/unencrypted -O confidential-image-ssh-key
-
-chmod 600 confidential-image-ssh-key
-```
-
-The following command will run a remote SSH command on the container to check if SEV memory encryption is active:
-
-```shell
-ssh -i confidential-image-ssh-key \
-  -o "StrictHostKeyChecking no" \
-  -t root@${pod_ip} \
-  'dmesg | grep "Memory Encryption Features"'
-```
-
-If SEV is enabled and active, the output should return:
-
-```shell
-[    0.150045] Memory Encryption Features active: AMD SEV
-```
-
-## Create an Encrypted Image
-
-If SSH access to the container is desired, create a keypair:
-
-```shell
-ssh-keygen -t ed25519 -f encrypted-image-tests -P "" -C "" <<< y
-```
-
-The above command will save the keypair in a file named `encrypted-image-tests`.
-
-Here is a sample Dockerfile to create a docker image:
-
-```Dockerfile
-FROM alpine:3.16
-
-# Update and install openssh-server
-RUN apk update && apk upgrade && apk add openssh-server
-
-# Generate container ssh key
-RUN ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -P ""
-
-# A password needs to be set for login to work. An empty password is
-# unproblematic as password-based login to root is not allowed.
-RUN passwd -d root
-
-# Copy the remote generated public key to the container authorized_keys
-# Generate with 'ssh-keygen -t ed25519 -f encrypted-image-tests -P "" -C ""'
-COPY encrypted-image-tests.pub /root/.ssh/authorized_keys
-
-# Entry point - run sshd
-ENTRYPOINT /usr/sbin/sshd -D
-```
-
-Store this `Dockerfile` in the same directory as the `encrypted-image-tests` ssh keypair.
-
-Build image:
-
-```shell
-docker build -t encrypted-image-tests .
-```
-
-Tag and upload this unencrypted docker image to a registry:
-
-```shell
-docker tag encrypted-image-tests:latest [REGISTRY_URL]:unencrypted
-docker push [REGISTRY_URL]:unencrypted
-```
-
-Be sure to replace `[REGISTRY_URL]` with the desired registry URL.
-
-[skopeo](https://github.com/containers/skopeo) is required to encrypt the container image. Follow the instructions here to install `skopeo`:
-
-[skopeo Installation](https://github.com/containers/skopeo/blob/main/install.md)
-
-The Attestation Agent hosts a grpc service to support encrypting the image. Clone the repository:
-
-```shell
-attestation_agent_tag="v0.1.0"
-git clone https://github.com/confidential-containers/attestation-agent.git
-(cd attestation-agent && git checkout -b "branch_${attestation_agent_tag}" "${attestation_agent_tag}")
-```
-
-Run the offline_fs_kbs:
-
-```shell
-(cd attestation-agent/sample_keyprovider/src/enc_mods/offline_fs_kbs \
-&& cargo run --release --features offline_fs_kbs -- --keyprovider_sock 127.0.0.1:50001 &)
-```
-
-Create the Attestation Agent keyprovider:
-
-```shell
-cat > attestation-agent/sample_keyprovider/src/enc_mods/offline_fs_kbs/ocicrypt.conf <<EOF
-{
-  "key-providers": {
-    "attestation-agent": {
-      "grpc": "127.0.0.1:50001"
-}}}
-EOF
-```
-
-Set a desired value for the encryption key that should be a 32-bytes and base64 encoded value:
-
-```shell
-enc_key="RcHGava52DPvj1uoIk/NVDYlwxi0A6yyIZ8ilhEX3X4="
-```
-
-Create a Key file:
-
-```shell
-cat > keys.json <<EOF
-{
-    "key_id1":"${enc_key}"
-}
-EOF
-```
-
-Run skopeo to encrypt the image created in the previous section:
-
-```shell
-sudo OCICRYPT_KEYPROVIDER_CONFIG=$(pwd)/attestation-agent/sample_keyprovider/src/enc_mods/offline_fs_kbs/ocicrypt.conf \
-skopeo copy --insecure-policy \
-docker:[REGISTRY_URL]:unencrypted \
-docker:[REGISTRY_URL]:encrypted \
---encryption-key provider:attestation-agent:$(pwd)/keys.json:key_id1
-```
-
-Again, be sure to replace `[REGISTRY_URL]` with the desired registry URL.
-`--insecure-policy` flag is used to connect to the attestation agent and will not impact the security of the project.
-
-Make sure to use the `docker` prefix in the source and destination URL when running the `skopeo copy` command as demonstrated above.
-Utilizing images via the local `docker-daemon` is known to have issues, and the `skopeo copy` command does not return an adequate error
-response. A remote registry known to support encrypted images like GitHub Container Registry (GHCR) is required.
-
-At this point it is a good idea to inspect the image was really encrypted as skopeo can silently leave it unencrypted. Use
-`skopeo inspect` as shown below to check that the layers MIME types are **application/vnd.oci.image.layer.v1.tar+gzip+encrypted**:
-
-```shell
-skopeo inspect docker-daemon:[REGISTRY_URL]:encrypted
-```
-
-Push the encrypted image to the registry:
-
-```shell
-docker push [REGISTRY_URL]:encrypted
-```
-
-`mysql-client` is required to insert the key into the `simple-kbs` database. `jq` is required to json parse responses on the command line.
-
-* Debian / Ubuntu:
-  
-  ```shell
-  sudo apt install mysql-client jq
-  ```
-
-* CentOS / Fedora / RHEL:
-  
-  ```shell
-  sudo dnf install [ mysql | mariadb | community-mysql ] jq
-  ```
-
-The `mysql-client` package name may differ depending on OS flavor and version.
-
-The `simple-kbs` uses default settings and credentials for the MySQL database. These settings can be changed by the `simple-kbs` administrator and saved into a credential file. For the purposes of this quick start, set them in the environment for use with the MySQL client command line:
-
-```shell
-KBS_DB_USER="kbsuser"
-KBS_DB_PW="kbspassword"
-KBS_DB="simple_kbs"
-KBS_DB_TYPE="mysql"
-```
-
-Retrieve the host address of the MySQL database container:
-
-```shell
-KBS_DB_HOST=$(docker network inspect simple-kbs_default \
-  | jq -r '.[].Containers[] | select(.Name | test("simple-kbs[_-]db.*")).IPv4Address' \
-  | sed "s|/.*$||g")
-```
-
-Add the key to the `simple-kbs` database without any verification policy:
-
-```shell
-mysql -u${KBS_DB_USER} -p${KBS_DB_PW} -h ${KBS_DB_HOST} -D ${KBS_DB} <<EOF
-  REPLACE INTO secrets VALUES (10, 'key_id1', '${enc_key}', NULL);
-  REPLACE INTO keysets VALUES (10, 'KEYSET-1', '["key_id1"]', NULL);
-EOF
-```
-
-The second value in the keysets table (`KEYSET-1`) must match the `guest_pre_attestation_keyset` value specified in the SEV kata configuration file located here:
-
-`/opt/confidential-containers/share/defaults/kata-containers/configuration-qemu-sev.toml`
-
-Return to step [Launch the Pod and Verify SEV Encryption](#launch-the-pod-and-verify-sev-encryption) and finish the remaining process. Make sure to change the `encrypted-image-tests.yaml` to reflect the new `[REGISTRY_URL]`.
-
-To learn more about creating custom policies, see the section on [Creating a simple-kbs Policy to Verify the SEV Firmware Measurement](#creating-a-simple-kbs-policy-to-verify-the-sev-guest-firmware-measurement).
-

guides/snp.md

@@ -1,107 +0,0 @@
-# SNP Guide
-
-## Platform Setup
-
-> [!WARNING]
->
-> In order to launch SEV or SNP memory encrypted guests, the host must be prepared with a compatible kernel, `6.8.0-rc5-next-20240221-snp-host-cc2568386`. AMD custom changes and required components and repositories will eventually be upstreamed. 
-
-> [Sev-utils](https://github.com/amd/sev-utils/blob/coco-202402240000/docs/snp.md) is an easy way to install the required host kernel, but it will unnecessarily build AMD compatible guest kernel, OVMF, and QEMU components. The additional components can be used with the script utility to test launch and attest a base QEMU SNP guest. However, for the CoCo use case, they are already packaged and delivered with Kata.
-
-Alternatively, refer to the [AMDESE guide](https://github.com/confidential-containers/amdese-amdsev/tree/amd-snp-202402240000?tab=readme-ov-file#prepare-host) to manually build the host kernel and other components.
-
-## Getting Started
-
-This guide covers platform-specific setup for SNP and walks through the complete flows for the different CoCo use cases:
-
-- [Container Launch with Memory Encryption](#container-launch-with-memory-encryption)
-
-## Container Launch With Memory Encryption
-
-### Launch a Confidential Service 
-
-To launch a container with SNP memory encryption, the SNP runtime class (`kata-qemu-snp`) must be specified as an annotation in the yaml. A base alpine docker container ([Dockerfile](https://github.com/kata-containers/kata-containers/blob/main/tests/integration/kubernetes/runtimeclass_workloads/confidential/unencrypted/Dockerfile)) has been previously built for testing purposes. This image has also been prepared with SSH access and provisioned with a [SSH public key](https://github.com/kata-containers/kata-containers/blob/main/tests/integration/kubernetes/runtimeclass_workloads/confidential/unencrypted/ssh/unencrypted.pub) for validation purposes.
-
-Here is a sample service yaml specifying the SNP runtime class: 
-
-```yaml
-kind: Service
-apiVersion: v1
-metadata:
-  name: "confidential-unencrypted"
-spec:
-  selector:
-    app: "confidential-unencrypted"
-  ports:
-  - port: 22
----
-kind: Deployment
-apiVersion: apps/v1
-metadata:
-  name: "confidential-unencrypted"
-spec:
-  selector:
-    matchLabels:
-      app: "confidential-unencrypted"
-  template:
-    metadata:
-      labels:
-        app: "confidential-unencrypted"
-       annotations:
-        io.containerd.cri.runtime-handler: kata-qemu-snp
-    spec:
-      runtimeClassName: kata-qemu-snp
-      containers:
-      - name: "confidential-unencrypted"
-        image: ghcr.io/kata-containers/test-images:unencrypted-nightly
-        imagePullPolicy: Always
-```
-
-Save the contents of this yaml to a file called `confidential-unencrypted.yaml`.
-
-Start the service:
- 
-```shell
-kubectl apply -f confidential-unencrypted.yaml
-```
-
-Check for errors:
-
-```shell
-kubectl describe pod confidential-unencrypted
-```
-
-If there are no errors in the Events section, then the container has been successfully created with SNP memory encryption.
-
-### Validate SNP Memory Encryption
-
-The container dmesg log can be parsed to indicate that SNP memory encryption is enabled and active. The container image defined in the yaml sample above was built with a predefined key that is authorized for SSH access.
-
-Get the pod IP:
-
-```shell
-pod_ip=$(kubectl get pod -o wide | grep confidential-unencrypted | awk '{print $6;}')
-```
-
-Download and save the SSH private key and set the permissions.
-
-```shell
-wget https://github.com/kata-containers/kata-containers/raw/main/tests/integration/kubernetes/runtimeclass_workloads/confidential/unencrypted/ssh/unencrypted -O confidential-image-ssh-key
-
-chmod 600 confidential-image-ssh-key
-```
-
-The following command will run a remote SSH command on the container to check if SNP memory encryption is active:
-
-```shell
-ssh -i confidential-image-ssh-key \
-  -o "StrictHostKeyChecking no" \
-  -t root@${pod_ip} \
-  'dmesg | grep "Memory Encryption Features""'
-```
-
-If SNP is enabled and active, the output should return:
-
-```shell
-[    0.150045] Memory Encryption Features active: AMD SNP
-```

quickstart.md

@@ -22,7 +22,7 @@ hardware support and limitations.
 You can enable Confidential Containers in an existing Kubernetes cluster using the Confidential Containers Operator.
 When installation is finished, your cluster will have new runtime classes for different hardware platforms,
 including a generic runtime for testing CoCo without confidential hardware support, a runtime using a remote hypervisor
-that allows for cloud integration, a runtime for process-based isolation using SGX, as well as runtimes for TDX and SEV.
+that allows for cloud integration, a runtime for process-based isolation using SGX, as well as runtimes for TDX and SNP.
 
 ## Prerequisites
 
@@ -148,7 +148,6 @@ Details on each of the runtime classes:
 - *kata-clh* - standard kata runtime using the cloud hypervisor 
 - *kata-qemu* - same as kata
 - *kata-qemu-coco-dev* - standard kata runtime using the QEMU hypervisor including all CoCo building blocks for a non CC HW
-- *kata-qemu-sev* - using QEMU, and support for AMD SEV HW
 - *kata-qemu-snp* - using QEMU, and support for AMD SNP HW
 - *kata-qemu-tdx* -using QEMU, and support Intel TDX HW based on what's provided by [Ubuntu](https://github.com/canonical/tdx) and [CentOS 9 Stream](https://sigs.centos.org/virt/tdx/).
 
@@ -202,8 +201,7 @@ With some TEEs, the CoCo use cases and/or configurations are implemented differe
 [guide](./guides) section. To get started using CoCo without TEE hardware, follow the CoCo-dev guide below:
 
 - [CoCo-dev](./guides/coco-dev.md)
-- [SEV(-ES)](./guides/sev.md)
-- [SNP](./guides/snp.md)
+- [SNP](https://confidentialcontainers.org/docs/getting-started/prerequisites/hardware/snp/)
 - TDX: No additional steps required.
 - [SGX](./guides/enclave-cc.md)
 - [IBM Secure Execution](./guides/ibm-se.md)

releases/v0.14.0.md

@@ -0,0 +1,89 @@
+# Release Notes for v0.14.0
+
+Release Date: May 23rd, 2025
+
+This release is based on [3.17.0](https://github.com/kata-containers/kata-containers/releases/tag/3.17.0) of Kata Containers
+and [v0.11.0](https://github.com/confidential-containers/enclave-cc/releases/tag/v0.11.0) of enclave-cc.
+
+Kata and the CoCo components share an MSRV of 1.80.0.
+
+Please see the [quickstart guide](https://confidentialcontainers.org/docs/getting-started/) or [project documentation](https://confidentialcontainers.org/docs) for more information.
+
+## What's new
+
+* Init-data is supported on bare metal Confidential Containers (see limitations below)
+* [Peer Pods](https://github.com/confidential-containers/cloud-api-adaptor) is now supported by [Alibaba Cloud](https://github.com/confidential-containers/cloud-api-adaptor/tree/main/src/cloud-api-adaptor/alibabacloud).
+* Image-rs supports registry configuration file for fine-grained proxying and remapping of container registries.
+* KBS Client can be used to set reference values for Trustee.
+* KBS Client has a few simple resource policies built-in.
+* Trustee supports native verification of CCA guests in addition to verification via veraison.
+* Trustee artifacts are built and tested for ARM.
+* Trustee can extract reference values from TCG RIMs.
+* Trustee can be configured to support a larger payload size to accomodate guests with large evidence.
+* The confidential guest kernel configuration disables virtio MMIO transport and rng to reduce host attack surface.
+
+## Bug Fixes
+* CDH configuration file no longer requires `coco_as` and `kbs_token` fields to be set when not in use.
+* Trustee with docker compose can attest TDX evidence without any changes to QCNL configuration.
+* Trustee no longer errors when parsing the CCEl of a guest booted with grub. 
+* Trustee default policy matches parsed claims generated by SNP verifier.
+* Trustee k8s deployment and Kata tests updated for new AKS interfaces
+
+## Hardware Support
+
+Attestation is supported and tested on three platforms: Intel TDX, AMD SEV-SNP, and IBM SE.
+Not all features have been tested on every platform, but those based on attestation
+are expected to work on the platforms above.
+
+Make sure your host platform is compatible with the hypervisor and guest kernel
+provisioned by CoCo.
+
+This release has been tested on the following stacks:
+
+### AMD SEV-SNP
+
+* Processor: AMD EPYC 7413
+* Kernel: 6.12.0-snp-host-adc218676 (upstream 6.11+) 
+* OS: Ubuntu 22.04.4 LTS
+* k8s: v1.30.1 (Kubeadm)
+* Kustomize: v4.5.4
+
+### Intel TDX
+
+* Kernel: [6.8.0-1022-intel](https://git.launchpad.net/~kobuk-team/ubuntu/+source/linux-intel/tree/?h=noble-main-next)
+* OS: Ubuntu 24.04 LTS
+* k8s: v1.30.2 (Kubeadm)
+* Kustomize: v5.0.4-0.20230601165947-6ce0bf390ce3
+
+### Secure Execution on IBM zSystems (s390x) running LinuxONE
+
+* Hardware: IBM Z16 LPAR
+* Kernel: 5.15.0-113-generic
+* OS: Ubuntu 22.04.1 LTS
+* k8s: v1.28.4 (Kubeadm)
+* Kustomize: v5.3.0
+
+## Limitations
+
+The following are limitations and known issues with this release.
+
+* Bare metal initdata is only tested on TDX and non-tee.
+* Plaintext initdata is not forwarded to Trustee.
+* Credentials for authenticated registries are exposed to the host.
+* Not all features are tested on all platforms.
+* Nydus snapshotter support is not mature.
+  * Nydus snapshotter sometimes fails to pull an image.
+  * Host pulling with Nydus snapshotter is not yet enabled.
+  * Nydus snapshotter is not supported with enclave-cc.
+* Pulling container images inside guest may have negative performance implications including greater resource usage and slower startup.
+* `crio` support is still evolving.
+* Platform support is rapidly changing
+* SELinux is not supported on the host and must be set to permissive if in use.
+* Complete integration with Kubernetes is still in progress.
+  * Existing APIs do not fully support the CoCo security and threat model. [More info](https://github.com/confidential-containers/confidential-containers/issues/53)
+  * Some commands accessing confidential data, such as `kubectl exec`, may either fail to work, or incorrectly expose information to the host
+* The CoCo community aspires to adopting open source security best practices, but not all practices are adopted yet.
+
+## CVE Fixes
+
+None

releases/v0.15.0.md

@@ -0,0 +1,88 @@
+# Release Notes for v0.15.0
+
+Release Date: July 23rd, 2025
+
+This release is based on [3.19.1](https://github.com/kata-containers/kata-containers/releases/tag/3.19.1) of Kata Containers
+and [v0.11.0](https://github.com/confidential-containers/enclave-cc/releases/tag/v0.11.0) of enclave-cc.
+
+Trustee and the guest components use KBS protocol v0.4.0.
+
+Please see the [quickstart guide](https://confidentialcontainers.org/docs/getting-started/) or [project documentation](https://confidentialcontainers.org/docs) for more information.
+
+
+## What's New
+
+* Attestation can account for confidential devices attached to a guest in addition to the CPU.
+  So far only one confidential device is supported (the Deep Computing Unit from Hygon),
+  but more are coming soon.
+* The full (plaintext) Init-Data is transmitted to Trustee where it is provided as input
+  to the KBS policy. This allows KBS policies to check configuration fields in the Init-Data.
+* The image-rs registry config file can be specified directly in the CDH config, allowing it
+  to be provisioned via Init-Data.
+* Trustee has Prometheus support, allowing admins to track attestation metrics. 
+* Trustee can store resources with HashiCorp Vault.
+* Trustee can be configured to allow cross-origin requests, such as from browser-based tools.
+* Trustee supports reference values of any type that can be represented as JSON including
+  complex types like maps and lists.
+* The KBS-Client can be used to set reference values of multiple types using the KBS admin
+  interface.
+* Trustee has more sophisticated [CC eventlog](https://uefi.org/specs/UEFI/2.11/38_Confidential_Computing.html#virtual-platform-cc-event-log) parsing, allowing boot information to be parsed into TCB claims.
+* When using Trustee with Docker compose, the required admin keypair is automatically generated. 
+* Trustee can attest SNP guests on Milan, Genoa, and Turin hosts with report version 3 or 4.
+
+## Hardware Support
+
+Attestation is supported and tested on three platforms: Intel TDX, AMD SEV-SNP, and IBM SE.
+Not all features have been tested on every platform, but those based on attestation
+are expected to work on the platforms above.
+
+Make sure your host platform is compatible with the hypervisor and guest kernel
+provisioned by CoCo.
+
+This release has been tested on the following stacks:
+
+### AMD SEV-SNP
+
+* Processor: AMD EPYC 7413
+* Kernel: 6.12.0-snp-host-adc218676 (upstream 6.11+) 
+* OS: Ubuntu 22.04.4 LTS
+* k8s: v1.30.1 (Kubeadm)
+* Kustomize: v4.5.4
+
+### Intel TDX
+
+* Kernel: [6.8.0-1022-intel](https://git.launchpad.net/~kobuk-team/ubuntu/+source/linux-intel/tree/?h=noble-main-next)
+* OS: Ubuntu 24.04 LTS
+* k8s: v1.30.2 (Kubeadm)
+* Kustomize: v5.0.4-0.20230601165947-6ce0bf390ce3
+
+### Secure Execution on IBM zSystems (s390x) running LinuxONE
+
+* Hardware: IBM Z16 LPAR
+* Kernel: 5.15.0-113-generic
+* OS: Ubuntu 22.04.1 LTS
+* k8s: v1.28.4 (Kubeadm)
+* Kustomize: v5.3.0
+
+## Limitations
+
+The following are limitations and known issues with this release.
+
+* Credentials for authenticated registries are exposed to the host.
+* Not all features are tested on all platforms.
+* Nydus snapshotter support is not mature.
+  * Nydus snapshotter sometimes fails to pull an image.
+  * Host pulling with Nydus snapshotter is not yet enabled.
+  * Nydus snapshotter is not supported with enclave-cc.
+* Pulling container images inside guest may have negative performance implications including greater resource usage and slower startup.
+* `crio` support is still evolving.
+* Platform support is rapidly changing
+* SELinux is not supported on the host and must be set to permissive if in use.
+* Complete integration with Kubernetes is still in progress.
+  * Existing APIs do not fully support the CoCo security and threat model. [More info](https://github.com/confidential-containers/confidential-containers/issues/53)
+  * Some commands accessing confidential data, such as `kubectl exec`, may either fail to work, or incorrectly expose information to the host
+* The CoCo community aspires to adopting open source security best practices, but not all practices are adopted yet.
+
+## CVE Fixes
+
+None

releases/v0.16.0.md

@@ -0,0 +1,104 @@
+# Release Notes for v0.16.0
+
+Release Date: September 26th, 2025
+
+This release is based on [3.21.0](https://github.com/kata-containers/kata-containers/releases/tag/3.21.0) of Kata Containers
+and [v0.11.0](https://github.com/confidential-containers/enclave-cc/releases/tag/v0.11.0) of enclave-cc.
+
+Trustee and the guest components use KBS protocol v0.4.0.
+
+Please see the [quickstart guide](https://confidentialcontainers.org/docs/getting-started/) or [project documentation](https://confidentialcontainers.org/docs) for more information.
+
+## Deprecation Notices
+* Support for process-based confidential computing via enclave-cc will be removed in the v0.18.0 release.
+  The enclave-cc project will be archived.
+* This will be the last release of CoCo and Trustee that supports `simple` attestation tokens.
+  Today, EAR attestation tokens are the default, but Trustee can be configured to use `simple` tokens.
+  This option will be removed.
+
+## Breaking Changes
+
+* Previously the Init-Data was set for a pod via the `io.katacontainers.config.runtime.cc_init_data` annotation.
+  Now the `io.katacontainers.config.hypervisor.cc_init_data` annotation must be used.
+
+## What's New
+
+* The affirming resource policy now checks that every submod is affirming. With multi-device attestation,
+  KBS policies should be aware of all submods.
+* Experimental support for attesting some NVIDIA GPUs, such as the H100. Either ITA or the Trustee
+  Attestation Service can be used to verify the device evidence, with some limitations.
+* Experimental support for using pre-provisioned VMs in cloud-api-adaptor via "bring-your-own-machine (BYOM) provider
+* Runtime measurements can be extended from inside a workload container using a REST API.
+* Improved support for runtime measurements with AAEL on TDX
+* Trustee supports AMD-SNP guests with report version 5.
+* Additional/improved TCB claims generated by TDX verifier.
+* Extractor modules can now receive configuration if required.
+* SWID/RIM-IM extractor improved.
+* Confidential guest kernel updated to v6.16.7 with certain security-focused configs.
+* CSV verifier supports AAEL.
+* Eventlog parsing supports SM3 hashes.
+* A default GPU attestation policy is provided by Trustee, but it is very limited.
+* eHSM sealed secret backend no longer enabled by default.
+* evidence-getter tool now exposes primary and additional evidence.
+
+## Bug Fixes
+
+* Fixed issue where Init-Data checks could be maliciously bypassed when using dm-verity rootfs, such as with the base `kata-qemu-tdx` runtime. [GHSA-989w-4xr2-ww9m](https://github.com/kata-containers/kata-containers/security/advisories/GHSA-989w-4xr2-ww9m)
+* Fixed issue with attestation service policy endpoint permissions
+* Fixed issue with AR4SI vectors containing underscores
+* Various RUST advisories resolved by updating crates.
+
+## Hardware Support
+
+Attestation is supported and tested on three platforms: Intel TDX, AMD SEV-SNP, and IBM SE.
+Not all features have been tested on every platform, but those based on attestation
+are expected to work on the platforms above.
+
+Make sure your host platform is compatible with the hypervisor and guest kernel
+provisioned by CoCo.
+
+This release has been tested on the following stacks:
+
+### AMD SEV-SNP
+
+* Processor: AMD EPYC 7413
+* Kernel: 6.16.1
+* OS: Ubuntu 22.04.4 LTS
+* k8s: v1.33.0 (Kubeadm)
+* Kustomize: v5.6.0
+
+### Intel TDX
+
+* Kernel: [6.8.0-1028-intel](https://git.launchpad.net/~kobuk-team/ubuntu/+source/linux-intel/tree/?h=noble-main-next)
+* OS: Ubuntu 24.04 LTS
+* k8s: v1.30.2 (Kubeadm)
+* Kustomize: v5.0.4-0.20230601165947-6ce0bf390ce3
+
+### IBM Secure Execution for Linux
+
+* Hardware: IBM Z16 LPAR
+* Kernel: 6.8.0-60-generic
+* OS: Ubuntu Ubuntu 24.04.2 LTS
+* k8s: v1.31.1 (Kubeadm)
+* Kustomize: v5.7.1
+
+## Limitations
+
+The following are limitations and known issues with this release.
+
+* Credentials for authenticated registries are exposed to the host.
+* Not all features are tested on all platforms.
+* Nydus snapshotter support is not mature.
+  * Nydus snapshotter sometimes fails to pull an image.
+  * Host pulling with Nydus snapshotter is not yet enabled.
+  * Nydus snapshotter is not supported with enclave-cc.
+  * There is an experimental option to force guest image pull without a snapshotter.
+    This is also not mature.
+* Pulling container images inside guest may have negative performance implications including greater resource usage and slower startup.
+* `crio` support is still evolving.
+* Platform support is rapidly changing
+* SELinux is not supported on the host and must be set to permissive if in use.
+* Complete integration with Kubernetes is still in progress.
+  * Existing APIs do not fully support the CoCo security and threat model. [More info](https://github.com/confidential-containers/confidential-containers/issues/53)
+  * Some commands accessing confidential data, such as `kubectl exec`, may either fail to work, or incorrectly expose information to the host
+* The CoCo community aspires to adopting open source security best practices, but not all practices are adopted yet.