mirror of
https://github.com/kata-containers/kata-containers.git
synced 2025-04-28 03:42:09 +00:00
Merge pull request #4931 from jpecholt/snp-support
Added SNP-Support for Kata-Containers
This commit is contained in:
Peng Tao
GitHub
commit
8a2df6b31c
@ -33,51 +33,41 @@ You need to install the following to build Kata Containers components:
|
||||
- `make`.
|
||||
- `gcc` (required for building the shim and runtime).
|
||||
|
||||
# Build and install the Kata Containers runtime
|
||||
# Build and install Kata Containers
|
||||
## Build and install the Kata Containers runtime
|
||||
|
||||
```
|
||||
$ go get -d -u github.com/kata-containers/kata-containers
|
||||
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/src/runtime
|
||||
$ make && sudo -E PATH=$PATH make install
|
||||
```bash
|
||||
$ git clone https://github.com/kata-containers/kata-containers.git
|
||||
$ pushd kata-containers/src/runtime
|
||||
$ make && sudo -E "PATH=$PATH" make install
|
||||
$ sudo mkdir -p /etc/kata-containers/
|
||||
$ sudo install -o root -g root -m 0640 /usr/share/defaults/kata-containers/configuration.toml /etc/kata-containers
|
||||
$ popd
|
||||
```
|
||||
|
||||
The build will create the following:
|
||||
|
||||
- runtime binary: `/usr/local/bin/kata-runtime` and `/usr/local/bin/containerd-shim-kata-v2`
|
||||
- configuration file: `/usr/share/defaults/kata-containers/configuration.toml`
|
||||
|
||||
# Check hardware requirements
|
||||
|
||||
You can check if your system is capable of creating a Kata Container by running the following:
|
||||
|
||||
```
|
||||
$ sudo kata-runtime check
|
||||
```
|
||||
|
||||
If your system is *not* able to run Kata Containers, the previous command will error out and explain why.
|
||||
- configuration file: `/usr/share/defaults/kata-containers/configuration.toml` and `/etc/kata-containers/configuration.toml`
|
||||
|
||||
## Configure to use initrd or rootfs image
|
||||
|
||||
Kata containers can run with either an initrd image or a rootfs image.
|
||||
|
||||
If you want to test with `initrd`, make sure you have `initrd = /usr/share/kata-containers/kata-containers-initrd.img`
|
||||
in your configuration file, commenting out the `image` line:
|
||||
If you want to test with `initrd`, make sure you have uncommented `initrd = /usr/share/kata-containers/kata-containers-initrd.img`
|
||||
in your configuration file, commenting out the `image` line in
|
||||
`/etc/kata-containers/configuration.toml`. For example:
|
||||
|
||||
`/usr/share/defaults/kata-containers/configuration.toml` and comment out the `image` line with the following. For example:
|
||||
|
||||
```
|
||||
$ sudo mkdir -p /etc/kata-containers/
|
||||
$ sudo install -o root -g root -m 0640 /usr/share/defaults/kata-containers/configuration.toml /etc/kata-containers
|
||||
```bash
|
||||
$ sudo sed -i 's/^\(image =.*\)/# \1/g' /etc/kata-containers/configuration.toml
|
||||
$ sudo sed -i 's/^# \(initrd =.*\)/\1/g' /etc/kata-containers/configuration.toml
|
||||
```
|
||||
You can create the initrd image as shown in the [create an initrd image](#create-an-initrd-image---optional) section.
|
||||
|
||||
If you want to test with a rootfs `image`, make sure you have `image = /usr/share/kata-containers/kata-containers.img`
|
||||
If you want to test with a rootfs `image`, make sure you have uncommented `image = /usr/share/kata-containers/kata-containers.img`
|
||||
in your configuration file, commenting out the `initrd` line. For example:
|
||||
|
||||
```
|
||||
$ sudo mkdir -p /etc/kata-containers/
|
||||
$ sudo install -o root -g root -m 0640 /usr/share/defaults/kata-containers/configuration.toml /etc/kata-containers
|
||||
```bash
|
||||
$ sudo sed -i 's/^\(initrd =.*\)/# \1/g' /etc/kata-containers/configuration.toml
|
||||
```
|
||||
The rootfs image is created as shown in the [create a rootfs image](#create-a-rootfs-image) section.
|
||||
@ -90,7 +80,7 @@ rootfs `image`(100MB+).
|
||||
|
||||
Enable seccomp as follows:
|
||||
|
||||
```
|
||||
```bash
|
||||
$ sudo sed -i '/^disable_guest_seccomp/ s/true/false/' /etc/kata-containers/configuration.toml
|
||||
```
|
||||
|
||||
@ -100,9 +90,7 @@ This will pass container seccomp profiles to the kata agent.
|
||||
|
||||
Enable full debug as follows:
|
||||
|
||||
```
|
||||
$ sudo mkdir -p /etc/kata-containers/
|
||||
$ sudo install -o root -g root -m 0640 /usr/share/defaults/kata-containers/configuration.toml /etc/kata-containers
|
||||
```bash
|
||||
$ sudo sed -i -e 's/^# *\(enable_debug\).*=.*$/\1 = true/g' /etc/kata-containers/configuration.toml
|
||||
$ sudo sed -i -e 's/^kernel_params = "\(.*\)"/kernel_params = "\1 agent.log=debug initcall_debug"/g' /etc/kata-containers/configuration.toml
|
||||
```
|
||||
@ -175,7 +163,7 @@ and offers possible workarounds and fixes.
|
||||
it stores. When messages are suppressed, it is noted in the logs. This can be checked
|
||||
for by looking for those notifications, such as:
|
||||
|
||||
```sh
|
||||
```bash
|
||||
$ sudo journalctl --since today | fgrep Suppressed
|
||||
Jun 29 14:51:17 mymachine systemd-journald[346]: Suppressed 4150 messages from /system.slice/docker.service
|
||||
```
|
||||
@ -200,7 +188,7 @@ RateLimitBurst=0
|
||||
|
||||
Restart `systemd-journald` for the changes to take effect:
|
||||
|
||||
```sh
|
||||
```bash
|
||||
$ sudo systemctl restart systemd-journald
|
||||
```
|
||||
|
||||
@ -214,25 +202,24 @@ $ sudo systemctl restart systemd-journald
|
||||
|
||||
The agent is built with a statically linked `musl.` The default `libc` used is `musl`, but on `ppc64le` and `s390x`, `gnu` should be used. To configure this:
|
||||
|
||||
```
|
||||
$ export ARCH=$(uname -m)
|
||||
```bash
|
||||
$ export ARCH="$(uname -m)"
|
||||
$ if [ "$ARCH" = "ppc64le" -o "$ARCH" = "s390x" ]; then export LIBC=gnu; else export LIBC=musl; fi
|
||||
$ [ ${ARCH} == "ppc64le" ] && export ARCH=powerpc64le
|
||||
$ rustup target add ${ARCH}-unknown-linux-${LIBC}
|
||||
$ [ "${ARCH}" == "ppc64le" ] && export ARCH=powerpc64le
|
||||
$ rustup target add "${ARCH}-unknown-linux-${LIBC}"
|
||||
```
|
||||
|
||||
To build the agent:
|
||||
|
||||
```
|
||||
$ go get -d -u github.com/kata-containers/kata-containers
|
||||
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/src/agent && make
|
||||
```bash
|
||||
$ make -C kata-containers/src/agent
|
||||
```
|
||||
|
||||
The agent is built with seccomp capability by default.
|
||||
If you want to build the agent without the seccomp capability, you need to run `make` with `SECCOMP=no` as follows.
|
||||
|
||||
```
|
||||
$ make -C $GOPATH/src/github.com/kata-containers/kata-containers/src/agent SECCOMP=no
|
||||
```bash
|
||||
$ make -C kata-containers/src/agent SECCOMP=no
|
||||
```
|
||||
|
||||
> **Note:**
|
||||
@ -240,13 +227,6 @@ $ make -C $GOPATH/src/github.com/kata-containers/kata-containers/src/agent SECCO
|
||||
> - If you enable seccomp in the main configuration file but build the agent without seccomp capability,
|
||||
> the runtime exits conservatively with an error message.
|
||||
|
||||
## Get the osbuilder
|
||||
|
||||
```
|
||||
$ go get -d -u github.com/kata-containers/kata-containers
|
||||
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/osbuilder
|
||||
```
|
||||
|
||||
## Create a rootfs image
|
||||
### Create a local rootfs
|
||||
|
||||
@ -254,24 +234,26 @@ As a prerequisite, you need to install Docker. Otherwise, you will not be
|
||||
able to run the `rootfs.sh` script with `USE_DOCKER=true` as expected in
|
||||
the following example.
|
||||
|
||||
```
|
||||
$ export ROOTFS_DIR=${GOPATH}/src/github.com/kata-containers/kata-containers/tools/osbuilder/rootfs-builder/rootfs
|
||||
$ sudo rm -rf ${ROOTFS_DIR}
|
||||
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/osbuilder/rootfs-builder
|
||||
$ script -fec 'sudo -E GOPATH=$GOPATH USE_DOCKER=true ./rootfs.sh ${distro}'
|
||||
```bash
|
||||
$ export distro="ubuntu" # example
|
||||
$ export ROOTFS_DIR="$(realpath kata-containers/tools/osbuilder/rootfs-builder/rootfs)"
|
||||
$ sudo rm -rf "${ROOTFS_DIR}"
|
||||
$ pushd kata-containers/tools/osbuilder/rootfs-builder
|
||||
$ script -fec 'sudo -E USE_DOCKER=true ./rootfs.sh "${distro}"'
|
||||
$ popd
|
||||
```
|
||||
|
||||
You MUST choose a distribution (e.g., `ubuntu`) for `${distro}`.
|
||||
You can get a supported distributions list in the Kata Containers by running the following.
|
||||
|
||||
```
|
||||
$ ./rootfs.sh -l
|
||||
```bash
|
||||
$ ./kata-containers/tools/osbuilder/rootfs-builder/rootfs.sh -l
|
||||
```
|
||||
|
||||
If you want to build the agent without seccomp capability, you need to run the `rootfs.sh` script with `SECCOMP=no` as follows.
|
||||
|
||||
```
|
||||
$ script -fec 'sudo -E GOPATH=$GOPATH AGENT_INIT=yes USE_DOCKER=true SECCOMP=no ./rootfs.sh ${distro}'
|
||||
```bash
|
||||
$ script -fec 'sudo -E AGENT_INIT=yes USE_DOCKER=true SECCOMP=no ./rootfs.sh "${distro}"'
|
||||
```
|
||||
|
||||
> **Note:**
|
||||
@ -287,17 +269,18 @@ $ script -fec 'sudo -E GOPATH=$GOPATH AGENT_INIT=yes USE_DOCKER=true SECCOMP=no
|
||||
>
|
||||
> - You should only do this step if you are testing with the latest version of the agent.
|
||||
|
||||
```
|
||||
$ sudo install -o root -g root -m 0550 -t ${ROOTFS_DIR}/usr/bin ../../../src/agent/target/x86_64-unknown-linux-musl/release/kata-agent
|
||||
$ sudo install -o root -g root -m 0440 ../../../src/agent/kata-agent.service ${ROOTFS_DIR}/usr/lib/systemd/system/
|
||||
$ sudo install -o root -g root -m 0440 ../../../src/agent/kata-containers.target ${ROOTFS_DIR}/usr/lib/systemd/system/
|
||||
```bash
|
||||
$ sudo install -o root -g root -m 0550 -t "${ROOTFS_DIR}/usr/bin" "${ROOTFS_DIR}/../../../../src/agent/target/x86_64-unknown-linux-musl/release/kata-agent"
|
||||
$ sudo install -o root -g root -m 0440 "${ROOTFS_DIR}/../../../../src/agent/kata-agent.service" "${ROOTFS_DIR}/usr/lib/systemd/system/"
|
||||
$ sudo install -o root -g root -m 0440 "${ROOTFS_DIR}/../../../../src/agent/kata-containers.target" "${ROOTFS_DIR}/usr/lib/systemd/system/"
|
||||
```
|
||||
|
||||
### Build a rootfs image
|
||||
|
||||
```
|
||||
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/osbuilder/image-builder
|
||||
$ script -fec 'sudo -E USE_DOCKER=true ./image_builder.sh ${ROOTFS_DIR}'
|
||||
```bash
|
||||
$ pushd kata-containers/tools/osbuilder/image-builder
|
||||
$ script -fec 'sudo -E USE_DOCKER=true ./image_builder.sh "${ROOTFS_DIR}"'
|
||||
$ popd
|
||||
```
|
||||
|
||||
> **Notes:**
|
||||
@ -313,21 +296,26 @@ $ script -fec 'sudo -E USE_DOCKER=true ./image_builder.sh ${ROOTFS_DIR}'
|
||||
|
||||
### Install the rootfs image
|
||||
|
||||
```
|
||||
$ commit=$(git log --format=%h -1 HEAD)
|
||||
$ date=$(date +%Y-%m-%d-%T.%N%z)
|
||||
```bash
|
||||
$ pushd kata-containers/tools/osbuilder/image-builder
|
||||
$ commit="$(git log --format=%h -1 HEAD)"
|
||||
$ date="$(date +%Y-%m-%d-%T.%N%z)"
|
||||
$ image="kata-containers-${date}-${commit}"
|
||||
$ sudo install -o root -g root -m 0640 -D kata-containers.img "/usr/share/kata-containers/${image}"
|
||||
$ (cd /usr/share/kata-containers && sudo ln -sf "$image" kata-containers.img)
|
||||
$ popd
|
||||
```
|
||||
|
||||
## Create an initrd image - OPTIONAL
|
||||
### Create a local rootfs for initrd image
|
||||
```
|
||||
$ export ROOTFS_DIR="${GOPATH}/src/github.com/kata-containers/kata-containers/tools/osbuilder/rootfs-builder/rootfs"
|
||||
$ sudo rm -rf ${ROOTFS_DIR}
|
||||
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/osbuilder/rootfs-builder
|
||||
$ script -fec 'sudo -E GOPATH=$GOPATH AGENT_INIT=yes USE_DOCKER=true ./rootfs.sh ${distro}'
|
||||
|
||||
```bash
|
||||
$ export distro="ubuntu" # example
|
||||
$ export ROOTFS_DIR="$(realpath kata-containers/tools/osbuilder/rootfs-builder/rootfs)"
|
||||
$ sudo rm -rf "${ROOTFS_DIR}"
|
||||
$ pushd kata-containers/tools/osbuilder/rootfs-builder/
|
||||
$ script -fec 'sudo -E AGENT_INIT=yes USE_DOCKER=true ./rootfs.sh "${distro}"'
|
||||
$ popd
|
||||
```
|
||||
`AGENT_INIT` controls if the guest image uses the Kata agent as the guest `init` process. When you create an initrd image,
|
||||
always set `AGENT_INIT` to `yes`.
|
||||
@ -335,14 +323,14 @@ always set `AGENT_INIT` to `yes`.
|
||||
You MUST choose a distribution (e.g., `ubuntu`) for `${distro}`.
|
||||
You can get a supported distributions list in the Kata Containers by running the following.
|
||||
|
||||
```
|
||||
$ ./rootfs.sh -l
|
||||
```bash
|
||||
$ ./kata-containers/tools/osbuilder/rootfs-builder/rootfs.sh -l
|
||||
```
|
||||
|
||||
If you want to build the agent without seccomp capability, you need to run the `rootfs.sh` script with `SECCOMP=no` as follows.
|
||||
|
||||
```
|
||||
$ script -fec 'sudo -E GOPATH=$GOPATH AGENT_INIT=yes USE_DOCKER=true SECCOMP=no ./rootfs.sh ${distro}'
|
||||
```bash
|
||||
$ script -fec 'sudo -E AGENT_INIT=yes USE_DOCKER=true SECCOMP=no ./rootfs.sh "${distro}"'
|
||||
```
|
||||
|
||||
> **Note:**
|
||||
@ -351,28 +339,31 @@ $ script -fec 'sudo -E GOPATH=$GOPATH AGENT_INIT=yes USE_DOCKER=true SECCOMP=no
|
||||
|
||||
Optionally, add your custom agent binary to the rootfs with the following commands. The default `$LIBC` used
|
||||
is `musl`, but on ppc64le and s390x, `gnu` should be used. Also, Rust refers to ppc64le as `powerpc64le`:
|
||||
```
|
||||
$ export ARCH=$(uname -m)
|
||||
$ [ ${ARCH} == "ppc64le" ] || [ ${ARCH} == "s390x" ] && export LIBC=gnu || export LIBC=musl
|
||||
$ [ ${ARCH} == "ppc64le" ] && export ARCH=powerpc64le
|
||||
$ sudo install -o root -g root -m 0550 -T ../../../src/agent/target/${ARCH}-unknown-linux-${LIBC}/release/kata-agent ${ROOTFS_DIR}/sbin/init
|
||||
```bash
|
||||
$ export ARCH="$(uname -m)"
|
||||
$ [ "${ARCH}" == "ppc64le" ] || [ "${ARCH}" == "s390x" ] && export LIBC=gnu || export LIBC=musl
|
||||
$ [ "${ARCH}" == "ppc64le" ] && export ARCH=powerpc64le
|
||||
$ sudo install -o root -g root -m 0550 -T "${ROOTFS_DIR}/../../../../src/agent/target/${ARCH}-unknown-linux-${LIBC}/release/kata-agent" "${ROOTFS_DIR}/sbin/init"
|
||||
```
|
||||
|
||||
### Build an initrd image
|
||||
|
||||
```
|
||||
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/osbuilder/initrd-builder
|
||||
$ script -fec 'sudo -E AGENT_INIT=yes USE_DOCKER=true ./initrd_builder.sh ${ROOTFS_DIR}'
|
||||
```bash
|
||||
$ pushd kata-containers/tools/osbuilder/initrd-builder
|
||||
$ script -fec 'sudo -E AGENT_INIT=yes USE_DOCKER=true ./initrd_builder.sh "${ROOTFS_DIR}"'
|
||||
$ popd
|
||||
```
|
||||
|
||||
### Install the initrd image
|
||||
|
||||
```
|
||||
$ commit=$(git log --format=%h -1 HEAD)
|
||||
$ date=$(date +%Y-%m-%d-%T.%N%z)
|
||||
```bash
|
||||
$ pushd kata-containers/tools/osbuilder/initrd-builder
|
||||
$ commit="$(git log --format=%h -1 HEAD)"
|
||||
$ date="$(date +%Y-%m-%d-%T.%N%z)"
|
||||
$ image="kata-containers-initrd-${date}-${commit}"
|
||||
$ sudo install -o root -g root -m 0640 -D kata-containers-initrd.img "/usr/share/kata-containers/${image}"
|
||||
$ (cd /usr/share/kata-containers && sudo ln -sf "$image" kata-containers-initrd.img)
|
||||
$ popd
|
||||
```
|
||||
|
||||
# Install guest kernel images
|
||||
@ -391,44 +382,44 @@ Kata Containers makes use of upstream QEMU branch. The exact version
|
||||
and repository utilized can be found by looking at the [versions file](../versions.yaml).
|
||||
|
||||
Find the correct version of QEMU from the versions file:
|
||||
```
|
||||
$ source ${GOPATH}/src/github.com/kata-containers/kata-containers/tools/packaging/scripts/lib.sh
|
||||
$ qemu_version=$(get_from_kata_deps "assets.hypervisor.qemu.version")
|
||||
$ echo ${qemu_version}
|
||||
```bash
|
||||
$ source kata-containers/tools/packaging/scripts/lib.sh
|
||||
$ qemu_version="$(get_from_kata_deps "assets.hypervisor.qemu.version")"
|
||||
$ echo "${qemu_version}"
|
||||
```
|
||||
Get source from the matching branch of QEMU:
|
||||
```
|
||||
$ go get -d github.com/qemu/qemu
|
||||
$ cd ${GOPATH}/src/github.com/qemu/qemu
|
||||
$ git checkout ${qemu_version}
|
||||
$ your_qemu_directory=${GOPATH}/src/github.com/qemu/qemu
|
||||
```bash
|
||||
$ git clone -b "${qemu_version}" https://github.com/qemu/qemu.git
|
||||
$ your_qemu_directory="$(realpath qemu)"
|
||||
```
|
||||
|
||||
There are scripts to manage the build and packaging of QEMU. For the examples below, set your
|
||||
environment as:
|
||||
```
|
||||
$ go get -d github.com/kata-containers/kata-containers
|
||||
$ packaging_dir="${GOPATH}/src/github.com/kata-containers/kata-containers/tools/packaging"
|
||||
```bash
|
||||
$ packaging_dir="$(realpath kata-containers/tools/packaging)"
|
||||
```
|
||||
|
||||
Kata often utilizes patches for not-yet-upstream and/or backported fixes for components,
|
||||
including QEMU. These can be found in the [packaging/QEMU directory](../tools/packaging/qemu/patches),
|
||||
and it's *recommended* that you apply them. For example, suppose that you are going to build QEMU
|
||||
version 5.2.0, do:
|
||||
```
|
||||
$ cd $your_qemu_directory
|
||||
$ $packaging_dir/scripts/apply_patches.sh $packaging_dir/qemu/patches/5.2.x/
|
||||
```bash
|
||||
$ "$packaging_dir/scripts/apply_patches.sh" "$packaging_dir/qemu/patches/5.2.x/"
|
||||
```
|
||||
|
||||
To build utilizing the same options as Kata, you should make use of the `configure-hypervisor.sh` script. For example:
|
||||
```
|
||||
$ cd $your_qemu_directory
|
||||
$ $packaging_dir/scripts/configure-hypervisor.sh kata-qemu > kata.cfg
|
||||
```bash
|
||||
$ pushd "$your_qemu_directory"
|
||||
$ "$packaging_dir/scripts/configure-hypervisor.sh" kata-qemu > kata.cfg
|
||||
$ eval ./configure "$(cat kata.cfg)"
|
||||
$ make -j $(nproc --ignore=1)
|
||||
# Optional
|
||||
$ sudo -E make install
|
||||
$ popd
|
||||
```
|
||||
|
||||
If you do not want to install the respective QEMU version, the configuration file can be modified to point to the correct binary. In `/etc/kata-containers/configuration.toml`, change `path = "/path/to/qemu/build/qemu-system-x86_64"` to point to the correct QEMU binary.
|
||||
|
||||
See the [static-build script for QEMU](../tools/packaging/static-build/qemu/build-static-qemu.sh) for a reference on how to get, setup, configure and build QEMU for Kata.
|
||||
|
||||
### Build a custom QEMU for aarch64/arm64 - REQUIRED
|
||||
@ -439,11 +430,33 @@ See the [static-build script for QEMU](../tools/packaging/static-build/qemu/buil
|
||||
> under upstream review for supporting NVDIMM on aarch64.
|
||||
>
|
||||
You could build the custom `qemu-system-aarch64` as required with the following command:
|
||||
```bash
|
||||
$ git clone https://github.com/kata-containers/tests.git
|
||||
$ script -fec 'sudo -E tests/.ci/install_qemu.sh'
|
||||
```
|
||||
$ go get -d github.com/kata-containers/tests
|
||||
$ script -fec 'sudo -E ${GOPATH}/src/github.com/kata-containers/tests/.ci/install_qemu.sh'
|
||||
|
||||
## Build `virtiofsd`
|
||||
|
||||
When using the file system type virtio-fs (default), `virtiofsd` is required
|
||||
|
||||
```bash
|
||||
$ pushd kata-containers/tools/packaging/static-build/virtiofsd
|
||||
$ ./build-static-virtiofsd.sh
|
||||
$ popd
|
||||
```
|
||||
|
||||
Modify `/etc/kata-containers/configuration.toml` and update value `virtio_fs_daemon = "/path/to/kata-containers/tools/packaging/static-build/virtiofsd/virtiofsd/virtiofsd"` to point to the binary.
|
||||
|
||||
# Check hardware requirements
|
||||
|
||||
You can check if your system is capable of creating a Kata Container by running the following:
|
||||
|
||||
```bash
|
||||
$ sudo kata-runtime check
|
||||
```
|
||||
|
||||
If your system is *not* able to run Kata Containers, the previous command will error out and explain why.
|
||||
|
||||
# Run Kata Containers with Containerd
|
||||
Refer to the [How to use Kata Containers and Containerd](how-to/containerd-kata.md) how-to guide.
|
||||
|
||||
@ -474,7 +487,7 @@ See [Set up a debug console](#set-up-a-debug-console).
|
||||
|
||||
## Checking Docker default runtime
|
||||
|
||||
```
|
||||
```bash
|
||||
$ sudo docker info 2>/dev/null | grep -i "default runtime" | cut -d: -f2- | grep -q runc && echo "SUCCESS" || echo "ERROR: Incorrect default Docker runtime"
|
||||
```
|
||||
## Set up a debug console
|
||||
@ -491,7 +504,7 @@ contain either `/bin/sh` or `/bin/bash`.
|
||||
|
||||
Enable debug_console_enabled in the `configuration.toml` configuration file:
|
||||
|
||||
```
|
||||
```toml
|
||||
[agent.kata]
|
||||
debug_console_enabled = true
|
||||
```
|
||||
@ -502,7 +515,7 @@ This will pass `agent.debug_console agent.debug_console_vport=1026` to agent as
|
||||
|
||||
For Kata Containers `2.0.x` releases, the `kata-runtime exec` command depends on the`kata-monitor` running, in order to get the sandbox's `vsock` address to connect to. Thus, first start the `kata-monitor` process.
|
||||
|
||||
```
|
||||
```bash
|
||||
$ sudo kata-monitor
|
||||
```
|
||||
|
||||
@ -564,10 +577,10 @@ an additional `coreutils` package.
|
||||
|
||||
For example using CentOS:
|
||||
|
||||
```
|
||||
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/osbuilder/rootfs-builder
|
||||
$ export ROOTFS_DIR=${GOPATH}/src/github.com/kata-containers/kata-containers/tools/osbuilder/rootfs-builder/rootfs
|
||||
$ script -fec 'sudo -E GOPATH=$GOPATH USE_DOCKER=true EXTRA_PKGS="bash coreutils" ./rootfs.sh centos'
|
||||
```bash
|
||||
$ pushd kata-containers/tools/osbuilder/rootfs-builder
|
||||
$ export ROOTFS_DIR="$(realpath ./rootfs)"
|
||||
$ script -fec 'sudo -E USE_DOCKER=true EXTRA_PKGS="bash coreutils" ./rootfs.sh centos'
|
||||
```
|
||||
|
||||
#### Build the debug image
|
||||
@ -582,9 +595,10 @@ Install the image:
|
||||
>**Note**: When using an initrd image, replace the below rootfs image name `kata-containers.img`
|
||||
>with the initrd image name `kata-containers-initrd.img`.
|
||||
|
||||
```
|
||||
```bash
|
||||
$ name="kata-containers-centos-with-debug-console.img"
|
||||
$ sudo install -o root -g root -m 0640 kata-containers.img "/usr/share/kata-containers/${name}"
|
||||
$ popd
|
||||
```
|
||||
|
||||
Next, modify the `image=` values in the `[hypervisor.qemu]` section of the
|
||||
@ -593,7 +607,7 @@ to specify the full path to the image name specified in the previous code
|
||||
section. Alternatively, recreate the symbolic link so it points to
|
||||
the new debug image:
|
||||
|
||||
```
|
||||
```bash
|
||||
$ (cd /usr/share/kata-containers && sudo ln -sf "$name" kata-containers.img)
|
||||
```
|
||||
|
||||
@ -604,7 +618,7 @@ to avoid all subsequently created containers from using the debug image.
|
||||
|
||||
Create a container as normal. For example using `crictl`:
|
||||
|
||||
```
|
||||
```bash
|
||||
$ sudo crictl run -r kata container.yaml pod.yaml
|
||||
```
|
||||
|
||||
@ -617,7 +631,7 @@ those for firecracker / cloud-hypervisor.
|
||||
|
||||
Add `agent.debug_console` to the guest kernel command line to allow the agent process to start a debug console.
|
||||
|
||||
```
|
||||
```bash
|
||||
$ sudo sed -i -e 's/^kernel_params = "\(.*\)"/kernel_params = "\1 agent.debug_console"/g' "${kata_configuration_file}"
|
||||
```
|
||||
|
||||
@ -638,7 +652,7 @@ between the host and the guest. The kernel command line option `agent.debug_cons
|
||||
|
||||
Add the parameter `agent.debug_console_vport=1026` to the kernel command line
|
||||
as shown below:
|
||||
```
|
||||
```bash
|
||||
sudo sed -i -e 's/^kernel_params = "\(.*\)"/kernel_params = "\1 agent.debug_console_vport=1026"/g' "${kata_configuration_file}"
|
||||
```
|
||||
|
||||
@ -651,7 +665,7 @@ Next, connect to the debug console. The VSOCKS paths vary slightly between each
|
||||
VMM solution.
|
||||
|
||||
In case of cloud-hypervisor, connect to the `vsock` as shown:
|
||||
```
|
||||
```bash
|
||||
$ sudo su -c 'cd /var/run/vc/vm/${sandbox_id}/root/ && socat stdin unix-connect:clh.sock'
|
||||
CONNECT 1026
|
||||
```
|
||||
@ -659,7 +673,7 @@ CONNECT 1026
|
||||
**Note**: You need to type `CONNECT 1026` and press `RETURN` key after entering the `socat` command.
|
||||
|
||||
For firecracker, connect to the `hvsock` as shown:
|
||||
```
|
||||
```bash
|
||||
$ sudo su -c 'cd /var/run/vc/firecracker/${sandbox_id}/root/ && socat stdin unix-connect:kata.hvsock'
|
||||
CONNECT 1026
|
||||
```
|
||||
@ -668,7 +682,7 @@ CONNECT 1026
|
||||
|
||||
|
||||
For QEMU, connect to the `vsock` as shown:
|
||||
```
|
||||
```bash
|
||||
$ sudo su -c 'cd /var/run/vc/vm/${sandbox_id} && socat "stdin,raw,echo=0,escape=0x11" "unix-connect:console.sock"'
|
||||
```
|
||||
|
||||
@ -681,7 +695,7 @@ If the image is created using
|
||||
[osbuilder](../tools/osbuilder), the following YAML
|
||||
file exists and contains details of the image and how it was created:
|
||||
|
||||
```
|
||||
```bash
|
||||
$ cat /var/lib/osbuilder/osbuilder.yaml
|
||||
```
|
||||
|
||||
|
||||
@ -42,4 +42,5 @@
|
||||
- [How to setup swap devices in guest kernel](how-to-setup-swap-devices-in-guest-kernel.md)
|
||||
- [How to run rootless vmm](how-to-run-rootless-vmm.md)
|
||||
- [How to run Docker with Kata Containers](how-to-run-docker-with-kata.md)
|
||||
- [How to run Kata Containers with `nydus`](how-to-use-virtio-fs-nydus-with-kata.md)
|
||||
- [How to run Kata Containers with `nydus`](how-to-use-virtio-fs-nydus-with-kata.md)
|
||||
- [How to run Kata Containers with AMD SEV-SNP](how-to-run-kata-containers-with-SNP-VMs.md)
|
||||
|
||||
@ -77,8 +77,8 @@ $ command -v containerd
|
||||
You can manually install CNI plugins as follows:
|
||||
|
||||
```bash
|
||||
$ go get github.com/containernetworking/plugins
|
||||
$ pushd $GOPATH/src/github.com/containernetworking/plugins
|
||||
$ git clone https://github.com/containernetworking/plugins.git
|
||||
$ pushd plugins
|
||||
$ ./build_linux.sh
|
||||
$ sudo mkdir /opt/cni
|
||||
$ sudo cp -r bin /opt/cni/
|
||||
@ -93,8 +93,8 @@ $ popd
|
||||
You can install the `cri-tools` from source code:
|
||||
|
||||
```bash
|
||||
$ go get github.com/kubernetes-sigs/cri-tools
|
||||
$ pushd $GOPATH/src/github.com/kubernetes-sigs/cri-tools
|
||||
$ git clone https://github.com/kubernetes-sigs/cri-tools.git
|
||||
$ pushd cri-tools
|
||||
$ make
|
||||
$ sudo -E make install
|
||||
$ popd
|
||||
|
||||
159
docs/how-to/how-to-run-kata-containers-with-SNP-VMs.md
Normal file
159
docs/how-to/how-to-run-kata-containers-with-SNP-VMs.md
Normal file
@ -0,0 +1,159 @@
|
||||
# Kata Containers with AMD SEV-SNP VMs
|
||||
|
||||
## Disclaimer
|
||||
|
||||
This guide is designed for developers and is - same as the Developer Guide - not intended for production systems or end users. It is advisable to only follow this guide on non-critical development systems.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
To run Kata Containers in SNP-VMs, the following software stack is used.
|
||||
|
||||
|
||||
|
||||
The host BIOS and kernel must be capable of supporting AMD SEV-SNP and configured accordingly. For Kata Containers, the host kernel with branch [`sev-snp-iommu-avic_5.19-rc6_v3`](https://github.com/AMDESE/linux/tree/sev-snp-iommu-avic_5.19-rc6_v3) and commit [`3a88547`](https://github.com/AMDESE/linux/commit/3a885471cf89156ea555341f3b737ad2a8d9d3d0) is known to work in conjunction with SEV Firmware version 1.51.3 (0xh\_1.33.03) available on AMD's [SEV developer website](https://developer.amd.com/sev/). See [AMD's guide](https://github.com/AMDESE/AMDSEV/tree/sev-snp-devel) to configure the host accordingly. Verify that you are able to run SEV-SNP encrypted VMs first. The guest components required for Kata Containers are built as described below.
|
||||
|
||||
**Tip**: It is easiest to first have Kata Containers running on your system and then modify it to run containers in SNP-VMs. Follow the [Developer guide](../Developer-Guide.md#warning) and then follow the below steps. Nonetheless, you can just follow this guide from the start.
|
||||
|
||||
## How to build
|
||||
|
||||
Follow all of the below steps to install Kata Containers with SNP-support from scratch. These steps mostly follow the developer guide with modifications to support SNP
|
||||
|
||||
__Steps from the Developer Guide:__
|
||||
- Get all the [required components](../Developer-Guide.md#requirements-to-build-individual-components) for building the kata-runtime
|
||||
- [Build the and install kata-runtime](../Developer-Guide.md#build-and-install-the-kata-containers-runtime)
|
||||
- [Build a custom agent](../Developer-Guide.md#build-a-custom-kata-agent---optional)
|
||||
- [Create an initrd image](../Developer-Guide.md#create-an-initrd-image---optional) by first building a rootfs, then building the initrd based on the rootfs, use a custom agent and install. `ubuntu` works as the distribution of choice.
|
||||
- Get the [required components](../../tools/packaging/kernel/README.md#requirements) to build a custom kernel
|
||||
|
||||
__SNP-specific steps:__
|
||||
- Build the SNP-specific kernel as shown below (see this [guide](../../tools/packaging/kernel/README.md#build-kata-containers-kernel) for more information)
|
||||
```bash
|
||||
$ pushd kata-containers/tools/packaging/kernel/
|
||||
$ ./build-kernel.sh -a x86_64 -x snp setup
|
||||
$ ./build-kernel.sh -a x86_64 -x snp build
|
||||
$ sudo -E PATH="${PATH}" ./build-kernel.sh -x snp install
|
||||
$ popd
|
||||
```
|
||||
- Build a current OVMF capable of SEV-SNP:
|
||||
```bash
|
||||
$ pushd kata-containers/tools/packaging/static-build/ovmf
|
||||
$ ./build.sh
|
||||
$ tar -xvf edk2-x86_64.tar.gz
|
||||
$ popd
|
||||
```
|
||||
- Build a custom QEMU
|
||||
```bash
|
||||
$ source kata-containers/tools/packaging/scripts/lib.sh
|
||||
$ qemu_url="$(get_from_kata_deps "assets.hypervisor.qemu.snp.url")"
|
||||
$ qemu_branch="$(get_from_kata_deps "assets.hypervisor.qemu.snp.branch")"
|
||||
$ qemu_commit="$(get_from_kata_deps "assets.hypervisor.qemu.snp.commit")"
|
||||
$ git clone -b "${qemu_branch}" "${qemu_url}"
|
||||
$ pushd qemu
|
||||
$ git checkout "${qemu_commit}"
|
||||
$ ./configure --target-list=x86_64-softmmu --enable-debug
|
||||
$ make -j "$(nproc)"
|
||||
$ popd
|
||||
```
|
||||
|
||||
### Kata Containers Configuration for SNP
|
||||
|
||||
The configuration file located at `/etc/kata-containers/configuration.toml` must be adapted as follows to support SNP-VMs:
|
||||
- Use the SNP-specific kernel for the guest VM (change path)
|
||||
```toml
|
||||
kernel = "/usr/share/kata-containers/vmlinuz-snp.container"
|
||||
```
|
||||
- Enable the use of an initrd (uncomment)
|
||||
```toml
|
||||
initrd = "/usr/share/kata-containers/kata-containers-initrd.img"
|
||||
```
|
||||
- Disable the use of a rootfs (comment out)
|
||||
```toml
|
||||
# image = "/usr/share/kata-containers/kata-containers.img"
|
||||
```
|
||||
- Use the custom QEMU capable of SNP (change path)
|
||||
```toml
|
||||
path = "/path/to/qemu/build/qemu-system-x86_64"
|
||||
```
|
||||
- Use `virtio-9p` device since `virtio-fs` is unsupported due to bugs / shortcomings in QEMU version [`snp-v3`](https://github.com/AMDESE/qemu/tree/snp-v3) for SEV and SEV-SNP (change value)
|
||||
```toml
|
||||
shared_fs = "virtio-9p"
|
||||
```
|
||||
- Disable `virtiofsd` since it is no longer required (comment out)
|
||||
```toml
|
||||
# virtio_fs_daemon = "/usr/libexec/virtiofsd"
|
||||
```
|
||||
- Disable NVDIMM (uncomment)
|
||||
```toml
|
||||
disable_image_nvdimm = true
|
||||
```
|
||||
- Disable shared memory (uncomment)
|
||||
```toml
|
||||
file_mem_backend = ""
|
||||
```
|
||||
- Enable confidential guests (uncomment)
|
||||
```toml
|
||||
confidential_guest = true
|
||||
```
|
||||
- Enable SNP-VMs (uncomment)
|
||||
```toml
|
||||
sev_snp_guest = true
|
||||
```
|
||||
- Configure an OVMF (add path)
|
||||
```toml
|
||||
firmware = "/path/to/kata-containers/tools/packaging/static-build/ovmf/opt/kata/share/ovmf/OVMF.fd"
|
||||
```
|
||||
|
||||
## Test Kata Containers with Containerd
|
||||
|
||||
With Kata Containers configured to support SNP-VMs, we use containerd to test and deploy containers in these VMs.
|
||||
|
||||
### Install Containerd
|
||||
If not already present, follow [this guide](./containerd-kata.md#install) to install containerd and its related components including `CNI` and the `cri-tools` (skip Kata Containers since we already installed it)
|
||||
|
||||
### Containerd Configuration
|
||||
|
||||
Follow [this guide](./containerd-kata.md#configuration) to configure containerd to use Kata Containers
|
||||
|
||||
## Run Kata Containers in SNP-VMs
|
||||
|
||||
Run the below commands to start a container. See [this guide](./containerd-kata.md#run) for more information
|
||||
```bash
|
||||
$ sudo ctr image pull docker.io/library/busybox:latest
|
||||
$ sudo ctr run --cni --runtime io.containerd.run.kata.v2 -t --rm docker.io/library/busybox:latest hello sh
|
||||
```
|
||||
|
||||
### Check for active SNP:
|
||||
|
||||
Inside the running container, run the following commands to check if SNP is active. It should look something like this:
|
||||
```
|
||||
/ # dmesg | grep -i sev
|
||||
[ 0.299242] Memory Encryption Features active: AMD SEV SEV-ES SEV-SNP
|
||||
[ 0.472286] SEV: Using SNP CPUID table, 31 entries present.
|
||||
[ 0.514574] SEV: SNP guest platform device initialized.
|
||||
[ 0.885425] sev-guest sev-guest: Initialized SEV guest driver (using vmpck_id 0)
|
||||
```
|
||||
|
||||
### Obtain an SNP Attestation Report
|
||||
|
||||
To obtain an attestation report inside the container, the `/dev/sev-guest` must first be configured. As of now, the VM does not perform this step, however it can be performed inside the container, either in the terminal or in code.
|
||||
|
||||
Example for shell:
|
||||
```
|
||||
/ # SNP_MAJOR=$(cat /sys/devices/virtual/misc/sev-guest/dev | awk -F: '{print $1}')
|
||||
/ # SNP_MINOR=$(cat /sys/devices/virtual/misc/sev-guest/dev | awk -F: '{print $2}')
|
||||
/ # mknod -m 600 /dev/sev-guest c "${SNP_MAJOR}" "${SNP_MINOR}"
|
||||
```
|
||||
|
||||
## Known Issues
|
||||
|
||||
- Support for cgroups v2 is still [work in progress](https://github.com/kata-containers/kata-containers/issues/927). If issues occur due to cgroups v2 becoming the default in newer systems, one possible solution is to downgrade cgroups to v1:
|
||||
```bash
|
||||
sudo sed -i 's/^\(GRUB_CMDLINE_LINUX=".*\)"/\1 systemd.unified_cgroup_hierarchy=0"/' /etc/default/grub
|
||||
sudo update-grub
|
||||
sudo reboot
|
||||
```
|
||||
- If both SEV and SEV-SNP are supported by the host, Kata Containers uses SEV-SNP by default. You can verify what features are enabled by checking `/sys/module/kvm_amd/parameters/sev` and `sev_snp`. This means that Kata Containers can not run both SEV-SNP-VMs and SEV-VMs at the same time. If SEV is to be used by Kata Containers instead, reload the `kvm_amd` kernel module without SNP-support, this will disable SNP-support for the entire platform.
|
||||
```bash
|
||||
sudo rmmod kvm_amd && sudo modprobe kvm_amd sev_snp=0
|
||||
```
|
||||
|
||||
4
docs/how-to/images/SNP-stack.svg
Normal file
4
docs/how-to/images/SNP-stack.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 9.0 KiB |
@ -56,6 +56,7 @@ BINLIBEXECLIST :=
|
||||
BIN_PREFIX = $(PROJECT_TYPE)
|
||||
PROJECT_DIR = $(PROJECT_TAG)
|
||||
IMAGENAME = $(PROJECT_TAG).img
|
||||
INITRDNAME = $(PROJECT_TAG)-initrd.img
|
||||
|
||||
TARGET = $(BIN_PREFIX)-runtime
|
||||
RUNTIME_OUTPUT = $(CURDIR)/$(TARGET)
|
||||
@ -110,6 +111,7 @@ PKGLIBEXECDIR := $(LIBEXECDIR)/$(PROJECT_DIR)
|
||||
KERNELDIR := $(PKGDATADIR)
|
||||
|
||||
IMAGEPATH := $(PKGDATADIR)/$(IMAGENAME)
|
||||
INITRDPATH := $(PKGDATADIR)/$(INITRDNAME)
|
||||
FIRMWAREPATH :=
|
||||
FIRMWAREVOLUMEPATH :=
|
||||
|
||||
@ -401,6 +403,8 @@ USER_VARS += FCVALIDJAILERPATHS
|
||||
USER_VARS += SYSCONFIG
|
||||
USER_VARS += IMAGENAME
|
||||
USER_VARS += IMAGEPATH
|
||||
USER_VARS += INITRDNAME
|
||||
USER_VARS += INITRDPATH
|
||||
USER_VARS += MACHINETYPE
|
||||
USER_VARS += KERNELDIR
|
||||
USER_VARS += KERNELTYPE
|
||||
|
||||
@ -15,6 +15,7 @@
|
||||
path = "@QEMUPATH@"
|
||||
kernel = "@KERNELPATH@"
|
||||
image = "@IMAGEPATH@"
|
||||
# initrd = "@INITRDPATH@"
|
||||
machine_type = "@MACHINETYPE@"
|
||||
|
||||
# Enable confidential guest support.
|
||||
@ -33,6 +34,12 @@ machine_type = "@MACHINETYPE@"
|
||||
# Default false
|
||||
# confidential_guest = true
|
||||
|
||||
# Choose AMD SEV-SNP confidential guests
|
||||
# In case of using confidential guests on AMD hardware that supports both SEV
|
||||
# and SEV-SNP, the following enables SEV-SNP guests. SEV guests are default.
|
||||
# Default false
|
||||
# sev_snp_guest = true
|
||||
|
||||
# Enable running QEMU VMM as a non-root user.
|
||||
# By default QEMU VMM run as root. When this is set to true, QEMU VMM process runs as
|
||||
# a non-root random user. See documentation for the limitations of this mode.
|
||||
|
||||
@ -231,6 +231,9 @@ const (
|
||||
// SEVGuest represents an SEV guest object
|
||||
SEVGuest ObjectType = "sev-guest"
|
||||
|
||||
// SNPGuest represents an SNP guest object
|
||||
SNPGuest ObjectType = "sev-snp-guest"
|
||||
|
||||
// SecExecGuest represents an s390x Secure Execution (Protected Virtualization in QEMU) object
|
||||
SecExecGuest ObjectType = "s390-pv-guest"
|
||||
// PEFGuest represent ppc64le PEF(Protected Execution Facility) object.
|
||||
@ -295,6 +298,8 @@ func (object Object) Valid() bool {
|
||||
case TDXGuest:
|
||||
return object.ID != "" && object.File != "" && object.DeviceID != ""
|
||||
case SEVGuest:
|
||||
fallthrough
|
||||
case SNPGuest:
|
||||
return object.ID != "" && object.File != "" && object.CBitPos != 0 && object.ReducedPhysBits != 0
|
||||
case SecExecGuest:
|
||||
return object.ID != ""
|
||||
@ -349,6 +354,8 @@ func (object Object) QemuParams(config *Config) []string {
|
||||
deviceParams = append(deviceParams, fmt.Sprintf("config-firmware-volume=%s", object.FirmwareVolume))
|
||||
}
|
||||
case SEVGuest:
|
||||
fallthrough
|
||||
case SNPGuest:
|
||||
objectParams = append(objectParams, string(object.Type))
|
||||
objectParams = append(objectParams, fmt.Sprintf("id=%s", object.ID))
|
||||
objectParams = append(objectParams, fmt.Sprintf("cbitpos=%d", object.CBitPos))
|
||||
|
||||
@ -86,6 +86,7 @@ const defaultVhostUserStorePath string = "/var/run/kata-containers/vhost-user/"
|
||||
const defaultRxRateLimiterMaxRate = uint64(0)
|
||||
const defaultTxRateLimiterMaxRate = uint64(0)
|
||||
const defaultConfidentialGuest = false
|
||||
const defaultSevSnpGuest = false
|
||||
const defaultGuestSwap = false
|
||||
const defaultRootlessHypervisor = false
|
||||
const defaultDisableSeccomp = false
|
||||
|
||||
@ -149,6 +149,7 @@ type hypervisor struct {
|
||||
DisableVhostNet bool `toml:"disable_vhost_net"`
|
||||
GuestMemoryDumpPaging bool `toml:"guest_memory_dump_paging"`
|
||||
ConfidentialGuest bool `toml:"confidential_guest"`
|
||||
SevSnpGuest bool `toml:"sev_snp_guest"`
|
||||
GuestSwap bool `toml:"enable_guest_swap"`
|
||||
Rootless bool `toml:"rootless"`
|
||||
DisableSeccomp bool `toml:"disable_seccomp"`
|
||||
@ -827,6 +828,7 @@ func newQemuHypervisorConfig(h hypervisor) (vc.HypervisorConfig, error) {
|
||||
GuestMemoryDumpPath: h.GuestMemoryDumpPath,
|
||||
GuestMemoryDumpPaging: h.GuestMemoryDumpPaging,
|
||||
ConfidentialGuest: h.ConfidentialGuest,
|
||||
SevSnpGuest: h.SevSnpGuest,
|
||||
GuestSwap: h.GuestSwap,
|
||||
Rootless: h.Rootless,
|
||||
LegacySerial: h.LegacySerial,
|
||||
@ -1221,6 +1223,7 @@ func GetDefaultHypervisorConfig() vc.HypervisorConfig {
|
||||
TxRateLimiterMaxRate: defaultTxRateLimiterMaxRate,
|
||||
SGXEPCSize: defaultSGXEPCSize,
|
||||
ConfidentialGuest: defaultConfidentialGuest,
|
||||
SevSnpGuest: defaultSevSnpGuest,
|
||||
GuestSwap: defaultGuestSwap,
|
||||
Rootless: defaultRootlessHypervisor,
|
||||
DisableSeccomp: defaultDisableSeccomp,
|
||||
|
||||
@ -422,6 +422,8 @@ func (clh *cloudHypervisor) enableProtection() error {
|
||||
|
||||
case sevProtection:
|
||||
return errors.New("SEV protection is not supported by Cloud Hypervisor")
|
||||
case snpProtection:
|
||||
return errors.New("SEV-SNP protection is not supported by Cloud Hypervisor")
|
||||
|
||||
default:
|
||||
return errors.New("This system doesn't support Confidentian Computing (Guest Protection)")
|
||||
|
||||
@ -348,6 +348,10 @@ type HypervisorConfig struct {
|
||||
// Enable or disable different hardware features, ranging
|
||||
// from memory encryption to both memory and CPU-state encryption and integrity.
|
||||
ConfidentialGuest bool
|
||||
|
||||
// Enables SEV-SNP guests in case both AMD SEV and SNP are supported.
|
||||
// SEV is default.
|
||||
SevSnpGuest bool
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
@ -530,6 +530,9 @@ type HypervisorConfig struct {
|
||||
// from memory encryption to both memory and CPU-state encryption and integrity.
|
||||
ConfidentialGuest bool
|
||||
|
||||
// Enable SEV-SNP guests on AMD machines capable of both
|
||||
SevSnpGuest bool
|
||||
|
||||
// BootToBeTemplate used to indicate if the VM is created to be a template VM
|
||||
BootToBeTemplate bool
|
||||
|
||||
@ -873,6 +876,11 @@ const (
|
||||
// Exclude from lint checking for it won't be used on arm64 code
|
||||
sevProtection
|
||||
|
||||
// AMD Secure Encrypted Virtualization - Secure Nested Paging (SEV-SNP)
|
||||
// https://developer.amd.com/sev/
|
||||
// Exclude from lint checking for it won't be used on arm64 code
|
||||
snpProtection
|
||||
|
||||
// IBM POWER 9 Protected Execution Facility
|
||||
// https://www.kernel.org/doc/html/latest/powerpc/ultravisor.html
|
||||
// Exclude from lint checking for it won't be used on arm64 code
|
||||
@ -889,6 +897,7 @@ var guestProtectionStr = [...]string{
|
||||
pefProtection: "pef",
|
||||
seProtection: "se",
|
||||
sevProtection: "sev",
|
||||
snpProtection: "snp",
|
||||
tdxProtection: "tdx",
|
||||
}
|
||||
|
||||
|
||||
@ -13,6 +13,8 @@ const (
|
||||
tdxCPUFlag = "tdx"
|
||||
|
||||
sevKvmParameterPath = "/sys/module/kvm_amd/parameters/sev"
|
||||
|
||||
snpKvmParameterPath = "/sys/module/kvm_amd/parameters/sev_snp"
|
||||
)
|
||||
|
||||
// Implementation of this function is architecture specific
|
||||
@ -26,6 +28,13 @@ func availableGuestProtection() (guestProtection, error) {
|
||||
if d, err := os.Stat(tdxSysFirmwareDir); (err == nil && d.IsDir()) || flags[tdxCPUFlag] {
|
||||
return tdxProtection, nil
|
||||
}
|
||||
// SEV-SNP is supported and enabled when the kvm module `sev_snp` parameter is set to `Y`
|
||||
// SEV-SNP support infers SEV (-ES) support
|
||||
if _, err := os.Stat(snpKvmParameterPath); err == nil {
|
||||
if c, err := os.ReadFile(snpKvmParameterPath); err == nil && len(c) > 0 && (c[0] == 'Y') {
|
||||
return snpProtection, nil
|
||||
}
|
||||
}
|
||||
// SEV is supported and enabled when the kvm module `sev` parameter is set to `1` (or `Y` for linux >= 5.12)
|
||||
if _, err := os.Stat(sevKvmParameterPath); err == nil {
|
||||
if c, err := os.ReadFile(sevKvmParameterPath); err == nil && len(c) > 0 && (c[0] == '1' || c[0] == 'Y') {
|
||||
|
||||
@ -24,6 +24,8 @@ type qemuAmd64 struct {
|
||||
// inherit from qemuArchBase, overwrite methods if needed
|
||||
qemuArchBase
|
||||
|
||||
snpGuest bool
|
||||
|
||||
vmFactory bool
|
||||
|
||||
devLoadersCount uint32
|
||||
@ -122,6 +124,7 @@ func newQemuArch(config HypervisorConfig) (qemuArch, error) {
|
||||
legacySerial: config.LegacySerial,
|
||||
},
|
||||
vmFactory: factory,
|
||||
snpGuest: config.SevSnpGuest,
|
||||
}
|
||||
|
||||
if config.ConfidentialGuest {
|
||||
@ -169,6 +172,21 @@ func (q *qemuAmd64) bridges(number uint32) {
|
||||
q.Bridges = genericBridges(number, q.qemuMachine.Type)
|
||||
}
|
||||
|
||||
func (q *qemuAmd64) cpuModel() string {
|
||||
var err error
|
||||
cpuModel := defaultCPUModel
|
||||
|
||||
// Temporary until QEMU cpu model 'host' supports AMD SEV-SNP
|
||||
protection, err := availableGuestProtection()
|
||||
if err == nil {
|
||||
if protection == snpProtection && q.snpGuest {
|
||||
cpuModel = "EPYC-v4"
|
||||
}
|
||||
}
|
||||
|
||||
return cpuModel
|
||||
}
|
||||
|
||||
func (q *qemuAmd64) memoryTopology(memoryMb, hostMemoryMb uint64, slots uint8) govmmQemu.Memory {
|
||||
return genericMemoryTopology(memoryMb, hostMemoryMb, slots, q.memoryOffset)
|
||||
}
|
||||
@ -197,6 +215,11 @@ func (q *qemuAmd64) enableProtection() error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// Configure SNP only if specified in config
|
||||
if q.protection == snpProtection && !q.snpGuest {
|
||||
q.protection = sevProtection
|
||||
}
|
||||
|
||||
logger := hvLogger.WithFields(logrus.Fields{
|
||||
"subsystem": "qemuAmd64",
|
||||
"machine": q.qemuMachine,
|
||||
@ -219,6 +242,13 @@ func (q *qemuAmd64) enableProtection() error {
|
||||
q.qemuMachine.Options += "confidential-guest-support=sev"
|
||||
logger.Info("Enabling SEV guest protection")
|
||||
return nil
|
||||
case snpProtection:
|
||||
if q.qemuMachine.Options != "" {
|
||||
q.qemuMachine.Options += ","
|
||||
}
|
||||
q.qemuMachine.Options += "confidential-guest-support=snp"
|
||||
logger.Info("Enabling SNP guest protection")
|
||||
return nil
|
||||
|
||||
// TODO: Add support for other x86_64 technologies
|
||||
|
||||
@ -263,6 +293,16 @@ func (q *qemuAmd64) appendProtectionDevice(devices []govmmQemu.Device, firmware,
|
||||
CBitPos: cpuid.AMDMemEncrypt.CBitPosition,
|
||||
ReducedPhysBits: cpuid.AMDMemEncrypt.PhysAddrReduction,
|
||||
}), "", nil
|
||||
case snpProtection:
|
||||
return append(devices,
|
||||
govmmQemu.Object{
|
||||
Type: govmmQemu.SNPGuest,
|
||||
ID: "snp",
|
||||
Debug: false,
|
||||
File: firmware,
|
||||
CBitPos: cpuid.AMDMemEncrypt.CBitPosition,
|
||||
ReducedPhysBits: 1,
|
||||
}), "", nil
|
||||
case noneProtection:
|
||||
return devices, firmware, nil
|
||||
|
||||
|
||||
@ -293,6 +293,26 @@ func TestQemuAmd64AppendProtectionDevice(t *testing.T) {
|
||||
|
||||
assert.Equal(expectedOut, devices)
|
||||
|
||||
// snp protection
|
||||
amd64.(*qemuAmd64).protection = snpProtection
|
||||
|
||||
devices, bios, err = amd64.appendProtectionDevice(devices, firmware, "")
|
||||
assert.NoError(err)
|
||||
assert.Empty(bios)
|
||||
|
||||
expectedOut = append(expectedOut,
|
||||
govmmQemu.Object{
|
||||
Type: govmmQemu.SNPGuest,
|
||||
ID: "snp",
|
||||
Debug: false,
|
||||
File: firmware,
|
||||
CBitPos: cpuid.AMDMemEncrypt.CBitPosition,
|
||||
ReducedPhysBits: 1,
|
||||
},
|
||||
)
|
||||
|
||||
assert.Equal(expectedOut, devices)
|
||||
|
||||
// tdxProtection
|
||||
amd64.(*qemuAmd64).protection = tdxProtection
|
||||
|
||||
|
||||
@ -209,6 +209,13 @@ func TestQemuArm64AppendProtectionDevice(t *testing.T) {
|
||||
assert.Empty(bios)
|
||||
assert.NoError(err)
|
||||
|
||||
// SNP protection
|
||||
arm64.(*qemuArm64).protection = snpProtection
|
||||
devices, bios, err = arm64.appendProtectionDevice(devices, firmware, "")
|
||||
assert.Empty(devices)
|
||||
assert.Empty(bios)
|
||||
assert.NoError(err)
|
||||
|
||||
// TDX protection
|
||||
arm64.(*qemuArm64).protection = tdxProtection
|
||||
devices, bios, err = arm64.appendProtectionDevice(devices, firmware, "")
|
||||
|
||||
@ -79,6 +79,12 @@ func TestQemuPPC64leAppendProtectionDevice(t *testing.T) {
|
||||
assert.Error(err)
|
||||
assert.Empty(bios)
|
||||
|
||||
//SNP protection
|
||||
ppc64le.(*qemuPPC64le).protection = snpProtection
|
||||
devices, bios, err = ppc64le.appendProtectionDevice(devices, firmware, "")
|
||||
assert.Error(err)
|
||||
assert.Empty(bios)
|
||||
|
||||
//TDX protection
|
||||
ppc64le.(*qemuPPC64le).protection = tdxProtection
|
||||
devices, bios, err = ppc64le.appendProtectionDevice(devices, firmware, "")
|
||||
|
||||
@ -136,6 +136,12 @@ func TestQemuS390xAppendProtectionDevice(t *testing.T) {
|
||||
assert.Error(err)
|
||||
assert.Empty(bios)
|
||||
|
||||
// SNP protection
|
||||
s390x.(*qemuS390x).protection = snpProtection
|
||||
devices, bios, err = s390x.appendProtectionDevice(devices, firmware, "")
|
||||
assert.Error(err)
|
||||
assert.Empty(bios)
|
||||
|
||||
// Secure Execution protection
|
||||
s390x.(*qemuS390x).protection = seProtection
|
||||
|
||||
|
||||
@ -9,7 +9,8 @@ automates the process to build a kernel for Kata Containers.
|
||||
The `build-kernel.sh` script requires an installed Golang version matching the
|
||||
[component build requirements](../../../docs/Developer-Guide.md#requirements-to-build-individual-components).
|
||||
It also requires [yq](https://github.com/mikefarah/yq) version 3.4.1
|
||||
> **Hint**: `snap install yq --channel=v3/stable`
|
||||
> **Hint**: `snap install yq --channel=v3/stable` \
|
||||
> **or** `go install github.com/mikefarah/yq/v3@latest`
|
||||
|
||||
|
||||
The Linux kernel scripts further require a few packages (flex, bison, and libelf-dev)
|
||||
@ -53,7 +54,7 @@ Options:
|
||||
```
|
||||
|
||||
Example:
|
||||
```
|
||||
```bash
|
||||
$ ./build-kernel.sh -v 5.10.25 -g nvidia -f -d setup
|
||||
```
|
||||
> **Note**
|
||||
@ -68,8 +69,8 @@ $ ./build-kernel.sh -v 5.10.25 -g nvidia -f -d setup
|
||||
## Setup kernel source code
|
||||
|
||||
```bash
|
||||
$ go get -d -u github.com/kata-containers/kata-containers
|
||||
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kernel
|
||||
$ git clone github.com/kata-containers/kata-containers
|
||||
$ cd kata-containers/tools/packaging/kernel
|
||||
$ ./build-kernel.sh setup
|
||||
```
|
||||
|
||||
|
||||
@ -101,7 +101,7 @@ Options:
|
||||
-t <hypervisor> : Hypervisor_target.
|
||||
-u <url> : Kernel URL to be used to download the kernel tarball.
|
||||
-v <version> : Kernel version to use if kernel path not provided.
|
||||
-x <type> : Confidential guest protection type, such as sev and tdx
|
||||
-x <type> : Confidential guest protection type, such as sev, snp and tdx
|
||||
EOF
|
||||
exit "$exit_code"
|
||||
}
|
||||
@ -525,7 +525,7 @@ main() {
|
||||
x)
|
||||
conf_guest="${OPTARG}"
|
||||
case "$conf_guest" in
|
||||
sev|tdx) ;;
|
||||
sev|snp|tdx) ;;
|
||||
*) die "Confidential guest type '$conf_guest' not supported" ;;
|
||||
esac
|
||||
;;
|
||||
|
||||
10
tools/packaging/kernel/configs/fragments/x86_64/snp/snp.conf
Normal file
10
tools/packaging/kernel/configs/fragments/x86_64/snp/snp.conf
Normal file
@ -0,0 +1,10 @@
|
||||
# !s390x !ppc64le !arm64
|
||||
# enable sev-snp support
|
||||
CONFIG_AMD_MEM_ENCRYPT=y
|
||||
CONFIG_SEV_GUEST=y
|
||||
CONFIG_VIRT_DRIVERS=y
|
||||
|
||||
# Prepare kernel for direct boot using OVMF
|
||||
CONFIG_EFI=y
|
||||
CONFIG_EFI_STUB=y
|
||||
|
||||
@ -102,6 +102,11 @@ assets:
|
||||
description: "VMM that uses KVM and supports TDX"
|
||||
url: "https://github.com/intel/qemu-dcp"
|
||||
tag: "SPR-BKC-QEMU-v2.5"
|
||||
snp:
|
||||
description: "VMM that uses KVM and supports AMD SEV-SNP"
|
||||
url: "https://github.com/AMDESE/qemu"
|
||||
branch: "snp-v3"
|
||||
commit: "ffa95097ee"
|
||||
|
||||
qemu-experimental:
|
||||
description: "QEMU with virtiofs support"
|
||||
@ -162,6 +167,10 @@ assets:
|
||||
description: "Linux kernel that supports SEV"
|
||||
url: "https://cdn.kernel.org/pub/linux/kernel/v5.x/"
|
||||
version: "v5.19.2"
|
||||
snp:
|
||||
description: "Linux kernel that supports AMD SEV-SNP for VMs"
|
||||
url: "https://cdn.kernel.org/pub/linux/kernel/v5.x/"
|
||||
version: "v5.19.2"
|
||||
|
||||
kernel-experimental:
|
||||
description: "Linux kernel with virtio-fs support"
|
||||
|
||||
Loading…
Reference in New Issue
Block a user