From 2174fee48dfcc05b105b16006201761752bf4d22 Mon Sep 17 00:00:00 2001 From: Hui Zhu Date: Wed, 14 Jul 2021 16:32:22 +0800 Subject: [PATCH 1/3] docs: Add swap annotations introduction Add swap annotations introduction of `io.katacontainers.config.hypervisor.enable_guest_swap`, `io.katacontainers.container.resource.swappiness"` and `io.katacontainers.container.resource.swap_in_bytes"` to how-to-set-sandbox-config-kata.md. Fixes: #2326 Signed-off-by: Hui Zhu --- docs/how-to/how-to-set-sandbox-config-kata.md | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/docs/how-to/how-to-set-sandbox-config-kata.md b/docs/how-to/how-to-set-sandbox-config-kata.md index 044e1576c0..4cb15968a4 100644 --- a/docs/how-to/how-to-set-sandbox-config-kata.md +++ b/docs/how-to/how-to-set-sandbox-config-kata.md @@ -91,6 +91,13 @@ There are several kinds of Kata configurations and they are listed below. | `io.katacontainers.config.hypervisor.virtio_fs_cache` | string | the cache mode for virtio-fs, valid values are `always`, `auto` and `none` | | `io.katacontainers.config.hypervisor.virtio_fs_daemon` | string | virtio-fs `vhost-user` daemon path | | `io.katacontainers.config.hypervisor.virtio_fs_extra_args` | string | extra options passed to `virtiofs` daemon | +| `io.katacontainers.config.hypervisor.enable_guest_swap` | `boolean` | enable swap in the guest | + +## Container Options +| Key | Value Type | Comments | +|-------| ----- | ----- | +| `io.katacontainers.container.resource.swappiness"` | `uint64` | specify the `Resources.Memory.Swappiness` | +| `io.katacontainers.container.resource.swap_in_bytes"` | `uint64` | specify the `Resources.Memory.Swap` | # CRI-O Configuration @@ -100,11 +107,12 @@ In case of CRI-O, all annotations specified in the pod spec are passed down to K For containerd, annotations specified in the pod spec are passed down to Kata starting with version `1.3.0` of containerd. Additionally, extra configuration is -needed for containerd, by providing a `pod_annotations` field in the containerd config -file. The `pod_annotations` field is a list of annotations that can be passed down to -Kata as OCI annotations. It supports golang match patterns. Since annotations supported -by Kata follow the pattern `io.katacontainers.*`, the following configuration would work -for passing annotations to Kata from containerd: +needed for containerd, by providing `pod_annotations` field and +`container_annotations` field in the containerd config +file. The `pod_annotations` field and `container_annotations` field are two lists of +annotations that can be passed down to Kata as OCI annotations. They support golang match +patterns. Since annotations supported by Kata follow the pattern `io.katacontainers.*`, +the following configuration would work for passing annotations to Kata from containerd: ``` $ cat /etc/containerd/config @@ -113,6 +121,7 @@ $ cat /etc/containerd/config [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata] runtime_type = "io.containerd.kata.v2" pod_annotations = ["io.katacontainers.*"] + container_annotations = ["io.katacontainers.*"] .... ``` From 74d645cd2130337d8598d974f656c58d7e6c736b Mon Sep 17 00:00:00 2001 From: Hui Zhu Date: Thu, 29 Jul 2021 11:12:33 +0800 Subject: [PATCH 2/3] how-to: Add how-to-setup-swap-devices-in-guest-kernel.md Add how-to-setup-swap-devices-in-guest-kernel.md to how-to to introduce how to setup swap device in guest kernel. Fixes: #2326 Signed-off-by: Hui Zhu --- docs/how-to/README.md | 1 + ...w-to-setup-swap-devices-in-guest-kernel.md | 59 +++++++++++++++++++ 2 files changed, 60 insertions(+) create mode 100644 docs/how-to/how-to-setup-swap-devices-in-guest-kernel.md diff --git a/docs/how-to/README.md b/docs/how-to/README.md index ea0c508a33..07a46bcc22 100644 --- a/docs/how-to/README.md +++ b/docs/how-to/README.md @@ -34,3 +34,4 @@ - [How to set sandbox Kata Containers configurations with pod annotations](how-to-set-sandbox-config-kata.md) - [How to monitor Kata Containers in K8s](how-to-set-prometheus-in-k8s.md) - [How to use hotplug memory on arm64 in Kata Containers](how-to-hotplug-memory-arm64.md) +- [How to setup swap devices in guest kernel](how-to-setup-swap-devices-in-guest-kernel.md) diff --git a/docs/how-to/how-to-setup-swap-devices-in-guest-kernel.md b/docs/how-to/how-to-setup-swap-devices-in-guest-kernel.md new file mode 100644 index 0000000000..8ab9e89e5d --- /dev/null +++ b/docs/how-to/how-to-setup-swap-devices-in-guest-kernel.md @@ -0,0 +1,59 @@ +# Setup swap device in guest kernel + +## Introduction + +Setup swap device in guest kernel can help to increase memory capacity, handle some memory issues and increase file access speed sometimes. +Kata Containers can insert a raw file to the guest as the swap device. + +## Requisites + +The swap config of the containers should be set by [annotations](how-to-set-sandbox-config-kata.md#container-options). So [extra configuration is needed for containerd](how-to-set-sandbox-config-kata.md#containerd-configuration). + +Kata Containers just supports setup swap device in guest kernel with QEMU. +Install and setup Kata Containers as shown [here](../install/README.md). + +Enable setup swap device in guest kernel as follows: +``` +$ sudo sed -i -e 's/^#enable_guest_swap.*$/enable_guest_swap = true/g' /etc/kata-containers/configuration.toml +``` + +## Run a Kata Container utilizing swap device + +Use following command to start a Kata Container with swappiness 60 and 1GB swap device (swap_in_bytes - memory_limit_in_bytes). +``` +$ pod_yaml=pod.yaml +$ container_yaml=container.yaml +$ image="quay.io/prometheus/busybox:latest" +$ cat << EOF > "${pod_yaml}" +metadata: + name: busybox-sandbox1 +EOF +$ cat << EOF > "${container_yaml}" +metadata: + name: busybox-test-swap +annotations: + io.katacontainers.container.resource.swappiness: "60" + io.katacontainers.container.resource.swap_in_bytes: "2147483648" +linux: + resources: + memory_limit_in_bytes: 1073741824 +image: + image: "$image" +command: +- top +EOF +$ sudo crictl pull $image +$ podid=$(sudo crictl runp $pod_yaml) +$ cid=$(sudo crictl create $podid $container_yaml $pod_yaml) +$ sudo crictl start $cid +``` + +Kata Container setups swap device for this container only when `io.katacontainers.container.resource.swappiness` is set. + +The following table shows the swap size how to decide if `io.katacontainers.container.resource.swappiness` is set. +|`io.katacontainers.container.resource.swap_in_bytes`|`memory_limit_in_bytes`|swap size| +|---|---|---| +|set|set| `io.katacontainers.container.resource.swap_in_bytes` - `memory_limit_in_bytes`| +|not set|set| `memory_limit_in_bytes`| +|not set|not set| `io.katacontainers.config.hypervisor.default_memory`| +|set|not set|cgroup doesn't support this usage| From d67a414b2b5713d9d70fb12b395d9ef73cf48d45 Mon Sep 17 00:00:00 2001 From: Hui Zhu Date: Fri, 10 Sep 2021 10:41:53 +0800 Subject: [PATCH 3/3] src/runtime/README.md: Fix URL of Licence Fix URL of Licence of src/runtime/README.md. Fixes: #2326 Signed-off-by: Hui Zhu --- src/runtime/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/runtime/README.md b/src/runtime/README.md index a13636096c..f8a3efe5d8 100644 --- a/src/runtime/README.md +++ b/src/runtime/README.md @@ -27,7 +27,7 @@ to work seamlessly with both Docker and Kubernetes respectively. The code is licensed under an Apache 2.0 license. -See [the license file](LICENSE) for further details. +See [the license file](../../LICENSE) for further details. ## Platform support