From 9948b1892a99036c9f069e7c788c93c701bc547f Mon Sep 17 00:00:00 2001 From: Julio Montes Date: Mon, 20 Aug 2018 15:26:33 -0500 Subject: [PATCH] usecases: add vsock documentation Add vsock documentation fixes #226 Signed-off-by: Julio Montes --- VSocks.md | 126 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 126 insertions(+) create mode 100644 VSocks.md diff --git a/VSocks.md b/VSocks.md new file mode 100644 index 0000000000..dc1357dd9b --- /dev/null +++ b/VSocks.md @@ -0,0 +1,126 @@ +# Kata Containers and vsocks + +- [Introduction](#introduction) + - [proxy communication diagram](#proxy-communication-diagram) + - [vsock communication diagram](#vsock-communication-diagram) +- [System requirements](#system-requirements) +- [Advantages of using vsocks](#advantages-of-using-vsocks) + - [High density](#high-density) + - [Reliability](#reliability) + +## Introduction + +There are two different ways processes in the virtual machine can communicate +with processes in the host. The first one is by using serial ports, where the +processes in the virtual machine can read/write data from/to a serial port +device and the processes in the host can read/write data from/to a unix socket. +Most GNU/Linux distributions have support for serial ports, making it the most +portable solution. However, the serial link limits read/write access to one +process at a time. To deal with this limitation the resources (serial port and +unix socket) must be multiplexed. In Kata Containers those resources are +multiplexed by using [kata-proxy][2] and [yamux][3], the following diagram shows +how it's implemented. + + +### proxy communication diagram + +``` +.----------------------. +| .------------------. | +| | .-----. .-----. | | +| | |cont1| |cont2| | | +| | `-----' `-----' | | +| | \ / | | +| | .---------. | | +| | | agent | | | +| | `---------' | | +| | | | | +| | .-----------. | | +| |POD |serial port| | | +| `----|-----------|-' | +| | socket | | +| `-----------' | +| | | +| .-------. | +| | proxy | | +| `-------' | +| | | +| .------./ \.------. | +| | shim | | shim | | +| `------' `------' | +| Host | +`----------------------' +``` + +A newer, simpler method is [vsocks][4], which can accept connections from +multiple clients and does not require multiplexers ([kata-proxy][2] and +[yamux][3]). The following diagram shows how it's implemented in Kata Containers. + + +### vsock communication diagram + +``` +.----------------------. +| .------------------. | +| | .-----. .-----. | | +| | |cont1| |cont2| | | +| | `-----' `-----' | | +| | | | | | +| | .---------. | | +| | | agent | | | +| | `---------' | | +| | | | | | +| | POD .-------. | | +| `-----| vsock |----' | +| `-------' | +| | | | +| .------. .------. | +| | shim | | shim | | +| `------' `------' | +| Host | +`----------------------' +``` + +## System requirements + +The host Linux kernel version must be greater than or equal to v4.8, and the +`vhost_vsock` module must be loaded or built-in (CONFIG_VHOST_VSOCK=y). To +load the module run the following command: + +``` +$ sudo modprobe -i vhost_vsock +``` + +The Kata Containers version must be greater than or equal to 1.2.0 and `use_vsock` +must be set to `true` in the runtime [configuration file][1]. + +## Advantages of using vsocks + +### High density + +Using a proxy for multiplexing the connections between the VM and the host uses +4.5MB per [POD][5]. In a high density deployment this could add up to GBs of +memory that could have been used to host more PODs. When we talk about density +each kilobyte matters and it might be the decisive factor between run another +POD or not. For example if you have 500 PODs running in a server, the same +amount of [kata-proxy][2] processes will be running and consuming for around +2250MB of RAM. Before making the decision not to use vsocks, you should ask +yourself, how many more containers can run with the memory RAM consumed by the +kata-proxies?. + +### Reliability + +[kata-proxy][2] is in charge of multiplexing the connections between virtual +machine and host processes, if it dies all connections get broken. For example +if you have a [POD][5] with 10 containers running, if kata-proxy dies it would +be impossible to contact your containers, though they would still be running. +Since communication via vsocks is direct, the only way to lose communication +with the containers is if the VM itself or the [shim][6] dies, if this happens +the containers are removed automatically. + +[1]: https://github.com/kata-containers/runtime#configuration +[2]: https://github.com/kata-containers/proxy +[3]: https://github.com/hashicorp/yamux +[4]: https://wiki.qemu.org/Features/VirtioVsock +[5]: https://github.com/kata-containers/documentation/blob/master/constraints/cpu.md#virtual-cpus-and-kubernetes-pods +[6]: https://github.com/kata-containers/shim