diff --git a/docs/design/virtualization.md b/docs/design/virtualization.md index 9467fa2a08..18cc76eed4 100644 --- a/docs/design/virtualization.md +++ b/docs/design/virtualization.md @@ -41,14 +41,9 @@ Each hypervisor or VMM varies on how or if it handles each of these. ## Kata Containers Hypervisor and VMM support -Kata Containers is designed to support multiple virtual machine monitors (VMMs) and hypervisors. -Kata Containers supports: - - [ACRN hypervisor](https://projectacrn.org/) - - [Cloud Hypervisor](https://github.com/cloud-hypervisor/cloud-hypervisor)/[KVM](https://www.linux-kvm.org/page/Main_Page) - - [Firecracker](https://github.com/firecracker-microvm/firecracker)/KVM - - [QEMU](http://www.qemu-project.org/)/KVM +Kata Containers [supports multiple hypervisors](../hypervisors.md). -Which configuration to use will depend on the end user's requirements. Details of each solution and a summary are provided below. +Details of each solution and a summary are provided below. ### QEMU/KVM @@ -122,8 +117,6 @@ Devices used: | Solution | release introduced | brief summary | |-|-|-| -| QEMU | 1.0 | upstream QEMU, with support for hotplug and filesystem sharing | -| NEMU | 1.4 | Deprecated, removed as of 1.10 release. Slimmed down fork of QEMU, with experimental support of virtio-fs | -| Firecracker | 1.5 | upstream Firecracker, rust-VMM based, no VFIO, no FS sharing, no memory/CPU hotplug | -| QEMU-virtio-fs | 1.7 | upstream QEMU with support for virtio-fs. Will be removed once virtio-fs lands in upstream QEMU | | Cloud Hypervisor | 1.10 | rust-VMM based, includes VFIO and FS sharing through virtio-fs, no hotplug | +| Firecracker | 1.5 | upstream Firecracker, rust-VMM based, no VFIO, no FS sharing, no memory/CPU hotplug | +| QEMU | 1.0 | upstream QEMU, with support for hotplug and filesystem sharing | diff --git a/docs/hypervisors.md b/docs/hypervisors.md new file mode 100644 index 0000000000..08ab847385 --- /dev/null +++ b/docs/hypervisors.md @@ -0,0 +1,68 @@ +# Hypervisors + +* [Hypervisors](#hypervisors) + * [Introduction](#introduction) + * [Types](#types) + * [Determine currently configured hypervisor](#determine-currently-configured-hypervisor) + * [Choose a Hypervisor](#choose-a-hypervisor) + +## Introduction + +Kata Containers supports multiple hypervisors. This document provides a very +high level overview of the available hypervisors, giving suggestions as to +which hypervisors you may wish to investigate further. + +> **Note:** +> +> This document is not prescriptive or authoritative: +> +> - It is up to you to decide which hypervisors may be most appropriate for +> your use-case. +> - Refer to the official documentation for each hypervisor for further details. + +## Types + +Since each hypervisor offers different features and options, Kata Containers +provides a separate +[configuration file](/src/runtime/README.md#configuration) +for each. The configuration files contain comments explaining which options +are available, their default values and how each setting can be used. + +> **Note:** +> +> The simplest way to switch between hypervisors is to create a symbolic link +> to the appropriate hypervisor-specific configuration file. + +| Hypervisor | Written in | Architectures | Type | Configuration file | +|-|-|-|-|-| +[ACRN] | C | `x86_64` | Type 1 (bare metal) | `configuration-acrn.toml` | +[Cloud Hypervisor] | rust | `aarch64`, `x86_64` | Type 2 ([KVM]) | `configuration-clh.toml` | +[Firecracker] | rust | `aarch64`, `x86_64` | Type 2 ([KVM]) | `configuration-fc.toml` | +[QEMU] | C | all | Type 2 ([KVM]) | `configuration-qemu.toml` | + +## Determine currently configured hypervisor + +```bash +$ kata-runtime kata-env | awk -v RS= '/\[Hypervisor\]/' | grep Path +``` + +## Choose a Hypervisor + +The table below provides a brief summary of some of the differences between +the hypervisors: + + +| Hypervisor | Summary | Features | Limitations | Container Creation speed | Memory density | Use cases | Comment | +|-|-|-|-|-|-|-|-| +[ACRN] | Safety critical and real-time workloads | | | excellent | excellent | Embedded and IOT systems | For advanced users | +[Cloud Hypervisor] | Low latency, small memory footprint, small attack surface | Minimal | | excellent | excellent | High performance modern cloud workloads | | +[Firecracker] | Very slimline | Extremely minimal | Doesn't support all device types | excellent | excellent | Serverless / FaaS | | +[QEMU] | Lots of features | Lots | | good | good | Good option for most users | | All users | + +For further details, see the [Virtualization in Kata Containers](design/virtualization.md) document and the official documentation for each hypervisor. + +[ACRN]: https://projectacrn.org +[Cloud Hypervisor]: https://github.com/cloud-hypervisor/cloud-hypervisor +[Firecracker]: https://github.com/firecracker-microvm/firecracker +[KVM]: https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine +[QEMU]: http://www.qemu-project.org diff --git a/src/runtime/README.md b/src/runtime/README.md index b5ee5dc3ad..02ffb0d8b3 100644 --- a/src/runtime/README.md +++ b/src/runtime/README.md @@ -19,6 +19,8 @@ For details of the other Kata Containers repositories, see the * [Quick start for developers](#quick-start-for-developers) * [Architecture overview](#architecture-overview) * [Configuration](#configuration) + * [Hypervisor specific configuration](#hypervisor-specific-configuration) + * [Stateless systems](#stateless-systems) * [Logging](#logging) * [Kata OCI](#kata-oci) * [Kata containerd shimv2](#kata-containerd-shimv2) @@ -106,6 +108,15 @@ The file contains comments explaining all options. > You may need to modify this file to optimise or tailor your system, or if you have > specific requirements. +### Hypervisor specific configuration + +Kata Containers supports multiple hypervisors so your `configuration.toml` +configuration file may be a symbolic link to a hypervisor-specific +configuration file. See +[the hypervisors document](../../docs/hypervisors.md) for further details. + +### Stateless systems + Since the runtime supports a [stateless system](https://clearlinux.org/about), it checks for this configuration file in multiple locations, two of which are