docs: document erofs deployment requirements

Document operational requirements for using the EROFS snapshotter
with Kata Containers.

Call out that EROFS layer conversion can happen during
CreateContainer, not only during image pull, so kubelet
runtimeRequestTimeout may need to be increased for large images.
Also document the kernel modules needed for EROFS and dm-verity,
the erofs-utils version required by the configured mkfs options,
and the distinction between host-side fs-verity and guest-visible
dm-verity.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Assisted-by: OpenAI <support@openai.com>
This commit is contained in:
Manuel Huber
2026-07-10 22:17:28 +00:00
parent af212219b8
commit bc5ae6437d

View File

@@ -15,7 +15,7 @@ This section provides a quick overview of the steps to get started with EROFS sn
### Quick Steps
1. **Install erofs-utils**: Install erofs-utils (version >= 1.8) on your host system
1. **Install erofs-utils**: Install erofs-utils (version >= 1.8.2) on your host system
2. **Configure containerd**: Enable EROFS snapshotter and differ in containerd configuration
3. **Configure Kata Containers**: Set up runtime-rs with appropriate hypervisor settings
4. **Run a container**: Use `ctr` or Kubernetes to run containers with EROFS snapshotter
@@ -25,17 +25,29 @@ This section provides a quick overview of the steps to get started with EROFS sn
| Component | Version Requirement |
|-----------|-------------------|
| Linux kernel | >= 5.4 (with `erofs` module, higher recommended) |
| erofs-utils | >= 1.8 (fsmerge `mkfs_options` require >= 1.8.2) |
| erofs-utils | >= 1.8.2 |
| containerd | >= 2.2 (with EROFS snapshotter and differ support, higher recommended) |
| Kata Containers | Latest `main` branch with runtime-rs |
| QEMU | >= 5.0 (VMDK flat-extent support and >= 9.0 higher recommended) |
> **Note**: When enabling the EROFS snapshotter on a node that is already in
> use, existing images in the containerd image store may not yet have EROFS
> snapshots. Since those images are already present, kubelet may skip pulling
> them again, so containerd can end up converting their layers into EROFS
> snapshots during `CreateContainer` rather than during the image pull phase.
> Configure kubelet's `runtimeRequestTimeout` to cover the slowest expected
> EROFS preparation path for the target node and workload. Large images can
> otherwise exceed kubelet's CRI request deadline before the sandbox starts.
## Installation Guide
This section provides detailed step-by-step instructions for installing and configuring EROFS snapshotter with Kata Containers.
### Step 1: Install erofs-utils
The configuration below uses fsmerge `mkfs_options` that require erofs-utils
>= 1.8.2.
```bash
# Debian/Ubuntu
$ sudo apt install erofs-utils
@@ -48,7 +60,7 @@ Verify the version:
```bash
$ mkfs.erofs --version
# Should show 1.8 or higher (1.8.2+ required for fsmerge mkfs_options)
# Should show 1.8.2 or higher
```
Load the kernel module:
@@ -57,11 +69,25 @@ Load the kernel module:
$ sudo modprobe erofs
```
When using EROFS dm-verity mode, also load the dm-verity device-mapper target:
```bash
$ sudo modprobe dm-mod
$ sudo modprobe dm-verity
```
Make these modules load persistently before containerd starts, for example via
your distribution's modules-load mechanism. If the required kernel support is
missing when containerd starts, enabling the EROFS snapshotter can fail early
instead of producing a useful workload-level error.
### Step 2: Configure containerd
#### Enable the EROFS snapshotter and differ
> **Note**: The following settings target containerd v2.3.0-beta.0 and erofs-utils v1.8.10. Compatibility for other versions is not guaranteed, as configuration options evolve. Always cross-reference with the official documentation for your current version.
> **Note**: The following settings target containerd v2.3.0. The
> configured EROFS fsmerge `mkfs_options` require erofs-utils >= 1.8.2. Always
> cross-reference with the official documentation for your current versions.
Edit your containerd configuration (typically `/etc/containerd/config.toml`):
@@ -91,8 +117,46 @@ version = 3
[plugins.'io.containerd.snapshotter.v1.erofs']
default_size = '<SIZE>' # SIZE=6G or 10G or other size
max_unmerged_layers = 0
enable_fsverity = true
```
`enable_fsverity = true` enables host-side fs-verity handling for EROFS layer
blobs when the kernel and backing filesystem support it. This is separate from
the dm-verity mode described below: fs-verity protects layer blob files on the
host, while dm-verity protects the block devices mounted by the Kata guest.
Containerd treats fs-verity support as best-effort and may skip it if the host
kernel or backing filesystem does not support it. If fs-verity protection is
required, make sure the filesystem backing containerd's EROFS snapshotter state
supports fs-verity. For example, on ext4 this requires the `verity` filesystem
feature on the relevant device:
```bash
$ sudo tune2fs -O verity <device-backing-containerd-state>
```
For dm-verity, enable metadata generation in the differ and force strict
dm-verity use in the snapshotter:
```toml
[plugins.'io.containerd.differ.v1.erofs']
enable_dmverity = true
[plugins.'io.containerd.snapshotter.v1.erofs']
dmverity_mode = 'on'
```
Avoid switching a persistent node back and forth between dm-verity and
non-dm-verity EROFS modes without cleaning or rebuilding the EROFS snapshotter
state. Keep dm-verity consistently enabled or consistently disabled for a given
snapshotter state, because existing layers may have been prepared without
dm-verity metadata.
When dm-verity is enabled, avoid `dmverity_mode = 'auto'`. Auto mode does not
solve stale layer state, because layers prepared without dm-verity metadata can
still be used without dm-verity protection. Strict `dmverity_mode = 'on'` makes
such stale layers fail instead of silently falling back to an unprotected path.
#### Verify the EROFS plugins are loaded
Check if EROFS module is loaded