2acb94ef2d
Kernel 6.18.x has an issue with DAX, which is not yet fixed upstream: ``` [ 0.737679] EXT4-fs (pmem0p1): mounted filesystem 79676804-7c8b-491a-b2a6-9bae3c72af70 ro with ordered data mode. Quota mode: disabled. [ 0.737891] VFS: Mounted root (ext4 filesystem) readonly on device 259:1. [ 0.739119] devtmpfs: mounted [ 0.739476] Freeing unused kernel memory: 1920K [ 0.740156] Run /sbin/init as init process [ 0.740229] with arguments: [ 0.740286] /sbin/init [ 0.740321] with environment: [ 0.740369] HOME=/ [ 0.740400] TERM=linux [ 0.743162] Unable to handle kernel paging request at virtual address fffffdffbf000008 [ 0.743285] Mem abort info: [ 0.743316] ESR = 0x0000000096000006 [ 0.743371] EC = 0x25: DABT (current EL), IL = 32 bits [ 0.743444] SET = 0, FnV = 0 [ 0.743489] EA = 0, S1PTW = 0 [ 0.743545] FSC = 0x06: level 2 translation fault [ 0.743610] Data abort info: [ 0.743656] ISV = 0, ISS = 0x00000006, ISS2 = 0x00000000 [ 0.743720] CM = 0, WnR = 0, TnD = 0, TagAccess = 0 [ 0.743785] GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0 [ 0.743848] swapper pgtable: 4k pages, 48-bit VAs, pgdp=00000000b9d17000 [ 0.743931] [fffffdffbf000008] pgd=10000000bfa3d403, p4d=10000000bfa3d403, pud=1000000040bfe403, pmd=0000000000000000 [ 0.744070] Internal error: Oops: 0000000096000006 [#1] SMP [ 0.748888] CPU: 0 UID: 0 PID: 1 Comm: init Not tainted 6.18.4 #1 NONE [ 0.749421] pstate: 004000c5 (nzcv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 0.749969] pc : dax_disassociate_entry.constprop.0+0x20/0x50 [ 0.750444] lr : dax_insert_entry+0xcc/0x408 [ 0.750802] sp : ffff80008000b9e0 [ 0.751083] x29: ffff80008000b9e0 x28: 0000000000000000 x27: 0000000000000000 [ 0.751682] x26: 0000000001963d01 x25: ffff0000004f7d90 x24: 0000000000000000 [ 0.752264] x23: 0000000000000000 x22: ffff80008000bcc8 x21: 0000000000000011 [ 0.752836] x20: ffff80008000ba90 x19: 0000000001963d01 x18: 0000000000000000 [ 0.753407] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000 [ 0.753970] x14: ffffbf3154b9ae70 x13: 0000000000000000 x12: ffffbf3154b9ae70 [ 0.754548] x11: ffffffffffffffff x10: 0000000000000000 x9 : 0000000000000000 [ 0.755122] x8 : 000000000000000d x7 : 000000000000001f x6 : 0000000000000000 [ 0.755707] x5 : 0000000000000000 x4 : 0000000000000000 x3 : fffffdffc0000000 [ 0.756287] x2 : 0000000000000008 x1 : 0000000040000000 x0 : fffffdffbf000000 [ 0.756871] Call trace: [ 0.757107] dax_disassociate_entry.constprop.0+0x20/0x50 (P) [ 0.757592] dax_iomap_pte_fault+0x4fc/0x808 [ 0.757951] dax_iomap_fault+0x28/0x30 [ 0.758258] ext4_dax_huge_fault+0x80/0x2dc [ 0.758594] ext4_dax_fault+0x10/0x3c [ 0.758892] __do_fault+0x38/0x12c [ 0.759175] __handle_mm_fault+0x530/0xcf0 [ 0.759518] handle_mm_fault+0xe4/0x230 [ 0.759833] do_page_fault+0x17c/0x4dc [ 0.760144] do_translation_fault+0x30/0x38 [ 0.760483] do_mem_abort+0x40/0x8c [ 0.760771] el0_ia+0x4c/0x170 [ 0.761032] el0t_64_sync_handler+0xd8/0xdc [ 0.761371] el0t_64_sync+0x168/0x16c [ 0.761677] Code: f9453021 f2dfbfe3 cb813080 8b001860 (f9400401) [ 0.762168] ---[ end trace 0000000000000000 ]--- [ 0.762550] note: init[1] exited with irqs disabled [ 0.762631] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ``` For now, we limit the rootfs that we ship to ARM64 to not use DAX, in the future we'll re-enable it as soon as the patch lands on mainstream kernel. Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
runtime-rs
Wath's runtime-rs
runtime-rs is a new component introduced in Kata Containers 3.0, it is a Rust version of runtime(shim). It like runtime, but they have many difference:
runtime-rsis written in Rust, andruntimeis written in Go.runtimeis the default shim in Kata Containers 3.0,runtime-rsis still under heavy development.runtime-rshas a completed different architecture thanruntime, you can check at the architecture overview.
Note:
runtime-rs is still under heavy development, you should avoid using it in critical system.
Architecture overview
Also, runtime-rs provides the following features:
- Turn key solution with builtin
DragonballSandbox, all components in one process - Async I/O to reduce resource consumption
- Extensible framework for multiple services, runtimes and hypervisors
- Lifecycle management for sandbox and container associated resources
See the architecture overview
for details on the runtime-rs design.
runtime-rs is a runtime written in Rust, it is composed of several crates.
This picture shows the overview about the crates under this directory and the relation between crates.
Not all the features have been implemented yet, for details please check the roadmap.
Crates
The runtime-rs directory contains some crates in the crates directory that compose the containerd-shim-kata-v2.
| Crate | Description |
|---|---|
shim |
containerd shimv2 implementation |
service |
services for containers, includes task service |
runtimes |
container runtimes |
resource |
sandbox and container resources |
hypervisor |
hypervisor that act as a sandbox |
agent |
library used to communicate with agent in the guest OS |
persist |
persist container state to disk |
shim
shim is the entry point of the containerd shim process, it implements containerd shim's binary protocol:
- start: start a new shim process
- delete: delete exist a shim process
- run: run ttRPC service in shim
containerd will launch a shim process and the shim process will serve as a ttRPC server to provide shim service through TaskService from service crate.
service
The runtime-rs has an extensible framework, includes extension of services, runtimes, and hypervisors.
Currently, only containerd compatible TaskService is implemented.
TaskService has implemented the containerd shim protocol,
and interacts with runtimes through messages.
runtimes
Runtime is a container runtime, the runtime handler handles messages from task services to manage containers. Runtime handler and Runtime instance is used to deal with the operation for sandbox and container.
Currently, only VirtContainer has been implemented.
resource
In runtime-rs, all networks/volumes/rootfs are abstracted as resources.
Resources are classified into two types:
- sandbox resources: network, share-fs
- container resources: rootfs, volume, cgroup
Here is a detailed description of the resources.
hypervisor
For VirtContainer, there will be more hypervisors to choose.
Currently, built-in Dragonball has been implemented. We have also added initial support for cloud-hypervisor with CI being added next.
agent
agent is used to communicate with agent in the guest OS from the shim side. The only supported agent is KataAgent.
persist
Persist defines traits and functions to help different components save state to disk and load state from disk.
helper libraries
Some helper libraries are maintained in the library directory so that they can be shared with other rust components.
Build and install
See the build from the source section of the rust runtime installation guide.
Configuration
runtime-rs has the same configuration as runtime with some limitations.
Logging
See the debugging section of the developer guide.
Debugging
See the debugging section of the developer guide.
An experimental alternative binary is available that removes containerd dependencies and makes it easier to run the shim proper outside of the runtime's usual deployment environment (i.e. on a developer machine).
Limitations
For Kata Containers limitations, see the limitations file for further details.
runtime-rs is under heavy developments, and doesn't support all features as the Golang version runtime, check the roadmap for details.