sparrow-kata-full/README.md

# This project has been renamed

Due to a naming mishap, we've had to rename "Kata" to "Cantrip". This repository
will be removed December 1st.

Please use
[https://github.com/AmbiML/sparrow-cantrip-full](https://github.com/AmbiML/sparrow-cantrip-full)
instead.

# Project Sparrow: KataOS

Sparrow is a project to build a low-power secure embedded platform
for Ambient ML applications. The target platform leverages
[RISC-V](https://riscv.org/) and [OpenTitan](https://opentitan.org/).

The Sparrow
software includes a home-grown operating system named KataOS, that runs
on top of [seL4](https://github.com/seL4) and (ignoring the seL4 kernel)
is written almost entirely in [Rust](https://www.rust-lang.org/).

This is a CAmkES project that assembles the entire KataOS. It exists outside
the seL4 source trees since it contains code not intended to go to upstream
seL4.

This uses the [standard CAmkES build system](https://docs.sel4.systems/projects/camkes/manual.html#running-a-simple-example)
by symlinking CMakeLists.txt and assumes
the [CAmkES dependencies](https://docs.sel4.systems/projects/buildsystem/host-dependencies.html#camkes-build-dependencies)
are already installed.
It also symlinks settings.cmake, and so retains
the notion of "apps," which enables the build system to switch which assembly
it builds using the CAMKES\_APP CMake cache value. KataOS just has one app,
*system*.

## Sparrow Rust crates (what's included here).

[More software will be published as we deem it ready for sharing until eventually
all of Sparrow (software and hardware designs) will be available.]

Many KataOS Rust crates are in the *apps/system/components* directory.
Common/shared code is in *kata-os-common*:

- *allocator*: a heap allocator built on the linked-list-allocator crate
- *camkes*: support for writing CAmkES components in Rust
- *capdl*: support for reading the capDL specification generated by capDL-tool
- *copyregion*: a helper for temporarily mapping physical pages into a thread's VSpace
- *cspace-slot*: an RAII helper for the *slot-allocator*
- *logger*: seL4 integration with the Rust logger crate
- *model*: support for processing capDL; used by the kata-os-rootserver
- *panic*: an seL4-specific panic handler
- *sel4-config*: build glue for seL4 kernel configuration
- *sel4-sys*: seL4 system interfaces & glue
- *slot-allocator*: an allocator for slots in the top-level CNode

KataOS system services make up the remaining components:

- *DebugConsole*: a command line interface intended for debug builds
- *MailboxDriver*: a driver for the Mailbox interface used to communicate betwen the security and management cores
- *MemoryManager*: the memory / object manager service that supports dynamic memory management
- *MlCoordinator*: the service that manages the vector core (using the Vector Core driver)
- *OpenTitanUARTDriver*: a driver for the UART on the management core
- *ProcessManager*: the service that creates & manages execution of applications
- *SDKRuntime*: the service that handles application runtime requests
- *SecurityCoordinator*: the service that provides an interface to the security core (using the MailboxDriver)
- *TimerService*: a service built on top of the management core timer hardware

At the moment all the above services are included in both debug and release builds.
Production builds replace the *DebugConsole* with a more limited interface and drop
the UART driver when there is no serial console. The *SDKRuntime* is more a proof of
concept than anything else. There are test applications written in C and Rust in the
*apps* tree that exercise the experimental api. Production systems are likely to have
their own SDK and associated runtime tailored to their needs.

### Depending on Rust crates

To use crates from Sparrow you can reference them from a local repository or
directly from GitHub using git; e.g. in a Config.toml:
```
kata-os-common = { path = "../system/components/kata-os-common" }
kata-os-common = { git = "https://github.com/AmbiML/sparrow-kata" }
```
NB: the git usage depends on cargo's support for searching for a crate
named "kata-os-common" in the kata repo.
When using a git dependency a git tag can be used to lock the crate version.

Note that many Sparrow crates need the seL4 kernel configuration
(e.g. to know whether MCS is configured). This is handled by the
kata-os-common/sel4-config crate that is used by a build.rs to import
kernel configuration parameters as Cargo features. In a Cargo.toml create
a features manifest with the kernel parameters you need e.g.

```
[features]
default = []
# Used by sel4-config to extract kernel config
CONFIG_PRINTING = []
```

then specify build-dependencies:

```
[build-dependencies]
# build.rs depends on SEL4_OUT_DIR = "${ROOTDIR}/out/kata/kernel"
sel4-config = { path = "../../kata/apps/system/components/kata-os-common/src/sel4-config" }
```

and use a build.rs that includes at least:

```
extern crate sel4_config;
use std::env;

fn main() {
    // If SEL4_OUT_DIR is not set we expect the kernel build at a fixed
    // location relative to the ROOTDIR env variable.
    println!("SEL4_OUT_DIR {:?}", env::var("SEL4_OUT_DIR"));
    let sel4_out_dir = env::var("SEL4_OUT_DIR")
        .unwrap_or_else(|_| format!("{}/out/kata/kernel", env::var("ROOTDIR").unwrap()));
    println!("sel4_out_dir {}", sel4_out_dir);

    // Dredge seL4 kernel config for settings we need as features to generate
    // correct code: e.g. CONFIG_KERNEL_MCS enables MCS support which changes
    // the system call numbering.
    let features = sel4_config::get_sel4_features(&sel4_out_dir);
    println!("features={:?}", features);
    for feature in features {
        println!("cargo:rustc-cfg=feature=\"{}\"", feature);
    }
}
```

Note how build.rs expects an SEL4_OUT_DIR environment variable that has the path to
the top of the kernel build area. The build-sparrow.sh script sets this for you but, for
example, if you choose to run ninja directly you will need it set in your environment.

Similar to SEL4_OUT_DIR the kata-os-common/src/sel4-sys crate that has the seL4 system
call wrappers for Rust programs requires an SEL4_DIR envronment variable that has the
path to the top of the kernel sources. This also is set by build-sparrow.sh.

## Memory Footprint

The initial Sparrow target platform was intended to have 4MiB of
memory. A production build of the included services fit in <3MiB
of memory but due to the overhead of CAmkES and the rootserver
boostrap mechanism actually require ~2x that to reach a running
state. The kmem.sh script can be used to analyze memory use. These
services should easily fit in <1MiB but due to CAmkES overhead
(e.g. per-thread cost) are significantly bloated. We use kmem and the
[bloaty tool](https://github.com/google/bloaty) to evaluate memory use.
Reducing memory use to <1MiB likely requires replacing CAmkES by a native
Rust framework like [embassy](https://github.com/embassy-rs/embassy).

We also expect that replacing CAmkES C-based templates by native Rust code
will significantly reduce memory use (as well as improve robustness by
extending the scope of the borrow checker). The RPC mechanism used for
communication between applications and the *SDKRuntime* is a prototype
of a native Rust implementation that demonstrates this.

## Portability

There are three areas in the software where per-architecture support is required:

- *sel4-sys*: system call wrappers
- *kata-os-model*: capDL support for the kata-os-rootserver
- *kata-proc-manager/sel4bundle*: application construction

The first two have approximately the same architectue coverage as
(upstream) seL4; there are likely issues in *kata-os-model* given the
rootserver has never been run on any architectures but riscv32imac and
aarch64. The *sel4bundle* supports _only_ the riscv32 architecture used
by Sparrow and in some areas is very simple (e.g. it only sets up
page tables for a minimal VSpace). The goal was to unify *kata-os-model*
and *sel4bundle* such that architectural support could be directly shared.

## Source Code Headers

Every file containing source code includes copyright and license
information. For dependent / non-Google code these are inherited from
the upstream repositories. If there are Google modifications you may find
the Google Apache license found below.

Apache header:

    Copyright 2022 Google LLC

    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

        https://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.