Commit Graph

32 Commits

Author SHA1 Message Date
Sam Leffler
34809a47e1 kata-memory-interface: improve portability
Change-Id: I8b436a32aabbc5e0e57d14680cc7d09f46b7bc30
GitOrigin-RevId: 79e28f7b6f5ac4bb96a2e09f152ac92bb91dc5f7
2022-10-18 15:35:45 -07:00
Sam Leffler
173e7211c7 Merge "kata-memory-manager: remove dead code"
GitOrigin-RevId: e6f4bd529b9f934e3488cc7c162b4f825511b0a4
2022-10-06 19:01:24 +00:00
Sam Leffler
601e7f5c78 kata: compile-out debug & trace log calls in release builds
Change-Id: Ibde5a49141daa67c57e24340b73b436d08e68e7a
GitOrigin-RevId: 4a77833a6cd3e16850d2e4d90b7e7ed916770b50
2022-10-06 19:00:28 +00:00
Sam Leffler
b099005951 kata-os-logger: support no logging interface connection out of a component
When a CAmkES component lacks an outbound connection to send log msgs
there will be no logger_log symbol. Use a weak ref here to handle that
without resorting to a feature or similar.

Mark logger connections as "maybe" so they are optional.

Change-Id: I6ecd939014d26a612d115741fd2ac673afa40857
GitOrigin-RevId: 0b1bf2611cbb628500cae37889c6547a996d50e9
2022-10-06 19:00:19 +00:00
Sam Leffler
326ec0d6c9 MemoryManager: sort untyped slabs by available space
Change-Id: I3ee5a717c506f8a969dd7e3465ad612863835b7a
GitOrigin-RevId: 4f75f4a24d6a4a72dd4fcf3879bae342eee0ce7d
2022-10-06 19:00:10 +00:00
June Tate-Gans
1a068b0143 kata: Prepend the Google copyright and Apache 2.0 license
This includes adding it to TOML files and CAmkES assembly files.

Change-Id: I263e7a566df91fccc04f9b2186edab13331290c5
GitOrigin-RevId: 48a35b06ca868a0a4d379f881068cc4dad491669
2022-10-06 18:59:16 +00:00
June Tate-Gans
a5ca02b0e7 cbindgen: Dynamically generate header files during toplevel build
Change-Id: I9e62df409506f9e9e79c984f966b34cf09c1b95f
GitOrigin-RevId: 319803d00536e972f6aa65eacb5c471db97be9af
2022-10-06 18:58:29 +00:00
Cindy Liu
50cd809320 Merge changes If949288d,I9849065a,I121edee2
* changes:
  kata: LSC: rustfmt with global rustfmt.toml
  kata: update global rustfmt flags
  kata: Promote rustfmt.toml to all kata crates

GitOrigin-RevId: 452e66fa7231f39cff81dc2fbe37ad1a1fc62fb9
2022-10-06 18:57:32 +00:00
Sam Leffler
846993767c Merge "kata-os-model: reclaim rootserver resources"
GitOrigin-RevId: d4cefc38720351494ff4c96848112c11ce36c2a6
2022-10-06 18:57:00 +00:00
Sam Leffler
0fdb94a938 Merge changes from topic "kata-os-bootinfo"
* changes:
  MemoryManager: add kernel reserved memory to startup logging
  sel4-sys: add kernel reserved memory size to BootInfo

GitOrigin-RevId: f5a2227c1b4c919db4658e361c8c0479c67f313d
2022-10-06 18:56:53 +00:00
Sam Leffler
13c433c647 Merge "MemoryManager: log system-wide memory stats at boot"
GitOrigin-RevId: f62d293c8b6ac0f11a8ee163b4bbb8b19b7030fc
2022-10-06 18:56:30 +00:00
Sam Leffler
ce1543c466 kata: rustfmt
Change-Id: I1edc1d0aa241983fb9336d9bd7e42f2dfa362a15
GitOrigin-RevId: 1a10df3451ce2dfb7d54c6553cfa5d237bf96431
2022-10-06 18:56:15 +00:00
Sam Leffler
05233af12c Add capscan suport.
Add support to output the contents of the top-level CNode of a CAmkES
service or KataOS application to the serial console. This is dependent
on kernel support that is enabled with CONFIG_PRINTING. Applications
must be running; otherwise there is no CSpace to dump.

Specific changes:
- add a "capscan" shell command
- add capscan method to each CAmkES interface
- add capscan_bundle method to the ProcessControlInterface
- add Camkes::capscan() to dump the top-level CNode
- add ProcessManager support to dump the CNode for a bundle

TODO: fix syscall wrapper error return

Change-Id: If6ca222decdb4c40a1d3a63e69792eb3feb30f6a
GitOrigin-RevId: 504c0182ccccf287b5d58cd8e33981c11d7539d7
2022-10-06 18:56:08 +00:00
Sam Leffler
c2accc33b0 MemoryManager: correct initial memory use and add mdebug command
- Use seL4_Untyped_Describe to get an accurate view of each
  UntypedMemory slab being managed; this makes mstats reflect
  rootserver allocations.
- Track memory allocated before we run as "overhead" (was meant to
  track fragmentation but was always zero).
- Add an "mdebug" command to describe each managed memory slab;
  this is useful to see whether the kernel's view of memory use is
  consistent with MemoryManager.

Change-Id: I53b2738c430ad3356ecd16a1cad29ca92dc74beb
GitOrigin-RevId: 2ad43f9b7760c722a6590ea049a3814c8dcccba7
2022-10-06 18:56:00 +00:00
Sam Leffler
70b4072428 kata-memory-manager: add CONFIG_NOISY_UNTYPEDS feature
Enabling CONFIG_NOISY_UNTYPEDS causes the UntypedMemory slabs received
at boot to be logged to the console.

Change-Id: Ieceedbe17b09c4bb72a2e40e44daa041990019af
GitOrigin-RevId: b5f598cac4302e24b501a8df0c6e0c194b27b991
2022-10-06 18:55:32 +00:00
Cindy Liu
c9f36f4da8 Merge "Changes for updating the toolchain to nightly-1.58.0"
GitOrigin-RevId: 6eadbd3c3652d88b3079635f2649b43e370fe4bd
2022-10-06 18:54:11 +00:00
Cindy Liu
70b86c9c84 Update the cbindgen toml to re-enable clang-format check
Change-Id: I826b172087a09111c4f9934794f2a5c9e195e1ad
GitOrigin-RevId: ed65bc8643bf5e909760dc632d856048e8aace62
2022-10-06 18:53:48 +00:00
Sam Leffler
b2bd86e43b kata-security-coordinator::fakeimpl: correct LoadModel & LoadApplication
Change the fake to behave as the real impl will for LoadModel &
LoadApplication: return a deep copy of the saved package contents as
would happen if the data were pulled from flash. Match this behaviour in
the kata-shell SecurityCoordinator test commands and the MlCoordinator
by taking ownership of the received objects and free'ing them when no
longer needed.

With these changes one can install a bundle and repeatedly load_application
without leaking any memory, capabilities, or slots in the toplevel CNode
of the caller (DebugConsole in this case).

Likewise doing install of a model, test_mlexecute, and then uninstall
of the model's bundle returns all resources.

Specific changes:
- correctly release resources in kata-shell load_application & load_model
- correct release of bundle_frames in seL4BundleImpl::stop
- release resources in MlCoordinator::load_model
- connect the MemoryInterface to the MlCoordinator so it can return memory
- setup two copyregions in the SecurityCoordinator to do the deep copy
- add ObjdDescBundle::cptr_iter for iterating over the set of seL4_CPtr's
- hack kata_frame_alloc_in_cnode to split requests according to the
  kernel's config on the max Retype count
- while here switch test_mailbox to use one of the copyregions

TODO:
- deep_copy allocates all frames at once which requires a band-aid;
  either hide that in MemoryManager or maybe allocate a page at a time

Change-Id: Ia425976b31ea7a32b1d0e4affc3a0ef9ba966c87
GitOrigin-RevId: 31d5bc99b569a5eab9c33c7e1014793bfe57161e
2022-10-06 18:53:33 +00:00
Sam Leffler
b0d1b6efac kata_object_alloc_in_cnode: misc fixups
- optimize CNode size calc
- remove an extranous map_err that hid the true error

Change-Id: I63a43dff7c242d49f2f968abdde4134ac300d0b1
GitOrigin-RevId: 9e8e9ec4b3bb33dca53ee45aed231836dec7d6e4
2022-10-06 18:53:04 +00:00
Sam Leffler
9920153e10 kata-memory-component: add debug asserts for attached capabilities
Change-Id: I1ef3fa98583a7ac59269d8c3aa15c886a0d8b768
GitOrigin-RevId: 815a4358d6516986e57dc2987f3a9ade46cbffb6
2022-10-06 18:52:45 +00:00
Sam Leffler
0d27b4a3f0 kata-os-camkes: wrap request ipc buffer capability handling
Add Camkes::set_request_cap to attach an seL4 capability to an outbound
ipc message. The return value is an RAII wrapper that cleans up state
and must be held until after the CAmkES rpc call completes

Change-Id: I0672c59e0b5e43e39c9ea3fb16809270a33f51ef
GitOrigin-RevId: 56be13a2c05fcc1b4a1aa5c8e0eab47bcd0f2345
2022-10-06 18:52:18 +00:00
Sam Leffler
67442dc8f0 kata-os-camkes: export C bindings for better CAmkES integration
In particular this exports the CAP_RELEASE flag so there is a single
source for it's definition and KataOS-specific support in the CAmkES
templates is enabled only for KataOS components.

Specific changes:
- adds cbindgen-generated CamkesBindings.h
- adds include of CamkesBindings.h in component cbindgen files
- add a dep on cbindgen.toml in all Makefiles
- update generated interface files

Change-Id: Ib6239d3ac0036b7a04bb36afccf25a05737b0e56
GitOrigin-RevId: af10117fa253f0c7c67969a5852ced9d992c6274
2022-10-06 18:52:07 +00:00
Sam Leffler
ddacc3762e Add kata-os-common::camkes support.
Add a new "camkes" submodule that consolidates KataOS CAmkES component
integration boilerplate. Each component is expected to declare:

static mut CAMKES: Camkes = Camkes::new("ProcessManager");

and then (typically) use "pre_init" to setup the logger, heap, and the
slot allocator. More fine-grained control is provided by:

fn init_logger(self: &Cmakes, level: Log::LevelFilter);
fn init_allocator(self: &Camkes, heap: &'static mut [u8]);
fn init_slot_allocator(self: &Camkes, first_slot: seL4_CPtr, last_slot: seL4_CPtr);

When receiving capabilities use "init_recv_path" to setup the IPCBuffer
receive path and "assert_recv_path" & "check_recv_path" calls to verify
noting has clobbered the setting.

The debug_assert_slot_* macros are wrapped in Camkes:: functions and a
"top_level_path" function for constructing seL4_CPath objects. Altogether
this normally allows a component to be written without direct use of the
CAmkES global static identifiers SELF_CNODE*.

Change-Id: Ia1351e411a5355789cf74bc0fcfe0e41a418b7d4
GitOrigin-RevId: fb81a8e0687ed9321c9961410edd5dbd54093ce5
2022-10-06 18:48:10 +00:00
Sam Leffler
e2ec09e001 MemoryManager: clippy findings
Change-Id: I9a82a9cd7628c7a770752baccf382a7c4e7b625c
GitOrigin-RevId: 7cf81bf93076305d733ea658fae08e58c3e6f7f1
2022-10-06 18:46:07 +00:00
Sam Leffler
ea5b34e20b MemoryManager: workaround release build problem
Running test_obj_alloc and similar (e.g. install+start+stop+uninstall)
on a  release build yields kernel complaints about using a null
capability. Traced this down to delete_caps for ObjDesc's with count >
1. Checking the result from the syscall resolves things. Seems like an
optimizer bug.

Change-Id: Ib09095ac4255261b1ef67fec68ef3267d6be4883
GitOrigin-RevId: 7149d33b88d2b383d4ab65903c62c478e1659d1d
2022-10-06 18:45:52 +00:00
Sam Leffler
4f17bb33b3 MemoryManager: integrate with the slot allocator and many api updates
- track CAmkES well-known name changes (e.g. RECV_CNODE -> MEMORT_RECV_CNODE)
- initialize the slot allocator for the component
- fix size_bytes to use the object count in its calculation
- add can_combine method to help optimize ObjDescBundle construction
- enhance ObjDescBundle:
  - len, count, size_bytes, count_log2, maybe_combine_last
  - move_objects_to_toplevel, move_objects_from_toplevel
  - fmt (show SELF_CNODE & MEMORY_RECV_CNODE symbolically)
- add MmeCapAllocFailed
- add kata_object_alloc_in_toplevel & kata_object_free_toplevel
- add kata_object_alloc_in_cnode & kata_object_free_in_cnode
- add kata_frame_alloc_in_cnode
- avoid underflow in stats bookkeeping to not crash until we fix
  the root cause

Change-Id: I826b55f1f0b07bef58a420e32efbc02cd1d6363f
GitOrigin-RevId: 43bd8ae5cf41fd510fae502c7cd8e234c04edb85
2022-10-06 18:44:07 +00:00
Austin Appleby
10f0ace32e Apparently we still need resolver = 2 for the build to work. :/
Change-Id: I45df25651d19b28ea5335aea8b394ae17394e60f
GitOrigin-RevId: 6eb5747eed94fd2f34686edd46169677f46b644a
2022-10-06 18:42:44 +00:00
June Tate-Gans
57a4ee2f13 cargo: Bump to 2021 edition
Change-Id: Icb2deaff58da7f92970d8f57f7853349769323f5
GitOrigin-RevId: f8303df543a9d84c066b9a4a1bd1c0239eb75857
2022-10-06 18:42:30 +00:00
Sam Leffler
9188a29bd4 Move common crates to kata-os-common and shorten names.
- kata-allocator, kata-kata-logger, kata-panic, and kata-slot-allocator
  are now submodules of kata-os-common
- be more consistent on use stmts (and remove unneeded deps)
- cleanup various deps on kata-panic that no longer seem to be needed
  (likely due to using resolver=2)

Change-Id: I5d3f4b399e3be66c09c2f97c75d5e3053993ebdd
GitOrigin-RevId: 393e28fbb975959cba35388bab749b256cda0095
2022-10-06 18:42:22 +00:00
Sam Leffler
fdc3a88437 Impprove cbindgen integration.
- add Makefile's to re-generate CAmkES interface defs
- move cbindgen.toml files into the interface crate(s)

Note: still need to integrate running make with the build
Change-Id: I16b3e3b831b66ad1c976402e38df6be4a126d923
GitOrigin-RevId: 78b8b9024e4acb89590410a439f121fcb1fdc996
2022-10-06 18:42:15 +00:00
Sam Leffler
957f429c52 Mark KataOS components for special CAmkES processing.
Mark each component that depends on component.kataos.c features with a
"kataos" attribute.

Change-Id: I7e2e3f9010a7aa717d4fe3dfc2f088cbd1a66488
GitOrigin-RevId: 2967036cf4e70edc3f96d2ead45ecfe9e42fd7fa
2022-10-06 18:41:39 +00:00
Sam Leffler
18c7660244 Add MemoryManager service.
The MemoryManager service allocates & frees seL4 objects. Requests can be
batched. Capabilities to dynamically allocated objects are moved in CNode
containers attached to IPC requests.

Specific changes:
- Add new CAmkES MemoryManager component.
- Add api's for allocating & freeing singleton objects (e.g.
  kata_cnode_alloc) and batches of objects (kata_object_alloc &
  kata_object_free).
- Add support to kata-os-rootserver to hand-off UntypedMemory objects
  just before terminating. The objects are placed directly in the
  MemoryManager's top-level CNode and a BootInfo frame is constructed
  that describes where the objects are.
- Switch the rootserver to kata-os-rootserver as the C version lacks
  the UntypedMemory hand-off.
- Add test_bootinfo kata-shell command to dump the MemoryManager BootInfo
  frame contents (broken for now because it directlry references the
  shared page).
- Add test_obj_alloc kata-shell command that exercises the MemoryManager
  singleton and batch api's

While here, did some cleanup of arg handling in kata-shell.

TODO: top-level object allocations use a simplistic capability allocator
TODO: move test_bootinfo to the MemoryManager and add an interface rpc

Change-Id: I778b2d5fe7f2f9b65ee642ff905cf56d4b2b02fd
GitOrigin-RevId: 7fc72d1927bba165234955e68f8b9ad1b556f6fb
2022-10-06 18:41:15 +00:00