This CL enables multiple outstanding periodic executions. To do so the
MlCoordinator now includes an array of started models and a queue of
models that are ready to be executed immediately. Additionally, each
periodic model has an associated timer. When that timer fires the model
is added to the execution queue. When a model finishes executing, the
next model is popped off the queue and executed. If a model becomes
ready when there's already an execution for it queued, that execution
is dropped and a warning printed. A cancel command is added to remove
periodic or outstanding executions.
A state debug command was also added.
Currently we can only load a single model due to limitations with the
StorageManager, but we can do so multiple times.
Tests:
Single shot: https://paste.googleplex.com/6704629669691392
Two periodic execs: https://paste.googleplex.com/5288292800004096
Overloaded warning: https://paste.googleplex.com/4549962219126784
Debug State:
KATA> state_mlcoord
kata_ml_coordinator::Running model: fubar:mobilenet_v1_emitc_static.model
kata_ml_coordinator::Loaded model: fubar:mobilenet_v1_emitc_static.model
kata_ml_coordinator::Loadable Models:
kata_ml_coordinator:: LoadableModel { bundle_id: "fubar", model_id: "mobilenet_v1_emitc_static.model", rate_in_ms: Some(2000) }
kata_ml_coordinator:: LoadableModel { bundle_id: "fubar", model_id: "mobilenet_v1_emitc_static.model", rate_in_ms: Some(6000) }
kata_ml_coordinator::Execution Queue:
kata_ml_coordinator:: fubar:mobilenet_v1_emitc_static.model
kata_ml_coordinator::Statistics: Statistics { load_failures: 0, already_queued: 0 }
Change-Id: I7637c9c390eb6ffd9ae22088f37b98c056a441c2
GitOrigin-RevId: 18c0d3fe740a37381f7f1eddee8f2224f679fd61
Having CSpaceSlot be a submodule of kata-slot-allocator caused an sel-4sys
dependency that broke unit tests. Split it out to it's own crate.
Change-Id: Iea531e36284b58e932eb0cbd84f90abc81823fe3
GitOrigin-RevId: 39abb4510f66cfe026b12fcacf78b429ddd8f0b5
Running test_obj_alloc and similar (e.g. install+start+stop+uninstall)
on a release build yields kernel complaints about using a null
capability. Traced this down to delete_caps for ObjDesc's with count >
1. Checking the result from the syscall resolves things. Seems like an
optimizer bug.
Change-Id: Ib09095ac4255261b1ef67fec68ef3267d6be4883
GitOrigin-RevId: 7149d33b88d2b383d4ab65903c62c478e1659d1d
- replace the memory-mapped elf file by a cpio archive of bundle
objects (BundleImages until we fill in what a bundle is)
- add a new "builtins" command to list the contents of the cpio
archive (similar to cpio -t)
- extend the "install" command to load from the builtins archive
- switch the connection to ProcessManager to support the larger
ObjDescBundle's coming from the cpio archive
Change-Id: I5d7c195b58937df3921f925de3637f325f53fa2f
GitOrigin-RevId: 410813e62ae8f38685a1b32deb2e80de538085a4
- change zmodem uploads to get memory from the MemoryManager
- add a "-z" option to the "install" command to start a zmodem upload
to generate the package contents to send to ProcessManager
- increase CNode headroom for loading package contents
Change-Id: I5f329cdd044368e5568ad891245d67a4a13f8468
GitOrigin-RevId: 2853cde48cb8232f3ba75fe7e8efdbd3032bcb66
- remove vestiges of mapped ELF file; models now come from
SecurityCoordinator using a LoadModel request in the form of a BundleImage
- change execute api to take a bundle_id & model_id
- track loded bundle_id & model_id to handle switching between models
- load_elf -> load_image (which loads a BundleImage)
- add a client-side wrapper to hide marshalling details
- add a copyregion (VSpace hole) for processing BundleImages
- integrate with slot allocator
- setup the heap (needed for various things)
Change-Id: I50f3526bddefcdb67b742d42ca396bfefc10801f
GitOrigin-RevId: 80aa0da34b057b334b7d76932c7ad25146364569
- replace the opaque memory placeholder with MemoryManager integration
(e.g. ObjDescBundle's and BundleImage's)
- set CNode headroom so there is space to construct a 4MB application
- arrange to get capabilities for an ASIDPool, the global SchedController,
and the global DomainController for crafting new TCB's
- arrange for copregions (VSpace holes) for working with BundleImage's
and spilling arguments to the stack of a new TCB
- integrate with the slot allocator
- accept an ObjDescBundle for the pkg_contents of an install request
- add BundleImplInterface traits in suport of mocking/faking the bundle
implementation (unit tests not included)
- new error codes: ObjCapInvalid, SuspendFailed, ResumeFailed
- integrate with the kernel so TCB construction honors settings like MCS
- track kata_security_* wrapper additions
- add seL4BundleImpl (derived from kata-os-rootserver)
TODO: build+test only on RISCV32
TODO: verify seL4BundleImpl features all build at least
TODO: cleanup VSpace layout (is probably arch specific)
TODO: review+cleanup error handling (esp resources)
TODO: fault endpoints are not conencted
TODO: SDK runtime is not setup
TODO: scheduling is likely not the way we want
TODO: security eview
Change-Id: I025b008eeb8b47af2fe3894149da6576642bb8ed
GitOrigin-RevId: 539bf11739a9e756936168a9cbd3e2e2e4b0ba11
- break dependency loop with kata-security-interface
- use new kata_security_* wrappers
Change-Id: I65b98a406f18c82354e5425b37612789d4ab340d
GitOrigin-RevId: 5744715a439a5305ead57a99eacc1108b5d10750
This mostly cleans up my bad idea of how to process SecurityRequests.
- accept an ObjDescBundle attached in an InstallRequest
- return a BundleImage in LoadApplication & LoadNModel replies
- integrate with the slot allocator
- integrate with MemoryManager (for the fake)
Change-Id: I695efbecabfa3e71d7d2cfdd013c113a5a915b40
GitOrigin-RevId: 623ffdf19f5550918da530b57a299659061832aa
- hide marshalling details in kata_security_* wrappers
- replace the placeholder opaque memory representation in rpc's with
MemoryManager integration
- uses proper struct's for all Request/Response msgs (except for Echo):
this mostly eases the way for switching away from postcard + CAmkES
to comething like flatbuffers
- extend SecurityRequest structs with SecurityCapability traits to
support passing capabilities (specifically CNode's holding dynamically
allocated objects)
- new error codes: SreDeleteFirst, SreCapAllocFailed, SreCapMoveFailed,
and SreObjCapInvalid
- shuffle integration with StorageManager to resolve a dependency loop
- redo SecurityCoordinatorInterface to enable server-side changes (in
a separate cl)
- add headroom to the component CNode for the fake impl to hold package
contents
Change-Id: Ief6931d451023b511dbe1bdc3ffba4bf18070962
GitOrigin-RevId: 066f6cfa72f47fb8b4f8edc861dd9e3759ec6054
Add support for BundleImage, a file format for loading applications and
models from a bundle. BundleImage is simple, compact, and streamable,
BundleImage files are constructed with the prepare_bundle_image tool.
TODO: add compression
TODO: check crc32
Change-Id: I0770608a075cac9754a54e0bb244d75673ae1be6
GitOrigin-RevId: 368dabd3a5af19d47fe7f8084b8a0a0b57b8471d
- track CAmkES well-known name changes (e.g. RECV_CNODE -> MEMORT_RECV_CNODE)
- initialize the slot allocator for the component
- fix size_bytes to use the object count in its calculation
- add can_combine method to help optimize ObjDescBundle construction
- enhance ObjDescBundle:
- len, count, size_bytes, count_log2, maybe_combine_last
- move_objects_to_toplevel, move_objects_from_toplevel
- fmt (show SELF_CNODE & MEMORY_RECV_CNODE symbolically)
- add MmeCapAllocFailed
- add kata_object_alloc_in_toplevel & kata_object_free_toplevel
- add kata_object_alloc_in_cnode & kata_object_free_in_cnode
- add kata_frame_alloc_in_cnode
- avoid underflow in stats bookkeeping to not crash until we fix
the root cause
Change-Id: I826b55f1f0b07bef58a420e32efbc02cd1d6363f
GitOrigin-RevId: 43bd8ae5cf41fd510fae502c7cd8e234c04edb85
This fixes debug builds which otherwise have an undefined reference
to SELF_CNODE.
Change-Id: I55384cfff19b99cf49b147d396b274c74bdf5db5
GitOrigin-RevId: d055b070ee5a67f1648566d1c1c511446d02b3cc
The slot allocator has CAmkES-specific support that will not build for
a non-CAmkES target. Add a "camkes_support" feature flag that controls
whether the slot allocator is included. This is enabled by default but
allows users like kata-os-rootserver to workaround the problem.
Change-Id: I2decb9fc794c215455e223d17c24c6a19cc52d56
GitOrigin-RevId: d7a3811b4a9a000b0f34fbc2abdfc628250f64d4
- kata-allocator, kata-kata-logger, kata-panic, and kata-slot-allocator
are now submodules of kata-os-common
- be more consistent on use stmts (and remove unneeded deps)
- cleanup various deps on kata-panic that no longer seem to be needed
(likely due to using resolver=2)
Change-Id: I5d3f4b399e3be66c09c2f97c75d5e3053993ebdd
GitOrigin-RevId: 393e28fbb975959cba35388bab749b256cda0095
- add Makefile's to re-generate CAmkES interface defs
- move cbindgen.toml files into the interface crate(s)
Note: still need to integrate running make with the build
Change-Id: I16b3e3b831b66ad1c976402e38df6be4a126d923
GitOrigin-RevId: 78b8b9024e4acb89590410a439f121fcb1fdc996
Mark each component that depends on component.kataos.c features with a
"kataos" attribute.
Change-Id: I7e2e3f9010a7aa717d4fe3dfc2f088cbd1a66488
GitOrigin-RevId: 2967036cf4e70edc3f96d2ead45ecfe9e42fd7fa
When calculating the landing spot for handing off the UntypedMemory
objects we need to take into account empty slots. Calculate the max
occupied slot # and add one. This works so long as empty slots have
not been allocated (by CAmkES) at the end of the CNode--which should
be safe since we control the MemoryManager CAmkES specification.
Change-Id: I5bfb75bd7134254c4475a91653d54e2aeb126721
GitOrigin-RevId: cf14850894ad21be97ea094f977c3420edf322ab
The MemoryManager service allocates & frees seL4 objects. Requests can be
batched. Capabilities to dynamically allocated objects are moved in CNode
containers attached to IPC requests.
Specific changes:
- Add new CAmkES MemoryManager component.
- Add api's for allocating & freeing singleton objects (e.g.
kata_cnode_alloc) and batches of objects (kata_object_alloc &
kata_object_free).
- Add support to kata-os-rootserver to hand-off UntypedMemory objects
just before terminating. The objects are placed directly in the
MemoryManager's top-level CNode and a BootInfo frame is constructed
that describes where the objects are.
- Switch the rootserver to kata-os-rootserver as the C version lacks
the UntypedMemory hand-off.
- Add test_bootinfo kata-shell command to dump the MemoryManager BootInfo
frame contents (broken for now because it directlry references the
shared page).
- Add test_obj_alloc kata-shell command that exercises the MemoryManager
singleton and batch api's
While here, did some cleanup of arg handling in kata-shell.
TODO: top-level object allocations use a simplistic capability allocator
TODO: move test_bootinfo to the MemoryManager and add an interface rpc
Change-Id: I778b2d5fe7f2f9b65ee642ff905cf56d4b2b02fd
GitOrigin-RevId: 7fc72d1927bba165234955e68f8b9ad1b556f6fb
