From d24179550ec0e02e9ff98a94307ba2dcc0a8b775 Mon Sep 17 00:00:00 2001 From: Geoffroy Van Cutsem Date: Fri, 29 Jun 2018 18:01:59 +0200 Subject: [PATCH] Documentation: move Trusty documentation to doc/ Move the existing Trusty document to the doc/ folder (where it belongs) and convert the text to ReST. The Documentation/ folder under hypervisor/ is removed as all documents should be put under doc/. All technical information has been preserved or was already available in other documents. Signed-off-by: Geoffroy Van Cutsem --- doc/acrn.doxyfile | 1 + doc/developer-guides/index.rst | 1 + doc/developer-guides/trusty.rst | 163 ++++++++++++++++++++++ doc/developer-guides/trusty/boot-flow.dot | 10 ++ hypervisor/Documentation/ACRN_UEFI.txt | 13 -- hypervisor/Documentation/Trusty.txt | 157 --------------------- hypervisor/arch/x86/trusty.c | 25 ++++ hypervisor/include/common/hypercall.h | 33 ++++- 8 files changed, 228 insertions(+), 175 deletions(-) create mode 100644 doc/developer-guides/trusty.rst create mode 100644 doc/developer-guides/trusty/boot-flow.dot delete mode 100644 hypervisor/Documentation/ACRN_UEFI.txt delete mode 100644 hypervisor/Documentation/Trusty.txt diff --git a/doc/acrn.doxyfile b/doc/acrn.doxyfile index 2c44681e3..19ad47d41 100644 --- a/doc/acrn.doxyfile +++ b/doc/acrn.doxyfile @@ -794,6 +794,7 @@ INPUT = custom-doxygen/mainpage.md \ ../hypervisor/include/common/hypercall.h \ ../hypervisor/include/public/acrn_common.h \ ../hypervisor/include/public/acrn_hv_defs.h \ + ../hypervisor/arch/x86/trusty.c \ ../devicemodel/include/virtio.h diff --git a/doc/developer-guides/index.rst b/doc/developer-guides/index.rst index e1ad2c29f..e272fe971 100644 --- a/doc/developer-guides/index.rst +++ b/doc/developer-guides/index.rst @@ -9,6 +9,7 @@ Developer Guides primer.rst ../api/index.rst ../reference/kconfig/index.rst + trusty.rst Contributing to the project diff --git a/doc/developer-guides/trusty.rst b/doc/developer-guides/trusty.rst new file mode 100644 index 000000000..f8dbb6db5 --- /dev/null +++ b/doc/developer-guides/trusty.rst @@ -0,0 +1,163 @@ +.. _trusty_tee: + +Trusty TEE on ACRN +################## + +Introduction +************ + +`Trusty`_ is a set of software components supporting a Trusted Execution Environment (TEE). +TEE is commonly known as an isolated processing environment in which applications can be +securely executed irrespective of the rest of the system. For more information about TEE, +please visit the `Trusted Execution Environment wikipage `_. +Trusty consists of: + +1. An operating system (the Trusty OS) that runs on a processor intended to provide a TEE +#. Drivers for the Android kernel (Linux) to facilitate communication with applications + running under the Trusty OS +#. A set of libraries for Android/Linux systems software to facilitate communication with + trusted applications executed within the Trusty OS using the kernel drivers + +LK (`Little Kernel`_) is a tiny operating system suited for small embedded devices, bootloaders, +and other environments where OS primitives such as threads, mutexes, and timers are needed, but +there's a desire to keep things small and lightweight. LK has been chosen as the Trusty OS kernel. + +Trusty Architecture +******************* + +Trusty Architectural diagram:: + + +---------------------------+ + |VMn | + | ...... | + +------------+ +---------------------------+ | + |VM0 | |VM1 | | + | | | +--------+ +--------+ | | + | | | | | | | | | + | SOS | | | Normal | | Secure | | | + | | | | World | | World | |-+ + | | | | | | | | + | | | +--------+ +--------+ | + +------------+ +---------------------------+ + +-------------------------------------------+ + | ACRN Hypervisor | + +-------------------------------------------+ + +-------------------------------------------+ + | HW | + +-------------------------------------------+ + +.. note:: + Trusty OS is running in Secure World in the architecture drawing above. + +Trusty specific Hypercalls +************************** + +There are a few :ref:`hypercall_apis` that are related to Trusty. + +.. doxygengroup:: trusty_hypercall + :project: Project ACRN + :content-only: + +Trusty Boot flow +**************** + +By design, the User OS bootloader (``UOS_Loader``) will trigger the Trusty boot process. The complete bootflow is illustrated below. + +.. graphviz:: trusty/boot-flow.dot + :name: trusty-boot-flow + :align: center + :caption: Trusty Boot Flow + +As shown in the above figure, here are some details about the Trusty boot flow processing: + +1. UOS_Loader + + a. Load and verify Trusty image from virtual disk + #. Allocate runtime memory for trusty + #. Do ELF relocation of trusty image and get entry address + #. Call ``hcall_initialize_trusty`` with trusty memory base and entry address +#. ACRN (``hcall_initialize_trusty``) + + a. Save World context for Normal World + #. Init World context for Secure World (RIP, RSP, EPT, etc.) + #. Resume to Secure World +#. Trusty + + a. Booting up + #. Call ``hcall_world_switch`` to switch back to Normal World if boot completed +#. ACRN (``hcall_world_switch``) + + a. Save World context for the World which caused this ``vmexit`` (Secure World) + #. Restore World context for next World (Normal World (UOS_Loader)) + #. Resume to next World (UOS_Loader) +#. UOS_Loader + + a. Continue to boot + +EPT Hierarchy +************* + +As per the Trusty design, Trusty can access Normal World's memory, but Normal World cannot +access Secure World's memory. Hence it means Secure World EPTP page table hierarchy +must contain normal world GPA address space, while Trusty world's GPA address space +must be removed from the Normal world EPTP page table hierarchy. + +Design +====== + +Put Secure World's GPA to very high position: 511 GB - 512 GB. The PML4/PDPT for Trusty +World are separated from Normal World. PD/PT for low memory (< 511 GB) are shared in +both Trusty World's EPT and Normal World's EPT. PD/PT for high memory (>= 511 GB) are +valid for Trusty World's EPT only. + +Benefit +======= + +This design will benefit the EPT changes of Normal World. There are requirement to +modify Normal World's EPT during runtime such as increasing memory, changing attributes, +etc. If such behavior happened, only PD and PT for Normal World need to +be updated. + +:: + + ABSTRACT EPT hierarchy for 2 Worlds: + ==================================================================== ================================================== + : Normal World : : Secure World : + : PML4 : : PML4 : + : +--------+ : : +--------+ : + : | | : : | | : + : | | : : PD | | : + : | | : : +-------+ | | : + : | | : : | | | | : + : | 0-512G |--+ : : | | +--| 0-512G | : + :EPTP -->+--------+ | : : | | | +--------+<-- EPTP : + : | PDPT : : | | PDPT | : + : | +--------+ : : | | +--------+ | : + : | | >=511G |---> Not present : : +-------+<--| >=511G | | : + : | |________| : : |________| | : + : | | | : : | | | : + : | | <511G |->+<----------------------------:--------:--------------| <511G | | : + : | | | | : : | | | : + : +-->+--------+ | PD PT : : +--------+<-+ : + : | ... ... : ================================================== + : | +-------+ +-------+ : + : | +-------+| +-------+| : + : | | || | || : + : | | || | || : + : | | PDE |--+ | || : + : | | || | | || : + : | | |+ | | |+ : + : +-->+-------+ +-->+-------+ : + : : + ==================================================================== + + +API +=== + +.. doxygengroup:: trusty_apis + :project: Project ACRN + :content-only: + +.. _Trusty: https://source.android.com/security/trusty/ +.. _Little Kernel: https://github.com/littlekernel/lk diff --git a/doc/developer-guides/trusty/boot-flow.dot b/doc/developer-guides/trusty/boot-flow.dot new file mode 100644 index 000000000..828bb1050 --- /dev/null +++ b/doc/developer-guides/trusty/boot-flow.dot @@ -0,0 +1,10 @@ +digraph G { + rankdir=LR; + rank=same; + bgcolor="transparent"; + uosl1 [label="UOS_Loader"] + acrn_init [shape=box style="rounded,filled" label="ACRN"] + acrn_switch [shape=box style="rounded,filled" label="ACRN"] + uosl2 [label="UOS_Loader"] + uosl1 -> acrn_init -> "Trusty" -> acrn_switch -> uosl2; +} diff --git a/hypervisor/Documentation/ACRN_UEFI.txt b/hypervisor/Documentation/ACRN_UEFI.txt deleted file mode 100644 index 8f230bfbd..000000000 --- a/hypervisor/Documentation/ACRN_UEFI.txt +++ /dev/null @@ -1,13 +0,0 @@ -******************* -Usage of acrn.efi -******************* - -Suggest to follow the following step: -a. Build out the acrn.efi image with command "make PLATFORM=uefi". -b. Rename the original os loader file named bootloaderx64.efi that is located at \\EFI\\org.clearlinux\\ to bootloaderx64_origin.efi -which name can be user defined in the config file bsp/uefi/include/bsp/bsp_cfg.h, the default name is "EFI\\org.clearlinux\\bootloaderx64_origin.efi". -c. Copy acrn.efi to \\EFI\\org.clearlinux\\, and rename it to bootloaderx64.efi. -d. Update the bsp/uefi/clearlinux/acrn.conf file by filling the field with the your native rootfs partition uuid. -e. Copy bsp/uefi/clearlinux/acrn.conf to loader\\entries\\ directory. -f. Reboot. -g. If you want to recover to original os loader, just replace the bootloaderx64.efi with bootloaderx64_origin.efi. diff --git a/hypervisor/Documentation/Trusty.txt b/hypervisor/Documentation/Trusty.txt deleted file mode 100644 index 38b8a85e5..000000000 --- a/hypervisor/Documentation/Trusty.txt +++ /dev/null @@ -1,157 +0,0 @@ - Trusty on ACRN -Overview -Trusty Architecture -Trusty specific Hypercalls -Trusty Boot flow -EPT Hierarchy - -******** -Overview -******** -Trusty is a set of software components supporting a Trusted Execution Environment (TEE). -Trusty consists of: - 1. An operating system (the Trusty OS) that runs on a processor intended to provide a TEE - 2. Drivers for the Android kernel (Linux) to facilitate communication with applications - running under the Trusty OS - 3. A set of libraries for Android/Linux systems software to facilitate communication with - trusted applications executed within the Trusty OS using the kernel drivers - -LK (Little Kernel) is a tiny operating system suited for small embedded devices, bootloaders, -and other environments where OS primitives like threads, mutexes, and timers are needed, but -there's a desire to keep things small and lightweight. LK has been chosen as the Trusty OS kernel. - - -******************* -Trusty Architecture -******************* - +---------------------------+ - |VMn | - | ...... | -+------------+ +---------------------------+ | -|VM0 | |VM1 | | -| | | +--------+ +--------+ | | -| | | | | | | | | -| SOS | | | Normal | | Secure | | | -| | | | World | | World | |-+ -| | | | | | | | -| | | +--------+ +--------+ | -+------------+ +---------------------------+ -+-------------------------------------------+ -| ACRN Hypervisor | -+-------------------------------------------+ -+-------------------------------------------+ -| HW | -+-------------------------------------------+ - -Note: Trusty OS is running in Secure World in the architecture above. - - -************************** -Trusty specific Hypercalls -************************** -1. HC_INITIALIZE_TRUSTY - ->This Hypercall is used by UOS_Loader to request ACRN to initialize Trusty. - ->The Trusty memory region range, entry point must be specified. - ->Hypervisor needs to save current vCPU contexts (Normal World). -2. HC_WORLD_SWITCH - ->Simulate ARM SMC (Secure Monitor Call, or SMC) instruction to do world switch. - ->Hypervisor needs to save current world vCPU contexts, and load next world vCPU contexts. - ->Update rdi, rsi, rdx, rbx to next world vCPU contexts. - -API ---- -1. hcall_initialize_trusty(vm_t *vm); -2. hcall_world_switch(vm_t *vm); - - -**************** -Trusty Boot flow -**************** -Per design, UOSloader will trigger boot of Trusty. So the boot flow will be: - UOSloader --> ACRN --> Trusty --> ACRN --> UOSloader - -Detail: -1. UOS_Loader - 1.1 load and verify trusty image from virtual disk. - 1.2 allocate runtime memory for trusty. - 1.3 do ELF relocation of trusty image and get entry address. - 1.4 call HC_INITIALIZE_TRUSTY with trusty memory base and entry address. -2. ACRN(HC_INITIALIZE_TRUSTY) - 2.1 save World context for Normal World. - 2.2 init World context for Secure World(RIP, RSP, EPT, etc.). - 2.3 resume to Secure World. -3. Trusty - 3.1 booting up - 3.2 call HC_WORLD_SWITCH to switch back to Normal World if boot completed. -4. ACRN(HC_WORLD_SWITCH) - 4.1 save World context for the World which caused this vmexit(Secure World) - 4.2 restore World context for next World(Normal World(UOS_Loader)) - 4.3 resume to next World(UOS_Loader) -5. UOS_Loader - 5.1 continue to boot. - - -************* -EPT Hierarchy -************* -Per Trusty design, Trusty can access Normal World's memory, but Normal World cannot -access Secure World's memory. Hence it means Secure World EPTP page table hierarchy -must contain normal world GPA address space, while Trusty world's GPA address space -must be removed from the Normal world EPTP page table hierarchy. - -Design: -Put Secure World's GPA to very high position: 511G-512G. The PML4/PDPT for Trusty -World are separated from Normal World. PD/PT for low memory (<511G) are shared in -both Trusty World's EPT and Normal World's EPT. PD/PT for high memory (>=511G) are -valid for Trusty World's EPT only. - -Benefit: -This design will benefit the EPT changes of Normal World. There are requirement to -modify Normal World's EPT during runtime such as memory increasing, attribute -change, etc. If such behavior happened, only PD and PT for Normal World need to -be updated. - -ABSTRACT EPT hierarchy for 2 Worlds: -==================================================================== ================================================== -: Normal World : : Secure World : -: PML4 : : PML4 : -: +--------+ : : +--------+ : -: | | : : | | : -: | | : : PD | | : -: | | : : +-------+ | | : -: | | : : | | | | : -: | 0-512G |--+ : : | | +--| 0-512G | : -:EPTP -->+--------+ | : : | | | +--------+<-- EPTP : -: | PDPT : : | | PDPT | : -: | +--------+ : : | | +--------+ | : -: | | >=511G |---> Not present : : +-------+<--| >=511G | | : -: | |________| : : |________| | : -: | | | : : | | | : -: | | <511G |->+<----------------------------:--------:--------------| <511G | | : -: | | | | : : | | | : -: +-->+--------+ | PD PT : : +--------+<-+ : -: | ... ... : ================================================== -: | +-------+ +-------+ : -: | +-------+| +-------+| : -: | | || | || : -: | | || | || : -: | | PDE |--+ | || : -: | | || | | || : -: | | |+ | | |+ : -: +-->+-------+ +-->+-------+ : -: : -==================================================================== - -API ----- -/* -Create Secure World EPT hierarchy, construct new PML4/PDPT, reuse PD/PT parse from -vm->arch_vm->ept - -Parameters: - vm: VM with 2 Worlds - gpa: original gpa allocated from vSBL - size: LK size(16M by default) - rebase_offset: rebase the gpa to offset xxx(511G_OFFSET) -*/ -int create_secure_world_ept(vm_t *vm, uint64_t gpa, uint64_t size, uint64_t rebase_offset) diff --git a/hypervisor/arch/x86/trusty.c b/hypervisor/arch/x86/trusty.c index 3b5df7c80..303a26de7 100644 --- a/hypervisor/arch/x86/trusty.c +++ b/hypervisor/arch/x86/trusty.c @@ -49,6 +49,27 @@ static struct key_info g_key_info = { exec_vmwrite(VMX_GUEST_##SEG_NAME##_ATTR, seg.attr); \ } +/** + * @defgroup trusty_apis Trusty APIs + * + * This is a special group that includes all APIs + * related to Trusty + * + * @{ + */ + +/** + * @brief Create Secure World EPT hierarchy + * + * Create Secure World EPT hierarchy, construct new PML4/PDPT, reuse PD/PT parse from + * vm->arch_vm->ept + * + * @param vm pointer to a VM with 2 Worlds + * @param gpa_orig original gpa allocated from vSBL + * @param size LK size (16M by default) + * @param gpa_rebased gpa rebased to offset xxx (511G_OFFSET) + * + */ static void create_secure_world_ept(struct vm *vm, uint64_t gpa_orig, uint64_t size, uint64_t gpa_rebased) { @@ -464,3 +485,7 @@ void trusty_set_dseed(void *dseed, uint8_t dseed_num) memcpy_s(&g_key_info.dseed_list, sizeof(struct seed_info) * dseed_num, dseed, sizeof(struct seed_info) * dseed_num); } + +/** + * @} + */ // End of trusty_apis diff --git a/hypervisor/include/common/hypercall.h b/hypervisor/include/common/hypercall.h index 5f24bb9c5..3a57fcc8c 100644 --- a/hypervisor/include/common/hypercall.h +++ b/hypervisor/include/common/hypercall.h @@ -322,9 +322,23 @@ int64_t hcall_setup_sbuf(struct vm *vm, uint64_t param); int64_t hcall_get_cpu_pm_state(struct vm *vm, uint64_t cmd, uint64_t param); +/** + * @defgroup trusty_hypercall Trusty Hypercalls + * + * This is a special group that includes all hypercalls + * related to Trusty + * + * @{ + */ /** - * @brief Switch VCPU state between Normal/Secure World. + * @brief Switch vCPU state between Normal/Secure World. + * + * * Hypervisor uses the Secure Monitor Code (SMC) instruction to do + * the world switch + * * The hypervisor needs to save current world vCPU contexts, and load + * the next world vCPU contexts + * * Updates ``rdi``, ``rsi``, ``rdx``, ``rbx`` to next world vCPU contexts * * @param vcpu Pointer to VCPU data structure * @@ -334,11 +348,16 @@ int64_t hcall_get_cpu_pm_state(struct vm *vm, uint64_t cmd, uint64_t param); int64_t hcall_world_switch(struct vcpu *vcpu); /** - * @brief Initialize environment for Trusty-OS on a VCPU. + * @brief Initialize environment for Trusty-OS on a vCPU. * - * @param vcpu Pointer to VCPU data structure + * * It is used by the User OS bootloader (``UOS_Loader``) to request ACRN + * to initialize Trusty + * * The Trusty memory region range, entry point must be specified + * * The hypervisor needs to save current vCPU contexts (Normal World) + * + * @param vcpu Pointer to vCPU data structure * @param param guest physical address. This gpa points to - * struct trusty_boot_param + * trusty_boot_param structure * * @return 0 on success, non-zero on error. */ @@ -346,6 +365,10 @@ int64_t hcall_initialize_trusty(struct vcpu *vcpu, uint64_t param); /** * @} - */ + */ // End of trusty_hypercall + +/** + * @} + */ // End of acrn_hypercall #endif /* HYPERCALL_H*/