diff --git a/doc/developer-guides/hld/hld-hypervisor.rst b/doc/developer-guides/hld/hld-hypervisor.rst index cd4c8e1c3..57b343d21 100644 --- a/doc/developer-guides/hld/hld-hypervisor.rst +++ b/doc/developer-guides/hld/hld-hypervisor.rst @@ -8,6 +8,7 @@ Hypervisor high-level design :maxdepth: 1 hv-startup + hv-vm-management hv-cpu-virt Memory management I/O Emulation diff --git a/doc/developer-guides/hld/hld-vm-management.rst b/doc/developer-guides/hld/hld-vm-management.rst deleted file mode 100644 index 9d5b96ccb..000000000 --- a/doc/developer-guides/hld/hld-vm-management.rst +++ /dev/null @@ -1,161 +0,0 @@ -.. _hld-vm-management: - -VM Management high-level design -############################### - -Management of a Virtual Machine (VM) means to switch a VM to the right -state, according to the requirements of applications or system power -operations. - -VM state -******** - -Generally, a VM is not running at the beginning: it is in a 'stopped' -state. After its UOS is launched successfully, the VM enter a 'running' -state. When the UOS powers off, the VM returns to a 'stopped' state again. -A UOS can sleep when it is running, so there is also a 'paused' state. - -Because VMs are designed to work under an SOS environment, a VM can -only run and change its state when the SOS is running. A VM must be put to -'paused' or 'stopped' state before the SOS can sleep or power-off. -Otherwise the VM may be damaged and user data would be lost. - -Scenarios of VM state change -**************************** - -Button-initiated System Power On -================================ - -When the user presses the power button to power on the system, -everything is started at the beginning. VMs that run user applications -are launched automatically after the SOS is ready. - -Button-initiated VM Power on -============================ - -At SOS boot up, SOS-Life-Cycle-Service and Acrnd are automatically started -as system services. SOS-Life-Cycle-Service notifies Acrnd that SOS is -started, then Acrnd starts an Acrn-DM for launching each UOS, whose state -changes from 'stopped' to 'running'. - -Button-initiated VM Power off -============================= - -When SOS is about to shutdown, IOC powers off all VMs. -SOS-Life-Cycle-Service delays the SOS shutdown operation using heartbeat, -and waits for Acrnd to notify it can shutdown. - -Acrnd keeps query states of all VMs. When all of them are 'stopped', -it notifies SOS-Life-Cycle-Service. SOS-Life-Cycle-Service stops the send delay -shutdown heartbeat, allowing SOS to continue the shutdown process. - -RTC S3/S5 entry -=============== - -UOS asks Acrnd to resume/restart itself later by sending an RTC timer request, -and suspends/powers-off. SOS suspends/powers-off before that RTC -timer expires. Acrnd stores the RTC resume/restart time to a file, and -send the RTC timer request to SOS-Life-Cycle-Service. -SOS-Life-Cycle-Service sets the RTC timer to IOC. Finally, the SOS is -suspended/powered-off. - -RTC S3/S5 exiting -================= - -SOS is resumed/started by IOC RTC timer. SOS-Life-Cycle-Service notifies -Acrnd SOS has become alive again. Acrnd checks that the wakeup reason -was because SOS is resumed/started by IOC RTC. It then reads UOS -resume/restart time from the file, and resumes/restarts the UOS when -time is expired. - -VM State management -******************* - -Overview of VM State Management -=============================== - -Management of VMs on SOS uses the -SOS-Life-Cycle-Service, Acrnd, and Acrn-dm, working together and using -Acrn-Manager-AIP as IPC interface. - -* The Lifecycle-Service get the Wakeup-Reason from IOC controller. It can set - different power cycle method, and RTC timer, by sending a heartbeat to IOC - with proper data. - -* The Acrnd get Wakeup Reason from Lifecycle-Service and forwards it to - Acrn-dm. It coordinates the lifecycle of VMs and SOS and handles IOC-timed - wakeup/poweron. - -* Acrn-Dm is the device model of a VM running on SOS. Virtual IOC - inside Acrn-DM is responsible to control VM power state, usually triggered by Acrnd. - -SOS Life Cycle Service -====================== - -SOS-Life-Cycle-Service (SOS-LCS) is a daemon service running on SOS. - -SOS-LCS listens on ``/dev/cbc-lifecycle`` tty port to receive "wakeup -reason" information from IOC controller. SOS-LCS keeps reading system -status from IOC, to discover which power cycle method IOC is -doing. SOS-LCS should reply a heartbeat to IOC. This heartbeat can tell -IOC to keep doing this power cycle method, or change to another power -cycle method. SOS-LCS heartbeat can also set RTC timer to IOC. - -SOS-LCS handles SHUTDOWN, SUSPEND, and REBOOT acrn-manager messages -request from Acrnd. When these messages are received, SOS-LCS switches IOC -power cycle method to shutdown, suspend, and reboot, respectively. - -SOS-LCS handles WAKEUP_REASON acrn-manager messages request from Acrnd. -When it receives this message, SOS-LCS sends "wakeup reason" to Acrnd. - -SOS-LCS handles RTC_TIMER acrn-manager messages request from Acrnd. -When it receives this message, SOS-LCS setup IOC RTC timer for Acrnd. - -SOS-LCS notifies Acrnd at the moment system becomes alive from other -status. - -Acrnd -===== - -Acrnd is a daemon service running on SOS. - -Acrnd can start/resume VMs and query VM states for SOS-LCS, helping -SOS-LCS to decide which power cycle method is right. It also helps UOS -to be started/resumed by timer, required by S3/S5 feature. - -Acrnd forwards wakeup reason to acrn-dm. Acrnd is responsible to retrieve -wakeup reason from SOS-LCS service and attach the wakeup reason to -acrn-dm parameter for ioc-dm. - -When SOS is about to suspend/shutdown, SOS lifecycle service will send a -request to Acrnd to guarantee all guest VMs are suspended or shutdown -before SOS suspending/shutdown process continue. On receiving the -request, Acrnd starts polling the guest VMs state, and notifies SOS -lifecycle service when all guest VMs are put in proper state gracefully. - -Guest UOS may need to -resume/start in a future time for some tasks. To -setup a timed resume/start, ioc-dm will send a request to acrnd to -maintain a list of timed requests from guest VMs. acrnd selects the -nearest request and sends it to SOS lifecycle service who will setup the -physical IOC. - -Acrn-DM -======= - -Acrn-Dm is the device model of VM running on SOS. Dm-IOC inside Acrn-DM -operates virtual IOC to control VM power state, and collects VM power -state information. Acrn-DM Monitor abstracts these Virtual IOC -functions into monitor-vm-ops, and allows Acrnd to use them via -Acrn-Manager IPC helper functions. - -Acrn-manager IPC helper -======================= - -SOS-LCS, Acrnd, and Acrn-DM use sockets to do IPC. Acrn-Manager IPC helper API -makes socket transparent for them. These are: - -- int mngr_open_un() - create a descriptor for vm management IPC -- void mngr_close() - close descriptor and release the resources -- int mngr_add_handler() - add a handler for message specified by message -- int mngr_send_msg() - send a message and wait for acknowledgement diff --git a/doc/developer-guides/hld/hv-vm-management.rst b/doc/developer-guides/hld/hv-vm-management.rst new file mode 100644 index 000000000..2aa392170 --- /dev/null +++ b/doc/developer-guides/hld/hv-vm-management.rst @@ -0,0 +1,63 @@ +.. _hv-vm-management: + +VM Management +############# + +The ACRN hypervisor maintained a dedicated VM structure instance for each +running VM, and a series VM APIs like create_vm, start_vm, reset_vm, shutdown_vm +etc are used to switch a VM to the right state, according to the requirements of +applications or system power operations. + +VM structure +************ + +The ``acrn_vm`` structure is defined to manage a VM instance, this structure +maintained a VM's HW resources like vcpu, vpic, vioapic, vuart, vpci. And at +the same time ``acrn_vm`` structure also recorded a bunch of SW information +related with corresponding VM, like info for VM indentifier, info for SW +loader, info for memory e820 entries, info for IO/MMIO handlers, info for +platform level cpuid entries, and so on. + +The ``acrn_vm`` structure instance will be created by create_vm API, and then +work as the first parameter for other VM APIs. + +VM state +******** + +Generally, a VM is not running at the beginning: it is in a 'powered off' +state. After it got created successfully, the VM enter a 'created' state. +Then the VM could be kick to run, and enter a 'started' state. When the +VM powers off, the VM returns to a 'powered off' state again. +A VM can be paused to wait some operation when it is running, so there is +also a 'paused' state. + +:numref:`hvvm-state` illustrates the state-machine of a VM state transition, +please refer to :ref:`hv-cpu-virt` for related VCPU state. + +.. figure:: images/hld-image108.png + :align: center + :name: hvvm-state + + Hypervisor VM State Transition + +VM State Management +******************* + +Pre-launched and Service VM +=========================== + +The hypervisor is the owner to control pre-launched and Service VM's state +by calling VM APIs directly, and it follows the design of system power +management. Please refer to ACRN power management design for more details. + + +Post-launched User VMs +====================== + +DM is taking control of post-launched User VMs' state transition after SOS +boot up, and it calls VM APIs through hypercalls. + +SOS user level service like Life-Cycle-Service and tool like Acrnd may work +together with DM to launch or stop a User VM. Please refer to ACRN tool +introduction for more details. + diff --git a/doc/developer-guides/hld/images/hld-image108.png b/doc/developer-guides/hld/images/hld-image108.png new file mode 100644 index 000000000..4276aea06 Binary files /dev/null and b/doc/developer-guides/hld/images/hld-image108.png differ