misc: upgrade dependency for acrn-configurator

upgrade crossbeam-channel to 0.5.15 which is used by tauri's component `tao`

Tracked-On: #8769
Signed-off-by: DongpingX Wu <dongpingx.wu@intel.com>

.gitignore

@@ -31,6 +31,7 @@ debian/acrn-lifemngr
 debian/acrn-system
 debian/acrn-tools
 debian/acrnd
+debian/changelog
 debian/debhelper-build-stamp
 debian/files
 debian/grub-acrn

CODEOWNERS

@@ -13,9 +13,9 @@
 *                                        @dongyaozu @NanlinXie
 
 Makefile                                 @terryzouhao @NanlinXie
-/hypervisor/                             @dongyaozu @lifeix
-/devicemodel/                            @ywan170
-/doc/                                    @dbkinder @NanlinXie
+/hypervisor/                             @dongyaozu @lifeix @junjiemao1
+/devicemodel/                            @ywan170 @chejianj
+/doc/                                    @NanlinXie
 /misc/debug_tools/acrn_crashlog/         @ywan170 @lifeix
 /misc/debug_tools/acrn_log/              @ywan170 @lifeix
 /misc/debug_tools/acrn_trace/            @ywan170 @lifeix
@@ -27,4 +27,4 @@ Makefile                                 @terryzouhao @NanlinXie
 /misc/packaging/                         @terryzouhao @NanlinXie
 /misc/hv_prebuild/                       @terryzouhao @NanlinXie
 
-*.rst                                    @dbkinder @NanlinXie
+*.rst                                    @NanlinXie

CONTRIBUTING.rst

@@ -48,7 +48,7 @@ the TSC and its membership, are described in the project's `technical-charter`_.
 
 These are the current TSC voting members and chair person:
 
-- Junjie Mao (chair): junjie.mao@intel.com
+- Yu Wang (chair): yu1.wang@intel.com
 - Helmut Buchsbaum: helmut.buchsbaum@tttech-industrial.com
 - Thomas Gleixner: thomas.gleixner@intel.com
 

VERSION

@@ -1,3 +1,3 @@
 MAJOR_VERSION=3
-MINOR_VERSION=2
+MINOR_VERSION=4
 EXTRA_VERSION=-unstable

debian/changelog

@@ -1,53 +0,0 @@
-acrn-hypervisor (3.0~3.gbp9074bb) UNRELEASED; urgency=medium
-
-  ** SNAPSHOT build @9074bb42223cf0a18a01ca7f5b481834ba79fc6a **
-
-  * Initial native packaging release for ACRN.
-  * d/rules: Update regarding config detection
-  * d/control: Pin elementpath and xmlschema
-  * debian: Add docker build helpers
-  * debian: Create local apt repo after build
-  * debian: Update and rename README.md
-  * debian: Add nuc7i7dnh/shared+initrd
-  * debian: Add kontron-COMe-mAL10/shared+initrd
-  * debian: Update changelog for UNRELEASED package build
-  * debian: Preinstall build requirements in docker helper
-  * debian: Silence build user creation
-  * life_mngr: Makefile: Use sysconfdir
-  * life_mngr: Makefile: Fix install path creation of CONF_DIR
-  * life_mngr: Makefile: Fix permission of life_mngr.conf
-  * debian: Fixup ACRN lifemngr package build
-  * debian/rules: Store board and scenario xml
-  * debian/grub: Adapt Linux commandline to ACRN scenario settings
-  * debian/grub: Remove override variable support
-  * debian: Use original acrnd.service
-  * acrn_log: Fix log path to comply with Linux FSSTD
-  * doc: Adapt documentation to change of acrnlog log path
-  * debian: Install acrnprobe.xml
-  * debian: linitian suppression rule update
-  * debian/control: Use compat level 12
-  * debian/rules: override_dh_strip: Fix wrong parameter
-  * debian: Trigger grub-acrn on acrn-hypervisor install
-  * debian/rules: Do not start services on install
-  * debian: acrn-tools: Add helper scripts
-  * debian: acrn-hypervisor: Refactor debconf
-  * debian: nuc7i7dnh/shared+initrd: Add  boot parameters
-  * debian/configs: Add simatic-ipc227g/shared+initrd
-  * debian/rules: Add simatic-ipc227g
-  * debian: Add default configuration for acrnlog
-  * debian/configs: Add kontron-COMe-mAL10:shared
-  * debian/configs: Add nuc7i7dnh:shared
-  * debian/configs: Add simatic-ipc227g:shared
-  * debian: Separate build config items to acrn-hypervisor.conf.mk
-  * debian/rules: Generate launch scripts
-  * debian: Switch to elementpath 2.5.0-1 and xmlschema 1.10.0-1
-  * debian/docker: Add source package handling for local apt repository
-  * debian: Add build and installation description
-  * debian: Update README.md
-  * debian: Update changelog for UNRELEASED package build
-  * debian: Convert README.md to README.rst
-  * debian: Convert INSTALL.md to INSTALL.rst
-  * debian/acrn-doc.doc-base: Fix typo
-  * debian/configs: Remove proprietary configs, just provide a hook directory
-
- -- Helmut Buchsbaum <helmut.buchsbaum@opensource.tttech-industrial.com>  Mon, 09 May 2022 14:00:35 +0200

debian/debian_build.sh

@@ -8,6 +8,7 @@ usage() {
   echo "Usage: $0 [--board_list ACRN_BOARDLIST] [--scenario_list ACRN_SCENARIOLIST] [--config_path CONFIGDIRS] [--release n|y] [acrn | board_inspector | clean]"
   echo "Optional arguments:"
   echo "  -h, --help           show this help message and exit"
+  echo "  -v, --verbose        show verbose output"
   echo "  -b, --board_list     list the boards to build, seperated by blank; build all scanned boards in the config path if specified as \"\"; build the default boards in debian rules if not specified"
   echo "  -s, --scenario_list  list the scenarios to build, seperated by blank; build all scanned scenarios in the config path if specified as \"\"; build the default scenarios in debian rules if not specified"
   echo "  -c, --config_path    specify the config path for the board and scenario configuration files, default use misc/config_tools/data if not specified"
@@ -59,6 +60,10 @@ while [[ $# -gt 0 ]]; do
       release="$2"
       shift 2
       ;;
+    -v|--verbose)
+      verbose=1
+      shift
+      ;;
     -h|--help)
       usage
       exit 0
@@ -76,6 +81,9 @@ done
 set -- "${POSITIONAL_ARGS[@]}"
 
 cmd="debuild"
+if [ -n "$verbose" ]; then
+  cmd="$cmd -eDH_VERBOSE=1"
+fi
 if [ "$board_list" != "default" ]; then
   echo "ACRN_BOARDLIST    = ${board_list@Q}"
   cmd="$cmd -eACRN_BOARDLIST=${board_list@Q}"

debian/grub/20_acrn-board-inspector

@@ -103,7 +103,7 @@ linux_entry ()
   if [ -z "$boot_device_id" ]; then
       boot_device_id="$(grub_get_device_id "${GRUB_DEVICE}")"
   fi
-  echo "menuentry '$(echo "$os" | grub_quote)' ${CLASS} \$menuentry_id_option 'gnulinux-acrn-board-inspector-$boot_device_id' {"
+  echo "menuentry '$(echo "$os, with Linux ${version}" | grub_quote)' ${CLASS} \$menuentry_id_option 'gnulinux-acrn-board-inspector-$boot_device_id' {"
 
   # Use ELILO's generic "efifb" when it's known to be available.
   # FIXME: We need an interface to select vesafb in case efifb can't be used.
@@ -181,7 +181,7 @@ prepare_root_cache=
 boot_device_id=
 
 while [ "x$list" != "x" ] ; do
-  linux=`version_find_latest $list`
+  linux=$(echo ${list} | tr ' ' '\n' | sed -e 's/\.old$/ 1/; / 1$/! s/$/ 2/' | LC_ALL=C sort -V -r | sed -e 's/ 1$/.old/; s/ 2$//' | head -n 1)
   case $linux in
     *.efi.signed)
       # We handle these in linux_entry.

debian/grub/25_linux_acrn

@@ -208,7 +208,7 @@ linux_entry ()
       boot_device_id="$(grub_get_device_id "${GRUB_DEVICE}")"
   fi
   title="$(gettext_printf "%s with ACRN hypervisor" "${os}")"
-  echo "menuentry '$(echo "$title" | grub_quote)' ${CLASS} \$menuentry_id_option 'acrn-gnulinux-$boot_device_id' {"
+  echo "menuentry '$(echo "$title, with Linux ${version} (ACRN ${acrn_version})" | grub_quote)' ${CLASS} \$menuentry_id_option 'acrn-gnulinux-$boot_device_id' {"
 
   if [ -z "${prepare_boot_cache}" ]; then
     prepare_boot_cache="$(prepare_grub_to_access_device ${GRUB_DEVICE_BOOT} | grub_add_tab)"
@@ -311,7 +311,7 @@ boot_device_id=
 acrn_first_entry=
 
 while [ "x${acrn_list}" != "x" ] ; do
-    current_acrn=$(version_find_latest $acrn_list)
+    current_acrn=$(echo ${acrn_list} | tr ' ' '\n' | sed -e 's/\.old$/ 1/; / 1$/! s/$/ 2/' | LC_ALL=C sort -V -r | sed -e 's/ 1$/.old/; s/ 2$//' | head -n 1)
     acrn_basename=$(basename ${current_acrn})
     acrn_dirname=$(dirname ${current_acrn})
     rel_acrn_dirname=$(make_system_path_relative_to_its_root $acrn_dirname)
@@ -327,7 +327,7 @@ while [ "x${acrn_list}" != "x" ] ; do
         else
             title="$(gettext_printf "%s with ACRN hypervisor %s" "${OS}" "${acrn_version}")"
         fi
-        echo "menuentry '$(echo "$title" | grub_quote)' ${CLASS} \$menuentry_id_option 'acrn-gnulinux-partitioned-${acrn_version}' {"
+        echo "menuentry '$(echo "$title, with Linux ${version} (ACRN ${acrn_version})" | grub_quote)' ${CLASS} \$menuentry_id_option 'acrn-gnulinux-partitioned-${acrn_version}' {"
         message="$(gettext_printf "Loading ACRN hypervisor %s ..." ${acrn_version})"
         cat << EOF
 	echo	'$(echo "$message" | grub_quote)'
@@ -342,7 +342,7 @@ EOF
 
     # only if we have at least one ACRN capable kernel and a Service VM entry defined
     while [ "x$list" != "x" ] && [ "x${ACRN_SERVICE_VM_ID}" != "x" ] ; do
-        linux=$(version_find_latest $list)
+        linux=$(echo ${list} | tr ' ' '\n' | sed -e 's/\.old$/ 1/; / 1$/! s/$/ 2/' | LC_ALL=C sort -V -r | sed -e 's/ 1$/.old/; s/ 2$//' | head -n 1)
         gettext_printf "Found ACRN linux image: %s\n" "$linux" >&2
         basename=$(basename $linux)
         dirname=$(dirname $linux)

debian/rules

@@ -32,8 +32,8 @@ rwildcard=$(foreach d,$(wildcard $(1:=/*)),$(call rwildcard,$d,$2) $(filter $(su
 unquote = $(subst $\",,$1)
 
 # set these variables to define build of certain boards/scenarios, e.g.
-ACRN_BOARDLIST ?= whl-ipc-i5 nuc11tnbi5 cfl-k700-i7 tgl-vecow-spc-7100-Corei7
-ACRN_SCENARIOLIST ?= partitioned shared hybrid hybrid_rt
+ACRN_BOARDLIST ?=
+ACRN_SCENARIOLIST ?=
 
 # for now build the debug versions
 # set to y for RELEASE build
@@ -62,7 +62,7 @@ $(eval $(call unquote,$(shell xmllint --xpath '/acrn-config/@board' $1 2>/dev/nu
 $(eval $(if $(board),                                                                     \
     $(eval config_$(board) := $1)                                                         \
     $(eval boardlist := $(sort $(boardlist) $(board)))                                    \
-    $(foreach f,$(wildcard $(addprefix $(dir $1),*)),                                     \
+    $(foreach f,$(wildcard $(addprefix $(dir $1),*.xml)),                                 \
         $(if $(strip $(shell xmllint --xpath '/acrn-config/@board' $f 2>/dev/null)),,     \
             $(if $(subst scenario.xml,,$(notdir $f)),                                     \
                 $(eval scenario = $(basename $(notdir $f))),                              \

devicemodel/Makefile

@@ -80,6 +80,7 @@ LDFLAGS += -L$(TOOLS_OUT)/services
 LIBS = -lrt
 LIBS += -lpthread
 LIBS += -lcrypto
+LIBS += -luring
 LIBS += -lpciaccess
 LIBS += -lusb-1.0
 LIBS += -lacrn-mngr
@@ -187,6 +188,8 @@ SRCS += core/cmd_monitor/socket.c
 SRCS += core/cmd_monitor/command.c
 SRCS += core/cmd_monitor/command_handler.c
 SRCS += core/cmd_monitor/cmd_monitor.c
+SRCS += core/sbuf.c
+SRCS += core/vm_event.c
 
 # arch
 SRCS += arch/x86/pm.c

devicemodel/arch/x86/pm.c

@@ -42,6 +42,7 @@
 #include "lpc.h"
 #include "monitor.h"
 #include "log.h"
+#include "vm_event.h"
 
 static pthread_mutex_t pm_lock = PTHREAD_MUTEX_INITIALIZER;
 static struct mevent *power_button;
@@ -192,7 +193,7 @@ void
 pm_backto_wakeup(struct vmctx *ctx)
 {
 	/* According to ACPI 5.0 Table 4-16: bit 15, WAK_STS should be
-	 * set when system trasition to the working state
+	 * set when system transition to the working state
 	 */
 	pm1_status |= PM1_WAK_STS;
 }
@@ -242,6 +243,14 @@ power_button_handler(int signal, enum ev_type type, void *arg)
 		inject_power_button_event(arg);
 }
 
+static void
+send_poweroff_event(void)
+{
+	struct vm_event event;
+	event.type = VM_EVENT_POWEROFF;
+	dm_send_vm_event(&event);
+}
+
 static int
 pm1_control_handler(struct vmctx *ctx, int vcpu, int in, int port, int bytes,
 		    uint32_t *eax, void *arg)
@@ -265,6 +274,7 @@ pm1_control_handler(struct vmctx *ctx, int vcpu, int in, int port, int bytes,
 		 */
 		if (*eax & VIRTUAL_PM1A_SLP_EN) {
 			if ((pm1_control & VIRTUAL_PM1A_SLP_TYP) >> 10 == 5) {
+				send_poweroff_event();
 				vm_suspend(ctx, VM_SUSPEND_POWEROFF);
 			}
 

devicemodel/bios/MD5SUM_ovmf

@@ -1,5 +1,5 @@
-3102701dfdaeab846577e139398af5b8  OVMF.fd
-75f5f310da41cd7bd1dd96ddcbcff8cf  OVMF_debug.fd
-dca1ed6265062454a9c445d6e6e33d05  OVMF_CODE.fd
-c2e1d7a8e1e4036a1d3f203dfc66bac0  OVMF_CODE_debug.fd
+16c698cb5e6354f514e307eb016237dd  OVMF_CODE_debug.fd
+e4b15beab40a0482bbe821e151c96d75  OVMF_CODE.fd
+0166b812b014a4c1809d22199823132a  OVMF_debug.fd
+bf8d9d3c85ba06ac940ee9d6d6f9f811  OVMF.fd
 aa9703e68b787f1a391bcbf201c84e02  OVMF_VARS.fd

devicemodel/bios/SHA512SUM_ovmf

@@ -1,5 +1,5 @@
-6a29eba62832a4e94aec79fb726cfd507b019e5c9039e0abf2efa023fd21373f911e8e7ba0ca1e20c495409b738d513ac9ac7912c05ef7db7159569af3cb3da8  OVMF.fd
-91bdb40d49a994a4b45e25144917fbedefa0710d9d95aedf633197ab3cd7d6ca9c3a075bf120c15667f863840f48612297bd48275080a31d84217da076f2b629  OVMF_debug.fd
-ffbce1003d166751f842c34cdde6ae67df02a27f4388c58707d8ec3ca6da692cdeb6bae9b5eae8a8f0c55709be1852bfc84af8a2c6a8ab8e45fb0b84c3c383fd  OVMF_CODE.fd
-24e927b075be8071efcb5ca988537bd6593205bee5a5750e6036370a08add296d4e4966e2e326700a096167d6a8a55e0078936ae3dedd3492c9d46cc9bb0ac4a  OVMF_CODE_debug.fd
+ed00f44b07d375d036b67d9b50b878c9c5b1bee59082b08f9e0126c06dd91914f6e9e2dc248fb1bc61350c377da284fa4150ffd2028a4daa34a6c3217047e32e  OVMF_CODE_debug.fd
+a958f4f5dbe8ad98565e3ac4883fcac40a67bbdd184b2e3d7c1a7974181628c20498159dec64596879af12e0710e8e70748c27337060aab7da164a83b0124f08  OVMF_CODE.fd
+05e4996dd3d7a1e83aca8176ea0da4a92ea704116ea21f9b2df59f386b8bdbdb9e697731a111eaa31d994c66b66b732c105b979e4c5a7e8b6f7960145ebdf898  OVMF_debug.fd
+4647bfe6a4bc0430e0c751d51d06665b5500eeaf0170826abbc6f38f94c19946a6683ce4fc2f9c2e09753546e2f4f30d65643d5b511ecd28a9e73a362db133cc  OVMF.fd
 6c5f0629cda9a950dd76ea4162c3e7d3483615fa6e9918da365900dbdcb84681192df70c4629237f0db19d0a27fbc574bb3d9f38a522246e5b91356cd5e5a1e5  OVMF_VARS.fd

devicemodel/bios/changelog_ovmf.txt

@@ -1,3 +1,12 @@
+OVMF release v3.3
+
+- OvmfPkg: resolve AcrnS3Lib
+- OvmfPkg: add AcrnS3Lib to support S3
+- OvmfPkg: introduce AcrnS3Lib class
+- OVMF:ACRN:PCI: Try to load ROM image for the PCI device with PCI_ROM
+- OVMF:ACRN:PCI: Add LoadOpRomImageLight to Load the PCI Rom
+- OVMF:ACRN:PCI: Write back the original value of PCI ROM
+
 OVMF release v3.0
 
 - VGA interface of virtio-gpu adapter support

devicemodel/core/cmd_monitor/cmd_monitor.c

@@ -99,6 +99,7 @@ static void register_socket_message_handlers(struct vmctx *ctx)
 	arg.ctx_arg = ctx;
 	register_command_handler(user_vm_destroy_handler, &arg, DESTROY);
 	register_command_handler(user_vm_blkrescan_handler, &arg, BLKRESCAN);
+	register_command_handler(user_vm_register_vm_event_client_handler, &arg, REGISTER_VM_EVENT_CLIENT);
 }
 
 int init_cmd_monitor(struct vmctx *ctx)

devicemodel/core/cmd_monitor/command.c

@@ -26,6 +26,7 @@
 #define CMD_OBJS \
 	GEN_CMD_OBJ(DESTROY), \
 	GEN_CMD_OBJ(BLKRESCAN), \
+	GEN_CMD_OBJ(REGISTER_VM_EVENT_CLIENT), \
 
 struct command dm_command_list[CMDS_NUM] = {CMD_OBJS};
 

devicemodel/core/cmd_monitor/command.h

@@ -10,8 +10,9 @@
 
 #define DESTROY "destroy"
 #define BLKRESCAN "blkrescan"
+#define REGISTER_VM_EVENT_CLIENT "register_vm_event_client"
 
-#define CMDS_NUM 2U
+#define CMDS_NUM 3U
 #define CMD_NAME_MAX 32U
 #define CMD_ARG_MAX 320U
 

devicemodel/core/cmd_monitor/command_handler.c

@@ -67,6 +67,78 @@ static int send_socket_ack(struct socket_dev *sock, int fd, bool normal)
 	return ret;
 }
 
+static struct socket_client *vm_event_client = NULL;
+static pthread_mutex_t vm_event_client_mutex = PTHREAD_MUTEX_INITIALIZER;
+
+static void vm_event_free_cb(struct socket_client *self)
+{
+	vm_event_client = NULL;
+}
+
+static int set_vm_event_client(struct socket_client *client)
+{
+	if (vm_event_client != NULL) {
+		pr_err("vm event client already registerred.\n");
+		return -1;
+	} else {
+		vm_event_client = client;
+		client->per_client_mutex = &vm_event_client_mutex;
+		client->free_client_cb = vm_event_free_cb;
+		return 0;
+	}
+}
+
+int vm_monitor_send_vm_event(const char *msg)
+{
+	int ret = -1;
+	struct socket_client *client;
+	pthread_mutex_t *per_client_mutex = &vm_event_client_mutex;
+
+	pthread_mutex_lock(per_client_mutex);
+	client = vm_event_client;
+	if (msg == NULL || client == NULL) {
+		pthread_mutex_unlock(per_client_mutex);
+		return -1;
+	}
+	memset(client->buf, 0, CLIENT_BUF_LEN);
+	memcpy(client->buf, msg, strlen(msg));
+	client->len = strlen(msg);
+	ret = write_socket_char(client);
+	pthread_mutex_unlock(per_client_mutex);
+	return ret;
+}
+
+/* When a client issues the REGISTER_VM_EVENT_CLIENT command,
+ * this handler will register that client as this VM's only vm_event receiver,
+ * and keeps the socket connection. Then vm events will be sent to
+ * the client through this connection.
+ */
+int user_vm_register_vm_event_client_handler(void *arg, void *command_para)
+{
+	int ret;
+	struct command_parameters *cmd_para = (struct command_parameters *)command_para;
+	struct handler_args *hdl_arg = (struct handler_args *)arg;
+	struct socket_dev *sock = (struct socket_dev *)hdl_arg->channel_arg;
+	struct socket_client *client = NULL;
+	bool cmd_completed = false;
+
+	client = find_socket_client(sock, cmd_para->fd);
+	if (client == NULL)
+		return -1;
+
+	if (set_vm_event_client(client) == 0) {
+		cmd_completed = true;
+	}
+
+	pr_dbg("%s: client with fd %d registerred\n", __func__, client->fd);
+
+	ret = send_socket_ack(sock, cmd_para->fd, cmd_completed);
+	if (ret < 0) {
+		pr_err("%s: Failed to send ACK message by socket.\n", __func__);
+	}
+	return ret;
+}
+
 int user_vm_destroy_handler(void *arg, void *command_para)
 {
 	int ret;

devicemodel/core/cmd_monitor/command_handler.h

@@ -9,4 +9,6 @@ extern struct socket_dev *sock_server;
 
 int user_vm_destroy_handler(void *arg, void *command_para);
 int user_vm_blkrescan_handler(void *arg, void *command_para);
+int user_vm_register_vm_event_client_handler(void *arg, void *command_para);
+
 #endif

devicemodel/core/cmd_monitor/socket.c

@@ -47,13 +47,23 @@ err:
 }
 static void free_socket_client(struct socket_dev *sock, struct socket_client *client)
 {
+	pthread_mutex_t *per_client_mutex = client->per_client_mutex;
 	pthread_mutex_lock(&sock->client_mtx);
 	LIST_REMOVE(client, list);
 	pthread_mutex_unlock(&sock->client_mtx);
 
+	if (per_client_mutex) {
+		pthread_mutex_lock(per_client_mutex);
+	}
+	if (client->free_client_cb) {
+		client->free_client_cb(client);
+	}
 	close(client->fd);
 	client->fd = -1;
 	free(client);
+	if (per_client_mutex) {
+		pthread_mutex_unlock(per_client_mutex);
+	}
 }
 
 int write_socket_char(struct socket_client *client)
@@ -142,7 +152,8 @@ static struct socket_client *new_socket_client(struct socket_dev *sock)
 					__func__);
 		goto alloc_client;
 	}
-
+	/* If per client mutex is needed, init in callback */
+	client->per_client_mutex = NULL;
 	client->addr_len = sizeof(client->addr);
 	client->fd =
 	    accept(sock->sock_fd, (struct sockaddr *)&client->addr,
@@ -153,7 +164,6 @@ static struct socket_client *new_socket_client(struct socket_dev *sock)
 					__func__, sock->sock_fd, strerror(errno));
 		goto accept_con;
 	}
-
 	pthread_mutex_lock(&sock->client_mtx);
 	LIST_INSERT_HEAD(&sock->client_head, client, list);
 	pthread_mutex_unlock(&sock->client_mtx);

devicemodel/core/cmd_monitor/socket.h

@@ -21,7 +21,11 @@ struct socket_client {
 	socklen_t addr_len;
 	char buf[CLIENT_BUF_LEN];
 	int len; /* buf len */
-
+	/* When a client is registered as vm_event receiver, we need this per_client_mutex
+	 * to make sure it is safe to free the client when client disconnects.
+	 */
+	pthread_mutex_t *per_client_mutex;
+	void (*free_client_cb)(struct socket_client *self);
 	LIST_ENTRY(socket_client) list;
 };
 

devicemodel/core/hugetlb.c

@@ -446,7 +446,7 @@ static bool release_larger_freepage(int level_limit)
 		if (hugetlb_priv[level].pages_delta >= 0)
 			continue;
 
-		/* free one unsed larger page */
+		/* free one un-used larger page */
 		orig_pages = read_sys_info(hugetlb_priv[level].nr_pages_path);
 		total_pages = orig_pages - 1;
 		snprintf(cmd_buf, MAX_PATH_LEN, "echo %d > %s",
@@ -483,7 +483,7 @@ static bool release_larger_freepage(int level_limit)
  *.D.enough higher level free pages, but not enough free memory for
  *    lower level gap pages, so release some higher level free pages for that.
  * other info:
- *.   even enough free memory, it is eaiser to reserve smaller pages than
+ *.   even enough free memory, it is easier to reserve smaller pages than
  * lager ones, for example:2MB easier than 1GB. One flow of current solution:
  *.it could leave Service VM very small free memory.
  *.return value: true: success; false: failure
@@ -507,8 +507,8 @@ static bool hugetlb_reserve_pages(void)
 
 		/* probably system allocates fewer pages than needed
 		 * especially for larger page like 1GB, even there is enough
-		 * free memory, it stil can fail to allocate 1GB huge page.
-		 * so if that,it needs the next level to handle it.
+		 * free memory, it still can fail to allocate 1GB huge page.
+		 * so if that, it needs the next level to handle it.
 		 */
 		if (level > HUGETLB_LV1) {
 			left_gap = hugetlb_priv[level].pages_delta;
@@ -555,8 +555,13 @@ bool init_hugetlb(void)
 			path[i] = 0;
 			if (access(path, F_OK) != 0) {
 				if (mkdir(path, 0755) < 0) {
-					pr_err("mkdir %s failed.\n", path);
-					return -1;
+					/* We might have multiple acrn-dm instances booting VMs at
+					 * the same time
+					 */
+					if (errno != EEXIST) {
+						pr_err("mkdir %s failed: %s\n", path, errormsg(errno));
+						return false;
+					}
 				}
 			}
 			path[i] = '/';
@@ -901,7 +906,7 @@ bool vm_allow_dmabuf(struct vmctx *ctx)
 	}
 
 	if (ctx->lowmem) {
-		/* Check the lowhmem is used by HUGETLB_LV1/HUGETLB_LV2 */
+		/* Check the lowmem is used by HUGETLB_LV1/HUGETLB_LV2 */
 		mem_flags = 0;
 		if ((hugetlb_priv[HUGETLB_LV1].fd > 0) &&
 			(hugetlb_priv[HUGETLB_LV1].lowmem))

devicemodel/core/iothread.c

@@ -14,46 +14,45 @@
 #include <sys/queue.h>
 #include <pthread.h>
 #include <signal.h>
+#include <string.h>
 
 #include "iothread.h"
 #include "log.h"
 #include "mevent.h"
+#include "dm.h"
 
 
 #define MEVENT_MAX 64
-#define MAX_EVENT_NUM 64
-struct iothread_ctx {
-	pthread_t tid;
-	int epfd;
-	bool started;
-	pthread_mutex_t mtx;
-};
-static struct iothread_ctx ioctx;
+
+static struct iothread_ctx ioctxes[IOTHREAD_NUM];
+static int ioctx_active_cnt;
+/* mutex to protect the free ioctx slot allocation */
+static pthread_mutex_t ioctxes_mutex = PTHREAD_MUTEX_INITIALIZER;
 
 static void *
 io_thread(void *arg)
 {
 	struct epoll_event eventlist[MEVENT_MAX];
 	struct iothread_mevent *aevp;
-	int i, n, status;
-	char buf[MAX_EVENT_NUM];
+	int i, n;
+	struct iothread_ctx *ioctx_x = (struct iothread_ctx *)arg;
 
-	while(ioctx.started) {
-		n = epoll_wait(ioctx.epfd, eventlist, MEVENT_MAX, -1);
+	set_thread_priority(PRIO_IOTHREAD, true);
+
+	while(ioctx_x->started) {
+		n = epoll_wait(ioctx_x->epfd, eventlist, MEVENT_MAX, -1);
 		if (n < 0) {
-			if (errno == EINTR)
-				pr_info("%s: exit from epoll_wait\n", __func__);
-			else
+			if (errno == EINTR) {
+				/* EINTR may happen when io_uring fd is monitored, it is harmless. */
+				continue;
+			} else {
 				pr_err("%s: return from epoll wait with errno %d\r\n", __func__, errno);
-			break;
+				break;
+			}
 		}
 		for (i = 0; i < n; i++) {
 			aevp = eventlist[i].data.ptr;
 			if (aevp && aevp->run) {
-				/* Mitigate the epoll_wait repeat cycles by reading out the events as more as possile.*/
-				do {
-					status = read(aevp->fd, buf, sizeof(buf));
-				} while (status == MAX_EVENT_NUM);
 				(*aevp->run)(aevp->arg);
 			}
 		}
@@ -63,36 +62,54 @@ io_thread(void *arg)
 }
 
 static int
-iothread_start(void)
+iothread_start(struct iothread_ctx *ioctx_x)
 {
-	pthread_mutex_lock(&ioctx.mtx);
+	int ret;
 
-	if (ioctx.started) {
-		pthread_mutex_unlock(&ioctx.mtx);
+	pthread_mutex_lock(&ioctx_x->mtx);
+
+	if (ioctx_x->started) {
+		pthread_mutex_unlock(&ioctx_x->mtx);
 		return 0;
 	}
 
-	if (pthread_create(&ioctx.tid, NULL, io_thread, NULL) != 0) {
-		pthread_mutex_unlock(&ioctx.mtx);
+	if (pthread_create(&ioctx_x->tid, NULL, io_thread, ioctx_x) != 0) {
+		pthread_mutex_unlock(&ioctx_x->mtx);
 		pr_err("%s", "iothread create failed\r\n");
 		return -1;
 	}
-	ioctx.started = true;
-	pthread_setname_np(ioctx.tid, "iothread");
-	pthread_mutex_unlock(&ioctx.mtx);
-	pr_info("iothread started\n");
+
+	ioctx_x->started = true;
+	pthread_setname_np(ioctx_x->tid, ioctx_x->name);
+
+	if (CPU_COUNT(&(ioctx_x->cpuset)) != 0) {
+		ret = pthread_setaffinity_np(ioctx_x->tid, sizeof(cpuset_t), &(ioctx_x->cpuset));
+		if (ret != 0) {
+			pr_err("pthread_setaffinity_np fails %d \n", ret);
+		}
+	}
+
+	pthread_mutex_unlock(&ioctx_x->mtx);
+	pr_info("%s started\n", ioctx_x->name);
+
 	return 0;
 }
 
 int
-iothread_add(int fd, struct iothread_mevent *aevt)
+iothread_add(struct iothread_ctx *ioctx_x, int fd, struct iothread_mevent *aevt)
 {
 	struct epoll_event ee;
 	int ret;
+
+	if (ioctx_x == NULL) {
+		pr_err("%s: ioctx_x is NULL \n", __func__);
+		return -1;
+	}
+
 	/* Create a epoll instance before the first fd is added.*/
 	ee.events = EPOLLIN;
 	ee.data.ptr = aevt;
-	ret = epoll_ctl(ioctx.epfd, EPOLL_CTL_ADD, fd, &ee);
+	ret = epoll_ctl(ioctx_x->epfd, EPOLL_CTL_ADD, fd, &ee);
 	if (ret < 0) {
 		pr_err("%s: failed to add fd, error is %d\n",
 			__func__, errno);
@@ -100,7 +117,7 @@ iothread_add(int fd, struct iothread_mevent *aevt)
 	}
 
 	/* Start the iothread after the first fd is added.*/
-	ret = iothread_start();
+	ret = iothread_start(ioctx_x);
 	if (ret < 0) {
 		pr_err("%s: failed to start iothread thread\n",
 			__func__);
@@ -109,12 +126,17 @@ iothread_add(int fd, struct iothread_mevent *aevt)
 }
 
 int
-iothread_del(int fd)
+iothread_del(struct iothread_ctx *ioctx_x, int fd)
 {
 	int ret = 0;
 
-	if (ioctx.epfd) {
-		ret = epoll_ctl(ioctx.epfd, EPOLL_CTL_DEL, fd, NULL);
+	if (ioctx_x == NULL) {
+		pr_err("%s: ioctx_x is NULL \n", __func__);
+		return -1;
+	}
+
+	if (ioctx_x->epfd) {
+		ret = epoll_ctl(ioctx_x->epfd, EPOLL_CTL_DEL, fd, NULL);
 		if (ret < 0)
 			pr_err("%s: failed to delete fd from epoll fd, error is %d\n",
 				__func__, errno);
@@ -126,40 +148,215 @@ void
 iothread_deinit(void)
 {
 	void *jval;
+	int i;
+	struct iothread_ctx *ioctx_x;
 
-	if (ioctx.tid > 0) {
-		pthread_mutex_lock(&ioctx.mtx);
-		ioctx.started = false;
-		pthread_mutex_unlock(&ioctx.mtx);
-		pthread_kill(ioctx.tid, SIGCONT);
-		pthread_join(ioctx.tid, &jval);
+	pthread_mutex_lock(&ioctxes_mutex);
+	for (i = 0; i < ioctx_active_cnt; i++) {
+		ioctx_x = &ioctxes[i];
+
+		if (ioctx_x->tid > 0) {
+			pthread_mutex_lock(&ioctx_x->mtx);
+			ioctx_x->started = false;
+			pthread_mutex_unlock(&ioctx_x->mtx);
+			pthread_kill(ioctx_x->tid, SIGCONT);
+			pthread_join(ioctx_x->tid, &jval);
+		}
+		if (ioctx_x->epfd > 0) {
+			close(ioctx_x->epfd);
+			ioctx_x->epfd = -1;
+		}
+		pthread_mutex_destroy(&ioctx_x->mtx);
+		pr_info("%s stop \n", ioctx_x->name);
 	}
-	if (ioctx.epfd > 0) {
-		close(ioctx.epfd);
-		ioctx.epfd = -1;
-	}
-	pthread_mutex_destroy(&ioctx.mtx);
-	pr_info("iothread stop\n");
+	ioctx_active_cnt = 0;
+	pthread_mutex_unlock(&ioctxes_mutex);
 }
 
-int
-iothread_init(void)
+/*
+ * Create @ioctx_num iothread context instances
+ * Return NULL if fails. Otherwise, return the base of those iothread context instances.
+ *
+ * Notes:
+ * The caller of iothread_create() shall call iothread_free_options() afterwards to free the resources that
+ * are dynamically allocated during iothread_parse_options(), such as iothr_opt->cpusets.
+ *
+ * A general calling sequence from the virtual device owner is like:
+ * 1. Call iothread_parse_options() to parse the options from the user.
+ * 2. Call iothread_create() to create the iothread instances.
+ * 3. Call iothread_free_options() to free the dynamic resources.
+ */
+struct iothread_ctx *
+iothread_create(struct iothreads_option *iothr_opt)
 {
 	pthread_mutexattr_t attr;
+	int i, ret, base, end;
+	struct iothread_ctx *ioctx_x;
+	struct iothread_ctx *ioctx_base = NULL;
+	ret = 0;
 
-	pthread_mutexattr_init(&attr);
-	pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE);
-	pthread_mutex_init(&ioctx.mtx, &attr);
-	pthread_mutexattr_destroy(&attr);
-
-	ioctx.tid = 0;
-	ioctx.started = false;
-	ioctx.epfd = epoll_create1(0);
-
-	if (ioctx.epfd < 0) {
-		pr_err("%s: failed to create epoll fd, error is %d\r\n",
-			__func__, errno);
-		return -1;
+	if (iothr_opt == NULL) {
+		pr_err("%s: iothr_opt is NULL \n", __func__);
+		return ioctx_base;
 	}
+
+	pthread_mutex_lock(&ioctxes_mutex);
+	base = ioctx_active_cnt;
+	end = base + iothr_opt->num;
+
+	if (end > IOTHREAD_NUM) {
+		ret = -1;
+		pr_err("%s: fails to create new iothread context, max number of instances is %d \n",
+			__func__, IOTHREAD_NUM);
+	} else {
+		for (i = base; i < end; i++) {
+			ioctx_x = &ioctxes[i];
+
+			pthread_mutexattr_init(&attr);
+			pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE);
+			pthread_mutex_init(&(ioctx_x->mtx), &attr);
+			pthread_mutexattr_destroy(&attr);
+
+			ioctx_x->idx = i;
+			ioctx_x->tid = 0;
+			ioctx_x->started = false;
+			ioctx_x->epfd = epoll_create1(0);
+
+			CPU_ZERO(&(ioctx_x->cpuset));
+			if (iothr_opt->cpusets != NULL) {
+				memcpy(&(ioctx_x->cpuset), iothr_opt->cpusets + (i - base), sizeof(cpu_set_t));
+			}
+
+			if (snprintf(ioctx_x->name, PTHREAD_NAME_MAX_LEN,
+				"iothr-%d-%s", ioctx_x->idx, iothr_opt->tag) >= PTHREAD_NAME_MAX_LEN) {
+				pr_err("%s: iothread name too long \n", __func__);
+			}
+
+			if (ioctx_x->epfd < 0) {
+				ret = -1;
+				pr_err("%s: failed to create epoll fd, error is %d\r\n",
+					__func__, errno);
+				break;
+			}
+		}
+		if (ret == 0) {
+			ioctx_base = &ioctxes[base];
+			ioctx_active_cnt = end;
+		}
+	}
+	pthread_mutex_unlock(&ioctxes_mutex);
+
+	return ioctx_base;
+}
+
+/*
+ * Parse the iothread options from @str and fill the options in @iothr_opt if successes.
+ * Return -1 if fails to parse. Otherwise, return 0.
+ */
+int
+iothread_parse_options(char *str, struct iothreads_option *iothr_opt)
+{
+	char *tmp_num = NULL;
+	char *tmp_cpusets = NULL;
+	char *tmp_cpux = NULL;
+	int service_vm_cpuid, iothread_sub_idx, num;
+	cpu_set_t *cpuset_list = NULL;
+
+	/*
+	 * Create one iothread instance if DM parameters contain 'iothread', but the number is not specified.
+	 */
+	num = 1;
+
+	/*
+	 * Valid 'iothread' setting examples:
+	 * - create 1 iothread instance for virtio-blk
+	 *   ... virtio-blk iothread,...
+	 *
+	 * - create 1 iothread instance for virtio-blk
+	 *   ... virtio-blk iothread=1,...
+	 *
+	 * - create 3 iothread instances for virtio-blk
+	 *   ... virtio-blk iothread=3,...
+	 *
+	 * - create 3 iothread instances for virtio-blk with CPU affinity settings
+	 *   ... virtio-blk iothread=3@0:1:2/0:1,...
+	 *   CPU affinity of iothread instances for this virtio-blk device:
+	 *   - 1st iothread instance <-> Service VM CPU 0,1,2
+	 *   - 2nd iothread instance <-> Service VM CPU 0,1
+	 *   - 3rd iothread instance <-> No CPU affinity settings
+	 *
+	 */
+	if (str != NULL) {
+		/*
+		 * "@" is used to separate the following two settings:
+		 * - the number of iothread instances
+		 * - the CPU affinity settings for each iothread instance.
+		 */
+		tmp_num = strsep(&str, "@");
+
+		if (tmp_num != NULL) {
+			if (dm_strtoi(tmp_num, &tmp_num, 10, &num) || (num <= 0)) {
+				pr_err("%s: invalid iothread number %s \n", __func__, tmp_num);
+				return -1;
+			}
+
+			cpuset_list = calloc(num, sizeof(cpu_set_t));
+			if (cpuset_list == NULL) {
+				pr_err("%s: calloc cpuset_list returns NULL \n", __func__);
+				return -1;
+			}
+
+			iothread_sub_idx = 0;
+			while ((str != NULL) && (*str !='\0') && (iothread_sub_idx < num)) {
+				/* "/" is used to separate the CPU affinity setting for each iothread instance. */
+				tmp_cpusets = strsep(&str, "/");
+
+				CPU_ZERO(cpuset_list + iothread_sub_idx);
+				while ((tmp_cpusets != NULL) && (*tmp_cpusets !='\0')) {
+					/* ":" is used to separate different CPU cores. */
+					tmp_cpux = strsep(&tmp_cpusets, ":");
+
+					/*
+					 * char '*' can be used to skip the setting for the
+					 * specific iothread instance.
+					 */
+					if (*tmp_cpux == '*') {
+						break;
+					}
+
+					if (dm_strtoi(tmp_cpux, &tmp_cpux, 10, &service_vm_cpuid) ||
+						(service_vm_cpuid < 0)) {
+						pr_err("%s: invalid CPU affinity setting %s \n",
+							__func__, tmp_cpux);
+
+						free(cpuset_list);
+						return -1;
+					}
+
+					CPU_SET(service_vm_cpuid, cpuset_list + iothread_sub_idx);
+					pr_err("%s: iothread[%d]: set service_vm_cpuid %d \n",
+						__func__, iothread_sub_idx, service_vm_cpuid);
+				}
+				iothread_sub_idx++;
+			}
+		}
+	}
+	iothr_opt->num = num;
+	iothr_opt->cpusets = cpuset_list;
+
 	return 0;
 }
+
+/*
+ * This interface is used to free the elements that are allocated dynamically in iothread_parse_options(),
+ * such as iothr_opt->cpusets.
+ */
+void iothread_free_options(struct iothreads_option *iothr_opt)
+{
+	if ((iothr_opt != NULL) && (iothr_opt->cpusets != NULL)) {
+		free(iothr_opt->cpusets);
+		iothr_opt->cpusets = NULL;
+	}
+
+	return;
+}

devicemodel/core/main.c

@@ -71,6 +71,7 @@
 #include "cmd_monitor.h"
 #include "vdisplay.h"
 #include "iothread.h"
+#include "vm_event.h"
 
 #define	VM_MAXCPU		16	/* maximum virtual cpus */
 
@@ -101,6 +102,7 @@ bool vtpm2;
 bool is_winvm;
 bool skip_pci_mem64bar_workaround = false;
 bool gfx_ui = false;
+bool ovmf_loaded = false;
 
 static int guest_ncpus;
 static int virtio_msix = 1;
@@ -166,7 +168,7 @@ usage(int code)
 		"       -v: version\n"
 		"       --ovmf: ovmf file path\n"
 		"       --iasl: iasl compiler path\n"
-		"       --ssram: Congfiure Software SRAM parameters\n"
+		"       --ssram: Configure Software SRAM parameters\n"
 		"       --cpu_affinity: list of Service VM vCPUs assigned to this User VM, the vCPUs are"
 		"	     identified by their local APIC IDs.\n"
 		"       --enable_trusty: enable trusty for guest\n"
@@ -176,7 +178,7 @@ usage(int code)
 		"       --cmd_monitor: enable command monitor\n"
 		"            its params: unix domain socket path\n"
 		"       --virtio_poll: enable virtio poll mode with poll interval with ns\n"
-		"       --acpidev_pt: acpi device ID args: HID in ACPI Table\n"
+		"       --acpidev_pt: ACPI device ID args: HID in ACPI Table\n"
 		"       --mmiodev_pt: MMIO resources args: physical MMIO regions\n"
 		"       --vtpm2: Virtual TPM2 args: sock_path=$PATH_OF_SWTPM_SOCKET\n"
 		"       --lapic_pt: enable local apic passthrough\n"
@@ -232,6 +234,12 @@ virtio_uses_msix(void)
 	return virtio_msix;
 }
 
+int
+guest_cpu_num(void)
+{
+	return guest_ncpus;
+}
+
 size_t
 high_bios_size(void)
 {
@@ -240,6 +248,64 @@ high_bios_size(void)
 	return roundup2(size, 2 * MB);
 }
 
+/*
+ * set nice value of current pthread
+ * input range: [-20, 19]
+ * Lower priorities cause more favorable scheduling.
+ */
+void
+set_thread_priority(int priority, bool reset_on_fork)
+{
+	int ret, policy;
+	char tname[MAXCOMLEN + 1];
+	struct sched_param sp = { .sched_priority = 0 };
+
+	memset(tname, 0, sizeof(tname));
+	pthread_getname_np(pthread_self(), tname, sizeof(tname));
+
+	policy = sched_getscheduler(0);
+	if (policy == -1) {
+		pr_err("%s(%s), sched_getscheduler failed, errno = %d\n",
+			__func__, tname, errno);
+	}
+
+	if ((policy & SCHED_RESET_ON_FORK) && !reset_on_fork)
+		policy &= ~SCHED_RESET_ON_FORK;
+	else if (((policy & SCHED_RESET_ON_FORK) == 0) && reset_on_fork)
+		policy |= SCHED_RESET_ON_FORK;
+
+	ret = sched_setscheduler(0, policy, &sp);
+	if (ret == -1) {
+		pr_err("%s(%s), sched_setscheduler failed, errno = %d\n",
+			__func__, tname, errno);
+	}
+
+	errno = 0;
+	ret = getpriority(PRIO_PROCESS, 0);
+	if (errno && (ret == -1)) {
+		pr_err("%s(%s), getpriority failed, errno = %d\n",
+			__func__, tname, errno);
+	} else {
+		pr_info("%s(%s), orig prio = %d\n",
+			__func__, tname, ret);
+	}
+
+	ret = setpriority(PRIO_PROCESS, 0, priority);
+	if (ret) {
+		pr_err("%s(%s), setpriority failed, errno = %d\n",
+			__func__, tname, errno);
+	}
+
+	ret = getpriority(PRIO_PROCESS, 0);
+	if (ret != priority) {
+		pr_err("%s(%s), getpriority(%d) != setpriority(%d)\n",
+			__func__, tname, ret, priority);
+	} else {
+		pr_info("%s(%s), new priority = %d\n",
+			__func__, tname, ret);
+	}
+}
+
 static void *
 start_thread(void *param)
 {
@@ -609,6 +675,8 @@ vm_reset_vdevs(struct vmctx *ctx)
 	pci_irq_deinit(ctx);
 	ioapic_deinit();
 
+	iothread_deinit();
+
 	pci_irq_init(ctx);
 	atkbdc_init(ctx);
 	vrtc_init(ctx);
@@ -674,6 +742,7 @@ vm_system_reset(struct vmctx *ctx)
 static void
 vm_suspend_resume(struct vmctx *ctx)
 {
+	struct acrn_vcpu_regs bsp_regs;
 	/*
 	 * If we get warm reboot request, we don't want to exit the
 	 * vcpu_loop/vm_loop/mevent_loop. So we do:
@@ -685,6 +754,8 @@ vm_suspend_resume(struct vmctx *ctx)
 	 *   6. hypercall restart vm
 	 */
 	vm_pause(ctx);
+	if (ovmf_loaded)
+		vrtc_suspend(ctx);
 
 	vm_clear_ioreq(ctx);
 	vm_stop_watchdog(ctx);
@@ -694,8 +765,32 @@ vm_suspend_resume(struct vmctx *ctx)
 	vm_reset_watchdog(ctx);
 	vm_reset(ctx);
 
+	bsp_regs = ctx->bsp_regs;
+	/* for bzImage or elf */
+	if (!ovmf_loaded) {
+		uint32_t *guest_wakeup_vec32;
+		/* 64BIT_WAKE_SUPPORTED_F is not set */
+		guest_wakeup_vec32 = paddr_guest2host(ctx,
+						      get_acpi_wakingvector_offset(),
+						      get_acpi_wakingvector_length());
+		/* set the BSP waking vector */
+		bsp_regs.vcpu_regs.cs_sel = (uint16_t)((*guest_wakeup_vec32 >> 4U) & 0xFFFFU);
+		bsp_regs.vcpu_regs.cs_base = bsp_regs.vcpu_regs.cs_sel << 4U;
+		/* real mode code segment */
+		bsp_regs.vcpu_regs.cs_ar = 0x009FU;
+		bsp_regs.vcpu_regs.cs_limit = 0xFFFFU;
+		bsp_regs.vcpu_regs.rip = 0x0U;
+		/* CR0_ET | CR0_NE */
+		bsp_regs.vcpu_regs.cr0 = 0x30;
+		/* real mode gdt */
+		bsp_regs.vcpu_regs.gdt.limit = 0xFFFFU;
+		bsp_regs.vcpu_regs.gdt.base = 0UL;
+		/* real mode idt */
+		bsp_regs.vcpu_regs.idt.limit = 0xFFFFU;
+		bsp_regs.vcpu_regs.idt.base = 0UL;
+	}
 	/* set the BSP init state */
-	vm_set_vcpu_regs(ctx, &ctx->bsp_regs);
+	vm_set_vcpu_regs(ctx, &bsp_regs);
 	vm_run(ctx);
 }
 
@@ -735,8 +830,7 @@ vm_loop(struct vmctx *ctx)
 			break;
 		}
 
-		/* RTVM can't be reset */
-		if ((VM_SUSPEND_SYSTEM_RESET == vm_get_suspend_mode()) && (!is_rtvm)) {
+		if (VM_SUSPEND_SYSTEM_RESET == vm_get_suspend_mode()) {
 			vm_system_reset(ctx);
 		}
 
@@ -852,7 +946,7 @@ vm_init_asyncio(struct vmctx *ctx, uint64_t base)
 	sbuf->overrun_cnt = 0;
 	sbuf->head = 0;
 	sbuf->tail = 0;
-	return vm_setup_sbuf(ctx, ACRN_ASYNCIO, base);
+	return vm_setup_asyncio(ctx, base);
 }
 
 int
@@ -949,6 +1043,7 @@ main(int argc, char *argv[])
 		case CMD_OPT_OVMF:
 			if (!vsbl_file_name && acrn_parse_ovmf(optarg) != 0)
 				errx(EX_USAGE, "invalid ovmf param %s", optarg);
+			ovmf_loaded = true;
 			skip_pci_mem64bar_workaround = true;
 			break;
 		case CMD_OPT_IASL:
@@ -1132,18 +1227,18 @@ main(int argc, char *argv[])
 			goto mevent_fail;
 		}
 
-		error = iothread_init();
-		if (error) {
-			pr_err("Unable to initialize iothread (%d)\n", errno);
-			goto iothread_fail;
-		}
-
 		pr_notice("vm_init_vdevs\n");
 		if (vm_init_vdevs(ctx) < 0) {
 			pr_err("Unable to init vdev (%d)\n", errno);
 			goto dev_fail;
 		}
 
+		pr_notice("vm setup vm event\n");
+		error = vm_event_init(ctx);
+		if (error) {
+			pr_warn("VM_EVENT is not supported by kernel or hyperviosr!\n");
+		}
+
 		/*
 		 * build the guest tables, MP etc.
 		 */
@@ -1198,6 +1293,7 @@ main(int argc, char *argv[])
 			break;
 		}
 
+		vm_event_deinit();
 		vm_deinit_vdevs(ctx);
 		mevent_deinit();
 		iothread_deinit();
@@ -1210,13 +1306,14 @@ main(int argc, char *argv[])
 	}
 
 vm_fail:
+	vm_event_deinit();
+
 	vm_deinit_vdevs(ctx);
 	if (ssram)
 		clean_vssram_configs();
 
 dev_fail:
 	iothread_deinit();
-iothread_fail:
 	mevent_deinit();
 mevent_fail:
 	vm_unsetup_memory(ctx);

devicemodel/core/mevent.c

@@ -72,7 +72,7 @@ struct mevent {
 };
 
 static LIST_HEAD(listhead, mevent) global_head;
-/* List holds the mevent node which is requested to deleted */
+/* List holds the mevent node which is requested to be deleted */
 static LIST_HEAD(del_listhead, mevent) del_head;
 
 static void

devicemodel/core/monitor.c

@@ -124,10 +124,10 @@ static void *intr_storm_monitor_thread(void *arg)
 		}
 
 		/*
-		 * calc the delta of the two times count of interrupt;
-		 * compare the IRQ num first, if not same just drop it,
-		 * for it just happens rarelly when devices dynamically
-		 * allocation in Service VM or User VM, it can be calc next time
+		 * calculate the delta of the two times count of interrupt;
+		 * compare the IRQ number first, if not same just drop it,
+		 * for it just happens rarely when devices dynamically
+		 * allocation in Service VM or User VM, it can be calculated next time
 		 */
 		for (i = 0; i < hdr->buf_cnt; i += 2) {
 			if (hdr->buffer[i] != intr_cnt_buf[i])
@@ -350,7 +350,7 @@ static void handle_stop(struct mngr_msg *msg, int client_fd, void *param)
 	ack.msgid = msg->msgid;
 	ack.timestamp = msg->timestamp;
 
-	if (msg->data.acrnd_stop.force && !is_rtvm) {
+	if (msg->data.acrnd_stop.force) {
 		pr_info("%s: setting VM state to %s\n", __func__, vm_state_to_str(VM_SUSPEND_POWEROFF));
 		vm_set_suspend_mode(VM_SUSPEND_POWEROFF);
 		ack.data.err = 0;

devicemodel/core/sbuf.c

@@ -0,0 +1,125 @@
+/*
+ * Copyright (C) 2018-2023 Intel Corporation.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include <asm/errno.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdio.h>
+#include <stdbool.h>
+#include "sbuf.h"
+#include <errno.h>
+
+static inline uint32_t sbuf_next_ptr(uint32_t pos_arg,
+		uint32_t span, uint32_t scope)
+{
+	uint32_t pos = pos_arg;
+	pos += span;
+	pos = (pos >= scope) ? (pos - scope) : pos;
+	return pos;
+}
+
+uint32_t sbuf_get(struct shared_buf *sbuf, uint8_t *data)
+{
+	const void *from;
+
+	if ((sbuf == NULL) || (data == NULL))
+		return -EINVAL;
+
+	if (sbuf_is_empty(sbuf)) {
+		/* no data available */
+		return 0;
+	}
+
+	from = (void *)sbuf + SBUF_HEAD_SIZE + sbuf->head;
+
+	memcpy(data, from, sbuf->ele_size);
+
+	mb();
+
+	sbuf->head = sbuf_next_ptr(sbuf->head, sbuf->ele_size, sbuf->size);
+
+	return sbuf->ele_size;
+}
+
+int sbuf_clear_buffered(struct shared_buf *sbuf)
+{
+	if (sbuf == NULL)
+		return -EINVAL;
+
+	sbuf->head = sbuf->tail;
+
+	return 0;
+}
+
+/**
+ * The high caller should guarantee each time there must have
+ * sbuf->ele_size data can be write form data.
+ * Caller should provide the max length of the data for safety reason.
+ *
+ * And this function should guarantee execution atomically.
+ *
+ * flag:
+ * If OVERWRITE_EN set, buf can store (ele_num - 1) elements at most.
+ * Should use lock to guarantee that only one read or write at
+ * the same time.
+ * if OVERWRITE_EN not set, buf can store (ele_num - 1) elements
+ * at most. Shouldn't modify the sbuf->head.
+ *
+ * return:
+ * ele_size:	write succeeded.
+ * 0:		no write, buf is full
+ * UINT32_MAX:	failed, sbuf corrupted.
+ */
+uint32_t sbuf_put(struct shared_buf *sbuf, uint8_t *data, uint32_t max_len)
+{
+	uint32_t ele_size = sbuf->ele_size;
+	void *to;
+	uint32_t next_tail;
+	uint32_t ret;
+	bool trigger_overwrite = false;
+
+	next_tail = sbuf_next_ptr(sbuf->tail, ele_size, sbuf->size);
+
+	if ((next_tail == sbuf->head) && ((sbuf->flags & OVERWRITE_EN) == 0U)) {
+		/* if overrun is not enabled, return 0 directly */
+		ret = 0U;
+	} else if (ele_size <= max_len) {
+		if (next_tail == sbuf->head) {
+			/* accumulate overrun count if necessary */
+			sbuf->overrun_cnt += sbuf->flags & OVERRUN_CNT_EN;
+			trigger_overwrite = true;
+		}
+		to = (void *)sbuf + SBUF_HEAD_SIZE + sbuf->tail;
+
+		memcpy(to, data, ele_size);
+		/* make sure write data before update head */
+		mb();
+
+		if (trigger_overwrite) {
+			sbuf->head = sbuf_next_ptr(sbuf->head,
+					ele_size, sbuf->size);
+		}
+		sbuf->tail = next_tail;
+		ret = ele_size;
+	} else {
+		/* there must be something wrong */
+		ret = UINT32_MAX;
+	}
+
+	return ret;
+}
+
+void sbuf_init(struct shared_buf *sbuf, uint32_t total_size, uint32_t ele_size)
+{
+	sbuf->magic = SBUF_MAGIC;
+	sbuf->ele_size = ele_size;
+	sbuf->ele_num = (total_size - SBUF_HEAD_SIZE) / sbuf->ele_size;
+	sbuf->size = sbuf->ele_size * sbuf->ele_num;
+	sbuf->flags = 0;
+	sbuf->overrun_cnt = 0;
+	sbuf->head = 0;
+	sbuf->tail = 0;
+}

devicemodel/core/sw_load_common.c

@@ -297,7 +297,7 @@ acrn_sw_load(struct vmctx *ctx)
 {
 	if (vsbl_file_name)
 		return acrn_sw_load_vsbl(ctx);
-	else if ((ovmf_file_name != NULL) ^ (ovmf_code_file_name && ovmf_vars_file_name))
+	else if (ovmf_loaded)
 		return acrn_sw_load_ovmf(ctx);
 	else if (kernel_file_name)
 		return acrn_sw_load_bzimage(ctx);

devicemodel/core/vm_event.c

@@ -0,0 +1,510 @@
+/*
+ * Copyright (C) 2018-2023 Intel Corporation.
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include <errno.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include <stdbool.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <sys/epoll.h>
+#include <sys/queue.h>
+#include <pthread.h>
+#include <sys/ioctl.h>
+#include <sys/eventfd.h>
+#include <acrn_common.h>
+
+#include "vm_event.h"
+#include "hsm_ioctl_defs.h"
+#include "sbuf.h"
+#include "log.h"
+#include <cjson/cJSON.h>
+#include "monitor.h"
+#include "timer.h"
+
+#define VM_EVENT_ELE_SIZE (sizeof(struct vm_event))
+
+#define HV_VM_EVENT_TUNNEL 0
+#define DM_VM_EVENT_TUNNEL 1
+#define MAX_VM_EVENT_TUNNELS 2
+#define MAX_EPOLL_EVENTS MAX_VM_EVENT_TUNNELS
+
+#define THROTTLE_WINDOW	1U /* time window for throttle counter, in secs*/
+
+#define BROKEN_TIME ((time_t)-1)
+
+typedef void (*vm_event_handler)(struct vmctx *ctx, struct vm_event *event);
+typedef void (*vm_event_generate_jdata)(cJSON *event_obj, struct vm_event *event);
+
+static int epoll_fd;
+static bool started = false;
+static char hv_vm_event_page[4096] __aligned(4096);
+static char dm_vm_event_page[4096] __aligned(4096);
+static pthread_t vm_event_tid;
+
+static void general_event_handler(struct vmctx *ctx, struct vm_event *event);
+static void rtc_chg_event_handler(struct vmctx *ctx, struct vm_event *event);
+
+static void gen_rtc_chg_jdata(cJSON *event_obj, struct vm_event *event);
+
+enum event_source_type {
+	EVENT_SOURCE_TYPE_HV,
+	EVENT_SOURCE_TYPE_DM,
+};
+
+struct vm_event_tunnel {
+	enum event_source_type type;
+	struct shared_buf *sbuf;
+	uint32_t sbuf_size;
+	int kick_fd;
+	pthread_mutex_t mtx;
+	bool enabled;
+};
+
+struct event_throttle_ctl {
+	struct acrn_timer timer;
+	pthread_mutex_t mtx;
+	uint32_t event_counter;
+	uint32_t throttle_count;	/* how many events has been throttled(dropped) */
+	bool	is_up;
+};
+
+struct vm_event_proc {
+	vm_event_handler ve_handler;
+	uint32_t	throttle_rate; /* how many events allowed per sec */
+	struct event_throttle_ctl throttle_ctl;
+	vm_event_generate_jdata gen_jdata_handler; /* how to transtfer vm_event data to json txt */
+};
+
+static struct vm_event_proc ve_proc[VM_EVENT_COUNT] = {
+	[VM_EVENT_RTC_CHG] = {
+		.ve_handler = rtc_chg_event_handler,
+		.gen_jdata_handler = gen_rtc_chg_jdata,
+		.throttle_rate = 1,
+	},
+	[VM_EVENT_POWEROFF] = {
+		.ve_handler = general_event_handler,
+		.gen_jdata_handler = NULL,
+		.throttle_rate = 1,
+	},
+	[VM_EVENT_TRIPLE_FAULT] = {
+		.ve_handler = general_event_handler,
+		.gen_jdata_handler = NULL,
+		.throttle_rate = 1,
+	},
+};
+
+static inline struct vm_event_proc *get_vm_event_proc(struct vm_event *event)
+{
+	struct vm_event_proc *proc = NULL;
+	if (event->type < VM_EVENT_COUNT) {
+		proc = &ve_proc[event->type];
+	}
+	return proc;
+}
+
+static bool event_throttle(struct vm_event *event)
+{
+	struct vm_event_proc *proc;
+	struct event_throttle_ctl *ctl;
+	uint32_t current_rate;
+	bool ret = false;
+
+	proc = get_vm_event_proc(event);
+	if (proc) {
+		ctl = &proc->throttle_ctl;
+		if (ctl->is_up) {
+			pthread_mutex_lock(&ctl->mtx);
+			current_rate = ctl->event_counter / THROTTLE_WINDOW;
+			if (current_rate < proc->throttle_rate) {
+				ctl->event_counter++;
+				ret = false;
+			} else {
+				ret = true;
+				ctl->throttle_count++;
+				pr_notice("event %d throttle: %d dropped\n",
+					event->type, ctl->throttle_count);
+			}
+			pthread_mutex_unlock(&ctl->mtx);
+		}
+	}
+	return ret;
+}
+
+void throttle_timer_cb(void *arg, uint64_t nexp)
+{
+	struct event_throttle_ctl *ctl = (struct event_throttle_ctl *)arg;
+	pthread_mutex_lock(&ctl->mtx);
+	ctl->event_counter = 0;
+	pthread_mutex_unlock(&ctl->mtx);
+}
+
+static void vm_event_throttle_init(struct vmctx *ctx)
+{
+	int i;
+	struct event_throttle_ctl *ctl;
+	int ret = 0;
+	struct itimerspec timer_spec;
+
+	for (i = 0; i < ARRAY_SIZE(ve_proc); i++) {
+		ctl = &ve_proc[i].throttle_ctl;
+		ctl->event_counter = 0U;
+		ctl->throttle_count = 0U;
+		ctl->is_up = false;
+		pthread_mutex_init(&ctl->mtx, NULL);
+		ctl->timer.clockid = CLOCK_MONOTONIC;
+		ret = acrn_timer_init(&ctl->timer, throttle_timer_cb, ctl);
+		if (ret < 0) {
+			pr_warn("failed to create timer for vm_event %d, throttle disabled\n", i);
+			continue;
+		}
+		timer_spec.it_value.tv_sec = THROTTLE_WINDOW;
+		timer_spec.it_value.tv_nsec = 0;
+		timer_spec.it_interval.tv_sec = THROTTLE_WINDOW;
+		timer_spec.it_interval.tv_nsec = 0;
+		ret = acrn_timer_settime(&ctl->timer, &timer_spec);
+		if (ret < 0) {
+			pr_warn("failed to set timer for vm_event %d, throttle disabled\n", i);
+			continue;
+		}
+		ctl->is_up = true;
+	}
+}
+
+static void vm_event_throttle_deinit(void)
+{
+	int i;
+	struct event_throttle_ctl *ctl;
+
+	for (i = 0; i < ARRAY_SIZE(ve_proc); i++) {
+		ctl = &ve_proc[i].throttle_ctl;
+		if (ctl->timer.fd != -1) {
+			acrn_timer_deinit(&ctl->timer);
+		}
+	}
+}
+
+static char *generate_vm_event_message(struct vm_event *event)
+{
+	char *event_msg = NULL;
+	cJSON *val;
+	cJSON *event_obj = cJSON_CreateObject();
+	struct vm_event_proc *proc;
+
+	if (event_obj == NULL)
+		return NULL;
+	val = cJSON_CreateNumber(event->type);
+	if (val == NULL)
+		return NULL;
+	cJSON_AddItemToObject(event_obj, "vm_event", val);
+
+	proc = get_vm_event_proc(event);
+	if (proc && proc->gen_jdata_handler) {
+		(proc->gen_jdata_handler)(event_obj, event);
+	}
+
+	event_msg = cJSON_Print(event_obj);
+	if (event_msg == NULL)
+		fprintf(stderr, "Failed to generate vm_event message.\n");
+
+	cJSON_Delete(event_obj);
+
+	return event_msg;
+}
+
+static void emit_vm_event(struct vmctx *ctx, struct vm_event *event)
+{
+	if (!event_throttle(event)) {
+		char *msg = generate_vm_event_message(event);
+		if (msg != NULL) {
+			vm_monitor_send_vm_event(msg);
+			free(msg);
+		}
+	}
+}
+
+static void general_event_handler(struct vmctx *ctx, struct vm_event *event)
+{
+	emit_vm_event(ctx, event);
+}
+
+static void gen_rtc_chg_jdata(cJSON *event_obj, struct vm_event *event)
+{
+	struct rtc_change_event_data *data = (struct rtc_change_event_data *)event->event_data;
+	cJSON *val;
+
+	val = cJSON_CreateNumber(data->delta_time);
+	if (val != NULL) {
+		cJSON_AddItemToObject(event_obj, "delta_time", val);
+	}
+	val = cJSON_CreateNumber(data->last_time);
+	if (val != NULL) {
+		cJSON_AddItemToObject(event_obj, "last_time", val);
+	}
+}
+
+/* assume we only have one unique rtc source */
+
+static struct acrn_timer rtc_chg_event_timer = {
+	.clockid = CLOCK_MONOTONIC,
+};
+static pthread_mutex_t rtc_chg_mutex = PTHREAD_MUTEX_INITIALIZER;
+static struct timespec time_window_start;
+static time_t last_time_cached = BROKEN_TIME;
+static time_t delta_time_sum = 0;
+#define RTC_CHG_WAIT_TIME 1 /* 1 second */
+static void rtc_chg_event_handler(struct vmctx *ctx, struct vm_event *event)
+{
+	struct itimerspec timer_spec;
+	struct rtc_change_event_data *data = (struct rtc_change_event_data *)event->event_data;
+
+	/*
+	 * RTC time is not reliable until guest finishes updating all RTC date/time regs.
+	 * So wait for some time, if no more change happens, we can conclude that the RTC
+	 * change has been done.
+	 */
+	timer_spec.it_value.tv_sec = RTC_CHG_WAIT_TIME;
+	timer_spec.it_value.tv_nsec = 0;
+	timer_spec.it_interval.tv_sec = 0;
+	timer_spec.it_interval.tv_nsec = 0;
+	pthread_mutex_lock(&rtc_chg_mutex);
+	if (last_time_cached == BROKEN_TIME) {
+		last_time_cached = data->last_time;
+	}
+	delta_time_sum += data->delta_time;
+	/* The last timer will be overwriten if it is not triggered yet. */
+	acrn_timer_settime(&rtc_chg_event_timer, &timer_spec);
+	clock_gettime(CLOCK_MONOTONIC, &time_window_start);
+	pthread_mutex_unlock(&rtc_chg_mutex);
+}
+
+static void rtc_chg_timer_cb(void *arg, uint64_t nexp)
+{
+	struct timespec now, delta;
+	struct timespec time_window_size = {RTC_CHG_WAIT_TIME, 0};
+	struct vmctx *ctx = arg;
+	struct vm_event send_event;
+	struct rtc_change_event_data *data = (struct rtc_change_event_data *)send_event.event_data;
+
+	pthread_mutex_lock(&rtc_chg_mutex);
+	clock_gettime(CLOCK_MONOTONIC, &now);
+	delta = now;
+	timespecsub(&delta, &time_window_start);
+	/* possible racing problem here. make sure this is the right timer cb for the vm_event */
+	if (timespeccmp(&delta, &time_window_size, >=)) {
+		data->delta_time = delta_time_sum;
+		data->last_time = last_time_cached;
+		emit_vm_event(ctx, &send_event);
+		last_time_cached = BROKEN_TIME;
+		delta_time_sum = 0;
+	}
+	pthread_mutex_unlock(&rtc_chg_mutex);
+}
+
+static void *vm_event_thread(void *param)
+{
+	int n, i;
+	struct vm_event ve;
+	eventfd_t val;
+	struct vm_event_tunnel *tunnel;
+	struct vmctx *ctx = param;
+
+	struct epoll_event eventlist[MAX_EPOLL_EVENTS];
+
+	while (started) {
+		n = epoll_wait(epoll_fd, eventlist, MAX_EPOLL_EVENTS, -1);
+		if (n < 0) {
+			if (errno != EINTR) {
+				pr_err("%s: epoll failed %d\n", __func__, errno);
+			}
+			continue;
+		}
+		for (i = 0; i < n; i++) {
+			if (i < MAX_EPOLL_EVENTS) {
+				tunnel = eventlist[i].data.ptr;
+				eventfd_read(tunnel->kick_fd, &val);
+				if (tunnel && tunnel->enabled) {
+					while (!sbuf_is_empty(tunnel->sbuf)) {
+						struct vm_event_proc *proc;
+						sbuf_get(tunnel->sbuf, (uint8_t*)&ve);
+						pr_dbg("%ld vm event from%d %d\n", val, tunnel->type, ve.type);
+						proc = get_vm_event_proc(&ve);
+						if (proc && proc->ve_handler) {
+							(proc->ve_handler)(ctx, &ve);
+						} else {
+							pr_warn("%s: unhandled vm event type %d\n", __func__, ve.type);
+						}
+
+					}
+				}
+			}
+		}
+	}
+	return NULL;
+}
+
+static struct vm_event_tunnel ve_tunnel[MAX_VM_EVENT_TUNNELS] = {
+	{
+		.type = EVENT_SOURCE_TYPE_HV,
+		.sbuf = (struct shared_buf *)hv_vm_event_page,
+		.sbuf_size = 4096,
+		.enabled = false,
+	},
+	{
+		.type = EVENT_SOURCE_TYPE_DM,
+		.sbuf = (struct shared_buf *)dm_vm_event_page,
+		.sbuf_size = 4096,
+		.enabled = false,
+	},
+};
+
+static int create_event_tunnel(struct vmctx *ctx, struct vm_event_tunnel *tunnel, int epoll_fd)
+{
+	struct epoll_event ev;
+	enum event_source_type type = tunnel->type;
+	struct shared_buf *sbuf = tunnel->sbuf;
+	int kick_fd = -1;
+	int error;
+
+	sbuf_init(sbuf, tunnel->sbuf_size, VM_EVENT_ELE_SIZE);
+
+	if (type == EVENT_SOURCE_TYPE_HV) {
+		error = ioctl(ctx->fd, ACRN_IOCTL_SETUP_VM_EVENT_RING, sbuf);
+		if (error) {
+			pr_err("%s: Setting vm_event ring failed %d\n", __func__, error);
+			goto out;
+		}
+	}
+
+	kick_fd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC);
+	if (kick_fd < 0) {
+		pr_err("%s: eventfd failed %d\n", __func__, errno);
+		goto out;
+	}
+
+	if (type == EVENT_SOURCE_TYPE_HV) {
+		error = ioctl(ctx->fd, ACRN_IOCTL_SETUP_VM_EVENT_FD, kick_fd);
+		if (error) {
+			pr_err("%s: Setting vm_event fd failed %d\n", __func__, error);
+			goto out;
+		}
+	}
+
+	ev.events = EPOLLIN;
+	ev.data.ptr = tunnel;
+	error = epoll_ctl(epoll_fd, EPOLL_CTL_ADD, kick_fd, &ev);
+	if (error < 0) {
+		pr_err("%s: failed to add fd, error is %d\n", __func__, errno);
+		goto out;
+	}
+
+	tunnel->kick_fd = kick_fd;
+	pthread_mutex_init(&tunnel->mtx, NULL);
+	tunnel->enabled = true;
+
+	return 0;
+
+out:
+	if (kick_fd >= 0) {
+		close(kick_fd);
+	}
+	return -1;
+}
+
+void destory_event_tunnel(struct vm_event_tunnel *tunnel)
+{
+	if (tunnel->enabled) {
+		close(tunnel->kick_fd);
+		tunnel->enabled = false;
+		pthread_mutex_destroy(&tunnel->mtx);
+	}
+}
+
+int vm_event_init(struct vmctx *ctx)
+{
+	int error;
+
+	epoll_fd = epoll_create1(0);
+	if (epoll_fd < 0) {
+		pr_err("%s: failed to create epoll %d\n", __func__, errno);
+		goto out;
+	}
+
+	error = create_event_tunnel(ctx, &ve_tunnel[HV_VM_EVENT_TUNNEL], epoll_fd);
+	if (error) {
+		goto out;
+	}
+
+	error = create_event_tunnel(ctx, &ve_tunnel[DM_VM_EVENT_TUNNEL], epoll_fd);
+	if (error) {
+		goto out;
+	}
+
+	vm_event_throttle_init(ctx);
+
+	error = pthread_create(&vm_event_tid, NULL, vm_event_thread, ctx);
+	if (error) {
+		pr_err("%s: vm_event create failed %d\n", __func__, errno);
+		goto out;
+	}
+
+	acrn_timer_init(&rtc_chg_event_timer, rtc_chg_timer_cb, ctx);
+
+	started = true;
+	return 0;
+
+out:
+	if (epoll_fd >= 0) {
+		close(epoll_fd);
+	}
+	destory_event_tunnel(&ve_tunnel[HV_VM_EVENT_TUNNEL]);
+	destory_event_tunnel(&ve_tunnel[DM_VM_EVENT_TUNNEL]);
+	return -1;
+}
+
+int vm_event_deinit(void)
+{
+	void *jval;
+
+	if (started) {
+		started = false;
+		vm_event_throttle_deinit();
+		pthread_kill(vm_event_tid, SIGCONT);
+		pthread_join(vm_event_tid, &jval);
+		close(epoll_fd);
+		destory_event_tunnel(&ve_tunnel[HV_VM_EVENT_TUNNEL]);
+		destory_event_tunnel(&ve_tunnel[DM_VM_EVENT_TUNNEL]);
+	}
+	return 0;
+}
+
+/* Send a dm generated vm_event by putting it to sbuf.
+ * A thread will receive and process those events.
+ * Events will be dropped if sbuf is full.
+ * They also maight be dropped due to event throttle control in receive thread.
+ */
+int dm_send_vm_event(struct vm_event *event)
+{
+	struct vm_event_tunnel *tunnel = &ve_tunnel[DM_VM_EVENT_TUNNEL];
+	struct shared_buf *sbuf;
+	int32_t ret = -1;
+	uint32_t size_sent;
+
+	if (!tunnel->enabled) {
+		return -1;
+	}
+	sbuf = tunnel->sbuf;
+
+	if (sbuf != NULL) {
+		pthread_mutex_lock(&tunnel->mtx);
+		size_sent = sbuf_put(sbuf, (uint8_t *)event, sizeof(*event));
+		pthread_mutex_unlock(&tunnel->mtx);
+		if (size_sent == VM_EVENT_ELE_SIZE) {
+			eventfd_write(tunnel->kick_fd, 1UL);
+			ret = 0;
+		}
+	}
+	return ret;
+}

devicemodel/core/vmmapi.c

@@ -295,19 +295,14 @@ vm_destroy(struct vmctx *ctx)
 }
 
 int
-vm_setup_sbuf(struct vmctx *ctx, uint32_t sbuf_id, uint64_t base)
+vm_setup_asyncio(struct vmctx *ctx, uint64_t base)
 {
 	int error;
-	struct acrn_sbuf sbuf_param;
 
-	bzero(&sbuf_param, sizeof(sbuf_param));
-	sbuf_param.sbuf_id = sbuf_id;
-	sbuf_param.base = base;
-
-	error = ioctl(ctx->fd, ACRN_IOCTL_SETUP_SBUF, &sbuf_param);
+	error = ioctl(ctx->fd, ACRN_IOCTL_SETUP_ASYNCIO, base);
 
 	if (error) {
-		pr_err("ACRN_IOCTL_SBUF_PAGE ioctl() returned an error: %s\n", errormsg(errno));
+		pr_err("ACRN_IOCTL_SETUP_ASYNCIO ioctl() returned an error: %s\n", errormsg(errno));
 	}
 
 	return error;

devicemodel/hw/pci/ahci.c

@@ -2405,7 +2405,7 @@ pci_ahci_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts, int atapi)
 		 */
 		snprintf(bident, sizeof(bident), "%02x:%02x:%02x", dev->slot,
 		    dev->func, p);
-		bctxt = blockif_open(opts, bident);
+		bctxt = blockif_open(opts, bident, 1, NULL);
 		if (bctxt == NULL) {
 			ahci_dev->ports = p;
 			ret = 1;

devicemodel/hw/pci/core.c

@@ -1918,31 +1918,29 @@ pci_bus_write_dsdt(int bus)
 	dsdt_line("        ,, , AddressRangeMemory, TypeStatic)");
 	dsdt_line("    })");
 
-	if (!is_rtvm) {
-		count = pci_count_lintr(bus);
-		if (count != 0) {
-			dsdt_indent(2);
-			dsdt_line("Name (PPRT, Package ()");
-			dsdt_line("{");
-			pci_walk_lintr(bus, pci_pirq_prt_entry, NULL);
-			dsdt_line("})");
-			dsdt_line("Name (APRT, Package ()");
-			dsdt_line("{");
-			pci_walk_lintr(bus, pci_apic_prt_entry, NULL);
-			dsdt_line("})");
-			dsdt_line("Method (_PRT, 0, NotSerialized)");
-			dsdt_line("{");
-			dsdt_line("  If (PICM)");
-			dsdt_line("  {");
-			dsdt_line("    Return (APRT)");
-			dsdt_line("  }");
-			dsdt_line("  Else");
-			dsdt_line("  {");
-			dsdt_line("    Return (PPRT)");
-			dsdt_line("  }");
-			dsdt_line("}");
-			dsdt_unindent(2);
-		}
+	count = pci_count_lintr(bus);
+	if (count != 0) {
+		dsdt_indent(2);
+		dsdt_line("Name (PPRT, Package ()");
+		dsdt_line("{");
+		pci_walk_lintr(bus, pci_pirq_prt_entry, NULL);
+		dsdt_line("})");
+		dsdt_line("Name (APRT, Package ()");
+		dsdt_line("{");
+		pci_walk_lintr(bus, pci_apic_prt_entry, NULL);
+		dsdt_line("})");
+		dsdt_line("Method (_PRT, 0, NotSerialized)");
+		dsdt_line("{");
+		dsdt_line("  If (PICM)");
+		dsdt_line("  {");
+		dsdt_line("    Return (APRT)");
+		dsdt_line("  }");
+		dsdt_line("  Else");
+		dsdt_line("  {");
+		dsdt_line("    Return (PPRT)");
+		dsdt_line("  }");
+		dsdt_line("}");
+		dsdt_unindent(2);
 	}
 
 	dsdt_indent(2);
@@ -2012,8 +2010,6 @@ pci_msix_enabled(struct pci_vdev *dev)
  *
  * @param dev Pointer to struct pci_vdev representing virtual PCI device.
  * @param index MSIx table entry index.
- *
- * @return None
  */
 void
 pci_generate_msix(struct pci_vdev *dev, int index)
@@ -2041,8 +2037,6 @@ pci_generate_msix(struct pci_vdev *dev, int index)
  *
  * @param dev Pointer to struct pci_vdev representing virtual PCI device.
  * @param index Message data index.
- *
- * @return None
  */
 void
 pci_generate_msi(struct pci_vdev *dev, int index)
@@ -2163,8 +2157,6 @@ pci_lintr_route(struct pci_vdev *dev)
  * @brief Assert INTx pin of virtual PCI device
  *
  * @param dev Pointer to struct pci_vdev representing virtual PCI device.
- *
- * @return None
  */
 void
 pci_lintr_assert(struct pci_vdev *dev)
@@ -2189,8 +2181,6 @@ pci_lintr_assert(struct pci_vdev *dev)
  * @brief Deassert INTx pin of virtual PCI device
  *
  * @param dev Pointer to struct pci_vdev representing virtual PCI device.
- *
- * @return None
  */
 void
 pci_lintr_deassert(struct pci_vdev *dev)

devicemodel/hw/pci/gvt.c

@@ -256,7 +256,7 @@ gvt_init_config(struct pci_gvt *gvt)
 	/* capability */
 	pci_set_cfgdata8(gvt->gvt_pi, PCIR_CAP_PTR, gvt->host_config[0x34]);
 	cap_ptr = gvt->host_config[0x34];
-	while (cap_ptr != 0) {
+	while (cap_ptr != 0 && cap_ptr <= PCI_REGMAX - 15) {
 		pci_set_cfgdata32(gvt->gvt_pi, cap_ptr,
 			gvt->host_config[cap_ptr]);
 		pci_set_cfgdata32(gvt->gvt_pi, cap_ptr + 4,

devicemodel/hw/pci/ivshmem.c

@@ -53,6 +53,7 @@
 #define	IVSHMEM_DEVICE_ID	0x1110
 #define	IVSHMEM_CLASS		0x05
 #define	IVSHMEM_REV		0x01
+#define IVSHMEM_INTEL_SUBVENDOR_ID      0x8086U
 
 
 /* IVSHMEM MMIO Registers */
@@ -249,13 +250,13 @@ pci_ivshmem_read(struct vmctx *ctx, int vcpu, struct pci_vdev *dev,
 static int
 pci_ivshmem_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 {
-	uint32_t size;
-	char *tmp, *name, *orig;
+	uint32_t size, region_id = 0;
+	char *tmp, *name, *size_str, *orig;
 	struct pci_ivshmem_vdev *ivshmem_vdev = NULL;
 	bool is_hv_land;
 	int rc;
 
-	/* ivshmem device usage: "-s N,ivshmem,shm_name,shm_size" */
+	/* ivshmem device usage: "-s N,ivshmem,shm_name,shm_size,region_id" */
 	tmp = orig = strdup(opts);
 	if (!orig) {
 		pr_warn("No memory for strdup\n");
@@ -277,8 +278,9 @@ pci_ivshmem_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 		goto err;
 	}
 
-	if (dm_strtoui(tmp, &tmp, 10, &size) != 0) {
-		pr_warn("the shared memory size is incorrect, %s\n", tmp);
+	size_str = strsep(&tmp, ",");
+	if (dm_strtoui(size_str, &size_str, 10, &size) != 0) {
+		pr_warn("the shared memory size is incorrect, %s\n", size_str);
 		goto err;
 	}
 	size *= 0x100000; /* convert to megabytes */
@@ -289,6 +291,13 @@ pci_ivshmem_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 		goto err;
 	}
 
+	if (tmp) {
+		if (dm_strtoui(tmp, &tmp, 10, &region_id) != 0) {
+			pr_warn("shared memory region ID is incorrect, %s, 0 will used.\n", tmp);
+			region_id = 0;
+		}
+	}
+
 	ivshmem_vdev = calloc(1, sizeof(struct pci_ivshmem_vdev));
 	if (!ivshmem_vdev) {
 		pr_warn("failed to allocate ivshmem device\n");
@@ -304,6 +313,8 @@ pci_ivshmem_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 	pci_set_cfgdata16(dev, PCIR_DEVICE, IVSHMEM_DEVICE_ID);
 	pci_set_cfgdata16(dev, PCIR_REVID, IVSHMEM_REV);
 	pci_set_cfgdata8(dev, PCIR_CLASS, IVSHMEM_CLASS);
+	pci_set_cfgdata16(dev, PCIR_SUBDEV_0, (uint16_t)region_id);
+	pci_set_cfgdata16(dev, PCIR_SUBVEND_0, IVSHMEM_INTEL_SUBVENDOR_ID);
 
 	pci_emul_alloc_bar(dev, IVSHMEM_MMIO_BAR, PCIBAR_MEM32, IVSHMEM_REG_SIZE);
 	pci_emul_alloc_bar(dev, IVSHMEM_MSIX_BAR, PCIBAR_MEM32, IVSHMEM_MSIX_PBA_SIZE);

devicemodel/hw/pci/lpc.c

@@ -59,27 +59,31 @@ SYSRES_IO(NMISC_PORT, 1);
 
 static struct pci_vdev *lpc_bridge;
 
-#define	LPC_UART_NUM	2
+#define	LPC_UART_NUM	5
 static struct lpc_uart_vdev {
 	struct uart_vdev *uart;
 	const char *opts;
 	int	iobase;
 	int	irq;
-	int	enabled;
+	int	enabled;	/* enabled/configured by user */
 } lpc_uart_vdev[LPC_UART_NUM];
+#define LPC_S5_UART_NAME "COM5"
 
-static const char *lpc_uart_names[LPC_UART_NUM] = { "COM1", "COM2" };
+static const char *lpc_uart_names[LPC_UART_NUM] = { "COM1", "COM2", "COM3", "COM4", LPC_S5_UART_NAME};
 
 /*
  * LPC device configuration is in the following form:
  * <lpc_device_name>[,<options>]
  * For e.g. "com1,stdio"
+ * For S5 e.g. "com5,/dev/pts/0,0x9000,5"
  */
 int
 lpc_device_parse(const char *opts)
 {
 	int unit, error;
 	char *str, *cpy, *lpcdev;
+        char *lpcopt, *lpcport, *endptr;
+	int s5_port = 0, s5_irq = 0;
 
 	error = -1;
 	str = cpy = strdup(opts);
@@ -87,7 +91,26 @@ lpc_device_parse(const char *opts)
 	if (lpcdev != NULL) {
 		for (unit = 0; unit < LPC_UART_NUM; unit++) {
 			if (strcasecmp(lpcdev, lpc_uart_names[unit]) == 0) {
-				lpc_uart_vdev[unit].opts = str;
+				lpc_uart_vdev[unit].enabled = 1;
+				if(strcasecmp(lpcdev,LPC_S5_UART_NAME) == 0){
+					lpcopt = strsep(&str,",");
+					if(lpcopt != NULL){
+						lpc_uart_vdev[unit].opts = lpcopt;
+					}
+					lpcport = strsep(&str, ",");
+					if(lpcport != NULL){
+						if(dm_strtoul(lpcport, &endptr, 0, (long unsigned int*)&s5_port))
+							goto done;
+						if(dm_strtoul(str, &endptr, 0, (long unsigned int*)&s5_irq))
+							goto done;
+					}
+					if((s5_port != 0) && (s5_irq != 0)){
+						uart_legacy_reinit_res(unit, s5_port, s5_irq);
+					}
+				}
+				else{
+					lpc_uart_vdev[unit].opts = str;
+				}
 				error = 0;
 				goto done;
 			}
@@ -184,7 +207,6 @@ lpc_deinit(struct vmctx *ctx)
 		uart_release_backend(lpc_uart->uart, lpc_uart->opts);
 		uart_legacy_dealloc(unit);
 		lpc_uart->uart = NULL;
-		lpc_uart->enabled = 0;
 	}
 }
 
@@ -202,6 +224,9 @@ lpc_init(struct vmctx *ctx)
 		lpc_uart = &lpc_uart_vdev[unit];
 		name = lpc_uart_names[unit];
 
+		if (lpc_uart->enabled == 0)
+			continue;
+
 		if (uart_legacy_alloc(unit,
 				      &lpc_uart->iobase,
 				      &lpc_uart->irq) != 0) {
@@ -229,7 +254,6 @@ lpc_init(struct vmctx *ctx)
 		error = register_inout(&iop);
 		if (error)
 			goto init_failed;
-		lpc_uart->enabled = 1;
 	}
 
 	return 0;

devicemodel/hw/pci/passthrough.c

@@ -48,6 +48,7 @@
 #include "dm.h"
 #include "passthru.h"
 #include "ptm.h"
+#include "igd_pciids.h"
 
 /* Some audio drivers get topology data from ACPI NHLT table.
  * For such drivers, we need to copy the host NHLT table to make it
@@ -60,8 +61,9 @@
 
 extern uint64_t audio_nhlt_len;
 
-uint32_t gpu_dsm_hpa = 0;
-uint32_t gpu_dsm_gpa = 0;
+uint64_t gpu_dsm_hpa = 0;
+uint64_t gpu_dsm_gpa = 0;
+uint32_t gpu_dsm_size = 0;
 uint32_t gpu_opregion_hpa = 0;
 uint32_t gpu_opregion_gpa = 0;
 
@@ -552,7 +554,62 @@ get_gpu_rsvmem_base_gpa()
 uint32_t
 get_gpu_rsvmem_size()
 {
-	return GPU_OPREGION_SIZE + GPU_DSM_SIZE;
+	return GPU_OPREGION_SIZE + gpu_dsm_size;
+}
+
+static const struct igd_device igd_device_tbl[] = {
+	IGD_RPLP_DEVICE_IDS,
+	IGD_RPLS_DEVICE_IDS,
+	IGD_ADLN_DEVICE_IDS,
+	IGD_ADLP_DEVICE_IDS,
+	IGD_ADLS_DEVICE_IDS,
+	IGD_RKL_DEVICE_IDS,
+	IGD_TGL_DEVICE_IDS,
+	IGD_JSL_DEVICE_IDS,
+	IGD_EHL_DEVICE_IDS,
+	IGD_ICL_DEVICE_IDS,
+	IGD_CFL_DEVICE_IDS,
+	IGD_KBL_DEVICE_IDS,
+	IGD_GLK_DEVICE_IDS,
+	IGD_BXT_DEVICE_IDS,
+	IGD_SKL_DEVICE_IDS,
+	{ 0 }
+};
+
+int igd_gen(uint16_t device) {
+	const struct igd_device *entry;
+
+	for (entry = igd_device_tbl; entry->device != 0; entry++) {
+		if (entry->device == device) {
+			return entry->gen;
+		}
+	}
+	return 0;
+}
+
+uint32_t igd_dsm_region_size(struct pci_device *igddev)
+{
+	uint16_t ggc;
+	uint8_t gms;
+
+	ggc = read_config(igddev, PCIR_GGC, 2);
+	gms = ggc >> PCIR_GGC_GMS_SHIFT;
+
+	switch (gms) {
+		case 0x00 ... 0x10:
+			return gms * 32 * MB;
+		case 0x20:
+			return 1024 * MB;
+		case 0x30:
+			return 1536 * MB;
+		case 0x40:
+			return 2048 * MB;
+		case 0xf0 ... 0xfe:
+			return (gms - 0xf0 + 1) * 4 * MB;
+	}
+
+	pr_err("%s: Invalid GMS value in GGC register. GGC = %04x\n", __func__, ggc);
+	return 0;	/* Should never reach here */
 }
 
 /*
@@ -563,59 +620,26 @@ passthru_gpu_dsm_opregion(struct vmctx *ctx, struct passthru_dev *ptdev,
 			struct acrn_pcidev *pcidev, uint16_t device)
 {
 	uint32_t opregion_phys, dsm_mask_val;
+	int gen;
 
 	/* get opregion hpa */
 	opregion_phys = read_config(ptdev->phys_dev, PCIR_ASLS_CTL, 4);
 	gpu_opregion_hpa = opregion_phys & PCIM_ASLS_OPREGION_MASK;
 
-	switch (device) {
-	/* ElkhartLake */
-	case 0x4500:
-	case 0x4541:
-	case 0x4551:
-	case 0x4571:
-	/* TigerLake */
-	case 0x9a40:
-	case 0x9a49:
-	case 0x9a59:
-	case 0x9a60:
-	case 0x9a68:
-	case 0x9a70:
-	case 0x9a78:
-	case 0x9ac0:
-	case 0x9ac9:
-	case 0x9ad9:
-	case 0x9af8:
-	/* AlderLake */
-	case 0x4680:
-	case 0x4681:
-	case 0x4682:
-	case 0x4683:
-	case 0x4690:
-	case 0x4691:
-	case 0x4692:
-	case 0x4693:
-	case 0x4698:
-	case 0x4699:
-	/* ADL-P GT graphics */
-	case 0x4626:
-	case 0x4628:
-	case 0x462a:
-	case 0x46a0:
-	case 0x46a1:
-	case 0x46a2:
-	case 0x46a3:
-	case 0x46a6:
-	case 0x46a8:
-	case 0x46aa:
-	case 0x46b0:
-	case 0x46b1:
-	case 0x46b2:
-	case 0x46b3:
-	case 0x46c0:
-	case 0x46c1:
-	case 0x46c2:
-	case 0x46c3:
+	gen = igd_gen(device);
+	if (!gen) {
+		pr_warn("Device 8086:%04x is not an igd device in allowlist, assuming it is gen 11+. " \
+			"GVT-d may not working properly\n", device);
+		gen = 11;
+	}
+
+	gpu_dsm_size = igd_dsm_region_size(ptdev->phys_dev);
+	if (!gpu_dsm_size) {
+		pr_err("Invalid igd dsm region size, check DVMT Pre-Allocated option in BIOS\n");
+		return;
+	}
+
+	if (gen >= 11) {
 		/* BDSM register has 64 bits.
 		 * bits 63:20 contains the base address of stolen memory
 		 */
@@ -634,9 +658,7 @@ passthru_gpu_dsm_opregion(struct vmctx *ctx, struct passthru_dev *ptdev,
 		pci_set_cfgdata32(ptdev->dev, PCIR_GEN11_BDSM_DW1, 0);
 
 		ptdev->has_virt_pcicfg_regs = &has_virt_pcicfg_regs_on_ehl_gpu;
-		break;
-	/* If on default platforms, such as KBL,WHL  */
-	default:
+	} else {
 		/* bits 31:20 contains the base address of stolen memory */
 		gpu_dsm_hpa = read_config(ptdev->phys_dev, PCIR_BDSM, 4);
 		dsm_mask_val = gpu_dsm_hpa & ~PCIM_BDSM_MASK;
@@ -646,15 +668,14 @@ passthru_gpu_dsm_opregion(struct vmctx *ctx, struct passthru_dev *ptdev,
 		pci_set_cfgdata32(ptdev->dev, PCIR_BDSM, gpu_dsm_gpa | dsm_mask_val);
 
 		ptdev->has_virt_pcicfg_regs = &has_virt_pcicfg_regs_on_def_gpu;
-		break;
 	}
 
 	gpu_opregion_gpa = gpu_dsm_gpa - GPU_OPREGION_SIZE;
 	pci_set_cfgdata32(ptdev->dev, PCIR_ASLS_CTL, gpu_opregion_gpa | (opregion_phys & ~PCIM_ASLS_OPREGION_MASK));
 
 	/* initialize the EPT mapping for passthrough GPU dsm region */
-	vm_unmap_ptdev_mmio(ctx, 0, 2, 0, gpu_dsm_gpa, GPU_DSM_SIZE, gpu_dsm_hpa);
-	vm_map_ptdev_mmio(ctx, 0, 2, 0, gpu_dsm_gpa, GPU_DSM_SIZE, gpu_dsm_hpa);
+	vm_unmap_ptdev_mmio(ctx, 0, 2, 0, gpu_dsm_gpa, gpu_dsm_size, gpu_dsm_hpa);
+	vm_map_ptdev_mmio(ctx, 0, 2, 0, gpu_dsm_gpa, gpu_dsm_size, gpu_dsm_hpa);
 
 	/* initialize the EPT mapping for passthrough GPU opregion */
 	vm_unmap_ptdev_mmio(ctx, 0, 2, 0, gpu_opregion_gpa, GPU_OPREGION_SIZE, gpu_opregion_hpa);
@@ -693,22 +714,25 @@ parse_vmsix_on_msi_bar_id(char *s, int *id, int base)
 static int
 passthru_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 {
-	int bus, slot, func, idx, error;
+	int bus, slot, func, idx, irq, error;
 	struct passthru_dev *ptdev;
 	struct pci_device_iterator *iter;
 	struct pci_device *phys_dev;
-	char *opt;
+	char *opt, *s_irq;
 	bool keep_gsi = false;
 	bool need_reset = true;
 	bool d3hot_reset = false;
 	bool enable_ptm = false;
+	bool enable_irq = false;
 	int vrp_sec_bus = 0;
 	int vmsix_on_msi_bar_id = -1;
 	struct acrn_pcidev pcidev = {};
 	uint16_t vendor = 0, device = 0;
 	uint8_t class = 0;
 	char rom_file[256];
+	char dsdt_path[256];
 	bool need_rombar = false;
+	bool need_dsdt = false;
 
 	ptdev = NULL;
 	error = -EINVAL;
@@ -725,6 +749,7 @@ passthru_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 	}
 
 	memset(rom_file, 0, sizeof(rom_file));
+	memset(dsdt_path, 0, sizeof(dsdt_path));
 	while ((opt = strsep(&opts, ",")) != NULL) {
 		if (!strncmp(opt, "keep_gsi", 8))
 			keep_gsi = true;
@@ -744,7 +769,30 @@ passthru_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 		} else if (!strncmp(opt, "romfile=", 8)) {
 			need_rombar = true;
 			opt += 8;
-			strcpy(rom_file, opt);
+			if (strnlen(opt, PATH_MAX) >= sizeof(rom_file)) {
+				pr_err("romfile path too long, max supported path length is 255");
+				return -EINVAL;
+			}
+			strncpy(rom_file, opt, sizeof(rom_file));
+		} else if (!strncmp(opt, "irq=", 4)) {
+			if(dm_strtoi(opt + 4, &s_irq, 10, &irq) == 0) {
+				enable_irq = true;
+				pr_warn("IRQ %d might be shared by multiple devices!\n", irq);
+			}
+			else {
+				pr_err("Input IRQ number cannnot be recognized.\n");
+				return -EINVAL;
+			}
+		} else if (!strncmp(opt, "dsdt=", 5)) {
+			need_dsdt = true;
+			opt += 5;
+			if (strlen(opt) >= sizeof(dsdt_path)) {
+				pr_err("dsdt file path too long, max supported path length is %d\n",
+					sizeof(dsdt_path)-1);
+				return -EINVAL;
+			}
+			strncpy(dsdt_path, opt, sizeof(dsdt_path) - 1);
+			pr_info("dsdt file path is %s\n", dsdt_path);
 		} else
 			pr_warn("Invalid passthru options:%s", opt);
 	}
@@ -868,7 +916,12 @@ passthru_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 		/* Allocates the virq if ptdev only support INTx */
 		pci_lintr_request(dev);
 
-		ptdev->phys_pin = read_config(ptdev->phys_dev, PCIR_INTLINE, 1);
+		if(enable_irq) {
+			ptdev->phys_pin = irq;
+		}
+		else {
+			ptdev->phys_pin = read_config(ptdev->phys_dev, PCIR_INTLINE, 1);
+		}
 
 		if (ptdev->phys_pin == -1 || ptdev->phys_pin > 256) {
 			pr_err("ptdev %x/%x/%x has wrong phys_pin %d, likely fail!",
@@ -878,6 +931,10 @@ passthru_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 		}
 	}
 
+	ptdev->need_dsdt = need_dsdt;
+	memset(ptdev->dsdt_path, 0, sizeof(ptdev->dsdt_path));
+	strncpy(ptdev->dsdt_path, dsdt_path, sizeof(ptdev->dsdt_path) - 1);
+
 	if (enable_ptm) {
 		error = ptm_probe(ctx, ptdev, &vrp_sec_bus);
 
@@ -946,7 +1003,7 @@ passthru_deinit(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 		phys_bdf = ptdev->phys_bdf;
 
 	if (is_intel_graphics_dev(dev)) {
-		vm_unmap_ptdev_mmio(ctx, 0, 2, 0, gpu_dsm_gpa, GPU_DSM_SIZE, gpu_dsm_hpa);
+		vm_unmap_ptdev_mmio(ctx, 0, 2, 0, gpu_dsm_gpa, gpu_dsm_size, gpu_dsm_hpa);
 		vm_unmap_ptdev_mmio(ctx, 0, 2, 0, gpu_opregion_gpa, GPU_OPREGION_SIZE, gpu_opregion_hpa);
 	}
 
@@ -955,13 +1012,10 @@ passthru_deinit(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 	pciaccess_cleanup();
 	free(ptdev);
 
-	if (!is_rtvm) {
-		/* Let the HV to deassign the pt device for RTVM, In this case, the RTVM
-		 * could still be alive if DM died.
-		 */
-		vm_deassign_pcidev(ctx, &pcidev);
-	}
-	if (!is_rtvm && phys_bdf) {
+	/*Let device model to deassign pt device for all VMs, including RTVM if the Service VM plays 
+         *supervisor role.*/
+	vm_deassign_pcidev(ctx, &pcidev);
+	if (phys_bdf) {
 		memset(reset_path, 0, sizeof(reset_path));
 		snprintf(reset_path, 40,
 			"/sys/bus/pci/devices/0000:%02x:%02x.%x/reset",
@@ -1784,9 +1838,38 @@ write_dsdt_tsn(struct pci_vdev *dev, uint16_t device)
 	dsdt_line("");
 }
 
+static void
+write_dsdt_file(struct pci_vdev *dev)
+{
+	struct passthru_dev *ptdev = NULL;
+	FILE *fp;
+	char *line = NULL;
+	char *dsdt_path = NULL;
+	size_t len = 0;
+	ssize_t read;
+
+	ptdev = (struct passthru_dev *) dev->arg;
+	dsdt_path = ptdev->dsdt_path;
+	fp = fopen(dsdt_path, "r");
+	if (fp == NULL) {
+		pr_err("Cannot open dsdt file %s", dsdt_path);
+		return;
+	}
+
+	dsdt_line("");
+	/* Read each line of dsdt file */
+	while ((read = getline(&line, &len, fp)) != -1) {
+		dsdt_line(line);
+	}
+	if (line)
+		free(line);
+	fclose(fp);
+}
+
 static void
 passthru_write_dsdt(struct pci_vdev *dev)
 {
+	struct passthru_dev *ptdev = NULL;
 	uint16_t vendor = 0, device = 0;
 
 	vendor = pci_get_cfgdata16(dev, PCIR_VENDOR);
@@ -1795,6 +1878,7 @@ passthru_write_dsdt(struct pci_vdev *dev)
 		return;
 
 	device = pci_get_cfgdata16(dev, PCIR_DEVICE);
+	ptdev = (struct passthru_dev *) dev->arg;
 
 	/* Provides ACPI extra info */
 	if (device == 0x5aaa)
@@ -1817,6 +1901,9 @@ passthru_write_dsdt(struct pci_vdev *dev)
 		write_dsdt_sdc(dev);
 	else if ((device == 0x4b32) || (device == 0x4ba0) || (device == 0x4bb0))
 		write_dsdt_tsn(dev, device);
+	else if (ptdev->need_dsdt)
+		/* load DSDT by input file */
+                write_dsdt_file(dev);
 }
 
 struct pci_vdev_ops passthru = {

devicemodel/hw/pci/virtio/vhost_vsock.c

@@ -298,14 +298,16 @@ virtio_vhost_vsock_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 	virtio_set_modern_bar(&vsock->base, false);
 
 	vsock->vhost_vsock = vhost_vsock_init(&vsock->base, 0);
+	if (!vsock->vhost_vsock) {
+		pr_err("vhost vosck init failed.");
+		free(vsock);
+		return -1;
+	}
 	vhost_vsock_set_guest_cid(&vsock->vhost_vsock->vdev, vsock->config.guest_cid);
 
 	if (virtio_interrupt_init(&vsock->base, virtio_uses_msix())) {
-		if (vsock) {
-			if (vsock->vhost_vsock)
-				vhost_vsock_deinit(vsock->vhost_vsock);
-			free(vsock);
-		}
+		vhost_vsock_deinit(vsock->vhost_vsock);
+		free(vsock);
 		return -1;
 	}
 	return 0;

devicemodel/hw/pci/virtio/virtio.c

@@ -66,13 +66,19 @@ void iothread_handler(void *arg)
 	struct virtio_base *base = viothrd->base;
 	int idx = viothrd->idx;
 	struct virtio_vq_info *vq = &base->queues[idx];
+	eventfd_t val;
+
+	/* Mitigate the epoll_wait repeat cycles by reading out the event */
+	if (eventfd_read(vq->viothrd.iomvt.fd, &val) == -1) {
+		pr_err("%s: eventfd_read fails \r\n", __func__);
+		return;
+	}
 
 	if (viothrd->iothread_run) {
-		if (base->mtx)
-			pthread_mutex_lock(base->mtx);
+		pthread_mutex_lock(&vq->mtx);
+		/* only vq specific data can be accessed in qnotify callback */
 		(*viothrd->iothread_run)(base, vq);
-		if (base->mtx)
-			pthread_mutex_unlock(base->mtx);
+		pthread_mutex_unlock(&vq->mtx);
 	}
 }
 
@@ -106,12 +112,12 @@ virtio_set_iothread(struct virtio_base *base,
 			vq->viothrd.iomvt.run = iothread_handler;
 			vq->viothrd.iomvt.fd = vq->viothrd.kick_fd;
 
-			if (!iothread_add(vq->viothrd.kick_fd, &vq->viothrd.iomvt))
+			if (!iothread_add(vq->viothrd.ioctx, vq->viothrd.kick_fd, &vq->viothrd.iomvt))
 				if (!virtio_register_ioeventfd(base, idx, true, vq->viothrd.kick_fd))
 					vq->viothrd.ioevent_started = true;
 		} else {
 			if (!virtio_register_ioeventfd(base, idx, false, vq->viothrd.kick_fd))
-				if (!iothread_del(vq->viothrd.kick_fd)) {
+				if (!iothread_del(vq->viothrd.ioctx, vq->viothrd.kick_fd)) {
 					vq->viothrd.ioevent_started = false;
 					if (vq->viothrd.kick_fd) {
 						close(vq->viothrd.kick_fd);
@@ -187,8 +193,6 @@ virtio_poll_timer(void *arg, uint64_t nexp)
  * @param pci_virtio_dev Pointer to instance of certain virtio device.
  * @param dev Pointer to struct pci_vdev which emulates a PCI device.
  * @param queues Pointer to struct virtio_vq_info, normally an array.
- *
- * @return None
  */
 void
 virtio_linkup(struct virtio_base *base, struct virtio_ops *vops,
@@ -196,13 +200,27 @@ virtio_linkup(struct virtio_base *base, struct virtio_ops *vops,
 	      struct virtio_vq_info *queues,
 	      int backend_type)
 {
-	int i;
+	int i, rc;
+	pthread_mutexattr_t attr;
 
 	/* base and pci_virtio_dev addresses must match */
 	if ((void *)base != pci_virtio_dev) {
 		pr_err("virtio_base and pci_virtio_dev addresses don't match!\n");
 		return;
 	}
+
+	rc = pthread_mutexattr_init(&attr);
+	if (rc) {
+		pr_err("%s, pthread_mutexattr_init failed\n", __func__);
+		return;
+	}
+
+	rc = pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE);
+	if (rc) {
+		pr_err("%s, pthread_mutexattr_settype failed\n", __func__);
+		return;
+	}
+
 	base->vops = vops;
 	base->dev = dev;
 	dev->arg = base;
@@ -212,6 +230,11 @@ virtio_linkup(struct virtio_base *base, struct virtio_ops *vops,
 	for (i = 0; i < vops->nvq; i++) {
 		queues[i].base = base;
 		queues[i].num = i;
+		rc = pthread_mutex_init(&queues[i].mtx, &attr);
+		if (rc) {
+			pr_err("%s, pthread_mutex_init failed\n", __func__);
+			return;
+		}
 	}
 }
 
@@ -226,8 +249,6 @@ virtio_linkup(struct virtio_base *base, struct virtio_ops *vops,
  * If MSI-X is enabled, this also resets all the vectors to NO_VECTOR.
  *
  * @param base Pointer to struct virtio_base.
- *
- * @return None
  */
 void
 virtio_reset_dev(struct virtio_base *base)
@@ -275,8 +296,6 @@ virtio_reset_dev(struct virtio_base *base)
  *
  * @param base Pointer to struct virtio_base.
  * @param barnum Which BAR[0..5] to use.
- *
- * @return None
  */
 void
 virtio_set_io_bar(struct virtio_base *base, int barnum)
@@ -765,8 +784,6 @@ vq_endchains(struct virtio_vq_info *vq, int used_all_avail)
  *
  * @param base Pointer to struct virtio_base.
  * @param vq Pointer to struct virtio_vq_info.
- *
- * @return None
  */
 void vq_clear_used_ring_flags(struct virtio_base *base, struct virtio_vq_info *vq)
 {
@@ -1929,8 +1946,6 @@ virtio_pci_read(struct vmctx *ctx, int vcpu, struct pci_vdev *dev,
  * @param offset Register offset in bytes within a BAR region.
  * @param size Access range in bytes.
  * @param value Data value to be written into register.
- *
- * @return None
  */
 void
 virtio_pci_write(struct vmctx *ctx, int vcpu, struct pci_vdev *dev,
@@ -1998,12 +2013,12 @@ int virtio_register_ioeventfd(struct virtio_base *base, int idx, bool is_registe
 	int rc = 0;
 
 	if (!is_register)
-		ioeventfd.flags = ACRN_IOEVENTFD_FLAG_DEASSIGN;
-	else if (base->iothread)
+		ioeventfd.flags |= ACRN_IOEVENTFD_FLAG_DEASSIGN;
+	if (base->iothread)
 		/* Enable ASYNCIO by default. If ASYNCIO is not supported by kernel
 		 * or hyperviosr, this flag will be ignored.
 		 */
-		ioeventfd.flags = ACRN_IOEVENTFD_FLAG_ASYNCIO;
+		ioeventfd.flags |= ACRN_IOEVENTFD_FLAG_ASYNCIO;
 	/* register ioeventfd for kick */
 	if (base->device_caps & (1UL << VIRTIO_F_VERSION_1)) {
 		/*

devicemodel/hw/pci/virtio/virtio_block.c

@@ -61,7 +61,7 @@
 
 /* Device can toggle its cache between writeback and writethrough modes */
 #define	VIRTIO_BLK_F_CONFIG_WCE	(1 << 11)
-
+#define	VIRTIO_BLK_F_MQ		(1 << 12)	/* support more than one vq */
 #define	VIRTIO_BLK_F_DISCARD	(1 << 13)
 
 /*
@@ -101,8 +101,8 @@ struct virtio_blk_config {
 	} topology;
 	uint8_t	writeback;
 	uint8_t unused;
-	/* Reserve for num_queues when VIRTIO_BLK_F_MQ is support*/
-	uint16_t reserve;
+	/* num_queues when VIRTIO_BLK_F_MQ is support*/
+	uint16_t num_queues;
 	/* The maximum discard sectors (in 512-byte sectors) for one segment */
 	uint32_t max_discard_sectors;
 	/* The maximum number of discard segments */
@@ -156,13 +156,16 @@ struct virtio_blk_ioreq {
 struct virtio_blk {
 	struct virtio_base base;
 	pthread_mutex_t mtx;
-	struct virtio_vq_info vq;
+	struct virtio_vq_info *vqs;
 	struct virtio_blk_config cfg;
 	bool dummy_bctxt; /* Used in blockrescan. Indicate if the bctxt can be used */
 	struct blockif_ctxt *bc;
 	char ident[VIRTIO_BLK_BLK_ID_BYTES + 1];
-	struct virtio_blk_ioreq ios[VIRTIO_BLK_RINGSZ];
+	struct virtio_blk_ioreq *ios;
 	uint8_t original_wce;
+	int num_vqs;
+	struct iothreads_info iothrds_info;
+	struct virtio_ops ops;
 };
 
 static void virtio_blk_reset(void *);
@@ -170,18 +173,6 @@ static void virtio_blk_notify(void *, struct virtio_vq_info *);
 static int virtio_blk_cfgread(void *, int, int, uint32_t *);
 static int virtio_blk_cfgwrite(void *, int, int, uint32_t);
 
-static struct virtio_ops virtio_blk_ops = {
-	"virtio_blk",		/* our name */
-	1,			/* we support 1 virtqueue */
-	sizeof(struct virtio_blk_config), /* config reg size */
-	virtio_blk_reset,	/* reset */
-	virtio_blk_notify,	/* device-wide qnotify */
-	virtio_blk_cfgread,	/* read PCI config */
-	virtio_blk_cfgwrite,	/* write PCI config */
-	NULL,			/* apply negotiated features */
-	NULL,			/* called on guest set status */
-};
-
 static void
 virtio_blk_reset(void *vdev)
 {
@@ -199,6 +190,7 @@ virtio_blk_done(struct blockif_req *br, int err)
 {
 	struct virtio_blk_ioreq *io = br->param;
 	struct virtio_blk *blk = io->blk;
+	struct virtio_vq_info *vq = blk->vqs + br->qidx;
 
 	if (err)
 		DPRINTF(("virtio_blk: done with error = %d\n\r", err));
@@ -215,10 +207,10 @@ virtio_blk_done(struct blockif_req *br, int err)
 	 * Return the descriptor back to the host.
 	 * We wrote 1 byte (our status) to host.
 	 */
-	pthread_mutex_lock(&blk->mtx);
-	vq_relchain(&blk->vq, io->idx, 1);
-	vq_endchains(&blk->vq, !vq_has_descs(&blk->vq));
-	pthread_mutex_unlock(&blk->mtx);
+	pthread_mutex_lock(&vq->mtx);
+	vq_relchain(vq, io->idx, 1);
+	vq_endchains(vq, !vq_has_descs(vq));
+	pthread_mutex_unlock(&vq->mtx);
 }
 
 static void
@@ -235,13 +227,14 @@ virtio_blk_proc(struct virtio_blk *blk, struct virtio_vq_info *vq)
 {
 	struct virtio_blk_hdr *vbh;
 	struct virtio_blk_ioreq *io;
-	int i, n;
+	int i, n, qidx;
 	int err;
 	ssize_t iolen;
 	int writeop, type;
 	struct iovec iov[BLOCKIF_IOV_MAX + 2];
 	uint16_t idx, flags[BLOCKIF_IOV_MAX + 2];
 
+	qidx = vq - blk->vqs;
 	idx = vq->qsize;
 	n = vq_getchain(vq, &idx, iov, BLOCKIF_IOV_MAX + 2, flags);
 
@@ -259,7 +252,7 @@ virtio_blk_proc(struct virtio_blk *blk, struct virtio_vq_info *vq)
 		return;
 	}
 
-	io = &blk->ios[idx];
+	io = &blk->ios[qidx * VIRTIO_BLK_RINGSZ + idx];
 	if ((flags[0] & VRING_DESC_F_WRITE) != 0) {
 		WPRINTF(("%s: the type for hdr should not be VRING_DESC_F_WRITE\n", __func__));
 		virtio_blk_abort(vq, idx);
@@ -420,6 +413,9 @@ virtio_blk_get_caps(struct virtio_blk *blk, bool wb)
 	if (blockif_is_ro(blk->bc))
 		caps |= VIRTIO_BLK_F_RO;
 
+	if (blk->num_vqs > 1)
+		caps |= VIRTIO_BLK_F_MQ;
+
 	return caps;
 }
 
@@ -447,6 +443,7 @@ virtio_blk_update_config_space(struct virtio_blk *blk)
 	    (sto != 0) ? ((sts - sto) / sectsz) : 0;
 	blk->cfg.topology.min_io_size = 0;
 	blk->cfg.writeback = blockif_get_wce(blk->bc);
+	blk->cfg.num_queues = (uint16_t)blk->num_vqs;
 	blk->original_wce = blk->cfg.writeback; /* save for reset */
 	if (blockif_candiscard(blk->bc)) {
 		blk->cfg.max_discard_sectors = blockif_max_discard_sectors(blk->bc);
@@ -456,6 +453,18 @@ virtio_blk_update_config_space(struct virtio_blk *blk)
 	blk->base.device_caps =
 		virtio_blk_get_caps(blk, !!blk->cfg.writeback);
 }
+
+static void
+virtio_blk_init_ops(struct virtio_blk *blk, int num_vqs)
+{
+	blk->ops.name = "virtio_blk";
+	blk->ops.nvq = num_vqs;
+	blk->ops.cfgsize = sizeof(struct virtio_blk_config);
+	blk->ops.reset = virtio_blk_reset;
+	blk->ops.cfgread = virtio_blk_cfgread;
+	blk->ops.cfgwrite = virtio_blk_cfgwrite;
+}
+
 static int
 virtio_blk_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 {
@@ -468,14 +477,21 @@ virtio_blk_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 	u_char digest[16];
 	struct virtio_blk *blk;
 	bool use_iothread;
-	int i;
+	struct iothread_ctx *ioctx_base = NULL;
+	struct iothreads_info iothrds_info;
+	int num_vqs;
+	int i, j;
 	pthread_mutexattr_t attr;
 	int rc;
+	struct iothreads_option iot_opt;
+
+	memset(&iot_opt, 0, sizeof(iot_opt));
 
 	bctxt = NULL;
 	/* Assume the bctxt is valid, until identified otherwise */
 	dummy_bctxt = false;
 	use_iothread = false;
+	num_vqs = 1;
 
 	if (opts == NULL) {
 		pr_err("virtio_blk: backing device required\n");
@@ -501,17 +517,75 @@ virtio_blk_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 		return -1;
 	}
 	if (strstr(opts, "nodisk") == NULL) {
-		opt = strsep(&opts_tmp, ",");
-		if (strcmp("iothread", opt) == 0) {
-			use_iothread = true;
-		} else {
-			/* The opts_start is truncated by strsep, opts_tmp is also
-			 * changed by strsetp, so use opts which points to the
-			 * original parameter string
-			 */
-			opts_tmp = opts;
+		/*
+		 * both ",iothread" and ",mq=int" are consumed by virtio-blk
+		 * and must be specified before any other opts which will
+		 * be used by blockif_open.
+		 */
+		char *p = opts_start;
+		while (opts_tmp != NULL) {
+			opt = strsep(&opts_tmp, ",");
+
+			if (!strncmp(opt, "iothread", strlen("iothread"))) {
+				use_iothread = true;
+				strsep(&opt, "=");
+
+				if (iothread_parse_options(opt, &iot_opt) < 0) {
+					free(opts_start);
+					return -1;
+				}
+
+				p = opts_tmp;
+			} else if (!strncmp(opt, "mq", strlen("mq"))) {
+				strsep(&opt, "=");
+				if (opt != NULL) {
+					if (dm_strtoi(opt, &opt, 10, &num_vqs) ||
+						(num_vqs <= 0)) {
+						WPRINTF(("%s: incorrect num queues %s\n",
+							__func__, opt));
+						free(opts_start);
+						return -1;
+					}
+					/* the max vq number allowed by FE is guest cpu num */
+					if (num_vqs > guest_cpu_num())
+						num_vqs = guest_cpu_num();
+				}
+				p = opts_tmp;
+			} else {
+				/* The opts_start is truncated by strsep, opts_tmp is also
+				 * changed by strsetp, so use opts which points to the
+				 * original parameter string
+				 */
+				p = opts + (p - opts_start);
+				break;
+			}
 		}
-		bctxt = blockif_open(opts_tmp, bident);
+
+		if (use_iothread) {
+			/*
+			 * Creating more iothread instances than the number of virtqueues is not necessary.
+			 * - One or more vqs can be handled in one iothread.
+			 * - The mapping between virtqueues and iothreads is based on round robin.
+			 */
+			if (iot_opt.num > num_vqs) {
+				iot_opt.num = num_vqs;
+			}
+
+			if (snprintf(iot_opt.tag, sizeof(iot_opt.tag), "blk%s", bident) >= sizeof(iot_opt.tag)) {
+				pr_err("%s: virtio-blk ioctx_tag too long \n", __func__);
+			}
+
+			ioctx_base = iothread_create(&iot_opt);
+			iothread_free_options(&iot_opt);
+			if (ioctx_base == NULL) {
+				pr_err("%s: Fails to create iothread context instance \n", __func__);
+				return -1;
+			}
+		}
+		iothrds_info.ioctx_base = ioctx_base;
+		iothrds_info.num = iot_opt.num;
+
+		bctxt = blockif_open(p, bident, num_vqs, &iothrds_info);
 		if (bctxt == NULL) {
 			pr_err("Could not open backing file");
 			free(opts_start);
@@ -530,17 +604,39 @@ virtio_blk_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 		return -1;
 	}
 
+	blk->iothrds_info.ioctx_base = ioctx_base;
+	blk->iothrds_info.num = iot_opt.num;
+
 	blk->bc = bctxt;
 	/* Update virtio-blk device struct of dummy ctxt*/
 	blk->dummy_bctxt = dummy_bctxt;
 
-	for (i = 0; i < VIRTIO_BLK_RINGSZ; i++) {
-		struct virtio_blk_ioreq *io = &blk->ios[i];
+	blk->num_vqs = num_vqs;
+	blk->vqs = calloc(blk->num_vqs, sizeof(struct virtio_vq_info));
+	if (!blk->vqs) {
+		WPRINTF(("virtio_blk: calloc vqs returns NULL\n"));
+		free(blk);
+		return -1;
+	}
+	blk->ios = calloc(blk->num_vqs * VIRTIO_BLK_RINGSZ,
+		sizeof(struct virtio_blk_ioreq));
+	if (!blk->ios) {
+		WPRINTF(("virtio_blk: calloc ios returns NULL\n"));
+		free(blk->vqs);
+		free(blk);
+		return -1;
+	}
 
-		io->req.callback = virtio_blk_done;
-		io->req.param = io;
-		io->blk = blk;
-		io->idx = i;
+	for (j = 0; j < num_vqs; j++) {
+		for (i = 0; i < VIRTIO_BLK_RINGSZ; i++) {
+			struct virtio_blk_ioreq *io = &blk->ios[j * VIRTIO_BLK_RINGSZ + i];
+
+			io->req.callback = virtio_blk_done;
+			io->req.param = io;
+			io->req.qidx = j;
+			io->blk = blk;
+			io->idx = i;
+		}
 	}
 
 	/* init mutex attribute properly to avoid deadlock */
@@ -557,13 +653,20 @@ virtio_blk_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 		DPRINTF(("virtio_blk: pthread_mutex_init failed with "
 					"error %d!\n", rc));
 
+	virtio_blk_init_ops(blk, num_vqs);
+
 	/* init virtio struct and virtqueues */
-	virtio_linkup(&blk->base, &virtio_blk_ops, blk, dev, &blk->vq, BACKEND_VBSU);
+	virtio_linkup(&blk->base, &(blk->ops), blk, dev, blk->vqs, BACKEND_VBSU);
 	blk->base.iothread = use_iothread;
 	blk->base.mtx = &blk->mtx;
 
-	blk->vq.qsize = VIRTIO_BLK_RINGSZ;
-	/* blk->vq.vq_notify = we have no per-queue notify */
+	for (j = 0; j < num_vqs; j++) {
+		blk->vqs[j].qsize = VIRTIO_BLK_RINGSZ;
+		blk->vqs[j].notify = virtio_blk_notify;
+		if (use_iothread) {
+			blk->vqs[j].viothrd.ioctx = ioctx_base + j % (iot_opt.num);
+		}
+	}
 
 	/*
 	 * Create an identifier for the backing file. Use parts of the
@@ -645,6 +748,10 @@ virtio_blk_deinit(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 			blockif_close(bctxt);
 		}
 		virtio_reset_dev(&blk->base);
+		if (blk->ios)
+			free(blk->ios);
+		if (blk->vqs)
+			free(blk->vqs);
 		free(blk);
 	}
 }
@@ -744,7 +851,7 @@ virtio_blk_rescan(struct vmctx *ctx, struct pci_vdev *dev, char *newpath)
 
 	pr_err("name=%s, Path=%s, ident=%s\n", dev->name, newpath, bident);
 	/* update the bctxt for the virtio-blk device */
-	bctxt = blockif_open(newpath, bident);
+	bctxt = blockif_open(newpath, bident, blk->num_vqs, &blk->iothrds_info);
 	if (bctxt == NULL) {
 		pr_err("Error opening backing file\n");
 		goto end;

devicemodel/hw/pci/virtio/virtio_gpu.c

@@ -702,6 +702,12 @@ virtio_gpu_cmd_resource_create_2d(struct virtio_gpu_command *cmd)
 	}
 	r2d = (struct virtio_gpu_resource_2d*)calloc(1, \
 			sizeof(struct virtio_gpu_resource_2d));
+	if (!r2d) {
+		pr_err("%s: memory allocation for r2d failed.\n", __func__);
+		resp.type = VIRTIO_GPU_RESP_ERR_OUT_OF_MEMORY;
+		goto response;
+	}
+
 	r2d->resource_id = req.resource_id;
 	r2d->width = req.width;
 	r2d->height = req.height;
@@ -774,15 +780,42 @@ virtio_gpu_cmd_resource_attach_backing(struct virtio_gpu_command *cmd)
 	struct virtio_gpu_ctrl_hdr resp;
 	int i;
 	uint8_t *pbuf;
+	struct iovec *iov;
 
 	memcpy(&req, cmd->iov[0].iov_base, sizeof(req));
 	memset(&resp, 0, sizeof(resp));
 
+	/*
+	 * 1. Per VIRTIO GPU specification,
+	 *    'cmd->iovcnt' = 'nr_entries' of 'struct virtio_gpu_resource_attach_backing' + 2,
+	 *    where 'nr_entries' is number of instance of 'struct virtio_gpu_mem_entry'.
+	 *    case 'cmd->iovcnt < 3' means above 'nr_entries' is zero, which is invalid
+	 *    and ignored.
+	 *    2. Function 'virtio_gpu_ctrl_bh(void *data)' guarantees cmd->iovcnt >=1.
+	 */
+	if (cmd->iovcnt < 2) {
+		resp.type = VIRTIO_GPU_RESP_ERR_INVALID_PARAMETER;
+		memcpy(cmd->iov[cmd->iovcnt - 1].iov_base, &resp, sizeof(resp));
+		pr_err("%s : invalid memory entry.\n", __func__);
+		return;
+	}
+
 	r2d = virtio_gpu_find_resource_2d(cmd->gpu, req.resource_id);
-	if (r2d) {
-		r2d->iov = malloc(req.nr_entries * sizeof(struct iovec));
+	if (r2d && req.nr_entries > 0) {
+		iov = malloc(req.nr_entries * sizeof(struct iovec));
+		if (!iov) {
+			resp.type = VIRTIO_GPU_RESP_ERR_OUT_OF_MEMORY;
+			goto exit;
+		}
+
+		r2d->iov = iov;
 		r2d->iovcnt = req.nr_entries;
-		entries = malloc(req.nr_entries * sizeof(struct virtio_gpu_mem_entry));
+		entries = calloc(req.nr_entries, sizeof(struct virtio_gpu_mem_entry));
+		if (!entries) {
+			free(iov);
+			resp.type = VIRTIO_GPU_RESP_ERR_OUT_OF_MEMORY;
+			goto exit;
+		}
 		pbuf = (uint8_t*)entries;
 		for (i = 1; i < (cmd->iovcnt - 1); i++) {
 			memcpy(pbuf, cmd->iov[i].iov_base, cmd->iov[i].iov_len);
@@ -796,13 +829,13 @@ virtio_gpu_cmd_resource_attach_backing(struct virtio_gpu_command *cmd)
 			r2d->iov[i].iov_len = entries[i].length;
 		}
 		free(entries);
+		resp.type = VIRTIO_GPU_RESP_OK_NODATA;
 	} else {
 		pr_err("%s: Illegal resource id %d\n", __func__, req.resource_id);
 		resp.type = VIRTIO_GPU_RESP_ERR_INVALID_RESOURCE_ID;
 	}
-
+exit:
 	cmd->iolen = sizeof(resp);
-	resp.type = VIRTIO_GPU_RESP_OK_NODATA;
 	virtio_gpu_update_resp_fence(&cmd->hdr, &resp);
 	memcpy(cmd->iov[cmd->iovcnt - 1].iov_base, &resp, sizeof(resp));
 }
@@ -1166,6 +1199,7 @@ virtio_gpu_cmd_create_blob(struct virtio_gpu_command *cmd)
 	struct virtio_gpu_ctrl_hdr resp;
 	int i;
 	uint8_t *pbuf;
+	struct iovec *iov;
 
 	memcpy(&req, cmd->iov[0].iov_base, sizeof(req));
 	cmd->iolen = sizeof(resp);
@@ -1177,7 +1211,19 @@ virtio_gpu_cmd_create_blob(struct virtio_gpu_command *cmd)
 		resp.type = VIRTIO_GPU_RESP_ERR_INVALID_RESOURCE_ID;
 		memcpy(cmd->iov[cmd->iovcnt - 1].iov_base, &resp, sizeof(resp));
 		return;
+	}
 
+	/*
+	 * 1. Per VIRTIO GPU specification,
+	 *    'cmd->iovcnt' = 'nr_entries' of 'struct virtio_gpu_resource_create_blob' + 2,
+	 *    where 'nr_entries' is number of instance of 'struct virtio_gpu_mem_entry'.
+	 *    2. Function 'virtio_gpu_ctrl_bh(void *data)' guarantees cmd->iovcnt >=1.
+	 */
+	if (cmd->iovcnt < 2) {
+		resp.type = VIRTIO_GPU_RESP_ERR_INVALID_PARAMETER;
+		memcpy(cmd->iov[cmd->iovcnt - 1].iov_base, &resp, sizeof(resp));
+		pr_err("%s : invalid memory entry.\n", __func__);
+		return;
 	}
 
 	if ((req.blob_mem != VIRTIO_GPU_BLOB_MEM_GUEST) ||
@@ -1200,48 +1246,73 @@ virtio_gpu_cmd_create_blob(struct virtio_gpu_command *cmd)
 
 	r2d = (struct virtio_gpu_resource_2d *)calloc(1,
 			sizeof(struct virtio_gpu_resource_2d));
+	if (!r2d) {
+		pr_err("%s : memory allocation for r2d failed.\n", __func__);
+		resp.type = VIRTIO_GPU_RESP_ERR_OUT_OF_MEMORY;
+		memcpy(cmd->iov[cmd->iovcnt - 1].iov_base, &resp, sizeof(resp));
+		return;
+	}
+
 	r2d->resource_id = req.resource_id;
 
-	entries = malloc(req.nr_entries * sizeof(struct virtio_gpu_mem_entry));
-	pbuf = (uint8_t *)entries;
-	for (i = 1; i < (cmd->iovcnt - 1); i++) {
-		memcpy(pbuf, cmd->iov[i].iov_base, cmd->iov[i].iov_len);
-		pbuf += cmd->iov[i].iov_len;
-	}
-	if (req.size > CURSOR_BLOB_SIZE) {
-		/* Try to create the dma buf */
-		r2d->dma_info = virtio_gpu_create_udmabuf(cmd->gpu,
-							  entries,
-							  req.nr_entries);
-		if (r2d->dma_info == NULL) {
-			free(entries);
-			resp.type = VIRTIO_GPU_RESP_ERR_UNSPEC;
+	if (req.nr_entries > 0) {
+		entries = calloc(req.nr_entries, sizeof(struct virtio_gpu_mem_entry));
+		if (!entries) {
+			pr_err("%s : memory allocation for entries failed.\n", __func__);
+			free(r2d);
+			resp.type = VIRTIO_GPU_RESP_ERR_OUT_OF_MEMORY;
 			memcpy(cmd->iov[cmd->iovcnt - 1].iov_base, &resp, sizeof(resp));
 			return;
 		}
-		r2d->blob = true;
-	} else {
-		/* Cursor resource with 64x64 and PIXMAN_a8r8g8b8 format.
-		 * Or when it fails to create dmabuf
-		 */
-		r2d->width = 64;
-		r2d->height = 64;
-		r2d->format = PIXMAN_a8r8g8b8;
-		r2d->image = pixman_image_create_bits(
-				r2d->format, r2d->width, r2d->height, NULL, 0);
-
-		r2d->iov = malloc(req.nr_entries * sizeof(struct iovec));
-		r2d->iovcnt = req.nr_entries;
-		for (i = 0; i < req.nr_entries; i++) {
-			r2d->iov[i].iov_base = paddr_guest2host(
-					cmd->gpu->base.dev->vmctx,
-					entries[i].addr,
-					entries[i].length);
-			r2d->iov[i].iov_len = entries[i].length;
+		pbuf = (uint8_t *)entries;
+		for (i = 1; i < (cmd->iovcnt - 1); i++) {
+			memcpy(pbuf, cmd->iov[i].iov_base, cmd->iov[i].iov_len);
+			pbuf += cmd->iov[i].iov_len;
 		}
-	}
+		if (req.size > CURSOR_BLOB_SIZE) {
+			/* Try to create the dma buf */
+			r2d->dma_info = virtio_gpu_create_udmabuf(cmd->gpu,
+					entries,
+					req.nr_entries);
+			if (r2d->dma_info == NULL) {
+				free(entries);
+				resp.type = VIRTIO_GPU_RESP_ERR_UNSPEC;
+				memcpy(cmd->iov[cmd->iovcnt - 1].iov_base, &resp, sizeof(resp));
+				return;
+			}
+			r2d->blob = true;
+		} else {
+			/* Cursor resource with 64x64 and PIXMAN_a8r8g8b8 format.
+			 * Or when it fails to create dmabuf
+			 */
+			r2d->width = 64;
+			r2d->height = 64;
+			r2d->format = PIXMAN_a8r8g8b8;
+			r2d->image = pixman_image_create_bits(
+					r2d->format, r2d->width, r2d->height, NULL, 0);
 
-	free(entries);
+			iov = malloc(req.nr_entries * sizeof(struct iovec));
+			if (!iov) {
+				free(entries);
+				free(r2d);
+				resp.type = VIRTIO_GPU_RESP_ERR_OUT_OF_MEMORY;
+				memcpy(cmd->iov[cmd->iovcnt - 1].iov_base, &resp, sizeof(resp));
+				return;
+			}
+			r2d->iov = iov;
+
+			r2d->iovcnt = req.nr_entries;
+			for (i = 0; i < req.nr_entries; i++) {
+				r2d->iov[i].iov_base = paddr_guest2host(
+						cmd->gpu->base.dev->vmctx,
+						entries[i].addr,
+						entries[i].length);
+				r2d->iov[i].iov_len = entries[i].length;
+			}
+		}
+
+		free(entries);
+	}
 	resp.type = VIRTIO_GPU_RESP_OK_NODATA;
 	LIST_INSERT_HEAD(&cmd->gpu->r2d_list, r2d, link);
 	memcpy(cmd->iov[cmd->iovcnt - 1].iov_base, &resp, sizeof(resp));
@@ -1552,7 +1623,7 @@ virtio_gpu_vga_render(void *param)
 	gpu->vga.surf.stride = 0;
 	/* The below logic needs to be refined */
 	while(gpu->vga.enable) {
-		if(gpu->vga.gc->gc_image->vgamode) {
+		if ((gpu->vga.gc->gc_image->vgamode) && (gpu->vga.dev != NULL)) {
 			vga_render(gpu->vga.gc, gpu->vga.dev);
 			break;
 		}
@@ -1801,6 +1872,9 @@ virtio_gpu_deinit(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 	int i;
 
 	gpu = (struct virtio_gpu *)dev->arg;
+	if (!gpu)
+		return;
+
 	gpu->vga.enable = false;
 
 	pthread_mutex_lock(&gpu->vga_thread_mtx);
@@ -1860,10 +1934,8 @@ virtio_gpu_deinit(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 
 	vdpy_deinit(gpu->vdpy_handle);
 
-	if (gpu) {
-		pthread_mutex_destroy(&gpu->mtx);
-		free(gpu);
-	}
+	pthread_mutex_destroy(&gpu->mtx);
+	free(gpu);
 	virtio_gpu_device_cnt--;
 }
 

devicemodel/hw/pci/xhci.c

@@ -384,6 +384,12 @@ struct pci_xhci_vbdp_dev_state {
 	uint8_t	state;
 };
 
+struct pci_xhci_async_request_node {
+	STAILQ_ENTRY(pci_xhci_async_request_node) link;
+	uint64_t offset;
+	uint64_t value;
+};
+
 struct pci_xhci_vdev {
 	struct pci_vdev *dev;
 	pthread_mutex_t mtx;
@@ -424,6 +430,12 @@ struct pci_xhci_vdev {
 	int		vbdp_dev_num;
 	struct pci_xhci_vbdp_dev_state vbdp_devs[XHCI_MAX_VIRT_PORTS];
 
+	pthread_t	async_thread;
+	bool	async_transfer;
+	pthread_cond_t	async_cond;
+	pthread_mutex_t	async_tmx;
+	STAILQ_HEAD(, pci_xhci_async_request_node) async_head;
+
 	/*
 	 * native_ports uses for record the command line assigned native root
 	 * hub ports and its child external hub ports.
@@ -485,6 +497,7 @@ static int pci_xhci_parse_extcap(struct pci_xhci_vdev *xdev, char *opts);
 static int pci_xhci_convert_speed(int lspeed);
 static void pci_xhci_free_usb_xfer(struct pci_xhci_dev_emu *dev, struct usb_xfer *xfer);
 static void pci_xhci_isoc_handler(void *arg, uint64_t param);
+static void pci_xhci_async_enqueue(struct pci_xhci_vdev *xdev, uint64_t offset, uint64_t value);
 
 #define XHCI_OPT_MAX_LEN 32
 static struct pci_xhci_option_elem xhci_option_table[] = {
@@ -1982,7 +1995,9 @@ pci_xhci_cmd_disable_slot(struct pci_xhci_vdev *xdev, uint32_t slot)
 				slot, di->path.bus, usb_dev_path(&di->path));
 
 		/* release all the resource allocated for virtual device */
+		pthread_mutex_unlock(&xdev->mtx);
 		pci_xhci_dev_destroy(dev);
+		pthread_mutex_lock(&xdev->mtx);
 	} else
 		UPRINTF(LWRN, "invalid slot %d\r\n", slot);
 
@@ -2056,6 +2071,7 @@ pci_xhci_cmd_address_device(struct pci_xhci_vdev *xdev,
 	struct usb_native_devinfo *di;
 	uint32_t cmderr;
 	uint8_t rh_port;
+	int ret;
 
 	input_ctx = XHCI_GADDR(xdev, trb->qwTrb0 & ~0xFUL);
 	if (!input_ctx) {
@@ -2110,7 +2126,10 @@ pci_xhci_cmd_address_device(struct pci_xhci_vdev *xdev,
 				"port %d\r\n", di->path.bus,
 				usb_dev_path(&di->path), rh_port);
 
+		pthread_mutex_unlock(&xdev->mtx);
 		dev = pci_xhci_dev_create(xdev, di);
+		pthread_mutex_lock(&xdev->mtx);
+
 		if (!dev) {
 			UPRINTF(LFTL, "fail to create device for %d-%s\r\n",
 					di->path.bus,
@@ -2142,8 +2161,15 @@ pci_xhci_cmd_address_device(struct pci_xhci_vdev *xdev,
 	dev->hci.hci_address = slot;
 	dev->dev_ctx = dev_ctx;
 
-	if (dev->dev_ue->ue_reset == NULL ||
-	    dev->dev_ue->ue_reset(dev->dev_instance) < 0) {
+	if (dev->dev_ue->ue_reset == NULL) {
+		cmderr = XHCI_TRB_ERROR_ENDP_NOT_ON;
+		goto done;
+	}
+
+	pthread_mutex_unlock(&xdev->mtx);
+	ret = dev->dev_ue->ue_reset(dev->dev_instance);
+	pthread_mutex_lock(&xdev->mtx);
+	if (ret < 0) {
 		cmderr = XHCI_TRB_ERROR_ENDP_NOT_ON;
 		goto done;
 	}
@@ -2391,6 +2417,7 @@ pci_xhci_cmd_reset_ep(struct pci_xhci_vdev *xdev,
 
 	devep = &dev->eps[epid];
 	pthread_mutex_lock(&devep->mtx);
+	pthread_mutex_unlock(&xdev->mtx);
 
 	xfer = devep->ep_xfer;
 	for (i = 0; i < xfer->max_blk_cnt; ++i) {
@@ -2413,6 +2440,7 @@ pci_xhci_cmd_reset_ep(struct pci_xhci_vdev *xdev,
 	UPRINTF(LDBG, "reset ep[%u] %08x %08x %016lx %08x\r\n",
 		epid, ep_ctx->dwEpCtx0, ep_ctx->dwEpCtx1, ep_ctx->qwEpCtx2,
 		ep_ctx->dwEpCtx4);
+	pthread_mutex_lock(&xdev->mtx);
 	pthread_mutex_unlock(&devep->mtx);
 
 done:
@@ -3022,8 +3050,10 @@ pci_xhci_try_usb_xfer(struct pci_xhci_vdev *xdev,
 
 	/* outstanding requests queued up */
 	if (dev->dev_ue->ue_data != NULL) {
+		pthread_mutex_unlock(&xdev->mtx);
 		err = dev->dev_ue->ue_data(dev->dev_instance, xfer, epid & 0x1 ?
 					   USB_XFER_IN : USB_XFER_OUT, epid/2);
+		pthread_mutex_lock(&xdev->mtx);
 		if (err == USB_ERR_CANCELLED) {
 			if (USB_DATA_GET_ERRCODE(&xfer->data[xfer->head]) ==
 			    USB_NAK)
@@ -3286,8 +3316,11 @@ retry:
 
 	if (epid == 1) {
 		err = USB_ERR_NOT_STARTED;
-		if (dev->dev_ue->ue_request != NULL)
+		if (dev->dev_ue->ue_request != NULL) {
+			pthread_mutex_unlock(&xdev->mtx);
 			err = dev->dev_ue->ue_request(dev->dev_instance, xfer);
+			pthread_mutex_lock(&xdev->mtx);
+		}
 		setup_trb = NULL;
 	} else {
 		/* handle data transfer */
@@ -3658,21 +3691,24 @@ pci_xhci_write(struct vmctx *ctx,
 
 	xdev = dev->arg;
 
-	pthread_mutex_lock(&xdev->mtx);
 	if (offset < XHCI_CAPLEN)	/* read only registers */
 		UPRINTF(LWRN, "write RO-CAPs offset %ld\r\n", offset);
-	else if (offset < xdev->dboff)
+	else if (offset < xdev->dboff) {
+		pthread_mutex_lock(&xdev->mtx);
 		pci_xhci_hostop_write(xdev, offset, value);
-	else if (offset < xdev->rtsoff)
-		pci_xhci_dbregs_write(xdev, offset, value);
-	else if (offset < xdev->rtsend)
+		pthread_mutex_unlock(&xdev->mtx);
+	} else if (offset < xdev->rtsoff) {
+		pci_xhci_async_enqueue(xdev, offset, value);
+	} else if (offset < xdev->rtsend) {
+		pthread_mutex_lock(&xdev->mtx);
 		pci_xhci_rtsregs_write(xdev, offset, value);
-	else if (offset < xdev->regsend)
+		pthread_mutex_unlock(&xdev->mtx);
+	} else if (offset < xdev->regsend) {
+		pthread_mutex_lock(&xdev->mtx);
 		pci_xhci_excap_write(xdev, offset, value);
-	else
+		pthread_mutex_unlock(&xdev->mtx);
+	} else
 		UPRINTF(LWRN, "write invalid offset %ld\r\n", offset);
-
-	pthread_mutex_unlock(&xdev->mtx);
 }
 
 static uint64_t
@@ -3901,25 +3937,29 @@ pci_xhci_read(struct vmctx *ctx,
 	uint32_t	value;
 
 	xdev = dev->arg;
-
-	pthread_mutex_lock(&xdev->mtx);
-	if (offset < XHCI_CAPLEN)
+	if (offset < XHCI_CAPLEN) {
+		pthread_mutex_lock(&xdev->mtx);
 		value = pci_xhci_hostcap_read(xdev, offset);
-	else if (offset < xdev->dboff)
+		pthread_mutex_unlock(&xdev->mtx);
+	} else if (offset < xdev->dboff) {
+		pthread_mutex_lock(&xdev->mtx);
 		value = pci_xhci_hostop_read(xdev, offset);
-	else if (offset < xdev->rtsoff)
+		pthread_mutex_unlock(&xdev->mtx);
+	} else if (offset < xdev->rtsoff) {
 		value = pci_xhci_dbregs_read(xdev, offset);
-	else if (offset < xdev->rtsend)
+	} else if (offset < xdev->rtsend) {
+		pthread_mutex_lock(&xdev->mtx);
 		value = pci_xhci_rtsregs_read(xdev, offset);
-	else if (offset < xdev->regsend)
+		pthread_mutex_unlock(&xdev->mtx);
+	} else if (offset < xdev->regsend) {
+		pthread_mutex_lock(&xdev->mtx);
 		value = pci_xhci_excap_read(xdev, offset);
-	else {
+		pthread_mutex_unlock(&xdev->mtx);
+	} else {
 		value = 0;
 		UPRINTF(LDBG, "read invalid offset %ld\r\n", offset);
 	}
 
-	pthread_mutex_unlock(&xdev->mtx);
-
 	switch (size) {
 	case 1:
 		value &= 0xFF;
@@ -4005,6 +4045,7 @@ pci_xhci_dev_intr(struct usb_hci *hci, int epctx)
 	struct xhci_endp_ctx	*ep_ctx;
 	int	dir_in;
 	int	epid;
+	int ret = 0;
 
 	dir_in = epctx & 0x80;
 	epid = epctx & ~0x80;
@@ -4018,10 +4059,11 @@ pci_xhci_dev_intr(struct usb_hci *hci, int epctx)
 	xdev = dev->xdev;
 
 	/* check if device is ready; OS has to initialise it */
+	pthread_mutex_lock(&xdev->mtx);
 	if (xdev->rtsregs.erstba_p == NULL ||
 	    (xdev->opregs.usbcmd & XHCI_CMD_RS) == 0 ||
 	    dev->dev_ctx == NULL)
-		return 0;
+		goto out;
 
 	p = XHCI_PORTREG_PTR(xdev, hci->hci_port);
 
@@ -4030,16 +4072,18 @@ pci_xhci_dev_intr(struct usb_hci *hci, int epctx)
 		p->portsc &= ~XHCI_PS_PLS_MASK;
 		p->portsc |= XHCI_PS_PLS_SET(UPS_PORT_LS_RESUME);
 		if ((p->portsc & XHCI_PS_PLC) != 0)
-			return 0;
+			goto out;
 
 		p->portsc |= XHCI_PS_PLC;
 
 		pci_xhci_set_evtrb(&evtrb, hci->hci_port,
 				   XHCI_TRB_ERROR_SUCCESS,
 				   XHCI_TRB_EVENT_PORT_STS_CHANGE);
+
 		if (pci_xhci_insert_event(xdev, &evtrb, 0) != 0) {
 			UPRINTF(LFTL, "Failed to inject port status change event!\r\n");
-			return -ENAVAIL;
+			ret = -ENAVAIL;
+			goto out;
 		}
 	}
 
@@ -4048,14 +4092,15 @@ pci_xhci_dev_intr(struct usb_hci *hci, int epctx)
 	if ((ep_ctx->dwEpCtx0 & 0x7) == XHCI_ST_EPCTX_DISABLED) {
 		UPRINTF(LWRN, "device interrupt on disabled endpoint %d\r\n",
 			 epid);
-		return 0;
+		goto out;
 	}
 
 	UPRINTF(LDBG, "device interrupt on endpoint %d\r\n", epid);
-
 	pci_xhci_device_doorbell(xdev, hci->hci_port, epid, 0);
 
-	return 0;
+out:
+	pthread_mutex_unlock(&xdev->mtx);
+	return ret;
 }
 
 static int
@@ -4127,11 +4172,6 @@ pci_xhci_parse_bus_port(struct pci_xhci_vdev *xdev, char *opts)
 		goto errout;
 	}
 
-	if (bus >= USB_NATIVE_NUM_BUS || port >= USB_NATIVE_NUM_PORT) {
-		rc = -1;
-		goto errout;
-	}
-
 	if (!usb_native_is_bus_existed(bus) ||
 			!usb_native_is_port_existed(bus, port)) {
 		rc = -2;
@@ -4418,6 +4458,55 @@ pci_xhci_isoc_handler(void *arg, uint64_t param)
 				? "under" : "over", pdata->slot, pdata->epnum);
 }
 
+static void
+pci_xhci_async_enqueue(struct pci_xhci_vdev *xdev, uint64_t offset, uint64_t value)
+{
+	struct pci_xhci_async_request_node *request;
+
+	request = malloc(sizeof(struct pci_xhci_async_request_node));
+	if (request == NULL) {
+		UPRINTF(LFTL, "%s: malloc memory fail\r\n", __func__);
+		return;
+	}
+	request->offset = offset;
+	request->value = value;
+
+	pthread_mutex_lock(&xdev->async_tmx);
+	if (STAILQ_EMPTY(&xdev->async_head)) {
+		STAILQ_INSERT_HEAD(&xdev->async_head, request, link);
+	} else {
+		STAILQ_INSERT_TAIL(&xdev->async_head, request, link);
+	}
+	pthread_cond_signal(&xdev->async_cond);
+	pthread_mutex_unlock(&xdev->async_tmx);
+}
+
+static void *
+pci_xhci_ansyc_thread(void *data)
+{
+	struct pci_xhci_vdev *xdev;
+	struct pci_xhci_async_request_node *request;
+
+	xdev = data;
+	pthread_mutex_lock(&xdev->async_tmx);
+	while (xdev->async_transfer || !STAILQ_EMPTY(&xdev->async_head)) {
+		if(STAILQ_EMPTY(&xdev->async_head))
+			pthread_cond_wait(&xdev->async_cond, &xdev->async_tmx);
+		if ((request = STAILQ_FIRST(&xdev->async_head)) == NULL) {
+			continue;
+		}
+		pthread_mutex_unlock(&xdev->async_tmx);
+		pthread_mutex_lock(&xdev->mtx);
+		pci_xhci_dbregs_write(xdev, request->offset, request->value);
+		pthread_mutex_unlock(&xdev->mtx);
+		pthread_mutex_lock(&xdev->async_tmx);
+		STAILQ_REMOVE_HEAD(&xdev->async_head, link);
+		free(request);
+	}
+	pthread_mutex_unlock(&xdev->async_tmx);
+	return NULL;
+}
+
 static int
 pci_xhci_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 {
@@ -4548,6 +4637,15 @@ pci_xhci_init(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 	if (error)
 		goto done;
 
+	xdev->async_transfer = true;
+	pthread_cond_init(&xdev->async_cond, NULL);
+	pthread_mutex_init(&xdev->async_tmx, NULL);
+	STAILQ_INIT(&xdev->async_head);
+	error = pthread_create(&xdev->async_thread, NULL, pci_xhci_ansyc_thread,
+			(void *)xdev);
+	if (error)
+		goto done;
+
 	xhci_in_use = 1;
 done:
 	if (error) {
@@ -4603,6 +4701,14 @@ pci_xhci_deinit(struct vmctx *ctx, struct pci_vdev *dev, char *opts)
 	pthread_join(xdev->vbdp_thread, NULL);
 	sem_close(&xdev->vbdp_sem);
 
+	pthread_mutex_lock(&xdev->async_tmx);
+	xdev->async_transfer = false;
+	pthread_cond_signal(&xdev->async_cond);
+	pthread_mutex_unlock(&xdev->async_tmx);
+	pthread_join(xdev->async_thread, NULL);
+	pthread_cond_destroy(&xdev->async_cond);
+	pthread_mutex_destroy(&xdev->async_tmx);
+
 	pthread_mutex_destroy(&xdev->mtx);
 	free(xdev);
 	xhci_in_use = 0;

devicemodel/hw/platform/acpi/acpi.c

@@ -93,6 +93,10 @@
 #define RTCT_OFFSET		0xF00
 #define DSDT_OFFSET		0x1100
 
+/* Define the byte offset and byte length in FACS table */
+#define WAKING_VECTOR_OFFSET   12
+#define WAKING_VECTOR_LEN      4
+
 #define	ASL_TEMPLATE	"dm.XXXXXXX"
 #define ASL_SUFFIX	".aml"
 
@@ -879,6 +883,7 @@ basl_fwrite_dsdt(FILE *fp, struct vmctx *ctx)
 
 	acpi_dev_write_dsdt(ctx);
 
+	osc_write_ospm_dsdt(ctx, basl_ncpu);
 	pm_write_dsdt(ctx, basl_ncpu);
 
 	dsdt_line("}");
@@ -1112,6 +1117,18 @@ get_acpi_table_length(void)
 	return ACPI_LENGTH;
 }
 
+uint32_t
+get_acpi_wakingvector_offset(void)
+{
+	return basl_acpi_base + FACS_OFFSET + WAKING_VECTOR_OFFSET;
+}
+
+uint32_t
+get_acpi_wakingvector_length(void)
+{
+	return WAKING_VECTOR_LEN;
+}
+
 int
 get_default_iasl_compiler(void)
 {

devicemodel/hw/platform/acpi/acpi_pm.c

@@ -22,15 +22,15 @@ static inline int get_vcpu_pm_info(struct vmctx *ctx, int vcpu_id,
 	return vm_get_cpu_state(ctx, pm_info);
 }
 
-static inline uint8_t get_vcpu_px_cnt(struct vmctx *ctx, int vcpu_id)
+static inline int get_vcpu_px_cnt(struct vmctx *ctx, int vcpu_id, uint8_t *px_cnt)
 {
-	uint64_t px_cnt;
+	uint64_t px_cnt_u64;
+	int ret;
 
-	if (get_vcpu_pm_info(ctx, vcpu_id, ACRN_PMCMD_GET_PX_CNT, &px_cnt)) {
-		return 0;
-	}
+	ret = get_vcpu_pm_info(ctx, vcpu_id, ACRN_PMCMD_GET_PX_CNT, &px_cnt_u64);
+	*px_cnt = (uint8_t)px_cnt_u64;
 
-	return (uint8_t)px_cnt;
+	return ret;
 }
 
 uint8_t get_vcpu_cx_cnt(struct vmctx *ctx, int vcpu_id)
@@ -259,9 +259,13 @@ static int dsdt_write_pss(struct vmctx *ctx, int vcpu_id)
 	uint8_t vcpu_px_cnt;
 	int i;
 	struct acrn_pstate_data *vcpu_px_data;
+	int ret;
 
-	vcpu_px_cnt = get_vcpu_px_cnt(ctx, vcpu_id);
-	if (!vcpu_px_cnt) {
+	ret = get_vcpu_px_cnt(ctx, vcpu_id, &vcpu_px_cnt);
+	/* vcpu_px_cnt = 0 Indicates vcpu supports continuous pstate.
+	 * Then we should write _CPC instate of _PSS
+	 */
+	if (ret || !vcpu_px_cnt) {
 		return -1;
 	}
 
@@ -316,10 +320,49 @@ static int dsdt_write_pss(struct vmctx *ctx, int vcpu_id)
 	return 0;
 }
 
+/* _CPC: Continuous Performance Control
+ * Hard code a V3 CPC table, describing HWP register interface.
+ */
+static void dsdt_write_cpc(void)
+{
+	dsdt_line("");
+	dsdt_line("    Method (_CPC, 0, NotSerialized)");
+	dsdt_line("    {");
+	dsdt_line("        Return (Package (0x17)");
+	dsdt_line("        {");
+	dsdt_line("            0x17,");
+	dsdt_line("            0x03,");
+	dsdt_line("            ResourceTemplate() {Register(FFixedHW, 0x08, 0x00, 0x0000000000000771, 0x04, )},");
+	dsdt_line("            ResourceTemplate() {Register(FFixedHW, 0x08, 0x08, 0x00000000000000CE, 0x04, )},");
+	dsdt_line("            ResourceTemplate() {Register(FFixedHW, 0x08, 0x10, 0x0000000000000771, 0x04, )},");
+	dsdt_line("            ResourceTemplate() {Register(FFixedHW, 0x08, 0x18, 0x0000000000000771, 0x04, )},");
+	dsdt_line("            ResourceTemplate() {Register(FFixedHW, 0x08, 0x08, 0x0000000000000771, 0x04, )},");
+	dsdt_line("            ResourceTemplate() {Register(FFixedHW, 0x08, 0x10, 0x0000000000000774, 0x04, )},");
+	dsdt_line("            ResourceTemplate() {Register(FFixedHW, 0x08, 0x00, 0x0000000000000774, 0x04, )},");
+	dsdt_line("            ResourceTemplate() {Register(FFixedHW, 0x08, 0x08, 0x0000000000000774, 0x04, )},");
+	dsdt_line("            ResourceTemplate() {Register(SystemMemory, 0x00, 0x00, 0x0000000000000000, , )},");
+	dsdt_line("            ResourceTemplate() {Register(SystemMemory, 0x00, 0x00, 0x0000000000000000, , )},");
+	dsdt_line("            ResourceTemplate() {Register(SystemMemory, 0x00, 0x00, 0x0000000000000000, , )},");
+	dsdt_line("            ResourceTemplate() {Register(FFixedHW, 0x40, 0x00, 0x00000000000000E7, 0x04, )},");
+	dsdt_line("            ResourceTemplate() {Register(FFixedHW, 0x40, 0x00, 0x00000000000000E8, 0x04, )},");
+	dsdt_line("            ResourceTemplate() {Register(FFixedHW, 0x02, 0x01, 0x0000000000000777, 0x04, )},");
+	dsdt_line("            ResourceTemplate() {Register(FFixedHW, 0x01, 0x00, 0x0000000000000770, 0x04, )},");
+	dsdt_line("            One,");
+	dsdt_line("            ResourceTemplate() {Register(FFixedHW, 0x0A, 0x20, 0x0000000000000774, 0x04, )},");
+	dsdt_line("            ResourceTemplate() {Register(FFixedHW, 0x08, 0x18, 0x0000000000000774, 0x04, )},");
+	dsdt_line("            Zero,");
+	dsdt_line("            Zero,");
+	dsdt_line("            Zero");
+	dsdt_line("        })");
+	dsdt_line("    }");
+}
+
 void pm_write_dsdt(struct vmctx *ctx, int ncpu)
 {
 	int i;
 	int ret;
+	bool is_cpc = false;
+	uint8_t px_cnt;
 
 	/* Scope (_PR) */
 	dsdt_line("");
@@ -364,6 +407,85 @@ void pm_write_dsdt(struct vmctx *ctx, int ncpu)
 			}
 		}
 
+		ret = get_vcpu_px_cnt(ctx, i, &px_cnt);
+		if (ret == 0 && px_cnt == 0) {
+			/* px_cnt = 0 Indicates vcpu supports continuous pstate.
+			 * Then we can write _CPC
+			 */
+			is_cpc = true;
+		}
+
+		if (is_cpc) {
+			if (i == 0) {
+				dsdt_write_cpc();
+			} else {
+				dsdt_line("    Method (_CPC, 0, NotSerialized)");
+				dsdt_line("    {");
+				dsdt_line("      Return (^^PR00._CPC)");
+				dsdt_line("    }");
+				dsdt_line("");
+			}
+		}
+
 		dsdt_line("  }");
 	}
 }
+
+/* _OSC: Operating System Capabilities
+ * Currently only support CPPC v2 capability.
+ * CPPC v2 capability: revision 2 of the _CPC object.
+ * If all vcpus don't support _CPC object, no need to add _OSC in DSDT.
+ */
+void osc_write_ospm_dsdt(struct vmctx *ctx, int ncpu)
+{
+	int ret;
+	bool support_cpc = false;
+	uint8_t px_cnt;
+
+	/* check px_cnt on vBSP */
+	ret = get_vcpu_px_cnt(ctx, 0, &px_cnt);
+	if (ret == 0 && px_cnt == 0) {
+		/* px_cnt = 0 Indicates vcpu supports continuous pstate.
+		 */
+		support_cpc = true;
+	}
+	if (support_cpc) {
+		/* Scope (_SB._OSC) */
+		dsdt_line("");
+		dsdt_line("  Scope (_SB)");
+		dsdt_line("  {");
+		dsdt_line("    Method (_OSC, 4, NotSerialized) // _OSC: Operating System Capabilities");
+		dsdt_line("    {");
+		dsdt_line("      CreateDWordField (Arg3, 0x00, STS0)");
+		dsdt_line("      CreateDWordField (Arg3, 0x04, CAP0)");
+		dsdt_line("      If ((Arg0 == ToUUID (\"0811b06e-4a27-44f9-8d60-3cbbc22e7b48\") /* Platform-wide OSPM Capabilities */))");
+		dsdt_line("      {");
+		dsdt_line("        If ((Arg1 == One))");
+		dsdt_line("        {");
+		dsdt_line("          If ((CAP0 & 0x40))");
+		dsdt_line("          {");
+		dsdt_line("            CAP0 &= 0x00000040");
+		dsdt_line("          }");
+		dsdt_line("          Else");
+		dsdt_line("          {");
+		dsdt_line("            STS0 &= 0xFFFFFF00");
+		dsdt_line("            STS0 |= 0x02");
+		dsdt_line("          }");
+		dsdt_line("        }");
+		dsdt_line("        Else");
+		dsdt_line("        {");
+		dsdt_line("          STS0 &= 0xFFFFFF00");
+		dsdt_line("          STS0 |= 0x0A");
+		dsdt_line("        }");
+		dsdt_line("      }");
+		dsdt_line("      Else");
+		dsdt_line("      {");
+		dsdt_line("        STS0 &= 0xFFFFFF00");
+		dsdt_line("        STS0 |= 0x06");
+		dsdt_line("      }");
+		dsdt_line("      Return (Arg3)");
+		dsdt_line("    }");
+		dsdt_line("  }");
+		dsdt_line("");
+	}
+}

devicemodel/hw/platform/hpet.c

@@ -202,7 +202,7 @@ vhpet_counter(struct vhpet *vhpet, struct timespec *nowptr)
 	val = vhpet->countbase;
 
 	if (vhpet_counter_enabled(vhpet)) {
-		if (clock_gettime(CLOCK_REALTIME, &now))
+		if (clock_gettime(CLOCK_MONOTONIC, &now))
 			pr_dbg("clock_gettime returned: %s", strerror(errno));
 
 		/* delta = now - countbase_ts */
@@ -225,7 +225,7 @@ vhpet_counter(struct vhpet *vhpet, struct timespec *nowptr)
 		 */
 		if (nowptr) {
 			pr_warn("vhpet unexpected nowptr");
-			if (clock_gettime(CLOCK_REALTIME, nowptr))
+			if (clock_gettime(CLOCK_MONOTONIC, nowptr))
 				pr_dbg("clock_gettime returned: %s", strerror(errno));
 		}
 	}
@@ -366,7 +366,7 @@ vhpet_timer_handler(void *a, uint64_t nexp)
 
 	vhpet_timer_interrupt(vhpet, n);
 
-	if (clock_gettime(CLOCK_REALTIME, &now))
+	if (clock_gettime(CLOCK_MONOTONIC, &now))
 		pr_dbg("clock_gettime returned: %s", strerror(errno));
 
 	if (acrn_timer_gettime(vhpet_tmr(vhpet, n), &tmrts))
@@ -548,7 +548,7 @@ vhpet_start_counting(struct vhpet *vhpet)
 {
 	int i;
 
-	if (clock_gettime(CLOCK_REALTIME, &vhpet->countbase_ts))
+	if (clock_gettime(CLOCK_MONOTONIC, &vhpet->countbase_ts))
 		pr_dbg("clock_gettime returned: %s", strerror(errno));
 
 	/* Restart the timers based on the main counter base value */
@@ -639,7 +639,7 @@ vhpet_timer_update_config(struct vhpet *vhpet, int n, uint64_t data,
 			 *   - Timer remains in periodic mode
 			 */
 			if (!vhpet_timer_enabled(vhpet, n)) {
-				if (clock_gettime(CLOCK_REALTIME, &now))
+				if (clock_gettime(CLOCK_MONOTONIC, &now))
 					pr_dbg("clock_gettime returned: %s", strerror(errno));
 				vhpet_stop_timer(vhpet, n, &now, true);
 			} else if (!(oldval & (HPET_TCNF_TYPE | HPET_TCNF_INT_ENB)) ||
@@ -998,7 +998,7 @@ vhpet_init(struct vmctx *ctx)
 			arg->timer_num = i;
 
 			tmr = &vhpet->timer[i].tmrlst[j].t;
-			tmr->clockid = CLOCK_REALTIME;
+			tmr->clockid = CLOCK_MONOTONIC;
 			error = acrn_timer_init(tmr, vhpet_timer_handler, arg);
 
 			if (error) {

devicemodel/hw/platform/pit.c

@@ -107,7 +107,7 @@ ticks_elapsed_since(const struct timespec *since)
 {
 	struct timespec ts;
 
-	if (clock_gettime(CLOCK_REALTIME, &ts))
+	if (clock_gettime(CLOCK_MONOTONIC, &ts))
 		pr_dbg("clock_gettime returned: %s", strerror(errno));
 
 	if (timespeccmp(&ts, since, <=))
@@ -192,7 +192,7 @@ pit_load_ce(struct channel *c)
 		c->nullcnt = false;
 		c->crbyte = 0;
 
-		if (clock_gettime(CLOCK_REALTIME, &c->start_ts))
+		if (clock_gettime(CLOCK_MONOTONIC, &c->start_ts))
 			pr_dbg("clock_gettime returned: %s", strerror(errno));
 
 		if (c->initial == 0 || c->initial > 0x10000) {
@@ -330,7 +330,7 @@ pit_timer_start_cntr0(struct vpit *vpit)
 	sigevt.sigev_notify = SIGEV_THREAD;
 	sigevt.sigev_notify_function = vpit_timer_handler;
 
-	if (timer_create(CLOCK_REALTIME, &sigevt, &c->timer_id))
+	if (timer_create(CLOCK_MONOTONIC, &sigevt, &c->timer_id))
 		pr_dbg("timer_create returned: %s", strerror(errno));
 
 	vpit_timer_arg[c->timer_idx].active = true;
@@ -360,7 +360,7 @@ pit_update_counter(struct vpit *vpit, struct channel *c, bool latch,
 
 		c->initial = PIT_HZ_TO_TICKS(100);
 		delta_ticks = 0;
-		if (clock_gettime(CLOCK_REALTIME, &c->start_ts))
+		if (clock_gettime(CLOCK_MONOTONIC, &c->start_ts))
 			pr_dbg("clock_gettime returned: %s", strerror(errno));
 	} else
 		delta_ticks = ticks_elapsed_since(&c->start_ts);

devicemodel/hw/platform/rtc.c

@@ -41,6 +41,7 @@
 #include "timer.h"
 #include "acpi.h"
 #include "lpc.h"
+#include "vm_event.h"
 
 #include "log.h"
 
@@ -80,6 +81,7 @@ struct vrtc {
 	u_int		addr;               /* RTC register to read or write */
 	time_t		base_uptime;
 	time_t		base_rtctime;
+	time_t		halted_rtctime;
 	struct rtcdev	rtcdev;
 };
 
@@ -220,6 +222,17 @@ update_enabled(struct vrtc *vrtc)
 	return true;
 }
 
+/* monotonic time is number of seconds that the system has been running
+ * since it was booted. It is none setable. It is more suitable to be used
+ * as base time.
+ */
+static time_t monotonic_time(void)
+{
+	struct timespec ts;
+	clock_gettime(CLOCK_MONOTONIC, &ts);
+	return ts.tv_sec;
+}
+
 static time_t
 vrtc_curtime(struct vrtc *vrtc, time_t *basetime)
 {
@@ -229,7 +242,7 @@ vrtc_curtime(struct vrtc *vrtc, time_t *basetime)
 	t = vrtc->base_rtctime;
 	*basetime = vrtc->base_uptime;
 	if (update_enabled(vrtc)) {
-		now = time(NULL);
+		now = monotonic_time();
 		delta = now - vrtc->base_uptime;
 		secs = delta;
 		t += secs;
@@ -714,6 +727,18 @@ vrtc_set_reg_c(struct vrtc *vrtc, uint8_t newval)
 	}
 }
 
+static void
+send_rtc_chg_event(time_t newtime, time_t lasttime)
+{
+	struct vm_event event;
+	struct rtc_change_event_data *data = (struct rtc_change_event_data *)event.event_data;
+
+	event.type = VM_EVENT_RTC_CHG;
+	data->delta_time = newtime - lasttime;
+	data->last_time = lasttime;
+	dm_send_vm_event(&event);
+}
+
 static int
 vrtc_set_reg_b(struct vrtc *vrtc, uint8_t newval)
 {
@@ -736,16 +761,23 @@ vrtc_set_reg_b(struct vrtc *vrtc, uint8_t newval)
 	if (changed & RTCSB_HALT) {
 		if ((newval & RTCSB_HALT) == 0) {
 			rtctime = rtc_to_secs(vrtc);
-			basetime = time(NULL);
+			basetime = monotonic_time();
 			if (rtctime == VRTC_BROKEN_TIME) {
 				if (rtc_flag_broken_time)
 					return -1;
+			} else {
+				/* send rtc change event if rtc time changed during halt */
+				if (vrtc->halted_rtctime != VRTC_BROKEN_TIME &&
+					rtctime != vrtc->halted_rtctime) {
+					send_rtc_chg_event(rtctime, vrtc->halted_rtctime);
+					vrtc->halted_rtctime = VRTC_BROKEN_TIME;
+				}
 			}
 		} else {
 			curtime = vrtc_curtime(vrtc, &basetime);
 			if (curtime != vrtc->base_rtctime)
 				return -1;
-
+			vrtc->halted_rtctime = curtime;
 			/*
 			 * Force a refresh of the RTC date/time fields so
 			 * they reflect the time right before the guest set
@@ -810,7 +842,7 @@ vrtc_set_reg_a(struct vrtc *vrtc, uint8_t newval)
 		 * maintain the illusion that the RTC date/time was frozen
 		 * while the dividers were disabled.
 		 */
-		vrtc->base_uptime = time(NULL);
+		vrtc->base_uptime = monotonic_time();
 		RTC_DEBUG("RTC divider out of reset at %#lx/%#lx\n",
 				vrtc->base_rtctime, vrtc->base_uptime);
 	} else {
@@ -882,6 +914,12 @@ vrtc_addr_handler(struct vmctx *ctx, int vcpu, int in, int port,
 	return 0;
 }
 
+static inline bool vrtc_is_time_register(uint32_t offset)
+{
+	return ((offset == RTC_SEC) || (offset == RTC_MIN) || (offset == RTC_HRS) || (offset == RTC_DAY)
+			|| (offset == RTC_MONTH) || (offset == RTC_YEAR) || (offset == RTC_CENTURY));
+}
+
 int
 vrtc_data_handler(struct vmctx *ctx, int vcpu, int in, int port,
 		  int bytes, uint32_t *eax, void *arg)
@@ -960,14 +998,23 @@ vrtc_data_handler(struct vmctx *ctx, int vcpu, int in, int port,
 		}
 
 		/*
-		 * XXX some guests (e.g. OpenBSD) write the century byte
-		 * outside of RTCSB_HALT so re-calculate the RTC date/time.
+		 * Some guests (e.g. OpenBSD) write the century byte outside of RTCSB_HALT,
+		 * and some guests (e.g. WaaG) write all date/time outside of RTCSB_HALT,
+		 * so re-calculate the RTC date/time.
 		 */
-		if (offset == RTC_CENTURY && !rtc_halted(vrtc)) {
+		if (vrtc_is_time_register(offset) && !rtc_halted(vrtc)) {
+			time_t last_time = curtime;
 			curtime = rtc_to_secs(vrtc);
-			error = vrtc_time_update(vrtc, curtime, time(NULL));
-			if ((error != 0) || (curtime == VRTC_BROKEN_TIME && rtc_flag_broken_time))
+			error = vrtc_time_update(vrtc, curtime, monotonic_time());
+			if ((error != 0) || (curtime == VRTC_BROKEN_TIME && rtc_flag_broken_time)) {
 				error = -1;
+			} else {
+				/* We don't know when the Guest has finished the RTC change action.
+				 * So send an event each time the date/time regs has been updated.
+				 * The event handler will process those events.
+				 */
+				send_rtc_chg_event(rtc_to_secs(vrtc), last_time);
+			}
 		}
 	}
 
@@ -982,7 +1029,7 @@ vrtc_set_time(struct vrtc *vrtc, time_t secs)
 	int error;
 
 	pthread_mutex_lock(&vrtc->mtx);
-	error = vrtc_time_update(vrtc, secs, time(NULL));
+	error = vrtc_time_update(vrtc, secs, monotonic_time());
 	pthread_mutex_unlock(&vrtc->mtx);
 
 	if (error)
@@ -993,6 +1040,17 @@ vrtc_set_time(struct vrtc *vrtc, time_t secs)
 	return error;
 }
 
+/* set CMOS shutdown status register (index 0xF) as S3_resume(0xFE)
+ * BIOS will read it and start S3 resume at POST Entry
+ */
+void vrtc_suspend(struct vmctx *ctx)
+{
+	struct vrtc *vrtc = ctx->vrtc;
+	struct rtcdev *rtc = &vrtc->rtcdev;
+
+	*((uint8_t *)rtc + 0xF) = 0xFE;
+}
+
 int
 vrtc_init(struct vmctx *ctx)
 {
@@ -1087,6 +1145,7 @@ vrtc_init(struct vmctx *ctx)
 	/* Reset the index register to a safe value. */
 	vrtc->addr = RTC_STATUSD;
 
+	vrtc->halted_rtctime = VRTC_BROKEN_TIME;
 	/*
 	 * Initialize RTC time to 00:00:00 Jan 1, 1970 if curtime = 0
 	 */
@@ -1095,16 +1154,16 @@ vrtc_init(struct vmctx *ctx)
 
 	pthread_mutex_lock(&vrtc->mtx);
 	vrtc->base_rtctime = VRTC_BROKEN_TIME;
-	vrtc_time_update(vrtc, curtime, time(NULL));
+	vrtc_time_update(vrtc, curtime, monotonic_time());
 	secs_to_rtc(curtime, vrtc, 0);
 	pthread_mutex_unlock(&vrtc->mtx);
 
 	/* init periodic interrupt timer */
-	vrtc->periodic_timer.clockid = CLOCK_REALTIME;
+	vrtc->periodic_timer.clockid = CLOCK_MONOTONIC;
 	acrn_timer_init(&vrtc->periodic_timer, vrtc_periodic_timer, vrtc);
 
 	/* init update interrupt timer(1s)*/
-	vrtc->update_timer.clockid = CLOCK_REALTIME;
+	vrtc->update_timer.clockid = CLOCK_MONOTONIC;
 	acrn_timer_init(&vrtc->update_timer, vrtc_update_timer, vrtc);
 	vrtc_start_timer(&vrtc->update_timer, 1, 0);
 

devicemodel/hw/uart_core.c

@@ -29,6 +29,7 @@
 
 #include <stdio.h>
 #include <stdlib.h>
+#include <signal.h>
 #include <fcntl.h>
 #include <termios.h>
 #include <unistd.h>
@@ -52,6 +53,12 @@
 #define COM1_IRQ	4
 #define	COM2_BASE	0x2F8
 #define COM2_IRQ	3
+#define COM3_BASE	0x3E8
+#define COM3_IRQ	6
+#define COM4_BASE	0x2E8
+#define COM4_IRQ	7
+#define COM5_BASE       0x9000   /*for S5 connection*/
+#define COM5_IRQ        10
 
 #define	DEFAULT_RCLK	1843200
 #define	DEFAULT_BAUD	9600
@@ -83,8 +90,13 @@ static struct {
 } uart_lres[] = {
 	{ COM1_BASE, COM1_IRQ, false},
 	{ COM2_BASE, COM2_IRQ, false},
+	{ COM3_BASE, COM3_IRQ, false},
+	{ COM4_BASE, COM4_IRQ, false},
+	{ COM5_BASE, COM5_IRQ, false},
 };
 
+static bool stdio_ctrl_a_pressed = false;
+
 #define	UART_NLDEVS	(ARRAY_SIZE(uart_lres))
 
 enum uart_be_type {
@@ -618,6 +630,18 @@ uart_legacy_alloc(int which, int *baseaddr, int *irq)
 	return 0;
 }
 
+int 
+uart_legacy_reinit_res(int which, int baseaddr, int irq)
+{
+	if (which < 0 || which >= UART_NLDEVS || uart_lres[which].inuse)
+		return -1;
+
+	uart_lres[which].baseaddr = baseaddr;
+	uart_lres[which].irq = irq;
+
+	return 0;
+}
+
 void
 uart_legacy_dealloc(int which)
 {
@@ -696,6 +720,18 @@ uart_backend_read(struct uart_backend *be)
 
 	switch (be->be_type) {
 	case UART_BE_STDIO:
+		rc = read(be->fd, &rb, 1);
+		if (rb == 0x01) {  // Ctrl-a
+			DPRINTF(("%s: Got Ctrl-a\n", __func__));
+			stdio_ctrl_a_pressed = true;
+		} else if (stdio_ctrl_a_pressed) {
+			if (rb == 'x') {
+				DPRINTF(("%s: Got Ctrl-a x\n", __func__));
+				kill(getpid(), SIGINT);
+			}
+			stdio_ctrl_a_pressed = false;
+		}
+		break;
 	case UART_BE_TTY:
 		/* fd is used to read */
 		rc = read(be->fd, &rb, 1);

devicemodel/hw/vdisplay_sdl.c

@@ -1369,13 +1369,16 @@ int vdpy_parse_cmd_option(const char *opts)
 
 	error = 0;
 	vdpy.vscrs = calloc(VSCREEN_MAX_NUM, sizeof(struct vscreen));
+	if (!vdpy.vscrs) {
+		pr_err("%s, memory allocation for vscrs failed.", __func__);
+		return -1;
+	}
 	vdpy.vscrs_num = 0;
 
 	stropts = strdup(opts);
 	while ((str = strsep(&stropts, ",")) != NULL) {
 		vscr = vdpy.vscrs + vdpy.vscrs_num;
-		tmp = strcasestr(str, "geometry=");
-		if (str && strcasestr(str, "geometry=fullscreen")) {
+		if ((tmp = strcasestr(str, "geometry=fullscreen")) != NULL) {
 			snum = sscanf(tmp, "geometry=fullscreen:%d", &vscr->pscreen_id);
 			if (snum != 1) {
 				vscr->pscreen_id = 0;
@@ -1388,7 +1391,7 @@ int vdpy_parse_cmd_option(const char *opts)
 			pr_info("virtual display: fullscreen on monitor %d.\n",
 					vscr->pscreen_id);
 			vdpy.vscrs_num++;
-		} else if (str && strcasestr(str, "geometry=")) {
+		} else if ((tmp = strcasestr(str, "geometry=")) != NULL) {
 			snum = sscanf(tmp, "geometry=%dx%d+%d+%d",
 					&vscr->guest_width, &vscr->guest_height,
 					&vscr->org_x, &vscr->org_y);

devicemodel/hw/vga.c

@@ -1291,6 +1291,10 @@ vga_init(struct gfx_ctx *gc, int io_only)
 	int port, error;
 
 	vd = calloc(1, sizeof(struct vga_vdev));
+	if (!vd) {
+		pr_err("%s: out of memory.\n", __func__);
+		return NULL;
+	}
 
 	bzero(&iop, sizeof(struct inout_port));
 	iop.name = "VGA";
@@ -1326,8 +1330,12 @@ vga_init(struct gfx_ctx *gc, int io_only)
 		return NULL;
 	}
 
-	vd->vga_ram = malloc(256 * KB);
-	memset(vd->vga_ram, 0, 256 * KB);
+	vd->vga_ram = calloc(256, KB);
+	if (!vd->vga_ram) {
+		pr_err("%s: failed to allocate vga_ram.\n", __func__);
+		free(vd);
+		return NULL;
+	}
 
 	{
 		static uint8_t palette[] = {

devicemodel/include/acpi.h

@@ -99,6 +99,8 @@ struct acpi_madt_local_apic {
 void acpi_table_enable(int num);
 uint32_t get_acpi_base(void);
 uint32_t get_acpi_table_length(void);
+uint32_t get_acpi_wakingvector_offset(void);
+uint32_t get_acpi_wakingvector_length(void);
 
 struct vmctx;
 
@@ -125,4 +127,6 @@ int acrn_parse_iasl(char *arg);
 int get_iasl_compiler(void);
 int check_iasl_version(void);
 
+void osc_write_ospm_dsdt(struct vmctx *ctx, int ncpu);
+
 #endif /* _ACPI_H_ */

devicemodel/include/block_if.h

@@ -39,19 +39,68 @@
 #include <sys/uio.h>
 #include <sys/unistd.h>
 
+#include "iothread.h"
+
 #define BLOCKIF_IOV_MAX		256	/* not practical to be IOV_MAX */
 
+/*
+ *  |<------------------------------------- bounced_size --------------------------------->|
+ *  |<-------- alignment ------->|                            |<-------- alignment ------->|
+ *  |<--- head --->|<------------------------ org_size ---------------------->|<-- tail -->|
+ *  |              |             |                            |               |            |
+ *  *--------------$-------------*----------- ... ------------*---------------$------------*
+ *  |              |             |                            |               |            |
+ *  |              start                                                      end          |
+ *  aligned_dn_start                                          aligned_dn_end
+ *  |__________head_area_________|                            |__________tail_area_________|
+ *  |<--- head --->|             |                            |<-- end_rmd -->|<-- tail -->|
+ *  |<-------- alignment ------->|                            |<-------- alignment ------->|
+ *
+ */
+struct br_align_info {
+	uint32_t	alignment;
+
+	bool		is_iov_base_aligned;
+	bool		is_iov_len_aligned;
+	bool		is_offset_aligned;
+
+	/*
+	 * Needs to convert the misaligned request to an aligned one when
+	 * O_DIRECT is used, but the request (either buffer address/length, or offset) is not aligned.
+	 */
+	bool		need_conversion;
+
+	uint32_t	head;
+	uint32_t	tail;
+	uint32_t	org_size;
+	uint32_t	bounced_size;
+
+	off_t		aligned_dn_start;
+	off_t		aligned_dn_end;
+
+	/*
+	 * A bounce_iov for aligned read/write access.
+	 * bounce_iov.iov_base is aligned to @alignment
+	 * bounce_iov.iov_len is @bounced_size (@head + @org_size + @tail)
+	 */
+	struct iovec	bounce_iov;
+};
+
 struct blockif_req {
-	struct iovec	iov[BLOCKIF_IOV_MAX];
-	int		iovcnt;
-	off_t		offset;
-	ssize_t		resid;
-	void		(*callback)(struct blockif_req *req, int err);
-	void		*param;
+	struct iovec		iov[BLOCKIF_IOV_MAX];
+	int			iovcnt;
+	off_t			offset;
+	ssize_t			resid;
+	void			(*callback)(struct blockif_req *req, int err);
+	void			*param;
+	int			qidx;
+
+	struct br_align_info	align_info;
 };
 
 struct blockif_ctxt;
-struct blockif_ctxt *blockif_open(const char *optstr, const char *ident);
+struct blockif_ctxt *blockif_open(const char *optstr, const char *ident, int queue_num,
+	struct iothreads_info *iothrds_info);
 off_t	blockif_size(struct blockif_ctxt *bc);
 void	blockif_chs(struct blockif_ctxt *bc, uint16_t *c, uint8_t *h,
 		    uint8_t *s);

devicemodel/include/dm.h

@@ -30,6 +30,8 @@
 #define	_DM_H_
 
 #include <stdbool.h>
+#include <sys/resource.h>
+
 #include "types.h"
 #include "dm_string.h"
 #include "acrn_common.h"
@@ -51,6 +53,15 @@ extern bool pt_tpm2;
 extern bool ssram;
 extern bool vtpm2;
 extern bool is_winvm;
+extern bool ovmf_loaded;
+
+enum acrn_thread_prio {
+	PRIO_VCPU = PRIO_MIN,
+	PRIO_IOTHREAD = PRIO_MIN,
+	PRIO_VIRTIO_SND,
+	PRIO_VIRTIO_IPU,
+	PRIO_VIRTIO_GPU
+};
 
 /**
  * @brief Convert guest physical address to host virtual address
@@ -63,7 +74,9 @@ extern bool is_winvm;
  */
 void *paddr_guest2host(struct vmctx *ctx, uintptr_t gaddr, size_t len);
 int  virtio_uses_msix(void);
+int  guest_cpu_num(void);
 size_t high_bios_size(void);
 void init_debugexit(void);
 void deinit_debugexit(void);
+void set_thread_priority(int priority, bool reset_on_fork);
 #endif

devicemodel/include/igd_pciids.h

@@ -0,0 +1,273 @@
+/*
+ * Copyright (C) 2023 Intel Corporation.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sub license, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the
+ * next paragraph) shall be included in all copies or substantial portions
+ * of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ * 
+ * Device ids are available in linux kernel source tree and Intel website.
+ * https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/drm/i915_pciids.h
+ * https://dgpu-docs.intel.com/devices/hardware-table.html
+ */
+
+#ifndef _IGD_PCIIDS_H_
+#define _IGD_PCIIDS_H_
+
+#include <stdint.h>
+
+struct igd_device {
+	uint16_t device;
+	int gen;
+};
+
+#define IGD_DEVICE_ENTRY(id, gen)	\
+	{ id, gen }
+
+
+/* Skylake, Gen 9 */
+#define IGD_SKL_DEVICE_IDS	\
+	IGD_DEVICE_ENTRY(0x1906, 9),	\
+	IGD_DEVICE_ENTRY(0x1913, 9),	\
+	IGD_DEVICE_ENTRY(0x190E, 9),	\
+	IGD_DEVICE_ENTRY(0x1915, 9),	\
+	IGD_DEVICE_ENTRY(0x1902, 9),	\
+	IGD_DEVICE_ENTRY(0x190A, 9),	\
+	IGD_DEVICE_ENTRY(0x190B, 9),	\
+	IGD_DEVICE_ENTRY(0x1917, 9),	\
+	IGD_DEVICE_ENTRY(0x1916, 9),	\
+	IGD_DEVICE_ENTRY(0x1921, 9),	\
+	IGD_DEVICE_ENTRY(0x191E, 9),	\
+	IGD_DEVICE_ENTRY(0x1912, 9),	\
+	IGD_DEVICE_ENTRY(0x191A, 9),	\
+	IGD_DEVICE_ENTRY(0x191B, 9),	\
+	IGD_DEVICE_ENTRY(0x191D, 9),	\
+	IGD_DEVICE_ENTRY(0x1923, 9),	\
+	IGD_DEVICE_ENTRY(0x1926, 9),	\
+	IGD_DEVICE_ENTRY(0x1927, 9),	\
+	IGD_DEVICE_ENTRY(0x192A, 9),	\
+	IGD_DEVICE_ENTRY(0x192B, 9),	\
+	IGD_DEVICE_ENTRY(0x192D, 9),	\
+	IGD_DEVICE_ENTRY(0x1932, 9),	\
+	IGD_DEVICE_ENTRY(0x193A, 9),	\
+	IGD_DEVICE_ENTRY(0x193B, 9),	\
+	IGD_DEVICE_ENTRY(0x193D, 9)
+
+/* Apollo Lake, Gen 9 */
+#define IGD_BXT_DEVICE_IDS	\
+	IGD_DEVICE_ENTRY(0x0A84, 9),	\
+	IGD_DEVICE_ENTRY(0x1A84, 9),	\
+	IGD_DEVICE_ENTRY(0x1A85, 9),	\
+	IGD_DEVICE_ENTRY(0x5A84, 9),	\
+	IGD_DEVICE_ENTRY(0x5A85, 9)
+
+/* Gemini Lake, Gen 9 */
+#define IGD_GLK_DEVICE_IDS	\
+	IGD_DEVICE_ENTRY(0x3184, 9),	\
+	IGD_DEVICE_ENTRY(0x3185, 9)
+
+/* Kaby Lake, Gen 9 */
+#define IGD_KBL_DEVICE_IDS	\
+	IGD_DEVICE_ENTRY(0x5906, 9),	\
+	IGD_DEVICE_ENTRY(0x5913, 9),	\
+	IGD_DEVICE_ENTRY(0x590E, 9),	\
+	IGD_DEVICE_ENTRY(0x5915, 9),	\
+	IGD_DEVICE_ENTRY(0x5902, 9),	\
+	IGD_DEVICE_ENTRY(0x5908, 9),	\
+	IGD_DEVICE_ENTRY(0x590A, 9),	\
+	IGD_DEVICE_ENTRY(0x590B, 9),	\
+	IGD_DEVICE_ENTRY(0x5916, 9),	\
+	IGD_DEVICE_ENTRY(0x5921, 9),	\
+	IGD_DEVICE_ENTRY(0x591E, 9),	\
+	IGD_DEVICE_ENTRY(0x5912, 9),	\
+	IGD_DEVICE_ENTRY(0x5917, 9),	\
+	IGD_DEVICE_ENTRY(0x591A, 9),	\
+	IGD_DEVICE_ENTRY(0x591B, 9),	\
+	IGD_DEVICE_ENTRY(0x591D, 9),	\
+	IGD_DEVICE_ENTRY(0x5926, 9),	\
+	IGD_DEVICE_ENTRY(0x5923, 9),	\
+	IGD_DEVICE_ENTRY(0x5927, 9),	\
+	IGD_DEVICE_ENTRY(0x593B, 9),	\
+	IGD_DEVICE_ENTRY(0x591C, 9),	\
+	IGD_DEVICE_ENTRY(0x87C0, 9)
+
+/* Coffee Lake/Comet Lake, Gen 9 */
+#define IGD_CFL_DEVICE_IDS	\
+	IGD_DEVICE_ENTRY(0x87CA, 9),	\
+	IGD_DEVICE_ENTRY(0x9BA2, 9),	\
+	IGD_DEVICE_ENTRY(0x9BA4, 9),	\
+	IGD_DEVICE_ENTRY(0x9BA5, 9),	\
+	IGD_DEVICE_ENTRY(0x9BA8, 9),	\
+	IGD_DEVICE_ENTRY(0x9B21, 9),	\
+	IGD_DEVICE_ENTRY(0x9BAA, 9),	\
+	IGD_DEVICE_ENTRY(0x9BAC, 9),	\
+	IGD_DEVICE_ENTRY(0x9BC2, 9),	\
+	IGD_DEVICE_ENTRY(0x9BC4, 9),	\
+	IGD_DEVICE_ENTRY(0x9BC5, 9),	\
+	IGD_DEVICE_ENTRY(0x9BC6, 9),	\
+	IGD_DEVICE_ENTRY(0x9BC8, 9),	\
+	IGD_DEVICE_ENTRY(0x9BE6, 9),	\
+	IGD_DEVICE_ENTRY(0x9BF6, 9),	\
+	IGD_DEVICE_ENTRY(0x9B41, 9),	\
+	IGD_DEVICE_ENTRY(0x9BCA, 9),	\
+	IGD_DEVICE_ENTRY(0x9BCC, 9),	\
+	IGD_DEVICE_ENTRY(0x3E90, 9),	\
+	IGD_DEVICE_ENTRY(0x3E93, 9),	\
+	IGD_DEVICE_ENTRY(0x3E99, 9),	\
+	IGD_DEVICE_ENTRY(0x3E91, 9),	\
+	IGD_DEVICE_ENTRY(0x3E92, 9),	\
+	IGD_DEVICE_ENTRY(0x3E96, 9),	\
+	IGD_DEVICE_ENTRY(0x3E98, 9),	\
+	IGD_DEVICE_ENTRY(0x3E9A, 9),	\
+	IGD_DEVICE_ENTRY(0x3E9C, 9),	\
+	IGD_DEVICE_ENTRY(0x3E94, 9),	\
+	IGD_DEVICE_ENTRY(0x3E9B, 9),	\
+	IGD_DEVICE_ENTRY(0x3EA9, 9),	\
+	IGD_DEVICE_ENTRY(0x3EA5, 9),	\
+	IGD_DEVICE_ENTRY(0x3EA6, 9),	\
+	IGD_DEVICE_ENTRY(0x3EA7, 9),	\
+	IGD_DEVICE_ENTRY(0x3EA8, 9),	\
+	IGD_DEVICE_ENTRY(0x3EA1, 9),	\
+	IGD_DEVICE_ENTRY(0x3EA4, 9),	\
+	IGD_DEVICE_ENTRY(0x3EA0, 9),	\
+	IGD_DEVICE_ENTRY(0x3EA3, 9),	\
+	IGD_DEVICE_ENTRY(0x3EA2, 9)
+
+/* Ice Lake, Gen 11 */
+#define IGD_ICL_DEVICE_IDS	\
+	IGD_DEVICE_ENTRY(0x8A50, 11),	\
+	IGD_DEVICE_ENTRY(0x8A52, 11),	\
+	IGD_DEVICE_ENTRY(0x8A53, 11),	\
+	IGD_DEVICE_ENTRY(0x8A54, 11),	\
+	IGD_DEVICE_ENTRY(0x8A56, 11),	\
+	IGD_DEVICE_ENTRY(0x8A57, 11),	\
+	IGD_DEVICE_ENTRY(0x8A58, 11),	\
+	IGD_DEVICE_ENTRY(0x8A59, 11),	\
+	IGD_DEVICE_ENTRY(0x8A5A, 11),	\
+	IGD_DEVICE_ENTRY(0x8A5B, 11),	\
+	IGD_DEVICE_ENTRY(0x8A5C, 11),	\
+	IGD_DEVICE_ENTRY(0x8A70, 11),	\
+	IGD_DEVICE_ENTRY(0x8A71, 11),	\
+	IGD_DEVICE_ENTRY(0x8A51, 11),	\
+	IGD_DEVICE_ENTRY(0x8A5D, 11)
+
+/* Elkhart Lake, Gen 12 */
+#define IGD_EHL_DEVICE_IDS	\
+	IGD_DEVICE_ENTRY(0x4541, 12),	\
+	IGD_DEVICE_ENTRY(0x4551, 12),	\
+	IGD_DEVICE_ENTRY(0x4555, 12),	\
+	IGD_DEVICE_ENTRY(0x4557, 12),	\
+	IGD_DEVICE_ENTRY(0x4570, 12),	\
+	IGD_DEVICE_ENTRY(0x4571, 12)
+
+/* Jasper Lake, Gen 12 */
+#define IGD_JSL_DEVICE_IDS	\
+	IGD_DEVICE_ENTRY(0x4E51, 12),	\
+	IGD_DEVICE_ENTRY(0x4E55, 12),	\
+	IGD_DEVICE_ENTRY(0x4E57, 12),	\
+	IGD_DEVICE_ENTRY(0x4E61, 12),	\
+	IGD_DEVICE_ENTRY(0x4E71, 12)
+
+/* Tiger Lake, Gen 12 */
+#define IGD_TGL_DEVICE_IDS	\
+	IGD_DEVICE_ENTRY(0x9A60, 12),	\
+	IGD_DEVICE_ENTRY(0x9A68, 12),	\
+	IGD_DEVICE_ENTRY(0x9A70, 12),	\
+	IGD_DEVICE_ENTRY(0x9A40, 12),	\
+	IGD_DEVICE_ENTRY(0x9A49, 12),	\
+	IGD_DEVICE_ENTRY(0x9A59, 12),	\
+	IGD_DEVICE_ENTRY(0x9A78, 12),	\
+	IGD_DEVICE_ENTRY(0x9AC0, 12),	\
+	IGD_DEVICE_ENTRY(0x9AC9, 12),	\
+	IGD_DEVICE_ENTRY(0x9AD9, 12),	\
+	IGD_DEVICE_ENTRY(0x9AF8, 12)
+
+/* Rocket Lake, Gen 12 */
+#define IGD_RKL_DEVICE_IDS	\
+	IGD_DEVICE_ENTRY(0x4C80, 12),	\
+	IGD_DEVICE_ENTRY(0x4C8A, 12),	\
+	IGD_DEVICE_ENTRY(0x4C8B, 12),	\
+	IGD_DEVICE_ENTRY(0x4C8C, 12),	\
+	IGD_DEVICE_ENTRY(0x4C90, 12),	\
+	IGD_DEVICE_ENTRY(0x4C9A, 12)
+
+/* Alder Lake-S, Gen 12 */
+#define IGD_ADLS_DEVICE_IDS	\
+	IGD_DEVICE_ENTRY(0x4680, 12),	\
+	IGD_DEVICE_ENTRY(0x4682, 12),	\
+	IGD_DEVICE_ENTRY(0x4688, 12),	\
+	IGD_DEVICE_ENTRY(0x468A, 12),	\
+	IGD_DEVICE_ENTRY(0x468B, 12),	\
+	IGD_DEVICE_ENTRY(0x4690, 12),	\
+	IGD_DEVICE_ENTRY(0x4692, 12),	\
+	IGD_DEVICE_ENTRY(0x4693, 12)
+
+/* Alder Lake-P, Gen 12 */
+#define IGD_ADLP_DEVICE_IDS	\
+	IGD_DEVICE_ENTRY(0x46A0, 12),	\
+	IGD_DEVICE_ENTRY(0x46A1, 12),	\
+	IGD_DEVICE_ENTRY(0x46A2, 12),	\
+	IGD_DEVICE_ENTRY(0x46A3, 12),	\
+	IGD_DEVICE_ENTRY(0x46A6, 12),	\
+	IGD_DEVICE_ENTRY(0x46A8, 12),	\
+	IGD_DEVICE_ENTRY(0x46AA, 12),	\
+	IGD_DEVICE_ENTRY(0x462A, 12),	\
+	IGD_DEVICE_ENTRY(0x4626, 12),	\
+	IGD_DEVICE_ENTRY(0x4628, 12),	\
+	IGD_DEVICE_ENTRY(0x46B0, 12),	\
+	IGD_DEVICE_ENTRY(0x46B1, 12),	\
+	IGD_DEVICE_ENTRY(0x46B2, 12),	\
+	IGD_DEVICE_ENTRY(0x46B3, 12),	\
+	IGD_DEVICE_ENTRY(0x46C0, 12),	\
+	IGD_DEVICE_ENTRY(0x46C1, 12),	\
+	IGD_DEVICE_ENTRY(0x46C2, 12),	\
+	IGD_DEVICE_ENTRY(0x46C3, 12)
+
+/* Alder Lake-N, Gen 12 */
+#define IGD_ADLN_DEVICE_IDS	\
+	IGD_DEVICE_ENTRY(0x46D0, 12),	\
+	IGD_DEVICE_ENTRY(0x46D1, 12),	\
+	IGD_DEVICE_ENTRY(0x46D2, 12),	\
+	IGD_DEVICE_ENTRY(0x46D3, 12),	\
+	IGD_DEVICE_ENTRY(0x46D4, 12)
+
+/* Raptor Lake-S, Gen 12 */
+#define IGD_RPLS_DEVICE_IDS	\
+	IGD_DEVICE_ENTRY(0xA780, 12),	\
+	IGD_DEVICE_ENTRY(0xA781, 12),	\
+	IGD_DEVICE_ENTRY(0xA782, 12),	\
+	IGD_DEVICE_ENTRY(0xA783, 12),	\
+	IGD_DEVICE_ENTRY(0xA788, 12),	\
+	IGD_DEVICE_ENTRY(0xA789, 12),	\
+	IGD_DEVICE_ENTRY(0xA78A, 12),	\
+	IGD_DEVICE_ENTRY(0xA78B, 12)
+
+/* Raptor Lake-P, Gen 12 */
+#define IGD_RPLP_DEVICE_IDS	\
+	IGD_DEVICE_ENTRY(0xA721, 12),	\
+	IGD_DEVICE_ENTRY(0xA7A1, 12),	\
+	IGD_DEVICE_ENTRY(0xA7A9, 12),	\
+	IGD_DEVICE_ENTRY(0xA7AC, 12),	\
+	IGD_DEVICE_ENTRY(0xA7AD, 12),	\
+	IGD_DEVICE_ENTRY(0xA720, 12),	\
+	IGD_DEVICE_ENTRY(0xA7A0, 12),	\
+	IGD_DEVICE_ENTRY(0xA7A8, 12),	\
+	IGD_DEVICE_ENTRY(0xA7AA, 12),	\
+	IGD_DEVICE_ENTRY(0xA7AB, 12)
+
+#endif

devicemodel/include/iothread.h

@@ -7,14 +7,48 @@
 #ifndef	_iothread_CTX_H_
 #define	_iothread_CTX_H_
 
+#define IOTHREAD_NUM			40
+
+/*
+ * The pthread_setname_np() function can be used to set a unique name for a thread,
+ * which can be useful for debugging multithreaded applications.
+ * The thread name is a meaningful C language string,
+ * whose length is restricted to 16 characters, including the terminating null byte ('\0').
+ */
+#define PTHREAD_NAME_MAX_LEN		16
+
 struct iothread_mevent {
 	void (*run)(void *);
 	void *arg;
 	int fd;
 };
-int iothread_add(int fd, struct iothread_mevent *aevt);
-int iothread_del(int fd);
-int iothread_init(void);
+
+struct iothread_ctx {
+	pthread_t tid;
+	int epfd;
+	bool started;
+	pthread_mutex_t mtx;
+	int idx;
+	cpu_set_t cpuset;
+	char name[PTHREAD_NAME_MAX_LEN];
+};
+
+struct iothreads_option {
+	char tag[PTHREAD_NAME_MAX_LEN];
+	int num;
+	cpu_set_t *cpusets;
+};
+
+struct iothreads_info {
+	struct iothread_ctx *ioctx_base;
+	int num;
+};
+
+int iothread_add(struct iothread_ctx *ioctx_x, int fd, struct iothread_mevent *aevt);
+int iothread_del(struct iothread_ctx *ioctx_x, int fd);
 void iothread_deinit(void);
+struct iothread_ctx *iothread_create(struct iothreads_option *iothr_opt);
+int iothread_parse_options(char *str, struct iothreads_option *iothr_opt);
+void iothread_free_options(struct iothreads_option *iothr_opt);
 
 #endif

devicemodel/include/monitor.h

@@ -36,4 +36,7 @@ unsigned get_wakeup_reason(void);
 int set_wakeup_timer(time_t t);
 int acrn_parse_intr_monitor(const char *opt);
 int vm_monitor_blkrescan(void *arg, char *devargs);
+
+int vm_monitor_send_vm_event(const char *msg);
+
 #endif

devicemodel/include/passthru.h

@@ -9,6 +9,7 @@
 #define __PASSTHRU_H__
 
 #include <types.h>
+#include <limits.h>
 
 #include "pciaccess.h"
 #include "pci_core.h"
@@ -37,7 +38,9 @@ struct passthru_dev {
 	bool need_reset;
 	bool d3hot_reset;
 	bool need_rombar;
+	bool need_dsdt;
 	char *rom_buffer;
+	char dsdt_path[256];
 	bool (*has_virt_pcicfg_regs)(int offset);
 };
 

devicemodel/include/pci_core.h

@@ -296,7 +296,6 @@ void destory_io_rsvd_rgns(struct pci_vdev *vdev);
  * For OpRegion 2.0:  ASLE.rvda = physical address, not support currently
  */
 #define GPU_DSM_GPA			0x7C000000
-#define GPU_DSM_SIZE			0x4000000
 #define GPU_OPREGION_SIZE		0x5000
 /*
  * TODO: Forced DSM/OPREGION size requires native BIOS configuration.
@@ -350,8 +349,6 @@ int	pci_emul_add_pciecap(struct pci_vdev *pi, int pcie_device_type);
  *
  * @param dev Pointer to struct pci_vdev representing virtual PCI device.
  * @param index Message data index.
- *
- * @return None
  */
 void	pci_generate_msi(struct pci_vdev *dev, int index);
 
@@ -360,8 +357,6 @@ void	pci_generate_msi(struct pci_vdev *dev, int index);
  *
  * @param dev Pointer to struct pci_vdev representing virtual PCI device.
  * @param index MSIs table entry index.
- *
- * @return None
  */
 void	pci_generate_msix(struct pci_vdev *dev, int index);
 
@@ -369,8 +364,6 @@ void	pci_generate_msix(struct pci_vdev *dev, int index);
  * @brief Assert INTx pin of virtual PCI device
  *
  * @param dev Pointer to struct pci_vdev representing virtual PCI device.
- *
- * @return None
  */
 void	pci_lintr_assert(struct pci_vdev *dev);
 
@@ -378,8 +371,6 @@ void	pci_lintr_assert(struct pci_vdev *dev);
  * @brief Deassert INTx pin of virtual PCI device
  *
  * @param dev Pointer to struct pci_vdev representing virtual PCI device.
- *
- * @return None
  */
 void	pci_lintr_deassert(struct pci_vdev *dev);
 
@@ -417,8 +408,6 @@ struct pci_vdev *pci_get_vdev_info(int slot);
  * @param dev Pointer to struct pci_vdev representing virtual PCI device.
  * @param offset Offset in configuration space.
  * @param val Value in 1 byte.
- *
- * @return None
  */
 static inline void
 pci_set_cfgdata8(struct pci_vdev *dev, int offset, uint8_t val)
@@ -436,8 +425,6 @@ pci_set_cfgdata8(struct pci_vdev *dev, int offset, uint8_t val)
  * @param dev Pointer to struct pci_vdev representing virtual PCI device.
  * @param offset Offset in configuration space.
  * @param val Value in 2 bytes.
- *
- * @return None
  */
 static inline void
 pci_set_cfgdata16(struct pci_vdev *dev, int offset, uint16_t val)
@@ -455,8 +442,6 @@ pci_set_cfgdata16(struct pci_vdev *dev, int offset, uint16_t val)
  * @param dev Pointer to struct pci_vdev representing virtual PCI device.
  * @param offset Offset in configuration space.
  * @param val Value in 4 bytes.
- *
- * @return None
  */
 static inline void
 pci_set_cfgdata32(struct pci_vdev *dev, int offset, uint32_t val)

devicemodel/include/pcireg.h

@@ -1066,6 +1066,8 @@
 #define	PCIM_OSC_CTL_PCIE_CAP_STRUCT	0x10 /* Various Capability Structures */
 
 /* Graphics definitions */
+#define PCIR_GGC			0x50 /* GMCH Graphics Control */
+#define PCIR_GGC_GMS_SHIFT		8    /*   Bit 15:8  Graphics Memory Size (GMS) */
 #define PCIR_BDSM			0x5C /* BDSM graphics base data of stolen memory register */
 #define PCIR_GEN11_BDSM_DW0		0xC0
 #define PCIR_GEN11_BDSM_DW1		0xC4

devicemodel/include/public/hsm_ioctl_defs.h

@@ -107,8 +107,6 @@
 	_IOW(ACRN_IOCTL_TYPE, 0x41, struct acrn_vm_memmap)
 #define ACRN_IOCTL_UNSET_MEMSEG		\
 	_IOW(ACRN_IOCTL_TYPE, 0x42, struct acrn_vm_memmap)
-#define ACRN_IOCTL_SETUP_SBUF	\
-	_IOW(ACRN_IOCTL_TYPE, 0x43, struct acrn_sbuf)
 
 /* PCI assignment*/
 #define ACRN_IOCTL_SET_PTDEV_INTR	\
@@ -138,6 +136,15 @@
 #define ACRN_IOCTL_IRQFD		\
 	_IOW(ACRN_IOCTL_TYPE, 0x71, struct acrn_irqfd)
 
+/* Asynchronous IO */
+#define ACRN_IOCTL_SETUP_ASYNCIO	\
+	_IOW(ACRN_IOCTL_TYPE, 0x90, __u64)
+
+/* VM EVENT */
+#define ACRN_IOCTL_SETUP_VM_EVENT_RING	\
+	_IOW(ACRN_IOCTL_TYPE, 0xa0, __u64)
+#define ACRN_IOCTL_SETUP_VM_EVENT_FD	\
+	_IOW(ACRN_IOCTL_TYPE, 0xa1, int)
 
 #define	ACRN_MEM_ACCESS_RIGHT_MASK	0x00000007U
 #define	ACRN_MEM_ACCESS_READ		0x00000001U
@@ -250,13 +257,4 @@ struct acrn_irqfd {
        struct acrn_msi_entry msi;
 };
 
-/**
- * @brief data structure to register a share buffer by ioctl
- */
-struct acrn_sbuf {
-	/** Type of the sbuf. */
-	uint32_t sbuf_id;
-	/** Base address of the sbuf. */
-	uint64_t base;
-};
 #endif /* VHM_IOCTL_DEFS_H */

devicemodel/include/rtc.h

@@ -43,6 +43,7 @@ struct vrtc;
 struct vmctx;
 
 int vrtc_init(struct vmctx *ctx);
+void vrtc_suspend(struct vmctx *ctx);
 void vrtc_enable_localtime(int l_time);
 void vrtc_deinit(struct vmctx *ctx);
 int vrtc_set_time(struct vrtc *vrtc, time_t secs);

devicemodel/include/sbuf.h

@@ -0,0 +1,39 @@
+/*
+ * Copyright (C) 2018-2023 Intel Corporation.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#ifndef SHARED_BUF_H
+#define SHARED_BUF_H
+
+#include <linux/types.h>
+#include "acrn_common.h"
+
+
+static inline bool sbuf_is_empty(struct shared_buf *sbuf)
+{
+	return (sbuf->head == sbuf->tail);
+}
+
+static inline void sbuf_clear_flags(struct shared_buf *sbuf, uint64_t flags)
+{
+        sbuf->flags &= ~flags;
+}
+
+static inline void sbuf_set_flags(struct shared_buf *sbuf, uint64_t flags)
+{
+        sbuf->flags = flags;
+}
+
+static inline void sbuf_add_flags(struct shared_buf *sbuf, uint64_t flags)
+{
+        sbuf->flags |= flags;
+}
+
+uint32_t sbuf_get(struct shared_buf *sbuf, uint8_t *data);
+uint32_t sbuf_put(struct shared_buf *sbuf, uint8_t *data, uint32_t max_len);
+int sbuf_clear_buffered(struct shared_buf *sbuf);
+void sbuf_init(struct shared_buf *sbuf, uint32_t total_size, uint32_t ele_size);
+
+#endif /* SHARED_BUF_H */

devicemodel/include/uart_core.h

@@ -36,6 +36,7 @@ struct uart_vdev;
 
 typedef void (*uart_intr_func_t)(void *arg);
 int	uart_legacy_alloc(int unit, int *ioaddr, int *irq);
+int     uart_legacy_reinit_res(int which, int baseaddr, int irq);
 void	uart_legacy_dealloc(int which);
 uint8_t	uart_read(struct uart_vdev *uart, int offset);
 void	uart_write(struct uart_vdev *uart, int offset, uint8_t value);

devicemodel/include/usb_core.h

@@ -221,12 +221,6 @@ enum USB_ERRCODE {
 	do { if (lvl <= usb_log_level) pr_dbg(LOG_TAG fmt, ##args); } while (0)
 
 #define NATIVE_USBSYS_DEVDIR "/sys/bus/usb/devices"
-#define NATIVE_USB2_SPEED "480"
-#define NATIVE_USB3_SPEED "5000"
-#define USB_NATIVE_NUM_PORT 20
-#define USB_NATIVE_NUM_BUS 5
-
-#define USB_DROPPED_XFER_MAGIC	0xaaaaaaaa55555555
 
 inline bool
 index_valid(int head, int tail, int maxcnt, int idx) {

devicemodel/include/virtio.h

@@ -427,6 +427,7 @@ struct virtio_iothread {
 	int idx;
 	int kick_fd;
 	bool	ioevent_started;
+	struct iothread_ctx *ioctx;
 	struct iothread_mevent iomvt;
 	void (*iothread_run)(void *, struct virtio_vq_info *);
 };
@@ -439,6 +440,7 @@ struct virtio_vq_info {
 	struct virtio_base *base;
 				/**< backpointer to virtio_base */
 	uint16_t num;		/**< the num'th queue in the virtio_base */
+	pthread_mutex_t mtx;	/**< per queue mutex */
 
 	uint16_t flags;		/**< flags (see above) */
 	uint16_t last_avail;	/**< a recent value of avail->idx */
@@ -510,8 +512,6 @@ vq_has_descs(struct virtio_vq_info *vq)
  *
  * @param vb Pointer to struct virtio_base.
  * @param vq Pointer to struct virtio_vq_info.
- *
- * @return None
  */
 static inline void
 vq_interrupt(struct virtio_base *vb, struct virtio_vq_info *vq)
@@ -533,8 +533,6 @@ vq_interrupt(struct virtio_base *vb, struct virtio_vq_info *vq)
  * MSI-X or a generic MSI interrupt with config changed event.
  *
  * @param vb Pointer to struct virtio_base.
- *
- * @return None
  */
 static inline void
 virtio_config_changed(struct virtio_base *vb)
@@ -567,8 +565,6 @@ struct iovec;
  * @param dev Pointer to struct pci_vdev which emulates a PCI device.
  * @param queues Pointer to struct virtio_vq_info, normally an array.
  * @param backend_type can be VBSU, VBSK or VHOST
- *
- * @return None
  */
 void virtio_linkup(struct virtio_base *base, struct virtio_ops *vops,
 		   void *pci_virtio_dev, struct pci_vdev *dev,
@@ -624,8 +620,6 @@ int virtio_intr_init(struct virtio_base *base, int barnum, int use_msix);
  * If MSI-X is enabled, this also resets all the vectors to NO_VECTOR.
  *
  * @param base Pointer to struct virtio_base.
- *
- * @return None
  */
 void virtio_reset_dev(struct virtio_base *base);
 
@@ -634,8 +628,6 @@ void virtio_reset_dev(struct virtio_base *base);
  *
  * @param base Pointer to struct virtio_base.
  * @param barnum Which BAR[0..5] to use.
- *
- * @return None
  */
 void virtio_set_io_bar(struct virtio_base *base, int barnum);
 
@@ -660,8 +652,6 @@ int vq_getchain(struct virtio_vq_info *vq, uint16_t *pidx,
  * available ring.
  *
  * @param vq Pointer to struct virtio_vq_info.
- *
- * @return None
  */
 void vq_retchain(struct virtio_vq_info *vq);
 
@@ -672,8 +662,6 @@ void vq_retchain(struct virtio_vq_info *vq);
  * @param vq Pointer to struct virtio_vq_info.
  * @param idx Pointer to available ring position, returned by vq_getchain().
  * @param iolen Number of data bytes to be returned to frontend.
- *
- * @return None
  */
 void vq_relchain(struct virtio_vq_info *vq, uint16_t idx, uint32_t iolen);
 
@@ -685,8 +673,6 @@ void vq_relchain(struct virtio_vq_info *vq, uint16_t idx, uint32_t iolen);
  *
  * @param vq Pointer to struct virtio_vq_info.
  * @param used_all_avail Flag indicating if driver used all available chains.
- *
- * @return None
  */
 void vq_endchains(struct virtio_vq_info *vq, int used_all_avail);
 
@@ -699,8 +685,6 @@ void vq_endchains(struct virtio_vq_info *vq, int used_all_avail);
  *
  * @param base Pointer to struct virtio_base.
  * @param vq Pointer to struct virtio_vq_info.
- *
- * @return None
  */
 void vq_clear_used_ring_flags(struct virtio_base *base, struct virtio_vq_info *vq);
 
@@ -735,8 +719,6 @@ uint64_t virtio_pci_read(struct vmctx *ctx, int vcpu, struct pci_vdev *dev,
  * @param offset Register offset in bytes within a BAR region.
  * @param size Access range in bytes.
  * @param value Data value to be written into register.
- *
- * @return None
  */
 void virtio_pci_write(struct vmctx *ctx, int vcpu, struct pci_vdev *dev,
 		      int baridx, uint64_t offset, int size, uint64_t value);

devicemodel/include/vm_event.h

@@ -0,0 +1,18 @@
+/*
+ * Copyright (C) 2019-2023 Intel Corporation.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#ifndef VM_EVENT_H
+#define VM_EVENT_H
+
+#include <types.h>
+#include "vmmapi.h"
+
+int vm_event_init(struct vmctx *ctx);
+int vm_event_deinit(void);
+int dm_send_vm_event(struct vm_event *event);
+uint32_t get_dm_vm_event_overrun_count(void);
+
+#endif /* VM_EVENT_H */

devicemodel/include/vmmapi.h

@@ -107,7 +107,7 @@ int	vm_create_ioreq_client(struct vmctx *ctx);
 int	vm_destroy_ioreq_client(struct vmctx *ctx);
 int	vm_attach_ioreq_client(struct vmctx *ctx);
 int	vm_notify_request_done(struct vmctx *ctx, int vcpu);
-int	vm_setup_sbuf(struct vmctx *ctx, uint32_t sbuf_type, uint64_t base);
+int	vm_setup_asyncio(struct vmctx *ctx, uint64_t base);
 void	vm_clear_ioreq(struct vmctx *ctx);
 const char *vm_state_to_str(enum vm_suspend_how idx);
 void	vm_set_suspend_mode(enum vm_suspend_how how);

doc/_templates/layout.html

@@ -5,7 +5,7 @@
         <p class="admonition-title">Important</p>
         <p>This is the latest documentation for the unstable development branch of
         Project ACRN (master).<br/>Use the drop-down menu on the left to select
-        documentation for a stable release such as <a href="/3.1/">v3.1</a> or
+        documentation for a stable release such as <a href="/3.2/">v3.2</a> or
         <a href="/3.0/">v3.0</a>.</p>
     </div>
   {% endif %}

doc/acrn.doxyfile

@@ -241,6 +241,8 @@ TAB_SIZE               = 4
 # Allow for rst directives and advanced functions e.g. grid tables
 ALIASES  = "rst=\verbatim embed:rst:leading-asterisk"
 ALIASES += "endrst=\endverbatim"
+ALIASES += consistency="\par consistency:^^"
+ALIASES += alignment="\par alignment:^^"
 
 # This tag can be used to specify a number of word-keyword mappings (TCL only).
 # A mapping has the form "name=value". For example adding "class=itcl::class"
@@ -1809,7 +1811,7 @@ LATEX_HIDE_INDICES     = NO
 # The default value is: NO.
 # This tag requires that the tag GENERATE_LATEX is set to YES.
 
-LATEX_SOURCE_CODE      = NO
+# LATEX_SOURCE_CODE      = NO
 
 # The LATEX_BIB_STYLE tag can be used to specify the style to use for the
 # bibliography, e.g. plainnat, or ieeetr. See
@@ -1891,7 +1893,7 @@ RTF_EXTENSIONS_FILE    =
 # The default value is: NO.
 # This tag requires that the tag GENERATE_RTF is set to YES.
 
-RTF_SOURCE_CODE        = NO
+# RTF_SOURCE_CODE        = NO
 
 #---------------------------------------------------------------------------
 # Configuration options related to the man page output
@@ -1989,7 +1991,7 @@ DOCBOOK_OUTPUT         = docbook
 # The default value is: NO.
 # This tag requires that the tag GENERATE_DOCBOOK is set to YES.
 
-DOCBOOK_PROGRAMLISTING = NO
+# DOCBOOK_PROGRAMLISTING = NO
 
 #---------------------------------------------------------------------------
 # Configuration options for the AutoGen Definitions output
@@ -2187,7 +2189,7 @@ EXTERNAL_PAGES         = YES
 # powerful graphs.
 # The default value is: YES.
 
-CLASS_DIAGRAMS         = YES
+# CLASS_DIAGRAMS         = YES
 
 # You can define message sequence charts within doxygen comments using the \msc
 # command. Doxygen will then run the mscgen tool (see:

doc/asa.rst

@@ -3,6 +3,45 @@
 Security Advisory
 #################
 
+Addressed in ACRN v3.0.2
+************************
+We recommend that all developers using v3.0.1 or earlier upgrade to this v3.0.2
+release (or later), which addresses the following security issue discovered in
+previous releases. For v3.1 users, these issues are addressed in the v3.2
+release:
+
+-----
+
+- Board_inspector: use executables found under system paths
+    Using partial executable paths in the board inspector may cause unintended
+    results when another executable has the same name and is also detectable in
+    the search paths.
+
+    Introduce a wrapper module (`external_tools`) which locates executables
+    only under system paths such as /usr/bin and /usr/sbin and converts partial
+    executable paths to absolute ones before executing them via the subprocess
+    module. All invocations to `subprocess.run` or `subprocess.Popen`
+    throughout the board inspector are replaced with `external_tools.run`, with
+    the only exception being the invocation to the legacy board parser which
+    already uses an absolute path to the current Python interpreter.
+
+    **Affected Release:** v3.1, v3.0.1 and earlier
+
+- Add tarfile member sanitization to extractall()
+    A directory traversal vulnerability in the Python tarfile module extractall() functions
+    could allow user-assisted remote attackers to overwrite arbitrary files via
+    a ``..`` (dot dot) sequence in filenames in a tar archive, related to CVE-2001-1267.
+    (Addresses security issue tracked by CVE-2007-4559)
+
+    **Affected Release:** v3.1, v3.0.1 and earlier
+
+- PMU (Performance Monitoring Unit) is passed through to an RTVM only for debug mode
+    Enabling Pass-through PMU counters to RTVM can cause workload interference
+    in a release build, so enable PMU passthrough only when building ACRN in
+    debug mode.
+
+    **Affected Release:** v3.1, v3.0.1 and earlier
+
 Addressed in ACRN v3.0.1
 ************************
 We recommend that all developers upgrade to this v3.0.1 release (or later), which

doc/conf.py

@@ -46,7 +46,8 @@ extensions = [
 # extlinks provides a macro template
 
 extlinks = {
-            'acrn-issue': ('https://github.com/projectacrn/acrn-hypervisor/issues/%s', '#')
+            'acrn-issue': ('https://github.com/projectacrn/acrn-hypervisor/issues/%s', '#'),
+            'acrn-pr': ('https://github.com/projectacrn/acrn-hypervisor/pull/%s', '#')
            }
 
 # use intersphinx linking to link to previous version release notes
@@ -133,7 +134,7 @@ language = 'en'
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This patterns also effect to html_static_path and html_extra_path
-exclude_patterns = ['_build', 'misc/README.rst' ]
+exclude_patterns = ['_build', 'misc/README.rst', 'venv' ]
 
 # The name of the Pygments (syntax highlighting) style to use.
 pygments_style = 'sphinx'
@@ -198,6 +199,7 @@ html_context = {
    'docs_title': docs_title,
    'is_release': is_release,
    'versions': ( ("latest", "/latest/"),
+                 ("3.2", "/3.2/"),
                  ("3.1", "/3.1/"),
                  ("3.0", "/3.0/"),
                  ("2.7", "/2.7/"),

doc/develop.rst

@@ -29,6 +29,7 @@ User VM Tutorials
    tutorials/using_xenomai_as_user_vm
    tutorials/using_vxworks_as_user_vm
    tutorials/using_zephyr_as_user_vm
+   tutorials/using_celadon_as_user_vm
 
 Configuration Tutorials
 ***********************

doc/developer-guides/c_coding_guidelines.rst

@@ -3312,8 +3312,7 @@ each function:
     ``@post <post-condition description>``.
 12) The brief description of the function return value shall be documented
     with the format ``@return <brief description of return value>``.
-13) A void-returning function shall be documented with the format
-    ``@return None``.
+13) A void-returning function shall not be documented with ``@return``.
 14) The comments explaining the actual return values shall be documented with
     the format ``@retval <return value> <return value explanation>``.
 15) If the description of one element needs to span multiple lines, each line

doc/developer-guides/hld/hv-console.rst

@@ -70,10 +70,24 @@ Specifically:
    the hypervisor shell. Inputs to the physical UART will be
    redirected to the vUART starting from the next timer event.
 
--  The vUART is deactivated after a :kbd:`Ctrl` + :kbd:`Space` hotkey is received
-   from the physical UART. Inputs to the physical UART will be
-   handled by the hypervisor shell starting from the next timer
-   event.
+-  The vUART enters escaping mode after a BREAK character is received from
+   the physical UART. When in escaping mode, next received character will
+   be a command. After processing this command, the vUART exits the escaping
+   mode. So far, following escaping commands are supported:
+
+   - BREAK charater. If user sends break charater again in escaping mode,
+     one break charater will be sent to vUART.
+
+   - Character "e". This will deactive vUART. Inputs to the physical UART will
+     be handled by the hypervisor shell starting from the next timer event.
+
+   Other characters are not supported. The physical UART will prompt out an
+   "Unknown escaping key" message and the active vUART exits escaping mode.
+
+   Note that the BREAK character is a control character and different serial
+   terminals have different ways to send it, for example, `<Ctrl-A> + F`
+   in minicom, `<Ctrl-a> + <Ctrl-\>` in picocom, right click -> special
+   command -> break in putty serial terminal.
 
 The workflows are described as follows:
 

doc/developer-guides/hld/ivshmem-hld.rst

@@ -106,12 +106,12 @@ MMIO Registers Definition
    * - IVSHMEM\_IRQ\_MASK\_REG
      - 0x0
      - R/W
-     - Interrupt Status register is used for legacy interrupt.
+     - Interrupt Mask register is used for legacy interrupt.
        ivshmem doesn't support interrupts, so this register is reserved.
    * - IVSHMEM\_IRQ\_STA\_REG
      - 0x4
      - R/W
-     - Interrupt Mask register is used for legacy interrupt.
+     - Interrupt Status register is used for legacy interrupt.
        ivshmem doesn't support interrupts, so this register is reserved.
    * - IVSHMEM\_IV\_POS\_REG
      - 0x8

doc/getting-started/getting-started.rst

@@ -53,11 +53,6 @@ Before you begin, make sure your machines have the following prerequisites:
   - USB keyboard and mouse
   - Monitor
   - Ethernet cable and Internet access
-  - A second USB disk with minimum 16GB capacity. Format your USB disk with a
-    file system that supports files greater than 4GB: extFAT or NTFS, but not
-    FAT32. We'll use this USB disk to copy files between the development
-    computer and target system. Instead of a USB drive, you can copy files
-    between systems over the network using the ``scp`` command.
   - Local storage device (NVMe or SATA drive, for example).  We recommend having
     40GB or more of free space.
 
@@ -122,10 +117,11 @@ To set up the ACRN build environment on the development computer:
            python3 python3-pip libblkid-dev e2fslibs-dev \
            pkg-config libnuma-dev libcjson-dev liblz4-tool flex bison \
            xsltproc clang-format bc libpixman-1-dev libsdl2-dev libegl-dev \
-           libgles-dev libdrm-dev gnu-efi libelf-dev \
+           libgles-dev libdrm-dev gnu-efi libelf-dev liburing-dev \
            build-essential git-buildpackage devscripts dpkg-dev equivs lintian \
-           apt-utils pristine-tar dh-python python3-lxml python3-defusedxml \
-           python3-tqdm python3-xmlschema python3-elementpath acpica-tools
+           apt-utils pristine-tar dh-python acpica-tools python3-tqdm \
+           python3-elementpath python3-lxml python3-xmlschema python3-defusedxml
+      
 
 #. Get the ACRN hypervisor and ACRN kernel source code, and check out the
    current release branch.
@@ -135,20 +131,12 @@ To set up the ACRN build environment on the development computer:
       cd ~/acrn-work
       git clone https://github.com/projectacrn/acrn-hypervisor.git
       cd acrn-hypervisor
-      git checkout master
+      git checkout release_3.3
 
       cd ..
       git clone https://github.com/projectacrn/acrn-kernel.git
       cd acrn-kernel
-      git checkout master
-
-
-#. Configure git with your name and email address:
-
-   .. code-block:: none
-
-      git config --global user.name "David Developer"
-      git config --global user.email "david.developer@company.com"
+      git checkout release_3.3
 
 .. _gsg-board-setup:
 
@@ -184,11 +172,11 @@ To set up the target hardware environment:
 
 #. Connect the monitor and power supply cable.
 
-#. Connect the target system to the LAN with the Ethernet cable.
+#. Connect the target system to the LAN with the Ethernet cable or wifi.
 
 Example of a target system with cables connected:
 
-.. image:: ./images/gsg_vecow.png
+.. image:: ./images/gsg_asus_minipc64.png
    :align: center
 
 Install OS on the Target
@@ -260,9 +248,10 @@ Configure Target BIOS Settings
 #. Boot your target and enter the BIOS configuration editor.
 
    Tip: When you are booting your target, you'll see an option (quickly) to
-   enter the BIOS configuration editor, typically by pressing :kbd:`F2` during
-   the boot and before the GRUB menu (or Ubuntu login screen) appears.  If you
-   are not quick enough, you can reboot the system to try again.
+   enter the BIOS configuration editor, typically by pressing :kbd:`F2` 
+   or :kbd:`DEL` during the boot and before the GRUB menu (or Ubuntu login
+   screen) appears. If you are not quick enough, you can still choose
+   ``UEFI settings`` in the GRUB menu or just reboot the system to try again.
 
 #. Configure these BIOS settings:
 
@@ -297,47 +286,11 @@ Generate a Board Configuration File
       In a few seconds, the build generates a board_inspector Debian package in the
       parent (``~/acrn-work``) directory.
 
-#. Copy the Board Inspector Debian package from the development computer to the
-   target system.
+#. Use the ``scp`` command to copy the board inspector Debian package from your
+   development computer to the ``/tmp`` directory on the target system.  Replace
+   ``10.0.0.200`` with the target system's IP address you found earlier::
 
-   Option 1: Use ``scp``
-      Use the ``scp`` command to copy the Debian package from your development
-      computer to the ``/tmp`` directory on the target
-      system.  Replace ``10.0.0.200`` with the target system's IP address you found earlier::
-
-         scp ~/acrn-work/python3-acrn-board-inspector*.deb acrn@10.0.0.200:/tmp
-
-   Option 2: Use a USB disk
-       a. On the development computer, insert the USB disk that you intend to use to
-          copy files.
-
-       #. Ensure that there is only one USB disk inserted by running the following
-          command:
-
-          .. code-block:: bash
-
-             ls /media/$USER
-
-          Confirm that only one disk name appears. You'll use that disk name in the following steps.
-
-       #. Copy the Board Inspector Debian package to the USB disk:
-
-          .. code-block:: bash
-
-             cd ~/acrn-work/
-             disk="/media/$USER/"$(ls /media/$USER)
-             cp -r python3-acrn-board-inspector*.deb "$disk"/
-             sync && sudo umount "$disk"
-
-       #. Remove the USB disk from the development computer and insert it into the target system.
-
-       #. Copy the Board Inspector Debian package from the USB disk to the target:
-
-          .. code-block:: bash
-
-             mkdir -p ~/acrn-work
-             disk="/media/$USER/"$(ls /media/$USER)
-             cp -r "$disk"/python3-acrn-board-inspector*.deb  /tmp
+     scp ~/acrn-work/python3-acrn-board-inspector*.deb acrn@10.0.0.200:/tmp
 
 #. Now that we've got the Board Inspector Debian package on the target system, install it there:
 
@@ -349,9 +302,9 @@ Generate a Board Configuration File
 
    .. code-block:: bash
 
-      reboot
+      sudo reboot
 
-#. Run the Board Inspector to generate the board configuration file. This
+#. Run the Board Inspector on the target system to generate the board configuration file. This
    example uses the parameter ``my_board`` as the file name. The Board Inspector
    can take a few minutes to scan your target system and create the board XML
    file with your target system's information.
@@ -359,7 +312,7 @@ Generate a Board Configuration File
    .. code-block:: bash
 
       cd ~/acrn-work
-      sudo board_inspector my_board
+      sudo acrn-board-inspector my_board
 
    .. note::
 
@@ -373,37 +326,12 @@ Generate a Board Configuration File
 
       ls ./my_board.xml
 
-#. Copy ``my_board.xml`` from the target to the development computer.  Again we
-   have two options:
+#. From your development computer, use the ``scp`` command to copy the board
+   configuration file on your target system back to the ``~/acrn-work``
+   directory on your development computer. Replace ``10.0.0.200`` with the
+   target system's IP address you found earlier::
 
-   Option 1: Use ``scp``
-      From your development computer, use the ``scp`` command to copy the board
-      configuration file from your target system back to the
-      ``~/acrn-work`` directory on your development computer. Replace
-      ``10.0.0.200`` with the target system's IP address you found earlier::
-
-         scp acrn@10.0.0.200:~/acrn-work/my_board.xml ~/acrn-work/
-
-   Option 2: Use a USB disk
-       a. Make sure the USB disk is connected to the target.
-
-       #. Copy ``my_board.xml`` to the USB disk:
-
-          .. code-block:: bash
-
-             disk="/media/$USER/"$(ls /media/$USER)
-             cp ~/acrn-work/my_board.xml "$disk"/
-             sync && sudo umount "$disk"
-
-       #. Insert the USB disk into the development computer.
-
-       #. Copy ``my_board.xml`` from the USB disk to the development computer:
-
-          .. code-block:: bash
-
-             disk="/media/$USER/"$(ls /media/$USER)
-             cp "$disk"/my_board.xml ~/acrn-work
-             sync && sudo umount "$disk"
+     scp acrn@10.0.0.200:~/acrn-work/my_board.xml ~/acrn-work/
 
 .. _gsg-dev-setup:
 
@@ -413,7 +341,7 @@ Generate a Scenario Configuration File and Launch Script
 ********************************************************
 
 In this step, you will download, install, and use the `ACRN Configurator
-<https://github.com/projectacrn/acrn-hypervisor/releases/download/v3.1/acrn-configurator-3.2-unstable.deb>`__
+<https://github.com/projectacrn/acrn-hypervisor/releases/download/v3.3/acrn-configurator-3.3.deb>`__
 to generate a scenario configuration file and launch script.
 
 A **scenario configuration file** is an XML file that holds the parameters of
@@ -429,8 +357,7 @@ post-launched User VM. Each User VM has its own launch script.
    .. code-block:: bash
 
       cd ~/acrn-work
-      wget https://github.com/projectacrn/acrn-hypervisor/releases/download/v3.1/acrn-configurator-3.2-unstable.deb
-      cp acrn-configurator-3.2-unstable.deb /tmp
+      wget https://github.com/projectacrn/acrn-hypervisor/releases/download/v3.3/acrn-configurator-3.3.deb -P /tmp
 
    If you already have a previous version of the acrn-configurator installed,
    you should first remove it:
@@ -443,7 +370,7 @@ post-launched User VM. Each User VM has its own launch script.
 
    .. code-block:: bash
 
-      sudo apt install -y /tmp/acrn-configurator-3.2-unstable.deb
+      sudo apt install -y /tmp/acrn-configurator-3.3.deb
 
 #. Launch the ACRN Configurator:
 
@@ -508,7 +435,7 @@ post-launched User VM. Each User VM has its own launch script.
       :class: drop-shadow
 
    The Configurator does consistency and validation checks when you load or save
-   a scenario.  Notice the Hypervisor and VM1 tabs both have an error icon,
+   a scenario. Notice the Hypervisor and VM1 tabs both have an error icon,
    meaning there are issues with configuration options in two areas.  Since the
    Hypervisor tab is currently highlighted, we're seeing an issue we can resolve
    on the Hypervisor settings.  Once we resolve all the errors and save the
@@ -541,26 +468,29 @@ post-launched User VM. Each User VM has its own launch script.
    #. Confirm that the **VM type** is ``Standard``. In the previous step,
       ``STD`` in the VM name is short for Standard.
 
-   #. Scroll down to **Memory size (MB)** and change the value to ``1024``. For
+   #. Scroll down to **Memory size (MB)** and change the value to ``2048``. For
       this example, we will use Ubuntu 22.04 to boot the post-launched VM.
-      Ubuntu 22.04 needs at least 1024 MB to boot.
+      Ubuntu 22.04 needs at least 2048 MB to boot.
 
    #. For **Physical CPU affinity**, select pCPU ID ``0``, then click **+** and
       select pCPU ID ``1`` to affine (or pin) the VM to CPU cores 0 and 1. (That will
       resolve the missing physical CPU affinity assignment error.)
 
    #. For **Virtio console device**, click **+** to add a device and keep the
-      default options. This parameter specifies the console that you will use to
-      log in to the User VM later in this guide.
+      default options. 
 
    #. For **Virtio block device**, click **+** and enter
-      ``/home/acrn/acrn-work/ubuntu-22.04.1-desktop-amd64.iso``. This parameter
+      ``/home/acrn/acrn-work/ubuntu-22.04.4-desktop-amd64.iso``. This parameter
       specifies the VM's OS image and its location on the target system. Later
       in this guide, you will save the ISO file to that directory. (If you used
       a different username when installing Ubuntu on the target system, here's
       where you'll need to change the ``acrn`` username to the username you used.)
 
-   .. image:: images/configurator-postvm.png
+   .. image:: images/configurator_postvm01.png
+      :align: center
+      :class: drop-shadow
+
+   .. image:: images/configurator_postvm02.png
       :align: center
       :class: drop-shadow
 
@@ -585,14 +515,14 @@ post-launched User VM. Each User VM has its own launch script.
 .. rst-class:: numbered-step
 
 Build ACRN
-***************
+**********
 
 #. On the development computer, build the ACRN hypervisor:
 
    .. code-block:: bash
 
       cd ~/acrn-work/acrn-hypervisor
-      debian/debian_build.sh clean && debian/debian_build.sh -c ~/acrn-work/MyConfiguration -b my_board.board -s scenario
+      debian/debian_build.sh clean && debian/debian_build.sh -c ~/acrn-work/MyConfiguration
 
    The build typically takes a few minutes. When done, the build generates several
    Debian packages in the parent (``~/acrn-work``) directory:
@@ -611,7 +541,7 @@ Build ACRN
          acrn-tools_*.deb
          grub-acrn_*.deb
 
-   The Debian packages contain the ACRN hypervisor and tools to ease installing
+   These Debian packages contain the ACRN hypervisor and tools to ease installing
    ACRN on the target.
 
 #. Build the ACRN kernel for the Service VM:
@@ -642,49 +572,20 @@ Build ACRN
    .. code-block:: bash
 
       cd ..
-      ls *.deb
-         linux-headers-5.15.44-acrn-service-vm_5.15.44-acrn-service-vm-1_amd64.deb
-         linux-image-5.15.44-acrn-service-vm_5.15.44-acrn-service-vm-1_amd64.deb
-         linux-image-5.15.44-acrn-service-vm-dbg_5.15.44-acrn-service-vm-1_amd64.deb
-         linux-libc-dev_5.15.44-acrn-service-vm-1_amd64.deb
+      ls *acrn-service-vm*.deb  
+         linux-headers-6.1.80-acrn-service-vm_6.1.80-acrn-service-vm-1_amd64.deb
+         linux-image-6.1.80-acrn-service-vm_6.1.80-acrn-service-vm-1_amd64.deb
+         linux-libc-dev_6.1.80-acrn-service-vm-1_amd64.deb
 
-#. Copy all the necessary files generated on the development computer to the
-   target system, using one of these two options:
+#. Use the ``scp`` command to copy files from your development computer to the
+   target system.  Replace ``10.0.0.200`` with the target system's IP address
+   you found earlier::
 
-   Option 1: Use ``scp``
-      Use the ``scp`` command to copy files from your development computer to 
-      the target system.
-      Replace ``10.0.0.200`` with the target system's IP address you found earlier::
-
-         sudo scp ~/acrn-work/acrn*.deb \
-             ~/acrn-work/grub*.deb \
-             ~/acrn-work/*acrn-service-vm*.deb \
-             ~/acrn-work/MyConfiguration/launch_user_vm_id1.sh \
-             acrn@10.0.0.200:~/acrn-work
-
-   Option 2: by USB disk
-       a. Insert the USB disk into the development computer and run these commands:
-
-          .. code-block:: bash
-
-             disk="/media/$USER/"$(ls /media/$USER)
-             cp ~/acrn-work/acrn*.deb "$disk"/
-             cp ~/acrn-work/grub*.deb "$disk"/
-             cp ~/acrn-work/*acrn-service-vm*.deb "$disk"/
-             cp ~/acrn-work/MyConfiguration/launch_user_vm_id1.sh "$disk"/
-             sync && sudo umount "$disk"
-
-       #. Insert the USB disk you just used into the target system and run these
-          commands to copy the files locally:
-
-          .. code-block:: bash
-
-             disk="/media/$USER/"$(ls /media/$USER)
-             cp "$disk"/acrn*.deb ~/acrn-work
-             cp "$disk"/grub*.deb ~/acrn-work
-             cp "$disk"/*acrn-service-vm*.deb ~/acrn-work
-             cp "$disk"/launch_user_vm_id1.sh ~/acrn-work
-             sync && sudo umount "$disk"
+     sudo scp ~/acrn-work/acrn*.deb \
+         ~/acrn-work/grub*.deb \
+         ~/acrn-work/*acrn-service-vm*.deb \
+         ~/acrn-work/MyConfiguration/launch_user_vm_id1.sh \
+         acrn@10.0.0.200:~/acrn-work
 
 .. _gsg-install-acrn:
 
@@ -699,28 +600,46 @@ Install ACRN
    .. code-block:: bash
 
       cd ~/acrn-work
-      sudo apt install ./acrn*.deb ./grub*.deb
-      sudo apt install ./*acrn-service-vm*.deb
+      cp ./acrn*.deb ./grub*.deb ./*acrn-service-vm*.deb /tmp
+      sudo apt install /tmp/acrn*.deb /tmp/grub*.deb /tmp/*acrn-service-vm*.deb
+
+#. Modify the GRUB menu display using ``sudo vi /etc/default/grub``, comment out the hidden style  
+   and changing the timeout to 5 seconds (leave other lines as they are), as shown:: 
+
+      #GRUB_TIMEOUT_STYLE=hidden
+      GRUB_TIMEOUT=5
+
+   and install the new GRUB menu using::
+   
+      sudo update-grub
 
 #. Reboot the system:
 
    .. code-block:: bash
 
-      reboot
+      sudo reboot
 
-#. Confirm that you see the GRUB menu with the "ACRN multiboot2" entry. Select
-   it and proceed to booting ACRN. (It may be auto-selected, in which case it
+   The target system will reboot into the ACRN hypervisor and
+   start the Ubuntu Service VM.
+
+#. Confirm that you see the GRUB menu with "Ubuntu with ACRN hypervisor, with Linux 6.1.80-acrn-service-vm (ACRN 3.3)"
+   entry. Select it and proceed to booting ACRN. (It may be auto-selected, in which case it
    will boot with this option automatically in 5 seconds.)
 
+   Example grub menu shown as below:
+
    .. code-block:: console
-      :emphasize-lines: 5
 
                               GNU GRUB version 2.04
       ────────────────────────────────────────────────────────────────────────────────
       Ubuntu
       Advanced options for Ubuntu
-      *Ubuntu GNU/Linux, with ACRN hypervisor
-      Advanced options for Ubuntu GNU/Linux (with ACRN hypervisor)
+      Ubuntu-ACRN Board Inspector, with Linux 6.5.0-18-generic
+      Ubuntu-ACRN Board Inspector, with Linux 6.1.80-acrn-service-vm
+      Memory test (memtest86+x64.efi)
+      Memory test (memtest86+x64.efi, serial console)
+      Ubuntu with ACRN hypervisor, with Linux 6.5.0-18-generic (ACRN 3.3)
+      *Ubuntu with ACRN hypervisor, with Linux 6.1.80-acrn-service-vm (ACRN 3.3)
       UEFI Firmware Settings
 
 .. _gsg-run-acrn:
@@ -732,7 +651,8 @@ Run ACRN and the Service VM
 
 The ACRN hypervisor boots the Ubuntu Service VM automatically.
 
-#. On the target, log in to the Service VM. (It will look like a normal
+#. On the target, log in to the Service VM using the ``acrn`` username and
+   password you set up previously. (It will look like a normal
    graphical Ubuntu session.)
 
 #. Verify that the hypervisor is running by checking ``dmesg`` in the Service
@@ -753,8 +673,12 @@ The ACRN hypervisor boots the Ubuntu Service VM automatically.
    so the Device Model can create a bridge device (acrn-br0) that provides User VMs with
    wired network access:
 
+   .. warning::
+      The IP address of Service VM may change after executing the following command.
+
    .. code-block:: bash
 
+      sudo cp /usr/share/doc/acrnd/examples/* /etc/systemd/network
       sudo systemctl enable --now systemd-networkd
 
 .. _gsg-user-vm:
@@ -764,12 +688,12 @@ The ACRN hypervisor boots the Ubuntu Service VM automatically.
 Launch the User VM
 *******************
 
-#. On the target system, use the web browser to go to the `official Ubuntu website <https://releases.ubuntu.com/jammy/>`__ to
+#. On the target system, use the web browser to visit the `official Ubuntu website <https://releases.ubuntu.com/jammy/>`__ and
    get the Ubuntu Desktop 22.04 LTS ISO image
-   ``ubuntu-22.04.1-desktop-amd64.iso`` for the User VM. (The same image you
+   ``ubuntu-22.04.4-desktop-amd64.iso`` for the User VM. (The same image you
    specified earlier in the ACRN Configurator UI.) Alternatively, instead of
-   downloading it again, you can use a USB drive or ``scp`` to copy the ISO
-   image file to the ``~/acrn-work`` directory on the target system.
+   downloading it again, you could use ``scp`` to copy the ISO
+   image file from the development system to the ``~/acrn-work`` directory on the target system.
 
 #. If you downloaded the ISO file on the target system, copy it from the
    Downloads directory to the ``~/acrn-work/`` directory (the location we said
@@ -778,7 +702,7 @@ Launch the User VM
 
    .. code-block:: bash
 
-      cp ~/Downloads/ubuntu-22.04.1-desktop-amd64.iso ~/acrn-work
+      cp ~/Downloads/ubuntu-22.04.4-desktop-amd64.iso ~/acrn-work
 
 #. Launch the User VM:
 
@@ -788,32 +712,27 @@ Launch the User VM
       sudo ~/acrn-work/launch_user_vm_id1.sh
 
 #. It may take about a minute for the User VM to boot and start running the
-   Ubuntu image. You will see a lot of output, then the console of the User VM
+   Ubuntu image. You will see a lot of output, then the console of the User VM 
    will appear as follows:
-
+   
    .. code-block:: console
 
-      Ubuntu 22.04.1 LTS ubuntu hvc0
-
-      ubuntu login:
-
-#. Log in to the User VM. For the Ubuntu 22.04 ISO, the user is ``ubuntu``, and
-   there's no password.
-
-#. Confirm that you see output similar to this example:
-
-   .. code-block:: console
-
-      Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 5.15.0-43-generic x86_64)
+      Welcome to Ubuntu 22.04.4 LTS (GNU/Linux 6.5.0-18-generic x86_64)
 
       * Documentation:  https://help.ubuntu.com
       * Management:     https://landscape.canonical.com
       * Support:        https://ubuntu.com/advantage
 
-      0 packages can be updated.
-      0 updates are security updates.
+      Expanded Security Maintenance for Applications is not enabled.
 
-      Your Hardware Enablement Stack (HWE) is supported until April 2025.
+      0 updates can be applied immediately.
+
+      Enable ESM Apps to receive additional future security updates.
+      See https://ubuntu.com/esm or run: sudo pro status
+
+
+      The list of available updates is more than a week old.
+      To check for new updates run: sudo apt update
 
       The programs included with the Ubuntu system are free software;
       the exact distribution terms for each program are described in the
@@ -828,20 +747,20 @@ Launch the User VM
       ubuntu@ubuntu:~$
 
 #. This User VM and the Service VM are running different Ubuntu images. Use this
-   command to see that the User VM is running the downloaded Ubuntu ISO image:
+   command to see that the User VM is running the downloaded Ubuntu image:
 
    .. code-block:: console
 
-      ubuntu@ubuntu:~$ uname -r
-      5.15.0-43-generic
+      acrn@ubuntu:~$ uname -r
+      6.5.0-18-generic
 
    Then open a new terminal window and use the command to see that the Service
    VM is running the ``acrn-kernel`` Service VM image:
 
    .. code-block:: console
 
-      acrn@vecow:~$ uname -r
-      5.15.44-acrn-service-vm
+      acrn@asus-MINIPC-PN64:~$ uname -r
+      6.1.80-acrn-service-vm
 
    The User VM has launched successfully. You have completed this ACRN setup.
 

doc/getting-started/sample-app.rst

@@ -76,7 +76,7 @@ Preparing the Target System
 ===========================
 
 On the target system, reboot and choose the regular Ubuntu image (not the
-Multiboot2 choice created when following the Getting Started Guide).
+Ubuntu-ACRN Board Inspector choice created when following the Getting Started Guide).
 
 1. Log in as the **acrn** user. We'll be making ssh connections to the target system
    later in these steps, so install the ssh server on the target system using::
@@ -108,7 +108,7 @@ As a normal (e.g., **acrn**) user, follow these steps:
 1. Install some additional packages in your development computer used for
    building the sample application::
 
-     sudo apt install -y cloud-guest-utils schroot kpartx qemu-kvm
+     sudo apt install -y cloud-guest-utils schroot kpartx qemu-utils
 
 #. Check out the ``acrn-hypervisor`` source code branch (already cloned from the
    ``acrn-hypervisor`` repo when you followed the :ref:`gsg`). We've tagged a
@@ -117,7 +117,7 @@ As a normal (e.g., **acrn**) user, follow these steps:
 
      cd ~/acrn-work/acrn-hypervisor
      git fetch --all
-     git checkout master 
+     git checkout release_3.3
 
 #. Build the ACRN sample application source code::
 
@@ -189,10 +189,10 @@ Make the RT_VM Image
 
    .. code-block:: console
 
-     linux-headers-5.15.44-rt46-acrn-kernel-rtvm+_5.15.44-rt46-acrn-kernel-rtvm+-1_amd64.deb
-     linux-image-5.15.44-rt46-acrn-kernel-rtvm+-dbg_5.15.44-rt46-acrn-kernel-rtvm+-1_amd64.deb
-     linux-image-5.15.44-rt46-acrn-kernel-rtvm+_5.15.44-rt46-acrn-kernel-rtvm+-1_amd64.deb
-     linux-libc-dev_5.15.44-rt46-acrn-kernel-rtvm+-1_amd64.deb
+     linux-headers-5.15.71-rt46-acrn-kernel-rtvm+_5.15.71-rt46-acrn-kernel-rtvm+-1_amd64.deb
+     linux-image-5.15.71-rt46-acrn-kernel-rtvm+-dbg_5.15.71-rt46-acrn-kernel-rtvm+-1_amd64.deb
+     linux-image-5.15.71-rt46-acrn-kernel-rtvm+_5.15.71-rt46-acrn-kernel-rtvm+-1_amd64.deb
+     linux-libc-dev_5.15.71-rt46-acrn-kernel-rtvm+-1_amd64.deb
 
 #. Make the RT VM image::
 
@@ -394,43 +394,21 @@ Build the ACRN Hypervisor and Service VM Images
      cd ~/acrn-work/acrn-hypervisor
 
      make clean
-     make BOARD=~/acrn-work/MyConfiguration/my_board.board.xml SCENARIO=~/acrn-work/MyConfiguration/scenario.xml
+     debian/debian_build.sh clean && debian/debian_build.sh -c ~/acrn-work/MyConfiguration 
 
    The build typically takes about a minute. When done, the build
-   generates a Debian package in the build directory with your board and
-   working folder name.
+   generates several Debian packages in the build directory. Only one 
+   with your board and working folder name among these Debian packages 
+   is different from genetated in the Getting Started Guide. So we only 
+   need to copy and reinstall one Debian package to the target system. 
 
    This Debian package contains the ACRN hypervisor and tools for
    installing ACRN on the target.
 
-#. Build the ACRN kernel for the Service VM (the sample application
-   requires a newer version of the Service VM than generated in the
-   Getting Started Guide, so we'll need to generate it again) using a tagged
-   version of the ``acrn-kernel``::
-
-     cd ~/acrn-work/acrn-kernel
-     git fetch --all
-
-     git checkout acrn-v3.1
-
-     make distclean
-     cp kernel_config_service_vm .config
-     make olddefconfig
-     make -j $(nproc) deb-pkg
-
-   The kernel build can take 15 minutes or less on a fast computer, but
-   could take one to two hours depending on the performance of your development
-   computer. When done, the build generates four Debian packages in the
-   directory above the build root directory:
-
-   .. code-block:: console
-
-      $ ls ../*acrn-service*.deb
-
-      linux-headers-5.15.44-acrn-service-vm_5.15.44-acrn-service-vm-1_amd64.deb
-      linux-image-5.15.44-acrn-service-vm_5.15.44-acrn-service-vm-1_amd64.deb
-      linux-image-5.15.44-acrn-service-vm-dbg_5.15.44-acrn-service-vm-1_amd64.deb
-      linux-libc-dev_5.15.44-acrn-service-vm-1_amd64.deb
+#. Use the ACRN kernel for the Service VM already on your development computer 
+   when you followed the Getting Started Guide (the sample application
+   requires the same version of the Service VM as generated in the
+   Getting Started Guide, so no need to generate it again).
 
 .. rst-class:: numbered-step
 
@@ -439,91 +417,56 @@ Copy Files from the Development Computer to Your Target System
 
 1. Copy all the files generated on the development computer to the
    target system. This includes the sample application executable files,
-   HMI_VM and RT_VM images, Debian packages for the Service VM and
-   Hypervisor, launch scripts, and the iasl tool built following the
-   Getting Started Guide. You can use ``scp`` to copy across the local network,
-   or use a USB stick:
+   HMI_VM and RT_VM images, Debian packages for ACRN Hypervisor,
+   and the launch scripts. 
 
-   Option 1: use ``scp`` to copy files over the local network
-     Use ``scp`` to copy files from your development computer to the
-     ``~/acrn-work`` directory on the target (replace the IP address used in
-     this example with the target system's IP address you found earlier)::
+   Use ``scp`` to copy files from your development computer to the
+   ``~/acrn-work`` directory on the target (replace the IP address used in
+   this example with the target system's IP address you found earlier)::
 
-       cd ~/acrn-work
+     cd ~/acrn-work
 
-       scp acrn-hypervisor/misc/sample_application/image_builder/build/*_vm.img \
-           acrn-hypervisor/build/acrn-my_board-MyConfiguration*.deb \
-           *acrn-service-vm*.deb MyConfiguration/launch_user_vm_id*.sh \
-           acpica-unix-20210105/generate/unix/bin/iasl \
-           acrn@10.0.0.200:~/acrn-work
+     scp acrn-hypervisor/misc/sample_application/image_builder/build/*_vm.img \
+         acrn-hypervisor*.deb \
+         MyConfiguration/launch_user_vm_id*.sh \
+         acrn@10.0.0.200:~/acrn-work
 
-     Then on the target system, run these commands::
-
-       sudo cp ~/acrn-work/iasl /usr/sbin
-       sudo ln -s /usr/sbin/iasl /usr/bin/iasl
-
-   Option 2: use a USB stick to copy files
-     Because the VM image files are large, format your USB stick with a file
-     system that supports files greater than 4GB: extFAT or NTFS, but not FAT32.
-
-     Insert a USB stick into the development computer and run these commands::
-
-        disk="/media/$USER/"$(ls /media/$USER)
-
-        cd ~/acrn-work
-        cp acrn-hypervisor/misc/sample_application/image_builder/build/*_vm.img rt_vm.img "$disk"
-        cp acrn-hypervisor/build/acrn-my_board-MyConfiguration*.deb "$disk"
-        cp *acrn-service-vm*.deb "$disk"
-        cp MyConfiguration/launch_user_vm_id*.sh "$disk"
-        cp acpica-unix-20210105/generate/unix/bin/iasl "$disk"
-        sync && sudo umount "$disk"
-
-     Move the USB stick you just used to the target system and run
-     these commands to copy the files locally::
-
-        disk="/media/$USER/"$(ls /media/$USER)
-
-        cp "$disk"/*_vm.img ~/acrn-work
-        cp "$disk"/acrn-my_board-MyConfiguration*.deb ~/acrn-work
-        cp "$disk"/*acrn-service-vm*.deb ~/acrn-work
-        cp "$disk"/launch_user_vm_id*.sh ~/acrn-work
-        sudo cp "$disk"/iasl /usr/sbin/
-        sudo ln -s /usr/sbin/iasl /usr/bin/iasl
-        sync && sudo umount "$disk"
 
 .. rst-class:: numbered-step
 
 Install and Run ACRN on the Target System
 *****************************************
 
-1. On your target system, install the ACRN Debian package and ACRN
+1. On the target system, configure your network according to instruction of below link:
+
+   https://www.ubuntupit.com/how-to-configure-and-use-network-bridge-in-ubuntu-linux/
+
+#. On your target system, install the ACRN Debian package and ACRN
    kernel Debian packages using these commands::
 
      cd ~/acrn-work
+     cp ./acrn-hypervisor*.deb ./*acrn-service-vm*.deb /tmp
      sudo apt purge acrn-hypervisor
-     sudo apt install ./acrn-my_board-MyConfiguration*.deb
-     sudo apt install ./*acrn-service-vm*.deb
+     sudo apt install /tmp/acrn-hypervisor*.deb  /tmp/*acrn-service-vm*.deb
 
-#. Enable networking services for sharing with the HMI User VM::
+#. Enable networking services for sharing with the HMI User VM:
 
+   .. warning::
+      The IP address of Service VM may change after executing the following command.
+
+   .. code-block:: bash
+
+     cp /usr/share/doc/acrnd/examples/* /etc/systemd/network
      sudo systemctl enable --now systemd-networkd
 
 #. Reboot the system::
 
      reboot
 
-#. Confirm that you see the GRUB menu with the "ACRN multiboot2" entry.  Select
-   it and press :kbd:`Enter` to proceed to booting ACRN. (It may be
-   auto-selected, in which case it will boot with this option automatically in 5
-   seconds.)
-
-   .. image:: images/samp-image016.png
-      :class: drop-shadow
-      :align: center
-
-   This will boot the ACRN hypervisor and launch the Service VM.
-
-#. Log in to the Service VM (using the target's keyboard and HDMI monitor) using
+#. The target system will boot automatically into the ACRN hypervisor and 
+   launch the Service VM. 
+   
+   Log in to the Service VM (using the target's keyboard and HDMI monitor) using
    the ``acrn`` username.
 
 #. Find the Service VM's IP address (the first IP address shown by this command):
@@ -579,6 +522,7 @@ Install and Run ACRN on the Target System
 
    and then the ``histapp.py`` application::
 
+     pip install "numpy<2"
      sudo python3 /root/histapp.py
 
    At this point, the HMI_VM is running and we've started the HMI parts of
@@ -606,7 +550,7 @@ Install and Run ACRN on the Target System
 
       ubuntu login: root
       Password:
-      Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 5.15.44-rt46-acrn-kernel-rtvm+ x86_64)
+      Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 5.15.71-rt46-acrn-kernel-rtvm+ x86_64)
 
       . . .
 

doc/index.rst

@@ -78,6 +78,7 @@ license.
    contribute
    release_notes/index
    asa
+   projects/index
    glossary
    genindex
 

doc/projects/index.rst

@@ -0,0 +1,13 @@
+.. _projects:
+
+Projects
+########
+
+Here is documentation for projects that build on the initial and continuing work
+from the ACRN development team at Intel.
+
+.. toctree::
+   :maxdepth: 1
+
+   multi-arch-support
+

doc/projects/multi-arch-support.rst

@@ -0,0 +1,38 @@
+.. _multi-arch-support:
+
+Hypervisor Multi-Architecture and RISC-V Support
+################################################
+
+.. note:: This is a preliminary draft of a planned and as yet unreleased effort
+   to port the ACRN Hypervisor to non-Intel architectures.
+
+From its first release in July 2018, the ACRN Hypervisor was designed for and
+targeted to Intel platforms and relied on Intel Virtualization Technology (Intel
+VT). From that base, we're expanding support to enable the ACRN hypervisor to
+RISC-V64 architecture with a Hypervisor Extension.
+
+RISC-V Support
+**************
+
+Adding multi-architecture support begins by refining the current architecture
+abstraction layer and defining architecture-neutral APIs covering the management
+of cores, caches, memory, interrupts, timers, and hardware virtualization
+facilities.  Then an implementation of those APIs for RISC-V will be introduced.
+
+Based on its wide availability and flexibility, QEMU is the first RISC-V
+(virtual) platform this project targets. Real platforms may be selected later
+based on business and community interests.
+
+Current State
+=============
+
+This project is currently under development and is not yet ready for production.
+Once this support is implemented and has sufficient quality, this port will
+become a part of the upstream ACRN project and we'll continue development there
+and encourage contributions by the ACRN community.
+
+License
+=======
+
+This project will be released under the BSD-3-Clause license, the same as the
+rest of project ACRN.

doc/reference/hardware.rst

@@ -66,47 +66,60 @@ level includes the activities described in the lower levels.
 .. _ASRock iEP-9010E:
    https://www.asrockind.com/en-gb/iEP-9010E
 
-+------------------------+----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------+
-|                        |                            | .. rst-class::                                                                                                                                                |
-|                        |                            |    centered                                                                                                                                                   |
-|                        |                            |                                                                                                                                                               |
-|                        |                            |    ACRN Version                                                                                                                                               |
-|                        |                            +-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+
-| Intel Processor Family | Tested Products            | .. rst-class::    | .. rst-class::    | .. rst-class::    | .. rst-class::    | .. rst-class::    | .. rst-class::    | .. rst-class::    | .. rst-class::    |
-|                        |                            |    centered       |    centered       |    centered       |    centered       |    centered       |    centered       |    centered       |    centered       |
-|                        |                            |                   |                   |                   |                   |                   |                   |                   |                   |
-|                        |                            |    v1.0           |    v1.6.1         |    v2.0           |    v2.5           |    v2.6           |    v2.7           |    v3.0           |    v3.1           |
-+========================+============================+===================+===================+===================+===================+===================+===================+===================+===================+
-| Alder Lake             | | `ASRock iEPF-9010S-EY4`_,|                                                                                                                       | .. rst-class::    | .. rst-class::    |
-|                        | | `ASRock iEP-9010E`_      |                                                                                                                       |    centered       |    centered       |
-|                        |                            |                                                                                                                       |                   |                   |
-|                        |                            |                                                                                                                       |    Release        |    Community      |
-+------------------------+----------------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+
-| Tiger Lake             | `Vecow SPC-7100`_          |                                                                                                                       | .. rst-class::                        |
-|                        |                            |                                                                                                                       |    centered                           |
-|                        |                            |                                                                                                                       |                                       |
-|                        |                            |                                                                                                                       |    Maintenance                        |
-+------------------------+----------------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+---------------------------------------+
-| Tiger Lake             | `NUC11TNHi5`_              |                   |                   |                   | .. rst-class::    | .. rst-class::                        | .. rst-class::                        |
-|                        |                            |                   |                   |                   |    centered       |    centered                           |    centered                           |
-|                        |                            |                   |                   |                   |                   |                                       |                                       |
-|                        |                            |                   |                   |                   |    Release        |    Maintenance                        |    Community                          |
-+------------------------+----------------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+---------------------------------------+
-| Whiskey Lake           | `WHL-IPC-I5`_              |                   |                   | .. rst-class::    | .. rst-class::                        | .. rst-class::                                            |
-|                        |                            |                   |                   |    centered       |    centered                           |    centered                                               |
-|                        |                            |                   |                   |                   |                                       |                                                           |
-|                        |                            |                   |                   |    Release        |    Maintenance                        |    Community                                              |
-+------------------------+----------------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-----------------------------------------------------------+
-| Kaby Lake              | `NUC7i7DNHE`_              |                   | .. rst-class::    | .. rst-class::                        | .. rst-class::                                                                |
-|                        |                            |                   |    centered       |    centered                           |    centered                                                                   |
-|                        |                            |                   |                   |                                       |                                                                               |
-|                        |                            |                   |    Release        |    Maintenance                        |    Community                                                                  |
-+------------------------+----------------------------+-------------------+-------------------+---------------------------------------+-------------------------------------------------------------------------------+
-| Apollo Lake            | | `NUC6CAYH`_,             | .. rst-class::    | .. rst-class::    | .. rst-class::                                                                                                        |
-|                        | | `UP2-N3350`_,            |    centered       |    centered       |    centered                                                                                                           |
-|                        | | `UP2-N4200`_,            |                   |                   |                                                                                                                       |
-|                        | | `UP2-x5-E3940`_          |    Release        |    Maintenance    |    Community                                                                                                          |
-+------------------------+----------------------------+-------------------+-------------------+-----------------------------------------------------------------------------------------------------------------------+
+.. _ASUS PN64-E1:
+   https://www.asus.com/displays-desktops/mini-pcs/pn-series/asus-expertcenter-pn64-e1/
+
+.. important::
+   We recommend you use a system configuration that includes a serial port.
+
+.. # Note For easier editing, I'm using unicode non-printing spaces in this table to help force the width of the first two columns to help prevent wrapping (using   isn't compact enough)
+
++------------------------+---------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|                        |                                 | .. rst-class::                                                                                                                                                                                        |
+|                        |                                 |    centered                                                                                                                                                                                           |
+|                        |                                 |                                                                                                                                                                                                       |
+|                        |                                 |    ACRN Version                                                                                                                                                                                       |
+|                        |                                 +-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+
+| Intel Processor Family | Tested Products                 | .. rst-class::    | .. rst-class::    | .. rst-class::    | .. rst-class::    | .. rst-class::    | .. rst-class::    | .. rst-class::    | .. rst-class::    | .. rst-class::    | .. rst-class::    |
+| Code Name              |                                 |    centered       |    centered       |    centered       |    centered       |    centered       |    centered       |    centered       |    centered       |    centered       |    centered       |
+|                        |                                 |                   |                   |                   |                   |                   |                   |                   |                   |                   |                   |
+|                        |                                 |    v1.0           |    v1.6.1         |    v2.0           |    v2.5           |    v2.6           |    v2.7           |    v3.0           |    v3.1           |    v3.2           |    v3.3           |
++========================+=================================+===================+===================+===================+===================+===================+===================+===================+===================+===================+===================+
+| Raptor Lake            | `ASUS PN64-E1`_                 |                                                                                                                                                               | .. rst-class::    | .. rst-class::    |
+|                        |                                 |                                                                                                                                                               |    centered       |    centered       |
+|                        |                                 |                                                                                                                                                               |                   |                   |
+|                        |                                 |                                                                                                                                                               |    Community      |    Maintenance    |
++------------------------+---------------------------------+-----------------------------------------------------------------------------------------------------------------------+-------------------+-------------------+-------------------+-------------------+
+| Alder Lake             | | `ASRock iEPF-9010S-EY4`_,     |                                                                                                                       | .. rst-class::    | .. rst-class::                                            |
+|                        | | `ASRock iEP-9010E`_           |                                                                                                                       |    centered       |    centered                                               |
+|                        |                                 |                                                                                                                       |                   |                                                           |
+|                        |                                 |                                                                                                                       |    Release        |    Community                                              |
++------------------------+---------------------------------+-----------------------------------------------------------------------------------------------------------------------+-------------------+---------------------------------------+-------------------+
+| Tiger Lake             | `Vecow SPC-7100`_               |                                                                                                                       | .. rst-class::                                            | .. rst-class::    |
+|                        |                                 |                                                                                                                       |    centered                                               |    centered       |
+|                        |                                 |                                                                                                                       |                                                           |                   |
+|                        |                                 |                                                                                                                       |    Maintenance                                            |    Community      |
++------------------------+---------------------------------+-----------------------------------------------------------+-------------------+---------------------------------------+-----------------------------------------------------------+-------------------+
+| Tiger Lake             | `NUC11TNHi5`_                   |                                                           | .. rst-class::    | .. rst-class::                        | .. rst-class::                                                                |
+|                        |                                 |                                                           |    centered       |    centered                           |    centered                                                                   |
+|                        |                                 |                                                           |                   |                                       |                                                                               |
+|                        |                                 |                                                           |    Release        |    Maintenance                        |    Community                                                                  |
++------------------------+---------------------------------+---------------------------------------+-------------------+-------------------+-------------------+-------------------+-------------------------------------------------------------------------------+
+| Whiskey Lake           | `WHL-IPC-I5`_                   |                                       | .. rst-class::    | .. rst-class::                        | .. rst-class::                                                                                    |
+|                        |                                 |                                       |    centered       |    centered                           |    centered                                                                                       |
+|                        |                                 |                                       |                   |                                       |                                                                                                   |
+|                        |                                 |                                       |    Release        |    Maintenance                        |    Community                                                                                      |
++------------------------+---------------------------------+-------------------+-------------------+-------------------+-------------------+-------------------+---------------------------------------------------------------------------------------------------+
+| Kaby Lake              | `NUC7i7DNHE`_                   |                   | .. rst-class::    | .. rst-class::                        | .. rst-class::                                                                                                        |
+|                        |                                 |                   |    centered       |    centered                           |    centered                                                                                                           |
+|                        |                                 |                   |                   |                                       |                                                                                                                       |
+|                        |                                 |                   |    Release        |    Maintenance                        |    Community                                                                                                          |
++------------------------+---------------------------------+-------------------+-------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------------------+
+| Apollo Lake            | | `NUC6CAYH`_,                  | .. rst-class::    | .. rst-class::    | .. rst-class::                                                                                                                                                |
+|                        | | `UP2-N3350`_,                 |    centered       |    centered       |    centered                                                                                                                                                   |
+|                        | | `UP2-N4200`_,                 |                   |                   |                                                                                                                                                               |
+|                        | | `UP2-x5-E3940`_               |    Release        |    Maintenance    |    Community                                                                                                                                                  |
++------------------------+---------------------------------+-------------------+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+
 
 * **Release**: New ACRN features are complete and tested for the listed product.
   This product is recommended for this ACRN version. Support for older products
@@ -136,4 +149,4 @@ you will use to configure the ACRN hypervisor, as described in the
 acrn-user@lists.projectacrn.org mailing list on your findings about
 unlisted products.
 
-.. # vim: tw=200
+.. # vim: tw=300

doc/release_notes/release_notes_3.0.2.rst

@@ -0,0 +1,56 @@
+.. _release_notes_3.0.2:
+
+ACRN v3.0.2 (Nov 2022)
+######################
+
+We are pleased to announce the release of the Project ACRN hypervisor
+version 3.0.2 with hot fixes to the v3.0 release.
+
+ACRN is a flexible, lightweight reference hypervisor that is built with
+real-time and safety-criticality in mind. It is optimized to streamline
+embedded development through an open-source platform. See the
+:ref:`introduction` introduction for more information.
+
+All project ACRN source code is maintained in the
+https://github.com/projectacrn/acrn-hypervisor repository and includes
+folders for the ACRN hypervisor, the ACRN device model, tools, and
+documentation. You can download this source code either as a zip or
+tar.gz file (see the `ACRN v3.0.2 GitHub release page
+<https://github.com/projectacrn/acrn-hypervisor/releases/tag/v3.0.2>`_) or
+use Git ``clone`` and ``checkout`` commands::
+
+   git clone https://github.com/projectacrn/acrn-hypervisor
+   cd acrn-hypervisor
+   git checkout v3.0.2
+
+The project's online technical documentation is also tagged to
+correspond with a specific release: generated v3.0 documents can be
+found at https://projectacrn.github.io/3.0/.  Documentation for the
+latest development branch is found at https://projectacrn.github.io/latest/.
+
+ACRN v3.0.2 requires Ubuntu 20.04 (as does v3.0).  Follow the instructions in the
+:ref:`gsg` to get started with ACRN.
+
+
+What's New in v3.0.2
+********************
+
+Passthrough PMU (performance monitor unit) to user VM only in debug builds
+  ACRN v2.6 introduced PMU passthrough to RT VMs that have LAPIC passthrough
+  enabled. This is useful for performance profiling at development time but can
+  cause workload interference in a production build. PMU passthrough is only
+  enabled now for a hypervisor debug mode build.
+
+Added tarfile member sanitization to Python tarfile package extractall() calls
+  A vulnerability in the ACRN Configurator is patched, where files extracted
+  from a maliciously crafted tarball could  be written to somewhere outside the
+  target directory and cause unsafe behavior.
+
+Run executables with absolute paths in board inspector
+  Using partial executable paths in the board inspector may cause unintended
+  results when another executable has the same name and is found via PATH
+  settings. The board inspector now uses absolute paths to executable.
+
+
+
+See :ref:`release_notes_3.0` and :ref:`release_notes_3.0.1` for additional release information.

doc/release_notes/release_notes_3.2.rst

@@ -0,0 +1,188 @@
+.. _release_notes_3.2:
+
+ACRN v3.2 (Aug 2023)
+####################
+
+We are pleased to announce the release of the Project ACRN hypervisor
+version 3.2.
+
+ACRN is a flexible, lightweight reference hypervisor that is built with
+real-time and safety-criticality in mind. It is optimized to streamline
+embedded development through an open-source platform. See the
+:ref:`introduction` introduction for more information.
+
+All project ACRN source code is maintained in the
+https://github.com/projectacrn/acrn-hypervisor repository and includes
+folders for the ACRN hypervisor, the ACRN device model, tools, and
+documentation. You can download this source code either as a zip or
+tar.gz file (see the `ACRN v3.2 GitHub release page
+<https://github.com/projectacrn/acrn-hypervisor/releases/tag/v3.2>`_) or
+use Git ``clone`` and ``checkout`` commands::
+
+   git clone https://github.com/projectacrn/acrn-hypervisor
+   cd acrn-hypervisor
+   git checkout v3.2
+
+The project's online technical documentation is also tagged to
+correspond with a specific release: generated v3.2 documents can be
+found at https://projectacrn.github.io/3.2/.  Documentation for the
+latest development branch is found at https://projectacrn.github.io/latest/.
+
+ACRN v3.2 requires Ubuntu 22.04.  Follow the instructions in the
+:ref:`gsg` to get started with ACRN.
+
+
+What's New in v3.2
+******************
+
+Enabling New Generation Intel® Processors
+  ACRN v3.2 release now supports 12th Generation Intel® Atom N-Series Processors
+  (formerly code named Alder Lake N) and 13th Generation Intel® Core™ Mobile and
+  Desktop Processors (formerly code named Raptor Lake) with real-time SKUs.
+
+Hypervisor-Managed Processor Performance Policy Controls
+  The ACRN hypervisor Configurator now provides processor performance policy
+  control for CPU frequency if the system supports hardware-controlled
+  performance states (HWP). This ensures that loaded CPUs can run at least at
+  their guaranteed frequency level.
+
+New Debianization Solution for ACRN
+  The v3.2 release provides a standardized approach for ACRN debianization.
+  We provide an option to build each component as a separate Debian package and
+  allow users to select the binary to deploy at package installation time. Users
+  can also reselect the binary by reconfiguring the installed package.
+
+Service VM Upgraded to use Ubuntu 22.04
+  The v3.2 release upgrades the Service VM OS from Ubuntu 20.04 to 22.04.
+
+Upgrading to v3.2 from Previous Releases
+****************************************
+
+We recommend you generate a new board XML for your target system with the v3.2
+Board Inspector. You should also use the v3.2 Configurator to generate a new
+scenario XML file and launch scripts. Scenario XML files and launch scripts
+created by previous ACRN versions will not work with the v3.2 ACRN hypervisor
+build process and could produce unexpected errors during the build.
+
+Given the scope of changes for the v3.2 release, we have recommendations for how
+to upgrade from prior ACRN versions:
+
+1. Start fresh from our :ref:`gsg`. This is the best way to ensure you have a
+   v3.2-ready board XML file from your target system and generate a new scenario
+   XML and launch scripts from the new ACRN Configurator that are consistent and
+   will work for the v3.2 build system.
+#. Use the :ref:`upgrader tool <upgrading_configuration>` to attempt upgrading
+   your configuration files that worked with prior releases. You'll need the
+   matched pair of scenario XML and launch XML files from a prior configuration,
+   and use them to create a new merged scenario XML file. See
+   :ref:`upgrading_configuration` for details.
+#. Manually edit your previous older scenario XML and launch XML files to make them
+   compatible with v3.2.  This is not our recommended approach.
+
+Here are some additional details about upgrading to the v3.2 release.
+
+Generate New Board XML
+======================
+
+Board XML files, generated by ACRN Board Inspector, contain board information
+that is essential for building the ACRN hypervisor and setting up User VMs.
+Compared to previous versions, ACRN v3.2 adds the following information to the
+board XML file for supporting new features and fixes:
+
+* Add CPU frequency information. (See :acrn-pr:`8174`)
+* Get connected displays and add them as child nodes to a corresponding graphics
+  card. (See :acrn-pr:`8230`)
+* Add bdf information to an ioport serial controller. (See :acrn-pr:`8237`)
+* Stop running and report an error if VMD is enabled in the BIOS setting. (See
+  :acrn-pr:`8328`)
+* Report an error if a USB device is unplugged or disconnected while extracting
+  USB device information. (See :acrn-pr:`8326`)
+* Handle PCI functions with an undefined header layout. (See :acrn-pr:`8233`)
+
+See the :ref:`board_inspector_tool` documentation for a complete list of steps
+to install and run the tool.
+
+Update Configuration Options
+============================
+
+As explained in this :ref:`upgrading_configuration` document, we do provide a
+tool that can assist upgrading your existing pre-v3.2 scenario XML files in the
+new merged v3.2 format. From there, you can use the v3.2 ACRN Configurator UI to
+open the upgraded scenario file for viewing and further editing if the upgrader
+tool lost meaningful data during the conversion.
+
+The ACRN Configurator adds the following features and fixes to improve the user
+experience:
+
+* Support virtio GPU configuration. (See :acrn-pr:`8248`)
+* Determine SSRAM_ENABLED value automatically. (See :acrn-pr:`8232`)
+* Add "CPU performance policy type" option. (See :acrn-pr:`8174`)
+* Add "exclusively owns physical CPUs" checkbox to pre-launched and
+  post-launched VMs. (See :acrn-pr:`8290`)
+* Generate ``config_summary.rst`` when saving scenario XML and launch scripts.
+  (See :acrn-pr:`8309`)
+
+See the :ref:`scenario-config-options` documentation for details about all the
+available configuration options in the new Configurator.
+
+
+Document Updates
+****************
+
+Here are some of the more significant documentation updates from the v3.1 release:
+
+.. rst-class:: rst-columns2
+
+* :ref:`asa`
+* :ref:`hld-security`
+* :ref:`hv-cpu-virt`
+* :ref:`gsg`
+* :ref:`GSG_sample_app`
+* :ref:`release_notes_3.2`
+* :ref:`release_notes_3.0.2`
+* :ref:`acrn_configurator_tool`
+* :ref:`acrn_doc`
+* :ref:`enable_multiple_displays`
+* :ref:`acrn-dm_parameters-and-launch-script`
+* :ref:`scenario-config-options`
+
+Fixed Issues Details
+********************
+
+.. comment example item
+   - :acrn-issue:`5626` - Host Call Trace once detected
+
+- :acrn-issue:`8435` - Post-launch RTVM and WaaG running simultaneously will cause Windows kernel crash
+- :acrn-issue:`8445` - Fix security vulnerability for configurator dependent library
+- :acrn-issue:`8454` - Fail to boot RTVM or UaaG when passthru Ethernet controller
+- :acrn-issue:`8448` - The script in Sample Application Guide is not working
+- :acrn-issue:`8352` - Sample app fails to build for v3.2 RC1
+- :acrn-issue:`8439` - Possible null pointer dereference/uninitialized variable/buffer overflow in code
+- :acrn-issue:`8435` - Post-launch RTVM and WaaG running simultaneously will cause Windows kernel crash
+- :acrn-issue:`8432` - Flickering screen when passing ADL-N and RPL-P platforms
+- :acrn-issue:`8413` - hypervisor: 'vm_config' may be used uninitialized [-Werror=maybe-uninitialized]
+- :acrn-issue:`8382` - Failed to build with gcc 12
+- :acrn-issue:`8422` - Failed to generate config summary and launch scripts if CAT is enabled in configurator
+- :acrn-issue:`8395` - Configurator load fails because it needs to download RstCloth packages.
+- :acrn-issue:`8380` - Cannot generate XML file for target system
+- :acrn-issue:`8388` - Fail to generate board XML because of non-ASCII characters
+- :acrn-issue:`8385` - Failed to generate config_summary.rst when board.xml has "module" node under the "processors/die"
+- :acrn-issue:`8359` - GSG: change the method of checking kernel version of grub menuentry
+- :acrn-issue:`8246` - Debianization improvement
+- :acrn-issue:`8344` - debian/debian_build.sh fails when a work folder contains files other than XML
+- :acrn-issue:`8111` - Sync between Service VM OS and RTVM failed when startup hence life_mngr cannot work
+- :acrn-issue:`8315` - Invoking a command with partial executable path in Board Inspector Python file
+- :acrn-issue:`8274` - Wrong kernel cmdline added in grub menu when install acrn-hypervisor
+
+
+Known Issues
+************
+
+- :acrn-issue:`6631` - Kata support is broken since v2.7
+- :acrn-issue:`6978` - openstack failed since ACRN v2.7
+- :acrn-issue:`7827` - Pre_launched standard VMs cannot share CPU with Service VM in configurator
+- :acrn-issue:`8202` - HV fail to boot acrn on QEMU
+- :acrn-issue:`8471` - PTM enabling failure on i225 NIC
+- :acrn-issue:`8472` - Failed to clear memory for post-launched standard VM
+- :acrn-issue:`8473` - Missing VirtIO GPU Windows VF driver
+

doc/release_notes/release_notes_3.3.rst

@@ -0,0 +1,199 @@
+.. _release_notes_3.3:
+
+ACRN v3.3 (Aug 2024)
+####################
+
+We are pleased to announce the release of the Project ACRN hypervisor
+version 3.3.
+
+ACRN is a flexible, lightweight reference hypervisor that is built with
+real-time and safety-criticality in mind. It is optimized to streamline
+embedded development through an open-source platform. See the
+:ref:`introduction` introduction for more information.
+
+All project ACRN source code is maintained in the
+https://github.com/projectacrn/acrn-hypervisor repository and includes
+folders for the ACRN hypervisor, the ACRN device model, tools, and
+documentation. You can download this source code either as a zip or
+tar.gz file (see the `ACRN v3.3 GitHub release page
+<https://github.com/projectacrn/acrn-hypervisor/releases/tag/v3.3>`_) or
+use Git ``clone`` and ``checkout`` commands::
+
+   git clone https://github.com/projectacrn/acrn-hypervisor
+   cd acrn-hypervisor
+   git checkout v3.3
+
+The project's online technical documentation is also tagged to
+correspond with a specific release: generated v3.3 documents can be
+found at https://projectacrn.github.io/3.3/. Documentation for the
+latest development branch is found at https://projectacrn.github.io/latest/.
+
+ACRN v3.3 requires Ubuntu 22.04. Follow the instructions in the
+:ref:`gsg` to get started with ACRN.
+
+
+What's New in v3.3
+******************
+
+Generic Main VM Support
+  The v3.3 release now supports a new scenario called "Main VM". A "Service VM"
+  has two characteristics: (1) it is the default owner of physical resources 
+  and (2) it can invoke VM management hypercalls. This release adds support 
+  to configure a VM with only the physical resource ownership characteristic 
+  and calling this a "Main VM". An example scenario is a pre-launched TEE
+  (Trusted Execution Environment) VM and a main REE (Rich Execution Environment)
+  VM.
+
+Enabling Celadon as User VM
+  The acrn hypervisor now supports Celadon as User VM OS. Celadon is an 
+  open-source project by Intel that provides a reference software stack for Android
+  on Intel architecture platforms, aiming to enable developers to optimize and test
+  Android on Intel-based devices.
+
+Virtual Processor Performance Controls (vHWP)
+  The v3.3 release provides virtual HWP feature to a VM so that the VM can check
+  hardware performance ranges and adjust performance levels for performance or 
+  power consumption.
+
+Virtual Thermal Monitor and Software Controlled Clock Facilities
+  This release is able to virtualize processor thermal sensors and
+  controls for thermal management in VMs.
+
+Hypervisor Runtime Core PM 
+  The v3.3 release enhances processor power management in the hypervisor
+  at runtime to reduce power consumption when a core is idle.
+
+Guest S3 Support
+  The v3.3 release supports suspend-to-RAM of post-launched VMs running
+  with OVMF.
+
+System Performance Optimization - Virtio-blk Multi-Virtqueue Support
+  This release optimizes the virtio-block backend performance by allowing
+  multiple virtqueues between a frontend driver and the backend.
+
+Notification of VM Events 
+  Emit events (such as RTC changes and power cycles) to the monitor socket for 
+  customizing further actions upon such events.
+
+Enhance device model passthrough
+  This release support passthrough PCI device with legacy interrupt, some ACPI device like
+  GPIO controller, legacy UART.
+
+ServiceVM supervisor role
+  User can config ServicVM as supervisor role in result it can manage the power status of 
+  any guest VM.
+
+
+Upgrading to v3.3 from Previous Releases
+****************************************
+
+We recommend you generate a new board XML for your target system with the v3.3
+Board Inspector. You should also use the v3.3 Configurator to generate a new
+scenario XML file and launch scripts. Scenario XML files and launch scripts
+created by previous ACRN versions will not work with the v3.3 ACRN hypervisor
+build process and could produce unexpected errors during the build.
+
+Given the scope of changes for the v3.3 release, we have recommendations for how
+to upgrade from prior ACRN versions:
+
+1. Start fresh from our :ref:`gsg`. This is the best way to ensure you have a
+   v3.3-ready board XML file from your target system and generate a new scenario
+   XML and launch scripts from the new ACRN Configurator that are consistent and
+   will work for the v3.3 build system.
+#. Use the :ref:`upgrader tool <upgrading_configuration>` to attempt upgrading
+   your configuration files that worked with prior releases. You'll need the
+   matched pair of scenario XML and launch XML files from a prior configuration,
+   and use them to create a new merged scenario XML file. See
+   :ref:`upgrading_configuration` for details.
+#. Manually edit your previous older scenario XML and launch XML files to make them
+   compatible with v3.3.  This is not our recommended approach.
+
+Here are some additional details about upgrading to the v3.3 release.
+
+Generate New Board XML
+======================
+
+Board XML files, generated by ACRN Board Inspector, contain board information
+that is essential for building the ACRN hypervisor and setting up User VMs.
+Compared to previous versions, ACRN v3.3 adds the following information to the
+board XML file for supporting new features and fixes:
+
+* Fix typo in PCIe PTM Capability name (See :acrn-pr:`8607`)
+* Support motherboard which exposes MCFG1/MCFG2 instad of one ACPI MCFG 
+  table. (See :acrn-pr:`8513`)
+
+See the :ref:`board_inspector_tool` documentation for a complete list of steps
+to install and run the tool.
+
+Update Configuration Options
+============================
+
+As explained in this :ref:`upgrading_configuration` document, we do provide a
+tool that can assist upgrading your existing pre-v3.3 scenario XML files in the
+new merged v3.3 format. From there, you can use the v3.3 ACRN Configurator UI to
+open the upgraded scenario file for viewing and further editing if the upgrader
+tool lost meaningful data during the conversion.
+
+The ACRN Configurator adds the following features and fixes to improve the user
+experience:
+
+* Support Main VM configuration. (See :acrn-pr:`8658`)
+* Change Service VM to supervisor role. (See :acrn-pr:`8630`)
+* Fix Vue3 version and update braces version (See :acrn-pr:`8627`)
+* Fix openssl's vulnerability for tauri (See :acrn-pr:`8670`)
+* Fix v-model used on props for Vue3 making strictly checking (See :acrn-pr:`8597`)
+* Support vUART two options in configurator (See :acrn-pr:`8649`)
+* Add checking cpu affinity and serial port for post-launch VM and hypervisor
+  while the user click to append a new VM. (See :acrn-pr:`8602`)
+
+See the :ref:`scenario-config-options` documentation for details about all the
+available configuration options in the new Configurator.
+
+
+Document Updates
+****************
+
+Here are some of the more significant documentation updates from the v3.2 release:
+
+.. rst-class:: rst-columns2
+
+* :ref:`gsg`
+* :ref:`using_celadon_as_user_vm`
+* :ref:`release_notes_3.3`
+* :ref:`hv-config`
+* :ref:`acrn_configurator_tool`
+* :ref:`GSG_sample_app`
+* :ref:`acrnshell`
+
+
+Fixed Issues Details
+********************
+
+.. comment example item
+   - :acrn-issue:`5626` - Host Call Trace once detected
+
+- :acrn-issue:`8608` - hybrid vcpuid support 
+- :acrn-issue:`8590` - Hypervisor crashes after rebooting post-launched vm with passthrogh device for lots of times 
+- :acrn-issue:`8599` - Should clear pcpu_active_bitmap in start_pcpu 
+- :acrn-issue:`8590` - Hypervisor crashes after rebooting post-launched vm with passthrogh device for lots of times 
+- :acrn-issue:`8576` - Update-grub failed with GRUB 2.12 
+- :acrn-issue:`8518` - Initial boot log is lost in vm_console 
+- :acrn-issue:`8509` - S3 feature of Service VM OS is not available 
+- :acrn-issue:`8506` - Unable to passthrough USB device on bus 5 to guest 
+- :acrn-issue:`8500` - Add weight support for BVT scheduler 
+- :acrn-issue:`8495` - Service VM dead loops when booting up on platform with reserved memory as the last e820 entry 
+- :acrn-issue:`8492` - passthru multifunction device at function 0 will cause sub-function devices lost 
+- :acrn-issue:`8537` - Emulate COM3/4 in devicemodel 
+- :acrn-issue:`8491` - need to expose service vm config pointer 
+- :acrn-issue:`8579` - debian: fix broken grub config with grub 2.12 
+- :acrn-issue:`6631` - Fix Kata support with modify network configuration
+
+Known Issues
+************
+
+- :acrn-issue:`6978` - openstack failed since ACRN v2.7
+- :acrn-issue:`7827` - Pre_launched standard VMs cannot share CPU with Service VM in configurator
+- :acrn-issue:`8471` - PTM enabling failure on i225 NIC
+- :acrn-issue:`8472` - Failed to clear memory for post-launched standard VM
+
+ 

doc/scripts/filter-known-issues.py

@@ -36,7 +36,6 @@ import logging
 import mmap
 import os
 import re
-import sre_constants
 import sys
 import traceback
 
@@ -69,7 +68,7 @@ def config_import_file(filename):
                 regex = gd['regex']
                 try:
                     r = re.compile(regex, re.MULTILINE)
-                except sre_constants.error as e:
+                except re.error as e:
                     logging.error("%s: bytes %d-%d: bad regex: %s",
                                   filename, m.start(), m.end(), e)
                     raise

doc/static/acrn-custom.css

@@ -291,7 +291,8 @@ body {
    counter-reset: step-count;
 }
 
-div.numbered-step h2::before {
+div.numbered-step h2::before,
+section.numbered-step h2::before {
   counter-increment: step-count;
   content: counter(step-count);
   background: #cccccc;

doc/tutorials/acrn_configurator_tool.rst

@@ -444,16 +444,18 @@ how to build the Debian package from source code.
 
    .. code-block:: bash
 
-      sudo apt install -y libwebkit2gtk-4.0-dev \
-         build-essential \
+      sudo apt install -y build-essential \
          curl \
          wget \
          libssl-dev \
          libgtk-3-dev \
-         libappindicator3-dev \
          librsvg2-dev \
          python3-venv
 
+      cd /tmp/
+      wget http://security.ubuntu.com/ubuntu/pool/main/w/webkit2gtk/libwebkit2gtk-4.0-37_2.44.2-0ubuntu0.22.04.1_amd64.deb http://mirrors.kernel.org/ubuntu/pool/main/i/icu/libicu70_70.1-2_amd64.deb http://security.ubuntu.com/ubuntu/pool/main/w/webkit2gtk/libjavascriptcoregtk-4.0-18_2.44.2-0ubuntu0.22.04.1_amd64.deb
+      sudo apt install ./libwebkit2gtk-4.0-37_2.44.2-0ubuntu0.22.04.1_amd64.deb ./libicu70_70.1-2_amd64.deb ./libjavascriptcoregtk-4.0-18_2.44.2-0ubuntu0.22.04.1_amd64.deb
+
 #. Install Node.js (npm included) as follows:
 
    a. We recommend using nvm to manage your Node.js runtime. It allows you to

doc/tutorials/enable_ivshmem.rst

@@ -67,6 +67,9 @@ For the shared memory region:
 #. Enter a name for the shared memory region.
 #. Select the source of the emulation, either Hypervisor or Device Model.
 #. Select the size of the shared memory region.
+#. **Enter shared memory region ID, which can be in hexadecimal or decimal format**.
+    .. note::
+       Default value is 0 and IDs in 0x001 ~ 0xFFF  are reserved, 0x1000 ~ 0xFFFF are available.
 #. Select at least two VMs that can use the shared memory region.
 #. Enter a virtual Board:Device.Function (BDF) address for each VM or leave it
    blank. If the field is blank, the tool provides an address when the
@@ -88,4 +91,4 @@ Learn More
 ACRN supports multiple inter-VM communication methods. For a comparison, see
 :ref:`inter-vm_communication`.
 
-For details on ACRN IVSHMEM high-level design, see :ref:`ivshmem-hld`.
+For details on ACRN IVSHMEM high-level design, see :ref:`ivshmem-hld`.

doc/tutorials/user_vm_guide.rst

@@ -168,6 +168,7 @@ when using one of these OSs:
 
 * :ref:`using_ubuntu_as_user_vm`
 * :ref:`using_windows_as_user_vm`
+* :ref:`using_celadon_as_user_vm`
 
 Real-time VM OS Considerations
 ******************************

doc/tutorials/using_celadon_as_user_vm.rst

@@ -0,0 +1,197 @@
+.. _using_celadon_as_user_vm:
+
+Run Celadon as the User VM OS
+#############################
+
+Introduction to Celadon
+***********************
+`Celadon`_ ---- An open source Android software reference stack for Intel architecture.
+
+This tutorial describes how to run Celadon Android as the User VM OS on ACRN hypervisor.
+
+If you want to learn more about Celadon, refer to the 
+official `Celadon documentation <https://projectceladon.github.io>`__.
+
+.. _Celadon:
+   http://github.com/projectceladon
+   
+
+Build Celadon Image from Source
+*******************************
+
+Before building the Celadon image, please make sure your development workstation 
+meets the following requirements: A 64-bit workstation running Ubuntu with **64GB memory** and 
+**350GB of free disk space**. If your workstation does not meet these requirements, 
+you may encouter unexpected errors.
+
+Follow these instructions to build the Celadon images:
+
+#. Install the repo tools:
+   
+   .. code-block:: none
+
+      mkdir -p ~/bin
+      curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
+      chmod a+x ~/bin/repo
+      export PATH=~/bin:$PATH
+
+#. Install the required building packages:
+
+   .. code-block:: none
+
+      sudo apt update
+      sudo apt install openjdk-8-jdk git ccache automake \
+       lzop bison gperf build-essential zip curl \
+       zlib1g-dev g++-multilib python3-networkx \
+       libxml2-utils bzip2 libbz2-dev libbz2-1.0 \
+       libghc-bzlib-dev squashfs-tools pngcrush \
+       schedtool dpkg-dev liblz4-tool make optipng maven \
+       libssl-dev bc bsdmainutils gettext python3-mako \
+       libelf-dev sbsigntool dosfstools mtools efitools \
+       python3-pystache git-lfs python3 flex clang libncurses5 \
+       fakeroot ncurses-dev xz-utils python3-pip ninja-build \
+       cryptsetup-bin cutils cmake pkg-config xorriso mtools
+      sudo pip3 install mako==1.1.0 meson==0.60.0 dataclasses
+      sudo su
+      cd /usr/local/
+      wget https://github.com/KhronosGroup/glslang/releases/download/SDK-candidate-26-Jul-2020/glslang-master-linux-Release.zip && \
+      unzip glslang-master-linux-Release.zip bin/glslangValidator 
+
+#. Download the source code   
+
+   .. code-block:: none
+      
+      mkdir ~/civ
+      cd ~/civ
+
+   We choose Celadon Android 14 Base Releases `CIV_00.23.04.51_A14 <https://projectceladon.github.io/celadon-documentation/release-notes/base-releases-A14.html#civ-00-23-04-51-a14>`__. 
+   Repo tool will download the entire source code into your local environments and it will cost several hours depending on your network.
+
+   .. code-block:: none
+
+      repo init -u https://github.com/projectceladon/manifest -b master -m stable-build/CIV_00.23.04.51_A14.xml
+      repo sync -c -q -j5
+
+#. Disable Trusty:
+
+   Trusty is a mandatory component since Android Oreo Desert onwards. However, it's easier to boot Celadon as VM without Trusty feature.
+   So we recommend to **disable trusty**. To disable, set the 'trusty' and 'tpm' options to false in the mixins config file 
+   ``civ/device/intel/projectceladon/caas/mixins.spec`` as follows:
+   
+   .. code-block:: none
+
+      [groups]
+      device-specific: celadon
+      treble: true
+      .... 
+      tpm: false
+      ....
+      trusty: false
+
+   After modifing ``mixins.spec``, you must run the ``civ/device/intel/mixins/mixin-update`` script to apply the changes.
+
+   Enter the following commands to initialize your build variables and specifiy your celadon lunch target using ``lunch`` target:
+
+   .. code-block:: none
+
+      source build/envsetup.sh
+      lunch caas 
+
+   Meanwhile, the following trusty related configs in **Android kernel** at
+   ``device/intel/mixins/groups/kernel/gmin64/config-lts/linux-intel-lts2022/x86_64_defconfig``
+   should be disabled as:
+   
+   .. code-block:: none
+      
+      # CONFIG_TCG_TPM is not set
+      # CONFIG_HW_RANDOM_TPM is not set
+      # CONFIG_TRUSTY is not set
+      # CONFIG_TRUSTY_LOG is not set
+      # CONFIG_TRUSTY_VIRTIO is not set
+      # CONFIG_TRUSTY_VIRTIO_IPC is not set
+      # CONFIG_TRUSTY_X86_64 is not set
+      # CONFIG_TRUSTY_BACKUP_TIMER is not set
+
+   Run `mixin` command to apply this changes:
+
+   .. code-block:: none
+
+      cd ~/civ
+      mixin
+
+#. Build Celadon flash image:
+
+   Then you are ready to build Celadon images. Build progress may cost several hours or even more depends on your building system:
+
+   .. code-block:: none
+
+      make flashfiles -j $(nproc)
+
+
+#. Flash Celadon image into disk:
+
+   Caution: Please **remain only one hard disk** (the disk will be entirely removed and flashed) in your destination platform, otherwise
+   Celadon may flash into the wrong disk and cause data loss. There are two ways to do: i. Physically
+   remove the hard disk. ii. Disable the sata(or nvme) slot in the BIOS settings.
+
+   We test this VM in an ASUS MiniPC with two disk slots: one is a m.2 nvme slot and one is a sata slot. We run service OS 
+   (Ubuntu) on a sata disk and run guest OS(Celadon Android) on a nvme disk. 
+   
+   Prepare an empty USB disk and plug it into your **development platform**, run ``lsblk`` command to find it. Assume it's ``/dev/sdc`` here.
+
+   .. code-block:: none
+
+      cd ~/civ/out/target/product/caas
+      unzip caas-flashfile-eng.dot.iso.zip
+      sudo dd if=~/civ/caas-flashfile-eng.dot.iso of=/dev/sdc status=progress   
+      sudo eject /dev/sdc
+   
+   Unplug the USB disk and plug it into your **destination platform**. Power on your destination platform and boot into this USB disk via BIOS settings. The flash progress
+   will require you press :kbd:`UP` or :kbd:`PgUp` to continue. When flash done, you can boot into Celadon Android.
+
+#. ACRN Service VM Setup
+
+   Follow the steps in this :ref:`gsg` to set up ACRN based Ubuntu and launch the Service VM.
+   Modifiy the ACRN device model parameters in ``launch_user_vm_id1.sh`` as follows:
+
+   .. code-block:: none 
+
+      dm_params=(
+         `add_cpus                                 8 9 16 17`
+         -m 8192M
+         --ovmf /usr/share/acrn/bios/OVMF.fd
+         `add_virtual_device                       1:0 lpc`
+         `add_virtual_device                       0:0 hostbridge`
+         `add_virtual_device                       3 virtio-console @stdio:stdio_port`
+         `add_passthrough_device                   2 0000:00:02.0`
+         `add_passthrough_device                   4 0000:00:14.0`
+         `add_interrupt_storm_monitor              10000 10 1 100`
+         `add_passthrough_device                   5 0000:01:00.0`
+         `add_logger_settings                      console=4 kmsg=3 disk=5`
+         VM1
+      )
+
+#. Boot Celadon VM
+
+   Remotely connect to the target system via SSH and Boot Celadon VM via the launch script ``launch_user_vm_id1``:
+
+   .. code-block:: none
+
+      sudo chmod +x ./launch_user_vm_id1.sh
+      sudo ./launch_user_vm_id1.sh
+
+   Then the screen will temperatory go off. Wait for about one minute and the Android UI will appear as:
+
+   .. figure:: images/celadon_uservm_01.png
+      :align: center
+      :name: Android-screenlock
+      :class: drop-shadow
+
+   .. figure:: images/celadon_uservm_02.png
+      :align: center
+      :name: Android-desktop
+      :class: drop-shadow
+
+
+
+