The QEMU log file is essentially about fine grain tracing of QEMU
internals and mostly useful for developpers, not production. Notably,
the log file isn't limited in size, nor rotated in any way. It means
that a container running in the VM could possibly flood the log file
with a guest triggerable trace. For example, on openshift, the log
file is supposed to reside on a per-VM 14 GiB tmpfs mount. This means
that each pod running with the kata runtime could potentially consume
this amount of host RAM which is not acceptable.
Error messages are best collected from QEMU's stderr as kata is doing
now since PR #5736 was merged. Drop support for the QEMU log file
because it doesn't bring any value but can certainly do harm.
Fixes#6173
Signed-off-by: Greg Kurz <groug@kaod.org>
(cherry picked from commit 334c4b8bdc)
Signed-off-by: Greg Kurz <groug@kaod.org>
LaunchQemu now connects a pipe to QEMU's stderr and makes it
usable by callers through a Go io.ReadCloser object. As
explained in [0], all messages should be read from the pipe
before calling cmd.Wait : introduce a LogAndWait helper to handle
that.
Fixes#5780
Signed-off-by: Greg Kurz <groug@kaod.org>
(cherry picked from commit 39fe4a4b6f)
Signed-off-by: Greg Kurz <groug@kaod.org>
QEMU has always been started daemonized since the beginning. I
could not find any justification for that though, but it certainly
introduces a problem : QEMU stops logging errors when started this
way, which isn't accaptable from a support standpoint. The QEMU
community discourages the use of -daemonize ; mostly because
libvirt, QEMU's primary consummer, doesn't use this option and
prefers getting errors from QEMU's stderr through a pipe in order
to enforce rollover.
Now that virtcontainers knows how to start QEMU with a pre-
established QMP connection, let's start QEMU without -daemonize.
This requires to handle the reaping of QEMU when it terminates.
Since cmd.Wait() is blocking, call it from a goroutine.
Signed-off-by: Greg Kurz <groug@kaod.org>
(cherry picked from commit a5319c6be6)
Signed-off-by: Greg Kurz <groug@kaod.org>
LaunchCustomQemu() currently starts QEMU with cmd.Run() which is
supposed to block until the child process terminates. This assumes
that QEMU daemonizes itself, otherwise LaunchCustomQemu() would
block forever. The virtcontainers package indeed enables the
Daemonize knob in the configuration but having such an implicit
dependency on a supposedly configurable setting is ugly and fragile.
cmd.Run() is :
func (c *Cmd) Run() error {
if err := c.Start(); err != nil {
return err
}
return c.Wait()
}
Let's open-code this : govmm calls cmd.Start() and returns the
cmd to virtcontainers which calls cmd.Wait().
If QEMU doesn't start, e.g. missing binary, there won't be any
errors to collect from QEMU output. Just drop these lines in govmm.
Similarily there won't be any log file to read from in virtcontainers.
Drop that as well.
Signed-off-by: Greg Kurz <groug@kaod.org>
(cherry picked from commit bf4e3a618f)
Signed-off-by: Greg Kurz <groug@kaod.org>
Running QEMU daemonized ensures that the QMP socket is ready to
accept connections when LaunchQemu() returns. In order to be
able to run QEMU undaemonized, let's handle that part upfront.
Create a listener socket and connect to it. Pass the listener
to QEMU and pass the connected socket to QMP : this ensures
that we cannot fail to establish QMP connection and that we
can detect if QEMU exits before accepting the connection.
This is basically what libvirt does.
Signed-off-by: Greg Kurz <groug@kaod.org>
(cherry picked from commit 8a1723a5cb)
Signed-off-by: Greg Kurz <groug@kaod.org>
QEMU's -qmp option can be passed the file descriptor of a socket that
is already in listening mode. This is done with by passing `fd=XXX`
to `-qmp` instead of a path. Note that these two options are mutually
exclusive : QEMU errors out if both are passed, so we check that as
well in the validation function.
While here add the `path=` stanza in the path based case for clarity.
Signed-off-by: Greg Kurz <groug@kaod.org>
(cherry picked from commit 8a4f08cb0f)
Signed-off-by: Greg Kurz <groug@kaod.org>
When QEMU is launched daemonized, we have the guarantee that the
QMP socket is available. In order to launch a non-daemonized QEMU,
the QMP connection should be created before QEMU is started in order
to avoid a race. Introduce a variant of QMPStart() that can use such
an existing connection.
Signed-off-by: Greg Kurz <groug@kaod.org>
(cherry picked from commit 219bb8e7d0)
Signed-off-by: Greg Kurz <groug@kaod.org>
The compile option link-self-contained=yes asks rustc to use
C library startup object files that come with the compiler,
which are not available on the target s390x-unknown-linux-gnu.
A build does not contain any startup files leading to a
broken executable entry point (causing segmentation fault).
Fixes: #5522 for stable-3.0
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
(cherry picked from commit 43fcb8fd09)
Signed-off-by: Greg Kurz <groug@kaod.org>
This is to make a docker version to v20.10 in docker upstream image ubuntu:20.04 for s390x and ppc64le.
Fixes: #6211 for stable-3.0
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
(cherry picked from commit f49b89b632)
Signed-off-by: Greg Kurz <groug@kaod.org>
The key steps in how-to-hotplug-memory-arm64.md are missing, resulting in the kata qemu pod not being created successfully.
Fixes: #6105
Signed-off-by: SinghWang <wangxin_0611@126.com>
The key steps in how-to-hotplug-memory-arm64.md are missing, resulting in the kata qemu pod not being created successfully.
Fixes: #6105
Signed-off-by: SinghWang <wangxin_0611@126.com>
The key steps in how-to-hotplug-memory-arm64.md are missing, resulting in the kata qemu pod not being created successfully.
Fixes: #6105
Signed-off-by: SinghWang <wangxin_0611@126.com>
The key steps in how-to-hotplug-memory-arm64.md are missing, resulting in the kata qemu pod not being created successfully.
Fixes: #6105
Signed-off-by: SinghWang <wangxin_0611@126.com>
- stable-3.0 | kata-deploy: Fix the pod of kata deploy starts to occur an error
- Stable-3.0 | Upgrade to Cloud Hypervisor v28.0
- stable-3.0 | Snap CI backports
- stable-3.0 | package: add nydus to release artifacts
19f51c7cc release: Adapt kata-deploy for 3.0.1
d3f7b829f versions: update nydusd version
1bf7f2f68 package: add nydus to release artifacts
9cf1af873 runtime: clh: Re-generate the client code
4d6ca7623 versions: Upgrade to Cloud Hypervisor v28.0
719017d68 clh: return faster with dead clh process from isClhRunning
569ecdbe7 clh: fast exit from isClhRunning if the process was stopped
fa8a0ad49 clh: don't try to stop clh multiple times
8fbf862fa cloud-hypervisor: Fix GetThreadIDs function
9141acd94 versions: Update Cloud Hypervisor to b4e39427080
9a0ab92f6 runtime: clh: Use the new API to boot with TDX firmware (td-shim)
f3eac35b5 runtime: clh: Re-generate the client code
8a7e0efd1 versions: Upgrade to Cloud Hypervisor v27.0
a152f6034 runk: Ignore an error when calling kill cmd with --all option
50bf4434d log-parser: Simplify check
74791ed38 runtime: Fix gofmt issues
778ebb6e6 golang: Stop using io/ioutils
b5661e988 versions: Update golangci-lint
88c13b682 versions: bump containerd version
b8ce291dd build: update golang version to 1.19.2
f5e5ca427 github: Parallelise static checks
eaa7ab746 snap: Unbreak docker install
8d2fd2449 snap: Use metadata for dependencies
ab83ab6be snap: Build virtiofsd using the kata-deploy scripts
1772df5ac snap: Create a task for installing docker
2e4958644 virtiofsd: Build inside a container
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
kata-deploy files must be adapted to a new release. The cases where it
happens are when the release goes from -> to:
* main -> stable:
* kata-deploy-stable / kata-cleanup-stable: are removed
* stable -> stable:
* kata-deploy / kata-cleanup: bump the release to the new one.
There are no changes when doing an alpha release, as the files on the
"main" branch always point to the "latest" and "stable" tags.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
If a pod of kata is deployed on a machine, after the machine restarts, the pod status of kata-deploy will be CrashLoopBackOff.
Fixes: #5868
Signed-off-by: SinghWang <wangxin_0611@126.com>
This patch re-generates the client code for Cloud Hypervisor v28.0.
Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by openapi-generator.
Fixes: #5683
Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit 36545aa81a)
Through proactively checking if Cloud Hypervisor process is dead,
this patch provides a faster path for isClhRunning
Fixes: #5623
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
(cherry picked from commit 0e3ac66e76)
Use atomic operations instead of acquiring a mutex in isClhRunning.
This stops isClhRunning from generating a deadlock by trying to
reacquire an already-acquired lock when called via StopVM->terminate.
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
(cherry picked from commit 9ef68e0c7a)
Avoid executing StopVM concurrently when virtiofs dies as a result of clh
being stopped in StopVM.
Fixes: #5622
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
(cherry picked from commit 2631b08ff1)
Get vcpu thread-ids by reading cloud-hypervisor process tasks information.
Fixes: #5568
Signed-off-by: Guanglu Guo <guoguanglu@qiyi.com>
(cherry picked from commit daeee26a1e)
An API change, done a long time ago, has been exposed on Cloud
Hypervisor and we should update it on the Kata Containers side to ensure
it doesn't affect Cloud Hypervisor CI and because the change is needed
for an upcoming work to get QAT working with Cloud Hypervisor.
Fixes: #5492
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 9d286af7b4)
The new way to boot from TDX firmware (e.g. td-shim) is using the
combination of '--platform tdx=on' with '--firmware tdshim'.
Fixes: #5309
Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit 067e2b1e33)
This release has been tracked in our new [roadmap project ](https://github.com/orgs/cloud-hypervisor/projects/6) as iteration v27.0.
**Community Engagement**
A new mailing list has been created to support broader community discussions.
Please consider [subscribing](https://lists.cloudhypervisor.org/g/dev/); an announcement of a regular meeting will be
announced via this list shortly.
**Prebuilt Packages**
Prebuilt packages are now available. Please see this [document](https://github.com/cloud-hypervisor/obs-packaging/blob/main/README.md)
on how to install. These packages also include packages for the different
firmware options available.
**Network Device MTU Exposed to Guest**
The MTU for the TAP device associated with a virtio-net device is now exposed
to the guest. If the user provides a MTU with --net mtu=.. then that MTU is
applied to created TAP interfaces. This functionality is also exposed for
vhost-user-net devices including those created with the reference backend.
**Boot Tracing**
Support for generating a trace report for the boot time has been added
including a script for generating an SVG from that trace.
**Simplified Build Feature Flags**
The set of feature flags, for e.g. experimental features, have been simplified:
* msvh and kvm features provide support for those specific hypervisors
(with kvm enabled by default),
* tdx provides support for Intel TDX; and although there is no MSHV support
now it is now possible to compile with the mshv feature,
* tracing adds support for boot tracing,
* guest_debug now covers both support for gdbing a guest (formerly gdb
feature) and dumping guest memory.
The following feature flags were removed as the functionality was enabled by
default: amx, fwdebug, cmos and common.
**Asynchronous Kernel Loading**
AArch64 has gained support for loading the guest kernel asynchronously like
x86-64.
**GDB Support for AArch64**
GDB stub support (accessed through --gdb under guest_debug feature) is now
available on AArch64 as well as as x86-64.
**Notable Bug Fixes**
* This version incorporates a version of virtio-queue that addresses an issue
where a rogue guest can potentially DoS the VMM,
* Improvements around PTY handling for virtio-console and serial devices,
* Improved error handling in virtio devices.
**Deprecations**
Deprecated features will be removed in a subsequent release and users should
plan to use alternatives.
* Booting legacy firmware (compiled without a PVH header) has been deprecated.
All the firmware options (Cloud Hypervisor OVMF and Rust Hypervisor Firmware)
support booting with PVH so support for loading firmware in a legacy mode is no
longer needed. This functionality will be removed in the next release.
Details can be found: https://github.com/cloud-hypervisor/cloud-hypervisor/releases/tag/v27.0
Note: To have the new API of loading firmware for booting (e.g. boot
from td-shim), a specific commit revision after the v27.0 release is
used as the Cloud Hypervisor version from the 'versions.yaml'.
Fixes: #5309
Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit fe61070426)
Ignore an error handling that is triggered when the kill command is called
with `--all option` to the stopped container.
High-level container runtimes such as containerd call the kill command with
`--all` option in order to terminate all processes inside the container
even if the container already is stopped. Hence, a low-level runtime
should allow `kill --all` regardless of the container state like runc.
This commit reverts to the previous behavior.
Fixes: #5555
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
(cherry picked from commit 16dca4ecd4)
```
14:13:15 parse.go:306:5: S1009: should omit nil check; len() for github.com/kata-containers/kata-containers/src/tools/log-parser.kvPairs is defined as zero (gosimple)
14:13:15 if pairs == nil || len(pairs) == 0 {
14:13:15 ^
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 2f5f575a43)
It seems that bumping the version of golang and golangci-lint new format
changes are required.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit d94718fb30)
The package has been deprecated as part of 1.16 and the same
functionality is now provided by either the io or the os package.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 16b8375095)
Let's bump the golangci-lint in order to fix issues that popped up after
updating Golang to its 1.19.2 version.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 66aa330d0d)
v1.5.2 cannot be built from source by newer golang. Let's bump
containerd version to 1.6.8. The GO runtime dependency has
been moved to v1.6.6 for some time already.
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
(cherry picked from commit b3a4a16294)
So that we get the latest language fixes.
There is little use to maitain compiler backward compatibility.
Let's just set the default golang version to the latest 1.19.2.
Fixes: #5494
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
(cherry picked from commit eab8d6be13)
Although introducing an awful amount of code duplication, let's
parallelise the static checks in order to reduce its time and the space
used in the VMs running those.
While I understand there may be ways to make the whole setup less
repetitive and error prone, I'm taking the approach of:
* Make it work
* Make it right
* Make it fast
So, it's clear that I'm only attempting to make it work, and I'd
appreciate community help in order to improve the situation here. But,
for now, this is a stopgap solution.
JFYI, the time needed for run the tests on the `main` branch went down
from ~110 minutes to ~60 minutes. Plus, we're not running those on a
single VM anymore, which decreases the change to hit the space limit.
Reference: https://github.com/kata-containers/kata-containers/actions/runs/3393468605/jobs/5640842041
Ideally, each one of the following tests should be also split into
smaller tests, each test for one component, for instance.
* static-checks
* compiler-checks
* unit-tests
* unit-tests-as-root
Fixes: #5585
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 40d514aa2c)
It appears that _either_ the GitHub workflow runners have changed their
environment, or the Ubuntu archive has changed package dependencies,
resulting in the following error when building the snap:
```
Installing build dependencies: bc bison build-essential cpio curl docker.io ...
:
The following packages have unmet dependencies:
docker.io : Depends: containerd (>= 1.2.6-0ubuntu1~)
E: Unable to correct problems, you have held broken packages.
```
This PR uses the simplest solution: install the `containerd` and `runc`
packages. However, we might want to investigate alternative solutions in
the future given that the docker and containerd packages seem to have
gone wild in the Ubuntu GitHub workflow runner environment. If you
include the official docker repo (which the snap uses), a _subset_ of
the related packages is now:
- `containerd`
- `containerd.io`
- `docker-ce`
- `docker.io`
- `moby-containerd`
- `moby-engine`
- `moby-runc`
- `runc`
Fixes: #5545.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
(cherry picked from commit 990e6359b7)
Rather than hard-coding the package manager into the docker part,
use the `build-packages` section to specify the parts package
dependencies in a distro agnostic manner.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
(cherry picked from commit ca69a9ad6d)
