Commit Graph

5549 Commits

Author SHA1 Message Date
Bo Chen
e3efcfd40f runtime: Fix the format of the client code of cloud-hypervisor APIs
Regenerate the client code with the added `go-fmt` step. No functional
changes.

Fixes: #1606

Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit 0c38d9ecc4)
2021-04-01 15:15:54 -07:00
Bo Chen
5a92333f4b runtime: Format auto-generated client code for cloud-hypervisor API
This patch extends the current process of generating client code for
cloud-hypervisor API with an additional step, `go-fmt`, which will remove
the generated `client/go.mod` file and format all auto-generated code.

Fixes: #1606

Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit 52cacf8838)
2021-04-01 15:15:54 -07:00
Bo Chen
ec0424e153 versions: Update cloud-hypervisor to release v0.14.1
Highlights for cloud-hypervisor version 0.14.0 include: 1) Structured
event monitoring; 2) MSHV improvements; 3) Improved aarch64 platform; 4)
Updated hotplug documentation; 6) PTY control for serial and
virtio-console; 7) Block device rate limiting; 8) Plan to deprecate the
support of "LinuxBoot" protocol and support PVH protocol only.

Highlights for cloud-hypervisor version 0.13.0 include: 1) Wider VFIO
device support; 2) Improve huge page support; 3) MACvTAP support; 4) VHD
disk image support; 5) Improved Virtio device threading; 6) Clean
shutdown support via synthetic power button.

Details can be found:
https://github.com/cloud-hypervisor/cloud-hypervisor/releases

Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by `openapi-generator` [1-2]. As the API changes do not
impact usages in Kata, no additional changes in kata's runtime are
needed to work with the latest version of cloud-hypervisor.

[1] https://github.com/OpenAPITools/openapi-generator
[2] https://github.com/kata-containers/kata-containers/blob/main/src/runtime/virtcontainers/pkg/cloud-hypervisor/README.md

Fixes: #1591

Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit 84b62dc3b1)
2021-04-01 15:15:06 -07:00
Bin Liu
b26e94ffba
Merge pull request #1595 from bergwolf/backports-stable-2.0
kata-deploy: QEMU, for 2.x, already includes virtiofs
2021-03-31 17:00:21 +08:00
Fabiano Fidêncio
f6f4023508 kata-deploy: Get rid of references to the docker script
The docker script has been removed as part of
62cbaf4de4, but references to it were left
behind in the artifact-list.sh, release/kata-deploy-binaries.sh, and
kata-deploy/Dockerfile.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-31 14:34:42 +08:00
Fabiano Fidêncio
814e7d7285 packaging: Remove NEMU mentions
There's no more NEMU, for some time already.  Considering this, let's
just remove any mention to it as part of our project.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-31 14:34:19 +08:00
Fabiano Fidêncio
92d1197f10 kata-deploy: QEMU, for 2.x, already includes virtiofs
There's no reason to ship qemu & qemu-virtiofs when the former already
includes vitiofs support (and that's the default for 2.x deployments).

In case we will enable experimental qemu DAX support, we should add a
new target, a "qemu-experimental" target, as Carlos has been working on.

Fixes: #1424

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-31 12:13:19 +08:00
Eric Ernst
a2484d0088
Merge pull request #1574 from bergwolf/2.0.2-branch-bump
# Kata Containers 2.0.2
2021-03-30 06:05:23 -07:00
Peng Tao
9e2cbe8ea1 release: Kata Containers 2.0.2
- container: on cleanup, rm container directory for mounts path
- backport: bindmount fixes
- [backport] runtime: increase dial timeout
- kata-deploy: Use the correct image for 2.0.2 release
- [backport] shimv2: return the hypervisor's pid as the container pid
- runtime: clh-config: add runtime hooks to the clh toml
- runtime: cpuset: when creating container, don't pass cpuset details
- [backport] runtime: Fix missing 'name' field on containerd-shim-v2 logs
- [backport] Fixup systemd cgroup handling
- Backports stable 2.0
- [backport] Fix Snap CI

ac9f838 container: on cleanup, rm container directory for mounts path
2c1b957 runtime: remove unused functions
dfe5ef3 tools: fix missing SPDX license header
50aa89f runtime: fix virtiofsd RO volume sharing
57aa746 runtime: mount shared mountpoint readonly
ce2798b runtime: readonly mounts should be readonly bindmount on the host
b7208b3 runtime: increase dial timeout
a649d33 kata-deploy: Use the correct image
d87076e runtime: return hypervisor Pid in TaskExit event
2dd859b shimv2: return the hypervisor's pid as the container pid
c7db337 versions: Use release-1.18 (commit ee9128444bec10)
72af86f runtime: Fix missing 'name' field on containerd-shim-v2 logs
95b2cad clh: Use vanilla kernel.
506f4f2 cgroups: Add systemd detection when creating cgroup manager
a3e35e7 cgroups: remove unused SystemdCgroup variable and accessor/mutators
fdf69ab kernel: ACPI: Always build evged for stable kernel
56b94e2 agent: set edition = "2018" in .rustfmt.toml to fix rustfmt about async fn
0533bee packaging: add missing SPDX headers
2114576 docs: fix fluentbit logfmt parser link
bcd8fd5 versions: Update cloud-hypervisor to release v0.12.0
6fe3f33 ci: default branch has renamed to main
3f3a253 agent: Agent invokes OCI hooks with wrong PID
fc72d39 release: Rename runtime-release-notes to release-notes
ef4ebfb release: We're not compatible with Docker.
336b806 rustjail: fix blkio conversion
dd3c5fc kata-deploy: Ensure CRI-O uses the VM runtime type
93bd2e4 kata-deploy: Move the containerd workarounds to their own functions
7eb882a kata-deploy: Update README to reflect the current distributed artifacts
a60cf37 snap: fix kernel setup
ca64387 Agent: OCI hooks return malformed json
32feb10 runtime: cpuset: when creating container, don't pass cpuset details
3c618a6 agent: Remove bogus check from list_interfaces() unit test
7c888b3 osbuilder: Fix USE_DOCKER on s390x
cf81d40 ci: snap: run snap CI on every pull request
79ed33a snap: fix kernel setup
f1cea9a ci: snap: build targets that not need sudo first
4f802cc ci: snap: define proxy variables
dda4279 kernel: Don't fail if "experimental" dir doesn't exist
5888971 packaging: Fix vmlinux kernel install on s390x
ca28ca4 qemu: Add security fixes for CVE-2020-35517
50ad323 agent: Build for glibc on s390x
f8314be docs: add katacontainers end-to-end arch image
99d9a24 kata-monitor: set buildmode to exe to avoid build failing
0091b89 osbuilder: Enforcing LIBC=gnu to rootfs build for ppc64le
9da2707 kernel: Updates to kernel config for ppc64le
2a0ff0b osbuilder: Add "Agent init" on terms glossary
fa581d3 agent: add secure_join to prevent softlink escape
a3967e9 osbuilder: Fix urls to repositories
272d39b osbuilder: Use Fedora and CentOS registries
7a86c2e shimv2: log a warning and continue on post-start hook failure
5096bd6 shimv2: log a warning and continue on post-stop hook failure
3fe59a9 snap: Don't release Kata Alpha/RC in snap store
61fa4a3 rustjail: fix the issue of missing destroy contaienr cgroups
856af1a runtime: cpuset: when creating container, don't pass cpuset details
74b5874 runtime: clh-config: add runtime hooks to the clh toml

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-30 07:55:15 +00:00
Peng Tao
fc676f76de
Merge pull request #1567 from egernst/cleanup-pr-stable-2.0
container: on cleanup, rm container directory for mounts path
2021-03-30 11:12:25 +08:00
Eric Ernst
ac9f838e33 container: on cleanup, rm container directory for mounts path
A wrong path was being used for container directory when
virtiofs is utilized. This resulted in a warning message in
logs when a container is killed, or completes:

level=warning msg="Could not remove container share dir"

Without proper removal, they'd later be cleaned up when the shared
path is removed as part of stopping the sandbox.

Fixes: #1559

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-03-29 14:16:22 -07:00
Fabiano Fidêncio
9ea851ee53
Merge pull request #1485 from egernst/backport-bindmount-fixes
backport: bindmount fixes
2021-03-26 18:45:54 +01:00
Peng Tao
2c1b957642 runtime: remove unused functions
archConvertStatFs is not called by anyone and causes static checker
failures.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-26 15:30:09 +08:00
Peng Tao
dfe5ef36b4 tools: fix missing SPDX license header
As reported by the static checker.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-26 15:08:40 +08:00
Bin Liu
8a374af6b7
Merge pull request #1548 from snir911/timeout_stable2.0
[backport] runtime: increase dial timeout
2021-03-26 14:43:03 +08:00
Peng Tao
50aa89fa05 runtime: fix virtiofsd RO volume sharing
Right now we rely heavily on mount propagation to share host
files/directories to the guest. However, because virtiofsd
pivots and moves itself to a separate mount namespace, the remount
mount is not present in virtiofsd's mount. And it causes guest to be
able to write to the host RO volume.

To fix it, create a private RO mount and then move it to the host mounts
dir so that it will be present readonly in the host-guest shared dir.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-26 12:53:04 +08:00
Peng Tao
57aa746d0d runtime: mount shared mountpoint readonly
bindmount remount events are not propagated through mount subtrees,
so we have to remount the shared dir mountpoint directly.

E.g.,
```
mkdir -p source dest foo source/foo

mount -o bind --make-shared source dest

mount -o bind foo source/foo
echo bind mount rw
mount | grep foo
echo remount ro
mount -o remount,bind,ro source/foo
mount | grep foo
```
would result in:
```
bind mount rw
/dev/xvda1 on /home/ubuntu/source/foo type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda1 on /home/ubuntu/dest/foo type ext4 (rw,relatime,discard,data=ordered)
remount ro
/dev/xvda1 on /home/ubuntu/source/foo type ext4 (ro,relatime,discard,data=ordered)
/dev/xvda1 on /home/ubuntu/dest/foo type ext4 (rw,relatime,discard,data=ordered)
```

The reason is that bind mount creats new mount structs and attaches them to different mount subtrees.
However, MS_REMOUNT only looks for existing mount structs to modify and does not try to propagate the
change to mount structs in other subtrees.

Fixes: #1061
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-26 12:00:28 +08:00
Peng Tao
ce2798b688 runtime: readonly mounts should be readonly bindmount on the host
So that we get protected at the VM boundary not just the guest kernel.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-26 12:00:27 +08:00
Snir Sheriber
b7208b3c6c runtime: increase dial timeout
On some setups, starting multiple kata pods (qemu) simultaneously on the same node
might cause kata VMs booting time to increase and the pods to fail with:
Failed to check if grpc server is working: rpc error: code = DeadlineExceeded desc = timed
out connecting to vsock 1358662990:1024: unknown

Increasing default dialing timeout to 30s should cover most cases.

Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
Fixes: #1543
(backport https://github.com/kata-containers/kata-containers/pull/1544)
2021-03-25 10:01:36 +02:00
Fabiano Fidêncio
7e4dc08b0e
Merge pull request #1541 from fidencio/wip/kata-deploy-use-the-correct-image
kata-deploy: Use the correct image for 2.0.2 release
2021-03-25 08:31:44 +01:00
Fabiano Fidêncio
a649d33a45 kata-deploy: Use the correct image
Let's tag the yet non existent 2.0.2 image, so whoever deploys kata
using kata-deploy from the release tarball is pointed to the correct
image.

Fixes: #1493

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-24 00:22:32 +01:00
Peng Tao
c628ecf298
Merge pull request #1528 from fgiudici/stable-2.0_hypervisor_pid
[backport] shimv2: return the hypervisor's pid as the container pid
2021-03-23 16:31:18 +08:00
bin
d87076eea5 runtime: return hypervisor Pid in TaskExit event
Other RPC calls return Pid of hypervisor, the TaskExit should
return the same Pid.

Fixes: #1497

Signed-off-by: bin <bin@hyper.sh>

(backport https://github.com/kata-containers/kata-containers/pull/1498)
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
[ fix missing GetHypervisorPid method in MockSandbox ]
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-23 15:34:45 +08:00
fupan.lfp
2dd859bfce shimv2: return the hypervisor's pid as the container pid
Since the kata's hypervisor process is in the network namespace,
which is close to container's process, and some host metrics
such as cadvisor can use this pid to access the network namespace
to get some network metrics. Thus this commit replace the shim's
pid with the hypervisor's pid.

Fixes: #1451

Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>

(backport https://github.com/kata-containers/kata-containers/pull/1452)
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
2021-03-23 15:34:45 +08:00
Peng Tao
4c9af982e6
Merge pull request #1426 from egernst/stable-2.0-clh-hooks
runtime: clh-config: add runtime hooks to the clh toml
2021-03-23 15:18:45 +08:00
Peng Tao
06f964843a
Merge pull request #1427 from egernst/stable-2.0-cpusets
runtime: cpuset: when creating container, don't pass cpuset details
2021-03-23 15:17:36 +08:00
Peng Tao
c27c3c40dd
Merge pull request #1529 from wainersm/for_stable-2.0
[backport] runtime: Fix missing 'name' field on containerd-shim-v2 logs
2021-03-23 15:14:15 +08:00
Peng Tao
476467115f
Merge pull request #1522 from fgiudici/stable-2.0
[backport] Fixup systemd cgroup handling
2021-03-23 15:13:53 +08:00
Fabiano Fidêncio
73645d1742
Merge pull request #1466 from bergwolf/backports-stable-2.0
Backports stable 2.0
2021-03-22 19:30:24 +01:00
Fabiano Fidêncio
c7db337f10 versions: Use release-1.18 (commit ee9128444bec10)
Let's update CRI-O version to the commit which introduced the fix for
the "k8s-copy-file" tests.

Fixes: #1080

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-22 20:35:15 +08:00
Wainer dos Santos Moschetta
72af86f686 runtime: Fix missing 'name' field on containerd-shim-v2 logs
Each Kata Containers application should generate log records with a specified
structure. Currently on containerd-shim-v2's logs, the required 'name' field
is missing. This changed its logger to append the application name on each
and every emitted entries.

Fixes #1479
Related-to: github.com/kata-containers/tests/issues/3260
Suggested-by: James O. D. Hunt <james.o.hunt@intel.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-03-18 14:53:22 -04:00
Carlos Venegas
95b2cad095 clh: Use vanilla kernel.
Qemu config alredy use vanilla kernel build for virtiofs.

Lets make cosisntent the usage of kernel.

Fixes: #1302

Depends-on: github.com/kata-containers/tests#3325
Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-18 11:22:33 +08:00
Eric Ernsteernst
506f4f2adc cgroups: Add systemd detection when creating cgroup manager
Look at the provided cgroup path to determine whether systemd is being
used to manage the cgroups. With this, systemd cgroups are being detected
and created appropriately for the sandbox.

Fixes: #599

Signed-off-by: Eric Ernsteernst <eric@amperecomputing.com>

(forward port of https://github.com/kata-containers/runtime/pull/2817)
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
2021-03-17 17:50:14 +01:00
Eric Ernsteernst
a3e35e7e92 cgroups: remove unused SystemdCgroup variable and accessor/mutators
Since we are now detecting, no longer to keep this state.

Signed-off-by: Eric Ernsteernst <eric@amperecomputing.com>

(forward port of https://github.com/kata-containers/runtime/pull/2817)
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
2021-03-17 17:50:07 +01:00
Carlos Venegas
fdf69ab84c kernel: ACPI: Always build evged for stable kernel
Path required to enable CPU/memory hotplug in cloud-hypervisor.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-17 11:42:08 +08:00
Tim Zhang
56b94e200c agent: set edition = "2018" in .rustfmt.toml to fix rustfmt about async fn
Got:
find . -type f -name "*.rs"  | egrep -v "target/|grpc-rs/|protocols/" | xargs rustfmt --check

error[E0670]: `async fn` is not permitted in the 2015 edition

This commit fixes this issue.

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-03-17 11:31:32 +08:00
Peng Tao
0533bee222 packaging: add missing SPDX headers
Otherwise static checker complains and fails.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-17 11:31:32 +08:00
Peng Tao
2114576be5 docs: fix fluentbit logfmt parser link
The old link is invalid and causes CI failure.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-17 11:31:32 +08:00
Bo Chen
bcd8fd538d versions: Update cloud-hypervisor to release v0.12.0
Highlights for cloud-hypervisor version v0.12.0 include: removal of
`vhost-user-net` and `vhost-user-block` self spawning, migration of
`vhost-user-fs` backend, ARM64 enhancements with full support of
`--watchdog` for rebooting, and enhanced `info` HTTP API to include the
details of devices used by the VM including VFIO devices.

Fixes: #1315

Signed-off-by: Bo Chen <chen.bo@intel.com>
2021-03-17 11:31:32 +08:00
Peng Tao
6fe3f331c9 ci: default branch has renamed to main
2.0-dev branch does not exist any more and it causes travis to fail.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-17 11:31:32 +08:00
Maksym Pavlenko
3f3a2533a3 agent: Agent invokes OCI hooks with wrong PID
Agent sends -1 PID when invoking OCI hooks.

OCI state struct is initialized before obtaining PID, so this PR moves
`oci_state` call down, right after we get the id.

Fixes: #1458

Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-03-17 11:31:32 +08:00
Fabiano Fidêncio
fc72d392b7 release: Rename runtime-release-notes to release-notes
There's no runtime repo anymore, let's avoid making a reference to it,
which may end up confusing people reading the Release-Process file.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-17 11:31:32 +08:00
Fabiano Fidêncio
ef4ebfba48 release: We're not compatible with Docker.
We don't support Docker as part of 2.x repository.

Fixes: #1449

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-17 11:31:32 +08:00
Manabu Sugimoto
336b80626c rustjail: fix blkio conversion
BFQ weight controller is using the same BFQ weight scheme (i.e 1->1000).
Therefore, there is no need to do the conversion.

More details here: https://github.com/opencontainers/runc/pull/2786

Fixes: #1440

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2021-03-17 11:31:32 +08:00
Fabiano Fidêncio
dd3c5fc617 kata-deploy: Ensure CRI-O uses the VM runtime type
For Kata Containers 2.x, CRI-O should always be using the
`containerd-shim-kata-v2` binary, and always be configured to use the
"vm" runtime type, developed specifically for the shimv2, instead of the
default "oci" runtime type.

I've taken the liberty to try to simplify the CRI-O script and make it
less error prone.  In the future, we can start dropping a configuration
file to /etc/crio/crio.conf.d and just removing it as part of the
cleanup, but that's for the future.

Fixes: #1357

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-17 11:31:32 +08:00
Fabiano Fidêncio
93bd2e4716 kata-deploy: Move the containerd workarounds to their own functions
Factoring those pieces of code to their own functions allows us to
easily re-use them when creating & cleaning up the CRI-O configuration
files, as CRI-O is also affected by the issues that are still opened.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-17 11:31:32 +08:00
Fabiano Fidêncio
7eb882a797 kata-deploy: Update README to reflect the current distributed artifacts
Our list was based on what we used to ship for Kata Containers 1.x, not
even taking into account the shimv2 binary.

Let's update it in order to reflect better what we currently distribute.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-17 11:31:32 +08:00
Julio Montes
a60cf37879 snap: fix kernel setup
kernel setup fails when `yes "n"` is used and `make oldconfig` doesn't
read anything from STDIN, `yes "n"` was added in the past as a
workaround to fix incomplete kernel configs.
Enable `build-kernel.sh` debug.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-03-17 11:31:32 +08:00
Maksym Pavlenko
ca6438728d Agent: OCI hooks return malformed json
This PR fixes wrong serialization of OCI state object.
OCI hooks end up with a JSON string with double quotes in `state` field.

This happens because of confusion `Debug` and `Display` traits. Debug trait
returns a string representation with double quotes.

Ideally we should not use Debug as a part of serialization process, so a bit
more safer fix would be to move container states to `oci` crate and simply
disallow wrong values in that field.

`ContainerState` in go spec: https://github.com/opencontainers/runtime-spec/blob/master/specs-go/state.go#L4

Fixes: #1404

Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
[ backport to stable-2.0 ]
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-17 11:31:32 +08:00
Eric Ernst
32feb10331 runtime: cpuset: when creating container, don't pass cpuset details
Today we only clear out the cpuset details when doing an update call on
existing container/pods. This works in the case of Kubernetes, but not
in the case where we are explicitly setting the cpuset details at boot
time. For example, if you are running a single container via docker ala:

docker run --cpuset-cpus 0-3 -it alpine sh

What would happen is the cpuset info would be passed in with the
container spec for create container request to the agent. At that point
in time, there'd only be the defualt number of CPUs available in the
guest (1), so you'd be left with cpusets set to 0. Next, we'd hotplug
the vCPUs, providing 0-4 CPUs in the guest, but the cpuset would never
be updated, leaving the application tied to CPU 0.

Ouch.

Until the day we support cpusets in the guest, let's make sure that we
start off clearing the cpuset fields.

Fixes: #1405

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-03-17 11:31:32 +08:00