Commit Graph

5564 Commits

Author SHA1 Message Date
Tim Zhang
56b94e200c agent: set edition = "2018" in .rustfmt.toml to fix rustfmt about async fn
Got:
find . -type f -name "*.rs"  | egrep -v "target/|grpc-rs/|protocols/" | xargs rustfmt --check

error[E0670]: `async fn` is not permitted in the 2015 edition

This commit fixes this issue.

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-03-17 11:31:32 +08:00
Peng Tao
0533bee222 packaging: add missing SPDX headers
Otherwise static checker complains and fails.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-17 11:31:32 +08:00
Peng Tao
2114576be5 docs: fix fluentbit logfmt parser link
The old link is invalid and causes CI failure.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-17 11:31:32 +08:00
Bo Chen
bcd8fd538d versions: Update cloud-hypervisor to release v0.12.0
Highlights for cloud-hypervisor version v0.12.0 include: removal of
`vhost-user-net` and `vhost-user-block` self spawning, migration of
`vhost-user-fs` backend, ARM64 enhancements with full support of
`--watchdog` for rebooting, and enhanced `info` HTTP API to include the
details of devices used by the VM including VFIO devices.

Fixes: #1315

Signed-off-by: Bo Chen <chen.bo@intel.com>
2021-03-17 11:31:32 +08:00
Peng Tao
6fe3f331c9 ci: default branch has renamed to main
2.0-dev branch does not exist any more and it causes travis to fail.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-17 11:31:32 +08:00
Maksym Pavlenko
3f3a2533a3 agent: Agent invokes OCI hooks with wrong PID
Agent sends -1 PID when invoking OCI hooks.

OCI state struct is initialized before obtaining PID, so this PR moves
`oci_state` call down, right after we get the id.

Fixes: #1458

Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-03-17 11:31:32 +08:00
Fabiano Fidêncio
fc72d392b7 release: Rename runtime-release-notes to release-notes
There's no runtime repo anymore, let's avoid making a reference to it,
which may end up confusing people reading the Release-Process file.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-17 11:31:32 +08:00
Fabiano Fidêncio
ef4ebfba48 release: We're not compatible with Docker.
We don't support Docker as part of 2.x repository.

Fixes: #1449

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-17 11:31:32 +08:00
Manabu Sugimoto
336b80626c rustjail: fix blkio conversion
BFQ weight controller is using the same BFQ weight scheme (i.e 1->1000).
Therefore, there is no need to do the conversion.

More details here: https://github.com/opencontainers/runc/pull/2786

Fixes: #1440

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2021-03-17 11:31:32 +08:00
Fabiano Fidêncio
dd3c5fc617 kata-deploy: Ensure CRI-O uses the VM runtime type
For Kata Containers 2.x, CRI-O should always be using the
`containerd-shim-kata-v2` binary, and always be configured to use the
"vm" runtime type, developed specifically for the shimv2, instead of the
default "oci" runtime type.

I've taken the liberty to try to simplify the CRI-O script and make it
less error prone.  In the future, we can start dropping a configuration
file to /etc/crio/crio.conf.d and just removing it as part of the
cleanup, but that's for the future.

Fixes: #1357

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-17 11:31:32 +08:00
Fabiano Fidêncio
93bd2e4716 kata-deploy: Move the containerd workarounds to their own functions
Factoring those pieces of code to their own functions allows us to
easily re-use them when creating & cleaning up the CRI-O configuration
files, as CRI-O is also affected by the issues that are still opened.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-17 11:31:32 +08:00
Fabiano Fidêncio
7eb882a797 kata-deploy: Update README to reflect the current distributed artifacts
Our list was based on what we used to ship for Kata Containers 1.x, not
even taking into account the shimv2 binary.

Let's update it in order to reflect better what we currently distribute.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-17 11:31:32 +08:00
Julio Montes
a60cf37879 snap: fix kernel setup
kernel setup fails when `yes "n"` is used and `make oldconfig` doesn't
read anything from STDIN, `yes "n"` was added in the past as a
workaround to fix incomplete kernel configs.
Enable `build-kernel.sh` debug.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-03-17 11:31:32 +08:00
Maksym Pavlenko
ca6438728d Agent: OCI hooks return malformed json
This PR fixes wrong serialization of OCI state object.
OCI hooks end up with a JSON string with double quotes in `state` field.

This happens because of confusion `Debug` and `Display` traits. Debug trait
returns a string representation with double quotes.

Ideally we should not use Debug as a part of serialization process, so a bit
more safer fix would be to move container states to `oci` crate and simply
disallow wrong values in that field.

`ContainerState` in go spec: https://github.com/opencontainers/runtime-spec/blob/master/specs-go/state.go#L4

Fixes: #1404

Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
[ backport to stable-2.0 ]
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-17 11:31:32 +08:00
Eric Ernst
32feb10331 runtime: cpuset: when creating container, don't pass cpuset details
Today we only clear out the cpuset details when doing an update call on
existing container/pods. This works in the case of Kubernetes, but not
in the case where we are explicitly setting the cpuset details at boot
time. For example, if you are running a single container via docker ala:

docker run --cpuset-cpus 0-3 -it alpine sh

What would happen is the cpuset info would be passed in with the
container spec for create container request to the agent. At that point
in time, there'd only be the defualt number of CPUs available in the
guest (1), so you'd be left with cpusets set to 0. Next, we'd hotplug
the vCPUs, providing 0-4 CPUs in the guest, but the cpuset would never
be updated, leaving the application tied to CPU 0.

Ouch.

Until the day we support cpusets in the guest, let's make sure that we
start off clearing the cpuset fields.

Fixes: #1405

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-03-17 11:31:32 +08:00
David Gibson
3c618a61d6 agent: Remove bogus check from list_interfaces() unit test
The unit test for list_interfaces() checks that the hardware address
returned for each interface has non-zero length.  However, that need not be
the case.  Point-to-point devices, such as ppp, or tun devices in certain
configurations may not have a hardware address, which is represented as
a zero length address here.

This happens on my machine with a tun0 device created by OpenVPN.

fixes #1377

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-17 11:31:32 +08:00
Jakob-Naucke
7c888b34be osbuilder: Fix USE_DOCKER on s390x
- Install the required protobuf-compiler on Ubuntu
- Install correct libc Rust target (glibc on s390x)
- Do not skip Rust installation on s390x

Fixes: #1266

Signed-off-by: Jakob-Naucke <jakob.naucke@ibm.com>
2021-03-17 11:31:32 +08:00
GabyCT
234d53b6df
Merge pull request #1408 from devimc/2021-02-12/FixSnapCI
[backport] Fix Snap CI
2021-03-05 09:50:43 -06:00
Julio Montes
cf81d400d8 ci: snap: run snap CI on every pull request
Make sure a pull request doesn't break the snap packages,
run snap CI on every pull request.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-03-04 15:58:49 -06:00
Julio Montes
79ed33adb5 snap: fix kernel setup
kernel setup fails when `yes "n"` is used and `make oldconfig` doesn't
read anything from STDIN, `yes "n"` was added in the past as a
workaround to fix incomplete kernel configs.
Enable `build-kernel.sh` debug.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-03-04 15:58:46 -06:00
Julio Montes
f1cea9a022 ci: snap: build targets that not need sudo first
`sudo` is required to build the image, once the image has been built
the permission of some directories may change, let's build first the
targerts that not need `sudo`

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-03-04 15:58:42 -06:00
Julio Montes
4f802cc993 ci: snap: define proxy variables
define proxy variables before using them to fix `unbound variable`
error

fixes #1386

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-03-04 15:58:39 -06:00
Fabiano Fidêncio
dda4279a2b kernel: Don't fail if "experimental" dir doesn't exist
This directory has been automatically removed as there's no files inside
it, as part of d3c9862059

Let's improve the logic in the scripts to avoid failing in case the
"experimental" dir is not present.

Fixes: #1328

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-26 00:58:59 +08:00
Jakob-Naucke
5888971e18 packaging: Fix vmlinux kernel install on s390x
Installing the built uncompressed vmlinux kernel will not work on s390x,
QEMU will complain:
Linux kernel boot failure: An attempt to boot a vmlinux ELF image
failed.
This image does not contain all parts necessary for starting up. Use
bzImage or arch/s390/boot/compressed/vmlinux instead.
Hence, use that kernel image on s390x.

Fixes: #1264

Signed-off-by: Jakob-Naucke <jakob.naucke@ibm.com>
2021-02-26 00:58:40 +08:00
Fabiano Fidêncio
ca28ca422c qemu: Add security fixes for CVE-2020-35517
This series is based on
https://lists.gnu.org/archive/html/qemu-devel/2021-02/msg01787.html, and
was kindly brought up by David Gilbert.

Fixes: #1361

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-26 00:58:14 +08:00
Jakob Naucke
50ad323a21 agent: Build for glibc on s390x
Since there is no Rust target for musl on s390x, builds on s390x should
use glibc. This commit makes glibc the default on s390x as per the agent
Makefile.

Fixes: #1262

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-02-26 00:57:03 +08:00
Adel Zaalouk
f8314bedb0 docs: add katacontainers end-to-end arch image
Added a Katacontainers e2e flow figure, the figure can be used in many places or simply just as a reference.
It contains pieces from Kata 1.x and Kata 2.0 (e.g., kata-monitor).

The figures are produced via excalidraw.com, also in the commit is the
source .excalidraw which can be used to modify the figure if needed.

Fixes #1185

Signed-off-by: Adel Zaalouk <azaalouk@redhat.com>
2021-02-26 00:56:53 +08:00
bin
99d9a24a51 kata-monitor: set buildmode to exe to avoid build failing
CGO_ENABLED=0 and -buildmode=pie are not compatible and may lead build failing in some OS.
Specify buildmode=exe to overwrite the value set in BUILDFLAGS

Fixes: #1343

Signed-off-by: bin <bin@hyper.sh>
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-02-26 00:54:59 +08:00
Jing Wang
0091b89184 osbuilder: Enforcing LIBC=gnu to rootfs build for ppc64le
To enforce LIBC=gnu to rootfs.sh for ppc64le, instead of error and exit if not set.

Fixes: #1339
Signed-off-by: Jing Wang <jing.wang4@ibm.com>
2021-02-26 00:53:32 +08:00
Jing Wang
9da2707202 kernel: Updates to kernel config for ppc64le
Need to enable virtiofs and ipv6 to kernel config for ppc64le.

Fixes: #1333
Signed-off-by: Jing Wang <jing.wang4@ibm.com>
2021-02-26 00:53:22 +08:00
Wainer dos Santos Moschetta
2a0ff0bec3 osbuilder: Add "Agent init" on terms glossary
Include on the terms glossary some words about the agent init as
later on README its explain how the agent can be switched between
systemd and init.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-02-26 00:33:40 +08:00
Qingyuan Hou
fa581d334f agent: add secure_join to prevent softlink escape
This patch fixed the security issue if the container images has
unsafe symlink to the container rootfs and hackers can be exploit
this symlink to hack the guest system. e.g. make directory or files
on guest.

CVE-2015-3629

Fixes: #1219

Signed-off-by: Qingyuan Hou <qingyuan.hou@linux.alibaba.com>
2021-02-26 00:33:18 +08:00
Wainer dos Santos Moschetta
a3967e9a59 osbuilder: Fix urls to repositories
Changed the user-visible urls to point to the right Kata Containers
files/repositories.

Fixes #234

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-02-26 00:33:09 +08:00
Wainer dos Santos Moschetta
272d39bc87 osbuilder: Use Fedora and CentOS registries
To avoid hitting quota limit on docker.io, this changes the Fedora
and CentOS dockerfiles to pull the image from the projects registries.

Fixes #1324

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-02-26 00:33:05 +08:00
Snir Sheriber
7a86c2eedd shimv2: log a warning and continue on post-start hook failure
According to runtime-spec:
The poststart hooks MUST be invoked by the runtime. If any poststart
hook fails, the runtime MUST log a warning, but the remaining hooks
and lifecycle continue as if the hook had succeeded

Fixes: #1252

Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2021-02-26 00:32:49 +08:00
Snir Sheriber
5096bd6a11 shimv2: log a warning and continue on post-stop hook failure
According to runtime-spec:
The poststop hooks MUST be invoked by the runtime. If any
poststop hook fails, the runtime MUST log a warning, but
the remaining hooks and lifecycle continue as if the hook
had succeeded.

Fixes: #1252

Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2021-02-26 00:32:38 +08:00
Julio Montes
3fe59a99ff snap: Don't release Kata Alpha/RC in snap store
Stable and candidate snapcraft's channels are used to release Kata
Containers 1.x and 2.x respectively. Alpha and RC releases shouldn't
be pushed to the snap store because there are not enough channels
for all the releases of Kata Containers 1.x and 2.x.

fixes #1303

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-02-26 00:32:17 +08:00
fupan.lfp
61fa4a3c75 rustjail: fix the issue of missing destroy contaienr cgroups
In the container's destroy method, it should destroy
the container's cgroups.

Fixes: #1291

Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
2021-02-26 00:31:50 +08:00
Eric Ernst
856af1a886 runtime: cpuset: when creating container, don't pass cpuset details
Today we only clear out the cpuset details when doing an update call on
existing container/pods. This works in the case of Kubernetes, but not
in the case where we are explicitly setting the cpuset details at boot
time. For example, if you are running a single container via docker ala:

docker run --cpuset-cpus 0-3 -it alpine sh

What would happen is the cpuset info would be passed in with the
container spec for create container request to the agent. At that point
in time, there'd only be the defualt number of CPUs available in the
guest (1), so you'd be left with cpusets set to 0. Next, we'd hotplug
the vCPUs, providing 0-4 CPUs in the guest, but the cpuset would never
be updated, leaving the application tied to CPU 0.

Ouch.

Until the day we support cpusets in the guest, let's make sure that we
start off clearing the cpuset fields.

Fixes: #1405

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-02-17 16:55:56 -08:00
Eric Ernst
74b587431f runtime: clh-config: add runtime hooks to the clh toml
Today hooks are only described in the QEMU toml. This shouldn't be VMM
specific -- let's make sure these are advertised for Cloud Hypervisor as
well.

Fixes: #1401

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-02-17 16:54:31 -08:00
Eric Ernst
3df65f4f3a
Merge pull request #1282 from egernst/fix-2.0-stable-release
Fix 2.0 stable release
2021-01-15 17:30:17 -08:00
Eric Ernst
c5a6354718 actions: w/a deprecated set-env
set-env is no longer allowed. Updated to use the new recommended syntax.

Fixes: #1273

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-01-15 16:29:37 -08:00
Eric Ernst
867d8bc9b4 packaging: should tag/update tests repo when releasing
We should still bump/version the tests repository, just as we do for
1.x.

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-01-15 14:39:39 -08:00
Eric Ernst
cfe9470ff1
Merge pull request #1270 from egernst/2.0.1-branch-bump
# Kata Containers 2.0.1
2021-01-14 13:56:03 -08:00
Fabiano Fidêncio
9820459a0f
Merge pull request #1271 from devimc/2021-01-14/stable-2.0/fixSnap
[backport] snap: tag yq version
2021-01-14 22:41:56 +01:00
Julio Montes
4e141a96ed snap: tag yq version
yq major releases are not backward compatible, install the same
major version used in the CI to avoid conflics building the kata
components.
We should update yq when the CI updates it, not before.

fixes #1232

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-01-14 14:03:39 -06:00
Eric Ernst
c8028da3c6 release: Kata Containers 2.0.1
- volume cleanup, RO blk device support
- Backport to stable-2.0 branch
- [stable 2.0] backport VFIO fixes
- [backport] snap: fix snap release channel
- [backport] snap: add GH actions jobs to release the snap package
- backport fixes to stable-2.0.0
- Backport: Backport doc changes from 2.0 dev

e4cea92a blk-dev: hotplug readonly if applicable
0590fedd volumes: cleanup / minor refactoring
6b666899 vendor: revendor govmm from intel to kata-containers
65ae1271 runtime: clh: update cloud-hypervisor
9bc6fe6c runtime: clh: disable virtiofs DAX when FS cache size is 0
349d496f versions: Update cloud-hypervisor to release v0.11.0
60050264 rootfs: Fix indentation inside a switch
91b43a99 rootfs: apparmor=unconfined is needed for non Red Hat host OSes
2478b8f4 rootfs: Always add SYS_ADMIN, CHROOT, and MKNOD caps to docker cmdline
499aa24d rootfs: Don't fallthrough in the docker_extra_args() switch
1edb7fe7 rustjail: fix the issue of sync read
607a892f rustjail: fix the issue of bind mount /dev
26f176e2 rustjail: allow network sysctls
3306195f agent: Avoid container stats panic caused by cgroup controller non-exist
a7568b52 agent: Clean up commented use declarations
e6d68349 agent: Fix temp prefix on Namespace::test_setup_persistent_ns
1f943bd6 agent: Return error on trying to persist a pid namespace
9a41d09f shimv2: Avoid double removing of container from sandbox
8fdb85e0 jail/validator: avoid unwrap() for safety
49516ef6 rustjail: add more context info for errors
21fad464 oci: fix two incompatible issues with OCI spec
b745e5ff agent: consume ttrpc crate from crates.io
40316f68 qemu: no state to save if QEMU isn't running
35b619ff oci: fix a typo in "addtionalGids"
662e8db5 agent/sandbox:  Don't update cpuset when ncpus = 0
9117dd40 runtime/network: Fix error reporting in listRoutes()
fce14f36 runtime/network: Correct error reporting in listInterfaces()
0fd70f7e rootfs-builder: add support for gentoo
4727a9c3 rootfs: reduce size of debian image
7ab8f62d runtime: Allow to overwrite DESTDIR
7e92833b packaging: Make qemu/apply_patches.sh common
14b18b55 packaging/qemu: Delete the temporary container
1dde0de1 packaging/qemu: Build and package completely in the container
d4c1b768 packaging/qemu: Add QEMU_DESTDIR argument to dockerfiles
3c36ce81 rootfs-builder: add functions to run before and after the container
c9d4e2c4 agent-ctl: Add void "install" target
5fadc5fc trace-forwarder: Add void "install" target
5f887506 snap: fix snap release channel
7526ee93 snap: add GH actions jobs to release the snap package
21ed9dc2 agent: update proto file copyright
5f1520bd agent: generate proto files properly
e30bd673 agent-ctl: update cargo.lock
78df4a0c runtime: remove the unused proto files
7daf9cff agent: move gogo.proto out of the github.com namespance
293be9d0 agent: types.pb.go is not regenerated
84e1a34f agent/protocols: Move agent.proto out of the mock folder of agent
cf56307e agent/protocols: Fix copyright header checking
359f76d2 agent/protocols: Stop generate agent proto files in the shellscript
ca8f1399 agent/protocols: Ignore generated files and remove these files from repo
0bb559a4 agent/protocols: Generate proto files programmatically
4ca4412f docs: fix spell check
3d80c848 docs: Update how-to Readme with hypervisor information.
f0fdc8e1 docs: Update Readme to remove hypervisor information
e53645ec docs: Remove docs for nemu

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2021-01-14 11:05:32 -08:00
Eric Ernst
0aa68ccfef
Merge pull request #1258 from egernst/ro-stable
volume cleanup, RO blk device support
2021-01-14 09:04:54 -08:00
Eric Ernst
e4cea92ad3 blk-dev: hotplug readonly if applicable
If a block based volume is read only, let's make sure we add as a RO
device

Fixes: #1246

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-01-13 14:44:45 -08:00
Eric Ernst
0590fedd98 volumes: cleanup / minor refactoring
Update some headers, very minor refactoring

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-01-13 14:44:45 -08:00