Commit Graph

5570 Commits

Author SHA1 Message Date
Fabiano Fidêncio
e5df408f64
Merge pull request #1653 from fidencio/wip/stable-2.0-fix-kata-deploy-test-script
stable-2.0 | kata-deploy: Fix test-kata.sh and do some small cleanups / improvements in the kata-deploy script
2021-04-08 17:43:54 +02:00
Chelsea Mafrica
985b9fa479
Merge pull request #1652 from fidencio/wip/stable-2.0-only-keep-one-VERSION-file
stable-2.0 | build: Only keep one VERSION file
2021-04-08 08:19:46 -07:00
Fabiano Fidêncio
6d5e47bab1
Merge pull request #1649 from fidencio/wip/stable-2.0-cherry-pick-github-kata-deploy-stuff
stable-2.0 | cherry-pick fixes for test-kata-deploy GitHub actions
2021-04-08 16:58:16 +02:00
Fabiano Fidêncio
514af3624b
Merge pull request #1659 from fidencio/wip/stable-2.0-update-docs
stable-2.0 | A bunch of backports fixing documentation
2021-04-08 14:07:23 +02:00
Fupan Li
a6e3fb6514
Merge pull request #1658 from Tim-Zhang/fix-ttrpc-version-for-agent-ctl
stable-2.0 | agent-ctl: update ttrpc version
2021-04-08 20:06:47 +08:00
Fabiano Fidêncio
55bdd1fcf4 kata-deploy: Adapt regex for testing kata-deploy
On commit a649d33 we've started adding the specific version of the image
to be used, in order to ensure people using our content from a tarball
would be relying on the correct image.

However, later on, @bergwolf figured out it had some undesired side
effects, such as
https://github.com/kata-containers/kata-containers/runs/2235812941?check_suite_focus=true

What happens there is that the regular expression used to point the
image to a testing one doesn't take into consideration the $VERSION, and
that breaks the deployment.

Fixes: #1632

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
(cherry picked from commit d43098ec21)
2021-04-08 13:39:57 +02:00
bin
6586f3b725 docs: update configuration for passing annotations in conatinerd
Using "io.containerd.kata.v2" instead of deprecated "io.containerd.runc.v1".

Fixes: #1629

Signed-off-by: bin <bin@hyper.sh>
(cherry picked from commit b0e4618e84)
2021-04-08 12:34:52 +02:00
Fabiano Fidêncio
f5adc4c114 docs: Remove ubuntu installation guide
The installation guide points to 1.x packages from OBS.  For 2.x we
decided to stop building packages on OBS in favour of advertising
kata-deploy.

Apart from this, Ubuntu itself doesn't provide packages for
kata-containers.

Fixes: #1588

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
(cherry picked from commit ede1ab8670)
2021-04-08 12:34:17 +02:00
Fabiano Fidêncio
a67bdc369a docs: Update snap install guide
As this repo is specific to the kata-containers 2.x, let's stop
mentioning / referring to the 1.x here, including how to setup and use
the snap package for 1.x.

Fixes: #1601

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
(cherry picked from commit 4a38ff41f0)
2021-04-08 12:33:58 +02:00
bin
67be55834d docs: update how-to-use-k8s-with-cri-containerd-and-kata.md
Update how-to-use-k8s-with-cri-containerd-and-kata.md to fit the latest
Kubernetes way.
And also changed CNI plugin from flannel to bridge, that will be easy to run.

Fixes: #1325

Signed-off-by: bin <bin@hyper.sh>
(cherry picked from commit 2c47277ca1)
2021-04-08 12:27:01 +02:00
Fabiano Fidêncio
abfff68de6 docs: Update CentOS install docs
There are two changes here.  There first one being relying on the
`centos-release-advanced-virtualization` package instead providing the
content of the repo ourselves; and the second one being installing
`kata-containers` (2.x) instead of the `kata-runtime` one (1.x).

Fixes: #1583

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
(cherry picked from commit 1ce29fc959)
2021-04-08 12:26:54 +02:00
Fabiano Fidêncio
0466ee04da docs: Update Fedora install docs
The package to be installed on Fedora is `kata-containers` instead of
`kata-runtime`.  The difference being `kata-runtime` is the 1.x package,
while `kata-containers` is the 2.x one.

Fixes: #1582

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
(cherry picked from commit 3f90561bf1)
2021-04-08 12:26:47 +02:00
Fabiano Fidêncio
6b223194a9 docs: Remove SLE installation guide
The content of the SLE installation guide is related to the 1.x
packages, as SUSE doesn't provide katacontainers 2.x packages.

Fixes: #1586

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
(cherry picked from commit 2888ceb024)
2021-04-08 12:26:41 +02:00
Tim Zhang
fb01d51573 agent-ctl: update ttrpc version
Fixes: #1657

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-04-08 18:24:20 +08:00
Fabiano Fidêncio
144be14547 release: Get rid of "master"
We don't use the "master" branch for anything in
`kata-containers/kata-containers`.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
(cherry picked from commit ca4dccf980)
2021-04-07 21:20:43 +02:00
Fabiano Fidêncio
017c7cf249 release: Use sudo to install hub
This doesn't make much difference for the automated process we have in
place, but makes a whole lot of difference for those trying to have the
binaries deployed locally.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
(cherry picked from commit c2197cbf2b)
2021-04-07 21:20:37 +02:00
Fabiano Fidêncio
52c6b0737c build: Only keep one VERSION file
Instead of having different VERSION files spread accross the project,
let's always use the one in the topsrcdir and remove all the others,
keeping only a synlink to the topsrcdir one.

Fixes: #1579

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
(cherry picked from commit 572aff53e8)
2021-04-07 19:12:10 +02:00
Fabiano Fidêncio
e7bdeb49b9 github: Fix slash-command-action usage
`/test-kata-deploy` command does **not** work, and the output returned
is:
```
Error: Comment didn't contain a valid slash command
```

So, why does this happen?

This is the regex used: `^\/([\w]+)\b *(.*)?$`, being the important part
of the command "\/([\w]+)\b", with the rest being arguments to it.
Okay, `\w` is the key here, as `\w` means: a-z, A-Z, 0-9, including the
_.

Our command is `/test-kata-deploy`, and `-` is not present as part of
`\w`.  Knowing this we need to update the command to something like:
`/test_kata_deploy`

Fixes: #1645

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
(cherry picked from commit 7873b7a1f9)
2021-04-07 18:22:45 +02:00
Fabiano Fidêncio
c0ca9f9a90 github: Revert "github: Remove kata-deploy-test action"
This partially reverts commit 1cce930071.

As mentioned in #1635, the malformed yaml wouldn't allow us to actually
test changes that were supposed to be test by this action.

So, this is now reverted and adapted accordingly.

Main differences from what we had before:
* As it tests kata-deploy itself, not the statically built binaries,
  let's just use the binaries from 2.0.0 release;
* Adapt download and deploy location to the
  `kata-containers/kata-containers` repo, as the original action was
  based on 1.x repos;

Fixes: #1640

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
(cherry picked from commit eda8da1ec5)
2021-04-07 18:22:26 +02:00
Fabiano Fidêncio
81f389903a github: Remove kata-deploy-test action
Currently the action is not running because it's broken, and it was
broken by 50fea9f.

Sadly, I cannot just test a fix on a PR as every single time we end up
triggering what's currently on main, rather than triggering the content
of the PR itself.

With this in mind, let me just remove the file and re-add it as part of
a new PR and, hopefully, have it tested in this way.

Sorry for the breakage, by the way.

Fixes: #1634

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
(cherry picked from commit 1cce930071)
2021-04-07 18:21:00 +02:00
Chelsea Mafrica
179a98d678
Merge pull request #1605 from likebreath/0331/clh_v0.14.1_stable_2.0
[backport] versions: Update cloud-hypervisor to release v0.14.1
2021-04-05 10:38:01 -07:00
Bo Chen
e3efcfd40f runtime: Fix the format of the client code of cloud-hypervisor APIs
Regenerate the client code with the added `go-fmt` step. No functional
changes.

Fixes: #1606

Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit 0c38d9ecc4)
2021-04-01 15:15:54 -07:00
Bo Chen
5a92333f4b runtime: Format auto-generated client code for cloud-hypervisor API
This patch extends the current process of generating client code for
cloud-hypervisor API with an additional step, `go-fmt`, which will remove
the generated `client/go.mod` file and format all auto-generated code.

Fixes: #1606

Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit 52cacf8838)
2021-04-01 15:15:54 -07:00
Bo Chen
ec0424e153 versions: Update cloud-hypervisor to release v0.14.1
Highlights for cloud-hypervisor version 0.14.0 include: 1) Structured
event monitoring; 2) MSHV improvements; 3) Improved aarch64 platform; 4)
Updated hotplug documentation; 6) PTY control for serial and
virtio-console; 7) Block device rate limiting; 8) Plan to deprecate the
support of "LinuxBoot" protocol and support PVH protocol only.

Highlights for cloud-hypervisor version 0.13.0 include: 1) Wider VFIO
device support; 2) Improve huge page support; 3) MACvTAP support; 4) VHD
disk image support; 5) Improved Virtio device threading; 6) Clean
shutdown support via synthetic power button.

Details can be found:
https://github.com/cloud-hypervisor/cloud-hypervisor/releases

Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by `openapi-generator` [1-2]. As the API changes do not
impact usages in Kata, no additional changes in kata's runtime are
needed to work with the latest version of cloud-hypervisor.

[1] https://github.com/OpenAPITools/openapi-generator
[2] https://github.com/kata-containers/kata-containers/blob/main/src/runtime/virtcontainers/pkg/cloud-hypervisor/README.md

Fixes: #1591

Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit 84b62dc3b1)
2021-04-01 15:15:06 -07:00
Bin Liu
b26e94ffba
Merge pull request #1595 from bergwolf/backports-stable-2.0
kata-deploy: QEMU, for 2.x, already includes virtiofs
2021-03-31 17:00:21 +08:00
Fabiano Fidêncio
f6f4023508 kata-deploy: Get rid of references to the docker script
The docker script has been removed as part of
62cbaf4de4, but references to it were left
behind in the artifact-list.sh, release/kata-deploy-binaries.sh, and
kata-deploy/Dockerfile.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-31 14:34:42 +08:00
Fabiano Fidêncio
814e7d7285 packaging: Remove NEMU mentions
There's no more NEMU, for some time already.  Considering this, let's
just remove any mention to it as part of our project.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-31 14:34:19 +08:00
Fabiano Fidêncio
92d1197f10 kata-deploy: QEMU, for 2.x, already includes virtiofs
There's no reason to ship qemu & qemu-virtiofs when the former already
includes vitiofs support (and that's the default for 2.x deployments).

In case we will enable experimental qemu DAX support, we should add a
new target, a "qemu-experimental" target, as Carlos has been working on.

Fixes: #1424

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-31 12:13:19 +08:00
Eric Ernst
a2484d0088
Merge pull request #1574 from bergwolf/2.0.2-branch-bump
# Kata Containers 2.0.2
2021-03-30 06:05:23 -07:00
Peng Tao
9e2cbe8ea1 release: Kata Containers 2.0.2
- container: on cleanup, rm container directory for mounts path
- backport: bindmount fixes
- [backport] runtime: increase dial timeout
- kata-deploy: Use the correct image for 2.0.2 release
- [backport] shimv2: return the hypervisor's pid as the container pid
- runtime: clh-config: add runtime hooks to the clh toml
- runtime: cpuset: when creating container, don't pass cpuset details
- [backport] runtime: Fix missing 'name' field on containerd-shim-v2 logs
- [backport] Fixup systemd cgroup handling
- Backports stable 2.0
- [backport] Fix Snap CI

ac9f838 container: on cleanup, rm container directory for mounts path
2c1b957 runtime: remove unused functions
dfe5ef3 tools: fix missing SPDX license header
50aa89f runtime: fix virtiofsd RO volume sharing
57aa746 runtime: mount shared mountpoint readonly
ce2798b runtime: readonly mounts should be readonly bindmount on the host
b7208b3 runtime: increase dial timeout
a649d33 kata-deploy: Use the correct image
d87076e runtime: return hypervisor Pid in TaskExit event
2dd859b shimv2: return the hypervisor's pid as the container pid
c7db337 versions: Use release-1.18 (commit ee9128444bec10)
72af86f runtime: Fix missing 'name' field on containerd-shim-v2 logs
95b2cad clh: Use vanilla kernel.
506f4f2 cgroups: Add systemd detection when creating cgroup manager
a3e35e7 cgroups: remove unused SystemdCgroup variable and accessor/mutators
fdf69ab kernel: ACPI: Always build evged for stable kernel
56b94e2 agent: set edition = "2018" in .rustfmt.toml to fix rustfmt about async fn
0533bee packaging: add missing SPDX headers
2114576 docs: fix fluentbit logfmt parser link
bcd8fd5 versions: Update cloud-hypervisor to release v0.12.0
6fe3f33 ci: default branch has renamed to main
3f3a253 agent: Agent invokes OCI hooks with wrong PID
fc72d39 release: Rename runtime-release-notes to release-notes
ef4ebfb release: We're not compatible with Docker.
336b806 rustjail: fix blkio conversion
dd3c5fc kata-deploy: Ensure CRI-O uses the VM runtime type
93bd2e4 kata-deploy: Move the containerd workarounds to their own functions
7eb882a kata-deploy: Update README to reflect the current distributed artifacts
a60cf37 snap: fix kernel setup
ca64387 Agent: OCI hooks return malformed json
32feb10 runtime: cpuset: when creating container, don't pass cpuset details
3c618a6 agent: Remove bogus check from list_interfaces() unit test
7c888b3 osbuilder: Fix USE_DOCKER on s390x
cf81d40 ci: snap: run snap CI on every pull request
79ed33a snap: fix kernel setup
f1cea9a ci: snap: build targets that not need sudo first
4f802cc ci: snap: define proxy variables
dda4279 kernel: Don't fail if "experimental" dir doesn't exist
5888971 packaging: Fix vmlinux kernel install on s390x
ca28ca4 qemu: Add security fixes for CVE-2020-35517
50ad323 agent: Build for glibc on s390x
f8314be docs: add katacontainers end-to-end arch image
99d9a24 kata-monitor: set buildmode to exe to avoid build failing
0091b89 osbuilder: Enforcing LIBC=gnu to rootfs build for ppc64le
9da2707 kernel: Updates to kernel config for ppc64le
2a0ff0b osbuilder: Add "Agent init" on terms glossary
fa581d3 agent: add secure_join to prevent softlink escape
a3967e9 osbuilder: Fix urls to repositories
272d39b osbuilder: Use Fedora and CentOS registries
7a86c2e shimv2: log a warning and continue on post-start hook failure
5096bd6 shimv2: log a warning and continue on post-stop hook failure
3fe59a9 snap: Don't release Kata Alpha/RC in snap store
61fa4a3 rustjail: fix the issue of missing destroy contaienr cgroups
856af1a runtime: cpuset: when creating container, don't pass cpuset details
74b5874 runtime: clh-config: add runtime hooks to the clh toml

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-30 07:55:15 +00:00
Peng Tao
fc676f76de
Merge pull request #1567 from egernst/cleanup-pr-stable-2.0
container: on cleanup, rm container directory for mounts path
2021-03-30 11:12:25 +08:00
Eric Ernst
ac9f838e33 container: on cleanup, rm container directory for mounts path
A wrong path was being used for container directory when
virtiofs is utilized. This resulted in a warning message in
logs when a container is killed, or completes:

level=warning msg="Could not remove container share dir"

Without proper removal, they'd later be cleaned up when the shared
path is removed as part of stopping the sandbox.

Fixes: #1559

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-03-29 14:16:22 -07:00
Fabiano Fidêncio
9ea851ee53
Merge pull request #1485 from egernst/backport-bindmount-fixes
backport: bindmount fixes
2021-03-26 18:45:54 +01:00
Peng Tao
2c1b957642 runtime: remove unused functions
archConvertStatFs is not called by anyone and causes static checker
failures.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-26 15:30:09 +08:00
Peng Tao
dfe5ef36b4 tools: fix missing SPDX license header
As reported by the static checker.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-26 15:08:40 +08:00
Bin Liu
8a374af6b7
Merge pull request #1548 from snir911/timeout_stable2.0
[backport] runtime: increase dial timeout
2021-03-26 14:43:03 +08:00
Peng Tao
50aa89fa05 runtime: fix virtiofsd RO volume sharing
Right now we rely heavily on mount propagation to share host
files/directories to the guest. However, because virtiofsd
pivots and moves itself to a separate mount namespace, the remount
mount is not present in virtiofsd's mount. And it causes guest to be
able to write to the host RO volume.

To fix it, create a private RO mount and then move it to the host mounts
dir so that it will be present readonly in the host-guest shared dir.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-26 12:53:04 +08:00
Peng Tao
57aa746d0d runtime: mount shared mountpoint readonly
bindmount remount events are not propagated through mount subtrees,
so we have to remount the shared dir mountpoint directly.

E.g.,
```
mkdir -p source dest foo source/foo

mount -o bind --make-shared source dest

mount -o bind foo source/foo
echo bind mount rw
mount | grep foo
echo remount ro
mount -o remount,bind,ro source/foo
mount | grep foo
```
would result in:
```
bind mount rw
/dev/xvda1 on /home/ubuntu/source/foo type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda1 on /home/ubuntu/dest/foo type ext4 (rw,relatime,discard,data=ordered)
remount ro
/dev/xvda1 on /home/ubuntu/source/foo type ext4 (ro,relatime,discard,data=ordered)
/dev/xvda1 on /home/ubuntu/dest/foo type ext4 (rw,relatime,discard,data=ordered)
```

The reason is that bind mount creats new mount structs and attaches them to different mount subtrees.
However, MS_REMOUNT only looks for existing mount structs to modify and does not try to propagate the
change to mount structs in other subtrees.

Fixes: #1061
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-26 12:00:28 +08:00
Peng Tao
ce2798b688 runtime: readonly mounts should be readonly bindmount on the host
So that we get protected at the VM boundary not just the guest kernel.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-26 12:00:27 +08:00
Snir Sheriber
b7208b3c6c runtime: increase dial timeout
On some setups, starting multiple kata pods (qemu) simultaneously on the same node
might cause kata VMs booting time to increase and the pods to fail with:
Failed to check if grpc server is working: rpc error: code = DeadlineExceeded desc = timed
out connecting to vsock 1358662990:1024: unknown

Increasing default dialing timeout to 30s should cover most cases.

Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
Fixes: #1543
(backport https://github.com/kata-containers/kata-containers/pull/1544)
2021-03-25 10:01:36 +02:00
Fabiano Fidêncio
7e4dc08b0e
Merge pull request #1541 from fidencio/wip/kata-deploy-use-the-correct-image
kata-deploy: Use the correct image for 2.0.2 release
2021-03-25 08:31:44 +01:00
Fabiano Fidêncio
a649d33a45 kata-deploy: Use the correct image
Let's tag the yet non existent 2.0.2 image, so whoever deploys kata
using kata-deploy from the release tarball is pointed to the correct
image.

Fixes: #1493

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-24 00:22:32 +01:00
Peng Tao
c628ecf298
Merge pull request #1528 from fgiudici/stable-2.0_hypervisor_pid
[backport] shimv2: return the hypervisor's pid as the container pid
2021-03-23 16:31:18 +08:00
bin
d87076eea5 runtime: return hypervisor Pid in TaskExit event
Other RPC calls return Pid of hypervisor, the TaskExit should
return the same Pid.

Fixes: #1497

Signed-off-by: bin <bin@hyper.sh>

(backport https://github.com/kata-containers/kata-containers/pull/1498)
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
[ fix missing GetHypervisorPid method in MockSandbox ]
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-23 15:34:45 +08:00
fupan.lfp
2dd859bfce shimv2: return the hypervisor's pid as the container pid
Since the kata's hypervisor process is in the network namespace,
which is close to container's process, and some host metrics
such as cadvisor can use this pid to access the network namespace
to get some network metrics. Thus this commit replace the shim's
pid with the hypervisor's pid.

Fixes: #1451

Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>

(backport https://github.com/kata-containers/kata-containers/pull/1452)
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
2021-03-23 15:34:45 +08:00
Peng Tao
4c9af982e6
Merge pull request #1426 from egernst/stable-2.0-clh-hooks
runtime: clh-config: add runtime hooks to the clh toml
2021-03-23 15:18:45 +08:00
Peng Tao
06f964843a
Merge pull request #1427 from egernst/stable-2.0-cpusets
runtime: cpuset: when creating container, don't pass cpuset details
2021-03-23 15:17:36 +08:00
Peng Tao
c27c3c40dd
Merge pull request #1529 from wainersm/for_stable-2.0
[backport] runtime: Fix missing 'name' field on containerd-shim-v2 logs
2021-03-23 15:14:15 +08:00
Peng Tao
476467115f
Merge pull request #1522 from fgiudici/stable-2.0
[backport] Fixup systemd cgroup handling
2021-03-23 15:13:53 +08:00
Fabiano Fidêncio
73645d1742
Merge pull request #1466 from bergwolf/backports-stable-2.0
Backports stable 2.0
2021-03-22 19:30:24 +01:00