Commit Graph

6164 Commits

Author SHA1 Message Date
Chelsea Mafrica
543f9da3ba runtime: Disable trace for healthcheck
With tracing enabled, grpc health check generates a large number of
spans which creates too much data for tasks running longer than a few
minutes. To solve this, remove span creation from kata agent check() and
sendReq() where the majority of the spans come from. Leave contexts in
functions for subsequent calls that create spans.

Fixes #1395

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2021-04-09 15:47:00 -07:00
Fabiano Fidêncio
4e62d596db
Merge pull request #1675 from fidencio/2.1.0-alpha2-branch-bump
# Kata Containers 2.1.0-alpha2
2021-04-09 20:14:19 +02:00
Fabiano Fidêncio
4f164b5246 release: Kata Containers 2.1.0-alpha2
- release: Do not git add kata-{deploy,cleanup}.yaml for the tests repo
- kata-deploy: add runtimeclass that includes pod overhead
- release: automatically bump the version of the kata-deploy images
- Refine uevent matching conditions
- docs: update dev-guide to include fixes from 1.x
- virtcontainers: replace newStore by store in Sandbox struct
- agent: log the mount point if it is already mounted
- tools/agent-ctl: Update Cargo.lock
- agent: Rework the debug console
- oci: Update seccomp configuration
- kernel: update experimental kernel to 5.10.x
- kata-deploy: Fix `test-kata.sh` and do some small cleanups / improvements in the kata-deploy script
- github: Fix slash-command-action usage
- rustjail: fix the issue of missing default home env
- Make uevent watching mechanism more flexible
- ci/openshift-ci: Prepare to build on CentOS 8
- docs: update configuration for passing annotations in conatinerd
- Revert "github: Remove kata-deploy-test action"
- runtime: increase dial timeout
- qemu experimental: Move to latest tree on virtio-fs-dev (qemu 6.0 + DAX patches).
- github: Remove kata-deploy-test action
- agent: s390x statfs constants
- kernel: upgrade kernel to 5.10.x for arm64.
- Don't do anything in Pipestream::shutdown
- Fix fsgroup
- agent: Remove many "panic message is not string literal" warnings
- osbuilder: Update QAT Dockerfile with new QAT driver version
- osbuilder: update dockerfiles to utilize IMAGE_REGISTRY
- Only keep one VERSION file
- Dechat deruntime
- runtime: Format auto-generated client code for cloud-hypervisor API
- runtime: use concrete KataAgentConfig instead of interface type
- versions: Update cloud-hypervisor to release v0.14.1
- runtime: import runtime/v2/runc/options to decode request from Docker
- virtcontainers/fc: Upgrade Firecracker to v0.23.1
- docs: Remove ubuntu installation guide
- docs: Update snap install guide
- docs: update how-to-use-k8s-with-cri-containerd-and-kata.md
- Update install docs for Fedora and CentOS
- action: fix missing qemu tag
- Remove installation guides for SLE and openSUSE
- kernel: Enable OVERLAY_FS_{METACOPY,XINO_AUTO}
- versions: kernel 5.10.x
- virtcontainers: Fix missing contexts in s390x
- runtime: makefile allow override DAX value

11897248 release: Do not git add kata-{deploy,cleanup}.yaml for the tests repo
2b5f79d6 release: automatically bump the version of the kata-deploy images
8682d6b7 docs: update dev-guide to include fixes from 1.x
f444adb5 kata-cleanup: Explicitly add tag to the container image
12582c2f kata-deploy: add runtimeclass that includes pod overhead
d75fe956 virtcontainers: replace newStore by store in Sandbox struct
342eb765 tools/agent-ctl: Update Cargo.lock
24b0703f agent: fix test for the debug console
79033257 agent: async the debug console
8ea2ce9a agent/device: Remove legacy uevent matching
5d007743 agent/device: Refine uevent matching for pmem devices
9017e110 agent: start to rework the debug console
a59e07c1 agent/define: Refine uevent matching for virtio-scsi devices
484a3647 agent/device: Rework uevent handling for virtio-blk devices
7873b7a1 github: Fix slash-command-action usage
eda8da1e github: Revert "github: Remove kata-deploy-test action"
a938d903 rustjail: fix the issue of missing default home env
b0e4618e docs: update configuration for passing annotations in conatinerd
d43098ec kata-deploy: Adapt regex for testing kata-deploy
107ceca6 kernel: update experimental kernel to 5.10.x
ca4dccf9 release: Get rid of "master"
c2197cbf release: Use sudo to install hub
49eec920 agent: log the tag and mount point if it is already mounted
16f732fc ci/lib: Use git to clone the tests repository
9281e567 ci/openshift-ci: Add build root dockerfile
1cce9300 github: Remove kata-deploy-test action
0828f9ba agent/uevent: Introduce wait_for_uevent() helper
16ed55e4 agent/device: Use consistent matching for past and future uevents
4b16681d agent/uevent: Put matcher object rather than "device address" in watch list
b8b32248 agent/uevent: Consolidate event matching logic
d2caff6c agent: Re-organize uevent processing
55ed2ddd agent: Store uevent watchers in Vec rather than HashMap
91e0ef5c agent/uevent: Report whole Uevents to device watchers
36420054 agent: Store whole Uevent in map, rather than just /dev name
06162025 agent/device: Move GLOBAL_DEVICE_WATCHER into Sandbox
11ae32e3 agent/device: Fix path matching for PCI devices
4f608804 agent/device: Update test_get_device_name()
ee6a590d agent: add test test_pipestream_shutdown
4a2d4370 agent: don't do anything in Pipestream::shutdown
e3e670c5 agent/device: Forward port test for get_device_name() from Kata 1.x
ed08980f agent: Remove many "panic message is not string literal" warnings
f365bdb7 versions: qemu-experimental: 6.0~rc 470dd6
6491b9d7 qemu: Add support to build static qemu for dev tree
13653e7b runtime: increase dial timeout
935460e5 osbuilder: update dockerfiles to utilize IMAGE_REGISTRY
010d57f4 osbuilder: Update QAT Dockerfile with new QAT driver version
adb866ad kata-deploy: Adapt to the correct tag name
60adc7f0 VERSION: Use the correct form
a4c125a8 trace: move gRPC requests from debug to trace
50fff977 trace: move trace span chatter to trace rather than info
28bd8c11 kernel: upgrade kernel to 5.10.x for arm64.
6fe48329 runtime: use concrete KataAgentConfig instead of interface type
64939425 mount: fix the issue of missing set fsGroup
88e58a4f agent: fix the issue of missing pass fsGroup
572aff53 build: Only keep one VERSION file
0c38d9ec runtime: Fix the format of the client code of cloud-hypervisor APIs
52cacf88 runtime: Format auto-generated client code for cloud-hypervisor API
84b62dc3 versions: Update cloud-hypervisor to release v0.14.1
4a38ff41 docs: Update snap install guide
ede1ab86 docs: Remove ubuntu installation guide
6255cc19 virtcontainers/fc: Upgrade Firecracker to v0.23.1
2c47277c docs: update how-to-use-k8s-with-cri-containerd-and-kata.md
317f55f8 docs: Update minimum version for Fedora
1ce29fc9 docs: Update CentOS install docs
3f90561b docs: Update Fedora install docs
8a1c6c3f action: fix missing qemu tag
a9ff9c87 docs: Remove openSUSE installation guide
2888ceb0 docs: Remove SLE installation guide
09d454ac runtime: import runtime/v2/runc/options to decode request from Docker
0b502d15 runtime: makefile allow override DAX value
a65519b9 versions: keep using kernel 5.4.x for ARM
31ced01e virtcontainers: Fix missing contexts in s390x
52a276fb agent: Fix type for PROC_SUPER_MAGIC on s390x
5b7c8b7d agent: Update cgroups-rs to 0.2.5
c035cdb3 versions: kernel 5.10.x
660b0473 oci: Update seccomp configuration
8c1e0d30 kernel: Enable OVERLAY_FS_{METACOPY,XINO_AUTO}

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-09 17:56:29 +02:00
Fabiano Fidêncio
73aa74b4bb
Merge pull request #1673 from fidencio/wip/release-dont-git-add-kata-deploy-cleanup-on-tests-repo
release: Do not git add kata-{deploy,cleanup}.yaml for the tests repo
2021-04-09 16:21:14 +02:00
Fabiano Fidêncio
1189724822 release: Do not git add kata-{deploy,cleanup}.yaml for the tests repo
I was, mistakenly, `git add`ing those files unconditionally.

Fixes: #1672

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-09 14:38:13 +02:00
Fabiano Fidêncio
43a9d4e90a
Merge pull request #1666 from egernst/rc-overhead
kata-deploy: add runtimeclass that includes pod overhead
2021-04-09 12:44:41 +02:00
Peng Tao
efd5d6f1fe
Merge pull request #1667 from fidencio/wip/automatically-bump-kata-deploy-image-version
release: automatically bump the version of the kata-deploy images
2021-04-09 16:06:07 +08:00
David Gibson
0e04d6299b
Merge pull request #1642 from dgibson/ueventplus
Refine uevent matching conditions
2021-04-09 13:10:52 +10:00
Fabiano Fidêncio
2b5f79d685 release: automatically bump the version of the kata-deploy images
Let's teach `update-repository-version.sh` to automatically bump the
version of the kata-deploy images to be used within that release, when
running against the `kata-containers` repo.

Fixes: #1665

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-09 00:31:27 +02:00
GabyCT
368ab486f6
Merge pull request #1662 from egernst/docs-cleanup
docs: update dev-guide to include fixes from 1.x
2021-04-08 15:43:43 -05:00
Eric Ernst
2334b858a0
Merge pull request #1661 from liubin/1660-replace-newStore-by-store
virtcontainers: replace newStore by store in Sandbox struct
2021-04-08 13:17:44 -07:00
Eric Ernst
8682d6b7ea docs: update dev-guide to include fixes from 1.x
This addresses a few gaps with respect to fixes in 1.x docs:
  - Cleanup QEMU information in order to drop references to qemu-lite
  - Make sure we include directions for debug console in case of QEMU

Fixes: #574

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-04-08 12:55:59 -07:00
Fabiano Fidêncio
f444adb51b kata-cleanup: Explicitly add tag to the container image
We have the tags explicitly set on kata-deploy, let's do the same for
kata-cleanup.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-08 21:43:59 +02:00
Eric Ernst
12582c2f6d kata-deploy: add runtimeclass that includes pod overhead
The overhead values may not be perfect, but this is a start, and a good
reference.

Fixes: #580

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-04-08 12:42:15 -07:00
bin
d75fe95685 virtcontainers: replace newStore by store in Sandbox struct
The property name make newcomers confused when reading code.
Since in Kata Containers 2.0 there will only be one type of store,
so it's safe to replace it by `store` simply.

Fixes: #1660

Signed-off-by: bin <bin@hyper.sh>
2021-04-08 23:59:16 +08:00
Eric Ernst
324b026a77
Merge pull request #1604 from wainersm/agent_mount-1
agent: log the mount point if it is already mounted
2021-04-08 08:26:12 -07:00
Eric Ernst
6d3053be4c
Merge pull request #1656 from fidencio/wip/update-agent-ctl-cargo-lock
tools/agent-ctl: Update Cargo.lock
2021-04-08 08:25:17 -07:00
Fupan Li
521887db16
Merge pull request #1648 from Tim-Zhang/rework-debug-console
agent: Rework the debug console
2021-04-08 23:14:52 +08:00
Fabiano Fidêncio
342eb765c2 tools/agent-ctl: Update Cargo.lock
While performing a `make` in the top directory of the project, I've
noticed that agent-ctl's Cargo.lock file was not up-to-date.

Fixes: #1655

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-08 10:24:08 +02:00
Tim Zhang
24b0703fda agent: fix test for the debug console
Fix test for the debug console.

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-04-08 14:57:40 +08:00
Tim Zhang
790332575b agent: async the debug console
Make the debug console in this commit.
Finish the rework of debug console.

Fixes: #1647

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-04-08 14:57:36 +08:00
David Gibson
8ea2ce9a31 agent/device: Remove legacy uevent matching
DevAddrMatcher existed purely as a transitional step as we refined the
uevent matching logic for each of the different device types we care about.
We've now done that, so it can be removed along with several related
pieces.

fixes #1628

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-04-08 12:30:18 +10:00
David Gibson
5d007743c1 agent/device: Refine uevent matching for pmem devices
Use the new uevent matching infrastructure to refine the matching for pmem
devices to something more pinned down to that device type.  While we're
there, fix a few anciliary problems with get_pmem_device_name():

- The name is poor - the *input* to this function is the expected device
  name, so the result isn't helpful, except that it needs to wait for the
  device to be ready in the guest.  Change it to wait_for_pmem_device() and
  explicitly check that the returned device name matches the one expected.
- Remove an incorrect comment in nvdimm_storage_handler() (the only caller)
  which appears to have been copied from the virtio-blk path, but then
  become stale.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-04-08 12:02:39 +10:00
James O. D. Hunt
9017e1100b agent: start to rework the debug console
It's the first commit of the rework.

Fixes: #1647

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-04-08 09:57:48 +08:00
David Gibson
a59e07c1f9 agent/define: Refine uevent matching for virtio-scsi devices
Current get_scsi_device_name() uses the legacy uevent matching which
isn't very precise.  This refines it to use a specific matcher
implementation.  While we're at it:

- No longer insist on the SCSI controller being under the PCI root.
  It generally will be, but there's no particular reason to require
  it.

The matcher still has a problem in that it won't work sensibly if
there are multiple SCSI busses in the guest.  Fixing that requires
changes on the runtime side as well, though, so it's beyond scope for
this change.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-04-08 11:13:00 +10:00
David Gibson
484a364729 agent/device: Rework uevent handling for virtio-blk devices
There are some problems with get_pci_device_name():

1) It's misnamed: in fact it is only used for handling virtio-blk PCI
   devices.  It's also only correct for virtio-blk devices, the event
   matching doesn't locate the "raw" PCI device, but rather the block
   device created by virtio-blk as a child of the PCI device itself.

2) The uevent matching is imprecise.  As all things using the legacy
   DevAddrMatcher, it matches on a bunch of conditions used across several
   different device types, not all of which make sense for virtio-blk pci
   devices specifically.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-04-08 11:13:00 +10:00
Eric Ernst
15c2d7ed30
Merge pull request #1400 from ManaSugi/update-oci-seccomp
oci: Update seccomp configuration
2021-04-07 15:18:19 -07:00
Carlos Venegas
ec36883fe3
Merge pull request #1638 from jcvenegas/2021-04-06/linux-5.10.x-qemu-experimental
kernel: update experimental kernel to 5.10.x
2021-04-07 16:02:44 -05:00
Wainer Moschetta
69dbcaa32b
Merge pull request #1633 from fidencio/wip/fix-test-kata-deploy-script
kata-deploy: Fix `test-kata.sh` and do some small cleanups / improvements in the kata-deploy script
2021-04-07 16:13:17 -03:00
GabyCT
a374b007bd
Merge pull request #1646 from fidencio/wip/fix-slash-command-action-usage
github: Fix slash-command-action usage
2021-04-07 10:17:04 -05:00
GabyCT
d922070c50
Merge pull request #1644 from lifupan/fix_env
rustjail: fix the issue of missing default home env
2021-04-07 10:16:07 -05:00
GabyCT
81bcded9a3
Merge pull request #1492 from dgibson/uevent
Make uevent watching mechanism more flexible
2021-04-07 10:15:33 -05:00
Fabiano Fidêncio
d2fda148fa
Merge pull request #1637 from wainersm/openshift_ci_centos8
ci/openshift-ci: Prepare to build on CentOS 8
2021-04-07 15:37:41 +02:00
Fabiano Fidêncio
514ba369fd
Merge pull request #1630 from liubin/1629-update-containerd-config
docs: update configuration for passing annotations in conatinerd
2021-04-07 15:36:21 +02:00
Fabiano Fidêncio
7873b7a1f9 github: Fix slash-command-action usage
`/test-kata-deploy` command does **not** work, and the output returned
is:
```
Error: Comment didn't contain a valid slash command
```

So, why does this happen?

This is the regex used: `^\/([\w]+)\b *(.*)?$`, being the important part
of the command "\/([\w]+)\b", with the rest being arguments to it.
Okay, `\w` is the key here, as `\w` means: a-z, A-Z, 0-9, including the
_.

Our command is `/test-kata-deploy`, and `-` is not present as part of
`\w`.  Knowing this we need to update the command to something like:
`/test_kata_deploy`

Fixes: #1645

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-07 14:46:21 +02:00
Fabiano Fidêncio
b09fda36bd
Merge pull request #1641 from fidencio/wip/re-add-kata-deploy-test-action
Revert "github: Remove kata-deploy-test action"
2021-04-07 13:18:25 +02:00
Fabiano Fidêncio
eda8da1ec5 github: Revert "github: Remove kata-deploy-test action"
This partially reverts commit 1cce930071.

As mentioned in #1635, the malformed yaml wouldn't allow us to actually
test changes that were supposed to be test by this action.

So, this is now reverted and adapted accordingly.

Main differences from what we had before:
* As it tests kata-deploy itself, not the statically built binaries,
  let's just use the binaries from 2.0.0 release;
* Adapt download and deploy location to the
  `kata-containers/kata-containers` repo, as the original action was
  based on 1.x repos;

Fixes: #1640

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-07 12:07:30 +02:00
fupan.lfp
a938d90310 rustjail: fix the issue of missing default home env
first get the "HOME" env from "/etc/passwd", if
there's no corresponding uid entry in /etc/passwd,
then set "/" as the home env.

Fixes: #1643

Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
2021-04-07 15:11:28 +08:00
bin
b0e4618e84 docs: update configuration for passing annotations in conatinerd
Using "io.containerd.kata.v2" instead of deprecated "io.containerd.runc.v1".

Fixes: #1629

Signed-off-by: bin <bin@hyper.sh>
2021-04-07 10:16:02 +08:00
Fabiano Fidêncio
d43098ec21 kata-deploy: Adapt regex for testing kata-deploy
On commit 60f6315 we've started adding the specific version of the image
to be used, in order to ensure people using our content from a tarball
would be relying on the correct image.

However, later on, @bergwolf figured out it had some undesired side
effects, such as
https://github.com/kata-containers/kata-containers/runs/2235812941?check_suite_focus=true

What happens there is that the regular expression used to point the
image to a testing one doesn't take into consideration the $VERSION, and
that breaks the deployment.

Depends-on: github.com/kata-containers/kata-containers#1641
Fixes: #1632

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-06 23:41:24 +02:00
GabyCT
0b87fd436f
Merge pull request #1544 from snir911/timeout
runtime: increase dial timeout
2021-04-06 16:10:51 -05:00
Carlos Venegas
84e453a643
Merge pull request #1625 from jcvenegas/2021-04-05/qemu-experimental-6.0
qemu experimental: Move to latest tree on virtio-fs-dev (qemu 6.0 + DAX patches).
2021-04-06 15:29:05 -05:00
Carlos Venegas
107ceca680 kernel: update experimental kernel to 5.10.x
Relevant changes for experimental :

42d3e2d04 virtiofs: calculate number of scatter-gather elements accurately
413daa1a3 fuse: connection remove fix
bf109c640 fuse: implement crossmounts
1866d779d fuse: Allow fuse_fill_super_common() for submounts
fcee216be fuse: split fuse_mount off of fuse_conn
8f622e949 fuse: drop fuse_conn parameter where possible
24754db27 fuse: store fuse_conn in fuse_req
c6ff213fe fuse: add submount support to <uapi/linux/fuse.h>
d78092e49 fuse: fix page dereference after free
9a752d18c virtiofs: add logic to free up a memory range
d0cfb9dcb virtiofs: maintain a list of busy elements
6ae330cad virtiofs: serialize truncate/punch_hole and dax fault path
9483e7d58 virtiofs: define dax address space operations
2a9a609a0 virtiofs: add DAX mmap support
c2d0ad00d virtiofs: implement dax read/write operations
ceec02d43 virtiofs: introduce setupmapping/removemapping commands
fd1a1dc6f virtiofs: implement FUSE_INIT map_alignment field
45f2348ec virtiofs: keep a list of free dax memory ranges
1dd539577 virtiofs: add a mount option to enable dax
22f3787e9 virtiofs: set up virtio_fs dax_device
f4fd4ae35 virtiofs: get rid of no_mount_options
b43b7e81e virtiofs: provide a helper function for virtqueue initialization

Fixes: #1639

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-04-06 19:06:09 +00:00
Fabiano Fidêncio
ca4dccf980 release: Get rid of "master"
We don't use the "master" branch for anything in
`kata-containers/kata-containers`.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-06 20:28:42 +02:00
Fabiano Fidêncio
c2197cbf2b release: Use sudo to install hub
This doesn't make much difference for the automated process we have in
place, but makes a whole lot of difference for those trying to have the
binaries deployed locally.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-06 20:28:42 +02:00
Fabiano Fidêncio
dc373d0161
Merge pull request #1635 from fidencio/wip/remove-kata-deploy-test-action
github: Remove kata-deploy-test action
2021-04-06 20:27:54 +02:00
Wainer dos Santos Moschetta
49eec92038 agent: log the tag and mount point if it is already mounted
On commit 17e9a2cff5 it was introduced a guard for the case the mount point is already
mounted. Instead of log only the mount tag ("kataShared") with this change it will print
both tag and mount point path.

Fixes: #1398
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-04-06 14:14:59 -04:00
Wainer dos Santos Moschetta
16f732fc18 ci/lib: Use git to clone the tests repository
On clone_tests_repo() use git instead of `go get` to clone and/or
update the tests repository.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-04-06 13:44:02 -04:00
Wainer dos Santos Moschetta
9281e56705 ci/openshift-ci: Add build root dockerfile
This adds the dockerfile which is used by the OpenShift CI operator to build
the build root image. It is installed git as it is required by the operator
to clone repositories. The sudo package is also installed because many scripts
relies on the command but it is not installed by tests/.ci/setup_env_centos.sh.

Fixes #1636

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-04-06 13:44:02 -04:00
Fabiano Fidêncio
1cce930071 github: Remove kata-deploy-test action
Currently the action is not running because it's broken, and it was
broken by 50fea9f.

Sadly, I cannot just test a fix on a PR as every single time we end up
triggering what's currently on main, rather than triggering the content
of the PR itself.

With this in mind, let me just remove the file and re-add it as part of
a new PR and, hopefully, have it tested in this way.

Sorry for the breakage, by the way.

Fixes: #1634

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-06 19:01:53 +02:00