Commit Graph

6833 Commits

Author SHA1 Message Date
Gabriela Cervantes
1d25d7d43f docs: Remove kata-proxy and binaries reference
Kata-proxy is not longer used in kata 2.x, this PR removes the
reference as well to an script that is not longer existing.

Fixes #2391

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2021-08-04 22:00:57 +00:00
Chelsea Mafrica
eac05ad6d6
Merge pull request #2375 from sameo/upstream/topic/process-cwd
agent: Create the process CWD when it does not exist
2021-08-04 11:35:11 -07:00
Anastassios Nanos
64dd35ba4f virtcontainers: fc: properly remove jailed block device
When running a firecracker instance jailed, block devices
are not removed correctly, as the jailerRoot path is not
stripped from the PATCH command sent to the FC API.

This patch differentiates the jailed case from the non-jailed
one and allows the firecracker instance to be properly
terminated.

Fixes #2387

Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
2021-08-04 16:31:56 +00:00
Julio Montes
03325f0612
Merge pull request #2382 from dgibson/prep-qemu-6.1
Update Kata to allow it to use Qemu 6.1
2021-08-04 09:16:06 -05:00
GabyCT
48bb104660
Merge pull request #2385 from Jakob-Naucke/dracut-libs
osbuilder/dracut: Add missing libraries
2021-08-04 09:04:46 -05:00
Jakob Naucke
b8133a188c
osbuilder/dracut: Add missing libraries
When the guest is built using dracut and the agent uses glibc (esp.
ppc64le/s390x), libraries might be missing. In my case, it was
`libutil.so`, but more can be added easily. Add a script to configure
`install_items` for dracut w.r.t. `ldd` of the agent.

Fixes: #2384
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-08-04 13:50:42 +02:00
Jakob Naucke
d473967120
Merge pull request #2379 from Jakob-Naucke/env-os-version
osbuilder: pass env OS_VERSION
2021-08-04 10:40:17 +02:00
David Gibson
831c2feead packaging: Remove reference to sheepdog driver
The QEMU sheepdog driver was deprecated in 5.2.0 and removed entirely in
6.1.  Explicitly disabling, therefore is unnecessary from 5.2.0 and will
give an error from 6.1.

fixes #2337

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-08-04 15:04:36 +10:00
David Gibson
2e28b71473 packaging: Drop support for qemu < 5.0
We only test qemu 5.2 in the CI (5.1 for ARM), and I believe we already
have some subtle dependencies that will stop things working on older qemu
versions.

We just updated govmm to a version that explicitly only works with qemu 5.0
and later, so we can drop stale checks for older qemu versions.  More
specifically that means we can drop patches for older qemu versions, and
remove checks for older qemu versions from configure-hypervisor.sh.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-08-04 15:04:36 +10:00
David Gibson
d5f85698e1 vendor: Update govmm
Update to commit 3c64244cbb, in particular to get these fixes which
are needed to work with qemu-6.0 and later:

https://github.com/kata-containers/govmm/pull/192
https://github.com/kata-containers/govmm/pull/194

Git log

d27256f (qmp: Don't use deprecated 'props' field for object-add, 2021-08-03)
d8cdf9a (qemu: Drop support for versions older than 5.0, 2021-08-03)
1b02192 (Use 'host_device' driver for blockdev backends, 2021-07-29)
9518675 (add support for "sandbox" feature to qemu, 2021-07-20)
335fa81 (qemu: fix golangci-lint errors, 2021-07-21)
61b6378 (.github/workflows: reimplement github actions CI, 2021-07-21)
9d6e797 (go: support go modules, 2021-07-21)
0d21263 (qemu: support read-only nvdimm, 2021-07-21)

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-08-04 15:04:30 +10:00
David Gibson
3165095669 runtime/qemu: Use explicit "on" for kernel_irqchip parameter
Kata uses the 'kernel_irqchip' machine option to qemu.  By default it
uses it in what qemu calls the "short-form boolean" with no parameter.
That style was deprecated by qemu between 5.2 and 6.0 (commit
ccd3b3b8112b) and effectively removed entirely between 6.0 and 6.1
(commit d8fb7d0969d5).

Update ourselves for newer qemus by using an explicit
"kernel_irqchip=on".

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-08-04 14:34:11 +10:00
Carlos Venegas
b24ee4b11e
Merge pull request #2369 from converge/issue_2354
tools: shorten directory path
2021-08-03 15:32:56 -05:00
Carlos Venegas
27b9a68189
Merge pull request #2365 from sameo/topic/clh-tracing
virtcontainers: clh: Do not use the default HTTP client
2021-08-03 12:54:09 -05:00
Jakob Naucke
a72b08117f
osbuilder: pass env OS_VERSION
With lines like
0a2e2c6038/tools/osbuilder/rootfs-builder/fedora/config.sh (L8)
we imply that one can set another OS_VERSION and it will get picked up.
This is not the case when building inside Docker/Podman because the
variable is not passed to the container, which can lead to confusion.
Forward this env.

Fixes: #2378
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-08-03 18:28:17 +02:00
Samuel Ortiz
0a2e2c6038
Merge pull request #2358 from YchauWang/wyc-deploy-test
docs: update kata deploy README doc to add cloud-hypervisor test command
2021-08-03 10:55:13 +02:00
Fupan Li
eda55195fb
Merge pull request #2373 from teawater/fix_swap2
Container: Add initConfigResourcesMemory and call it in newContainer
2021-08-03 10:58:48 +08:00
Joao Vanzuita
d007bb8550 kata-deploy: shorten directory path
long file paths are difficult to read, this change adds a new readonly variable to shorten the full file path of the static build folder files.

Fixes: #2354
Signed-off-by: Joao Vanzuita <joaovanzuita@me.com>
2021-08-02 22:37:39 +02:00
Hui Zhu
e6408fe670 Container: Add initConfigResourcesMemory and call it in newContainer
The swappiness is not right if just set
io.katacontainers.container.resource.swappiness:
$ pod_yaml=pod.yaml
$ container_yaml=container.yaml
$ image="quay.io/prometheus/busybox:latest"
$ cat << EOF > "${pod_yaml}"
metadata:
  name: busybox-sandbox1
EOF
$ cat << EOF > "${container_yaml}"
metadata:
  name: busybox-killed-vmm
annotations:
  io.katacontainers.container.resource.swappiness: "100"
image:
  image: "$image"
command:
- top
EOF
$ sudo crictl pull $image
$ podid=$(sudo crictl runp $pod_yaml)
$ cid=$(sudo crictl create $podid $container_yaml $pod_yaml)
$ sudo crictl start $cid
crictl exec $cid cat /sys/fs/cgroup/memory/memory.swappiness
60

The cause of this issue is there are two elements store the resources
infomation.  They are c.config.Resources for calculateSandboxMemory and
c.GetPatchedOCISpec() for agent.
This add initConfigResourcesMemory to Container and call it in
newContainer to handle the issue.

Fixes: #2372

Signed-off-by: Hui Zhu <teawater@antfin.com>
2021-08-02 16:02:12 +08:00
Fupan Li
fdc42ca7ff
Merge pull request #2324 from jongwu/ro_nv
qemu/arm: remove nvdimm/"ReadOnly" option on arm64
2021-08-02 14:14:06 +08:00
Fupan Li
186a02593a
Merge pull request #2368 from teawater/fix_swap
Fix issue container start fail if io.katacontainers.container.resource.swap_in_bytes and memory_limit_in_bytes are not set
2021-08-02 10:08:30 +08:00
Samuel Ortiz
49083bfa31 agent: Create the process CWD when it does not exist
Although the OCI specification does not explictly requires that, we
should create the process CWD if it does not exist, before chdir'ing
to it. Without that fizx, the kata-agent fails to create a container
and returns a grpc error when it's trying to change the containerd
working directory to an non existing folder.

runc, the OCI runtime reference implementation, also creates the process
CWD when it's not part of the container rootfs.

Fixes #2374

Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
2021-08-01 04:27:03 +02:00
Hui Zhu
ee90affc18 newContainer: Initialize c.config.Resources.Memory if it is nil
container start fail if
io.katacontainers.container.resource.swap_in_bytes and
memory_limit_in_bytes are not set.
$ pod_yaml=pod.yaml
$ container_yaml=container.yaml
$ image="quay.io/prometheus/busybox:latest"
$ cat << EOF > "${pod_yaml}"
metadata:
  name: busybox-sandbox1
EOF
$ cat << EOF > "${container_yaml}"
metadata:
  name: busybox-killed-vmm
annotations:
  io.katacontainers.container.resource.swappiness: "60"
image:
  image: "$image"
command:
- top
EOF
$ sudo crictl pull $image
$ podid=$(sudo crictl runp $pod_yaml)
$ cid=$(sudo crictl create $podid $container_yaml $pod_yaml)
$ sudo crictl start $cid
DEBU[0000] get runtime connection
DEBU[0000] connect using endpoint
'unix:///var/run/containerd/containerd.sock' with '10s' timeout
DEBU[0000] connected successfully using endpoint:
unix:///var/run/containerd/containerd.sock
DEBU[0000] StartContainerRequest:
&StartContainerRequest{ContainerId:4fea91d16f661931fe33acd247efe831ef9e571588ba18b5a16f04c278fd61b8,}
DEBU[0000] StartContainerResponse: nil
FATA[0000] starting the container
"4fea91d16f661931fe33acd247efe831ef9e571588ba18b5a16f04c278fd61b8": rpc
error: code = Unknown desc = failed to create containerd task: failed to
create shim: ttrpc: closed: unknown

The cause of fail if if c.config.Resources.Memory is nil, values of
io.katacontainers.container.resource.swappiness and
io.katacontainers.container.resource.swap_in_bytes will be store in
newContainer.

This commit initialize c.config.Resources.Memory if it is nil in
newContainer.

Fixes: #2367

Signed-off-by: Hui Zhu <teawater@antfin.com>
2021-08-01 10:03:27 +08:00
Hui Zhu
767a41ce56 updateResources: Log result after calculateSandboxMemory
Log result after calculateSandboxMemory in updateResources.

Fixes: #2367

Signed-off-by: Hui Zhu <teawater@antfin.com>
2021-08-01 09:57:44 +08:00
Samuel Ortiz
760ec4e58a virtcontainers: clh: Do not use the default HTTP client
When enabling tracing with Cloud Hypervisor, we end up establishing 2
connections to 2 different HTTP servers: The Cloud Hypervisor API one
that runs over a UNIX socket and the Jaeger endpoint running over UDP.

Both connections use the default HTTP golang client instance, and thus
share the same transport layer. As the Cloud Hypervisor implementation
sets it up to be over a Unix socket, the jaeger uploader ends up going
through that transport as well, and sending its spans to the Cloud
Hypervisor API server.

We fix that by giving the Cloud Hypervisor implementation its own HTTP
client instance and we avoid sharing it with anything else in the shim.

Fixes #2364

Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
2021-07-30 16:51:01 +02:00
GabyCT
b94252a5fa
Merge pull request #2062 from jodh-intel/add-tracing-design-proposals
docs: Add tracing proposals doc
2021-07-30 09:37:38 -05:00
Fabiano Fidêncio
0c913040b6
Merge pull request #2023 from jodh-intel/docs-rm-tocs
docs: Remove table of contents
2021-07-30 15:44:50 +02:00
Jakob Naucke
d9359440e2
Merge pull request #2361 from fidencio/wip/workflows-fix-skip-ci
static-checks: Check for the `force-skip-ci` label on each step
2021-07-30 15:07:25 +02:00
Fabiano Fidêncio
3fe6695b0a static-checks: Check for the force-skip-ci label on each step
This is not the most beautiful solution, but when do the check on every
single step we ensure the test at least started, and consequently will
succeed.

Without this the tests wouldn't even start, making any PR using the
`force-skip-ci` label not mergeable.

Fixes: #2362

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-07-30 13:51:56 +02:00
James O. D. Hunt
7df56301be CI: Call agent shutdown test
Run the agent shutdown test as part of CI testing code in this repo.

Fixes: #1808.

Depends-on:github.com/kata-containers/tests#3495

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-07-30 11:03:25 +01:00
James O. D. Hunt
57b696a5ec docs: Removed mention of 1.x
All users should be running 2.x releases so remove the legacy details
since it's arguably confusing to have two sets of details.

Reworked the components listed in the main README so that rather than
being sorted alphabetically, they are now sorted in semi-order of
importance and split into two tables to make the point more clearly.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-07-30 10:58:22 +01:00
James O. D. Hunt
4f0726bc49 docs: Remove table of contents
Removed all TOCs now that GitHub auto-generates them.

Also updated the documentation requirements doc removing the requirement
to add a TOC.

Fixes: #2022.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-07-30 10:58:22 +01:00
James O. D. Hunt
f186c5e284 docs: Fix invalid URLs
Correct broken / stale URLs as detected by the CI URL checker.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-07-30 10:58:22 +01:00
James O. D. Hunt
7c610a6ff1 docs: Fix shell code
Correct the shell code in the packaging tools README to keep the CI
happy.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-07-30 10:58:22 +01:00
Fabiano Fidêncio
1d70523a59
Merge pull request #2356 from YchauWang/wyc-doc-bug-fix
docs: update the kata release url in the kata deploy document
2021-07-30 11:57:26 +02:00
wangyongchao.bj
80afba15ee docs: update kata deploy README doc to add cloud-hypervisor test command
Kata deploy README document only contains Firecracker and Qemu. This PR adds
 cloud-hypervisor test command to the README.md file.

Fixes: #2357

Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
2021-07-30 10:01:13 +08:00
wangyongchao.bj
5a0d3c4fac docs: update the kata release url in the kata deploy document
fixed the url error, updated the path to kata 2.x release
(https://github.com/kata-containers/kata-containers/releases) from kata 1.x release
(https://github.com/kata-containers/runtime/releases) in the kata-deploy README.md file.

Fixes: #2355.

Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
2021-07-30 09:50:30 +08:00
Fabiano Fidêncio
2d142bc92d
Merge pull request #2155 from jcvenegas/kata-deploy-2021-06-29
kata-deploy: Allow build  kata-deploy  tarball from HEAD
2021-07-29 22:50:03 +02:00
Fabiano Fidêncio
7ed6423dd8
Merge pull request #2347 from fidencio/wip/update-runc-and-containerd-deps
mod: unify runc and containerd dependencies
2021-07-29 22:29:06 +02:00
Peng Tao
9514dda52e mod: unity containerd dependency
The old ones are carrying CVEs, do not use them.

PS: In order to update the modules, we're running `make handle_vendor`
target from the runtime's Makefile.  This is now part of the CI and
ensures that the vendored code is up-to-date.  It's important to note
that older versions of golang may generate different results for those,
but those versions are not supported anymore, so we're good to go with
what we have in the CI (1.15 and 1.16).

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-07-29 20:51:02 +02:00
Peng Tao
6ffe37b949 mod: unify runc dependency
Since the old ones are carrying CVEs. Do not use them.

PS: In order to update the modules, we're running `make handle_vendor`
target from the runtime's Makefile.  This is now part of the CI and
ensures that the vendored code is up-to-date.  It's important to note
that older versions of golang may generate different results for those,
but those versions are not supported anymore, so we're good to go with
what we have in the CI (1.15 and 1.16).

Fixes: #2338

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-07-29 20:48:52 +02:00
GabyCT
8508e52fdc
Merge pull request #2349 from teawater/fix_vm_doc
how-to-use-virtio-mem-with-kata.md: Remove undefined ${REPORT_DIR}
2021-07-29 10:42:44 -05:00
James O. D. Hunt
5b514177b0 docs: Add tracing proposals doc
Create a document summarising the tracing design proposals
from PR #1937.

Fixes: #2061.

Signed-off-by: bin <bin@hyper.sh>
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-07-29 16:17:48 +01:00
Fabiano Fidêncio
6b2ad64aea
Merge pull request #2344 from fidencio/wip/run-static-checks-also-when-prs-are-edited-or-syncrhonized
ci: Run static checks when PRs are updated
2021-07-29 10:23:12 +02:00
Hui Zhu
b53e84054e how-to-use-virtio-mem-with-kata.md: Remove undefined ${REPORT_DIR}
Remove undefined ${REPORT_DIR} in how-to-use-virtio-mem-with-kata.md.

Fixes: #2348

Signed-off-by: Hui Zhu <teawater@antfin.com>
2021-07-29 14:50:08 +08:00
Fabiano Fidêncio
5957bc7d9c ci: Run static checks when PRs are updated
Looking at the changes that could cause the static-checks not to run
when a PR is updated I think 7db8a85a1f
could be the one that introduced such a regression.

Let's (try to) fix this by enforcing the workflow to run also when the
PR has been "edited" and "synchronized".

Fixes: #2343

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-07-28 22:26:55 +02:00
Carlos Venegas
81e6bf6f2c kata-deploy: Split shimv2 build in a separate container.
Instead of install golang in the base container, split the shimv2 build.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-28 19:45:35 +00:00
Carlos Venegas
d46ae3248e kernel: build: Add container build
Add script to build kernel in a container.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-28 19:45:35 +00:00
Carlos Venegas
b789a935cf actions: release: Use new kata-deploy scripts.
Update release action to build kata assets with new
scripts.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-28 19:45:35 +00:00
Carlos Venegas
85987c6d79 kata-deploy: Add Makefile
Add makefile to document possible options to run.

e.g
Default: Create a kata tarball, it will build assets concurrently.
```
$ make

```

Create a tarball build for cloud-hypervisor.
```
$ make cloud-hypervisor
```

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-28 19:45:35 +00:00
Carlos Venegas
b9d2eea39b kata-deploy: Add script to merge kata tarballs.
After each asset is build it is needed to merge them all into one single
tarball.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-28 19:45:35 +00:00