Commit Graph

6800 Commits

Author SHA1 Message Date
Jianyong Wu
6871aeaa60 snap: enable snap build for arm64
snap build for arm64 fail for a long time, here we enable it.
the changes:
1. correct the variable of "branch"
2. add v5.1.0 under tag_patchs

Fixes: #2194
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-08-06 17:11:58 +08:00
Fabiano Fidêncio
15e0a3c8f0 kata-deploy: Remove unneeded yum cached files
Let's just remove the cached failes as those are not needed for anything
we do when using this image.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-08-06 11:01:58 +02:00
Fabiano Fidêncio
d01aebebae kata-deploy: Ensure the system is up-to-date
In order to avoid providing an image with security issues, let's ensure
we run `yum update` as part of our image build process.  This is needed
as even with the latest CentOS images there may be fix provided by some
CVE that's already part of the updates but not yet part of the image.

In our case, it's even more needed as the `centos/systemd` image has not
been updated for 3 years or so and those are the vulnerabilities found
in the current images:
https://quay.io/repository/kata-containers/kata-deploy?tab=tags

Fixes: #2303

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-08-06 11:01:58 +02:00
Fabiano Fidêncio
b4b843178c
Merge pull request #2406 from fidencio/wip/workflows-login-to-the-correct-registry
workflows: Actually login to quay.io
2021-08-06 11:01:18 +02:00
Fabiano Fidêncio
77160e591a workflows: Actually login to quay.io
9fa1febfd9 added the support to also push
the image to quay.io.  However, we didn't try explicitly pass quay.io as
the registry server, causing then to login to fail.

Fixes: #2306

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-08-06 10:07:25 +02:00
Fabiano Fidêncio
2f5a77d144
Merge pull request #2307 from fidencio/wip/kata-deploy-also-push-images-to-quay-io
kata-deploy: Update our content to use / point to quay.io/kata-containers rather than katadocker
2021-08-06 09:23:35 +02:00
Fabiano Fidêncio
b9e03a1cf1 docs: update the image repository to quay.io
This can help our users to **not** hit the pull limitation imposed by
dockerhub.

Fixes: #2306

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-08-05 22:53:20 +02:00
Fabiano Fidêncio
f47cad3d95 tools: Update the image repository to quay.io
This can help our users to **not** hit the pull limitation imposed by
dockerhub.

Fixes: #2306

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-08-05 22:53:20 +02:00
Fabiano Fidêncio
9fa1febfd9 workflows: Also push the image to quay.io
Now that the project owns https://quay.io/kata-containers, let's also
push our images there and start making it our primary image repository.

Fixes: #2306

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-08-05 22:53:19 +02:00
Samuel Ortiz
233b53c048 agent: Fix cargo 1.54 clippy warning
Mostly the needless borrow one, plus a few others that are now enforced.

Fixes #2408

Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
2021-08-05 18:41:55 +02:00
Francesco Giudici
2d8386ea52 kata-monitor: add few unit tests
Add cri.go unit tests

Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
2021-08-05 11:41:54 +02:00
Francesco Giudici
8714a35063 kata-monitor: make code to identify kata pods simpler
just search for the "kata" substring in the runtime value and log at
info level when the runtime name/type is not found.

Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
2021-08-05 11:41:54 +02:00
Francesco Giudici
68a6f011b5 kata-monitor: drop the runtime info from the sandbox cache
We keep the container engine info in the sandbox cache map, as the value
associated to the pod id (the key). Since we used that in
getMonitorAddress() only (which is gone) we can avoid storing that
information. Let's drop it.
Keep the map structure and the [put,delete]IfExists functions as we may
want to move to an event based cache update process sooner or later, and
we will need those.

Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
2021-08-05 11:41:54 +02:00
Francesco Giudici
97dcc5f78a kata-monitor: drop getMonitorAddress()
since the shim socket path is statically defined in the containerd-shimv2
code, we don't need to retrieve the socket name from the filesystem:
construct the socket name using the containerd-shimv2 code.

Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
2021-08-05 11:41:54 +02:00
Francesco Giudici
0b03d97d0b vendor: update vendors for kata-monitor
kata-monitor switched from containerd client to CRI. Update the
dependencies and vendored code.

go mod tidy
go mod vendor

Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
2021-08-05 11:41:54 +02:00
Francesco Giudici
c2f03e8993 kata-monitor: talk to the container engine via the CRI
kata-monitor uses containerd client to retrieve information from the
container engine. This makes kata-monitor work with the containerd
container engine only.
Bin Liu (bin <bin@hyper.sh>) worked on a kata-monitor version able
to talk to any container engine leveraging the standard CRI[1].
Here, the original work of Bin Lui has been adapted on the current
kata-monitor to make it container engine independent.

[1] https://github.com/liubin/kata-containers/tree/fix/1030-use-cri-in-kata-monitor

Fixes: #1030
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
2021-08-05 11:41:54 +02:00
David Gibson
c867d1e069 osbuilder: Drop Go agent support
With Kata 1.x EOL, the Go agent is no more.  So, remove support for it from
the osbuilder scripts.  This removes the RUST_AGENT variable, treating it
as always true.

fixes #2396

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-08-05 16:10:10 +10:00
Gabriela Cervantes
1d25d7d43f docs: Remove kata-proxy and binaries reference
Kata-proxy is not longer used in kata 2.x, this PR removes the
reference as well to an script that is not longer existing.

Fixes #2391

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2021-08-04 22:00:57 +00:00
Chelsea Mafrica
eac05ad6d6
Merge pull request #2375 from sameo/upstream/topic/process-cwd
agent: Create the process CWD when it does not exist
2021-08-04 11:35:11 -07:00
Anastassios Nanos
64dd35ba4f virtcontainers: fc: properly remove jailed block device
When running a firecracker instance jailed, block devices
are not removed correctly, as the jailerRoot path is not
stripped from the PATCH command sent to the FC API.

This patch differentiates the jailed case from the non-jailed
one and allows the firecracker instance to be properly
terminated.

Fixes #2387

Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
2021-08-04 16:31:56 +00:00
Julio Montes
03325f0612
Merge pull request #2382 from dgibson/prep-qemu-6.1
Update Kata to allow it to use Qemu 6.1
2021-08-04 09:16:06 -05:00
GabyCT
48bb104660
Merge pull request #2385 from Jakob-Naucke/dracut-libs
osbuilder/dracut: Add missing libraries
2021-08-04 09:04:46 -05:00
Jakob Naucke
b8133a188c
osbuilder/dracut: Add missing libraries
When the guest is built using dracut and the agent uses glibc (esp.
ppc64le/s390x), libraries might be missing. In my case, it was
`libutil.so`, but more can be added easily. Add a script to configure
`install_items` for dracut w.r.t. `ldd` of the agent.

Fixes: #2384
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-08-04 13:50:42 +02:00
Jakob Naucke
d473967120
Merge pull request #2379 from Jakob-Naucke/env-os-version
osbuilder: pass env OS_VERSION
2021-08-04 10:40:17 +02:00
David Gibson
831c2feead packaging: Remove reference to sheepdog driver
The QEMU sheepdog driver was deprecated in 5.2.0 and removed entirely in
6.1.  Explicitly disabling, therefore is unnecessary from 5.2.0 and will
give an error from 6.1.

fixes #2337

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-08-04 15:04:36 +10:00
David Gibson
2e28b71473 packaging: Drop support for qemu < 5.0
We only test qemu 5.2 in the CI (5.1 for ARM), and I believe we already
have some subtle dependencies that will stop things working on older qemu
versions.

We just updated govmm to a version that explicitly only works with qemu 5.0
and later, so we can drop stale checks for older qemu versions.  More
specifically that means we can drop patches for older qemu versions, and
remove checks for older qemu versions from configure-hypervisor.sh.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-08-04 15:04:36 +10:00
David Gibson
d5f85698e1 vendor: Update govmm
Update to commit 3c64244cbb, in particular to get these fixes which
are needed to work with qemu-6.0 and later:

https://github.com/kata-containers/govmm/pull/192
https://github.com/kata-containers/govmm/pull/194

Git log

d27256f (qmp: Don't use deprecated 'props' field for object-add, 2021-08-03)
d8cdf9a (qemu: Drop support for versions older than 5.0, 2021-08-03)
1b02192 (Use 'host_device' driver for blockdev backends, 2021-07-29)
9518675 (add support for "sandbox" feature to qemu, 2021-07-20)
335fa81 (qemu: fix golangci-lint errors, 2021-07-21)
61b6378 (.github/workflows: reimplement github actions CI, 2021-07-21)
9d6e797 (go: support go modules, 2021-07-21)
0d21263 (qemu: support read-only nvdimm, 2021-07-21)

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-08-04 15:04:30 +10:00
David Gibson
3165095669 runtime/qemu: Use explicit "on" for kernel_irqchip parameter
Kata uses the 'kernel_irqchip' machine option to qemu.  By default it
uses it in what qemu calls the "short-form boolean" with no parameter.
That style was deprecated by qemu between 5.2 and 6.0 (commit
ccd3b3b8112b) and effectively removed entirely between 6.0 and 6.1
(commit d8fb7d0969d5).

Update ourselves for newer qemus by using an explicit
"kernel_irqchip=on".

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-08-04 14:34:11 +10:00
Carlos Venegas
b24ee4b11e
Merge pull request #2369 from converge/issue_2354
tools: shorten directory path
2021-08-03 15:32:56 -05:00
Carlos Venegas
27b9a68189
Merge pull request #2365 from sameo/topic/clh-tracing
virtcontainers: clh: Do not use the default HTTP client
2021-08-03 12:54:09 -05:00
Jakob Naucke
a72b08117f
osbuilder: pass env OS_VERSION
With lines like
0a2e2c6038/tools/osbuilder/rootfs-builder/fedora/config.sh (L8)
we imply that one can set another OS_VERSION and it will get picked up.
This is not the case when building inside Docker/Podman because the
variable is not passed to the container, which can lead to confusion.
Forward this env.

Fixes: #2378
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-08-03 18:28:17 +02:00
Samuel Ortiz
0a2e2c6038
Merge pull request #2358 from YchauWang/wyc-deploy-test
docs: update kata deploy README doc to add cloud-hypervisor test command
2021-08-03 10:55:13 +02:00
Fupan Li
eda55195fb
Merge pull request #2373 from teawater/fix_swap2
Container: Add initConfigResourcesMemory and call it in newContainer
2021-08-03 10:58:48 +08:00
Joao Vanzuita
d007bb8550 kata-deploy: shorten directory path
long file paths are difficult to read, this change adds a new readonly variable to shorten the full file path of the static build folder files.

Fixes: #2354
Signed-off-by: Joao Vanzuita <joaovanzuita@me.com>
2021-08-02 22:37:39 +02:00
Hui Zhu
e6408fe670 Container: Add initConfigResourcesMemory and call it in newContainer
The swappiness is not right if just set
io.katacontainers.container.resource.swappiness:
$ pod_yaml=pod.yaml
$ container_yaml=container.yaml
$ image="quay.io/prometheus/busybox:latest"
$ cat << EOF > "${pod_yaml}"
metadata:
  name: busybox-sandbox1
EOF
$ cat << EOF > "${container_yaml}"
metadata:
  name: busybox-killed-vmm
annotations:
  io.katacontainers.container.resource.swappiness: "100"
image:
  image: "$image"
command:
- top
EOF
$ sudo crictl pull $image
$ podid=$(sudo crictl runp $pod_yaml)
$ cid=$(sudo crictl create $podid $container_yaml $pod_yaml)
$ sudo crictl start $cid
crictl exec $cid cat /sys/fs/cgroup/memory/memory.swappiness
60

The cause of this issue is there are two elements store the resources
infomation.  They are c.config.Resources for calculateSandboxMemory and
c.GetPatchedOCISpec() for agent.
This add initConfigResourcesMemory to Container and call it in
newContainer to handle the issue.

Fixes: #2372

Signed-off-by: Hui Zhu <teawater@antfin.com>
2021-08-02 16:02:12 +08:00
Fupan Li
fdc42ca7ff
Merge pull request #2324 from jongwu/ro_nv
qemu/arm: remove nvdimm/"ReadOnly" option on arm64
2021-08-02 14:14:06 +08:00
Fupan Li
186a02593a
Merge pull request #2368 from teawater/fix_swap
Fix issue container start fail if io.katacontainers.container.resource.swap_in_bytes and memory_limit_in_bytes are not set
2021-08-02 10:08:30 +08:00
Samuel Ortiz
49083bfa31 agent: Create the process CWD when it does not exist
Although the OCI specification does not explictly requires that, we
should create the process CWD if it does not exist, before chdir'ing
to it. Without that fizx, the kata-agent fails to create a container
and returns a grpc error when it's trying to change the containerd
working directory to an non existing folder.

runc, the OCI runtime reference implementation, also creates the process
CWD when it's not part of the container rootfs.

Fixes #2374

Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
2021-08-01 04:27:03 +02:00
Hui Zhu
ee90affc18 newContainer: Initialize c.config.Resources.Memory if it is nil
container start fail if
io.katacontainers.container.resource.swap_in_bytes and
memory_limit_in_bytes are not set.
$ pod_yaml=pod.yaml
$ container_yaml=container.yaml
$ image="quay.io/prometheus/busybox:latest"
$ cat << EOF > "${pod_yaml}"
metadata:
  name: busybox-sandbox1
EOF
$ cat << EOF > "${container_yaml}"
metadata:
  name: busybox-killed-vmm
annotations:
  io.katacontainers.container.resource.swappiness: "60"
image:
  image: "$image"
command:
- top
EOF
$ sudo crictl pull $image
$ podid=$(sudo crictl runp $pod_yaml)
$ cid=$(sudo crictl create $podid $container_yaml $pod_yaml)
$ sudo crictl start $cid
DEBU[0000] get runtime connection
DEBU[0000] connect using endpoint
'unix:///var/run/containerd/containerd.sock' with '10s' timeout
DEBU[0000] connected successfully using endpoint:
unix:///var/run/containerd/containerd.sock
DEBU[0000] StartContainerRequest:
&StartContainerRequest{ContainerId:4fea91d16f661931fe33acd247efe831ef9e571588ba18b5a16f04c278fd61b8,}
DEBU[0000] StartContainerResponse: nil
FATA[0000] starting the container
"4fea91d16f661931fe33acd247efe831ef9e571588ba18b5a16f04c278fd61b8": rpc
error: code = Unknown desc = failed to create containerd task: failed to
create shim: ttrpc: closed: unknown

The cause of fail if if c.config.Resources.Memory is nil, values of
io.katacontainers.container.resource.swappiness and
io.katacontainers.container.resource.swap_in_bytes will be store in
newContainer.

This commit initialize c.config.Resources.Memory if it is nil in
newContainer.

Fixes: #2367

Signed-off-by: Hui Zhu <teawater@antfin.com>
2021-08-01 10:03:27 +08:00
Hui Zhu
767a41ce56 updateResources: Log result after calculateSandboxMemory
Log result after calculateSandboxMemory in updateResources.

Fixes: #2367

Signed-off-by: Hui Zhu <teawater@antfin.com>
2021-08-01 09:57:44 +08:00
Samuel Ortiz
760ec4e58a virtcontainers: clh: Do not use the default HTTP client
When enabling tracing with Cloud Hypervisor, we end up establishing 2
connections to 2 different HTTP servers: The Cloud Hypervisor API one
that runs over a UNIX socket and the Jaeger endpoint running over UDP.

Both connections use the default HTTP golang client instance, and thus
share the same transport layer. As the Cloud Hypervisor implementation
sets it up to be over a Unix socket, the jaeger uploader ends up going
through that transport as well, and sending its spans to the Cloud
Hypervisor API server.

We fix that by giving the Cloud Hypervisor implementation its own HTTP
client instance and we avoid sharing it with anything else in the shim.

Fixes #2364

Signed-off-by: Samuel Ortiz <samuel.e.ortiz@protonmail.com>
2021-07-30 16:51:01 +02:00
GabyCT
b94252a5fa
Merge pull request #2062 from jodh-intel/add-tracing-design-proposals
docs: Add tracing proposals doc
2021-07-30 09:37:38 -05:00
Fabiano Fidêncio
0c913040b6
Merge pull request #2023 from jodh-intel/docs-rm-tocs
docs: Remove table of contents
2021-07-30 15:44:50 +02:00
Jakob Naucke
d9359440e2
Merge pull request #2361 from fidencio/wip/workflows-fix-skip-ci
static-checks: Check for the `force-skip-ci` label on each step
2021-07-30 15:07:25 +02:00
Fabiano Fidêncio
3fe6695b0a static-checks: Check for the force-skip-ci label on each step
This is not the most beautiful solution, but when do the check on every
single step we ensure the test at least started, and consequently will
succeed.

Without this the tests wouldn't even start, making any PR using the
`force-skip-ci` label not mergeable.

Fixes: #2362

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-07-30 13:51:56 +02:00
James O. D. Hunt
7df56301be CI: Call agent shutdown test
Run the agent shutdown test as part of CI testing code in this repo.

Fixes: #1808.

Depends-on:github.com/kata-containers/tests#3495

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-07-30 11:03:25 +01:00
James O. D. Hunt
57b696a5ec docs: Removed mention of 1.x
All users should be running 2.x releases so remove the legacy details
since it's arguably confusing to have two sets of details.

Reworked the components listed in the main README so that rather than
being sorted alphabetically, they are now sorted in semi-order of
importance and split into two tables to make the point more clearly.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-07-30 10:58:22 +01:00
James O. D. Hunt
4f0726bc49 docs: Remove table of contents
Removed all TOCs now that GitHub auto-generates them.

Also updated the documentation requirements doc removing the requirement
to add a TOC.

Fixes: #2022.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-07-30 10:58:22 +01:00
James O. D. Hunt
f186c5e284 docs: Fix invalid URLs
Correct broken / stale URLs as detected by the CI URL checker.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-07-30 10:58:22 +01:00
James O. D. Hunt
7c610a6ff1 docs: Fix shell code
Correct the shell code in the packaging tools README to keep the CI
happy.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-07-30 10:58:22 +01:00