Compare commits

...

59 Commits
2.2.2 ... 2.1.1

Author SHA1 Message Date
Fabiano Fidêncio
0e2be438bd Merge pull request #2007 from fidencio/2.1.1-branch-bump
# Kata Containers 2.1.1
2021-06-11 17:09:11 +02:00
Fabiano Fidêncio
55dede1bce release: Kata Containers 2.1.1
- stable-2.1 | week 23: weekly backports
- stable-2.1 | versions: Update kubernetes to 1.21.1
- stable-2.1 | Port fd leak fixes
- [stable-2.1] Weekly backports to stable-2.1 branch, May 31st 2021
- [backport]runtime: and cgroup and SandboxCgroupOnly check for check sub-command
- Weekly stable 2.1 backports may 24th
- [backport-2.1] workflows: release kata 2.x snap through the stable channel
- [2.1] how-to-use-virtio-mem-with-kata.md: Update doc to make it clear
- github: Do not run require porting labels on stable-2.1

492729f4 tools/packaging: clone meson and dependencies before building QEMU
db8d853b runtime: remove covertool from cli test
3fad5277 docs: Fix Release Process document
175970c9 versions: Update kubernetes to 1.21.1
1cc2ad3f agent: Fix fd leak caused by netlink
ac34f6df agent: Upgrade tokio-vsock to fix fd leak of vsock socket
915fea7b cgroup: fix the issue of set mem.limit and mem.swap
a05e1377 agent: re-enable the standard SIGPIPE behavior
8019f732 virtiofsd: Fix file descriptors leak and return correct PID
e48c9d42 runtime: and cgroup and SandboxCgroupOnly check for check sub-command
7874ab33 agent: fix start container failed when dropping all capabilities
536634e9 qemu: align before memory hotplug on arm64
c51891fe sandbox-bindmount: persist mount information
b137c7ac sandbox: Cleanup if failure to setup sandbox-bindmount occurs
68a77a7d workflows: release kata 2.x snap through the stable channel
550269ff how-to-use-virtio-mem-with-kata.md: Update doc to make it clear
1ea0dc98 github: Do not run require porting labels on stable-2.1

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-06-11 09:48:55 +02:00
Fabiano Fidêncio
4d514ba1fb Merge pull request #1978 from fidencio/wip/weekly-backports-week-23
stable-2.1 | week 23: weekly backports
2021-06-10 10:19:52 +02:00
Julio Montes
492729f443 tools/packaging: clone meson and dependencies before building QEMU
In some distros (Ubuntu 18 and 20) it's not possible to clone meson
and QEMU dependencies from https://git.qemu.org due to problems with
its certificates, let's pull these dependencies from github before
building QEMU.

fixes #1965

Signed-off-by: Julio Montes <julio.montes@intel.com>
(cherry picked from commit 9ec9bbbabc)
2021-06-08 10:37:42 +02:00
Fabiano Fidêncio
645e950b8e Merge pull request #1963 from fidencio/wip/stable-2.1-update-kubernetes-1.21.0-to-1.21.1
stable-2.1 | versions: Update kubernetes to 1.21.1
2021-06-08 10:02:21 +02:00
Shengjing Zhu
db8d853b99 runtime: remove covertool from cli test
covertool has no active since 2018 and is not compatible with go1.16

  ../vendor/github.com/dlespiau/covertool/pkg/cover/cover.go:76:29: cannot use f (type dummyTestDeps) as type testing.testDeps in argument to testing.MainStart:
  dummyTestDeps does not implement testing.testDeps (missing SetPanicOnExit0 method)

Fixes: #1862

Signed-off-by: Shengjing Zhu <zhsj@debian.org>
(cherry picked from commit 1b60705646)
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-06-08 10:00:51 +02:00
Gabriela Cervantes
3fad527734 docs: Fix Release Process document
This PR updates the correct url for github actions as well as it
corrects a misspelling.

Fixes #1960

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
(cherry picked from commit 9158ec68cc)
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-06-08 10:00:47 +02:00
Fabiano Fidêncio
175970c93a versions: Update kubernetes to 1.21.1
The reason for doing such is to (try to) avoid random crashes we've been
facing as part of our CI, such as the one reported as part of
https://github.com/kata-containers/tests/issues/3473

Fixes: #1850

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
(cherry picked from commit dbef2b2931)
2021-06-04 15:43:28 +02:00
Tim Zhang
6cea1b146c Merge pull request #1959 from Tim-Zhang/port-fix-fd-leak-for-stable-2.1
stable-2.1 | Port fd leak fixes
2021-06-03 20:15:23 +08:00
Tim Zhang
1cc2ad3f34 agent: Fix fd leak caused by netlink
See also: little-dude/netlink#165

Fixes: #1952

Because the author of netlink has no time to maintain the crate
(https://github.com/little-dude/netlink/issues/161), so we
need to switch the dependency to github temporarily.

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-06-03 17:24:09 +08:00
Tim Zhang
ac34f6dfd9 agent: Upgrade tokio-vsock to fix fd leak of vsock socket
Fixes: #1950

The further information: rust-vsock/vsock-rs#15

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-06-03 10:29:33 +08:00
Julio Montes
ff206cf6cf Merge pull request #1946 from fidencio/wip/weekly-backports-to-stable-2.1
[stable-2.1] Weekly backports to stable-2.1 branch, May 31st 2021
2021-06-01 16:00:42 -05:00
Bin Liu
57f7ffbe39 Merge pull request #1940 from liubin/backport/1934
[backport]runtime: and cgroup and SandboxCgroupOnly check for check sub-command
2021-06-01 08:32:45 +08:00
fupan.lfp
915fea7b1f cgroup: fix the issue of set mem.limit and mem.swap
When update memory limit, we should adapt the write sequence
for memory and swap memory, so it won't fail because
the new value and the old value don't fit kernel's
validation.

Fixes: #1917

Signed-off-by: fupan.lfp <fupan.lfp@antgroup.com>
(cherry picked from commit 30f4834c5b)
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-05-31 16:56:15 +02:00
fupan.lfp
a05e137710 agent: re-enable the standard SIGPIPE behavior
The Rust standard library had suppressed the default SIGPIPE
behavior, see https://github.com/rust-lang/rust/pull/13158.
Since the parent's signal handler would be inherited by it's child
process, thus we should re-enable the standard SIGPIPE behavior as a
workaround.

Fixes: #1887

Signed-off-by: fupan.lfp <fupan.lfp@antgroup.com>
(cherry picked from commit 0ae364c8eb)
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-05-31 16:56:15 +02:00
bin
8019f7322d virtiofsd: Fix file descriptors leak and return correct PID
This commit will fix two problems:
- Virtiofsd process ID returned to the caller will always be 0,
   the pid var is never being assigned a value.
- Socket listen fd may leak in case of failure of starting virtiofsd process.
  This is a port of be9ca0d58b

Fixes: #1931

Signed-off-by: bin <bin@hyper.sh>
(cherry picked from commit 773deca2f6)
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-05-31 16:56:15 +02:00
bin
e48c9d426d runtime: and cgroup and SandboxCgroupOnly check for check sub-command
In kata-runtime check sub-command, checks cgroups and SandboxCgroupOnly
to show message if the SandboxCgroupOnly is not set to true
and cgroup v2 is used.

Fixes: #1927

Signed-off-by: bin <bin@hyper.sh>
2021-05-28 16:36:49 +08:00
Peng Tao
6a7e6a8d0a Merge pull request #1921 from fidencio/wip/weekly-stable-2.1-backports-may-24th
Weekly stable 2.1 backports may 24th
2021-05-25 10:17:18 +08:00
quanweiZhou
7874ab33d4 agent: fix start container failed when dropping all capabilities
When starting a container and dropping all capabilities,
the init child process has no permission to read the exec.fifo
file because the parent set the file mode 0o622. So change the exec.fifo file mode to 0o644.

fixes #1913

Signed-off-by: quanweiZhou <quanweiZhou@linux.alibaba.com>
(cherry picked from commit 3e4ebe10ac)
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-05-24 15:08:15 +02:00
Yuanzhe Liu
536634e909 qemu: align before memory hotplug on arm64
When hotplug memory on arm64 in kata, kernel will shout:

[ 0.396551] Block size [0x40000000] unaligned hotplug range: start 0xc8000000, size 0x40000000
[ 0.396556] acpi PNP0C80:01: add_memory failed
[ 0.396834] acpi PNP0C80:01: acpi_memory_enable_device() error
[ 0.396948] acpi PNP0C80:01: Enumeration failure

It means that kernel will check if the memory range to be hotplugged
align with 1G before plug the memory. So we should twist the qemu to
make sure the memory range align with 1G to pass the kernel check.

Fixes: #1841

Signed-off-by: Yuanzhe Liu <yuanzheliu09@gmail.com>
(cherry picked from commit bc36b7b49f)
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-05-24 15:08:11 +02:00
Eric Ernst
c51891fee7 sandbox-bindmount: persist mount information
Without this, if the shim dies, we will not have a reliable way to
identify what mounts should be cleaned up if `containerd-shim-kata-v2
cleanup` is called for the sandbox.

Before this, if you `ctr run` with a sandbox bindmount defined and SIGKILL the
containerd-shim-kata-v2, you'll notice the sandbox bindmount left on
host.

With this change, the shim is able to get the sandbox bindmount
information from disk and do the appropriate cleanup.

Fixes #1896

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
(cherry picked from commit 7f1030d303)
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-05-24 15:08:08 +02:00
Eric Ernst
b137c7ac33 sandbox: Cleanup if failure to setup sandbox-bindmount occurs
If for any reason there's an error when trying to setup the sandbox
bindmounts, make sure we roll back any mounts already created when
setting up the sandbox.

Without this, we'd leave shared directory mount and potentially
sandbox-bindmounts on the host.

Fixes: #1895

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
(cherry picked from commit 089a7484e1)
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-05-24 15:08:04 +02:00
Fabiano Fidêncio
a407e53b94 Merge pull request #1911 from devimc/2021-05-21/stable-2.1/updateChannels
[backport-2.1] workflows: release kata 2.x snap through the stable channel
2021-05-22 09:22:10 +02:00
Julio Montes
68a77a7dec workflows: release kata 2.x snap through the stable channel
kata 1.x has been deprecated, now kata 2.x can be released through
the stable channel

fixes #1909

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-05-21 15:47:50 -05:00
Bin Liu
169cf133c9 Merge pull request #1873 from teawater/vm_doc2.1
[2.1] how-to-use-virtio-mem-with-kata.md: Update doc to make it clear
2021-05-20 17:36:03 +08:00
Hui Zhu
550269ff89 how-to-use-virtio-mem-with-kata.md: Update doc to make it clear
Update this howto because the virtio-mem support of kata, qemu and Linux
was updated.

Fixes: #1845

Signed-off-by: Hui Zhu <teawater@antfin.com>
2021-05-18 14:11:43 +08:00
Chelsea Mafrica
d6d16dc597 Merge pull request #1849 from GabyCT/topic/removeprportinglabel
github: Do not run require porting labels on stable-2.1
2021-05-17 13:19:25 -07:00
Gabriela Cervantes
1ea0dc9804 github: Do not run require porting labels on stable-2.1
When we are creating a PR in stable-2.1 we do not need to run
the github action of porting labels as we are doing backports or
new releases in stable-2.1 and we it is unnecessary to put labels
like no-backport-needed or no-forwardport-needed, etc.

Fixes #1847

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2021-05-14 16:37:44 -05:00
Fabiano Fidêncio
0f82291926 Merge pull request #1856 from fidencio/2.1.0-branch-bump
# Kata Containers 2.1.0
2021-05-14 19:59:06 +02:00
Fabiano Fidêncio
5d3610e25f release: Kata Containers 2.1.0
- stable-2.1 | The last round of backports before releasing 2.1.0
- back port: image_build: align image size to 128M for arm64
- stable-2.1 | runtime: make dialing timeout configurable
- stable-2.1 | agent: avoid reaping the exit signal of execute_hook in the reaper
- stable-2.1 | Get sandbox metrics cli
- packaging/kata-cleanup: add k3s containerd volume
- stable-2.1: First round of backports
- [backport]runtime: use s.ctx instead ctx for checking cancellation
- [2.1.0] kernel: configs: Open CONFIG_VIRTIO_MEM in x86_64 Linux kernel
- [2.1.0] Fix issue of virtio-mem

9266c246 rustjail: separated the propagation flags from mount flags
7086f91e runtime: sandbox delete should succeed after verifying sandbox state
0a7befa6 docs: Fix spell-check errors found after new text is discovered
eff70d2e docs: Remove horizontal ruler markers that disable spell checks
260f59df image_build: align image size to 128M for arm64
c0bdba23 runtime: make dialing timeout configurable
1b3cf2fb kata-monitor: export get stats for sandbox
59b9e5d0 kata-runtime: add `metrics` command
828a3048 agent: avoid reaping the exit signal of execute_hook in the reaper
d3690952 runtime: shim: dedup client, socket addr code
7f7c794d runtime: Short the shim-monitor path
3f1b7c91 cli: delete tracing code for kata-runtime binary
68cad377 agent: Set fixed NOFILE limit value for kata-agent
7c9067cc docs: add per-Pod Kata configurations for enable_pprof
dba86ef3 ci/install_yq.sh: install_yq: Check version before return
3883e4e2 kernel: configs: Open CONFIG_VIRTIO_MEM in x86_64 Linux kernel
79831faf runtime: use s.ctx instead ctx for checking cancellation
3212c7ae packaging/kata-cleanup: add k3s containerd volume
7f7c3fc8 qemu.go: qemu: resizeMemory: Fix virtio-mem resize overflow issue
c9053ea3 qemu.go: qemu: setupVirtioMem: let sizeMB be multiple of 2Mib

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-05-14 16:05:01 +02:00
Fabiano Fidêncio
ed01ac3e0c Merge pull request #1853 from fidencio/wip/last-round-of-backports-for-2.1.0
stable-2.1 | The last round of backports before releasing 2.1.0
2021-05-14 14:35:25 +02:00
fupan.lfp
9266c2460a rustjail: separated the propagation flags from mount flags
Since the propagation flags couldn't be combinted with the
standard mount flags, and they should be used with the remount,
thus it's better to split them from the standard mount flags.

Fixes: #1699

Signed-off-by: fupan.lfp <fupan.lfp@antgroup.com>
(cherry picked from commit e5fe572f51)
2021-05-14 09:42:00 +02:00
Peng Tao
7086f91e1f runtime: sandbox delete should succeed after verifying sandbox state
Otherwise we might block delete and create orphan containers.

Fixes: #1039

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
(cherry picked from commit 35151f1786)
2021-05-14 09:41:38 +02:00
Christophe de Dinechin
0a7befa645 docs: Fix spell-check errors found after new text is discovered
The spell-checker scripts has some bugs that caused large chunks of texts to not
be spell checked at all (see #1793). The previous commit worked around this bug,
which exposed another bug:

The following source text:

    are discussions about using VM save and restore to
    give [`criu`](https://github.com/checkpoint-restore/criu)-like
    functionality, which might provide a solution

yields the surprising error below:

    WARNING: Word 'givelike': did you mean one of the following?: give like, give-like, wavelike

Apparently, an extra space is removed, which is another issue with the
spell-checking script. This case is somewhat contrived because of the URL link,
so for now, I decided for a creative rewriting, inserting the word "a" knowing
that "alike" is a valid word ;-)

Fixes: #1793

Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
(cherry picked from commit 5fdf617e7f)
2021-05-14 09:41:38 +02:00
Christophe de Dinechin
eff70d2eea docs: Remove horizontal ruler markers that disable spell checks
There is a bug in the CI script checking spelling that causes it
to skip any text that follows a horizontal ruler.
(https://github.com/kata-containers/tests/issues/3448)

Solution: replace one horizontal ruler marker with another that
does not trip the spell-checking script.

Fixes: #1793

Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
(cherry picked from commit 42425456e7)
2021-05-14 09:38:50 +02:00
Fabiano Fidêncio
dd26aa5838 Merge pull request #1840 from jongwu/stable-2.1_image_align
back port: image_build: align image size to 128M for arm64
2021-05-13 10:37:50 +02:00
Jianyong Wu
260f59df38 image_build: align image size to 128M for arm64
There is an inconformity between qemu and kernel of memory alignment
check in memory hotplug. Both of qemu and kernel will do the start
address alignment check in memory hotplug. But it's 2M in qemu
while 128M in kernel. It leads to an issue when memory hotplug.

Currently, the kata image is a nvdimm device, which will plug into the VM as
a dimm. If another dimm is pluged, it will reside on top of that nvdimm.
So, the start address of the second dimm may not pass the alginment
check in kernel if the nvdimm size doesn't align with 128M.

There are 3 ways to address this issue I think:
1. fix the alignment size in kernel according to qemu. I think people
in linux kernel community will not accept it.
2. do alignment check in qemu and force the start address of hotplug
in alignment with 128M, which means there maybe holes between memory blocks.
3. obey the rule in user end, which means fix it in kata.

I think the second one is the best, but I can't do that for some reason.
Thus, the last one is the choice here.

Fixes: #1769
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-05-13 10:09:25 +08:00
Chelsea Mafrica
9a32a3e16d Merge pull request #1835 from snir911/backport_configure_timeout
stable-2.1 | runtime: make dialing timeout configurable
2021-05-12 13:14:37 -07:00
Fabiano Fidêncio
123f7d53cb Merge pull request #1830 from Tim-Zhang/fix-reap-for-stable-2.1
stable-2.1 | agent: avoid reaping the exit signal of execute_hook in the reaper
2021-05-12 20:26:42 +02:00
Fabiano Fidêncio
aa213fdc28 Merge pull request #1833 from fidencio/wip/stable-2.1-backport-of-1816
stable-2.1 | Get sandbox metrics cli
2021-05-12 19:34:20 +02:00
Snir Sheriber
c0bdba2350 runtime: make dialing timeout configurable
allow to set dialing timeout in configuration.toml
default is 30s

Fixes: #1789
(cherry-picked 01b56d6cbf)
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2021-05-12 14:17:34 +03:00
Eric Ernst
1b3cf2fb7d kata-monitor: export get stats for sandbox
Gathering stats for a given sandbox is pretty useful; let's export a
function from katamonitor pkg to do this.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
(cherry picked from commit 3787306107)
2021-05-12 11:44:58 +02:00
Eric Ernst
59b9e5d0f8 kata-runtime: add metrics command
For easier debug, let's add subcommand to kata-runtime for gathering
metrics associated with a given sandbox.

kata-runtime metrics --sandbox-id foobar

Fixes: #1815

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
(cherry picked from commit 8068a4692f)
2021-05-12 11:44:53 +02:00
Tim Zhang
828a304883 agent: avoid reaping the exit signal of execute_hook in the reaper
Fixes: #1826

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-05-12 16:33:44 +08:00
Fabiano Fidêncio
70734dfa17 Merge pull request #1803 from nubificus/stable-2.1
packaging/kata-cleanup: add k3s containerd volume
2021-05-11 19:38:57 +02:00
Fabiano Fidêncio
f170df6201 Merge pull request #1821 from fidencio/wip/first-round-of-backports
stable-2.1: First round of backports
2021-05-11 08:52:18 +02:00
Eric Ernst
d3690952e6 runtime: shim: dedup client, socket addr code
(1) Add an accessor function, SocketAddress, to the shim-v2 code for
determining the shim's abstract domain socket address, given the sandbox
ID.

(2) In kata monitor, create a function, BuildShimClient, for obtaining the appropriate
http.Client for communicating with the shim's monitoring endpoint.

(3) Update the kata CLI and kata-monitor code to make use of these.

(4) Migrate some kata monitor methods to be functions, in order to ease
future reuse.

(5) drop unused namespace from functions where it is no longer needed.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
(cherry picked from commit 3caed6f88d)
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-05-10 15:35:53 +02:00
Fabiano Fidêncio
7f7c794da4 runtime: Short the shim-monitor path
Instead of having something like
"/containerd-shim/$namespace/$sandboxID/shim-monitor.sock", let's change
the approach to:
* create the file in a more neutral location "/run/vc", instead of
  "/containerd-shim";
* drop the namespace, as the sandboxID should be unique;
* remove ".sock" from the socket name.

This will result on a name that looks like:
"/run/vc/$sandboxID/shim-monitor"

Fixes: #497

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
(cherry picked from commit 4bc006c8a4)
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-05-10 15:35:47 +02:00
bin
3f1b7c9127 cli: delete tracing code for kata-runtime binary
There are no pod/container operations in kata-runtime binary,
tracing in this package is meaningless.

Fixes: #1748

Signed-off-by: bin <bin@hyper.sh>
(cherry picked from commit 13c23fec11)
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-05-10 15:35:36 +02:00
Snir Sheriber
68cad37720 agent: Set fixed NOFILE limit value for kata-agent
Some applications may fail if NOFILE limit is set to unlimited.
Although in some environments this value is explicitly overridden,
lets set it to a more sane value in case it doesn't.

Fixes #1715
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
(cherry picked from commit a188577ebf)
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-05-10 15:34:28 +02:00
bin
7c9067cc9d docs: add per-Pod Kata configurations for enable_pprof
Now enabling enable_pprof for individual pods is supported,
but not documented.

This commit will add per-Pod Kata configurations for `enable_pprof`
in file `docs/how-to/how-to-set-sandbox-config-kata.md`

Fixes: #1744

Signed-off-by: bin <bin@hyper.sh>
(cherry picked from commit 95e54e3f48)
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-05-10 15:34:05 +02:00
Hui Zhu
dba86ef31a ci/install_yq.sh: install_yq: Check version before return
Check the yq version before return.

Fixes: #1776

Signed-off-by: Hui Zhu <teawater@antfin.com>
(cherry picked from commit d8896157df)
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-05-10 15:33:33 +02:00
Tim Zhang
0e2df80bda Merge pull request #1814 from liubin/fix/1804-select-sandbox-ctx
[backport]runtime: use s.ctx instead ctx for checking cancellation
2021-05-07 19:43:14 +08:00
Bin Liu
8c4e187049 Merge pull request #1813 from teawater/open_vm
[2.1.0] kernel: configs: Open CONFIG_VIRTIO_MEM in x86_64 Linux kernel
2021-05-07 19:31:23 +08:00
Fabiano Fidêncio
3bcdc26008 Merge pull request #1812 from teawater/fix_vm
[2.1.0] Fix issue of virtio-mem
2021-05-07 08:14:19 +02:00
bin
79831fafaf runtime: use s.ctx instead ctx for checking cancellation
s.ctx should be used for checking cancellation, and the
local ctx is used for tracing.

Fixes: #1804

Signed-off-by: bin <bin@hyper.sh>
2021-05-06 17:22:53 +08:00
Orestis Lagkas Nikolos
3212c7ae00 packaging/kata-cleanup: add k3s containerd volume
kata-deploy cleanup expects to find containerd configuration
in /etc/containerd/config.toml. In case of k3s mount the k3s
containerd config as a volume.

Original PR #1802

Fixes #1801

Signed-off-by: Orestis Lagkas Nikolos <olagkasn@nubificus.co.uk>
2021-05-06 03:36:38 -05:00
Hui Zhu
7f7c3fc8ec qemu.go: qemu: resizeMemory: Fix virtio-mem resize overflow issue
This commit change sizeByte from uint32 to uint64 to fix overflow issue.

Fixes: #1796

Signed-off-by: Hui Zhu <teawater@antfin.com>
2021-05-06 14:13:50 +08:00
Hui Zhu
c9053ea3fb qemu.go: qemu: setupVirtioMem: let sizeMB be multiple of 2Mib
Got:
FATA[0000] run pod sandbox: rpc error: code = Unknown desc = failed to
create containerd task: Add 189759MB virtio-mem-pci fail QMP command
failed: backend memory size must be multiple of 0x200000: unknown

This commit let sizeMB be multiple of 2Mib to fix the issue.

Fixes: #1796

Signed-off-by: Hui Zhu <teawater@antfin.com>
2021-05-06 14:13:48 +08:00
56 changed files with 561 additions and 676 deletions

View File

@@ -6,6 +6,9 @@
name: Ensure PR has required porting labels
on:
pull_request:
branches:
- main
pull_request_target:
types:
- opened

View File

@@ -33,5 +33,5 @@ jobs:
snap_file="kata-containers_${snap_version}_amd64.snap"
# Upload the snap if it exists
if [ -f ${snap_file} ]; then
snapcraft upload --release=candidate ${snap_file}
snapcraft upload --release=stable ${snap_file}
fi

View File

@@ -1 +1 @@
2.1.0-rc0
2.1.1

View File

@@ -18,7 +18,9 @@ function install_yq() {
GOPATH=${GOPATH:-${HOME}/go}
local yq_path="${GOPATH}/bin/yq"
local yq_pkg="github.com/mikefarah/yq"
[ -x "${GOPATH}/bin/yq" ] && return
local yq_version=3.4.1
[ -x "${GOPATH}/bin/yq" ] && [ "`${GOPATH}/bin/yq --version`"X == "yq version ${yq_version}"X ] && return
read -r -a sysInfo <<< "$(uname -sm)"
@@ -56,8 +58,6 @@ function install_yq() {
die "Please install curl"
fi
local yq_version=3.4.1
## NOTE: ${var,,} => gives lowercase value of var
local yq_url="https://${yq_pkg}/releases/download/${yq_version}/yq_${goos,,}_${goarch}"
curl -o "${yq_path}" -LSsf "${yq_url}"

View File

@@ -28,7 +28,7 @@
* [Appendices](#appendices)
* [The constraints challenge](#the-constraints-challenge)
---
***
# Overview
@@ -94,7 +94,9 @@ This section lists items that might be possible to fix.
### checkpoint and restore
The runtime does not provide `checkpoint` and `restore` commands. There
are discussions about using VM save and restore to give [`criu`](https://github.com/checkpoint-restore/criu)-like functionality, which might provide a solution.
are discussions about using VM save and restore to give us a
`[criu](https://github.com/checkpoint-restore/criu)`-like functionality,
which might provide a solution.
Note that the OCI standard does not specify `checkpoint` and `restore`
commands.

View File

@@ -60,7 +60,7 @@
### Check Git-hub Actions
We make use of [GitHub actions](https://github.com/features/actions) in this [file](https://github.com/kata-containers/kata-containers/blob/master/.github/workflows/main.yaml) in the `kata-containers/kata-containers` repository to build and upload release artifacts. This action is auto triggered with the above step when a new tag is pushed to the `kata-containers/kata-conatiners` repository.
We make use of [GitHub actions](https://github.com/features/actions) in this [file](https://github.com/kata-containers/kata-containers/blob/main/.github/workflows/main.yaml) in the `kata-containers/kata-containers` repository to build and upload release artifacts. This action is auto triggered with the above step when a new tag is pushed to the `kata-containers/kata-containers` repository.
Check the [actions status page](https://github.com/kata-containers/kata-containers/actions) to verify all steps in the actions workflow have completed successfully. On success, a static tarball containing Kata release artifacts will be uploaded to the [Release page](https://github.com/kata-containers/kata-containers/releases).

View File

@@ -26,6 +26,7 @@ There are several kinds of Kata configurations and they are listed below.
| `io.katacontainers.config.runtime.disable_new_netns` | `boolean` | determines if a new netns is created for the hypervisor process |
| `io.katacontainers.config.runtime.internetworking_model` | string| determines how the VM should be connected to the container network interface. Valid values are `macvtap`, `tcfilter` and `none` |
| `io.katacontainers.config.runtime.sandbox_cgroup_only`| `boolean` | determines if Kata processes are managed only in sandbox cgroup |
| `io.katacontainers.config.runtime.enable_pprof` | `boolean` | enables Golang `pprof` for `containerd-shim-kata-v2` process |
## Agent Options
| Key | Value Type | Comments |

View File

@@ -13,26 +13,23 @@ Kata Containers with `virtio-mem` supports memory resize.
## Requisites
Kata Containers with `virtio-mem` requires Linux and the QEMU that support `virtio-mem`.
The Linux kernel and QEMU upstream version still not support `virtio-mem`. @davidhildenbrand is working on them.
Please use following unofficial version of the Linux kernel and QEMU that support `virtio-mem` with Kata Containers.
Kata Containers just supports `virtio-mem` with QEMU.
Install and setup Kata Containers as shown [here](../install/README.md).
The Linux kernel is at https://github.com/davidhildenbrand/linux/tree/virtio-mem-rfc-v4.
The Linux kernel config that can work with Kata Containers is at https://gist.github.com/teawater/016194ee84748c768745a163d08b0fb9.
The QEMU is at https://github.com/teawater/qemu/tree/kata-virtio-mem. (The original source is at https://github.com/davidhildenbrand/qemu/tree/virtio-mem. Its base version of QEMU cannot work with Kata Containers. So merge the commit of `virtio-mem` to upstream QEMU.)
Set Linux and the QEMU that support `virtio-mem` with following line in the Kata Containers QEMU configuration `configuration-qemu.toml`:
```toml
[hypervisor.qemu]
path = "qemu-dir"
kernel = "vmlinux-dir"
### With x86_64
The `virtio-mem` config of the x86_64 Kata Linux kernel is open.
Enable `virtio-mem` as follows:
```
$ sudo sed -i -e 's/^#enable_virtio_mem.*$/enable_virtio_mem = true/g' /etc/kata-containers/configuration.toml
```
Enable `virtio-mem` with following line in the Kata Containers configuration:
```toml
enable_virtio_mem = true
### With other architectures
The `virtio-mem` config of the others Kata Linux kernel is not open.
You can open `virtio-mem` config as follows:
```
CONFIG_VIRTIO_MEM=y
```
Then you can build and install the guest kernel image as shown [here](../../tools/packaging/kernel/README.md#build-kata-containers-kernel).
## Run a Kata Container utilizing `virtio-mem`
@@ -41,13 +38,35 @@ Use following command to enable memory overcommitment of a Linux kernel. Becaus
$ echo 1 | sudo tee /proc/sys/vm/overcommit_memory
```
Use following command start a Kata Container.
Use following command to start a Kata Container.
```
$ docker run --rm -it --runtime=kata --name test busybox
$ pod_yaml=pod.yaml
$ container_yaml=${REPORT_DIR}/container.yaml
$ image="quay.io/prometheus/busybox:latest"
$ cat << EOF > "${pod_yaml}"
metadata:
name: busybox-sandbox1
EOF
$ cat << EOF > "${container_yaml}"
metadata:
name: busybox-killed-vmm
image:
image: "$image"
command:
- top
EOF
$ sudo crictl pull $image
$ podid=$(sudo crictl runp $pod_yaml)
$ cid=$(sudo crictl create $podid $container_yaml $pod_yaml)
$ sudo crictl start $cid
```
Use following command set the memory size of test to default_memory + 512m.
Use the following command to set the container memory limit to 2g and the memory size of the VM to its default_memory + 2g.
```
$ docker update -m 512m --memory-swap -1 test
$ sudo crictl update --memory $((2*1024*1024*1024)) $cid
```
Use the following command to set the container memory limit to 1g and the memory size of the VM to its default_memory + 1g.
```
$ sudo crictl update --memory $((1*1024*1024*1024)) $cid
```

View File

@@ -14,7 +14,7 @@ Kata Containers can be installed in any Linux distribution that supports
Run the following command to install **Kata Containers**:
```sh
$ sudo snap install kata-containers --candidate --classic
$ sudo snap install kata-containers --stable --classic
```
## Configure Kata Containers

110
src/agent/Cargo.lock generated
View File

@@ -104,12 +104,6 @@ dependencies = [
"iovec",
]
[[package]]
name = "bytes"
version = "0.5.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0e4cec68f03f32e44924783795810fa50a7035d8c8ebe78580ad7e6c703fba38"
[[package]]
name = "bytes"
version = "1.0.1"
@@ -347,7 +341,7 @@ dependencies = [
"futures-sink",
"futures-task",
"memchr",
"pin-project-lite 0.2.4",
"pin-project-lite",
"pin-utils",
"proc-macro-hack",
"proc-macro-nested",
@@ -411,7 +405,7 @@ dependencies = [
"futures-core",
"inotify-sys",
"libc",
"tokio 1.2.0",
"tokio",
]
[[package]]
@@ -478,8 +472,8 @@ dependencies = [
"libc",
"log",
"logging",
"netlink-packet-utils",
"netlink-sys",
"netlink-packet-utils 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)",
"netlink-sys 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)",
"nix 0.17.0",
"oci",
"prctl",
@@ -497,7 +491,7 @@ dependencies = [
"slog-scope",
"slog-stdlog",
"tempfile",
"tokio 1.2.0",
"tokio",
"tokio-vsock",
"ttrpc",
]
@@ -614,28 +608,26 @@ checksum = "2eb04b9f127583ed176e163fb9ec6f3e793b87e21deedd5734a69386a18a0151"
[[package]]
name = "netlink-packet-core"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "559c08b3ea479ace788a8f75388e3db15210561a4c0f37c4ec7bdf40a3be564e"
version = "0.2.4"
source = "git+https://github.com/little-dude/netlink?rev=a9367bc4700496ddebc088110c28f40962923326#a9367bc4700496ddebc088110c28f40962923326"
dependencies = [
"anyhow",
"byteorder",
"libc",
"netlink-packet-utils",
"netlink-packet-utils 0.4.0 (git+https://github.com/little-dude/netlink?rev=a9367bc4700496ddebc088110c28f40962923326)",
]
[[package]]
name = "netlink-packet-route"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da1bf86b4324996fb58f8e17752b2a06176f4c5efc013928060ac94a3a329b71"
source = "git+https://github.com/little-dude/netlink?rev=a9367bc4700496ddebc088110c28f40962923326#a9367bc4700496ddebc088110c28f40962923326"
dependencies = [
"anyhow",
"bitflags",
"byteorder",
"libc",
"netlink-packet-core",
"netlink-packet-utils",
"netlink-packet-utils 0.4.0 (git+https://github.com/little-dude/netlink?rev=a9367bc4700496ddebc088110c28f40962923326)",
]
[[package]]
@@ -650,18 +642,28 @@ dependencies = [
"thiserror",
]
[[package]]
name = "netlink-packet-utils"
version = "0.4.0"
source = "git+https://github.com/little-dude/netlink?rev=a9367bc4700496ddebc088110c28f40962923326#a9367bc4700496ddebc088110c28f40962923326"
dependencies = [
"anyhow",
"byteorder",
"paste",
"thiserror",
]
[[package]]
name = "netlink-proto"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b72e0d34a11ff4253bc7d38fbc246750c4d2d2718b318c9ad981d450ed62aa8"
source = "git+https://github.com/little-dude/netlink?rev=a9367bc4700496ddebc088110c28f40962923326#a9367bc4700496ddebc088110c28f40962923326"
dependencies = [
"bytes 0.5.6",
"bytes 1.0.1",
"futures",
"log",
"netlink-packet-core",
"netlink-sys",
"tokio 1.2.0",
"netlink-sys 0.6.0 (git+https://github.com/little-dude/netlink?rev=a9367bc4700496ddebc088110c28f40962923326)",
"tokio",
"tokio-util",
]
@@ -674,7 +676,18 @@ dependencies = [
"futures",
"libc",
"log",
"tokio 1.2.0",
"tokio",
]
[[package]]
name = "netlink-sys"
version = "0.6.0"
source = "git+https://github.com/little-dude/netlink?rev=a9367bc4700496ddebc088110c28f40962923326#a9367bc4700496ddebc088110c28f40962923326"
dependencies = [
"futures",
"libc",
"log",
"tokio",
]
[[package]]
@@ -846,12 +859,6 @@ dependencies = [
"fixedbitset",
]
[[package]]
name = "pin-project-lite"
version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c917123afa01924fc84bb20c4c03f004d9c38e5127e3c039bbf7f4b9c76a2f6b"
[[package]]
name = "pin-project-lite"
version = "0.2.4"
@@ -877,7 +884,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "059a34f111a9dee2ce1ac2826a68b24601c4298cfeb1a587c3cb493d5ab46f52"
dependencies = [
"libc",
"nix 0.19.1",
"nix 0.20.0",
]
[[package]]
@@ -1138,8 +1145,7 @@ dependencies = [
[[package]]
name = "rtnetlink"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d7c188dd8ddd43e195e483d255f4975d457a78021acf185ea2269e0667ef8b1a"
source = "git+https://github.com/little-dude/netlink?rev=a9367bc4700496ddebc088110c28f40962923326#a9367bc4700496ddebc088110c28f40962923326"
dependencies = [
"byteordered",
"futures",
@@ -1148,7 +1154,7 @@ dependencies = [
"netlink-proto",
"nix 0.19.1",
"thiserror",
"tokio 1.2.0",
"tokio",
]
[[package]]
@@ -1186,7 +1192,7 @@ dependencies = [
"slog",
"slog-scope",
"tempfile",
"tokio 1.2.0",
"tokio",
]
[[package]]
@@ -1438,16 +1444,6 @@ dependencies = [
"winapi",
]
[[package]]
name = "tokio"
version = "0.2.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "099837d3464c16a808060bb3f02263b412f6fafcb5d01c533d309985fbeebe48"
dependencies = [
"bytes 0.5.6",
"pin-project-lite 0.1.11",
]
[[package]]
name = "tokio"
version = "1.2.0"
@@ -1461,7 +1457,7 @@ dependencies = [
"mio",
"num_cpus",
"once_cell",
"pin-project-lite 0.2.4",
"pin-project-lite",
"signal-hook-registry",
"tokio-macros",
"winapi",
@@ -1480,28 +1476,28 @@ dependencies = [
[[package]]
name = "tokio-util"
version = "0.2.0"
version = "0.6.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "571da51182ec208780505a32528fc5512a8fe1443ab960b3f2f3ef093cd16930"
checksum = "1caa0b0c8d94a049db56b5acf8cba99dc0623aab1b26d5b5f5e2d945846b3592"
dependencies = [
"bytes 0.5.6",
"bytes 1.0.1",
"futures-core",
"futures-sink",
"log",
"pin-project-lite 0.1.11",
"tokio 0.2.24",
"pin-project-lite",
"tokio",
]
[[package]]
name = "tokio-vsock"
version = "0.3.0"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "70c401a3d0e82328c6fa900e9e2c4152737a4caa304558b762f0154a8b306456"
checksum = "9e0723fc001950a3b018947b05eeb45014fd2b7c6e8f292502193ab74486bdb6"
dependencies = [
"bytes 0.4.12",
"futures",
"libc",
"tokio 1.2.0",
"tokio",
"vsock",
]
@@ -1520,7 +1516,7 @@ dependencies = [
"protobuf",
"protobuf-codegen-pure",
"thiserror",
"tokio 1.2.0",
"tokio",
"tokio-vsock",
]
@@ -1577,9 +1573,9 @@ checksum = "6a02e4885ed3bc0f2de90ea6dd45ebcbb66dacffe03547fadbb0eeae2770887d"
[[package]]
name = "vsock"
version = "0.2.3"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "50e2ef09834e1d203d24556512c0e58e66de203440bd9d74c30a33f7240091c6"
checksum = "c932be691560e8f3f7b2be5a47df1b8f45387e1d1df40d45b2e62284b9e9150e"
dependencies = [
"libc",
"nix 0.19.1",

View File

@@ -24,8 +24,11 @@ async-trait = "0.1.42"
tokio = { version = "1.2.0", features = ["rt", "rt-multi-thread", "sync", "macros", "io-util", "time", "signal", "io-std", "process", "fs"] }
futures = "0.3.12"
netlink-sys = { version = "0.6.0", features = ["tokio_socket",]}
tokio-vsock = "0.3.0"
rtnetlink = "0.7.0"
tokio-vsock = "0.3.1"
# Because the author has no time to maintain the crate, we switch the dependency to github,
# Once the new version released on crates.io, we switch it back.
# https://github.com/little-dude/netlink/issues/161
rtnetlink = { git = "https://github.com/little-dude/netlink", rev = "a9367bc4700496ddebc088110c28f40962923326" }
netlink-packet-utils = "0.4.0"
ipnetwork = "0.17.0"

View File

@@ -15,7 +15,7 @@ Wants=kata-containers.target
StandardOutput=tty
Type=simple
ExecStart=@BINDIR@/@AGENT_NAME@
LimitNOFILE=infinity
LimitNOFILE=1048576
# ExecStop is required for static agent tracing; in all other scenarios
# the runtime handles shutting down the VM.
ExecStop=/bin/sync ; /usr/bin/systemctl --force poweroff

View File

@@ -349,14 +349,34 @@ fn set_memory_resources(cg: &cgroups::Cgroup, memory: &LinuxMemory, update: bool
mem_controller.set_kmem_limit(-1)?;
}
set_resource!(mem_controller, set_limit, memory, limit);
set_resource!(mem_controller, set_soft_limit, memory, reservation);
set_resource!(mem_controller, set_kmem_limit, memory, kernel);
set_resource!(mem_controller, set_tcp_limit, memory, kernel_tcp);
// If the memory update is set to -1 we should also
// set swap to -1, it means unlimited memory.
let mut swap = memory.swap.unwrap_or(0);
if memory.limit == Some(-1) {
swap = -1;
}
if let Some(swap) = memory.swap {
// set memory swap
let swap = if cg.v2() {
if memory.limit.is_some() && swap != 0 {
let memstat = get_memory_stats(cg)
.into_option()
.ok_or_else(|| anyhow!("failed to get the cgroup memory stats"))?;
let memusage = memstat.get_usage();
// When update memory limit, the kernel would check the current memory limit
// set against the new swap setting, if the current memory limit is large than
// the new swap, then set limit first, otherwise the kernel would complain and
// refused to set; on the other hand, if the current memory limit is smaller than
// the new swap, then we should set the swap first and then set the memor limit.
if swap == -1 || memusage.get_limit() < swap as u64 {
mem_controller.set_memswap_limit(swap)?;
set_resource!(mem_controller, set_limit, memory, limit);
} else {
set_resource!(mem_controller, set_limit, memory, limit);
mem_controller.set_memswap_limit(swap)?;
}
} else {
set_resource!(mem_controller, set_limit, memory, limit);
swap = if cg.v2() {
convert_memory_swap_to_v2_value(swap, memory.limit.unwrap_or(0))?
} else {
swap
@@ -366,6 +386,10 @@ fn set_memory_resources(cg: &cgroups::Cgroup, memory: &LinuxMemory, update: bool
}
}
set_resource!(mem_controller, set_soft_limit, memory, reservation);
set_resource!(mem_controller, set_kmem_limit, memory, kernel);
set_resource!(mem_controller, set_tcp_limit, memory, kernel_tcp);
if let Some(swappiness) = memory.swappiness {
if (0..=100).contains(&swappiness) {
mem_controller.set_swappiness(swappiness as u64)?;

View File

@@ -48,6 +48,7 @@ use oci::State as OCIState;
use std::collections::HashMap;
use std::os::unix::io::FromRawFd;
use std::str::FromStr;
use std::sync::Arc;
use slog::{info, o, Logger};
@@ -57,6 +58,7 @@ use crate::sync_with_async::{read_async, write_async};
use async_trait::async_trait;
use rlimit::{setrlimit, Resource, Rlim};
use tokio::io::AsyncBufReadExt;
use tokio::sync::Mutex;
use crate::utils;
@@ -106,6 +108,9 @@ pub type Config = CreateOpts;
type NamespaceType = String;
lazy_static! {
// This locker ensures the child exit signal will be received by the right receiver.
pub static ref WAIT_PID_LOCKER: Arc<Mutex<bool>> = Arc::new(Mutex::new(false));
static ref NAMESPACES: HashMap<&'static str, CloneFlags> = {
let mut m = HashMap::new();
m.insert("user", CloneFlags::CLONE_NEWUSER);
@@ -817,7 +822,7 @@ impl BaseContainer for LinuxContainer {
if stat::stat(fifo_file.as_str()).is_ok() {
return Err(anyhow!("exec fifo exists"));
}
unistd::mkfifo(fifo_file.as_str(), Mode::from_bits(0o622).unwrap())?;
unistd::mkfifo(fifo_file.as_str(), Mode::from_bits(0o644).unwrap())?;
fifofd = fcntl::open(
fifo_file.as_str(),
@@ -1467,6 +1472,8 @@ async fn execute_hook(logger: &Logger, h: &Hook, st: &OCIState) -> Result<()> {
})
.collect();
// Avoid the exit signal to be reaped by the global reaper.
let _wait_locker = WAIT_PID_LOCKER.lock().await;
let mut child = tokio::process::Command::new(path)
.args(args.iter())
.envs(env.iter())

View File

@@ -68,6 +68,8 @@ lazy_static! {
m.insert("rprivate", MsFlags::MS_PRIVATE | MsFlags::MS_REC);
m.insert("slave", MsFlags::MS_SLAVE);
m.insert("rslave", MsFlags::MS_SLAVE | MsFlags::MS_REC);
m.insert("unbindable", MsFlags::MS_UNBINDABLE);
m.insert("runbindable", MsFlags::MS_UNBINDABLE | MsFlags::MS_REC);
m
};
static ref OPTIONS: HashMap<&'static str, (bool, MsFlags)> = {
@@ -93,17 +95,6 @@ lazy_static! {
m.insert("nodiratime", (false, MsFlags::MS_NODIRATIME));
m.insert("bind", (false, MsFlags::MS_BIND));
m.insert("rbind", (false, MsFlags::MS_BIND | MsFlags::MS_REC));
m.insert("unbindable", (false, MsFlags::MS_UNBINDABLE));
m.insert(
"runbindable",
(false, MsFlags::MS_UNBINDABLE | MsFlags::MS_REC),
);
m.insert("private", (false, MsFlags::MS_PRIVATE));
m.insert("rprivate", (false, MsFlags::MS_PRIVATE | MsFlags::MS_REC));
m.insert("shared", (false, MsFlags::MS_SHARED));
m.insert("rshared", (false, MsFlags::MS_SHARED | MsFlags::MS_REC));
m.insert("slave", (false, MsFlags::MS_SLAVE));
m.insert("rslave", (false, MsFlags::MS_SLAVE | MsFlags::MS_REC));
m.insert("relatime", (false, MsFlags::MS_RELATIME));
m.insert("norelatime", (true, MsFlags::MS_RELATIME));
m.insert("strictatime", (false, MsFlags::MS_STRICTATIME));
@@ -192,7 +183,7 @@ pub fn init_rootfs(
let mut bind_mount_dev = false;
for m in &spec.mounts {
let (mut flags, data) = parse_mount(&m);
let (mut flags, pgflags, data) = parse_mount(&m);
if !m.destination.starts_with('/') || m.destination.contains("..") {
return Err(anyhow!(
"the mount destination {} is invalid",
@@ -234,13 +225,15 @@ pub fn init_rootfs(
// effective.
// first check that we have non-default options required before attempting a
// remount
if m.r#type == "bind" {
for o in &m.options {
if let Some(fl) = PROPAGATION.get(o.as_str()) {
let dest = secure_join(rootfs, &m.destination);
mount(None::<&str>, dest.as_str(), None::<&str>, *fl, None::<&str>)?;
}
}
if m.r#type == "bind" && !pgflags.is_empty() {
let dest = secure_join(rootfs, &m.destination);
mount(
None::<&str>,
dest.as_str(),
None::<&str>,
pgflags,
None::<&str>,
)?;
}
}
}
@@ -657,26 +650,27 @@ pub fn ms_move_root(rootfs: &str) -> Result<bool> {
Ok(true)
}
fn parse_mount(m: &Mount) -> (MsFlags, String) {
fn parse_mount(m: &Mount) -> (MsFlags, MsFlags, String) {
let mut flags = MsFlags::empty();
let mut pgflags = MsFlags::empty();
let mut data = Vec::new();
for o in &m.options {
match OPTIONS.get(o.as_str()) {
Some(v) => {
let (clear, fl) = *v;
if clear {
flags &= !fl;
} else {
flags |= fl;
}
if let Some(v) = OPTIONS.get(o.as_str()) {
let (clear, fl) = *v;
if clear {
flags &= !fl;
} else {
flags |= fl;
}
None => data.push(o.clone()),
} else if let Some(fl) = PROPAGATION.get(o.as_str()) {
pgflags |= *fl;
} else {
data.push(o.clone());
}
}
(flags, data.join(","))
(flags, pgflags, data.join(","))
}
// This function constructs a canonicalized path by combining the `rootfs` and `unsafe_path` elements.
@@ -922,7 +916,7 @@ pub fn finish_rootfs(cfd_log: RawFd, spec: &Spec) -> Result<()> {
for m in spec.mounts.iter() {
if m.destination == "/dev" {
let (flags, _) = parse_mount(m);
let (flags, _, _) = parse_mount(m);
if flags.contains(MsFlags::MS_RDONLY) {
mount(
Some("/dev"),

View File

@@ -248,6 +248,7 @@ fn main() -> std::result::Result<(), Box<dyn std::error::Error>> {
}
if args.len() == 2 && args[1] == "init" {
reset_sigpipe();
rustjail::container::init_child();
exit(0);
}
@@ -358,5 +359,16 @@ fn sethostname(hostname: &OsStr) -> Result<()> {
}
}
// The Rust standard library had suppressed the default SIGPIPE behavior,
// see https://github.com/rust-lang/rust/pull/13158.
// Since the parent's signal handler would be inherited by it's child process,
// thus we should re-enable the standard SIGPIPE behavior as a workaround to
// fix the issue of https://github.com/kata-containers/kata-containers/issues/1887.
fn reset_sigpipe() {
unsafe {
libc::signal(libc::SIGPIPE, libc::SIG_DFL);
}
}
use crate::config::AgentConfig;
use std::os::unix::io::{FromRawFd, RawFd};

View File

@@ -22,6 +22,9 @@ async fn handle_sigchild(logger: Logger, sandbox: Arc<Mutex<Sandbox>>) -> Result
info!(logger, "handling signal"; "signal" => "SIGCHLD");
loop {
// Avoid reaping the undesirable child's signal, e.g., execute_hook's
// The lock should be released immediately.
rustjail::container::WAIT_PID_LOCKER.lock().await;
let result = wait::waitpid(
Some(Pid::from_raw(-1)),
Some(WaitPidFlag::WNOHANG | WaitPidFlag::__WALL),

View File

@@ -150,6 +150,10 @@ block_device_driver = "@DEFBLOCKSTORAGEDRIVER_ACRN@"
#debug_console_enabled = true
# Agent connection dialing timeout value in seconds
# (default: 30)
#dial_timeout = 30
[netmon]
# If enabled, the network monitoring process gets started when the
# sandbox is created. This allows for the detection of some additional

View File

@@ -165,6 +165,10 @@ block_device_driver = "virtio-blk"
#debug_console_enabled = true
# Agent connection dialing timeout value in seconds
# (default: 30)
#dial_timeout = 30
[netmon]
# If enabled, the network monitoring process gets started when the
# sandbox is created. This allows for the detection of some additional

View File

@@ -287,6 +287,10 @@ kernel_modules=[]
#debug_console_enabled = true
# Agent connection dialing timeout value in seconds
# (default: 30)
#dial_timeout = 30
[netmon]
# If enabled, the network monitoring process gets started when the
# sandbox is created. This allows for the detection of some additional

View File

@@ -437,6 +437,10 @@ kernel_modules=[]
#debug_console_enabled = true
# Agent connection dialing timeout value in seconds
# (default: 30)
#dial_timeout = 30
[netmon]
# If enabled, the network monitoring process gets started when the
# sandbox is created. This allows for the detection of some additional

View File

@@ -25,6 +25,7 @@ import (
"strings"
"syscall"
"github.com/containerd/cgroups"
"github.com/kata-containers/kata-containers/src/runtime/pkg/katautils"
vc "github.com/kata-containers/kata-containers/src/runtime/virtcontainers"
"github.com/kata-containers/kata-containers/src/runtime/virtcontainers/pkg/oci"
@@ -389,13 +390,6 @@ EXAMPLES:
if verbose {
kataLog.Logger.SetLevel(logrus.InfoLevel)
}
ctx, err := cliContextToContext(context)
if err != nil {
return err
}
span, _ := katautils.Trace(ctx, "check")
defer span.End()
if !context.Bool("no-network-checks") && os.Getenv(noNetworkEnvVar) == "" {
cmd := RelCmdCheck
@@ -407,8 +401,7 @@ EXAMPLES:
if os.Geteuid() == 0 {
kataLog.Warn("Not running network checks as super user")
} else {
err = HandleReleaseVersions(cmd, version, context.Bool("include-all-releases"))
err := HandleReleaseVersions(cmd, version, context.Bool("include-all-releases"))
if err != nil {
return err
}
@@ -424,7 +417,12 @@ EXAMPLES:
return errors.New("check: cannot determine runtime config")
}
err = setCPUtype(runtimeConfig.HypervisorType)
// check if cgroup can work use the same logic for creating containers
if _, err := vc.V1Constraints(); err != nil && err == cgroups.ErrMountPointNotExist && !runtimeConfig.SandboxCgroupOnly {
return fmt.Errorf("Cgroup v2 requires the following configuration: `sandbox_cgroup_only=true`.")
}
err := setCPUtype(runtimeConfig.HypervisorType)
if err != nil {
return err
}
@@ -437,7 +435,6 @@ EXAMPLES:
}
err = hostIsVMContainerCapable(details)
if err != nil {
return err
}

View File

@@ -13,7 +13,6 @@ import (
"strings"
"github.com/BurntSushi/toml"
"github.com/kata-containers/kata-containers/src/runtime/pkg/katautils"
"github.com/kata-containers/kata-containers/src/runtime/pkg/utils"
vc "github.com/kata-containers/kata-containers/src/runtime/virtcontainers"
exp "github.com/kata-containers/kata-containers/src/runtime/virtcontainers/experimental"
@@ -448,14 +447,6 @@ var kataEnvCLICommand = cli.Command{
},
},
Action: func(context *cli.Context) error {
ctx, err := cliContextToContext(context)
if err != nil {
return err
}
span, _ := katautils.Trace(ctx, "kata-env")
defer span.End()
return handleSettings(defaultOutputFile, context)
},
}

View File

@@ -14,7 +14,6 @@ import (
"net/http"
"net/url"
"os"
"path/filepath"
"strings"
"sync"
@@ -26,7 +25,6 @@ import (
clientUtils "github.com/kata-containers/kata-containers/src/runtime/virtcontainers/pkg/agent/protocols/client"
"github.com/pkg/errors"
"github.com/urfave/cli"
"go.opentelemetry.io/otel/label"
)
const (
@@ -38,10 +36,8 @@ const (
subCommandName = "exec"
// command-line parameters name
paramRuntimeNamespace = "runtime-namespace"
paramDebugConsolePort = "kata-debug-port"
defaultKernelParamDebugConsoleVPortValue = 1026
defaultRuntimeNamespace = "k8s.io"
)
var (
@@ -57,34 +53,16 @@ var kataExecCLICommand = cli.Command{
Name: subCommandName,
Usage: "Enter into guest by debug console",
Flags: []cli.Flag{
cli.StringFlag{
Name: paramRuntimeNamespace,
Usage: "Namespace that containerd or CRI-O are using for containers. (Default: k8s.io, only works for containerd)",
},
cli.Uint64Flag{
Name: paramDebugConsolePort,
Usage: "Port that debug console is listening on. (Default: 1026)",
},
},
Action: func(context *cli.Context) error {
ctx, err := cliContextToContext(context)
if err != nil {
return err
}
span, _ := katautils.Trace(ctx, subCommandName)
defer span.End()
namespace := context.String(paramRuntimeNamespace)
if namespace == "" {
namespace = defaultRuntimeNamespace
}
span.SetAttributes(label.Key("namespace").String(namespace))
port := context.Uint64(paramDebugConsolePort)
if port == 0 {
port = defaultKernelParamDebugConsoleVPortValue
}
span.SetAttributes(label.Key("port").Uint64(port))
sandboxID := context.Args().Get(0)
@@ -92,9 +70,8 @@ var kataExecCLICommand = cli.Command{
return err
}
span.SetAttributes(label.Key("sandbox").String(sandboxID))
conn, err := getConn(sandboxID, port)
conn, err := getConn(namespace, sandboxID, port)
if err != nil {
return err
}
@@ -177,9 +154,8 @@ func (s *iostream) Read(data []byte) (n int, err error) {
return s.conn.Read(data)
}
func getConn(namespace, sandboxID string, port uint64) (net.Conn, error) {
socketAddr := filepath.Join(string(filepath.Separator), "containerd-shim", namespace, sandboxID, "shim-monitor.sock")
client, err := kataMonitor.BuildUnixSocketClient(socketAddr, defaultTimeout)
func getConn(sandboxID string, port uint64) (net.Conn, error) {
client, err := kataMonitor.BuildShimClient(sandboxID, defaultTimeout)
if err != nil {
return nil, err
}
@@ -190,7 +166,7 @@ func getConn(namespace, sandboxID string, port uint64) (net.Conn, error) {
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("Failed to get %s: %d", socketAddr, resp.StatusCode)
return nil, fmt.Errorf("Failure from %s shim-monitor: %d", sandboxID, resp.StatusCode)
}
defer resp.Body.Close()

View File

@@ -0,0 +1,38 @@
// Copyright (c) 2021 Apple Inc.
//
// SPDX-License-Identifier: Apache-2.0
//
package main
import (
"fmt"
kataMonitor "github.com/kata-containers/kata-containers/src/runtime/pkg/kata-monitor"
"github.com/kata-containers/kata-containers/src/runtime/pkg/katautils"
"github.com/urfave/cli"
)
var kataMetricsCLICommand = cli.Command{
Name: "metrics",
Usage: "gather metrics associated with infrastructure used to run a sandbox",
UsageText: "metrics <sandbox id>",
Action: func(context *cli.Context) error {
sandboxID := context.Args().Get(0)
if err := katautils.VerifyContainerID(sandboxID); err != nil {
return err
}
// Get the metrics!
metrics, err := kataMonitor.GetSandboxMetrics(sandboxID)
if err != nil {
return err
}
fmt.Printf("%s\n", metrics)
return nil
},
}

View File

@@ -27,9 +27,6 @@ import (
specs "github.com/opencontainers/runtime-spec/specs-go"
"github.com/sirupsen/logrus"
"github.com/urfave/cli"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/label"
otelTrace "go.opentelemetry.io/otel/trace"
)
// specConfig is the name of the file holding the containers configuration
@@ -125,6 +122,7 @@ var runtimeCommands = []cli.Command{
kataCheckCLICommand,
kataEnvCLICommand,
kataExecCLICommand,
kataMetricsCLICommand,
factoryCLICommand,
}
@@ -132,10 +130,6 @@ var runtimeCommands = []cli.Command{
// parsing occurs.
var runtimeBeforeSubcommands = beforeSubcommands
// runtimeAfterSubcommands is the function to run after the command-line
// has been parsed.
var runtimeAfterSubcommands = afterSubcommands
// runtimeCommandNotFound is the function to handle an invalid sub-command.
var runtimeCommandNotFound = commandNotFound
@@ -168,10 +162,6 @@ func init() {
// setupSignalHandler sets up signal handling, starting a go routine to deal
// with signals as they arrive.
//
// Note that the specified context is NOT used to create a trace span (since the
// first (root) span must be created in beforeSubcommands()): it is simply
// used to pass to the crash handling functions to finalise tracing.
func setupSignalHandler(ctx context.Context) {
signals.SetLogger(kataLog)
@@ -181,10 +171,6 @@ func setupSignalHandler(ctx context.Context) {
signal.Notify(sigCh, sig)
}
dieCb := func() {
katautils.StopTracing(ctx)
}
go func() {
for {
sig := <-sigCh
@@ -198,7 +184,6 @@ func setupSignalHandler(ctx context.Context) {
if signals.FatalSignal(nativeSignal) {
kataLog.WithField("signal", sig).Error("received fatal signal")
signals.Die(dieCb)
} else if debug && signals.NonFatalSignal(nativeSignal) {
kataLog.WithField("signal", sig).Debug("handling signal")
signals.Backtrace()
@@ -210,18 +195,6 @@ func setupSignalHandler(ctx context.Context) {
// setExternalLoggers registers the specified logger with the external
// packages which accept a logger to handle their own logging.
func setExternalLoggers(ctx context.Context, logger *logrus.Entry) {
var span otelTrace.Span
// Only create a new span if a root span already exists. This is
// required to ensure that this function will not disrupt the root
// span logic by creating a span before the proper root span has been
// created.
if otelTrace.SpanFromContext(ctx) != nil {
span, ctx = katautils.Trace(ctx, "setExternalLoggers")
defer span.End()
}
// Set virtcontainers logger.
vci.SetLogger(ctx, logger)
@@ -244,7 +217,6 @@ func beforeSubcommands(c *cli.Context) error {
var configFile string
var runtimeConfig oci.RuntimeConfig
var err error
var traceFlushFunc func()
katautils.SetConfigOptions(name, defaultRuntimeConfiguration, defaultSysConfRuntimeConfiguration)
@@ -270,7 +242,6 @@ func beforeSubcommands(c *cli.Context) error {
// Issue: https://github.com/kata-containers/runtime/issues/2428
ignoreConfigLogs := false
var traceRootSpan string
subCmdIsCheckCmd := (c.NArg() >= 1 && ((c.Args()[0] == "kata-check") || (c.Args()[0] == "check")))
if subCmdIsCheckCmd {
@@ -302,16 +273,13 @@ func beforeSubcommands(c *cli.Context) error {
cmdName := c.Args().First()
if c.App.Command(cmdName) != nil {
kataLog = kataLog.WithField("command", cmdName)
// Name for the root span (used for tracing) now the
// sub-command name is known.
traceRootSpan = name + " " + cmdName
}
// Since a context is required, pass a new (throw-away) one - we
// cannot use the main context as tracing hasn't been enabled yet
// (meaning any spans created at this point will be silently ignored).
setExternalLoggers(context.Background(), kataLog)
ctx, err := cliContextToContext(c)
if err != nil {
return err
}
setExternalLoggers(ctx, kataLog)
if c.NArg() == 1 && (c.Args()[0] == "kata-env" || c.Args()[0] == "env") {
// simply report the logging setup
@@ -325,20 +293,6 @@ func beforeSubcommands(c *cli.Context) error {
}
if !subCmdIsCheckCmd {
debug = runtimeConfig.Debug
if traceRootSpan != "" {
// Create the tracer.
//
// Note: no spans are created until the command-line has been parsed.
// This delays collection of trace data slightly but benefits the user by
// ensuring the first span is the name of the sub-command being
// invoked from the command-line.
traceFlushFunc, err = setupTracing(c, traceRootSpan, &runtimeConfig)
if err != nil {
return err
}
defer traceFlushFunc()
}
}
args := strings.Join(c.Args(), " ")
@@ -377,36 +331,6 @@ func handleShowConfig(context *cli.Context) {
}
}
func setupTracing(context *cli.Context, rootSpanName string, config *oci.RuntimeConfig) (func(), error) {
flush, err := katautils.CreateTracer(name, config)
if err != nil {
return nil, err
}
ctx, err := cliContextToContext(context)
if err != nil {
return nil, err
}
// Create the root span now that the sub-command name is
// known.
//
// Note that this "Before" function is called (and returns)
// before the subcommand handler is called. As such, we cannot
// "Finish()" the span here - that is handled in the .After
// function.
tracer := otel.Tracer("kata")
newCtx, span := tracer.Start(ctx, rootSpanName)
span.SetAttributes(label.Key("subsystem").String("runtime"))
// Add tracer to metadata and update the context
context.App.Metadata["tracer"] = tracer
context.App.Metadata["context"] = newCtx
return flush, nil
}
// add supported experimental features in context
func addExpFeatures(clictx *cli.Context, runtimeConfig oci.RuntimeConfig) error {
ctx, err := cliContextToContext(clictx)
@@ -420,22 +344,11 @@ func addExpFeatures(clictx *cli.Context, runtimeConfig oci.RuntimeConfig) error
}
ctx = exp.ContextWithExp(ctx, exps)
// Add tracer to metadata and update the context
// Add experimental features to metadata and update the context
clictx.App.Metadata["context"] = ctx
return nil
}
func afterSubcommands(c *cli.Context) error {
ctx, err := cliContextToContext(c)
if err != nil {
return err
}
katautils.StopTracing(ctx)
return nil
}
// function called when an invalid command is specified which causes the
// runtime to error.
func commandNotFound(c *cli.Context, command string) {
@@ -502,7 +415,6 @@ func createRuntimeApp(ctx context.Context, args []string) error {
app.Flags = runtimeFlags
app.Commands = runtimeCommands
app.Before = runtimeBeforeSubcommands
app.After = runtimeAfterSubcommands
app.EnableBashCompletion = true
// allow sub-commands to access context
@@ -578,12 +490,5 @@ func cliContextToContext(c *cli.Context) (context.Context, error) {
func main() {
// create a new empty context
ctx := context.Background()
dieCb := func() {
katautils.StopTracing(ctx)
}
defer signals.HandlePanic(dieCb)
createRuntime(ctx)
}

View File

@@ -20,7 +20,6 @@ import (
"strings"
"testing"
"github.com/dlespiau/covertool/pkg/cover"
ktu "github.com/kata-containers/kata-containers/src/runtime/pkg/katatestutils"
"github.com/kata-containers/kata-containers/src/runtime/pkg/katautils"
"github.com/kata-containers/kata-containers/src/runtime/pkg/utils"
@@ -152,16 +151,6 @@ func runUnitTests(m *testing.M) {
// TestMain is the common main function used by ALL the test functions
// for this package.
func TestMain(m *testing.M) {
// Parse the command line using the stdlib flag package so the flags defined
// in the testing package get populated.
cover.ParseAndStripTestFlags()
// Make sure we have the opportunity to flush the coverage report to disk when
// terminating the process.
defer func() {
cover.FlushProfiles()
}()
// If the test binary name is kata-runtime.coverage, we've are being asked to
// run the coverage-instrumented kata-runtime.
if path.Base(os.Args[0]) == name+".coverage" ||

View File

@@ -6,7 +6,6 @@
package main
import (
"github.com/kata-containers/kata-containers/src/runtime/pkg/katautils"
"github.com/urfave/cli"
)
@@ -14,14 +13,6 @@ var versionCLICommand = cli.Command{
Name: "version",
Usage: "display version details",
Action: func(context *cli.Context) error {
ctx, err := cliContextToContext(context)
if err != nil {
return err
}
span, _ := katautils.Trace(ctx, "version")
defer span.End()
cli.VersionPrinter(context)
return nil
},

View File

@@ -16,7 +16,6 @@ import (
"strconv"
"strings"
"github.com/containerd/containerd/namespaces"
cdshim "github.com/containerd/containerd/runtime/v2/shim"
vc "github.com/kata-containers/kata-containers/src/runtime/virtcontainers"
vcAnnotations "github.com/kata-containers/kata-containers/src/runtime/virtcontainers/pkg/annotations"
@@ -129,11 +128,7 @@ func decodeAgentMetrics(body string) []*dto.MetricFamily {
func (s *service) startManagementServer(ctx context.Context, ociSpec *specs.Spec) {
// metrics socket will under sandbox's bundle path
metricsAddress, err := socketAddress(ctx, s.id)
if err != nil {
shimMgtLog.WithError(err).Error("failed to create socket address")
return
}
metricsAddress := SocketAddress(s.id)
listener, err := cdshim.NewSocket(metricsAddress)
if err != nil {
@@ -166,7 +161,7 @@ func (s *service) startManagementServer(ctx context.Context, ociSpec *specs.Spec
svr.Serve(listener)
}
// mountServeDebug provides a debug endpoint
// mountPprofHandle provides a debug endpoint
func (s *service) mountPprofHandle(m *http.ServeMux, ociSpec *specs.Spec) {
// return if not enabled
@@ -188,10 +183,8 @@ func (s *service) mountPprofHandle(m *http.ServeMux, ociSpec *specs.Spec) {
m.Handle("/debug/pprof/trace", http.HandlerFunc(pprof.Trace))
}
func socketAddress(ctx context.Context, id string) (string, error) {
ns, err := namespaces.NamespaceRequired(ctx)
if err != nil {
return "", err
}
return filepath.Join(string(filepath.Separator), "containerd-shim", ns, id, "shim-monitor.sock"), nil
// SocketAddress returns the address of the abstract domain socket for communicating with the
// shim management endpoint
func SocketAddress(id string) string {
return filepath.Join(string(filepath.Separator), "run", "vc", id, "shim-monitor")
}

View File

@@ -142,7 +142,7 @@ func watchOOMEvents(ctx context.Context, s *service) {
for {
select {
case <-ctx.Done():
case <-s.ctx.Done():
return
default:
containerID, err := s.sandbox.GetOOMEvent(ctx)

View File

@@ -18,7 +18,6 @@ require (
github.com/containerd/typeurl v1.0.1-0.20190228175220-2a93cfde8c20
github.com/containernetworking/plugins v0.8.2
github.com/cri-o/cri-o v1.0.0-rc2.0.20170928185954-3394b3b2d6af
github.com/dlespiau/covertool v0.0.0-20180314162135-b0c4c6d0583a
github.com/docker/distribution v2.7.1+incompatible // indirect
github.com/docker/docker v1.13.1 // indirect
github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c // indirect

View File

@@ -108,8 +108,6 @@ github.com/d2g/hardwareaddr v0.0.0-20190221164911-e7d9fbe030e4/go.mod h1:bMl4RjI
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dlespiau/covertool v0.0.0-20180314162135-b0c4c6d0583a h1:+cYgqwB++gEE09SluRYGqJyDhWmLmdWZ2cXlOXSGV8w=
github.com/dlespiau/covertool v0.0.0-20180314162135-b0c4c6d0583a/go.mod h1:/eQMcW3eA1bzKx23ZYI2H3tXPdJB5JWYTHzoUPBvQY4=
github.com/docker/distribution v2.7.1+incompatible h1:a5mlkVzth6W5A4fOsS3D2EO5BUmsJpcB+cRlLU7cSug=
github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker v1.13.1 h1:IkZjBSIc8hBjLpqeAbeE5mca5mNgeatLHBy3GO78BWo=
@@ -200,7 +198,6 @@ github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt
github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=
github.com/golang/protobuf v0.0.0-20161109072736-4bd1920723d7/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1 h1:YF8+flBXS5eO826T4nzqPrxfhQThhXl0YzfuUPu4SBg=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
@@ -218,9 +215,7 @@ github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Z
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1 h1:Xye71clBPdm5HgqGwUkwhbynsUJZhDbS20FvLhQ2izg=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0 h1:xsAVV57WRhGj6kEIi8ReJzQlHHqcBYCElAvkovg3B/4=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
@@ -242,7 +237,6 @@ github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hf
github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.1.0 h1:Jf4mxPC/ziBnoPIdpQdPJ9OeiomAUHLvxmPRSPH9m4s=
github.com/google/uuid v1.1.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.1.1 h1:Gkbcsh/GbpXz7lPftLA3P6TYMwjCLYm83jiFQZF/3gY=
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
@@ -274,10 +268,6 @@ github.com/juju/errors v0.0.0-20180806074554-22422dad46e1/go.mod h1:W54LbzXuIE0b
github.com/juju/loggo v0.0.0-20190526231331-6e530bcce5d8/go.mod h1:vgyd7OREkbtVEN/8IXZe5Ooef3LQePvuBm9UWj6ZL8U=
github.com/juju/testing v0.0.0-20190613124551-e81189438503/go.mod h1:63prj8cnj0tU0S9OHjGJn+b1h0ZghCndfnbQolrYTwA=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/kata-containers/govmm v0.0.0-20210112013750-7d320e8f5dca h1:UdXFthwasAPnmv37gLJUEFsW9FaabYA+mM6FoSi8kzU=
github.com/kata-containers/govmm v0.0.0-20210112013750-7d320e8f5dca/go.mod h1:VmAHbsL5lLfzHW/MNL96NVLF840DNEV5i683kISgFKk=
github.com/kata-containers/govmm v0.0.0-20210329205824-7fbc685865d2 h1:oDJsQ5avmW8PFUkSJdsuaGL3SR4/YQRno51GNGl+IDU=
github.com/kata-containers/govmm v0.0.0-20210329205824-7fbc685865d2/go.mod h1:VmAHbsL5lLfzHW/MNL96NVLF840DNEV5i683kISgFKk=
github.com/kata-containers/govmm v0.0.0-20210428163604-f0e9a35308ee h1:M4N7AdSHgWz/ubV5AZQdeqmK+9Ztpea6oqeXgk8GCHk=
github.com/kata-containers/govmm v0.0.0-20210428163604-f0e9a35308ee/go.mod h1:VmAHbsL5lLfzHW/MNL96NVLF840DNEV5i683kISgFKk=
github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=
@@ -307,15 +297,12 @@ github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3Rllmb
github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+o7JKHSa8/e818NopupXU1YMK5fe1lsApnBw=
github.com/onsi/ginkgo v0.0.0-20151202141238-7f8ab55aaf3b h1:Ey6yH0acn50T/v6CB75bGP4EMJqnv9WvnjN7oZaj+xE=
github.com/onsi/ginkgo v0.0.0-20151202141238-7f8ab55aaf3b/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.10.1 h1:q/mM8GF/n0shIN8SaAZ0V+jnLPzen6WIVZdiwrRlMlo=
github.com/onsi/ginkgo v1.10.1/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.11.0 h1:JAKSXpt1YjtLA7YpPiqO9ss6sNXEsPfSGdwN0UHqzrw=
github.com/onsi/ginkgo v1.11.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/gomega v0.0.0-20151007035656-2152b45fa28a h1:KfNOeFvoAssuZLT7IntKZElKwi/5LRuxY71k+t6rfaM=
github.com/onsi/gomega v0.0.0-20151007035656-2152b45fa28a/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA=
github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA=
github.com/onsi/gomega v1.7.0 h1:XPnZz8VVBHjVsy1vzJmRwIcSwiUO+JFfrv/xGiigmME=
@@ -335,7 +322,6 @@ github.com/pborman/uuid v1.2.0 h1:J7Q5mO4ysT1dv8hyrUGHb9+ooztCXu1D8MY8DZYsu3g=
github.com/pborman/uuid v1.2.0/go.mod h1:X/NO0urCmaxf9VXbdlT7C2Yzkj2IKimNn4k+gtPdI/k=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.8.1-0.20171018195549-f15c970de5b7/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
@@ -379,7 +365,6 @@ github.com/stretchr/objx v0.1.1 h1:2vfRuCMp5sSVIDSqO8oNnWJq7mPa6KVP3iPIwFBuy8A=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.6.1 h1:hDPOHmpOpP40lSULcqw7IrRb/u7w6RpDC9399XyoNd0=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
@@ -431,7 +416,6 @@ golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTk
golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190930215403-16217165b5de h1:5hukYrvBGR8/eNkX5mdUezrA6JiaEZDtJb9Ei+1LlBs=
golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=
golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
@@ -463,7 +447,6 @@ golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLL
golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191004110552-13f9640d40b9/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191108221443-4ba9e2ef068c h1:SRpq/kuj/xNci/RdvEs+RSvpfxqvLAzTKuKGlzoGdZQ=
golang.org/x/net v0.0.0-20191108221443-4ba9e2ef068c/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
@@ -477,7 +460,6 @@ golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/
golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200707034311-ab3426394381 h1:VXak5I6aEWmAXeQjA+QSZzlgNrpq9mjcfDemuexIKsU=
golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200822124328-c89045814202 h1:VvcQYSHwXgi7W+TpUR6A9g6Up98WAHf3f/ulnJ62IyA=
golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
@@ -493,7 +475,6 @@ golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e h1:vcxGaoTs7kV8m5Np9uUNQin4BrLOthgV7252N8V+FwY=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208 h1:qwRHBd0NqMbJxfbotnDhm2ByMI1Shq4Y6oRJo21SGJA=
@@ -535,17 +516,14 @@ golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1 h1:ogLJMz+qpzav7lGMh10LMvAkM/fAoGlaiiHYiFYdm80=
golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f h1:Fqb3ao1hUmOR3GkUOg/Y+BadLwykBIzs5q8Ez2SbHyc=
golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.0.0-20160726164857-2910a502d2bf/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3 h1:cokOdA+Jmi5PJGXLlLllQSgYigAEfHXJAERHVMaCc2k=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
@@ -597,7 +575,6 @@ golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc
golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -620,7 +597,6 @@ google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz513
google.golang.org/api v0.32.0 h1:Le77IccnTqEa8ryp9wIpX5W3zYm7Gf9LhOp9PHcwFts=
google.golang.org/api v0.32.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0 h1:/wp5JvzpHIxhs/dumFmF7BXTf3Z+dd4uXta4kVyO508=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=
@@ -637,14 +613,12 @@ google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQ
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.0 h1:4MY060fB1DLGMB/7MBTLnwQUY6+F09GEiz6SsrNqyzM=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.25.0 h1:Ejskq+SyPohKW+1uil0JJMtmHCgJPJ/qWTxr8qp+R4c=
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
gopkg.in/airbrake/gobrake.v2 v2.0.9/go.mod h1:/h5ZAUhDkGaJfjzjKLSjv6zCL6O0LLBxU4K+aSYdM/U=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
@@ -658,10 +632,8 @@ gopkg.in/mgo.v2 v2.0.0-20180705113604-9856a29383ce/go.mod h1:yeKp02qBN3iKW1OzL3M
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.5 h1:ymVxjfMaHvXD8RqPRmzHHsB3VvucivSkIAvJFDI5O3c=
gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=

View File

@@ -176,7 +176,7 @@ func (km *KataMonitor) aggregateSandboxMetrics(encoder expfmt.Encoder) error {
for sandboxID, namespace := range sandboxes {
wg.Add(1)
go func(sandboxID, namespace string, results chan<- []*dto.MetricFamily) {
sandboxMetrics, err := km.getSandboxMetrics(sandboxID, namespace)
sandboxMetrics, err := getParsedMetrics(sandboxID)
if err != nil {
monitorLog.WithError(err).WithField("sandbox_id", sandboxID).Errorf("failed to get metrics for sandbox")
}
@@ -229,13 +229,12 @@ func (km *KataMonitor) aggregateSandboxMetrics(encoder expfmt.Encoder) error {
return err
}
}
return nil
}
// getSandboxMetrics will get sandbox's metrics from shim
func (km *KataMonitor) getSandboxMetrics(sandboxID, namespace string) ([]*dto.MetricFamily, error) {
body, err := km.doGet(sandboxID, namespace, defaultTimeout, "metrics")
func getParsedMetrics(sandboxID string) ([]*dto.MetricFamily, error) {
body, err := doGet(sandboxID, defaultTimeout, "metrics")
if err != nil {
return nil, err
}
@@ -243,6 +242,16 @@ func (km *KataMonitor) getSandboxMetrics(sandboxID, namespace string) ([]*dto.Me
return parsePrometheusMetrics(sandboxID, body)
}
// GetSandboxMetrics will get sandbox's metrics from shim
func GetSandboxMetrics(sandboxID string) (string, error) {
body, err := doGet(sandboxID, defaultTimeout, "metrics")
if err != nil {
return "", err
}
return string(body), nil
}
// parsePrometheusMetrics will decode metrics from Prometheus text format
// and return array of *dto.MetricFamily with an ASC order
func parsePrometheusMetrics(sandboxID string, body []byte) ([]*dto.MetricFamily, error) {

View File

@@ -87,13 +87,8 @@ func (km *KataMonitor) GetAgentURL(w http.ResponseWriter, r *http.Request) {
commonServeError(w, http.StatusBadRequest, err)
return
}
namespace, err := km.getSandboxNamespace(sandboxID)
if err != nil {
commonServeError(w, http.StatusBadRequest, err)
return
}
data, err := km.doGet(sandboxID, namespace, defaultTimeout, "agent-url")
data, err := doGet(sandboxID, defaultTimeout, "agent-url")
if err != nil {
commonServeError(w, http.StatusBadRequest, err)
return

View File

@@ -11,6 +11,8 @@ import (
"net"
"net/http"
"time"
shim "github.com/kata-containers/kata-containers/src/runtime/containerd-shim-v2"
)
const (
@@ -33,16 +35,13 @@ func getSandboxIDFromReq(r *http.Request) (string, error) {
return "", fmt.Errorf("sandbox not found in %+v", r.URL.Query())
}
func (km *KataMonitor) buildShimClient(sandboxID, namespace string, timeout time.Duration) (*http.Client, error) {
socketAddr, err := km.getMonitorAddress(sandboxID, namespace)
if err != nil {
return nil, err
}
return BuildUnixSocketClient(socketAddr, timeout)
// BuildShimClient builds and returns an http client for communicating with the provided sandbox
func BuildShimClient(sandboxID string, timeout time.Duration) (*http.Client, error) {
return buildUnixSocketClient(shim.SocketAddress(sandboxID), timeout)
}
// BuildUnixSocketClient build http client for Unix socket
func BuildUnixSocketClient(socketAddr string, timeout time.Duration) (*http.Client, error) {
// buildUnixSocketClient build http client for Unix socket
func buildUnixSocketClient(socketAddr string, timeout time.Duration) (*http.Client, error) {
transport := &http.Transport{
DisableKeepAlives: true,
Dial: func(proto, addr string) (conn net.Conn, err error) {
@@ -61,8 +60,8 @@ func BuildUnixSocketClient(socketAddr string, timeout time.Duration) (*http.Clie
return client, nil
}
func (km *KataMonitor) doGet(sandboxID, namespace string, timeoutInSeconds time.Duration, urlPath string) ([]byte, error) {
client, err := km.buildShimClient(sandboxID, namespace, timeoutInSeconds)
func doGet(sandboxID string, timeoutInSeconds time.Duration, urlPath string) ([]byte, error) {
client, err := BuildShimClient(sandboxID, timeoutInSeconds)
if err != nil {
return nil, err
}

View File

@@ -154,6 +154,7 @@ type agent struct {
Debug bool `toml:"enable_debug"`
Tracing bool `toml:"enable_tracing"`
DebugConsoleEnabled bool `toml:"debug_console_enabled"`
DialTimeout uint32 `toml:"dial_timeout"`
}
type netmon struct {
@@ -471,6 +472,10 @@ func (a agent) debugConsoleEnabled() bool {
return a.DebugConsoleEnabled
}
func (a agent) dialTimout() uint32 {
return a.DialTimeout
}
func (a agent) debug() bool {
return a.Debug
}
@@ -920,6 +925,7 @@ func updateRuntimeConfigAgent(configPath string, tomlConf tomlConfig, config *oc
TraceType: agent.traceType(),
KernelModules: agent.kernelModules(),
EnableDebugConsole: agent.debugConsoleEnabled(),
DialTimeout: agent.dialTimout(),
}
}

View File

@@ -1,201 +0,0 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright {yyyy} {name of copyright owner}
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@@ -1,82 +0,0 @@
// Copyright (c) 2017 Intel Corporation
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package cover
import (
"flag"
"io"
"os"
"strings"
"testing"
)
// ParseAndStriptestFlags runs flag.Parse to parse the standard flags of a test
// binary and remove them from os.Args.
func ParseAndStripTestFlags() {
// Parse the command line using the stdlib flag package so the flags defined
// in the testing package get populated.
flag.Parse()
// Strip command line arguments that were for the testing package and are now
// handled.
var runtimeArgs []string
for _, arg := range os.Args {
if strings.HasPrefix(arg, "-test.") ||
strings.HasPrefix(arg, "-httptest.") {
continue
}
runtimeArgs = append(runtimeArgs, arg)
}
os.Args = runtimeArgs
}
type dummyTestDeps func(pat, str string) (bool, error)
func (d dummyTestDeps) MatchString(pat, str string) (bool, error) { return false, nil }
func (d dummyTestDeps) StartCPUProfile(io.Writer) error { return nil }
func (d dummyTestDeps) StopCPUProfile() {}
func (f dummyTestDeps) StartTestLog(w io.Writer) {}
func (f dummyTestDeps) StopTestLog() error { return nil }
func (d dummyTestDeps) WriteHeapProfile(io.Writer) error { return nil }
func (d dummyTestDeps) WriteProfileTo(string, io.Writer, int) error { return nil }
func (f dummyTestDeps) ImportPath() string { return "" }
// FlushProfiles flushes test profiles to disk. It works by build and executing
// a dummy list of 1 test. This is to ensure we execute the M.after() function
// (a function internal to the testing package) where all reporting (cpu, mem,
// cover, ... profiles) is flushed to disk.
func FlushProfiles() {
// Redirect Stdout/err temporarily so the testing code doesn't output the
// regular:
// PASS
// coverage: 21.4% of statements
// Thanks to this, we can test the output of the instrumented binary the same
// way the normal binary is.
oldstdout := os.Stdout
oldstderr := os.Stderr
os.Stdout, _ = os.Open(os.DevNull)
os.Stderr, _ = os.Open(os.DevNull)
tests := []testing.InternalTest{}
benchmarks := []testing.InternalBenchmark{}
examples := []testing.InternalExample{}
var f dummyTestDeps
dummyM := testing.MainStart(f, tests, benchmarks, examples)
dummyM.Run()
// restore stdout/err
os.Stdout = oldstdout
os.Stderr = oldstderr
}

View File

@@ -145,7 +145,6 @@ github.com/cri-o/cri-o/pkg/annotations
github.com/davecgh/go-spew/spew
# github.com/dlespiau/covertool v0.0.0-20180314162135-b0c4c6d0583a
## explicit
github.com/dlespiau/covertool/pkg/cover
# github.com/docker/distribution v2.7.1+incompatible
## explicit
github.com/docker/distribution/digestset

View File

@@ -217,6 +217,7 @@ type KataAgentConfig struct {
ContainerPipeSize uint32
TraceMode string
TraceType string
DialTimeout uint32
KernelModules []string
}
@@ -236,6 +237,7 @@ type kataAgent struct {
keepConn bool
dynamicTracing bool
dead bool
dialTimout uint32
kmodules []string
vmSocket interface{}
@@ -344,6 +346,7 @@ func (k *kataAgent) init(ctx context.Context, sandbox *Sandbox, config KataAgent
disableVMShutdown = k.handleTraceSettings(config)
k.keepConn = config.LongLiveConn
k.kmodules = config.KernelModules
k.dialTimout = config.DialTimeout
return disableVMShutdown, nil
}
@@ -380,7 +383,7 @@ func (k *kataAgent) internalConfigure(h hypervisor, id string, config KataAgentC
return nil
}
func setupSandboxBindMounts(sandbox *Sandbox) error {
func (k *kataAgent) setupSandboxBindMounts(sandbox *Sandbox) (err error) {
if len(sandbox.config.SandboxBindMounts) == 0 {
return nil
}
@@ -391,6 +394,20 @@ func setupSandboxBindMounts(sandbox *Sandbox) error {
if err := os.MkdirAll(sandboxMountDir, DirMode); err != nil {
return fmt.Errorf("Creating sandbox shared mount directory: %v: %w", sandboxMountDir, err)
}
var mountedList []string
defer func() {
if err != nil {
for _, mnt := range mountedList {
if derr := syscall.Unmount(mnt, syscall.MNT_DETACH|UmountNoFollow); derr != nil {
k.Logger().WithError(derr).Errorf("cleanup: couldn't unmount %s", mnt)
}
}
if derr := os.RemoveAll(sandboxMountDir); derr != nil {
k.Logger().WithError(derr).Errorf("cleanup: failed to remove %s", sandboxMountDir)
}
}
}()
for _, m := range sandbox.config.SandboxBindMounts {
mountDest := filepath.Join(sandboxMountDir, filepath.Base(m))
@@ -398,6 +415,7 @@ func setupSandboxBindMounts(sandbox *Sandbox) error {
if err := bindMount(context.Background(), m, mountDest, true, "private"); err != nil {
return fmt.Errorf("Mounting sandbox directory: %v to %v: %w", m, mountDest, err)
}
mountedList = append(mountedList, mountDest)
mountDest = filepath.Join(sandboxShareDir, filepath.Base(m))
if err := remountRo(context.Background(), mountDest); err != nil {
@@ -409,19 +427,30 @@ func setupSandboxBindMounts(sandbox *Sandbox) error {
return nil
}
func cleanupSandboxBindMounts(sandbox *Sandbox) error {
func (k *kataAgent) cleanupSandboxBindMounts(sandbox *Sandbox) error {
if sandbox.config == nil || len(sandbox.config.SandboxBindMounts) == 0 {
return nil
}
var retErr error
bindmountShareDir := filepath.Join(getMountPath(sandbox.id), sandboxMountsDir)
for _, m := range sandbox.config.SandboxBindMounts {
mountPath := filepath.Join(getMountPath(sandbox.id), sandboxMountsDir, filepath.Base(m))
mountPath := filepath.Join(bindmountShareDir, filepath.Base(m))
if err := syscall.Unmount(mountPath, syscall.MNT_DETACH|UmountNoFollow); err != nil {
return fmt.Errorf("Unmounting observe directory: %v: %w", mountPath, err)
if retErr == nil {
retErr = err
}
k.Logger().WithError(err).Errorf("Failed to unmount sandbox bindmount: %v", mountPath)
}
}
if err := os.RemoveAll(bindmountShareDir); err != nil {
if retErr == nil {
retErr = err
}
k.Logger().WithError(err).Errorf("Failed to remove sandbox bindmount directory: %s", bindmountShareDir)
}
return nil
return retErr
}
func (k *kataAgent) configure(ctx context.Context, h hypervisor, id, sharePath string, config KataAgentConfig) error {
@@ -470,7 +499,7 @@ func (k *kataAgent) configureFromGrpc(h hypervisor, id string, config KataAgentC
return k.internalConfigure(h, id, config)
}
func (k *kataAgent) setupSharedPath(ctx context.Context, sandbox *Sandbox) error {
func (k *kataAgent) setupSharedPath(ctx context.Context, sandbox *Sandbox) (err error) {
// create shared path structure
sharePath := getSharePath(sandbox.id)
mountPath := getMountPath(sandbox.id)
@@ -485,9 +514,16 @@ func (k *kataAgent) setupSharedPath(ctx context.Context, sandbox *Sandbox) error
if err := bindMount(ctx, mountPath, sharePath, true, "slave"); err != nil {
return err
}
defer func() {
if err != nil {
if umountErr := syscall.Unmount(sharePath, syscall.MNT_DETACH|UmountNoFollow); umountErr != nil {
k.Logger().WithError(umountErr).Errorf("failed to unmount vm share path %s", sharePath)
}
}
}()
// Setup sandbox bindmounts, if specified:
if err := setupSandboxBindMounts(sandbox); err != nil {
if err = k.setupSandboxBindMounts(sandbox); err != nil {
return err
}
@@ -1794,7 +1830,7 @@ func (k *kataAgent) connect(ctx context.Context) error {
}
k.Logger().WithField("url", k.state.URL).Info("New client")
client, err := kataclient.NewAgentClient(k.ctx, k.state.URL)
client, err := kataclient.NewAgentClient(k.ctx, k.state.URL, k.dialTimout)
if err != nil {
k.dead = true
return err
@@ -2146,8 +2182,8 @@ func (k *kataAgent) markDead(ctx context.Context) {
}
func (k *kataAgent) cleanup(ctx context.Context, s *Sandbox) {
if err := cleanupSandboxBindMounts(s); err != nil {
k.Logger().WithError(err).Errorf("failed to cleanup observability logs bindmount")
if err := k.cleanupSandboxBindMounts(s); err != nil {
k.Logger().WithError(err).Errorf("failed to cleanup sandbox bindmounts")
}
// Unmount shared path

View File

@@ -1216,3 +1216,116 @@ func TestKataAgentDirs(t *testing.T) {
assert.Equal(ephemeralPath(), defaultEphemeralPath)
}
}
func TestSandboxBindMount(t *testing.T) {
if os.Getuid() != 0 {
t.Skip("Test disabled as requires root user")
}
assert := assert.New(t)
// create temporary files to mount:
testMountPath, err := ioutil.TempDir("", "sandbox-test")
assert.NoError(err)
defer os.RemoveAll(testMountPath)
// create a new shared directory for our test:
kataHostSharedDirSaved := kataHostSharedDir
testHostDir, err := ioutil.TempDir("", "kata-cleanup")
assert.NoError(err)
kataHostSharedDir = func() string {
return testHostDir
}
defer func() {
kataHostSharedDir = kataHostSharedDirSaved
}()
m1Path := filepath.Join(testMountPath, "foo.txt")
f1, err := os.Create(m1Path)
assert.NoError(err)
defer f1.Close()
m2Path := filepath.Join(testMountPath, "bar.txt")
f2, err := os.Create(m2Path)
assert.NoError(err)
defer f2.Close()
// create sandbox for mounting into
sandbox := &Sandbox{
ctx: context.Background(),
id: "foobar",
config: &SandboxConfig{
SandboxBindMounts: []string{m1Path, m2Path},
},
}
k := &kataAgent{ctx: context.Background()}
// make the shared directory for our test:
dir := kataHostSharedDir()
err = os.MkdirAll(path.Join(dir, sandbox.id), 0777)
assert.Nil(err)
defer os.RemoveAll(dir)
sharePath := getSharePath(sandbox.id)
mountPath := getMountPath(sandbox.id)
err = os.MkdirAll(sharePath, DirMode)
assert.Nil(err)
err = os.MkdirAll(mountPath, DirMode)
assert.Nil(err)
// setup the expeted slave mount:
err = bindMount(sandbox.ctx, mountPath, sharePath, true, "slave")
assert.Nil(err)
defer syscall.Unmount(sharePath, syscall.MNT_DETACH|UmountNoFollow)
// Test the function. We expect it to succeed and for the mount to exist
err = k.setupSandboxBindMounts(sandbox)
assert.NoError(err)
// Test the cleanup function. We expect it to succeed for the mount to be removed.
err = k.cleanupSandboxBindMounts(sandbox)
assert.NoError(err)
// After successful cleanup, verify there are not any mounts left behind.
stat := syscall.Stat_t{}
mount1CheckPath := filepath.Join(getMountPath(sandbox.id), sandboxMountsDir, filepath.Base(m1Path))
err = syscall.Stat(mount1CheckPath, &stat)
assert.Error(err)
assert.True(os.IsNotExist(err))
mount2CheckPath := filepath.Join(getMountPath(sandbox.id), sandboxMountsDir, filepath.Base(m2Path))
err = syscall.Stat(mount2CheckPath, &stat)
assert.Error(err)
assert.True(os.IsNotExist(err))
// Now, let's setup the cleanup to fail. Setup the sandbox bind mount twice, which will result in
// extra mounts being present that the sandbox description doesn't account for (ie, duplicate mounts).
// We expect cleanup to fail on the first time, since it cannot remove the sandbox-bindmount directory because
// there are leftover mounts. If we run it a second time, however, it should succeed since it'll remove the
// second set of mounts:
err = k.setupSandboxBindMounts(sandbox)
assert.NoError(err)
err = k.setupSandboxBindMounts(sandbox)
assert.NoError(err)
// Test the cleanup function. We expect it to succeed for the mount to be removed.
err = k.cleanupSandboxBindMounts(sandbox)
assert.Error(err)
err = k.cleanupSandboxBindMounts(sandbox)
assert.NoError(err)
//
// Now, let's setup the sandbox bindmount to fail, and verify that no mounts are left behind
//
sandbox.config.SandboxBindMounts = append(sandbox.config.SandboxBindMounts, "oh-nos")
err = k.setupSandboxBindMounts(sandbox)
assert.Error(err)
// Verify there aren't any mounts left behind
stat = syscall.Stat_t{}
err = syscall.Stat(mount1CheckPath, &stat)
assert.Error(err)
assert.True(os.IsNotExist(err))
err = syscall.Stat(mount2CheckPath, &stat)
assert.Error(err)
assert.True(os.IsNotExist(err))
}

View File

@@ -191,6 +191,8 @@ func (s *Sandbox) dumpConfig(ss *persistapi.SandboxState) {
Cgroups: sconfig.Cgroups,
}
ss.Config.SandboxBindMounts = append(ss.Config.SandboxBindMounts, sconfig.SandboxBindMounts...)
for _, e := range sconfig.Experimental {
ss.Config.Experimental = append(ss.Config.Experimental, e.Name)
}
@@ -459,6 +461,7 @@ func loadSandboxConfig(id string) (*SandboxConfig, error) {
DisableGuestSeccomp: savedConf.DisableGuestSeccomp,
Cgroups: savedConf.Cgroups,
}
sconfig.SandboxBindMounts = append(sconfig.SandboxBindMounts, savedConf.SandboxBindMounts...)
for _, name := range savedConf.Experimental {
sconfig.Experimental = append(sconfig.Experimental, *exp.Get(name))

View File

@@ -275,6 +275,9 @@ type SandboxConfig struct {
DisableGuestSeccomp bool
// SandboxBindMounts - list of paths to mount into guest
SandboxBindMounts []string
// Experimental enables experimental features
Experimental []string

View File

@@ -62,15 +62,21 @@ type dialer func(string, time.Duration) (net.Conn, error)
// model, and mediates communication between AF_UNIX sockets (on the host end)
// and AF_VSOCK sockets (on the guest end).
// - mock://<path>. just for test use.
func NewAgentClient(ctx context.Context, sock string) (*AgentClient, error) {
func NewAgentClient(ctx context.Context, sock string, timeout uint32) (*AgentClient, error) {
grpcAddr, parsedAddr, err := parse(sock)
if err != nil {
return nil, err
}
dialTimeout := defaultDialTimeout
if timeout > 0 {
dialTimeout = time.Duration(timeout) * time.Second
agentClientLog.WithField("timeout", timeout).Debug("custom dialing timeout has been set")
}
var conn net.Conn
var d = agentDialer(parsedAddr)
conn, err = d(grpcAddr, defaultDialTimeout)
conn, err = d(grpcAddr, dialTimeout)
if err != nil {
return nil, err
}
@@ -92,7 +98,7 @@ func NewAgentClient(ctx context.Context, sock string) (*AgentClient, error) {
grpc.WithStreamInterceptor(otgrpc.OpenTracingStreamClientInterceptor(tracer)))
}
ctx, cancel := context.WithTimeout(ctx, defaultDialTimeout)
ctx, cancel := context.WithTimeout(ctx, dialTimeout)
defer cancel()
conn, err := grpc.DialContext(ctx, grpcAddr, dialOpts...)
if err != nil {

View File

@@ -768,8 +768,8 @@ func (q *qemu) setupVirtioMem() error {
if err != nil {
return err
}
// 1024 is size for nvdimm
sizeMB := int(maxMem) - int(q.config.MemorySize)
// backend memory size must be multiple of 2Mib
sizeMB := (int(maxMem) - int(q.config.MemorySize)) >> 2 << 2
share, target, memoryBack, err := q.getMemArgs()
if err != nil {
@@ -2043,8 +2043,9 @@ func (q *qemu) resizeMemory(ctx context.Context, reqMemMB uint32, memoryBlockSiz
var addMemDevice memoryDevice
if q.config.VirtioMem && currentMemory != reqMemMB {
q.Logger().WithField("hotplug", "memory").Debugf("resize memory from %dMB to %dMB", currentMemory, reqMemMB)
sizeByte := (reqMemMB - q.config.MemorySize) * 1024 * 1024
err := q.qmpMonitorCh.qmp.ExecQomSet(q.qmpMonitorCh.ctx, "virtiomem0", "requested-size", uint64(sizeByte))
sizeByte := uint64(reqMemMB - q.config.MemorySize)
sizeByte = sizeByte * 1024 * 1024
err := q.qmpMonitorCh.qmp.ExecQomSet(q.qmpMonitorCh.ctx, "virtiomem0", "requested-size", sizeByte)
if err != nil {
return 0, memoryDevice{}, err
}

View File

@@ -686,13 +686,13 @@ func (s *Sandbox) Delete(ctx context.Context) error {
for _, c := range s.containers {
if err := c.delete(ctx); err != nil {
return err
s.Logger().WithError(err).WithField("cid", c.id).Debug("failed to delete container")
}
}
if !rootless.IsRootless() {
if err := s.cgroupsDelete(); err != nil {
return err
s.Logger().WithError(err).Error("failed to cleanup cgroups")
}
}

View File

@@ -75,6 +75,8 @@ func (v *virtiofsd) getSocketFD() (*os.File, error) {
if err != nil {
return nil, err
}
// no longer needed since fd is a dup
defer listener.Close()
listener.SetUnlinkOnClose(false)
@@ -98,6 +100,7 @@ func (v *virtiofsd) Start(ctx context.Context) (int, error) {
if err != nil {
return 0, err
}
defer socketFD.Close()
cmd.ExtraFiles = append(cmd.ExtraFiles, socketFD)
@@ -128,7 +131,7 @@ func (v *virtiofsd) Start(ctx context.Context) (int, error) {
v.wait = waitVirtiofsReady
}
return pid, socketFD.Close()
return cmd.Process.Pid, nil
}
func (v *virtiofsd) Stop(ctx context.Context) error {

View File

@@ -63,11 +63,8 @@ readonly -a systemd_files=(
# Set a default value
AGENT_INIT=${AGENT_INIT:-no}
# Align image to (size in MB) according to different architecture.
case "$(uname -m)" in
aarch64) readonly mem_boundary_mb=16 ;;
*) readonly mem_boundary_mb=128 ;;
esac
# Align image to 128M
readonly mem_boundary_mb=128
# shellcheck source=../scripts/lib.sh
source "${lib_file}"

View File

@@ -2,7 +2,8 @@
* [How to use Kata workloads for `ccloudvm`](#how-to-use-kata-workloads-for-ccloudvm)
* [Create Docker\* and Kata Containers virtualized environment](#create-docker-and-kata-containers-virtualized-environment)
---
***
The [ccloudvm](https://github.com/intel/ccloudvm/) tool is a command
to create development and demo environments. The tool sets up these development

View File

@@ -18,7 +18,7 @@ spec:
katacontainers.io/kata-runtime: cleanup
containers:
- name: kube-kata-cleanup
image: katadocker/kata-deploy:2.1.0-rc0
image: katadocker/kata-deploy:2.1.1
imagePullPolicy: Always
command: [ "bash", "-c", "/opt/kata-artifacts/scripts/kata-deploy.sh reset" ]
env:

View File

@@ -0,0 +1,5 @@
bases:
- ../../base
patchesStrategicMerge:
- mount_k3s_conf.yaml

View File

@@ -0,0 +1,17 @@
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: kubelet-kata-cleanup
namespace: kube-system
spec:
template:
spec:
containers:
- name: kube-kata-cleanup
volumeMounts:
- name: containerd-conf
mountPath: /etc/containerd/
volumes:
- name: containerd-conf
hostPath:
path: /var/lib/rancher/k3s/agent/etc/containerd/

View File

@@ -16,7 +16,7 @@ spec:
serviceAccountName: kata-label-node
containers:
- name: kube-kata
image: katadocker/kata-deploy:2.1.0-rc0
image: katadocker/kata-deploy:2.1.1
imagePullPolicy: Always
lifecycle:
preStop:

View File

@@ -0,0 +1,41 @@
From 0a153bdd2c31932c8357970c8f71f710dbfc7d11 Mon Sep 17 00:00:00 2001
From: Yuanzhe Liu <yuanzheliu09@gmail.com>
Date: Wed, 19 May 2021 14:17:03 +0000
Subject: [PATCH] qemu: align before memory hotplug on arm64
When hotplug memory on arm64 in kata, kernel will shout:
[ 0.396551] Block size [0x40000000] unaligned hotplug range: start 0xc8000000, size 0x40000000
[ 0.396556] acpi PNP0C80:01: add_memory failed
[ 0.396834] acpi PNP0C80:01: acpi_memory_enable_device() error
[ 0.396948] acpi PNP0C80:01: Enumeration failure
It means that kernel will check if the memory range to be hotplugged
align with 1G before plug the memory. So we should twist the qemu to
make sure the memory range align with 1G to pass the kernel check.
Signed-off-by: Yuanzhe Liu <yuanzheliu09@gmail.com>
---
hw/arm/virt.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 09eb6c06e4..513f191dfe 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -2217,8 +2217,11 @@ static void virt_memory_plug(HotplugHandler *hotplug_dev,
MachineState *ms = MACHINE(hotplug_dev);
bool is_nvdimm = object_dynamic_cast(OBJECT(dev), TYPE_NVDIMM);
Error *local_err = NULL;
+ PCDIMMDevice *dimm = PC_DIMM(dev);
- pc_dimm_plug(PC_DIMM(dev), MACHINE(vms), &local_err);
+ dimm->addr = QEMU_ALIGN_UP(dimm->addr, 0x40000000);
+
+ pc_dimm_plug(dimm, MACHINE(vms), &local_err);
if (local_err) {
goto out;
}
--
2.20.1

View File

@@ -59,6 +59,9 @@ ARG QEMU_VERSION
RUN git checkout "${QEMU_VERSION}"
RUN git clone https://github.com/qemu/capstone.git capstone
RUN git clone https://github.com/qemu/keycodemapdb.git ui/keycodemapdb
RUN git clone https://github.com/qemu/meson.git meson
RUN git clone https://github.com/qemu/berkeley-softfloat-3.git tests/fp/berkeley-softfloat-3
RUN git clone https://github.com/qemu/berkeley-testfloat-3.git tests/fp/berkeley-testfloat-3
ADD scripts/configure-hypervisor.sh /root/configure-hypervisor.sh
ADD qemu /root/kata_qemu

View File

@@ -203,7 +203,7 @@ externals:
uscan-url: >-
https://github.com/kubernetes/kubernetes/tags
.*/v?([\d\.]+)\.tar\.gz
version: "1.21.0-00"
version: "1.21.1-00"
runc:
description: "OCI CLI reference runtime implementation"