Commit Graph

8543 Commits

Author SHA1 Message Date
snir911
18dc578134
Merge pull request #3999 from fgiudici/kata-monitor_fix_help
kata-monitor: fix duplicated output when printing usage
2022-03-30 18:56:59 +03:00
Francesco Giudici
a63bbf9793 kata-monitor: fix duplicated output when printing usage
(default: "/run/containerd/containerd.sock") is duplicated when
printing kata-monitor usage:

[root@kubernetes ~]# kata-monitor --help
Usage of kata-monitor:
  -listen-address string
        The address to listen on for HTTP requests. (default ":8090")
  -log-level string
        Log level of logrus(trace/debug/info/warn/error/fatal/panic). (default "info")
  -runtime-endpoint string
        Endpoint of CRI container runtime service. (default: "/run/containerd/containerd.sock") (default "/run/containerd/containerd.sock")

the golang flag package takes care of adding the defaults when printing
usage. Remove the explicit print of the value so that it would not be
printed on screen twice.

Fixes: #3998

Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
2022-03-30 11:58:53 +02:00
David Esparza
9e4ca0c4f8 doc: Improve kata-deploy README.md by changing sh blocks to bash blocks
The idea is to pass this README file to kata-doc-to-script.sh script and
then execute the result.

Added comments with a file name on top of each YAML snippet.
This helps in assigning a file name when we cat the YAML to a file.

Fixes: #3943

Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
2022-03-30 05:30:41 -04:00
Peng Tao
6837ab7213
Merge pull request #3989 from liubin/fix/3815-redue-oom-logs
Stop getting OOM events from agent for "ttrpc closed" error
2022-03-30 17:02:05 +08:00
snir911
f1a88371c8
Merge pull request #3991 from gkurz/fix-kata-deploy-binaries-sh
tools/packaging: Fix error path in `kata-deploy-binaries.sh -s`
2022-03-30 11:51:43 +03:00
Hui Zhu
e1a39bde8b
Merge pull request #3987 from bergwolf/kata-deploy
kata-deploy: fix version bump from -rc to stable
2022-03-30 16:13:27 +08:00
Fabiano Fidêncio
e1875d1879
Merge pull request #3974 from fidencio/wip/release-include-all-rust-vendored-code-to-the-vendored-tarball
release: Include all the rust vendored code into the vendored tarball
2022-03-29 23:25:17 +02:00
Greg Kurz
a779e19bee tools/packaging: Fix error path in 'kata-deploy-binaries.sh -s'
`make kata-tarball` relies on `kata-deploy-binaries.sh -s` which
silently ignores errors, and you may end up with an incomplete
tarball without noticing it because `make`'s exit status is 0.

`kata-deploy-binaries.sh` does set the `errexit` option and all the
code in the script seems to assume that since it doesn't do error
checking. Unfortunately, bash automatically disables `errexit` when
calling a function from a conditional pipeline, like done in the `-s`
case:

	if [ "${silent}" == true ]; then
		if ! handle_build "${t}" &>"$log_file"; then
                ^^^^^^
           this disables `errexit`

and `handle_build` ends with a `tar tvf` that always succeeds.

Adding error checking all over the place isn't really an option
as it would seriously obfuscate the code. Drop the conditional
pipeline instead and print the final error message from a `trap`
handler on the special ERR signal. This requires the `errtrace`
option as `trap`s aren't propagated to functions by default.

Since all outputs of `handle_build` are redirected to the build
log file, some file descriptor duplication magic is needed for
the handler to be able to write to the orignal stdout and stderr.

Fixes #3757

Signed-off-by: Greg Kurz <groug@kaod.org>
2022-03-29 19:00:46 +02:00
Greg Kurz
0baebd2b37 tools/packaging: Fix usage of kata-deploy-binaries.sh
Add missing documentation for -s .

Signed-off-by: Greg Kurz <groug@kaod.org>
2022-03-29 19:00:46 +02:00
GabyCT
2dc092fe60
Merge pull request #3947 from GabyCT/topic/removevpp
docs: Remove VPP documentation
2022-03-29 10:45:21 -06:00
Fabiano Fidêncio
3606923ac8 workflows,release: Ship *all* the rust vendored code
Instead of only vendoring the code needed by the agent, let's ensure we
vendor all the needed rust code, and let's do it using the newly
introduced enerate_vendor.sh script.

Fixes: #3973

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-03-29 12:37:00 +02:00
Fabiano Fidêncio
2eb07455d0 tools: Add a generate_vendor.sh script
This script is responsible for generating a tarball with all the rust
vendored code that is needed for fully building kata-containers on a
disconnected environment.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-03-29 12:36:36 +02:00
bin
5e1c30d484 runtime: add logs around sandbox monitor
For debugging purposes, add some logs.

Fixes: #3815

Signed-off-by: bin <bin@hyper.sh>
2022-03-29 16:59:12 +08:00
bin
fb8be96194 runtime: stop getting OOM events when ttrpc: closed error
getOOMEvents is a long-waiting call, it will retry when failed.
For cases of agent shutdown, the retry should stop.

When the agent hasn't detected agent has died, we can also check
whether the error is "ttrpc: closed".

Fixes: #3815

Signed-off-by: bin <bin@hyper.sh>
2022-03-29 16:39:01 +08:00
Peng Tao
93d03cc064 kata-deploy: fix version bump from -rc to stable
In such case, we should bump from "latest" tag rather than from
current_version.

Fixes: #3986
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2022-03-29 03:41:12 +00:00
Bin Liu
9495316145
Merge pull request #3962 from yaoyinnan/fix/3750-VirtioMem
runtime: Remove the explicit VirtioMem set and fix the comment
2022-03-29 10:20:05 +08:00
David Gibson
025fa60268
Merge pull request #3969 from gkurz/kata-deploy-copy-yq-installer
tools/packaging/kata-deploy: Copy install_yq.sh before starting parallel builds
2022-03-29 12:56:09 +11:00
Julio Montes
c9178b0750
Merge pull request #3981 from GabyCT/topic/removekata-proxy
docs: Remove kata-proxy references in documentation
2022-03-28 14:52:41 -06:00
Gabriela Cervantes
a931402375 docs: Remove kata-proxy references in documentation
This PR removes the kata-proxy references in VSocks documentation,
as this is not a component in kata 2.0 and all the examples that
were used belonged to kata 1.x.

Fixes #3980

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-03-28 16:36:22 +00:00
yaoyinnan
66f05c5bcb runtime: Remove the explicit VirtioMem set and fix the comment
Modify the 2Mib in the comment to 4Mib.
VirtioMem is set by configuration file or annotation. And setupVirtioMem is called only when VirtioMem is true.

Fixes: #3750

Signed-off-by: yaoyinnan <yaoyinnan@foxmail.com>
2022-03-28 21:21:38 +08:00
Yu Li
800e4a9cfb agent: use ms as unit of cputime instead of ticks
For the library `procfs`, the unit of values in `CpuTime` is ticks,
and we do not know how many ticks per second from metrics because the
`tps` in `CpuTime` is private.

But there are some implements in `CpuTime` for getting these values,
e.g., `user_ms()` for `user`, and `nice_ms()` for `nice`.  With these
values, accurate time can be obtained.

Fixes: #3979

Acked-by: zhaojizhuang <571130360@qq.com>
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
2022-03-28 19:30:09 +08:00
Peng Tao
e723dd5bba
Merge pull request #3955 from fengwang666/container-leak
agent: Signal the whole process group
2022-03-28 17:11:34 +08:00
Feng Wang
0928eb9f4e agent: Kill the all the container processes of the same cgroup
Otherwise the container process might leak and cause an unclean exit

Fixes: #3913

Signed-off-by: Feng Wang <feng.wang@databricks.com>
2022-03-27 10:06:58 -07:00
GabyCT
a07956a369
Merge pull request #3966 from devimc/2022-03-22/fixOsbuilderQAT
osbuilder/qat: don't pull kata sources if exist
2022-03-25 15:12:03 -06:00
Jakob Naucke
ff17c756d2
runtime: Allow and require no initrd for SE
Previously, it was not permitted to have neither an initrd nor an image.
However, this is the exact config to use for Secure Execution, where the
initrd is part of the image to be specified as `-kernel`. Require the
configuration of no initrd for Secure Execution.

Also
- remove redundant code for image/initrd checking -- no need to check in
  `newQemuHypervisorConfig` (calling) when it is also checked in
  `getInitrdAndImage` (called)
- use `QemuCCWVirtio` constant when possible

Fixes: #3922
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2022-03-25 18:36:12 +01:00
Julio Montes
c27963276b osbuilder/qat: don't pull kata sources if exist
don't pull kata sources if they already exist under GOPATH

fixes #3965

Signed-off-by: Julio Montes <julio.montes@intel.com>
2022-03-25 09:09:52 -06:00
Greg Kurz
154c8b03d3 tools/packaging/kata-deploy: Copy install_yq.sh in a dedicated script
'make kata-tarball' sometimes fails early with:

cp: cannot create regular file '[...]/tools/packaging/kata-deploy/local-build/dockerbuild/install_yq.sh': File exists

This happens because all assets are built in parallel using the same
`kata-deploy-binaries-in-docker.sh` script, and thus all try to copy
the `install_yq.sh` script to the same location with the `cp` command.
This is a well known race condition that cannot be avoided without
serialization of `cp` invocations.

Move the copying of `install_yq.sh` to a separate script and ensure
it is called *before* parallel builds. Make the presence of the copy
a prerequisite for each sub-build so that they still can be triggered
individually. Update the GH release workflow to also call this script
before calling `kata-deploy-binaries-in-docker.sh`.

Fixes #3756

Signed-off-by: Greg Kurz <groug@kaod.org>
2022-03-25 15:59:24 +01:00
David Gibson
1ed7da8fc7 packaging: Eliminate TTY_OPT and NO_TTY variables in kata-deploy
NO_TTY configured whether to add the -t option to docker run.  It makes no
sense for the caller to configure this, since whether you need it depends
on the commands you're running.  Since the point here is to run
non-interactive build scripts, we don't need -t, or -i either.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Greg Kurz <groug@kaod.org>
2022-03-25 15:52:02 +01:00
David Gibson
bad859d2f8 tools/packaging/kata-deploy/local-build: Add build to gitignore
This directory consists entirely of files built during a make kata-tarball,
so it should not be committed to the tree. A symbolic link to this directory
might be created during 'make tarball', ignore it as well.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
[greg: - rearranged the subject to make the subsystem checker happy
       - also ignore the symbolic link created by
         `kata-deploy-binaries-in-docker.sh`]
Signed-off-by: Greg Kurz <groug@kaod.org>
2022-03-25 15:52:02 +01:00
James O. D. Hunt
486322a0f1
Merge pull request #3930 from liubin/fix/3929-doc-for-dind
docs: fix markdown issues in how-to-run-docker-with-kata.md
2022-03-25 10:49:19 +00:00
Feng Wang
19f372b5f5 runtime: Add more debug logs for container io stream copy
This can help debugging container lifecycle issues

Fixes: #3913

Signed-off-by: Feng Wang <feng.wang@databricks.com>
2022-03-24 21:35:16 -07:00
GabyCT
4776e346a0
Merge pull request #3952 from devimc/2022-03-23/fixQATCI
osbuilder/qat: use centos as base OS
2022-03-24 10:10:52 -06:00
Julio Montes
459f4bfedb osbuilder/qat: use centos as base OS
move away from ubuntu, since now it's easier to build using
CentOS as base OS

fixes #3936

Signed-off-by: Julio Montes <julio.montes@intel.com>
2022-03-24 08:18:29 -06:00
Peng Tao
853dd98b7b
Merge pull request #3951 from GabyCT/topic/vcpusdoc
docs: Update vcpu handling document
2022-03-24 16:02:59 +08:00
Peng Tao
098374b179
Merge pull request #3934 from dcmiddle/fix-agent-check
Agent: fix unneeded late initialization lint
2022-03-24 16:02:11 +08:00
GabyCT
d9cd8cde2b
Merge pull request #3909 from fidencio/wip/clh-allow-testing-a-specific-pr
static-build,clh: Add the ability to build from a PR
2022-03-23 15:24:34 -06:00
Gabriela Cervantes
9a5b477062 docs: Update vcpu handling document
This PR updates the vcpu handling document by removing docker information
which is not longer being used in kata 2.x and leaving only k8s information.

Fixes #3950

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-03-23 17:58:49 +00:00
Fabiano Fidêncio
7a8b96b857
Merge pull request #3942 from dgibson/kata1420
Don't use a globally installed mock hook for hook tests
2022-03-23 17:57:16 +01:00
Gabriela Cervantes
ecf71d6dd6 docs: Remove VPP documentation
This PR is removing VPP documentation as it is not longer valid with
kata 2.x, all the instructions were used for kata 1.x

Fixes #3946

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-03-23 15:50:37 +00:00
David Gibson
c77e34de33 runtime: Move mock hook source
src/runtime/virtcontainers/hook/mock contains a simple example hook in Go.
The only thing this is used for is for some tests in
src/runtime/pkg/katautils/hook_test.go.  It doesn't really have anything
to do with the rest of the virtcontainers package.

So, move it next to the test code that uses it.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-03-23 19:37:35 +11:00
David Gibson
86723b51ae virtcontainers: Remove unused install/uninstall targets
We've now removed the need to install the mock hook binary for unit tests.
However, it turns out that managing that was the *only* thing that the
install and uninstall targets in the virtcontainers Makefile handled.

So, remove them.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-03-23 19:37:18 +11:00
David Gibson
0e83c95fac virtcontainers: Run mock hook from build tree rather than system bin dir
Running unit tests should generally have minimal dependencies on
things outside the build tree.  It *definitely* shouldn't modify
system wide things outside the build tree.  Currently the runtime
"make test" target does so, though.

Several of the tests in src/runtime/pkg/katautils/hook_test.go require a
sample hook binary.  They expect this hook in
/usr/bin/virtcontainers/bin/test/hook, so the makefile, as root, installs
the test binary to that location.

Go tests automatically run within the package's directory though, so
there's no need to use a system wide path.  We can use a relative path to
the binary build within the tree just as easily.

fixes #3941

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-03-23 19:34:50 +11:00
bin
7743486413 docs: fix markdown issues in how-to-run-docker-with-kata.md
Some links in how-to-run-docker-with-kata.md is not correct,
and some typos.

Fixes: #3929

Signed-off-by: bin <bin@hyper.sh>
2022-03-23 08:15:02 +08:00
Dan Middleton
32131cb8ba Agent: fix unneeded late initialization lint
Clippy v1.58 added needless_late_init

Fixes #3933

Signed-off-by: Dan Middleton <dan.middleton@intel.com>
2022-03-22 10:17:24 -05:00
David Gibson
e65db838ff virtcontainers: Remove VC_BIN_DIR
The VC_BIN_DIR variable in the virtcontainers Makefile is almost unused.
It's used to generate TEST_BIN_DIR, and it's created in the install target.
However, we also create TEST_BIN_DIR, which is a subdirectory of VC_BIN_DIR
with mkdir -p, so it will necessarily create VC_BIN_DIR along the way.

So we can drop the unnecessary mkdir and expand the definition of
VC_BIN_DIR in the definition of TEST_BIN_DIR.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-03-22 16:53:59 +11:00
David Gibson
c20ad2836c virtcontainers: Remove unused Makefile defines
The INSTALL_EXEC and UNINSTALL_EXEC definitions from the virtcontainers
Makefile (unlike those from the runtime Makefile in the parent directory)
are entirely unused.  Remove them.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-03-22 16:40:57 +11:00
David Gibson
c776bdf4a8 virtcontainers: Remove unused parameter from go-test.sh
The check-go-test target passes the path to the mock hook test binary to
go-test.sh when it invokes it.  But go-test.sh just calls run_go_test from
ci/lib.sh, which invokes a script from the tests repo *without* any
parameters.

That is, this parameter is ignored anyway, so remove it.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-03-22 16:39:22 +11:00
Fabiano Fidêncio
aa6886f1ed
Merge pull request #2482 from Bevisy/main-815
ci: Weekly check whether the docs url is alive
2022-03-21 17:15:40 +01:00
James O. D. Hunt
3edf25b6c9
Merge pull request #3682 from Jakob-Naucke/cross
Multistrap Ubuntu & enable cross-building guest
2022-03-21 11:11:47 +00:00
James O. D. Hunt
f8fb0d3bb6
Merge pull request #3322 from Kvasscn/kata_dev_block_driver_option
device: using const strings for block-driver option instead of hard coding
2022-03-21 10:56:25 +00:00