Commit Graph

6037 Commits

Author SHA1 Message Date
Tim Zhang
b58fb25d88
Merge pull request #1555 from liubin/fix/1554-install-hook-before-test
test: install mock hook binary before test
2021-03-30 14:01:56 +08:00
Eric Ernst
05680b86c4
Merge pull request #1537 from lifupan/main
cgroups: fix the issue of get wrong online cpus
2021-03-29 15:56:03 -07:00
Eric Ernst
460117a1a6
Merge pull request #1510 from littlejawa/issue_1003
build: remove unused variables from Makefile
2021-03-29 14:54:09 -07:00
Eric Ernst
24214a536a
Merge pull request #1560 from egernst/fix-1559
container: on cleanup, rm container directory for mounts path
2021-03-29 14:14:52 -07:00
GabyCT
17840cb573
Merge pull request #1546 from devimc/2021-03-24/supportQEMU6
runtime: add support for QEMU 6
2021-03-29 14:33:16 -06:00
Eric Ernst
6dfe3acf93
Merge pull request #1535 from jodh-intel/agent-shutdown
agent: Enable clean shutdown
2021-03-29 12:26:01 -07:00
Eric Ernst
9a4e866654 container: on cleanup, rm container directory for mounts path
A wrong path was being used for container directory when
virtiofs is utilized. This resulted in a warning message in
logs when a container is killed, or completes:

level=warning msg="Could not remove container share dir"

Without proper removal, they'd later be cleaned up when the shared
path is removed as part of stopping the sandbox.

Fixes: #1559

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-03-29 11:39:39 -07:00
bin
48e5e4f2f3 test: install mock hook binary before test
`make test` depends mock hook in virtcontainers directory,
before test, install it first.

And also run test as normal user and root in GitHub actions.

Fixes: #1554

Signed-off-by: bin <bin@hyper.sh>
2021-03-29 22:40:45 +08:00
James O. D. Hunt
1d448813a1 uevent: Add shutdown channel for task
Allow the uevent task to shutdown on request.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:12 +01:00
James O. D. Hunt
d8d5b4cd1d signal: Move to a new module
Move the signal handling code into a new module and refactor into the
main handler and a new SIGCHLD handling function to make the code
simpler and easier to understand.

Also added a unit test for shutdown.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:12 +01:00
James O. D. Hunt
011f7d785a logging: Rework for shutdown
Make changes to logger thread to allow the logger to be replaced with
a NOP logger (required for agent shutdown).

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:12 +01:00
James O. D. Hunt
7d5f88c0ad agent: Enable clean shutdown
The agent doesn't normally shutdown: it doesn't need to be as it is
killed *after* the workload has finished. However, a clean and ordered
shutdown sequence is required to support agent tracing, since all trace
spans need to be completed to ensure a valid trace transaction.

Enable a controlled shutdown by allowing the main threads (tasks) to be
stopped.

To allow this to happen, each thread is now passed a shutdown channel
which it must listen to asynchronously, and shut down the thread if
activity is detected on that channel.

Since some threads are created for I/O and since the standard `io::copy`
cannot be stopped, added a new `interruptable_io_copier()` function
which shares the same semantics as `io::copy()`, but which is also
passed a shutdown channel to allow asynchronous I/O operations to be
stopped cleanly.

Fixes: #1531.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:12 +01:00
James O. D. Hunt
dcb39c61f1 main: Create logger task
Encapsulate the logic for handling the task that displays logger output
into a new function to simplify the code and remove another anonymous
async block.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:11 +01:00
James O. D. Hunt
2cf2897d31 main: Use task list for stopping tasks
Maintain a list of tasks and wait on them all before main returns.

This is preparatory work for the agent shutdown: all tasks that are
started need to be added to the list. This aggregation makes it easier
to identify what needs to stop before the agent can exit cleanly.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:11 +01:00
James O. D. Hunt
039df1d727 main: Refactor main logic into new async function
Move most of the main logic into a separate async function. This makes
the code clearer and avoids the anonymous async block.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:11 +01:00
James O. D. Hunt
2a648fa760 logging: Use guard to make threaded logging safe
Return a guard variable from `create_logger()` which the caller can
implicitly drop to guarantee that all threads started by the async log
drain are stopped.

This fixes a long-standing bug [1] whereby the agent could panic with
the following error, generated by the `slog` logging crate:

```
slog::Fuse Drain: Custom { kind: Other, error: "serde serialization error: Bad file descriptor (os error 9)" }
```

[1] - See https://github.com/kata-containers/kata-containers/issues/171.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:11 +01:00
James O. D. Hunt
38f0d8d3ce config: Fix assert_error testing macro
Fixed the `assert_error!()` test macro so that it correctly handles the
scenario where the test expects an error, but the actual result was `Ok`
(no error).

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:11 +01:00
Bin Liu
594c47ab6c
Merge pull request #1553 from bergwolf/ro-volumes
runtime: fix virtiofsd RO volume sharing
2021-03-29 20:43:34 +08:00
fupan.lfp
3f46e6379d cgroups: fix the issue of getting wrong online cpus
It's better to get the online cpus from
"/sys/devices/system/cpu/online" instead of from
cpuset cgroup, cause there would be an latency
between one cpu online and present in the root
cpuset cgroup.

Fixes: #1536

Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
2021-03-29 15:49:15 +08:00
Fupan Li
6932ac6522
Merge pull request #1460 from liubin/fix/1455
runtime: update virtcontainers API documentation
2021-03-29 15:24:41 +08:00
Peng Tao
e34924488b runtime: fix virtiofsd RO volume sharing
Right now we rely heavily on mount propagation to share host
files/directories to the guest. However, because virtiofsd
pivots and moves itself to a separate mount namespace, the remount
mount is not present in virtiofsd's mount. And it causes guest to be
able to write to the host RO volume.

To fix it, create a private RO mount and then move it to the host mounts
dir so that it will be present readonly in the host-guest shared dir.

Fixes: #1552
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-29 13:54:25 +08:00
bin
532ff7c909 runtime: update virtcontainers API documentation
Virtcontainers API documentation is outdated, update documentation from the latest
source.

Fixes: #1455

Signed-off-by: bin <bin@hyper.sh>
2021-03-29 11:50:53 +08:00
Chelsea Mafrica
b9489e6c6e
Merge pull request #1551 from cmaf/fix-span-ordering-static-check
runtime: Fix trace span ordering and static checks
2021-03-26 16:35:23 -07:00
Chelsea Mafrica
6fcfea8dcf runtime: Fix static check errors
Fix comment formatting and unused variable to make static checks pass.

Fixes #1550

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2021-03-25 12:59:01 -07:00
Chelsea Mafrica
f3ebbb1f1a runtime: Fix trace span ordering
Return ctx in trace() functions to correct span ordering.

Fixes #1550

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2021-03-25 11:43:04 -07:00
Bin Liu
5b5b5cc611
Merge pull request #1539 from bergwolf/ut
fix runtime UTs and enable static check
2021-03-25 16:29:45 +08:00
Fabiano Fidêncio
deca207608
Merge pull request #1542 from fidencio/wip/kata-deploy-use-the-correct-image-for-alpha
kata-deploy: Use the correct tag for 2.1-alpha1 release
2021-03-25 08:32:06 +01:00
Carlos Venegas
753c7270ed
Merge pull request #1422 from jcvenegas/2021-02-17/experimental-virtiofs
ci: fix virtiofs-experimental build
2021-03-24 14:47:27 -06:00
Carlos Venegas
5a3ee7d7e9 snap: Use qemu.version to build snap
Use only one key to refer the version used by kata.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-24 17:52:52 +00:00
Carlos Venegas
0f78a5dc96 kernel: rename exeperimental kernel symlink.
use -experimental suffix instead of virtiofs

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-24 17:52:52 +00:00
Carlos Venegas
f79105231c qemu: Build experimental qemu.
Split qemu script to build qemu experimental using
same dockerfile.

Fixes: #1421

Depends-on: github.com/kata-containers/tests#3255

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-24 17:52:07 +00:00
Julio Montes
1555bfd8b5 runtime: add support for QEMU 6
Use `on` and `off` to enable or disable features,
`no` prefix is deprecated

fixes #1545

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-03-24 10:55:35 -06:00
James O. D. Hunt
2fc7f75724
Merge pull request #1521 from jodh-intel/verify-cid
Verify container ID
2021-03-24 13:27:58 +00:00
Peng Tao
fc0f93aef9 actions: enable unit tests in PR check
Right now we only run UTs for agent. We need to run it for *ALL*
components.

Fixes: #1538
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-24 20:12:14 +08:00
Peng Tao
74192d179d runtime: fix static check errors
It turns out we have managed to break the static checker in many
difference places with the absence of static checker in github action.
Let's fix them while enabling static checker in github actions...

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-24 20:10:19 +08:00
Peng Tao
a2dee1f6a0 runtime: fix vm factory UT failure
We need to use different mocked socket otherwise they conflict with each
other.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-24 18:21:21 +08:00
Peng Tao
076bc5075f agent-ctl: update Cargo.lock
Just build would result in these diffs. Let's include them in git.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-24 15:03:52 +08:00
Peng Tao
0153f76b07 runtime: gofmt code
Looks like we have merged a lot of code that is not properly formated.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-24 14:37:46 +08:00
Bin Liu
018454be44
Merge pull request #1534 from Tim-Zhang/rework-execute_hook
rustjail: rework execute_hook
2021-03-24 14:09:09 +08:00
Fabiano Fidêncio
60f6315b2d kata-deploy: Use the correct tag for 2.1-alpha1 release
Let's ensure we use the appropriate tag for the release, even before it
was actually created.

Fixes: #1493

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-24 00:29:03 +01:00
Carlos Venegas
b0e51e59fa qemu: Improve cache build
Add arguments and files as needed, if only of them
changes the build will start from the change and
not from scratch.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-23 20:52:39 +00:00
Carlos Venegas
bc587da9f5 qemu: Add suffix for qemu binaries.
To build different qemu versions with the
same qemu code add a prefix on install.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-23 20:52:39 +00:00
Carlos Venegas
5493517b4f qemu: add CACHE_TIMEOUT
Add docker ARG to provide a date to invalid cache, if the date changes
the image will be rebuild. This is required to keep build dependencies
with security fixes, but still take advantage of build qemu faster using
docker cache.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-23 20:52:39 +00:00
Carlos Venegas
98d01ce6b9 qemu: Apply patches for specific versions.
Today we apply patches per base branch. Having
two qemu versions in a similar base version can make
can have problems if one of the trees already has a patch.
If a patch is needed only for one specific tag/commit
add only the patch to that version.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-23 20:52:39 +00:00
Tim Zhang
40861fbab5
Merge pull request #1517 from jodh-intel/agent-server-address-cmdline
agent: Allow server address to be specified on kernel command-line
2021-03-23 19:33:25 +08:00
Peng Tao
190f813427 runtime/katautils: PFlash should be initialized
newQemuHypervisorConfig() sets it to an empty slice. We have to set the
same in the test config otherwise it is nil and reflect DeepEqual would
fail.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-23 17:49:47 +08:00
Peng Tao
b2ec5a43d5 runtime: fix cleanupSandboxBindMounts panic
Found in UT:
--- FAIL: TestKataCleanupSandbox (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-23 16:44:47 +08:00
Peng Tao
9b689ea1d7 runtime/cli: fix TestMainBeforeSubCommandsLoadConfigurationFail failure
Now that it is `kata-config`...

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-23 16:40:38 +08:00
Peng Tao
8e71c4fc7a runtime: fix missing context argument in mocked sandbox APIs
Missing context.Context in several APIs.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-23 16:19:46 +08:00
Peng Tao
8ff62beeb4 runtime: fix vcmock build failure
github.com/kata-containers/kata-containers/src/runtime/virtcontainers/pkg/vcmock
virtcontainers/pkg/vcmock/container.go:19:10: cannot use c.MockSandbox
(type *Sandbox) as type virtcontainers.VCSandbox in return argument:
        *Sandbox does not implement virtcontainers.VCSandbox (missing
	GetHypervisorPid method)
github.com/kata-containers/kata-containers/src/runtime/pkg/katautils

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-23 15:57:07 +08:00