Commit Graph

5998 Commits

Author SHA1 Message Date
fupan.lfp
81607e348e rustjail: fix the issue of home_dir function
Since the crate dirs::home_dir function depends on the
libc's api: getpwuid_r, but this api function wouldn't
be static linked on glibc, thus we'd better to figure
out an alternative way to get the home dir from /etc/passwd.
For much more info about this glibc's issue, please see:
https://sourceware.org/bugzilla/show_bug.cgi?id=19341.

This commit read and parse the "/etc/passwd" directly and
fetch the corresponding uid's home dir.

Fixes: #675

Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
2021-03-08 21:51:23 +08:00
James O. D. Hunt
83d5a49941
Merge pull request #1472 from jodh-intel/unbreak-agent-ctl
agent-ctl: Unbreak build
2021-03-08 10:32:22 +00:00
Fupan Li
f6630ddd49
Merge pull request #1478 from lifupan/fix_device
rustjail: fix the issue of bind mount device file from guest
2021-03-08 09:55:00 +08:00
James O. D. Hunt
c258ea25d4 agent-ctl: Function parameter cleanup
Remove unused function parameters from the following types:

- `AgentCmdFp`: Removed the config parameter and made
  the context parameter the first (à la golang).

- `BuiltinCmdFp`: Removed the config and options parameters.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-05 09:54:50 +00:00
James O. D. Hunt
fcd45def53 agent-ctl: Unbreak build
The recent switch to an async rust agent broke the `agent-ctl` tool.
However, we didn't notice because that isn't being built by the CI.

Fix the breakage by passing a ttRPC context to all ttRPC API calls and
also build the tool as part of the static checks CI.

Fixes: #1471.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-05 09:54:50 +00:00
James O. D. Hunt
efe625dfc1 build: Remove whitespace
Zap trailing whitespace.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-05 09:54:50 +00:00
Eric Ernst
48ed8f3c4a runtime: add support for readonly sandbox bindmounts
If specified, sandbox_bind_mounts identifies host paths to be
mounted (ro) into the sandboxes shared path. This is only valid
if filesystem sharing is utilized.

The provided path(s) will be bindmounted (ro) into the shared fs directory on
the host, and thus mapped into the guest. If defaults are utilized,
these mounts should be available in the guest at
`/var/run/kata-containers/shared/containers/sandbox-mounts`

These will not be exposed to the container workloads, and are only
added for potential guest-services to consume (example: expose certs
into the guest that are available on the host).

Fixes: #1464

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-03-04 10:04:25 -08:00
Carlos Venegas
7ae349c511 agent: makefile: Add codecov target
Add target to run codecov report locally.

Useful to identify what are the missing lines
to be covered by unit test.

Fixes: #1487

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-04 16:32:52 +00:00
Fabiano Fidêncio
acc4bc57f4
Merge pull request #1491 from jongwu/musl_tar
musl/arm64: decompression before use the tarball.
2021-03-04 08:58:03 +01:00
Jianyong Wu
f580d33cc9 musl/arm64: decompression before use the tarball.
In the last fix, the decompression ops is deleted by mistake and need
add it back.

Fixes: #1490
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-03-04 10:12:29 +08:00
GabyCT
ed3cb9bfa8
Merge pull request #1482 from jongwu/musl
osbuilder/arm64: build musl toolchain from source code if needed
2021-03-03 16:49:31 -06:00
Jianyong Wu
2da058ed7b osbuild: build musl toolchain from source if needed
Currently, musl toolchain installation on arm64 is just downloading from
a website. It's unsafe in case the website corrupts. So build musl
toolchain from source if it can't be downloaded.

Fixes: #1481
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-03-03 17:22:39 +08:00
Eric Ernst
6e1ef809e1
Merge pull request #1480 from wainersm/fix_shim_log
runtime: Fix missing 'name' field on containerd-shim-v2 logs
2021-03-02 11:46:09 -08:00
Adams, Eric
6417067d62 osbuilder: Port QAT Dockerfile to 2.0 repo
Update the Intel QAT Dockerfile to work with the 2.0 repos, fix some
bugs with building Debian/Ubuntu rootfs, and update the latest QAT
driver. Updated copyright.

Fixes: #1419

Signed-off-by: Adams, Eric <eric.adams@intel.com>
2021-03-01 17:34:04 -08:00
Wainer dos Santos Moschetta
85601cd360 snap: Update for QEMU 5.2.0
QEMU 5.2.0 needs ninja-build package installed on the build environment.

The default-configs were copied to $QEMU_SRC/default-configs but that does
take any effect, so instead it is now copied to $QEMU_SRC/default-configs/devices
and the configs for i386 were updated.

Also it had to change some arguments being passed to configure as Meson was failing
due inconsistent paths:

  ./meson.build:1:0: ERROR: The value of the 'libdir' option is '/usr/lib/qemu' which must be a subdir of the prefix '/snap/kata-containers/current/usr'.
  Note that if you pass a relative path, it is assumed to be a subdir of prefix.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-03-01 16:57:58 -05:00
Wainer dos Santos Moschetta
88cef33b76 versions: update QEMU to 5.2.0
This change the version of QEMU used in the tests and CI.

The scripts/configure-hypervisor.sh was changed so that:
  - Passing the `--enable-virtiofsd` flag
  - Do not compiling with -O3 to avoid the warning:

    Program python3 found: YES (/usr/bin/python3)
    ../meson.build:104: WARNING: Consider using the built-in optimization level instead of using "-O3".
    ../meson.build:108: WARNING: Consider using the built-in optimization level instead of using "-O3".

The qemu.blacklist files was changed so that new and uneeded firmware files are removed from the
final tarball. Except for qboot.rom which is new but kept, since it can be used with microvm
machine type (in case we want to enable microvm in the future).

The patches which are applied on QEMU sources:
 - 0001-virtiofsd-Allow-to-build-it-without-the-tools.patch
   (Build fix for Meson - allows passing `--disable-tools --enable-virtiofsd`)
 - 0002-virtiofsd-extract-lo_do_open-from-lo_open.patch
   0003-virtiofsd-optionally-return-inode-pointer-from-lo_do.patch
   0004-virtiofsd-prevent-opening-of-special-files-CVE-2020-.patch
   0005-virtiofsd-Add-_llseek-to-the-seccomp-whitelist.patch
   0006-virtiofsd-Add-restart_syscall-to-the-seccomp-whiteli.patch
   (Security fixes for virtiofsd)
 - 0007-9p-removing-coroutines-of-9p-to-increase-the-I-O-per.patch
   (Performance improvement for 9p driver)
 - 0008-hw-s390x-fix-build-for-virtio-9p-ccw.patch
   (Build fix for virtio-9p-ccw machine type)

Fixes: #1238

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-03-01 16:57:50 -05:00
Wainer dos Santos Moschetta
21bdaaf84f runtime: Fix missing 'name' field on containerd-shim-v2 logs
Each Kata Containers application should generate log records with a specified
structure. Currently on containerd-shim-v2's logs, the required 'name' field
is missing. This changed its logger to append the application name on each
and every emitted entries.

Fixes #1479
Related-to: github.com/kata-containers/tests/issues/3260
Suggested-by: James O. D. Hunt <james.o.hunt@intel.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-03-01 16:52:24 -05:00
Wainer dos Santos Moschetta
74a893f732 packaging: Refactor version comparisons on configure-hypervisor.sh
The scripts/configure-hypervisor.sh split the QEMU and GCC version
in major and minor versions then use those values on shell conditionals
to compare versions. This is error prone, so instead this change the script
to use the `sort -V -C ` command for version comparisons.

Fixes: #1349
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-03-01 14:46:24 -05:00
Eric Ernst
90a18e228b
Merge pull request #1457 from mxpv/shared
agent: don't error of virtiofs share is already mounted
2021-03-01 11:16:18 -08:00
fupan.lfp
34dc861cde rustjail: fix the issue of bind mount device file from guest
When do pass guest device files to container, the source
file wouldn't be a regular file, but we also need to create
a corresponding destination file to bind mount source file
to it. Thus it's better to check whether the source file
was a directory instead of regular file.

Fixes: #1477

Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
2021-03-01 21:20:01 +08:00
Bin Liu
61f0291d63
Merge pull request #1452 from lifupan/main
shimv2: return the hypervisor's pid as the container pid
2021-03-01 15:48:01 +08:00
Eric Ernst
ddb283dd43
Merge pull request #1475 from egernst/yiiiiikes
runtime: check if error loading runtime config
2021-02-26 15:01:22 -08:00
Eric Ernst
0f7098339b runtime: check if error loading runtime config
Looks like we inadvertantly removed the check on the loadRuntimeConfig
error return value. Adding back...

Fixes: #1474

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-02-26 10:51:38 -08:00
Bin Liu
7587d2a8d6
Merge pull request #1462 from Tim-Zhang/fix-clippy-for-rust1.5
agent: fix clippy for rustc 1.5
2021-02-26 15:52:03 +08:00
Fupan Li
b5282fa224
Merge pull request #1305 from Tim-Zhang/upgrade-tokio-to-1.0
agent: Upgrade tokio to 1.2.0
2021-02-26 13:33:24 +08:00
Tim Zhang
6f720761ed agent: fix clippy for rustc 1.5
Fixes: #1461

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-02-25 17:04:54 +08:00
Bin Liu
735fe3f94a
Merge pull request #1444 from ManaSugi/fix-blkio-weight
rustjail: fix blkio conversion
2021-02-25 15:20:20 +08:00
Tim Zhang
4a214720e8 agent: Fix test
Struct TtrpcContext has been changed in ttrpc@0.5

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-02-25 14:38:38 +08:00
Tim Zhang
02079dbb4f agent: upgrade tokio to 1.0
Fixes: #1257

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-02-25 14:38:32 +08:00
Eric Ernst
2f591f5642
Merge pull request #1459 from mxpv/pid
agent: Agent invokes OCI hooks with wrong PID
2021-02-24 20:08:56 -08:00
Maksym Pavlenko
a42dc74898 agent: Agent invokes OCI hooks with wrong PID
Agent sends -1 PID when invoking OCI hooks.

OCI state struct is initialized before obtaining PID, so this PR moves
`oci_state` call down, right after we get the id.

Fixes: #1458

Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-02-24 18:16:17 -08:00
Fupan Li
91ad176a06
Merge pull request #1434 from fidencio/wip/kata-deploy-cleanups
kata-deploy: stop mentioning qemu-virtiofsd, as the default qemu supports virtiofsd already (plus some cleanups)
2021-02-25 09:50:10 +08:00
Maksym Pavlenko
17e9a2cff5 agent: don't error of virtiofs share is already mounted
Port kata-containers/agent#883 to the Rust Agent.

In the event that the virtiofs device is already mounted at the
requested destination, don't error out. We'll check before attempting to
mount to see if the destination is already a mount point. If so, skip
doing the mount in the agent.

This facilitates mounting the sharedfs automatically in the guest before
the agent service starts.

Signed-off-by: Eric Ernst eric.g.ernst@gmail.com

Fixes: #1398

Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-02-24 17:05:59 -08:00
Tim Zhang
947913f6e5 agent/protocols: Remove cargo:rerun-if-changed in build.rs
So that the build.rs will be re-runed if any file
within the package is changed.

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-02-24 21:50:47 +08:00
Tim Zhang
2c42be0ca6
Merge pull request #1447 from liubin/liubin/fix-1369
agent: Stop receive message from Receiver if got None
2021-02-24 16:48:24 +08:00
Bin Liu
974f926292
Merge pull request #1450 from fidencio/wip/release-notes-script-still-mentions-docker
Remove "Docker" & "[kata] runtime" references from the release scripts & process documentation
2021-02-24 15:59:36 +08:00
fupan.lfp
bc0ac526a2 shimv2: return the hypervisor's pid as the container pid
Since the kata's hypervisor process is in the network namespace,
which is close to container's process, and some host metrics
such as cadvisor can use this pid to access the network namespace
to get some network metrics. Thus this commit replace the shim's
pid with the hypervisor's pid.

Fixes: #1451

Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
2021-02-24 13:26:05 +08:00
Fabiano Fidêncio
10ed3da4eb release: Rename runtime-release-notes to release-notes
There's no runtime repo anymore, let's avoid making a reference to it,
which may end up confusing people reading the Release-Process file.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-23 17:25:40 +01:00
Fabiano Fidêncio
f5dab6af2e release: We're not compatible with Docker.
We don't support Docker as part of 2.x repository.

Fixes: #1449

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-23 17:25:40 +01:00
Fabiano Fidêncio
2c8ea0a8d1 kata-deploy: Add copyright to the kata-deploy's Dockerfile
All the work done on this file, apart from merging the 2.x repo, and now
removing unused lines, comes from Intel.

The reason it's being added is to silent a complaint from the static
checker.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-23 14:25:00 +01:00
Fabiano Fidêncio
4e494e34c9 packaging: Remove NEMU mentions
There's no more NEMU, for some time already.  Considering this, let's
just remove any mention to it as part of our project.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-23 14:24:54 +01:00
Fabiano Fidêncio
f21c54a985 kata-deploy: QEMU, for 2.x, already includes virtiofs
There's no reason to ship qemu & qemu-virtiofs when the former already
includes vitiofs support (and that's the default for 2.x deployments).

In case we will enable experimental qemu DAX support, we should add a
new target, a "qemu-experimental" target, as Carlos has been working on.

Fixes: #1424

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-23 14:24:48 +01:00
Fabiano Fidêncio
657bd789a3 kata-deploy: Get rid of references to the docker script
The docker script has been removed as part of
62cbaf4de4, but references to it were left
behind in the artifact-list.sh, release/kata-deploy-binaries.sh, and
kata-deploy/Dockerfile.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-23 14:20:39 +01:00
Fabiano Fidêncio
81109f89bd
Merge pull request #1423 from fidencio/wip/kata-deploy-crio-plus-shimv2
kata-deploy: Ensure CRI-O uses shimv2 & the "vm" runtime type
2021-02-22 20:28:30 +01:00
Manabu Sugimoto
dcea08697a rustjail: fix blkio conversion
BFQ weight controller is using the same BFQ weight scheme (i.e 1->1000).
Therefore, there is no need to do the conversion.

More details here: https://github.com/opencontainers/runc/pull/2786

Fixes: #1440

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2021-02-23 00:26:57 +09:00
bin
bc34cbbce5 agent: Stop receive message from Receiver if got None
If the container has exited, the sender in notifier watching OOM events
will be dropped after the loop exited, and recv() from the according
receiver will get None.

This will lead two problems for get_oom_event rpc all from agent:

- return an wrong OOM event.
- continuously return OOM events.

Fixes: #1369

Signed-off-by: bin <bin@hyper.sh>
2021-02-22 21:56:07 +08:00
Fabiano Fidêncio
01481d6ac0 kata-deploy: Ensure CRI-O uses the VM runtime type
For Kata Containers 2.x, CRI-O should always be using the
`containerd-shim-kata-v2` binary, and always be configured to use the
"vm" runtime type, developed specifically for the shimv2, instead of the
default "oci" runtime type.

I've taken the liberty to try to simplify the CRI-O script and make it
less error prone.  In the future, we can start dropping a configuration
file to /etc/crio/crio.conf.d and just removing it as part of the
cleanup, but that's for the future.

Fixes: #1357

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-20 09:27:33 +01:00
Fabiano Fidêncio
d1c717363d kata-deploy: Move the containerd workarounds to their own functions
Factoring those pieces of code to their own functions allows us to
easily re-use them when creating & cleaning up the CRI-O configuration
files, as CRI-O is also affected by the issues that are still opened.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-20 09:27:24 +01:00
Fabiano Fidêncio
5013634e23 kata-deploy: Stop shipping kata-{clh,fc,qemu,qemu-virtiofs} binaries
Those binaries are not revelant for 2.x deployments.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-20 09:24:12 +01:00
Fabiano Fidêncio
2270f19ee1 kata-deploy: Update README to reflect the current distributed artifacts
Our list was based on what we used to ship for Kata Containers 1.x, not
even taking into account the shimv2 binary.

Let's update it in order to reflect better what we currently distribute.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-20 09:24:12 +01:00