Commit Graph

2078 Commits

Author SHA1 Message Date
Archana Shinde
ebd9fcc2c3 actions: Run static checks before make agent
Run static checks prior to building the agent.Checks
fail if run after since the compilation process
produces new rust code.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2021-01-08 11:04:54 -06:00
Fabiano Fidêncio
ce27c00ee2 Merge pull request #1217 from snir911/fix_hanging_pods
shimv2: Avoid double removing of container from sandbox
2021-01-08 15:00:54 +01:00
Eric Ernst
542e93d987 Merge pull request #1180 from egernst/qemu-cleanup-check
qemu: no state to save if QEMU isn't running
2021-01-06 11:17:54 -08:00
Snir Sheriber
5c464018ed shimv2: Avoid double removing of container from sandbox
RemoveContainerRequest results in calling to deleteContainer, according
to spec calling to RemoveContainer is idempotent and "must not return
an error if the container has already been removed", hence, don't
return error if the error reports that the container is not found.

Fixes: #836

Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2020-12-27 18:04:06 +02:00
Eric Ernst
9a7bcccc8e qemu: no state to save if QEMU isn't running
On pod delete, we were looking to read files that we had just deleted. In particular,
stopSandbox for QEMU was called (we cleanup up vmpath), and then QEMU's
save function was called, which immediately checks for the PID file.

Let's only update the persist store for QEMU if QEMU is actually
running. This'll avoid Error messages being displayed when we are
stopping and deleting a sandbox:

```
level=error msg="Could not read qemu pid file"
```

I reviewed CLH, and it looks like it is already taking appropriate
action, so no changes needed.

Ideally we won't spend much time saving state to persist.json unless
there's an actual error during stop/delete/shutdown path, as the persist will
also be removed after the pod is removed. We may want to optimize this,
as currently we are doing a persist store when deleting each container
(after the sandbox is stopped, VM is killed), and when we stop the sandbox.
This'll require more rework... tracked in:
  https://github.com/kata-containers/kata-containers/issues/1181

Fixes: #1179

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2020-12-21 11:29:44 -08:00
David Gibson
e004616b02 runtime/network: Fix error reporting in listRoutes()
If the upcast from resultingRoutes to *grpc.IRoutes fails, we return
(nil, err), but previous code ensures that err is nil at that point, so we
return no error.

fixes #1206

Forward port of
0ffaeeb5d8

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2020-12-18 14:36:09 +11:00
David Gibson
1ae8e81abb runtime/network: Correct error reporting in listInterfaces()
If the upcast from resultingInterfaces to *grpc.Interfaces fails, we
return (nil, err), but previous code ensures that err is nil at that
point, so we return no error.

Forward port of
b86e904c2d

fixes #1206

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2020-12-18 14:35:50 +11:00
Bin Liu
caa6965c17 Merge pull request #1183 from wainersm/runtime_destdir
runtime: Allow to overwrite DESTDIR
2020-12-17 14:10:56 +08:00
David Gibson
a19263e58d agent/protocols: Remove unneeded import from oci.proto
oci.proto imports "google/protobuf/wrappers.proto", but doesn't appear to
use it, which causes a warning from protoc when we compile it.  Remove the
import to fix the warning.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2020-12-17 13:06:41 +11:00
Wainer dos Santos Moschetta
10e9bfc6f7 runtime: Allow to overwrite DESTDIR
On runtime/Makefile the value of DESTDIR is set to "/", unless one
pass that variable as an argument to `make`. This change will
allow its overwrite if DESTDIR is exported in the environment as
well.

Fixes #1182

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2020-12-09 09:04:04 -05:00
Chelsea Mafrica
49e7151d3d shimv2: Add tracing
Add trace calls to shimv2 that create spans for functions in service.go.
Tracing starts in New(), which is forked twice and is followed by either
StartShim() or Create().

Tracing cannot start without the value for Trace enabled from the
runtime config so load the config in New(), which results in it being
loaded every time New() is called in addition to where it is originally
loaded after Create().

Fixes #903

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2020-12-04 19:38:44 -08:00
Fabiano Fidêncio
f7383ef835 Merge pull request #1166 from cmaf/fix-ctx-port
shimv2: handle ctx passed by containerd
2020-12-03 19:45:52 +01:00
Bin Liu
4e0a7e31f9 Merge pull request #1103 from likebreath/1111/clh_fix_cleanupVM
runtime: clh: Enforce to call 'cleanupVM' for 'stopSandbox'
2020-12-03 17:34:26 +08:00
Chelsea Mafrica
0155fe1260 shimv2: handle ctx passed by containerd
Sometimes shim process cannot be shutdown because of container list
is not empty. This container list is written in shim service, while
creating container. We find that if containerd cancel its Create
Container Request due to timeout, but runtime didn't handle it properly
and continue creating action, then this container cannot be deleted at
all. So we should make sure the ctx passed to Create Service rpc call
is effective.

Fixes #1088

Signed-off-by: Yves Chan <shanks.cyp@gmail.com>
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2020-12-02 14:28:31 -08:00
Bo Chen
647331ace6 runtime: clh: Enforce to call 'cleanupVM' for 'stopSandbox'
We should always cleanup the vm directory when doing `stopSandbox`,
while we are skipping the cleanup process on some error code paths when
using cloud-hypervisor driver.

Fixes: #1098

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-12-01 17:27:44 -08:00
Eric Ernst
2f1cb7995f kata-monitor: allow for building for alpine
- add a reference Dockerfile to tools
- update kata-monitor build to:
  1) utilize the kata buildflags, which were dropped before
  2) disable CGO, so we have option for building in alpine

From root of the repository, example build:
 $ docker build -f tools/packaging/kata-monitor/Dockerfile -t kata-monitor .

Fixes: #1135

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2020-12-01 10:28:59 -08:00
Julio Montes
70f198d78e cli: check modules and permissions before loading a module
Before loading a module, the check subcommand should check if the
current user can load it.

fixes #3085

Signed-off-by: Julio Montes <julio.montes@intel.com>
2020-11-26 11:55:42 -06:00
Julio Montes
cb684cf8ea cli: don't fail if rate limit is exceeded
Don't fail if rate limit is exceeded since this is a
limitation/restriction of Github not a problem in the host.
Print a warning when the rate limit is exceeded.

For more information about Github's rate limit, see
https://developer.github.com/v3/#rate-limiting

Signed-off-by: Julio Montes <julio.montes@intel.com>
2020-11-26 11:50:14 -06:00
bin liu
c388ec5bef runtime: don't wait the second shim process in shim start
In first shim v2 startup(with `start` command-line option), it will start
the second shim v2 process running as ttrpc server, there is no needs to
wait the second process, because the current shim v2 process will exit immediately.

Fixes: #1127

Signed-off-by: bin liu <bin@hyper.sh>
2020-11-18 17:18:59 +08:00
Julio Montes
1dd77e204f Merge pull request #1120 from liubin/fix/1119-revert-cleanupcontainer-api
virtcontainers: revert CleanupContainer from PR 1079
2020-11-17 09:11:29 -06:00
bin liu
fdbf7d3222 virtcontainers: revert CleanupContainer from PR 1079
In PR 1079, CleanupContainer's parameter of sandboxID is changed to VCSandbox, but at cleanup,
there is no VCSandbox is constructed, we should load it from disk by loadSandboxConfig() in
persist.go. This commit reverts parts of #1079

Fixes: #1119

Signed-off-by: bin liu <bin@hyper.sh>
2020-11-17 10:31:33 +08:00
James O. D. Hunt
91a390f072 docs: Create hypervisor summary document
Split some of the core hypervisor details out of the virtualisation
document and present in a simpler fashion for new users.

Fixes: #1063.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2020-11-16 11:52:40 +00:00
Tim Zhang
06b9294c7d Merge pull request #1110 from liubin/fix/1109-add-enable_pprof
runtime: change configuration key name from EnablePprof to enable_pprof
2020-11-13 17:44:34 +08:00
bin liu
14a21c3ab1 runtime: change configuration key name from EnablePprof to enable_pprof
Key name in configuration file is in snake case but not camel case.
And the key is processed as `enable_pprof` in code, the configuration
template file should replace `EnablePprof` it by `enable_pprof`

Fixes: #1109

Signed-off-by: bin liu <bin@hyper.sh>
2020-11-13 14:52:56 +08:00
bin liu
4e3a8c0124 runtime: remove global sandbox variable
Remove global sandbox variable, and save *Sandbox to hypervisor struct.
For some needs, hypervisor may need to use methods from Sandbox.

Signed-off-by: bin liu <bin@hyper.sh>
2020-11-13 09:47:09 +08:00
bin liu
290203943c runtime: delete sandboxlist.go and sandboxlist_test.go
Delete sandboxlist.go and sandboxlist_test.go under virtcontainers package.

Fixes: #1078

Signed-off-by: bin liu <bin@hyper.sh>
2020-11-13 09:47:09 +08:00
Julio Montes
36f65ce182 runtime: clh: update cloud-hypervisor
Update cloud-hypervisor to commit 2706319.
Fixes a limitation in OpenAPITools/openapi-generator tool,
it's impossible to send go zero types, like false and 0 to
cloud-hypervisor because `omitempty` is added if a field is not
required.
See cloud-hypervisor/cloud-hypervisor#1961 for more information

Signed-off-by: Julio Montes <julio.montes@intel.com>
2020-11-12 09:33:56 -06:00
Julio Montes
e1396f0402 runtime: clh: disable virtiofs DAX when FS cache size is 0
Guest consumes 120Mb more of memory when DAX is enabled and the default
FS cache size (8G) is used. Disable dax when it is not required
reducing guest's memory footprint.

Without this patch:

```
7fdea4000000-7fdee4000000 rw-s 18850589 /memfd:ch_ram (deleted)
Size:            1048576 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:              187876 kB
```

With this patch:

```
7fa970000000-7fa9b0000000 rw-s 612001  /memfd:ch_ram (deleted)
Size:            1048576 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:               57308 kB
Pss:               56722 kB
```

fixes #1100

Signed-off-by: Julio Montes <julio.montes@intel.com>
2020-11-12 09:33:56 -06:00
James O. D. Hunt
8f38265be4 release: Fix release candidate to major version upgrade check
Fix `kata-runtime kata-check`'s network version check which was failing
when the user was running a release candidate build and the latest
release was a major one, two examples of the error being:

- `BUG: unhandled scenario: current version: 1.12.0-rc0, latest version: 1.12.0`
- `BUG: unhandled scenario: current version: 2.0.0-rc0, latest version: 2.0.0`

Fixes: #1104.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2020-11-12 10:07:18 +00:00
James O. D. Hunt
2e0bf40adb tests: Ensure semver build metadata is ignored
According to the Semantic Versioning specification, build metadata must
be ignored for version comparisions, so add some explicit tests for this
scenario to `TestGetNewReleaseType()`.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2020-11-12 10:06:15 +00:00
James O. D. Hunt
4024a8274b release: Make error format string consistent
Use `%s` for both semver parameters.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2020-11-12 10:04:00 +00:00
Fupan Li
671a2be313 Merge pull request #1094 from liubin/fix/991
runtime: sleep 1 second after GetOOMEvent failed
2020-11-11 14:33:57 +08:00
Peng Tao
3c88106f65 Merge pull request #1084 from liubin/fix/1081-clean-codes
runtime: clean/refactor code
2020-11-11 10:09:10 +08:00
bin liu
cb0e6094ff runtime: sleep 1 second after GetOOMEvent failed
In some cases, for example agent crashed and not marked dead yet, the GetOOMEvent
will return errors like `connection reset by peer` or `ttrpc: closed`. Do a sleep
with 1 second (agent check interval) and let agent health check to do the check.

Fixes: #991

Signed-off-by: bin liu <bin@hyper.sh>
2020-11-10 12:02:31 +08:00
Bo Chen
359ab16a8f Merge pull request #1090 from likebreath/1106/clh_upgrade_v0.11.0
versions: Update cloud-hypervisor to release v0.11.0
2020-11-09 15:51:09 -08:00
bin liu
b8414045bf runtime: remove nsenter
remove code for nsenter

Fixes: #1081

Signed-off-by: bin liu <bin@hyper.sh>
2020-11-09 11:42:51 +08:00
bin liu
e3510be867 runtime: use one line if statement to check if err is nil for qemu.go
Use `if err := q.qmpSetup(); err != nil` to reduce code and make it easy
to read. And remove checking err if last function call also return an error,
return the function call directly.

Fixes: #1081

Signed-off-by: bin liu <bin@hyper.sh>
2020-11-09 11:42:45 +08:00
Fupan Li
d22c7cf00b Merge pull request #1013 from liubin/feature/1012-dump-guest-memroy-on-panic
Dump guest memory when kernel panic for QEMU
2020-11-09 09:46:28 +08:00
Bo Chen
92c1c4c690 versions: Update cloud-hypervisor to release v0.11.0
The release v0.11.0 of cloud-hypervisor features the following changes:
1) Improved Linux Boot Time, 2) `SIGTERM/SIGINT` Interrupt Signal,
Handling 3) Default Log Level Changed, 4) `io_uring` support by default
for `virtio-block` (on host kernel version 5.8+), 5) Windows Guest
Support, 6) New `--balloon` Parameter Added, 7) Experimental
`virtio-watchdog` Support, 8) Bug fixes.

Fixes: #1089

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-11-06 16:19:31 -08:00
Archana Shinde
6160043c01 Merge pull request #1077 from likebreath/1103/clh_refactor_device_unplug
clh: Consolidate the code path for device unplug
2020-11-06 16:00:56 -08:00
Peng Tao
c7a2b12fab Merge pull request #1086 from jodh-intel/2.0-dev-fix-annotations
annotations: Improve asset annotation handling
2020-11-06 10:29:22 +08:00
James O. D. Hunt
5ced96e96d hypervisor: Remove unused methods
Deleted `HypervisorConfig`'s unused  `CustomFirmwareAsset()` and
`JailerAssetPath()` methods.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2020-11-05 12:15:47 +00:00
James O. D. Hunt
e82c9daec3 annotations: Improve asset annotation handling
Make `asset.go` the arbiter of asset annotations by removing all asset
annotations lists from other parts of the codebase.

This makes the code simpler, easier to maintain, and more robust.

Specifically, the previous behaviour was inconsistent as the following
ways:

- `createAssets()` in `sandbox.go` was not handling the following asset
  annotations:

    - firmware:
      - `io.katacontainers.config.hypervisor.firmware`
      - `io.katacontainers.config.hypervisor.firmware_hash`

    - hypervisor:
      - `io.katacontainers.config.hypervisor.path`
      - `io.katacontainers.config.hypervisor.hypervisor_hash`

    - hypervisor control binary:
      - `io.katacontainers.config.hypervisor.ctlpath`
      - `io.katacontainers.config.hypervisor.hypervisorctl_hash`

    - jailer:
      - `io.katacontainers.config.hypervisor.jailer_path`
      - `io.katacontainers.config.hypervisor.jailer_hash`

- `addAssetAnnotations()` in the `oci` package was not handling the
  following asset annotations:

    - hypervisor:
      - `io.katacontainers.config.hypervisor.path`
      - `io.katacontainers.config.hypervisor.hypervisor_hash`

    - hypervisor control binary:
      - `io.katacontainers.config.hypervisor.ctlpath`
      - `io.katacontainers.config.hypervisor.hypervisorctl_hash`

    - jailer:
      - `io.katacontainers.config.hypervisor.jailer_path`
      - `io.katacontainers.config.hypervisor.jailer_hash`

This change fixes the bug where specifying a custom hypervisor path via an
asset annotation was having no effect.

Fixes: #1085.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2020-11-05 12:15:42 +00:00
James O. D. Hunt
0f26f1cd6f annotations: Add missing hypervisor control annotation
Add missing annotation definitions for a hypervisor control binary:

- `io.katacontainers.config.hypervisor.ctlpath`
- `io.katacontainers.config.hypervisor.hypervisorctl_hash`

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2020-11-05 12:12:58 +00:00
James O. D. Hunt
76064e3e2d asset: Formatting, grammar and whitespace
Improve formatting, grammar and whitespace.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2020-11-05 12:12:51 +00:00
bin liu
40418f6d88 runtime: add geust memory dump
When guest panic, dump guest kernel memory to host filesystem.
And also includes:
- hypervisor config
- hypervisor version
- and state of sandbox

Fixes: #1012

Signed-off-by: bin liu <bin@hyper.sh>
2020-11-05 16:04:21 +08:00
Peng Tao
a958eaa8d3 runtime: mount shared mountpoint readonly
bindmount remount events are not propagated through mount subtrees,
so we have to remount the shared dir mountpoint directly.

E.g.,
```
mkdir -p source dest foo source/foo

mount -o bind --make-shared source dest

mount -o bind foo source/foo
echo bind mount rw
mount | grep foo
echo remount ro
mount -o remount,bind,ro source/foo
mount | grep foo
```
would result in:
```
bind mount rw
/dev/xvda1 on /home/ubuntu/source/foo type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda1 on /home/ubuntu/dest/foo type ext4 (rw,relatime,discard,data=ordered)
remount ro
/dev/xvda1 on /home/ubuntu/source/foo type ext4 (ro,relatime,discard,data=ordered)
/dev/xvda1 on /home/ubuntu/dest/foo type ext4 (rw,relatime,discard,data=ordered)
```

The reason is that bind mount creats new mount structs and attaches them to different mount subtrees.
However, MS_REMOUNT only looks for existing mount structs to modify and does not try to propagate the
change to mount structs in other subtrees.

Fixes: #1061
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-11-04 17:51:49 +08:00
Peng Tao
125e21cea3 runtime: readonly mounts should be readonly bindmount on the host
So that we get protected at the VM boundary not just the guest kernel.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-11-04 17:51:49 +08:00
AIsland
b6f8a1d5af docs: Fix incorrect docs in config file
Correct the default configuration of [hypervisor.qemu] shared_fs in configuration-qemu.toml to virtio-fs in kata 2.0.

Fixes: #1054

Signed-off-by: AIsland <yuchunyu01@inspur.com>
2020-11-04 09:58:02 +08:00
Bo Chen
93d7962510 clh: Consolidate the code path for device unplug
In cloud-hypervisor, it provides a single unified way of unplugging
devices, e.g. the `/vm.RemoveDevice` HTTP API. Taking advantage of this
API, we can simplify our implementation of `hotplugRemoveDevice` in
`clh.go`, where we can consolidate similar code paths for different
device unplug (e.g. no need to implement `hotplugRemoveBlockDevice` and
`hotplugRemoveVfioDevice` separately). We will only need to retrieve the
right `deviceID` based on the type of devices, and use the single
unified HTTP API for device unplug.

Fixes: #1076

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-11-03 15:46:38 -08:00