Commit Graph

5350 Commits

Author SHA1 Message Date
Julio Montes
60770f4629
Merge pull request #566 from Jakob-Naucke/revendor-govmm-vfio-ap
runtime: Re-vendor GoVMM for hotplugging IBM Adjunct Processor (AP) devices over VFIO
2020-08-25 08:46:25 -05:00
Julio Montes
e6757de2ac
Merge pull request #543 from likebreath/clh_backporting
clh: Port cloud-hypervisor related changes from kata-runtime
2020-08-25 08:42:03 -05:00
Jakob-Naucke
4c30b2554f
runtime: Re-vendor GoVMM for VFIO-AP support
This is a re-vendor of intel/govmm, with support for hot-plugging IBM
Adjunct Processor (AP) devices over VFIO. This is necessary for
enabling AP device pass-through in Kata (see #491).

    39c372a Add support for hot-plugging IBM VFIO-AP devices
    f5bdd53 travis: disable amd64 jobs
    1af1c0d github: enable github actions
    4831c6e travis: Run coveralls after success
    cf0f05d qemu: add iommu_platform knob for qemuParams
    175ac49 typo fix

Fixes: #565

Signed-off-by: Jakob-Naucke <jakob.naucke@ibm.com>
2020-08-25 10:35:11 +02:00
Evan Foster
282bff9f94 sandbox: Disconnect from agent after VM shutdown
When a one-shot pod dies in CRI-O, the shimv2 process isn't killed until
the pod is actually deleted, even though the VM is shut down. In this
case, the shim appears to busyloop when attempting to talk to the (now
dead) agent via VSOCK. To address this, we disconnect from the agent
after the VM is shut down.

This is especially catastrophic for one-shot pods that may persist for
hours or days, but it also applies to any shimv2 pod where Kata is
configured to use VSOCK for communication.

See github.com/kata-containers/runtime#2719 for details.
Fixes #2719

Signed-off-by: Evan Foster <efoster@adobe.com>
2020-08-24 09:53:28 -06:00
Salvador Fuentes
29aae01779
Merge pull request #559 from snir911/doc
docs: remove outdated dependencies from agent docs
2020-08-24 10:19:29 -05:00
Julio Montes
5b707234d8
Merge pull request #511 from Jakob-Naucke/forward-s390x-packaging
[forward-port] packaging: s390x kernel config fragments
2020-08-24 09:44:04 -05:00
Alice Frosi
9f1a3d15db
kernel: add s390x fragment
Support for s390x kernel fragments.

Fixes: #1041

Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
Signed-off-by: Jakob-Naucke <jakob.naucke@ibm.com>
2020-08-24 14:36:01 +02:00
Alice Frosi
f13506163b
kernel: config CONFIG_GENERIC_MSI_IRQ_DOMAIN
Moved CONFIG_GENERIC_MSI_IRQ_DOMAIN in arch base.conf.
The config is not selected for s390x

Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
Signed-off-by: Jakob-Naucke <jakob.naucke@ibm.com>
2020-08-24 14:36:01 +02:00
Alice Frosi
b67325c3ac
kernel: add missing configs
Some kernel configs need additional dependencies:
  - CONFIG_NO_HZ depends on
      CONFIG_GENERIC_CLOCKEVENTS
  - CONFIG_CGROUP_PERF depends on
      CONFIG_PERF_EVENTS
        CONFIG_HAVE_PERF_EVENTS
  - CONFIG_BLK_DEV_LOOP depends on
      CONFIG_BLK_DEV
        CONFIG_BLOCK

Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
Signed-off-by: Jakob-Naucke <jakob.naucke@ibm.com>
2020-08-24 14:36:00 +02:00
Alice Frosi
454dd85432
kernel: config CONFIG_ PARAVIRT
Moved CONFIG_ PARAVIRT to each arch base.conf.

CONFIG_ PARAVIRT only defined in x86, arm64, arm in arch/$arch/Kconfig.

Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
Signed-off-by: Jakob-Naucke <jakob.naucke@ibm.com>
2020-08-24 14:36:00 +02:00
Alice Frosi
62b45064d4
kernel: config CONFIG_NO_HZ_FULL
Moved CONFIG_NO_HZ_FULL config to each arch base.conf.

The config CONFIG_NO_HZ_FULL depends on CONFIG_HAVE_CONTEXT_TRACKING.
See https://github.com/torvalds/linux/blob/
a811c1fa0a02c062555b54651065899437bacdbe/kernel/time/Kconfig#L96

The context tracking is not supported on s390x yet.
See https://github.com/torvalds/linux/blob/
a811c1fa0a02c062555b54651065899437bacdbe/Documentation/features/time/
context-tracking/arch-support.txt#L27

Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
Signed-off-by: Jakob-Naucke <jakob.naucke@ibm.com>
2020-08-24 14:35:59 +02:00
Alice Frosi
6dca74ba72
kernel: moved acpi hotplug config
Moved:
---
CONFIG_HOTPLUG_PCI_ACPI=y
CONFIG_PNPACPI=y
---
from hotplug to acpi.
In this way, it is possible to skip these config if the acpi feature is
not supported.

Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
Signed-off-by: Jakob-Naucke <jakob.naucke@ibm.com>
2020-08-24 14:35:59 +02:00
Alice Frosi
7c85decc82
kernel: config CONFIG_PCI_MSI_IRQ_DOMAIN
For certain kernel version CONFIG_PCI_MSI_IRQ_DOMAIN is only supported
on x86 and arm64.
See https://github.com/torvalds/linux/blob/
219d54332a09e8d8741c1e1982f5eae56099de85/drivers/pci/Kconfig#L55

Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
Signed-off-by: Jakob-Naucke <jakob.naucke@ibm.com>
2020-08-24 14:35:44 +02:00
Alice Frosi
efe51b291c
kernel: fragment for pmem
The option CONFIG_VIRTIO_PMEM is not supported on s390x.
It requires nvdimm support.

Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
Signed-off-by: Jakob-Naucke <jakob.naucke@ibm.com>
2020-08-24 14:25:34 +02:00
Alice Frosi
08d046d994
kernel: config CONFIG_HAVE_NET_DSA
The kernel config CONFIG_HAVE_NET_DSA is not supported on s390x.
See https://github.com/torvalds/linux/blob/master/net/dsa/Kconfig#L4.
Config moved to arch directory.

Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
Signed-off-by: Jakob-Naucke <jakob.naucke@ibm.com>
2020-08-24 14:25:34 +02:00
Alice Frosi
7b49fa121b
kernel: fragments not supported on s390x
Add !s390x tag to skip these group of fragments for s390x.

Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
Signed-off-by: Jakob-Naucke <jakob.naucke@ibm.com>
2020-08-24 14:25:30 +02:00
Peng Tao
ccfb73cb8c agent/agent-ctl: update Cargo.lock
rustc 1.45.2 would just ask for these depenedencies.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-08-24 19:31:08 +08:00
Chelsea Mafrica
fd13c93c6a virtcontainers: Add msg to existing utils unit tests
Add message to existing unit tests that use arrays and loops for easier
debugging.

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2020-08-23 19:49:40 -07:00
Chelsea Mafrica
c3fc09b95b virtcontainers: Add to utils unit tests
Expand unit tests for virtcontainers/utils/utils.go to include testing
CleanupFds, CPU calculations, ID string creation, and memory alignment
functions.

Fixes #490

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2020-08-23 19:49:40 -07:00
Peng Tao
942041ecd7
Merge pull request #548 from Tim-Zhang/fix-action-check-subsystem
action: Fix subsystem check
2020-08-24 10:27:30 +08:00
Snir Sheriber
965825564d docs: remove outdated dependencies from agent docs
switched from grpc to ttrpc (a7041c27d)

Fixes: #558

Signed-off-by: Snir Sheriber <snir911@gmail.com>
2020-08-23 14:26:40 +03:00
James O. D. Hunt
d12f920b3f console: Fix crash if debug console disabled
The logic for the debug console meant that if the debug console was
_disabled_, the agent was guaranteed to crash on function exit due to
the unsafe code block. Fixed by simplifying the code to use the standard
`Option` idiom for optional values.

Fixes: #554.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2020-08-21 17:52:58 +01:00
James O. D. Hunt
572de288f0 sandbox: Remove unnecessary thread
Don't create a thread to wait for the ttRPC server to end - it isn't
required as the operation should be blocked on.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2020-08-21 17:52:58 +01:00
James O. D. Hunt
d5fbba3b0a main: Remove commented out and redundant code
Remove confusing commented out code and some stray testing code.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2020-08-21 17:52:58 +01:00
James O. D. Hunt
1b2fe4a5be agent: Refactor main function
Move the sandbox creation into a new function.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2020-08-21 17:52:58 +01:00
James O. D. Hunt
bac79eeef0 main: Display config in announce
Add the configuration details in the announcement log call.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2020-08-21 17:52:58 +01:00
James O. D. Hunt
e2952b5354 main: Simplify version handling
Print a simple version string rather than delaying the output
to display a structured version string. The structured output
is potentially more useful but:

- This output is not consistent with other components.

- Delaying the output makes `--version` unusable in some
  environments (since a lot of setup is called before the
  version string can be output).

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2020-08-21 17:52:15 +01:00
Tim Zhang
cfa35a90b2 action: Fix subsystem check
\h is not a valid metacharacter in javascript which is used in
github-action.
Use \s\t to replace it.

Fixes: #551

Signed-off-by: Tim Zhang <tim@hyper.sh>
2020-08-21 23:49:43 +08:00
Pradipta Kumar
bba2773d7d
Merge pull request #482 from Amulyam24/fix-ppc64le-rootfs
osbuilder : ppc64le support for rust agent based rootfs/initrd image
2020-08-21 12:46:10 +05:30
Jose Carlos Venegas Munoz
39b53f4467 clh: enable build using Podman
[ Port from packaging commit 4e1b5729f47d5f67902e1344521bc5b121673046 ]

Build clh with Podman, allow build the vmm in the Podman CI

Virtiofs qemu has to be build as this is requried by clh.

Fixes: #461

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-20 14:00:31 -07:00
Jose Carlos Venegas Munoz
04b156f604 qemu-virtiofs: Update to qemu 5.0 + virtiofs + dax
[ Port from packaging commit cbe53bdb14e303830fa9f2d5a7f3c9161a32f033 ]

Update build scripts for qemu-virtiofs.

- virtiofs-0.3 patches are not needed
- Sync build on how vanilla qemu is built
- Apply patches for virtiofsd if any (none today)
- Apply patches that are used for the qemu vanilla
- Apply patches in order

Fixes: #461

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-20 14:00:28 -07:00
Bo Chen
3ec05a9f95 clh: Add support to unplug block devices
[ Port from runtime commit 44b58e4151d1fc7debed41274b65c37233a437e3 ]

This patch enables kata+clh to unplug block devices, which is required
to pass cri-o integration tests.

Fixes: #461

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-20 14:00:24 -07:00
Bo Chen
45e32e1b77 clh: Set 'Id' explicitly while hotplugging block device
[ Port from runtime commit 03fb9c50c180d3359178c30e06f1122df312ae76 ]

To support unplug block device, we need to set the 'Id' explicitly while
hotplugging devices with cloud-hypervisor HTTP API.

Fixes: #461

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-20 14:00:21 -07:00
Jose Carlos Venegas Munoz
895959d007 clh: Provide cpu topology to API
[ Port from runtime commit 39897867bc89667daaafdd141367ec4a5fdc9247 ]

API now requires cpu topology.

Fixes: #461

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-20 14:00:18 -07:00
Jose Carlos Venegas Munoz
3159438701 clh: opeanapi: update api for cloud hypervisor
[ Port from runtime commit 40f49312a4881c904a1cbdace04c4c697bd2d429 ]

Update api geneated by openapi.

Fixes: #461

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-20 14:00:15 -07:00
Jose Carlos Venegas Munoz
89836cd31a versions: cloud-hypervisor 0.9.0
[ Port from runtime commit 0dcbbd8dc113878c2aa8c78b5300e4853a7e64be ]

Highlights for cloud-hypervisor version 0.9.0 include:
virtiofs updates to new dax implementation based in qemu 5.0
Fixed random issues caused due to seccomp filters

io_uring Based Block Device Support

If the io_uring feature is enabled and the host kernel supports it then io_uring will be used for block devices. This results a very significant performance improvement.
Block and Network Device Statistics

Statistics for activity of the virtio network and block devices is now exposed through a new vm.counters HTTP API entry point. These take the form of simple counters which can be used to observe the activity of the VM.
HTTP API Responses

The HTTP API for adding devices now responds with the name that was assigned to the device as well the PCI BDF.
CPU Topology

A topology parameter has been added to --cpus which allows the configuration of the guest CPU topology allowing the user to specify the numbers of sockets, packages per socket, cores per package and threads per core.
Release Build Optimization

Our release build is now built with LTO (Link Time Optimization) which results in a ~20% reduction in the binary size.
Hypervisor Abstraction

A new abstraction has been introduced, in the form of a hypervisor crate so as to enable the support of additional hypervisors beyond KVM.
Snapshot/Restore Improvements

Multiple improvements have been made to the VM snapshot/restore support that was added in the last release. This includes persisting more vCPU state and in particular preserving the guest paravirtualized clock in order to avoid vCPU hangs inside the guest when running with multiple vCPUs.
Virtio Memory Ballooning Support

A virtio-balloon device has been added, controlled through the resize control, which allows the reclamation of host memory by resizing a memory balloon inside the guest.
Enhancements to ARM64 Support

The ARM64 support introduced in the last release has been further enhanced with support for using PCI for exposing devices into the guest as well as multiple bug fixes. It also now supports using an initramfs when booting.
Intel SGX Support

The guest can now use Intel SGX if the host supports it. Details can be found in the dedicated SGX documentation.
Seccomp Sandbox Improvements

The most frequently used virtio devices are now isolated with their own seccomp filters. It is also now possible to pass --seccomp=log which result in the logging of requests that would have otherwise been denied to further aid development.
Notable Bug Fixes

    Our virtio-vsock implementation has been resynced with the implementation from Firecracker and includes multiple bug fixes.
    CPU hotplug has been fixed so that it is now possible to add, remove, and re-add vCPUs (#1338)
    A workaround is now in place for when KVM reports MSRs available MSRs that are in fact unreadable preventing snapshot/restore from working correctly (#1543).
    virtio-mmio based devices are now more widely tested (#275).
    Multiple issues have been fixed with virtio device configuration (#1217)
    Console input was wrongly consumed by both virtio-console and the serial. (#1521)

Fixes: #461

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-20 14:00:11 -07:00
Jose Carlos Venegas Munoz
8d5a60ac0a versions: Update qemu-virtiofs to 5.0
[ Port from runtime commit d803f077c6fd26e4d020643eda415ea315f47e0c ]

Update to qemu 5.0.x with support for virtiofs + dax.

Fixes: #461

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-20 14:00:06 -07:00
Bo Chen
76a64667d5 clh: Remove the use of deprecated '--memory file=' parameter
[ Port from runtime commit 30b40f5505fd46d23b89eb5fb38301d2f7454f35 ]

Along with the release of cloud-hypervisor v0.8.0, this option has been
deprecated. clh now enforces to use the alternative controls,
e.g. "shared" and "hugepages", which can infer the backing file
paths. Also, we don't use "hugepages" in kata, so we are fine now as the
"shared" control  is already enabled.

Fixes: #461

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-20 13:51:43 -07:00
Bin Liu
5f2a9fad4d
Merge pull request #536 from Kvasscn/packaging-build-kernel-sh-test
packaging: add usage instructions for -a (arch_target) option
2020-08-20 17:53:52 +08:00
zhanghj
bfd78104b2 packaging: add usage instructions for -a (arch_target) option
Add usage instructions for -a option in script and README,
currently supported architecture are aarch64/ppc64le/s390x/x86_64.

Fixes: #534

Signed-off-by: zhanghj <zhanghj.lc@inspur.com>
2020-08-18 22:26:11 -04:00
Yang Bo
03b7bc17de
Merge pull request #499 from lifupan/2.0-dev
rustjail: add the "HOME" env for process
2020-08-19 09:35:06 +08:00
Yang Bo
485fc8ddc8
Merge pull request #531 from lifupan/propagation
rustjail: fix the issue of missing set propagation for bind mount
2020-08-19 09:33:53 +08:00
Julio Montes
2d28043cb2
Merge pull request #527 from devimc/2020-08-13/unittest/rustjail-process
agent: add unit tests for rustjail/process.rs
2020-08-18 12:08:31 -05:00
Bo Chen
646148b2d4
Merge pull request #533 from GabyCT/topic/backportversions
ci: Update experimental kernel tag to enable CLH CI
2020-08-18 09:16:26 -07:00
Gabriela Cervantes
ecaa1f9e25 clh: Enable versions and kernel tag to enable CLH CI for kata 2.0
This PR updates the versions for the virtiofs kernel branch and
as there is a tag based in kernel 5.6 move patches to uses the tag name.
This PR is needed to enable CLH CI for kata 2.0. This PR is backporting
kata-containers/runtime#2843 and kata-containers/packaging#1098.

Fixes #532

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2020-08-18 10:44:02 -05:00
Bin Liu
37a331a1e8
Merge pull request #506 from lining2020/2.0-dev
virtcontainers: fix outdated example code in api document
2020-08-18 19:36:51 +08:00
Amulya Meka
64b0694436 ppc64le: Support for rust agent based rootfs
For building rust agent on ppc64le, the rust toolchain is built using
the LIBC implementation - gnu instead of musl.

Fixes: #481

Signed-off-by: Amulya Meka <amulmek1@in.ibm.com>
2020-08-18 16:00:26 +05:30
Li Ning
2511cabbc3 virtcontainers: fix outdated example code in api document
Some type declarations were changed. The example code here is outdated
according to the example_pod_run_test.go under virtcontainers directory.
And add the imports to make where the types from clear.

Fixes: #507

Signed-off-by: Li Ning <lining_yewu@cmss.chinamobile.com>
2020-08-18 17:42:18 +08:00
Bin Liu
17e2a35cbe
Merge pull request #509 from lifupan/fix_lo
agent: setup the "lo" interface run agent as init
2020-08-18 17:24:19 +08:00
Bin Liu
de3fd3f325
Merge pull request #521 from Tim-Zhang/fix-github-actions
Fix commit-message-check and do some updates about github actions
2020-08-18 15:40:24 +08:00