Commit Graph

844 Commits

Author SHA1 Message Date
James O. D. Hunt
c200b28dc7 tracing: Add context to virtcontainers API
Add a `context.Context` parameter to all the virtcontainers API's to
support tracing.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-08-22 08:24:58 +01:00
James O. D. Hunt
f0073bec2f tracing: Create trace function
Simplify code slightly be creating a `trace()` function.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-08-22 08:24:58 +01:00
James O. D. Hunt
a193366b3d tracing: Add subsystem tag to runtime
Set a tag on the root span to denote the subsystem refers to the
runtime.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-08-22 08:24:58 +01:00
Julio Montes
d6a773c90c
Merge pull request #595 from amshinde/use-main-bus-for-hotplug
vfio: Add configuration to support VFIO hotplug on root bus
2018-08-21 11:09:49 -05:00
Graham Whaley
0c0e17f924
Merge pull request #609 from devimc/topic/vendorAgent
vendor: update kata agent
2018-08-21 11:10:26 +01:00
Julio Montes
8bbd9b5878
Merge pull request #602 from kata-containers/vhu-issue-601
network: fix vhost-user net creation
2018-08-20 14:04:52 -05:00
Archana Shinde
31e2925a9a vfio: Add configuration to support VFIO hotplug on root bus
We need this configuration due to a limitation in seabios
firmware in handling hotplug for PCI devices with large BARS.
Long term, this needs to be fixed in the firmware.

Fixes #594

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-08-20 11:36:21 -07:00
Julio Montes
49690b04ed vendor: update kata agent
Increase dial timeout to fix context deadline exceeded when
using vsocks

Shortlog:
93fe84e protocols: increase dial timeout
730b977 client: bypass grpc dialer backoff strategy
41c09c2 agent: enable memory hierarchical account
4f70b1c channel: remove unused functions
f6486e7 channel: support communication channel hotplug
28afabb release: Kata Containers 1.2.0

fixes #576

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-08-20 09:18:29 -05:00
zhangwei_cs
d748b26f40
Merge pull request #604 from jcvenegas/remove-not-Num-var
vc: Remove unused variable NumInterfaces
2018-08-20 10:06:30 +08:00
Eric Ernst
ee1ea36d0a network: fix vhost-user net creation
When creating a device structure to be added to the hypervisor, make
sure that the device includes the vhost-user type.  In particular,
for network devices, specificy VhostUserNet.

Fixes: #601

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2018-08-17 15:09:34 -07:00
Archana Shinde
70edc56fc1 disk: Pass the --share-rw option for hotplugging disks
With qemu 2.10, a write lock was added for qcow images that
prevents the same image to be passed more than once.
This can be over-ridden using the --share-rw option which is
desired for raw images.

This solves an issue with running Kata with devicemapper
using the privileged mode as in this case all devices on the host
are passed to the container including the block device associated
with the rootfs, causing it to be passed twice to qemu.

Fixes #606

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-08-17 15:08:33 -07:00
Archana Shinde
b9e0ca340d vendor: vendor latest govmm
Includes --share-rw option for hotplugging disks.

govmm Shortlog:

2706a07 qemu: Use the supplied context.Context for launching
e46092e qemu: Do not try and generate invalid RTC parameters
fcaf61d qemu/qmp: add vfio mediated device support
4461c45 disk: Add --share-rw option for hotplugging disks
6851999 qemu/qmp: add addr and bus to hotplug vsock devices

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-08-17 14:31:41 -07:00
Jose Carlos Venegas Munoz
161e3a771b vc: Remove unused variable NumInterfaces
Remove unsed variable, the variable is set just in one place, and
never is used again.

Fixes: #603

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-08-17 15:53:48 -05:00
Peng Tao
f70d6d2acb
Merge pull request #590 from devimc/topic/kataEnvSupportVSocks
cli: kata-env: show if vsocks are supported
2018-08-17 15:14:49 +08:00
Julio Montes
88d79f3517 cli: kata-env: show if vsocks are supported
Add a new field in `HostInfo` structure to show if vsocks are
supported in the host.

fixes #589

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-08-16 15:36:36 -05:00
Sebastien Boeuf
26f31071ca
Merge pull request #287 from caoruidong/hotplug
api: add sandbox hotplug network
2018-08-16 08:19:37 -07:00
Peng Tao
b473dc4ac9
Merge pull request #579 from lifupan/master
CI: bump the CI travis's go to 1.10
2018-08-16 22:51:59 +08:00
Ruidong Cao
7beb309faf test: add UTs for network hotplug
add UTs for network hotplug related fuctions

Fixes #113

Signed-off-by: Ruidong Cao <caoruidong@huawei.com>
2018-08-16 21:31:20 +08:00
Ruidong Cao
72df219306 cli: add network commands
Add update and list commands for notwork hotplug

Signed-off-by: Ruidong Cao <caoruidong@huawei.com>
2018-08-16 16:10:29 +08:00
Ruidong Cao
1a17200cc8 virtcontainers: add sandbox hotplug network API
Add sandbox hotplug network API to meet design

Signed-off-by: Ruidong Cao <caoruidong@huawei.com>
2018-08-16 16:10:10 +08:00
Ruidong
6666426739 vendor: update govmm and agent changes
To support tap network hotplug. Implement netdev_add, netdev_del and corresponding device_add QMP commands.
Full list in govmm:
    10efa84 qemu/qmp: add function for hotplug network by fds
    80ed88e qemu/qmp: implement function to hotplug serial ports
    ca46f21 qemu/qmp: implement function to hotplug character devices
    03f1a1c qemu/qmp: implement getfd
    84b212f qemu: add vhostfd and disable-modern to vsock hotplug
    12dfa87 qemu/qmp: implement function for hotplug network
    4ca232e qmp_test: Fix Warning and Error level logs
    430e72c qemu,qmp: Enable gas security checker
    ffc06e6 qemu,qmp: Add staticcheck to travis and fix errors

Add agent ListInterfaces and ListRoutes APIs.
Full list in agent:
    7c287c6 agent: add ListInterfaces and ListRoutes rpc

Signed-off-by: Ruidong Cao <caoruidong@huawei.com>
2018-08-16 16:06:16 +08:00
Archana Shinde
3424444be3
Merge pull request #578 from grahamwhaley/20180813_env_runtime_path
cli: kata-env: add runtime path to output
2018-08-15 10:13:21 -07:00
James O. D. Hunt
cd514b69fd
Merge pull request #568 from amshinde/remove-unused-cni-code
Remove unused cni code
2018-08-15 16:36:48 +01:00
zhangwei_cs
e0c179d954
Merge pull request #550 from WeiZhang555/add-device-before-container-create
devices: add interface "sandbox.AddDevice"
2018-08-15 17:56:51 +08:00
fupan
4850579da3 CI: bump the CI travis's go to 1.10
Bump the CI travis's go to 1.10.x to meet
the building kata shim v2 for containerd.

Fixes: #575

Signed-off-by: fupan <lifupan@gmail.com>
2018-08-15 17:28:13 +08:00
Wei Zhang
6e6be98b15 devices: add interface "sandbox.AddDevice"
Fixes #50 .

Add new interface sandbox.AddDevice, then for Frakti use case, a device
can be attached to sandbox before container is created.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-08-15 15:24:12 +08:00
Archana Shinde
474111c1af tests: Add a cleanup for the CNM tests
The CNM test was missing a cleanup which was handled earlier
with the CNI tests.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-08-14 22:56:50 -07:00
Archana Shinde
0d7b476923 network: Remove unused cni vendored code
Most of the libcni code can now be removed.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-08-14 22:56:50 -07:00
Archana Shinde
99fa758423 network: Remove unused CNI code
We dont really call CNI from Kata, this is done by the CRI layer.
All of the CNI code is unused.

Fixes #567

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-08-14 22:56:50 -07:00
Sebastien Boeuf
83008d4959
Merge pull request #583 from amshinde/update-qemu-commit
versions: Update the commit for qemu-lite
2018-08-14 22:32:50 -07:00
Archana Shinde
f6bfb857c0 versions: Update the commit for qemu-lite
The commit for qemu needs to be updated to the tip
of the qemu-lite-2.11.0 branch. The qemu packages for
1.12.0 also contain the latest commit.

Fixes #582

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-08-14 17:30:04 -07:00
Graham Whaley
dd2acd26eb
Merge pull request #565 from jodh-intel/support-opentracing
tracing: Add initial opentracing support
2018-08-14 10:32:52 +01:00
Graham Whaley
8b69c75144 cli: kata-env: add runtime path to output
`kata-env` did not include the path to the runtime exe itself.
Add that into the Runtime section.

Fixes: #577

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2018-08-13 14:08:02 +01:00
James O. D. Hunt
3a1bbd0271 tracing: Add initial opentracing support
Add initial support for opentracing by using the `jaeger` package.
Since opentracing uses the `context` package, add a `context.Context`
as the first parameter to all the functions that we might want to
trace. Trace "spans" (trace points) are then added by extracting the
trace details from the specified context parameter.

Notes:

- Although the tracer is created in `main()`, the "root span"
  (aka the first trace point) is not added until `beforeSubcommands()`.

  This is by design and is a compromise: by delaying the creation of the
  root span, the spans become much more readable since using the web-based
  JaegerUI, you will see traces like this:

  ```
  kata-runtime: kata-runtime create
  ------------  -------------------
       ^                ^
       |                |
  Trace name        First span name
                    (which clearly shows the CLI command that was run)
  ```

  Creating the span earlier means it is necessary to expand 'n' spans in
  the UI before you get to see the name of the CLI command that was run.
  In adding support, this became very tedious, hence my design decision to
  defer the creation of the root span until after signal handling has been
  setup and after CLI options have been parsed, but still very early in
  the code path.

  - At this stage, the tracing stops at the `virtcontainers` call
  boundary.

- Tracing is "always on" as there doesn't appear to be a way to toggle
  it. However, its resolves to a "nop" unless the tracer can talk to a
  jaeger agent.

Note that this commit required a bit of rework to `beforeSubcommands()`
to reduce the cyclomatic complexity.

Fixes #557.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-08-10 16:13:48 +01:00
Sebastien Boeuf
4220a7a9be
Merge pull request #564 from nitkon/error_message
cli: Make message of using initrd OR rootfs clearer
2018-08-10 07:36:44 -07:00
James O. D. Hunt
0ede467256 tests: Add cli.Context helper functions
Created two new helper functions to create a `cli.Context` with and without a
`cli.App`.

Calling these functions simplifies a lot of test code.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-08-10 15:25:00 +01:00
James O. D. Hunt
41d1c14c68 tests: Move assert closer to function call
Move a test assertion which seemed to have become separated from its
correct location.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-08-10 15:25:00 +01:00
Sebastien Boeuf
800369d58a
Merge pull request #574 from jodh-intel/update-dep-lock-file-format
vendor: Update dep lock file for new format
2018-08-10 07:17:14 -07:00
Nitesh Konkar
bee8d66ead cli: Make message of using initrd OR rootfs clearer
When starting a kata container, if both initrd and rootfs
are provided in the configuration file then the error message
presented is "cannot specify an image and an initrd in
configuration file" which might be a bit confusing. This
fix makes the error message more explicit.

Fixes:  #563

Signed-off-by: Nitesh Konkar niteshkonkar@in.ibm.com
2018-08-10 17:38:18 +05:30
James O. D. Hunt
d9fa73c054 vendor: Update dep lock file for new format
NOP update that simply updates the file format of the `dep` `Gopkg.lock`
file to the latest format.

Note: `dep ensure` run using `dep` at commit
6b79ccc405.

Fixes #573.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-08-10 09:03:50 +01:00
Sebastien Boeuf
66e05e7d13
Merge pull request #507 from amshinde/error-out-host-net
network: Error out when host-networking is requested
2018-08-09 16:12:00 -07:00
Archana Shinde
2f3f375b88 network: Error out when host-networking is requested
Instead of continuing with the network setup, we should detect
if host network namespace was requested and error out early.

Fixes #499

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-08-09 14:50:58 -07:00
Sebastien Boeuf
aedd749168
Merge pull request #562 from nitkon/master
virtcontainers: ppc64le: Add nvdimm to defaultQemuMachineOption
2018-08-09 14:44:01 -07:00
Graham Whaley
e863e55cdb
Merge pull request #559 from sboeuf/enable_virt_machine_type
virtcontainers: qemu: x86: Support "virt" machine type
2018-08-09 09:19:26 +01:00
Nitesh Konkar
ef3a7e80c6 virtcontainers: ppc64le: Add nvdimm to defaultQemuMachineOption
nvdimm is fundamental to get rootfs approach
working for Kata Containers on ppc64le. It should
be added to the default qemu machine option list.

Fixes: #561

Signed-off-by: Nitesh Konkar niteshkonkar@in.ibm.com
2018-08-09 11:36:48 +05:30
Sebastien Boeuf
ef74bc533d virtcontainers: qemu: x86: Support "virt" machine type
In order to start playing with the "virt" machine type for Kata,
we need this new machine type to be part of the list of supported
machines for qemu on x86 architecture.

Fixes #558

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2018-08-08 14:30:26 -07:00
Graham Whaley
0bcb32f704
Merge pull request #555 from jcvenegas/1.2.0-branch-bump
# Kata Containers 1.2.0
2018-08-08 14:40:08 +01:00
Jose Carlos Venegas Munoz
9f0c167bf9 release: Kata Containers 1.2.0
- versions: update openshift to 3.10.0
- qemu: create vm directory before launching qemu
- cli: support factory status command
- re-add: refactor device manager
- virtcontainers: fix the issue of cleanup the vm's path
- Reseed guest random number generator after booting from vm factory
- kernel: Remove initcall_debug boot option
- codecov: remove codecov.yml
- qemu: clear qmp state before wait for qemu process
- do not append sandbox is to kernel parameters
- Use VSOCK when is available in the host.
- Ensure containerID and sandboxID fields logged as early as possible
- revert: "virtcontainers: support pre-add storage for frakti"
- factory: add SetLogger API
- virtcontainers: support pre-add storage for frakti
- versions: Update docker-ce to 18.06
- agent: Increase timeout for check request.
- cli: add AMD support to kata-check
- codecov: Explicitly disable codecov/patch coverage
- virtconainers: rollback the NetNs when createNetwork failed
- sandbox: change container slice to a map
- virtcontainers: process the case that kata-agent doesn't start in VM
- virtcontainers: keep qmp connection whenever possible
- cli: create vm factory if failed to load existing one
- Ensure tests clean their tempfiles
- add vm factory support
- kata_agent: print request details
- virtcontainers: Add support for ephemeral volumes
- makefile: update PREFIX to remove redundant slash
- kata-env: Add ability to output as JSON
- versions: Update CRI-O supported version
- Fix no exit static check
- api: To stop its monitor after a sandbox paused
- virtcontainers: Set ppc64le maxmem depending on qemu version
- proxy: Pass sandbox to proxy
- hypervisor/qemu: add memory hotplug support
- ci: add specfic image-type for aarch64
- main: Pass runtime CLI command to vc logger
- qemu: Pass sandboxID to agent for logging purposes
- logging: Reset external loggers when cid+sid available
- api: To watch the vm console in FetchSandbox api

4fc3041 versions: update openshift to 3.10.0
bd50761 qemu: create vm directory before launching qemu
568b65c qemu: remove redundant code
024a38b cli: print to defaultOutputFile in factory.go
03f2459 cli: support factory status command
1586018 virtcontainers: fix the issue of cleanup the vm's path
6156341 factory: reseed guest RNG device after resuming
d43a435 vendor: update agent dependency
a8f5e2b kernel: Remove initcall_debug boot option
b200163 kata_agent: send sandbox id in CreateSandbox request
e69ebe5 vendor: update agent grpc
0541a5f cli: fix factory assignment
476f21f codecov: remove codecov.yml
3364379 virtcontainers: Use vsock if host support it
3c15bc5 virtcontainers: remove parseVSOCKAddr function
0527691 virtcontainers: implement function to cold plug vsocks
1515bd0 virtcontainers: update KataAgentConfig to support vsocks
3adc862 virtcontainers: log type of proxy started
4680e58 cli: add configuration option to enable/disable vsocks
f389b94 kata-agent: Remove GRPCSock unused variable.
9b28325 virtcontainers: Implement function to get a free context ID
2339ac3 virtcontainers/utils: Implement function to check vsocks support
8ae2888 vendor: update govmm
44a3a44 qemu: wait on disconnected channel in qmp shutdown
c8b4fab qemu: clear qmp state before wait for qemu process
44b9214 devices: fix typo
f474af1 devices: add some test cases
b746489 devices: address some comments
44c37bf devices: rename VFIODrive to VFIODev
7d9d66d devices: remove unused functions
a93e2d0 unit-tests: fix unit tests
f905c16 device-manager: refactor device manger
eec7fa3 devices: don't use drivers package directly.
5db5f42 devices: remove interface VhostUserDevice
1194154 devices: use device manager to manage all devices
763a1b6 logging: Remove unnecessary fields and use standard names
3323c08 logging: Add cid logging to update command
3d5ed66 logging: Improve cid+sid logging
58448bb logging: Allow SetLogger to be called multiple times
029e7ca api: Change logger functions to accept a log entry
dfb758a logging: Remove duplicate arch field in vc
acdd0b8 logging: Split logging source into two fields
c59394d network: Make better use of log fields
a0be57f network: Always call network logger function
927487c revert: "virtcontainers: support pre-add storage for frakti"
b3015dd devices: fix typo
da77124 versions: Update docker-ce to 18.06
9a497fe factory: add SetLogger API
198a069 devices: add some test cases
8391b20 devices: address some comments
04f4f52 devices: rename VFIODrive to VFIODev
daf5abc devices: remove unused functions
1b062b3 unit-tests: fix unit tests
7f5989f device-manager: refactor device manger
c08a263 devices: don't use drivers package directly.
b54df7e devices: remove interface VhostUserDevice
2885eb0 devices: use device manager to manage all devices
5fc7219 agent: check: Increase timeout check request.
12e1911 kata-agent: Improve error message.
4326ea8 cli: add AMD support to kata-check
0e5f6b2 codecov: Explicitly disable codecov/patch coverage
2993cb3 virtcontainers: fix kata-agent fail to start
7103c4f virtcontainers: add qemu process rollback
c2651a8 virtcontainers: add kata-proxy rollback
daebbd1 virtcontainers: add rollback to remove sandbox network
f9d5072 sandbox: change container slice to a map
c6fda44 virtconainers: rollback the NetNs when createNetwork failed
50b445c cli: tests: Clarify who cleans up tmpdir
73c8286 cli: tests: remove the tmpdir to the config.json
d6d38da cli: update_test: defer remove tmpfile
d69fbcf sandbox: add stateful sandbox config
4656a72 cli: create vm factory if failed to load existing one
7a6f205 virtcontainers: keep qmp connection when possible
c9bd12a qemu: cleanup qmp channel setup and teardown
0a11230 factory: add UTs
7cdc0fe cli: do not set ip based kernel parameter
0309e59 cli: add vm factory management subcommand
a7d888f virtconainers: add SetFactory API
bdd5c66 virtcontainers: add vm factory support
8dda2dd virtcontainers: add a vm abstraction layer
28b6104 qemu: prepare for vm templating support
057214f agent: prepare for vm factory
7f20dd8 hypervisor: cleanup valid method
18e6a6e hypervisor: decouple hypervisor from sandbox
4ac6754 qemu: remove append9PVolumes
5d6da35 kata_agent: print request details
b821a5d virtcontainers: Add support for ephemeral volumes
c72a720 makefile: update PREFIX to remove redundant slash
bb0bab2 versions: Update CRI-O supported version
bd6db30 kata-env: tests: add JSON out/in verify test
e45f591 kata-env: Do not leave temp files on test
1f8f762 kata-env: Fix test cases for kata-env JSON
031632d ci: no-exit: Do not run no-exit check on test files
63c06be kata-env: Add ability to output as JSON
62495d4 ci: no-exit: Allow path override for os.Exit check
6268ba4 ci: no-exit: Skip check if no files to check
1c453a3 ci: no-exit: Document and echo what the check is
66a3e81 hypervisor/qemu: add memory hotplug support
67de5d4 ci: reconstructure image-type and initrd for architecture-independant
114482e api: To stop its monitor after a sandbox paused
a38b251 tests: Refactor CC proxy test for Kata
23eb13f proxy: Pass sandbox to proxy
0646a39 vendor: fix dep warning and update yamux dependency
0f20b6b vendor: update govmm changes
830d50e main: Pass runtime CLI command to vc logger
64e2ca3 logging: Reset external loggers when cid+sid available
03d4d49 logging: Add function to handle external loggers
79e8da0 create: Remove redundant logging code
793a220 qemu: Pass sandboxID to agent for logging purposes
9155412 api: To watch the vm console in FetchSandbox api
d0bccab virtcontainers: Set ppc64le maxmem depending on qemu version

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-08-07 09:42:51 -05:00
Graham Whaley
e35caa60ad
Merge pull request #553 from chavafg/topic/update-openshift-version
versions: update openshift to 3.10.0
2018-08-07 09:54:54 +01:00
Salvador Fuentes
4fc3041a6e versions: update openshift to 3.10.0
Last week openshift origin v3.10.0 was released,
this PR updates our supported version from 3.9.0 to
3.10.0

This also updates the cri-o version that we use for
openshift, which is now cri-o 1.10.

Fixes: #552.

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2018-08-06 13:01:11 -05:00