Commit Graph

488 Commits

Author SHA1 Message Date
Peng Tao
cfbc974fec
Merge pull request #521 from bergwolf/log
factory: add SetLogger API
2018-07-27 15:52:24 +08:00
zhangwei_cs
2c3215c018
Merge pull request #301 from WeiZhang555/pre-addstorage-based-devmanager
virtcontainers: support pre-add storage for frakti
2018-07-27 15:19:30 +08:00
z00280905
b3015dda26 devices: fix typo
Fix typo.

Signed-off-by: z00280905 <zhangwei555@huawei.com>
2018-07-27 09:33:50 +08:00
Eric Ernst
2a670ce022
Merge pull request #522 from chavafg/topic/update-docker-version
versions: Update docker-ce to 18.06
2018-07-26 16:15:50 -07:00
Salvador Fuentes
da77124898 versions: Update docker-ce to 18.06
Docker 18.06 was released last week, update our
supported docker to this new version.

Fixes: #510

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2018-07-26 10:52:43 -05:00
Sebastien Boeuf
c5075d08ed
Merge pull request #517 from jcvenegas/issue-516-timeout-centos
agent: Increase timeout for check request.
2018-07-26 06:59:37 -07:00
Peng Tao
9a497fedf5 factory: add SetLogger API
So that we actually use the same logger as other packages when being
invoked by CLI.

Fixes: #520

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-26 20:56:31 +08:00
James O. D. Hunt
daa65a5526
Merge pull request #514 from gkennedy12/work
cli: add AMD support to kata-check
2018-07-26 13:28:14 +01:00
Wei Zhang
198a0695ab devices: add some test cases
Add test cases for device manager reworks.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-26 14:15:52 +08:00
Wei Zhang
8391b20805 devices: address some comments
Address some review comments:
* remove unnecessary rollback logics
* add vfio hot unplug handling.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-26 14:15:52 +08:00
Zhang Wei
04f4f528f7 devices: rename VFIODrive to VFIODev
Rename VFIODrive to VFIODev, also rename device interface "GetDeviceDrive()" to
"GetDeviceInfo()".

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-07-26 14:15:52 +08:00
Zhang Wei
daf5abce2d devices: remove unused functions
cleanup: remove ununsed device interface function "GetDeviceInfo()"

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-07-26 14:14:02 +08:00
Wei Zhang
1b062b3db4 unit-tests: fix unit tests
Fix #50

Fix unit tests

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-26 14:14:02 +08:00
Wei Zhang
7f5989f06c device-manager: refactor device manger
Fixes #50

This commit imports a big logic change:
* host device to be attached or appended now is sandbox level resources,
one device should bind to sandbox/hypervisor first, then container could
reference it via device's unique ID.
* attach or detach device should go through the device manager interface
instead of the device interface.
* allocate device ID in global device mapper to guarantee every device
has a uniq device ID and there won't be any ID collision.

With this change, there will some changes on data format on disk for sandbox
and container, these changes also make a breakage of backward compatibility.

New persist data format:
* every sandbox will get a new "devices.json" file under "/run/vc/sbs/<sid>/"
which saves detailed device information, this also conforms to the concept that
device should be sandbox level resource.
* every container uses a "devices.json" file but with new data format:
```
[
  {
    "ID": "b80d4736e70a471f",
    "ContainerPath": "/dev/zero"
  },
  {
    "ID": "6765a06e0aa0897d",
    "ContainerPath": "/dev/null"
  }
]
```
`ID` should reference to a device in a sandbox, `ContainerPath` indicates device
path inside a container.

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-07-26 14:09:53 +08:00
Wei Zhang
c08a26397e devices: don't use drivers package directly.
Instead of using drivers.XXXDevice directly, we should use exported
struct from device structure. package drivers should be internal struct
and other package should avoid read it's struct content directly.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-26 14:09:53 +08:00
Wei Zhang
b54df7e127 devices: remove interface VhostUserDevice
The interface "VhostUserDevice" has duplicate functions and fields with
Device, so we can merge them into one interface and manage them with one
group of interfaces.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-26 11:33:28 +08:00
Wei Zhang
2885eb0532 devices: use device manager to manage all devices
Fixes #50

Previously the devices are created with device manager and laterly
attached to hypervisor with "device.Attach()", this could work, but
there's no way to remember the reference count for every device, which
means if we plug one device to hypervisor twice, it's truly inserted
twice, but actually we only need to insert once but use it in many
places.

Use device manager as a consolidated entrypoint of device management can
give us a way to handle many "references" to single device, because it
can save all devices and remember it's use count.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-26 11:33:28 +08:00
Jose Carlos Venegas Munoz
5fc7219315 agent: check: Increase timeout check request.
In some slow enviroments the agent is taking more than 5 seconds
to start to serve grpc request.

This was reproducible in a Centos VM with 4 cpus running 8 pods in
parallel.

Fixes: #516

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-07-25 21:51:32 -05:00
Jose Carlos Venegas Munoz
12e1911aab kata-agent: Improve error message.
If the grpc connection check fails we only return the grpc error.
To make more clear what failed add more information to the error.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-07-25 20:25:23 -05:00
George Kennedy
4326ea874a cli: add AMD support to kata-check
Added support for identifying AMD CPUs in the `kata-check` CLI command.

Signed-off-by: George Kennedy <george.kennedy@oracle.com>

Fixes #476.
2018-07-25 12:05:47 -04:00
James O. D. Hunt
67b5841153
Merge pull request #512 from sboeuf/disable_codecov_patch
codecov: Explicitly disable codecov/patch coverage
2018-07-25 11:12:56 +01:00
Sebastien Boeuf
0e5f6b27e9 codecov: Explicitly disable codecov/patch coverage
Because codecov coverage regarding the patch is very inconsistent,
this commit introduces codecov.yml config file in order to disable
this check.

Fixes #511

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2018-07-24 11:20:07 -07:00
Eric Ernst
cd133dc9cb
Merge pull request #509 from lifupan/kata-integration
virtconainers: rollback the NetNs when createNetwork failed
2018-07-24 08:13:16 -07:00
Eric Ernst
20066270b9
Merge pull request #503 from bergwolf/container
sandbox: change container slice to a map
2018-07-24 08:10:39 -07:00
Haomin Tsai
8939fd802f
Merge pull request #351 from woshijpf/fix-no-kata-agent
virtcontainers: process the case that kata-agent doesn't start in VM
2018-07-24 19:47:08 +08:00
flyflypeng
2993cb3dd4 virtcontainers: fix kata-agent fail to start
If kata-agent doesn't start in VM, we need to do some rollback
operations to release related resources.

add grpc check() to check kata-agent is running or not

Fixes: #297

Signed-off-by: flyflypeng <jiangpengfei9@huawei.com>
2018-07-25 00:54:33 +08:00
flyflypeng
7103c4f14a virtcontainers: add qemu process rollback
If some errors occur after qemu process start, then we need to
rollback to kill qemu process

Fixes: #297

Signed-off-by: flyflypeng <jiangpengfei9@huawei.com>
2018-07-24 21:36:57 +08:00
flyflypeng
c2651a85a8 virtcontainers: add kata-proxy rollback
If some errors occur after kata-proxy start, we need to
rollback to kill kata-proxy process

Fixes: #297

Signed-off-by: flyflypeng <jiangpengfei9@huawei.com>
2018-07-24 21:36:57 +08:00
flyflypeng
daebbd1e93 virtcontainers: add rollback to remove sandbox network
If error occurs after sandbox network created successfully, we need to rollback
to remove the created sandbox network

Fixes: #297

Signed-off-by: flyflypeng <jiangpengfei9@huawei.com>
2018-07-24 21:34:58 +08:00
Peng Tao
99954d5025
Merge pull request #501 from bergwolf/qemu
virtcontainers: keep qmp connection whenever possible
2018-07-24 13:23:32 +08:00
Peng Tao
f9d50723b9 sandbox: change container slice to a map
ContainerID is supposed to be unique within a sandbox. It is better to use
a map to describe containers of a sandbox.

Fixes: #502

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-24 12:17:02 +08:00
fupan
c6fda444b7 virtconainers: rollback the NetNs when createNetwork failed
When createNetwork failed, cleanup the NetNs if it created.

Fixes: #508

Signed-off-by: fupan <lifupan@gmail.com>
2018-07-24 12:09:13 +08:00
Peng Tao
b244410443
Merge pull request #505 from bergwolf/create_factory
cli: create vm factory if failed to load existing one
2018-07-24 10:43:41 +08:00
Peng Tao
8bdceb92be
Merge pull request #496 from grahamwhaley/20180713_clean_tests
Ensure tests clean their tempfiles
2018-07-24 10:38:05 +08:00
Graham Whaley
50b445cf35 cli: tests: Clarify who cleans up tmpdir
Add a comment to clarify that the caller of
testRunContainerSetup() cleans up the tmpdir.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2018-07-23 17:32:52 +01:00
Graham Whaley
73c8286c7e cli: tests: remove the tmpdir to the config.json
We were defer removing the temporary config.json files
but not the tmpdir path we had created to store them in.
Expose that path out so we can defer removeall it.

Fixes: #480

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2018-07-23 17:32:52 +01:00
Graham Whaley
d6d38dae13 cli: update_test: defer remove tmpfile
Ensure we remove the tmpfile used for testing.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2018-07-23 17:32:52 +01:00
Peng Tao
d69fbcf17f sandbox: add stateful sandbox config
When enabled, do not release in memory sandbox resources in VC APIs,
and callers are expected to call sandbox.Release() to release the in
memory resources.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-23 09:54:02 +08:00
Peng Tao
4656a72b92 cli: create vm factory if failed to load existing one
When vm factory is enabled, create vm factory automatically
during creation.

Fixes: #504

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-23 08:46:53 +08:00
Peng Tao
7a6f205970 virtcontainers: keep qmp connection when possible
For each time a sandbox structure is created, we ensure s.Release()
is called. Then we can keep the qmp connection as long as Sandbox
pointer is alive.

All VC interfaces are still stateless as s.Release() is called before
each API returns.

OTOH, for VCSandbox APIs, FetchSandbox() must be paired with s.Release,
the same as before.

Fixes: #500

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-23 08:37:55 +08:00
Peng Tao
c9bd12aa19 qemu: cleanup qmp channel setup and teardown
Unify qmp channel setup and teardown. This also fixes the issue that
sometimes qmp pointer is not reset after qmp is shutdown.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-23 08:36:58 +08:00
Lai Jiangshan
14d25b82a4
Merge pull request #303 from bergwolf/vmfactory
add vm factory support
2018-07-22 06:47:34 +08:00
Peng Tao
0a11230bfb factory: add UTs
Add UTs to all factory components.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 12:44:58 +08:00
Peng Tao
7cdc0fe912 cli: do not set ip based kernel parameter
For one thing, it is not used by any kata components. For another thing,
it breaks vm factory hypervisor config check.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 12:44:58 +08:00
Peng Tao
0309e59cf8 cli: add vm factory management subcommand
Add enable_template option to the config file.
When it is set, enable the vm template factory.

cache factory cannot be used by kata cli directly because
it requires a running daemon to maintain the cache VMs.

`kata-runtime factory init` would initialize the vm factory and
`kata-runtime factory destroy` would destroy the vm factory.

When configured, a vm factory is loaded before creating new sandboxes.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 12:44:58 +08:00
Peng Tao
a7d888febc virtconainers: add SetFactory API
Add SetFactory to allow virtcontainers consumers to set a vm factory.
And use it to create new VMs whenever the factory is set.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 12:44:58 +08:00
Peng Tao
bdd5c66fc5 virtcontainers: add vm factory support
Add vm factory support per design in the VM Factory plugin section.
The vm factory controls how a new vm is created:

1. direct: vm is created directly
2. template: vm is created via vm template. A template vm is pre-created
   and saved. Later vm is just a clone of the template vm so that they
   readonly share a portion of initial memory (including kernel, initramfs
   and the kata agent). CPU and memory are hot plugged when necessary.
3. cache: vm is created via vm caches. A set of cached vm are pre-created
   and maintained alive. New vms are created by just picking a cached vm.
   CPU and memory are hot plugged when necessary.

Fixes: #303

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 12:44:58 +08:00
Peng Tao
8dda2dd7a5 virtcontainers: add a vm abstraction layer
As representation of a guest without actual sandbox attached to it.
This prepares for vm factory support.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 12:44:58 +08:00
Peng Tao
28b6104710 qemu: prepare for vm templating support
1. support qemu migration save operation
2. setup vm templating parameters per hypervisor config
3. create vm storage path when it does not exist. This can happen when
an empty guest is created without a sandbox.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 12:44:58 +08:00
Peng Tao
057214f0fe agent: prepare for vm factory
There are a few changes we need on kata agent to introduce vm factory
support:
1. decouple agent creation from sandbox config
2. setup agent without creating a sandbox
3. expose vm storage path and share mount point

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 12:44:55 +08:00