Commit Graph

550 Commits

Author SHA1 Message Date
Graham Whaley
e863e55cdb
Merge pull request #559 from sboeuf/enable_virt_machine_type
virtcontainers: qemu: x86: Support "virt" machine type
2018-08-09 09:19:26 +01:00
Sebastien Boeuf
ef74bc533d virtcontainers: qemu: x86: Support "virt" machine type
In order to start playing with the "virt" machine type for Kata,
we need this new machine type to be part of the list of supported
machines for qemu on x86 architecture.

Fixes #558

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2018-08-08 14:30:26 -07:00
Graham Whaley
0bcb32f704
Merge pull request #555 from jcvenegas/1.2.0-branch-bump
# Kata Containers 1.2.0
2018-08-08 14:40:08 +01:00
Jose Carlos Venegas Munoz
9f0c167bf9 release: Kata Containers 1.2.0
- versions: update openshift to 3.10.0
- qemu: create vm directory before launching qemu
- cli: support factory status command
- re-add: refactor device manager
- virtcontainers: fix the issue of cleanup the vm's path
- Reseed guest random number generator after booting from vm factory
- kernel: Remove initcall_debug boot option
- codecov: remove codecov.yml
- qemu: clear qmp state before wait for qemu process
- do not append sandbox is to kernel parameters
- Use VSOCK when is available in the host.
- Ensure containerID and sandboxID fields logged as early as possible
- revert: "virtcontainers: support pre-add storage for frakti"
- factory: add SetLogger API
- virtcontainers: support pre-add storage for frakti
- versions: Update docker-ce to 18.06
- agent: Increase timeout for check request.
- cli: add AMD support to kata-check
- codecov: Explicitly disable codecov/patch coverage
- virtconainers: rollback the NetNs when createNetwork failed
- sandbox: change container slice to a map
- virtcontainers: process the case that kata-agent doesn't start in VM
- virtcontainers: keep qmp connection whenever possible
- cli: create vm factory if failed to load existing one
- Ensure tests clean their tempfiles
- add vm factory support
- kata_agent: print request details
- virtcontainers: Add support for ephemeral volumes
- makefile: update PREFIX to remove redundant slash
- kata-env: Add ability to output as JSON
- versions: Update CRI-O supported version
- Fix no exit static check
- api: To stop its monitor after a sandbox paused
- virtcontainers: Set ppc64le maxmem depending on qemu version
- proxy: Pass sandbox to proxy
- hypervisor/qemu: add memory hotplug support
- ci: add specfic image-type for aarch64
- main: Pass runtime CLI command to vc logger
- qemu: Pass sandboxID to agent for logging purposes
- logging: Reset external loggers when cid+sid available
- api: To watch the vm console in FetchSandbox api

4fc3041 versions: update openshift to 3.10.0
bd50761 qemu: create vm directory before launching qemu
568b65c qemu: remove redundant code
024a38b cli: print to defaultOutputFile in factory.go
03f2459 cli: support factory status command
1586018 virtcontainers: fix the issue of cleanup the vm's path
6156341 factory: reseed guest RNG device after resuming
d43a435 vendor: update agent dependency
a8f5e2b kernel: Remove initcall_debug boot option
b200163 kata_agent: send sandbox id in CreateSandbox request
e69ebe5 vendor: update agent grpc
0541a5f cli: fix factory assignment
476f21f codecov: remove codecov.yml
3364379 virtcontainers: Use vsock if host support it
3c15bc5 virtcontainers: remove parseVSOCKAddr function
0527691 virtcontainers: implement function to cold plug vsocks
1515bd0 virtcontainers: update KataAgentConfig to support vsocks
3adc862 virtcontainers: log type of proxy started
4680e58 cli: add configuration option to enable/disable vsocks
f389b94 kata-agent: Remove GRPCSock unused variable.
9b28325 virtcontainers: Implement function to get a free context ID
2339ac3 virtcontainers/utils: Implement function to check vsocks support
8ae2888 vendor: update govmm
44a3a44 qemu: wait on disconnected channel in qmp shutdown
c8b4fab qemu: clear qmp state before wait for qemu process
44b9214 devices: fix typo
f474af1 devices: add some test cases
b746489 devices: address some comments
44c37bf devices: rename VFIODrive to VFIODev
7d9d66d devices: remove unused functions
a93e2d0 unit-tests: fix unit tests
f905c16 device-manager: refactor device manger
eec7fa3 devices: don't use drivers package directly.
5db5f42 devices: remove interface VhostUserDevice
1194154 devices: use device manager to manage all devices
763a1b6 logging: Remove unnecessary fields and use standard names
3323c08 logging: Add cid logging to update command
3d5ed66 logging: Improve cid+sid logging
58448bb logging: Allow SetLogger to be called multiple times
029e7ca api: Change logger functions to accept a log entry
dfb758a logging: Remove duplicate arch field in vc
acdd0b8 logging: Split logging source into two fields
c59394d network: Make better use of log fields
a0be57f network: Always call network logger function
927487c revert: "virtcontainers: support pre-add storage for frakti"
b3015dd devices: fix typo
da77124 versions: Update docker-ce to 18.06
9a497fe factory: add SetLogger API
198a069 devices: add some test cases
8391b20 devices: address some comments
04f4f52 devices: rename VFIODrive to VFIODev
daf5abc devices: remove unused functions
1b062b3 unit-tests: fix unit tests
7f5989f device-manager: refactor device manger
c08a263 devices: don't use drivers package directly.
b54df7e devices: remove interface VhostUserDevice
2885eb0 devices: use device manager to manage all devices
5fc7219 agent: check: Increase timeout check request.
12e1911 kata-agent: Improve error message.
4326ea8 cli: add AMD support to kata-check
0e5f6b2 codecov: Explicitly disable codecov/patch coverage
2993cb3 virtcontainers: fix kata-agent fail to start
7103c4f virtcontainers: add qemu process rollback
c2651a8 virtcontainers: add kata-proxy rollback
daebbd1 virtcontainers: add rollback to remove sandbox network
f9d5072 sandbox: change container slice to a map
c6fda44 virtconainers: rollback the NetNs when createNetwork failed
50b445c cli: tests: Clarify who cleans up tmpdir
73c8286 cli: tests: remove the tmpdir to the config.json
d6d38da cli: update_test: defer remove tmpfile
d69fbcf sandbox: add stateful sandbox config
4656a72 cli: create vm factory if failed to load existing one
7a6f205 virtcontainers: keep qmp connection when possible
c9bd12a qemu: cleanup qmp channel setup and teardown
0a11230 factory: add UTs
7cdc0fe cli: do not set ip based kernel parameter
0309e59 cli: add vm factory management subcommand
a7d888f virtconainers: add SetFactory API
bdd5c66 virtcontainers: add vm factory support
8dda2dd virtcontainers: add a vm abstraction layer
28b6104 qemu: prepare for vm templating support
057214f agent: prepare for vm factory
7f20dd8 hypervisor: cleanup valid method
18e6a6e hypervisor: decouple hypervisor from sandbox
4ac6754 qemu: remove append9PVolumes
5d6da35 kata_agent: print request details
b821a5d virtcontainers: Add support for ephemeral volumes
c72a720 makefile: update PREFIX to remove redundant slash
bb0bab2 versions: Update CRI-O supported version
bd6db30 kata-env: tests: add JSON out/in verify test
e45f591 kata-env: Do not leave temp files on test
1f8f762 kata-env: Fix test cases for kata-env JSON
031632d ci: no-exit: Do not run no-exit check on test files
63c06be kata-env: Add ability to output as JSON
62495d4 ci: no-exit: Allow path override for os.Exit check
6268ba4 ci: no-exit: Skip check if no files to check
1c453a3 ci: no-exit: Document and echo what the check is
66a3e81 hypervisor/qemu: add memory hotplug support
67de5d4 ci: reconstructure image-type and initrd for architecture-independant
114482e api: To stop its monitor after a sandbox paused
a38b251 tests: Refactor CC proxy test for Kata
23eb13f proxy: Pass sandbox to proxy
0646a39 vendor: fix dep warning and update yamux dependency
0f20b6b vendor: update govmm changes
830d50e main: Pass runtime CLI command to vc logger
64e2ca3 logging: Reset external loggers when cid+sid available
03d4d49 logging: Add function to handle external loggers
79e8da0 create: Remove redundant logging code
793a220 qemu: Pass sandboxID to agent for logging purposes
9155412 api: To watch the vm console in FetchSandbox api
d0bccab virtcontainers: Set ppc64le maxmem depending on qemu version

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-08-07 09:42:51 -05:00
Graham Whaley
e35caa60ad
Merge pull request #553 from chavafg/topic/update-openshift-version
versions: update openshift to 3.10.0
2018-08-07 09:54:54 +01:00
Salvador Fuentes
4fc3041a6e versions: update openshift to 3.10.0
Last week openshift origin v3.10.0 was released,
this PR updates our supported version from 3.9.0 to
3.10.0

This also updates the cri-o version that we use for
openshift, which is now cri-o 1.10.

Fixes: #552.

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2018-08-06 13:01:11 -05:00
Julio Montes
dec385abd4
Merge pull request #548 from bergwolf/cleanup
qemu: create vm directory before launching qemu
2018-08-03 07:12:51 -05:00
Peng Tao
bd5076101c qemu: create vm directory before launching qemu
Right now we create it in `createsandbox` and it would
create the vm dir unnecessarily for fetchsandbox() and
it ends up leaving an empty vm dir behind even after
DeleteSandbox.

Fixes: #547

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-08-03 16:40:02 +08:00
Peng Tao
568b65c275 qemu: remove redundant code
It looks to be left over due to merge conflicts.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-08-03 16:28:56 +08:00
Archana Shinde
244917c99d
Merge pull request #546 from bergwolf/status-factory
cli: support factory status command
2018-08-02 13:39:35 -07:00
Peng Tao
024a38bcb1 cli: print to defaultOutputFile in factory.go
To follow the same rule as other commands.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-08-03 00:41:53 +08:00
Peng Tao
03f2459388 cli: support factory status command
It checks vm factory status and prints the result.

Fixes: #545

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-08-03 00:26:55 +08:00
Sebastien Boeuf
16600efc1d
Merge pull request #531 from WeiZhang555/bugfix
re-add: refactor device manager
2018-08-02 07:32:02 -07:00
James O. D. Hunt
04fa125e95
Merge pull request #544 from lifupan/master
virtcontainers: fix the issue of cleanup the vm's path
2018-08-02 11:29:50 +01:00
Graham Whaley
c871a90b4d
Merge pull request #539 from bergwolf/crng
Reseed guest random number generator after booting from vm factory
2018-08-02 11:18:34 +01:00
Fupan Li
15860185d9 virtcontainers: fix the issue of cleanup the vm's path
To use the filepath.Join() instead of the simple
string append method to form the file path, otherwise
it will lose the "/" between the two parts.

Fixes #543.

Signed-off-by: Fupan Li <lifupan@gmail.com>
2018-08-02 16:21:55 +08:00
Peng Tao
6156341904 factory: reseed guest RNG device after resuming
So that shared memory guests can still generate different
random numbers.

Fixes: #538

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-08-02 09:43:10 +08:00
Peng Tao
d43a4350a6 vendor: update agent dependency
To include ReseedRandomDev grpc.

Full commits:
e398fd4 agent: add ReseedRandomDev rpc
8a51909 codecov: remove codecov.yml

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-08-02 09:26:04 +08:00
James O. D. Hunt
fc0142ec8e
Merge pull request #527 from jodh-intel/remove-initcall-debug-kernel-option
kernel: Remove initcall_debug boot option
2018-08-01 12:50:52 +01:00
Graham Whaley
02ef1a64ff
Merge pull request #541 from bergwolf/codecov
codecov: remove codecov.yml
2018-08-01 09:55:51 +01:00
James O. D. Hunt
a8f5e2becf kernel: Remove initcall_debug boot option
Remove the `initcall_debug` boot option from the kernel command-line as
we don't need it any more and it generates a ton of boot messages that
may well be impacting performance.

Fixes #526.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-08-01 09:52:13 +01:00
James O. D. Hunt
487f9efa57
Merge pull request #536 from bergwolf/qmp_clear
qemu: clear qmp state before wait for qemu process
2018-08-01 09:51:43 +01:00
Sebastien Boeuf
1dae056012
Merge pull request #524 from bergwolf/agent_log
do not append sandbox is to kernel parameters
2018-07-31 21:52:23 -07:00
Peng Tao
b200163de9 kata_agent: send sandbox id in CreateSandbox request
And do not append sandbox id to kernel arguments since that
would fail qemu args comparison in vm factory.

Fixes: #523

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-08-01 11:18:44 +08:00
Peng Tao
e69ebe5add vendor: update agent grpc
To include SandboxId in CreateSandboxRequest.

Included commits:
745fa71 (origin/pr/309) agent: set sandbox id log field in CreateSandbox
a3321f4 (origin/pr/304) vendor: Revendor netlink to fix PID 1 segfault
a9ade15 (origin/pr/293) agent: Add sandbox level reference counter for sandbox storage
5f89c07 (origin/pr/263) yamux: disable yamux keep alive in server channel
815f831 (origin/pr/303) agent: do not quit on grpc serve errors
8abefd1 (origin/pr/301) protocols/client: close yamux session when closing the stream
67643cf (origin/pr/297) agent: Fix CPU tests for both initrd and rootfs image
cf8d3c8 (origin/pr/295) agent: disable yamux keep alive
ed656df (origin/pr/284) grpc: Ignore WriteStdin after the standard input has been closed
1a52204 grpc: Don't close the terminal on CloseStdin() call
04457e3 (origin/pr/292) logging: Add sandbox field
ba2a32a (origin/pr/286) create: Reduce cyclomatic complexity of CreateContainer
bce5f78 (origin/pr/289) logging: Redirect yamux warnings/errors to logger
23311ea namespace: Add check to make sure PID namespace is not received
7fe2eba (origin/pr/273) release: Kata Containers 1.1.0
4ef4971 (origin/pr/277) travis: Enable travis ci for ppc64le
57bf4e6 (origin/pr/279) namespace: do not create path using rand.
a589bdb (origin/pr/275) CI: update static-checks.sh in travis.yml
1372df2 (origin/pr/271) logging: Fix handling of non-string fields when PID 1
4018add (origin/pr/269) agent: honour CPU constraints when agent is the init process
1cff660 (origin/pr/268) mount: Remove redundant mkdir
cc74b5d (origin/pr/266) network: Setup localhost when running as init
dc06ec7 network: Add extra sanity checks
b580ae7 network: Check all function parameters
ee106e9 main: Exit(1) on error
5bf3909 network: Reduce cyclomatic complexity
cdf9bce namespace: Simplify code
0d7f36f (origin/pr/260) agent: update container cpuset cgroup parents
4cf1809 (origin/pr/258) agent: update container cpuset cgroup before setting the new configuration
d70e5bb agent: don't modify container's config
a30395a (origin/pr/253) grpc: signal frozen containers

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-08-01 11:18:44 +08:00
Peng Tao
0541a5fc77 cli: fix factory assignment
A stupid typo...

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-08-01 11:17:56 +08:00
Peng Tao
476f21f22c codecov: remove codecov.yml
Now that we have a global team yaml, let's use it.

Fixes: #540

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-08-01 11:08:13 +08:00
Peng Tao
fc45d2e3d1
Merge pull request #497 from jcvenegas/vsock-runtime5
Use VSOCK when is available in the host.
2018-08-01 09:55:05 +08:00
Julio Montes
33643797ad virtcontainers: Use vsock if host support it
When the hypervisor option `use_vsock` is true the runtime will check for vsock
support. If vsock is supported, not proxy will be used and the shims
will connect to the VM using VSOCKS. This flag is true by default, so will use
VSOCK when possible and no proxy will be started.

fixes #383

Signed-off-by: Jose Carlos Venegas Munoz jose.carlos.venegas.munoz@intel.com
Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 15:38:45 -05:00
Julio Montes
3c15bc50d0 virtcontainers: remove parseVSOCKAddr function
parseVSOCKAddr function is no more needed since now agent config
contains a field to identify if vsocks should be used or not.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 13:52:44 -05:00
Julio Montes
052769196d virtcontainers: implement function to cold plug vsocks
`appendVSockPCI` function can be used to cold plug vocks, vhost file descriptor
holds the context ID and it's inherit by QEMU process, ID must be unique and
disable-modern prevents qemu from relying on fast MMIO.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 13:52:44 -05:00
Julio Montes
1515bd07a1 virtcontainers: update KataAgentConfig to support vsocks
add extra field in KataAgentConfig structure to specify if the
kata agent have to use a vsock instead of serial port.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 13:52:44 -05:00
Julio Montes
3adc8626e8 virtcontainers: log type of proxy started
In order to see what proxy was started or not, we should log
its type and the URL

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 13:52:44 -05:00
Julio Montes
4680e58e08 cli: add configuration option to enable/disable vsocks
Add `use_vsock` option to enable or disable the use of vsocks
for communication between host and guest.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 13:52:43 -05:00
Jose Carlos Venegas Munoz
f389b94d8a kata-agent: Remove GRPCSock unused variable.
We already save the URL used to connect to the agent in the `state.URL` this
variable is the used to connect the shim to agnet independently the socket type
(VSOCK or serial)

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-07-31 10:30:05 -05:00
Julio Montes
9b283254c3 virtcontainers: Implement function to get a free context ID
FindContextID generates a random number between 3 and max uint32
and uses it as context ID.
Using ioctl findContextID checks if the context ID is free, if
the context ID is being used by other process, this function
iterates from over all valid context IDs until one is available.

`/dev/vhost-vsock` is used to check what context IDs are free,
we need it to ensure we are using a unique context ID to
create the vsocks.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 10:30:05 -05:00
Julio Montes
2339ac3f93 virtcontainers/utils: Implement function to check vsocks support
Implement function to check if the system has support for vsocks.
This function looks for vsock and vhost-vsock devices returning
true if those exist, otherwise false.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 10:29:11 -05:00
Julio Montes
8ae28888e0 vendor: update govmm
add vhostfd and disable-modern to vhost-vsock-pci

shortlog:
3830b44 qemu: add vhostfd and disable-modern to vhost-vsock-pci
f700a97 qemu/qmp: implement function to hotplug vsock-pci

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 08:42:50 -05:00
Peng Tao
44a3a441aa qemu: wait on disconnected channel in qmp shutdown
That is how govmm ensures us that the qmp channel has been cleaned
up entirely.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-31 18:34:37 +08:00
James O. D. Hunt
17a2fb886f
Merge pull request #528 from jodh-intel/fix-log-fields
Ensure containerID and sandboxID fields logged as early as possible
2018-07-31 09:09:29 +01:00
Peng Tao
c8b4fabc37 qemu: clear qmp state before wait for qemu process
So that if there is any remaining state, we do not let it interfere
with the new one. This should fix the occasional vm factory hang.

Fixes: #535

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-31 11:48:40 +08:00
z00280905
44b9214141 devices: fix typo
Fix typo.

Signed-off-by: z00280905 <zhangwei555@huawei.com>
2018-07-31 10:05:56 +08:00
Wei Zhang
f474af1660 devices: add some test cases
Add test cases for device manager reworks.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-31 10:05:56 +08:00
Wei Zhang
b7464899ec devices: address some comments
Address some review comments:
* remove unnecessary rollback logics
* add vfio hot unplug handling.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-31 10:05:56 +08:00
Zhang Wei
44c37bf774 devices: rename VFIODrive to VFIODev
Rename VFIODrive to VFIODev, also rename device interface "GetDeviceDrive()" to
"GetDeviceInfo()".

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-07-31 10:05:56 +08:00
Zhang Wei
7d9d66d782 devices: remove unused functions
cleanup: remove ununsed device interface function "GetDeviceInfo()"

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-07-31 10:05:56 +08:00
Wei Zhang
a93e2d0421 unit-tests: fix unit tests
Fix #50

Fix unit tests

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-31 10:05:56 +08:00
Wei Zhang
f905c16f21 device-manager: refactor device manger
Fixes #50

This commit imports a big logic change:
* host device to be attached or appended now is sandbox level resources,
one device should bind to sandbox/hypervisor first, then container could
reference it via device's unique ID.
* attach or detach device should go through the device manager interface
instead of the device interface.
* allocate device ID in global device mapper to guarantee every device
has a uniq device ID and there won't be any ID collision.

With this change, there will some changes on data format on disk for sandbox
and container, these changes also make a breakage of backward compatibility.

New persist data format:
* every sandbox will get a new "devices.json" file under "/run/vc/sbs/<sid>/"
which saves detailed device information, this also conforms to the concept that
device should be sandbox level resource.
* every container uses a "devices.json" file but with new data format:
```
[
  {
    "ID": "b80d4736e70a471f",
    "ContainerPath": "/dev/zero"
  },
  {
    "ID": "6765a06e0aa0897d",
    "ContainerPath": "/dev/null"
  }
]
```
`ID` should reference to a device in a sandbox, `ContainerPath` indicates device
path inside a container.

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-07-31 10:03:57 +08:00
Wei Zhang
eec7fa394f devices: don't use drivers package directly.
Instead of using drivers.XXXDevice directly, we should use exported
struct from device structure. package drivers should be internal struct
and other package should avoid read it's struct content directly.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-31 09:59:29 +08:00
Wei Zhang
5db5f42b71 devices: remove interface VhostUserDevice
The interface "VhostUserDevice" has duplicate functions and fields with
Device, so we can merge them into one interface and manage them with one
group of interfaces.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-31 09:59:29 +08:00