Commit Graph

2150 Commits

Author SHA1 Message Date
Peng Tao
4656a72b92 cli: create vm factory if failed to load existing one
When vm factory is enabled, create vm factory automatically
during creation.

Fixes: #504

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-23 08:46:53 +08:00
Peng Tao
7a6f205970 virtcontainers: keep qmp connection when possible
For each time a sandbox structure is created, we ensure s.Release()
is called. Then we can keep the qmp connection as long as Sandbox
pointer is alive.

All VC interfaces are still stateless as s.Release() is called before
each API returns.

OTOH, for VCSandbox APIs, FetchSandbox() must be paired with s.Release,
the same as before.

Fixes: #500

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-23 08:37:55 +08:00
Peng Tao
c9bd12aa19 qemu: cleanup qmp channel setup and teardown
Unify qmp channel setup and teardown. This also fixes the issue that
sometimes qmp pointer is not reset after qmp is shutdown.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-23 08:36:58 +08:00
Lai Jiangshan
14d25b82a4
Merge pull request #303 from bergwolf/vmfactory
add vm factory support
2018-07-22 06:47:34 +08:00
Peng Tao
0a11230bfb factory: add UTs
Add UTs to all factory components.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 12:44:58 +08:00
Peng Tao
7cdc0fe912 cli: do not set ip based kernel parameter
For one thing, it is not used by any kata components. For another thing,
it breaks vm factory hypervisor config check.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 12:44:58 +08:00
Peng Tao
0309e59cf8 cli: add vm factory management subcommand
Add enable_template option to the config file.
When it is set, enable the vm template factory.

cache factory cannot be used by kata cli directly because
it requires a running daemon to maintain the cache VMs.

`kata-runtime factory init` would initialize the vm factory and
`kata-runtime factory destroy` would destroy the vm factory.

When configured, a vm factory is loaded before creating new sandboxes.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 12:44:58 +08:00
Peng Tao
a7d888febc virtconainers: add SetFactory API
Add SetFactory to allow virtcontainers consumers to set a vm factory.
And use it to create new VMs whenever the factory is set.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 12:44:58 +08:00
Peng Tao
bdd5c66fc5 virtcontainers: add vm factory support
Add vm factory support per design in the VM Factory plugin section.
The vm factory controls how a new vm is created:

1. direct: vm is created directly
2. template: vm is created via vm template. A template vm is pre-created
   and saved. Later vm is just a clone of the template vm so that they
   readonly share a portion of initial memory (including kernel, initramfs
   and the kata agent). CPU and memory are hot plugged when necessary.
3. cache: vm is created via vm caches. A set of cached vm are pre-created
   and maintained alive. New vms are created by just picking a cached vm.
   CPU and memory are hot plugged when necessary.

Fixes: #303

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 12:44:58 +08:00
Peng Tao
8dda2dd7a5 virtcontainers: add a vm abstraction layer
As representation of a guest without actual sandbox attached to it.
This prepares for vm factory support.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 12:44:58 +08:00
Peng Tao
28b6104710 qemu: prepare for vm templating support
1. support qemu migration save operation
2. setup vm templating parameters per hypervisor config
3. create vm storage path when it does not exist. This can happen when
an empty guest is created without a sandbox.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 12:44:58 +08:00
Peng Tao
057214f0fe agent: prepare for vm factory
There are a few changes we need on kata agent to introduce vm factory
support:
1. decouple agent creation from sandbox config
2. setup agent without creating a sandbox
3. expose vm storage path and share mount point

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 12:44:55 +08:00
Peng Tao
7f20dd89a3 hypervisor: cleanup valid method
The boolean return value is not necessary.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 10:49:25 +08:00
Peng Tao
18e6a6effc hypervisor: decouple hypervisor from sandbox
A hypervisor implementation does not need to depend on a sandbox
structure. Decouple them in preparation for vm factory.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 10:49:25 +08:00
Peng Tao
4ac675453f qemu: remove append9PVolumes
It is not used and we actully cannot append multiple 9pfs volumes to
a guest.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-19 10:49:25 +08:00
Graham Whaley
e0010619fc
Merge pull request #495 from bergwolf/agent_debug
kata_agent: print request details
2018-07-18 17:22:36 +01:00
Peng Tao
81c073f67d
Merge pull request #458 from harche/without_devices
virtcontainers: Add support for ephemeral volumes
2018-07-18 15:06:09 +08:00
Peng Tao
5d6da3517a kata_agent: print request details
It helps tracking each request that is sent and we can match with the
one printed by kata-agent on the guest side to find out any stack
requests in the middle.

Fixes: #494

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-18 11:26:50 +08:00
Harshal Patil
b821a5df4c virtcontainers: Add support for ephemeral volumes
Ephemeral volumes should not be passed at 9pfs mounts.
They should be created inside the VM.

This patch disables ephemeral volumes from getting
mounted as 9pfs from the host and instead a corresponding
tmpfs is created inside the VM.

Fixes : #61

Signed-off-by: Harshal Patil <harshal.patil@in.ibm.com>
2018-07-18 07:41:36 +05:30
Peng Tao
545fe0c873
Merge pull request #490 from kata-containers/makefile-fix
makefile: update PREFIX to remove redundant slash
2018-07-17 17:44:21 +08:00
Eric Ernst
c72a720b97 makefile: update PREFIX to remove redundant slash
Fixes: #488

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2018-07-16 11:01:33 -07:00
Julio Montes
77cbea5a42
Merge pull request #475 from grahamwhaley/20180706_env_json
kata-env: Add ability to output as JSON
2018-07-16 07:38:50 -05:00
Archana Shinde
02056b613e
Merge pull request #482 from chavafg/topic/update-cri-o
versions: Update CRI-O supported version
2018-07-12 10:30:16 -07:00
Salvador Fuentes
bb0bab2683 versions: Update CRI-O supported version
Some test fixes were introduced into the 1.9 and 1.10 branches
of cri-o. These fixes will help us minimize random failures.

Fixes #481.

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2018-07-12 09:47:03 -05:00
Graham Whaley
bd6db3031a kata-env: tests: add JSON out/in verify test
Add a test to ensure the JSON output passes the same
parameter check and write/re-read test as the TOML one.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2018-07-11 17:54:11 +01:00
Graham Whaley
e45f591219 kata-env: Do not leave temp files on test
One of the test cases was not defer removing the tmpfile
it uses. Add that defer.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2018-07-11 17:14:18 +01:00
Graham Whaley
1f8f7629a0 kata-env: Fix test cases for kata-env JSON
With the addition of the JSON kata-env output, we need
to fix up the tests:
 - add a test for the JSON flag
 - fix the format/layout of the other tests to take into
  account the change in function API and the additon of a
  flagset to the cmdline ctx.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2018-07-11 17:12:36 +01:00
Archana Shinde
8af3dd4511
Merge pull request #479 from grahamwhaley/20180710_no_exit_check
Fix no exit static check
2018-07-11 08:30:30 -07:00
Graham Whaley
031632d5b0 ci: no-exit: Do not run no-exit check on test files
The test files do not have access to our app level
exit() function, and are thus OK to call os.Exit() if
they need. Skip them from the check.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2018-07-10 17:04:27 +01:00
Graham Whaley
63c06bee70 kata-env: Add ability to output as JSON
Having a direct JSON output for kata-env will help record
results in our CIs in some instances. Add that ability with
a kata-env command line extension.

Fixes: #474

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2018-07-10 16:06:53 +01:00
Graham Whaley
62495d45be ci: no-exit: Allow path override for os.Exit check
Allow the path being checked by the os-no-exit script to be
passed in, and update the Makefile to use that to check the
current code paths of the cli and virtcontainers.

Fixes: #477

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2018-07-10 15:29:34 +01:00
Graham Whaley
6268ba4aa3 ci: no-exit: Skip check if no files to check
If we find no files to check, gracefully quit the test.
Formerly, if the list was empty we ended up trying to read
from stdin, and thus hung.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2018-07-10 15:29:34 +01:00
Graham Whaley
1c453a372e ci: no-exit: Document and echo what the check is
Make it clearer why we run the check.
Make it announce itself.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2018-07-10 15:29:34 +01:00
Sebastien Boeuf
ff7b4f6ed7
Merge pull request #473 from lifupan/hyper-integration
api: To stop its monitor after a sandbox paused
2018-07-09 19:28:01 -07:00
Sebastien Boeuf
cd842afca4
Merge pull request #417 from nitkon/maxmem
virtcontainers: Set ppc64le maxmem depending on qemu version
2018-07-09 12:07:12 -07:00
Sebastien Boeuf
5e17044c23
Merge pull request #464 from jodh-intel/pass-sandbox-to-proxy
proxy: Pass sandbox to proxy
2018-07-09 10:13:52 -07:00
Sebastien Boeuf
a8952fb79b
Merge pull request #470 from bergwolf/memory-hotplug
hypervisor/qemu: add memory hotplug support
2018-07-09 09:56:19 -07:00
Sebastien Boeuf
f084384501
Merge pull request #462 from Pennyzct/versions
ci: add specfic image-type for aarch64
2018-07-09 09:24:04 -07:00
Peng Tao
66a3e812f2 hypervisor/qemu: add memory hotplug support
So that we can add more memory to an existing guest.

Fixes: #469

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-09 15:29:50 +08:00
Penny Zheng
67de5d4347 ci: reconstructure image-type and initrd for architecture-independant
as default image-type and initrd weren't for non-x86_64 arch,
reconstructuring them to be architecture-specific.

Fixes: #461

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-07-09 02:27:01 +00:00
fupan
114482ed99 api: To stop its monitor after a sandbox paused
After the sandbox is paused, it's needed to stop its monitor,
Otherwise, its monitors will receive timeout errors if it is
paused for a long time, thus its monitor will not tell it's a
crash caused timeout or just a paused timeout.

Fixes: #472

Signed-off-by: fupan <lifupan@gmail.com>
2018-07-06 19:40:43 +08:00
James O. D. Hunt
a38b251120 tests: Refactor CC proxy test for Kata
Reworked `TestCCProxyStart` to create a generic `testProxyStart()` that
is now used for testing both CC and Kata proxies.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-07-06 11:52:22 +01:00
James O. D. Hunt
23eb13f23a proxy: Pass sandbox to proxy
Add the `-sandbox` option when launching the proxy. This isn't strictly
required by the proxy, but is extremely useful for log analysis to allow
log entries to be matched to sandboxes as the proxy will add a
`sandbox` field in each log entry.

Fixes #463.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-07-06 11:52:22 +01:00
Peng Tao
0646a39ff0 vendor: fix dep warning and update yamux dependency
I got following warning after upgrading dep tool:

Warning: the following project(s) have [[constraint]] stanzas in Gopkg.toml:

  ✗  github.com/hashicorp/yamux

However, these projects are not direct dependencies of the current project:
they are not imported in any .go files, nor are they in the 'required' list in
Gopkg.toml. Dep only applies [[constraint]] rules to direct dependencies, so
these rules will have no effect.

Either import/require packages from these projects so that they become direct
dependencies, or convert each [[constraint]] to an [[override]] to enforce rules
on these projects, if they happen to be transitive dependencies,

So let's convert constraint to override over yamux. In the meanwhile,
update the yamux vendor. Full commit list:

4c2fe0d (origin/b-consul-3040) Dont output keepalive error when the session is closed
f21aae5 Make sure to drain the timer channel on defer, and a clarifying comment
601ccd8 Make receive window update logic a bit cleaner
02d320c Uses timer pool in sendNoWait, like in waitForSendErr
cf433c5 window update unit test for partial read; benchmark large buffer
ca8dfd0 improve memory utilization in receive buffer, fix flow control
683f491 Fix race around read and write deadlines in Stream (#52)
40b86b2 Add public session CloseChan method (#44)

Note that commit 4c2fe0d might also help kata-containers/agent/issues/231.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-05 15:31:53 +08:00
Peng Tao
0f20b6b81b vendor: update govmm changes
To include vm factory related commits. Full list:
54caf78 (mine/templating, templating) qmp: add hotplug memory
e66a9b4 qemu: add appendMemoryKnobs helper
8aeca15 qmp: add migrate set arguments
a03d496 qmp: add set migration capabilities
0ace417 qemu: allow to set migration incoming
723bc5f qemu: allow to create a stopped guest
283d7df qemu: add file backed memory device support

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-05 15:25:19 +08:00
James O. D. Hunt
b2bec3362b
Merge pull request #449 from jodh-intel/add-runtime-cli-cmd-to-vc-log
main: Pass runtime CLI command to vc logger
2018-07-05 08:17:54 +01:00
James O. D. Hunt
830d50e9c5 main: Pass runtime CLI command to vc logger
Add the runtime CLI command name to the virtcontainers logger so that it
is clear when reading virtcontainers log entries which runtime command
they refer to.

Fixes #448.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-07-05 07:08:02 +01:00
James O. D. Hunt
47dfb7d6da
Merge pull request #466 from jodh-intel/pass-sandbox-id-to-agent
qemu: Pass sandboxID to agent for logging purposes
2018-07-05 06:49:12 +01:00
James O. D. Hunt
2865dc21aa
Merge pull request #468 from jodh-intel/improve-cid+sid-logging
logging: Reset external loggers when cid+sid available
2018-07-05 06:47:26 +01:00
Julio Montes
dfed5a5a6d
Merge pull request #442 from lifupan/hyper-integration
api: To watch the vm console in FetchSandbox api
2018-07-04 13:37:47 -05:00