Commit Graph

1760 Commits

Author SHA1 Message Date
Wei Zhang
341a988e06 persist: simplify persist api
Fixes #803

Simplify new store API to make the code easier to understand and use.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-04-30 11:54:42 +08:00
Julio Montes
fa5de87d84 virtcontainers: fix invalid CPU topology
sockets * cores * threads should be equal to maxcpus otherwise a
warning is thrown: 'warning: Invalid CPU topology deprecated:
    sockets * cores * threads != maxcpus'

This warning in the future will be an error and won't be possible to run
kata containers.

fixes #1605

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-04-29 15:13:53 -05:00
Ace-Tang
b7f51be8ce cli: do not fail on list when some containers bust
kata-runtime list command should list all valid container, not fail
when some containers information uncorrent, like rootfs not found.

Fixes: #1592

Signed-off-by: Ace-Tang <aceapril@126.com>
2019-04-29 17:04:15 +08:00
Ace-Tang
854cc86e8d shimv2: fix set status when container exit
in wait function, should send msg to exit channel after task status has
updated, since shim.Wait() is running in another goroutine, when it
receive msg from exit channel, it will stop waiting and return, then
someone who hold this Wait() get return, it can delete task, if exit msg
is send first, the container status may still be running.

Fixes: #1600

Signed-off-by: Ace-Tang <aceapril@126.com>
2019-04-29 13:22:26 +08:00
Julio Montes
582f20f489 virtcontainers: Use shim to print the agent logs if there is no proxy
The proxy is in charge to print the agent logs, but when `use_vsocks` is true
the runtime doesn't start the proxy, because it's not needed, hence the agent
logs are ignored. To mitigate this limitation and to make the debugging
processes easier, the fist shim started (the one who monitors the sandbox)
will read the console.sock and print the agent logs.

Depends-on: github.com/kata-containers/shim#172

fixes #1596

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-04-26 13:15:44 -05:00
Eric Ernst
437b3cb2f7
Merge pull request #1585 from rbradford/nemu-release-with-virtiofs
versions: Bump NEMU version to latest release
2019-04-25 21:12:55 -07:00
Archana Shinde
b5aa8d4f67
Merge pull request #1577 from chavafg/topic/revert-mount-pr
Revert "vc: change container rootfs to be a mount"
2019-04-25 09:41:15 -07:00
Graham Whaley
f4fe31e74d
Merge pull request #1442 from jodh-intel/add-agent-trace-support
Add agent trace support
2019-04-25 14:12:54 +01:00
Rob Bradford
da0ae03222 versions: Bump NEMU version to latest release
This release is based off of QEMU 4.0 and also includes support for
virtio-fs.

Fixes: #1580

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-04-25 12:35:10 +01:00
James O. D. Hunt
ed64240df2 agent: Support Kata agent tracing
Add configuration options to support the various Kata agent tracing
modes and types. See the comments in the built configuration files for
details:

- `cli/config/configuration-fc.toml`
- `cli/config/configuration-qemu.toml`

Fixes #1369.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-04-25 09:41:13 +01:00
James O. D. Hunt
b573d9bcb9 vendor: Update for agent tracing and fix issues
Updated the agent vendoring for `StartTracing` and `StopTracing`. This
only changed a single file - the auto-generated gRPC protocol
buffer file.

This change resolves four vendoring issues:

- The github.com/kubernetes-incubator/cri-o project was renamed to
  github.com/cri-o/cri-o. Although github redirects, `dep` complains that
  it cannot find the old `github.com/kubernetes-incubator/cri-o` files
  under `vendor/` so remove the old config, relying on the existing (and
  in other respects identical) `dep` config.

- There was a stale dependency on `github.com/clearcontainers/proxy`
  which should have been removed when the Clear Containers code was
  excised.

- The latest version of the agent code vendored into the runtime prior
  to this commit was a merge commit (commit
  `48dd1c031530fce9bf16b0f6a7305979cedd8fc9`). This somehow confused `dep`
  which did *not* correctly pull in the latest version of the
  auto-generated gRPC code
  (`vendor/github.com/kata-containers/agent/protocols/grpc/agent.pb.go`).
  This is clear because commit `48dd1c031530fce9bf16b0f6a7305979cedd8fc9`
  is newer than the agent commit that introduced the `StartTracing` and
  `StopTracing` APIs (`00cf907afcb7c8e56f077cf45ae3615f612fdc9d`).

  Resolving the other two issues above seems to have resolved this issue
  as the correct version of this file has now been included in the
  vendoring, however note there is no change to the `dep` files as this
  version of `agent.pb.go` should already have been included (!)

- Updating `agent.pb.go` also removed the `AddInterface` and
  `RemoveInterface` API calls which should again also have been removed
  already.

  Updated tests to remove these redundant calls.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-04-25 09:40:02 +01:00
James O. D. Hunt
b309dc5480 agent: Provide explicit config options for the agent
Previously, the agent behaviour was controlled entirely using the
`kernel_params=` config option. This mechanism suffers from a subtle
problem - the runtime is not aware of how the agent will behave.

From now on, all significant agent options will be controlled from the
agent section in the configuration file. This allows the runtime to be
more aware of -- and in control of -- such agent settings. It would also
allow the underlying kernel CLI options to be modified in the future if
required.

This PR adds the only useful agent option as an explicit option by
adding an `enable_debug=true` option to the Kata agent section in
`configuration.toml`. This allows controlling agent debug to be handled
in the same manner as the other debug options.

This change is somewhat foundational: it permits the agent to be handled
consistently with other config file sections which is useful, but
arguably not essential (the old way worked). However, the new way of
handling agent options will be essential when introducing agent tracing
control as the runtime must be aware of the agent trace mode to allow
the runtime to modify its behaviour accordingly.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-04-24 17:14:01 +01:00
James O. D. Hunt
87d91710b9 utils: Remove code duplication
Replace the two versions of `makeRuntimeConfigFileData()` with a single
`MakeRuntimeConfigFileData()` in a new `katatestutils` package and a new
`katautils.GetDefaultHypervisorConfig()` to query the default hypervisor
details.

This isn't ideal but a new package had to be created to avoid circular
dependencies. It was also required since test code cannot be exported
from a package.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-04-24 17:14:01 +01:00
James O. D. Hunt
ed248cef3b shim: Removed unused type and correct error message
Removed the unused `KataShimConfig` type and updated an error message
that incorrectly mentioned it.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-04-24 17:14:01 +01:00
James O. D. Hunt
97beb2b2d4 errors: Create a new standard error for invalid config
Refactor a common error into a new standard error object.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-04-24 17:14:01 +01:00
James O. D. Hunt
e803a7f870 agent: Return an error, not just an interface
Make `newAgentConfig()` return an explicit error rather than handling
the error scenario by simply returning the `error` object in the
`interface{}` return type. The old behaviour was confusing and
inconsistent with the other functions creating a new config type (shim,
proxy, etc).

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-04-24 17:14:01 +01:00
Julio Montes
63e1c440a1
Merge pull request #1393 from zhabinecho/noarp-feature
network: pass network interface RawFlags to agent
2019-04-23 12:14:49 -05:00
Salvador Fuentes
bc9b9e2af6 vc: Revert "vc: change container rootfs to be a mount"
This reverts commit 196661bc0d.

Reverting because cri-o with devicemapper started
to fail after this commit was merged.

Fixes: #1574.

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2019-04-23 08:56:36 -05:00
Hui Zhu
0c5cfcd302
Merge pull request #1565 from jodh-intel/fix-markdown
Fix markdown
2019-04-23 12:52:09 +08:00
Zha Bin
dd0808ae54 network: pass network interface RawFlags to agent
In order to support NOARP in ipvlan interface, the runtime
will pass the rawflags to agent, which also apply to other
network interfaces, not just ipvlan.

Fixes: #1391

Signed-off-by: Zha Bin <zhabin@linux.alibaba.com>
2019-04-23 09:42:00 +08:00
Hui Zhu
74fb9ff570
Merge pull request #1541 from stefanha/fix-parallel-make
Fix parallel make
2019-04-23 09:09:02 +08:00
Julio Montes
77309f4982
Merge pull request #1519 from saschagrunert/patch-1
Update golang to 1.12.3
2019-04-22 09:13:42 -05:00
Xu Wang
b850ab85e2
Merge pull request #1567 from bergwolf/mount
vc: change container rootfs to be a mount
2019-04-21 12:29:16 +08:00
Peng Tao
196661bc0d vc: change container rootfs to be a mount
We can use the same data structure to describe both of them.
So that we can handle them similarly.

Fixes: #1566

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2019-04-20 00:42:25 -07:00
Xu Wang
b218229589
Merge pull request #883 from WeiZhang555/rfc-persist-data-standard-wip
persist: baseline persist data format
2019-04-20 11:54:48 +08:00
Wei Zhang
989b3737c7 docs: fix lisence header to 2019
Modify lisense header from 2018 to 2019.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-04-20 10:04:30 +08:00
Fupan Li
fe6ff5c042
Merge pull request #1550 from teawater/template_path
factory: Add new factory option template_path
2019-04-19 17:20:47 +08:00
Wei Zhang
3262da0207 bugfix: fix potential panic
* Fix potential panic by nil pointer.
* Address comments.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-04-19 16:03:50 +08:00
Wei Zhang
9bd4e5008c store: address comments
Address review comments

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-04-19 15:38:10 +08:00
Wei Zhang
0f52c8b56d test: add unit test for new FS storage driver
add more unit tests.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-04-19 15:37:33 +08:00
Wei Zhang
02f21228dd test: fix unit test
For experimental features, state.json won't be updated, so modify some
unit test to skip.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-04-19 15:37:33 +08:00
Wei Zhang
e40dcb9376 storage: set new storage driver as "experimental"
Set new persist storage driver "virtcontainers/persist/" as "experimental"
feature.
One day when this can fully work and we're ready to move to 2.0, we'll move
it from "experimental" feature to formal feature.
At that time, the "virtcontainers/filesystem_resource_storage.go" can be removed
completely.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-04-19 15:35:33 +08:00
Wei Zhang
504c706bea storage: address comments
Address some comments:
* fix persist driver func names for better understanding
* modify some logic, add some returned error etc

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-04-19 15:33:53 +08:00
Wei Zhang
6e4149d86c persist: save and restore state from persist.json
Save and restore state from persist.json instead of state.json

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-04-19 15:33:53 +08:00
Wei Zhang
039ed4eeb8 persist: persist device data
Persist device information to relative file

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-04-19 15:33:53 +08:00
Wei Zhang
b42fde69c0 persist: demo code for persist api
Demonstrate how to make use of `virtcontainer/persist/api` data structure
package.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-04-19 15:33:53 +08:00
Wei Zhang
e14ffb40cf persist: baseline persist data format
Fixes #803

The disk persist data should be "versioned" and baselined, any modification in
persist data should be considered potential break of backward compatibility.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-04-19 15:33:53 +08:00
Hui Zhu
925193fb3e
Merge pull request #1563 from bergwolf/blockIndex
vc: remove BlockIndex from container state
2019-04-19 13:54:08 +08:00
Hui Zhu
0549a70d93 factory: Add new factory option template_path
Add new factory option template_path that specifies the path of template.

Fixes: #1549

Signed-off-by: Hui Zhu <teawater@hyper.sh>
2019-04-19 11:31:18 +08:00
Fupan Li
3bdc40bfd0
Merge pull request #1547 from teawater/vmtemplate_init_check
factory: Check if vm templateing is on before vm templateing initialize
2019-04-19 10:19:14 +08:00
Archana Shinde
f6b8387814 devices: Skip floppy drives while passing devices to guest
In privileged mode, all host devices are supposed to be passed
to the container in config.json. Skip floppy drives.

Fixes #1551

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2019-04-18 11:26:07 -07:00
James O. D. Hunt
82d105f759 doc: Fix markdown
Resolve a number of syntax issues that were lurking in the markdown.

Fixes: #1564.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-04-18 15:50:34 +01:00
James O. D. Hunt
1a0a4bc049 doc: Fix broken internal link
Link addresses must be all lower case to work.

Partially fixes: #1564.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-04-18 15:50:26 +01:00
James O. D. Hunt
92edeb11c6 doc: Simplify note
It seems that codeblocks inside block quotes are not well understood by
markdown parsers, so simplify by removing the codeblock in the note -
the note is clear enough without labouring the point.

Partially fixes: #1564.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-04-18 15:50:23 +01:00
Peng Tao
203728676a vc: remove BlockIndex from container state
No longer used.

Fixes: #1562

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2019-04-17 22:39:42 -07:00
Hui Zhu
76a5076e56 template: Add check if vm template is on before vm template initialize
Add check if vm template is on before vm template initialize.

Fixes: #1514

Signed-off-by: Hui Zhu <teawater@hyper.sh>
2019-04-18 10:53:08 +08:00
Eric Ernst
8097c54e79 Firecracker: update to version v0.15.2
Fixes: #1559

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2019-04-17 16:09:40 -07:00
Eric Ernst
717a30bfe0
Merge pull request #1556 from katacontainersbot/1.7.0-alpha1-branch-bump
# Kata Containers 1.7.0-alpha1
2019-04-17 13:26:18 -07:00
katacontainers bot
0ec4d799f8 release: Kata Containers 1.7.0-alpha1
- s390x: fix gofmt complain
- factory: Make VMCache and VM templating can work together
- agent: pass correct mount type to agent for ephemeral volumes
- network: Make tcfilter model as default
- netmon: Fix bug in how routes are converted
- shimv2 should return grpc error codes
- Deprecate hyperstart, CC shim, and CC proxy
- virtcontainers: inherit parent's CPU constraint
- fix ut failure due to incorrect cleanup and make it runnable with non-root
- storage: create k8s emptyDir inside VM
- virtcontainers: prepend a kata specific string to host cgroups path
- Factory: enable template feature for arm64
- cli: fix kata-check test
- shimv2: fix the issue of stop container failed
- split sandbox/container state
- Make factory can control VMCache server
- vc: remove spawner
- s390x: fix golangci-lint complains
- qemu: Remove the storage directories if qemu get from the factory
- katautils: fix the issue of missing proxy debug config
- vendor: update govmm
- virtcontainers: Use only supported namespaces
- cri-o: Update repository URLs
- travis: use xenial
- runtime: support memory hotplug via probe interface on aarch64
- s390x: not set socketID and threadID
- config: fix panic in type assertion
- Fix yq issues

a013f9b s390x: fix gofmt complain
3852682 network: Change the package level network default
33bae70 network: Make tcfilter model as default
343a0d3 factory: Make VMCache and VM templating can work together
b08ab6a vc: modify ioctl function to handle shim test
c425079 vc: Deprecate CC proxy and shim
c658770 vc: remove virtc api cli
d4ef9c0 vc: deprecate hyperstart agent
8abd2ec netmon: Fix bug in how routes are converted
59e3956 virtcontainers: inherit parent's CPU constraint
f512542 sandbox: return ErrNoSuchContainer when failing to find a container
8215a3c shimv2: convert vc errors to grpc errors
cf90751 vc: export vc error types
9040f6a ut: fix UT failure due to incorrect cleanup
a0f49a9 ut: fix UT failure due to non-root
1a1f93b virtcontainers: add a kata specific prefix to host cgroups path
f7223c6 shimv2: fix the issue of stop container failed
76c4639 storage: create k8s emptyDir inside VM
98687a3 Template: enable template for arm64
16fe855 qemu: Remove the storage directories if qemu get from the factory
ace8115 factory: Make factory status can show status of VMCache server
f639787 factory: Make factory destroy can stop VMCache server
9ac6831 cli: fix kata-check test
a63013d vc: remove spawner
c414599 types: remove pid from sandbox state
03ee25d agent: treat container as shared pidns whenever it has pidns path
616f26c types: split sandbox and container state
8041fc9 katautils: fix the issue of missing proxy debug config
bb347ac vendor: update govmm
0b430dc virtcontainers: Use only supported namespaces
e31b040 cri-o: Update repository URL
ea1df84 travis: use xenial
b50292a arm64: dep ensure to keep track of the revisons in kata agent
3bfcdf7 agent: add interface memHotplugByProbe
47670fc memoryDevice: reconstruct memoryDevice
30a6a7d agent: acquire memory hotplug probe info via GetGuestDetails
4993dff ci: Add a failed execution check for curl
7fa0390 ci: set $yq_version to 2.3.0
19458ec s390x: fix golangci-lint
49be8ee s390x: not set socketID and threadID
1b6affe config: fix panic in type assertion
2d422a8 agent: pass correct mount type to agent for ephemeral volumes
228d151 mount: Add check for k8s host empty directory
70c1931 mounts: Add check for system volumes

Signed-off-by: katacontainers bot <katacontainersbot@katacontainers.io>
2019-04-17 17:39:20 +00:00
Jose Carlos Venegas Munoz
57b1ce9328
Merge pull request #1554 from alicefr/fix-gofmt
s390x: fix gofmt complain
2019-04-17 11:05:16 -05:00