Commit Graph

3026 Commits

Author SHA1 Message Date
Sebastien Boeuf
20b087e3d2
Merge pull request #264 from devimc/topic/supportDAXandFC
image-builder: re-implement image builder script
2019-03-28 06:56:02 -07:00
Ace-Tang
096fa046f8 qemu: fix qemu leak when failed to start container
do cleanup inside startVM() if start vm get error

Fixes: #1426

Signed-off-by: Ace-Tang <aceapril@126.com>
2019-03-28 19:38:56 +08:00
Ace-Tang
1b6affe498 config: fix panic in type assertion
when use shim v2 interface to run container, no need to use kata-proxy
and kata-shim, remove kata-proxy and kata-shim in config file will cause
panic since type assertion. add check to avoid panic

Fixes: #1440

Signed-off-by: Ace-Tang <aceapril@126.com>
2019-03-28 19:28:51 +08:00
Xie Yongji
2d422a845b agent: pass correct mount type to agent for ephemeral volumes
The "ephemeral" is just used to indicate ephemeral volumes in
runtime. We should not pass it to agent. Instead, "bind" should be
the correct mount type to be passed.

Fixes: #1438

Signed-off-by: Xie Yongji <xieyongji@baidu.com>
2019-03-28 18:40:59 +08:00
Penny Zheng
2e5194e279 linter: remove deadcode linter check for generic item
After we switched golang linter to golangci-lint, we has extra 'deadcode'
linter check, and we need to remove this linter check for all
generic items.

Fixes: #1432

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2019-03-28 14:05:38 +08:00
zhangwei_cs
c89eb81dec
Merge pull request #1425 from bergwolf/proxy-path
config: validate proxy path
2019-03-27 11:31:39 +08:00
Julio Montes
d8cdd88ace image-builder: re-implement image builder script
Re-implement image builder script to generate an image with a double MBR +
a DAX metadata. The DAX metadata is read by the NVDIMM driver to know the
beginning of the data in the pmem device.
This new image format is required to enable DAX in the kernels and hypervisors
that support NVDIMM, without breaking the compatibility with the kernels and
hypervisors that don't support it.

Following diagram shows how the resulting image will look like

```
	.-----------.----------.---------------.-----------.
	| 0 - 512 B | 4 - 8 Kb |  2M - 2M+512B |    3M     |
	|-----------+----------+---------------+-----------+
	|   MBR #1  |   DAX    |    MBR #2     |  Rootfs   |
	'-----------'----------'---------------'-----------+
	      |          |      ^      |        ^
	      |          '-data-'      '--------'
	      |                                 |
	      '--------rootfs-partition---------'
```

MBR: Master boot record.
DAX: Metadata required by the NVDIMM driver to enable DAX in the guest [1][2]
(struct nd_pfn_sb).
Rootfs: partition that contains the root filesystem (/usr, /bin, etc).

Kernels and hypervisors that support DAX/NVDIMM read the MBR #2,
otherwise MBR #1 is read.

[1] - https://github.com/kata-containers/osbuilder/blob/master/image-builder/\
nsdax.gpl.c
[2] - https://github.com/torvalds/linux/blob/master/drivers/nvdimm/pfn.h

fixes #263

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-26 09:02:45 -06:00
Julio Montes
f32ae14883 tests: remove DAX env variable
DAX envar is no more required to generate images with support for DAX

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-26 09:02:45 -06:00
Julio Montes
f355c026c0 tests: enable DEBUG
run tests with debug enabled to detect errors easier

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-26 09:02:45 -06:00
Peng Tao
dca7a6f98b config: validate proxy path
Like shim, we should validate the proxy path if it is provided.

Fixes: #1424

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2019-03-26 03:46:36 -07:00
Wei Zhang
dd6e8eb82c tests: do cleanUp() always in the end
Fixes: #1422

Detect failing test case:

```
....
=== RUN   TestEnterContainerFailingContNotStarted
--- PASS: TestEnterContainerFailingContNotStarted (0.01s)
=== RUN   TestEnterContainer
--- FAIL: TestEnterContainer (0.00s)
 Error Trace: sandbox_test.go:1154
 Error:      	Expected value not to be nil.
 Messages:   	Entering non-running container should fail
 Error Trace: sandbox_test.go:1157
 Error:      	Expected nil, but got: &errors.errorString{s:"Can not
move from running to running"}
 Messages:   	Failed to start sandbox: Can not move from running to
running
FAIL
```

`TestEnterContainerFailingContNotStarted` calls `cleanUp` at function
begging but it doesn't clean its garbage after it ends.
`TestEnterContainer` only call `cleanUp` in the end but it doesn't do
cleanUp in the begging, that gives first test case a chance to impact
latter one.

This commit modifies all the test cases, let them all do the cleanUp()
in the end.

The policy here is: "everyone needs to take their garbage away when they
leave" :)

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-03-26 16:29:39 +08:00
Hui Zhu
c6804c8e2b
Merge pull request #1372 from ganeshmaharaj/go-linter-change
lint: Change go linter from gometalinter to golangci-lint
2019-03-26 12:32:38 +08:00
Wei Zhang
ad7d9b7bab cgroups: remove duplicate fields from state
Fixes: #1415

Container resources have been saved to ContainerConfig so there's no
need to save it again in state.json.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-03-26 10:34:03 +08:00
Archana Shinde
228d1512d9 mount: Add check for k8s host empty directory
k8s host empty-dir is equivalent to docker volumes.
For this case, we should just use the host directory even
for system directories.

Move the isEphemeral function to virtcontainers to not
introduce cyclic dependency.

Fixes #1417

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2019-03-25 14:06:23 -07:00
Archana Shinde
70c193132d mounts: Add check for system volumes
We handle system directories differently, if its a bind mount
we mount the guest system directory to the container mount and
skip the 9p share mount.
However, we should not do this for docker volumes which are directories
created by Docker.

This introduces a Docker specific check, but that is the only
information available to us at the OCI layer.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2019-03-25 10:49:27 -07:00
Graham Whaley
f56d26105b
Merge pull request #37 from ttx/packagecloud
Credit packagecloud for package hosting
2019-03-25 16:47:41 +00:00
Ganesh Maharaj Mahalingam
f4428761cb lint: Update go linter from gometalinter to golangci-lint.
gometalinter is deprecated and will be archived April '19. The
suggestion is to switch to golangci-lint which is apparently 5x faster
than gometalinter.

Partially Fixes: #1377

Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
2019-03-25 08:48:13 -07:00
Julio Montes
6c7f3077c9 image-builder: add xfsprogs to Dockerfile
xfsprogs is a package required to generate xfs images.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-25 08:20:49 -06:00
Julio Montes
ad6e1a9d59 scripts: implement error function
Implement function to log errors without exit.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-25 08:19:13 -06:00
Julio Montes
e888ef5a60
Merge pull request #260 from darfux/pass_debug_to_docker
builder: Pass the DEBUG flag when using docker
2019-03-25 07:25:12 -06:00
James O. D. Hunt
dcadf2cbfb
Merge pull request #256 from kata-containers/add-chrony-to-image
Add chrony to image
2019-03-25 10:59:16 +00:00
James O. D. Hunt
0b7e456d47
Merge pull request #250 from matthewfischer/max_attempts
Print attempt number after max check
2019-03-25 10:32:49 +00:00
Li Yuxuan
0d2ba4766e builder: Pass the DEBUG flag when using docker
When using docker, pass the `DEBUG` flag to trace the commands as well.

Fixes: #261

Signed-off-by: Li Yuxuan <liyuxuan04@baidu.com>
2019-03-25 17:36:38 +08:00
Hui Zhu
814e5de224
Merge pull request #1334 from jongwu/factory
Factory: Fix fake return value issue on creating template
2019-03-25 11:58:30 +08:00
Xu Wang
de9c42e80f
Merge pull request #1227 from WeiZhang555/experimental-support
config: Add config flag "experimental"
2019-03-23 08:59:45 +08:00
James O. D. Hunt
1f52f5e7bd
Merge pull request #1403 from jodh-intel/add-more-build-targets
Add more build targets
2019-03-22 11:52:32 +00:00
James O. D. Hunt
0d146738de
Merge pull request #1405 from jodh-intel/update-collect-script-for-dax-nvdimm-images
Update collect script for dax nvdimm images
2019-03-22 09:54:03 +00:00
James O. D. Hunt
36fce98517
Merge pull request #1387 from jodh-intel/allow-data-collection-to-be-hidden
Allow data collection to be hidden
2019-03-22 09:53:44 +00:00
Xu Wang
4f712b0657
Merge pull request #1401 from teawater/vmcache_vsock
config: Make VMCache can work with vsock
2019-03-22 10:51:35 +08:00
Salvador Fuentes
a06c82a120
Merge pull request #257 from jcvenegas/release-1.6.0
release: 1.6.0
2019-03-21 13:28:49 -06:00
Jose Carlos Venegas Munoz
75f4338350 release: 1.6.0
Version bump to kata 1.6.0.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2019-03-21 13:00:15 -06:00
James O. D. Hunt
e1eb28836a
Merge pull request #249 from GabyCT/topic/modifytest
test: Modify test_images script to handle FC
2019-03-21 17:04:32 +00:00
Julio Montes
a7ccc24c80
Merge pull request #1398 from teawater/curl
ci: check curl before use it
2019-03-21 08:17:38 -06:00
James O. D. Hunt
5d761cec76 scripts: Handle images with a DAX/NVDIMM header
osbuilder recently added the ability to create images with a DAX/NVDIMM
header [1], however this change broke the data collection script. Update
that script to handle images with and without this header.

The data collection script will now assume a header is present. However,
if it fails to find the required partition data, it will try again, this
time assuming the image does not have a DAX/NVDIMM header.

Fixes #1404.

[1] - https://github.com/kata-containers/osbuilder/pull/236

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-03-21 12:35:28 +00:00
James O. D. Hunt
bdf6b2d49d scripts: Handle missing partitions in collect script
Add an extra check in the data collection script to ensure partitions
are found in the image.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-03-21 12:28:58 +00:00
James O. D. Hunt
ad228e3c3b build: Add missing targets to show-usage
Add a number of useful build and install targets to the `show-usage`
target which are visible when the user runs `make help`.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-03-21 11:30:19 +00:00
James O. D. Hunt
e6a7091981 build: Allow runtime to be built+installed without shim
Add `install-runtime` and `install-netmon` targets. This allows the
`install` target to be simplified and also allows the runtime to be
built without having to build the `containerd-shim-v2` binary which is
slow to build:

```
$ make runtime && sudo -E PATH=$PATH make install-runtime
```

Fixes #1402.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-03-21 11:30:12 +00:00
James O. D. Hunt
206ffc66aa build: Don't build the runtime when building shim binary
The `containerd-shim-v2` binary does not need the `kata-runtime` binary
to be built first, so remove the dependency.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-03-21 11:30:07 +00:00
Graham Whaley
c70ba4844f
Merge pull request #1390 from devimc/topic/roFS
virtcontainers: improve security and mount the rootfs as read-only fs
2019-03-21 09:33:04 +00:00
Hui Zhu
639e8271de config: Make VMCache can work with vsock
After code check and test, found VMCache can work with vsock.
Remove the code that prohibit them from working together.

Fixes: #1400

Signed-off-by: Hui Zhu <teawater@hyper.sh>
2019-03-21 17:13:03 +08:00
Hui Zhu
aec0d263fa ci: check curl before use it
install-yq.sh use curl but not check if curl is available or not.
Add code to check curl before use it.

Fixes: #1379

Signed-off-by: Hui Zhu <teawater@hyper.sh>
2019-03-21 10:52:33 +08:00
Archana Shinde
e16ff37f86 chrony: Comment out any NTP sources for chrony
Reference:  https://chrony.tuxfamily.org/doc/3.4/chrony.conf.html

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2019-03-20 16:05:33 -07:00
Archana Shinde
3df19ff984 chrony: Add virtual PTP as source for chrony
KVM virtual PTP in linux kernel allows guest to sync its
clock to the host clock with high precision. kvm-ptp has been
enabled in our kernel. Add this as a source for `chrony` so that
it can be used to sync the guest system clock.
`chrony` needs to be started in the guest for time sync.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2019-03-20 14:05:42 -07:00
Sebastien Boeuf
f1ef63e5c6
Merge pull request #1394 from WeiZhang555/improve-readability
refactor: improve readability of `bumpAttachCount`
2019-03-20 08:13:45 -07:00
Julio Montes
9b73900ba6 katautils: mask systemd-random-seed
systemd-random-seed service fails if the rootfs is a read-only fs.
systemd-random-seed restores the random seed of the system at early
boot and saves it at shutdown, since kata containers are one boot machines
this service is not needed.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-20 09:02:20 -06:00
Julio Montes
64984667ad virtcontainers: improve security and mount the rootfs as read-only fs
Mounting the rootfs as read-only fs the binaries can't be modified.

fixes #1389

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-20 07:50:20 -06:00
Wei Zhang
26a9b72c34 refactor: improve readability of bumpAttachCount
Fixes #1392

Improve code readability of function `device.bumpAttachCount`

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-03-20 11:38:49 +08:00
Matt Fischer
ad5d879f8c rootfs: Print attempt number after max check
With the old code it was possible to see odd messages like:
"INFO: Create root disk image. Attempt 6 out of 5."

Move the attempt number print to after we check against the max

Fixes #251

Signed-off-by: Matt Fischer <matt@mattfischer.com>
2019-03-19 21:04:40 -06:00
Archana Shinde
8fe64058aa rootfs: Include chrony in the Dockerfile images
Add chrony to distro Dockerfile.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2019-03-19 14:34:14 -07:00
Archana Shinde
0b33519709 rootfs: add PACKAGE var to debian config
debian config seems to be missing PACKAGE variable altogether.
Add it along with appending chrony to the list.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2019-03-19 14:30:10 -07:00