Commit Graph

292 Commits

Author SHA1 Message Date
Julio Montes
d8cdd88ace image-builder: re-implement image builder script
Re-implement image builder script to generate an image with a double MBR +
a DAX metadata. The DAX metadata is read by the NVDIMM driver to know the
beginning of the data in the pmem device.
This new image format is required to enable DAX in the kernels and hypervisors
that support NVDIMM, without breaking the compatibility with the kernels and
hypervisors that don't support it.

Following diagram shows how the resulting image will look like

```
	.-----------.----------.---------------.-----------.
	| 0 - 512 B | 4 - 8 Kb |  2M - 2M+512B |    3M     |
	|-----------+----------+---------------+-----------+
	|   MBR #1  |   DAX    |    MBR #2     |  Rootfs   |
	'-----------'----------'---------------'-----------+
	      |          |      ^      |        ^
	      |          '-data-'      '--------'
	      |                                 |
	      '--------rootfs-partition---------'
```

MBR: Master boot record.
DAX: Metadata required by the NVDIMM driver to enable DAX in the guest [1][2]
(struct nd_pfn_sb).
Rootfs: partition that contains the root filesystem (/usr, /bin, etc).

Kernels and hypervisors that support DAX/NVDIMM read the MBR #2,
otherwise MBR #1 is read.

[1] - https://github.com/kata-containers/osbuilder/blob/master/image-builder/\
nsdax.gpl.c
[2] - https://github.com/torvalds/linux/blob/master/drivers/nvdimm/pfn.h

fixes #263

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-26 09:02:45 -06:00
Julio Montes
f32ae14883 tests: remove DAX env variable
DAX envar is no more required to generate images with support for DAX

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-26 09:02:45 -06:00
Julio Montes
f355c026c0 tests: enable DEBUG
run tests with debug enabled to detect errors easier

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-26 09:02:45 -06:00
Julio Montes
6c7f3077c9 image-builder: add xfsprogs to Dockerfile
xfsprogs is a package required to generate xfs images.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-25 08:20:49 -06:00
Julio Montes
ad6e1a9d59 scripts: implement error function
Implement function to log errors without exit.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-25 08:19:13 -06:00
Julio Montes
e888ef5a60
Merge pull request #260 from darfux/pass_debug_to_docker
builder: Pass the DEBUG flag when using docker
2019-03-25 07:25:12 -06:00
James O. D. Hunt
dcadf2cbfb
Merge pull request #256 from kata-containers/add-chrony-to-image
Add chrony to image
2019-03-25 10:59:16 +00:00
James O. D. Hunt
0b7e456d47
Merge pull request #250 from matthewfischer/max_attempts
Print attempt number after max check
2019-03-25 10:32:49 +00:00
Li Yuxuan
0d2ba4766e builder: Pass the DEBUG flag when using docker
When using docker, pass the `DEBUG` flag to trace the commands as well.

Fixes: #261

Signed-off-by: Li Yuxuan <liyuxuan04@baidu.com>
2019-03-25 17:36:38 +08:00
Salvador Fuentes
a06c82a120
Merge pull request #257 from jcvenegas/release-1.6.0
release: 1.6.0
2019-03-21 13:28:49 -06:00
Jose Carlos Venegas Munoz
75f4338350 release: 1.6.0
Version bump to kata 1.6.0.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2019-03-21 13:00:15 -06:00
James O. D. Hunt
e1eb28836a
Merge pull request #249 from GabyCT/topic/modifytest
test: Modify test_images script to handle FC
2019-03-21 17:04:32 +00:00
Archana Shinde
e16ff37f86 chrony: Comment out any NTP sources for chrony
Reference:  https://chrony.tuxfamily.org/doc/3.4/chrony.conf.html

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2019-03-20 16:05:33 -07:00
Archana Shinde
3df19ff984 chrony: Add virtual PTP as source for chrony
KVM virtual PTP in linux kernel allows guest to sync its
clock to the host clock with high precision. kvm-ptp has been
enabled in our kernel. Add this as a source for `chrony` so that
it can be used to sync the guest system clock.
`chrony` needs to be started in the guest for time sync.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2019-03-20 14:05:42 -07:00
Matt Fischer
ad5d879f8c rootfs: Print attempt number after max check
With the old code it was possible to see odd messages like:
"INFO: Create root disk image. Attempt 6 out of 5."

Move the attempt number print to after we check against the max

Fixes #251

Signed-off-by: Matt Fischer <matt@mattfischer.com>
2019-03-19 21:04:40 -06:00
Archana Shinde
8fe64058aa rootfs: Include chrony in the Dockerfile images
Add chrony to distro Dockerfile.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2019-03-19 14:34:14 -07:00
Archana Shinde
0b33519709 rootfs: add PACKAGE var to debian config
debian config seems to be missing PACKAGE variable altogether.
Add it along with appending chrony to the list.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2019-03-19 14:30:10 -07:00
Archana Shinde
510ddd28c8 rootfs: Add chrony service to rootfs
chrony will be used to schronize guest clock with host
using kvm_ptp kernel driver.
This does add another active component to the rootfs
but keeping time scychorized is crucial.

Fixes #255

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2019-03-19 14:12:21 -07:00
Gabriela Cervantes
523405e62d test: Modify test_images script to handle FC
We need to modify this test in order to handle firecracker.

Fixes #248

Depends-on:github.com/kata-containers/osbuilder#247

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2019-03-19 13:42:00 -06:00
Jose Carlos Venegas Munoz
d67aad893f
Merge pull request #247 from devimc/topic/fixFCNoDax
image-builder: make DAX support optional
2019-03-19 11:51:19 -06:00
Julio Montes
6217689cc0
Merge pull request #253 from bergwolf/1.6.0-rc2-branch-bump
# Kata Containers 1.6.0-rc2
2019-03-19 09:50:19 -06:00
Peng Tao
6570944b67 release: Kata Containers 1.6.0-rc2
- image-builder: fill out device namespace information into kata image
- tests: install kata components from master
- tests: skip euleros
- arch: Remove calls to arch command
- pullapprove: remove it
- memory_hotplug: MEM_BOUNDARY_MB should be arch-dependent

726f798 image-builder: fill out device namespace information into kata image
cbe5642 image-builder: add gcc as dependecy to generate the image
7620066 image-builder: sync rootfs data after copying it into the image
dc5bc07 image-builder: fix mem boundary recalculation
71ccc0a mage-builder: remove -s option and IMG_SIZE envar
8065bb6 rootfs-builder: delete dnf and rmp data
35588dd tests: install kata containers components from master
519bbe8 tests: remove docker before installing the newer version
b4de168 tests: skip euleros
f38c67d arch: Remove calls to arch command
8963b8e pullapprove: remove it
e4d5c1b memory_hotplug: MEM_BOUNDARY_MB should be arch-dependent

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2019-03-18 22:14:31 +08:00
Julio Montes
02fa22bbdd image-builder: make DAX support optional
Not all hypervisor support NVDIMM hence DAX support MUST BE enabled explicitly
setting the DAX environment variable to 'yes'

fixes #246

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-15 13:14:52 -06:00
James O. D. Hunt
ecd072430f
Merge pull request #236 from devimc/topic/fixDAX
image-builder: fill out device namespace information into kata image
2019-03-14 08:35:50 +00:00
Julio Montes
726f798ff7 image-builder: fill out device namespace information into kata image
The new NVDIMM driver implementation (kernel >= 4.16) needs to know the device
namespace information to map pages, this metadata is read from the nvdimm
namespace at 4k offset.

fixes #235

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-13 13:05:06 -06:00
Julio Montes
cbe5642b9d image-builder: add gcc as dependecy to generate the image
gcc is required to build the binary in charge to fill out the device
namespace information (matadata) into the kata containers image.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-13 13:05:06 -06:00
Julio Montes
7620066c8a image-builder: sync rootfs data after copying it into the image
Rootfs data must be sync'd after copying it into the image to avoid
data corruption

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-13 13:05:06 -06:00
Julio Montes
dc5bc07825 image-builder: fix mem boundary recalculation
$/${} is unnecessary on arithmetic variables. [SC2004]

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-13 13:05:06 -06:00
Julio Montes
71ccc0a6ea mage-builder: remove -s option and IMG_SIZE envar
guest kernel needs 64 bytes of DRAM per 4K page of emulated PMEM, hence the
image size should be as small as possible to reduce the container's memory
footprint. The image size is recalculated automatically if it's too small
to contain the rootfs.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-13 13:05:06 -06:00
Julio Montes
8065bb615a rootfs-builder: delete dnf and rmp data
dnf and rmp data are not needed in the final rootfs, removing them we save
2MB of disk

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-13 13:05:06 -06:00
Salvador Fuentes
da8101a2aa
Merge pull request #243 from devimc/topic/testMaster
tests: install kata components from master
2019-03-13 13:04:41 -06:00
Julio Montes
35588dd303 tests: install kata containers components from master
build and install kata containers components from master and
run osbuilder tests.

fixes #242

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-13 10:47:03 -06:00
Julio Montes
f9f2d925ba
Merge pull request #239 from devimc/topic/eulerosMirrors
tests: skip euleros
2019-03-11 16:29:47 -06:00
Julio Montes
519bbe8f66 tests: remove docker before installing the newer version
In some distros docker is already installed and the CI fails because the newer
version can't be installed because it conflicts with the old version. Remove
old version before installing the newer version of docker.

fixes #241

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-11 15:20:22 -06:00
Julio Montes
b4de168cf4 tests: skip euleros
euleros mirrors are down almost all time, don't fail if euleros rootfs
or image can't be generated.

fixes #238

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-11 11:46:17 -06:00
Graham Whaley
da9f541deb
Merge pull request #227 from jodh-intel/remove-arch-calls
arch: Remove calls to arch command
2019-02-28 11:30:08 +00:00
James O. D. Hunt
f38c67da0c arch: Remove calls to arch command
The `arch(1)` command is not available on some systems so use the
`uname(1)` command for the equivalent functionality.

Fixes #150.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-02-18 08:36:22 +00:00
James O. D. Hunt
56f1044e12
Merge pull request #229 from grahamwhaley/20190206_remove_pullapprove
pullapprove: remove it
2019-02-07 17:19:48 +00:00
Graham Whaley
8963b8e3c9 pullapprove: remove it
We are moving off pullapprove. Remove its config file.

Fixes: #228

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-02-06 16:14:24 +00:00
James O. D. Hunt
1e89f7ed68
Merge pull request #225 from Pennyzct/memory-hotplug
memory_hotplug: MEM_BOUNDARY_MB should be arch-dependent
2019-01-17 11:18:29 +00:00
Penny Zheng
e4d5c1b751 memory_hotplug: MEM_BOUNDARY_MB should be arch-dependent
As runtime/#624(https://github.com/kata-containers/runtime/pull/624#discussion_r212534586)
discussed before, the size of memory section is arch-dependent.
For arm64, it should be 1G, not 128MB.

Fixes: #224

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2019-01-16 11:16:26 +08:00
Eric Ernst
53215acc8e
Merge pull request #222 from bergwolf/1.5.0-rc2-branch-bump
# Kata Containers 1.5.0-rc2
2018-12-21 12:29:34 -08:00
Peng Tao
d890478fa2 release: Kata Containers 1.5.0-rc2
- docs: update compatibility matrix with s390
- setup: Add make and gcc as dependencies
- suse: fix port URL detection for aarch64
- s390x: set CC for fedora
- Add 'install' target to makefile
- rootfs: Bump golang version to 1.11.1
- tests: skip docker,kata install with KATA_DEV_MODE

c3d77ae docs: add s390 to the compatibility matrix
be97380 setup: Add make and gcc as dependencies
de7fe19 s390x: set CC for fedora
3366a32 README: add ARM architecture to openSUSE
804286f suse: fix port URL detection for aarch64
434fff8 rootfs_builder: Dont fail if GOPATH not defined.
1bbf1e4 rootfs: Allow use host repositories.
f994560 rootfs: Bump golang version to 1.11.1
06c8d88 makefile: Add make install
9c0773a test: avoid errors in exit_handler
eb92306 tests: skip docker,kata install with KATA_DEV_MODE

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-12-21 17:11:51 +08:00
Julio Montes
bb06a0498a
Merge pull request #221 from alicefr/s390_docu
docs: update compatibility matrix with s390
2018-12-17 07:42:08 -06:00
Alice Frosi
c3d77aea6a docs: add s390 to the compatibility matrix
Fixes: #198

Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
2018-12-17 10:39:09 +01:00
Salvador Fuentes
4fc6e33ccc
Merge pull request #220 from chavafg/topic/fix-dependencies-fedora
setup: Add make and gcc as dependencies
2018-12-14 10:58:32 -06:00
Salvador Fuentes
be97380a3b setup: Add make and gcc as dependencies
We require make and gcc to run our CI tools.

Fixes: #219.

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2018-12-14 08:09:52 -06:00
Julio Montes
121de2ccf3
Merge pull request #216 from marcov/suse-arm-repo-fix
suse: fix port URL detection for aarch64
2018-12-14 07:08:45 -06:00
Julio Montes
695702b61b
Merge pull request #218 from alicefr/master
s390x: set CC for fedora
2018-12-14 07:08:21 -06:00
Alice Frosi
de7fe193ab s390x: set CC for fedora
Fixes: #217

Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
2018-12-13 14:40:44 +01:00