Commit Graph

263 Commits

Author SHA1 Message Date
GabyCT
bdc9a66bd9
Merge pull request #1770 from jongwu/image_align
image_build: align image size to 128M for arm64
2021-05-12 09:38:06 -05:00
Chelsea Mafrica
e8038718aa osbuilder: Upgrade alpine version to 3.13.5
We are using an older version of alpine, so upgrade to latest 3.13.5.

Fixes #1817

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2021-05-07 16:34:45 -07:00
Jianyong Wu
2b0d5b252e image_build: align image size to 128M for arm64
There is an inconformity between qemu and kernel of memory alignment
check in memory hotplug. Both of qemu and kernel will do the start
address alignment check in memory hotplug. But it's 2M in qemu
while 128M in kernel. It leads to an issue when memory hotplug.

Currently, the kata image is a nvdimm device, which will plug into the VM as
a dimm. If another dimm is pluged, it will reside on top of that nvdimm.
So, the start address of the second dimm may not pass the alginment
check in kernel if the nvdimm size doesn't align with 128M.

There are 3 ways to address this issue I think:
1. fix the alignment size in kernel according to qemu. I think people
in linux kernel community will not accept it.
2. do alignment check in qemu and force the start address of hotplug
in alignment with 128M, which means there maybe holes between memory blocks.
3. obey the rule in user end, which means fix it in kata.

I think the second one is the best, but I can't do that for some reason.
Thus, the last one is the choice here.

Fixes: #1769
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-05-03 10:44:30 +08:00
GabyCT
503039482b
Merge pull request #1620 from eadamsintel/update-qat-dockerfile
osbuilder: Update QAT Dockerfile with new QAT driver version
2021-04-05 09:51:14 -05:00
Eric Ernst
d44412fe24
Merge pull request #1623 from egernst/custom-image-registry
osbuilder: update dockerfiles to utilize IMAGE_REGISTRY
2021-04-02 14:18:36 -07:00
Eric Ernst
935460e549 osbuilder: update dockerfiles to utilize IMAGE_REGISTRY
While we introduced IMAGE_REGISTRY, we didn't actually update the
corresponding Dockerfiles to utilize it. Let's add

Fixes: #1622

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-04-02 09:46:16 -07:00
Eric Adams
010d57f4e2 osbuilder: Update QAT Dockerfile with new QAT driver version
This fixes the QAT driver version and provides a check early in the
building process to make sure the driver exists. It also provides
hints to users on how to fix themselves if the driver changes again.

Fixes: #1618

Signed-off-by: Eric Adams <eric.adams@intel.com>
2021-04-01 19:20:44 +00:00
Fabiano Fidêncio
572aff53e8 build: Only keep one VERSION file
Instead of having different VERSION files spread accross the project,
let's always use the one in the topsrcdir and remove all the others,
keeping only a synlink to the topsrcdir one.

Fixes: #1579

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-31 23:51:20 +02:00
Chelsea Mafrica
8c1abc3b57
Merge pull request #1418 from Amulyam24/fix-docker-ppc64le
osbuiler: fixing USE_DOCKER for ppc64le
2021-03-19 17:22:12 -07:00
GabyCT
856ffb85fc
Merge pull request #1420 from eadamsintel/qat-dockerfile
osbuilder: Port QAT Dockerfile to 2.0 repo
2021-03-08 11:34:28 -06:00
Jianyong Wu
f580d33cc9 musl/arm64: decompression before use the tarball.
In the last fix, the decompression ops is deleted by mistake and need
add it back.

Fixes: #1490
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-03-04 10:12:29 +08:00
Jianyong Wu
2da058ed7b osbuild: build musl toolchain from source if needed
Currently, musl toolchain installation on arm64 is just downloading from
a website. It's unsafe in case the website corrupts. So build musl
toolchain from source if it can't be downloaded.

Fixes: #1481
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-03-03 17:22:39 +08:00
Adams, Eric
6417067d62 osbuilder: Port QAT Dockerfile to 2.0 repo
Update the Intel QAT Dockerfile to work with the 2.0 repos, fix some
bugs with building Debian/Ubuntu rootfs, and update the latest QAT
driver. Updated copyright.

Fixes: #1419

Signed-off-by: Adams, Eric <eric.adams@intel.com>
2021-03-01 17:34:04 -08:00
Adams, Eric
b412e159f9 osbuilder: Port QAT Dockerfile to 2.0 repo
Update the Intel QAT Dockerfile to work with the 2.0 repos, fix some
bugs with building Debian/Ubuntu rootfs, and update the latest QAT
driver. Updated copyright.

Fixes: #1419

Signed-off-by: Adams, Eric <eric.adams@intel.com>
2021-02-18 12:46:49 -08:00
Amulya Meka
5096103e7e osbuiler: fixing USE_DOCKER for ppc64le
For building rootfs with docker, glibc based rust target should be installed on ppc64le.
Additionally, protobuf-compiler would be required on ppc64le as it is not present by default.

Fixes: #1417

Signed-off-by: Amulya Meka <amulmek1@in.ibm.com>
2021-02-18 05:33:51 +00:00
Eric Ernst
49bdbac606 osbuilder: Allow image registry to be customizable
Give the user chance to specify their own registry in event the default
provided are not accessible, desirable.

Fixes: #1393

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-02-17 16:49:58 -08:00
Eric Ernst
cb6d2f3c40 osbuilder: alphabetize fields
Let's go ahead and list the usage info / fields in alphabetical order!

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-02-10 12:39:10 -08:00
Jakob-Naucke
11fe6a3552
osbuilder: Fix USE_DOCKER on s390x
- Install the required protobuf-compiler on Ubuntu
- Install correct libc Rust target (glibc on s390x)
- Do not skip Rust installation on s390x

Fixes: #1266

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-02-09 12:37:24 +01:00
Jakob Naucke
afb4197813 osbuilder: Build for glibc on s390x
Since there is no Rust target for musl on s390x, builds on s390x should use
glibc. This commit removes the "Cannot build Rust agent on s390x" restriction
in rootfs.sh and only installs musl in the build environment when it is
required.

Fixes: #1262

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-02-04 09:52:28 +01:00
Chelsea Mafrica
d4bc5952b1
Merge pull request #1340 from jing-wang4/rootfs-ppc64le
osbuilder: Enforcing LIBC=gnu to rootfs build for ppc64le
2021-02-03 16:06:05 -08:00
Jing Wang
71aeb920aa osbuilder: updates for feedback
Updates for review feedback.

Signed-off-by: Jing Wang <jing.wang4@ibm.com>
2021-02-02 14:41:23 +00:00
Tim Zhang
9a02c81eb9
Merge pull request #1313 from liubin/doc/add-distro-desc
osbuilder: add description for how to use DISTRO variable
2021-02-02 15:19:57 +08:00
bin
8e2b19ac77 osbuilder: add description for how to use DISTRO variable
Fixes: #1312

Signed-off-by: bin <bin@hyper.sh>
2021-02-02 14:52:09 +08:00
Jing Wang
9f7a7a4f86 osbuilder: Enforcing LIBC=gnu to rootfs build for ppc64le
To enforce LIBC=gnu to rootfs.sh for ppc64le, instead of error and exit if not set.

Fixes: #1339
Signed-off-by: Jing Wang <jing.wang4@ibm.com>
2021-01-29 20:22:31 +00:00
Bo Chen
03c5ecefef
Merge pull request #1306 from dgibson/no-cmake
osbuilder: remove traces of cmake
2021-01-26 16:26:02 -08:00
Wainer dos Santos Moschetta
fd39f0fa00 osbuilder: Add "Agent init" on terms glossary
Include on the terms glossary some words about the agent init as
later on README its explain how the agent can be switched between
systemd and init.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-01-26 10:51:27 -05:00
Wainer dos Santos Moschetta
1273e485d8 osbuilder: Fix urls to repositories
Changed the user-visible urls to point to the right Kata Containers
files/repositories.

Fixes #234

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-01-26 07:51:20 -05:00
Wainer dos Santos Moschetta
ba9fa49a53 osbuilder: Use Fedora and CentOS registries
To avoid hitting quota limit on docker.io, this changes the Fedora
and CentOS dockerfiles to pull the image from the projects registries.

Fixes #1324

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-01-26 07:51:07 -05:00
David Gibson
3718df69c2 osbuilder: Remove leftover pieces related to cmake
The osbuilder scripts include a bunch of code for installing cmake, even
though cmake is never invoked.  versions.yaml claims it's needed to build
grpc-rs, but that doesn't appear to be in our dependency graph.

Presumably, we used to need this, but don't any more.  So, remove all cmake
references.

Fixes #1309

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-01-23 15:24:25 +11:00
Fabiano Fidêncio
b329a74f18 rootfs: Fix indentation inside a switch
While touching this part of the code, let's help my OCD.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-01-08 22:46:24 +01:00
Fabiano Fidêncio
8879f9a09b rootfs: apparmor=unconfined is needed for non Red Hat host OSes
This is not needed for Fedora, RHEL, and CentOS, but it is required when
using any other host OS.  Having --security-opt apparmor=unconfined used
unconditionally is a no go as it'd break podman.

The reason this was only added when building for SUSE (as target distro)
was because debian and ubuntu condition would fall-through the switch to
the suse case (which makes me think that the fall-through was not
accidental).

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-01-08 21:35:10 +01:00
Fabiano Fidêncio
bbeebcdbba rootfs: Always add SYS_ADMIN, CHROOT, and MKNOD caps to docker cmdline
We use those, independently of the distro.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-01-08 21:35:10 +01:00
Fabiano Fidêncio
90ec2fa802 rootfs: Don't fallthrough in the docker_extra_args() switch
Falling through the switch cases in docker_extra_args() looks like a
typo and causes issues when building with podman, as `--security-opt
apparmor=unconfinded" shouldn't be passed if Apparmor is no enable on
the system.

Fixes: #1241

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-01-08 21:35:10 +01:00
GabyCT
a6d52d3da1
Merge pull request #1208 from GabyCT/topic/addgithu
github: Add github actions
2021-01-08 14:27:19 -06:00
Archana Shinde
ebd9fcc2c3 actions: Run static checks before make agent
Run static checks prior to building the agent.Checks
fail if run after since the compilation process
produces new rust code.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2021-01-08 11:04:54 -06:00
Peng Tao
d6a6f68fd7
Merge pull request #1177 from devimc/2020-12-07/osbuilder/gentoo
[forwardport] Add support for Gentoo
2020-12-23 18:39:12 +08:00
Julio Montes
8ac93f65fd rootfs-builder: add support for gentoo
Generate images based on gentoo

fixes #1178

Signed-off-by: Julio Montes <julio.montes@intel.com>
2020-12-10 09:36:15 -06:00
Julio Montes
9897238f3a rootfs: reduce size of debian image
Improve Kata Containers memory footprint by reducing debian
image size.

Without this change:
Debian image -> 256MB

With this change:
Debian image -> 128MB

Note: this change *will not* impact ubuntu image.

fixes #1188

Signed-off-by: Julio Montes <julio.montes@intel.com>
2020-12-10 09:23:48 -06:00
Julio Montes
faed2369a0 rootfs-builder: add functions to run before and after the container
Define `before_starting_container` and `after_stopping_container`
functions, these functions run before and after the container that
builds the rootfs respectively.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2020-12-08 10:14:00 -06:00
Ralf Haferkamp
e3cdc89b6c osbuilder: Create target directory for agent
When building with AGENT_SOURCE_BIN pointing to an already built
kata-agent binary, the target directory needs to be created in the
rootfs tree.

Fixes #873

Signed-off-by: Ralf Haferkamp <rhafer@suse.com>
2020-10-08 17:07:40 +02:00
zhanghj
ca501e5478 osbuilder: specify default toolchain verion in rust-init.
Specify default toolchain version in rust-init.

Fixes: #799

Signed-off-by: zhanghj <zhanghj.lc@inspur.com>
2020-09-24 23:20:43 -04:00
Archana Shinde
55354d5509
Merge pull request #651 from Kvasscn/fix-rust-agent-default-value-rootfs.sh
osbuilder: update usage of RUST_AGENT variable
2020-09-10 13:35:11 -07:00
Pradipta Kumar
8667df4d09
Merge pull request #679 from Amulyam24/fix-initrd-ppc64le
osbuilder: fix rootfs build on ppc64le
2020-09-09 12:32:28 +05:30
Amulya Meka
ebd3f316f2 osbuilder: fix rootfs build on ppc64le
The default toolchain used for building the
rust agent would be different for ppc64le.

Fixes: #678

Signed-off-by: Amulya Meka <amulmek1@in.ibm.com>
2020-09-09 03:09:03 +00:00
Pavel Mores
2dfb8bc549 rootfs-builder: fix unbootable dracut-based initramfs on Fedora
This is a forward port of Kata 1.x PR's
https://github.com/kata-containers/osbuilder/pull/480 and
https://github.com/kata-containers/osbuilder/pull/494 .

Fixes #646

Signed-off-by: Pavel Mores <pmores@redhat.com>
2020-09-08 20:10:38 +02:00
Bo Chen
d2a89d668e
Merge pull request #569 from devimc/2020-08-25/port/noReflink
[port] image-builder: disable reflink
2020-09-04 12:50:01 -07:00
zhanghj
dae6c7d967 osbuilder: update usage of RUST_AGENT variable
Update the default value of RUST_AGENT variable to 'yes' in
usage description.

fixes: #650

Signed-off-by: zhanghj <zhanghj.lc@inspur.com>
2020-09-01 05:23:29 -04:00
Peng Tao
c624fa7469 osbuilder: install musl for aarch64
So that it can do musl static build.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-08-31 22:10:32 +08:00
Peng Tao
cf1b72d6c6 osbuilder: install rust before sourcing cargo env
Otherwise we cannot find it.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-08-31 22:09:37 +08:00
Peng Tao
76c18aa345 osbuilder: fix alpine agent build
Since we always build musl kata-agent, there is no need to build
it inside a musl container. We can just build on the host and then
copy the binary to the target rootfs.

There are still a lot to clean up and it should be made so for ALL
target distros instead of just alpine. But this is at least working
for alpine first.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-08-31 22:09:37 +08:00
Peng Tao
dadab1febf osbuilder: build rust agent by default
Since we only have the rust agent.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-08-31 22:09:37 +08:00
Gabriela Cervantes
f879acd6e7 scripts: Foward port osbuilder scripts to update yq
This PR ports kata-containers/osbuilder#454 to kata 2.0

Fixes #576

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2020-08-27 15:00:03 -05:00
Julio Montes
f7ff6d3297 image-builder: disable reflink
Disable reflink when using DAX. Reflink is a xfs feature that cannot be
used together with DAX.

fixes kata-containers/osbuilder#456
fixes #577

Signed-off-by: Julio Montes <julio.montes@intel.com>
2020-08-26 09:42:17 -05:00
Amulya Meka
64b0694436 ppc64le: Support for rust agent based rootfs
For building rust agent on ppc64le, the rust toolchain is built using
the LIBC implementation - gnu instead of musl.

Fixes: #481

Signed-off-by: Amulya Meka <amulmek1@in.ibm.com>
2020-08-18 16:00:26 +05:30
liangxianlong
7bbb9e8122 rootfs-builder: Don't modify /sbin/init on the build host
Don't modify /sbin/init on the build host when using command `AGENT_INIT="yes" ./rootfs.sh centos` to build rootfs.

Fixes: #430

Signed-off-by: liangxianlong <liang.xianlong@zte.com.cn>
2020-07-18 14:19:06 +08:00
Peng Tao
a8430b3791 gitignore: ignore more files
We don't need to track them.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-06-29 01:27:05 -07:00
Peng Tao
ebfbca031b osbuilder: use newest golang
Instead of the oldest one.

Fixes: #345
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-06-27 20:42:30 -07:00
Peng Tao
a1ef594d2a cleanup: remove redundant files
And use top level VERSION for all components.

Fixes: #334
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-06-24 15:57:23 -07:00
Salvador Fuentes
81e11c9f7c osbuilder: remove references to agent and runtime repos
tools/osbuilder/scripts/lib.sh was making references to agent and
runtime repositories to get golang, rust, cmake and musl versions.
Since runtime and agent repos are consolidating, we only need to
make reference to our versions.yaml in this (kata-containers) repo.

Fixes: #234.

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2020-05-28 10:33:25 -05:00
Salvador Fuentes
629cc0ae8d rootfs-builder: remove EulerOS
EulerOS does not work with rust agent.

further info:
https://github.com/kata-containers/osbuilder/issues/387

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2020-04-30 08:27:25 -05:00
Salvador Fuentes
586d26480c rootfs-builder: Add license header to rootfs_lib_template.sh
Add license header to `tools/osbuilder/rootfs-builder/template/rootfs_lib_template.sh`

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2020-04-30 08:20:46 -05:00
Salvador Fuentes
11c3d81f68 docs: fix markdown check issues
- Removes `CODE_OF_CONDUCT.md` and `CONTRIBUTING.md` from osbuilder
directory.
- Fixes a reference from `image-builder/README.md` to
`rootfs-builder/README.md`
- Updates the main `README.md` making a reference to the local
`tools/osbuilder/README.md`

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2020-04-29 17:51:44 -05:00
Salvador Fuentes
715d342519 osbuilder: move code into tools directory
move all osbuilder files into `tools` directory to be able
to merge this into kata-containers repo.

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2020-04-29 16:45:00 -05:00