Commit Graph

110 Commits

Author SHA1 Message Date
Wainer dos Santos Moschetta
993dcc94ff osbuilder: delint dockerfiles
Removed all errors/warnings pointed out by hadolint version 2.7.0, except for the following
ignored rules:
  - "DL3008 warning: Pin versions in apt get install"
  - "DL3041 warning: Specify version with `dnf install -y <package>-<version>`"
  - "DL3033 warning: Specify version with `yum install -y <package>-<version>`"
  - "DL3048 style: Invalid label key"
  - "DL3003 warning: Use WORKDIR to switch to a directory"
  - "DL3018 warning: Pin versions in apk add. Instead of apk add <package> use apk add <package>=<version>"
  - "DL3037 warning: Specify version with zypper install -y <package>[=]<version>"

Fixes #3107
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2022-01-03 10:52:43 +02:00
Fabiano Fidêncio
1f61be842d osbuilder: Add protoc to the alpine container
It seems the lack of protoc in the alpine containers is causing issues
with some of our CIs, such as the VFIO one.

Fixes: #3323

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-12-29 14:40:24 +02:00
zhanghj
d2d8f9ac65 osbuilder: avoid to copy versions.txt which already deprecated
Currently the versions.txt in rootfs-builder dir is already removed,
so avoid to copy it in list of helper files.

Fixes: #3267

Signed-off-by: zhanghj <zhanghj.lc@inspur.com>
2021-12-29 14:39:34 +02:00
Snir Sheriber
572b25dd35 osbuilder: be runtime consistent also with podman build
Use the same runtime used for podman run also for the podman build cmd
Additionally remove "docker" from the docker_run_args variable

Fixes: #3239
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2021-12-29 14:38:32 +02:00
Jakob Naucke
d03e05e803
versions: Use fixed, minor version for Alpine
- Set Alpine guest rootfs to 3.13 on all instances.
- Specify a minor version rather than patch level as the Alpine
  repositories use that.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-09 16:47:43 +01:00
Jakob Naucke
0f7db91c0f
osbuilder: Revert to using apk.static for Alpine
#2399 partially reverted #418, missing on returning to bootstrapping a
rootfs with `apk.static` instead of copying the entire root, which can
result in drastically larger (more than 10x) images. Revert this as well
(requires some updates to URL building).

Fixes: #3216
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-09 16:47:43 +01:00
Binbin Zhang
df34e91978 osbuilder: fix missing cpio package when building rootfs-initrd image
1. install cpio package before building rootfs-initrd image
2. add `pipefaili;errexit` check to the scripts

Fixes: #3144

Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
(cherry picked from commit 8ee67aae4f)
2021-11-29 18:29:02 +01:00
Fabiano Fidêncio
5995efc0a6
Merge pull request #3143 from bergwolf/coreutils-2.3
stable-2.3 | osbuilder: add coreutils to guest rootfs
2021-11-29 12:31:38 +01:00
Peng Tao
f61e31cd84 osbuilder: add coreutils to guest rootfs
So that the debug console is more useful. In the meantime, remove
iptables as it is not used by kata-agent any more.

Fixes: #3138
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-11-29 16:53:04 +08:00
Julio Montes
cb7891e0b4 tools/osbuilder: build QAT kernel in fedora 34
kernel compiled in fedora 35 (latest) is not working, following error
is reported:

```
qemu-system-x86_64: Error loading uncompressed kernel without PVH ELF
Note
```

Build QAT kernel in fedora 34 container to fix it

fixes #3135

Signed-off-by: Julio Montes <julio.montes@intel.com>
(cherry picked from commit 857501d8dd)
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-11-29 08:24:31 +01:00
Wainer dos Santos Moschetta
a239a38f45 osbuilder: build image-builder image from Fedora 34
Currently the image-builder image is built from `fedora:latest` and
this is error-prone as any update of the base image can lead to
breakage. Instead let's create the image from Fedora 34, which is the
last known version to build fine.

Fixes #2960
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-11-03 13:07:42 -04:00
Manabu Sugimoto
a3647e3486 osbuilder: Set up libseccomp library
The osbuilder needs to set up libseccomp library to build the kata-agent
because the kata-agent supports seccomp currently.
The library is built from the sources to create a static library for musl libc.
In addition, environment variables for the libseccomp crate are set to
link the library statically.

Fixes: #1476

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2021-10-27 19:06:13 +09:00
Eric Adams
37fa453dd2 osbuilder: Update QAT driver in Dockerfile
This is to bump the OOT QAT 1.7 driver version to the
latest version. I dida test on my QAT enabled system and
everything functioned as expected.

Fixes: #2877

Signed-off-by: Eric Adams <eric.adams@intel.com>
2021-10-22 00:08:24 +00:00
Wainer dos Santos Moschetta
50da26d3e6 osbuilder: Call detect_rust_version() right before install_rust.sh
When building with dracut method the build_rootfs_distro() is not called, in turn
detect_rust_version() isn't either, so the install_rust.sh script is gave a null
rust version. This changed the script to call detect_rust_version() right before
install_rust.sh.

Related to commit: f34f67d610
Fixes #2862
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-10-18 16:54:53 -04:00
Jakob Naucke
f34f67d610
osbuilder: Specify version when installing Rust
and update the script in `ci/` accordingly.
When only parts of the Kata Containers repositories are checked out
(e.g. when building with Snap) and no Rust version is provided in
calling `install_rust.sh`, the scripts will attempt to clone the
appropriate repos to read the version, which will fail because the
directories already exist. Since we have read the version already, we
can just specify it.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-10-15 16:27:40 +02:00
Jakob Naucke
135a0802c5
osbuilder: Pass CI env to container agent build
The agent build inside a Docker or Podman container has been re-enabled,
but we have since introduced the `$CI` environment variable. Pass it to
avoid checking out the tests repo to main when there is a dependency.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-10-15 16:27:39 +02:00
Jakob Naucke
eb5dd76e9d
osbuilder: Re-enable building the agent in Docker
or Podman. This is a partial revert of
76c18aa345. The rationale behind that
commit was the fact that the agent could not be built on Alpine, and
then this capability was removed altogether. The issue in Alpine has
since been resolved (see
https://github.com/kata-containers/osbuilder/issues/386). At the same
time, this ensures being able to run a glibc agent on hosts with distros
more recent than the osbuilder distro used (i.e. as of now, when you
build the agent on the host, and its glibc is newer than the one used in
the guest, the agent may encounter unresolved symbols).

Fixes #2398
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-10-15 16:27:37 +02:00
Aaron Simmons
80f6b97710 osbuilder: fixing centos gpg key url for ppc64le
The centos ppc64le gpg key at mirror.centos.org doesn't exist (link rot?).
Replacing it with url from CentOS/sig-core-AltArch on github.

Fixes: #2676

Signed-off-by: Aaron Simmons <paleozogt@gmail.com>
2021-09-29 09:20:51 -06:00
Binbin Zhang
924a68d08d osbuilder: Change to "=" operator to make script more portable
zsh doesn't support "==" as equal comparison operator, so
replace "==" with "=" to make the script more portable

Fixes: #2584

Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
2021-09-08 16:57:34 +08:00
Bin Liu
3c9170ea0d
Merge pull request #2561 from rapiz1/patch-4
osbuilder: fix inconsistent calculation of fs size
2021-09-04 15:14:07 +08:00
Yujia Qiao
bfcee91164 osbuilder: fix inconsistent calculation of fs size
This patch fixes inconsistent calculations of the rootfs size.
For `du` and `df`, `-B 1MB` is different from `-BM`. The
former is the power of 1000, and the latter is the power of
1024. So comparing them doesn't make sense. The bug may result
in a larger image than needed.

Fixes: #2560

Signed-off-by: Yujia Qiao <rapiz3142@gmail.com>
2021-09-02 16:00:29 +08:00
Peng Tao
256c3b2747 license: drop redundent license files
There is no need to keep multiple copies of the license file in
different directory. We can just use the top level one for the project.

Fixes: #2553
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-09-01 15:10:04 +08:00
Marcel Apfelbaum
9de1129bf7 osbuilder: Fix rootfs-builder when running in VMs
The script runs apt sync at some point which scans all possible fds
in order to close them. The operation is incredibly slow on VMs
and may lead to build timeouts.

Fix it by limiting the container runtime fds to a sane limit.

Fixes: #2510

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2021-08-25 19:58:21 +00:00
Marcel Apfelbaum
65a1e13195 osbuilder: Allow running the tool several times
Once the ${ROOTFS_DIR} is created, the tool can't run the second
time since the directory is populated and the debootstrap tool
will fail.

Fix by deleting the contents of ${ROOTFS_DIR} if the directory exists.
Note that running make clean will also allow the re-run, it
is only an optimization for some cases the build fails in the middle.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2021-08-25 19:49:14 +00:00
Marcel Apfelbaum
a4214738b9 osbuilder: Fix Makefile
Let the DISTRO variable to be set from outside,
allowing "sudo -E DISTRO=<ANY> make clean" to delete the correct files.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2021-08-25 19:45:53 +00:00
Jakob Naucke
6a6dee7cc8
osbuilder: Document no Alpine support on s390x
Alpine used to work as guest under 1.x, but because there is no musl
target for Rust on s390x, Alpine will not work for 2.x. Document this.

Fixes: #2436
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-08-12 11:14:25 +02:00
Jakob Naucke
7effbdebcb
osbuilder: Upgrade Ubuntu guest to 20.04
- no need to create `/usr/lib/systemd/systemd` link any more
- install `chrony` as extra package and install extra packages in chroot
  rather than `debootstrap`, because `chrony` provides `time-daemon`,
  which under 20.04 is provided by `systemd-timesyncd`, which is
  required by `systemd`, and `debootstrap`'s conflict resolvement can't
  handle this, but `apt`'s can.

Fixes: #2147
Depends-on: github.com/kata-containers/tests#3636
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-08-10 16:31:21 +02:00
David Gibson
c867d1e069 osbuilder: Drop Go agent support
With Kata 1.x EOL, the Go agent is no more.  So, remove support for it from
the osbuilder scripts.  This removes the RUST_AGENT variable, treating it
as always true.

fixes #2396

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-08-05 16:10:10 +10:00
Jakob Naucke
b8133a188c
osbuilder/dracut: Add missing libraries
When the guest is built using dracut and the agent uses glibc (esp.
ppc64le/s390x), libraries might be missing. In my case, it was
`libutil.so`, but more can be added easily. Add a script to configure
`install_items` for dracut w.r.t. `ldd` of the agent.

Fixes: #2384
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-08-04 13:50:42 +02:00
Jakob Naucke
a72b08117f
osbuilder: pass env OS_VERSION
With lines like
0a2e2c6038/tools/osbuilder/rootfs-builder/fedora/config.sh (L8)
we imply that one can set another OS_VERSION and it will get picked up.
This is not the case when building inside Docker/Podman because the
variable is not passed to the container, which can lead to confusion.
Forward this env.

Fixes: #2378
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-08-03 18:28:17 +02:00
James O. D. Hunt
4f0726bc49 docs: Remove table of contents
Removed all TOCs now that GitHub auto-generates them.

Also updated the documentation requirements doc removing the requirement
to add a TOC.

Fixes: #2022.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-07-30 10:58:22 +01:00
Fabiano Fidêncio
2d142bc92d
Merge pull request #2155 from jcvenegas/kata-deploy-2021-06-29
kata-deploy: Allow build  kata-deploy  tarball from HEAD
2021-07-29 22:50:03 +02:00
Carlos Venegas
4895747f35 Rootfs: Add curl to alpine rootfs builder.
If alpine image is created inside a container,
it does not get any golang version data. It will try
to get it by installing yq. To install yq curl is used.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-28 19:45:35 +00:00
GabyCT
24cbb97f68
Merge pull request #2298 from fgiudici/yq_latest_fix
osbuilder/scripts: add support to yq version 4 and above
2021-07-23 12:19:46 -05:00
Fabiano Fidêncio
d75c01bd67
Merge pull request #2186 from YchauWang/yc-osbuilder-arm
osbuilder: update centos arm rootfs image config 'GPG_KEY_ARCH_URL'
2021-07-23 11:17:08 +02:00
Francesco Giudici
0f8c0dbc52 osbuilder/scripts: add support to yq version 4 and above
yq changed syntax in an incompatible way starting from version 4 and
above. Deal with that.

Fixes: #2297

Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
2021-07-22 16:01:57 +02:00
wangyongchao.bj
3882619471 osbuilder: update centos arm rootfs image config 'GPG_KEY_ARCH_URL'
fix GPG_KEY_ARCH_URL config of centos's config_aarch64,
update to "http://mirror.centos.org/altarch/7/os/aarch64/RPM-GPG-KEY-CentOS-7".

Fixes: #2181

Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
2021-07-20 13:49:17 +08:00
bin
b12b21f337 osbuilder: Skip installing golang for building rootfs
Building rootfs does not depend on golang, delete intalling
golang may save build time.

And there is only rust agent now, the code for golang agent should
be deleted too.

Fixes: #2170

Signed-off-by: bin <bin@hyper.sh>
2021-07-15 23:59:15 +08:00
Manabu Sugimoto
88e7075929 osbuilder: Fix the order of checking the distro config directory
Check if the distro config directory exists before using the directory

Fixes: #2148

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2021-07-11 15:24:25 +09:00
Eric Adams
a822cdf64d osbuilder: Update QAT driver version
This updates the QAT driver version to the latest version.

Fixes: #2103

Signed-off-by: Eric Adams <eric.adams@intel.com>
2021-06-23 23:49:19 +00:00
Tim Zhang
c881899903
Merge pull request #1990 from liubin/1989/fix-log-message
osbuilder: fix log message that is not error but seems like an error
2021-06-15 15:13:21 +08:00
bin
45fd58d11c osbuilder: fix log message that is not error but seems like an error
Only show checkout failed message if AGENT_VERSION is set
and the checkout is failed.

Fixes: #1989

Signed-off-by: bin <bin@hyper.sh>
2021-06-15 10:26:52 +08:00
Julio Montes
6e7b55baa9
Merge pull request #1995 from GabyCT/topic/removetravisreference
docs: Remove old travis reference
2021-06-11 09:23:47 -05:00
Fabiano Fidêncio
dc4307d3cc
Merge pull request #1974 from Jakob-Naucke/fix-cc-fedora-s390x
Update CC=gcc setting for Fedora s390x
2021-06-11 00:31:51 +02:00
Gabriela Cervantes
a668f310c3 docs: Remove old travis reference
This PR removes the travis reference as we currently for kata 2.0,
travis is not being supported.

Fixes #1994

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2021-06-10 17:17:17 -05:00
Jakob Naucke
a484d6db87
osbuilder: Streamline s390x CMake & musl handling
- Merge codepath in lib.sh with ppc64le -- do not install CMake
- Like ppc64le, do not install musl rather than just not using it

Fixes: #1975

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-06-07 17:39:31 +02:00
Jakob Naucke
da2d9ab813
osbuilder: Remove CC=gcc for Fedora s390x
since that was required specifically for the Go agent

Fixes: #1973

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-06-07 17:32:05 +02:00
GabyCT
bdc9a66bd9
Merge pull request #1770 from jongwu/image_align
image_build: align image size to 128M for arm64
2021-05-12 09:38:06 -05:00
Chelsea Mafrica
e8038718aa osbuilder: Upgrade alpine version to 3.13.5
We are using an older version of alpine, so upgrade to latest 3.13.5.

Fixes #1817

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2021-05-07 16:34:45 -07:00
Jianyong Wu
2b0d5b252e image_build: align image size to 128M for arm64
There is an inconformity between qemu and kernel of memory alignment
check in memory hotplug. Both of qemu and kernel will do the start
address alignment check in memory hotplug. But it's 2M in qemu
while 128M in kernel. It leads to an issue when memory hotplug.

Currently, the kata image is a nvdimm device, which will plug into the VM as
a dimm. If another dimm is pluged, it will reside on top of that nvdimm.
So, the start address of the second dimm may not pass the alginment
check in kernel if the nvdimm size doesn't align with 128M.

There are 3 ways to address this issue I think:
1. fix the alignment size in kernel according to qemu. I think people
in linux kernel community will not accept it.
2. do alignment check in qemu and force the start address of hotplug
in alignment with 128M, which means there maybe holes between memory blocks.
3. obey the rule in user end, which means fix it in kata.

I think the second one is the best, but I can't do that for some reason.
Thus, the last one is the choice here.

Fixes: #1769
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-05-03 10:44:30 +08:00