This allows to get released binaries again.
Fixes: #4151
Signed-off-by: Greg Kurz <groug@kaod.org>
(cherry picked from commit b658dccc5f)
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
If we fail to download the clh binary, we fall-back to build from source.
Unfortunately, `pull_clh_released_binary()` leaves a `cloud_hypervisor`
directory behind, which causes `build_clh_from_source()` not to clone
the git repo:
[ -d "${repo_dir}" ] || git clone "${cloud_hypervisor_repo}"
When building from a kata-containers git repo, the subsequent calls
to `git` in this function thus apply to the kata-containers repo and
eventually fail, e.g.:
+ git checkout v23.0
error: pathspec 'v23.0' did not match any file(s) known to git
It doesn't quite make sense actually to keep an existing directory the
content of which is arbitrary when we want to it to contain a specific
version of clh. Just remove it instead.
Fixes: #4151
Signed-off-by: Greg Kurz <groug@kaod.org>
(cherry picked from commit afbd60da27)
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
4c164afbac renamed extra_build_args to
features, but did it only in one place, leading to:
```
21:15:28 /home/jenkins/workspace/kata-containers-2.0-ubuntu-ARM-PR/go/src/github.com/kata-containers/kata-containers/tools/packaging/static-build/cloud-hypervisor/build-static-clh.sh: line 55: features: unbound variable
21:15:29 make[1]: *** [tools/packaging/kata-deploy/local-build/Makefile:30: cloud-hypervisor-tarball-build] Error 1
```
Fixes: #3775
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's bump the Cloud Hypervisor version to 5343e09e7b8db, as that brings
a few fixes we're interested in, such as:
* hypervisor, vmm: Handle TDX hypercalls with INVALID_OPERAND
- https://github.com/cloud-hypervisor/cloud-hypervisor/pull/3723
- This is needed for the TDX support on the cloud hypervisor driver,
which is part of this very same series.
* openapi: Update the PciBdf types
- https://github.com/cloud-hypervisor/cloud-hypervisor/pull/3748
- This is needed due to a change in a DeviceNode field, which would
cause a marshalling / demarshalling error when running with a
version of cloud-hypervisor that includes the TDX fixes mentioned
above.
* scripts: dev_cli: Don't quote $features_build
* scripts: dev_cli: Add --features option
- https://github.com/cloud-hypervisor/cloud-hypervisor/pull/3773
- This is needed due to changes in the scripts used to build Cloud
Hypervisor, which are used as part of Kata Containers CIs and
github actions.
Due to this change, we're also adapting the build scripts as part
of this very same commit.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
`tools/packaging/scripts/apply_patches.sh` uses `git apply $patch`, but
this will not apply to subdirectories. If one wanted to apply with
`git apply`, they'd have to run it with `--directory=...`
_relative to the Git tree's root_ (absolute will not work!). I suggest
we just use `patch`, which will do what we expected `git apply` would
do.
`patch` is also added to build containers that require it.
Fixes: #3690
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
We're downloading the released cloud-hypervisor binary from GitHub, but
we should also ensure we set the binary as executable.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's allow passing down a build flag to cargo, when building Cloud
Hypervisor.
By doing this we allow calling this script with:
```
extra_build_flags="--features tdx" ./build-static-clh.sh
```
Fixes: #3671
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The current code will always pull the release binaries in case the
version requested by Kata Containers matches with a released version.
This, however, has a limitation of preventing users / CIs to build
cloud-hypervisor from source for a reason or another, such as passing a
specific build flag to cloud-hypervisor.
This is a pre-req to solving
https://github.com/kata-containers/kata-containers/issues/3671.
While here, a small changes were needed in order to improve readability
and debugability of why we're building something from the sources rather
than simply downloading and using a pre-built binary.
Fixes: #3672
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The script `tools/packaging/static-build/qemu/build-base-qemu.sh`
previously failed on systems where the user's groupname differs from the
username
Fixes: #3461
Signed-off-by: Sebastian Hasler <sebastian.hasler@stuvus.uni-stuttgart.de>
Restore Debian as a rootfs.
1. revert of #3154, but some change
2. update debian version to 10.11
3. update `libstdc++-6-dev` to `libstdc++-8-dev`
4. changes discarded in QAT are not restored
Fixes: #3372
Signed-off-by: zhaojizhuang <571130360@qq.com>
Removed all errors/warnings pointed out by hadolint version 2.7.0, except for the following
ignored rules:
- "DL3008 warning: Pin versions in apt get install"
- "DL3041 warning: Specify version with `dnf install -y <package>-<version>`"
- "DL3033 warning: Specify version with `yum install -y <package>-<version>`"
- "DL3048 style: Invalid label key"
- "DL3003 warning: Use WORKDIR to switch to a directory"
- "DL3018 warning: Pin versions in apk add. Instead of apk add <package> use apk add <package>=<version>"
- "DL3037 warning: Specify version with zypper install -y <package>[=]<version>"
Fixes#3107
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Currently QEMU's submodules are git cloned but there is the scripts/git-submodule.sh
which is meant for that. Let's use that script.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The static build of QEMU takes a good amount of time on cloning the
source tree because we do a full git clone. In order to speed up that
operation this changed the Dockerfile so that it is carried out a
partial clone by using --depth=1 argument.
Fixes#3291
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Currently we do not have debian as part of the kata CI as we
do not have a mantainer, this PR removes debian as a supported
rootfs in order to have only the distros that we are supporting
and mantainining.
Fixes#3153
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
- Install OpenSSL for key generation in kernel build
- Do not install libpmem
- Do not exclude `*/share/*/*.img` files in QEMU tarball since among
them are boot loader files critical for IPLing.
Fixes: #2895
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Currently, there is cloud hypervisor binary released only for x86, thus
we must build from source code when install cloud hypervisor on arm64.
Fixes: #2410
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
kata-deploy buider now reuses the build directory, this
makes faster rebuilds. Update firecracker builder to
not fail if is called twice.
Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
Use Podman instead of Docker when $USE_PODMAN is set. This enables
running with Podman, e.g. to import images for CRI-O.
Fixes: #2067
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
In some distros (Ubuntu 18 and 20) it's not possible to clone meson
and QEMU dependencies from https://git.qemu.org due to problems with
its certificates, let's pull these dependencies from github before
building QEMU.
fixes#1965
Signed-off-by: Julio Montes <julio.montes@intel.com>
Update static build scripts to allow build qemu dev tree.
When qemu starts the process for a new version the patch number
from the qemu version is more than 50. Add logic to detect it
and not apply patches fro the base branch.
For example:
Qemu 5.2.50 means the beginning for 6.0 development. After detect a
development version, patches for 5.2.x will not be applied.
Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
Split qemu script to build qemu experimental using
same dockerfile.
Fixes: #1421
Depends-on: github.com/kata-containers/tests#3255
Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
Add arguments and files as needed, if only of them
changes the build will start from the change and
not from scratch.
Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
Add docker ARG to provide a date to invalid cache, if the date changes
the image will be rebuild. This is required to keep build dependencies
with security fixes, but still take advantage of build qemu faster using
docker cache.
Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
Today we apply patches per base branch. Having
two qemu versions in a similar base version can make
can have problems if one of the trees already has a patch.
If a patch is needed only for one specific tag/commit
add only the patch to that version.
Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
This change the version of QEMU used in the tests and CI.
The scripts/configure-hypervisor.sh was changed so that:
- Passing the `--enable-virtiofsd` flag
- Do not compiling with -O3 to avoid the warning:
Program python3 found: YES (/usr/bin/python3)
../meson.build:104: WARNING: Consider using the built-in optimization level instead of using "-O3".
../meson.build:108: WARNING: Consider using the built-in optimization level instead of using "-O3".
The qemu.blacklist files was changed so that new and uneeded firmware files are removed from the
final tarball. Except for qboot.rom which is new but kept, since it can be used with microvm
machine type (in case we want to enable microvm in the future).
The patches which are applied on QEMU sources:
- 0001-virtiofsd-Allow-to-build-it-without-the-tools.patch
(Build fix for Meson - allows passing `--disable-tools --enable-virtiofsd`)
- 0002-virtiofsd-extract-lo_do_open-from-lo_open.patch
0003-virtiofsd-optionally-return-inode-pointer-from-lo_do.patch
0004-virtiofsd-prevent-opening-of-special-files-CVE-2020-.patch
0005-virtiofsd-Add-_llseek-to-the-seccomp-whitelist.patch
0006-virtiofsd-Add-restart_syscall-to-the-seccomp-whiteli.patch
(Security fixes for virtiofsd)
- 0007-9p-removing-coroutines-of-9p-to-increase-the-I-O-per.patch
(Performance improvement for 9p driver)
- 0008-hw-s390x-fix-build-for-virtio-9p-ccw.patch
(Build fix for virtio-9p-ccw machine type)
Fixes: #1238
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
There's no reason to ship qemu & qemu-virtiofs when the former already
includes vitiofs support (and that's the default for 2.x deployments).
In case we will enable experimental qemu DAX support, we should add a
new target, a "qemu-experimental" target, as Carlos has been working on.
Fixes: #1424
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
Update all tools/packaging scripts to prefer the new options over the
old ones (e.g. `--config` instead of `--kata-config`).
Signed-off-by: Daniel Knittl-Frank <knittl89+git@googlemail.com>
It is used a temporary container to pull the QEMU tarball out
of the build image, but this container is never deleted. This
will ensure it gets deleted after its execution.
Fixes#1168
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Currently QEMU is built inside the container, its tarball pulled to
the host, files removed then packaged again. Instead, let's run all
those steps inside the container and the resulting tarball will
be the final version. For that end, it is introduced the
qemu-build-post.sh script which will remove the uneeded files and
create the tarball.
The patterns for directories on qemu.blacklist had to be changed
to work properly with `find -path`.
Fixes#1168
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The dockerfiles used to build qemu and qemu-virtiofs have the QEMU destination
path hardcoded, which in turn is also on the build scripts. This refactor
the dockerfiles to add the QEMU_DESTDIR argument, which value is passed by the scripts.
Fixes#1168
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Moved the qemu/apply_patches.sh to the common scripts directory and
refactor it so that it can be used as a generic and consistent way
to apply patches.
Fixes#1014
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
For experimental-virtiofs, we use it to test virtiofs with DAX. Let's
rename its virtiofsd to virtiofsd-dax.
Depends-on: github.com/kata-containers/tests#2951
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
In static-build/qemu-virtiofs/Dockerfile the code which
applies the virtiofs specific patches is spread in several
RUN instructions. Refactor this code so that it runs in a
single RUN and produce a single overlay image.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The qemu and qemu-virtiofs Dockerfile files repeat the code to apply
patches based on QEMU stable branch being built. Instead, this adds
a common script (qemu/apply_patches.sh) and make it called by the
respective Dockerfile files.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Fix a bug on qemu-virtiofs Dockerfile which end up not applying
the QEMU patches.
Fixes#786
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This patch add fall-back code path that builds cloud-hypervisor static
binary from source, when the downloading of cloud-hypervisor binary is
failing. This is useful when we experience network issues, and also
useful for upgrading clh to non-released version.
Together with the changes in the tests repo
(https://github.com/kata-containers/tests/pull/2862), the Jenkins config
file is also updated with new Execute shell script for the clh CI in the
kata-containers repo. Those two changes fix the regression on clh CI
here. Please check details in the issue below.
Fixes: #781
Fixes: https://github.com/kata-containers/tests/issues/2858
Signed-off-by: Bo Chen <chen.bo@intel.com>
We should just download the official static build binary instead of
trying to build on our own.
Fixes: #760
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
[ Port from packaging commit 4e1b5729f47d5f67902e1344521bc5b121673046 ]
Build clh with Podman, allow build the vmm in the Podman CI
Virtiofs qemu has to be build as this is requried by clh.
Fixes: #461
Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
Signed-off-by: Bo Chen <chen.bo@intel.com>
