Starting with version 1.80, the Rust linter does not accept an invalid
value for `target_arch` in configuration checks:
```
Compiling kata-sys-util v0.1.0 (/home/ddd/Work/kata/kata-containers/src/libs/kata-sys-util)
error: unexpected `cfg` condition value: `powerpc64le`
--> /home/ddd/Work/kata/kata-containers/src/libs/kata-sys-util/src/protection.rs:17:34
|
17 | #[cfg(any(target_arch = "s390x", target_arch = "powerpc64le"))]
| ^^^^^^^^^^^^^^-------------
| |
| help: there is a expected value with a similar name: `"powerpc64"`
|
= note: expected values for `target_arch` are: `aarch64`, `arm`, `arm64ec`, `avr`, `bpf`, `csky`, `hexagon`, `loongarch64`, `m68k`, `mips`, `mips32r6`, `mips64`, `mips64r6`, `msp430`, `nvptx64`, `powerpc`, `powerpc64`, `riscv32`, `riscv64`, `s390x`, `sparc`, `sparc64`, `wasm32`, `wasm64`, `x86`, and `x86_64`
= note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration
= note: `-D unexpected-cfgs` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(unexpected_cfgs)]`
```
According [to GitHub user @Urgau][explain], this is a new warning
introduced in Rust 1.80, but the problem exists before. The correct
architecture name should be `powerpc64`, and the differentiation
between `powerpc64le` and `powerpc64` should use the `target_endian =
"little"` check.
[explain]: #10072 (comment)
Fixes: #10067
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
[emlima: fix some more occurences and typos]
Signed-off-by: Emanuel Lima <emlima@redhat.com>
[stevenhorsman: fix some more occurences and typos]
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The new QEMU build needs python-tomli, now that we bumped Ubuntu
we can include the needed tomli package
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
In jammy we have the liburing package available, hence
remove the source build and include the package.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The kata containers hypervisior qemu configuration supports setting
block_device_aio="native", but the kata static build of qemu does
not add the linux aio feature.
The libaio-dev library is a necessary dependency for building qemu
with linux aio.
Fixes: #10130
Signed-off-by: Zhiwei Huang <ai.william@outlook.com>
Do not install the packages librados-dev and librbd-dev as they are not needed for building static qemu.
Add machine option cap-ail-mode-3=off while creating the VM to qemu cmdline.
Fixes: #9893
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
Since yq frequently updates, let's upgrade to a version from February to
bypass potential issues with versions 4.41-4.43 for now. We can always
upgrade to the newest version if necessary.
Fixes#9354
Depends-on:github.com/kata-containers/tests#5818
Signed-off-by: Beraldo Leal <bleal@redhat.com>
Let's update the QEMU to the one that's officially maintained by Intel
till all the TDX patches make their way upstream.
We've had to also update python to explicitly use python3 and add
python3-venv as part of the dependencies.
Fixes: #8810
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is required to allow creation of SNP coco on host kernel
(e.g. https://github.com/AMDESE/linux ,branch:snp-host-latest)
supporting guest private memory for SNP using gmem.
Note: This qemu does not work if the host kernel does not support
gmem/UPM.
Fixes: #9092
Signed-off-by: Niteesh Dubey <niteesh@us.ibm.com>
This PR uses a variable that is already defined in the build base
qemu script to have uniformity across the script as this variable
is already used in the script.
Fixes#8903
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
We've not been building QEMU experimental for a very long time, and the
entry there has only been serving the purpose to clutter the
versions.yaml (in the best case scenario) or even confuse new
contributors to the project.
Mind that the machinery to build the QEMU experimental is not touched,
and that's used to build the TEEs capabale artefacts.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
- statically linked qemu requires slof.bin to run, hence remove it from blacklist
- By default, initrd is used for Power, modify the configuration.toml accordingly
Fixes: #8458
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
Depends on mutiarch feature of ubuntu, we can set up cross build
environment easily and achive as good build performance as native
build.
Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Let's make sure the `qemu_suffix` and `qemu_tarball_name` can be
specified. With this we make it really easy to reuse this script for
any addition flavour of an experimental QEMU that ends up having to be
built (specifically looking at the ones for Confidential Containers
here).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add support for caching QEMU artefacts that are generated using
the kata-deploy local-build scripts.
Right now those are not used, but we'll switch to using them very soon
as part of upcoming changes of how we build the components we test in
our CI.
Fixes: #6480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
There's no need to pass repo_root_dir to get_last_modification() as the
variable used everywhere is exported from that very same file.
Fixes: #6431
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is used in several parts of the code, and can have a single
declaration as part of the `lib.sh` file, which is already imported by
all the places where it's used.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This, combined with the effort of caching builder images *and* only
performing the build itself inside the builder images, is the very first
step for reproducible builds for the project.
Reproducible builds are quite important when we talk about Confidential
Containers, as users may want to verify the content used / provided by
the CSPs, and this is the first step towards that direction.
Fixes: #5517
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the needed infra for only building and pushing the QEMU
builder image to the Kata Containers' quay.io registry.
Fixes: #5481
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's first try to pull a pre-existsing image, instead of building our
own, to be used as a builder image for QEMU.
This will save us some CI time.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Differently than every single other bit that's part of our repo, QEMU
has been using a single Dockerfile that prepares an environment where
the project can be built, but *also* building the project as part of
that very same Dockerfile.
This is a problem, for several different reasons, including:
* It's very hard to have a reproducible build if you don't have an
archived image of the builder
* One cannot cache / ipload the image of the builder, as that contains
already a specific version of QEMU
* Every single CI run we end up building the builder image, which
includes building dependencies (such as liburing)
Let's split the logic into a new build script, and pass the build script
to be executed inside the builder image, which will be only responsible
for providing an environment where QEMU can be built.
Fixes: #5464
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In the commit 54d6d01754 we ended up
removing the BUILD_SUFFIX argument passed to QEMU as it only seemed to
be used to generate the HYPERVISOR_NAME and PKGVERSION, which were added
as arguments to the dockerfile.
However, it turns out BUILD_SUFFIX is used by the `qemu-build-post.sh`
script, so it can rename the QEMU binary accordingly.
Let's just bring it back.
Fixes: #5078
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Dockerfile cannot decipher multiple conditional statements in the main RUN call.
Cannot segregate statements in Dockerfile with '{}' braces without wrapping entire statement in 'bash -c' statement.
Dockerfile does not support setting variables by bash command.
Must set HYPERVISOR_NAME and PKGVERSION from parent script: build-base-qemu.sh
Fixes: #5078
Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
io_uring is a Linux API for asynchronous I/O introduced in qemu 5.0.
It is designed to better performance than older aio API.
We could leverage this in order to get better storage performance.
We should be adding liburing-dev to qemu build to leverage this feature.
However liburing-dev package is not available in ubuntu 20.04,
it is avaiable in 22.04.
Upgrading the ubuntu version in the dockerfile to 22.04 is causing
issues in the static qemu build related to libpmem.
So instead we are building liburing from source until those build issues
are solved.
Fixes: #4645
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Currently $BUILD_DIR will be used to create a directory as:
/opt/kata/share/kata-qemu${BUILD_DIR}
It means that when passing a BUILD_DIR, like "foo", a name would be
built like /opt/kata/share/kata-qemufoo
We should, instead, be building it as /opt/kata/share/kata-qemu-foo.
Fixes: #4638
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Replaces calls of nproc with nproc with
nproc ${CI:+--ignore 1}
to run nproc with one less processing unit than the maximum to prevent
DOS-ing the local machine.
If process is being run in a container (determined via whether $CI is
null), all processing units avaliable will be used.
Fixes#3967
Signed-off-by: Derek Lee <derlee@redhat.com>
`tools/packaging/scripts/apply_patches.sh` uses `git apply $patch`, but
this will not apply to subdirectories. If one wanted to apply with
`git apply`, they'd have to run it with `--directory=...`
_relative to the Git tree's root_ (absolute will not work!). I suggest
we just use `patch`, which will do what we expected `git apply` would
do.
`patch` is also added to build containers that require it.
Fixes: #3690
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
The script `tools/packaging/static-build/qemu/build-base-qemu.sh`
previously failed on systems where the user's groupname differs from the
username
Fixes: #3461
Signed-off-by: Sebastian Hasler <sebastian.hasler@stuvus.uni-stuttgart.de>
Removed all errors/warnings pointed out by hadolint version 2.7.0, except for the following
ignored rules:
- "DL3008 warning: Pin versions in apt get install"
- "DL3041 warning: Specify version with `dnf install -y <package>-<version>`"
- "DL3033 warning: Specify version with `yum install -y <package>-<version>`"
- "DL3048 style: Invalid label key"
- "DL3003 warning: Use WORKDIR to switch to a directory"
- "DL3018 warning: Pin versions in apk add. Instead of apk add <package> use apk add <package>=<version>"
- "DL3037 warning: Specify version with zypper install -y <package>[=]<version>"
Fixes#3107
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Currently QEMU's submodules are git cloned but there is the scripts/git-submodule.sh
which is meant for that. Let's use that script.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The static build of QEMU takes a good amount of time on cloning the
source tree because we do a full git clone. In order to speed up that
operation this changed the Dockerfile so that it is carried out a
partial clone by using --depth=1 argument.
Fixes#3291
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
- Install OpenSSL for key generation in kernel build
- Do not install libpmem
- Do not exclude `*/share/*/*.img` files in QEMU tarball since among
them are boot loader files critical for IPLing.
Fixes: #2895
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Use Podman instead of Docker when $USE_PODMAN is set. This enables
running with Podman, e.g. to import images for CRI-O.
Fixes: #2067
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
In some distros (Ubuntu 18 and 20) it's not possible to clone meson
and QEMU dependencies from https://git.qemu.org due to problems with
its certificates, let's pull these dependencies from github before
building QEMU.
fixes#1965
Signed-off-by: Julio Montes <julio.montes@intel.com>
Update static build scripts to allow build qemu dev tree.
When qemu starts the process for a new version the patch number
from the qemu version is more than 50. Add logic to detect it
and not apply patches fro the base branch.
For example:
Qemu 5.2.50 means the beginning for 6.0 development. After detect a
development version, patches for 5.2.x will not be applied.
Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
Split qemu script to build qemu experimental using
same dockerfile.
Fixes: #1421
Depends-on: github.com/kata-containers/tests#3255
Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
Add arguments and files as needed, if only of them
changes the build will start from the change and
not from scratch.
Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
Add docker ARG to provide a date to invalid cache, if the date changes
the image will be rebuild. This is required to keep build dependencies
with security fixes, but still take advantage of build qemu faster using
docker cache.
Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
Today we apply patches per base branch. Having
two qemu versions in a similar base version can make
can have problems if one of the trees already has a patch.
If a patch is needed only for one specific tag/commit
add only the patch to that version.
Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
This change the version of QEMU used in the tests and CI.
The scripts/configure-hypervisor.sh was changed so that:
- Passing the `--enable-virtiofsd` flag
- Do not compiling with -O3 to avoid the warning:
Program python3 found: YES (/usr/bin/python3)
../meson.build:104: WARNING: Consider using the built-in optimization level instead of using "-O3".
../meson.build:108: WARNING: Consider using the built-in optimization level instead of using "-O3".
The qemu.blacklist files was changed so that new and uneeded firmware files are removed from the
final tarball. Except for qboot.rom which is new but kept, since it can be used with microvm
machine type (in case we want to enable microvm in the future).
The patches which are applied on QEMU sources:
- 0001-virtiofsd-Allow-to-build-it-without-the-tools.patch
(Build fix for Meson - allows passing `--disable-tools --enable-virtiofsd`)
- 0002-virtiofsd-extract-lo_do_open-from-lo_open.patch
0003-virtiofsd-optionally-return-inode-pointer-from-lo_do.patch
0004-virtiofsd-prevent-opening-of-special-files-CVE-2020-.patch
0005-virtiofsd-Add-_llseek-to-the-seccomp-whitelist.patch
0006-virtiofsd-Add-restart_syscall-to-the-seccomp-whiteli.patch
(Security fixes for virtiofsd)
- 0007-9p-removing-coroutines-of-9p-to-increase-the-I-O-per.patch
(Performance improvement for 9p driver)
- 0008-hw-s390x-fix-build-for-virtio-9p-ccw.patch
(Build fix for virtio-9p-ccw machine type)
Fixes: #1238
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
