Commit Graph

1801 Commits

Author SHA1 Message Date
Fabiano Fidêncio
2b5f79d685 release: automatically bump the version of the kata-deploy images
Let's teach `update-repository-version.sh` to automatically bump the
version of the kata-deploy images to be used within that release, when
running against the `kata-containers` repo.

Fixes: #1665

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-09 00:31:27 +02:00
Fabiano Fidêncio
f444adb51b kata-cleanup: Explicitly add tag to the container image
We have the tags explicitly set on kata-deploy, let's do the same for
kata-cleanup.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-08 21:43:59 +02:00
Eric Ernst
12582c2f6d kata-deploy: add runtimeclass that includes pod overhead
The overhead values may not be perfect, but this is a start, and a good
reference.

Fixes: #580

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-04-08 12:42:15 -07:00
Fabiano Fidêncio
342eb765c2 tools/agent-ctl: Update Cargo.lock
While performing a `make` in the top directory of the project, I've
noticed that agent-ctl's Cargo.lock file was not up-to-date.

Fixes: #1655

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-08 10:24:08 +02:00
Carlos Venegas
ec36883fe3 Merge pull request #1638 from jcvenegas/2021-04-06/linux-5.10.x-qemu-experimental
kernel: update experimental kernel to 5.10.x
2021-04-07 16:02:44 -05:00
Wainer Moschetta
69dbcaa32b Merge pull request #1633 from fidencio/wip/fix-test-kata-deploy-script
kata-deploy: Fix `test-kata.sh` and do some small cleanups / improvements in the kata-deploy script
2021-04-07 16:13:17 -03:00
Fabiano Fidêncio
d43098ec21 kata-deploy: Adapt regex for testing kata-deploy
On commit 60f6315 we've started adding the specific version of the image
to be used, in order to ensure people using our content from a tarball
would be relying on the correct image.

However, later on, @bergwolf figured out it had some undesired side
effects, such as
https://github.com/kata-containers/kata-containers/runs/2235812941?check_suite_focus=true

What happens there is that the regular expression used to point the
image to a testing one doesn't take into consideration the $VERSION, and
that breaks the deployment.

Depends-on: github.com/kata-containers/kata-containers#1641
Fixes: #1632

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-06 23:41:24 +02:00
Carlos Venegas
84e453a643 Merge pull request #1625 from jcvenegas/2021-04-05/qemu-experimental-6.0
qemu experimental: Move to latest tree on virtio-fs-dev (qemu 6.0 + DAX patches).
2021-04-06 15:29:05 -05:00
Carlos Venegas
107ceca680 kernel: update experimental kernel to 5.10.x
Relevant changes for experimental :

42d3e2d04 virtiofs: calculate number of scatter-gather elements accurately
413daa1a3 fuse: connection remove fix
bf109c640 fuse: implement crossmounts
1866d779d fuse: Allow fuse_fill_super_common() for submounts
fcee216be fuse: split fuse_mount off of fuse_conn
8f622e949 fuse: drop fuse_conn parameter where possible
24754db27 fuse: store fuse_conn in fuse_req
c6ff213fe fuse: add submount support to <uapi/linux/fuse.h>
d78092e49 fuse: fix page dereference after free
9a752d18c virtiofs: add logic to free up a memory range
d0cfb9dcb virtiofs: maintain a list of busy elements
6ae330cad virtiofs: serialize truncate/punch_hole and dax fault path
9483e7d58 virtiofs: define dax address space operations
2a9a609a0 virtiofs: add DAX mmap support
c2d0ad00d virtiofs: implement dax read/write operations
ceec02d43 virtiofs: introduce setupmapping/removemapping commands
fd1a1dc6f virtiofs: implement FUSE_INIT map_alignment field
45f2348ec virtiofs: keep a list of free dax memory ranges
1dd539577 virtiofs: add a mount option to enable dax
22f3787e9 virtiofs: set up virtio_fs dax_device
f4fd4ae35 virtiofs: get rid of no_mount_options
b43b7e81e virtiofs: provide a helper function for virtqueue initialization

Fixes: #1639

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-04-06 19:06:09 +00:00
Fabiano Fidêncio
ca4dccf980 release: Get rid of "master"
We don't use the "master" branch for anything in
`kata-containers/kata-containers`.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-06 20:28:42 +02:00
Fabiano Fidêncio
c2197cbf2b release: Use sudo to install hub
This doesn't make much difference for the automated process we have in
place, but makes a whole lot of difference for those trying to have the
binaries deployed locally.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-06 20:28:42 +02:00
GabyCT
201ad249c2 Merge pull request #1597 from jongwu/PA
kernel: upgrade kernel to 5.10.x for arm64.
2021-04-06 09:44:16 -05:00
Carlos Venegas
f365bdb7cf versions: qemu-experimental: 6.0~rc 470dd6
Move to next 6.0 dev tree for qemu experimental,
the qemu version is the same base as:

https://gitlab.com/virtio-fs/qemu/-/commits/virtio-fs-dev/

Using qemu 6.0-rc1 some patches does not apply.

Fixes: #1624

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-04-05 21:01:58 +00:00
Carlos Venegas
6491b9d7aa qemu: Add support to build static qemu for dev tree
Update static build scripts to allow build qemu dev tree.
When qemu starts the process for a new version the patch number
from the qemu version is more than 50. Add logic to detect it
and not apply patches fro the base branch.

For example:

Qemu 5.2.50 means the beginning for 6.0 development. After detect a
development version, patches for 5.2.x will not be applied.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-04-05 20:55:55 +00:00
GabyCT
503039482b Merge pull request #1620 from eadamsintel/update-qat-dockerfile
osbuilder: Update QAT Dockerfile with new QAT driver version
2021-04-05 09:51:14 -05:00
Eric Ernst
d44412fe24 Merge pull request #1623 from egernst/custom-image-registry
osbuilder: update dockerfiles to utilize IMAGE_REGISTRY
2021-04-02 14:18:36 -07:00
Eric Ernst
935460e549 osbuilder: update dockerfiles to utilize IMAGE_REGISTRY
While we introduced IMAGE_REGISTRY, we didn't actually update the
corresponding Dockerfiles to utilize it. Let's add

Fixes: #1622

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-04-02 09:46:16 -07:00
Eric Adams
010d57f4e2 osbuilder: Update QAT Dockerfile with new QAT driver version
This fixes the QAT driver version and provides a check early in the
building process to make sure the driver exists. It also provides
hints to users on how to fix themselves if the driver changes again.

Fixes: #1618

Signed-off-by: Eric Adams <eric.adams@intel.com>
2021-04-01 19:20:44 +00:00
Fabiano Fidêncio
adb866ad64 kata-deploy: Adapt to the correct tag name
Use 2.1.0-alpha1 instead of 2.1-alpha1

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-04-01 20:45:30 +02:00
Jianyong Wu
28bd8c1110 kernel: upgrade kernel to 5.10.x for arm64.
In kernel 5.10.x on arm64 side, When CONFIG_RANDOM_BASE is enabled,
physical base address can be a negative number. It may lead to bug
when a PA is taken as a unsigned number in comparison, as PA is
calculated based on the physical base address. The bug has been fixed
latest code by commit ee7febce051945be2 in memory hotplug zone. We can
eliminate the bug in an easy way by casting the PA as a signed value in
the current code base to avoid lots of backport.

Depends-on: github.com/kata-containers/tests#3388
Fixes: #1596
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-04-01 17:01:44 +08:00
Fabiano Fidêncio
572aff53e8 build: Only keep one VERSION file
Instead of having different VERSION files spread accross the project,
let's always use the one in the topsrcdir and remove all the others,
keeping only a synlink to the topsrcdir one.

Fixes: #1579

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-31 23:51:20 +02:00
Fabiano Fidêncio
bf707209df Merge pull request #1384 from fidencio/wip/update-kernel-config-for-overlayfs
kernel: Enable OVERLAY_FS_{METACOPY,XINO_AUTO}
2021-03-30 23:20:20 +02:00
Carlos Venegas
8e48fecc2c Merge pull request #1540 from jcvenegas/2021-03-23/kernel-5.10.x
versions: kernel 5.10.x
2021-03-30 12:12:53 -06:00
James O. D. Hunt
2a648fa760 logging: Use guard to make threaded logging safe
Return a guard variable from `create_logger()` which the caller can
implicitly drop to guarantee that all threads started by the async log
drain are stopped.

This fixes a long-standing bug [1] whereby the agent could panic with
the following error, generated by the `slog` logging crate:

```
slog::Fuse Drain: Custom { kind: Other, error: "serde serialization error: Bad file descriptor (os error 9)" }
```

[1] - See https://github.com/kata-containers/kata-containers/issues/171.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-29 14:32:11 +01:00
Carlos Venegas
c035cdb3ef versions: kernel 5.10.x
Linux 5.10.x is the new LTS branch, move
kata to a more recent kernel branch.

Fixes: #1288

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-26 17:58:09 +00:00
Bin Liu
5b5b5cc611 Merge pull request #1539 from bergwolf/ut
fix runtime UTs and enable static check
2021-03-25 16:29:45 +08:00
Fabiano Fidêncio
deca207608 Merge pull request #1542 from fidencio/wip/kata-deploy-use-the-correct-image-for-alpha
kata-deploy: Use the correct tag for 2.1-alpha1 release
2021-03-25 08:32:06 +01:00
Carlos Venegas
753c7270ed Merge pull request #1422 from jcvenegas/2021-02-17/experimental-virtiofs
ci: fix virtiofs-experimental build
2021-03-24 14:47:27 -06:00
Carlos Venegas
0f78a5dc96 kernel: rename exeperimental kernel symlink.
use -experimental suffix instead of virtiofs

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-24 17:52:52 +00:00
Carlos Venegas
f79105231c qemu: Build experimental qemu.
Split qemu script to build qemu experimental using
same dockerfile.

Fixes: #1421

Depends-on: github.com/kata-containers/tests#3255

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-24 17:52:07 +00:00
Peng Tao
076bc5075f agent-ctl: update Cargo.lock
Just build would result in these diffs. Let's include them in git.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-03-24 15:03:52 +08:00
Fabiano Fidêncio
60f6315b2d kata-deploy: Use the correct tag for 2.1-alpha1 release
Let's ensure we use the appropriate tag for the release, even before it
was actually created.

Fixes: #1493

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-03-24 00:29:03 +01:00
Carlos Venegas
b0e51e59fa qemu: Improve cache build
Add arguments and files as needed, if only of them
changes the build will start from the change and
not from scratch.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-23 20:52:39 +00:00
Carlos Venegas
bc587da9f5 qemu: Add suffix for qemu binaries.
To build different qemu versions with the
same qemu code add a prefix on install.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-23 20:52:39 +00:00
Carlos Venegas
5493517b4f qemu: add CACHE_TIMEOUT
Add docker ARG to provide a date to invalid cache, if the date changes
the image will be rebuild. This is required to keep build dependencies
with security fixes, but still take advantage of build qemu faster using
docker cache.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-23 20:52:39 +00:00
Carlos Venegas
98d01ce6b9 qemu: Apply patches for specific versions.
Today we apply patches per base branch. Having
two qemu versions in a similar base version can make
can have problems if one of the trees already has a patch.
If a patch is needed only for one specific tag/commit
add only the patch to that version.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-23 20:52:39 +00:00
Chelsea Mafrica
8c1abc3b57 Merge pull request #1418 from Amulyam24/fix-docker-ppc64le
osbuiler: fixing USE_DOCKER for ppc64le
2021-03-19 17:22:12 -07:00
Carlos Venegas
a09e58fa80 packaging: Use local file for assets.
When kata used multiple repositories, versions file was
downloaded. This is not needed anymore as the file is part
of the same repository.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-19 15:50:41 +00:00
Carlos Venegas
07cfa4ce22 qemu: patches: Fail if not patches directory
Fail if not patches directory is found.  Help to prevent
build a new qemu version with missing patches.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-18 20:07:40 +00:00
Carlos Venegas
5abdd2aaf0 qemu: move 5.0.0 patches to its own dir.
Some patches has conflicts with old experimental kernel. Move patches to its own specific version.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-18 19:25:55 +00:00
Wainer dos Santos Moschetta
0f7950fb2a packaging: configure QEMU with -O2
Currently the the configure-hypervisor.sh doesn't set any optimization flag when
compiling QEMU >= 5.2.0 since the configure script will implicitly set -O2 on Ubuntu. But
on other environments, for example CentOS 7, it won't be set any optimization and this
results on the compiler warn:

  # warning _FORTIFY_SOURCE requires compiling with optimization (-O)

To avoid this inconsistent behavior across different build environments, let's explicitly
set the -O2 flag.

Reported-by: Eric Ernst <eric.g.ernst@gmail.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-03-10 11:16:52 -03:00
Wainer Moschetta
6e496941a5 Merge pull request #1349 from wainersm/qemu_5_2
Bump to QEMU 5.2.0
2021-03-10 11:14:45 -03:00
GabyCT
856ffb85fc Merge pull request #1420 from eadamsintel/qat-dockerfile
osbuilder: Port QAT Dockerfile to 2.0 repo
2021-03-08 11:34:28 -06:00
James O. D. Hunt
c258ea25d4 agent-ctl: Function parameter cleanup
Remove unused function parameters from the following types:

- `AgentCmdFp`: Removed the config parameter and made
  the context parameter the first (à la golang).

- `BuiltinCmdFp`: Removed the config and options parameters.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-05 09:54:50 +00:00
James O. D. Hunt
fcd45def53 agent-ctl: Unbreak build
The recent switch to an async rust agent broke the `agent-ctl` tool.
However, we didn't notice because that isn't being built by the CI.

Fix the breakage by passing a ttRPC context to all ttRPC API calls and
also build the tool as part of the static checks CI.

Fixes: #1471.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-05 09:54:50 +00:00
James O. D. Hunt
efe625dfc1 build: Remove whitespace
Zap trailing whitespace.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-03-05 09:54:50 +00:00
Jianyong Wu
f580d33cc9 musl/arm64: decompression before use the tarball.
In the last fix, the decompression ops is deleted by mistake and need
add it back.

Fixes: #1490
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-03-04 10:12:29 +08:00
Jianyong Wu
2da058ed7b osbuild: build musl toolchain from source if needed
Currently, musl toolchain installation on arm64 is just downloading from
a website. It's unsafe in case the website corrupts. So build musl
toolchain from source if it can't be downloaded.

Fixes: #1481
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-03-03 17:22:39 +08:00
Adams, Eric
6417067d62 osbuilder: Port QAT Dockerfile to 2.0 repo
Update the Intel QAT Dockerfile to work with the 2.0 repos, fix some
bugs with building Debian/Ubuntu rootfs, and update the latest QAT
driver. Updated copyright.

Fixes: #1419

Signed-off-by: Adams, Eric <eric.adams@intel.com>
2021-03-01 17:34:04 -08:00
Wainer dos Santos Moschetta
85601cd360 snap: Update for QEMU 5.2.0
QEMU 5.2.0 needs ninja-build package installed on the build environment.

The default-configs were copied to $QEMU_SRC/default-configs but that does
take any effect, so instead it is now copied to $QEMU_SRC/default-configs/devices
and the configs for i386 were updated.

Also it had to change some arguments being passed to configure as Meson was failing
due inconsistent paths:

  ./meson.build:1:0: ERROR: The value of the 'libdir' option is '/usr/lib/qemu' which must be a subdir of the prefix '/snap/kata-containers/current/usr'.
  Note that if you pass a relative path, it is assumed to be a subdir of prefix.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-03-01 16:57:58 -05:00