Commit Graph

418 Commits

Author SHA1 Message Date
James O. D. Hunt
9db56ffd85 Merge pull request #2863 from wainersm/osbuilder_dracut_rust
osbuilder: Call detect_rust_version() right before install_rust.sh
2021-10-19 11:48:39 +01:00
bin
ed02bc9041 packaging: add containerd to versions.yaml
This commit will add containerd to versions.yaml.

Please at now there are both containerd and cri-containerd
in the versions.yaml.

After updating of kata-containers/tests repo, the cri-containerd
should be removed.

Fixes: #2791

Signed-off-by: bin <bin@hyper.sh>
2021-10-19 09:38:56 +08:00
Wainer dos Santos Moschetta
50da26d3e6 osbuilder: Call detect_rust_version() right before install_rust.sh
When building with dracut method the build_rootfs_distro() is not called, in turn
detect_rust_version() isn't either, so the install_rust.sh script is gave a null
rust version. This changed the script to call detect_rust_version() right before
install_rust.sh.

Related to commit: f34f67d610
Fixes #2862
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-10-18 16:54:53 -04:00
James O. D. Hunt
f16a99603c Merge pull request #2399 from Jakob-Naucke/container-osbuilder-respin
osbuilder: Re-enable building the agent in Docker
2021-10-18 12:06:37 +01:00
Jakob Naucke
f34f67d610 osbuilder: Specify version when installing Rust
and update the script in `ci/` accordingly.
When only parts of the Kata Containers repositories are checked out
(e.g. when building with Snap) and no Rust version is provided in
calling `install_rust.sh`, the scripts will attempt to clone the
appropriate repos to read the version, which will fail because the
directories already exist. Since we have read the version already, we
can just specify it.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-10-15 16:27:40 +02:00
Jakob Naucke
135a0802c5 osbuilder: Pass CI env to container agent build
The agent build inside a Docker or Podman container has been re-enabled,
but we have since introduced the `$CI` environment variable. Pass it to
avoid checking out the tests repo to main when there is a dependency.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-10-15 16:27:39 +02:00
Jakob Naucke
eb5dd76e9d osbuilder: Re-enable building the agent in Docker
or Podman. This is a partial revert of
76c18aa345. The rationale behind that
commit was the fact that the agent could not be built on Alpine, and
then this capability was removed altogether. The issue in Alpine has
since been resolved (see
https://github.com/kata-containers/osbuilder/issues/386). At the same
time, this ensures being able to run a glibc agent on hosts with distros
more recent than the osbuilder distro used (i.e. as of now, when you
build the agent on the host, and its glibc is newer than the one used in
the guest, the agent may encounter unresolved symbols).

Fixes #2398
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-10-15 16:27:37 +02:00
Fabiano Fidêncio
e42bc05c8a kata-deploy: add .dockerignore file
.dockerignore file is similar to .gitignore and serves the purpose to
simply ignore paths in the build context.

For now, let me just use it to fix the following problem:
```
docker build --build-arg KATA_ARTIFACTS=kata-static.tar.xz .
error checking context: 'no permission to read from
'(...)/local-build/build/firecracker/builddir/firecracker/(...)/crc64-1.0.0/.gitignore''.
```

Fixes: #2845

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2021-10-15 12:00:14 +02:00
Marcel Apfelbaum
9796babd92 Merge pull request #2311 from dgibson/mmconfig
packaging/kernel: Add CONFIG_PCI_MMCONFIG to x86 guest kernel configuration
2021-10-10 15:11:33 +03:00
Jakob Naucke
4152c45e4c Merge pull request #2706 from yuanzhe-liu0/qemu_link
qemu: use GitLab repos instead of qemu.org
2021-10-08 12:03:55 +02:00
David Gibson
c4236cb2d1 packaging/kernel: Add CONFIG_PCI_MMCONFIG to x86 guest kernel configuration
The guest kernel configuration suggested for Kata, and which is used by the
CI didn't include CONFIG_PCI_MMCONFIG.  That's kind of weird, MMCONFIG is
the modern normal way of handling configuration cycles.

In addition, due to a complex set of interactions through the ACPI code,
disabling MMCONFIG means that SHPC hotplug doesn't work: the driver is
included in the guest kernel, but will fail to probe on PCI to PCI bridges,
meaning it won't actually be activated.

Enable MMCONFIG so that we suggest and testa more typical guest kernel
configuration.

fixes #2288

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-10-07 13:21:48 +11:00
Chelsea Mafrica
1f6a551570 Merge pull request #2755 from paleozogt/centos-ppc64le-gpg
#2676: fixing centos gpg key url for ppc64le
2021-10-05 09:37:58 -07:00
Greg Kurz
cd1064b16f packaging: Configure QEMU with --enable-pie
We explicitely set the Postion Independant Executlable (PIE) options
in the extra CFLAGS and LDFLAGS that are passed to the QEMU configure
script for all archs. This means that these options are used pretty
much everywhere, including when building the sample plugins under the
test directory. These cannot be linked with -pie and break the build,
as experienced recently on ARM (see PR #2732).

This only broke on ARM because other archs are configured with
--disable-tcg : this disables plugins which are built by default
otherwise.

The --enable-pie option is all that is needed. The QEMU build system
knows which binaries should be created as PIE, e.g. the important
bits like QEMU and virtiofsd, and which ones should not, e.g. the
sample plugins that aren't used in production.

Rely on --enable-pie only, for all archs. This allows to drop the
workaround that was put in place in PR #2732.

Fixes: #2757
Signed-off-by: Greg Kurz <groug@kaod.org>
2021-09-30 11:17:41 +02:00
Aaron Simmons
80f6b97710 osbuilder: fixing centos gpg key url for ppc64le
The centos ppc64le gpg key at mirror.centos.org doesn't exist (link rot?).
Replacing it with url from CentOS/sig-core-AltArch on github.

Fixes: #2676

Signed-off-by: Aaron Simmons <paleozogt@gmail.com>
2021-09-29 09:20:51 -06:00
Chelsea Mafrica
20f4c252b8 Merge pull request #2519 from jcvenegas/kernel-experimental-5.13.10
kernel: Enable SGX in experimental kernel.
2021-09-28 11:00:46 -07:00
Carlos Venegas
dffc50928a kernel: Enable SGX in experimental kernel.
Enable Intel SGX support in experimental kernel.

Fixes: #2518

Signed-off-by: Carlos Venegas <jose.carlos.venegas.munoz@intel.com>
2021-09-27 03:39:50 +00:00
Carlos Venegas
ff6a677d16 kernel-build: Enable multiple config types.
Optional build types are common for early adoption.
Lets add a flag to build and optional config.

e.g.
kernel-build.sh -b experimental

In the future instead of add more flags just add a new build type.

Signed-off-by: Carlos Venegas <jose.carlos.venegas.munoz@intel.com>
2021-09-27 03:39:50 +00:00
Carlos Venegas
90046964ef experimental-kernel: bump 5.13.10
Upgrade Linux kernel to latest stable release.

Signed-off-by: Carlos Venegas <jose.carlos.venegas.munoz@intel.com>
2021-09-27 03:39:50 +00:00
Carlos Venegas
1fbb73041b build: kata-deploy kernel experimental
Allow build experimental kernel from kata-deploy.

Signed-off-by: Carlos Venegas <jose.carlos.venegas.munoz@intel.com>
2021-09-27 02:56:59 +00:00
Jianyong Wu
6d94957a14 kernel: reduce alignment size of memory hotplug to 128M
After 5.11-rc4, memory hotplug alignment size is reduced to 128M for 4K
page.
It works better for memory hotplug and nvdimm plug in kata on arm.
without this patch, memory hotplug will fail for the current memory
hotplug alignment is 1G but the nvdimm size align with 128M in kata.
After port it here, we can avoid a fix in qemu side.

Note: if you change the page size to other size than 4K, memory hotplug
will has no effect.

Fixes: #2707
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-09-26 15:33:33 +08:00
Jianyong Wu
48090f624a qemu: disable plug on arm64 when pie is added
For qemu 6.1.0 build on arm64, compile error occurs when "-pie" is added
 to ldflag.
tests/plugins/empty.c won't be linked as a sysmbol is missing.
I consider there maybe a bug.
Before figure it out, we should disable plugins for qemu 6.1.0 on arm64.

Fixes: #2707
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-09-26 15:33:33 +08:00
Chelsea Mafrica
e987632deb Merge pull request #2693 from Amulyam24/qemu-build
packaging: fix qemu build on ppc64le
2021-09-23 10:31:34 -07:00
Yuanzhe Liu
80463b445a qemu: use GitLab repos instead of qemu.org
arm using qemu 5.1.0, thus is affected by the wired submodules
link.

Fixes: #2705
Signed-off-by: Yuanzhe Liu <yuanzheliu09@gmail.com>
2021-09-23 12:07:44 +00:00
Samuel Ortiz
3276f3b5b6 Merge pull request #2453 from fidencio/wip/kata-deploy-use-stable-and-latest-tags
kata-deploy: Also provide "stable" & "latest" tags
2021-09-23 13:54:01 +02:00
Amulya Meka
439e5ac3b0 packaging: fix qemu build on ppc64le
Since the qemu upgrade to v6.1.0, the build fails
with a linking issue. Adding --disable-tcg to fix
it.

Fixes: #2710

Signed-off-by: Amulya Meka <amulmek1@in.ibm.com>
2021-09-23 06:27:15 +00:00
Jianyong Wu
0ca8c27241 qemu: add v5.1.0 dir under tag_patches
A related dir is needed when apply qemu patch using script. As qemu 5.1
is used for arm, a dir of "v5.1.0" is needed under tag_patches.

Fixes: #2696
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-09-22 18:07:24 +08:00
Fabiano Fidêncio
3bdcfaa658 kata-deploy: Add more info about the stable tag
Let's make it as clear as possible for the user that if they go for a
tagged version of kata-deploy, eg, 2.2.1, they'll have the kata runtime
2.2.1 deployed on their cluster.

Suggested-by: Eric Adams <eric.adams@intel.com>
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-09-21 23:13:45 +02:00
Fabiano Fidêncio
41c590fa0a kata-deploy: Improve README
Let's add more instructions in the README in order to make clear to the
reader what they can do to check whether kata-deploy is ready, or
whether they have to wait till proceeding with the next instruction.

Suggested-by: Eric Adams <eric.adams@intel.com>
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-09-21 23:13:45 +02:00
Fabiano Fidêncio
debf3c9fe9 kata-deploy: Remove qemu-virtiofs runtime class
There's only one QEMU runtime class deployed as part of kata-deploy, and
that includes virtiofs support (which is the default for quite some time
already).  Knowing this, let's just remove the `qemu-virtiofs` runtime
class definition.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-09-21 22:48:04 +02:00
Fabiano Fidêncio
43a72d76e2 release: update the kata-deploy yaml files accordingly
Let's teach our `update-repository-version.sh` script to properly update
the kata-deploy tags on both kata-deploy and kata-cleanup yaml files.

The 3 scenarios that we're dealing with, based on which branch we're
targetting, are:
```
 1) [main] ------> [main]        NO-OP
   "alpha0"       "alpha1"

                   +----------------+----------------+
                   |      from      |       to       |
  -----------------+----------------+----------------+
  kata-deploy      | "latest"       | "latest"       |
  -----------------+----------------+----------------+
  kata-deploy-base | "stable        | "stable"       |
  -----------------+----------------+----------------+

 2) [main] ------> [stable] Update kata-deploy and
   "alpha2"         "rc0"   get rid of kata-deploy-base

                   +----------------+----------------+
                   |      from      |       to       |
  -----------------+----------------+----------------+
  kata-deploy      | "latest"       | "rc0"          |
  -----------------+----------------+----------------+
  kata-deploy-base | "stable"       | REMOVED        |
  -----------------+----------------+----------------+

 3) [stable] ------> [stable]    Update kata-deploy
    "x.y.z"         "x.y.(z+1)"

                   +----------------+----------------+
                   |      from      |       to       |
  -----------------+----------------+----------------+
  kata-deploy      | "x.y.z"        | "x.y.(z+1)"    |
  -----------------+----------------+----------------+
  kata-deploy-base | NON-EXISTENT   | NON-EXISTENT   |
  -----------------+----------------+----------------+
```

And we can easily cover those 3 cases only with the information about
the "${target_branch}" and the "${new_version}", where:
* case 1) if "${target_branch}" is "main" *and* "${new_version}"
  contains "alpha", do nothing
* case 2) if "${target_branch}" is "main" *and* "${new_version}"
  contains "rc":
  * change the kata-deploy & kata-cleanup tags from "latest" to
    "${new_version}".
  * delete the kata-deploy-stable & kata-cleanup-stable files.
* case 3) if the "${target_branch}" contains "stable":
  * change the kata-deploy & kata-cleanup tags from "${current_version}"
    to "${new_version}".

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-09-21 22:48:04 +02:00
Fabiano Fidêncio
ea9b2f9c92 kata-deploy: Add "stable" info to the README
Similar to the instructions we have for the "latest" images, let's also
add instructions about the "stable" images.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-09-21 22:48:04 +02:00
Fabiano Fidêncio
e541105680 kata-deploy: Update the README
Let's just point to our repo URLs rather than assume users using
kata-deploy will have our repo cloned.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-09-21 22:48:04 +02:00
Fabiano Fidêncio
9acf4e5d32 kata-deploy: Add stable yaml files
This is **not** the nicest patch of my career, and I know it adds code
duplication.  However, I've decided to take this approach in order to
have easier / better instructions for users who're consuming
kata-deploy.

Having both stable & latest yaml on `main` will let us point to just one
place, without having to update the instructions.

I know, would be better to have those generated from a .in file,
wouldn't it?  For sure, but then we'd lose the ability to just point to
those files from kata-deploy pages (either on dockerhub or quay.io).

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-09-21 22:48:04 +02:00
Fabiano Fidêncio
a86babe0d0 kata-deploy: Point to the latest release
Instead of point to a specific release number, let's point to the
`latest` tag on the main branch.

There's still some work needed in order to point to the `stable` tag on
the stable-x.y branches, as this is something that should be done
automagically as part of the release process.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-09-21 22:48:04 +02:00
David Gibson
e7deee948a Merge pull request #2502 from dgibson/qemu-6.1
Update Kata to use qemu-6.1
2021-09-15 11:06:14 +10:00
Samuel Ortiz
057eb80ac9 Merge pull request #2596 from jongwu/qemu_mak
qemu: remove default config for arm64.
2021-09-13 11:23:35 +02:00
David Gibson
25670d3058 packaging/qemu: Update qemu-exerimental version to v6.1.0
This brings it back into line with the normal qemu version.  We refer to
v6.1.0 by full SHA in versions.yaml, rather than the tag, so that
apply_patches.sh sees it as different and applies the virtiofs DAX patches
which is what the experimental version is actually about having.

The virtiofs DAX patches themselves are updated to the version from
https://gitlab.com/virtio-fs/qemu, virtio-fs-dev branch as of commit
3620cb0a.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-11 16:43:26 +10:00
David Gibson
041a513f80 versions: Update qemu to v6.1.0
We need qemu-6.1 for ACPI PCI hotplug support for the q35 machine.  At the
moment qemu will use SHPC hotplug under the PCIe to PCI bridge on q35.
SHPC is too slow to use for our purposes (it requires a 5s delay).

Update the qemu version to v6.1.0.  This leaves the experimental version
*older* than the normal version, but we'll fix that up later.

We also need to tweak the snapcraft.yaml, since the location for configs
has changed in the new qemu version.

fixes #1691

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-11 16:43:22 +10:00
David Gibson
81de2d476b packaging: Correct error message in apply_patches.sh
If the script doesn't find a patches directory it expects, it gives an
error saying to create a dummy 'no_patches' file if you really don't want
any patches applied for that version.

But actual practice in the tree is to call the dummy file 'no_patches.txt'
rather than simply 'no_patches'.  Correct the message to match existing
practice.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-10 11:19:10 +10:00
Carlos Venegas
230eae3ff3 Merge pull request #2417 from jcvenegas/docker-build-fixes
kata-tarball: Build and test fixes
2021-09-09 14:14:26 -05:00
Carlos Venegas
78d99f5129 kata-deploy: Make verbose single builds
If a binary tarball for a single component is done,
the logs will be shown in stdout.

e.g.

make kernel-tarball

To build all a the same time still store logs in files.

make kata-tarball

Signed-off-by: Carlos Venegas <jose.carlos.venegas.munoz@intel.com>
2021-09-09 14:32:08 +00:00
Carlos Venegas
59486b855a kata-deploy: Add tarball suffix to makefile targets
Now that local-build kata-deploy makefile is inlucded in toplevel
makefile, lets use the suffix `-tarball` to avoid name collitions
and identify the tarball releted targets.

Signed-off-by: Carlos Venegas <jose.carlos.venegas.munoz@intel.com>
2021-09-09 14:32:08 +00:00
Binbin Zhang
924a68d08d osbuilder: Change to "=" operator to make script more portable
zsh doesn't support "==" as equal comparison operator, so
replace "==" with "=" to make the script more portable

Fixes: #2584

Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
2021-09-08 16:57:34 +08:00
Jianyong Wu
1fff9be707 qemu: remove default config for arm64.
The current default config in qemu for arm64 doesn't suit for qemu
version 5.1+, so remove them here.

Fixes: #2595
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-09-08 16:25:22 +08:00
Bin Liu
3c9170ea0d Merge pull request #2561 from rapiz1/patch-4
osbuilder: fix inconsistent calculation of fs size
2021-09-04 15:14:07 +08:00
James O. D. Hunt
f3a1bf3b45 Merge pull request #2552 from bergwolf/license
license: drop redundent license files
2021-09-02 14:31:18 +01:00
Yujia Qiao
bfcee91164 osbuilder: fix inconsistent calculation of fs size
This patch fixes inconsistent calculations of the rootfs size.
For `du` and `df`, `-B 1MB` is different from `-BM`. The
former is the power of 1000, and the latter is the power of
1024. So comparing them doesn't make sense. The bug may result
in a larger image than needed.

Fixes: #2560

Signed-off-by: Yujia Qiao <rapiz3142@gmail.com>
2021-09-02 16:00:29 +08:00
Peng Tao
256c3b2747 license: drop redundent license files
There is no need to keep multiple copies of the license file in
different directory. We can just use the top level one for the project.

Fixes: #2553
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-09-01 15:10:04 +08:00
Fabiano Fidêncio
18c95b9ab1 release: Kata Containers 2.3.0-alpha0
- tracing: Change runtime tracing tags to vars
- shimv2: add logging to shimv2 api calls
- drop qemu-lite support
- runtime: delete types or const that no longer needed
- runtime: Optimize the way slice created
- virtcontainers: simplify tests
- virtcontainers: clh: Upgrade to the openapi-generator v5.2.1
- build_image: Fix error soft link about initrd.img
- ci: Temporarily skip agent shutdown test on s390x
- Fix version parsing for firecracker version 0.25 and over
- Osbuilder fixes
- docs: update the GoDoc url from runtime project to kata-containers/sr…
- docs: update `how-to` README file for Firecracker config
- ci/openshift-ci: Pull centos from registry.centos.org
- docs: update containerd CRI plugin url

2250360b docs: remove mentioning of qemu-lite
a9de761d runtime: drop qemu-lite support
8ae3edbc runtime: fix default hypervisor path
0c7789fa runtime: Add container field to logs
72e3538e shimv2: add information to method comment
8dadca9c shimv2: add logging to shimv2 api calls
a99fcc3a virtcontainers: simplify tests
39ffd8ee runtime: delete types or const that no longer needed
ff37f5c7 runtime: Optimize the way slice created
8f0f949a tracing: Move dynamically added attributes to Trace()
932ee41b virtcontainers: clh: Workaround incorrect default values
bff38e4f virtcontainers: clh: Fix the unit test
d967d3cb virtcontainers: clh: Use constructors to ensure proper default value
87de26bd tracing: Modify Trace() to accept multiple tag maps
8058e972 tracing: Change runtime tracing tags to vars
a6a2e525 virtcontainers: clh: Migrate to use the updated client APIs
9de1129b osbuilder: Fix rootfs-builder when running in VMs
65a1e131 osbuilder: Allow running the tool several times
a4214738 osbuilder: Fix Makefile
b8717f35 ci: Temporarily skip agent shutdown test on s390x
938981be build_image: Fix error soft link about initrd.img
2304f935 docs: update the GoDoc url from kata 1.x to 2.x
2a614577 docs: update `how-to` README file for Firecracker config
486baba7 docs: update containerd CRI plugin url
46eb07e1 virtcontainers: clh: Re-generate the client code
80fba4d6 virtcontainers: clh: Upgrade to the openapi-generator v5.2.1
8594f80c ci/openshift-ci: Pull centos from registry.centos.org
87bbae1b fc: fix version parsing for fc >= 0.25

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-08-31 18:43:09 +02:00
GabyCT
d490704133 Merge pull request #2504 from Bevisy/main-2503
build_image: Fix error soft link about initrd.img
2021-08-26 13:26:52 -05:00