Commit Graph

55 Commits

Author SHA1 Message Date
Derek Lee
0e40ecf383 tools/snap: simplify nproc
Replaces calls of nproc	with nproc with

nproc ${CI:+--ignore 1}

to run nproc with one less processing unit than the maximum to prevent
DOS-ing the local machine.

If process is being run in a container (determined via whether $CI is
null), all processing units avaliable will be used.

Fixes #3967

Signed-off-by: Derek Lee <derlee@redhat.com>
2022-07-06 15:04:08 -07:00
Amulyam24
0bbbe70687 snap: fix snap build on ppc64le
Fixes the syntax error while building rustdeps.

Fixes: #4494

Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
2022-06-20 19:26:27 +05:30
Fabiano Fidêncio
1ef0b7ded0 runtime: Switch to using the rust version of virtiofsd (all but power)
So far this has been done for x86_64.  Now that the support for building
and testing has been added for all arches, let's do the second part of
the switch.

We're still not done yet for powerpc, as some a virtifosd crash on the
rust version has been found by the maintainer.

Fixes: #4258, #4260

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-06-13 10:41:26 +02:00
Gabriela Cervantes
168f325c43 docs: Update configuration reference for snap documentation
This PR updates the url link for the kata containers configuration
for the general snap documentation.

Fixes #4341

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-06-02 14:55:06 +00:00
James O. D. Hunt
002f2cd109 snap: Use helper script and cleanup
Move the common shell code to a helper script that is sourced by all
parts.

Add extra quoting to some variables in the snap config file
and simplify.

Fixes: #4304.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-06-01 16:09:29 +01:00
James O. D. Hunt
9b108d9937 docs: Improve snap formatting
Improve the snap docs by using more consistent formatting and proper
shell code in the shell example.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-06-01 12:00:40 +01:00
James O. D. Hunt
894f661cc4 docs: Add warning to snap build
Since we must build with `--destructive-mode`, add a warning that the
host environment could change the behaviour of the build, depending on
the packages installed on the system.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-06-01 12:00:40 +01:00
James O. D. Hunt
d759f6c3e5 snap: Fix CH architecture check
Correct the `cloud-hypervisor` part architecture check to use `x86_64`, not
`x64_64`.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-06-01 12:00:38 +01:00
James O. D. Hunt
56591804b3 docs: Improve snap build instructions
Make it clearer how to build the snap package manually.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-05-26 15:56:36 +01:00
James O. D. Hunt
60823abb9c docs: Move snap README
Move the snap README to a subdirectory to resolve the warning given by
`snapcraft` (folded and reformatted slightly for clarity):

```
The 'snap' directory is meant specifically for snapcraft,
but it contains the following non-snapcraft-related paths,
which is unsupported and will cause unexpected behavior:

- README.md

If you must store these files within the 'snap' directory,
move them to 'snap/local', which is ignored by snapcraft.
```

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-05-26 15:56:36 +01:00
James O. D. Hunt
49361749ed snap: Build and package rust version of virtiofsd
Update the snap config file to build the rust version of `virtiofsd` for
x86_64, but build QEMU's C version for other platforms.

Fixes: #4261.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-05-25 17:04:05 +01:00
James O. D. Hunt
27d903b76a snap: Put the yq binary in the staging bin directory
Rather than putting the `yq` binary in the staging directory itself,
put it in the `bin/` sub-directory.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-05-25 09:40:09 +01:00
James O. D. Hunt
d7b4ce049e snap: Remove unused variable
Remove the unused `kata_url` variable and use the value in the `website`
YAML metadata instead.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-05-25 09:40:09 +01:00
James O. D. Hunt
43de5440e5 snap: Fix unbound variable error
Don't assume `GITHUB_REF` is set.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-05-25 09:40:09 +01:00
James O. D. Hunt
c9b291509d snap: Fix whitespace
Remove trailing space.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-05-25 09:40:09 +01:00
James O. D. Hunt
799c2f4f2a
Merge pull request #3800 from jodh-intel/git-clone-depth-1-where-possible
snap: Use git clone depth 1 for QEMU and dependencies
2022-03-03 16:27:07 +00:00
Fabiano Fidêncio
c1fb4bb726 snap: Don't build cloud-hypevisor on ppc64le
snapcraft build is failing due to:
 ``
utils.mk:130: "WARNING: powerpc64le-unknown-linux-musl target is unavailable"
```

It seems to happen as powerpc64-unknown-linux-musl is a target that
although there's support for it, it's not exactly built or
automatically tested, at least according to:
https://doc.rust-lang.org/rustc/platform-support.html

Fixes: #3803

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-03-02 19:04:30 +01:00
James O. D. Hunt
58913694d3 snap: Use git clone depth 1 for QEMU and dependencies
Use `git clone --depth 1 ...` for QEMU and its dependencies
to speed up checkouts.

Fixes: #3799.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-03-02 08:31:06 +00:00
Fabiano Fidêncio
a67b93bb03 snap: clh: Re-use kata-deploy script here
The current snap build for clh is broken as it's not aware of how to
build the binary from sources.

Instead of fixing it here, let's take advantage of the kata-deploy
script, which is capable of building from sources, and re-use it here.

Fixes: #3693

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-03-01 09:03:51 +01:00
Fabiano Fidêncio
32e7845d31 snap: Build vanilla kernel for all arches
There's no need to build an experimental kernel for x86_64 as all the
bits which were part of the experimental one (SGX only, really) are now
part of the vanilla one.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-02-18 10:41:08 +01:00
Jianyong Wu
419d813427 snap: update qemu version to 6.1.0 for arm
Update qemu version of snap for arm to 6.1.0 thus the arch specific qemu
version for arm needs clean up.

Fixes: #3627
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2022-02-07 14:48:23 +08:00
James O. D. Hunt
6f9efb4043 docs: Move arch doc to separate directory
Move the architecture document into a new `docs/design/architecture/` directory
in preparation for splitting it into more manageable pieces.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-12-16 12:26:17 +00:00
Julio Montes
ff929fc081 snap: read initrd and image distros from version.yaml
Build initrd or image rootfs using the distro name specified
in the versions.yaml

fixes #3208

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-12-06 08:42:07 -06:00
Julio Montes
24fff57c23 snap: make curl commands consistent
remove -k and -Ssf from curl commands

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-10-19 08:36:06 -05:00
Julio Montes
2b9f79cfc9 snap: add cloud-hypervisor and experimental kernel
Add cloud-hypervisor and experimental kernel as part of the kata snap

fixes #2852

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-10-19 08:35:06 -05:00
David Gibson
041a513f80 versions: Update qemu to v6.1.0
We need qemu-6.1 for ACPI PCI hotplug support for the q35 machine.  At the
moment qemu will use SHPC hotplug under the PCIe to PCI bridge on q35.
SHPC is too slow to use for our purposes (it requires a 5s delay).

Update the qemu version to v6.1.0.  This leaves the experimental version
*older* than the normal version, but we'll fix that up later.

We also need to tweak the snapcraft.yaml, since the location for configs
has changed in the new qemu version.

fixes #1691

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-11 16:43:22 +10:00
David Gibson
4996f9b7da snap: Test variable instead of executing "branch"
In snapcraft.yaml we have a case statement on $(branch) - that is on the
output of executing a command "branch".  From the selections it appears
that what it actually wants is to simply select on the contents of the
$branch variable, which should be ${branch} instead.

fixes #2558

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-02 14:38:15 +10:00
Jianyong Wu
6871aeaa60 snap: enable snap build for arm64
snap build for arm64 fail for a long time, here we enable it.
the changes:
1. correct the variable of "branch"
2. add v5.1.0 under tag_patchs

Fixes: #2194
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-08-06 17:11:58 +08:00
James O. D. Hunt
4f0726bc49 docs: Remove table of contents
Removed all TOCs now that GitHub auto-generates them.

Also updated the documentation requirements doc removing the requirement
to add a TOC.

Fixes: #2022.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-07-30 10:58:22 +01:00
Fabiano Fidêncio
2d142bc92d
Merge pull request #2155 from jcvenegas/kata-deploy-2021-06-29
kata-deploy: Allow build  kata-deploy  tarball from HEAD
2021-07-29 22:50:03 +02:00
Carlos Venegas
3533a5b61d Packaging: stop using GOPATH for yq.
Use the yq installed in the env.  Needed
to build kata from docker. The container builder
has not an initial Go env.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-28 19:45:35 +00:00
Jakob Naucke
8e9ffe6f3a
snap: Substitute image configuration with initrd
rather than removing the other line because configuration only contains
the image line ever more and this is how we already do it in tests.

Fixes: #2330
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-07-27 18:39:14 +02:00
Jakob Naucke
9c0b8a7f5b
snap: do not export agent version
This causes the repository to be checked out to a version tag, which is
inconsistent with how we build runtime, and reverts us to a buggy
`snap/snapcraft.yaml`.

Fixes: #2313
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-07-26 17:32:18 +02:00
Julio Montes
5512da1739
Merge pull request #2280 from YchauWang/wyc-arm-qemu
snap: fixed snap aarch64 qemu patches dir in snapcraft.yaml file
2021-07-20 07:33:39 -05:00
wangyongchao.bj
a8649acfd0 snap: fixed snap aarch64 qemu patches dir in snapcraft.yaml file
fixed arm qemu patches dir in snap part. Clear the old `packaging/obs-packaging` path.

Fixes: #2279

Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
2021-07-20 15:24:42 +08:00
Jakob Naucke
558f1be62d
snap: Remove QEMU before clone
If you snap in an environment where you previously snapped,
`git clone`ing QEMU will fail. Remove the checkout directory.

Fixes: #2249
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-07-15 12:22:21 +02:00
Julio Montes
8d6dd2ad61 snap: support golang 1.16.x
Fix build process, set GO111MODULE environment variable to "auto"

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-07-13 12:22:33 -05:00
Jakob Naucke
007a656173
snap: Build initrd on ppc64le & s390x
instead of image, does not require privileged containers since `losetup`
is not used and is thus more portable for various build environments.

Fixes: #2218
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-07-12 15:17:30 +02:00
Jakob Naucke
2cc9006c26
snap: Miscellaneous s390x fixes
- Ported from https://github.com/kata-containers/tests/pull/3612:
  Install protobuf-compiler for agent build on ppc64le & s390x
- Fixes in image target for ppc64le & s390x
  - Install image instead of initrd since it's preferred
  - Use Ubuntu as base since Alpine requires a musl agent (cannot be
    built on ppc64le & s390x because there is no such Rust target)
- Ported from
  https://github.com/kata-containers/kata-containers/pull/1265:
  Fix vmlinux install path
- Install libseccomp-dev on all architectures, not just x86_64

Fixes: #2192
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-07-08 15:02:47 +02:00
Carlos Venegas
5a3ee7d7e9 snap: Use qemu.version to build snap
Use only one key to refer the version used by kata.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-03-24 17:52:52 +00:00
Wainer dos Santos Moschetta
224c50f420 snap: Package virtiofsd and fix path
This contain to fixes for the virtiofsd on snap:
 * removed the "-/usr/libexec" so that virtiofsd is copied to prime
 * The configuration.toml expects virtiofsd in /usr/libexec/kata-qemu so it should be passed "kata-qemu"
   to configure_hypervisor.sh script and it wil configure to install the executable onto the right directory.

Fixes #1238
Depends-on: github.com/kata-containers/kata-containers#1349
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-03-10 11:16:52 -03:00
Wainer dos Santos Moschetta
85601cd360 snap: Update for QEMU 5.2.0
QEMU 5.2.0 needs ninja-build package installed on the build environment.

The default-configs were copied to $QEMU_SRC/default-configs but that does
take any effect, so instead it is now copied to $QEMU_SRC/default-configs/devices
and the configs for i386 were updated.

Also it had to change some arguments being passed to configure as Meson was failing
due inconsistent paths:

  ./meson.build:1:0: ERROR: The value of the 'libdir' option is '/usr/lib/qemu' which must be a subdir of the prefix '/snap/kata-containers/current/usr'.
  Note that if you pass a relative path, it is assumed to be a subdir of prefix.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-03-01 16:57:58 -05:00
Julio Montes
919d51274d snap: fix kernel setup
kernel setup fails when `yes "n"` is used and `make oldconfig` doesn't
read anything from STDIN, `yes "n"` was added in the past as a
workaround to fix incomplete kernel configs.
Enable `build-kernel.sh` debug.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-02-12 09:16:09 -06:00
Julio Montes
d054841430 ci: snap: build targets that not need sudo first
`sudo` is required to build the image, once the image has been built
the permission of some directories may change, let's build first the
targerts that not need `sudo`

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-02-12 09:05:42 -06:00
Julio Montes
a115338ddd ci: snap: define proxy variables
define proxy variables before using them to fix `unbound variable`
error

fixes #1386

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-02-12 09:05:42 -06:00
Julio Montes
5ce74bab41 snap: tag yq version
yq major releases are not backward compatible, install the same
major version used in the CI to avoid conflics building the kata
components.
We should update yq when the CI updates it, not before.

fixes #1232

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-01-14 11:44:39 -06:00
Julio Montes
ef1feaf38f revert: "snap: Fix yq error in build"
This reverts commit 6cc1920c37.

Instead of updating the syntax of yq, let's use yq 3.x, otherwise
yq must be updated in the CI and the syntax updated in all the
tools (osbuilder, packging).

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-01-14 11:36:44 -06:00
James O. D. Hunt
6cc1920c37 snap: Fix yq error in build
The snap build pulls the latest release of `yq`, but `yq` version 4
changed the CLI syntax for reading a YAML file.

Update the snap config file to use the new `yq` v4 syntax.

Fixes: #1232.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-01-11 11:24:46 +00:00
Julio Montes
13a8e4e39e snap: update apps section
Add `kata-runtime` and `kata-collect-data.sh` commands to the apps
section, these two command will be accessible through the commands
`kata-containers.runtime` and `kata-containers.collect-data`
respectively.
Henceforth the snap command for `containerd-shim-kata-v2` will be
`kata-containers.shim`

fixes #1122

Signed-off-by: Julio Montes <julio.montes@intel.com>
2020-11-17 11:31:08 -06:00
Julio Montes
5b079a3ba7 snap: add GH actions jobs to release the snap package
Use Github actions to build and release the snap package automatically
when a new tag is pushed.

fixes #1006

Signed-off-by: Julio Montes <julio.montes@intel.com>
2020-10-20 14:34:59 -05:00