Commit Graph

666 Commits

Author SHA1 Message Date
Sebastian Hasler
adffd3f8b6 scripts: Use shebang /usr/bin/env bash
Not all distros have `/bin/bash`, e.g. NixOS.

Fixes: #3450

Signed-off-by: Sebastian Hasler <sebastian.hasler@stuvus.uni-stuttgart.de>
2022-01-13 22:53:28 +01:00
Fabiano Fidêncio
e4b7a12bf3 qat: Add Debian to the distro examples
Debian is a supported rootfs that uses systemd as init, thus, it should
be mentioned in the QAT README document.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-01-13 21:54:05 +01:00
Fabiano Fidêncio
6979d5be69 osbuilder: Remove gentoo rootfs-builder
As the gentoo rootfs is not tested in our CI, we can't guarantee it
actually works as expected.

Whenever we have someone willing to maintain this rootfs we can have it
added back, and also add a CI job to test it altogether, avoiding then
any possible regression.

Fixes: #2144

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-01-13 21:54:05 +01:00
Fabiano Fidêncio
22c1a093d7 osbuilder: Remove suse rootfs-builder
As the suse rootfs is not tested in our CI, we can't guarantee it
actually works as expected.

Whenver we have someone willing to maintain this rootfs we can have it
added back, and also add a CI job to test it altogether, avoiding then
any possible regression.

Fixes: #2145

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-01-13 21:54:05 +01:00
Fabiano Fidêncio
85dd587382 osbuilder: Remove fedora rootfs-builder
As the fedora rootfs is not tested in our CI, we can't guarantee it
actually works as expected.

Whenever we have someone willing to maintain the rootfs we can have it
added back, and also add a CI job to test it altogether, avoiding then
any possible regression.

Fixes: #2143

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-01-13 21:54:05 +01:00
Fabiano Fidêncio
06fae29f49 osbuilder: Remove centos rootfs-builder
As the centos rootfs is not tested in our CI, we can't guarantee it
actually works as expected.

Whenever we have someone willing to maintain the rootfs we can have it
added back, and also add a CI job to test it altogether, avoiding then
any possible regression.

Fixes: #2140

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-01-13 21:54:05 +01:00
Gabriela Cervantes
01005c5a9c docs: Remove ccloudvm reference
This PR removes the ccloudvm reference at the README document as the
setup of scripts of ccloudvm were removed.

Fixes #3448

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-01-13 18:22:26 +00:00
Snir Sheriber
ac7acbf87b kata-deploy: validate conf file can be created
As containerd doesn't exist at cleanup

Fixes: #3429
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2022-01-12 10:12:46 +02:00
Gabriela Cervantes
7e2bc4d764 packaging: Remove ccloudvm instructions and script
This PR removes ccloudvm for kata 2.0, ccloudvm was used in kata 1.x
and we are not longer using it for kata 2.0.

Fixes #3427

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-01-11 15:41:16 +00:00
Gabriela Cervantes
b48322d44e packaging: Remove obs packages testing for kata 2.0
This PR removes the scripts and the dockerfiles that were used in kata 1.x
to test the different kata components for different distributions in OBS.
Currently for kata 2.0 we are not generating packages in OBS so these scripts
are not longer being used.

Fixes #3404

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-01-07 17:06:20 +00:00
Snir Sheriber
e2c1e65e27 kata-deploy: fix tar command in dockerfile
tar params are passed wrongly

Fixes: #3394
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2022-01-05 20:07:52 +02:00
Bin Liu
b2166560fa
Merge pull request #3375 from zhaojizhuang/debianrootfs
osbuilder: Restore Debian as a rootfs
2022-01-05 10:27:47 +08:00
zhaojizhuang
3093f93a6f osbuilder: Restore Debian as a rootfs
Restore Debian as a rootfs.
1. revert of #3154, but some change
2. update debian version to 10.11
3. update  `libstdc++-6-dev` to `libstdc++-8-dev`
4.  changes discarded in QAT are not restored

Fixes: #3372
Signed-off-by: zhaojizhuang <571130360@qq.com>
2022-01-04 11:54:34 +08:00
Wainer Moschetta
820dc930db
Merge pull request #3109 from wainersm/delint_dockerfiles
Delint dockerfiles
2021-12-28 10:11:51 -03:00
Jakob Naucke
55bac67ac6
docs: Fix kernel configs README spelling errors
- `fragments` in backticks
- s/perfoms/performs/

Fixes: #3338
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-22 18:57:47 +01:00
Fabiano Fidêncio
67f0ab4092
Merge pull request #3294 from Kvasscn/kata_dev_osbuilder_makefile
osbuilder: avoid to copy versions.txt which already deprecated
2021-12-21 16:07:01 +01:00
Wainer dos Santos Moschetta
d79268ac65 tools/packaging: add copyright to kata-monitor's Dockerfile
The kata-monitor's Dockerfile was added by Eric Ernst on commit 2f1cb7995f
but for some reason the static checker did not catch the file misses the copyright statement
at the time it was added. But it is now complaining about it. So this assign the copyright to
him to make the static-checker happy.

Fixes #3329
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-12-21 10:01:11 -05:00
Fabiano Fidêncio
79153c3845
Merge pull request #3288 from gkurz/qemu-disable-libudev
qemu: Disable libudev for QEMU 5.2 and newer
2021-12-21 15:56:16 +01:00
Wainer dos Santos Moschetta
428cf0a685 packaging: delint tests dockerfiles
Removed all errors/warnings pointed out by hadolint version 2.7.0, except for the following
ignored rules:
  - "DL3008 warning: Pin versions in apt get install"
  - "DL3041 warning: Specify version with `dnf install -y <package>-<version>`"
  - "DL3033 warning: Specify version with `yum install -y <package>-<version>`"
  - "DL3048 style: Invalid label key"
  - "DL3003 warning: Use WORKDIR to switch to a directory"
  - "DL3018 warning: Pin versions in apk add. Instead of apk add <package> use apk add <package>=<version>"
  - "DL3037 warning: Specify version with zypper install -y <package>[=]<version>"

Fixes #3107
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-12-21 09:54:44 -05:00
Wainer dos Santos Moschetta
1ea9b70383 packaging: delint kata-deploy dockerfiles
Removed all errors/warnings pointed out by hadolint version 2.7.0, except for the following
ignored rules:
  - "DL3008 warning: Pin versions in apt get install"
  - "DL3041 warning: Specify version with `dnf install -y <package>-<version>`"
  - "DL3033 warning: Specify version with `yum install -y <package>-<version>`"
  - "DL3048 style: Invalid label key"
  - "DL3003 warning: Use WORKDIR to switch to a directory"
  - "DL3018 warning: Pin versions in apk add. Instead of apk add <package> use apk add <package>=<version>"
  - "DL3037 warning: Specify version with zypper install -y <package>[=]<version>"

Fixes #3107
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-12-21 09:54:44 -05:00
Wainer dos Santos Moschetta
aeb2b673b3 osbuilder: delint dockerfiles
Removed all errors/warnings pointed out by hadolint version 2.7.0, except for the following
ignored rules:
  - "DL3008 warning: Pin versions in apt get install"
  - "DL3041 warning: Specify version with `dnf install -y <package>-<version>`"
  - "DL3033 warning: Specify version with `yum install -y <package>-<version>`"
  - "DL3048 style: Invalid label key"
  - "DL3003 warning: Use WORKDIR to switch to a directory"
  - "DL3018 warning: Pin versions in apk add. Instead of apk add <package> use apk add <package>=<version>"
  - "DL3037 warning: Specify version with zypper install -y <package>[=]<version>"

Fixes #3107
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-12-21 09:54:44 -05:00
Wainer dos Santos Moschetta
bc120289ec packaging: delint kata-monitor dockerfiles
Removed all errors/warnings pointed out by hadolint version 2.7.0, except for the following
ignored rules:
  - "DL3008 warning: Pin versions in apt get install"
  - "DL3041 warning: Specify version with `dnf install -y <package>-<version>`"
  - "DL3033 warning: Specify version with `yum install -y <package>-<version>`"
  - "DL3048 style: Invalid label key"
  - "DL3003 warning: Use WORKDIR to switch to a directory"
  - "DL3018 warning: Pin versions in apk add. Instead of apk add <package> use apk add <package>=<version>"
  - "DL3037 warning: Specify version with zypper install -y <package>[=]<version>"

Fixes #3107
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-12-21 09:54:44 -05:00
Wainer dos Santos Moschetta
bc71dd5812 packaging: delint static-build dockerfiles
Removed all errors/warnings pointed out by hadolint version 2.7.0, except for the following
ignored rules:
  - "DL3008 warning: Pin versions in apt get install"
  - "DL3041 warning: Specify version with `dnf install -y <package>-<version>`"
  - "DL3033 warning: Specify version with `yum install -y <package>-<version>`"
  - "DL3048 style: Invalid label key"
  - "DL3003 warning: Use WORKDIR to switch to a directory"
  - "DL3018 warning: Pin versions in apk add. Instead of apk add <package> use apk add <package>=<version>"
  - "DL3037 warning: Specify version with zypper install -y <package>[=]<version>"

Fixes #3107
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-12-21 09:54:41 -05:00
Fabiano Fidêncio
99ef52a35d osbuilder: Add protoc to the alpine container
It seems the lack of protoc in the alpine containers is causing issues
with some of our CIs, such as the VFIO one.

Fixes: #3323

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-12-21 13:57:18 +01:00
Wainer dos Santos Moschetta
2938bb7f89 packaging/qemu: Use QEMU script to update submodules
Currently QEMU's submodules are git cloned but there is the scripts/git-submodule.sh
which is meant for that. Let's use that script.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-12-17 10:20:59 -03:00
Wainer dos Santos Moschetta
5d49ccd613 packaging/qemu: Use partial git clone
The static build of QEMU takes a good amount of time on cloning the
source tree because we do a full git clone. In order to speed up that
operation this changed the Dockerfile so that it is carried out a
partial clone by using --depth=1 argument.

Fixes #3291
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-12-17 10:20:29 -03:00
zhanghj
d1bc409d57 osbuilder: avoid to copy versions.txt which already deprecated
Currently the versions.txt in rootfs-builder dir is already removed,
so avoid to copy it in list of helper files.

Fixes: #3267

Signed-off-by: zhanghj <zhanghj.lc@inspur.com>
2021-12-17 17:23:05 +08:00
Greg Kurz
12c8e41c75 qemu: Disable libudev for QEMU 5.2 and newer
Commit 112ea25859 disabled libudev for static builds because it was
breaking snap. It turns out that the only users of libudev in QEMU are
qemu-pr-helper and USB. Kata already disables USB and doesn't use
qemu-pr-helper. Disable libudev for all builds if QEMU supports it, i.e.
version 5.2 or newer.

Fixes #3078

Signed-off-by: Greg Kurz <groug@kaod.org>
2021-12-16 16:12:02 +01:00
Jakob Naucke
a40e4877e9
Merge pull request #3266 from liubin/fix/3265-update-golang-to-1.16-and-remove-ioutil
runtime: update golang to 1.16 and remove ioutil package
2021-12-15 10:09:23 +01:00
bin
03546f75a6 runtime: change io/ioutil to io/os packages
Change io/ioutil to io/os packages because io/ioutil package
is deprecated from 1.16:

Discard => io.Discard
NopCloser => io.NopCloser
ReadAll => io.ReadAll
ReadDir => os.ReadDir
ReadFile => os.ReadFile
TempDir => os.MkdirTemp
TempFile => os.CreateTemp
WriteFile => os.WriteFile

Details: https://go.dev/doc/go1.16#ioutil

Fixes: #3265

Signed-off-by: bin <bin@hyper.sh>
2021-12-15 07:31:48 +08:00
Jakob Naucke
70274b9d39
Merge pull request #3258 from fidencio/wip/kata-deploy-count-with-a-non-existend-containerd-config-file
kata-deploy: Deal with empty containerd conf file
2021-12-14 20:14:41 +01:00
Bin Liu
6c34446f49
Merge pull request #3244 from bergwolf/reorg-code
src: reorg source code directory
2021-12-14 21:57:07 +08:00
Peng Tao
7c4263b3e1 src: reorg source directories
To make the code directory structure more clear:

└── src
    ├── agent
    ├── libs
    │   └── logging
    ├── runtime
    ├── runtime-rs (to be added)
    └── tools
        ├── agent-ctl
        └── trace-forwarder

Fixes: #3204
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-12-14 10:30:08 +08:00
Fabiano Fidêncio
8457150684 kata-deploy: Deal with empty containerd conf file
As containerd can properly run without having a existent
`/etc/containerd/config.toml` file (it'd run using the default
cobnfiguration), let's explicitly create the file in those cases.

This will avoid issues on ammending runtime classes to a non-existent
file.

Fixes: #3229

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Tested-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-13 11:20:22 +01:00
bin
3f7cf7ae67 osbuilder: show usage if no options/arguments specified
Now if no options/arguments specified, the shell scripts will return an error:

ERROR: Invalid rootfs directory: ''

This commit will show usage if no options/arguments specified.

Fixes: #3256

Signed-off-by: bin <bin@hyper.sh>
2021-12-13 16:10:55 +08:00
Bin Liu
978b13c9e8
Merge pull request #3235 from Kvasscn/kata_dev_image_builer_help
image_build: add help info for '-f' option and 'BLOCK_SIZE' env.
2021-12-09 22:55:24 +08:00
Julio Montes
70062e1563
Merge pull request #3238 from snir911/wip/build_with_runtime
osbuilder: be runtime consistent with podman build
2021-12-09 08:06:00 -06:00
Snir Sheriber
2ebaaac73d osbuilder: be runtime consistent also with podman build
Use the same runtime used for podman run also for the podman build cmd
Additionally remove "docker" from the docker_run_args variable

Fixes: #3239
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2021-12-09 11:28:16 +02:00
Jakob Naucke
2204ecac39
versions: Upgrade Alpine, using minor version
- Upgrade Alpine guest rootfs to 3.15
- Specify a minor version rather than patch level as the Alpine
  repositories use that.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-08 15:18:44 +01:00
Jakob Naucke
dfd0732ff9
osbuilder: Revert to using apk.static for Alpine
#2399 partially reverted #418, missing on returning to bootstrapping a
rootfs with `apk.static` instead of copying the entire root, which can
result in drastically larger (more than 10x) images. Revert this as well
(requires some updates to URL building).

Fixes: #3216
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-12-08 15:18:43 +01:00
zhanghj
6b3e4c212c image_build: add help info for '-f' option and 'BLOCK_SIZE' env.
The help information of '-f' option is missing, and same issue
with 'BLOCK_SIZE' env variables, fix it in usage() function.

Fixes: #3231

Signed-off-by: zhanghj <zhanghj.lc@inspur.com>
2021-12-08 17:33:07 +08:00
yuanke wei
b5b9de1de9 kata-deploy: Update API Version of RuntimeClass to v1
API Version of node.k8s.io/v1beta1 is deprecated in
v1.22+, unavailable in v1.25+

Fixes: #3185

Signed-off-by: yuanke wei <yuanke.wyk@alibaba-inc.com>
2021-12-08 14:18:57 +08:00
Binbin Zhang
8fae263170 packaging: Fix missing commit message in building kata-runtime
add `git` package to the shim-v2 build image

Fixes: #3196

Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
2021-12-04 11:59:59 +08:00
Bin Liu
86d9d2eed5
Merge pull request #3169 from Kvasscn/kata_dev_add_install_go_help
packaging: add help information for '-f' option in install_go.sh
2021-12-03 14:39:05 +08:00
zhanghj
1e6f58e562 packaging: add help information for '-f' option in install_go.sh
add help info for force install, and remove unused '-p' option.

Fixes: #3168

Signed-off-by: zhanghj <zhanghj.lc@inspur.com>
2021-12-02 02:58:12 -05:00
Feng Wang
6105e3ee85 runtime: enable FUSE_DAX kernel config for DAX
Otherwise DAX device cannot be set up.

Fixes #3165

Signed-off-by: Feng Wang <feng.wang@databricks.com>
2021-12-01 13:38:57 -08:00
Fupan Li
87f350db53
Merge pull request #3125 from jodh-intel/update-rust-crate-versions
Update rust crate versions
2021-12-01 18:00:33 +08:00
Gabriela Cervantes
923e098db6 osbuilder: Remove debian as a rootfs
Currently we do not have debian as part of the kata CI as we
do not have a mantainer, this PR removes debian as a supported
rootfs in order to have only the distros that we are supporting
and mantainining.

Fixes #3153

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2021-11-30 19:31:33 +00:00
James O. D. Hunt
13257986ae agent-ctl: Update rust lockfile
Ran `cargo update` to bump crate versions.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-30 12:58:15 +00:00
Binbin Zhang
8ee67aae4f osbuilder: fix missing cpio package when building rootfs-initrd image
1. install cpio package before building rootfs-initrd image
2. add `pipefaili;errexit` check to the scripts

Fixes: #3144

Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
2021-11-29 23:42:44 +08:00
Peng Tao
f59d3ff600 osbuilder: add coreutils to guest rootfs
So that the debug console is more useful. In the meantime, remove
iptables as it is not used by kata-agent any more.

Fixes: #3138
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-11-29 11:22:07 +08:00
Julio Montes
857501d8dd tools/osbuilder: build QAT kernel in fedora 34
kernel compiled in fedora 35 (latest) is not working, following error
is reported:

```
qemu-system-x86_64: Error loading uncompressed kernel without PVH ELF
Note
```

Build QAT kernel in fedora 34 container to fix it

fixes #3135

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-11-26 13:56:43 -06:00
Fabiano Fidêncio
abf39ddef0
Merge pull request #3089 from fidencio/wip/kata-deploy-remove-files-and-revert-removal-as-part-of-the-release-scripts
tools: Automatically revert kata-deploy changes
2021-11-25 15:23:52 +01:00
Fabiano Fidêncio
5ba2f52c73 tools: Quote functions arguments in the update repos script
Although this is not strictly needed, better be safe than sorry on those
cases.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-11-24 22:09:58 +01:00
Fabiano Fidêncio
5dbd752f8f tools: Remove the check for the VERSION file
All repos we release (https://github.com/kata-containers/kata-containers
and https://github.com/kata-containers/tests) have a VERSION file.

Keeping a check for it, although useful for a new repo, just complicates
the use-case we currently deal with.

While here, let's also anchor the '#' and potentially exclude blank
lines, following James' suggestion.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-11-24 22:09:49 +01:00
Fabiano Fidêncio
85eb743f46 tools: Make hub usage slightly less fragile
`grep`ing by a specific output, in a specific language, is quite fragile
and could easily break `hub`.  For now, let's work this around following
James' suggestion of setting `LC_ALL=C LANG=C` when calling `hub`.

> **Note**: I don't think we should invest much time on fixing `hub`
> usage, as it'll be soon replaced by `gh`, see:
> https://github.com/kata-containers/kata-containers/issues/3083

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-11-24 22:09:30 +01:00
Fabiano Fidêncio
76540dbdd1 tools: Automatically revert kata-deploy changes
When branching the "stable-x.y" branch, we need to do some quite
specific changes to kata-deploy / kata-cleanup files, such as:
* changing the tags from "latest" to "stable-x.y".
* removing the kata-deploy / kata-cleanup stable files.

However, after the branching is done, we need to get the `main` repo to
its original state, with the kata-deploy / kata-cleanup using the
"latest" tag, and with the stable files present there, and this commit
ensures that, during the release process, a new PR is automatically
created with these changes.

Fixes: #3069

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-11-24 22:07:53 +01:00
Fabiano Fidêncio
36d73c96c8 tools: Do the kata-deploy changes on its own commit
Rather than doing the kata-deploy changes as part of the release bump
commit, let's split those on its own changes, as it will both make the
life of the reviewer less confusing and also allows us to start
preparing the field for a possible automated revert of these changes,
whenever it becomes needed.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-11-24 22:07:52 +01:00
Fabiano Fidêncio
c8e22daf67 tools: Use vars for the registry in the update repo script
Similarly to what was done for the yaml files, let's use a var for
representing the registry where our images will be pushed to and avoid
repetition and too long lines.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-11-24 22:07:03 +01:00
Fabiano Fidêncio
ac958a3073 tools: Use vars for the yaml files used in the update repo script
Instead of always writing the full path of some files, let's just create
some vars and avoid both repetition (which is quite error prone) and too
long lines (which makes the file not so easy to read).

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-11-24 22:06:45 +01:00
Fabiano Fidêncio
edca829242 tools: Rewrite the logic around kata-deploy changes
We can simplify the code a little bit, as at least now we group common
operationr together.  Hopefully this will improve the maintainability
and the readability of the code.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-11-24 22:05:35 +01:00
Fabiano Fidêncio
31f6c2c2ea tools: Update comments about the kata-deploy yaml changes
The comments were mentioning kata-deploy-base files while it really
should mention kata-deploy-stable files.

While here, I've also added a missing '"' to one of the tags.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-11-24 21:17:40 +01:00
James O. D. Hunt
09abcd4dc6 agent-ctl: Remove some unwrap and expect calls
Replace some `unwrap()` and `expect()` calls with code to return the
error to the caller.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-24 11:43:49 +00:00
James O. D. Hunt
35db75baa1 agent-ctl: Remove redundant returns
Remove a number of redundant `return`'s.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-24 11:43:49 +00:00
James O. D. Hunt
46e459584d agent-ctl: Simplify main
Make the `main()` function simpler.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-24 11:43:49 +00:00
James O. D. Hunt
c7349d0bf1 agent-ctl: Simplify error handling
Replace `ok_or().map_err()` combinations with the simpler `ok_or_else()`
construct.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-24 11:43:49 +00:00
Fabiano Fidêncio
143fb27802 kata-deploy: Use the default notation for "Note"
Let's use the default GitHub notation for notes in documentation, as
describe here:
https://github.com/kata-containers/kata-containers/blob/main/docs/Documentation-Requirements.md#notes

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Suggested-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-23 13:24:42 +01:00
Fabiano Fidêncio
45d76407aa kata-deploy: Don't mention arch specific binaries in the README
Although the binary name of the shipped binary is `qemu-system-x86_64`,
and we only ship kata-deploy for `x86_64`, we better leaving the
architecture specific name out of our README file.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-11-23 13:21:37 +01:00
Fabiano Fidêncio
d08bcde7aa
Merge pull request #3068 from fidencio/wip/kata-deploy-re-add-latest-and-stable-tags
kata-deploy: Add back stable & latest tags
2021-11-19 15:58:55 +01:00
Greg Kurz
bdde8beb52
Merge pull request #3003 from Amulyam24/snap_ppc
qemu: fix snap build on ppc64le
2021-11-18 17:46:23 +01:00
Amulyam24
112ea25859 qemu: fix snap build by disabling libudev
While building snap, static qemu is considered. Disable libudev
as it doesn't have static libraries on most of the distros of all
archs.

Fixes: #3002

Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
2021-11-18 18:50:19 +05:30
Fabiano Fidêncio
6955d1442f kata-deploy: Add back stable & latest tags
stable-2.3 was the first time we branched the repo since
43a72d76e2 was merged.  One bit that I
didn't notice while working on this, regardless of being warned by
@amshinde (sorry!), was that the change would happen on `main` branch,
rather than on the branched `stable-2.3` one.

In my mind, the workflow was:
* we branch.
* we do the changes, including removing the files.
* we tag a release.

However, the workflow actually is:
* we do the changes, including removing the files.
* we branch.
* we tag a release.

A better way to deal with this has to be figured out before 2.4.0 is
out, but for now let's just re-add the files back.

Fixes: #3067

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-11-18 09:41:54 +01:00
James O. D. Hunt
8ab90e1068 agent-ctl: Allow API specification in JSON format
Update the `agent-ctl` tool to allow API fields to be specified in JSON
format, either directly on the command-line, or via a file URI.

This feature is made possible by enabling `serde` support in the agent
`protocols` crate. Careful use of the `serde` macros allows the
`agent-ctl` tool to accept _partially_ specified API objects in JSON
format; fields that are not specified are set to the default value for
their respective types.

`build.rs` changes based on work by Fupan.

Fixes: #2978.

Contributions-by: Fupan Li <lifupan@gmail.com>
Contributions-by: Bin Liu <bin@hyper.sh>
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-10 10:16:04 +00:00
James O. D. Hunt
18c47fe8f3
Merge pull request #2986 from jodh-intel/rm-dynamic-tracing-api
agent: Remove dynamic tracing APIs
2021-11-10 10:10:14 +00:00
Amulyam24
7566b736ac kernel: add VFIO kernel dependencies for ppc64le
Recently added VFIO kernel configs require addtional
dependencies on pcc64le.

Fixes: #2991

Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
2021-11-09 14:38:03 +05:30
James O. D. Hunt
87f676062c agent: Remove dynamic tracing APIs
Remove the `StartTracing` and `StopTracing` agent APIs that toggle
dynamic tracing. This is not supported in Kata 2.x, as documented in the
[tracing proposals document](https://github.com/kata-containers/kata-containers/pull/2062).

Fixes: #2985.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-09 08:39:06 +00:00
James O. D. Hunt
b192d388c1
Merge pull request #2970 from jodh-intel/logging-create-tests-and-checks
logging: Always run crate tests
2021-11-08 13:16:48 +00:00
Peng Tao
99c46be787 release: Kata Containers 2.3.0-rc0
- runtime# make sure the "Shutdown" trace span have a correct end
- tracing: Accept multiple dynamic tags
- logging: Enable agent debug output for release builds
- agent: "Revert agent: Disable seccomp feature on aarch64 temporarily"
- runtime: Enhancement for Makefile
- osbuilder: build image-builder image from Fedora 34
- agent: refactor process IO processing
- agent-ctl: Update for Hybrid VSOCK
- docs: Fix outdated links
- ci/install_libseccomp: Fix libseccomp build and misc improvement
- virtcontainers: simplify read-only mount handling
- runtime: add fast-test to let test exit on error
- test: Fix random failure for TestIoCopy
- cli: Show available guest protection in env output
- Update k8s, critools, and CRI-O to their 1.22 release
- package: assign proper value to redefined_string  in build-kernel.sh
- agent: Make wording of error message match CRI-O test suite
- docs: Moving from EOT to EOF
- virtcontainers: api: update the functions in the api.md docs
- release: Upload libseccomp sources with notice to release page
- virtcontainers: check that both initrd and image are not set
- agent: Fix the configuration sample file
- runtime: set tags for trace span
- agent-ctl: Implement Linux OCI spec handling
- runtime: Remove comments about unsupported features in config for clh
- tools/packaging: Add options for VFIO to guest kernel
- agent/runtime: Add seccomp feature
- ci: test-kata-deploy: Get rid of slash-command-action action
- This is to bump the OOT QAT 1.7 driver version to the latest version.…
- forwarder: Drop privileges when using hybrid VSOCK
- packaging/static-build: s390x fixes
- agent-ctl: improve the oci_to_grpc code
- agent: do not return error but print it if task wait failed
- virtcontainers: delete duplicated notify in watchHypervisor function
- agent: Handle uevent remove actions
- enable unit test on arm
- rustjail: Consistent coding style of LinuxDevice type
- cli: Fix outdated kata-runtime bash completion
- Allow VFIO devices to be used as VFIO devices in the container
- Expose top level hypervisor methods -
- Upgrade to Cloud Hypervisor v19.0
- docs: use-cases: Update Intel SGX use case
- virtcontainers: clh: Enable the `seccomp` feature
- runtime: delete cri containerd plugin from versions.yaml
- docs: Write tracing documentation
- runtime: delete useless src/runtime/cli/exit.go
- snap: add cloud-hypervisor and experimental kernel
- osbuilder: Call detect_rust_version() right before install_rust.sh
- docs: Updating Developer Guide re qemu-img
- versions: Add libseccomp and gperf version
- Enable agent tracing for hybrid VSOCK hypervisors
- runtime: optimize test code
- runtime: use containerd package instead of cri-containerd
- runtime: update sandbox root dir cleanup behavior in rootless hypervisor
- utils: kata-manager: Update kata-manager.sh for new containerd config
- osbuilder: Re-enable building the agent in Docker
- agent: Do not fail when trying to adding existing routes
- tracing: Fix typo in "package" tag name
- kata-deploy: add .dockerignore file
- runtime: change name in config settings back to "kata"
- tracing: Remove trace mode and trace type

09d5d88 runtime: tracing: Change method for adding tags
bcf3e82 logging: Enable agent debug output for release builds
a239a38 osbuilder: build image-builder image from Fedora 34
375ad2b runtime: Enhancement for Makefile
b468dc5 agent: Use dup3 system call in unit tests of seccomp
1aaa059 agent: "Revert agent: Disable seccomp feature on aarch64 temporarily"
1e331f7 agent: refactor process IO processing
9d3ec58 runtime: make sure the "Shutdown" trace span have a correct end
3f21af9 runtime: add fast-test to let test exit on error
9b270d7 ci/install_libseccomp: use a temporary work directory
98b4406 ci/install_libseccomp: Fix fail when DESTDIR is set
338ac87 virtcontainers: api: update the functions in the api.md docs
23496f9 release: Upload libseccomp sources with notice to release page
e610fc8 runtime: Remove comments about unsupported features in config for clh
7e40195 agent-ctl: Add stub for AddSwap API
82de838 agent-ctl: Update for Hybrid VSOCK
d1bcf10 forwarder: Remove quotes from socket path in doc
e66d047 virtcontainers: simplify read-only mount handling
bdf4824 tools/packaging: Add options for VFIO to guest kernel
c509a20 agent-ctl: Implement Linux OCI spec handling
42add7f agent: Disable seccomp feature on aarch64 temporarily
5dfedc2 docs: Add explanation about seccomp
45e7c2c static-checks: Add step for installing libseccomp
a3647e3 osbuilder: Set up libseccomp library
3be50ad agent: Add support for Seccomp
4280415 agent: Fix the configuration sample file
b0bc71f ci: test-kata-deploy: Get rid of slash-command-action action
309dae6 virtcontainers: check that both initrd and image are not set
a10cfff forwarder: Fix changing log level
6abccb9 forwarder: Drop privileges when using hybrid VSOCK
bf00b8d agent-ctl: improve the oci_to_grpc code
b67fa9e forwarder: Make explicit root check
e377578 forwarder: Fix docs socket path
5f30633 virtcontainers: delete duplicated notify in watchHypervisor function
5f5eca6 agent: do not return error but print it if task wait failed
d2a7b6f packaging/static-build: s390x fixes
6cc8000 cli: Show available guest protection in env output
2063b13 virtcontainers: Add func AvailableGuestProtections
a13e2f7 agent: Handle uevent remove actions
34273da runtime/device: Allow VFIO devices to be presented to guest as VFIO devices
68696e0 runtime: Add parameter to constrainGRPCSpec to control VFIO handling
d9e2e9e runtime: Rename constraintGRPCSpec to improve grammar
57ab408 runtime: Introduce "vfio_mode" config variable and annotation
730b9c4 agent/device: Create device nodes for VFIO devices
175f9b0 rustjail: Allow container devices in subdirectories
9891efc rustjail: Correct sanity checks on device path
d6b62c0 rustjail: Change mknod_dev() and bind_dev() to take relative device path
2680c0b rustjail: Provide useful context on device node creation errors
42b92b2 agent/device: Allow container devname to differ from the host
827a41f agent/device: Refactor update_spec_device_list()
8ceadcc agent/device: Sanity check guest IOMMU groups
ff59db7 agent/device: Add function to get IOMMU group for a PCI device
13b06a3 agent/device: Rebind VFIO devices to VFIO driver inside guest
e22bd78 agent/device: Add helper function for binding a guest device to a driver
b40eedc rustjail: Consistent coding style of LinuxDevice type
57c0f93 agent: fix race condition when test watcher
1a96b8b template: disable template unit test on arm
43b13a4 runtime: DefaultMaxVCPUs should not greater than defaultMaxQemuVCPUs
c59c367 runtime: current vcpu number should be limited
fa92251 runtime: kernel version with '+' as suffix panic in parse
52268d0 hypervisor: Expose the hypervisor itself
a72bed5 hypervisor: update tests based on createSandbox->CreateVM change
f434bcb hypervisor: createSandbox is CreateVM
76f1ce9 hypervisor: startSandbox is StartVM
fd24a69 hypervisor: waitSandbox is waitVM
a6385c8 hypervisor: stopSandbox is StopVM
f989078 hypervisor: resumeSandbox is ResumeVM
73b4f27 hypervisor: saveSandbox is SaveVM
7308610 hypervisor: pauseSandbox is nothing but PauseVM
8f78e1c hypervisor: The SandboxConsole is the VM's console
4d47aee hypervisor: Export generic interface methods
6baf258 hypervisor: Minimal exports of generic hypervisor internal fields
37fa453 osbuilder: Update QAT driver in Dockerfile
8030b6c virtcontainers: clh: Re-generate the client code
8296754 versions: Upgrade to Cloud Hypervisor v19.0
2b13944 docs: Fix outdated links
4f75ccb docs: use-cases: Update Intel SGX use case
4f018b5 runtime: delete useless src/runtime/cli/exit.go
7a80aeb docs: Moving from EOT to EOF
09a5e03 docs: Write tracing documentation
b625f62 runtime: delete cri containerd plugin from versions.yaml
24fff57 snap: make curl commands consistent
2b9f79c snap: add cloud-hypervisor and experimental kernel
273a1a9 runtime: optimize test code
76f16fd runtime: use containerd package instead of cri-containerd
6d55b1b docs: use containerd to replace cri-containerd
ed02bc9 packaging: add containerd to versions.yaml
50da26d osbuilder: Call detect_rust_version() right before install_rust.sh
b4fadc9 docs: Updating Developer Guide re qemu-img
b8e69ce versions: Add libseccomp and gperf version
17a8c5c runtime: Fix random failure for TestIoCopy
f34f67d osbuilder: Specify version when installing Rust
135a080 osbuilder: Pass CI env to container agent build
eb5dd76 osbuilder: Re-enable building the agent in Docker
bcffa26 tracing: Fix typo in "package" tag name
e61f5e2 runtime: Show socket path in kata-env output
5b3a349 trace-forwarder: Support Hybrid VSOCK
e42bc05 kata-deploy: add .dockerignore file
321be0f tracing: Remove trace mode and trace type
7d0b616 agent: Do not fail when trying to adding existing routes
3f95469 runtime: logging: Add variable for syslog tag
adc9e0b runtime: fix two bugs in rootless hypervisor
51cbe14 runtime: Add option "disable_seccomp" to config hypervisor.clh
98b7350 virtcontainers: clh: Enable the `seccomp` feature
46720c6 runtime: set tags for trace span
d789b42 package: assign proper value to redefined_string
4d7ddff utils: kata-manager: Update kata-manager.sh for new containerd config
f5172d1 cli: Fix outdated kata-runtime bash completion
d45c86d versions: Update CRI-O to its 1.22 release
c4a6426 versions: Update k8s & critools to v1.22
881b996 agent: Make wording of error message match CRI-O test suite

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-11-05 10:49:54 +00:00
James O. D. Hunt
d47484e7c1 logging: Always run crate tests
Ensure the tests in the local `logging` crate are run for all consumers
of it.

Additionally, add a new test which checks that output is generated by a
range of different log level `slog` macros. This is designed to ensure
debug level output is always available for the consumers of the
`logging` crate.

Fixes: #2969.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-04 17:26:52 +00:00
Wainer dos Santos Moschetta
a239a38f45 osbuilder: build image-builder image from Fedora 34
Currently the image-builder image is built from `fedora:latest` and
this is error-prone as any update of the base image can lead to
breakage. Instead let's create the image from Fedora 34, which is the
last known version to build fine.

Fixes #2960
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-11-03 13:07:42 -04:00
Fupan Li
1c81d7e0b6
Merge pull request #2915 from jodh-intel/agent-ctl-handle-hybrid-vsock
agent-ctl: Update for Hybrid VSOCK
2021-11-02 09:55:16 +08:00
GabyCT
e009b58c93
Merge pull request #2629 from Kvasscn/kata_dev_kbuild
package: assign proper value to redefined_string  in build-kernel.sh
2021-10-29 17:26:40 -05:00
Julio Montes
2751a13bbd
Merge pull request #2923 from ManaSugi/add-libseccomp-info
release: Upload libseccomp sources with notice to release page
2021-10-29 09:44:24 -05:00
Manabu Sugimoto
23496f94be release: Upload libseccomp sources with notice to release page
The `kata-agent` binaries inside the Kata Containers images provided
with release are statically linked with the GNU LGPL-2.1 licensed
libseccomp library by default.
Therefore, we attach the complete source code of the libseccomp
to the release page in order to comply with the LGPL-2.1 (6(a)).
In addition, we add the description about the libseccomp license
to the release page.

Fixes: #2922

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2021-10-29 12:38:14 +09:00
Bin Liu
00a20c840b
Merge pull request #2716 from liudalibj/linux_oci_spec
agent-ctl: Implement Linux OCI spec handling
2021-10-29 10:53:04 +08:00
James O. D. Hunt
7e401952f8 agent-ctl: Add stub for AddSwap API
Add a basic implementation for the `AddSwap` agent API call.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-10-28 09:22:52 +01:00
James O. D. Hunt
82de838e5f agent-ctl: Update for Hybrid VSOCK
Allow the `agent-ctl` tool to connect to a Hybrid VSOCK hypervisor such
as Cloud Hypervisor or Firecracker.

Fixes: #2914.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-10-28 09:22:35 +01:00
James O. D. Hunt
d1bcf105ff forwarder: Remove quotes from socket path in doc
Update the trace forwarder README to remove the quotes around the socket
path, which makes manipulating that path easier.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-10-28 09:20:38 +01:00
David Gibson
bdf4824145 tools/packaging: Add options for VFIO to guest kernel
Pull #2795 recently added support for a closer-to-OCI behaviour for
VFIO devices, in which they appear to the container as VFIO devices,
rather than being interpreted by the guest kernel.  However, in order
to use this, the Kata guest kernel needs to include the VFIO PCI
driver, along with dependencies like the Intel IOMMU driver.

The kernel as built by the scripts within Kata don't currently include
those, so this patch adds them.

fixes #2913

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-10-28 11:38:51 +11:00
Da Li Liu
c509a204f3 agent-ctl: Implement Linux OCI spec handling
- convert linux field from oci spec to grpc spec
- include all the fields below linux oci spec

Fixes: #2715

Signed-off-by: Da Li Liu <liudali@cn.ibm.com>
2021-10-27 13:41:57 +00:00
Manabu Sugimoto
a3647e3486 osbuilder: Set up libseccomp library
The osbuilder needs to set up libseccomp library to build the kata-agent
because the kata-agent supports seccomp currently.
The library is built from the sources to create a static library for musl libc.
In addition, environment variables for the libseccomp crate are set to
link the library statically.

Fixes: #1476

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2021-10-27 19:06:13 +09:00
Peng Tao
03a9411884
Merge pull request #2878 from eadamsintel/update-qat-dockerfile
This is to bump the OOT QAT 1.7 driver version to the latest version.…
2021-10-27 17:00:04 +08:00
Chelsea Mafrica
8f33e6f593
Merge pull request #2896 from Jakob-Naucke/static
packaging/static-build: s390x fixes
2021-10-26 11:53:34 -07:00
James O. D. Hunt
3120b489e3
Merge pull request #2687 from genjuro214/improve-oci-to-grpc
agent-ctl: improve the oci_to_grpc code
2021-10-26 13:00:02 +01:00
Lei Li
bf00b8df87 agent-ctl: improve the oci_to_grpc code
The oci_to_grpc function just handles part of oci fields,
and others are not copied from oci spec to grpc spec,
such as process.env, process.capabilities, mounts and so on.
Try to implement more handlings to convert thoses fields.

Fixes #2686

Signed-off-by: Lei Li <cdlleili@cn.ibm.com>
2021-10-26 16:54:28 +08:00
Jakob Naucke
d2a7b6ff4a
packaging/static-build: s390x fixes
- Install OpenSSL for key generation in kernel build
- Do not install libpmem
- Do not exclude `*/share/*/*.img` files in QEMU tarball since among
  them are boot loader files critical for IPLing.

Fixes: #2895
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-10-25 18:47:35 +02:00
Eric Adams
37fa453dd2 osbuilder: Update QAT driver in Dockerfile
This is to bump the OOT QAT 1.7 driver version to the
latest version. I dida test on my QAT enabled system and
everything functioned as expected.

Fixes: #2877

Signed-off-by: Eric Adams <eric.adams@intel.com>
2021-10-22 00:08:24 +00:00
James O. D. Hunt
9db56ffd85
Merge pull request #2863 from wainersm/osbuilder_dracut_rust
osbuilder: Call detect_rust_version() right before install_rust.sh
2021-10-19 11:48:39 +01:00
bin
ed02bc9041 packaging: add containerd to versions.yaml
This commit will add containerd to versions.yaml.

Please at now there are both containerd and cri-containerd
in the versions.yaml.

After updating of kata-containers/tests repo, the cri-containerd
should be removed.

Fixes: #2791

Signed-off-by: bin <bin@hyper.sh>
2021-10-19 09:38:56 +08:00
Wainer dos Santos Moschetta
50da26d3e6 osbuilder: Call detect_rust_version() right before install_rust.sh
When building with dracut method the build_rootfs_distro() is not called, in turn
detect_rust_version() isn't either, so the install_rust.sh script is gave a null
rust version. This changed the script to call detect_rust_version() right before
install_rust.sh.

Related to commit: f34f67d610
Fixes #2862
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2021-10-18 16:54:53 -04:00
James O. D. Hunt
f16a99603c
Merge pull request #2399 from Jakob-Naucke/container-osbuilder-respin
osbuilder: Re-enable building the agent in Docker
2021-10-18 12:06:37 +01:00
Jakob Naucke
f34f67d610
osbuilder: Specify version when installing Rust
and update the script in `ci/` accordingly.
When only parts of the Kata Containers repositories are checked out
(e.g. when building with Snap) and no Rust version is provided in
calling `install_rust.sh`, the scripts will attempt to clone the
appropriate repos to read the version, which will fail because the
directories already exist. Since we have read the version already, we
can just specify it.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-10-15 16:27:40 +02:00
Jakob Naucke
135a0802c5
osbuilder: Pass CI env to container agent build
The agent build inside a Docker or Podman container has been re-enabled,
but we have since introduced the `$CI` environment variable. Pass it to
avoid checking out the tests repo to main when there is a dependency.

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-10-15 16:27:39 +02:00
Jakob Naucke
eb5dd76e9d
osbuilder: Re-enable building the agent in Docker
or Podman. This is a partial revert of
76c18aa345. The rationale behind that
commit was the fact that the agent could not be built on Alpine, and
then this capability was removed altogether. The issue in Alpine has
since been resolved (see
https://github.com/kata-containers/osbuilder/issues/386). At the same
time, this ensures being able to run a glibc agent on hosts with distros
more recent than the osbuilder distro used (i.e. as of now, when you
build the agent on the host, and its glibc is newer than the one used in
the guest, the agent may encounter unresolved symbols).

Fixes #2398
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-10-15 16:27:37 +02:00
Fabiano Fidêncio
e42bc05c8a kata-deploy: add .dockerignore file
.dockerignore file is similar to .gitignore and serves the purpose to
simply ignore paths in the build context.

For now, let me just use it to fix the following problem:
```
docker build --build-arg KATA_ARTIFACTS=kata-static.tar.xz .
error checking context: 'no permission to read from
'(...)/local-build/build/firecracker/builddir/firecracker/(...)/crc64-1.0.0/.gitignore''.
```

Fixes: #2845

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2021-10-15 12:00:14 +02:00
Marcel Apfelbaum
9796babd92
Merge pull request #2311 from dgibson/mmconfig
packaging/kernel: Add CONFIG_PCI_MMCONFIG to x86 guest kernel configuration
2021-10-10 15:11:33 +03:00
Jakob Naucke
4152c45e4c
Merge pull request #2706 from yuanzhe-liu0/qemu_link
qemu: use GitLab repos instead of qemu.org
2021-10-08 12:03:55 +02:00
David Gibson
c4236cb2d1 packaging/kernel: Add CONFIG_PCI_MMCONFIG to x86 guest kernel configuration
The guest kernel configuration suggested for Kata, and which is used by the
CI didn't include CONFIG_PCI_MMCONFIG.  That's kind of weird, MMCONFIG is
the modern normal way of handling configuration cycles.

In addition, due to a complex set of interactions through the ACPI code,
disabling MMCONFIG means that SHPC hotplug doesn't work: the driver is
included in the guest kernel, but will fail to probe on PCI to PCI bridges,
meaning it won't actually be activated.

Enable MMCONFIG so that we suggest and testa more typical guest kernel
configuration.

fixes #2288

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-10-07 13:21:48 +11:00
Chelsea Mafrica
1f6a551570
Merge pull request #2755 from paleozogt/centos-ppc64le-gpg
#2676: fixing centos gpg key url for ppc64le
2021-10-05 09:37:58 -07:00
Greg Kurz
cd1064b16f packaging: Configure QEMU with --enable-pie
We explicitely set the Postion Independant Executlable (PIE) options
in the extra CFLAGS and LDFLAGS that are passed to the QEMU configure
script for all archs. This means that these options are used pretty
much everywhere, including when building the sample plugins under the
test directory. These cannot be linked with -pie and break the build,
as experienced recently on ARM (see PR #2732).

This only broke on ARM because other archs are configured with
--disable-tcg : this disables plugins which are built by default
otherwise.

The --enable-pie option is all that is needed. The QEMU build system
knows which binaries should be created as PIE, e.g. the important
bits like QEMU and virtiofsd, and which ones should not, e.g. the
sample plugins that aren't used in production.

Rely on --enable-pie only, for all archs. This allows to drop the
workaround that was put in place in PR #2732.

Fixes: #2757
Signed-off-by: Greg Kurz <groug@kaod.org>
2021-09-30 11:17:41 +02:00
Aaron Simmons
80f6b97710 osbuilder: fixing centos gpg key url for ppc64le
The centos ppc64le gpg key at mirror.centos.org doesn't exist (link rot?).
Replacing it with url from CentOS/sig-core-AltArch on github.

Fixes: #2676

Signed-off-by: Aaron Simmons <paleozogt@gmail.com>
2021-09-29 09:20:51 -06:00
Chelsea Mafrica
20f4c252b8
Merge pull request #2519 from jcvenegas/kernel-experimental-5.13.10
kernel: Enable SGX in experimental kernel.
2021-09-28 11:00:46 -07:00
Carlos Venegas
dffc50928a kernel: Enable SGX in experimental kernel.
Enable Intel SGX support in experimental kernel.

Fixes: #2518

Signed-off-by: Carlos Venegas <jose.carlos.venegas.munoz@intel.com>
2021-09-27 03:39:50 +00:00
Carlos Venegas
ff6a677d16 kernel-build: Enable multiple config types.
Optional build types are common for early adoption.
Lets add a flag to build and optional config.

e.g.
kernel-build.sh -b experimental

In the future instead of add more flags just add a new build type.

Signed-off-by: Carlos Venegas <jose.carlos.venegas.munoz@intel.com>
2021-09-27 03:39:50 +00:00
Carlos Venegas
90046964ef experimental-kernel: bump 5.13.10
Upgrade Linux kernel to latest stable release.

Signed-off-by: Carlos Venegas <jose.carlos.venegas.munoz@intel.com>
2021-09-27 03:39:50 +00:00
Carlos Venegas
1fbb73041b build: kata-deploy kernel experimental
Allow build experimental kernel from kata-deploy.

Signed-off-by: Carlos Venegas <jose.carlos.venegas.munoz@intel.com>
2021-09-27 02:56:59 +00:00
Jianyong Wu
6d94957a14 kernel: reduce alignment size of memory hotplug to 128M
After 5.11-rc4, memory hotplug alignment size is reduced to 128M for 4K
page.
It works better for memory hotplug and nvdimm plug in kata on arm.
without this patch, memory hotplug will fail for the current memory
hotplug alignment is 1G but the nvdimm size align with 128M in kata.
After port it here, we can avoid a fix in qemu side.

Note: if you change the page size to other size than 4K, memory hotplug
will has no effect.

Fixes: #2707
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-09-26 15:33:33 +08:00
Jianyong Wu
48090f624a qemu: disable plug on arm64 when pie is added
For qemu 6.1.0 build on arm64, compile error occurs when "-pie" is added
 to ldflag.
tests/plugins/empty.c won't be linked as a sysmbol is missing.
I consider there maybe a bug.
Before figure it out, we should disable plugins for qemu 6.1.0 on arm64.

Fixes: #2707
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-09-26 15:33:33 +08:00
Chelsea Mafrica
e987632deb
Merge pull request #2693 from Amulyam24/qemu-build
packaging: fix qemu build on ppc64le
2021-09-23 10:31:34 -07:00
Yuanzhe Liu
80463b445a qemu: use GitLab repos instead of qemu.org
arm using qemu 5.1.0, thus is affected by the wired submodules
link.

Fixes: #2705
Signed-off-by: Yuanzhe Liu <yuanzheliu09@gmail.com>
2021-09-23 12:07:44 +00:00
Samuel Ortiz
3276f3b5b6
Merge pull request #2453 from fidencio/wip/kata-deploy-use-stable-and-latest-tags
kata-deploy: Also provide "stable" & "latest" tags
2021-09-23 13:54:01 +02:00
Amulya Meka
439e5ac3b0 packaging: fix qemu build on ppc64le
Since the qemu upgrade to v6.1.0, the build fails
with a linking issue. Adding --disable-tcg to fix
it.

Fixes: #2710

Signed-off-by: Amulya Meka <amulmek1@in.ibm.com>
2021-09-23 06:27:15 +00:00
Jianyong Wu
0ca8c27241 qemu: add v5.1.0 dir under tag_patches
A related dir is needed when apply qemu patch using script. As qemu 5.1
is used for arm, a dir of "v5.1.0" is needed under tag_patches.

Fixes: #2696
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-09-22 18:07:24 +08:00
Fabiano Fidêncio
3bdcfaa658 kata-deploy: Add more info about the stable tag
Let's make it as clear as possible for the user that if they go for a
tagged version of kata-deploy, eg, 2.2.1, they'll have the kata runtime
2.2.1 deployed on their cluster.

Suggested-by: Eric Adams <eric.adams@intel.com>
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-09-21 23:13:45 +02:00
Fabiano Fidêncio
41c590fa0a kata-deploy: Improve README
Let's add more instructions in the README in order to make clear to the
reader what they can do to check whether kata-deploy is ready, or
whether they have to wait till proceeding with the next instruction.

Suggested-by: Eric Adams <eric.adams@intel.com>
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-09-21 23:13:45 +02:00
Fabiano Fidêncio
debf3c9fe9 kata-deploy: Remove qemu-virtiofs runtime class
There's only one QEMU runtime class deployed as part of kata-deploy, and
that includes virtiofs support (which is the default for quite some time
already).  Knowing this, let's just remove the `qemu-virtiofs` runtime
class definition.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-09-21 22:48:04 +02:00
Fabiano Fidêncio
43a72d76e2 release: update the kata-deploy yaml files accordingly
Let's teach our `update-repository-version.sh` script to properly update
the kata-deploy tags on both kata-deploy and kata-cleanup yaml files.

The 3 scenarios that we're dealing with, based on which branch we're
targetting, are:
```
 1) [main] ------> [main]        NO-OP
   "alpha0"       "alpha1"

                   +----------------+----------------+
                   |      from      |       to       |
  -----------------+----------------+----------------+
  kata-deploy      | "latest"       | "latest"       |
  -----------------+----------------+----------------+
  kata-deploy-base | "stable        | "stable"       |
  -----------------+----------------+----------------+

 2) [main] ------> [stable] Update kata-deploy and
   "alpha2"         "rc0"   get rid of kata-deploy-base

                   +----------------+----------------+
                   |      from      |       to       |
  -----------------+----------------+----------------+
  kata-deploy      | "latest"       | "rc0"          |
  -----------------+----------------+----------------+
  kata-deploy-base | "stable"       | REMOVED        |
  -----------------+----------------+----------------+

 3) [stable] ------> [stable]    Update kata-deploy
    "x.y.z"         "x.y.(z+1)"

                   +----------------+----------------+
                   |      from      |       to       |
  -----------------+----------------+----------------+
  kata-deploy      | "x.y.z"        | "x.y.(z+1)"    |
  -----------------+----------------+----------------+
  kata-deploy-base | NON-EXISTENT   | NON-EXISTENT   |
  -----------------+----------------+----------------+
```

And we can easily cover those 3 cases only with the information about
the "${target_branch}" and the "${new_version}", where:
* case 1) if "${target_branch}" is "main" *and* "${new_version}"
  contains "alpha", do nothing
* case 2) if "${target_branch}" is "main" *and* "${new_version}"
  contains "rc":
  * change the kata-deploy & kata-cleanup tags from "latest" to
    "${new_version}".
  * delete the kata-deploy-stable & kata-cleanup-stable files.
* case 3) if the "${target_branch}" contains "stable":
  * change the kata-deploy & kata-cleanup tags from "${current_version}"
    to "${new_version}".

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-09-21 22:48:04 +02:00
Fabiano Fidêncio
ea9b2f9c92 kata-deploy: Add "stable" info to the README
Similar to the instructions we have for the "latest" images, let's also
add instructions about the "stable" images.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-09-21 22:48:04 +02:00
Fabiano Fidêncio
e541105680 kata-deploy: Update the README
Let's just point to our repo URLs rather than assume users using
kata-deploy will have our repo cloned.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-09-21 22:48:04 +02:00
Fabiano Fidêncio
9acf4e5d32 kata-deploy: Add stable yaml files
This is **not** the nicest patch of my career, and I know it adds code
duplication.  However, I've decided to take this approach in order to
have easier / better instructions for users who're consuming
kata-deploy.

Having both stable & latest yaml on `main` will let us point to just one
place, without having to update the instructions.

I know, would be better to have those generated from a .in file,
wouldn't it?  For sure, but then we'd lose the ability to just point to
those files from kata-deploy pages (either on dockerhub or quay.io).

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-09-21 22:48:04 +02:00
Fabiano Fidêncio
a86babe0d0 kata-deploy: Point to the latest release
Instead of point to a specific release number, let's point to the
`latest` tag on the main branch.

There's still some work needed in order to point to the `stable` tag on
the stable-x.y branches, as this is something that should be done
automagically as part of the release process.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-09-21 22:48:04 +02:00
David Gibson
e7deee948a
Merge pull request #2502 from dgibson/qemu-6.1
Update Kata to use qemu-6.1
2021-09-15 11:06:14 +10:00
zhanghj
d789b42937 package: assign proper value to redefined_string
Fixes: #2624

Signed-off-by: zhanghj <zhanghj.lc@inspur.com>
2021-09-14 14:38:36 +08:00
Samuel Ortiz
057eb80ac9
Merge pull request #2596 from jongwu/qemu_mak
qemu: remove default config for arm64.
2021-09-13 11:23:35 +02:00
David Gibson
25670d3058 packaging/qemu: Update qemu-exerimental version to v6.1.0
This brings it back into line with the normal qemu version.  We refer to
v6.1.0 by full SHA in versions.yaml, rather than the tag, so that
apply_patches.sh sees it as different and applies the virtiofs DAX patches
which is what the experimental version is actually about having.

The virtiofs DAX patches themselves are updated to the version from
https://gitlab.com/virtio-fs/qemu, virtio-fs-dev branch as of commit
3620cb0a.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-11 16:43:26 +10:00
David Gibson
041a513f80 versions: Update qemu to v6.1.0
We need qemu-6.1 for ACPI PCI hotplug support for the q35 machine.  At the
moment qemu will use SHPC hotplug under the PCIe to PCI bridge on q35.
SHPC is too slow to use for our purposes (it requires a 5s delay).

Update the qemu version to v6.1.0.  This leaves the experimental version
*older* than the normal version, but we'll fix that up later.

We also need to tweak the snapcraft.yaml, since the location for configs
has changed in the new qemu version.

fixes #1691

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-11 16:43:22 +10:00
David Gibson
81de2d476b packaging: Correct error message in apply_patches.sh
If the script doesn't find a patches directory it expects, it gives an
error saying to create a dummy 'no_patches' file if you really don't want
any patches applied for that version.

But actual practice in the tree is to call the dummy file 'no_patches.txt'
rather than simply 'no_patches'.  Correct the message to match existing
practice.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-09-10 11:19:10 +10:00
Carlos Venegas
230eae3ff3
Merge pull request #2417 from jcvenegas/docker-build-fixes
kata-tarball: Build and test fixes
2021-09-09 14:14:26 -05:00
Carlos Venegas
78d99f5129 kata-deploy: Make verbose single builds
If a binary tarball for a single component is done,
the logs will be shown in stdout.

e.g.

make kernel-tarball

To build all a the same time still store logs in files.

make kata-tarball

Signed-off-by: Carlos Venegas <jose.carlos.venegas.munoz@intel.com>
2021-09-09 14:32:08 +00:00
Carlos Venegas
59486b855a kata-deploy: Add tarball suffix to makefile targets
Now that local-build kata-deploy makefile is inlucded in toplevel
makefile, lets use the suffix `-tarball` to avoid name collitions
and identify the tarball releted targets.

Signed-off-by: Carlos Venegas <jose.carlos.venegas.munoz@intel.com>
2021-09-09 14:32:08 +00:00
Binbin Zhang
924a68d08d osbuilder: Change to "=" operator to make script more portable
zsh doesn't support "==" as equal comparison operator, so
replace "==" with "=" to make the script more portable

Fixes: #2584

Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
2021-09-08 16:57:34 +08:00
Jianyong Wu
1fff9be707 qemu: remove default config for arm64.
The current default config in qemu for arm64 doesn't suit for qemu
version 5.1+, so remove them here.

Fixes: #2595
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-09-08 16:25:22 +08:00
Bin Liu
3c9170ea0d
Merge pull request #2561 from rapiz1/patch-4
osbuilder: fix inconsistent calculation of fs size
2021-09-04 15:14:07 +08:00
James O. D. Hunt
f3a1bf3b45
Merge pull request #2552 from bergwolf/license
license: drop redundent license files
2021-09-02 14:31:18 +01:00
Yujia Qiao
bfcee91164 osbuilder: fix inconsistent calculation of fs size
This patch fixes inconsistent calculations of the rootfs size.
For `du` and `df`, `-B 1MB` is different from `-BM`. The
former is the power of 1000, and the latter is the power of
1024. So comparing them doesn't make sense. The bug may result
in a larger image than needed.

Fixes: #2560

Signed-off-by: Yujia Qiao <rapiz3142@gmail.com>
2021-09-02 16:00:29 +08:00
Peng Tao
256c3b2747 license: drop redundent license files
There is no need to keep multiple copies of the license file in
different directory. We can just use the top level one for the project.

Fixes: #2553
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2021-09-01 15:10:04 +08:00
Fabiano Fidêncio
18c95b9ab1 release: Kata Containers 2.3.0-alpha0
- tracing: Change runtime tracing tags to vars
- shimv2: add logging to shimv2 api calls
- drop qemu-lite support
- runtime: delete types or const that no longer needed
- runtime: Optimize the way slice created
- virtcontainers: simplify tests
- virtcontainers: clh: Upgrade to the openapi-generator v5.2.1
- build_image: Fix error soft link about initrd.img
- ci: Temporarily skip agent shutdown test on s390x
- Fix version parsing for firecracker version 0.25 and over
- Osbuilder fixes
- docs: update the GoDoc url from runtime project to kata-containers/sr…
- docs: update `how-to` README file for Firecracker config
- ci/openshift-ci: Pull centos from registry.centos.org
- docs: update containerd CRI plugin url

2250360b docs: remove mentioning of qemu-lite
a9de761d runtime: drop qemu-lite support
8ae3edbc runtime: fix default hypervisor path
0c7789fa runtime: Add container field to logs
72e3538e shimv2: add information to method comment
8dadca9c shimv2: add logging to shimv2 api calls
a99fcc3a virtcontainers: simplify tests
39ffd8ee runtime: delete types or const that no longer needed
ff37f5c7 runtime: Optimize the way slice created
8f0f949a tracing: Move dynamically added attributes to Trace()
932ee41b virtcontainers: clh: Workaround incorrect default values
bff38e4f virtcontainers: clh: Fix the unit test
d967d3cb virtcontainers: clh: Use constructors to ensure proper default value
87de26bd tracing: Modify Trace() to accept multiple tag maps
8058e972 tracing: Change runtime tracing tags to vars
a6a2e525 virtcontainers: clh: Migrate to use the updated client APIs
9de1129b osbuilder: Fix rootfs-builder when running in VMs
65a1e131 osbuilder: Allow running the tool several times
a4214738 osbuilder: Fix Makefile
b8717f35 ci: Temporarily skip agent shutdown test on s390x
938981be build_image: Fix error soft link about initrd.img
2304f935 docs: update the GoDoc url from kata 1.x to 2.x
2a614577 docs: update `how-to` README file for Firecracker config
486baba7 docs: update containerd CRI plugin url
46eb07e1 virtcontainers: clh: Re-generate the client code
80fba4d6 virtcontainers: clh: Upgrade to the openapi-generator v5.2.1
8594f80c ci/openshift-ci: Pull centos from registry.centos.org
87bbae1b fc: fix version parsing for fc >= 0.25

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-08-31 18:43:09 +02:00
GabyCT
d490704133
Merge pull request #2504 from Bevisy/main-2503
build_image: Fix error soft link about initrd.img
2021-08-26 13:26:52 -05:00
Marcel Apfelbaum
9de1129bf7 osbuilder: Fix rootfs-builder when running in VMs
The script runs apt sync at some point which scans all possible fds
in order to close them. The operation is incredibly slow on VMs
and may lead to build timeouts.

Fix it by limiting the container runtime fds to a sane limit.

Fixes: #2510

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2021-08-25 19:58:21 +00:00
Marcel Apfelbaum
65a1e13195 osbuilder: Allow running the tool several times
Once the ${ROOTFS_DIR} is created, the tool can't run the second
time since the directory is populated and the debootstrap tool
will fail.

Fix by deleting the contents of ${ROOTFS_DIR} if the directory exists.
Note that running make clean will also allow the re-run, it
is only an optimization for some cases the build fails in the middle.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2021-08-25 19:49:14 +00:00
Marcel Apfelbaum
a4214738b9 osbuilder: Fix Makefile
Let the DISTRO variable to be set from outside,
allowing "sudo -E DISTRO=<ANY> make clean" to delete the correct files.

Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
2021-08-25 19:45:53 +00:00
Binbin Zhang
938981be1d build_image: Fix error soft link about initrd.img
fix error soft link about initrd.img

Fixes #2503

Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
2021-08-25 16:00:55 +08:00
Fabiano Fidêncio
aaf37d72b2 release: Kata Containers 2.2.0-rc0
- use CRI in kata-monitor
- config: Enable jailer by default when using firecracker
- workflows: Actually push the release to quay.io
- docs: update general wording for installation documentation
- Cleanup kernel packaging
- tracing: Return context in runHooks() span creation
- osbuilder: Document no Alpine support on s390x
- osbuilder: Upgrade Ubuntu guest to 20.04
- agent: watcher / inotify stability fixes
- enable snap build for arm64
- agent: Fix cargo 1.54 clippy warning
- osbuilder: Drop Go agent support
- kernel: PTP_KVM support for arm/arm64 in Kata
- docs: update the docs project url from kata 1.x to 2.x
- clh: correct cloud-hypervisor installation on non-x86
- virtcontainers: fc: properly remove jailed block device
- CI: Call agent shutdown test
- kata deploy: always update the base image
- docs: Remove kata-proxy and invalid script reference
- workflows: Actually login to quay.io
- kata-deploy: Update our content to use / point to quay.io/kata-containers rather than katadocker
- agent: Create the process CWD when it does not exist
- Update Kata to allow it to use Qemu 6.1
- osbuilder/dracut: Add missing libraries
- osbuilder: pass env OS_VERSION
- tools: shorten directory path
- virtcontainers: clh: Do not use the default HTTP client
- docs: update kata deploy README doc to add cloud-hypervisor test command
- Container: Add initConfigResourcesMemory and call it in newContainer
- qemu/arm: remove nvdimm/"ReadOnly" option on arm64
- Fix issue container start fail if io.katacontainers.container.resource.swap_in_bytes and memory_limit_in_bytes are not set
- docs: Add tracing proposals doc
- docs: Remove table of contents
- static-checks: Check for the `force-skip-ci` label on each step
- docs: update the kata release url in the kata deploy document
- kata-deploy: Allow build  kata-deploy  tarball from HEAD
- mod: unify runc and containerd dependencies
- how-to-use-virtio-mem-with-kata.md: Remove undefined ${REPORT_DIR}
- ci: Run static checks when PRs are updated
- docs: update url for log parser in how-to-import-kata-logs-with-fluen…
- versions: Upgrade to Cloud Hypervisor v17.0
- snap: Substitute image configuration with initrd
- docs: Update url for log parser in Developer guide
- mount: fix the issue of missing check file exists
- build(deps): bump github.com/containerd/containerd from 1.5.2 to 1.5.4 in /src/runtime
- docs: Update experimental documentation
- snap: do not export agent version
- Upgrade runc to 1.0.1
- runtime: read-only NVDIMM
- osbuilder/scripts: add support to yq version 4 and above
- osbuilder: update centos arm rootfs image config 'GPG_KEY_ARCH_URL'
- monitor: mv the monitor socket into sbs directory
- fix govet fieldalignment
- docs: added a glossary to support SEO tactics
- ci: expand $CI to nothing
- Add swap support
- snap: fixed snap aarch64 qemu patches dir in snapcraft.yaml file
- agent: clear MsFlags if the option has clear flag set
- snap: Remove QEMU before clone
- docs: fix minikube installation guide runtimeclasses error
- docs: fixed kata-deploy path for kata logs with fluentd doc
- agent/agent-ctl: update tokio to 1.8.1
- ci: set -o nounset
- static-checks: Add a make target to run static-checks locally
- virtiofsd: fix the issue of missing stop virtiofsd
- docs: Update containerd configuration format
- osbuilder: Skip installing golang for building rootfs
- agent-ctl: Use a common Makefile style like other components
- vsock-exporter: switch to tokio runtime
- config: Fix description for OCI hooks
- shimv2: fix the issue of kata-runtime exec failed

7a5ffd4a config: Enable jailer by default when using firecracker
2cb7b513 docs: update general wording for installation documentation
76f4588f workflows: Actually push the release to quay.io
b980c62f packaging/kernel: Update kernel build doc
99e9a6ad packaging/kernel: Update versions.yaml kernel urls
c23ffef4 packaging/kernel: Remove old Jenkins pipeline
9586d482 tracing: Return context in runHooks() span creation
6a6dee7c osbuilder: Document no Alpine support on s390x
71f304ce agent: watcher: cleanup mount if needed when container is removed
f1a505db agent: Temporarily allow unknown linters
961aaff0 agent: watcher: fixes to make more robust
7effbdeb osbuilder: Upgrade Ubuntu guest to 20.04
99ab91df docs: update the docs project url from kata 1.x to 2.x
4fe23b19 kernel: PTP_KVM support for arm/arm64 in Kata
f981fc64 clh: correct cloud-hypervisor installation
f87cee9d kata-deploy: Rely directly on a centos:7 image
6871aeaa snap: enable snap build for arm64
15e0a3c8 kata-deploy: Remove unneeded yum cached files
d01aebeb kata-deploy: Ensure the system is up-to-date
77160e59 workflows: Actually login to quay.io
b9e03a1c docs: update the image repository to quay.io
f47cad3d tools: Update the image repository to quay.io
9fa1febf workflows: Also push the image to quay.io
233b53c0 agent: Fix cargo 1.54 clippy warning
2d8386ea kata-monitor: add few unit tests
8714a350 kata-monitor: make code to identify kata pods simpler
68a6f011 kata-monitor: drop the runtime info from the sandbox cache
97dcc5f7 kata-monitor: drop getMonitorAddress()
0b03d97d vendor: update vendors for kata-monitor
c2f03e89 kata-monitor: talk to the container engine via the CRI
c867d1e0 osbuilder: Drop Go agent support
1d25d7d4 docs: Remove kata-proxy and binaries reference
64dd35ba virtcontainers: fc: properly remove jailed block device
b8133a18 osbuilder/dracut: Add missing libraries
831c2fee packaging: Remove reference to sheepdog driver
2e28b714 packaging: Drop support for qemu < 5.0
d5f85698 vendor: Update govmm
31650956 runtime/qemu: Use explicit "on" for kernel_irqchip parameter
a72b0811 osbuilder: pass env OS_VERSION
d007bb85 kata-deploy: shorten directory path
e6408fe6 Container: Add initConfigResourcesMemory and call it in newContainer
49083bfa agent: Create the process CWD when it does not exist
ee90affc newContainer: Initialize c.config.Resources.Memory if it is nil
767a41ce updateResources: Log result after calculateSandboxMemory
760ec4e5 virtcontainers: clh: Do not use the default HTTP client
3fe6695b static-checks: Check for the `force-skip-ci` label on each step
7df56301 CI: Call agent shutdown test
57b696a5 docs: Removed mention of 1.x
4f0726bc docs: Remove table of contents
f186c5e2 docs: Fix invalid URLs
7c610a6f docs: Fix shell code
80afba15 docs: update kata deploy README doc to add cloud-hypervisor test command
5a0d3c4f docs: update the kata release url in the kata deploy document
9514dda5 mod: unity containerd dependency
6ffe37b9 mod: unify runc dependency
5b514177 docs: Add tracing proposals doc
b53e8405 how-to-use-virtio-mem-with-kata.md: Remove undefined ${REPORT_DIR}
5957bc7d ci: Run static checks when PRs are updated
81e6bf6f kata-deploy: Split shimv2 build in a separate container.
d46ae324 kernel: build: Add container build
b789a935 actions: release: Use new kata-deploy scripts.
85987c6d kata-deploy: Add Makefile
b9d2eea3 kata-deploy: Add script to merge kata tarballs.
4895747f Rootfs: Add curl to alpine rootfs builder.
fc90bb53 Actions: Add new workflow to create static tarballs
bbb06c49 actions: Remove scripts from actions directory.
2f9859ab build: Reuse firecracker directory on builds.
3533a5b6 Packaging: stop using GOPATH for yq.
0c5ded4b kata-deploy: build kata only with docker in host
2ec31093 docs: update url for log parser in how-to-import-kata-logs-with-fluentd.md
cc0bb9ae versions: Upgrade to Cloud Hypervisor v17.0
8e9ffe6f snap: Substitute image configuration with initrd
8b15eafa docs: Update url for log parser in Developer guide
77604de8 qemu/arm: remove nvdimm/"ReadOnly" option on arm64
4fbae549 docs: Update experimental documentation
07f7ad9d build(deps): bump github.com/containerd/containerd in /src/runtime
9c0b8a7f snap: do not export agent version
3727caf7 versions: Update runc to 1.0.1
116c29c8 cgroups: manager's Set() now takes Resources as its parameter
c0f801c0 rootless: RunningInUserNS() is now part of userns namespace
b5293c52 runtime: update runc dependency to 1.0.1
2859600a runtime: virtcontainers: make rootfs image read-only
8befb1f3 kata-deploy: Refactor builder options.
7125f5d8 image-builder: Allow build image and initrd independently.
0f8c0dbc osbuilder/scripts: add support to yq version 4 and above
070590fb vendor: update govmm
b4c45df8 runtime: tools/packaging/cmd/kata-pkgsync: fix govet fieldalignment
aec53090 runtime: virtcontainers/utils: fix govet fieldalignment
1e4f7faa runtime: virtcontainers/types: fix govet fieldalignment
bb9495c0 runtime: virtcontainers/pkg: fix govet fieldalignment
80ab91ac runtime: virtcontainers/persist: fix govet fieldalignment
54bdd018 runtime: virtcontainers/factory: fix govet fieldalignment
dd58de36 runtime: virtcontainers/device: fix govet fieldalignment
47d95dc1 runtime: virtcontainers: fix govet fieldalignment
8ca7a7c5 runtime: netmon: fix govet fieldalignment
31de8eb7 runtime: pkg: fix govet fieldalignment
2b80091e runtime: containerd-shim-v2: fix govet fieldalignment
0dc59df6 runtime: cli: fix govet fieldalignment
c1042523 ci: expand $CI to nothing
add480ed monitor: mv the monitor socket into sbs directory
f7c6f170 docs: added a glossary to support SEO tactics
a8649acf snap: fixed snap aarch64 qemu patches dir in snapcraft.yaml file
38826194 osbuilder: update centos arm rootfs image config 'GPG_KEY_ARCH_URL'
c5fdc0db docs: fix minikube installation guide runtimeclasses error
f2ef25c6 docs: fixed kata-deploy path for kata logs with fluentd doc
cb6b7667 runtime: Add option "enable_guest_swap" to config hypervisor.qemu
a733f537 runtime: newContainer: Handle the annotations of SWAP
2c835b60 ContainerConfig: Set ocispec.Annotations to containerConfig.Annotations
243d4b86 runtime: Sandbox: Add addSwap and removeSwap
e1b91986 runtime: Update golang proto code for AddSwap
4f066db8 agent: agent.proto: Add AddSwap
4f23b8cd ci: set -o nounset
35cbc93d agent: clear MsFlags if the option has clear flag set
ff87da72 config: Fix description for OCI hooks
8e0daf67 shimv2: fix the issue of kata-runtime exec failed
b12b21f3 osbuilder: Skip installing golang for building rootfs
558f1be6 snap: Remove QEMU before clone
5371b921 mount: fix the issue of missing check file exists
27b299b2 agent-ctl: Use a common Makefile style like other components
05084699 agent-ctl: bump to latest tokio
acf69328 agent: update tokio to 1.8.1
dcd29867 static-checks: Call the static-checks make target
afd97850 makefile: Add static-checks target
34828df9 virtiofsd: fix the issue of missing stop virtiofsd
73d3798c vsock-exporter: switch to tokio runtime
7960689e tracing: replace SimpleSpanProcessor with BatchSpanProcessor
e887b39e docs: Update containerd configuration format

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-08-20 14:03:43 +02:00
Dan Middleton
b980c62f43 packaging/kernel: Update kernel build doc
Clarify dependencies, correct typos, and fill in some gaps.

Fixes: #2422

Signed-off-by: Dan Middleton <dan.middleton@intel.com>
2021-08-12 12:14:58 -05:00
Dan Middleton
c23ffef4eb packaging/kernel: Remove old Jenkins pipeline
This Jenkins pipeline is no longer used and it references now archived
repos.

Fixes: #2422

Signed-off-by: Dan Middleton <dan.middleton@intel.com>
2021-08-12 12:14:58 -05:00
Jakob Naucke
6a6dee7cc8
osbuilder: Document no Alpine support on s390x
Alpine used to work as guest under 1.x, but because there is no musl
target for Rust on s390x, Alpine will not work for 2.x. Document this.

Fixes: #2436
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-08-12 11:14:25 +02:00
Jakob Naucke
7effbdebcb
osbuilder: Upgrade Ubuntu guest to 20.04
- no need to create `/usr/lib/systemd/systemd` link any more
- install `chrony` as extra package and install extra packages in chroot
  rather than `debootstrap`, because `chrony` provides `time-daemon`,
  which under 20.04 is provided by `systemd-timesyncd`, which is
  required by `systemd`, and `debootstrap`'s conflict resolvement can't
  handle this, but `apt`'s can.

Fixes: #2147
Depends-on: github.com/kata-containers/tests#3636
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-08-10 16:31:21 +02:00
GabyCT
1ab55e5afd
Merge pull request #2397 from dgibson/no-go-agent
osbuilder: Drop Go agent support
2021-08-10 09:13:00 -05:00
GabyCT
e287708435
Merge pull request #2246 from damon-kwok/main
kernel: PTP_KVM support for arm/arm64 in Kata
2021-08-10 09:11:48 -05:00
Jakob Naucke
f152284f1b
Merge pull request #2411 from jongwu/clh
clh: correct cloud-hypervisor installation on non-x86
2021-08-10 10:39:57 +02:00
Damon Kwok
4fe23b190f kernel: PTP_KVM support for arm/arm64 in Kata
This work patched the 4.19, 5.4 and 5.10 kernels, and now ptp_kvm can work
correctly when the host and guest use different kernel versions..

Fixes: #2123

Signed-off-by: Damon Kwok <damon-kwok@outlook.com>
2021-08-10 11:04:28 +08:00
Jianyong Wu
f981fc6456 clh: correct cloud-hypervisor installation
Currently, there is cloud hypervisor binary released only for x86, thus
we must build from source code when install cloud hypervisor on arm64.

Fixes: #2410
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-08-09 15:56:28 +08:00
Fabiano Fidêncio
f87cee9d11 kata-deploy: Rely directly on a centos:7 image
Instead of relying on a centos/docker image, present only on dockerhub,
let's rely on the centos:7 image from the centos registry, and apply
the same modifications applied when generating the centos/systemd image.

The main reason for doing this is avoiding to update an image from 3
years ago, making the delta of the packages updated smaller.

If you're curious why we keep using CentOS 7 though, the reason is
because CentOS 8, and UBI images have a different systemd configuration
that works quite well when mounting the image using podman, but systemd
can't connect dbus when running on environments like AKS or even
minikube.  So, in order to be as compatible as possible, let's keep
using the CentOS 7 image for now, at least till we find a suitable
substitute for that.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-08-06 13:22:45 +02:00
Fabiano Fidêncio
15e0a3c8f0 kata-deploy: Remove unneeded yum cached files
Let's just remove the cached failes as those are not needed for anything
we do when using this image.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-08-06 11:01:58 +02:00
Fabiano Fidêncio
d01aebebae kata-deploy: Ensure the system is up-to-date
In order to avoid providing an image with security issues, let's ensure
we run `yum update` as part of our image build process.  This is needed
as even with the latest CentOS images there may be fix provided by some
CVE that's already part of the updates but not yet part of the image.

In our case, it's even more needed as the `centos/systemd` image has not
been updated for 3 years or so and those are the vulnerabilities found
in the current images:
https://quay.io/repository/kata-containers/kata-deploy?tab=tags

Fixes: #2303

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-08-06 11:01:58 +02:00
Fabiano Fidêncio
f47cad3d95 tools: Update the image repository to quay.io
This can help our users to **not** hit the pull limitation imposed by
dockerhub.

Fixes: #2306

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-08-05 22:53:20 +02:00
David Gibson
c867d1e069 osbuilder: Drop Go agent support
With Kata 1.x EOL, the Go agent is no more.  So, remove support for it from
the osbuilder scripts.  This removes the RUST_AGENT variable, treating it
as always true.

fixes #2396

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-08-05 16:10:10 +10:00
Julio Montes
03325f0612
Merge pull request #2382 from dgibson/prep-qemu-6.1
Update Kata to allow it to use Qemu 6.1
2021-08-04 09:16:06 -05:00
Jakob Naucke
b8133a188c
osbuilder/dracut: Add missing libraries
When the guest is built using dracut and the agent uses glibc (esp.
ppc64le/s390x), libraries might be missing. In my case, it was
`libutil.so`, but more can be added easily. Add a script to configure
`install_items` for dracut w.r.t. `ldd` of the agent.

Fixes: #2384
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-08-04 13:50:42 +02:00
Jakob Naucke
d473967120
Merge pull request #2379 from Jakob-Naucke/env-os-version
osbuilder: pass env OS_VERSION
2021-08-04 10:40:17 +02:00
David Gibson
831c2feead packaging: Remove reference to sheepdog driver
The QEMU sheepdog driver was deprecated in 5.2.0 and removed entirely in
6.1.  Explicitly disabling, therefore is unnecessary from 5.2.0 and will
give an error from 6.1.

fixes #2337

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-08-04 15:04:36 +10:00
David Gibson
2e28b71473 packaging: Drop support for qemu < 5.0
We only test qemu 5.2 in the CI (5.1 for ARM), and I believe we already
have some subtle dependencies that will stop things working on older qemu
versions.

We just updated govmm to a version that explicitly only works with qemu 5.0
and later, so we can drop stale checks for older qemu versions.  More
specifically that means we can drop patches for older qemu versions, and
remove checks for older qemu versions from configure-hypervisor.sh.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-08-04 15:04:36 +10:00
Carlos Venegas
b24ee4b11e
Merge pull request #2369 from converge/issue_2354
tools: shorten directory path
2021-08-03 15:32:56 -05:00
Jakob Naucke
a72b08117f
osbuilder: pass env OS_VERSION
With lines like
0a2e2c6038/tools/osbuilder/rootfs-builder/fedora/config.sh (L8)
we imply that one can set another OS_VERSION and it will get picked up.
This is not the case when building inside Docker/Podman because the
variable is not passed to the container, which can lead to confusion.
Forward this env.

Fixes: #2378
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-08-03 18:28:17 +02:00
Samuel Ortiz
0a2e2c6038
Merge pull request #2358 from YchauWang/wyc-deploy-test
docs: update kata deploy README doc to add cloud-hypervisor test command
2021-08-03 10:55:13 +02:00
Joao Vanzuita
d007bb8550 kata-deploy: shorten directory path
long file paths are difficult to read, this change adds a new readonly variable to shorten the full file path of the static build folder files.

Fixes: #2354
Signed-off-by: Joao Vanzuita <joaovanzuita@me.com>
2021-08-02 22:37:39 +02:00
James O. D. Hunt
4f0726bc49 docs: Remove table of contents
Removed all TOCs now that GitHub auto-generates them.

Also updated the documentation requirements doc removing the requirement
to add a TOC.

Fixes: #2022.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-07-30 10:58:22 +01:00
James O. D. Hunt
f186c5e284 docs: Fix invalid URLs
Correct broken / stale URLs as detected by the CI URL checker.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-07-30 10:58:22 +01:00
James O. D. Hunt
7c610a6ff1 docs: Fix shell code
Correct the shell code in the packaging tools README to keep the CI
happy.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-07-30 10:58:22 +01:00
wangyongchao.bj
80afba15ee docs: update kata deploy README doc to add cloud-hypervisor test command
Kata deploy README document only contains Firecracker and Qemu. This PR adds
 cloud-hypervisor test command to the README.md file.

Fixes: #2357

Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
2021-07-30 10:01:13 +08:00
wangyongchao.bj
5a0d3c4fac docs: update the kata release url in the kata deploy document
fixed the url error, updated the path to kata 2.x release
(https://github.com/kata-containers/kata-containers/releases) from kata 1.x release
(https://github.com/kata-containers/runtime/releases) in the kata-deploy README.md file.

Fixes: #2355.

Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
2021-07-30 09:50:30 +08:00
Fabiano Fidêncio
2d142bc92d
Merge pull request #2155 from jcvenegas/kata-deploy-2021-06-29
kata-deploy: Allow build  kata-deploy  tarball from HEAD
2021-07-29 22:50:03 +02:00
Carlos Venegas
81e6bf6f2c kata-deploy: Split shimv2 build in a separate container.
Instead of install golang in the base container, split the shimv2 build.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-28 19:45:35 +00:00
Carlos Venegas
d46ae3248e kernel: build: Add container build
Add script to build kernel in a container.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-28 19:45:35 +00:00
Carlos Venegas
85987c6d79 kata-deploy: Add Makefile
Add makefile to document possible options to run.

e.g
Default: Create a kata tarball, it will build assets concurrently.
```
$ make

```

Create a tarball build for cloud-hypervisor.
```
$ make cloud-hypervisor
```

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-28 19:45:35 +00:00
Carlos Venegas
b9d2eea39b kata-deploy: Add script to merge kata tarballs.
After each asset is build it is needed to merge them all into one single
tarball.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-28 19:45:35 +00:00
Carlos Venegas
4895747f35 Rootfs: Add curl to alpine rootfs builder.
If alpine image is created inside a container,
it does not get any golang version data. It will try
to get it by installing yq. To install yq curl is used.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-28 19:45:35 +00:00
Carlos Venegas
2f9859ab2f build: Reuse firecracker directory on builds.
kata-deploy buider now reuses the build directory, this
makes faster rebuilds. Update firecracker builder to
not fail if is called twice.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-28 19:45:35 +00:00
Carlos Venegas
3533a5b61d Packaging: stop using GOPATH for yq.
Use the yq installed in the env.  Needed
to build kata from docker. The container builder
has not an initial Go env.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-28 19:45:35 +00:00
Carlos Venegas
0c5ded4bd7 kata-deploy: build kata only with docker in host
Add script to build kata using docker.

Allow build kata-deploy binaries using docker.
kata-deploy-binaries-in-docker.sh is a wrapper of
kata-deploy-binaries.sh it will call kata-deploy-binaries.sh in a
container with all the dependencies installed.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-28 19:45:35 +00:00
GabyCT
24cbb97f68
Merge pull request #2298 from fgiudici/yq_latest_fix
osbuilder/scripts: add support to yq version 4 and above
2021-07-23 12:19:46 -05:00
Fabiano Fidêncio
d75c01bd67
Merge pull request #2186 from YchauWang/yc-osbuilder-arm
osbuilder: update centos arm rootfs image config 'GPG_KEY_ARCH_URL'
2021-07-23 11:17:08 +02:00
Carlos Venegas
8befb1f39f kata-deploy: Refactor builder options.
Update kata-deploy-binaries.sh cli options.

Add options to allow ask build a tarball for a specific asset.
It will help developers build a specific component and update
a kata-deploy installation. Also build each asset independetly
can help to create cache tarballs per asset in the future.

e.g. Build a tarball with shimv2.

```
./kata-deploy-binaries.sh --build=shim-v2
```

Additionally, the script path is moved to a new directory
as not only will work for releases.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-22 20:58:54 +00:00
Carlos Venegas
7125f5d8cf image-builder: Allow build image and initrd independently.
This will help to do concurrent builds and speedup CI.

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-07-22 20:49:38 +00:00
Francesco Giudici
0f8c0dbc52 osbuilder/scripts: add support to yq version 4 and above
yq changed syntax in an incompatible way starting from version 4 and
above. Deal with that.

Fixes: #2297

Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
2021-07-22 16:01:57 +02:00
Julio Montes
b4c45df885 runtime: tools/packaging/cmd/kata-pkgsync: fix govet fieldalignment
Fix structures alignment

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-07-20 12:09:54 -05:00
Bin Liu
6b00806bb8
Merge pull request #2243 from egernst/bump-tokio
agent/agent-ctl: update tokio to 1.8.1
2021-07-20 13:56:32 +08:00
wangyongchao.bj
3882619471 osbuilder: update centos arm rootfs image config 'GPG_KEY_ARCH_URL'
fix GPG_KEY_ARCH_URL config of centos's config_aarch64,
update to "http://mirror.centos.org/altarch/7/os/aarch64/RPM-GPG-KEY-CentOS-7".

Fixes: #2181

Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
2021-07-20 13:49:17 +08:00
Bin Liu
1da8fa1655
Merge pull request #2171 from liubin/feature/delete-golang-for-rootfs-builder
osbuilder: Skip installing golang for building rootfs
2021-07-19 17:06:57 +08:00
bin
b12b21f337 osbuilder: Skip installing golang for building rootfs
Building rootfs does not depend on golang, delete intalling
golang may save build time.

And there is only rust agent now, the code for golang agent should
be deleted too.

Fixes: #2170

Signed-off-by: bin <bin@hyper.sh>
2021-07-15 23:59:15 +08:00