Commit Graph

10911 Commits

Author SHA1 Message Date
Jeremi Piotrowski
70ed46169c workflows: static-checks: Set GOPATH only once
{{ runner.workspace }}/kata-containers and {{ github.workspace }} resolve to
the same value, but they're being used multiple times in the workflow. Remove
multiple definitions and define the GOPATH var at job level once.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-20 12:19:35 +01:00
Jeremi Piotrowski
e201544145 tools/osbuilder/tests: Remove TRAVIS variable
The last remaining user of the TRAVIS variable in this repo is
tools/osbuilder/tests and it is only used to skip spinning up VMs. Travis
didn't support virtualization and the same is true for github actions hosted
runners. Replace the variable with KVM_MISSING and determine availability of
/dev/kvm at runtime.

TRAVIS is also used by '.ci/setup.sh' in kata-containers/tests to reduce the
set of dependencies that gets installed, but this is also in the process of
being removed.

Fixes: #3544
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-20 12:19:35 +01:00
Jeremi Piotrowski
cdb2efe003 workflows: static-checks: Remove TRAVIS_XXX variables
These variables are unused since we don't use travis CI. This also allows to
remove two steps:

- 'Setup GOPATH' only printed variables
- 'Setup travis reference' modified some shell local variables that don't have
  any influence on the rest of the steps

The TRAVIS var is still used by tools/osbuilder/tests to determine if
virtualization is available.

Fixes: #3544
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-20 12:19:35 +01:00
Steve Horsman
4bd6103253
Merge pull request #6491 from jepio/jepio/kata-deploy-push-more-space
CC | kata-deploy-push more space
2023-03-20 10:34:51 +00:00
Fabiano Fidêncio
20452402b5
Merge pull request #6498 from fidencio/topic/cc-switch-back-the-permission-of-cache-components-to-0755
cache_components: Switch back the permission to 755
2023-03-20 09:09:22 +01:00
Fabiano Fidêncio
77783153b6 cache_components: Switch back the permission to 755
2b41dbe broke all the cached jobs as it changed the permission of the
cache_components.sh file from 755 to 644.

Fixes: #6497

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-17 23:50:47 +01:00
Fabiano Fidêncio
45759e82b1
Merge pull request #6474 from UnmeshDeodhar/fix-sev-cached-kernel-bug
packaging: fix sev kernel caching
2023-03-17 23:01:40 +01:00
Jeremi Piotrowski
26f80ff468 workflows: kata-deploy-push: Remove unneeded tools from runner
`make kata-tarball` needs a lot of disk space and github action runners don't
have that much of it. Remove unneeded tools from the runner, which frees
another ~10GB of space.

Fixes: #6490
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-17 17:46:38 +01:00
Jeremi Piotrowski
a394798b75 kata-deploy: Remove temporary directory after creating tarball
The various tarballs are unpacked into a temporary directory, and then that
directory is compressed into kata-static.tar.xz. After we have the tarball,
there is no reason to keep the temporary directory. Dispose of it as the last
step.

Fixes: #6490
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-17 17:43:42 +01:00
Unmesh Deodhar
2b41dbe00f packaging: fix sev kernel caching
Check if tarball exists. If yes, skip the tarball create command.

Fixes: #6473

signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
2023-03-17 10:17:54 -05:00
Fabiano Fidêncio
9c16268b0a
Merge pull request #6486 from fidencio/topic/cache-artefacts-fix-kernel-version
CC | kernel: Ensure `kata_config_version` is taken into account
2023-03-17 10:17:09 +01:00
Fabiano Fidêncio
b3a2caa527 kernel: Ensure kata_config_version is taken into account
We need to ensure `kata_config_version` is taken into account when:
* consuming a cached kernel, otherwise we may introduce changes to a
  kernel that will never be validated as part of the PR
* caching the kernel, otherwise we won't update the artefacts if just a
  config is changed

Fixes: #6485

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-16 17:44:06 +01:00
Steve Horsman
c3059d0bd9
Merge pull request #6207 from BbolroC/s390x-prepare-se-image
CC: Prepare secure execution image tarball for s390x
2023-03-15 14:33:59 +00:00
Steve Horsman
5da86f3e25
Merge pull request #6440 from GeorginaKin/CCv0
CCv0: Merge main into CCv0 branch
2023-03-15 10:40:55 +00:00
Fabiano Fidêncio
0bcbbfe44f
Merge pull request #5665 from jimcadden/online_sev_kbc
CC | runtime: Add online-kbs to kernel params
2023-03-14 21:19:37 +01:00
Fabiano Fidêncio
c9d6b46ea0
Merge pull request #6447 from fidencio/topic/kata-deploy-cc-switch-to-using-ubuntu
kata-deploy-cc: Switch to using an ubuntu image
2023-03-13 17:39:40 +01:00
Georgina Kinge
77e5351102 Bumping the kata config version
Bumping the version to 101cc to fix the static checks

Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2023-03-13 16:21:21 +00:00
Georgina Kinge
818569be79 Reverting container image names
Reverting the container image names to pick up the lib.sh methods.

Signed-off-by: Georgina Kinge georgina.kinge@ibm.com
Co-authored-by: Steve Horsman <steven@uk.ibm.com>
2023-03-13 16:21:21 +00:00
Fabiano Fidêncio
1eeed31e23 kata-deploy-cc: Switch to using Ubuntu image
This will help us in several ways:
* The first one is not using an image that's close to be EOLed, and
  which doesn't officially provide multi-arch images.
* The second is getting closer to what's been already done on main.
* The third is simplifying the logic to build the payload image.

Fixes: #6446

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-10 20:38:30 +01:00
Fabiano Fidêncio
a854355bea kata-deploy: Use more descriptive names for IMG_{TAG,NAME}
On main we're using BASE_IMAGE_TAG and BASE_IMAGE_NAME to avoid
any kind of confusion.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-10 15:54:29 +01:00
Georgina Kinge
eba66de646 Revert "runtime: clh: Re-generate the client code"
Reverting as cloud-hypervisor shouldn't be upgraded in CC.

Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2023-03-10 11:28:04 +00:00
Wainer Moschetta
a6bbc8af4e
Merge pull request #6416 from UnmeshDeodhar/remove-duplicate-sev-kernel-build
packaging: fix sev kernel build
2023-03-09 15:51:33 -03:00
Georgina Kinge
b5526b4ab6 Reverting cloud-hypervisor version
Reverting the cloud-hypervisor bump back to v29

Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2023-03-09 16:25:39 +00:00
Georgina Kinge
ec6ce46e7e CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #6428
Signed-off-by: Georgina Kinge <georgina.kinge@ibm.com>
2023-03-09 16:14:53 +00:00
Christophe de Dinechin
7566a7eae4
Merge pull request #6432 from fidencio/topic/simplify-get-last-modification
packaging: Simplify get_last_modification()
2023-03-09 10:57:58 +01:00
Hyounggyu Choi
f831098852 CC: Prepare secure execution image tarball for s390x
This is to prepare a secure image tarball to run a confidential
container for IBM Z SE(TEE).

Fixes: #6206

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-03-09 13:55:48 +09:00
Fabiano Fidêncio
43ce3f7588 packaging: Simplify get_last_modification()
There's no need to pass repo_root_dir to get_last_modification() as the
variable used everywhere is exported from that very same file.

Fixes: #6431

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-08 21:22:03 +01:00
Fabiano Fidêncio
33c5c49719 packaging: Move repo_root_dir to lib.sh
This is used in several parts of the code, and can have a single
declaration as part of the `lib.sh` file, which is already imported by
all the places where it's used.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-08 21:10:53 +01:00
James O. D. Hunt
614d1817ce
Merge pull request #6410 from tg5788re/kata-manager-use-runtime-checks
utils: Make kata-manager.sh runs checks
2023-03-08 09:55:03 +00:00
Chao Wu
fef268a7de
Merge pull request #6413 from xuejun-xj/xuejun/pmu
dragonball: support pmu on aarch64
2023-03-08 14:24:31 +08:00
Unmesh Deodhar
9afa38f11b packaging: add kernel modules dir
Create a directory where kernel modules will be extracted.

Fixes: #6418

Signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
2023-03-07 10:15:43 -06:00
Unmesh Deodhar
7267237660 packaging: remove duplicate sev kernel build
Remove duplicate SEV kernel builds.

Fixes: #6415

Signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
2023-03-07 10:15:41 -06:00
Steve Horsman
cc1821fb8b
Merge pull request #6409 from Sig00rd/patch-1
docs: fix typo in key filename in AWS installation guide
2023-03-07 15:19:46 +00:00
Fabiano Fidêncio
861552c305
Merge pull request #6414 from jepio/jepio/backport-3.1-rustjail-systemd-cgroup-fix-6331
backport rustjail systemd cgroup fix #6331 to 3.1
2023-03-07 12:51:08 +01:00
Jeremi Piotrowski
23488312f5 agent: always use cgroupfs when running as init
The logic to decide which cgroup driver is used is currently based on the
cgroup path that the host provides. This requires host and guest to use the
same cgroup driver. If the guest uses kata-agent as init, then systemd can't be
used as the cgroup driver. If the host requests a systemd cgroup, this
currently results in a rustjail panic:

  thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: I/O error: No such file or directory (os error 2)

  Caused by:
      No such file or directory (os error 2)', rustjail/src/cgroups/systemd/manager.rs:44:51
  stack backtrace:
     0:     0x7ff0fe77a793 - std::backtrace_rs::backtrace::libunwind::trace::h8c197fa9a679d134
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
     1:     0x7ff0fe77a793 - std::backtrace_rs::backtrace::trace_unsynchronized::h9ee19d58b6d5934a
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
     2:     0x7ff0fe77a793 - std::sys_common::backtrace::_print_fmt::h4badc450600fc417
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:65:5
     3:     0x7ff0fe77a793 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::had334ddb529a2169
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:44:22
     4:     0x7ff0fdce815e - core::fmt::write::h1aa7694f03e44db2
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/fmt/mod.rs:1209:17
     5:     0x7ff0fe74e0c4 - std::io::Write::write_fmt::h61b2bdc565be41b5
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/io/mod.rs:1682:15
     6:     0x7ff0fe77cd3f - std::sys_common::backtrace::_print::h4ec69798b72ff254
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:47:5
     7:     0x7ff0fe77cd3f - std::sys_common::backtrace::print::h0e6c02048dec3c77
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:34:9
     8:     0x7ff0fe77c93f - std::panicking::default_hook::{{closure}}::hcdb7e705dc37ea6e
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:267:22
     9:     0x7ff0fe77d9b8 - std::panicking::default_hook::he03a933a0f01790f
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:286:9
    10:     0x7ff0fe77d9b8 - std::panicking::rust_panic_with_hook::he26b680bfd953008
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:688:13
    11:     0x7ff0fe77d482 - std::panicking::begin_panic_handler::{{closure}}::h559120d2dd1c6180
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:579:13
    12:     0x7ff0fe77d3ec - std::sys_common::backtrace::__rust_end_short_backtrace::h36db621fc93b005a
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:137:18
    13:     0x7ff0fe77d3c1 - rust_begin_unwind
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:575:5
    14:     0x7ff0fda52ee2 - core::panicking::panic_fmt::he7679b415d25c5f4
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/panicking.rs:65:14
    15:     0x7ff0fda53182 - core::result::unwrap_failed::hb71caff146724b6b
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/result.rs:1791:5
    16:     0x7ff0fe5bd738 - <rustjail::cgroups::systemd::manager::Manager as rustjail::cgroups::Manager>::apply::hd46958d9d807d2ca
    17:     0x7ff0fe606d80 - <rustjail::container::LinuxContainer as rustjail::container::BaseContainer>::start::{{closure}}::h1de806d91fcb878f
    18:     0x7ff0fe604a76 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h1749c148adcc235f
    19:     0x7ff0fdc0c992 - kata_agent::rpc::AgentService::do_create_container::{{closure}}::{{closure}}::hc1b87a15dfdf2f64
    20:     0x7ff0fdb80ae4 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h846a8c9e4fb67707
    21:     0x7ff0fe3bb816 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h53de16ff66ed3972
    22:     0x7ff0fdb519cb - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h1cbece980286c0f4
    23:     0x7ff0fdf4019c - <tokio::future::poll_fn::PollFn<F> as core::future::future::Future>::poll::hc8e72d155feb8d1f
    24:     0x7ff0fdfa5fd8 - tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut::h0a407ffe2559449a
    25:     0x7ff0fdf033a1 - tokio::runtime::task::raw::poll::h1045d9f1db9742de
    26:     0x7ff0fe7a8ce2 - tokio::runtime::scheduler::multi_thread::worker::Context::run_task::h4924ae3464af7fbd
    27:     0x7ff0fe7afb85 - tokio::runtime::task::raw::poll::h5c843be39646b833
    28:     0x7ff0fe7a05ee - std::sys_common::backtrace::__rust_begin_short_backtrace::ha7777c55b98a9bd1
    29:     0x7ff0fe7a9bdb - core::ops::function::FnOnce::call_once{{vtable.shim}}::h27ec83c953360cdd
    30:     0x7ff0fe7801d5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hed812350c5aef7a8
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/alloc/src/boxed.rs:1987:9
    31:     0x7ff0fe7801d5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hc7df8e435a658960
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/alloc/src/boxed.rs:1987:9
    32:     0x7ff0fe7801d5 - std::sys::unix:🧵:Thread:🆕:thread_start::h575491a8a17dbb33
                                 at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys/unix/thread.rs:108:17

Forward the value of "init_mode" to AgentService, so that we can force cgroupfs
when systemd is unavailable.

Fixes: #5779
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-06 20:34:21 +01:00
Jeremi Piotrowski
8546387348 agent: determine value of use_systemd_cgroup before LinuxContainer::new()
Right now LinuxContainer::new() gets passed a CreateOpts struct, but then
modifies the use_systemd_cgroup field inside that struct. Pull the cgroups path
parsing logic into do_create_container, so that CreateOpts can be immutable in
LinuxContainer::new. This is just moving things around, there should be no
functional changes.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-06 20:34:21 +01:00
Jeremi Piotrowski
736aae47a4 rustjail: print type of cgroup manager
Since the cgroup manager is wrapped in a dyn now, the print in
LinuxContainer::new has been useless and just says "CgroupManager". Extend the
Debug trait for 'dyn Manager' to print the type of the cgroup manager so that
it's easier to debug issues.

Fixes: #5779
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2023-03-06 20:34:21 +01:00
Fabiano Fidêncio
0749657c73
Merge pull request #6359 from singhwang/main
main | kata-deploy: Fix kata deploy arm64 image build error
2023-03-06 16:48:03 +01:00
Fabiano Fidêncio
dbae281924 workflows: Properly set the kata-tarball architecture
Let's make sure the kata-tarball architecture upload / downloaded / used
is exactly the same one that we need as part of the architecture we're
using to generate the image.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-06 13:18:51 +01:00
Fabiano Fidêncio
76b4591e2b tools: Adjust the build-and-upload-payload.sh script
Now that we've switched the base container image to using Ubuntu instead
of CentOS, we don't need any kind of extra logic to correctly build the
image for different architectures, as Ubuntu is a multi-arch image that
supports all the architectures we're targetting.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-06 13:18:51 +01:00
SinghWang
cd2aaeda2a kata-deploy: Switch to using an ubuntu image
Let's make sure we use a multi-arch image for building kata-deploy.
A few changes were also added in order to get systemd working inside the
kata-deploy image, due to the switch from CentOS to Ubuntu.

Fixes: #6358
Signed-off-by: SinghWang <wangxin_0611@126.com>
2023-03-06 13:18:51 +01:00
Szymon Fugas
2d43e13102
docs: fix typo in AWS installation guide
Fixes referring to previously created key file with .pen extension instead of .pem.

Fixes: #6412
Signed-off-by: Sig00rd <sfugas@virtuslab.com>
2023-03-06 13:18:08 +01:00
xuejun-xj
760f78137d dragonball: support pmu on aarch64
This commit adds support for pmu virtualization on aarch64. The
initialization of pmu is in the following order:
1. Receive pmu parameter(vpmu_feature) from runtime-rs to determine the
VpmuFeatureLevel.
2. Judge whether to initialize pmu devices and add pmu device node into
fdt on aarch64, according to VpmuFeatureLevel.

Fixes: #6168

Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
2023-03-06 18:55:13 +08:00
Fabiano Fidêncio
93a40cb35e
Merge pull request #6402 from fidencio/topic/yet-more-fixes-for-the-publish-kata-deploy-payload-work
workflows: Yet more fixes for publishing the kata-deploy payload after every PR merged
2023-03-06 10:43:32 +01:00
Fabiano Fidêncio
df35f8f885
Merge pull request #6331 from jepio/jepio/fix-agent-init-cgroups
rustjail: fix cgroup handling in agent-init mode
2023-03-05 20:29:40 +01:00
Fabiano Fidêncio
98d611623f
Merge pull request #6361 from etrunko/main
runtime/Makefile: Fix install-containerd-shim-v2 dependency
2023-03-04 13:47:11 +01:00
Fabiano Fidêncio
9bc7bef3d6 kata-deploy: Fix path to the Dockerfile
As part of bd1ed26c8d, we've pointed to
the Dockerfile that's used in the CC branch, which is wrong.

For what we're doing on main, we should be pointing to the one under the
`kata-deploy` folder, and not the one under the non-existent
`kata-deploy-cc` one.

Fixes: #6343

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-04 12:18:38 +01:00
Fabiano Fidêncio
78ba363f8e kata-deploy: Use different images for s390x and aarch64
As the image provided as part of registry.centos.org is not a multi-arch
one, at least not for CentOS 7, we need to expand the script used to
build the image to pass images that are known to work for s390x (ClefOS)
and aarch64 (CentOS, but coming from dockerhub).

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-04 12:18:32 +01:00
Fabiano Fidêncio
6267909501 kata-deploy: Allow passing BASE_IMAGE_{NAME,TAG}
Let's break the IMAGE build parameter into BASE_IMAGE_NAME and
BASE_IMAGE_TAG, as it makes it easier to replace the default CentOS
image by something else.

Spoiler alert, the default CentOS image is **not** multi-arch, and we do
want to support at least aarch64 and s390x in the near term future.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-03-04 12:16:41 +01:00
tg5788re
f8e44172f6 utils: Make kata-manager.sh runs checks
Updated the `kata-manager.sh` script to make it run all the checks on
the host system before attempting to create a container. If any checks
fail, they will indicate to the user what the problem is in a clearer
manner than those reported by the container manager.

Fixes: #6281.

Signed-off-by: tg5788re <jfokugas@gmail.com>
2023-03-03 09:56:12 -06:00