We don't need to store the kernel headers anymore. We do need to store
the kernel modules, instead.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
We've done some bad file based driver determination,
now with versions.yaml there is a single source of truth.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
We need to package the build modules for the rootfs
to be able to consume it. We package the whole
/lib/modules/$(uname -r) directory strip=2.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
We want to have deterministic behaviour and only
one valid driver version acceptable via versions.yaml
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
We actually never installed yq to the kernel build,
there are some path that use yq but were never hit,
for the GPU use-case we need to read values from versions.yaml
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This is needed as the 580 driver doesn't build against 6.18.x, and the
590 driver is not yet fully working for our case, thus we stick to the
previous version that worked before.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Bump both the kernel and kernel-confidential versions from v6.12.x and
v6.16.x to v6.18.4, aligning with the new LTS release.
Kernel 6.18 introduced several configuration changes that required
updates to our kernel config fragments:
* CRYPTO_FIPS dependencies changed:
- In 6.12: depended on !CRYPTO_MANAGER_DISABLE_TESTS
- In 6.18: now depends on CRYPTO_SELFTESTS (which requires EXPERT)
Added CONFIG_EXPERT=y and CONFIG_CRYPTO_SELFTESTS=y to crypto.conf
to satisfy the new dependency chain.
* CONFIG_EXPERT is a naughty one, as it disables / enables a bunch
of things behind ones back, probably just to prove a point that
it is for experts ;-) ... regardless, a reasonable amount of
options had to be re-added in order to make sure anything ends
up broken.
* Legacy iptables support:
Kernel 6.18 requires explicit legacy xtables/iptables configs for
IP_NF_* options. Added CONFIG_NETFILTER_XTABLES_LEGACY,
CONFIG_IP_NF_IPTABLES_LEGACY, and CONFIG_IP6_NF_IPTABLES_LEGACY
to netfilter.conf.
* Module signing dependencies:
Added CONFIG_MODULES=y and other required dependencies to
module_signing.conf to ensure MODULE_SIG can be properly enabled.
* Whitelist updates:
- Added CONFIG_NF_CT_PROTO_DCCP (removed in 6.18+)
- Added CONFIG_CRYPTO_SELFTESTS, CONFIG_NETFILTER_XTABLES_LEGACY,
CONFIG_IP_NF_IPTABLES_LEGACY, CONFIG_IP6_NF_IPTABLES_LEGACY
(added in 6.18+, not present in older kernels like 6.12)
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
This commit adds a Github workflow for building a Github Pages site for the markdown
files in the docs/ directory. Zensical is a new markdown-based static site generation
framework built by the creators of Material for Mkdocs. https://zensical.org/
This commit does not clean the doc structure, so site navigation is initially going to
be messy.
Signed-off-by: LandonTClipp <11232769+LandonTClipp@users.noreply.github.com>
Remove the agent hotplug timeout parameter from the kernel
command line. Having shifted to VFIO cold-plug, this parameter is
no longer needed.
Remove the no longer required parameter for TDX and thus align the
SNP and TDX configurations.
Add a parameter to avoid the kernel to mount the /dev tmpfs. NVRC
and later on kata-agent attempt this. While kata-agent does not
panic when mounting /dev fails, NVRC makes mounting /dev a hard
requirement.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
set_container_command() previously appended command arguments
one-by-one with
'.command += [...]'. This makes the helper non-idempotent and can
lead to unexpected command arrays when invoked multiple times.
Update the helper to set the full command array in a single yq v4
expression and print the target YAML path plus the command being
applied to simplify debugging when tests fail.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The pod config file created by new_pod_config() was generated via
mktemp using the template "pod-config.yaml.in.XXX", which produces
filenames that do not end with ".yaml" (e.g. pod-config.yaml.in.ABC).
If the random combination of special suffix with ".Csv" or ".Xml", etc.
the following operations with yq will fail.
Some helpers and tooling assume the config path ends with ".yaml".
Switch the mktemp template to place the random suffix before the
extension so the returned path always ends with ".yaml".
Fixes: #12268, #12319
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This is a suggestion from Choi, so we can easily test with a specific
kubectl version and also easily understand which kubectl version is
being used in case of failure.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
This image will be used by our helm charts to verify that a
kata-containers deployment is correct.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Enhance the wait_for_migration implementation to reliably wait for
QEMU migration completion and avoid the previous `sleep(280ms)`
delay.
(1) Add an initial fast-path query to return immediately if
migration is already completed/failed/cancelled.
(2) Use a hard deadline to enforce timeouts deterministically.
(3) Implement adaptive polling with backoff and a maximum interval
to reduce QMP load while keeping responsiveness.
(4) Unify migration status handling and return clear errors on
failed/cancelled states.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Return information about current migration process. And the input
and output as below:
{ 'command': 'query-migrate', 'returns': 'MigrationInfo' }
But note that the Qemu API is valid within qapi-rs(v0.15+)
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The detailed information about the updated versions as below:
```
qapi = { version = "0.15", features = ["qmp", "async-tokio-all"] }
qapi-spec = "0.3.2"
qapi-qmp = "0.15.0"
```
and it will correct some corresonding structures.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Change the secure_storage_integrity option's default value to true.
With this, integrity protection for encrypted block device contents
will be requested from the confidential data hub by default, see the
agent's cdh_handler_trusted_storage function in rpc.rs.
This behavior can be disabled by explicitly setting the
agent.secure_storage_integrity parameter to 0 or false via kernel
command line parameters.
This will affect the trusted storage implementation for the guest-pull
mechanism, and it will affect future implementations using this code
path, such as implementations for ephemeral secure storage.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
In some builds we are seeing:
```
error: could not create temp file /opt/rustup/tmp/r2xu46kwuyc7k2kr_file: Permission denied (os error 13)
```
in the agent-ctl build, so try and port a fix from #12313 to the tools build
to try and resolve this.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Fixes deploying kata-containers using k3s. The deploy script fails with /opt/kata-artifacts/scripts/kata-deploy.sh: line 397: [: too many arguments
Signed-off-by: Federico A. Corazza <git@facorazza.com>
yamllint complains that there is only one space before the comment,
so add a second to prevent this annoying message showing up.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Create a new page for a reference implementation for Kubernetes
using QEMU, the go shim and an NVIDIA rootfs. The new page
contains information on:
- components involved in the NVIDIA (TEE) GPU scenario
- orchestration flow for GPU passthrough scenarios
- deployment guidance
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
- Apply a few structural/grouping changes and improve flow
- Group build sections together
- Move usage examples to last section
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
The following error was observed during virtiofsd static build:
```
error: could not create temp file /opt/rustup/tmp/p44enysfaxwdbvw4_file:
Permission denied (os error 13)
```
This occurs because RUSTUP_HOME and CARGO_HOME were initialized by the
root user during `docker build`, but `cargo build` is executed as a
non-root user via 'docker run --user'.
Ensure these directories are writable by adjusting the permission after
the toolchain installation is complete.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
OVMF build for Intel TDX (aka "TDVF") was disabled in favor of Ubuntu/
CentOS pre-upstream releases of Intel TDX.
See 4292c4c3b1.
It's time to re-enable the build and move runtime configurations to
use it (the latter will be done in a later commit).
This is a partial revert of 4292c4c3b with the following changes:
- Stop calling OVMF for Intel TDX "TDVF" and follow the naming distros
use for TDX enabled build: OVMF.inteltdx.fd.
- Single binary OVMF.inteltdx.fd is supported using -bios QEMU param.
- Secure Boot infrastructure is disabled since Kata does not support it.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Actually this method is indeed called, just add attribute of
`#[allow(dead_code)]` to allow UT pass. And the warning looks like:
warning: method `send_message_with_payload` is never used
|
224 | impl<R: Req> Endpoint<R> {
| ------------------------ method in this implementation
...
522 | pub fn send_message_with_payload<T: Sized, P: Sized>(
| ^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: `#[warn(dead_code)]` on by default
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
warning: unused `std::result::Result` that must be used
-->
src/dragonball/dbs_virtio_devices/src/vhost/vhost_user/net.rs:679:9
|
679 | / VirtioDevice::<Arc<GuestMemoryMmap<()>>, QueueSync,
GuestRegionMmap>::write_config(
680 | | &mut dev, 0, &config,
681 | | );
| |_________^
|
= note: this `Result` may be an `Err` variant, which should be
handled
= note: `#[warn(unused_must_use)]` on by default
help: use `let _ = ...` to ignore the resulting value
|
679 | let _ = VirtioDevice::<Arc<GuestMemoryMmap<()>>,
QueueSync, GuestRegionMmap>::write_config(
| +++++++
warning: unused `std::result::Result` that must be used
-->
src/dragonball/dbs_virtio_devices/src/vhost/vhost_user/net.rs:683:9
|
683 | / VirtioDevice::<Arc<GuestMemoryMmap<()>>, QueueSync,
GuestRegionMmap>::read_config(
684 | | &mut dev, 0, &mut data,
685 | | );
| |_________^
|
= note: this `Result` may be an `Err` variant, which should be
handled
help: use `let _ = ...` to ignore the resulting value
|
683 | let _ = VirtioDevice::<Arc<GuestMemoryMmap<()>>,
QueueSync, GuestRegionMmap>::read_config(
| +++++++
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
the WARNING looks like as:
...
warning: variable does not need to be mutable
--> src/dragonball/dbs_virtio_devices/src/vsock/csm/txbuf.rs:217:13
|
217 | let mut tmp: Vec<u8> = vec![0; TxBuf::SIZE - 2];
| ----^^^
| |
| help: remove this `mut`
|
= note: `#[warn(unused_mut)]` on by default
...
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Till k8s 1.34 we could grep by "Started containerd". From k8s 1.35
onwards the event message changed and we should, instead, grep by
"Container started".
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
QEMU v10.2.0 was released on December 24th, 2025.
The experimental GPU SNP / TDX are also pointing to v10.2.0 release with
their gpu-{snp,tdx}-20260107 branch.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
sha2 0.9.3 includes the use of cpuid-bool, which was renamed to cpufeatures
around 5 years ago. Try moving to a workspace dependency of sha2
and bumping to the latest version to remediate RUSTSEC-2021-0064
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
While the use-case of Intel QuickAssist (QAT) accelerated crypto
and/or compression with k8s and Kata Containers is still valid,
the setup instructions are outdated:
Starting with Intel Xeon Gen4 (Sapphire Rapids), QAT driver
stack moved to in-tree drivers without a separete SR-IOV VF
driver.
Drop all the setup instructions but keep the use-cases doc
for reference. Users wanting to enable the use-case, should consult
with Intel QAT Device plugins or Intel QAT DRA driver authors.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
The nontee job (run-k8s-tests-coco-nontee) for qemu-coco-dev-runtime-rs
is running well and it's time to make it required when the CI runs.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Using the built in size_of_val is easier to read and less error-prone
than doing this calculation manually
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
#[cfg(feature = "cargo-clippy")] has been deprecated for years,
so should be replaced with `#[cfg(clippy)]`
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
There are many, many null pointer dereferences in the bindgen code
when moving between rust 1.85.1 and 1.86 and no docs of the source
that it was generated from, so try and skip
these test from running until an SME can look at them @lifupan
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Clippy is recommending that format args are inlined for
better clarity, so update our code to remove these warnings
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Clippy is recommending that format args are inlined for
better clarity, so update our code to remove these warnings
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Clippy is recommending that format args are inlined for
better clarity, so update our code to remove these warnings
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
runtime-rs crates are pulled into kata-ctl and some of these have
bumped recently, so update these in kata-ctl as well
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Clippy is recommending that format args are inlined for
better clarity, so update our code to remove these warnings
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Clippy is recommending that format args are inlined for
better clarity, so update our code to remove these warnings
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Clippy is recommending that format args are inlined for
better clarity, so update our code to remove these warnings
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Clippy is recommending that format args are inlined for
better clarity, so update our code to remove these warnings
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Clippy is recommending that format args are inlined for
better clarity, so ensure our docs include this
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Clippy is recommending that format args are inlined for
better clarity, so update our code to remove these warnings
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
In #12151 the version was bumped in cargo.toml, but the update not
done, so run `cargo update -p container-device-interface` to apply it
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Since #12204 was merged, the following error has been observed:
```
bats warning: Executed 1 instead of expected 2 tests
[run_kubernetes_tests.sh:162] ERROR: Tests FAILED from suites: k8s-empty-dirs.bats
```
The cause is that `pod_logs_file` is re-declared as a local variable
in the second test before skipping, which makes it inaccessible
in `teardown()` and leads to an error.
This commit removes the re-declaration of the variable.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The Rust kata-deploy binary calls list_runtimeclasses() during NFD
setup, but the ClusterRole only granted get and patch permissions.
Add the list verb to the runtimeclasses resource permissions to fix
the RBAC error:
runtimeclasses.node.k8s.io is forbidden: User
\"system:serviceaccount:kube-system:kata-deploy-sa\" cannot list
resource \"runtimeclasses\" in API group \"node.k8s.io\" at the
cluster scope
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
KVM is not available in our ARM runners, let's skip those tests
accordingly, while making the rest test cases remain tested on machines
with KVM present and access to KVM device.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
There are test cases require interaction with KVM device, introduce
skip_if_kvm_unaccessable macro to skip them.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Changes in NIM/RAG samples:
- update image references
- update memory requirements, timeouts, model name
- sanitize some of the probes and print-out
Further refinements can be made in the future.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
cargo test was trying to evaluate the documentation comment and failing,
so try and make the comment explicitly text to avoid this
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
A few structs in genpolicy are never constructed, so add
`#[allow(dead_code)]` to prevent this clipped warning
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
In unicode you can have multi-byte characters, so it's better to
user char_indices than enumerate the bytes
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
VirtioBlkCcwDeviceHandler and VirtioBlkCcwHandler
are only constructed on s390x, so add #[cfg(target_arch = "s390x")]
to all the code
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
We can use the new Error::other options rather than
Error:new(Error:Kind:Other and drop our own macro that did this mapping
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Fix the warning throw up:
```
warning: hiding a lifetime that's elided elsewhere is confusing
--> /root/go/src/github.com/kata-containers/kata-containers/src/libs/kata-types/src/utils/u32_set.rs:50:17
|
50 | pub fn iter(&self) -> Iter<u32> {
| ^^^^^ --------- the same lifetime is hidden here
| |
| the lifetime is elided here
|
= help: the same lifetime is referred to in inconsistent ways, making the signature confusing
= note: `#[warn(mismatched_lifetime_syntaxes)]` on by default
help: use `'_` for type paths
|
50 | pub fn iter(&self) -> Iter<'_, u32> {
| +++
```
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Update virtiofsd to its latest release.
Here we also need to update the alpine version used by the builder as we
need a version of musl-dev new enough to have wrappers for pread2 and
pwrite2. As bumping, bump to the latest.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Add two attestation tests. The first one sets a resource policy that
requires CPU0 to have an affirming trust level. This is a negative test
which can run on any platform. Setting this policy without setting any
reference values should result in an attestation failure.
Next, a second test will set the same policy, but this time it will use
the journal log to find the QEMU command line from the previous test and
calculate the expected reference values. Currently this is only
supported on SNP using the sev-snp-measure tool, but the same flow
should work on other platforms.
Signed-off-by: Tobin Feldman-Fitzthum <tfeldmanfitz@nvidia.com>
The five tests are set to the same vhost socket path, which could lead
to racing with one another. Use unique name to avoid this.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Let's bump experimental {tdx,snp} QEMU to the tags created Today in the
Confidential Containers repo, which match with QEMU 10.2.0-rc3.
This bump is mostly for early testing what will become 10.2.0, which
will be bumped everywhere then.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
It will address the issue:
"# bats warning: Executed 0 instead of expected 1 tests"
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
As each case need such preparation of get_pod_config_dir,
a better method is directly move it into the setup_common method.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
To measure the duration for journal, we need clearly print the journal
start time and end time for each case which helps to ensure the journal
log is for the specified period for the case.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
For failure cases within CI, we need dump the kata log to help
address issues, but currently large log messages cause partial
log we can see.
We remove initdata log output and increase log level to reduce
log output.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Currently policy_settings_dir is created only when
BATS_TEST_NUMBER == "1",
but delete_tmp_policy_settings_dir "${policy_settings_dir}" is
called in teardown() for every test. This means that for tests
after the first one teardown() may attempt to delete a directory
that was already removed by a previous test, or rely on a value
that does not belong to the current test execution.
Adjust teardown logic so that policy_settings_dir is only deleted
for the first test case (BATS_TEST_NUMBER == "1") and ignored for
subsequent tests. This keeps the original optimization of running
genpolicy only once, while avoiding unnecessary or confusing cleanup
attempts in later test cases.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
the previous pod_name is set as local which can not be captured
within the teardown() function, causing failure.
This commit just remove the `local pod_name` to make it a global
variable.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Otherwise we may hit a `no space left on device` when building the rust
kata-deploy binary.
This happens mostly because of the muli-staging build used to generate a
distroless final container.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
There we ensure labels are added to better deal with ownership of the
runtimeclasses. It's not strictly needed here as helm does take care of
the ownership, but also doesn't hurt to follow what seems to be a common
practice.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Let's shamelessly duplicate the nightly job to have at least nightly
runs using the rust implementation of kata-deploy.
The reason for doing that is to be pragmatic, as pragmatic as possible,
and avoid switching away of the scripts before 3.24.0 release, while
still testing both ways till the switch happens.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Differently than the scripts, which are called as `bash -c ...`, the
kata-deploy rust binary must be invoked directly we do not even have
shell in its container.
For now, the rust version is used in the used image has the "-rust"
suffix, which will help us to have both ways being used / tested for a
little while.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
kata-deploy shell script is not THAT bad and, to be honest, it's quite
handy for quick hacks and quick changes. However, it's been
increasingly becoming harder to maintain as it's grown its scope from a
testing tool to the proper project's front door, lacking unit tests, and
with an abundacy of complex regular expressions and bashisms to be able
to properly parse the environment variables it consumes.
Morever, the fact it is a Frankstein's monster glued together using
python packages, golang binaries, and a distro dependent container makes
the situation VERY HARD to use it from a distroless container (thus,
avoiding security issues), preventing further integration with
components that require a higher standard of security than we've been
requiring.
With everything said, with the help of Cursor (mostly on generating the
tests cases), here comes the oxidized version of the script, which runs
from a distroless container image.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
The ORAS cache helper needs PUSH_TO_REGISTRY to be set to 'yes' to
push new artifacts to the cache. However, this environment variable
was not being passed to the Docker container during agent, tools, and
busybox builds.
Moreover, for ghcr.io authentication, add support for using GH_TOKEN and
GITHUB_ACTOR as fallbacks when explicit credentials
(ARTEFACT_REGISTRY_USERNAME/PASSWORD) are not provided.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
The GPG key used for gperf was incorrectly set to the busybox
maintainer's key (Denis Vlasenko) instead of the gperf maintainer's
key (Marcel Schaible).
Wrong key (busybox): C9E9416F76E610DBD09D040F47B70C55ACC9965B
Denis Vlasenko <vda.linux@googlemail.com>
Correct key (gperf): EDEB87A500CC0A211677FBFD93C08C88471097CD
Marcel Schaible <marcel.schaible@studium.fernuni-hagen.de>
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
kata-remote is a runtime class that cloud-api-adaptor relies on to work.
kata-remote by itself does nothing, and that's the reason it's disabled
by default. We're only adding it here so cloud-api-adaptor charts can
simply do something like `--set shims.remote.enabled=true`.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
When updating ephemeral storages, MS_REMOUNT is explicitly passed as,
for instance, `/dev/shm` should be remounted after memory is hotplugged.
Till now Kata Containers has been explicitly ignoring such updates,
leading to the containers' `/dev/shm` having the size of "half of the
memory allocated, during the startup time", which goes against the
expected behaviour.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
We're only releasing those for amd64 as that's the only architecture
we've been building the packages for.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Let's ensure we can create a specific "tools" tarball, which will help
those who only need to pull those either for testing or production
usage.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
After runtime-rs workspace merged into root workspace, features passed
when building runtime-rs needs to be refactored to be correctly
propagated. Taking dragonball for example, runtime-rs requires runtimes
to depend on virt_conttainers feature, and virt_containers needs to
handle hypervisor features specifically.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
After the workspace integration of runtime-rs, now the output of
runtime-rs is under the repo root, instead of src/runtime-rs. Change the
TARGET_PATH accordingly to tell Makefile where to lookup output.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Some cases in dragonball crates requires interaction with KVM module to
complete, which requires root privilege. Skip those tests under non-root
user.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
MMIODeviceInfo inside the test module of dbs_boot on aarch64 is used for
testing purpose, but `pub` attribute requires it to have documentation.
Since this is used only for testing purpose, let's allow missing_docs
for it.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Test set of dbs_utils's tap module is missing test attribute, which
makes dev-dependencies unusable. Marking tests of tap as test module.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
This is a follow-up of 3fbe693.
Remove runtime-rs from exclude list, and make it as a member of root
workspace.
Specify shim and shim-ctl as the binary of runtime-rs package, make
runtime-rs and all its members into root workspace.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Make runtime-rs a package produces shim and shim-ctl as its binary
product, which enables Makefile to work after it's incorporated into
root workspace.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Init the storage options with original rootfs options.
Addition: XFS, append nouuid to the mount options if not exist.
Signed-off-by: shezhang.lau <shezhang.lau@antgroup.com>
To protect against upstream download failures for gperf and busybox,
implement ORAS-based caching to GHCR.
This adds:
- download-with-oras-cache.sh: Core helper for downloading with cache
- populate-oras-tarball-cache.sh: Script to manually populate cache
- warn() function to lib.sh for consistency
Modified build scripts to:
- Try ORAS cache first (from ghcr.io/kata-containers/kata-containers)
- Fall back to upstream download on cache miss
- Automatically push to cache when PUSH_TO_REGISTRY=yes
The cache is automatically populated during CI builds, and parallel
architecture builds check for existing versions before pushing to avoid
race conditions.
Forks benefit from upstream cache but can override with their own:
ARTEFACT_REPOSITORY=myorg/kata make agent-tarball
Generated-By: Cursor IDE with Claude
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
The runtime handles the fsGroup field of the pod security context by
adding a mount option to the generated storage object [1]. This commit
changes genpolicy to expect this option.
Instead of passing another side input to
yaml::get_container_mounts_and_storages, we pass the entire PodSpec.
This reduces the necessary changes in the pod-generating resources and
allows for possible future use of other PodSpec fields.
[1]: https://github.com/kata-containers/kata-containers/blob/0c6fcde1/src/runtime/virtcontainers/kata_agent.go#L1620-L1625Fixes: #11934
Signed-off-by: Markus Rudy <mr@edgeless.systems>
I've seen this happening with the GPU SNP CI every now and then, but I
don't really understand how this was not caught by the TDX / SNP CI
themselves before.
In any case, the error seen is:
```
Error from server (Forbidden): error when applying patch:
{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"nfd.k8s-sigs.io/v1alpha1\",\"kind\":\"NodeFeatureRule\",\"metadata\":{\"annotations\":{},\"name\":\"amd64-tee-keys\"},\"spec\":{\"rules\":[{\"extendedResources\":{\"sev-snp.amd.com/esids\":\"@cpu.security.sev.encrypted_state_ids\"},\"labels\":{\"amd.feature.node.kubernetes.io/snp\":\"true\"},\"matchFeatures\":[{\"feature\":\"cpu.security\",\"matchExpressions\":{\"sev.snp.enabled\":{\"op\":\"Exists\"}}}],\"name\":\"amd.sev-snp\"},{\"extendedResources\":{\"tdx.intel.com/keys\":\"@cpu.security.tdx.total_keys\"},\"labels\":{\"intel.feature.node.kubernetes.io/tdx\":\"true\"},\"matchFeatures\":[{\"feature\":\"cpu.security\",\"matchExpressions\":{\"tdx.enabled\":{\"op\":\"Exists\"}}}],\"name\":\"intel.tdx\"}]}}\n"}}}
to:
Resource: "nfd.k8s-sigs.io/v1alpha1, Resource=nodefeaturerules", GroupVersionKind: "nfd.k8s-sigs.io/v1alpha1, Kind=NodeFeatureRule"
Name: "amd64-tee-keys", Namespace: ""
for: "/opt/kata-artifacts/node-feature-rules/x86_64-tee-keys.yaml": error when patching "/opt/kata-artifacts/node-feature-rules/x86_64-tee-keys.yaml": nodefeaturerules.nfd.k8s-sigs.io "amd64-tee-keys" is forbidden: User "system:serviceaccount:kube-system:kata-deploy-sa" cannot patch resource "nodefeaturerules" in API group "nfd.k8s-sigs.io" at the cluster scope
```
And the fix is as simple as allowing patching and updating a
nodefeaturerule in our service account RBAC.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Since the CI issue for s390x was resolved on Dec 5th,
the nightly test result has gone green for 10 consecutive days.
This commit puts the e2e tests for s390x again into the required job list.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Let's remove the deprecated features that were marked for removal
after Kata Containers 3.23.0:
kata-deploy.sh:
- Remove non-arch-specific variable fallbacks (SHIMS, DEFAULT_SHIM,
SNAPSHOTTER_HANDLER_MAPPING, ALLOWED_HYPERVISOR_ANNOTATIONS,
PULL_TYPE_MAPPING, EXPERIMENTAL_FORCE_GUEST_PULL). Each arch now
has its own default value.
- Remove CREATE_RUNTIMECLASSES and CREATE_DEFAULT_RUNTIMECLASS
variables and associated functions (create_runtimeclasses,
delete_runtimeclasses, adjust_shim_for_nfd). RuntimeClasses are
now managed by Helm chart, not the daemonset script.
- Unsupported architectures now fail with an error instead of
falling back to non-arch-specific defaults.
Helm chart:
- Remove all deprecated env values (createRuntimeClasses,
createDefaultRuntimeClass, debug, shims, shims_*, defaultShim,
defaultShim_*, allowedHypervisorAnnotations, snapshotterHandlerMapping,
snapshotterHandlerMapping_*, agentHttpsProxy, agentNoProxy,
pullTypeMapping, pullTypeMapping_*, _experimentalSetupSnapshotter,
_experimentalForceGuestPull, _experimentalForceGuestPull_*).
- Remove backward compatibility code from _helpers.tpl that checked
for legacy env values.
- Remove legacy env.shims check from runtimeclasses.yaml.
- Remove CREATE_RUNTIMECLASSES and CREATE_DEFAULT_RUNTIMECLASS env
vars from kata-deploy.yaml and post-delete-job.yaml.
- Update RBAC to only include runtimeclasses get/patch permissions
(needed for NFD patching), removing create/delete/list/update/watch.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
- Replace generic errors in sandbox operations with typed SandboxError variants (InvalidContainerId, InitProcessNotFound, InvalidExecId).
- This enables the kata shim to handle specific failure cases differently.
Fixes#12120
Signed-off-by: Adeet Phanse <adeet.phanse@mongodb.com>
Add better error handling to runtime rs to handle when the sandbox itself is killed and recreated.
- Update the kill_process function to skip sending a signal when the process is stopped.
- Always set ProcessStatus::Stopped even when wait_process fails
- In state_process return synthetic state for sandbox container when using Sandbox API
Fixes#12120
Signed-off-by: Adeet Phanse <adeet.phanse@mongodb.com>
Align with other test logic - declare the KATA_HYPERVISOR in the
run bash script, then declare the RUNTIME_CLASS_NAME variable in
the bats files.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Now that we have a more restrictive resource policy for KBS, let
us start adopting it across all NVIDIA test cases. This policy was
previously introduced by the NVIDIA attestation test.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
It aims to upgrade rtnetlink to mitigate netlink log noise.
This commit upgrades the `rtnetlink` dependency (and corresponding
libraries like `netlink-packet-route`) to address excessive and
unnecessary netlink-related logging during sandbox startup.
Problem:
The previously used `rtnetlink v0.16` (depending on `netlink-proto
v0.11.3`) generates a high volume of DEBUG/INFO level netlink messages
during sandbox initialization. This noise:
1. Overloads the logging system, often leading to warnings like
"slog-async: logger dropped messages due to channel overflow."
2. Interferes with effective troubleshooting by distracting developers
from legitimate Kata errors.
Solution:
We upgrade to `rtnetlink v0.19` (and `netlink-proto v0.12`), as testing
confirms that the latest versions have correctly elevated the verbosity
of these netlink internal events to the TRACE level.
This change significantly enhances the log analysis experience by
suppressing unnecessary network-related logs during startup.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
With these changes, we create pod security policies when running
against NVIDIA TEE GPU handlers where AUTO_GENERATE_POLICY is set.
For the non-TEE GPU tests, the added functions bail out by design.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Following existing patterns, we adapt the common policy settings
for NVIDIA GPU CI platforms. For instance, for our CI runners, we
use containerd 2.x.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Enable auto-generate policy for qemu-nvidia-gpu-* if the user
didn't specify an AUTO_GENERATE_POLICY value.
Setting this in run_kubernetes_nv_tests.sh is too late as
gha-run.sh calls into run_tests, setup.sh, and then into
create_common_genpolicy_settings() where the rules.rego and
genpolicy-settings file are being copied to the right locations.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Add one valid test case with 2 GPUs with proper VFIO device
entries and CDI annotations.
Add seven test cases with invalid combinations of VFIO device
entries and CDI annotations.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Add rules for vfio passthrough GPUs. When creating the security
policy document, parse GPU resource limits and derive CDI
annotation patterns and VFIO device entries.
With various values for CDI annotations and device paths being
runtime-dependent, use regular expressions.
For now, this enables passthrough of NVIDIA GPUs, but the changes
are designed to allow for other VFIO device types.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Add the block device specific annotations which is dedicated within
runtime-rs for num_queues and queue_sie to the document to help
users set the two parameters.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit introduces the capability to dynamically configure
`queue_size` and `num_queues` parameters via Pod annotations.
Currently, `kata-runtime` allows for static configuration of
`queue_size` and `num_queues` for block devices through its config
file. However, a critical issue arises when a Pod is allocated fewer
CPU cores than the statically configured `num_queues` value. In such
scenarios, the Pod fails to start, leading to operational instability
and limiting flexibility in resource allocation.
To address this, this feature enables users to override the default
queue_size and num_queues parameters by specifying them in Pod
annotations.This allows for fine-grained control and dynamic adjustment
of these parameters based on the specific resource allocation of a Pod.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The runner is down for a few weeks. I may end up bringing in my personal
runner, but I'm not confident I can easily do this before the holidays,
thus I'm skipping the tests for now.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Set the attestation policy for GPU0 to affirming. This requires
the GPU, for instance, to have production properties, such as
properly signed VBIOS firmware.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
As some reasons that this CI is continuously failed, we'd like to
temporarily skip it for the s390x platform. And it will be enabled
when we addressed related issues.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
As the default enable_annotations in runtime-rs is different with
runtime-go, we should make it align with configuration in runtime-go.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit refactors the vCPU resource management within runtime's
`CpuResource` structure and related calculation logic to use
floating-point numbers (`f32`) instead of integers (`u32`).
This migration is necessary to fully support the fractional vCPU
allocation introduced in the `kata-types` library, ensuring better
precision in:
1.Allocation Tracking: `current_vcpu` now tracks the precise
fractional value (e.g., 1.5 vCPUs).
2.Resource Calculation: `calc_cpu_resources` now returns a precise
`f32` sum of container vCPU requests, including normalization logic
based on the maximum period, removing the previous integer rounding
steps in the calculation.
3.Hypervisor Interaction: The integer vCPU requirement for the
hypervisor remains, so `ceil()` is now explicitly applied only when
interacting with the hypervisor or agent APIs
(`do_update_cpu_resources`, `current_vcpu`, `online_cpu_mem`).
And key changes as below:
1. `CpuResource::current_vcpu` updated from `u32` to `f32`.
2. `calc_cpu_resources` return type changed from `u32` to `f32`.
3. CPU hotplug logic now uses `f32` for the target vCPU count and applies
4. `ceil()` before calling `hypervisor.resize_vcpu()`.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Refactors `LinuxContainerCpuResources` and `LinuxSandboxCpuResources`
to track calculated vCPU allocation using `f64` (fractional float)
instead of `u64` (milliseconds).
This ensures more precise resource calculation (`quota / period`) and
aggregation by avoiding rounding errors inherent in millisecond-based
integer tracking.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit updates the non-TEE tests to disable two specific test
cases: `k8s-number-cpus.bats` and `k8s-sandbox-vcpus-allocation.bats`.
These tests are designed to cover CPU elasticity/dynamic scaling
capabilities. In the non-TEE scenario, we are enforcing the disabling of
this capability by setting the default configuration to
`static_sandbox_resource_mgmt=true`.
Although the tests currently pass, allowing them to run is logically
inconsistent with the intended non-TEE configuration. Therefore, we are
disabling them for all non-TEE runtimes, specifically targeting:
- `qemu-coco-dev`
- `qemu-coco-dev-runtime-rs`
This change ensures that our non-TEE CI accurately reflects the static
resource management policy and prevents misleading test results.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
As runtime-rs doesn't support block device hotplug in s390 arch,
with this fact, we just disable or skip the test when it is the
s390.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
To support such feature, the item in Makefile should be enabled,
and it can be set true when make build, just like this:
`DEFSTATICRESOURCEMGMT_QEMU := false`
When users don't want this feature, they can set it with true via
the configuration.toml.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Enable the cpu hotplug tests within the k8s-number-cpus.bats for both
cloud-hypervisor and qemu-runtime-rs.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
We have support cpu hotplug features within dragonball and clh, this
commit is to enable the test within the CI.
Fixes: #8660
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
As previous failure within the case, we choose to skip it, but now
the cpu hotplug has been corrected, and it's time to re-enable it.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Adding additional cases for the IOMMUFDID method to check for
non-IOMMUFD paths are passed. The method should do the right
thing.
Signed-off-by: LandonTClipp <11232769+LandonTClipp@users.noreply.github.com>
Logging the QMP commands gives us a lot of flexibility to
troubleshoot issues with what is being sent to QEMU.
Signed-off-by: LandonTClipp <11232769+LandonTClipp@users.noreply.github.com>
An import cycle was introduced because of a mutual need
for the constant that describes the prefix of IOMMUFD files.
We need to extract this out into a higher-level package.
Signed-off-by: LandonTClipp <11232769+LandonTClipp@users.noreply.github.com>
The QMP commands sent to QEMU did not properly set up
IOMMUFD objects in the codepath that handles VFIO device
hot-plugging. This is mainly relevant in the Kubernetes
use-case where the VFIO devices are not available when
QEMU is first launched.
Signed-off-by: LandonTClipp <11232769+LandonTClipp@users.noreply.github.com>
The function assumes that the runner is a Ubuntu machine, which so far
has been true as part of our CI.
However, the new ARM runner is running on Debian, and those mirror
additions would simply break.
With this in mind, for any distro that's not ubuntu, let's just make
sure to inform the owner of the system to have bats already installed as
part of the environment provided.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
This reverts commit 5a81b010f2, as we now
have all the infrastructure properly set up as part of our CI node.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Remove the existing containerd guest pull stability tests workflow
as we're going to rebuild all the VMs used for testing and introduce
new, more focused stability tests for nydus-snapshotter.
The new tests will be added soon, as part of another PR.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Now that we've bumped to QEMU 10.2.0-rc1, we can take advantage of a fix
that's present there, which fixes the double memory allocation for the
cases where GPUs are being cold-plugged.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
We've made the pods require a ridiculous amount of memory, just for the
sake of getting them running.
Now that those are running, tests are passing, CI is required, let's
work to lower the amount of mmemory needed as everything else is working
as expected.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Clean-up shellcheck warnings:
SC2030 (info): Modification of cmd_out is local (to subshell caused by (..) group).
SC2031 (info): cmd_out was modified in a subshell. That change might be lost.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Clean-up shellcheck warnings:
SC2250 (style): Prefer putting braces around variable references even
when not strictly required.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Let's add a simple backup and restore logic for the CDI configuration
file nvidia.com-pgpu.yaml in the k8s-nvidia-*.bats and
k8s-confidential-attestation.bats test files.
Althought not optimal, this is a temporary workaround needed until
NVIDIA releases what's needed for the GPU Operator to properly deal with
cold plugged devices for the Confidential Containers cases, which is
work in progress right now.
After that's released, we can revert/drop this patch.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Let's bump experimental {tdx,snp} QEMU to the tags created Today in the
Confidential Containers repo, which match with QEMU 10.2.0-rc1.
This bump is specially beneficial for us, as we can get rid of QEMU's
double memory allocation when **cold plugging** a GPU.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
If the sandbox has cold-plugged a IOMMUFD device but the
device-plugins sends us a /dev/vfio/<NUM> device we need to
check if the IOMMUFD device and the VFIO device are the same
We have the sibling.BDF we now need to extract the BDF of the
devPath that is either /dev/vfio/<NUM> or /dev/vfio/devices/vfio<NUM>
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Bump the github.com/sirupsen/logrus version to 1.9.3
across our components where it is back-level to bring us
up-to-date and resolve high severity CVE-2025-65637
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Add the attestation bats test case to the NVIDIA CI and provide a
second pod manifest for the attestation test with a GPU. This will
enable composite attestation in a subsequent step.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Bump to pull in a fix for composite attestation with GPUs. The new
commit ID corresponds to the fix (change for default GPU policy),
currently being the top commit of the main branch.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
This brings two fixes:
- use the test_key variable to check against the aatest value.
- properly check the run command invocation (run w/o bash does not
seem to like the pipe which leads to ALWAYS evaluating the
status result to 1. With this, the deny-all test would ALWAYS
succeed regardless of whether aatest was actually returned or not.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
When running these tests repeatedly locally, the default policy is not
being reset after the test completes, then subsequent runs fail.
Similar to k8s-sealed-secrets.bats, we set the default policy in an if
condition.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
This allows setting a GPU0 resource policy, enabling GPU
attestation tests to not use the default resource policy.
For now, the policy requires attestation's ear status to
not be contraindicated. In a future change we will require
this to be affirming once our CI runners' vBIOS version is
properly configured.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
This enables attestation tests to figure out whether composite
attestation with a GPU can be executed.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Add the NVIDIA TEE hypervisors. With this, attestation tests can be run
against the NVIDIA handlers, for instance.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
This reverts commit e4a13b9a4a, as it
caused some issues with the GPU workflows.
Reverting it is better, as it unblocks other PRs.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
vfio-ap passthrough has been introduced for runtime-rs,
requiring that the existing test verify this new functionality.
This commit adds:
- containerd config specific to runtime-rs
- extensions to the existing test functions to cover vfio-ap
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The following have been made for the enablement:
1. Make `MediatedPci` and `MediatedAp` in `VfioDeviceType`
2. Make HostDevice without BDF for `MediatedAp`
3. Add `CCW` to VFioBusMode and set it to VfioConfig as `bus_type`
4. Return `vfio-ap` driver type for `CCW` bus type
5. Set `bus_mode` for `VfioDevice` based on `bus_type`
6. Set `vfio-ap` to the agent device's `field_type`
7. Prepare a different argument for `vfio-ap` for QMP command
8. Set None to all PCI relevant fields
Please keep in mind that `vfio-ap` does not belong to any
types of port togologies like PCI (e.g., root or switch)
because devices on s390x are controlled by CCW.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Until now, we relied on `VMROOTFSDRIVER` to determine
whether a system uses a native CCW bus.
However, this method is not canonical and can be error-prone
depending on the configuration.
This commit introduces a new function that checks
for the presence of CCW bus infrastructure in sysfs
and verifies that native mainframe drivers are available.
It replaces all previous uses of the old detection method.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Add the small and normal variants of the qemu-runtime-rs
tests to the required-tests list now that they are stable.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
An oci-spec can be passed to the runtime without annotations
(e.g., `ctr run`). In this case, runtime panics with:
```
src/runtime-rs/crates/runtimes/src/manager.rs:391: called `Option::unwrap()` on a `None` value
```
This commit checks if the annotation is None, and instantiates
the hashmap as an empty map if it is missing. It also adds a None
check for `netns`.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Currently, the protection device configuration is constructed
automatically even if `confidential_guest` is not set.
This commit puts a condition to check the flag and allows the
construction accordingly.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Updates to the shim-v2 build and the binaries.sh script.
Makeing sure that both variants "confidential" AND
"nvidia-gpu-confidential" are handled.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Create an initial version of our toolchain policy as agreed in
Architecture Committee meetings and the PTG
Fixes: #9841
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
As tags are mutable and digests are not, lets pin our image
by digest to give our CI a better chance of stability
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Swap out the hard-coded nginx registry and verisons for reading
the test image details for version.yaml
which can also ensure that the quay.io mirror is used
rather than the docker hub versions which can hit pull limits
- Try setting imagePullPoliycy Always to fix issues with the arm CI
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Using make tarball targets for tools locally, binaries may exist
for both debug and release builds. In this case, cryptic errors
are shown as we try to install multiple binaries.
This change require exactly one binary to be found and errors out
in other cases.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
When tests regress, the CI wait time can increase significantly
with the current kubectly_retry attempt logic. Thus, align with
other tests and remove kubectl_retry invocations. Instead, rely on
proper timeouts.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
SEV-SNP machine is failing due to nydus not being deployed in the
machine.
We cannot easily contact the maintainers due to the US Holidays, and I
think this should become a criteria for a machine not be added as
required again (different regions coverage).
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
So far we've only been building the initrd for the nvidia rootfs.
However, we're also interested on having the image beind used for a few
use-cases.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
We hit a case that gatekeeper was failing due to thinking the WIP check
had failed, but since it ran the PR had been edited to remove that from
the title. We should listen to edits and unlabels of the PR to ensure that
gatekeeper doesn't get outdated in situations like this.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
When using the multiInstallSuffix we must be cautelous on using the shim
name, as qemu-nvidia-gpu* doesn't actually have a matching QEMU itself,
but should rather be mapped to:
qemu-nvidia-gpu -> qemu
qemu-nvidia-gpu-snp -> qemu-snp-experimental
qemu-nvidia-gpu-tdx -> qemu-tdx-experimental
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Fixes: #12123
`include` in #12069, introduced to choose a different runner
based on component, leads to another set of redundant jobs
where `matrix.command` is empty.
This commit gets back to the `runs-on` solution, but makes
the condition human-readable.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Containerd configuration syntax (`config.toml`) varies across versions,
requiring per-version logic for fields like `runtime`.
However, testing confirms that containerd LTS (1.7.x) and newer
versions fully support the v3 schema for the nydus remote snapshotter.
This commit changes the previous containerd v1 settings in `config.toml`.
Instead, it introduces a unified v3-style configuration for nydus, which
can be vailid for lts and active containerds.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
In the CoCo tests jobs @wainersm create a report tests step
that summarises the jobs, so they are easier to understand and
get results for. This is very useful, so let's roll it out to all the bats
tests.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
In order to have a better way to set things up using a toml editor, we
should take the containerd approach and actually have everything
uncommnted. This will help us to unify how we deal with such values in
the future from the kata-deploy POV.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
We need to ensure that we do not blindly append nor blindly override the
kernel parameters set by default, but rather modify the values in case
they exist, and append in case they do not.
Now we're actually making golang and rust runtime behave the same, as so
far they were behaving differently, each version wrong in its own way.
:-p.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
QEMU netdev_add QMP command requires the 'mq' (multi-queue) argument
to be of boolean type (`true` / `false`). In runtime-rs the virtio-net
device hotplug logic currently passes a string value (e.g. "on"/"off"),
which causes QEMU to reject the command:
```
Invalid parameter type for 'mq', expected: boolean
```
This patch modifies `hotplug_network_device` to insert 'mq' as a proper
boolean value of `true . This fixes sandbox startup failures when
multi-queue is enabled.
Fixes#12136
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Allow users to build the Kata Agent using INIT_DATA=no to disable the
detect_initdata_device() code loop and associated debug log output.
Future additional improvements related to Init Data are tracked by #11532.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
On 69c4fc4e76, I've mistakenly changed the
nvidia-gpu podOverhead while I should only have changed the TEE
nvidia-gpu ones.
Let's move it back to its original value.
Reported-by: Joji Mekkattuparamban <jojim@nvidia.com>
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
This settings is not needed anymore with Ubuntu 25.10
and the newest QEMU releases for TDX by Ubuntu.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
With issue 11777 being resolved, this commit enables openvpn
policy testing. The remaining work on the security policy
required to successfully run this test case was to enable UDP
ports for Service kinds and to use the mount path's last component
instead of the volume name to construct the expected storage
source path.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Use the mount path's last component instead of the volume name to
construct the expected storage source path. Example: Name of a
volumeMount is 'openvpn-config' and its mountPath is
'/etc/openvpn/'. Without this change, we use 'openvpn-config' to
calculate the expected storage source path. However, we need to
use 'openvpn', because the shim uses the basename of the
destination path as the source suffix and not the volume name.
For reference, see 'fs_hsare_linux.go"'s 'ShareFile' function
where the filename variable uses 'filepath.Base(m.Destionation))'.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
For Service kinds using the UDP protocol as port. An example is
the openvpn-server-service.yaml file part of the openvpn CI test.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
We've added logic to properly do the book keeping of the TEE keys when
using NFD **AND** creating the runtime classes. However, we need to also
take into consideration the case where the runtimeclasses are being
created by the helm template, and in that case we just update what helm
has deployed.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Remove the nvrc.smi.srs=1 parameter from the kernel command line.
In CC use cases, the attestation agent is expected to set the GPU
ready state. For the CUDA vectorAdd case where attestation agent
is not being used, we set the ready state by adding the kernel
command line parameter through an annotation.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Add an allow-all policy for the CC GPU tests and ensure the init-data
device is being created (hypervisor annotations).
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
The add_allow_all_policy_to_yaml in tests_common.sh needs some
improvements so that this function can support pod manifests with
different resource kinds. For now, moving the Secret definition
to the bottom so that we can create a default policy for the Pod.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
The qemu-nvida-gpu handlers should not cause is_aks_cluster to
return 1. Otherwise, CI logic will assume these hypervisors run on
AKS hosts, see the following message in CI w/o this change:
INFO: Adapting common policy settings for AKS Hosts
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
We currently start a pod that does a `wget` to the KBS address, and
fails after 5 seconds.
By the time it fails and reports back, we can see that KBS is actually
running, but the workflow failed as the checker failed. :-/
Let's give it more time for the KBS to show up, and the flakeness should
go away.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Utilize Kubelet's Pod Resource API to determine device allocations
for the Pod during sandbox creation. Use CDI files to translate the device
IDs to corresponding device paths and perform device injection.
Fixes#12009
Signed-off-by: Joji Mekkattuparamban <jojim@nvidia.com>
Use the pod name variable so that kubectl wait finds the pod. Currently,
kubectl waits for nvidia-nim-llama-3-2-nv-embedqa-1b-v2, not for
nvidia-nim-llama-3-2-nv-embedqa-1b-v2-tee
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Introduce a new devkit parameter which will produce a rootfs
without chisselling. This results in a larger rootfs with various
packages and binaries being included, for instance, enabling the
use of the debug console.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
There are rust packages being cloned and built inside
tools/packaging/kata-deploy/local-build/build folder, which may mislead
those packages to think they are part of the kata root workspace.
Exclude the directory to avoid that.
Reported-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
The person who introduced the check, someone named Fabiano Fidêncio,
forgot a `$` in a variable assignment.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
The snp CI has not been required for a while and has recently been
broken, so comment it out from the list of required jobs.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The run-nydus tests are not stable and blocking PRs, so make them
non-required temporarily until they can be looked at
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Enable auto-generate policy on cbl-mariner Hosts for
qemu-coco-dev-runtime-rs if the user didn't specify an
AUTO_GENERATE_POLICY value.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
We will re-enable this one later on once the changes to properly cold
plug multi GPUs are merged.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Let's just move the podOverhead to a gigantic value, as we do need pod
snadboxes as big as that, and we've noticed QEMU being OOM killed with
smaller overheads.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Those need to pull the models inside the guest, and the guest has 50% of
its memory "allowed" to be used as tmpfs, so, we gotta usa the RAM that
we have.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Yes, we're dealing with a combination of large images and image-rs
concurrent image layers being not optimal.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
We cannot use the same format used for docker, as it includes username
and password, while what's expected when using Trustee does not.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Now that we've bumped Trustee to a version that supports the NVIDIA
remote verifier, let's re-enable the tests.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Right now we have only been passing the env var to the deployment
script, but we really need to pass it to the tests script as well.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Try and reduce the page limit of each job request to avoid the chances of
us tripping over github's 10s api limit.
All credit to @burgerdev for the investigation and suggestion!
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Add the related block queue_size and num_queues in volumes based on
block devices, This very important for IO performance.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Previous Clh's settings with disk queue_size and num_queues are
hardcodes, they should be configurable with user-defined values.
This commit is to address such issue via passing these settings.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Usually, we pass the related block config via BlockConfig, and to reach
the goal of user-friendly setting queue_size and num_queues for users,
the queue_size and num_queues are introduced in BlockConfig.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Add two fields of queue_size and num_queues in BlockDeviceInfo to allow
users to set the related items via configurations
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Add related items for block device queue size and num queues in
configurations. And users can set the related items by configurations.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The current implementation causes issues with the Agent Policy
nontee CI tests, as Kata-Agent does not allow any configuration
for `count(Linux.Resources.Devices) == 0`.
This commit ensures that Linux.Resources.Devices, including all its
values, is completely cleared from the OCI Runtime Specification before
being passed to the Kata-Agent.
This addresses the CI failure by enforcing the required empty state for
the Devices cgroup configuration.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Previously, CopyFile implementation attempted to reuse existing guest
paths for subsequent containers within the same Pod. This prevented
correct bind mounting of shared configurations (e.g., ConfigMaps,
Service Accounts) into the later containers within a multi-containers
pod, as they lacked their own allocated guest path.
This commit modifies the logic to create a unique guest path for every
container that requires file propagation.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Crates with no workspace setup would think themselves are in the root
workspace, which our root workspace is not ready for them. Excluding
them for now.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Add Cargo.toml at repo root, use this root workspace for as many as
possible Rust components of Kata Containers. This would enable us to
share a common Cargo.lock file, and reduce the noise from dependabot.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Similar to #12075, bump-backtrace to 0.3.76 to remove the dependency
on adler, which is unmaintained - contributing to mitigating RUSTSEC-2025-0056
As a side effect this brought in loads of other crate changes, which I think are due
to it bumping the local dependencies that this package builds on.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Similar to #12075, bump-backtrace to remove the dependency
on adler, which is unmaintained - contributing to mitigating RUSTSEC-2025-0056
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Similar to #12075, bump-backtrace to remove the dependency
on adler, which is unmaintained - contributing to mitigating RUSTSEC-2025-0056
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Similar to #12075, bump-backtrace to remove the dependency
on adler, which is unmaintained - contributing to mitigating RUSTSEC-2025-0056
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Similar to #12075, bump flate2 and backtrace to remove the dependency
on adler, which is unmaintained - contributing to mitigating RUSTSEC-2025-0056
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Similar to #12075, bump flate2 and backtrace to remove the dependency
on adler, which is unmaintained - contributing to mitigating RUSTSEC-2025-0056
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Since the network device hotplug is an asynchronous operation,
it's possible that the hotplug operation had returned, but
the network device hasn't ready in guest, thus it's better to
retry on this operation to wait until the device ready in guest.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
This makes the user experience better, as the admin can deploy Kata
Containers without having to download / set up any additional file.
Of course, if the admin wants something more specific, examples are
provided.
Tests and documentation are updated to reflect this change.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
The current format of genpolicy request logs looks a bit like JSON, but
it does not parse out of the box and needs post-processing with sed, for
example.
This commit changes the log format to jsonlines[1], which is basically
newline-delimited compact JSON values. Compared to standard JSON, this
allows streaming output. The resulting file can be converted and
processed programmatically, for example with `jq -s`.
The fields are also adjusted to match the field names of TestRequest, so
that the logged requests can be used immediately in tests.
[1]: https://jsonlines.org/
Signed-off-by: Markus Rudy <mr@edgeless.systems>
This should allow keeping future diffs minimal.
The files were formatted with `jq -S`, which should be used after future
updates to the test case files.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
Storing the request type outside the request object has two benefits:
* The request JSON passed to the Rego engine matches more closely what
would be passed by the agent (no `type` field).
* If we want to update the requests, it's easier to insert them into a
dedicated field, rather than inserting them and amending the type
field.
This is a first step towards programmatic updates of testcase files.
This commit also adds the 'Request' suffix to the test case enum, such
that we can use the 'ep' input for allow_request directly.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
Add comments to make the "EnableIOThreads" flag as a switch
for virtio-blk(based on IndepIOThreads) driver.
Signed-off-by: zhangchen.kidd <zhangchen.kidd@jd.com>
Make hotplug virtio-blk device attach to Independent IOThread 0 as default
when enabled the EnableIOThreads and IndepIOThreads.
Signed-off-by: zhangchen.kidd <zhangchen.kidd@jd.com>
Qemu already support the device_add with iothread args.
Make KATA have ability to hotplug PCI device with IOThreads.
Currently, just support QEMU as the hypervisor, not sure it
works for stratovirt.
Signed-off-by: zhangchen.kidd <zhangchen.kidd@jd.com>
Make the original virtio-scsi iothread and the new independent
iothread to a dedicated method for handing the related logics.
Signed-off-by: zhangchen.kidd <zhangchen.kidd@jd.com>
Introduce independent IOThread framework for Kata container.
What is the indep_iothreads:
This new feature introduce a way to pre-alloc IOThreads
for QEMU hypervisor (maybe other hypervisor can support too).
Independent IOThreads enables IO to be processed in a separate thread.
To generally improve the performance of each module, avoid them
running in the QEMU main loop.
Why need indep_iothreads:
In Kata container implementation, many devices based on hotplug
mechanism. The real workload container may not sync the same
lifecycle with the VM. It may require to hotplug/unplug new disks
or other devices without destroying the VM. So we can keep the
IOThread with the VM as a IOThread pool(some devices need multi iothreads
for performance like virtio-blk vq-mapping), the hotplug devices
can attach/detach with the IOThread according to business needs.
At the same time, QEMU also support the "x-blockdev-set-iothread"
to change iothreads(but it need stop VM for data secure).
Current QEMU have many devices support iothread, virtio-blk,
virtio-scsi, virtio-balloon, monitor, colo-compare...etc...
How it works:
Add new item in hypervisor struct named "indep_iothreads" in toml.
The default value is 0, it reused the original "enable_iothreads" as
the switch. If the "indep_iothreads" != 0 and "enable_iothreads" = true
it will add qmp object -iothread indepIOThreadsPrefix_No when VM startup.
The first user is the virtio-blk, it will attach the indep_iothread_0
as default when enable iothread for virtio-blk.
Thanks
Chen
Signed-off-by: zhangchen.kidd <zhangchen.kidd@jd.com>
In commit 1f95d9401b
runtime-rs: change representation of default_vcpus from i32 to f32,
When the vCPU number is less than 1.0, directly converting an integer to
a floating-point number will automatically convert it to 0. Therefore,
it needs to be rounded up before converting it back to an integer.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Update the `cpath` variable in the policy template to support the
optional `/passthrough` subpath used by runtime-rs. This ensures
that mount source path validation works correctly for both runtime
implementations.
By changing `cpath` to include the `(?:/passthrough)?` regular
expression fragment, we make the `/passthrough` segment optional.
The updated `cpath`:
`/run/kata-containers/shared/containers(?:/passthrough)?`
This single regex pattern now correctly matches both:
1.`/run/kata-containers/shared/containers/<sandbox-id>/...`
(runtime-go)
2.`/run/kata-containers/shared/containers/passthrough/<sandbox-id>/...`
(runtime-rs)
This elegantly resolves the compatibility issue without needing to add
separate or conditional logic to the policy rules, making the policy
more robust and maintainable.
Fixes: #12063
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Add three example values files to make it easier for users to try out
different Kata Containers configurations:
- try-kata.values.yaml: Enables all available shims
- try-kata-tee.values.yaml: Enables only TEE/confidential computing shims
- try-kata-nvidia-gpu.values.yaml: Enables only NVIDIA GPU shims
These files use the new structured configuration format and serve as
ready-to-use examples for common deployment scenarios.
Also update the README.md to document these example files and how to use them.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Update the helm_helper function in gha-run-k8s-common.sh to use the
new structured configuration format instead of the legacy env.* format.
All possible settings have been migrated to the structured format:
- HELM_DEBUG now sets root-level 'debug' boolean
- HELM_SHIMS now enables shims in structured format with automatic
architecture detection based on shim name
- HELM_DEFAULT_SHIM now sets per-architecture defaultShim mapping
- HELM_EXPERIMENTAL_SETUP_SNAPSHOTTER now sets snapshotter.setup array
- HELM_ALLOWED_HYPERVISOR_ANNOTATIONS now sets per-shim allowedHypervisorAnnotations
- HELM_SNAPSHOTTER_HANDLER_MAPPING now sets per-shim containerd.snapshotter
- HELM_AGENT_HTTPS_PROXY and HELM_AGENT_NO_PROXY now set per-shim agent proxy settings
- HELM_PULL_TYPE_MAPPING now sets per-shim forceGuestPull/guestPull settings
- HELM_EXPERIMENTAL_FORCE_GUEST_PULL now sets per-shim forceGuestPull/guestPull
The test helper automatically determines supported architectures for
each shim (e.g., qemu-se supports s390x, qemu-cca supports arm64,
qemu-snp/qemu-tdx support amd64, etc.) and applies per-shim settings
to the appropriate shims based on HELM_SHIMS.
Only HELM_HOST_OS remains in legacy env.* format as it doesn't have
a structured equivalent yet.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Add comprehensive documentation for the new structured configuration
format, including:
- Migration guide from legacy env.* format
- List of deprecated fields with removal timeline (2 releases)
- Examples of the new structured format
- Explanation of key benefits
- Backward compatibility notes
The documentation makes it clear that the legacy format is deprecated
but will continue to work during the transition period.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
This commit adds backward compatibility support to ensure existing
configurations using the legacy env.* format continue to work.
The helper functions now check for legacy env.* values first, and
only fall back to the new structured format if legacy values are
not set. This allows for gradual migration without breaking
existing deployments.
Backward compatibility is maintained for:
- env.shims, env.shims_* (per architecture)
- env.defaultShim, env.defaultShim_* (per architecture)
- env.allowedHypervisorAnnotations
- env.snapshotterHandlerMapping_* (per architecture)
- env.pullTypeMapping_* (per architecture)
- env.agentHttpsProxy, env.agentNoProxy
- env._experimentalSetupSnapshotter
- env._experimentalForceGuestPull_* (per architecture)
- env.debug
Legacy env vars (SHIMS, DEFAULT_SHIM, etc.) are still set in the
DaemonSet when using the old format to maintain full compatibility.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
This commit introduces a new structured configuration format for
configuring Kata Containers shims in the Helm chart. The new format
provides:
- Per-shim configuration with enabled/supportedArches
- Per-shim snapshotter, guest pull, and agent proxy settings
- Architecture-aware default shim configuration
- Root-level debug and snapshotter setup configuration
All shims are disabled by default and must be explicitly enabled.
This provides better type safety and clearer organization compared
to the legacy env.* string-based format.
The templates are updated to use the new structure exclusively.
Backward compatibility will be added in a follow-up commit.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
As the some of the global vars can be empty, we should actually check
their _FOR_ARCH version instead.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
As we're making the values.yaml more user friendly, we actually have to
handle the https_proxy and no_proxy entries per shim, instead of having
this globally available, as this will only affect images being pulled
inside the guest (as in, when using TEE variations of the shims).
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Adds a practical set of kernel config used by docker-in-docker and kind
for network bridging and filtering. It also includes the matching IPv6
support to allow tools like kind that require IPv6 network policies to
work out of the box.
This support includes:
- nftables reject and filtering support for inet/ipv4/ipv6
- Bridge filtering for container-to-container traffic
- IPv6 NAT, filtering, and packet matching rules for network policies
- VXLAN and IPsec crypto support for network tunneling
- TMPFS POSIX ACL support for filesystem permissions
The configs are organized across fragment files:
- common/fs.conf: TMPFS ACL support
- common/crypto.conf: IPsec/VXLAN crypto algorithms
- common/network.conf: VXLAN, IPsec ESP, nftables bridge/ARP/netdev
- common/netfilter.conf: IPv6 netfilter stack and nftables advanced features
Fixes: #11886
Signed-off-by: Simon Kaegi <simon.kaegi@gmail.com>
Re-enable AUTO_GENERATE_POLICY for coco-dev Hosts, unless PULL_TYPE is
"experimental-force-guest-pull", or the caller specified a different
value for AUTO_GENERATE_POLICY.
Auto-generated Policy has been disabled accidentally and recently for
these Hosts, by a GHA workflow change.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Don't skip anymore parsing the pause container image when using the
recently updated AKS pause container handling - i.e. when
pause_container_id_policy == "v2".
This was the easiest CI fix for guest pull + new AKS given the *current*
tests. When adding *new* UID/GID/AdditionalGids tests in the future,
these workarounds might need additional updates.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The update removes the deprecated adler crate from our dependencies. In
addition, we're switching to the default backend (miniz_oxide), which is
a pure Rust implementation and thus much more portable. The performance
impact is negligible, because flate2 is only used for initdata
decompression, which is limited to a couple of MiB anyway.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
The update removes the deprecated adler crate from our dependencies. In
addition, we're switching to the default backend (miniz_oxide), which is
a pure Rust implementation and thus much more portable. The performance
impact is acceptable for a developer tool.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
The github API suggestions that `Authorization: Bearer <YOUR-TOKEN>`
is the way to set the auth token, but it also mentioned that `token`
should work, so it's unclear if this will help much, but it shouldn't harm.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The formatting wasn't quite right, so the `qemu-coco-dev-runtime-rs`
hypervisor wasn't skipping this test
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Introduce a flag `DEFSTATICRESOURCEMGMT_COCO` for setting static sandbox
resource management with default true. And then set it to the item of
`static_sandbox_resource_mgmt` in configuration.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The teardown_common will print the description of the running pods, kill
them all and print the system's syslogs afterwards.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
When testing this branch, on several occasions the Delete
AKS cluster step has hung for multiple hours, so add a timeout
to prevent this.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Since it aligns with the create_container_timeout definition in
runtime-go, we need to set the value in configuration.toml in seconds,
not milliseconds. We must also convert it to milliseconds when the
configuration is loaded for request_timeout_ms.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Create non-tee runtime class for runtime-rs qemu CoCo development
without requiring TEE hardware. Based on the qemu-runtime-rs
config, but with updated guest image, kernel and shared_fs
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The new environment of Power runners for agent checks is causing two test case failures
w.r.to selinux and inode which needs further understanding and is mostly an issue
due to environemnt change and not to do with the agent.
Fall back to running agent checks on original ppc64le self hosted runners.
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
As the arm 22.04 runner isn't working at the moment, let's test the
24.04 version to see if that is better.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The fact that we were not explicitly setting the VMM was leading to us
testing with the default runtime class (qemu). :-/
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
By doing this, the ones interested on RISC-V support can still have a
ood visibility of its state, without the extra noise in our CI.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
We have had those tests broken for months. It's time to get rid of
those.
NOTE that we could easily revert this commit and re-add those tests as
soon as we find someone to maintain and be responsible for such
integration.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
As stratovirt CI was removed in #12006 we should remove the
jobs from required.
Also the docker tests have been commented out for months, and
we are considering removing them, so clean this file up.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
sometimes it's hard to enumerate all blacklisted namespaces, lets add a
regular expression based only filter to allow specifying namespaces that
should be mutated.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Previous set for the Mount.type with `bind` is wrong, and for local
storage, the type of Mount should be `local`.
This commit aims to correct the type with "local".
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
As the disable_guest_empty_dir order is wrong which causes
the bool value is not correct and it got a wrong result.
This commit aims to correct the parameters order.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This is a bump pre-release, which brings several fixes and some
improvements related to initData, and NVIDIA's remote verifier.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
The test case designed to verify policy failures due to an "unexpected
capability" was misconfigured. It was using "CAP_SYS_CHROOT" as the
unexpected capability to be added.
This configuration was flawed for two main reasons:
1.Incorrect Syntax: Kubernetes Pod specs expect capability names without
the "CAP_" prefix (e.g., "SYS_CHROOT", not "CAP_SYS_CHROOT").
This made the test case's premise incorrect from a K8s API perspective.
2.Part of Default Set: "SYS_CHROOT" is already included in the
`default_caps` list for a standard container. Therefore, adding it would
not trigger a policy violation, defeating the purpose of the
"unexpected capability" test.
Furthermore, a related issue was observed where a malformed capability
like "CAP_CAP_SYS_CHROOT" was being generated, causing parsing failures
in the `oci-spec-rs` library. This was a symptom of incorrect string
manipulation when handling capabilities.
This commit corrects the test by selecting "SYS_NICE" as the unexpected
capability. "SYS_NICE" is a more suitable choice because:
- It is a valid Linux capability.
- It is relatively harmless.
- It is **not** part of the default capability set defined in
`genpolicy-settings.json`.
By using "SYS_NICE", the test now accurately simulates a scenario where
a Pod requests a legitimate but non-default capability, which the policy
(generated from a baseline Pod without this capability) should correctly
reject. This change fixes the test's logic and also resolves the
downstream `oci-spec-rs` parsing error by ensuring only valid capability
names are processed.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Detected a format mismatch in OCI Spec Capabilities fields between
`runtime-rs` (no `CAP_` prefix) and `runtime-go` (with `CAP_` prefix).
This introduces a normalization of caps in match_caps(p_caps, i_caps).
This ensures robust and consistent processing of Capabilities regardless
of whether the OCI Spec originates from `runtime-rs` or `runtime-go`.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Currently, the initdata module only detects virtio-blk devices
(/dev/vd*) when searching for the initdata block device. However,
when using virtio-scsi, the devices appear as /dev/sd* in the
guest, causing the initdata detection to fail.
This commit extends the device detection logic to support both
device types:
- virtio-blk devices: /dev/vda, /dev/vdb, etc.
- virtio-scsi devices: /dev/sda, /dev/sdb, etc.
This commits aims to address issue of theinitdata device not being
found when using virtio-scsi
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Thankfully there's only one piece that's still SNP specific (for the
supported TEEs). Let's adjust it so we can have an easy and smooth
execution when adding a TDX CI machine.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
There are several changes needed in order to get this test working with
CC, and yet we still are skipping it.
Basically, we need to:
* Pull an authenticated image inside the guest, which requires:
* Using Trustee to release the credential
* We still depend on a PR to be merged on Trustee side
* https://github.com/confidential-containers/trustee/pull/1035
* We still depend on a Trustee bump (including the PR above) on our
side
Apart from those changes, I ended up "duplicating" the tests by adding a
"-tee" version of those, which already have:
* The proper kbs annotations set up
* Dropped host mounts
* Increases the memory needed
Last but not least, as "bats" probably means "being a terrible script",
I had to re-arrange a few things otherwise the tests would not even run
due to bats-isms that I am sincerely not able to pin-point.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
We added the tests using virtio-9p as we knew it'd require incremental
changes to be able to use any kind of guest-pull method.
Now, as in the coming commits we'll be actually ensuring that guest-pull
works and is in use, we can enforce the experimental_force_guest_pull
usage for the nvidia cases.
Note: We're using experimental_force_guest_pull instead of
nydus-snapshotter due to stability concerns with the snapshotter.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
It was just missed when adding those configurations.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
It takes either a shim name or "", but we were treating this (thankfully
only in this specific file) as a boolean.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Adjust output to the setup_file and teardown_file behavior.
With this, we will be able to observe relevant logging rather than
adding to the output variable.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Previous commit enabled getting the physical address reduction from
processor but just stored it for later use. This commit adds handling
of the value to ProtectionDevice and enables the QEMU driver to use it.
Signed-off-by: Pavel Mores <pmores@redhat.com>
An implementation of cbitpos acquisition is supplied that was missing
so far. We also get the physical address reduction value from the same
source (CPUID Fn8000_001f function). This has been hardcoded at 1 so far,
following the Go runtime example, but it's better to get it from the
processor.
Signed-off-by: Pavel Mores <pmores@redhat.com>
- version.rs gets generated from version.rs.in
- version.rs.in contains values read from VERSION
- so version.rs (and maybe other Agent files too) must be
re-generated when the VERSION file changes
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The new image reference has changed to mcr.microsoft.com/oss/v2/kubernetes/pause:3.6
from mcr.microsoft.com/oss/kubernetes/pause:3.6.
The new image uses by default UID=0, GID=0 while the older. The older image had:
UID=65535, GID=65535.
There is a new pause_container_id_policy field in genpolicy-settings.json, informing
genpolicy about the way AdditionalGids gets updated - "v1" for the older behavior
and "v2" for the newer AKS version:
- When using v1, the default value of AdditionalGids is {65535}.
- When using v2, the default value of AdditionalGids is {}.
UID=65535 and GID=65535 are still hard-coded by default in genpolicy-settings.json.
We might be able to remove/ignore these fields in the future, if we'll stop relying
on policy::KataSpec::get_process_fields to use these fields.
A new CI function adapt_common_policy_settings_for_aks() changes the pause container
UID, GID, pause_container_id_policy, and image ref settings values when testing on
AKS Hosts - i.e., when testing coco-dev or mariner Hosts.
The genpolicy workarounds for the unexpected behavior with guest pull enabled have
been improved to use the current container's GID instead of hard-coding GID=0 as the
guest pull default. Also, AdditionalGids gets updated when the current container's GID is
changing, instead of always changing the AdditionalGids at the very end of
policy::AgentPolicy::get_container_process(), when the relevant evolution of the GID
value was no longer available.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Make it easier to understand the source of the UID/GID/AdditionalGids
values from the container in the auto-generated policy.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Parallelize busybox builds to build a bit faster and create the
build directory prior to Docker execution, which on my
environment, helps with permission issues when building busybox
without the kata-containers/build directory existing beforehand.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Correct the hardcoded value of disable_guest_empty_dir, instead,
we use the real value of it which comes from the configuration.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
A sandbox annotation that determines if it should create Kubernetes
emptyDir mounts on the guest filesystem.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
It acts as if it should create Kubernetes emptyDir mounts on the
guest filesystem. If enabled, the runtime will not create Kubernetes
emptyDir mounts on the guest filesystem.Instead, emptyDir mounts will
be created on the host and shared via virtio-fs.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Let's ensure Trustee is deployed as some of the tests rely images that
live behind authentication. /o\
The approach taken here to deploy Trustee is exactly the same one taken
on the other CoCo tests, apart from an env var passed to ensure we're
using the NVIDIA remote verifier (which will be in handy very very
soon).
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
This isn't really related to remote hypervisor though it was useful for
its debugging. It's a small helper I've been using regularly during
development for quite some time that I think might be useful more broadly.
Signed-off-by: Pavel Mores <pmores@redhat.com>
The remote hypervisor launches no VM, it just instructs the Cloud API
Adaptor to do so, therefore it has no need for an image or initrd to boot
from and should be exempt from the mandate for one or the other to be
specified.
Signed-off-by: Pavel Mores <pmores@redhat.com>
The go runtime's .proto file - which is also used by the Cloud API
Adaptor - puts the Hypervisor service into the "hypervisor" package.
runtime-rs has to do the same to avoid an "unimplemented" error.
Signed-off-by: Pavel Mores <pmores@redhat.com>
With the change made to the matrix when the CC GPU runner was added,
there was a change in the job name (@sprt saw that coming, but I
didn't).
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Same deal as the previous commut, just enabling the tests here, with the
same list of improvements that we will need to go through in order to
get is working in a perfect way.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
While the primary goal of this change is to detect regressions to the
NVIDIA SNP GPU scenario, various improvements to reflect a more
realistic CC setting are planned in subsequent changes, such as:
* moving away from the overlayfs snapshotter
* disabling filesystem sharing
* applying a pod security policy
* activating the GPUs only after attestation
* using a refined approach for GPU cold-plugging without requiring
annotations
* revisiting pod timeout and overhead parameters (the podOverhead value
was increased due to CUDA vectorAdd requiring about 6Gi of
podOverhead, as well as the inference and embedqa requiring at least
12Gi, respectively, 14Gi of podOverhead to run without invoking the
host's oom-killer. We will revisit this aspect after addressing
points 1. and 2.)
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
For the nvidia-gpu-snp and nvidia-gpu-tdx we must set containerd to
allow the CDI annotation to be passed to down.
This solution may become obsolete soon enough, but the cleanest way to
have it properly working is by adding it here (even if we remove it
before the next release).
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
It's been noticed that as more RAM is needed to run the CC tests, we
also need to update the podOverhead of the NVIDIA CC runtime classes to
avoid getting OOM Killed.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Since there's something wrong with the cpu hotplug
on qemu-runtime-rs, thus disable this test temporally.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
It should do nothing instead of return an error when
hot-unplug the memory to the size smaller than static
plugged memory size.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Since the nerdctl's network hook would call pselect6 syscall
by xtables-nft-multi, thus we'd better add it to the seccomp's
whitelist.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Let's add a new NVIDIA machine, which later on will be used for CC
related tests.
For now the current tests are skipped in the CC capable machine.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Let's now make sure that we don't add duplicated values to any of our
entries, making the script as sane as possible for sequential runs.
Vibed with Cursor's help!
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Let's add some helper functions, not yet used, to avoid adding
duplicated items.
This idea is an expansion of Choi's idea to avoid setting duplicated
items, and it'll help on making the whole script idempotent on
sequential runs.
Vibed with Cursor's help!
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
I know, this is not simplifying much things for now, but it has a good
intent in the background and will serve as base for making the
kata-deploy helm chart more user friendly.
With that said, let's add ALLOWED_HYPERVISOR_ANNOTATIONS per arch, while
adding support to set something like "qemu:foo,bar clh:bar foobar
barfoo". Why? Because in the future we'll have a better way to set this
per shim (and the shim is per arch ...).
More details of what we'll do in the future are being discussed here:
https://github.com/kata-containers/kata-containers/issues/12024
Anyways, the variables are **DELIBERATELY** not exposed to the chart for
now, as those will be later on when addressing the issue mentioned
above.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
When the runtimeClasses were added, as part of 7cfa826804, the
firecracker runtimeClass ended up missing from the dictionary.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
The Firecracker installation docs had an outaded containerd configuration for the devmapper plugin.
This commit updates the instructions so that they are compatible with more recent versions of containerd.
Signed-off-by: Anton Ippolitov <anton.ippolitov@datadoghq.com>
When added, I've mistakenly used the wrong test-type name, which is now
fixed and should be enough to trigger the tests correctly.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
On IBM actionspz P/Z runners, the following error was observed during
runtime tests:
```
host system doesn't support vsock: stat /dev/vhost-vsock: no such file or directory
```
Since loading the vsock module on the fly is not permitted, this commit
moves the runtime tests back to self-hosted runners for P/Z.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
On IBM actionspz Z runners, the following error occurs when running
`modprobe`:
```
modprobe: FATAL: Module bridge not found in directory /lib/modules/6.8.0-85-generic
```
Additionally, there are no files under `/lib/modules`, for example:
```
total 0
drwxr-xr-x 1 root root 0 Aug 5 13:09 .
drwxr-xr-x 1 root root 2.0K Oct 1 22:59 ..
```
This commit skips the `test_load_kernel_module` test if the module is
not found or if running `modprobe` is not permitted.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
On IBM actionspz Z runners, write operations on network interfaces
are not allowed, even for the root user.
This commit skips the `add_update_addresses` test if the operation
fails with EACCES (-13, permission denied).
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
On IBM actionspz Z runners, the ioctl system call is not allowed even
for the root user. There is likely an additional security mechanism
(such as AppArmor or seccomp) in place on Ubuntu runners.
This commit introduces a new helper, `is_permission_error()`,
which skips the test if ioctl operations in `reseed_rng()` are not
permitted.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The IBM actionspz Z runners mount /dev as tmpfs, while other systems
use devtmpfs. This difference causes an assertion failure for
test_already_baremounted.
This commit sets the detected filesystem for bare-mounted points
as the expected value.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The root filesystem for IBM actionspz Z runners is `btrfs` instead of `ext4`.
The error message differs when an unprivileged user tries to perform a bind mount.
This commit adjusts the handling of error messages based on the detected root
filesystem type.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Since the qemu & cloud-hypervisor support the cpu & memory
hotplug now, thus disable the static resource management
for qemu and cloud-hypervisor by default.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Since qemu-coco-dev-runtime-rs and qemu-coco-dev had disabled the
cpu&memory hotplug by enable static_sandbox_resource_mgmt, thus
we should disable the cpu hotplug test for those two runtime.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Since the qemu, cloud-hypervisor and dragonball had supported the
cpu hotplug on runtime-rs, thus enable the cpu hotplug test in CI.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
This commit introduces the configuration flag `disable_guest_empty_dir`
to control the placement of Kubernetes emptyDir volumes.
By default, the value is set to `false`, maintaining the current
behavior of creating emptyDirs within the guest VM
When set to `true`, emptyDirs will be created on the host filesystem.
This is essential for scenarios where users need to share data between
the host and the guest VM via an emptyDir.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
When handling a memory-based emptyDir, the runtime creates a tmpfs
mount inside the guest VM. The previous implementation just supports
mount options with only "rbind", which does not explicitly guarantee
the desired mount propagation behavior.
This commit hardens the mounting process by explicitly adding the
`rprivate` and `rw` mount flags.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit introduces the 'local' volume, which is specifically
designed to create and manage Kubernetes emptyDir volumes directly
within the VM's sandbox directory.
The core functionality ensures that local volume can be handled
correctly in handle volume procedure.
This capability is essential for allowing containers to leverage the
storage backend for shared volumes.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit implements the new 'local' storage type, enabling Kubernetes
emptyDir volumes to be created and managed directly inside the Kata VM
(in the sandbox directory).
The 'local' type instructs the kata-agent to provision the empty
directory within the VM.
This approach allows containers to share storage inside VM, Specially
useful within CoCo emptyDir scenarios.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Separated the checks for tmpfs and disk-based emptyDirs from an
`if-else if` block into two distinct `if` statements. This clarifies
the logic by treating each volume type detection as an independent task.
Additionally, updated the type for disk-based emptyDirs to the more
semantically accurate `KATA_K8S_LOCAL_STORAGE_TYPE`. This allows for
more specific handling downstream, distinguishing them from generic
host path mounts.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
In fact, emptyDir is not usually found in the proc mounts with the
previous logic and then it failed with the previous implementation.
Based on the related implementation within runtime-go,related
implementation within
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This introduces a new storage type: local. Local storage type will
tell kata-agent to create an empty directory with LocalStorgae handler
in the sandbox directory within the VM.
And it also makes it align with runtime-go `KataLocalDevType = "local"`.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Pod annotations from the outer runtime are being used for cold-plugging
CDI devices. We need to ensure that these annotations don't leak into
the inner runtime for which specific container (sibling) annotations
are being created. Without this change, the inner runtime receives both
annotations, leading to failing CDI injection as an outer runtime
annotation observed in the guest translates to an unresolvable CDI
device, for example, cdi.k8s.io/gpu: "nvidia.com/pgpu=0".
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Stratovirt has been failing for a considerable amount of time, with no
sign of someone watching it and being actively working on a fix.
With this we also stop building and shipping stratovirt as part of our
release as we cannot test it.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
A few weeks ago we've tested nydus-snapshotter with this approach, and
we DID find issues with it.
Now, let's also test this with `experimental_force_guest_pull`.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
It's just a follow-up on the previous commit where we move away from the
runtimeClass creation inside the script, and instead we do it using the
chart itself.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
This reverts commit be05e1370c, which is
not a problem as we never released such option.
Conflicts:
tools/packaging/kata-deploy/helm-chart/README.md
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
We had this logic inside the script when we didn't use the helm chart.
However, this only makes the shim script more convoluted for no reason.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
In order to fix:
```
=== Running govulncheck on containerd-shim-kata-v2 ===
Vulnerabilities found in containerd-shim-kata-v2:
=== Symbol Results ===
Vulnerability #1: GO-2025-4015
Excessive CPU consumption in Reader.ReadResponse in net/textproto
More info: https://pkg.go.dev/vuln/GO-2025-4015
Standard library
Found in: net/textproto@go1.24.6
Fixed in: net/textproto@go1.24.8
Vulnerable symbols found:
#1: textproto.Reader.ReadResponse
Vulnerability #2: GO-2025-4014
Unbounded allocation when parsing GNU sparse map in archive/tar
More info: https://pkg.go.dev/vuln/GO-2025-4014
Standard library
Found in: archive/tar@go1.24.6
Fixed in: archive/tar@go1.24.8
Vulnerable symbols found:
#1: tar.Reader.Next
Vulnerability #3: GO-2025-4013
Panic when validating certificates with DSA public keys in crypto/x509
More info: https://pkg.go.dev/vuln/GO-2025-4013
Standard library
Found in: crypto/x509@go1.24.6
Fixed in: crypto/x509@go1.24.8
Vulnerable symbols found:
#1: x509.Certificate.Verify
#2: x509.Certificate.Verify
Vulnerability #4: GO-2025-4012
Lack of limit when parsing cookies can cause memory exhaustion in net/http
More info: https://pkg.go.dev/vuln/GO-2025-4012
Standard library
Found in: net/http@go1.24.6
Fixed in: net/http@go1.24.8
Vulnerable symbols found:
#1: http.Client.Do
#2: http.Client.Get
#3: http.Client.Head
#4: http.Client.Post
#5: http.Client.PostForm
Use '-show traces' to see the other 9 found symbols
Vulnerability #5: GO-2025-4011
Parsing DER payload can cause memory exhaustion in encoding/asn1
More info: https://pkg.go.dev/vuln/GO-2025-4011
Standard library
Found in: encoding/asn1@go1.24.6
Fixed in: encoding/asn1@go1.24.8
Vulnerable symbols found:
#1: asn1.Unmarshal
#2: asn1.UnmarshalWithParams
Vulnerability #6: GO-2025-4010
Insufficient validation of bracketed IPv6 hostnames in net/url
More info: https://pkg.go.dev/vuln/GO-2025-4010
Standard library
Found in: net/url@go1.24.6
Fixed in: net/url@go1.24.8
Vulnerable symbols found:
#1: url.JoinPath
#2: url.Parse
#3: url.ParseRequestURI
#4: url.URL.Parse
#5: url.URL.UnmarshalBinary
Vulnerability #7: GO-2025-4009
Quadratic complexity when parsing some invalid inputs in encoding/pem
More info: https://pkg.go.dev/vuln/GO-2025-4009
Standard library
Found in: encoding/pem@go1.24.6
Fixed in: encoding/pem@go1.24.8
Vulnerable symbols found:
#1: pem.Decode
Vulnerability #8: GO-2025-4008
ALPN negotiation error contains attacker controlled information in
crypto/tls
More info: https://pkg.go.dev/vuln/GO-2025-4008
Standard library
Found in: crypto/tls@go1.24.6
Fixed in: crypto/tls@go1.24.8
Vulnerable symbols found:
#1: tls.Conn.Handshake
#2: tls.Conn.HandshakeContext
#3: tls.Conn.Read
#4: tls.Conn.Write
#5: tls.Dial
Use '-show traces' to see the other 4 found symbols
Vulnerability #9: GO-2025-4007
Quadratic complexity when checking name constraints in crypto/x509
More info: https://pkg.go.dev/vuln/GO-2025-4007
Standard library
Found in: crypto/x509@go1.24.6
Fixed in: crypto/x509@go1.24.9
Vulnerable symbols found:
#1: x509.CertPool.AppendCertsFromPEM
#2: x509.Certificate.CheckCRLSignature
#3: x509.Certificate.CheckSignature
#4: x509.Certificate.CheckSignatureFrom
#5: x509.Certificate.CreateCRL
Use '-show traces' to see the other 27 found symbols
Vulnerability #10: GO-2025-4006
Excessive CPU consumption in ParseAddress in net/mail
More info: https://pkg.go.dev/vuln/GO-2025-4006
Standard library
Found in: net/mail@go1.24.6
Fixed in: net/mail@go1.24.8
Vulnerable symbols found:
#1: mail.AddressParser.Parse
#2: mail.AddressParser.ParseList
#3: mail.Header.AddressList
#4: mail.ParseAddress
#5: mail.ParseAddressList
```
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
During the ${wait_time} for an expected condition, if
CreateContainerRequest was NOT expected to fail: detect possible
CreateContainerRequest failures early and abort the wait.
For example, before this change:
not ok 1 Successful replication controller with auto-generated policy in 123335ms
ok 2 Policy failure: unexpected container command in 14601ms
ok 3 Policy failure: unexpected volume mountPath in 14443ms
ok 4 Policy failure: unexpected host device mapping in 14515ms
ok 5 Policy failure: unexpected securityContext.allowPrivilegeEscalation in 14485ms
ok 6 Policy failure: unexpected capability in 14382ms
ok 7 Policy failure: unexpected UID = 1000 in 14578ms
After this change:
not ok 1 Successful replication controller with auto-generated policy in 17108ms
ok 2 Policy failure: unexpected container command in 14427ms
ok 3 Policy failure: unexpected volume mountPath in 14636ms
ok 4 Policy failure: unexpected host device mapping in 14493ms
ok 5 Policy failure: unexpected securityContext.allowPrivilegeEscalation in 14554ms
ok 6 Policy failure: unexpected capability in 15087ms
ok 7 Policy failure: unexpected UID = 1000 in 14371ms
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
During the ${wait_time} for an expected condition, if
CreateContainerRequest was NOT expected to fail: detect possible
CreateContainerRequest failures early and abort the wait.
For example, before this change:
not ok 1 Successful pod with auto-generated policy in 94852ms
ok 2 Policy failure: unexpected device mount in 17807ms
After this change:
not ok 1 Successful pod with auto-generated policy in 35194ms
ok 2 Policy failure: unexpected device mount in 21355ms
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
During the ${wait_time} for an expected condition, if
CreateContainerRequest was NOT expected to fail: detect possible
CreateContainerRequest failures early and abort the wait.
For example, before this change:
not ok 1 Logs empty when ReadStreamRequest is blocked in 102257ms
After this change:
not ok 1 Logs empty when ReadStreamRequest is blocked in 17339ms
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
During the ${wait_time} for an expected condition, if
CreateContainerRequest was NOT expected to fail: detect possible
CreateContainerRequest failures early and abort the wait.
For example, before this change:
not ok 1 Successful job with auto-generated policy in 107111ms
ok 2 Policy failure: unexpected environment variable in 7920ms
ok 3 Policy failure: unexpected command line argument in 7874ms
ok 4 Policy failure: unexpected emptyDir volume in 7823ms
ok 5 Policy failure: unexpected projected volume in 7812ms
ok 6 Policy failure: unexpected readOnlyRootFilesystem in 7903ms
ok 7 Policy failure: unexpected UID = 222 in 7720ms
After this change:
not ok 1 Successful job with auto-generated policy in 10271ms
ok 2 Policy failure: unexpected environment variable in 8018ms
ok 3 Policy failure: unexpected command line argument in 7886ms
ok 4 Policy failure: unexpected emptyDir volume in 7621ms
ok 5 Policy failure: unexpected projected volume in 7843ms
ok 6 Policy failure: unexpected readOnlyRootFilesystem in 7632ms
ok 7 Policy failure: unexpected UID = 222 in 7619ms
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
During the ${wait_time} for an expected condition, if
CreateContainerRequest was NOT expected to fail: detect possible
CreateContainerRequest failures early and abort the wait.
For example, before this change:
ok 1 Successful sc deployment with auto-generated policy and container image volumes in 14769ms
ok 2 Successful sc with fsGroup/supplementalGroup deployment with auto-generated policy and container image volumes in 8384ms
not ok 3 Successful sc deployment with security context choosing another valid user in 136149ms
ok 4 Successful layered sc deployment with auto-generated policy and container image volumes in 8862ms
ok 5 Policy failure: unexpected GID = 0 for layered securityContext deployment in 7941ms
ok 6 Policy failure: malicious root group added via supplementalGroups deployment in 11612ms
After:
ok 1 Successful sc deployment with auto-generated policy and container image volumes in 15230ms
ok 2 Successful sc with fsGroup/supplementalGroup deployment with auto-generated policy and container image volumes in 9364ms
not ok 3 Successful sc deployment with security context choosing another valid user in 11060ms
ok 4 Successful layered sc deployment with auto-generated policy and container image volumes in 9124ms
ok 5 Policy failure: unexpected GID = 0 for layered securityContext deployment in 7919ms
ok 6 Policy failure: malicious root group added via supplementalGroups deployment in 11666ms
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
During the ${wait_time} for an expected condition, if
CreateContainerRequest was NOT expected to fail: detect possible
CreateContainerRequest failures early and abort the wait.
For example, before this change:
not ok 1 Successful pod with auto-generated policy in 110801ms
not ok 2 Able to read env variables sourced from configmap using envFrom in 94104ms
not ok 3 Successful pod with auto-generated policy and runtimeClassName filter in 95838ms
not ok 4 Successful pod with auto-generated policy and custom layers cache path in 110712ms
ok 5 Policy failure: unexpected container image in 8113ms
ok 6 Policy failure: unexpected privileged security context in 7943ms
ok 7 Policy failure: unexpected terminationMessagePath in 11530ms
ok 8 Policy failure: unexpected hostPath volume mount in 7970ms
ok 9 Policy failure: unexpected config map in 7933ms
not ok 10 Policy failure: unexpected lifecycle.postStart.exec.command in 112677ms
ok 11 RuntimeClassName filter: no policy in 2302ms
not ok 12 ExecProcessRequest tests in 93946ms
not ok 13 Successful pod: runAsUser having the same value as the UID from the container image in 94003ms
ok 14 Policy failure: unexpected UID = 0 in 8016ms
ok 15 Policy failure: unexpected UID = 1234 in 7850ms
After:
not ok 1 Successful pod with auto-generated policy in 12182ms
not ok 2 Able to read env variables sourced from configmap using envFrom in 10121ms
not ok 3 Successful pod with auto-generated policy and runtimeClassName filter in 11738ms
not ok 4 Successful pod with auto-generated policy and custom layers cache path in 26592ms
ok 5 Policy failure: unexpected container image in 7742ms
ok 6 Policy failure: unexpected privileged security context in 7949ms
ok 7 Policy failure: unexpected terminationMessagePath in 7789ms
ok 8 Policy failure: unexpected hostPath volume mount in 7887ms
ok 9 Policy failure: unexpected config map in 7818ms
not ok 10 Policy failure: unexpected lifecycle.postStart.exec.command in 9120ms
ok 11 RuntimeClassName filter: no policy in 2081ms
not ok 12 ExecProcessRequest tests in 9883ms
not ok 13 Successful pod: runAsUser having the same value as the UID from the container image in 9870ms
ok 14 Policy failure: unexpected UID = 0 in 11161ms
ok 15 Policy failure: unexpected UID = 1234 in 7814ms
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
We've seen a few cases where we fail the test due to timeout and when we
print the pods we just see that they've been created.
With that in mind, let's just increase the timeout a little bit.
Example:
```
not ok 1 Parallel jobs in 6250ms
(in test file k8s-parallel.bats, line 41)
`kubectl wait --for=condition=Ready --timeout=$timeout pod -l jobgroup=${job_name}' failed
No resources found in kata-containers-k8s-tests namespace.
[bats-exec-test:71] INFO: k8s configured to use runtimeclass
job.batch/process-item-test1 created
job.batch/process-item-test2 created
job.batch/process-item-test3 created
NAME STATUS COMPLETIONS DURATION AGE
process-item-test1 Running 0/1 0s
process-item-test2 Running 0/1 0s
process-item-test3 Running 0/1 0s
error: no matching resources found
No resources found in kata-containers-k8s-tests namespace.
No resources found in kata-containers-k8s-tests namespace.
DEBUG: system logs of node 'aks-nodepool1-25989463-vmss000000' since test start time (2025-11-01 16:39:03)
-- No entries --
job.batch "process-item-test1" deleted
job.batch "process-item-test2" deleted
job.batch "process-item-test3" deleted
```
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Otherwise we'll face issues like:
```
Error: found in Chart.yaml, but missing in charts/ directory: node-feature-discovery
```
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
As we're failing on the uninstall, which seems related to a bug on NFD
itself, but I don't have access to a s390x machine to debug, let's skip
the enablement for now and enable it back once we've experimented it
better on s390x.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
As we're failing to install NFD on CBL Mariner, let's skip the
enablement there, and enable it once we've experimented it better there.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
As we have the ability to deploy NFD as a sub-chart of our chart, let's
make sure we test it during our CI.
We had to increase the timeout values, where we had timeouts set, to
deploy / undeploy kata, as now NFD is also deployed / undeployed.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Let's ensure that we add NFD as a weak dependency of the kata-deploy
helm chart.
What we're doing for now is leaving it up to the user / admin to enable
it, and if enabled then we do a explicit check for virtualization
support (x86_64 only for now).
In case NFD is already deployed, we fail the installation (in case it's
enabled on the kata-deploy helm chart) with a clear error message to the
user.
While I know that kata-remote **DOES NOT** require virtualization, I've
left this out (with a comment for when we add a peer-pods dependency on
kata-deploy) in order to simplify things for now, as kata-remote is not
a deployed shim by default.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
As Kata Containers can be consumed by other helm-charts, hard coding the
default runtime class name to `kata` is not optimal.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
All the options that take a specific shim as an argument MUST have
specific per arch settings, as not all the shims are available for all
the arches, leading to issues when setting up multi-arch deployments.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Let's ensure that we consume NVRC releases straight from GitHub instead
of building the binaries ourselves.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
We have here either /dev/vfio/<num> or /dev/vfio/devices/vfio<num>,
for IOMMUFD format /dev/vfio/devices/vfio<num>, strip "vfio" prefix
/dev/vfio/123 - basename "123" - vfioNum = "123" - cdi.k8s.io/vfio123
/dev/vfio/devices/vfio123 - basename "vfio123" - strip - vfioNum = "123" - cdi.k8s.io/vfio123
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This allows us to test privileged containers when using the webhook.
We can do this because kata-deploy sets privileged_without_host_devices = true for kata runtime by default.
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
We have 2 tests running on GitHub provided runners:
* devmapper
* CRI-O
- devmapper situation
For devmapper, we're currently testing devmapper with s390x as part of
one of its jobs.
More than that, this test has been failing here due to a lack of space
in the machine for quite some time, and no-action was taken to bring it
back either via GARM or some other way.
With that said, let's rely on the s390x CI to test devmapper and avoid
one extra failure on our CI by removing this one.
- cri-o situation
CRI-O is being tested with a fixed version of kubernetes that's already
reached its EOL, and a CRI-O version that matches that k8s version.
There has been attempts to raise issues, and also to provide a PR that
does at least part of the work ... leaving the debugging part for the
maintainers of the CI. However, there was no action on those from the
maintainers.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
- Explained the concept and benefits of VM templating
- Provided step-by-step instructions for enabling VM templating
- Detailed the setup for using snapshotter in place of VirtioFS for template-based VM creation
- Added performance test results comparing template-based and direct VM creation
Signed-off-by: ssc <741026400@qq.com>
- init: initialize the VM template factory
- status: check the current factory status
- destroy: clean up and remove factory resources
These commands provide basic lifecycle management for VM templates.
Signed-off-by: ssc <741026400@qq.com>
Use `ioctl_with_mut_ref` instead of `ioctl_with_ref` in the
`create_device` method as it needs to write to the `kvm_create_device`
struct passed to it, which was released in v0.12.1.
Signed-off-by: Siyu Tao <taosiyu2024@163.com>
Fix the cargo fmt issues and then we can make the libs tests required
again to avoid this regression happening again.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
kata-deploy helm chart is *THE* way to deploy kata-containers on
kubernetes environments, and kubernetes environments is basically the
only reliably tested deployment we have.
For now, let's just drop documentation that is outdated / incorrect, and
in the future let's ensure we update the linked docs, as we work on
update / upgrade for the helm chart.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
The libs in question were added when moving to developer.nvidia.com
but switching back to ubuntu only based builds they are not needed.
Remove them to keep the rootfs as minimal as possible.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
In the case of CC we need additional libraries in the rootfs.
Add them conditionally if type == confidential.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Add build_vm_from_template() that flips boot_from_template flag,
wires factory.template_path/{memory,state} into the hypervisor config,
and returns ready-to-use hypervisor & agent instances.
When factory.template is enabled, VirtContainer bypasses normal creation
and directly boots the VM by restoring the template through incoming migration,
completing the "create → save → clone" loop.
Fixes: #11413
Signed-off-by: ssc <741026400@qq.com>
Introduced factory::FactoryConfig with init/destroy/status commands to manage template pools.
Added template::Template to fetch, create and persist base VMs.
Introduced vm::{VM, VMConfig} exposing create, pause, save, resume, stop,
disconnect and migration helpers for sandbox integration.
Extended QemuInner to executes QMP incoming migration, pause/resume and status tracking.
Fixes: #11413
Signed-off-by: ssc <741026400@qq.com>
Added new fields in Hypervisor struct to support VM template creation,
template boot, memory and device state paths, shared path, and store
paths. Introduced a Factory struct in config to manage template path,
cache endpoint, cache number, and template enable flag. Integrated
Factory into TomlConfig for runtime configuration parsing.
Fixes: #11413
Signed-off-by: ssc <741026400@qq.com>
This change enables to run the Cloud Hypervisor VMM using a non-root user
when rootless flag is set true in configuration.
Fixes: #11414
Signed-off-by: stevenfryto <sunzitai_1832@bupt.edu.cn>
Pass the file descriptors of the tuntap device to the Cloud Hypervisor VMM process
so that the process could open the device without cap_net_admin
Signed-off-by: stevenfryto <sunzitai_1832@bupt.edu.cn>
There's no reason to keep the env var / input as it's never been used
and now kata-deploy detects automatically whether NFD is deployed or
not.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
When the NodeFeatureRule CRD is detected kata-deploy will:
* Create the specific NodeFeatureRules for the x86_64 TEEs
* Adapt the TEEs runtime classes to take into account the amount of keys
available in the system when spawning the podsandbox.
Note, we still do not have NFD as sub-dependency of the helm chart, and
I'm not even sure if we will have. However, it's important to integrate
better with the scenarios where the NFD is already present.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Change NIM bats file logic to allow skipping test cases which
require multiple GPUs. This can be helpful for test clusters where
there is only one node with a single GPU, or for local test
environments with a single-node cluster with a single GPU.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
Temporarily disables the new runners for building artifacts jobs. Will be re-enabled once they are stable.
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
This partially reverts 8dcd91c for the s390x because the
CI jobs are currently blocking the release. The new runners
will be re-introduced once they are stable and no longer
impact critical paths.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This allows us to do a full multi-arch deployment, as the user can
easily select which shim can be deployed per arch, as some of the VMMs
are not supported on all architectures, which would lead to a broken
installation.
Now, passing shims per arch we can easily have an heterogenous
deployment where, for instance, we can set qemu-se-runtime-rs for s390x,
qemu-cca for aarch64, and qemu-snp / qemu-tdx for x86_64 and call all of
those a default kata-confidential ... and have everything working with
the same deployment.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
The previous procedure failed to reliably ensure that all unused Device
Cgroups were completely removed, a failure consistently verified by CI
tests.
This change introduces a more robust and thorough cleanup mechanism. The
goal is to prevent previous issues—likely stemming from improper use of
Rust mutable references—that caused the modifications to be ineffective
or incomplete.
This ensures a clean environment and reliable CI test execution.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Build only from Ubuntu repositories do not mix with developer.nvidia.com
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Update tools/osbuilder/rootfs-builder/nvidia/nvidia_chroot.sh
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Migrate the k8s job to a different runner and use a long running cluster
instead of creating the cluster on every run.
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
This will help immensely projects consuming the kata-deploy helm chart
to use configuration options added during the development cycle that are
waiting for a release to be out ... allowing very early tests of the
stack.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
By default, `kubectl exec` inherits some capabilities from the
container, which could pose a security risk in a confidential
environment.
This change modifies the agent policy to strictly enforce that any
process started via `ExecProcessRequest` has no Linux capabilities.
This prevents potential privilege escalation within an exec session,
adhering to the principle of least privilege.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
As in CoCo cases, the ApparmorProfile setting within runtime-go is set with None,
we should align it with runtime-go.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Temporarily disable the auto-generated Agent Policy on Mariner hosts,
to workaround the new test failures on these hosts.
When re-enabling auto-generated policy in the future, that would be
better achieved with a tests/integration/kubernetes/gha-run.sh change.
Those changes are easier to test compared with GHA YAML changes.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
If a ConfigMap has more than 8 files it will not be mounted watchable
[1]. However, genpolicy assumes that ConfigMaps are always mounted at a
watchable path, so containers with large ConfigMap mounts fail
verification.
This commit allows mounting ConfigMaps from watchable and non-watchable
directories. ConfigMap mounts can't be meaningfully verified anyway, so
the exact location of the data does not matter, except that we stay in
the sandbox data dirs.
[1]: 0ce3f5fc6f/docs/design/inotify.md (L11-L21)Fixes: #11777
Signed-off-by: Markus Rudy <mr@edgeless.systems>
Every now and then, in case a failure happens, helm leaves the secret
behind without cleaning it up, leading to issues in the consecutive
runs.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Aurélien has moved to a reliable mirror for our tests, but we missed
that our tools Dockerfiles could benefit from the same change, which is
added now.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Although we saw this happening, we expected it to NOT happen ...
As the kernel is not signed, but we expect it to be (the cached
version), then we're bailing. :-/
Let's ensure a full rebuild of kernels happen and we'll be good from
that point onwards.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Add TDX QGS quote-generation-socket TDX QEMU object params for
attestation to work in NVGPU+TDX environment.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
By doing this we can ensure that more than one instance of
nydus-snapshotter can be running inside the cluster, which is super
useful for doing A-B "upgrades" (where we install a new version of
kata-containers + nydus on B, while A is still running, and then only
uninstall A after making sure that B is working as expected).
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
We've been wrongly trying to set up the `${shim}` (as the qemu-snp, for
instance) as the hypervisor name in the kata-containers configuration
file, leading to an `tomlq` breaking as all the .hypervisors.qemu* shims
are tied to the `qemu` hypervisor, and it happens regardless of the shim
having a different name, or the hypervisor being experimental or not.
```sh
$ grep "hypervisor.qemu*" src/runtime/config/configuration-*
src/runtime/config/configuration-qemu-cca.toml.in:[hypervisor.qemu]
src/runtime/config/configuration-qemu-coco-dev.toml.in:[hypervisor.qemu]
src/runtime/config/configuration-qemu-nvidia-gpu-snp.toml.in:[hypervisor.qemu]
src/runtime/config/configuration-qemu-nvidia-gpu-tdx.toml.in:[hypervisor.qemu]
src/runtime/config/configuration-qemu-nvidia-gpu.toml.in:[hypervisor.qemu]
src/runtime/config/configuration-qemu-se.toml.in:[hypervisor.qemu]
src/runtime/config/configuration-qemu-snp.toml.in:[hypervisor.qemu]
src/runtime/config/configuration-qemu-tdx.toml.in:[hypervisor.qemu]
src/runtime/config/configuration-qemu.toml.in:[hypervisor.qemu]
$ grep "hypervisor.qemu*" src/runtime-rs/config/configuration-*
src/runtime-rs/config/configuration-qemu-runtime-rs.toml.in:[hypervisor.qemu]
src/runtime-rs/config/configuration-qemu-se-runtime-rs.toml.in:[hypervisor.qemu]
```
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
`tests` module inside `memcg` module should be gated behind `test`, add
`[#cfg(test)]` to make those tests work properly.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Some tests from mem-agent requires root privilege, use
`skip_if_not_root` to skip those tests if they were not executed under
root user.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Prefixing with `#[allow(clippy::type_complexity)]` to silence this
warning, the return type is documented in comments.
```console
error: very complex type used. Consider factoring parts into `type` definitions
--> mem-agent/src/mglru.rs:184:6
|
184 | ) -> Result<HashMap<String, (usize, HashMap<usize, MGenLRU>)>> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#type_complexity
= note: `-D clippy::type-complexity` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::type_complexity)]`
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Manually fix `redundant_field_names ` clippy warning by testing equality
against 0 as suggested by rust 1.85.1, since `mem-agent` is now a member
of `libs` workspace.
```console
error: this comparison involving the minimum or maximum element for this type contains a case that is always true or always false
--> mem-agent/src/psi.rs:62:8
|
62 | if reader
| ________^
63 | | .read_line(&mut first_line)
64 | | .map_err(|e| anyhow!("reader.read_line failed: {}", e))?
65 | | <= 0
| |____________^
|
= help: because `0` is the minimum value for this type, the case where the two sides are not equal never occurs, consider using `reader
.read_line(&mut first_line)
.map_err(|e| anyhow!("reader.read_line failed: {}", e))? == 0` instead
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#absurd_extreme_comparisons
= note: `#[deny(clippy::absurd_extreme_comparisons)]` on by default
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Manually fix `redundant_field_names` clippy warning as suggested by rust
1.85.1, since `mem-agent` is now a member of `libs` workspace.
```console
error: redundant field names in struct initialization
--> mem-agent/src/memcg.rs:441:13
|
441 | numa_id: numa_id,
| ^^^^^^^^^^^^^^^^ help: replace it with: `numa_id`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_field_names
= note: `-D clippy::redundant-field-names` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::redundant_field_names)]`
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Manually fix `manual_strip` clippy warning as suggested by rust 1.85.1,
since `mem-agent` is now a member of `libs` workspace.
```console
error: stripping a prefix manually
--> mem-agent/src/mglru.rs:284:29
|
284 | u32::from_str_radix(&content[2..], 16)
| ^^^^^^^^^^^^^
|
note: the prefix was tested here
--> mem-agent/src/mglru.rs:283:13
|
283 | let r = if content.starts_with("0x") {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#manual_strip
= note: `-D clippy::manual-strip` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::manual_strip)]`
help: try using the `strip_prefix` method
|
283 ~ let r = if let Some(<stripped>) = content.strip_prefix("0x") {
284 ~ u32::from_str_radix(<stripped>, 16)
|
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Manually fix `field_reassign_with_default` clippy warning as suggested
by rust 1.85.1, since `mem-agent` is now a member of `libs` workspace.
```console
error: field assignment outside of initializer for an instance created with Default::default()
--> mem-agent/src/memcg.rs:874:21
|
874 | numa_cg.numa_id = numa;
| ^^^^^^^^^^^^^^^^^^^^^^^
|
note: consider initializing the variable with `memcg::CgroupConfig { numa_id: numa, ..Default::default() }` and removing relevant reassignments
--> mem-agent/src/memcg.rs:873:21
|
873 | let mut numa_cg = CgroupConfig::default();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#field_reassign_with_default
= note: `-D clippy::field-reassign-with-default` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::field_reassign_with_default)]`
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Fix `redundant_pattern_matching` clippy warning as suggested by rust
1.85.1, since `mem-agent` is now a member of `libs` workspace.
```console
error: redundant pattern matching, consider using `is_some()`
--> mem-agent/src/memcg.rs:595:40
|
595 | ... if let Some(_) = config_map.get_mut(path) {
| -------^^^^^^^--------------------------- help: try: `if config_map.get_mut(path).is_some()`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_pattern_matching
= note: `-D clippy::redundant-pattern-matching` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::redundant_pattern_matching)]`
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Fix `needless_bool` clippy warning as suggested by rust 1.85.1, since
`mem-agent` is now a member of `libs` workspace.
```console
error: this if-then-else expression returns a bool literal
--> mem-agent/src/memcg.rs:855:17
|
855 | / if configs.is_empty() {
856 | | true
857 | | } else {
858 | | false
859 | | }
| |_________________^ help: you can reduce it to: `configs.is_empty()`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_bool
= note: `-D clippy::needless-bool` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::needless_bool)]`
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Fix `for_kv_map` clippy warning as suggested by rust 1.85.1, since
`mem-agent` is now a member of `libs` workspace.
```console
error: you seem to want to iterate on a map's keys
--> mem-agent/src/memcg.rs:822:43
|
822 | for (single_config, _) in &secs_map.cgs {
| ^^^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#for_kv_map
help: use the corresponding method
|
822 | for single_config in secs_map.cgs.keys() {
| ~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Fix `into_iter_on_ref` clippy warning as suggested by rust 1.85.1, since
`mem-agent` is now a member of `libs` workspace.
```console
error: this `.into_iter()` call is equivalent to `.iter_mut()` and will not consume the `Vec`
--> mem-agent/src/memcg.rs:1122:27
|
1122 | for info in infov.into_iter() {
| ^^^^^^^^^ help: call directly: `iter_mut`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#into_iter_on_ref
= note: `-D clippy::into-iter-on-ref` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::into_iter_on_ref)]`
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Fix `legacy_numeric_constants` clippy warning as suggested by rust
1.85.1, since `mem-agent` is now a member of `libs` workspace.
```console
error: usage of a legacy numeric constant
--> mem-agent/src/compact.rs:132:47
|
132 | if self.config.compact_force_times == std::u64::MAX {
| ^^^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#legacy_numeric_constants
help: use the associated constant instead
|
132 | if self.config.compact_force_times == u64::MAX {
| ~~~~~~~~
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Fix `single_component_path_imports` clippy warning as suggested by rust
1.85.1, since `mem-agent` is now a member of `libs` workspace.
```console
error: this import is redundant
--> mem-agent/src/mglru.rs:345:5
|
345 | use slog_term;
| ^^^^^^^^^^^^^^ help: remove it entirely
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#single_component_path_imports
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Fix `from_str_radix_10` clippy warning as suggested by rust 1.85.1,
since `mem-agent` is now a member of `libs` workspace.
```console
error: this call to `from_str_radix` can be replaced with a call to `str::parse`
--> mem-agent/src/mglru.rs:29:14
|
29 | let id = usize::from_str_radix(words[1], 10)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try: `words[1].parse::<usize>()`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#from_str_radix_10
= note: `-D clippy::from-str-radix-10` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::from_str_radix_10)]`
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Fix `needless_borrow` clippy warning as suggested by rust 1.85.1, since
`mem-agent` is now a member of `libs` workspace.
```console
error: this expression creates a reference which is immediately dereferenced by the compiler
--> mem-agent/src/memcg.rs:1100:52
|
1100 | self.run_eviction_single_config(infov, &config)?;
| ^^^^^^^ help: change this to: `config`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Fix `ptr_arg` clippy warning as suggested by rust 1.85.1, since
`mem-agent` is now a member of `libs` workspace.
```console
error: writing `&PathBuf` instead of `&Path` involves a new object where a slice will do
--> mem-agent/src/memcg.rs:367:19
|
367 | psi_path: &PathBuf,
| ^^^^^^^^ help: change this to: `&Path`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#ptr_arg
= note: requested on the command line with `-D clippy::ptr-arg`
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Fix `crate_in_macro_def` clippy warning as suggested by rust 1.85.1,
since `mem-agent` is now a member of `libs` workspace.
```console
error: `crate` references the macro call's crate
--> mem-agent/src/misc.rs:12:22
|
12 | slog::error!(crate::misc::sl(), "{}", format_args!($($arg)*))
| ^^^^^ help: to reference the macro definition's crate, use: `$crate`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#crate_in_macro_def
= note: `-D clippy::crate-in-macro-def` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::crate_in_macro_def)]`
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Fix `len_zero` clippy warning as suggested by rust 1.85.1, since
`mem-agent` is now a member of `libs` workspace.
```console
error: length comparison to zero
--> mem-agent/src/memcg.rs:225:61
|
225 | let (keep, moved) = vec.drain(..).partition(|c| c.numa_id.len() > 0);
| ^^^^^^^^^^^^^^^^^^^ help: using `!is_empty` is clearer and more explicit: `!c.numa_id.is_empty()`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#len_zero
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Fix `bool_assert_comparison` clippy warning as suggested by rust 1.85.1,
since `mem-agent` is now a member of `libs` workspace.
```console
error: used `assert_eq!` with a literal bool
--> mem-agent/src/memcg.rs:1378:9
|
1378 | assert_eq!(m.get_timeout_list().len() > 0, true);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#bool_assert_comparison
= note: `-D clippy::bool-assert-comparison` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::bool_assert_comparison)]`
help: replace it with `assert!(..)`
|
1378 - assert_eq!(m.get_timeout_list().len() > 0, true);
1378 + assert!(m.get_timeout_list().len() > 0);
|
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`mem-agent` now does not ship example binaries and serves as a library
for `agent` to reference, so we move it into `libs` to better manage it.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Auto-generate policy in k8s-optional-empty-secret.bats, now that
genpolicy suppprts optional secret-based volumes.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
We've recently added support for:
* deploying and setting up a snapshotter, via
_experimentalSetupSnapshotter
* enabling experimental_force_guest_pull, via
_experimentalForceGuestPull
However, we never updated the documentation for those, thus let's do it
now.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Downloading Go from storage.googleapis.com fails intermittently with a 403
(see error below) so we switch to go.dev as referenced at
https://go.dev/dl/.
/tmp/install-go-tmp.Rw5Q4thEWr ~/work/kata-containers/kata-containers
/usr/bin/go
[install_go.sh:85] INFO: removing go version go1.24.9 linux/amd64
[install_go.sh:94] INFO: Download go version 1.24.6
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 298 100 298 0 0 2610 0 --:--:-- --:--:-- --:--:-- 2614
[install_go.sh:97] INFO: Install go
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
[install_go.sh:99] ERROR: sudo tar -C /usr/local/ -xzf go1.24.6.linux-amd64.tar.gz
https://github.com/kata-containers/kata-containers/actions/runs/18602801597/job/53045072109?pr=11947#step:5:17
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This change ensures that the NVIDIA package repository for nvidia-imex
and libnvidia-nspc is being used as source.
The NVIDIA repository does not publish these packages with a -580
version suffix, which made us fall back to the packages from the
Ubuntu repository.
These two packages were recently updated by Ubuntu to depend on
nvidia-kernel-common-580-server (this happened from version
580.82.07-0ubuntu1 to version 580.95.05-0ubuntu1). This conflicts
with nvidia-kernel-common-580 which gets installed by
nvidia-headless-no-dkms-580-open, thus causing a build failure.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
in CI helm is not yet installed and we don't have root access. Let's use
the current dir, which should be writable, and --no-sudo option to
install it.
Note when helm is installed it should not change anything and simply use
the syste-wide installation.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
While the local-build's folder's Makefile dependencies for the
confidential nvidia rootfs targets already declare the pause image
and coco-guest-components dependencies, the actual rootfs
composition does not contain the pause image bundle and relevant
certificates for guest pull. This change ensure the rootfs gets
composed with the relevant files.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
TOML was chosen for initdata particularly for the ability to include
policy docs and other configuration files without mangling them. The
default TOML encoding renders string values as single-line,
double-quoted strings, effectively depriving us of this feature.
This commit changes the encoding to use `to_string_pretty`, and includes
a test that verifies the desirable aspect of encoding: newlines are kept
verbatim.
Fixes: #11943
Signed-off-by: Markus Rudy <mr@edgeless.systems>
After supporting the Arm CCA, it will rely on the kernel kvm.h headers to build the
runtime. The kernel-headers currently quite new with the traditional one, so that we
rely on build the kernel header first and then inject it to the shim-v2 build container.
Signed-off-by: Kevin Zhao <kevin.zhao@linaro.org>
Co-authored-by: Seunguk Shin <seunguk.shin@arm.com>
The new initdata variants of the tests are failing on the tdx
runner, so as discussed, skip them for now: Issue #11945
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Now we have wider coverage of initdata testing in
k8s-guest-pull-image-signature.bats then remove
the old testing.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Our current set of authenticated registry tests involve setting
kernel_params to config the image pull process, but as of
kata-containers#11197
this approach is not the main way to set this configuration and the agent
config has been removed. Instead we should set the configuration in the
`cdh.toml` part of the initdata, so add new test cases for this. In future, when
we have been through the deprecation process, we should remove the old tests
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Our current set of signature tests involve setting kernel_parameters to
config the image pull process, but as of
https://github.com/kata-containers/kata-containers/pull/11197
this approach is not the main way to set this configuration and the agent
config has been removed. Instead we should set the configuration in the
`cdh.toml` part of the initdata, so add new test cases for this. In future, when
we have been through the deprecation process, we should remove the old tests
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Create a shared get_initdata method that injects a cdh image
section, so we don't duplicate the initdata structure everywhere
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
One problem that we've been having for a reasonable amount of time, is
containerd not behaving very well when we have multiple snapshotters.
Although I'm adding this test with my "CoCo" hat in mind, the issue can
happen easily with any other case that requires a different snapshotter
(such as, for instance, firecracker + devmapper).
With this in mind, let's do some stability tests, checking every hour a
simple case of running a few pre-defined containers with runc, and then
running the same containers with kata.
This should be enough to put us in the situation where containerd gets
confused about which snapshotter owns the image layers, and break on us
(or not break and show us that this has been solved ...).
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
With this change we namespace the stage one rootfs tarball name
and use the same name across all uses. This will help overcome
several subtle local build problems.
Signed-off-by: Manuel Huber <manuelh@nvidia.com>
We fix the following error:
```
thread 'sandbox::tests::add_and_get_container' panicked at src/sandbox.rs:901:10:
called `Result::unwrap()` on an `Err` value: Create cgroupfs manager
Caused by:
0: fs error caused by: Os { code: 17, kind: AlreadyExists, message: "File exists" }
1: File exists (os error 17)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
by ensuring that the cgroup path is unique for tests run in the same millisecond.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Use CDI exclusively from crates.io and not from a GH repository.
Cargo can easily check if a new version is available and we can
far more easier bump it if needed.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
with the shellcheck fixes we accidentally quoted the "-n NAMESPACE"
argument where we should have used array instead, which lead to oc
considering this as a pod name and returning error.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
We need to ensure that any change on the Dockerfile (and its dir) leads
to the build being retriggered, rather than using the cached version.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
We are seeing more protoc related failures on the new
runners, so try adding the protobuf-compiler dependency
to these steps to see if it helps.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- copy default-initdata.toml in create_tmp_policy_settings_dir, so it can be modified by other tests if needed
- make auto_generate_policy use default-initdata.toml by default
- add auto_generate_policy_no_added_flags, so it may be used by tests that don't want to use default-initdata.toml by default
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
On commit 9602ba6ccc, from February this
year, we've introduced a check to ensure that the files needed for
signing the kernel build are present. However, we've noticed last week
that there were a reasonable amount of wrong assumptions with the
workflow. :-)
Zvonko fixed the majority of those, but this bit was left and it'd cause
breakages when using kernel that was cached ... although passing when
building new kernels.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
This is needed to the kernel setup picks up the correct
config values from our fragments directories.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
We need to make sure that the kernel we're using has the
correct configs set, otherwise the module signing will not work.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Now that we have added the ability to deploy kata-containers with
experimental_force_guest_pull configured, let's make sure we test it to
avoid any kind of regressions.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Otherwise we have no way to differentiate running tests on qemu-coco-dev
with different snapshotters.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
What was done in the past, trying to set the env var on the same step
it'd be used, simply does not work.
Instead, we need to properly set it through the `env` set up, as done
now.
We're also bumping the kata_config_version to ensure we retrigger the
kernel builds.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
We have some scalable s390x and ppc runners, so
start to use them for build and test, to improve
the throughput of our CI
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
For some reason we didn't have the "Report tests" step as part of the
TEE jobs. This step immensely helps to check which tests are failing and
why, so let's add it while touching the workflow.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
There's no reason to have the code duplication between the SNP / TDX
tests for CoCo, as those are basically using the same configuration
nowadays.
Note that for the TEEs case, as the nydus-snapshotter is deployed by the
admin, once, instead of deploying it on every run ... I'm actually
removing the nydus-snapshotter steps so we make it clear that those
steps are not performed by the CI.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
As virtio-scsi has been set the default block device driver, the
runtime also need to correctly handle the virtio-scsi info, specially
the SCSI address required within kata-agent handling logic.
And getting and assigning the scsi_addr to kata agent device id
will be enough. This commit just do such work.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Since runtime-rs support the block device hotplug with
creating new containers, and the device would also be
removed when the container stopped, thus add the block
device unplug for clh.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
This commit introduces support for selecting `virtio-scsi` as the
block device driver for QEMU during initial setup.
The primary goal is to resolve a conflict in non-TEE environments:
1. The global block device configuration defaults to `virtio-scsi`.
2. The `initdata` device driver was previously designed and hardcoded
to `virtio-blk-pci`.
3. This conflict prevented unified block device usage.
By allowing `virtio-scsi` to be configured at cold boot, the `initdata`
device can now correctly adhere to the global setting, eliminating the
need for a hardcoded driver and ensuring consistent block device
configuration across all supported devices (excluding rootfs).
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The implementation of the seccomp feature in Dragonball currently has a basic framework.
But the actual restriction rules are empty.
This pull request includes the following changes:
- Modifiy configuration files to relevant configuration files.
- Modifiy seccomp framework to support different restrictions for different threads.
- Add new seccomp rules for the modified framework.
This commit primarily implements the changes 1 and 3 for runtime-rs.
Fixes: #11673
Signed-off-by: wangxinge <wangxinge@bupt.edu.cn>
For DGX like systems we need additional binaries and libraries,
enable the Kata AND CoCo use-case.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Update tools/osbuilder/rootfs-builder/nvidia/nvidia_rootfs.sh
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Fix all instances of template injection by using environment variables as
recommended by Zizmor, instead of directly injecting values into the
commands.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
The two ignored cases are strictly necessary for the CI to work today, and we
have various security mitigations in place.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
There are 62 such warnings and addressing them would take quite a bit of
time so just disable them for now.
help[undocumented-permissions]: permissions without explanatory comments
--> ./.github/workflows/release.yaml:71:7
|
71 | packages: write
| ^^^^^^^^^^^^^^^ needs an explanatory comment
72 | id-token: write
| ^^^^^^^^^^^^^^^ needs an explanatory comment
73 | attestations: write
| ^^^^^^^^^^^^^^^^^^^ needs an explanatory comment
|
= note: audit confidence → High
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
We can't test this PR because the workflow needs this trigger, so adding
this will allow testing future PRs.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
For those who are not willing to use the nydus-snapshotter for pulling
the image inside the guest, let's allow them setting the
experimetal_force_guest_pull, introduced by Edgeless, as part of our
helm-chart.
This option can be set as:
_experimentalForceGuestPull: "qemu-tdx,qemu-coco-dev"
Which would them ensure that the configuration for `qemu-tdx` and
`qemu-coco-dev` would have the option enabled.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
As the kata-deploy helm chart has been the only way we've been testing
kata-containers deployment as part of our CI, it's time to finally get
rid of the kustomize yamls and avoid us having to maintain two different
methods (with one of those not being tested).
Here I removed:
* kata-deploy yamls and kustomize yamls
* kata-cleanup yamls and kustomize yamls
* kata-rbac yals and kustomize yamls
* README.md for the kustomize yamls was removed
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
erofs-snapshotter can be used to leverage sharing the image from the
host to the guest without the need of a shared filesystem (such as
virtio-fs or virtio-9p).
This case is ideal for Confidential Computing enabled on Kata
Containers, and we can immensely benefit from this snapshotter, thus
let's test it as soon as possible so we can find issues, report bugs,
and ask for enhancement requests.
There are at least a few things that we know for sure to be problematic
now:
* Policy has to be adjusted to the erofs-snapshotter
* There is no support for signed nor encrypted images
* Tests that use the KBS are disabled for now
Even with the limitations, I do believe we should be testing the
snapshoitter, so we can team up and get those limitations addressed.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
As done in the previous commit, let's expand the vanilla k8s deployment
to also allow the erofs host side configuration.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
We already have support for deploying a few flavours of k8s that are
required for different tests we perform.
Let's also add the ability to deploy vanilla k8s, as that will be very
useful in the next commits in this series.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
The active version is 2.1.x, and the latest is 2.2.0-beta.0.
The latest is what we'll be using to test if the "to be released"
version of containerd works well for our use-cases.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Let's make sure that we can get non-official releases as well, otherwise
we won't be able to test a coming release of containerd, to know whether
it solves issues that we face or not, before it's actually released.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
NVRC introduced the confidential feature flag and we
haven't updated the rootfs build to accomodate.
If rootfs_type==confidential user --feature=confidential
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Canonical TDX release is not needed for vanilla Ubuntu 25.10 but
GRUB_CMDLINE_LINUX_DEFAULT needs to contain `nohibernate` and
`kvm_intel.tdx=1`
Signed-off-by: Szymon Klimek <szymon.klimek@intel.com>
Use grep_pod_exec_output to retry possible failing "kubectl exec"
commands. Other tests have been hitting such errors during CI in
the past.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This adds an integration test to verify that privileged containers work
properly when deploying Kata with kata-deploy.
This is a follow-up to #11878.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Let's rely on kata-deploy setting up the nydus snapshotter for us,
instead of doing this with external code.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
This allows us to stop setting up the snapshotter ourselves, and just
rely con kata-deploy to do so.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Let's expose the EXPERIMENTAL_SETUP_SNAPSHOTTER script environment
variable to our chart, allowing then users of our helm chart to take
advantage of this experimental feature.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
We may deploy in scenarios where we want to have both snapshotters set
up, sometimes even for simple test on which one behaves better.
With this in mind, let's allow EXTERNAL_SETUP_SNAPSHOTTER to receive a
comma separated list of snapshotters, such as:
```
EXPERIMENTAL_SETUP_SNAPSHOTTER="erofs,nydus"
```
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Similarly to what's been done for the nydus-snapshotter, let's allow
users to have erofs-snapshotter set up by simply passing:
```
EXPERIMENTAL_SETUP_SNAPSHOTTER="erofs".
```
Mind that erofs, although a built-in containerd snapshotter, has system
depdencies that we will *NOT* install and it's up to the admin to do so.
These dependencies are:
* erofs-utils
* fsverity
* erofs module loaded
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
In the previous commit we added the assumption that the
nydus-snapshotter version should be the same in two different places.
Now, with this test, we ensure those will always be in sync.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Let's introduce a new EXPERIMENTAL_SETUP_SNAPSHOTTER environemnt
variable that, when set, allows kata-deploy to put the nydus snapshotter
in the correct place, and configure containerd accordingly.
Mind, this is a stop gap till the nydus-snapshotter helm chart is ready
to be used and behaving well enough to become a weak dependency of our
helm chart. When that happens this code can be deleted entirely.
Users can have nydus-snapshotter deployed and configured for the
guest-pull use case by simply passing:
```
EXPERIMENTAL_SETUP_SNAPSHOTTER="nydus"
```
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Otherwise we'd end up adding a the file several times, which could lead
to problems when removing the entry, leading to containerd not being
able to start due to an import file not being present.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
The code, how it was, would lead to the following broke command:
`--header "Authorization: Bearer: "`
Let's only expand that part of the command if ${GH_TOKEN} is passed,
otherwise we don't even bother adding it.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Document that privileged containers with
privileged_without_host_devices=false are not generally supported.
When you try the above, the runtime will pass all the host devices to Kata
in the OCI spec, and Kata will fail to create the container for various
reasons depending on the setup, e.g.:
- Attempting to hotplug uninitialized loop devices.
- Attempting to remount /dev devices on themselves when the agent had
already created them as default devices (e.g. /dev/full).
- "Conflicting device updates" errors.
- And more...
privileged_without_host_devices was originally created to support
Kata [1][2] and lots of people are having issues when it's set to
false [3].
[1] https://github.com/kata-containers/runtime/issues/1568
[2] https://github.com/containerd/cri/pull/1225
[3] https://github.com/kata-containers/kata-containers/issues?q=is%3Aissue%20%20in%3Atitle%20privileged
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
We have noticed in the CI that the `gen_init_cpio ...` was returning 255
and breaking the build. Why? I am not sure.
When chatting with Steve, he suggested to split the command, so it'd be
easier to see what's actually breaking. But guess what? There's no
breakage when we split the command.
So, let's try it out and see whether the CI passes after it.
If someone is willing to educate us on this one, please, that would be
helpful! :-)
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Moving the CUDA repo to the top for all essential packages
and adding a repo priority favouring NVIDIA based repos.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Introduce new test case which verifies that openvpn clients and servers
can run as Kata pods and can successfully establish a connection.
Volatile certificates and keys are generated by an initialization
container and injected into the client and server containers.
This scenario requires TUN/TAP support for the UVM kernel.
Signed-off-by: Manuel Huber <mahuber@microsoft.com>
Co-authored-by: Manuel Huber <manuelh@nvidia.com>
No need to die when a Kind that does not require a policy annotation is
found in a pod manifest. Print an informational message instead.
Signed-off-by: Manuel Huber <mahuber@microsoft.com>
Currently, use of openvpn clients/servers is not possible in Kata UVMs.
Following error message can be expected:
ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such device (errno=19)
To support opevpn scenarios using bridging and TAP, we enable various
kernel networking config options.
Signed-off-by: Manuel Huber <mahuber@microsoft.com>
Manually added "hostPath" to main.txt then regenerated the dictionary
with `./kata-spell-check.sh make-dict`.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This change crystallizes and simplifies the current handling of /dev
hostPath mounts with virtually no functional change.
Before this change:
- If a mount DESTINATION is in /dev and it is a non-regular file on the HOST,
the shim passes the OCI bind mount as is to the guest (e.g.
/dev/kmsg:/dev/kmsg). The container rightfully sees the GUEST device.
- If the mount DESTINATION does not exist on the host, the shim relies on
k8s/containerd to automatically create a directory (ie. non-regular file) on
the HOST. The shim then also passes the OCI bind mount as is to the guest. The
container rightfully sees the GUEST device.
- For other /dev mounts, the shim passes the device major/minor to the guest
over virtio-fs. The container rightfully sees the GUEST device.
After this change:
- If a mount SOURCE is in /dev and it is a non-regular file on the HOST,
the shim passes the OCI bind mount as is to the guest. The container
rightfully sees the GUEST device.
- The shim does not anymore rely on k8s/containerd to create missing mount
directories. Instead it explicitely handles missing mount SOURCES, and
treats them like the previous bullet point.
- The shim no longer uses virtio-fs to pass /dev device major/minor to the
guest, instead it passes the OCI bind mount as is.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
test_add_one_arp_neighbor modifies the root network namespace, so we
should ensure that it does not interfere with normal network setup.
Adding an IP to a device results in automatic routes, which may affect
routing to non-test endpoints. Thus, we change the addresses used in the
test to come from TEST-NET-1, which is designated for tests and usually
not routable.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
list_routes and test_add_one_arp_neighbor have been flaky in the past
(#10856), but it's been hard to tell what exactly is going wrong.
This commit adds debug information for the most likely problem in
list_routes: devices being added/removed/modified concurrently.
Furthermore, it adds the exit code and stderr of the ip command, in case
it failed to list the ARP neighborhood.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
The previous code only checked the result of with_nix_path(), not statfs(),
thus leading to an uninitialized memory read if statfs() failed.
No functional change otherwise.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
85f3391bc added the support for TDX QGS port=0 but missed
defaultQgsPort in the default config. defaultQgsPort overrides
user provided tdx_quote_generation_service_socket_port=0.
After this change, defaultQgsPort is not needed anymore since
there's no default: any positive integer is OK and negative or
unset value becomes a parse error.
QEMUTDXQUOTEGENERATIONSERVICESOCKETPORT in the Makefile is used
to provide a sane default when tdx_quote_generation_service_socket_port
gets set in the configuration.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
genpolicy is a developer tool that should be usable on MacOS. Adding it
to the darwin CI job ensures that it can still be built after changes.
On an Apple M2, the output of `uname -m` is `arm64`, which is why a new
case is needed in the arch_to_* functions.
We're not going to cross-compile binaries on darwin, so don't install
any additional Rust targets.
Fixes: #11635
Signed-off-by: Markus Rudy <mr@edgeless.systems>
Most of the kata-types code is reusable across platforms. However, some
functions in the mount module require safe-path, which is Linux-specific
and can't be used on other platforms, notably darwin.
This commit adds a new feature `safe-path` to kata-types, which enables
the functions that use safe-path. The Linux-only callers kata-ctl and
runtime-rs enable this feature, whereas genpolicy only needs initdata
and does not need the functions from the mount module. Using a feature
instead of a target_os restriction ensures that the developer experience
for genpolicy remains the same.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
This commit adds changes to enable fs sharing between host/guest
using virtio-fs when booting a pod VM for testing. This primarily
enables sharing container rootfs for testing container lifecycle
commands.
Summary of changes is as below:
- adds minimal virtiofsd code to start userspace daemon (based on
`runtime-rs/crates/resource/src/share_fs`)
- adds the virtiofs device to the test vm
- prepares and mounts the container rootfs on host
- modifies container storage & oci specs
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
Fixing the shellcheck issues first so that they are not coupled to the
subsequent commit introducing Darwin support to the script.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
Auto-generate policy for nginx-deployment pods, instead of hard-coding
the "allow all" policy.
Note that the `busybox_pod` - created using `kubectl run` - still
doesn't have an Init Data annotation, so it is using the default policy
built into the Kata Guest rootfs image file.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Auto-generate agent policy in k8s-liveness-probes.bats, instead of using
the non-confidential "allow all" policy.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Auto-generate the agent policy for pod-secret-env.yaml, using
"genpolicy -c inject_secret.yaml".
Support for passing Secret specification files as "-c" arguments of
genpolicy has been added when fixing #10033 with PR #10986.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Since we cannot build all components with libc=musl and
static RUSTFLAG we still need to ship libcc for AA or other guest
components.
Without this change the guest components do not work and we see
/usr/local/bin/attestation-agent: error while loading shared
libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This fixes that error everywhere by adding a `name:` field to all jobs that
were missing it. We keep the same name as the job ID to ensure no
disturbance to the required job names.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
-o pipefail in particular ensures that exec_host() returns the right exit
code.
-u is also added for good measure. Note that $BATS_TEST_DIRNAME is set by
bats so we move its usage inside the function.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
As a consequence of moving away from Advanced Security for Zizmor, it now
checks the entire codebase and will error out on this PR and future.
To be reverted once we address all Zizmor findings in a future PR.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This PR fixes a test that failed on platforms like ppc64le due to a hardcoded mount option length.
* Test was failing on ppc64le due to larger system page size (e.g., 65536 bytes)
* Original test used a hardcoded 4097-byte string assuming 4KB page size
* Replaced with *MAX_MOUNT_PARAM_SIZE + 1 to reflect actual system limit
* Ensures test fails correctly across all architectures
Fixes: #11852
Signed-off-by: shwetha-s-poojary <shwetha.s-poojary@ibm.com>
The Hadolint warning DL3007 (pin the version explicitly) is no
longer applicable.
We have updated the base image to use a specific version
digest, which satisfies the linter's requirement for reproducible
builds. This commit removes the corresponding inline ignore comment.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This change enables to run the QEMU VMM using a non-root user when rootless flag is set true in the configuration.
Signed-off-by: stevenfryto <sunzitai_1832@bupt.edu.cn>
This commit introduces generic support for running the VMM in rootless mode in runtime-rs:
1.Detect whether the VMM is running in rootless mode.
2.Before starting the VMM process, create a non-root user and launch the VMM with that user’s UID and GID; also add the KVM user's group ID to the VMM process's supplementary groups so the VMM process can access /dev/kvm.
3.Add the setup of the rootless directory located in the dir /run/user/<uid> directory, and modify some path variables to be functions that return the path with the rootless directory prefix when running in rootless mode.
Fixes: #11414
Signed-off-by: stevenfryto <sunzitai_1832@bupt.edu.cn>
We recently hit the following error during build:
```
RUN ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -P ""
OpenSSL version mismatch. Built against 3050003f, you have 30500010
```
This happened because `alpine:latest` moved forward and the `ssh-keygen`
binary in the base image was compiled against a newer OpenSSL version
that is not available at runtime.
Pinning the base image to the stable release (3.20) avoids the mismatch
and ensures consistent builds.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This change fixes clean up logic when running tests
in a vm booted with qemu wrt to qmp.sock & console.sock
files, and no longer assumes any path for them.
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
Change the default block driver to virtio-scsi.
Since the latest qemu's commit:
https://gitlab.com/qemu-project/qemu/-/commit/
984a32f17e8dab0dc3d2328c46cb3e0c0a472a73
brings a bug for virtio-blk-pci with io_uring mode at line:
https://gitlab.com/qemu-project/qemu/-/commit/
984a32f17e8dab0dc3d2328c46cb3e0c0a472a73#
ce8eeb01f8b84f8cb8d3c35684d473fe1ee670f9_345_352
In order to avoid this issue, change the default block driver
to virtio-scsi.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
As OCI Spec annotation has been updated with adding or remove items,
we should use the updated annotation as the passed argument.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit removes the InitData annotation from the OCI Spec's
annotations.
Similar to the Policy annotation, InitData is now exclusively handled
and transmitted to the guest via the sandbox's init data mechanism.
Removing this redundant and potentially large annotation simplifies the
OCI Spec and streamlines the guest initialization process.
This change aligns the handling of InitData with existing practices
within runtime-go.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The repro below would show this error in the logs (in debug mode only):
fatal runtime error: IO Safety violation: owned file descriptor already closed
The issue was that the `pseudo.slave` file descriptor was being owned by
multiple variables simultaneously. When any of those variables would go out
of scope, they would close the same file descriptor, which is undefined
behavior.
To fix this, we clone: we create a new file descriptOR that refers to the same
file descriptION as the original. When the cloned descriptor is closed, this
affect neither the original descriptor nor the description. Only when the last
descriptor is closed does the kernel cleans up the description.
Note that we purposely consume (not clone) the original descriptor with
`child_stdin` as `pseudo` is NOT dropped automatically.
Repro
-----
Prerequisites:
- Use Rust 1.80+.
- Build the agent in debug mode.
$ cat busybox.yaml
apiVersion: v1
kind: Pod
metadata:
name: busybox
spec:
containers:
- image: busybox:latest
name: busybox
runtimeClassName: kata
$ kubectl apply -f busyboox.yaml
pod/busybox created
$ kubectl exec -it busybox -- sh
error: Internal error occurred: Internal error occurred: error executing
command in container: failed to exec in container: failed to start exec
"e6c602352849647201860c1e1888d99ea3166512f1cc548b9d7f2533129508a9":
cannot enter container 76a499cbf747b9806689e51f6ba35e46d735064a3f176f9be034777e93a242d5,
with err ttrpc: closed
Fixes: #11054
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Log how much time "kubectl get pods" and each test case are taking,
just in case that will reveal unusually slow test clusters, and/or
opportunities to improve tests.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
In the previous commit we've added some code that broke `cargo fmt --
--check` without even noticing, as the code didn't go through the CI
process (due to it being a security advisory).
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Trailing slash in DEFAULT_KATA_GUEST_SANDBOX_DIR caused double slashes
in mount_point (e.g. "/run/kata-containers/sandbox//shm"), which failed
OPA strict equality checks against policy mount_point. Removing it aligns
generated paths with policy and fixes CreateSandboxRequest denial.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
coco-guest-components tarball is used as is for both vanilla coco
rootfs and the nvidia enabled rootfs. nvidia-attester can be built
without nvml so make it globally enabled for coco-guest-components.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
`allow_interactive_exec` requires a sandbox-name annotation, however
this is only added for pods by genpolicy. Other pod-generating resources
have unpredictable sandbox names.
This patch instead uses a regex for the sandbox name in genpolicy, based
on the specified metadata and following Kubernetes' naming logic. The
generated regex is then used in the policy to correctly match the
sandbox name.
Fixes: #11823
Signed-off-by: Charlotte Hartmann Paludo <git@charlotteharludo.com>
Co-authored-by: Paul Meyer <katexochen0@gmail.com>
Co-authored-by: Markus Rudy <mr@edgeless.systems>
This commit addresses an issue where base64 output, when used with a
default configuration, would introduce newlines, causing decoding to
fail on the runtime.
The fix ensures base64 output is a single, continuous line using the -w0
flag. This guarantees the encoded string is a valid Base64 sequence,
preventing potential runtime errors caused by invalid characters.
Note that: When you use the base64 command without any parameters, it
typically automatically adds newlines to the output, usually every 76 chars.
In contrast, base64 -w0 explicitly tells the command not to add any
newlines (-w for wrap, and 0 for a width of zero), which results in a
continuous string with no whitespace.
This is a critical distinction because if you pass a Base64 string with
newlines to a runtime, it may be treated as an invalid string, causing
the decoding process to fail.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Correctly set dir's permissions and mode. This update ensures:
The dir_mode field of CopyFileRequest is set to DIR_MODE_PERMS
(equivalent to Go's 0o750 | os.ModeDir), which is primarily used for the
top-level directory creation permissions.
The file_mode field now directly uses metadata.mode() (equivalent to
Go's st.Mode) for the target entry.
This change aims to resolve potential permission issues or inconsistencies
during directory and file creation within the guest environment by precisely
matching the expected mode propagation of the Kata agent.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The core purpose of introducing volume_manager to VolumeResource is to
centralize the management of shared file system volumes. By creating a
single VolumeManager instance within VolumeResource, all shared file
volumes are managed by one central entity. This single volume_manager
can accurately track the references of all ShareFsVolume instances to
the shared volumes, ensuring correct reference counting, proper volume
lifecycle management, and preventing issues like volumes being
overwritten.
This new design ensures that all shared volumes are managed by a central
entity, which:
(1) Guarantees correct reference counting.
(2) Manages the volume lifecycle correctly, avoiding issues like volumes
being overwritten.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit integrates the new `VolumeManager` into the `ShareFsVolume`
lifecycle. Instead of directly copying files, `ShareFsVolume::new` now
uses the `VolumeManager` to get a guest path and determine if the volume
needs to be copied. It also updates the `cleanup` function to release
the volume's reference count, allowing the `VolumeManager` to manage its
state and clean up resources when no longer in use.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit introduces a new `VolumeManager` to track the state of shared
volumes, including their reference count and its corresponding container
ids.
The manager's goal is to handle the lifecycle of shared filesystem volumes,
including:
(1) Volume State Tracking: Tracks the mapping from host source paths to guest
destination paths.
(2) Reference Counting: Manages reference counts for each volume, preventing
premature cleanup when multiple containers share the same source.
(3) Deterministic guest paths: Generates unique guest paths using random string
to avoid naming conflicts.
(4) Improved Management: Provides a centralized way to handle volume creation,
copying, and release, including aborting file watchers when volumes are no longer
in use.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit refactors the `CopyFile` related code to streamline the
logic for creating guest directories and make the code structure
clearer.
Its main goal is to improve the overall maintainability and facilitate
future feature extensions.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit is designed to perform a full sync before starting monitoring
to ensure that files which exist before monitoring starts are also synced.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit updates the configuration for the initdata block
device to use the BlockDeviceAio::Native mode.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit enhances control over block device AIO modes via hotplug.
Previously, hotplugging block devices was set with default AIO mode (io_uring).
Even if users reset the AIO mode in the configuration file, the changes would
not be correctly applied to individual block devices.
With this update, users can now explicitly configure the AIO mode for hot-plugging
block devices via the configuration, and those settings will be correctly applied.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
We need more information about block device, just relapce the original
method get_block_driver with get_block_device_info and return its
BlockDeviceInfo.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
- Add disable_selinux and selinux_label fields to hypervisor for SELinux support.
- Implement related SELinux support functions.
Fixes: #9866
Signed-off-by: Caspian443 <scrisis843@gmail.com>
Add container_exec_with_retries(), useful for retrying if needed
commands similar to:
kubectl exec <pod_name> -c <container_name> -- <command>
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
On s390x, QEMU fails if maxmem is set to 0:
```
invalid value of maxmem: maximum memory size (0x0) must be at least the initial memory size
```
This commit sets maxmem to the initial memory size for s390x when hotplug is disabled,
resolving the error while still ensuring that memory hotplug remains off.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The setting '-m xM,slots=y,maxmem=zM' where maxmem is from
the host's memory capacity is failing with confidential VMs
on hosts having 1T+ of RAM.
slots/maxmem are necessary for setups where the container
memory is hotplugged to the VM during container creation based
on createContainer info.
This is not the case with CoCo since StaticResourceManagement
is enabled and memory hotplug flows have not been checked.
To avoid unexpeted errors with maxmem, disable slots/maxmem
in case ConfidentialGuest is requested.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Let's make sure that whenever we try to access the attestation agent
binariy, we only proceed the startup in case:
* the binary is found (CoCo case)
* the binary is not present (non-CoCo case)
In case any error that's not `NotFound`, we should simply abort as that
could mean a potential tampering with the binary (which would be
reported as an EIO).
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
Add the annotation of OCI bundle path to store its path.
As it'll be checked within agent policy, we need add them
to pass agent policy validations.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
With the help of `update_ocispec_annotations`, we'll add the contaienr
type key with "io.katacontainers.pkg.oci.container_type" and its
corresponding type "pod_sandbox" when it's pause container and
"pod_container" when it's an other containers.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
It'll updates OCI annotations by removing specified keys and adding
new ones. This function creates a new `HashMap` containing the updated
annotations, ensuring that the original map remains unchanged.
It is optimized for performance by pre-allocating the necessary capacity
and handling removals and additions efficiently.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
To enable access to the constants `POD_CONTAINER` and `POD_SANDBOX` from
other crates, their visibility has been updated to public. This change
addresses the previous limitation of restricted access and ensures these
values can be utilized across the codebase.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Add the annotation of nerdctl network namespace to let nerdctl know which namespace
to use when calling the selected CNI plugin with "nerdctl/network-namespace".
As it'll be checked within agent policy, we need add them to pass agent policy validations.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
On 63f6dcdeb9 we added the support to
download either a .xz or a .zst tarball file. However, we missed adding
the code to properly unpack a .zst tarball file.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
agent-ctl's make check has been failing with:
```
Checking kata-agent-ctl v0.0.1 (/home/ubuntu/runner/_layout/_work/kata-containers/kata-containers/src/tools/agent-ctl)
error[E0432]: unresolved import `hypervisor::ch`
--> src/vm/vm_ops.rs:10:5
|
10 | ch::CloudHypervisor,
| ^^ could not find `ch` in `hypervisor`
|
note: found an item that was configured out
--> /home/ubuntu/runner/_layout/_work/kata-containers/kata-containers/src/runtime-rs/crates/hypervisor/src/lib.rs:30:9
|
30 | pub mod ch;
| ^^
note: the item is gated here
--> /home/ubuntu/runner/_layout/_work/kata-containers/kata-containers/src/runtime-rs/crates/hypervisor/src/lib.rs:26:1
|
26 | / #[cfg(all(
27 | | feature = "cloud-hypervisor",
28 | | any(target_arch = "x86_64", target_arch = "aarch64")
29 | | ))]
| |___^
```
Let's just make sure that we include ch conditionally as well.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Docker containers support specifying the shm size using the --shm-size
option and support sandbox-level shm volumes, so we've added support for
shm volumes. Since Kubernetes doesn't support specifying the shm size,
it typically uses a memory-based emptydir as the container's shm, and
its size can be specified.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Since the rate limiter would be shared by cloud-hypervisor
and firecracker etc, thus move it from clh's config to
hypervisor config crate which would be shared by other vmm.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Given that Rust-based VMMs like cloud-hypervisor, Firecracker, and
Dragonball naturally offer user-level block I/O rate limiting, I/O
throttling has been implemented to leverage this capability for these
VMMs. This PR specifically introduces support for cloud-hypervisor.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
This commit introduces changes to parse the PciDeviceInfo received
in response payload when adding a network device to the VM with cloud
hypervisor. When hotplugging a network device for a given endpoint,
it rightly sets the PciPath of the plugged-in device in the endpoint.
In calls like virtcontainers/sandbox.go:AddInterface, the later call
to agent sends the pci info for uevents (instead of empty value) to
rightly update the interfaces instead of failing with `Link not found`
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
Support for the share-rw=true parameter has been added. While this
parameter is essential for maintaining data consistency across multiple
QEMU instances sharing a backend disk image, its implementation also
serves to standardize parameters with the block device hotplug
functionality in kata-runtime/qemu.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Docker tests have been broken for a while and should be removed if we
cannot maintain those.
For now, though, let's limit it to run only with one hypervisor and
avoid wasting resources for no reason.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
devmapper tests have been failing for a while. It's been breaking on the
kata-deploy deployment, which is most likely related to Disk Pressure.
Removing files was not enough to get the tests to run, so we'll just run
those with QEMU as a way to test fixes. Once we get the test working,
we can re-enable the other VMMs, but for now let's just not waste
resources for no reason.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
add_allow_all_policy_to_yaml now also sets the initdata annotation. So don't overwrite the
initdata annotation that was previously set by create_coco_pod_yaml_with_annotations.
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
Delete annotation from OCI spec and sandbox config. This is done after the optional initdata annotation value has been read.
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
Bump cri-o to 1.34.0 to try and remediate security advisories
CVE-2025-0750 and CVE-2025-4437.
Note: Running
```
go get github.com/cri-o/cri-o@v1.34.0
```
seems to bump a lot of other go modules, hence the size of the
vendor diff
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
In google.golang.org/grpc v1.72.0, `DialContext`, is deprecated, so
switch to use `NewClient` instead.
`grpc.WithBlock()` is deprecated and not recommend, so remove this
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
In `github.com/prometheus/common v0.62.0` expfmt.FmtText
is deprecated, so replace with `expfmt.NewFormat(expfmt.TypeTextPlain)`.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
In `github.com/prometheus/common v0.62.0` expfmt.FmtText
is deprecated, so replace with `expfmt.NewFormat(expfmt.TypeTextPlain)`.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This is following Steve's suggestion, based on what's been done on
cloud-api-adaptor.
The reason we're doing it here is because we've seen pods being evicted
due to disk pressure.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
I've hit this when using a machine with slow internet connection, which
took ages to download the kata-cleanup image, and then helm timed out in
the middle of the cleanup, leading to the cleanup job being restarted
and then bailing with an error as the runtimeclasses that kata-deploy
tries to delete were already deleted.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Since the cloud hypervisor's resize vCPU is an asynchronous operation,
it's possible that the previous resize operation hasn't completed when
the request is sent, causing the current call to return an error.
Therefore, several retries can be performed to avoid this error.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
A minor release of QEMU is out, so update to it for fixes and features.
QEMU changelog: https://wiki.qemu.org/ChangeLog/10.1
Notes:
* AVX support is not an option to be enabled / disabled anymore.
* Passt requires Glibc 2.40.+, which means a dependency on Ubuntu 25.04
or newer, thus we're disabling it.
Signed-off-by: Alex Tibbles <alex@bleg.org>
Although versions of slab prior to 0.4.10, don't have a security
vulnearability, we can bump them all to keep things in sync
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
```
Experimental fw_cfg Device Support
This feature enables passing configuration data and files, such as VM
boot configurations (kernel, kernel cmdline, e820 memory map, and ACPI
tables), from the host to the guest. (#7117)
Experimental ivshmem Device Support
Support for inter-VM shared memory has been added. For more information,
please refer to the ivshmem documentation. (#6703)
Firmware Boot Support on riscv64
In addition to direct kernel boot, firmware boot support has been added
on riscv64 hosts. (#7249)
Increased vCPU Limit on x86_64/kvm
The maximum number of supported vCPUs on x86_64 hosts using KVM has been
raised from 254 to 8192. (#7299)
Improved Block Performance with Small Block Sizes
Performance for virtio-blk with small block sizes (16KB and below)
is enhanced via submitting async IO requests in batches. (#7146)
Faster VM Pause Operation
The VM pause operation now is significantly faster particularly for VMs
with a large number of vCPUs. (#7290)
Updated Documentation on Windows Guest Support
Our Windows documentation now includes instructions to run Windows 11
guests, in addition to Windows Server guests. (#7218)
Policy on AI Generated Code
We will decline any contributions known to contain contents generated or
derived from using Large Language Models (LLMs). Details can be found
in our contributing documentation. (#7162)
Removed SGX Support
The SGX support has been removed, as announced in the deprecation notice two
release cycles ago. (#7093)
Notable Bug Fixes
Seccomp filter fixes with glibc v2.42 (#7327)
Various fixes related to (#7331, #7334, #7335)
```
From https://github.com/cloud-hypervisor/cloud-hypervisor/releases/tag/v48.0
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Exclude 'cgroup' namespace from namespace checks during `allow_linux`
validation. This complements the existing exclusion of the 'network'
namespace.
As runtime-rs has specific cgroup namespace configurations, and excluding it from
policy validation ensures parity between runtime-rs and runtime-go implementations.
This allows focusing validation on critical namespaces like PID, IPC, and MNT, while
avoiding potential policy mismatches due to another cgroup namespace management by
the runtime-rs.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Add `normalize_namespace_type()` function to map "mount"
(case-insensitive) to "mnt" while keeping other values unchanged.
This ensures namespace comparisons treat "mount" and "mnt" as
equivalent.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
- Use set comparison to ignore ordering differences when matching
capabilities.
- Add normalization to strip "CAP_" prefix to support both CAP_XXX and
XXX formats.
This makes capability matching more robust against different ordering
and naming formats.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
As prvious configure with overlayfs is incorrect, which causes the agent
policy validation failure. And it's also different with runtime-go's
configuration. In this patch, we'll correct its fstype with overlay and
align with runtime on this matter.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
When hot-removing a block device, the kernel must first unmount the
device and then destroy it on the VM. Therefore, a
prepare_remove_block_device procedure must be added to wait for the
kernel to unmount the device before destroying it on the VM.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
When hot-removing a block device, the kernel must first remove the
device and then destroy it on the VM. Therefore, a
prepare_remove_block_device procedure must be added to wait for the
kernel to unmount the device before destroying it on the VM.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Since Dragonball's MMIO bus only supports legacy interrupts, while
the PCI bus supports MSIX interrupts, to improve performance for block
devices, virtio-blk devices are set to PCI bus mode by default.
We had tested the virtio-blk's performance using the fio with the
following commands:
fio -filename=./test -direct=1 -iodepth 32 -thread -rw=randrw
-rwmixread=50 -ioengine=libaio -bs=4k -size=10G -numjobs=4
-group_reporting -name=mytest
When used the legacy interrupt, the io test is as below:
read : io=20485MB, bw=195162KB/s, iops=48790, runt=107485msec
write: io=20475MB, bw=195061KB/s, iops=48765, runt=107485msec
Once switched to msix innterrupt, the io test is as below:
read : io=20485MB, bw=260862KB/s, iops=65215, runt= 80414msec
write: io=20475MB, bw=260727KB/s, iops=65181, runt= 80414msec
We can get 34% performance improvement.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Added support for PCI buses for virtio-blk devices. This commit adds
support for PCI buses for both cold-plugged and hot-plugged
virtio-blk devices. Furthermore, during hot-plugging, support is added for
synchronous waiting for hot-plug completion. This ensures that multiple devices
can be hot-plugged successfully without causing upcall busy errors.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
In order to support the pci bus for virtio devices,
move the pci system manager from vfio manager to
device manager, thus it can be shared by both of
vfio and virtio pci devices.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Use @DEFENABLEANNOTATIONS_COCO@ in configuration-qemu-snp.toml,
for consistency with the tdx and coco-dev configuration files.
k8s-initdata.bats was failing during CI on SNP without this change,
because the cc_init_data annotation was disabled.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This runs Zizmor on pushes to any branch, not just main.
This is useful for:
1. Testing changes in feature branches with the manually-triggered CI.
2. Forked repos that may use a different name than "main" for their
default branch.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Test case for `get_uds_with_sid` with an empty run directory would not
hit the 0 match arm, i.e. "sandbox with the provided prefix {short_id:?}
is not found", because `get_uds_with_sid` will try to create the
directory with provided short id before detecting `target_id`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Preset directory `kata98654sandboxpath1` will produce more than one
`target_id` in `get_uds_with_sid`, which causes test to fail. Remove
that directory to make this test work.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`test_arch_guest_protection_*` test cases get triggered simultaneously,
which is impossible for a single machine to pass. Modify tests to detect
protection file before preceding.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Case 4 of `test_execute_hook` would fail because `args` could not be
empty, while by providing `build_oci_hook` with `vec![]` would result in
empty args at execution stage.
Modify `build_oci_hook` to set args as `None` when empty vector is
provided.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
The variable `addr` was used to store the log level string read from the
`LOG_LEVEL_ENV_VAR` environment variable. This name is misleading as it
implies a network address rather than a log level value.
This commit renames the variable to `level` to more accurately reflect
its purpose, improving the overall readability of the configuration code.
A minor whitespace formatting fix in a macro is also included.
Signed-off-by: Liang, Ma <liang3.ma@intel.com>
A new internal nightly test has been established for runtime-rs.
This commit adds a new entry `cc-se-e2e-tests-rs` to the existing
matrix and renames the existing entry `cc-se-e2e-tests` to
`cc-se-e2e-tests-go`.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Tests skipped because tests for `qemu-se` are skipped:
- k8s-empty-dirs.bats
- k8s-inotify.bats
- k8s-shared-volume.bats
Tests skipped because tests for `qemu-runtime-rs` are skipped:
- k8s-block-volume.bats
- k8s-cpu-ns.bats
- k8s-number-cpus.bats
Let's skip the tests above to run the nightly test
for runtime-rs on IBM SEL.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
There are still some issues to be address before we can mark `make test`
for `libs` as required. Mark this case as not required temporarily.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
SNP launch was failing after the confidential guest kernel was upgraded to 6.16.1.
Added required module CONFIG_MTRR enabled.
Added required module CONFIG_X86_PAT enabled.
Fixes: #11779
Signed-off-by: Ryan Savino <ryan.savino@amd.com>
Bump the version of runtime-rs' hypervisor crate
to upgrade (indirectly) protobug and remediate vulnerability
RUSTSEC-2024-0437
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The previous document about the default of create_container_timeout
is 30,000 millseconds which not keep alignment with runtime-go.
In this commit, we'll change it as 30 seconds.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Since it aligns with the create_container_timeout definition in
runtime-go, we need to set the value in configuration.toml in seconds,
not milliseconds. We must also convert it to milliseconds when the
configuration is loaded for request_timeout_ms.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
It's possible that tests take a long time to run and hence that the access
token expires before we delete the cluster. In this case `az cli` will try
to refresh the access token using the OIDC token (which will have
definitely also expired because its lifetime is ~5 minutes).
To address this we refresh the OIDC token manually instead. Automatic
refresh isn't supported per Azure/azure-cli#28708.
Fixes: #11758
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Introduce new test case in k8s-iptables.bats which verifies that
workloads can configure iptables in the UVM.
Users discovered that they weren't able to do this for common usecases
such as istio. Proper support for this should be built into UVM
kernels. This test ensures that current and future kernel
configurations don't regress this functionality.
Signed-off-by: Cameron Baird <cameronbaird@microsoft.com>
Currently, the UVM kernel fails for istio deployments (at least with the
version we tested, 1.27.0). This is because the istio sidecar container
uses ip6tables and the required kernel configs are not built-in:
```
iptables binary ip6tables has no loaded kernel support and cannot be used, err: exit status 3 out: ip6tables v1.8.10 (legacy):
can't initialize ip6tables table `filter': Table does not exist (do you need to insmod?)
Perhaps ip6tables or your kernel needs to be upgraded.
```
Signed-off-by: Cameron Baird <cameronbaird@microsoft.com>
In certain scenarios, particularly under CoCo/Agent Policy enforcement,
the default initial value of `Linux.Resources.Devices` is considered
non-compliant, leading to container creation failures. To address this
issue and ensure consistency with the behavior in `runtime-go`, this
commit removes the default value of `Linux.Resources.Devices` from the
OCI Spec.
This cleanup ensures that the OCI Spec aligns with runtime expectations
and prevents policy violations during container creation.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Only the StartContainer hook needs to be reserved for execution in the
guest, but we also make sure that the setting happens only when the OCI
Hooks does exist, otherwise we do nothing.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
In k8s-guest-pull-image.bats, `failed to pull image` is
not caught by assert_logs_contain() for runtime-rs.
To ensure consistency, this commit changes `failed` to
`Failed`, which works for both runtimes.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Mount validation for sealed secret requires the base path to start with
`/run/kata-containers/shared/containers`. Previously, it used
`/run/kata-containers/sandbox/passthrough`, which caused test
failures where volume mounts are used.
This commit renames the path to satisfy the validation check.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
ef642fe890 added a special case to avoid
moving cgroups that are on the "default" slice in case of deletion.
However, this special check should be done in the Parent() method
instead, which ensures that the default resource controller ID is
returned, instead of ".".
Fixes: #11599
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
- Set guest Storage.options for block rootfs to empty (do not propagate host mount options).
- Align behavior with Go runtime: only add xfs nouuid when needed.
Signed-off-by: Caspian443 <scrisis843@gmail.com>
We moved to `.zst`, but users still use the upstream kata-manager to
download older versions of the project, thus we need to support both
suffixes.
Fixes: #11714
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
Similar to what we've done for Cloud Hypervisor in the commit
9f76467cb7, we're backporting a runtime-rs
feature that would be benificial to have as part of the go runtime.
This allows users to use virito-balloon for the hypervisor to reclaim
memory freed by the guest.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
The default suggestion for top-level permissions was
`contents: read`, but scorecard notes anything other than empty,
so try updating it and see if there are any issues. I think it's
only needed if we run workflows from other repos.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Since the previous tightening a few workflow updates have
gone in and the zizmor job isn't flagging them as issues,
so address this to remove potential attack vectors
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This reverts commit cb5f143b1b, as the
cached packages have been regenerated after the switch to using zstd.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
As part of the go 1.24.6 bump there are errors about the incorrect
use of a errorf, so switch to the non-formatting version, or add
the format string as appropriate
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
golang 1.25 has been released, so 1.23 is EoL,
so we should update to ensure we don't end up with security issues
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Update the two workflows that used setup-go to
instead call `install_go.sh` script, which handles
installing the correct version of golang
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
`${kernel_name,,}` is bash 4.0 and not posix compliant, so doesn't
work on macos, so switch to `tr` which is more widely
supported
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
In #11693 the cc_init_data annotation was changes to be hypervisor
scoped, so each hypervisor needs to explicitly allow it in order to
use it now, so add this to both the go and rust runtime's remote
configurations
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
We need to get the root_hash.txt file from the image build, otherwise
there's no way to build the shim using those values for the
configuration files.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
Although the compress ratio is not as optimal as using xz, it's way
faster to compress / uncompress, and it's "good enough".
This change is not small, but it's still self-contained, and has to get
in at once, in order to help bisects in the future.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
As 3.18 is already EOL.
We need to add `--break-system-packages` to enforce the install of the
installation of the yq version that we rely on. The tests have shown
that no breakage actually happens, fortunately.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Currently, we change vm_rootfs_driver as the initdata device driver
with block_device_driver.
Fixes#11697
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
we also need support initdat within nonprotection even though the
platform is detected as NonProtection or usually is called nontee
host. Within these cases, there's no need to validate the item of
`confidential_guest=true`, we believe the result of the method
`available_guest_protection()?`.
Fixes#11697
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The default `reconnect_timeout` (3 seconds) was found to be insufficient for
IBM SEL when using VSOCK. This commit updates the timeouts as follows:
- `dial_timeout_ms`: Set to 90ms to match the value used in go-runtime for IBM SEL
- `reconnect_timeout_ms`: Increased to 5000ms based on empirical testing
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Add support for the `InitData` resource config on IBM SEL,
so that a corresponding block device is created and the
initdata is passed to the guest through this device.
Note that we skip passing the initdata hash via QEMU’s
object, since the hypervisor does not yet support this
mechanism for IBM SEL. It will be introduced separately
once QEMU adds the feature.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Linux v6.16 brings some useful features for the confidential guests.
Most importantly, it adds an ABI to extend runtime measurement registers
(RTMR) for the TEE platforms supporting it. This is currently enabled
on Intel TDX only.
The kernel version bump from v6.12.x to v6.16 forces some CONFIG_*
changes too:
MEMORY_HOTPLUG_DEFAULT_ONLINE was dropped in favor of more config
choices. The equivalent option is MHP_DEFAULT_ONLINE_TYPE_ONLINE_AUTO.
X86_5LEVEL was made unconditional. Since this was only a TDX
configuration, dropping it completely as part of v6.16 is fine.
CRYPTO_NULL2 was merged with CRYPTO_NULL. This was only added in
confidential guest fragments (cryptsetup) so we can drop it in this update.
CRYPTO_FIPS now depends on CRYPTO_SELFTESTS which further depends on
EXPERT which we don't have. Enable both in a separate config fragment
for confidential guests. This can be moved to a common setting once
other targets bump to post v6.16.
CRYPTO_SHA256_SSE3 arch optimizations were reworked and are now enabled
by default. Instead of adding it to whitelist.conf, just drop it completely
since it was only enabled as part of "measured boot" feature for
confidential guests. CONFIG_CRYPTO_CRC32_S390 was reworked the same way.
In this case, whitelist.conf is needed.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
This reverts commit ede773db17.
`cc_init_data` should be under a hypervisor category because
it is a hypervisor-specific feature. The annotation including
`runtime` also breaks a logic for `is_annotation_enabled()`.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
We need to include `cc_init_data` in the enable_annotations
array to pass the data. Since initdata is a CoCo-specific
feature, this commit introduces a new array,
`DEFENABLEANNOTATIONS_COCO`, which contains the required
string and applies it to the relevant CoCo configuration.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Currently, there are 2 issues for the empty initdata annotation
test:
- Empty string handling
- "\[CDH\] \[ERROR\]: Get Resource failed" not appearing
`add_hypervisor_initdata_overrides()` does not handle
an empty string, which might lead to panic like:
```
called `Result::unwrap()` on an `Err` value: gz decoder failed
Caused by:
failed to fill whole buffer
```
This commit makes the function return an empty string
for a given empty input and updates the assertion string
to one that appears in both go-runtime and runtime-rs.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Currently, runtime-rs related code within the libs directory lacks
sufficient CI protection. We frequently observe the following issues:
- Inconsistent Code Formatting: Code that has not been properly
formatted
is merged.
- Failing Tests: Code with failing unit or integration tests is merged.
To address these issues, we need introduce stricter CI checks for the
libs directory. This may specifically include:
- Code Formatting Checks
- Mandatory Test Runs
Fixes#11512
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
To make it aligned with the setting of runtime-go, we should keep
it as empty when users doesn't enable and set its specified path.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
We need to make sure that we use the latest kernel
and rebuild the initrd and image for the nvidia-gpu
use-cases otherwise the tests will fail since
the modules are not build against the new kernel and
they simply fail to load.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
OSV-Scanner highlights go.mod references to go stdlib 1.23.0 contrary to intention in versions.yaml, so synchronize them.
Make a converse comment for versions.yaml.
Fixes: #11700
Signed-off-by: Alex Tibbles <alex@bleg.org>
Let's rename the runtime-rs initdata annotation from
`io.katacontainers.config.runtime.cc_init_data` to
`io.katacontainers.config.hypervisor.cc_init_data`.
Rationale:
- initdata itself is a hypervisor-specific feature
- the new name aligns with the annotation handling logic:
c92bb1aa88/src/libs/kata-types/src/annotations/mod.rs (L514-L968)
This commit updates the annotation for go-runtime and tests accordingly.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Enable testing of initdata on the qemu-coco-dev and qemu-se
runtime classes, so we can validate the function on s390x
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This commit support the seccomp_sandbox option from the configuration.toml file
and add the logic for appending command-line arguments based on this new configuration parameter.
Fixes: #11524
Signed-off-by: wangxinge <wangxinge@bupt.edu.cn>
Previouly it is reusing the ovmf, which will enter some
issue for path checking, so move to aavmf as it should
be.
Signed-off-by: Kevin Zhao <kevin.zhao@linaro.org>
Read only the sealed secret prefix instead of the whole file.
Improves performance and reduces memory usage in I/O-heavy environments.
Fixes: #11643
Signed-off-by: Park.Jiyeon <jiyeonnn2@icloud.com>
Dependening on the platform configuration, users might want to
set a more secure policy than the QEMU default.
Signed-off-by: Paul Meyer <katexochen0@gmail.com>
This change introduces a new command line option `--vm`
to boot up a pod VM for testing. The tool connects with
kata agent running inside the VM to send the test commands.
The tool uses `hypervisor` crates from runtime-rs for VM
lifecycle management. Current implementation supports
Qemu & Cloud Hypervisor as VMMs.
In summary:
- tool parses the VMM specific runtime-rs kata config file in
/opt/kata/share/defaults/kata-containers/runtime-rs/*
- prepares and starts a VM using runtime-rs::hypervisor vm APIs
- retrieves agent's server address to setup connection
- tests the requested commands & shutdown the VM
Fixes#11566
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
The seccomp feature for Cloud Hypervisor and Firecracker is enabled by default.
This commit introduces an option to disable seccomp for both and updates the built-in configuration.toml file accordingly.
Fixes: #11535
Signed-off-by: wangxinge <wangxinge@bupt.edu.cn>
Route kata-shim logs directly to systemd-journald under 'kata' identifier.
This refactoring enables `kata-shim` logs to be properly attributed to
'kata' in systemd-journald, instead of inheriting the 'containerd'
identifier.
Previously, `kata-shim` logs were challenging to filter and debug as
they
appeared under the `containerd.service` unit.
This commit resolves this by:
1. Introducing a `LogDestination` enum to explicitly define logging
targets (File or Journal).
2. Modifying logger creation to set `SYSLOG_IDENTIFIER=kata` when
logging
to Journald.
3. Ensuring type safety and correct ownership handling for different
logging backends.
This significantly enhances the observability and debuggability of Kata
Containers, making it easier to monitor and troubleshoot Kata-specific
events.
Fixes: #11590
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
After moving Arm64 CI nodes to new one, we do faced an interesting
issue for timeout when it executes the command with crictl runp,
the error is usally: code = DeadlineExceeded
Fixes: #11662
Signed-off-by: Kevin Zhao <kevin.zhao@linaro.org>
This series should make runtime-rs's vcpu allocation behaviour match the
behaviour of runtime-go so we can now enable pertinent tests which were
skipped so far due the difference between both shims.
Signed-off-by: Pavel Mores <pmores@redhat.com>
Configuration information is adjusted after loading from file but so
far, there has been no similar check for configuration coming from
annotations. This commit introduces re-adjusting config after
annotations have been processed.
A small refactor was necessary as a prerequisite which introduces
function TomlConfig::adjust_config() to make it easier to invoke
the adjustment for a whole TomlConfig instance. This function is
analogous to the existing validate() function.
The immediate motivation for this change is to make sure that 0
in "default_vcpus" annotation will be properly adjusted to 1 as
is the case if 0 is loaded from a config file. This is required
to match the golang runtime behaviour.
Signed-off-by: Pavel Mores <pmores@redhat.com>
Also included (as commented out) is a test that does not pass although
it should. See source code comment for explanation why fixing this seems
beyond the scope of this PR.
Signed-off-by: Pavel Mores <pmores@redhat.com>
This commit focuses purely on the formal change of type. If any subsequent
changes in semantics are needed they are purposely avoided here so that the
commit can be reviewed as a 100% formal and 0% semantic change.
Signed-off-by: Pavel Mores <pmores@redhat.com>
This commit addresses a part of the same problem as PR #7623 did for the
golang runtime. So far we've been rounding up individual containers'
vCPU requests and then summing them up which can lead to allocation of
excess vCPUs as described in the mentioned PR's cover letter. We address
this by reversing the order of operations, we sum the (possibly fractional)
container requests and only then round up the total.
We also align runtime-rs's behaviour with runtime-go in that we now
include the default vcpu request from the config file ('default_vcpu')
in the total.
We diverge from PR #7623 in that `default_vcpu` is still treated as an
integer (this will be a topic of a separate commit), and that this
implementation avoids relying on 32-bit floating point arithmetic as there
are some potential problems with using f32. For instance, some numbers
commonly used in decimal, notably all of single-decimal-digit numbers
0.1, 0.2 .. 0.9 except 0.5, are periodic in binary and thus fundamentally
not representable exactly. Arithmetics performed on such numbers can lead
to surprising results, e.g. adding 0.1 ten times gives 1.0000001, not 1,
and taking a ceil() results in 2, clearly a wrong answer in vcpu
allocation.
So instead, we take advantage of the fact that container requests happen
to be expressed as a quota/period fraction so we can sum up quotas,
fundamentally integral numbers (possibly fractional only due to the need
to rewrite them with a common denominator) with much less danger of
precision loss.
Signed-off-by: Pavel Mores <pmores@redhat.com>
When hot-plugging CPUs on QEMU, we send a QMP command with JSON
arguments. QEMU 9.2 recently became more strict[1] enforcing the
JSON schema for QMP parameters. As a result, running Kata Containers
with QEMU 9.2 results in a message complaining that the core-id
parameter is expected to be an integer:
```
qmp hotplug cpu, cpuID=cpu-0 socketID=1, error:
QMP command failed:
Invalid parameter type for 'core-id', expected: integer
```
Fix that by changing the core-id, socket-id and thread-id to be
integer values.
[1]: be93fd5372Fixes: #11633
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
As we have changed the initdata annotation definition, Accordingly, we also
need correct its const definition with KATA_ANNO_CFG_RUNTIME_INIT_DATA.
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This adds SECURITY.md to the list of GH-native files that should be excluded by
the reference checker.
Today this is useful for downstreams who already have a SECURITY.md file for
compliance reasons. When Kata onboards that file, this commit will also be
required.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
When the network interface provisioned by the CNI has static ARP table entries,
the runtime calls AddARPNeighbor to propagate these to the agent. As of today,
these calls are simply rejected.
In order to allow the calls, we do some sanity checks on the arguments:
We must ensure that we don't unexpectedly route traffic to the host that was
not intended to leave the VM. In a first approximation, this applies to
loopback IPs and devices. However, there may be other sensitive ranges (for
example, VPNs between VMs), so there should be some flexibility for users to
restrict this further. This is why we introduce a setting, similar to
UpdateRoutes, that allows restricting the neighbor IPs further.
The only valid state of an ARP neighbor entry is NUD_PERMANENT, which has a
value of 128 [1]. This is already enforced by the runtime.
According to rtnetlink(7), valid flag values are 8 and 128, respectively [2],
thus we allow any combination of these.
[1]: https://github.com/torvalds/linux/blob/4790580/include/uapi/linux/neighbour.h#L72
[2]: https://github.com/torvalds/linux/blob/4790580/include/uapi/linux/neighbour.h#L49C20-L53Fixes: #11664
Signed-off-by: Markus Rudy <mr@edgeless.systems>
To make it work within CI, we do alignment with kata-runtime's definition
with "io.katacontainers.config.runtime.cc_init_data".
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Print more details about the behavior of "kubectl logs", trying to understand
errors like:
https://github.com/kata-containers/kata-containers/actions/runs/16662887973/job/47164791712
not ok 1 Check the number vcpus are correctly allocated to the sandbox
(in test file k8s-sandbox-vcpus-allocation.bats, line 37)
`[ `kubectl logs ${pods[$i]}` -eq ${expected_vcpus[$i]} ]' failed with status 2
No resources found in kata-containers-k8s-tests namespace.
...
k8s-sandbox-vcpus-allocation.bats: line 37: [: -eq: unary operator expected
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This auto-detects the repo by default (instead of having to specify
KATA_DEV_MODE=true) so that forked repos can leverage the static-checks.yaml CI
check without modification.
An alternative would have been to pass the repo in static-checks.yaml. However,
because of the matrix, this would've changed the check name, which is a pain to
handle in either the gatekeeper/GH UI.
Example fork failure:
https://github.com/microsoft/kata-containers/actions/runs/16656407213/job/47142421739#step:8:75
I've tested this change to work in a fork.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
In order to have a reproducible code generation process, we need to pin
the versions of the tools used. This is accomplished easiest by
generating inside a container.
This commit adds a container image definition with fixed dependencies
for Golang proto/ttrpc code generation, and changes the agent Makefile
to invoke the update-generated-proto.sh script from within that
container.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
The generated Go bindings for the agent are out of date. This commit
was produced by running
src/agent/src/libs/protocols/hack/update-generated-proto.sh with
protobuf compiler versions matching those of the last run, according to
the generated code comments.
Since there are new RPC methods, those needed to be added to the
HybridVSockTTRPCMockImp.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
Updated versions.yaml to use Firecracker v1.12.1.
Replaced firecracker and jailer binaries under /opt/kata/bin.
Tested with kata-fc runtime on Kubernetes:
- Deployed pods using gitpod/openvscode-server
- Verified microVM startup, container access, and Firecracker usage
- Confirmed Firecracker and jailer versions via CLI
Signed-off-by: Kumar Mohit <68772712+itsmohitnarayan@users.noreply.github.com>
- "confidential_emptyDir" becomes "emptyDir" in the settings file.
- "confidential_configMap" becomes "configMap" in settings.
- "mount_source_cpath" becomes "cpath".
- The new "root_path" gets used instead of the old "cpath" to point to
the container root path..
- "confidential_guest" is no longer used. By default it gets replaced
by "enable_configmap_secret_storages"=false, because CoCo is using
CopyFileRequest instead of the Storage data structures for ConfigMap
and/or Secret volume mounts during CreateContainerRequest.
- The value of "guest_pull" becomes true by default.
- "image_layer_verification" is no longer used - just CoCo's guest pull
is supported.
- The Request input files from unit tests are changing to reflect the
new default settings values described above.
- tests/integration/kubernetes/tests_common.sh adjusts the settings for
platforms that are not set-up for CoCo during CI (i.e., platforms
other than SNP, TDX, and CoCo Dev).
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Skip pulling container image layers when guest-pull=true. The contents
of these layers were ignored due to:
- #11162, and
- tarfs snapshotter support having been removed from genpolicy.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
AKS Confidential Containers are using the tarfs snapshotter. CoCo
upstream doesn't use this snapshotter, so remove this Policy complexity
from upstream.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
`mem-agent` here is now a library and do not contain examples, ignore
Cargo.lock to get rid of untracked file noise produced by `cargo run` or
`cargo test`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Re-generates the client code against Cloud Hypervisor v47.0.
Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by openapi-generator.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`MmapRegion` is only used while `virtio-fs` is enabled during testing
dragonball, gate the import behind `virtio-fs` feature.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Some variables went unused if certain features are not enabled, use
`#[allow(unused)]` to suppress those warnings at the time being.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`VcpuManagerError` is only needed when `host-device` feature is enabled,
gate the import behind that feature.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Code inside `test_mac_addr_serialization_and_deserialization` test does
not actually require this `with-serde` feature to test, removing the
assertion here to enable this test.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Add full cgroups support on host. Cgroups are managed by `FsManager` and
`SystemdManager`. As the names impies, the `FsManager` manages cgroups
through cgroupfs, while the `SystemdManager` manages cgroups through
systemd. The two manages support cgroup v1 and cgroup v2.
Two types of cgroups path are supported:
1. For colon paths, for example "foo.slice:bar:baz", the runtime manages
cgroups by `SystemdManager`;
2. For relative/absolute paths, the runtime manages cgroups by
`FsManager`.
vCPU threads are added into the sandbox cgroups in cgroup v1 + cgroupfs,
others, cgroup v1 + systemd, cgroup v2 + cgroupfs, cgroup v2 + systemd, VMM
process is added into the cgroups.
The systemd doesn't provide a way to add thread to a unit. `add_thread()`
in `SystemdManager` is equivalent to `add_process()`.
Cgroup v2 supports threaded mode. However, we should enable threaded mode
from leaf node to the root node (`/`) iteratively [1]. This means the
runtime needs to modify the cgroups created by container runtime (e.g.
containerd). Considering cgroupfs + cgroup v2 is not a common combination,
its behavior is aligned with systemd + cgroup v2, which is not allowed to
manage process at the thread level.
1: https://www.kernel.org/doc/html/v4.18/admin-guide/cgroup-v2.html#threadsFixes: #11356
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
As some reasons, it first should make it align with runtime-go, this
commit will do this work.
Fixes#11543
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
The actual memory usage on the host is equal to the hypervisor memory usage
plus the user memory usage. An OOM killer might kill the shim when the
memory limit on host is same with that of container and the container
consumes all available memory. In this case, the containerd will never
receive OOM event, but get "task exit" event. That makes the `k8s-oom.bats`
test fail.
The fix is to add a new container to increase the sandbox memory limit.
When the container "oom-test" is killed by OOM killer, there is still
available memory for the shim, so it will not be killed.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
When enabling systemd cgroup driver and sandbox cgroup only, the shim is
under a systemd unit. When the unit is stopping, systemd sends SIGTERM to
the shim. The shim can't exit immediately, as there are some cleanups to
do. Therefore, ignoring SIGTERM is required here. The shim should complete
the work within a period (Kata sets it to 300s by default). Once a timeout
occurs, systemd will send SIGKILL.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Our CI keeps on getting
```
jq: error (at <stdin>:1): Cannot index string with string "tag_name"
```
during the install dependencies phase, which I suspect
might be due to github rate limits being reduced, so try
to pass through the `GH_TOKEN` env and use it in the auth header.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
It is important that we continue to support VirtIO-SCSI. While
VirtIO-BLK is a common choice, virtio-scsi offers significant
performance advantages in specific scenarios, particularly when
utilizing iothreads and with NVMe Fabrics.
Maintaining Flexibility and Choice by supporting both virtio-blk and
virtio-scsi, we provide greater flexibility for users to choose the
optimal storage（virtio-blk, virtio-scsi) interface based on their
specific workload requirements and hardware configurations.
As virtio-scsi controller has been created when qemu vm starts with
block device driver is set to `virtio-scsi`. This commit is for blockdev_add
the backend block device and device_add frondend virtio-scsi device via qmp.
Fixes#11516
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
As block device index is an very important unique id of a block device
and can indicate a block device which is equivalent to device_id.
In case of index is required in calculating scsi LUN and reduce
useless arguments within reusing `hotplug_block_device`, we'd better
change the device_id with block device index.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
In this commit, block device aio are introduced within hotplug_block_device
within qemu via qmp and the "iouring" is set the default.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
It should be correctly handled within the device manager when do
create_block_device if the driver_option is virtio-scsi.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
It supports handling scsi device when block device driver is `scsi`.
And it will ensure a correct storage source with LUN.
Fixes#11516
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
It's used to help discover scsi devices inside guest and also add a
new const value `KATA_SCSI_DEV_TYPE` to help pass information.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
AIO is the I/O mechanism used by qemu with options:
- threads
Pthread based disk I/O.
- native
Native Linux I/O.
- io_uring (default mode)
Linux io_uring API. This provides the fastest I/O operations on
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Although Previous implementation of hotplugging block device via QMP
can successfully hot-plug the regular file based block device, but it
fails when the backend is /dev/xxx(e.g. /dev/loop0). With analysis about
it, we can know that it lacks the ablility to hotplug host block devices.
This commit will fill the gap, and make it work well for host block
devices.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
As there were a few moderate security vulnerability fixes missed as part
of the 3.19.0 release.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
For the release itself, let's simply copy the VERSION file to the
tarball.
To do so, we had to change the logic that merges the build, as at that
point the tag is not yet pushed to the repo.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
On commit 90bc749a19, we've changed the
QEMUTDXPATH in order to get it to work with GPUs, but the change broke
the non-GPU TDX use-case, which depends on the distro binary.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
- Add nodeSelector configuration to values.yaml with empty default
- Update DaemonSet template to conditionally include nodeSelector
- Add documentation and examples for nodeSelector usage in README
- Allows users to restrict kata-containers deployment to specific nodes by labeling them
Signed-off-by: Gus Minto-Cowcher <gus@basecamp-research.com>
According to the issue [1], Tokio will panic when we are giving a blocking
socket to Tokio's `from_std()` method, the information is as follows:
```
A panic occurred at crates/agent/src/sock/vsock.rs:59: Registering a
blocking socket with the tokio runtime is unsupported. If you wish to do
anyways, please add `--cfg tokio_allow_from_blocking_fd` to your RUSTFLAGS.
```
A workaround is to set the socket to non-blocking.
1: https://github.com/tokio-rs/tokio/issues/7172
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
The KERNEL_DEBUG_ENABLED was missing in the outer shell script
so overrides via make were not possible.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Bump these crates to remove the unmaintained dependency
proc-macro-error and remediate RUSTSEC-2024-0370
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Bump these crates across various components to remove the
dependency on unmaintained instant crate and remediate
RUSTSEC-2024-0384
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- The github generated template had an old version which
isn't valid for the pr-scan, so update to the latest
- The action needs also `actions: read` and `contents:read` to run in kata-containers
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Some of the nix apis we are using are now enabled by features,
so add these to resolve the compilation issues
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This new version of gc fixes s390x attestation, also introduces registry
configuration setting directly via initdata.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
The peer pods project is using the agent-ctl tool in some
tests, so tagging our cache will let them more easily identify
development versions of kata for testing between releases.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Sometimes, containers or execs do not use stdin, so there is no chance
to add parent stdin to the process's writer hashmap, resulting in the
parent stdin's fd not being closed when the process is cleaned up later.
Therefore, when creating a process, first explicitly add parent stdin to
the wirter hashmap. Make sure that the parent stdin's fd can be closed
when the process is cleaned up later.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
We want to be able to build a debug version of the kernel for various
use-cases like debugging, tracing and others.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The convention for rootfs-* names is:
* rootfs-${image_type}-${special_build}
If this is not followed, cache will never work as expected, leading to
building the initrd / image on every single build, which is specially
constly when building the nvidia specific targets.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
The init data could not be read properly within kata-agent because the
data length field was omitted, a consequence of a mismatch in the data
write format.
Fixes#11556
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Now AA supports to receive initdata toml plaintext and deliver it in the
attestation. This patch creates a file under
'/run/confidential-containers/initdata'
to store the initdata toml and give it to AA process.
When we have a separate component to handle initdata, we will move the
logic to that component.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
Update to https://github.com/teawater/mem-agent/tree/kata-20250627.
The commit list:
3854b3a Update nix version from 0.23.2 to 0.30.1
d9a4ced Update tokio version from 1.33 to 1.45.1
9115c4d run_eviction_single_config: Simplify check evicted pages after
eviction
68b48d2 get_swappiness: Use a rounding method to obtain the swappiness
value
14c4508 run_eviction_single_config: Add max_seq and min_seq check with
each info
8a3a642 run_eviction_single_config: Move infov update to main loop
b6d30cf memcg.rs: run_aging_single_config: Fix error of last_inc_time
check
54fce7e memcg.rs: Update anon eviction code
41c31bf cgroup.rs: Fix build issue with musl
0d6aa77 Remove lazy_static from dependencies
a66711d memcg.rs: update_and_add: Fix memcg not work after set memcg
issue
cb932b1 Add logs and change some level of some logs
93c7ad8 Add per-cgroup and per-numa config support
092a75b Remove all Cargo.lock to support different versions of rust
540bf04 Update mem-agent-srv, mem-agent-ctl and mem-agent-lib to
v0.2.0
81f39b2 compact.rs: Change default value of compact_sec_max to 300
c455d47 compact.rs: Fix psi_path error with cgroup v2 issue
6016e86 misc.rs: Fix log error
ded90e9 Set mem-agent-srv and mem-agent-ctl as bin
Fixes: #11478
Signed-off-by: teawater <zhuhui@kylinos.cn>
As the following job has passed 10 days in a row for the nightly test:
```
kata-containers-ci-on-push / run-k8s-tests-on-zvsi / run-k8s-tests (nydus, qemu-coco-dev, kubeadm)
```
this commit makes the job required again.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The teardown_common will print the description of the running pods, kill
them all and print the system's syslogs afterwards.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Set the node in the spec template of a Job manifest, allowing to use
set_node() on tests like k8s-parallel.bats
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The teardown_common will print the description of the running pods, kill
them all and print the system's syslogs afterwards.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The teardown_common will print the description of the running pods, kill
them all and print the system's syslogs afterwards.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The previous description for the `block_device_driver` was inaccurate or
outdated. This commit updates the documentation to provide a more
precise explanation of its function.
Fixes#11488
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
When we run a kata pod with runtime-rs/qemu and with a default
configuration toml, it will fail with error "unsupported driver type
virtio-scsi".
As virtio-scsi within runtime-rs is not so popular, we set default block
device driver with `virtio-blk-*`.
Fixes#11488
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This patch changes the container process HashMap to use exec_id as the primary
key instead of PID, preventing exec_id collisions that could be exploited in
Confidential Computing scenarios where the host is less trusted than the guest.
Key changes:
- Changed `processes: HashMap<pid_t, Process>` to `HashMap<String, Process>`
- Added exec_id collision detection in `start()` method
- Updated process lookup operations to use exec_id directly
- Simplified `get_process()` with direct HashMap access
This prevents multiple exec operations from reusing the same exec_id, which
could be problematic in CoCo use cases where process isolation and unique
identification are critical for security.
Signed-off-by: Ankita Pareek <ankitapareek@microsoft.com>
The `/opt/kata/VERSION` file, which is created using `git describe
--tags`, requires the newly released tag to be updated in order to be
accurate.
To do so, let's add a `fetch-tags: true` to the checkout action used
during the `create-kata-tarball` job.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
The CoCo non-TEE job (run-k8s-tests-coco-nontee) used to be required but
we had to withdraw it to fix a problem (#11156). Now the job is back
running and stable, so time to make it required again.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
tempdir hasn't been updated for seven years and pulls in
remove_dir_all@0.5.3 which has security advisory
GHSA-mc8h-8q98-g5hr, so replace this with using tempfile,
which the crate got merged into and we use elsewhere in the
project
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Ignore Cargo.lock in `libs` to prevent developers from accidentally
track lock files in `libs` workspace.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
This PR adds support for adding a network device before starting the
cloud-hypervisor VM.
This commit will get the host devices from NamedHypervisorConfig and
assign it to VmConfig's devices which is for vfio devices when clh
starts launching.
And with this, it successfully finish the vfio devices conversion from
a generic Hypervisor config to a clh specific VmConfig.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
This commit introduce `host_devices` to help convert vfio devices from
a generic hypervisor config to a cloud-hypervisor specific VmConfig.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
This PR adds support for adding a vfio device before starting the
cloud-hypervisor VM (or cold-plug vfio device).
This commit changes "pending_devices" for clh implementation via adding
DeviceType::Vfio() into pending_devices. And it will get shared host devices
after correctly handling vfio devices (Specially for primary device).
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
crates in `libs` workspace do not ship binaries, they are just libraries
for other workspace to reference, the `Cargo.lock` file hence would not
take effect. Removing Cargo.lock for `libs` workspace.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
In line with configuration for other TEEs, shared_fs should
be set to none for IBM SEL. This commit updates the value for
runtime/runtime-rs.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
As we're using a `kubectl wait --timeout ...` to check whether the
kata-deploy pod's been deleted or not, let's remove the `--wait` from
the `helm uninstall ...` call as k0s tests were failing because the
`kubectl wait --timeout...` was starting after the pod was deleted,
making the test fail.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
We've been pinning a specific version of k0s for CRI-O tests, which may
make sense for CRI-O, but doesn't make sense at all when it comes to
testing that we can install kata-deploy on latest k0s (and currently our
test for that is broken).
Let's bump to the latest, and from this point we start debugging,
instead of debugging on an ancient version of the project.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
Bump url, reqwests and idna crates in order to move away from
idna <1.0.3 and remediate CVE-2024-12224.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Previously, the rootlessDir variable in `src/runtime/virtcontainers/pkg/rootless.go` was initialized at
package load time using `os.Getenv("XDG_RUNTIME_DIR")`. However, in rootless
VMM mode, the correct value of $XDG_RUNTIME_DIR is set later during runtime
using os.Setenv(), so rootlessDir remained empty.
This patch defers the initialization of rootlessDir until the first call
to `GetRootlessDir()`, ensuring it always reflects the current environment
value of $XDG_RUNTIME_DIR.
Fixes: #11526
Signed-off-by: stevenfryto <sunzitai_1832@bupt.edu.cn>
There are workflows that rely on `az aks install-cli` to get kubectl
installed. There is a well-known problem on install-cli, related with
API usage rate limit, that has recently caused the command to fail
quite often.
This is replacing install-cli with the azure/setup-kubectl github
action which has no such as rate limit problem.
While here, removed the install_cli() function from gha-run-k8s-common.sh
so avoid developers using it by mistake in the future.
Fixes#11463
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Removing kernel config files realting
to SEV as part of the SEV deprecation
efforts.
Co-authored-by: Adithya Krishnan Kannan <AdithyaKrishnan.Kannan@amd.com>
Signed-off-by: Arvind Kumar <arvinkum@amd.com>
Removing runtime SEV functionality,
such as the kbs, ovmf, VMSA handling,
and SEV configs as part of deprecating
SEV from kata.
Co-authored-by: Adithya Krishnan Kannan <AdithyaKrishnan.Kannan@amd.com>
Signed-off-by: Arvind Kumar <arvinkum@amd.com>
Removing files related to SEV, responsible for
installing and configuring Kata containers.
Co-authored-by: Adithya Krishnan Kannan <AdithyaKrishnan.Kannan@amd.com>
Signed-off-by: Arvind Kumar <arvinkum@amd.com>
Add init data annotation within preparing remote hypervisor annotations
when prepare vm, so that it can be passed within CreateVMRequest.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
679cc9d47c was merged and bumped the
podoverhead for the gpu related runtimeclasses. However, the bump on the
`kata-runtimeClasses.yaml` as overlooked, making our tests fail due to
that discrepancy.
Let's just adjust the values here and move on.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
We cannot only rely only on default_cpu and default_memory in the
config, default is 1 and 2Gi but we need some overhead for QEMU and
the other related binaries running as the pod overhead. Especially
when QEMU is hot-plugging GPUs, CPUs, and memory it can consume more
memory.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
atty is unmaintained, with the last release almost 3 years
ago, so we don't need to check for updates, but instead will
remove it from out dependency tree.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
structopt features were integrated into clap v3 and so is not
actively updated and pulls in the atty crate which has a security
advisory, so update clap, remove structopts, update the code that
used it to remove the outdated dependencies.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
slog-term 2.9.0 included atty, which is unmaintained
as has a security advisory GHSA-g98v-hv3f-hcfr,
so bump the version across our components to remove
this dependency.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
We had the proper config.toml configuration for static builds
but were building the glibc target and not the musl target.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The way GH works, we can only require Zizmor results on ALL PR runs, or
none, so remove the path filter.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Previously, the source field was subject to mandatory checks. However,
in guest-pull mode, this field doesn't consistently provide useful
information. Our practical experience has shown that relying on this
field for critical data isn't always necessary.
In other aspect, not all cases need mandatory check for KataVirtualVolume.
based on this fact, we'd better to make from_base64 do only one thing and
remove the validate(). Of course, We also keep the previous capability to
make it easy for possible cases which use such method and we rename it
clearly with from_base64_and_validate.
This commit relaxes the mandatory checks on the KataVirtualVolume specifically
for guest-pull mode, acknowledging its diminished utility in this context.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
When hot plugging vcpu in dragonball hypervisor, use the synchronization
interface and wait until the hot plug cpu is executed in the guest
before returning. This ensures that the subsequent device hot plug will
not conflict with the previous call.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Let dragonball's resize_vcpu api support synchronization, and only
return after the hot-plug of the CPU is successfully executed in the
guest kernel. This ensures that the subsequent device hot-plug operation
can also proceed smoothly.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Got follow warning with make test of kata-agent:
Compiling rustjail v0.1.0 (/data/teawater/kata-containers/src/agent/rustjail)
Compiling kata-agent v0.1.0 (/data/teawater/kata-containers/src/agent)
warning: unused import: `std::os::unix::fs`
--> rustjail/src/mount.rs:1147:9
|
1147 | use std::os::unix::fs;
| ^^^^^^^^^^^^^^^^^
|
= note: `#[warn(unused_imports)]` on by default
This commit fixes it.
Fixes: #11508
Signed-off-by: teawater <zhuhui@kylinos.cn>
Introduce a const value `KATA_VIRTUAL_VOLUME_PREFIX` defined in the libs/kata-types,
and it'll be better import such const value from there.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This was originally implemented as a Jenkins skip and is only used in a few
workflows. Nowadays this would be better implemented via the gatekeeper.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This patch fixes the rules.rego file to ensure that the
policy is correctly parsed and applied by opa.
Signed-off-by: Archana Choudhary <archana1@microsoft.com>
This commit updates the `tests_common.sh` script
to enable the `confidential_guest`
setting for the coco tests in the Kubernetes
integration tests.
Signed-off-by: Archana Choudhary <archana1@microsoft.com>
This patch removes storages from the testcases.json file for execprocess.
This is because input storage objects are invalid for two reasons:
1. "io.katacontainers.fs-opt.layer=" is missing option in annotations.
2. by default, we don't have host-tarfs-dm-verity enabled, so the storage
objects are not created in policy.
Signed-off-by: Archana Choudhary <archana1@microsoft.com>
---
This patch introduces some basic checks for the
`image_guest_pull` storage type in the genpolicy tool.
Signed-off-by: Archana Choudhary <archana1@microsoft.com>
This patch improves the test framework for the
genpolicy tool by enabling the use of config maps.
Signed-off-by: Archana Choudhary <archana1@microsoft.com>
Add the definiation of variable DEFCREATECONTAINERTIMEOUT into
Makefile target with default timeout 30s.
Fixes: #485
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
It's used to indicate timeout value set for image pulling in
guest during creating container.
This allows users to set this timeout with annotation according to the
size of image to be pulled.
Fixes#10692
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
It allows users to set this create container timeout within
configuration.toml according to the size of image to be pulled
inside guest.
Fixes#10692
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
To better understand the impact of different timeout values on system
behavior, this section provides a more comprehensive explanation of the
request_timeout_ms:
This timeout value is used to set the maximum duration for the agent to
process a CreateContainerRequest. It's also used to ensure that workloads,
especially those involving large image pulls within the guest, have sufficient
time to complete.
Based on explaination above, it's renamed with `create_container_timeout`,
Specially, exposed in 'configuration.toml'
Fixes#10692
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This helps considerably to avoid patching the code, and just adjusting
the build environment to use a smaller alignment than the default one.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
These tests are not passing, or being maintained,
so as discussed on the AC meeting, we will skip them
from automatically running until they can be reviewed
and re-worked, so avoid wasting CI cycles.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This adds Zizmor GHA security scanning as a PR gate.
Note that this does NOT require that Zizmor returns 0 alerts, but rather
that Zizmor's invocation completes successfully (regardless of how many
alerts it raises).
I will set up the former after this commit is merged (through the GH UI).
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Enable GPU annotations by adding `default_gpus` and `default_gpu_model`
into the list of valid annotations `enable_annotations`.
Fixes#10484
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Add GPU specific annotations used by remote hypervisor for instance
selection during `prepare_vm`.
Fixes#10484
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Two annotations: `default_gpus and `default_gpu_model` as GPU annotations
are introduced for Kata VM configurations to improve instance selection on
remote hypervisors. By adding these annotations:
(1) `default_gpus`: Allows users to specify the minimum number of GPUs a VM
requires. This ensures that the remote hypervisor selects an instance
with at least that many GPUs, preventing resource under-provisioning.
(2) `default_gpu_model`: Lets users define the specific GPU model needed for
the VM. This is crucial for workloads that depend on particular GPU archs or
features, ensuring compatibility and optimal performance.
Fixes#10484
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
To provide the remote hypervisor with the necessary intelligence
to select the most appropriate instance for a given GPU instance,
leading to better resource allocation, two fields `default_gpus`
and `default_gpu_model` are introduced in `RemoteInfo`.
Fixes#10484
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
To better support containerd 2.1 and later versions, remove the
hardcoded `layer.erofs` and instead parse `/proc/mounts` to obtain the
real mount source (and `/sys/block/loopX/loop/backing_file` if needed).
If the mount source doesn't end with `layer.erofs`, it should be marked
as unsupported, as it may be a filesystem meta file generated by later
containerd versions for the EROFS flattened filesystem feature.
Also check whether the filesystem type is `overlay` or not, since the
containerd mount manager [1] may change it after being introduced.
[1] https://github.com/containerd/containerd/issues/11303
Fixes: f63ec50ba3 ("runtime: Add EROFS snapshotter with block device support")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Current Dockerfile fails when trying to build from the root of the repo
docker build -t kata-monitor -f tools/packaging/kata-monitor/Dockerfile .
with "invalid go version '1.23.0': must match format 1.23"
Using go 1.23 in the Dockerfile fixes the build error
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
I notices that agent-ctl is including a 9 month old version of
image-rs and the libs crates haven't been update for potentially
many years, so bump all of these.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This commit introduces the ability to run Pods without shared fs
mechanism in Kata.
The default shared fs can lead to unnecessary resource consumption
and security risks for certain use cases. Specifically, scenarios
where files only need to be copied into the VM once at Pod creation
(e.g., non-tee envs) and don't require dynamic updates make the shared
fs redundant and inefficient.
By explicitly disabling shared fs functionality, we reduce resource
overhead and shrink the attack surface. Users will need to employ
alternative methods(e.g. guest-pull) to ensure container images are
shared into the guest VM for these specific scenarios.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
In the pre commit:74eccc54e7b31cc4c9abd8b6e4007c3a4c1d4dd4,
it missed return the right rootfs volume.
In the is_block_rootfs fn, if the rootfs is based on a
block device such as devicemapper, it should clear the
volume's source and let the device_manager to use the
dev_id to get the device's host path.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
For containerd's Blockfile Snapshotter, it will pass
a rootfs mounts with a rawfile as a mount source
and mount options with "loop" embeded.
To support this type of rootfs, it is necessary to identify this as a
blockfile rootfs through the "loop" flag, and then use the volume source
of the rootfs as the source of the block device to hot-insert it into
the guest.
Fixes:#11464
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Instead of building it every time, we can store the regorus
binary in OCI registry using oras and download it from there.
This reduces the install time from ~1m40s to ~15s.
Signed-off-by: Paul Meyer <katexochen0@gmail.com>
This commit add support of resize_vcpu for cloud-hypervisor
using the it's vm resize api. It can support bothof vcpu hotplug
and hot unplug.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
For cloud-hypervisor, currently only hot plugging of memory is
supported, but hot unplugging of memory is not supported. In addition,
by default, cloud-hypervisor uses ACPI-based memory hot-plugging instead
of virtio-mem based memory hot-plugging.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Add API interfaces for get vminfo and resize. get vminfo can obtain the
memory size and number of vCPUs from the cloud hypervisor vmm in real
time. This interface provides information for the subsequent resize
memory and vCPU.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
The system's own Deserialize cannot implement parsing from string to
MacAddr, so we need to implement this trait ourself.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
To make it flexibility and extensibility This change modifies the Kata
Agent's handling of `InitData` to allow for unrecognized key-value pairs.
The `InitData` field now directly utilizes `HashMap<String, String>`,
enabling it to carry arbitrary metadata and information that may be
consumed by other components
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
During sandbox preparation, initdata should be specified to TdxConfig,
specially mrconfigid, which is used to pass to tdx guest report for
measurement.
Fixes#11180
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
SEV-SNP guest configuration utilizes a different set of properties
compared to the existing 'sev-guest' object. This change introduces
the `host-data` property within the sev-snp-guest object. This property
allows for configuring an SEV-SNP guest with host-provided data, which
is crucial for data integrity verification during attestation.
The `host-data` property is specifically valid for SEV-SNP guests
running
on a capable platform. It is configured as a base64-encoded string when
using the sev-snp-guest object.
the example cmdline looks like:
```shell
-object sev-snp-guest,id=sev-snp0,host-data=CGNkCHoBC5CcdGXir...
```
Fixes#11180
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
To facilitate the transfer of initdata generated during
`prepare_initdata_device_config`, a new parameter has been
introduced into the `prepare_protection_device_config` function.
Furthermore, to specifically pass initdata to SEV-SNP Guests, a
`host_data` field has been added to the `SevSnpConfig` structure.
However, this field is exclusively applicable to the SEV-SNP platform.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Retrieve the Initdata string content from the security_info of the
Configuration. Based on the Protection Platform type, calculate the
digest of the Initdata. Write the Initdata content to the block
device. Subsequently, construct the BlockConfig based on this block
device information.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
To correctly manage initdata as a block device, a new InitData
Resource type, inherently a block device, has been introduced
within the ResourceManager. As a component of the Sandbox's
resources, this InitData Resource needs to be appropriately
handled by the Device Manager's handler.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This commit implements the retrieval and processing of InitData provided
via a Pod annotation. Specifically, it enables runtime-rs to:
(1) Parse the "io.katacontainers.config.hypervisor.cc_init_data"
annotation from the Pod YAML.
(2) Perform reverse operations on the annotation value: base64 decoding
followed by gzip decompression.
(3) Deserialize the decompressed data into the internal InitData
structure.
(4) Serialize the resulting InitData into a string and store it in the
Configuration.
This allows users to inject configuration data into the TEE Guest by
encoding and compressing it and passing it as an annotation in the Pod
configuration. This mechanism supports scenarios where dynamic config
is required for Confidential Containers.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This commit introduces the Initdata Spec and the logic for
calculating its digest. It includes:
(1) Define a `ProtectedPlatform` enum to represent major TEE platform
types.
(2) Create an `InitData` struct to support building and serializing
initialization data in TOML format.
(3) Implement adaptation for SHA-256, SHA-384, and SHA-512 digest
algorithms.
(4) Provide a platform-specific mechanism for adjusting digest lengths
(zero-padding).
(5) Supporting the decoding and verification of base64+gzip encoded
Initdata.
The core functionality ensures the integrity of data injected by the
host through trusted algorithms, while also accommodating the
measurement requirements of different TEE platforms.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This commit introduces a new `initdata` field of type String to
hypervisor `SecurityInfo`.
In accordance with the Initdata Specification, this field will
facilitate the injection of well-defined data from an untrusted host
into the TEE. To ensure the integrity of this injected data, the TEE
evidence's hostdata capability or the (v)TPM dynamic measurement
capability will be leveraged, as outlined in the specification.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Don't use local launched_pods variable in test_rc_policy(), because
teardown() needs to use this variable to print a description of the
pods, for debugging purposes.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The locking mechanism around the layers cache file was insufficient to
prevent corruption of the file. This commit moves the layers cache's
management in-memory, only reading the cache file once at the beginning
of `genpolicy`, and only writing to it once, at the end of `genpolicy`.
In the case that obtaining a lock on the cache file fails,
reading/writing to it is skipped, and the cache is not used/persisted.
Signed-off-by: charludo <git@charlotteharludo.com>
`vmm-sys-util` was duplicated while updating the `ignore` list of
`rust-vmm` crates in #11431, remove duplicated one and sort the list.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
When moving from clap v2 to v4 a bunch of
functions have been removed, so update the code
to handle these replacements
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
When moving from clap v2 to v4 a bunch of
functions have been removed, so update the code
to handle these replacements
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Update dependabot ignore list in cargo ecosystem to ignore upgrades from
rust-vmm crates, since those crates need to be managed carefully and
manually.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Pin Github owned actions to specific hashes as recommended
as tags are mutable see https://pin-gh-actions.kammel.dev/.
This one of the recommendations that scorecard gives us.
Note this was generated with `frizbee actions`
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Fedora 40 is EoL, and I've seen the registry pull fail
a few times recently, so let's bump to fedora 42 which
has 10 months of support left.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Now we are decoupled from the image-rs crate,
we can bump the protobuf version across our project
to resolve the GHSA-2gh3-rmm4-6rq5 advisory
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This patch updates the container image for the CI test workloads:
- `k8s-layered-sc-deployment.yaml`
- `k8s-pod-sc-deployment.yaml`
- `k8s-pod-sc-nobodyupdate-deployment.yaml`
- `k8s-pod-sc-supplementalgroups-deployment.yaml`
- `k8s-policy-deployment.yaml`
Also updates unit tests:
- `test_create_container_security_context`
- `test_create_container_security_context_supplemental_groups`
This fixes tests failing due to an image pull error as the previous image is no longer available in
the container registry.
Signed-off-by: Archana Choudhary <archana1@microsoft.com>
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
Switch the hyper for an underscore, so the ghcr
helm publish can work properly.
Co-authored-by: Fabiano Fidêncio <fidencio@northflank.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Only sign the kernel if the user has provided the KBUILD_SIGN_PIN
otherwise ignore.
Whole here, let's move the functionality to the common fragments as it's
not a GPU specific functionality.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
At the moment if any of the tests in the matric fails
then the rest of the jobs are cancelled, so we have to
re-run everything. Add `fail-fast: false` to stop this
behaviour.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Remove the rule that causes gatekeeper to skip tests
if we've only updated the required-tests.yaml list.
Although update to just the required-tests.yaml
doesn't change the outcome of any of the CI tests, it
does change whether gatekeeper will still pass with the new
rules. Although it's a bit of a hit to run the CI, it's probably
worth it to keep gatekeeper validated.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This adds govulncheck vulnerability scanning as a non-blocking check in
the static checks workflow. The check scans Go runtime binaries for known
vulnerabilities while filtering out verified false positives.
Signed-off-by: Mitch Zhu <mitchzhu@microsoft.com>
Since #11197 was merged, all confidential k8s e2e tests for s390x
have been failing with the following errors:
```
attestation-agent: error while loading shared libraries:
libcurl.so.4: cannot open shared object file
libnghttp2.so.14: cannot open shared object file
```
In line with the update on x86_64, we need to upgrade the OS used
in rootfs-{image,initrd} on s390x.
This commit also bumps all 22.04 to 24.04 for all architectures.
For s390x, this ensures the missing packages listed above are
installed.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
After commit a3f973db3b merged, protection::GuestProtection::[Snp,Sev]
have changed to tuple variants, and can no longer be used in assert_eq
marco without tuple values, or some errors will raised:
```
assert_eq!(actual.unwrap(), GuestProtection::Snp);
| ^^^^^^^^^^^^^^^^^^^^ expected \
`GuestProtection`, found enum constructor
```
Signed-off-by: Lei Liu <liulei.pt@bytedance.com>
containerd-sandboxapi fails with `containerd v2.0.x` and passes with
`containerd v1.7.x` regardless kata-containers. And it was not tested
with `containerd v2.0.x` because `containerd v2.0.x` could not
recognize `[plugins.cri.containerd]` in `config.toml`.
Signed-off-by: Seunguk Shin <seunguk.shin@arm.com>
the latest Canonical TDX release supports 25.04 / Plucky as
well. Users experimenting with the latest goodies in the
25.04 TDX enablement won't get Kata deployed properly.
This change accepts 25.04 as supported distro for TDX.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Container image integrity protection is a critical practice involving a
multi-layered defense mechanism. While container images inherently offer
basic integrity verification through Content-Addressable Storage (CAS)
(ensuring pulled content matches stored hashes), a combination of other
measures is crucial for production environments. These layers include:
Encrypted Transport (HTTPS/TLS) to prevent tampering during transfer;
Image Signing to confirm the image originates from a trusted source;
Vulnerability Scanning to ensure the image content is "healthy"; and
Trusted Registries with stringent access controls.
In certain scenarios, such as when container image confidentiality
requirements are not stringent, and integrity is already ensured via the
aforementioned mechanisms (especially CAS and HTTPS/TLS), adopting
"force guest pull" can be a viable option. This implies that even when
pulling images from a container registry, their integrity remains
guaranteed through content hashes and other built-in mechanisms, without
relying on additional host-side verification or specialized transfer
methods.
Since this feature is already available in runtime-go and offers
synergistic benefits with guest pull, we have chosen to support force
guest pull.
Fixes#10690
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This commit introduces the `adjust_rootfs_mounts` function to manage
root filesystem mounts for guest-pull scenarios.
When the force guest-pull mechanism is active, this function ensures that
the rootfs is exclusively configured via a dedicated `KataVirtualVolume`.
It disregards any provided input mounts, instead generating a single,
default `KataVirtualVolume`. This volume is then base64-encoded and set
as the sole mount option for a new, singular `Mount` entry, which is
returned as the only item in the `Vec<Mount>`.
This change guarantees consistent and exclusive rootfs configuration
when utilizing guest-pull for container images.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
In CoCo scenarios, there's no image pulling on host side, and it will
disable such operations, that's to say, there's no files sharing between
host and guest, especially for container rootfs.
We introduce Kata Virtual Volume to help handle such cases:
(1) Introduce is_kata_virtual_volume to ensure the volume is kata
virtual volume.
(2) Introduce VirtualVolume Handling logic in handle_rootfs when the
mount is kata virtual volume.
Fixes#10690
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This commit introduces comprehensive support for rootfs mount mgmt
through Kata Virtual Volumes, specifically enabling the guest-pull
mechanism.
It enhances the runtime's ability to:
(1) Extract image references from container annotations (CRI/CRI-O).
(2) Process `KataVirtualVolume` objects, configuring them for guest-pull operations.
(3) Set up the agent's storage for guest-pulled images.
This functionality streamlines the process of pulling container images
directly within the guest for rootfs, aligning with guest-side image management strategies.
Fixes#10690
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
The Multistrap issue has been fixed in noble thus we can use the LTS.
Also, this will fix the error reported by CDH
```
/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found
```
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
The new version of AA allows the config not having a coco_as token
config. If not provided, it will mark as None.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
This patch updates the guest-components to new version with better
error logging for CDH. It also allows the config of AA not having a
coco_as token config.
Also, the new version of CDH requires to build aws-lc-sys thus needs to
install cmake for build.
See
https://github.com/kata-containers/kata-containers/actions/runs/15327923347/job/43127108813?pr=11197#step:6:1609
for details.
Besides, the new version of guest-components have some fixes for SNP
stack, which requires the updates of trustee side.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
This new parameter for kata-agent is used to control the timeout for a
guest pull request. Note that sometimes an image can be really big, so
we set default timeout to 1200 seconds (20 minutes).
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
After moving image pulling from kata-agent to CDH, the failed image pull
error messages have been slightly changed. This commit is to apply for
the change.
Note that in original and current image-rs implementation, both no key
or wrong key will result in a same error information.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
Now kata-agent by default supports both guest pull and host pull
abilities, thus we do not need to specify the PULL_TYPE env when
building kata-agent.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
After moving guest pull abilities to CDH, the document of guest pull
should be updated due to new workflow.
Also, replace the diagram of PNG into a mermaid one for better
maintaince.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
In previous version, only when the `guest-pull` feature is enabled
during the build time, the OCI process will be tried to be overrided
when the storage has a guest pull volume and also it is sandbox. After
getting rid of the feature, whether it is guest-pull is runtimely
determined thus we can always do this trying override, by checking if
there is kata guest pull volume in storages and it's sandbox.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
Now the ocicrypt configuration used by CDH is always the same and it's
not a good practics to write it into the rootfs during runtime by
kata-agent. Thus we now move it to coco-guest-components build script.
The config will be embedded into guest image/initrd together with CDH
binary.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
The feature `guest-pull` and `default-pull` are both removed, because
both guest pull and host pull are supported in building time without
without involving new dependencies like image-rs before. The guest pull
will depend on the CDH process, not the build time feature.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
This is a higher level calling to pull image inside guest. Now it should
call confidential_data_hub's API. As the previous pull_image API does
1. check is sandbox
2. generate bundle_path
inside the original logic, and the new API does not do them to keep the
API semantice clean, thus before we call the API, we explicitly do the
two things.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
now image pull ability is moved to CDH, thus the CDH process needs
environment variables of ocicrypt to help find the keyprovider(cdh) to
decrypt images.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
As image pull ability is moved to CDH, kata-agent does not need the
confugurations of image pulling anymore.
All these configurations reading from kernel cmdline is now implemented
by CDH.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
Image pull abilities are all moved to the separate component
Confidential Data Hub (CDH) and we only left the auxiliary functions
except pull_image in confidential_data_hub/image.rs
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
This is a little refactoring commit that moves the mod `cdh.rs` and
`image.rs` to a directory module `confidential_data_hub`. This is
because the image pull ability will be moved into confidential data
hub, thus it is better to handle image pull things in the confidential
data hub submodule.
Also, this commit does some changes upon the original code. It gets rid
of a static variable for CDH timeout config and directly use the global
config variable's member. Also, this changes the
`is_cdh_client_initialized` function to sync version as it does not need
to be async.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
CDH provides the image pull api. This commit adds the declaration of the
API in the CDH proto file. This will be used in following commits.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
This method is not used when guest-pull is not used.
Add a flag that prevents a compile error when building with rust version > 1.84.0 and not using guest-pull
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
Fixes a confusing log message shown when Virtio-FS is disabled.
Previously we logged “The virtiofsd had stopped” regardless of whether Virtio-FS was actually enabled or not.
Signed-off-by: Paweł Bęza <pawel.beza99@gmail.com>
Add the memory prealloc support for qemu hypervisor.
When it was enabled, all of the memory will be allocated
and locked. This is useful when you want to reserve all the
memory upfront or in the cases where you want memory latencies
to be very predictable.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
This reverts commit 2ee3470627.
This is mostly redundant given we already have workflow approval for external
contributors.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Bump `ttrpc-codegen` related dependencies in response to `ttrpc-codegen`
bump in `libs/protocol`.
Relates: #11376
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Fix `empty_line_after_doc_comments` clippy warning as suggested by rust
1.85.1.
```console
error: empty line after doc comment
--> dbs_boot/src/x86_64/layout.rs:11:1
|
11 | / /// Magic addresses externally used to lay out x86_64 VMs.
12 | |
| |_^
13 | /// Global Descriptor Table Offset
14 | pub const BOOT_GDT_OFFSET: u64 = 0x500;
| ------------------------------ the comment documents this constant
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#empty_line_after_doc_comments
= note: `-D clippy::empty-line-after-doc-comments` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::empty_line_after_doc_comments)]`
= help: if the empty line is unintentional remove it
help: if the documentation should include the empty line include it in the comment
|
12 | ///
|
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Manually fix `unstable_name_collisions` clippy warning reported by rust
1.85.1.
```console
error: a method with this name may be added to the standard library in the future
--> src/registry.rs:646:10
|
646 | file.unlock()?;
| ^^^^^^
|
= warning: once this associated item is added to the standard library, the ambiguity may cause an error or change in behavior!
= note: for more information, see issue #48919 <https://github.com/rust-lang/rust/issues/48919>
= help: call with fully qualified syntax `fs2::FileExt::unlock(...)` to keep using the current method
= note: `-D unstable-name-collisions` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(unstable_name_collisions)]`
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Bump `ttrpc-codegen` related dependencies in response to `ttrpc-codegen`
bump in `libs/protocol`.
Relates: #11376
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Manually fix `empty_line_after_outer_attr` clippy warning reported by
rust 1.85.1.
```console
error: empty line after outer attribute
--> src/check.rs:515:9
|
515 | / #[allow(dead_code)]
516 | |
| |_^
517 | struct TestData<'a> {
| ------------------- the attribute applies to this struct
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#empty_line_after_outer_attr
= note: `-D clippy::empty-line-after-outer-attr` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::empty_line_after_outer_attr)]`
= help: if the empty line is unintentional remove it
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Bump `ttrpc-codegen` related dependencies in response to `ttrpc-codegen`
bump in `libs/protocol`.
Relates: #11376
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Bump `ttrpc-codegen` related dependencies in response to `ttrpc-codegen`
bump in `libs/protocol`.
Relates: #11376
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Manually fix `empty_line_after_doc_comments` clippy warning reported by
rust 1.85.1.
```console
error: empty line after doc comment
--> src/linux_abi.rs:8:1
|
8 | / /// Linux ABI related constants.
9 | |
| |_^
10 | #[cfg(target_arch = "aarch64")]
11 | use std::fs;
| ------- the comment documents this import
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#empty_line_after_doc_comments
= note: `-D clippy::empty-line-after-doc-comments` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::empty_line_after_doc_comments)]`
= help: if the empty line is unintentional remove it
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
The users@0.11.0 has a high severity CVE-2025-5791
and doesn't seem to be maintained, so switch to
uzers which forked it.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
In the nvidia rootfs build, only copy in `kata-opa` if `AGENT_POLICY` is enabled. This fixes
builds when `AGENT_POLICY` is disabled and opa is not built.
Signed-off-by: Champ-Goblem <cameron@northflank.com>
Manually fix `unnecessary_get_then_check` clippy warning as suggested by
rust 1.85.1.
```console
warning: unnecessary use of `get(&shared_mount.src_ctr).is_none()`
--> src/sandbox.rs:431:25
|
431 | if src_ctrs.get(&shared_mount.src_ctr).is_none() {
| ---------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| help: replace it with: `!src_ctrs.contains_key(&shared_mount.src_ctr)`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_get_then_check
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Kata runtime employs a CapabilityBits mechanism for VMM capability
governance. Fundamentally, this mechanism utilizes predefined feature
flags to manage the VMM's operational boundaries.
To meet demands for storage performance and security, it's necessary
to explicitly enable capability flags such as `BlockDeviceSupport`
(basic block device support) and `BlockDeviceHotplugSupport` (block
device hotplug) which ensures the VMM provides the expected caps.
In CoCo scenarios, due to the potential risks of sensitive data leaks
or side-channel attacks introduced by virtio-fs through shared file
systems, the `FsSharingSupport` flag must be forcibly disabled. This
disables the virtio-fs feature at the capability set level, blocking
insecure data channels.
Fixes#11341
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Two key important scenarios:
(1) Support `virtio-blk-pci` cold plug capability for confidential guests
instead of nvdimm device in CVM due to security constraints in CoCo cases.
(2) Push initdata payload into compressed raw block device and insert it
in CVM through `virtio-blk-pci` cold plug mechanism.
Fixes#11341
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
By default the checkout action leave the credentials
in the checked-out repo's `.git/config`, which means
they could get exposed. Use persist-credentials: false
to prevent this happening.
Note: static-checks.yaml does use git diff after the checkout,
but the git docs state that git diff is just local, so doesn't
need authentication.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
On newer TDX platforms, checking `/sys/firmware/tdx` for `major_version` and
`minor_version` is no longer necessary. Instead, we only need to verify that
`/sys/module/kvm_intel/parameters/tdx` is set to `'Y'`.
This commit addresses the following:
(1) Removes the outdated check and corrects related code, primarily impacting
`cloud-hypervisor`.
(2) Refines the TDX platform detection logic within `arch_guest_protection`.
Fixes#11177
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Currently, the TDX Quote Generation Service (QGS) connection in
QEMU with default vsock port 4050 for TD attestation. To make it
flexible for users to modify the QGS port. Based on the introduced
qgs_port, This commit supports the QGS port to be configured via
configuration
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Currently, the TDX Quote Generation Service (QGS) connection in QEMU is
hardcoded to vsock port 4050, which limits flexibility for TD attestation.
While the users will be able to modify the QGS port. To address this
inflexibility, this commit introduces a new qgs_port field within security
info and make it default with 4050.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
During the prepare for `start sandbox` phase, this commit
ensures the correct `ProtectionDeviceConfig` is prepared
based on the `GuestProtection` type in a TEE platform.
Specifically, for the TDX platform, this commit sets the
essential parameters within the ProtectionDeviceConfig,
including the TDX ID, firmware path, and the default QGS
port (4050).
This information is then passed to the underlying VMM for
further processing using the existing ResourceManager and
DeviceManager infrastructure.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This patch introduces TdxConfig with key fields, firmare,
qgs_port, mrconfigid, and other useful things. With this config,
a new ProtectionDeviceConfig type `Tdx(TdxConfig)` is added.
With this new type supported, we finally add tdx protection device
into the cmdline to launch a TDX-based CVM.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This commit introduces the `tdx-guest` designed to facilitate
the launch of CVMs leveraging Intel's TDX.
Launching a TDX-based CVM requires various properties, including
`quote-generation-socket`, and `mrconfigid`,`sept-ve-disable` .etc.
(1) The `quote-generation-socket` property is added to the
`tdx-guest` object, which is of type `SocketAddress`, specifies the
address of the Quote Generation Service (QGS).
(2) The `mrconfigid` property, representing the SHA384 hash
for non-owner-defined configurations of the guest TD, is introduced as a
runtime or OS configuration parameter.
(3) And the `sept-ve-disable` property allows control over whether
EPT violation conversions to #VE exceptions are disabled when the guest
TD accesses PENDING pages.
With the introduction of the `tdx-guest` object and its associated
properties, launching TDX-based CVMs is now supported. For example, a
TDX guest can be configured via the command line as follows:
```shell
-object {"qom-type":"tdx-guest", "id":"tdx", "sept-ve-disable":true,\
"mrconfigid":"vHswGkzG4B3Kikg96sLQ5vPCYx4AtuB4Ubfzz9UOXvZtCGat8b8ok7Ubz4AxDDHh",\
"quote-generation-socket":{"type":"vsock","cid":"2","port":"4050"} \
-machine q35,accel=kvm,confidential-guest-support=tdx
```
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This enables consistent JSON representation of socket addresses
across system components:
(1) Add serde serialization/deserialization with standardized
field naming convention.
(2) Enforce string-based port/cid and unix/path representation
for protocol compatibility.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
For CoCo, shared_fs is prohibited as we cannot guarantee the security of
guest/host sharing. Therefore, this PR enables administrators to configure
shared_fs = "none" via the configuration.toml file, thereby enforcing the
disablement of sharing.
Fixes#10677
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Run the k8s tests on mariner with annotation disable_image_nvdimm=true,
to use virtio-blk instead of nvdimm for the guest rootfs block device.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Allow users to build using DEFDISABLEIMAGENVDIMM=true if they want to
set disable_image_nvdimm=true in configuration-clh.toml.
disable_image_nvdimm=false is the default config value.
Also, use virtio-blk instead of nvdimm if disable_image_nvdimm=true in
configuration-clh.toml.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Allow users to build using DEFDISABLEIMAGENVDIMM=true if they
want to set disable_image_nvdimm=true in configuration-qemu*.toml.
disable_image_nvdimm=false is the default configuration value.
Note that the value of disable_image_nvdimm gets ignored for
platforms using "confidential_guest = true".
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Comment out "disable_image_nvdimm = true" in:
- configuration-qemu-snp.toml
- configuration-qemu-nvidia-gpu-snp.toml
for consistency with the other configuration-qemu*.toml files.
Those two platforms are using "confidential_guest = true", and therefore
the value of disable_image_nvdimm gets ignored.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Bump chrono package to 0.4.41 and thereby
remove the time 0.1.43 dependency and remediate
CVE-2020-26235
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This removes the ok-to-test label on every push, except if the PR author
has write access to the repo (ie. permission to modify labels).
This protects against attackers who would initially open a genuine PR,
then push malicious code after the initial review.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This completely eliminates the Azure secret from the repo, following the below
guidance:
https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/configuring-openid-connect-in-azure
The federated identity is scoped to the `ci` environment, meaning:
* I had to specify this environment in some YAMLs. I don't believe there's any
downside to this.
* As previously, the CI works seamlessly both from PRs and in the manual
workflow.
I also deleted the tools/packaging/kata-deploy/action folder as it doesn't seem
to be used anymore, and it contains a reference to the secret.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Having secrets unconditionally being inherited is
bad practice, so update the workflows to only pass
through the minimal secrets that are needed
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
In this commit, hotplug_vfio_on_root_bus parameter is removed.
<dd422ccb69>
pcie_root_port parameter description
(`This value is valid when hotplug_vfio_on_root_bus is true and
machine_type is "q35"`) will have no value,
and not completely valid, since vrit or DB as also support for root-ports and CLH as well.
so removed.
Fixes: #11316
Co-authored-by: Zvonko Kaiser <zkaiser@nvidia.com>
Signed-off-by: Shunsuke Kimura <pbrehpuum@gmail.com>
Instead of looping over the users per group and parsing passwd for each
user, we can do the reverse lookup uid->user up front and then compare
the names directly. This has the nice side-effect of silencing warnings
about non-existent users mentioned in /etc/group, which is not relevant
for policy decisions.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
containerd does not automatically add groups to the list of additional
GIDs when the groups have the same name as the user:
https://github.com/containerd/containerd/blob/f482992/pkg/oci/spec_opts.go#L852-L854
This is a bug and should be corrected, but it has been present since at
least 1.6.0 and thus affects almost all containerd deployments in
existence. Thus, we adopt the same behavior and ignore groups with the
same name as the user when calculating additional GIDs.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
When connecting to guest through vsock, a log is printed for each failure.
The failure comes from two main reasons: (1) the guest is not ready or (2)
some real errors happen. Printing logs for the first case leads to log
clutter, and your logs will like this:
```
Feb 07 02:47:24 ubuntu containerd[520]: {"msg":"connect uds \"/run/kata/...
Feb 07 02:47:24 ubuntu containerd[520]: {"msg":"connect uds \"/run/kata/...
Feb 07 02:47:24 ubuntu containerd[520]: {"msg":"connect uds \"/run/kata/...
Feb 07 02:47:24 ubuntu containerd[520]: {"msg":"connect uds \"/run/kata/...
Feb 07 02:47:24 ubuntu containerd[520]: {"msg":"connect uds \"/run/kata/...
```
To avoid this, the sock implmentations save the last error and return it
after all retries are exhausted. Users are able to check all errors by
setting the log level to trace.
Reorganize the log format to "{sock type}: {message}" to make it clearer.
Apart from that, errors return by the socks use `self`, instead of
`ConnectConfig`, since the `ConnectConfig` doesn't provide any useful
information.
Disable infinite loop for the log forwarder. There is retry logic in the
sock implmentations. We can consider the agent-log unavailable if
`sock.connect()` encounters an error.
Fixes: #10847
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
The vhost-user-fs has been added to Dragonball, so we can remove
`update_memory`'s dead_code attribute.
Fixes: #8691
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Removed unnecessary dynamic dispatch for services. Properly dereferenced
service Box values and stored in Arc.
Co-authored-by: Ruoqing He <heruoqing@iscas.ac.cn>
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
Previous version of `ttrpc-codegen` is generating outdated
`#![allow(box_pointers)]` which was deprecated. Bump `ttrpc-codegen`
from v0.4.2 to v0.5.0 and `protobuf` from vx to v3.7.1 to get rid of
this.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
The additional GIDs are handled by genpolicy as a BTreeSet. This set is
then serialized to an ordered JSON array. On the containerd side, the
GIDs are added to a list in the order they are discovered in /etc/group,
and the main GID of the user is prepended to that list. This means that
we don't have any guarantees that the input GIDs will be sorted. Since
the order does not matter here, comparing the list of GIDs as sets is
close enough.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
The warning used to trigger even if the passwd file was not needed. This
commit moves it down to where it actually matters.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
We need more and accurate documentation. Let's start
by providing an Helm Chart install doc and as a second
step remove the kustomize steps.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Co-authored-by: Steve Horsman <steven@uk.ibm.com>
The Guest rootfs image file size is aligned up to 128M boundary,
since commmit 2b0d5b2. This change allows users to use a custom
alignment value - e.g., to align up to 2M, users will be able to
specify IMAGE_SIZE_ALIGNMENT_MB=2 for image_builder.sh.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
- Create groups for commonly seen cargo packages so that rather than
getting up to 9 PRs for each rust components, bumps to the same package
are grouped together.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Create a dependabot configuration to check for updates
to our rust and golang packages each day and our github
actions each month
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
After the last commit, the initdata test on SNP should be ok. Thus we
turn on this flag for CI.
Fixes#11300
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
the qemu commandline of SNP should start with `sev-snp-guest`, and then
following other parameters separeted by ','. This patch fixes the
parameter order.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
We're switching to using a rev as it may take some time for the package
to be updated on crates.io.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
There's no benefit on keeping those restricted to the dragonball build,
when they can be used with other VMMs as well (as long as they support
the mem-agent).
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
Currently, when a new sandbox resource controller is created with cgroupsv2 and sandbox_cgroup_only is disabled,
the cgroup management falls back to cgroupfs. During deletion, `IsSystemdCgroup` checks if the path contains `:`
and tries to delete the cgroup via systemd. However, the cgroup was originally set up via cgroupfs and this process
fails with `lstat /sys/fs/cgroup/kubepods.slice/kubepods-besteffort.slice/....scope: no such file or directory`.
This patch updates the deletion logic to take in to account the sandbox_cgroup_only=false option and in this case uses
the cgroupfs delete.
Fixes: #11036
Signed-off-by: Champ-Goblem <cameron@northflank.com>
Increase the NOFILE limit in the systemd service, this helps with
running databases in the Kata runtime.
Signed-off-by: Champ-Goblem <cameron@northflank.com>
Since 3.12 we're shipping the helm-chart per default
with each release. Update the documentation to use helm rather
then the kata-deploy manifests.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
We have a number of jobs that either need,or nest workflows
that need gh permissions, such as for pushing to ghcr,
or doing attest build provenance. This means they need write
permissions on things like `packages`, `id-token` and `attestations`,
so we need to set these permissions at the job-level
(along with `contents: read`), so they are not restricted by our
safe defaults.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
I shortsightedly forgot that gatekeeper would need
to read more than just the commit content in it's
python scripts, so add read permissions to actions
issues which it uses in it's processing
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
We have a number of jobs that nest the build-static-tarball
workflows later on. Due to these doing attest build provenance,
and pushing to ghcr.io, t hey need write permissions on
`packages`, `id-token` and `attestations`, so we need to set
these permissions on the top-level jobs (along with `contents: read`),
so they are not blocked.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Some legacy workflows require write access to github which
is a security weakness and don't provide much value,
so lets remove them.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
It frequently causes "Resource Temporarily Unavailable (OS Error 11)"
with the original 250ms read timeout When passing through devices via
VFIO in QEMU. The root cause lies in synchronization timeout windows
failing to accommodate inherent delays during critical hardware init
phases in kernel space. This commit would increase the timeout to 5000ms
which was determined through some tests. While not guaranteeing complete
resolution for all hardware combinations, this change significantly
reduces timeout failures.
Fixes # 10361
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This enables guest pull via config, without the need of any external
snapshotter. When the config enables runtime.experimental_force_guest_pull, instead of
relying on annotations to select the way to share the root FS, we always
use guest pull.
Co-authored-by: Markus Rudy <mr@edgeless.systems>
Signed-off-by: Paul Meyer <katexochen0@gmail.com>
The service name is specified as RFC 1035 lable name [1]. The svc_name
regex in the genpolicy settings is applied to the downward API env
variables created based on the service name. So it tries to match
RFC 1035 labels after they are transformed to downward API variable
names [2]. So the set of lower case alphanumerics and dashes is
transformed to upper case alphanumerics and underscores.
The previous regex wronly permitted use of numbers, but did allow
dot and dash, which shouldn't be allowed (dot not because they aren't
conform with RFC 1035, dash not because it is transformed to underscore).
We have to take care not to also try to use the regex in places where
we actually want to check for RFC 1035 label instead of the downward
API transformed version of it.
Further, we should consider using a format like JSON5/JSONC for the
policy settings, as these are far from trivial and would highly benefit
from proper documentation through comments.
[1]: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service
[2]: b2dfba4151/pkg/kubelet/envvars/envvars.go (L29-L70)
Signed-off-by: Paul Meyer <katexochen0@gmail.com>
Since kernel v6.3 the vsock packet is not split over two descriptors and
is instead included in a single one.
Therefore, we currently decide the specific method of obtaining
BufWrapper based on the length of descriptor.
Refer:
a2752fe04fhttps://git.kernel.org/torvalds/c/71dc9ec9ac7d
Signed-off-by: Xingru Li <lixingru.lxr@linux.alibaba.com>
[ Gao Xiang: port this patch from the internal branch to address Linux 6.1.63+. ]
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Currently, Kata EROFS support needs it, otherwise it will:
[ 0.564610] erofs: (device sda): mounted with root inode @ nid 36.
[ 0.564858] overlayfs: failed to set xattr on upper
[ 0.564859] overlayfs: ...falling back to index=off,metacopy=off.
[ 0.564860] overlayfs: ...falling back to xino=off.
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Some nvidia gpu pci address domain with 0001,
current runtime default deal with 0000:bdf,
which cause address errors during device initialization
and address conflicts during device registration.
Fixes#11252
Signed-off-by: yangsong <yunya.ys@antgroup.com>
Fixed "note: Not following: ./../../../tools/packaging/guest-image/lib_se.sh:
openBinaryFile: does not exist (No such file or directory) [SC1091]"
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Although the script will inherit that setting from the caller scripts,
expliciting it in the file will vanish shellcheck "warning: Use 'pushd
... || exit' or 'pushd ... || return' in case pushd fails. [SC2164]"
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Addressed the following shellcheck advices:
SC2046 (warning): Quote this to prevent word splitting.
SC2248 (style): Prefer double quoting even when variables don't contain special characters
SC2250 (style): Prefer putting braces around variable references even when not strictly required.
SC2292 (style): Prefer [[ ]] over [ ] for tests in Bash/Ksh
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
It's import to handle port allocation in a PCIe topology before vfio
deivce hotplug via QMP.
The code ensures that VFIO devices are properly allocated to available
ports (either root ports or switch ports) and updates the device's bus
and port information accordingly.
It'll first retrieves the PCIe port type from the topology using
pcie_topo.get_pcie_port(). And then, searches for an available node in
the PCIe topology with RootPort or SwitchPort type and allocates the
VFIO device to the found available port. Finally, Updates the device's
bus with the allocated port's ID and type.
Fixes # 10361
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This commit implements the `find_available_node` function,
which searches the PCIe topology for the first available
`TopologyPortDevice` or `SwitchDownPort`.
If no available node is found in either the `pcie_port_devices`
or the connected switches' downstream ports, the function returns
`None`.
Fixes # 10361
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This commit note that the current implementation restriction where
'multifunction=on' is temporarily unsupported. While the feature
isn't available in the present version, we explicitly acknowledge
this limitation and commit to addressing it in future iterations
to enhance functional completeness.
Tracking issue #11292 has been created to monitor progress towards
full multifunction support.
Fixes#10361
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
To support port devices for vfio devices, more fields need to be
introduced to help pass port type, bus and other information.
Fixes#10361
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
When try to delete a cgroup, it's needed to move all of the
tasks/procs in the cgroup into root cgroup and then delete it.
Since for cgroup v2, it doesn't support to move thread into
root cgroup, thus move the processes instead of moving tasks
can fix this issue.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
this script relies on temporary subscriptions and won't cleanup any
resources. Let's improve the logging to better describe what resources
were created and how to clean them, if the user needs to do so.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
We used hardcoded "ci/openshift-ci/cluster" location which expects this
script to be only executed from the root. Let's use SCRIPT_DIR instead
to allow execution from elsewhere eg. by user bisecting a failed CI run.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
In CI we hit problem where just after `az login` the first `az
network vnet list` command fails due to permission. We see
"insufficient permissions" or "pending permissions", suggesting we should
retry later. Manual tests and successful runs indicate we do have the
permissions, but not immediately after login.
Azure docs suggest using extra `az account set` but still the
propagation might take some time. Add a loop retrying
the first command a few times before declaring failure.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
kbs_k8s_svc_host() returns the ingress IP when the KBS service is
exposed via an ingress. In Azure AKS the ingress can time a while to be
fully ready and recently we have noticed on CI that kbs_k8s_svc_host()
has returned empty value. Maybe the problem is on current timeout being
too low, so let's increase it to 50 seconds to see if the situation
improves.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Added 'report-tests' command to gha-run.sh to print to stdout a report
of the tests executed.
For example:
```
SUMMARY (2025-02-17-14:43:53):
Pass: 0
Fail: 1
STATUSES:
not_ok foo.bats
OUTPUTS:
::group::foo.bats
1..3
not ok 1 test 1
not ok 2 test 2
ok 3 test 3
1..2
not ok 1 test 1
not ok 2 test 2
::endgroup::
```
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Currently run_kubernetes_tests.sh sends all the bats outputs to stdout
which can be very difficult to browse to find a problem, mainly on
CI. With this change, each bats execution have its output sent to
'reports/yyy-mm-dd-hh:mm:ss/<status>-<bats file>.log' where <status>
is either 'ok' (tests passed) or 'not_ok' (some tests failed).
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Added variable REPO_COMPONENTS (default: "main") which sets components
used by mmdebstrap for rootfs building.
This is useful for custom image builders who want to include EXTRA_PKGS
from components other than the default "main" (e.g. "universe").
Fixes: #11278
Signed-off-by: Jacek Tomasiak <jtomasiak@arista.com>
Signed-off-by: Jacek Tomasiak <jacek.tomasiak@gmail.com>
Use $(sandbox-namespace) wildcard in case none is specified in yaml. If wildcard is present, compare
input against annotation value.
Fixes regression introduced in https://github.com/microsoft/kata-containers/pull/273
where samples that use metadata.namespace env var were no longer working.
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
Validate more process fields for commands enabled using the
ExecProcessRequest "commands" and/or "regex" fields from the
settings file.
Add function to get the container from state based on container_id
matching instead of matching it against every policy container data
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Signed-off-by: Ankita Pareek <ankitapareek@microsoft.com>
Using process data inputs for allow_process() is easier to
read/understand compared with the older OCI data inputs.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
For security reasons, we have restricted directory copying.
Introduces the `is_allowlisted_copy_volume` function to verify
if a given volume path is present in an allowed copy directory.
This enhances security by ensuring only permitted volumes are
copied
Currently, only directories under the path
`/var/lib/kubelet/pods/<uid>/volumes/{kubernetes.io~configmap,
kubernetes.io~secret, kubernetes.io~downward-api,
kubernetes.io~projected}` are allowed to be copied into the
guest. Copying of other directories will be prohibited.
Fixes#11237
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
When synchronizing file changes on the host, a "symlink AlreadyExists"
issue occurs, primarily due to improper handling of symbolic links
(symlinks). Additionally, there are other related problems.
This patch will try to address these problems.
(1) Handle symlink target existence (files, dirs, symlinks) during host file
sync. Use appropriate removal methods (unlink, remove_file, remove_dir_all).
(2) Enhance temporary file handling for safer operations and implement truncate
only at offset 0 for resume support.
(3) Set permissions and ownership for parent directories.
(4) Check and clean target path for regular files before rename.
Fixes#11237
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Introduce event-driven file sync mechanism between host and guest when
sharedfs is disabled, which will help monitor the host path in time and
do sync files changes:
1. Introduce FsWatcher to monitor directory changes via inotify;
2. Support recursive watching with configurable filters;
3. Add debounce logic (default 500ms cooldown) to handle burst events;
4. Trigger `copy_dir_recursively` on stable state;
5. Handle CREATE/MODIFY/DELETE/MOVED/CLOSE_WRITE events;
Fixes#11237
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
In Kubernetes (k8s), while Kata Pods often use virtiofs for injecting
Service Accounts, Secrets, and ConfigMaps, security-sensitive
environments like CoCo disable host-guest sharing. Consequently, when
SharedFs is disabled, we propagate these configurations into the guest
via file copy and bind mount for correct container access.
Fixes#11237
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
There's several cases that block device plays very import roles:
1. Direct Volume:
In Kata cases, to achieve high-performance I/O, raw files on the host
are typically passed directly to the Guest via virtio-blk, and then
bond/mounted within the Guest for container usage.
2. Trusted Storage
In CoCo scenarios, particularly in Guest image pull mode, images are
typically pulled directly from the registry within the Guest. However,
due to constrained memory resources (prioritized for containers), CoCo
leverages externally attached encrypted storage to store images,
requiring hot-plug capability for block devices.
and as other vmms, like dragonball and cloud-hypervisor in runtime-rs or
qemu in kata-runtime have already supported such capabilities, we need
support block device with hot-plug method (QMP) in qemu-rs. Let's do it.
Fixes#11143
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This commit introduces block device hotplugging capability using
QMP commands.
The implementation enables attaching raw block devices to a running
VM through the following steps:
1.Block Device Configuration
Uses `blockdev-add` QMP command to define a raw block backend with
(1) Direct I/O mode
(2) Configurable read-only flag
(3) Host file/block device path (`/path/to/block`)
2.PCI Device Attachment, Attaches the block device via `device_add`
QMP command as a `virtio-blk-pci` device:
(1) Dynamically allocates PCI slots using `find_free_slot()`
(2) Binds to user-specified PCIe bus (e.g., `pcie.1`)
(3) Returns PCI path for further management
Fixes#11143
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
The get_pci_path_by_qdev_id function is designed to search for a PCI
device within a given list of devices based on a specified qdev_id.
It tracks the device's path in the PCI topology by recording the slot
values of the devices traversed during the search. If the device is
located behind a PCI bridge, the function recursively explores the
bridge's device list to find the target device. The function returns
the matching device along with its updated path if found, otherwise,
it returns None.
Fixes#11143
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
`containerd` command should be executed in the host environment.
(To generate the config that matches the host's containerd version.)
Fixes: #11092
Signed-off-by: Shunsuke Kimura <pbrehpuum@gmail.com>
Fixes: #11288
This commit appends hotplug devices (e.g., persistent volume)
to deviceInfos when `vfio_mod` is `vfio` and `cold_plug_vfio`
is set to one except `no-port`. For details, please visit the issue.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This exposes REPO_URL and adds REPO_URL_X86_64 which can be set to use
custom Ubuntu repo for building rootfs.
If only one architecture is built, REPO_URL can be set. Otherwise,
REPO_URL_X86_64 is used for x86_64 arch and REPO_URL for others.
Fixes: #11276
Signed-off-by: Jacek Tomasiak <jtomasiak@arista.com>
Signed-off-by: Jacek Tomasiak <jacek.tomasiak@gmail.com>
Now that memory hotplug should work, as we're using a firmware that
supports that, let's re-enable the tests that rely on hotplug.
Fixes: #10926, #10927
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
As the genpolicy from_files call makes network requests to container
registries, it has a chance to fail.
Harden us against flakes due to network by introducing a 6x retry loop
in genpolicy tests.
Signed-off-by: Cameron Baird <cameronbaird@microsoft.com>
Drop '-vmx-rdseed-exit' from '-cpu host' QEMU options. The history
of it is unknown but it's likely related to early TDX enablement.
TD pods start up fine without it (tested by manually editing the
configuration file) and it's also not used elsewhere.
Keep TDXCPUFEATURES for now in case a need for it shows up later.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Extract PortDevice relevant information, and then invoke different
processing methods based on the device type.
Fixes#10361
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Some data structures and methods are introduced to help handle vfio devices.
And mothods add_pcie_root_ports and add_pcie_switch_ports follow runtime's
related implementations of vfio devices.
Fixes#10361
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Prepare pcie port devices before starting VM with the help of
device manager and PCIe Topology.
Fixes#10361
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
A new resource type `PortDevice` is introduced which is dedicated
for handling root ports/switch ports during sandbox creation(VM).
Fixes#10361
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
PortDevice is for handling root ports or switch ports in PCIe
Topology. It will make it easy pass the root ports/switch ports
information during create VM with requirements of PCIe devices.
Fixes#10361
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This commit introduces an implementation for managing PCIe topologies,
focusing on the relationship between Root Ports and Switch Ports. The
design supports two strategies for generating Switch Ports:
Let's take the requirement of 4 switch ports as an example. There'll be
three possible solutions as below:
(1) Single Root Port + Single PCIe Switch: Uses 1 Root Port and 1 Switch
with 4 Downstream Ports.
(2) Multiple Root Ports + Multiple PCIe Switches: Uses 2 Root Ports and
2 Switches, each with 2 Downstream Ports.
The recommended strategy is Option 1 due to its simplicity, efficiency,
and scalability. The implementation includes data structures
(PcieTopology, RootPort, PcieSwitch, SwitchPort) and operations
(add_pcie_root_port, add_switch_to_root_port, add_switch_port_to_switch)
to manage the topology effectively.
Fxies #10361
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
(1) Introduce new field `pcie_switch_port` for switch ports.
(2) Add related checking logics in vmms(dragonball, qemu)
Fixes#10361
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
The edk2 is required for memory hot plug on qemu for arm64.
This adds the edk2 to configuration-qemu.toml for arm64.
Signed-off-by: Seunguk Shin <seunguk.shin@arm.com>
Reviewed-by: Nick Connolly <nick.connolly@arm.com>
The edk2 is required for memory hot plug on qemu for arm64.
This adds the edk2 to static tarball for arm64.
Signed-off-by: Seunguk Shin <seunguk.shin@arm.com>
Reviewed-by: Nick Connolly <nick.connolly@arm.com>
The github rest api truncated job names that are >100
characters (which doesn't seem to be documented).
There doesn't seem to be a way to easily make gatekeeper
handle this automatically, so lets update the required-tests
to expect the truncated job names
Fixes: #11176
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
to simplify gatekeeper development add support for DEBUG_INPUT which can
be used to report content from files gathered in DEBUG run.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
to avoid manual curling to analyze GK issues let's add a way to dump all
GK requests in a directory when the use specifies "DEBUG" env variable.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Let's take advantage that helm take and OCI registry as the charts, and
upload our charts to the OCI registries we've been using so far.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
The previous attempt to fix this issue only took in consideration the
QEMU binary, as I completely forgot that there were other pieces of the
config that we also adjusted.
Now, let's just check one of the configs before trying to adjust
anything else, and only do the changes if the suffix added with the
multi-install suffix is not yet added.{
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
Multistrap requires usrmerge package which was dropped in Ubuntu 24.04
(Noble). Based on details from [0], the rootfs build process was switched
to mmdebstrap.
Some additional minor tweaks were needed around chrony as the version
from Noble has very strict systemd sandboxing configured and it doesn't
work with readonly root by default.
[0] https://lists.debian.org/debian-dpkg/2023/05/msg00080.htmlFixes: #11245
Signed-off-by: Jacek Tomasiak <jtomasiak@arista.com>
Signed-off-by: Jacek Tomasiak <jacek.tomasiak@gmail.com>
Guest components is now less verbose with its error messages. This will
be fixed after the release but for now switch to a more generic error
message that is still found in the logs.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Guest components is less verbose with its error message now. This will
be fixed after the release, but for now, update the tests with the new
more general message.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Guets components is less verbose with its error messages. This will be
fixed after the release, but for now let's replace this with a more
generic message.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Fix up genpolicy test inputs to include required additionalGids
Include a test for the pod_container container in security_context tests
as these containers follow slightly different paths in containerd.
Introduce a test for fsGroup/supplementalGroups fields in the security
context.
Signed-off-by: Cameron Baird <cameronbaird@microsoft.com>
Introduce new test case to the security context bats file which verifies
that policy works properly for a deployment yaml containing fsGroup and
supplementalGroup configuration.
Signed-off-by: Cameron Baird <cameronbaird@microsoft.com>
With added support for parsing these fields in genpolicy, we can now
enable policy verification of AdditionalGids.
Signed-off-by: Cameron Baird <cameronbaird@microsoft.com>
Some changes in guest components have obscured the error message that we
show when we fail to get the credentials for an authenticated image. The
new error message is a little bit misleading since it references
decrypting an image. This will be udpated in a future release, but for
now look for this message.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Guest components prints out a different error when failing to decrypt an
image. Update the test to look for this new error.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Adding:
"-object rng-random,id=rng0,filename=/dev/urandom -device
virtio-rng-pci,rng=rng0"
for confidential guests is not necessary as the RNG source cannot
be trusted and the guest kernel has the driver already disable as well.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Linux CoCo x86 guest is hardened to ensure RDRAND provides enough
entropy to initialize Linux RNG. A failure will panic the guest.
For confidential guests any other RNG source is untrusted so disable
them.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
With #11076 merged, a VFIO configuration is needed in the runtime
when IBM SEL is involved (e.g., qemu-se or qemu-se-runtime-rs).
For the Go runtime, we already have a nightly test
(e.g., https://github.com/kata-containers/kata-containers/actions/runs/14964175872/job/42031097043)
in which this change has been applied.
For the Rust runtime, the feature has not yet been migrated.
Thus, this change serves as a placeholder and a reminder for future implementation.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Pick up changes to guest components. This hash is right before the
changes to GC to support image pull via the CDH.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
As the comment in the fragment suggests, this is for the firecracker builds
and not relevant for confidential guests, for example.
Exlude mmio.conf fragment by adding the new !confidential tag to drop
virtio MMIO transport for the confidential guest kernel (as virtio PCI is
enough for the use cases today).
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
build-kernel.sh supports exluding fragments from the common base
set based on the kernel target architecture.
However, there are also cases where the base set must be stripped
down for other reason. For example, confidential guest builds want to
exclude some drivers the untrusted host may try to add devices (e.g.,
virtio-rng).
Make build-kernel.sh to skip fragments tagged using '!confidential'
when confidential guest kernels are built.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
It looks like the 22.04 image got updated and broke
the docker tests (see #11247), so make these un-required
until we can get a resolution
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Update the tempfile crate to resolve security issue
[WS-2023-0045](7247a8b6ee)
that came with the remove_dir_all dependency in prior versions
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This aligns the workdir preparation more closely with the workdir
preparation for the generate integration test. Most notably, we
clean up the temporary directory before we execute the tests in it.
This way we better isolate different runs.
Signed-off-by: Leonard Cohnen <leonard.cohnen@gmail.com>
Signed-off-by: Markus Rudy <mr@edgeless.systems>
Add a new type of integration test to genpolicy. Now we can test flag handling
and how the CLI behaves with certain yaml inputs.
The first tests cover the case when a Pod references a Kubernetes secret of
config map in another file. Those need to be explicitly added via the
--config-files flag.
In the future we can easily add test suites that cover that all yaml fields
of all resources are understood by genpolicy.
Signed-off-by: Leonard Cohnen <leonard.cohnen@gmail.com>
In preparation for adding more types of integration tests, moving the
policy enforcements test into a separate folder.
Signed-off-by: Leonard Cohnen <leonard.cohnen@gmail.com>
Signed-off-by: Markus Rudy <mr@edgeless.systems>
This allows passing config maps and secrets (as well as any other
resource kinds relevant in the future) using the -c flag.
Fixes: #10033
Co-authored-by: Leonard Cohnen <leonard.cohnen@gmail.com>
Signed-off-by: Leonard Cohnen <leonard.cohnen@gmail.com>
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Switch imports to resolve:
```
SA1019: "github.com/opencontainers/runc/libcontainer/userns" is deprecated:
use github.com/moby/sys/userns
```
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
In the latest oci-spec, the prestart hook is deprecated.
However, the docker & nerdctl tests failed when I switched
to one of the newer hooks which don't run at quite the same time,
so ignore the deprecation warnings for now to unblock the security fix
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
We've been using the
github.com/containers/podman/v4/pkg/annotations module
to get cri-o annotations, which has some major CVEs in, but
in v5 most of the annotations were moved into crio (from 1.30)
(see https://github.com/cri-o/cri-o/pull/7867). Let's switch
to use the cri-o annotations module instead and remediate
CVE-2024-3056.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
When installing with kata-deploy,
usually `/opt/kata/bin` is not in the PATH.
Therefore, it will fail to execute.
so add it to the PATH.
Fixes: #11122
Signed-off-by: Shunsuke Kimura <pbrehpuum@gmail.com>
Co-authored-by: Jakob Naucke <jakob.naucke@ibm.com>
`musl` target is not yet available for riscv64 as of 1.80.0 rust
toolchain, set `FORTIFY_SOURCE` to 1 on riscv64 platforms.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`create_pci_root_bus_path` needs to be enabled on riscv64 for agent to
compile and work on those platforms.
Signed-off-by: Nikos Ch. Papadopoulos <ncpapad@cslab.ece.ntua.gr>
Since the ephemeral volume already has a separate volume type for
processing, the processing in the virtiofs share volume can be deleted.
Moreover, it is not appropriate to process the ephemeral in the share
fs.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
For k8s, there's two type of volumes based on ephemral memory,
one is emptydir volume based on ephemeral memory, and the other
one is used for shm device such as /dev/shm. Thus add a new volume
type ephemeral volume to support those two type volumes and remove
the legacy shm volume.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Implementing directory creation logic in the OverlayfsHandler to process
driver options with the KATA_VOLUME_OVERLAYFS_CREATE_DIR prefix
Signed-off-by: ChengyuZhu6 <hudson@cyzhu.com>
- Detection of EROFS options in container rootfs
- Creation of necessary EROFS devices
- Sharing of rootfs with EROFS via overlayfs
Fixes: #11163
Signed-off-by: ChengyuZhu6 <hudson@cyzhu.com>
This patch:
- adds a count check on mounts
- adds various test scenarios for mounts with emptyDir volume source
Signed-off-by: Archana Choudhary <archana1@microsoft.com>
Some cni plugins will set the MTU of some routes, such as cilium will
modify the MTU of the default route. If the mtu of the route is not set
correctly, it may cause excessive fragmentation or even packet loss of
network packets. Therefore, this PR adds the setting of the MTU of the
route. First, when obtaining the route, if the MTU is set, the MTU will
also be obtained and set to the route in the guest.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Bump `netlink-sys` to v0.8, `netlink-packet-route` to v0.22 and
`rtnetlink` to v0.16 to reach a consistent state of `rust-netlink`
dependencies.
`bitflags` is bumped to v2.9.0 since those crates requires it.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`rtnetlink`, `netlink-sys` and `netlink-packet-route` are from the same
organization, and some of them are depending on the others, which
implies the version of those crates should be chosen and dealt with
carefully, group them to provide better management.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Kernel Makefiles changed how to deduce the right arch
lets set it explicilty to enable arm and amd builds.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Put local dependencies (mostly `dbs` crates) into workspace to avoid
complex path dependencies all over the workspace. Simplify path
dependency referencing.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
It's better to open the log pipe file with read & write option,
otherwise, once the containerd reboot and closed the read
endpoint, kata shim would write the log pipe with broken pipe error.
Fixes: #11207
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Update the runtime-rs workspace packages to
use workspace package versions where applicable
to centralise the config and reduce maintenance
when updating these
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
As discussed in #9538, with anyhow >=1.0.77 we have test failures due to backtrace behaviour
changing, so set RUST_LIB_BACKTRACE=0,
so that we only have backtrace on panics
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Update all crossbeam-channel for all non-agent
packages (it was done separately in #11175)
to 0.5.15 to get them on latest version and remove
the versions with a vulnerability
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
When a PR has no new files the cargo deny runner fails with:
```
[cargo-deny-generator.sh:17] ERROR: changed_files_status=
```
so add `|| true` to try and help this
Co-authored-by: Ruoqing He <heruoqing@iscas.ac.cn>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
We are seeing failures in this test, where the output of
the kubectl exec command seems to be blank, so try
retrying the exec like #11024Fixes: #11133
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Since #10780 the dbs crates are managed as members
of the dragonball workspace, so we can remove the lockfile
as it's now workspace managed now
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
As #11076 introduces VFIO-AP bind/associate funtions for IBM Secure
Execution (SEL), a new internal nightly test has been established.
This PR adds a new entry `cc-vfio-ap-e2e-tests` to the existing matrix
to share the test result.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
For help with debugging add, logging of the KBS,
like the container system logs if the confidential test fails
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Bump golang.org/x/net to 0.38.0 as dependabot
isn't doing it for these packages to remediate
CVE-2025-22872
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Nydus+guest_pull has specific behavior where it improperly handles image layers on
the host, causing the CRI to not find /etc/passwd and /etc/group files
on container images which have them. The unfortunately causes different
outcomes w.r.t. GID used which we are trying to enforce with policy.
This behavior is observed/explained in https://github.com/kata-containers/kata-containers/issues/11162
Handle this exception with a config.settings.cluster_config.guest_pull
field. When this is true, simply ignore the /etc/* files in the
container image as they will not be parsed by the CRI.
Signed-off-by: Cameron Baird <cameronbaird@microsoft.com>
Add security context testcases for genpolicy, verifying that UID and GID
configurations controlled by the kubernetes security context are
enforced.
Also, fix the other CreateContainerRequest tests' expected contents to
reflect our new genpolicy parsing/enforcement of GIDs.
Signed-off-by: Cameron Baird <cameronbaird@microsoft.com>
Introduce tests to check for policy correctness on a redis deployment
with 1. a pod-level securityContext 2. a container-level securityContext
which shadows the pod-level securityContext 3. a pod-level
securityContext which selects an existing user (nobody), causing a new GID to be selected.
Redis is an interesting container image to test with because it includes
a /etc/passwd file with existing user/group configuration of 1000:1000 baked in.
Signed-off-by: Cameron Baird <cameronbaird@microsoft.com>
With fixes to align policy GID parsing with the CRI behavior, we can now
enable policy verification of GIDs.
Signed-off-by: Cameron Baird <cameronbaird@microsoft.com>
The GID used for the running process in an OCI container is a function of
1. The securityContext.runAsGroup specified in a pod yaml, 2. The UID:GID mapping in
/etc/passwd, if present in the container image layers, 3. Zero, even if
the userstr specifies a GID.
Make our policy engine align with this behavior by:
1. At the registry level, always obtain the GID from the /etc/passwd
file if present. Ignore GIDs specified in the userstr encoded in the
OCI container.
2. After an update to UID due to securityContexts, perform one final check against
the /etc/passwd file if present. The GID used for the running
process is the mapping in this file from UID->GID.
3. Override everything above with the GID of the securityContext
configuration if provided
Signed-off-by: Cameron Baird <cameronbaird@microsoft.com>
Our policy should cover these fields for securityContexts at the pod or
container level of granularity.
Signed-off-by: Cameron Baird <cameronbaird@microsoft.com>
The get_process logic in registry.rs did not account for all cases
(username:groupname), did not defer to contents of /etc/group,
/etc/passwd when it should, and was difficult to read.
Clean this implementation up, factoring the string parsing for
user/group strings into their own functions. Enable the
registry::Container class to query /etc/passwd and /etc/group, if they
exist.
Signed-off-by: Cameron Baird <cameronbaird@microsoft.com>
By running on "all" host type there are two consequences:
1) run the "normal" tests too (until now, it's only "small" tests), so
increasing the coverage
2) create AKS cluster with larger VMs. This is a new requirement due to
the current ingress controller for the KBS service eating too much
vCPUs and lefting only few for the tests (resulting on failures)
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
_print_instance_type() returns the instance type of the AKS nodes, based
on the host type. Tests are grouped per host type in "small" and "normal"
sets based on the CPU requirements: "small" tests require few CPUs and
"normal" more.
There is an 3rd case: "all" host type maps to the union of "small"
and "normal" tests, which should be handled by _print_instance_type()
properly. In this case, it should return the largest instance type
possible because "normal" tests will be executed too.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
It's used an AKS managed ingress controller which keeps two nginx pod
replicas where both request 500m of CPU. On small VMs like we've used on
CI for running the CoCo non-TEE tests, it left only a few amount of CPU
for the tests. Actually, one of these pod replicas won't even get
started. So let's patch the ingress controller to have only one replica
of nginx.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The Azure AKS addon-http-application-routing add-on is deprecated and
cannot be enabled on new clusters which has caused some CI jobs to fail.
Migrated our code to use approuting instead. Unlike
addon-http-application-routing, this add-on doesn't
configure a managed cluster DNS zone, but the created ingress has a
public IP. To avoid having to deal with DNS setup, we will be using that
address from now on. Thus, some functions no longer used are deleted.
Fixes#11156
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Once the multiInstallSuffix has been taken into account, we should not
keep appending it on every re-run/restart, as that would lead to a path
that does not exist.
Fixes: #11187
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
In #11044, `run-k8s-tests-coco-nontee` was set as requried by mistake.
This PR disables the test again.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
We're bringing to *Cloud Hypervisor only* the reclaim_guest_freed_memory
option already present in the runtime-rs.
This allows us to use virtio-balloon for the hypervisor to reclaim
memory freed by the guest.
The reason we're not touching other hypervisors is because we're very
much aware of avoiding to clutter the go code at this point, so we'll
leave it for whoever really needs this on other hypervisor (and trust
me, we really do need it for Cloud Hypervisor right now ;-)).
Signed-off-by: Champ-Goblem <cameron@northflank.com>
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
The AKS CLI recently introduced a regression that prevents using
aks-preview extensions (Azure/azure-cli#31345), and hence create
CI clusters.
To address this, we temporarily hardcode the last known good version of
aks-preview.
Note that I removed the comment about this being a Mariner requirement,
as aks-preview is also a requirement of AKS App Routing, which will
be introduced soon in #11164.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Knowing that the upstream project provides a "ready to use" version of
the kernel, it's good to include an easy way to users to monitor
performance, and that's what we're doing by enabling the TASKSTATS (and
related) kernel configs.
This has been present as part of older kernels, but I couldn't
reasonably find the reason why it's been dropped.
Signed-off-by: Champ-Goblem <cameron@northflank.com>
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
Let's add a RUNTIME_CHOICE env var that can be passed to be build
scripts, which allows the user to select whether they bulld the go
runtime, the rust runtime, or both.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
genpolicy is sending more HTTPS requests than other components during
CI so it's more likely to be affected by transient network errors
similar to:
ConnectError(
"dns error",
Custom {
kind: Uncategorized,
error: "failed to lookup address information: Try again",
},
)
Note that genpolicy is not the only component hitting network errors
during CI. Recent example from a different component:
"Message: failed to create containerd task: failed to create shim task:
failed to async pull blob stream HTTP status server error (502 Bad Gateway)"
This CI change might help just with the genpolicy errors.
Fixes: #11182
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
We can provide devices during cold-plug with CDI annotation on a Pod
level and add per container device information wit the device plugin.
Since the sandbox has already attached the VFIO device remove them
from consideration and just apply the inner runtime CDI annotation.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The addition of CDI devices is now done for single_container
and pod_sandbox and pod_container before the devmanager creates
the deviceinfos no need for extra parsing.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
After the outer runtime has processed the CDI annotation from the
spec we can delete them since they were converted into Linux
devices in the OCI spec.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Every so often the main gnu site has an outage, so
we can't download gperf. GNU providesthe generic URL https://ftpmirror.gnu.org to
automatically choose a nearby and up-to-date mirror,
so switch to this to help avoid this problem
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
If the correct version of go is already installed then
install_go.sh runs `exit`. When calling this as source from
cri-containerd/gha-run.sh it means all dependencies after
are skipped, so remove this.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Bump golang version to the latest minor 1.23.x release
now that 1.24 has been released and 1.22.x is no longer
stable and receiving security fixes
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Add apt/apt-get updates before we do
apt/apt-get installs to try and help with
issues where we fail to fetch packages
Co-authored-by: Fabiano Fidêncio <fidencio@northflank.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This test we will test initdata in the following logic
1. Enable image signature verification via kernel commandline
2. Set Trustee address via initdata
3. Pull an image from a banned registry
4. Check if the pulling fails with log `image security validation
failed` the initdata works.
Note that if initdata does not work, the pod still fails to launch. But
the error information is `[CDH] [ERROR]: Get Resource failed` which
internally means that the KBS URL has not been set correctly.
This test now only runs on qemu-coco-dev+x86_64 and qemu-tdx
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
For a long time, there has been unformatted code in the kata-types
codebase, for example:
```
if qemu.memory_info.enable_guest_swap {
- return Err(eother!(
- "Qemu hypervisor doesn't support enable_guest_swap"
- ));
+ return Err(eother!("Qemu hypervisor doesn't support
enable_guest_swap"));
}
...
- }, device::DRIVER_NVDIMM_TYPE, eother, resolve_path
+ },
+ device::DRIVER_NVDIMM_TYPE,
+ eother, resolve_path,
-use std::collections::HashMap;
-use anyhow::{Result, anyhow};
+use anyhow::{anyhow, Result};
use std::collections::hash_map::Entry;
+use std::collections::HashMap;
-/// DRIVER_VFIO_PCI_GK_TYPE is the device driver for vfio-pci
+/// DRIVER_VFIO_PCI_GK_TYPE is the device driver for vfio-pci
```
This has brought unnecessary difficulties in version maintenance and
commit difficulties. This commit will address this issue.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
The Coniguration initialization was observed to be significantly slow
due to the extensive system information gathering performed by
`sysinfo::System::new_all()`. This function collects data on CPU,
memory, disks, and network, most of which is unnecessary for Kata's
memory adjusting config phase, where only the total system memory is
required.
This commit optimizes the initialization process by implementing a more
targeted approach to retrieve only the total system memory. This avoids
the overhead of collecting a large amount of irrelevant data, resulting
in a noticeable performance improvement.
Fixes#11165
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
We need more information (BAR memory and other future
ures...)for
PCI devices when vfio devices passed through.
So the method get_bars_max_addressable_memory is introduced for vfio
devices to deduce the memory_reserve and pref64_reserve for NVIDIA
devices. But it will be extended for other devices.
Fixes#10556
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
It's the basic framework for getting information of pci devices.
Currently, we focus on the PCI Max bar memory size, but it'll be
extended in the future.
Fixes#10556
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Use `|=` instead of `+=` while calculating and iterating through a
vector of flags, which makes more sense and prevents situations like
duplicated flags in vector, which would cause problems.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Currently, s390x only tests cri-containerd. Partially converge to the
feature set of basic-ci-amd64:
- containerd-sandboxapi
- containerd-stability
- docker
with the appropriate hypervisors.
Do not run tests currently skipped on amd64, as well as
- agent-ctl, which we don't package for s390x
- nerdctl, does not package the `full` image for s390x
- nydus, does not package for s390x
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Co-authored-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Recent PR #10732 moved the deletion of systemd files and units that were
deemed uneccessary by 02b3b3b977 from `image_builder.sh` to `rootfs.sh`.
This unfortunately broke `rootfs.sh centos` and `rootfs.sh -r` as used by
some other downstream users like fedora and RHEL, with the following error :
Warning FailedCreatePodSandBox 1s (x5 over 63s) kubelet
Failed to create pod sandbox: rpc error: code = Unknown
desc = CreateContainer failed: Establishing a D-Bus connection
Caused by:
0: I/O error: Connection reset by peer (os error 104)
1: Connection reset by peer (os error 104)
This is because the aforementioned distros use dbus-broker [1] that requires
systemd-journald to be present.
It is questionable that systemd units or files should be deemed unnecessary
for _all_ distros but this has been around since 2019. There's now also a
long-standing expectation from CI that `make rootfs && make image` does
remove these files.
In order to accomodate all the expectations, add a `-d` flag to `rootfs.sh`
to delete the systemd files and have `make rootfs` to use it.
[1] https://github.com/bus1/dbus-broker
Reported-by: Niteesh Dubey <niteesh@us.ibm.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
The CoCo non-TEE job has failed due the removal of an add-on
from AKS, causing KBS to not get installed (see #11156).
The fix should be done in this repo as well as in trustee, which can
take some time. We don't want to hold kata-containers PRs from getting
merged anylonger, so removing the job from required list.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
this script will be used in a new OCP integration pipeline to monitor
basic workflows of OCP+peer-pods.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
io.katacontainers.config.runtime.cc_init_data specifies initdata used by
the pod in base64(gzip(initdata toml)) format. The initdata will be
encapsulated into an initdata image and mount it as a raw block device
to the guest.
The initdata image will be aligned with 512 bytes, which is chosen as a
usual sector size supported by different hypervisors like qemu, clh and
dragonball.
Note that this patch only adds support for qemu hypervisor.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
This commit adds changes to add input container_id and related
container data to state after a CreateContainerRequest is allowed. This
helps constrain reference container data for evaluating request
inputs to one instead of matching against every policy container data,
Ex: in ExecProcessRequest inputs.
Fixes#11109
Signed-off-by: Sumedh Sharma <sumsharma@microsoft.com>
We need to make sure the device files are created correctly
in the rootfs otherwise kata-agent will apply permission 0o000.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
It's been released for some time already ... and although we did have
the necessary patches in, we better to stick to a released version of
the project.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
This is mostly used for Kata Containers backing up Confidential
Computing use cases, this also has benefits for the normal Kata
Containers use cases, this it's left enabled by default.
However, let's allow users to specify whether or not they want to have
it enabled, as depending on their use-case, it just does not make sense.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
Right now we've had some logic to add EXTRA_PKGS, but those were
restrict to the nvidia builds, and would require changing the file
manually.
Let's make sure a user can add this just by specifying an env var.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
Kata Containers provides, since forever, a way to run OCI guest-hooks
from the rootfs, as long as the files are dropped in a specific location
defined in the configuration.toml.
However, so far, it's been up to the ones using it to hack the generated
image in order to add those guest hooks, which is far from handy.
Let's add a way for the ones interested on this feature to just drop a
tarball file under the same known build directory, spcificy an env var,
and let the guest hooks be installed during the rootfs build.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
Add a top-level rust-toolchain.toml with the version
that matches version.yaml to ensure that we stay in sync
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Kata-agent now will check if a device /dev/vd* with 'initdata' magic
number exists. If it exists, kata-agent will try to read it. Bytes 9~16
are the length of the compressed initdata toml in little endine.
Bytes starting from 17 is the compressed initdata.
The initdata image device layout looks like
0 8 16 16+length ... EOF
'initdata' length gzip(initdata toml) paddings
The initdata will be parsed and put as aa.toml, cdh.toml and
policy.rego to /run/confidential-containers/initdata.
When AgentPolicy is initialized, the default policy will be overwritten
by that.
When AA is to be launched, if initdata is once processed, the launch arg
will include --initdata parameter.
Also, if
/run/confidential-containers/initdata/aa.toml exists, the launch args
will include -c /run/confidential-containers/initdata/aa.toml.
When CDH is to be launched, if initdata is once processed, the launch
args will include -c /run/confidential-containers/initdata/cdh.toml
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
When making new tests required, or removing existing tests
from required, this doesn't impact the CI jobs, so we don't need
to run all the tests.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Remove metrics setup job
- Update some truncation typos of job names
- Add shellcheck-required
- Remove the ok-to-test as a required label on the build test
as it isn't needed as a trigger
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
To reduce duplication, we could update
the vsock-exporter crate to use settings and versions
from the agent, where applicable.
> [!NOTE]
> In order to use the workspace, this has bumped some crate versions
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- To reduce duplication, we could update
the rustjail crate to use settings and versions
from the agent, where applicable.
- Also switch to using the derive feature in serde crate
rather than the separate serde_derive to avoid keeping
both versions in sync
> [!NOTE]
> In order to use the workspace, this has bumped
some crate versions
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
To reduce duplication, we could update
the policy crate to use settings and versions
from the agent, where applicable.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Create agent workspace dependencies and packge info
so that the packages in the workspace can use them
- Group the local dependencies together for clarity
(like in #11129)
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Put local dependencies into workspace to avoid complex path dependencies all
over the workspace. This gives an overview of local dependencies this workspace
uses, where those crates are located, and simplifies the local dependencies
referencing process.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Judging by the layout of the `Cargo.toml` files, local dependencies are
intentionally separated from other dependencies, let's enforce it
workspace-wise.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Only `shim` and `shim-ctl` are incorporated in `runtime-rs`'s workspace, let's
extend it to cover all crates in `runtime-rs/crates`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Just get base name from iommu group symlink is enough. As the
validation will be handled in subsequent steps when constructing
the full path /sys/kernel/iommu_groups/$iommu_group.
In this PR, it will remove dupicalted validation of iommu_group.
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
For those not interested in CoCo, let's at least allow them to easily
build the agent without the guest-pull feature.
This reduces the binary size (already stripped) from 25M to 18M.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This commit introduces missing validations for input fields in ExecProcessRequest to
harden the security policy.
The changes include:
- Update rules.rego to add null/empty field enforcements for String_user, SelinuxLabel and ApparmorProfile
- Add unit test cases for ExecProcessRequest for each of the validations
Signed-off-by: Ankita Pareek <ankitapareek@microsoft.com>
Add how-to-use-memory-agent.md (How to use mem-agent to decrease the
memory usage of Kata container) to docs to show how to use mem-agent.
Fixes: #11013
Signed-off-by: Hui Zhu <teawater@gmail.com>
some of the e2e tests spawn a lot of workers which are mainly idle, but
the scheduler fails to schedule them due to cpu resource overcommit. For
our testing we are more focused on having actual pods running than the
speed of the scheduled pods so let's increase the amount of schedulable
pods by decreasing the default cpu requests.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Previously we introduced `build-kata-static-tarball-riscv64.yaml`,
enable that workflow in `ci.yaml`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
INSTALLATION_PREFIX must begin with a "/"
because it is being concatenated with /host.
If there is no /, displays a message and makes an error.
Fixes: #11096
Signed-off-by: Shunsuke Kimura <pbrehpuum@gmail.com>
Since it is difficult to update the README when modifying the options of ./build-kernel.sh,
instead of update the README, we encourage users to run the -h command.
Fixes: #11065
Signed-off-by: Shunsuke Kimura <pbrehpuum@gmail.com>
While running `cargo fmt -- --check` in `src/runtime-rs` directory, it
errors out and suggesting these is an redundant empty line, which
prevents `make check` of `runtime-rs` component from passing.
Remove redundant empty line to fix this.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
kata-deploy tests have been quite stable, working for more than 10 days
without any nightly failure (or any failure reported at all), and I'll
be the one maintaining those.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
`VMContainerCapable` requires a present `kvm` device, which is not yet
available in our RISC-V runners. Skipped related tests if it is running
on `riscv-builder`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Provide according tests to cover `kata-runtime` package, test
`kata-runtime`'s `check` functionality on riscv64 platforms.
Signed-off-by: Yuting Nie <nieyuting@iscas.ac.cn>
Add `getExpectedHostDetails` with expected value according to template
defined in `kata-check_data_riscv64_test.go`. This provides necessary
`HostInfo` for tests to cover `kata-check_riscv64.go`.
Signed-off-by: Yuting Nie <nieyuting@iscas.ac.cn>
Add definition of `testCPUInfoTemplate` which is retrieved from
`/proc/cpuinfo` of a QEMU emulated virtual machine on virt board.
Signed-off-by: Yuting Nie <nieyuting@iscas.ac.cn>
`testSetCPUTypeGeneric` will be used for writting `kata-check` in
`kata-runtime` on riscv64 platforms, enable building for later testing.
Signed-off-by: Yuting Nie <nieyuting@iscas.ac.cn>
Convert Rust arch to Go arch in Makefile, and add `riscv64-options.mk`
to provide definitions required for runtime to build on riscv64.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Enable `kata-runtime check` command to work on riscv64 platforms to make
sure required features/devices presents.
Co-authored-by: Yuting Nie <nieyuting@iscas.ac.cn>
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
We get the following error while writing containerd config
if a base dir `/etc/containerd` does not exist like:
```
sudo tee /etc/containerd/config.toml << EOF
...
EOF
tee: /etc/containerd/config.toml: No such file or directory
```
The commit makes sure a base directory for containerd before
writing config and drops the config file deletion because a
default behaviour of `tee` is overwriting.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
As reported in #11011, mounted secrets are available after
a container image is pulled by add_storage() for IBM SE.
But secure mount should be handled before the `add_storage()`.
Therefore, this commit divides cdh_handler() into:
- cdh_handler_trusted_storage()
- cdh_handler_sealed_secrets()
and calls cdh_handler_sealed_secrets() after add_storage()
while keeping cdh_handler_trusted_storage() unchanged.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The test `Cannot get CDH resource when deny-all policy is set`
completes with a KBS policy set to deny-all. This affects the
future TEE test (e.g. k8s-sealed-secrets.bats) which makes a
request against KBS.
This commit introduces kbs_set_default_policy() and puts it to
the setup() in k8s-sealed-secrets.bats.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Kata Containers has support for both the IBM Secure Execution trusted
execution environment and the IBM Crypto Express hardware security
module (used via the Adjunct Processor bus), but using them together
requires specific steps.
In Secure Execution, the Acceleration and Enterprise PKCS11 modes of
Crypto Express are supported. Both modes require the domain to be
_bound_ in the guest, and the latter also requires the domain to be
_associated_ with a _guest secret_. Guest secrets must be submitted to
the ultravisor from within the guest.
Each EP11 domain has a master key verification pattern (MKVP) that can
be established at HSM setup time. The guest secret and its ID are to
be provided at `/vfio_ap/{mkvp}/secret` and
`/vfio_ap/{mkvp}/secret_id` via a key broker service respectively.
Bind each domain, and for each EP11 domain,
- get the secret and secret ID from the addresses above,
- submit the secret to the ultravisor,
- find the index of the secret corresponding to the ID, and
- associate the domain to the index of this secret.
To bind, add the secret, parse the info about the domain, and
associate, the s390_pv_core crate is used. The code from this crate
also does the AP online check, which can be removed from here.
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
With this we switch to fully testing with helm, instead of testimg with
the kustomizations (which will soon be removed).
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Let's use what we have in the k8s functional tests to create a common
function to deploy kata containers using our helm charts. This will
help us immensely in the kata-deploy testing side in the near future.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This is not strictly needed, but it does help a lot when setting up a
cluster manually, while still relying on those scripts.
While here, let's also ensure the assignment is between quotes, to make
shellchecker happier.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This commit introduces changes to add test data for multiple request
type in a single testcases.json file. This allows for stateful testing,
for ex: enable testing ExecProcessRequest using policy state set after testing
a CreateContainerRequest.
Fixes#11073.
Signed-off-by: Sumedh Sharma <sumsharma@microsoft.com>
TDX Quote Generation Service (QGS) signs TDREPORT sent to it from
Qemu (GetQuote hypercall). Qemu needs quote-generation-socket
address configured for IPC.
Currently, Kata govmm only enables vsock based IPC for QGS but
QGS supports Unix Domain Sockets too which works well for host
process to process IPC (Qemu <-> QGS).
The QGS configuration to enable UDS is to run the service with "-port=0"
parameter. The same works well here too: setting
"tdx_quote_generation_service_socket_port=0" let's users to enable
UDS based IPC.
The socket path is fixed in QGS and cannot be configured: when "-port=0"
is used, the socket appears in /var/run/tdx-qgs/qgs.socket.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Use "cargo build --release" when BUILD_TYPE was not specified, or when
BUILD_TYPE=release. The default "cargo build" behavior is to build in
debug mode.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
It hangs when invalid arguments are specified.
```bash
kata-deploy-6sr2p:/# /opt/kata-artifacts/scripts/kata-deploy.sh xxx
Action:
* xxx
...
Usage: /opt/kata-artifacts/scripts/kata-deploy.sh [install/cleanup/reset]
ERROR: invalid arguments
...
^C <- hang
```
I changed it to behave the same as when there are no arguments.
```bash
kata-deploy-6sr2p:/# /opt/kata-artifacts/scripts/kata-deploy.sh
Usage: /opt/kata-artifacts/scripts/kata-deploy.sh [install/cleanup/reset]
ERROR: invalid arguments
kata-deploy-6sr2p:/# echo $?
1
```
Fixes: #11068
Signed-off-by: Shunsuke Kimura <pbrehpuum@gmail.com>
Since some files generated by protobuf were share between
runtime-rs and kata agent, and the kata agent's dependency
image-rs dependened protobuf@3.7.1, thus we'd better to keep
the protobuf version aligned between runtime-rs and agent,
otherwise, we couldn't compile the runtime-rs and agent
at the same time.
Fixes: https://github.com/kata-containers/kata-containers/issues/10650
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
`GITHUB_RUNNER_CI_ARM64` is turned on for self hosted runners without
virtualization to skipped those tests depend on virtualization. This may
happen to other archs/runners as well, let's generalize it to
`GITHUB_RUNNER_CI_NON_VIRT` so we can reuse it on other archs.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
More straightforward implementation of hard_coded_policy_tests_enabled,
that avoids ShellCheck warning:
warning: Remove quotes from right-hand side of =~ to match as a regex rather than literally. [SC2076]
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Fix unintended use of caller's variable. Use the corresponding function
parameter instead. ShellCheck:
warning: policy_settings_dir is referenced but not assigned. [SC2154]
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Avoid masking command return values by declaring and only then assigning.
ShellCheck:
warning: Declare and assign separately to avoid masking return values. [SC2155]
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Pick the the values exported by other scripts. ShellCheck:
warning: AUTO_GENERATE_POLICY is referenced but not assigned. [SC2154]
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
ShellCheck:
warning: This assignment is only seen by the forked process. [SC2097]
warning: This expansion will not see the mentioned assignment. [SC2098]
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
ShellCheck: add braces around variable references:
note: Prefer putting braces around variable references even when not strictly required. [SC2250]
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
ShellCheck: export variables used outside of tests_common.sh - e.g.,
warning: timeout appears unused. Verify use (or export if used externally). [SC2034]
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Replace [ ] with [[ ]] as advised by shellcheck:
note: Prefer [[ ]] over [ ] for tests in Bash/Ksh. [SC2292]
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The metrics tests haven't been stable, or required through
github for many week now, so update the required-tests.yaml
list to re-sync
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
with the latest CoCo guest-components, tdx-attester no longer
depends on libtdx attest. Stop installing it to the rootfs.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
`go.opentelemetry.io/otel/trace.NewNoopTracerProvider`
is deprectated now, so switch to
`go.opentelemetry.io/otel/trace/noop.NewTracerProvider`
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
SC2248 (style): Prefer double quoting even when variables don't contain
special characters, might result in arguments difference, shouldn't in
our cases.
Related to: #10951
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
SC2155 (warning): Declare and assign separately to avoid masking return
values, should be harmless.
Related to: #10951
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
SC2086 Double quote to prevent globbing and word splitting, might break
places where we deliberately use word splitting, but we are not using it
here.
Related to: #10951
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
SC2292 (style): Prefer [[ ]] over [ ] for tests in Bash/Ksh. This might
result in different handling of globs and some ops which we don't use.
Related to: #10951
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
`runtime-rs` is now buildable and testable on riscv64 platforms, enable
`build-check` on `runtime-rs`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`available_guets_protection` is required for `runtime-rs` to infer while
building it on riscv64 platforms. Set it to `NoProtection` as riscv64
does not support guest protection for now.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Same rationale as for runtime. With tests, the blackfriday replacement was
actually meaningful, so I refactored some imports.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
These replace directives aren't understood by dependabot, hence dependabot can
claim to upgrade a dependency, while a replace directive still makes the
dependency point to an old version.
Fixes: #11020
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
The SNP CI has been consistently passing and we request the @kata-containers/architecture-committee to mark this test as a required test.
Signed-Off-By: Adithya Krishnan Kannan <AdithyaKrishnan.Kannan@amd.com>
Retry "kubectl exec" a few times if it unexpectedly produced an empty
output string.
This is an attempt to work around test failures similar to:
https://github.com/kata-containers/kata-containers/actions/runs/13840930994/job/38730153687?pr=10983
not ok 1 Environment variables
(from function `grep_pod_exec_output' in file tests_common.sh, line 394,
in test file k8s-env.bats, line 36)
`grep_pod_exec_output "${pod_name}" "HOST_IP=\([0-9]\+\(\.\|$\)\)\{4\}" "${exec_command[@]}"' failed
That test obtained correct ouput from "sh -c printenv" one time, but the
second execution of the same command returned an empty output string.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
image-rs has gotten a number of significant updates, eliminating corner
cases with obscure containers, improving support for local certs, and
more.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Update to the latest hash of guest-components. This will pick up some
nice new features including using ec key for the rcar handshake.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Update to new hashes for Trustee. The MSRV for Trustee is now 1.80.0 so
bump the rust toolchain as well.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
In ef0e8669fb we
had been seeing some significantly lower minvalues in
the jitter.Result test, so I lowered the mid-value rather
than having a very high minpercent, but it appears that the
variability of this result is very high, so we are still getting
the occasional high value, so reset the midval and just
have a bigger ranges on both sides, to try and keep the test
stable.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The kubectl wait has a built in timeout of 30s, so
wrapping it in waitForProcess, means we have
180/2 * 30 delay, which is much longer than intended,
so just set the timeout directly.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This PR makes changes to remove steps to run scripts for
preparing and cleaning the runner and instead use runner
hooks env variables to manage them.
Fixes: #9934
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
For a use case, we want to set the SNP IDBlock, which allows
configuring the AMD ASP to enforce parameters like expected launch
digest at launch. The struct with the config that should be enforced
(IDBlock) is signed. The public key is placed in the auth block and
the signature is verified by the ASP before launch. The digest of the
public key is also part of the attestation report (ID_KEY_DIGESTS).
Signed-off-by: Paul Meyer <katexochen0@gmail.com>
Test using the host path /tmp/k8s-policy-pod-test instead of
/var/lib/kubelet/pods.
/var/lib/kubelet/pods might happen to contain files that CopyFileRequest
would try to send to the Guest before CreateContainerRequest. Such
CopyFileRequest was an unintended side effect of this test.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Validate sandbox name using a regex. If the YAML specifies metadata.name, use a regex that exact matches.
If the YAML specifies metadata.generateName, use a regex that matches the prefix of the generated name.
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
We have three SG2042 connected and labeled as `riscv-builder`, add that
entry to `actionlint.yaml` to help linting while setting up workflows.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Enable `kernel` and `virtiofsd` static-tarball build for riscv64. Since
`virtiofsd` was previously supported and `kernel` is supported now.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
AIA (Advanced Interrupt Architecture) is available and enabled by
default after v6.10 kernel, provide pci.conf to make proper use of IMSIC
of AIA.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Create `riscv` folder for riscv64 architecture to be inferred while
constructing kernel configuration, and introduce `base.conf` which
builds 64-bit kernel and with KVM built-in to kernel.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Introduce rule to block routes from source addresses which are the
loopback. Block routes added to the lo device.
Signed-off-by: Cameron Baird <cameronbaird@microsoft.com>
AddSwap send the pci path to guest kernel to let it add swap device.
But some mmio device doesn't have pci path. To support it add
AddSwapPath send virt_path to guest kernel as swap device.
Fixes: #10988
Signed-off-by: Hui Zhu <teawater@antgroup.com>
This commit add guest swap support.
When configuration enable_guest_swap is enabled, runtime-rs will start a
swap task.
When the VM start or update the guest memory, the swap task will be
waked up to create and insert a swap file.
Before this job, swap task will sleep some seconds (set by configuration
guest_swap_create_threshold_secs) to reduce the impact on guest kernel
boot performance and prevent the insertion of multiple swap files due to
frequent memory elasticity within a short period.
The size of swap file is set by configuration guest_swap_size_percent.
The percentage of the total memory to be used as swap device.
Fixes: #10988
Signed-off-by: Hui Zhu <teawater@antgroup.com>
Add is_direct to struct BlockConfig.
This option specifies cache-related options for block devices.
Denotes whether use of O_DIRECT (bypass the host page cache) is enabled.
If not set, use configurarion block_device_cache_direct.
Fixes: #10988
Signed-off-by: Hui Zhu <teawater@antgroup.com>
Log the "kubectl exec" ouput, just in case it helps investigate sporadic
test errors like:
https://github.com/kata-containers/kata-containers/actions/runs/13724022494/job/38387329321?pr=10973
not ok 1 Environment variables
(in test file k8s-env.bats, line 37)
`grep "HOST_IP=\([0-9]\+\(\.\|$\)\)\{4\}"' failed
It appears that the first exec from this test case produced the expected
output:
MY_POD_NAME=test-env
but the second exec produced something else - that will be logged after
this change.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Log the "kubectl exec" ouput, just in case it helps investigate sporadic
test errors like:
https://github.com/kata-containers/kata-containers/actions/runs/13724022494/job/38387329268?pr=10973
not ok 1 ConfigMap for a pod
(in test file k8s-configmap.bats, line 44)
`kubectl exec $pod_name -- "${exec_command[@]}" | grep "KUBE_CONFIG_2=value-2"' failed
It appears that the first exec from this test case produced the expected
output:
KUBE_CONFIG_1=value-1
but the second exec produced something else - that will be logged after
this change.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
grep_pod_exec_output invokes "kubectl exec", logs its output, and checks
that a grep pattern is present in the output.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
We can use the netlink update method to add a route or an interface
address. There is no need to delete it first and then add it. This can
save two system commissions.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Get the route entry's flags from the host and
pass it into kata-agent to add route entries
with flags support.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
We should support the flags when add the route from
host to guest. Otherwise, some route would be set
failed.
Fixes: #7934
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
./tests/git-helper.sh:20:5: note: Prefer [[ ]] over [ ] for tests in Bash/Ksh. [SC2292]
./tests/git-helper.sh:22:26: note: Double quote to prevent globbing and word splitting. [SC2086]
./tests/git-helper.sh:23:7: note: Prefer [[ ]] over [ ] for tests in Bash/Ksh. [SC2292]
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Switch to `docker.io` provided by Ubuntu sources. It is not necessary
for us to install docker through `get-docker.sh`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
- In the previous PR I only skipped the runtime/vendor
directory, but errors are showing up in other vendor
packages, so try a wildcard skip
- Also update the job step was we can distinguish between the
required and non-required versions
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- remove hard link to v.1.47.0 in go.mod
- run go mod tidy, go mod vendor to actually update to v1.58.3
- addresses CVE-2023-44487
Signed-off-by: Manuel Huber <mahuber@microsoft.com>
Cloud-Hypervisor currently only supports `x86_64` and `aarch64`, this
features should not be avaiable even if other architectures explicitly
requires it.
Restrict `cloud-hypervisor` feature to only `x86_64` and `aarch64`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Drop `target_arch = "s390x"` all over `runtime-rs`, it is strange to
have such predicates on features and code while we do not support it.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
As we'll touch this file during this series, let's already make sure we
solve all the needed warnings.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
He were fixing the few warnings we found in the files present in the
functional tests for kata-deploy.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
It doesn't make much sense to test different VMMs as that wouldn't
trigger a different code path.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
The previous PR mistakenly set all perms to 0o666 we should follow
what runc does and fetch the permission from the guest aka host
if the file_mode == 0. If we do not find the device on the guest aka
host fallback to 0.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
As we're testing against the LTS and the Active versions of
containers, let's upgrade the lts version from 1.6 to 1.7 and
active version from 1.7 to 2.0 to cover the sandboxapi tests.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
After the introduction of the following kernel parameters (see #6163):
```
CONFIG_SCLP_VT220_TTY=y
CONFIG_SCLP_VT220_CONSOLE=y
```
the system log for Kata components (e.g., the agent) no longer appeared
on the SCLP console (i.e., /dev/ttysclp0). Let's switch to the default
fallback console (likely /dev/console) for logging.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
When `KATA_HYPERVISOR` is set to `qemu-se-runtime-rs`,
a configuration file is properly referenced and a runtime class
should be created via kata-deploy.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
A configuration file, `configuration-qemu-se-runtime-rs.toml`,
is referenced when the `qemu-se-runtime-rs` runtime is configured.
This commit adds a template file and updates the Makefile configuration
accordingly.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
We had the default permissions set to 0o000 if the file_mode was not
present, for most container devices this is the wrong default. Since
those devices are meant also to be accessed by users and others add a
sane default of 0o666 to devices that do not have any permissions set.
Otherwise only root can acess those and we cannot run containers as a
user.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
`musl-tools` is only needed when a component needs `rust`, and the
`instance` running is of `x86_64` or `aarch64`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
We had a static mapping of host guest PCI addresses, which prevented to
use VFIO devices in initContainers. We're tracking now the host-guest
mapping per container and removing this mapping if a container is
removed.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Now we've added the double quotes around
`${K8S_TEST_UNION[@]}`, so platforms are
failing with:
```
Error: Test file "/home/ubuntu/runner/_layout/_work/kata-containers/kata-containers/tests/integration/kubernetes/k8s-nginx-connectivity.bats
" does not exist
```
due to the line continuation, so sanitise the value
to try and fix this.
Co-authored-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The space was missing before `]`, so fix this and also
swtich to double square brackets and variable braces
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This was checking that a literal string was non-zero.
I'm assume it instead wanted to check if the file exists
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
> In functions, use return instead of break.
> rationale: break or continue are used to abort or
continue a loop, and are not the right way to exit
a function. Use return instead.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
> Can only exit with status 0-255. Other data should be written to stdout/stderr.
Switch exit -1 to exit 1
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
> Argument mixes string and array. Use * or separate argument.
- Swap echos for printfs and improve formatting
- Replace $@ with $*
- Split arrays into separate arguments
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
I'm not sure if we use test_images anywhere, so before
we invest the time to fix the 120 shellcheck errors and warnings
we should decide if we want to keep it. See #10957
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Start with a required smaller set of shellchecks
to try and prevent regressions whilst we fix
the current problems
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Ignore the vendor directories in our shellcheck
workflow as we can't fix them. If there is a way to
set this in shellcheckrc that would be better, but
it doesn't seem to be implemented yet.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
When using `virtio-net-pci` for IBM SE, the following error occurs:
```
update interface: Link not found (Address: f2:21:48:25:f4:10)
```
On s390x, it is more appropriate to use the CCW type of virtio
network device.
This commit ensures that a subchannel is configured accordingly.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
For IBM SE, the following kernel parameters are not required:
- Basic parameters (reboot and systemd-related)
- Rootfs parameters
This commit suppresses these parameters when IBM SE is configured.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This commit configures the command line for IBM Secure Execution (SE)
and other TEEs. The following changes are made:
- Add a new item `Se` to ProtectionDeviceConfig and handle it at sandbox
- Introduce `add_se_protection_device()` for SE cmdline config
- Bypass rootfs image/initrd validity checks when SE is configured.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
`USE_BUILDIN_DB` is turned on by default for architectures do not
support `Dragonball`, which leads `s390x` is building `runtime-rs` with
`--features dragonball` presents.
Let's restrict `USE_BUILDIN_DB` to be enable only for architectures
supported by `Dragonball` (namely x86_64 and aarch64 as of now).
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
On qemu the run seems to error after ~4-7 runs, so try
a cut down version of repetitions to see if this helps us
get results in a stable way.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
We have a new metrics machine and environment
and the iperf jitter result failed as it finished too quickly,
so increase the minpercent to try and get it stable
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
We have a new metrics machine and environment
and the fio write.bw and iperf3 parallel.Results
tests failed for clh, as below
the minimum range, so increase the
minpercent to try and get it stable
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
We have a new metrics machine and environment
and the boot time test failed for clh, so increase the
maxpercent to try and get it stable
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The iperf deployment is quite a lot out of date
and uses `master` for it's affinity and toleration,
so update this to control-plane, so it can run on
newer Kubernetes clusters
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The new metrics runner seems slower, so we are
seeing errors like:
The iperf3 tests are failing with:
```
pod rejected: RuntimeClass "kata" not found
```
so give more time for it to succeed
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Move `kill_kata_components` from common.bash
into the metrics code base as the only user of it
- Increase the timeout on the start of containerd as
the last 10 nightlies metric tests have failed with:
```
223478 Killed sudo timeout -s SIGKILL "${TIMEOUT}" systemctl start containerd
```
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- As the metrics tests are largely independent
then allow subsequent tests to run even if previous
ones failed. The results might not be perfect if
clean-up is required, but we can work on that later.
- Move the test results check out of the latency
test that seems arbitrary and into it's own job step
- Add timeouts to steps that might fail/hang if there
are containerd/K8s issues
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Currently the run-metrics job runs a manual install
and does this in a separate job before the metrics
tests run. This doesn't make sense as if we have multiple
CI runs in parallel (like we often do), there is a high chance
that the setup for another PR runs between the metrics
setup and the runs, meaning it's not testing the correct
version of code. We want to remove this from happening,
so install (and delete to cleanup) kata as part of the metrics
test jobs.
Also switch to kata-deploy rather than manual install for
simplicity and in order to test what we recommend to users.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The drop-in path should be /etc/containerd (from the containers'
perspective), which mounts to the host path /etc/k0s/containerd.d.
With what we had we ended up dropping the file under the
/etc/k0s/containerd.d/containerd.d/, which is wrong.
This is a regression introduce by: 94b3348d3c
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Change kata-deploy script and Helm chart in order to be able to use kata-deploy on a microk8s cluster deployed with snap.
Fixes: #10830
Signed-off-by: Stephane Talbot <Stephane.Talbot@univ-savoie.fr>
Refator matrix setup and according dependencies installation logic in
`build-checks.yaml` and `build-checks-preview-riscv64.yaml` to provide
better readability and maintainability.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`install-libseccomp` is applied only for `agent` component, and we are
already combining matrix with `if`s in steps, drop `install-libseccomp`
in matrix to reduce complexity.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
At the proper step pass-through the var KBUILD_SIGN_PIN
so that the kernel_headers step has the PIN for encrypting
the signing key.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
In kata-deploy-binaries.sh we need to pass-through the var
KBUILD_SIGN_PIN to the other static builder scripts.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Update kata-deploy-binaries-in-docker.sh to read the
env variable KBUILD_SIGN_PIN that either can be set via
GHA or other means.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
We need to place the signing key and cert at the right place
and hide the KBUILD_SIGN_PIN from echo'ing or xtrace
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
If KBUILD_SIGN_PIN is provided we can encrypt the signing key
for out-of-tree builds and second round jobs in GHA
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The GitHub hosted runners for ARM64 do not provide virtualisation
support, thus we're just skipping the tests as those would check whether
or not the system is "VMContainerCapable".
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Update the code to install the version of k0s
that we have in our versions.yaml, rather than
just installing the latest, to help our CI being
less stable and prone to breaking due to things
we don't control.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Add external versions support for k0s and
initially pin it at v1.31.5 as our cri-o tests
started failing when v1.32 became the latest
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
In some cases, /init is not following two levels of symlinks
i.e. /init to /sbin/init to /lib/systemd/systemd
Setting /init directly to /lib/systemd/systemd when AGENT_INIT is not mandated
Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
Sysctls may be added to a container by the Kubernetes pod definition or
by containerd configuration. This commit adds support for the
corresponding PodSecurityContext field and an option to specify
environment-dependent sysctls in the settings file.
The sysctls requested in a CreateContainerRequest are checked against
the sysctls in the pod definition, or if not defined there in the
defaults in genpolicy-settings.json. There is no check for the presence
of expected sysctls, though, because Kubernetes might legitimately
omit unsafe syscalls itself and because default sysctls might not apply
to all containers.
Fixes: #10064
Signed-off-by: Markus Rudy <mr@edgeless.systems>
On s390x, a virtio-net device will use the CCW bus instead of PCI,
which impacts how its uevent should be handled. Take the respective
path accordingly.
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
To support virtio-net-ccw for s390x, add CCW devices to the Endpoint
interface. Add respective fields and functions to implementing structs.
Device paths may be empty. PciPath resolves this by being a list that
may be empty, but this design does not map to CcwDevice. Use a pointer
instead.
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Field is being used for both PCI and CCW devices. Name it devicePath
to avoid confusion when the device isn't a PCI device.
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
As part of device preparation in Sandbox we check available protection
and create a corresponding ProtectionDeviceConfig if appropriate. The
resource-side handling is trivial.
Signed-off-by: Pavel Mores <pmores@redhat.com>
As an example, or a test case, we add some implementation of SEV/SEV-SNP.
Within the QEMU command line generation, the 'Cpu' object is extended to
accomodate the EPYC-v4 CPU type for SEV-SNP.
'Machine' is extended to support the confidential-guest-support parameter
which is useful for other TEEs as well.
Support for emitting the -bios command line switch is added as that seems
to be the preferred way of supplying a path to firmware for SEV/SEV-SNP.
Support for emitting '-object sev-guest' and '-object sev-snp-guest'
with an appropriate set of parameters is added as well.
Signed-off-by: Pavel Mores <pmores@redhat.com>
ProtectionDevice is a new device type whose implementation structure
matches the one of other devices in the device module. It is split into
an inner "config" part which contains device details (we implement
SEV/SEV-SNP for now) and the customary outer "device" part which just adds
a device instance ID and the customary Device trait implementation.
Signed-off-by: Pavel Mores <pmores@redhat.com>
This matches the existing TDX handling where additional details are
retrieved right away after TDX is detected. Note that the actual details
(cbitpos) acquisition is NOT included at this time.
This change might seem bigger than it is. The change itself is just in
protection.rs, the rest are corresponding adjustments.
Signed-off-by: Pavel Mores <pmores@redhat.com>
This function was accidentally left unimplemented for CronJob, resulting
in runAsUser not being supported there.
Fixes: #10653
Signed-off-by: Markus Rudy <mr@edgeless.systems>
Since we have RISC-V builders available now, let's start with
`agent-ctl`, `trace-forwarder` and `genpolicy` components to run
build-checks on these `riscv-builder`s, and gradually add the rest
components when they are ready, to catch up with other architectures
eventually.
This workflow could be mannually triggered, `riscv-builder` will be the
default instance when that is the case.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Adding devices by CDI annotation can fail for a variety of reasons. If
that happens, it's helpful to know the root cause of the issue (CDI spec
missing, malformatted, requested device not present, etc.).
This commit adds the root cause of the CDI device addition to the errors
reported back to the caller. Since this error is bubbled up all the way
back to the shimv2 task.Create handler, it will be visible in Kubernetes
logs and enable fixing the root cause.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
Currently, if a layer can't be processed, we log this a warning and
continue execution, finally exit with a zero exit code. This can lead
to the generation of invalid policies. One reason a layer might not be
processed is that the pull of that layer fails.
We need all layers to be processed successfully to generate a valid
policy, as otherwise we will miss the verity hash for that layer or
we might miss the USER information from a passwd stored in that layer.
This will cause our VM to not get through the agent's policy validation.
Returning an error instead of printing a warning will cause genpolicy
to fail in such cases.
Signed-off-by: Paul Meyer <katexochen0@gmail.com>
As the guest-pull is a very Confidental Containers specific feature,
let's make sure we, at least, don't break folks who decide to build Kata
Containers' agent without having this feature enabled (for instance, for
the sake of the agent size).
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Architectures here with `musl` available are minority, which is more
suitable for enumeration.
With this change, we are implicitly choosing gnu target for `ppc64le`,
`riscv64` and `s390x`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
While installing Rust and Golang in our CI workflow, `arch_to_golang`
and `arch_to_rust` are needed for inferring the correct arch string for
riscv64 architecture.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Since `ARCH` for `riscv64` is `riscv64gc`, we'll need to override it in
`utils.mk`, and forcing `gnu` target for `riscv64` because `musl` target
is not yet made ready.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
We need a proper ID otherwise QEMU sometimes fails with invalid ID.
Use the same pattern as with the old VFIO implementation.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
I noticed that CodeQl using the default config hasn't
scanned since May 2024, so figured it would be worth
trying an explicit configuration to see if that gets better results.
It's mostly the template, but updated to be more relevant:
- Only scan PRs and pushes to the `main` branch
- Set a pinned runner version rather than latest (with mac support)
- Edit the list of languages to be scanned to be more relevant
for kata-containers
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Currently the ci-on-push workflow that runs on PRs runs
two jobs: gatekeeper-skipper.yaml and ci.yaml. In order
to test things like for the error
```
too many workflows are referenced, total: 21, limit: 20
```
on topic branches, we need ci-devel.yaml to have an
extra workflow to match ci-on-push, so add the build-checks
as this is helpful to run on topic branches anyway.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Replace the four different publish workflows with
a single one that take input parameters of the arch
and runner, so reduce the amount of duplicated code
and try and avoid the
```
too many workflows are referenced, total: 21, limit: 20
```
error
Let's take advantege of the current arm64 runners, and make sure we have
those tests running there as well.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Signed-off-by: Kevin Zhao <kevin.zhao@linaro.org>
There are many `override ARCH = powerpc64le` after where `utils.mk` is
included, which are redundant.
Drop those redundant `override`s.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
In the CI, test containers intermittently fail to start after creation,
with an error like below (see #10872 for more details):
# State: Terminated
# Reason: StartError
# Message: failed to start containerd task "afd43e77fae0815afbc7205eac78f94859e247968a6a4e8bcbb987690fcf10a6": No such file or directory (os error 2)
I've observed this error to repro with the following containers, which
have in common that they're all *very short-lived* by design (more tests
might be affected):
* k8s-job.bats
* k8s-seccomp.bats
* k8s-hostname.bats
* k8s-policy-job.bats
* k8s-policy-logs.bats
Furthermore, appending a `; sleep 1` to the command line for those
containers seemed to consistently get rid of the error.
Investigating further, I've uncovered a race between the end of the container
process and the setting up of the cgroup watchers (to report OOMs).
If the process terminates first, the agent will try to watch cgroup
paths that don't exist anymore, and it will fail to start the container.
The added error context in notifier.rs confirms that the error comes
from the missing cgroup:
https://github.com/kata-containers/kata-containers/actions/runs/13450787436/job/37585901466#step:17:6536
The fix simply consists in creating the watchers *before* we start the
container but still *after* we create it -- this is non-blocking, and IIUC the
cgroup is guaranteed to already be present then.
Fixes: #10872
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
I missed that when I added the other comments, so, for the sake of
consistency, let's just add it there as well.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
We never ever tested MEASURED_ROOTFS with initrd, and I sincerely do not
know why we've been setting that to "yes" in the initrd cases.
Let's drop it, as it may be causing issues with the jobs that rely on
the rootfs-initrd-confidential.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
For each IOMMUFD device create an object and assign
it to the device, we need additional information that
is populated now correctly to decide if we run the old VFIO
or new VFIO backend.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
As a follow-up for #10904, we do not need to set MEASURED_ROOTFS to no
on s390x explicitly. The GHA workflow already exports this variable.
This commit removes the redundant assignment.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This test verifies that, when ReadStreamRequest is blocked by the
policy, the logs are empty and the container does not deadlock.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This is the first attempt to remove the following code:
```
if [ "${ARCH}" == "s390x" ]; then
export MEASURED_ROOTFS=no
fi
```
from install_shimv2() in kata-deploy-binaries.sh.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
On Ubuntu 24.04, due to the /usr merge, system-provided unit files
now reside in `/usr/lib/systemd/system/` instead of `/lib/systemd/system/`.
For example, the command below now returns a different path:
```
$ systemctl show containerd.service -p FragmentPath
/usr/lib/systemd/system/containerd.service
```
Previously, on Ubuntu 22.04 and earlier, it returned:
```
/lib/systemd/system/containerd.service
```
The current pattern `if [[ $unit_file == /lib* ]]` fails to match the new path.
To ensure compatibility across versions, we update the pattern to match both
`/lib` and `/usr/lib` like:
```
if [[ $unit_file =~ ^/(usr/)?lib/ ]]
```
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Centralize all RustVMM crates to workspace.dependencies to prevent
having multiple versions of each RustVMM crate, which is error-prone and
inconsistent. With this setup, updates on RustVMM crates would be much
easier.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Setup workspace in dragonball, move `dbs` crates one level up to be
managed as members of dragonball workspace.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Add entries for dbs_* crates' README.md to pass `kata-spell-check.sh`
spell checking.
Changed British terms to American terms in README of `dbs_pci` to pass
`hunspell` check.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
sudo hides the environment variables that are sometimes
useful with the builds (for example: proxy settings).
While install_oras.sh could run completely without sudo in
the container it's COPY'd to, make minimal changes to it
to keep it functional outside the container too while still
addressing the problem of 'sudo curl' not working with proxy
env variables.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
To mitigate:
warning: `.../kata-containers/src/agent/.cargo/config` is deprecated in favor of `config.toml`
note: if you need to support cargo 1.38 or earlier, you can symlink `config` to `config.toml`
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
This picks up a security fix for confidential pulling of unsigned
images.
The crate moved permanently to oci-client, which required a few import
changes.
Co-authored-by: Paul Meyer <katexochen0@gmail.com>
Signed-off-by: Markus Rudy <mr@edgeless.systems>
We are running `header_check` for non-text files like binary files,
symbolic link files, image files (pictures) and etc., which does not
make sense.
Filter out non-text files and run `header_check` only for text files
changed.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
This was messed up a little when factoring out the policy crate.
Removing the dependencies no longer used by the agent and making the
import of kata-agent-policy optional again.
Signed-off-by: Paul Meyer <katexochen0@gmail.com>
An image `registry.k8s.io/hpa-example` only supports amd64.
Let's use a multi-arch image `quay.io/prometheus/prometheus`
for the QEMU example instead.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
`kata-dictionary.dic` changes after running `kata-spell-check.sh
make-dict`. This is due to someone forgot to first update entries in
data and run `make-dict`, but directly updated `kata-dictionary.dic`
instead.
Add mssing entries to data and re-run `make-dict` to generate correct
`kata-dictionary.dic`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
previously we used to deploy the webhook and then modified the cm from
our ci/openshift-ci/ script to the desired value, but sometimes it
happens that the webhook pod starts before we modify the cm and keeps
using the default value.
Let's change the approach and modify the deployments in-place. The only
cons is it leaves the git dirty, but since this script is only supposed
to be used in ci it should be safe.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
So attestation-agent and others have a version including the ttrpc bump
to v0.8.4, allowing us to use the latest LTS kernel.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
We've been appending to the wrong variable for quite some time, it
seems, leading to not actually regenerating the rootfs when needed.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Set CONFIG_BLK_DEV_WRITE_MOUNTED=y to restore previous kernel behaviour.
Kernel v6.8+ will by default block buffer writes to block devices mounted by filesystems.
This unfortunately is what we need to use mounted loop devices needed by some teams
to build OSIs and as an overlay backing store.
More info on this config item [here](https://cateee.net/lkddb/web-lkddb/BLK_DEV_WRITE_MOUNTED.html)
Fixes: #10808
Signed-off-by: Simon Kaegi <simon.kaegi@gmail.com>
Run:
```
cargo update -p cookie-store
cargo update -p publicsuffix
```
to update the version of idna and resolve CVE-2024-12224
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Removed a rogue printf and updated the logging to say
that we're waiting for CDI spec(s) to be generated rather
than saying there is an error, it's not we have a timeout
after that it is an error.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
With the create_container_timeout the dial_timeout is lest important.
Add the custom timeout for GPUs in create_container_timeout
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The tags created automatically for published Github releases
are probably not annotated, so by simply running `git describe` we are
not getting the correct tag. Use a `git describe --tags` to allow git
to look at all tags, not just annotated ones.
Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
AgentConfig now has the cdi_timeout from the kernel
cmdline, update the proper function signature and use
it in the for loop.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Some systems like a DGX where we have 8 H100 or 8 H800 GPUs
need some extended time to be initialized. We need to make
sure we can configure CDI timeout, to enable even systems with 16 GPUs.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Phase 1 of Issue #10840
AMD has deprecated SEV support on
Kata Containers, and going forward,
SNP will be the only AMD feature
supported. As a first step in this
deprecation process, we are removing
the SEV CI workflow from the test suite
to unblock the CI.
Will be adding future commits to
remove redundant SEV code paths.
Signed-Off-By: Adithya Krishnan Kannan <AdithyaKrishnan.Kannan@amd.com>
The block volume test has failed on 10/10 nightlies
and all the PRs I've seen, so skip it until it can be assessed.
See #10873
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Based on the guidance from @Xynnn007 in #10851
> The new version of image-rs will do attestation once
ClientBuilder.build().await() is called, while the old version
will do so lazily the first image pull request comes.
Looks like it's called in rpc::start() in kata-agent, when
I'm afraid the network hasn't been initialized yet.
> I am not sure if the guest network is prepared after
the DNS is configured (in create_sandbox),
if so we can move (the init_image_service) right after that.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
As this brings in the commit bumping ttrpc to 0.8.4, which fixes
connection issues with kernel 6.12.9+.
As image-rs has a new builder pattern and several of the values in the
image client config have been renamed, let's change the agent to account
for this.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@linux.ibm.com>
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
cgroups v2 enforces stricter delegation rules, preventing operations on
cgroups outside our ownership boundary. When running Docker-in-Docker (DinD),
processes must be attached to an "init" subcgroup within the systemd unit.
This fix detects and uses the init subcgroup when proxying process attachment.
Fixes#10733
Signed-off-by: Antoine Gaillard <antoine.gaillard@datadoghq.com>
When trying to deploy nydus on kcli locally we get the
following failure:
```
root@sh-kata-ci1:~# kubectl get pods -n nydus-system
NAMESPACE NAME READY STATUS RESTARTS AGE
nydus-system nydus-snapshotter-5kdqs 0/1 CrashLoopBackOff 4 (84s ago) 7m29s
```
Digging into this I found that the nydus-snapshotter service
is failing with:
```
ubuntu@kata-k8s-worker-0:~$ journalctl -u nydus-snapshotter.service
-- Logs begin at Wed 2025-02-12 15:06:08 UTC, end at Wed 2025-02-12 15:20:27 UTC. --
Feb 12 15:10:39 kata-k8s-worker-0 systemd[1]: Started nydus snapshotter.
Feb 12 15:10:39 kata-k8s-worker-0 containerd-nydus-grpc[6349]: /usr/local/bin/containerd-nydus-grpc:
/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required b>
Feb 12 15:10:39 kata-k8s-worker-0 containerd-nydus-grpc[6349]: /usr/local/bin/containerd-nydus-grpc:
/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required b>
Feb 12 15:10:39 kata-k8s-worker-0 systemd[1]: nydus-snapshotter.service: Main process exited, code=exited, status=1/FAILURE
```
I think this is because 20.04 has version:
```
ubuntu@kata-k8s-worker-0:~$ ldd --version
ldd (Ubuntu GLIBC 2.31-0ubuntu9.16) 2.31
```
so it's too old for the nydus snapshotter.
Also 20.04 is EoL soon, so bumping is better.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Some problem hidden in `dbs` crates are revealed after making these
crates workspace components, fix according to `cargo clippy` suggests.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Peer pods have a linux namespace of type network. We want to make sure that all
container in the same pod use the same namespace. Therefore, we add the first
namespace path to the state and check all other requests against that.
This commit also adds the corresponding integration test in the policy crate
showcasing the benefit of having rust integration tests for the policy.
Signed-off-by: Leonard Cohnen <lc@edgeless.systems>
The generated rego policies for `CreateContainerRequest` are stateful and that
state is handled in the policy crate. We use this policy crate in the
genpolicy integration test to be able to test if those state changes are
handled correctly without spinning up an agent or even a cluster.
This also allows to easily test on a e.g., CreateContainerRequest level
instead of relying on changing the yaml that is applied to a cluster.
Signed-off-by: Leonard Cohnen <lc@edgeless.systems>
This commit allows to programmatically invoke genpolicy. This allows for other
rust tools that don't want to consume genpolicy as binary to generate policies.
One such use-case is the policy integration test implemented in the following
commits.
Signed-off-by: Leonard Cohnen <lc@edgeless.systems>
The policy module augments the policy generated with genpolicy by keeping and
providing state to each invocation.
Therefore, it is not sufficient anymore to test the passing of requests in
the genpolicy crate.
Since in Rust, integration tests cannot call functions that are not exposed
publicly, this commit factors out the policy module of the agent into its
own crate and exposes the necessary functions to be consumed by the agent
and an integration tests. The integration test itself is implemented in the
following commits.
Signed-off-by: Leonard Cohnen <lc@edgeless.systems>
Since the CPU SNP is upstreamed and available via our
default QEMU target we're repurposing the SNP-experimental
for the GPU+SNP enablement.
First step is to update the version we're basing it off.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
the latest containerd had an issue for its e2e test, thus we should do
the following fix to workaround this issue. For much info about this issue,
please see:
https://github.com/containerd/containerd/pull/11240
Once this pr was merged and release new version, we can remove
this workaround.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
A test case is added based on the intergrated cri-containerd case.
The difference between cri containerd integrated testcase and sandbox
api testcase is the "sandboxer" setting in the sandbox runtime handler.
If the "sandboxer" is set to "" or "podsandbox", then containerd will
use the legacy shimv2 api, and if the "sandboxer" is set to "shim", then
it will use the sandbox api to launch the pod.
In addition, add a containerd v2.0.0 version. Because containerd officially
supports the sandbox api from version 2.0.0.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
add and resiger the sandbox api service, thus runtime-rs
can deal with the sandbox api rpc call from the containerd.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
For Kata-Containers, we add SandboxService for these new calls alongside
the existing
TaskService, including processing requests and replies, and properly
calling
VirtSandbox's interfaces. By splitting the start logic of the sandbox,
virt_container
is compatible with calls from the SandboxService and TaskService. In
addition, we modify
the processing of resource configuration to solve the problem that
SandboxService does not
have a spec file when creating a pod.
Sandbox api can be supported from containerd 1.7. But there's a
difference from container 2.0.
To enbale it from 2.0, you can support the sandbox api for a specific
runtime by adding:
sandboxer = "shim", take kata runtime as an example:
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata]
runtime_type = "io.containerd.kata.v2"
sandboxer = "shim"
privileged_without_host_devices = true
pod_annotations = ["io.katacontainers.*"]
For container version 1.7, you can enable it by:
1: add env ENABLE_CRI_SANDBOXES=true
2: add sandbox_mode = "shim" to runtime config.
Acknowledgement
This work was based on @wllenyj's POC code:
(f5b62a2d7c)
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
For the processing of init sandbox, the init of task
api has some more special processing procedures than
the init of sandbox api, so these two types of init
are separated here.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
When start the sandbox, the sandbox id would be passed from the
shim command line, and it only need to get the containerd id from
oci spec when starting the pod container instead of the pod sandbox.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
When the sandbox api was enabled, the pause container would
be removed and sandbox start api only pass an empty bundle
directory, which means there's no oci spec file under it, thus
the cgroup config couldn't get the cgroup path from pause container's
oci spec. So we should set a default cgroup path for sandbox api
case.
In the future, we can promote containerd to pass the cgroup path during
the sandbox start phase.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Remove block_device_cache_direct from config of fc in runtime-rs because
fc doesn't support this config.
Fixes: #10849
Signed-off-by: Hui Zhu <teawater@antgroup.com>
Add block_device_cache_direct to config of ch and dragonball in
runtime-rs because they support this config.
Fixes: #10849
Signed-off-by: Hui Zhu <teawater@antgroup.com>
This commit change config in CloudHypervisorInner to normal
HypervisorConfig to decrease the change of its type.
Fixes: #10849
Signed-off-by: Hui Zhu <teawater@antgroup.com>
Otherwise we may end up simply unpacking kata-containers specific
binaries into the same location that system ones are needed, leading to
a broken system (most likely what happened with the metrics CI, and also
what's happening with the GHA runners).
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
We've been hitting issues with the CentOS 9 Stream machine, which Intel
doesn't have cycles to debug.
After raising this up in the Confidential Containers community meeting
we got the green light from Red Hat (Ariel Adam) to just disable the CI
based on CentOS 9 Stream for now.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
IBM SE ensures to make initrd measured by genprotimg and verified by ultravisor.
Let's not build the measured rootf on s390x.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This is needed in order to avoid agent build issues, such as:
```
error[E0658]: use of unstable library feature 'lazy_cell'
--> /home/ansible/.cargo/git/checkouts/guest-components-1e54b222ad8d9630/514c561/ocicrypt-rs/src/lib.rs:10:5
|
10 | use std::sync::LazyLock;
| ^^^^^^^^^^^^^^^^^^^
|
= note: see issue #109736 <https://github.com/rust-lang/rust/issues/109736> for more information
```
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
As we want to make sure a new builder image is generated if the rust
version is bumped.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Due to the way that multi-arch support is done, on various platforms
we will get a clippy error:
```
error: this expression always evaluates to false
```
which might not be true on those other platforms, so
allow this code pattern to suppress the clippy error
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
k8s-policy-job is modeled after the older k8s-job, and it appears
that both of them fail occasionally on coco-dev.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Preparing to install nydus permanently on the AMD node,
so disabling deploy and delete command for SNP and SEV.
Signed-off-by: Arvind Kumar <arvinkum@amd.com>
We introduced extratarballs with a make target. The CI
currently only uploads tarballs that are listed in the matrix.
The NV kernel builds a headers package which needs to be uploaded
as well.
The get-artifacts has a glob to download all artifacts hence we
should be good.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
So it avoids us hitting
```
error[E0282]: type annotations needed for `Box<_>`
--> /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/time-0.3.31/src/format_description/parse/mod.rs:83:9
|
83 | let items = format_items
| ^^^^^
...
86 | Ok(items.into())
| ---- type must be known at this point
|
help: consider giving `items` an explicit type, where the placeholders `_` are specified
|
83 | let items: Box<_> = format_items
| ++++++++
```
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
On powerpc64le platform the ip neigh command has
a trailing space after the state, so the test is failing e.g.
```
assertion `left == right` failed
left: "169.254.1.1 lladdr 6a:92:3a:59:70:aa PERMANENT \n"
right: "169.254.1.1 lladdr 6a:92:3a:59:70:aa PERMANENT\n"
```
Trim the whitespace to make the test pass on all platforms
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
`host_is_vmcontainer_capable` is required, but wasn't
implemented for powerpc64, so copy the aarch64 approach
@Amulyam24
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
In #7236 the guest protection code was moved to kata-sys-utils,
but some of it was left behind, and the adjustment to the new
location wasn't completed, so the powerpc64 code doesn't
build now we've fixed the cfg to test it.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Some of the Kernel structs have `#[allow(dead_code)]`
but not all and this results in the clippy error:
```
error: fields `name` and `value` are never read
```
so complete the job started before to remove the error.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Clippy errors with:
```
error: field `driver` is never read
--> crates/resource/src/network/utils/link/driver_info.rs:77:9
|
76 | pub struct DriverInfo {
| ---------- field in this struct
77 | pub driver: String,
| ^^^^^^
```
We set this, but never read it, so clippy is correct,
but I'm not sure if it's useful for logging, or other purposes,
so I'll allow it for now.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Clippy fails with:
```
warning: unexpected `cfg` condition value: `test-mock`
--> /root/go/src/github.com/kata-containers/kata-containers/src/dragonball/src/dbs_pci/src/vfio.rs:1929:17
|
1929 | #[cfg(all(test, feature = "test-mock"))]
| ^^^^^^^^^^^^^^^^^^^^^ help: remove the condition
|
= note: no expected values for `feature`
= help: consider adding `test-mock` as a feature in `Cargo.toml`
```
So add it as an expected cfg in the linter to skip this
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Clippy fails with:
```
error: unexpected `cfg` condition value: `enable-vendor`
--> crates/hypervisor/src/device/driver/vfio.rs:180:11
|
180 | #[cfg(feature = "enable-vendor")]
| ^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: expected values for `feature` are: `ch-config`, `cloud-hypervisor`, `default`, and `dragonball`
= help: consider adding `enable-vendor` as a feature in `Cargo.toml`
```
So add it as an expected cfg in the linter to skip this
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Clippy complains about:
```
error: for loop over a `&Result`. This is more readably written as an `if let` statement
--> crates/hypervisor/src/firecracker/fc_api.rs:99:22
|
99 | for param in &kernel_params.to_string() {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
```
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Fix clippy error:
```
direct implementation of `ToString`
```
by switching to implement Display instead
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Clippy errors with:
```
error: field `0` is never read
--> crates/hypervisor/src/qemu/cmdline_generator.rs:375:25
|
375 | DeviceAlreadyExists(String), // Error when trying to add an existing device
| ------------------- ^^^^^^
```
but this is used when creating the error later, so add an allow
to ignore this warning
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Fix clippy error
```
error: usage of a legacy numeric constant
```
by swapping `std::u8::MAX` for `u8::MAX`
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Clippy errors with:
```
error: field `0` is never read
```
but the field is required for the `map_err`, so ignore this
error for now to avoid too much disruption
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
There were references to `config_manager::DeviceInfoGroup`
which doesn't exist, so I guess it means `DeviceConfigInfo`
instead, so update them
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Clippy errors with:
```
error: doc list item missing indentation
```
which I think is because the Return is between two list
items, so add a blank line to separate this into a separate
paragraph
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
clippy errors with:
```
error: initializer for `thread_local` value can be made `const`
```
so update as suggested
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Fix clippy error:
```
direct implementation of `ToString`
```
by switching to implement Display instead
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Fix clippy error
```
error: usage of a legacy numeric constant
```
by swapping `std::i32::<MIN/MAX>` for `i32::<MIN/MAX>`
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
```
error: file opened with `create`, but `truncate` behavior not defined
```
`truncate(true)` ensures the file is entirely overwritten with new data
which I believe is the behaviour we want
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
```
error: bound is defined in more than one place
```
Move Sized into the later definition of `R` & `W`
rather than defining them in two places
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
```
error: file opened with `create`, but `truncate` behavior not defined
```
`truncate(true)` ensures the file is entirely overwritten with new data
which I believe is the behaviour we want
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
```
error: field `image` is never read
--> src/registry.rs:35:9
|
34 | pub struct Container {
| --------- field in this struct
35 | pub image: String,
| ^^^^^
|
= note: `Container` has derived impls for the traits `Debug` and `Clone`, but these are intentionally ignored during dead code analysis
= note: `-D dead-code` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(dead_code)]`
error: field `use_cache` is never read
--> src/utils.rs:106:9
|
105 | pub struct Config {
| ------ field in this struct
106 | pub use_cache: bool,
| ^^^^^^^^^
|
= note: `Config` has derived impls for the traits `Debug` and `Clone`, but these are intentionally ignored during dead code analysis
error: could not compile `genpolicy` (bin "genpolicy") due to 2 previous errors
```
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Based on comments from @Amulyam24 we need to use
the `target_endian = "little"` as well as target_arch = "powerpc64"
to ensure we are working on powerpc64le.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Starting with version 1.80, the Rust linter does not accept an invalid
value for `target_arch` in configuration checks:
```
Compiling kata-sys-util v0.1.0 (/home/ddd/Work/kata/kata-containers/src/libs/kata-sys-util)
error: unexpected `cfg` condition value: `powerpc64le`
--> /home/ddd/Work/kata/kata-containers/src/libs/kata-sys-util/src/protection.rs:17:34
|
17 | #[cfg(any(target_arch = "s390x", target_arch = "powerpc64le"))]
| ^^^^^^^^^^^^^^-------------
| |
| help: there is a expected value with a similar name: `"powerpc64"`
|
= note: expected values for `target_arch` are: `aarch64`, `arm`, `arm64ec`, `avr`, `bpf`, `csky`, `hexagon`, `loongarch64`, `m68k`, `mips`, `mips32r6`, `mips64`, `mips64r6`, `msp430`, `nvptx64`, `powerpc`, `powerpc64`, `riscv32`, `riscv64`, `s390x`, `sparc`, `sparc64`, `wasm32`, `wasm64`, `x86`, and `x86_64`
= note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration
= note: `-D unexpected-cfgs` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(unexpected_cfgs)]`
```
According [to GitHub user @Urgau][explain], this is a new warning
introduced in Rust 1.80, but the problem exists before. The correct
architecture name should be `powerpc64`, and the differentiation
between `powerpc64le` and `powerpc64` should use the `target_endian =
"little"` check.
[explain]: #10072 (comment)
Fixes: #10067
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
[emlima: fix some more occurences and typos]
Signed-off-by: Emanuel Lima <emlima@redhat.com>
[stevenhorsman: fix some more occurences and typos]
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Add aarch64 and x86_64 handling. Especially build the Rust
dependency with the correct rust musl target.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Container logs are forwarded to the agent through a unix pipe. These
pipes have limited capacity and block the writer when full. If reading
logs is blocked by policy, a common setup for confidential containers,
the pipes fill up and eventually block the container.
This commit changes the implementation of ReadStream such that it
returns empty log messages instead of a policy failure (in case reading
log messages is forbidden by policy). As long as the runtime does not
encounter a failure, it keeps pulling logs periodically. In turn, this
triggers the agent to flush the pipes.
Fixes: #10680
Co-Authored-By: Aurélien Bombo <abombo@microsoft.com>
Signed-off-by: Markus Rudy <mr@edgeless.systems>
In v4.44.5 of `yq`, artifacts for riscv64 are released. Update the
version used for `yq` and enable `install_yq.sh` to work on riscv64.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
The static_checks_versions test uses yamllint which fails with:
```
[comments] too few spaces before comment
```
many times and so makes code reviews more annoying with
all these extra messages. Other it's probably not the worse issues,
I checked the
[yaml spec](https://yaml.org/spec/1.2.2/#66-comments)
and it does say
> Comments must be separated from other tokens by white space character*s*
so it's easiest to fix it and move on.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
I've also seen cases (the qemu, crio, k0s tests) where Delete kata-deploy is still
running for this test after 2 hours, and had to be manually
cancelled, so let's try adding a 5m timeout to the kata-deploy delete to stop CI jobs hanging.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
#10714 added support for building a specific commit,
but due to the clone only having `--depth=1`, we can only
reset to a commit if it's the latest on the `main` branch,
otherwise we will get:
```
+ git clone --depth 1 --branch main https://gitlab.com/virtio-fs/virtiofsd virtiofsd
Cloning into 'virtiofsd'...
warning: redirecting to https://gitlab.com/virtio-fs/virtiofsd.git/
+ pushd virtiofsd
+ git reset --hard cecc61bca981ab42aae6ec490dfd59965e79025e
...
fatal: Could not parse object 'cecc61bca981ab42aae6ec490dfd59965e79025e'.
```
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Referenced AMD developer page for latest SEV firmware.
Instructions to point to upstream 6.11 kernel or later.
Referenced sev-utils and AMDESE fork for kernel setup.
Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
use upstream qemu in snp and nvidia snp configs.
load ovmf with bios flag on qemu cmdline instead of file.
Fixes: #10750
Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
snp standard attestation with the upstream kernel and qemu do not support extended attestation with certs.
Fixes: #10750
Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
Previously, the test for VFIO-AP coldplug only checked whether a
passthrough device was attached to the VM guest. This commit expands
the test to include a full set of zcrypttest to verify that the device
functions properly within a container.
Additionally, since containerd has been upgraded to v1.7.25 on the
test machine, it is no longer necessary to run the test via crictl.
The commit removes all related codes/files.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This commit updates the device handler to call check_ap_device()
instead of wait_for_ap_device() for VFIO-AP coldplug.
The handler now returns a SpecUpdate for passthrough devices if
the device is online (e.g., `/sys/devices/ap/card05/05.001f/online`
is set to 1).
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This commit introduces a new gRPC device type, `vfio-ap-cold`, to support
VFIO-AP coldplug. This enables the VM guest to handle passthrough devices
differently from VFIO-AP hotplug.
With this new type, the guest no longer needs to wait for events (e.g., device
addition) because the device already exists at the time the device type is checked.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Even though ociSpec.Linux.Devices is preserved when vfio_mode is VFIO,
it has not been updated correctly for coldplug scenarios. This happens
because the device info passed to the agent via CreateContainerRequest
is dropped by the Kata runtime.
This commit ensures that the device info is added to the sandbox's
device manager when vfio_mode is VFIO and coldPlugVFIO is true
(e.g., vfio-ap-cold), allowing ociSpec.Linux.Devices to be properly
updated with the device information before the container is created on
the guest.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Since we're importing some build script for nvidia and we're
setting set -u we have some unbound variables in rootfs.sh
add initialization for those.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
stack-only types are handled properly with the
parse_cmdline_param macro advancted types like
String couldn't be guarded by a guard function since
it passed the variable by value rather than reference.
Now we can have guard functions for the String type
parse_cmdline_param!(
param,
CGROUP_NO_V1,
config.cgroup_no_v1,
get_string_value,
| no_v1 | no_v1 == "all"
);
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
For AGENT_INIT=yes we do not run systemd and hence
systemd.unified_... does not mean anything to other init
systems. Providing cgroup_no_v1=all is enough to signal
other init systemd to use cgroupV2.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Since we're defaulting to AGENT_INIT=no for all the initrd/images
adapt the NV build to properly get kata-agent installed.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
While I wish we could be bumping to the very same version everywhere,
it's not possible and it's been quite a ride to get a combination of
things that work.
Let me try to describe my approach here:
* Do *NOT* stay on 20.04
* This version will be EOL'ed by April
* This version has a very old version of systemd that causes a bug
when trying to online the cpusets for guests using systemd as
init, causing then a breakage on the qemu-coco-non-tee and TDX
non-attestation set of tests
* Bump to 22.04 when possible
* This was possible for the majority of the cases, but for the
confidential initrd & confidential images for x86_64, the reason
being failures on AMD SEV CI (which I didn't debug), and a kernel
panic on the CentOS 9 Stream TDX machine
* 22.04 is being used instead of 24.04 as multistrap is simply broken
on Ubuntu 24.04, and I'd prefer to stay on an LTS release whenever
it's possible
* Bump to 24.10 for x86_64 image confidential
* This was done as we got everything working with 24.10 in the CI.
* This requires using libtdx-attest from noble (Ubuntu 24.04), as
Intel only releases their sgx stuff for LTS releases.
* Stick to 20.04 for x86_64 initrd confidential
* 24.10 caused a panic on their CI
* This is only being used by AMD so far, so they can decide when to
bump, after doing the proper testing & debug that the bump will work
as expected for them
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
We first try without passing the `--break-system-packages` argument, as
that's not supported on Ubuntu 22.04 or older, but that's required on
Ubuntu 24.04 or newer.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Otherwise a bump in the os name and / or os version would lead to the CI
using a cached artefact.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
We have gotten Ubuntu 20.04 working pretty much "by luck", as multistrap
fails the deployment, and then a hacky function was introduced to add
the proper dbus links. However, this does not scale at all, and we
should:
* Fail if multistrap fails
* I won't do this for Ubuntu 20.04 as it's working for now and soon
enough it'll be EOL
* Add better logging to ensure someone can know when multistrap fails
Below you can find the failure that we're hitting on Ubuntu 20.04:
```sh
Errors were encountered while processing:
dbus
ERR: dpkg configure reported an error.
Native mode configuration reported an error!
I: Tidying up apt cache and list data.
Multistrap system reported 1 error in /rootfs/.
I: Tidying up apt cache and list data.
```
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Right now we're hitting an interesting situation with osbuilder, where
regardless of what's being passed Ubuntu 20.04 (focal) is being used
when building the rootfs-image, as shown in the snippets of the logs
below:
```
ffidenci@tatu:~/src/upstream/kata-containers/kata-containers$ make rootfs-image-confidential-tarball
/home/ffidenci/src/upstream/kata-containers/kata-containers/tools/packaging/kata-deploy/local-build//kata-deploy-copy-libseccomp-installer.sh "agent"
make agent-tarball-build
...
make pause-image-tarball-build
...
make coco-guest-components-tarball-build
...
make kernel-confidential-tarball-build
...
make rootfs-image-confidential-tarball-build
make[1]: Entering directory '/home/ffidenci/src/upstream/kata-containers/kata-containers'
/home/ffidenci/src/upstream/kata-containers/kata-containers/tools/packaging/kata-deploy/local-build//kata-deploy-binaries-in-docker.sh --build=rootfs-image-confidential
sha256:f16c57890b0e85f6e1bbe1957926822495063bc6082a83e6ab7f7f13cabeeb93
Build kata version 3.13.0: rootfs-image-confidential
INFO: DESTDIR /home/ffidenci/src/upstream/kata-containers/kata-containers/tools/packaging/kata-deploy/local-build/build/rootfs-image-confidential/destdir
INFO: Create image
build image
~/src/upstream/kata-containers/kata-containers/tools/osbuilder ~/src/upstream/kata-containers/kata-containers/tools/packaging/kata-deploy/local-build/build/rootfs-image-confidential/builddir
INFO: Build image
INFO: image os: ubuntu
INFO: image os version: latest
Creating rootfs for ubuntu
/home/ffidenci/src/upstream/kata-containers/kata-containers/tools/osbuilder/rootfs-builder/rootfs.sh -o 3.13.0-13f0807e9f5687d8e5e9a0f4a0a8bb57ca50d00c-dirty -r /home/ffidenci/src/upstream/kata-containers/kata-containers/tools/packaging/kata-deploy/local-build/build/rootfs-image-confidential/builddir/rootfs-image/ubuntu_rootfs ubuntu
INFO: rootfs_lib.sh file found. Loading content
~/src/upstream/kata-containers/kata-containers/tools/osbuilder/rootfs-builder/ubuntu ~/src/upstream/kata-containers/kata-containers/tools/osbuilder
~/src/upstream/kata-containers/kata-containers/tools/osbuilder
INFO: rootfs_lib.sh file found. Loading content
INFO: build directly
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB]
Get:2 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB]
Get:4 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [4276 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB]
Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [11.3 MB]
Get:7 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1297 kB]
Get:8 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [30.9 kB]
Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [4187 kB]
Get:10 http://archive.ubuntu.com/ubuntu focal/restricted amd64 Packages [33.4 kB]
Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages [1275 kB]
Get:12 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages [177 kB]
Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [4663 kB]
Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1589 kB]
Get:15 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [34.6 kB]
Get:16 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [4463 kB]
Get:17 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [55.2 kB]
Get:18 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [28.6 kB]
Fetched 34.1 MB in 5s (6284 kB/s)
...
```
The reason this is happening is due to a few issues in different places:
1. IMG_OS_VERSION, passed to osbuilder, is not used anywhere and
OS_VERSION should be used instead. And we should break if OS_VERSION
is not properly passed down
2. Using UBUNTU_CODENAME is simply wrong, as it'll use whatever comes as
the base container from kata-deploy's local-build scripts, and it has
just been working by luck
Note that at the same time this commit fixes the wrong behaviour, it
would break the rootfses build as they are, this we need to set the
versions.yaml to use 20.04 were it was already using 20.04 even without
us knowing.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
As this is required as part of the osbuilder tool to be able to properly
set the repositories used when building the rootfs.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
While having variables are nice, those are more extensive to write down,
and actually confusing for tired developer eyes to read, plus we're
mixing the use of the yaml variables here and there together with not
using them for some architectures.
With the best "all or nothing" spirit, let's just make it easier for our
developers to read the versions.yaml and easily understand what's being
used.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
As the devices controller works in a different way in cgroupsv2, the
"/sys/fs/cgroup/devices/devices.list" file simply doesn't exist.
For now, let's skip the test till the test maintainer decides to
re-enable it for cgroupsv2.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
The changes done are:
* cpu/cpu.shares was replaced by cpu.weight
* The weight, according to our reference[0], is calculated by:
weight = (1 + ((request - 2) * 9999) / 262142)
* cpu/cpu.cfs_quota_us & cpu/cpu.cfs_period_us were replaced by cpu.max,
where quota and period are written together (in this order)
[0]: https://github.com/containers/crun/blob/main/crun.1.md#cgroup-v2
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This reverts commit 091ad2a1b2, in order
to ensure tests would be running with cgroupsv2 on the guest.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
In the last couple of days I've seen the blogbench
metrics write latency test on clh fail a few times because
the latency was too low, so adjust the minimum range
to tolerate quicker finishes.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The static-checks targets are `pull_request`, so
they can run the PR workflow version, so we want to
update the required-tests.yaml so that static-check
workflow changes do trigger static checks in order
to test them properly.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Now we have the build-assets running on the gh-hosted
runners, try the same approach for the static-checks
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
I've noticed the following error when running the tests with SEV:
```
2025-01-21T17:10:28.7999896Z # @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2025-01-21T17:10:28.8000614Z # @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
2025-01-21T17:10:28.8001217Z # @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2025-01-21T17:10:28.8001857Z # IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
2025-01-21T17:10:28.8003009Z # Someone could be eavesdropping on you right now (man-in-the-middle attack)!
2025-01-21T17:10:28.8003348Z # It is also possible that a host key has just been changed.
2025-01-21T17:10:28.8004422Z # The fingerprint for the ED25519 key sent by the remote host is
2025-01-21T17:10:28.8005019Z # SHA256:x7wF8zI+LLyiwphzmUhqY12lrGY4gs5qNCD81f1Cn1E.
2025-01-21T17:10:28.8005459Z # Please contact your system administrator.
2025-01-21T17:10:28.8006734Z # Add correct host key in /home/kata/.ssh/known_hosts to get rid of this message.
2025-01-21T17:10:28.8007031Z # Offending ED25519 key in /home/kata/.ssh/known_hosts:178
2025-01-21T17:10:28.8007254Z # remove with:
2025-01-21T17:10:28.8008172Z # ssh-keygen -f "/home/kata/.ssh/known_hosts" -R "10.244.0.71"
```
And this was causing a failure to ssh into the confidential pod.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Relying on dmesg is really not ideal, as we may lose important info,
mainly those which happen very early in the boot, depending on the size
of kernel ring buffer.
So, for this specific test, let's increase the kernel ring buffer, by
default, to 4M.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Let's make sure that we don't use Kata Containers' agent as init for the
Confidential related rootfses*, as we don't want to increase the agent's
complexity for no reason ... mainly when we can rely on a proper init
system.
*:
- images already used systemd as init
- initrds are now using systemd as init
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Bumps the go_modules group with 1 update in the /src/runtime directory: [golang.org/x/net](https://github.com/golang/net).
Bumps the go_modules group with 1 update in the /src/tools/csi-kata-directvolume directory: [golang.org/x/net](https://github.com/golang/net).
Bumps the go_modules group with 1 update in the /tools/testing/kata-webhook directory: [golang.org/x/net](https://github.com/golang/net).
Updates `golang.org/x/net` from 0.25.0 to 0.33.0
- [Commits](https://github.com/golang/net/compare/v0.25.0...v0.33.0)
Updates `golang.org/x/net` from 0.23.0 to 0.33.0
- [Commits](https://github.com/golang/net/compare/v0.25.0...v0.33.0)
Updates `golang.org/x/net` from 0.23.0 to 0.33.0
- [Commits](https://github.com/golang/net/compare/v0.25.0...v0.33.0)
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: indirect
dependency-group: go_modules
- dependency-name: golang.org/x/net
dependency-type: direct:production
dependency-group: go_modules
- dependency-name: golang.org/x/net
dependency-type: indirect
dependency-group: go_modules
...
Signed-off-by: dependabot[bot] <support@github.com>
When the agent is run as the init process cgroupfs is being
setup. In the case of cgroupsV1 we needed to enable the memory hiearchy
this is now per default enabled in cgroupsV2. Additionally the file
/sys/fs/cgroup/memory/memory.use_hierarchy isn't even available with V2.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The `Create AKS cluster` step in `run-k8s-tests-on-aks.yaml` is likely
to fail fail since we are trying to issue `PUT` to `aks` in a relatively
high frequency, while the `aks` end has it's limit on `bucket-size` and
`refill-rate`, documented here [1].
Use `nick-fields/retry@v3` to retry in 10 seconds after request fail,
based on observations that AKS were request 7, or 8 second delays
before retry as part of their 429 response
[1] https://learn.microsoft.com/en-us/azure/aks/quotas-skus-regions#throttling-limits-on-aks-resource-provider-apisFixes: #10772
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
With this change, `virtiofsd` (gnu target) could be built and then to be
used with other components.
Depends: #10741Fixes: #10739
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
While working on #10559, I realized that some parts of the codebase use
$GH_PR_NUMBER, while other parts use $PR_NUMBER.
Notably, in that PR, since I used $GH_PR_NUMBER for CoCo non-TEE tests
without realizing that TEE tests use $PR_NUMBER, the tests on that PR
fail on TEEs:
https://github.com/kata-containers/kata-containers/actions/runs/12818127344/job/35744760351?pr=10559#step:10:45
...
44 error: error parsing STDIN: error converting YAML to JSON: yaml: line 90: mapping values are not allowed in this context
...
135 image: ghcr.io/kata-containers/csi-kata-directvolume:
...
So let's unify on $GH_PR_NUMBER so that this issue doesn't repro in the
future: I replaced all instances of PR_NUMBER with GH_PR_NUMBER.
Note that since some test scripts also refer to that variable, the CI
for this PR will fail (would have also happened with the converse
substitution), hence I'm not adding the ok-to-test label and we should
force-merge this after review.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
With ubuntu 20.04 image, virtiofsd gnu target couldn't be built due to
"unsupported ISA subset z" reported by "cc".
Updating to ubuntu 22.04 image addresses this problem.
Relates: #10739
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
With newer kernels we have a new backend for VFIO
called IOMMUFD this is a departure from VFIO IOMMU Groups
since it has only one device associated with an IOMMUFD entry.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The pyinstaller is located per default under /usr/local/bin
some prior versions were installing it to ${HOME}.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Skip logging empty lines of text from the Guest console output, if
there are any such lines.
Without this change, the Guest console log from CLH + /dev/pts/0 has
twice as many lines of text. Half of these lines are empty.
Fixes: #10737
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
On s390x, some tests for trusted storage occasionally failed due to:
```bash
etcdserver: request timed out
```
or
```bash
Internal error occurred: resource quota evaluation timed out
```
These timeouts were not observed previously on k3s but occur
sporadically on kubeadm. Importantly, they appear to be temporary
and transient, which means they can be ignored in most cases.
To address this, we introduced a new wrapper function, `retry_kubectl_apply()`,
for `kubectl create`. This function retries applying a given manifest up to 5
times if it fails due to a timeout. However, it will still catch and handle
any other errors during pod creation.
Fixes: #10651
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Move the deletion of unnecessary systemd units and files from
image_builder.sh into rootfs.sh.
The files being deleted can be applicable to other image file formats
too, not just to the rootfs-image format created by image_builder.sh.
Also, image_builder.sh was deleting these files *after* it calculated
the size of the rootfs files, thus missing out on the opportunity to
possibly create a smaller image file.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This update addresses an issue with token verification for SE and SNP
introduced in the last update by #10541.
Bumping the project to the latest commit resolves the issue.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The earlier implementation relied on using a specific mount-path prefix - `/sealed`
to determine that the referenced secret is a sealed secret.
However that was restrictive for certain use cases as it forced
the user to always use a specific mountpath naming convention.
This commit introduces an alternative implementation to relax the
restriction. A sealed secret can be mounted in any mount-path.
However it comes with a potential performance penality. The
implementation loops through all volume mounts and reads the file
to determine if it's a sealed secret or not.
Fixes: #10398
Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
We hit a failure with:
```
time="2025-01-09T09:55:58Z" level=warning msg="Failed Minval (0.017600 > 0.015000) for [network-iperf3]"
```
The range is very big, but in the last 3 test runs I reviewed we have got a minimum value of 0.015s
and a max value of 0.052, so there is a ~350% difference possible
so I think we need to have a wide range to make this stable.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Because az client restricts the name to be less than 64 characters. In
some cases (e.g. KATA_HYPERVISOR=qemu-runtime-rs) the generated name
will exceed the limit. This changed the function to shorten the name:
* SHA1 is computed from metadata then compound the cluster's name
* metadata as plain-text are passed as --tags
Fixes: #9850
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The bump to kernel 6.12 seems to have reduced the latency in
the metrics test, so increase the ranges for the minimal value,
to account for this.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Together with the bump, let's also bump the rust version needed to build
the package, with the caveat that virtiofsd doesn't actually use a
pinned version as part of their CI, so we're bumping to whatever is the
version on `alpine:rust` (which is used in their CI).
It's important to note that we're using a version which brings in one
extra patch apart from the release, as the next virtiofsd release will
happen at the end of February, 2025.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Right now we've been only building releases from virtiofsd, but we'll
need to pin a specific commit till v1.14.0 is out, thus let's add the
needed machinery to do so.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
The guest-components directory has been re-arranged slightly. Adjust the
installation path of the LUKS helper script to account for this.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Trustee has some new features including a plugin backend, support for
PKCS11 resources, improvements to token verification, and adjustments to
logging, and more.
Also update guest-components to pickup improvements and keep the KBS
protocol in sync.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Since we bumped to the 6.12.x LTS kernel, we've also adjusted the
aggressivity of the OOM test, which may be enough to allow us to
re-enable it for mariner.
Fixes: #8821
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Since from https://github.com/containerd/containerd/pull/9096
containerd removed cri-containerd-*.tar.gz release bundles,
thus we'd better change the tarball name to "containerd".
BTW, the containerd tarball containerd the follow files:
bin/
bin/containerd-shim
bin/ctr
bin/containerd-shim-runc-v1
bin/containerd-stress
bin/containerd
bin/containerd-shim-runc-v2
thus we should untar containerd into /usr/local directory instead of "/"
to keep align with the cri-containerd.
In addition, there's no containerd.service file,runc binary and cni-plugin
included, thus we should add a specific containerd.service file and
install install the runc binary and cni-pluginspecifically.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Got following issue:
kata-ctl direct-volume add /kubelet/kata-direct-vol-002/directvol002
"{\"device\": \"/home/t4/teawater/coco/t.img\", \"volume-type\":
\"directvol\", \"fstype\": \"\", \"metadata\":"{}", \"options\": []}"
subsystem: kata-ctl_main
Dec 30 09:43:41.150 ERRO Os {
code: 2,
kind: NotFound,
message: "No such file or directory",
}
The reason is KATA_DIRECT_VOLUME_ROOT_PATH is not exist.
This commit create_dir_all KATA_DIRECT_VOLUME_ROOT_PATH before join_path
to handle this issue.
Fixes: #10695
Signed-off-by: Hui Zhu <teawater@antgroup.com>
Unsurprisingly now we've got passed the containerd test
hangs on the ppc64le, we are hitting others in the "Prepare the
self-hosted runner" stage, so add timeouts to all of them
to avoid CI blockages.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Add information about what required jobs are and
our initial guidelines for how jobs are eligible for being
made required, or non-required
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
`set_hypervisor_config()` and `set_passfd_listener_port()` acquire inner
lock, so that `mut` for `hypervisor` is unneeded.
Fixes: #10682
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Fix the logic to make the test skipped on qemu-coco-dev,
rather than the opposite and update the syntax to make it
clearer as it incorrectly got written and reviewed by three
different people in it's prior form.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
In order to check that we don't accidentally overwrite
release artifacts, we should add a check if the release
name already exists and bail if it does.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Building static binaries for aarch64 requires disabling PIE
We get an GOT overflow and the OS libraries are only build with fpic
and not with fPIC which enables unlimited sized GOT tables.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The current script logic assigns an empty string to APID and APQI
when APQN consists entirely of zeros (e.g., "00.0000").
However, this behavior is incorrect, as "00" and "0000" are valid
values and should be represented as "0".
This commit ensures that the script assigns the default string “0”
to APID and APQI if their computed values are empty.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Due to the agent-api tests requiring the agent to be deployed in the
CI by the tarball, so in the short-term lets only do this on the release
stage, so that both kata-manager works with the release and the
agent-api tests work with the other CI builds.
In the longer term we need to re-evaluate what is in our tarballs
(issue #10619), but want to unblock the tests in the short-term.
Fixes: #10630
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
With the code I originally did I think there is potentially
a case where we can get a failure due to timing of steps.
Before this change the `build-asset-shim-v2`
job could start the `get-artifacts` step and concurrently
`remove-rootfs-binary-artifacts` could run and delete the artifact
during the download and result in the error. In this commit, I
try to resolve this by making sure that the shim build waits
for the artifact deletes to complete before starting.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
We want to have the latest QEMU version available
which is as of this writing v9.1.2
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
qemu: Add new options for 9.1.2
We need to fence specific options depending on the version
and disable ones that are not needed anymore
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
qemu: Add no_patches.txt
Since we do not have any patches for this version
let's create the appropriate files.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The new QEMU build needs python-tomli, now that we bumped Ubuntu
we can include the needed tomli package
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
We're disabling pmem support, it is heavilly broken with
Ubuntu's static build of QEMU and not needed
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
In jammy we have the liburing package available, hence
remove the source build and include the package.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This path fixes the logic of getting the type of volume: when the type of
OCI mount is Some("none") and the options have "bind" or "rbind", the
type will be considered as "bind".
Fixes: #10642
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This commit sets memory config `shared` to false in cloud hypervisor
when creating vm with shared_fs=None && hugePages = false.
Currently in runtime/virtcontainers/clh.go,the memory config shared is by default set to true.
As per the CLH memory document,
(a) shared=true is needed in case like when using virtio_fs since virtiofs daemon runs as separate process than clh.
(b) for shared_fs=none + hugespages=false, shared=false can be set to use private anonymous memory for guest (with no file backing).
(c) Another memory config thp (use transparent huge pages) is always enabled by default.
As per documentation, (b) + (c) can be used in combination.
However, with the current CLH implementation, the above combination cannot be used since shared=true is always set.
Fixes#10547
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
> A && B || C is not if-then-else. C may run when A is true
Refactor the echo so that we can't get into a situation where
the retry of workspace delete happens if the original one was
successful
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Bump some actions that are significantly out-of-date
and out of sync with the versions used in other workflows
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The tdx runners got split into two different
runners, so we need to update the known self-hosted
runner labels
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
On PRs that update anything in the workflows directory,
add an actionlint run to validate our workflow files for errors
and hopefully catch issues earlier.
Fixes: #9646
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
So users can simply download the chart and use it accordingly without
the need to download the full repo.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Add sub command MemAgentMemcgSet and MemAgentCompactSet to agent-ctl to
configate the mem-agent inside the running kata-containers.
Fixes: #10625
Signed-off-by: Hui Zhu <teawater@antgroup.com>
Add MemAgentMemcgSet and MemAgentCompactSet to agent API to set the config of
mem-agent memcg and compact.
Fixes: #10625
Signed-off-by: Hui Zhu <teawater@antgroup.com>
mem-agent is a component designed for managing memory in Linux
environments.
Sub-feature memcg: Utilizes the MgLRU feature to monitor each cgroup's
memory usage and periodically reclaim cold memory.
Sub-feature compact: Periodically compacts memory to facilitate the
kernel's free page reporting feature, enabling the release of more idle
memory from guests.
During memory reclamation and compaction, mem-agent monitors system
pressure using Pressure Stall Information (PSI). If the system pressure
becomes too high, memory reclamation or compaction will automatically
stop.
Fixes: #10625
Signed-off-by: Hui Zhu <teawater@antgroup.com>
Add mglru, debugfs and psi to dragonball-experimental/mem_agent.conf to
support mem_agent function.
Fixes: #10625
Signed-off-by: Hui Zhu <teawater@antgroup.com>
This is super useful for development / debugging scenarios, mainly when
dealing with limited hardware availability, as this change allows
multiple people to develop into one single machine, while still using
kata-deploy.
Fixes: #10546
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
At the same time that INSTALLATION_PREFIX was added, I was working on
the helm changes to properly do the cleanup / deletion when it's
removed. However, I missed adding the INSTALLATION_PREFIX env var
there. which I'm doing now.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
- Remove default_namespace from settings
- Ensure container namespaces in a pod match each other in case no namespace is specified in the YAML
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
The tests is unstable on this platform, so skip it for now to prevent
the regular known failures covering up other issues. See #10616
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Fix copy-paste errors in artifact filters for arm64 and ppc64le
- Remove the trailing wildcard filter that falsely ends up removing agent-ctl
and replace with the tarball-suffix, which should exactly match the artifacts
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
With the building/publishing step for the CSI driver validated, we can
set that as a requirement for the CoCo tests.
Depends on: #10561
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
The driver build recipe has a script to check the current Go version against
the go.mod version. However, the script is broken ($expected is unbound) and I
don't believe we do this for other components. On top of this, Go should be
backward-compatible. Let's keep things simple for now and we can evaluate
restoring this script in the future if need be.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This fully implements the compilation step for csi-kata-directvolume.
This component can now be built by the CI running:
$ cd tools/packaging/kata-deploy/local-build
$ make csi-kata-directvolume-tarball
A couple notes:
* When installing the binary, we rename it from directvolplugin to
csi-kata-directvolume on the fly to make it more readable.
* We add go to the tools builder Dockerfile to support building this
tool.
* I've noticed the file install_libseccomp.sh gets created by the build
process so I've added it to a .gitignore.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
The devno assignment logic is repeated in 5 different places
during device addition.
To improve code maintainability and readability, this commit
introduces a standalone function, `get_devno_ccw()`,
to handle the deduplication.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Before this patch there was a mismatch between the JSON path under which
the state of the rule evaluation is set in comparison to under which
it is retrieved.
This resulted in the behavior that each time the policy was evaluated,
it thought it was the _first_ time the policy was evaluated.
This also means that the consistency check for the `sandbox_name`
was ineffective.
Signed-off-by: Leonard Cohnen <lc@edgeless.systems>
With full debug logging enabled there might be around 1,500 redials
so log just ~15 of these redials to avoid flooding the log.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The default initrd confidential target will have a
variant=confidential we need to accomodate this
and make sure we also accomodate aaa-xxx-confidential targets.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
We've also seen the qemu metrics tests are failing due to the results
being slightly outside the max range for network-iperf3 parallel and minimum for network-iperf3 jitter tests on PRs that have no code changes,
so we've increase the bounds to not see false negatives.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
We've seen a couple of instances recently where the metrics
tests are failing due to the results being below the minimum
value by ~2%.
For tests like latency I'm not sure why values being too low would
be an issue, but I've updated the minpercent range of the failing tests
to try and get them passing.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
As the following CI job has been marked as required:
- kata-containers-ci-on-push / run-k8s-tests-on-zvsi / run-k8s-tests (devmapper, qemu, kubeadm)
we need to add it to the gatekeeper's required job list.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This reverts commit 559018554b.
As we've noticed that this is causing issues with initrd builds in the
CI.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This will help us to simply allow a new dummy build whenever a new
component is added.
As long as the format `$(call DUMMY,$@)` is followed, we should be good
to go without taking the risk of breaking the CI.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This will make our lives considerably easier when it comes to cleaning
up content added, while it's also a groundwork needed for having
multiple installations running in parallel.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
We need the publish certain artefacts for the rootfs,
like the agent, guest-components, pause bundle etc
as they are consumed in the `build-asset-rootfs` step.
However after this point they aren't needed and probably
shouldn't be included in the overall kata tarball, so delete
them once they aren't needed any more to avoid them
being included.
Fixes: #10575
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Let's actually mount the whole /etc/k0s as /etc/containerd, so we can
easily access the containerd configuration file which has the version in
it, allowing us to parse it instead of just making a guess based on
kubernetes distro being used.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
On Ubuntu 24.04, with the distro default containerd, we're already
getting:
```
$ containerd config default | grep "version = "
version = 3
```
With that in mind, let's make sure that we're ready to support this from
the next release.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
A new attribute named `devno` is added to DeviceVirtioScsi.
It will be used to specify a device number for a CCW bus type.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
A new attribute named `devno` is added to DeviceVhostUserFs.
It will be used to specify a device number for a CCW bus type.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
A new attribute named `devno` is added to VhostVsock.
It will be used to specify a device number for a CCW bus type.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
A new attribute named `devno` is added to DeviceVirtioSerial.
It will be used to specify a device number for a CCW bus type.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
A new attribute named `devno` is added to DeviceVirtioBlk.
It will be used to specify a device number for a CCW bus type.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
To explicitly specify a device number on the QEMU command line
for the following devices using the CCW transport on s390x:
- SerialDevice
- BlockDevice
- VhostUserDevice
- SCSIController
- VSOCKDevice
this commit introduces a new structure CcwSubChannel and implements
the following methods:
- add_device()
- remove_device()
- address_format_ccw()
- set_addr()
You can see the detailed explanation for each method in the comment.
This resolves the 1st part of #10573.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
let's print the also the existing result's id when printing the
information about ignoring older result id to simplify debugging.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
tha matching run_id means we're dealing with the same job but with
updated results and not with an older job. Update the results in such
case.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
This adds a new job to build and publish the CSI driver Docker image.
Of course this job will fail after we merge this PR because the CSI driver
compilation job hasn't been implemented yet. However that will be implemented
directly after in #10561.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This adds a no-op build step to compile the CSI driver. The actual compilation
will be implemented in an ulterior PR, so as to ensure we don't break the CI.
Addresses: #10560
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Build ubuntu rootfs with Docker failed with error:
`Unable to find libclang`
Fix this error by adding libclang-dev to the dependency.
Signed-off-by: Jitang Lei <leijitang@outlook.com>
We need to clean-up any created files/dirs otherwise
we cause problems on self-hosted runners. Using tempdir which
will be removed automatically.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Now we are downloading artifacts to create the rootfs
we need to ensure they are uploaded always,
even on releases
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
KinD checks for the presence of this (and other) kernel configuration
via scripts like
https://blog.hypriot.com/post/verify-kernel-container-compatibility/ or
attempts to directly use /proc/sys/kernel/keys/ without checking to see
if it exists, causing an exit when it does not see it.
Docker/it's consumers apparently expect to be able to use the kernel
keyring and it's associated syscalls from/for containers.
There aren't any known downsides to enabling this except that it would
by definition enable additional syscalls defined in
https://man7.org/linux/man-pages/man7/keyrings.7.html which are
reachable from userspace. This minimally increases the attack surface of
the Kata Kernel, but this attack surface is minimal (especially since
the kernel is most likely being executed by some kind of hypervisor) and
highly restricted compared to the utility of enabling this feature to
get further containerization compatibility.
Signed-off-by: Crypt0s <BryanHalf@gmail.com>
This will be helpful in order to increase the OS coverage (we'll be
using both Ubuntu 24.04 and CentOS 9 Stream), while also reducing the
amount spent on the tests (as one machine will only run attestation
related tests, and the other the tests that do *not* require
attestation).
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
We're doing this as, at Intel, we have two different kind of machines we
can plug into our CI. Without going much into details, only one of
those two kinds of machines will work for the attestation tests we
perform with ITA, thus in order to speed up the CI and improve test
coverage (OS wise), we're going to run different tests in different
machines.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
As discussed in the CI working group,
we are temporarily skipping the SNP CI
to unblock the remaining workflow.
Will revert after fixing the SNP runner.
Signed-Off-By: Adithya Krishnan Kannan <AdithyaKrishnan.Kannan@amd.com>
Use regorous engine's add_data method to add state to the policy.
This data can later be accessed inside rego context through the data namespace.
Support state modifications (json-patches) that may be returned as a result from policy evaluation.
Also initialize a policy engine data slice "pstate" dedicated for storing state.
Fixes#10087
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
The Clear Linux rootfs is not being tested anywhere, and it seems Intel
doesn't have the capacity to review the PRs related to this (combined
with the lack of interested from the rest of the community on reviewing
PRs that are specific to this untested rootfs).
With this in mind, I'm suggesting we drop Clear Linux support and focus
on what we can actually maintain.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This reverts commit f15e16b692, as we
don't have to do this since we're relying on the
`static_sandbox_resource_mgmt` feature, which gives us the correct
amount of memory and CPUs to be allocated.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Trustee's deployment must set the correct https_proxy as env var on the
container that will talk to the ITA / ITTS server, otherwise the kbs
service won't be able to start, causing then issues in our CI.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Signed-off-by: Krzysztof Sandowicz <krzysztof.sandowicz@intel.com>
Fedora F40 removed python3 from the base container, to avoid such issues
let's rely on the latest and greates official python container.
Fixes: #10497
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
This PR adds the get artifacts which are needed when installing kata
tools in stability workflow to avoid failures saying that artifacts
are missing.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
As discussed in the AC meeting, we don't have a maintainer,
(or users?) of runk, and the CI is unstable, so giving we can't
support it, we shouldn't waste CI cycles on it.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
As discussed on the AC call, we are lacking maintainers for the
metrics tests. As a starting point for potentially phasing them
out, we discussed starting with removing the test for stratovirt
as a non-core hypervisor and a job that is problematic in leaving
behind resources that need cleaning up.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This PR adds the install kata tools step as part of the k8s stability workflow.
To avoid the failures saying that certain kata components are not installed it.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Currently the error we are checking for is
`CreateContainerRequest timed out`, but this message
doesn't always seem to be printed to our pod log.
Try using a more general message that should be present
more reliably.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
We have an error with service name resolution with this test when using crio.
This error could not be reproduced outside of the CI for now.
Skipping it to keep the CI job running until we find a solution.
See: #10414
Signed-off-by: Julien Ropé <jrope@redhat.com>
By the moment we're testing it also with qemu-coco-dev, it becomes
easier for a developer without access to TEE to also test it locally.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
The qemu-coco-dev runtime class should be as close as possible to what
the TEEs runtime classes are doing, and this was one of the options that
ended up overlooked till now.
Shout out to Dan Mihai for noticing that!
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Most likely this was overlooked during the development / review, but
we're actually interested on the size rather than on the pagesize of the
hugepages.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
So far we were not prepared to deal with release candidates as those:
* Do not have a sha256sum in the sha256sums provided by the kernel cdn
* Come from a different URL (directly from Linus)
* Have a different suffix (.tar.gz, instead of .tar.xz)
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This doesn't change much on how we're doing things Today, but it
simplifies a lot cases that may be added later on (and will be) like
building -rc kernels.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
By doing this we can ensure this can be re-used, if needed (and it'll be
needed), for also getting the URL.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
-f forces the (re)generaton of the config when doing the setup, which
helps a lot on local development whilst not causing any harm in the CI
builds.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Let's just do a simple `sed` and **not** use the repo that became
private.
This is not a backport of https://github.com/tianocore/edk2/pull/6402,
but it's a similar approach that allows us to proceed without the need
to pick up a newer version of edk2.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This PR fixes the source to avoid duplication specially in the common.sh
script and avoid failures saying that certain script is not in the directory.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This change is motivated by a problem in peerpod's podvms. In this setup
the lifecycle of guest components is managed by systemd. The current code
skips over init steps like setting the ocicrypt-rs env and initialization
of a CDH client in this case.
To address this the launch of the processes has been isolated into its
own fn.
Signed-off-by: Magnus Kulke <magnuskulke@microsoft.com>
This reverts commit 973b8a1d8f.
As @danmihai1 points out https://github.com/bats-core/bats-core/issues/364
states that using traps in bats is error prone, so this could be the cause
of the confidential test instability we've been seeing, like it was
in the static checks, so let's try and revert this.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
At @danmihai1's suggestion add a die message in case
the call to setup_common fails, so we can see if in the test
output.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
We've seen some issues with tests not being run in
some of the Coco CI jobs (Issue #10451) and in the
envrionments that are more stable we noticed that
they had a newer version of bats installed.
Try updating the version to 1.10+ and print out
the version for debug purposes
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
We need to know if we're building a nvidia initrd or image
Additionally if we build a regular or confidential VARIANT
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Fixes: #9478
We want to keep track of the driver versions build during initrd/image build so update the artifact_name after the fact.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This commit introduces changes to enable testing kata-agent's container
APIs of CreateContainer/StartContainer/RemoveContainer. The changeset
include:
- using confidential-containers image-rs crate to pull/unpack/mount a
container image. Currently supports only un-authenicated registry pull
- re-factor api handlers to reduce cmdline complexity and handle
request generation logic in tool
- introduce an OCI config template for container creation
- add test case
Fixes#9707
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
This PR updates the virtualization document by removing a url link
which is not longer valid.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The current version of the oci-spec crate compiles RLimit structs only
for Linux and Solaris. Until this is fixed upstream, add compilation
conditions to the type converters for the affected structs.
Fixes: #10071
Signed-off-by: Markus Rudy <mr@edgeless.systems>
The parse_json_string function is specific to parsing capability strings
out of ttRPC proto definitions and does not benefit from being available
to other crates. Moving it into the protocols crate allows removing
kata-sys-util as a dependency, which in turn enables compiling the
library on darwin.
Fixes: #10071
Signed-off-by: Markus Rudy <mr@edgeless.systems>
The cgroups.rs source file was removed in
234d7bca04. With cgroups support handled
in runtime-rs, the cgroups dependency on kata-sys-util can be removed.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
This commit introduces changes to use ubuntu for statically
building kata tools. In the existing CI setup, the tools
currently build only for x86_64 architecture.
It also fixes the build error seen for agent-ctl PR#10395.
Fixes#10441
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
This PR adds missing steps in the gha run script for the kata stability
workflow.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR removes the kernel config repo variable at the build kernel
script as it is not used.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The cloud-api-adaptor needs to support different types of pod VM
instance.
We needs to pass some annotations like machine_type, default_vcpus and
default_memory to prepare the VMs.
Signed-off-by: Chasing1020 <643601464@qq.com>
This patch adds the support of the remote hypervisor type for runtime-rs.
The cloud-api-adaptor needs the annotations and network namespace path
to create the VMs.
The remote hypervisor opens a UNIX domain socket specified in the config
file, and sends ttrpc requests to a external process to control sandbox
VMs.
Fixes: #10350
Signed-off-by: Chasing1020 <643601464@qq.com>
Add GPU annotations for remote hypervisor to help
with the right instance selection based on number of GPUs
and model
Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Last November, SUSE discontinued support for s390x, leaving k3s
on this platform stuck at k8s version 1.28, while upstream k8s
has since reached 1.31. Fortunately, kubeadm allows us to create
a 1.30 Kubernetes cluster on s390x.
This commit switches the KUBERNETES option from k3s to kubeadm
for s390x and removes a dedicated cluster creation step.
Now, cluster setup and teardown occur in ACTIONS_RUNNER_HOOK_JOB_{STARTED,COMPLETED}.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
When creating a k8s cluster via kubeadm, the devmapper setup
for containerd requires a different configuration.
This commit introduces a new `kubeadm` option for the KUBERNETES
variable and adjusts the path to the containerd config file for
devmapper setup.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
All the oras push logic happens from inside `${workdir}`, while the
root_hash.txt extraction and renaming was not taking this into
consideration.
This was not caught during the manually triggered runs as those do not
perform the oras push.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
As we're now building everything needed to test TDX with measured rootfs
support, let's bring this test back in (for TDX only, at least for now).
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
The approach taken for now is to export MEASURED_ROOTFS=yes on the
workflow files for the architectures using confidential stuff, and leave
the "normal" build without having it set (to avoid any change of
expectation on the current bevahiour).
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Let's make sure we take the root_hashes into consideration to decide
whether the shim-v2 should or should not be used from the cached
artefacts.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Let's cache the root_hash.txt from the confidential image so we can use
them later on to decide whether there was a rootfs change that would
require shim-v2 to be rebuilt.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Let's ensure we remove the component and any extra tarball provided by
ORAS in case the cached component cannot be used.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
We may decide to add this later on, but for now this is only targetting
TEEs and the confidential image / initrd.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Let's ensure that we get the already built rootfs tarball from previous
steps of the action at the time we're building the shim-v2.
The reason we do that is because the rootfs binary tarballs has a
root_hash.txt file that contains the information needed the shim-v2
build scripts to add the measured rootfs arguments to the shim-v2
configuration files.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
As the rootfs will have what we need to add as part of the shim-v2
configuration files for measured rootfs, we **must** ensure this is
built **before** shim-v2.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
By doing this we can just re-use the dependencies already built, saving
us a reasonable amount of time.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This is a helper script that does basically what's already being done by
the s390x CI, which is:
* Move a folder with the components that we were stored / downloaded
during the GHA execution to the expected `build` location
* Get rid of the dependencies for a specific asset, as the dependencies
are already pulled in from previous GHA steps
For now this script is only being added but not yet executed anywhere,
and that will come as the next step in this series.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
So far we haven't been storing the rootfs dependencies as part of our
workflows, but we better do it to re-use them as part of the rootfs
build.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Return of proper error to the initiator is not guaranteed.
Method StopVM could kill shim process together with VM pieces.
Signed-off-by: Konstantin Khlebnikov <koct9i@gmail.com>
The only reason we had this one passing for amd64 is because the check
was done using the wrong variable (`matrix.stage`, while in the other
workflows the variable used is `inputs.stage`).
The commit that broke the release process is 67a8665f51, which
blindly copy & pasted the logic from the matrix assets.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Corrects device filemode permissions typo/regression in rustjail to `666` instead of `066`.
`666` is the standard and expected value for these devices in containers.
Fixes: #10454
Signed-off-by: Simon Kaegi <simon.kaegi@gmail.com>
This PR increase the time to run the stressng k8s tests for the
CoCo stability CI.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
More than one developer can and should be able to run this workflow at
the same time, without cancelling the job started by another developer.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Let's use "dev" instead of "manually-triggered" as it avoids the name
being too long, which results in failures to create AKS clusters.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This PR adds the trap statement into the kata doc
script to clean up properly the temporary files.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
As a new workflow was added for the cases where developers want to test
their changes in the workflow itself, let's make sure we stop allowing
manual triggers on this workflow, which can lead to a polluted /
misleading weather of the CI.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This workflow is intended to replace the `workflow_dispatch` trigger
currently present as part of the `ci-nightly.yaml`.
The reasoning behind having this done in this way is because of our good
and old GHA behaviour for `pull_request_target`, which requires a PR to
be merged in order to check the changes in the workflow itself, which
leads to:
* when a change in a workflow is done, developers (should) do:
* push their branch to the kata-containers repo
* manually trigger the "nightly" CI in order to ensure the changes
don't break anything
* this can result in the "nightly" CI weather being polluted
* we don't have the guarantee / assurance about the last n nightly
runs anymore
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Change the PACKAGES variable for the cbl-mariner rootfs-builder
to use the kata-packages-uvm meta package from
packages.microsoft.com to define the set of packages to be
contained in the UVM.
This aligns the UVM build for the Azure Linux distribution
with the UVM build done for the Kata Containers offering on
Azure Kubernetes Services (AKS).
Signed-off-by: Manuel Huber <mahuber@microsoft.com>
By doing this we can ensure that whenever the rootfs changes, we'll be
able to get the new root_hash.txt and use it.
This is the very first step to bring the measured rootfs tests back.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
When QEMU is terminated by signal 15, it deletes the PidFile.
Upon detecting that QEMU has exited, the shim executes the stopVM function.
If the PidFile is not found, the PID is set to 0.
Subsequently, the shim executes `kill -9 0`, which terminates the current process group.
This prevents any further logic from being executed, resulting in resources not being cleaned up.
Signed-off-by: wangyaqi54 <wangyaqi54@jd.com>
Semantics are lifted straight out of the go runtime for compatibility.
We introduce DeviceVirtioScsi to represent a virtio-scsi device and
instantiate it if block device driver in the configuration file is set
to virtio-scsi. We also introduce ObjectIoThread which is instantiated
if the configuration file additionally enables iothreads.
Signed-off-by: Pavel Mores <pmores@redhat.com>
When do update_container_namespaces updating namespaces, setting
all UTS(and IPC) namespace paths to None resulted in hostnames
set prior to the update becoming ineffective. This was primarily
due to an error made while aligning with the oci spec: in an attempt
to match empty strings with None values in oci-spec-rs, all paths
were incorrectly set to None.
Fixes#10325
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Moved the parsing of the oci image marker into its own step, since we
only need to perform that for attestation purposes and some cached
images might not have that file in the tarball.
Signed-off-by: Magnus Kulke <magnuskulke@microsoft.com>
This adds provenance attestation logic for agent binaries that are
published to an oci registry via ORAS.
As a downstream consumer of the kata-agent binary the Peerpod project
needs to verify that the artifact has been built on kata's CI.
To create an attestation we need to know the exact digest of the oci
artifact, at the point when the artifact was pushed.
Therefore we record the full oci image as returned by oras push.
The pushing and tagging logic has been slightly reworked to make this
task less repetetive.
The oras cli accepts multiple tags separated by comma on pushes, so a
push can be performed atomically instead of iterating through tags and
pushing each individually. This removes the risk of partially successful
push operations (think: rate limits on the oci registry).
So far the provenance creation has been only enabled for agent builds on
amd64 and xs390x.
Signed-off-by: Magnus Kulke <magnuskulke@microsoft.com>
This PR updates the CI documentation referring to the several tests and
in which kind of instances is running them.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Use rstest for unit test rather than TestData arrays where
possible to make the code more compact, easier to read
and open the possibility to enhance test cases with a
description more easily.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This PR replaces the arch uname -m to use the arch_to_golang
variable in the script to have a better uniformity across the script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
inotify-configmap-pod.yaml is using: "inotifywait --timeout 120",
so wait for up to 180 seconds for the pod termination to be
reported.
Hopefully, some of the sporadic errors from #10413 will be avoided
this way:
not ok 1 configmap update works, and preserves symlinks
waitForProcess "${wait_time}" "$sleep_time" "${command}" failed
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
While Kubernetes defines `binaryData` as `[]byte`,
when defined in a YAML file the raw bytes are
base64 encoded. Therefore, we need to read the YAML
value as `String` and not as `Vec<u8>`.
Fixes: #10410
Signed-off-by: Leonard Cohnen <lc@edgeless.systems>
Adds dependencies of 'clang' & 'protobuf' to be installed in runners
when building agent-ctl sources having image pull support.
Fixes#10400
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
This PR removes egrep command as it has been deprecated and it replaces by
grep in the test images script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Use the configuration used by AKS (static_sandbox_resource_mgmt=true)
for CI testing on Mariner hosts.
Hopefully pod startup will become more predictable on these hosts -
e.g., by avoiding the occasional hotplug timeouts described by #10413.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
When dealing with a specific release, it was easier to just do some
adjustments on the image that has to be used for ITA without actually
adding a new entry in the versions.yaml.
However, it's been proven to be more complicated than that when it comes
to dealing with staged images, and we better explicitly add (and
update) those versions altogether to avoid CI issues.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
The root cause is that the CDH client is a global variable, and unit tests `test_unseal_env` and `test_unseal_file`
share this lock-free global variable, leading to resource contention and destruction.
Merging the two unit tests into one test_sealed_secret will resolve this issue.
Fixes: #10403
Signed-off-by: ChengyuZhu6 <zhucy0405@gmail.com>
As mariner has switched to using an image instead of an initrd, let's
just drop the abiliy to build the initrd and avoid keeping something in
the tree that won't be used.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
As the mariner image is already in place, and the tests were modified to
use them (as part of this series), let's just stop building it as part
of the CI.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
The reason we're doing this is because mariner image uses, by default,
cgroups default-hierarchy as `unified` (aka, cgroupsv2).
In order to keep the same initrd behaviour for mariner, let's enforce
that `SYSTEMD_CGROUP_ENABLE_LEGACY_FORCE=1
systemd.legacy_systemd_cgroup_controller=yes
systemd.unified_cgroup_hierarchy=0` is passed to the kernel cmdline, at
least for now.
Other tests that are setting `kernel_params` are not running on mariner,
then we're safe taking this path as it's done as part of this PR.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
As an image has been added for mariner as part of the commit 63c1f81c2,
let's start using it in the CI, instead of using the initrd.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This reverts commit 31e09058af, as it's
breaking the agent unit tests CI.
This is a stop gap till Chengyu Zhu finds the time to properly address
the issue, avoiding the CI to be blocked for now.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
In https://github.com/confidential-containers/trustee/pull/521
the overlays logic was modified to add non-SE
s390x support and simplify non-ibm-se platforms.
We need to update the logic in `kbs_k8s_deploy`
to match and can remove the dummying of `IBM_SE_CREDS_DIR`
for non-SE now
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This PR adds the trap statement in the test images script to clean up
tmp files.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Kata CI will start testing the new rootfs-image-mariner instead of the
older rootfs-initrd-mariner image.
The "official" AKS images are moving from a rootfs-initrd-mariner
format to the rootfs-image-mariner format. Making the same change in
Kata CI is useful to keep this testing in sync with the AKS settings.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Refactor CDH-related operations into the cdh_handler function to make the `create_container` code clearer.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Users must set the mount path to `/sealed/<path>` for kata agent to detect the sealed secret mount
and handle it in createcontainer stage.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Signed-off-by: Linda Yu <linda.yu@intel.com>
Introduced `unseal_file` function to unseal secret as files:
- Implemented logic to handle symlinks and regular files within the sealed secret directory.
- For each entry, call CDH to unseal secrets and the unsealed contents are written to a new file, and a symlink is created to replace the sealed symlink.
Fixes: #8123
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Signed-off-by: Linda Yu <linda.yu@intel.com>
By checking for AGENT_POLICY we ensure we only try to read
allow-all.rego if AGENT_POLICY is set to "yes"
Signed-off-by: Emanuel Lima <emlima@redhat.com>
The behavior of Kata CI doesn't change.
For local testing using kubernetes/gha-run.sh and AUTO_GENERATE_POLICY=yes:
1. Before these changes users were forced to use:
- SEV, SNP, or TDX guests, or
- KATA_HOST_OS=cbl-mariner
2. After these changes users can also use other platforms that are
configured with "shared_fs = virtio-fs" - e.g.,
- KATA_HOST_OS=ubuntu + KATA_HYPERVISOR=qemu
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR removes duplicated arch variable definition in the rootfs script
as this variable and its value is already defined at the top of the
script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
add_device() now checks if QEMU is running already by checking if we have
a QMP connection. If we do a new function hotplug_device() is called
which hotplugs the device if it's a network one.
Signed-off-by: Pavel Mores <pmores@redhat.com>
With the helpers from previous commit, the actual hotplugging
implementation, though lengthy, is mostly just assembling a QMP command
to hotplug the network device backend and then doing the same for the
corresponding frontend.
Note that hotplug_network_device() takes cmdline_generator types Netdev
and DeviceVirtioNet. This is intentional and aims to take advantage of
the similarity between parameter sets needed to coldplug and hotplug
devices reuse and simplify our code. To enable using the types from qmp,
accessors were added as needed.
Signed-off-by: Pavel Mores <pmores@redhat.com>
Before adding network device hotplugging functionality itself we add
a couple of helpers in a separate commit since their functionality is
non-trivial.
To hotplug a device we need a free PCI slot. We add find_free_slot()
which can be called to obtain one. It looks for PCI bridges connected
to the root bridge and looks for an unoccupied slot on each of them. The
first found is returned to the caller. The algorithm explicitly doesn't
support any more complex bridge hierarchies since those are never produced
when coldplugging PCI bridges.
Sending netdev queue and vhost file descriptors to QEMU is slightly
involved and implemented in pass_fd(). The actual socket has to be passed
in an SCM_RIGHTS socket control message (also called ancillary data, see
man 3 cmsg) so we have to use the msghdr structure and sendmsg() call
(see man 2 sendmsg) to send the message. Since qapi-rs doesn't support
sending messages with ancillary data we have to do the sending sort of
"under it", manually, by retrieving qapi-rs's socket and using it directly.
Signed-off-by: Pavel Mores <pmores@redhat.com>
NetworkConfig::index has been used to generate an id for a network device
backend. However, it turns out that it's not unique (it's always zero
as confirmed by a comment at its definition) so it's not suitable to
generate an id that needs to be unique.
Use the host device name instead.
Signed-off-by: Pavel Mores <pmores@redhat.com>
Network device hotplugging will use the same infrastructure (Netdev,
DeviceVirtioNet) as coldplugging, i.e. QemuCmdLine. To make the code
of network device setup visible outside of QemuCmdLine we factor it out
to a non-member function `get_network_device()` and make QemuCmdLine just
delegate to it.
Signed-off-by: Pavel Mores <pmores@redhat.com>
The function takes a whole QemuCmdLine but only actually uses
HypervisorConfig. We increase callability of the function by limiting
its interface to what it needs. This will come handy shortly.
Signed-off-by: Pavel Mores <pmores@redhat.com>
At least one PCI bridge is necessary to hotplug PCI devices. We only
support PCI (at this point at least) since that's what the go runtime
does (note that looking at the code in virtcontainers it might seem that
other bus types are supported, however when the bridge objects are passed
to govmm, all but PCI bridges are actually ignored). The entire logic of
bridge setup is lifted from runtime-go for compatibility's sake.
Signed-off-by: Pavel Mores <pmores@redhat.com>
with multiple iterations/reruns we need to use the latest run of each
workflow. For that we can use the "run_id" and only update results of
the same or newer run_ids.
To do that we need to store the "run_id". To avoid adding individual
attributes this commit stores the full job object that contains the
status, conclussion as well as other attributes of the individual jobs,
which might come handy in the future in exchange for slightly bigger
memory overhead (still we only store the latest run of required jobs
only).
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
some tests require certain labels before they are executed. When our PR
is not labeled appropriately the gatekeeper detects skipped required
tests and reports a failure. With this change we add "required-labeles"
to the tests mapping and check the expected labels first informing the
user about the missing labeles before even checking the test statuses.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
the test names are using `;` and regexps were designed to use `,` but
during development simply joined the expressions by `|`. This should
work but might be confusing so let's go with the semi-colon separator
everywhere.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
The Github SHA of triggering PR should be exported in the environment
so that gatekeeper can fetch the right workflows/jobs.
Note: by default github will export GITHUB_SHA in the job's environment
but that value cannot be used if the gatekeeper was triggered from a
pull_request_target event, because the SHA correspond to the push
branch.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This will allow to cancel-in-progress the gatekeeper jobs.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
to allow selective testing as well as selective list of required tests
let's add a mapping of required jobs/tests in "skips.py" and a
"gatekeaper" workflow that will ensure the expected required jobs were
successful. Then we can only mark the "gatekeaper" as the required job
and modify the logic to suit our needs.
Fixes: #9237
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
The behavior of Kata CI doesn't change.
For local testing using kubernetes/gha-run.sh:
1. Before these changes:
- AUTO_GENERATE_POLICY=yes was always used by the users of SEV, SNP,
TDX, or KATA_HOST_OS=cbl-mariner.
2. After these changes:
- Users of SEV, SNP, TDX, or KATA_HOST_OS=cbl-mariner must specify
AUTO_GENERATE_POLICY=yes if they want to auto-generate policy.
- These users have the option to test just using hard-coded policies
(e.g., using the default policy built into the Guest rootfs) by
using AUTO_GENERATE_POLICY=no. AUTO_GENERATE_POLICY=no is the default
value of this env variable.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR adds the trap statement in the confidential kbs script
to clean up temporary files and ensure we are leaving them.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The tests is disabled for qemu-coco-dev / qemu-tdx, but it doesn't seen
to actually be failing on those. Plus, it's passing on SEV / SNP, which
means that we most likely missed re-enabling this one in the past.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
The quay.io registry returns the tags sorted alphabetically and doesn't
seem to provide a way to sort it by age. Let's use "git log" to get all
changes between the commits and print all tags that were actually
pushed.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Currently, `qemu-runtime-rs` does not support `virtio-scsi`,
which causes the `k8s-block-volume.bats` test to fail.
We should skip this test until `virtio-scsi` is supported by the runtime.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The nginx container seems to error out when using UID=123.
Depending on the timing between container initialization and "kubectl
wait", the test might have gotten lucky and found the pod briefly in
Ready state before nginx errored out. But on some of the nodes, the pod
never got reported as Ready.
Also, don't block in "kubectl wait --for=condition=Ready" when wrapping
that command in a waitForProcess call, because waitForProcess is
designed for short-lived commands.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR updates the fast footprint script to remove the use
of egrep as this command has been deprecated and change it
to use grep command.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This imports the k8s-block-volume test from the tests repo and modifies
it slightly to set up the host volume on the AKS host.
This is a follow-up to #7132.
Fixes: #7164
Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
The ita kustomization for Trustee, as well as previously used one
(DCAP), doesn't have a $(uname -m) directory after the deployment
directory name.
Let's follow the same logic used for the deploy-kbs script and clean
those up accordingly.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Intel Tiber Trust Services (formerly known as Intel Trust Authority) is
Intel's own attestation service, and we want to take advantage of the
TDX CI in order to ensure ITTS works as expected.
In order to do so, let's replace the former method used (DCAP) to use
ITTS instead.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Inorder to support sandbox api, intorduce the sandbox_config
struct and split the sandbox start stage from init process.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
There are many similar or duplicated code patterns in `teardown()`.
This commit consolidates them into a new function, `teardown_common()`,
which is now called within `teardown()`.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The current `exec_host()` accepts a given node name and
creates a node debugger pod, even if the name is invalid.
This could result in the creation of an unnecessary pending
pod (since we are using nodeAffinity; if the given name
does not match any actual node names, the pod won’t be scheduled),
which wastes resources.
This commit introduces validation for the node name to
prevent this situation.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This commit enables basic amd64 tests of docker for runtime-rs by adding
vmm types "dragonball" and "cloud-hypervisor".
Signed-off-by: Sicheng Liu <lsc2001@outlook.com>
Docker cannot exit normally after the container process exits when
used with runtime-rs since it doesn't receive the exit event. This
commit enable runtime-rs to send TaskExit to containerd after process
exits.
Also, it moves "system_time_into" and "option_system_time_into" from
crates/runtimes/common/src/types/trans_into_shim.rs to a new utility
mod.
Signed-off-by: Sicheng Liu <lsc2001@outlook.com>
When the sandbox api was enabled, the pasue container
wouldn't be created, thus the shared sandbox pidns
should be fallbacked to the first container's init process,
instead of return any error here.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
It was observed that the custom node debugger pod is not
cleaned up when a test times out.
This commit ensures the pod is cleaned up by triggering
the cleanup on EXIT, preventing any debugger pods from
being left behind.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
When using network adapters that support SR-IOV, a VFIO device can be
plugged into a guest VM and claimed as a network interface. This can
significantly enhance network performance.
Fixes: #9758
Signed-off-by: Lei Huang <leih@nvidia.com>
With #10232 merged, we now have a persistent node debugger pod throughout the test.
As a result, there’s no need to spawn another debugger pod using `kubectl debug`,
which could lead to false negatives due to premature pod termination, as reported
in #10081.
This commit removes the `print_node_journal()` call that uses `kubectl debug` and
instead uses `exec_host()` to capture the host journal. The `exec_host()` function
is relocated to `tests/integration/kubernetes/lib.sh` to prevent cyclical dependencies
between `tests_common.sh` and `lib.sh`.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
in b9d88f74ed the `runtime_class` CM was
added which overrides the one we previously set. Let's reorder our logic
to first deploy webhook and then override the default CM in order to use
the one we really want.
Since we need to change dirs we also have to use realpath to ensure the
files are located well.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
`assert_pod_fail()` currently calls `k8s_create_pod()` to ensure that a pod
does not become ready within the default 120s. However, this delays the test's
completion even if an error message is detected earlier in the journal.
This commit removes the use of `k8s_create_pod()` and modifies `assert_pod_fail()`
to fail as soon as the pod enters a failed state.
All failing pods end up in one of the following states:
- CrashLoopBackOff
- ImagePullBackOff
The function now polls the pod's state every 5 seconds to check for these conditions.
If the pod enters a failed state, the function immediately returns 0. If the pod
does not reach a failed state within 120 seconds, it returns 1.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This PR removes some qemu information which is not longer valid as
this is referring to the tests repository and to kata 1.x.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
rename the task_service to service, in order to
incopperate with the following added sandbox
services.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
In order to make different from sandbox request/response, this commit
changed the task request/response to TaskRequest/TaskResponse.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Since the wait_vm would be called before calling stop_vm,
which would take the reader lock, thus blocking the stop_vm
getting the writer lock, which would trigge the dead lock.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Since the block_on would block on the current thread
which would prevent other async tasks to be run on this
worker thread, thus change it to use the async task for
this task.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
- Reflect the need to update the versions in the Helm Chart
- Add the lock branch instruction
- Add clarity about the permissions needed to complete tasks
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
* Clarifies instructions for k0s.
* Adds kata-deploy step for each cluster type.
* Removes the old kata-deploy-stable step for vanilla k8s.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This PR introduces support for selectively compiling Dragonball in
runtime-rs. By default, Dragonball will continue to be compiled into
the containerd-shim-kata-v2 executable, but users now have the option
to disable Dragonball compilation.
Fixes#10310
Signed-off-by: sidney chang <2190206983@qq.com>
Now that the issue with handling loop devices has been resolved,
this commit re-enables the guest-pull-image tests for `qemu-coco-dev`.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Timeouts occur (e.g. `create_container_timeout` and `wait_time`)
when using qemu-coco-dev.
This commit increases these timeouts for the trusted image storage
test cases
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
If the host running the tests is different from the host where the cluster is running,
the *_loop_device() functions do not work as expected because the device is created
on the test host, while the cluster expects the device to be local.
This commit ensures that all commands for the relevant functions are executed via exec_host()
so that a device should be handled on a cluster node.
Additionally, it modifies exec_host() to return the exit code of the last executed command
because the existing logic with `kubectl debug` sometimes includes unexpected characters
that are difficult to handle. `kubectl exec` appears to properly return the exit code for
a given command to it.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Creating and deleting a node debugger pod for every `exec_host()`
call is inefficient.
This commit changes the test suite to create and delete the pod
only once, globally.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This commit addresses an issue with handling loop devices
via a node debugger due to restricted privileges.
It runs a pod with full privileges, allowing it to mount
the host root to `/host`, similar to the node debugger.
This change enables us to run tests for trusted image storage
using the `qemu-coco-dev` runtime class.
Fixes: #10133
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Add cdi devices including ContainerDevice definition and
annotation_container_device method to annotate vfio device
in OCI Spec annotations which is inserted into Guest with
its mapping of vendor-class and guest pci path.
Fixes#10145
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
We need vfio device's properties device, vendor and
class, but we can only get property device and vendor.
just extend it with class is ok.
Fixes#10145
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The kata webhook requires a configmap to define what runtime class it
should set for the newly created pods. Additionally, the configmap
allows others to modify the default runtime class name we wish to set
(in case the handler is kata but the name of the runtimeclass is
different).
Finally, this PR changes the webhook-check to compare the runtime of the
newly created pod against the specific runtime class in the configmap,
if said confimap doesn't exist, then it will default to "kata".
Signed-off-by: Martin <mheberling@microsoft.com>
This PR increases the timeout to run k8s tests for Kata CoCo TDX
to avoid the random failures of timeout.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
As we don't have any CI, nor maintainer to keep ACRN code around, we
better have it removed than give users the expectation that it should or
would work at some point.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This change technically affects the path for enabled guest selinux as well,
however since this is not implemented in runtime-rs anyway nothing should
break. When guest selinux support is added this change will come handy.
Signed-off-by: Pavel Mores <pmores@redhat.com>
If guest selinux is off the runtime has to ensure that container OCI spec
contains no selinux labels for the container rootfs and process. Failure
to do so causes kata agent to try and apply the labels which fails since
selinux is not enabled in guest, which in turn causes container launch
to fail.
This is largely inspired by golang runtime(*) with a slight deviation
in ordering of checks. This change simply checks the disable_guest_selinux
config setting and if it's true it clears both rootfs and process label if
necessary. Golang runtime, on the other hand, seems to first check if
process label is non-empty and only then it checks the config setting,
meaning that if process label is empty the rootfs label is not reset
even if it's non-empty. Frankly, this looks like a potential bug though
probably unlikely to manifest since it can be assumed that the labels are
either both empty, or both non-empty.
(*) 4fd4b02f2e/src/runtime/virtcontainers/kata_agent.go (L1005)
Signed-off-by: Pavel Mores <pmores@redhat.com>
In order to handle the setting we have to first parse it and make its
value available to the rest of the program.
The yes() function is added to comply with serde which seems to insist
on default values being returned from functions. Long term, this is
surely not the best place for this function to live, however given that
this is currently the first and only place where it's used it seems
appropriate to put it near its use. If it ends up being reused elsewhere
a better place will surely emerge.
Signed-off-by: Pavel Mores <pmores@redhat.com>
Azure internal mirrors for Ubuntu 20.04 have gone awry, leading to a
situation where dependencies cannot be installed (such as
libdevmapper-dev), blocking then our CI.
Let's bump the runners to 22.04 regardless, even knowing it'll cause an
issue with the runk tests, as the agent check tests are considered more
crucial to the project at this point.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
kata-shim was not reporting `inactive_file` in memory stat.
This memory is deducted by containerd when calculating the size of container working set, as it can be paged out by the operating
system under memory pressure. Without reporting `inactive_file`, containerd will over report container memory usage.
[Here](https://github.com/containerd/containerd/blob/v1.7.22/pkg/cri/server/container_stats_list_linux.go#L117) is where containerd
deducts `inactive_file` from memory usage.
Note that kata-shim correctly reports `total_inactive_file` for cgroup v1, but this was not implemented for cgroup v2.
This commit:
- Adds code in kata-shim to report "inactive_file" memory for cgroup v2
- Implements reporting of all available cgroup v2 memory stats to containerd
- Uses defensive coding to avoid assuming existence of any memory.stat fields
The list of available cgroup v2 memory stats defined by containerd can be found
[here](https://pkg.go.dev/github.com/containerd/cgroups/v2/stats#MemoryStat).
Fixes#10280
Signed-off-by: Alex Man <alexman@stripe.com>
Yq installation shouldn't force to use sudo in case yq is already installed in correct version.
Signed-off-by: Pawel Proskurnicki <pawel.proskurnicki@intel.com>
This patch adds support to call kata agents SetPolicy
API. Also adds tests for SetPolicy API using agent-ctl.
Fixes#9711
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
The `deploy_k0s` and `deploy_k3s` kubectl installs aren't failing
yet, but let get ahead of this and bump them as well
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This commit adds a step called `Rebase atop of the latest target branch`
to the job named `build-asset-boot-image-se` which can test the PR properly.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
agent built with policy feature initializes the policy engine using a
policy document from a default path, which is installed & linked during
UVM rootfs build. This commit adds support to provide a default agent
policy as environment variable.
This targets development/testing scenarios where kata-agent
is wanted to be started as a local process.
Fixes#10301
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
The following changes have been made:
- Remove unnecessary `sudo`
- Add an error message where an incorrect host key document is used
- Add a missing artifact `kernel-confidential-modules`
- Make a variable `kernel_version` and replace it with relevant hits
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
We've added s390x test container image, so add support
to use them based on the arch the test is running on
Fixes: #10302
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
fixuop
This commit brings some public parts of image pulling test series like
encrypted image pulling, pulling images from authenticated registry and
image verification. This would help to reduce the cost of maintainance.
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Close#8120
**Case 1**
Create a pod from an unsigned image, on an insecureAcceptAnything
registry works.
Image: quay.io/prometheus/busybox:latest
Policy rule:
```
"default": [
{
"type": "insecureAcceptAnything"
}
]
```
**Case 2**
Create a pod from an unsigned image, on a 'restricted registry' is
rejected.
Image: ghcr.io/confidential-containers/test-container-image-rs:unsigned
Policy rule:
```
"quay.io/confidential-containers/test-container-image-rs": [
{
"type": "sigstoreSigned",
"keyPath": "kbs:///default/cosign-public-key/test"
}
]
```
**Case 3**
Create a pod from a signed image, on a 'restricted registry' is
successful.
Image: ghcr.io/confidential-containers/test-container-image-rs:cosign-signed
Policy rule:
```
"ghcr.io/confidential-containers/test-container-image-rs": [
{
"type": "sigstoreSigned",
"keyPath": "kbs:///default/cosign-public-key/test"
}
]
```
**Case 4**
Create a pod from a signed image, on a 'restricted registry', but with
the wrong key is rejected
Image:
ghcr.io/confidential-containers/test-container-image-rs:cosign-signed-key2
Policy:
```
"ghcr.io/confidential-containers/test-container-image-rs": [
{
"type": "sigstoreSigned",
"keyPath": "kbs:///default/cosign-public-key/test"
}
]
```
**Case 5**
Create a pod from an unsigned image, on a 'restricted registry' works
if enable_signature_verfication is false
Image: ghcr.io/kata-containers/confidential-containers:unsigned
image security enable: false
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Add two parameters for enabling cosign signature image verification.
- `enable_signature_verification`: to activate signature verification
- `image_policy`: URI of the image policy
config
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
new version of the anyhow crate has changed the backtrace capture thus
unit tests of kata-agent that compares a raised error with an expected
one would fail. To fix this, we need only panics to have backtraces,
thus set `RUST_BACKTRACE=1` and `RUST_LIB_BACKTRACE=0` for tests due to
document
https://docs.rs/anyhow/latest/anyhow/
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
Change pod runAsUser value of a Replication Controller after generating
the RC's policy, and verify that the RC pods get rejected due to this
change.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Change pod runAsUser value of a Job after generating the Job's policy,
and verify that the Job gets rejected due to this change.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Change pod runAsUser value of a Deployment after generating the
Deployment's policy, and verify that the Deployment fails due to
this change.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Poll/wait for pod termination instead of sleeping 2 minutes. This
change typically saves ~90 seconds in my test cluster.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
It's a prerequisite PR to make built-in vmm dragonball compilation
options configurable.
Extract TAP device-related code from dragonball's dbs_utils into a
separate library within the runtime-rs hypervisor module.
To enhance functionality and reduce dependencies, the extracted code
has been reimplemented using the libc crate and the ifreq structure.
Fixes#10182
Signed-off-by: sidney chang <2190206983@qq.com>
In the latest `s390-tools`, there has been update on how to
verify a secure boot image. A host key revocation list (CRL),
which was optinoal, now becomes mandatory for verification.
This commit updates the relevant scripts and documentation accordingly.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
If the CI platform being tested doesn't support yet the prometheus
container image:
- Use busybox instead of prometheus.
- Skip the test cases that depend on the prometheus image.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR fixes the indentation in the cri containerd tests as we
have in several places a misalignment in the script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Remove the workaround for #9928, now that genpolicy is able to
convert user names from container images into the corresponding
UIDs from these images.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Remove the recently added default UID/GID values, because the genpolicy
design is to initialize those fields before this new code path gets
executed.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Some container images are configured such that the user (and group)
under which their entrypoint should run is not a number (or pair of
numbers), but a user name.
For example, in a Dockerfile, one might write:
> USER 185
indicating that the entrypoint should run under UID=185.
Some images, however, might have:
> RUN groupadd --system --gid=185 spark
> RUN useradd --system --uid=185 --gid=spark spark
> ...
> USER spark
indicating that the UID:GID pair should be resolved at runtime via
/etc/passwd.
To handle such images correctly, read through all /etc/passwd files in
all layers, find the latest version of it (i.e., the top-most layer with
such a file), and, in so doing, ensure that whiteouts of this file are
respected (i.e., if one layer adds the file and some subsequent layer
removes it, don't use it).
Signed-off-by: Hernan Gatta <hernan.gatta@opaque.co>
Disabling the UID Policy rule was a workaround for #9928. Re-enable
that rule here and add a new test/CI temporary workaround for this
issue. This new test workaround will be removed after fixing #9928.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Change quay.io/prometheus/busybox to quay.io/prometheus/prometheus in
this test. The prometheus image will be helpful for testing the future
fix for #9928 because it specifies user = "nobody".
Also, change:
sh -c "ls -l /"
to:
echo -n "readinessProbe with space characters"
as the test readinessProbe command line. Both include a command line
argument containing space characters, but "sh -c" behaves differently
when using the prometheus container image (causes the readinessProbe
to time out, etc.).
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR removes the remove_img variable in the metrics common script
as it is not being used.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR decreases the number of iterations in the kubernetes soak test
as this is already taking more than 2 hours for the kata coco ci
stability.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
kata-agent incorrectly reports CPU time for cgroup v2, causing 1000x underreporting.
For cgroup v2, kata-agent reads the cpu.stat file, which reports the time consumed by the processes in the cgroup in µs.
However, there was a bug in kata-agent where it returned this value in µs without converting it to ns.
This commit adds the necessary µs to ns conversion for cgroup v2, aligning it with v1 behavior and kata-shim's expectations.
This fixes#10278
Signed-off-by: Alex Man <alexman@stripe.com>
With an older version of image-rs, we were getting the following error:
```
Message: failed to create containerd task: failed to create shim task: failed to handle layer: failed to get decrypt key no suitable key found for decrypting layer key:
```
However, with the version of image-rs we are bumping to, the error comes
as:
```
Message: failed to create containerd task: failed to create shim task: failed to handle layer: failed to get decrypt key
Caused by:
no suitable key found for decrypting layer key:
keyprovider: failed to unwrap key by ttrpc
```
Due to this change, I'm splitting the check in two different ones.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
ITA stands for Intel Trust Authority, which is in the process to being
renamed to ITTS (Intel Tiber Trust Services).
Proper ITA / ITTS support on Trustee was finished as part of:
* 6f767fa15f
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
ITA stands for Intel Trust Authority, which is in the process to being
renamed to ITTS (Intel Tiber Trust Services).
As we've bumped guest-components on trustee, let's make sure we also
bump image-rs to the commit that brings ITA support in:
* 1db6c3a876
The reason we need to bump the dependency here is to avoid kbs_protocol
mismatch between the version used by the agent and the trustee one.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
In the process of switching the TDX CI machine we've noticed that
`hostname -i` in one of the machines returns an one and only IP address,
while in another machine it returns a full list of IPs.
As we're only interested in the first one, let's adapt the code to
always return the first one.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Since CDH/attestation related processes and its dependencies are not fully
available, the setup fails to start kata-agent as local process. This
fix removes these procs to prevent kata-agent from trying to start them.
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
The test setup starts kata-agent as a local process without the
UVM. The agent policy initialization fails due to missing policy
document at `/etc/kata-opa/default-policy.rego`. The fix
- installs a relaxed `allow-all.rego` policy document
- cleans up the install during exit
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
This commit introduces test cases for testing
CopyFile API using kata-agent-ctl with improved command
semantics and handling.
- copy a file to /run/kata-containers
- copy symlink to /run/kata-containers
- copy directory to /run/kata-containers
- copy file to /tmp
- copy large file to /run/kata-containers
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
In the existing implementation for the CopyFile subcommand,
- cmd line argument list is too long, including various metadata information.
- in case of a regular file, passing the actual data as bytes stream adds to the size and complexity of the input.
- the copy request will fail when the file size exceeds that of the allowed ttrpc max data length limit of 4Mb.
This change refactors the CopyFile handler and modifies the input to a known 'source' 'destination' syntax.
Fixes#9708
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
This PR increases the timeout to wait that the deployment for the soak
stability test is ready in order to avoid random failures saying that
the deployment is not ready yet.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
It will panic when users do GPU vfio passthrough with cdi in runtime.
The root cause is that CustomSpec.Annotations is nil when new element
added.
To address this issue, initialization is introduced when it's nil.
Fixes#10266
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Move device code with nvdimm driver to nvdimm_device_handler, including
nvdimm device and pmem device.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Removed the `StorageHandlerManager` struct and its associated implementations and
introduced a type alias `StorageHandlerManager` for `HandlerManager` to simplify the code.
The new type alias maintains the same functionality while reducing redundancy.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Introduced `HandlerManager` struct to manage registered handlers, which will be used to storage and device management for kata-agent.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
I know right now we're always passing a value for that, but this doesn't
really have to be set unless attestation is used. Thus, let's also omit
it in case it's empty.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This is a quick and simple pre-req for supporting initData, which will
take advantage of the mrconfigid in the TDX case.
While already adding mrconfigid, which is hardcoded empty right now,
let's do the same for mrowner and mrownerconfig, and leave it prepared
for future expansions.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
The reason we're relying on yet another function to do so is because the
TDX object will be used in its qom / qapi json format.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This commit enables running tests for kata agent apis.
The 'api-tests' directory will contain bats test files for
individual APIs.
Fixes#10269
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
This PR updates the machine learning tests references or urls for the
openVINO and oneDNN scripts as currently they are refering to a different
performance benchmark.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
enable CI to add test cases for testing kata-agent APIs. This commit
introduces:
- a workflow to run tests
- setup scripts to prepare the test environment
Fixes#10262
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
As kata-cleanup will only call `reset_runtime()`, there's absolutely no
need to export the other set of environment variables in its yaml file.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
metrics tests sometimes fail with kata components still running.
sending SIGKILL and waiting for the processes to reap.
Fixes#8651
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
This error is specific to SNP platforms, so let's make sure we only
error this out when an SNP platform is used.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Add CONFIG_PAGE_REPORTING, CONFIG_BALLOON_COMPACTION and
CONFIG_VIRTIO_BALLOON to dragonball-experimental configs to open
dragonball function and free page reporting function.
Signed-off-by: Hui Zhu <teawater@antgroup.com>
Under normal circumstances, the virtual machine only requests memory
from the host and does not actively release it back to host when it is
no longer needed, leading to a waste of memory resources.
Free page reporting is a sub-feature of virtio-balloon. When this
feature is enabled, the Linux guest kernel will send information about
released pages to dragonball via virtio-balloon, and dragonball will
then release these pages.
This commit adds an option enable_balloon_f_reporting to runtime-rs.
When this option is enabled, runtime-rs will insert a virtio-balloon
device with the f_reporting option enabled during the Dragonball virtual
machine startup.
Signed-off-by: Hui Zhu <teawater@antgroup.com>
...or by using a binary with additional suffix.
This allows having multiple versions of nydus-overlayfs installed on the
host, telling nydus-snapshotter which one to use while still detecting
Nydus is used.
Signed-off-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
This will help to avoid code duplication on what's needed on the helm
and non-helm cases.
The reason it's not been added as part of the commit which adds the
post-delete hook is simply for helping the reviewer (as the diff would
be less readable with this change).
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Instead of using a lifecycle.preStop hook, as done when we're using
using the helm chat, let's add a post-delete hook to take care of
properly cleaning up the node during when uninstalling kata-deploy.
The reason why the lifecyle.preStop hook would never work on our case is
simply because each helm chart operation follows the Kuberentes
"declarative" approach, meaning that an operation won't wait for its
previous operation to successfully finish before being called, leading
to us trying to access content that's defined by our RBAC, in an
operation that was started before our RBAC was deleted, but having the
RBAC being deleted before the operation actually started.
Unfortunately this hook brings in some code duplicatioon, mainly related
to the RBAC parts, but that's not new as the same happens with our
deamonset.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
On commit 51690bc157 we switched the installation from kubectl to helm
and used its `--wait` expecting the execution would continue when all
kata-deploy Pods were Ready. It turns out that there is a limitation on
helm install that won't wait properly when the daemonset is made of a
single replica and maxUnavailable=1. In order to fix that issue, let's
revert the changes partially to keep using kubectl and waitForProcess
to the exection while Pods aren't Running.
Fixes#10168
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Move `unseal_env` and `secure_mount` functions on the global `CDH_CLIENT` instance to access the CDH client.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Introduced a global `CDH_CLIENT` instance to hold the cdh client and
implemented `init_cdh_client` function to initialize the cdh client if not already set.
Fixes: #10231
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This PR removes the metrics report which is not longer being used
in Kata Containers.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The following steps are required for enabling KBS:
- Set environment variables `KBS` and `KBS_INGRESS`
- Uninstall and install `kbs-client`
- Deploy KBS
This commit adds the above stpes to the existing workflow
for `qemu-coco-dev`.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
To deploy KBS on s390x, the environment variable `IBM_SE_CREDS_DIR`
must be exported, and the corresponding directory must be created.
This commit enables KBS deployment for `qemu-coco-dev`, in addition
to the existing `qemu-se` support on the platform.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
- Added `driver_types` method to `StorageHandler` trait to return driver
types managed by each handler.
- Implemented driver_types method for all storage handlers.
- Updated `STORAGE_HANDLERS` initialization to use `driver_types` for
handler registration.
Fixes: #10242
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
- Updated the `add_handler` function in `StorageHandlerManager` to accept a slice of IDs (`&[&str]`) instead of a single ID (`&str`).
This change allows a single handler to be registered for multiple storage device types.
- Refactored calls to `add_handler` in `Storage` of kata-agent to use the new function, passing arrays of storage drivers instead of single driver.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This reverts commit 704da86e9b, as the
tests never became stable to run.
This was discussed and agreed with the maintainer.
Conflicts:
.github/workflows/basic-ci-amd64.yaml
tests/integration/stdio/gha-run.sh
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This PR increases the time to run the Kata CoCo stability tests as
this tests are design to run for more than 2 hours.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR enables the k8s soak stability test to run on the weekly
Kata CoCo stability CI.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
It's simply easier if we just use /host/opt/kata instead in our scripts,
which will simplify a lot the logic of adding an INSTALLATION_PREFIX
later on.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
As we build our binaries with the `/opt/kata` prefix, that's the value
of $dest_dir.
Later in thise series it'll become handy, as we'll introduce a way to
install the Kata Containers artefacts in a different location.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This PR updates the firecracker version to 1.8.0 which includes the
following changes:
- Added ACPI support to Firecracker for x86_64 microVMs. Currently, we pass ACPI tables with information about the available vCPUs, interrupt controllers, VirtIO and legacy x86 devices to the guest. This allows booting kernels without MPTable support. Please see our kernel policy documentation for more information regarding relevant kernel configurations.
- Added support for the Virtual Machine Generation Identifier (VMGenID) device on x86_64 platforms. VMGenID is a virtual device that allows VMMs to notify guests when they are resumed from a snapshot. Linux includes VMGenID support since version 5.18. It uses notifications from the device to reseed its internal CSPRNG. Please refer to snapshot support and random for clones documention for more info on VMGenID. VMGenID state is part of the snapshot format of Firecracker. As a result, Firecracker snapshot version is now 2.0.0.
- Changed T2CL template to pass through bit 27 and 28 of MSR_IA32_ARCH_CAPABILITIES (RFDS_NO and RFDS_CLEAR) since KVM consider they are able to be passed through and T2CL isn't designed for secure snapshot migration between different processors.
- Avoid setting kvm_immediate_exit to 1 if are already handling an exit, or if the vCPU is stopped. This avoids a spurious KVM exit upon restoring snapshots.
- Changed T2S template to set bit 27 of MSR_IA32_ARCH_CAPABILITIES (RFDS_NO) to 1 since it assumes that the fleet only consists of processors that are not affected by RFDS.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This commit includes a fix for pulling an image on platforms that do not
support xattr.
Some platforms/file-systems do not support xattrs, this would make the
image pull fail because of failing to set xattr. This commit will check
whether the target path supports xattr. If yes, the unpacking will
maintain xattrs; if not, it will not set xattrs.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
slog's is_enabled() is documented as:
- "best effort", and
- Sometime resulting in false positives.
Use AGENT_CONFIG.log_level.as_usize() instead, to avoid those false
positives.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
22.04 is the default today:
23da668261/README.md
Being more specific will avoid unexpected errors when Github updates the
default.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Also keeps the Rust installation step even though it's preinstalled, so that we
use the version specified in versions.yaml.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This PR adds the oneDNN benchmark information to the machine
learning metrics README.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
GH-9592 addressed a bug in a previous version of the AKS Mariner host
kernel that blocked the CH v39 upgrade. This bug has now been fixed so
we undo that PR.
Note we also specify a different OCI version for Mariner as it differs
from Ubuntu's.
Fixes: #9594
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This PR fixes the entry or use of the ci weekly GHA workflow
to run properly the weekly k8s tests.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
PhysicalEndpoint unbinds its VF interface and rebinds it as a VFIO device,
then cold-plugs the VFIO device into the guest kernel.
When `cold_plug_vfio` is set to "no-port", cold-plugging the VFIO device
will fail.
This change checks if `cold_plug_vfio` is enabled before creating PhysicalEndpoint
to avoid unnecessary VFIO rebind operations.
Fixes: #10162
Signed-off-by: Lei Huang <leih@nvidia.com>
This reverts commit 41b7577f08.
We were seeing a lot of issues in the TDX CI of the nature:
"Error: failed to create containerd container: create instance
470: object with key "470" already exists: unknown"
With the TDX CI, we moved to having the nydus snapsotter pre-installed.
Essentially the `deploy-snapshotter` step was performed once before any
actual CI runs.
We were seeing failures related to the error message above.
On reverting this change, we are no longer seeing errors related to
"key exists" with the TDX CI passing now.
The change reverted here is related to downloading incomplete images, but this
seems to be messing up TDX CI.
It is possible to pass --snapshotter to `ctr image check` but that does
not seem to have any effect on the data set returned.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This patch re-generates the client code for Cloud Hypervisor v41.0.
Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by openapi-generator.
Fixes: #10203
Signed-off-by: Bo Chen <chen.bo@intel.com>
This PR adds the OpenVINO benchmark general information into the
machine learning README metrics information.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
In certain environments (e.g., those with lower performance), `k8s_create_pod()`
may require additional wait time, especially when dealing with large images.
Since `k8s_wait_pod_be_ready()` — which is called by `k8s_create_pod()` — already
accepts `wait_time` as a second argument, it makes sense to introduce `wait_time`
to `k8s_create_pod()` and propagate it to the callee.
This commit adds `wait_time` to `k8s_create_pod()` as the 2nd (optional) argument.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Some of the tests call set_metadata_annotation() for updating the kernel
parameters. For `kata-qemu-se`, repack_secure_image() is called which is
defined in `lib_se.sh` and sourced by `confidential_kbs.sh`.
This commit ensures that the function call chain for the relevant
`KATA_HYPERVISOR` is properly handled.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This commit add a row for `cdh_api_timeout` to the agent options in
how-to-set-sandbox-config-kata.md.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This commit allows `cdh_api_timeout` to be configured from the configuration file.
The configuration is commented out with specifying a default value (50s) because
the default value is configured in the agent.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
therwise we may end up running into unexpected issues when calling the
cleanup option, as the same checks would be done, and files could end up
being copied again, overwriting the original content which was backked
up by the install option.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This commit updates CDH_API_TIMEOUT to use AGENT_CONFIG.cdh_api_timeout
and changes it from a `const` to `lazy_static` to accommodate runtime-determined values.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
To make the `cdh_api_timeout` variable configurable, it has been added to
the `AgentConfig` structure.
This change includes storing the variable as a `time::Duration` type and
generalizing the existing `hotplug_timeout` code to handle both timeouts.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This PR adds the pod deployment yaml for soak test which is part
of the stability k8s tests.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
* genpolicy: deny UpdateEphemeralMountsRequest
Deny UpdateEphemeralMountsRequest by default, because paths to
critical Guest components can be redirected using such request.
Signed-off-by: Dan Mihai <Daniel.Mihai@microsoft.com>
This PR adds a kubernetes parallel test that will launch multiple replicas
from a kubernetes deployment and we will iterate this multiple times to
verify that we are able to do this using CoCo Kata. This test will be
part of the CoCo Kata stability CI.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This partially reverts commit 94b3348d3c,
as there's more work needed in order to have this one done in a robust
way, and we are taking the safer path of reverting for now, and adding
it back as soon as the release is cut out.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This partially reverts commit 51690bc157,
as there's more work needed in order to have this one done in a robust
way, and we are taking the safer path of reverting for now, and adding
it back as soon as the release is cut out.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This reverts commit 1221ab73f9, as there's
more work needed in order to have this one done in a robust way, and we
are taking the safer path of reverting for now, and adding it back as
soon as the release is cut out.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This PR adds the Kata CoCo Stability workflow that will setup the
environment to run the k8s tests on a non-tee environment.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the Kata CoCo stability weekly yaml that will trigger
weekly the k8s stability tests.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The deploy-kbs.sh script generates the kbs.key that's used to install
KBS. This same file is used lately by kbs-client to authenticate. This ensures
that the file was created, otherwise fail.
Another problem solved here is that on bare-metal machines the key doesn't survive
a reboot as it is created in a temporary directory (/tmp/trustee). So let's save
the file to a non-temporary location.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Neither CRI-O nor containerd requires that, and removing such symlinks
makes everything less intrusive from our side.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It's already being used with CRi-O, let's simplify what we do and also
use this for containerd, which will allow us to do further cleanups in
the coming patches.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
"gha-run.sh" requires a `run` argument in order to run the tests, which
seems to be forgotten when the test was added.
This PR needs to get merged before the test can successfully run.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
I assume the PR that introduced this was based on an older version of
yq, and as the test couldn't run before it got merged we never noticed
the error.
However, this test has been failing for a reasonable amount of time,
which makes me think that we either need a maintainer for it, or just
remove it completely, but that's a discussion for another day.
For now, let's make it, at least, run.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR adds the k8s stability Kata CoCo GHA workflow to run weekly
the k8s stability tests.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This reverts commit d35320472c.
Although the commit in question does solve an issue related to the usage
of busybox from docker.io, as it's reasonably easy to hit the rate
limit, the commit also brings in functionalities that are causing issues
in, at least, the TDX CI, such as:
```sh
[2024-08-16T16:03:52Z INFO actix_web::middleware::logger] 10.244.0.1 "POST /kbs/v0/attest HTTP/1.1" 401 259 "-" "attestation-agent-kbs-client/0.1.0" 0.065266
[2024-08-16T16:03:53Z INFO kbs::http::attest] Auth API called.
[2024-08-16T16:03:53Z INFO actix_web::middleware::logger] 10.244.0.1 "POST /kbs/v0/auth HTTP/1.1" 200 74 "-" "attestation-agent-kbs-client/0.1.0" 0.000169
[2024-08-16T16:03:54Z INFO kbs::http::attest] Attest API called.
[2024-08-16T16:03:54Z INFO verifier::tdx] Quote DCAP check succeeded.
[2024-08-16T16:03:54Z INFO verifier::tdx] MRCONFIGID check succeeded.
[2024-08-16T16:03:54Z INFO verifier::tdx] CCEL integrity check succeeded.
[2024-08-16T16:03:54Z ERROR kbs::http::error] Attestation failed: Verifier evaluate failed: TDX Verifier: failed to parse AA Eventlog from evidence
Caused by:
at least one line should be included in AAEL
```
Let's revert this for now, and then once we get this one fixed on
trustee side we'll update again.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add bind mounts for volumes defined by docker container images, unless
those mounts have been defined in the input K8s YAML file too.
For example, quay.io/opstree/redis defines two mounts:
/data
/node-conf
Before these changes, if these mounts were not defined in the YAML file
too, the auto-generated policy did not allow this container image to
start.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Improved error handling to provide clearer feedback on request failures.
For example:
Improve createcontainer request timeout error message from
"Error: failed to create containerd task: failed to create shim task:context deadline exceed"
to "Error: failed to create containerd task: failed to create shim task: CreateContainerRequest timed out: context deadline exceed".
Fixes: #10173 -- part II
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
For the rare cases where containerd_conf_file does not exist, cp could fail
and let the pod in Error state. Let's make it a little bit more robust.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
Install luks-encrypt-storage script by guest-components. So that we can maintain a single source and prevent synchronization issues.
Fixes: #10173 -- part I
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Container/Sandbox clean up should not fail if root FS is not mounted.
This PR handles EINVAL errors when umount2 is called.
Fixes: #10166
Signed-off-by: Silenio Quarti <silenio_quarti@ca.ibm.com>
Instead of deploying and removing the snapshotter on every single run,
let's make sure the snapshotter is always deploy on the TDX case.
We're doing this as an experiment, in order to see if we'll be able to
reduce the failures we've been facing with the nydus snapshotter.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The `exec_host()` function often fails to capture the output of a given command
because the node debugger pod is prematurely terminated. To address this issue,
the function has been refactored to ensure consistent output capture by adjusting
the `kubectl debug` process as follows:
- Keep the node debugger pod running
- Wait until the pod is fully ready
- Execute the command using `kubectl exec`
- Capture the output and terminate the pod
This commit refactors `exec_host()` to implement the above steps, improving its reliability.
Fixes: #10081
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This reverts commit 8d9bec2e01, as it
causes issues in the operator and kata-deploy itself, leading to the
node to be NotReady.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
I can't set up loop device with `exec_host`, which the command is
necessary for qemu-coco-dev. See issue #10133.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Update error message in pulling image encrypted to "failed to get decrypt key no suitable key found for decrypting layer key".
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Introduces `secure_mount` API in the cdh. It includes:
- Adding the `SecureMountServiceClient`.
- Implementing the `secure_mount` function to handle secure mounting requests.
- Updating the confidential_data_hub.proto file to define SecureMountRequest and SecureMountResponse messages
and adding the SecureMountService service.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
renames the sealed_secret.proto file to confidential_data_hub.proto and
updates the corresponding API namespace from sealed_secret to confidential_data_hub.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
add tests for guest pull with configured timeout:
1) failed case: Test we cannot pull a large image that pull time exceeds a short creatcontainer timeout(10s) inside the guest
2) successful case: Test we can pull a large image inside the guest with increasing createcontainer timeout(120s)
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
add tests for pull images in the guest using trusted storage:
1) failed case: Test we cannot pull an image that exceeds the memory limit inside the guest
2) successful case: Test we can pull an image inside the guest using
trusted ephemeral storage.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This PR adds a k8s stability test that will be part of the CoCo Kata
stability tests that will run weekly.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR disables the k8s file volume test as we are having random failures
in multiple GHA CIs mainly because the exec_host function sometimes
does it not work properly.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds kubernetes stress-ng tests as part of the stability testing
for kata.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Hooks are executed on the host, so we don't expect to run hooks and thus
require that no hook paths are set.
Additional Kernel modules expand the attack surface, so require that
none are set. If a use case arises, modules should be allowlisted via
settings.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
CopyFile is invoked by the host's FileSystemShare.ShareFile function,
which puts all files into directories with a common pattern. Copying
files anywhere else is dangerous and must be prevented. Thus, we check
that the target path prefix matches the expected directory pattern of
ShareFile, and that this directory is not escaped by .. traversal.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
when use debug console, the shell run in child process may not be
exited, in some scenes.
eg. directly Ctrl-C in the host to terminate the kata-runtime process,
that will block the task handling the console connection，while waiting
for the child to exit.
Signed-off-by: soulfy <liukai254@jd.com>
Only do the checking in case the tarball was not explicitly passed by
the user. We have no control of what's passed and we cannot expect that
all the files are going to be under /opt.
Fixes: #10147
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Currently, setting `io.containerd.cri.runtime-handler` annotation in
the yaml is not necessary for pulling images in the guest. All TEE
hypervisors are already running tests with guest-pulling enabled.
Therefore, we can remove some duplicate tests and re-enable the
guest-pull test for running different runtime pods at the same time.
While considering to support different containerd version, I recommend
to keep setting "io.containerd.cri.runtime-handler".
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Initialize the trusted stroage when the device is defined
as "/dev/trusted_store" with shell script as first step.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
After enable secure storage integrity for trusted storage, the initialize
time will take more times, the default value will be NOT enabled but add this config to
allow the user to enable if they care more strict security.
Fixes: #8142
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
This PR updates the ubuntu image for stress Dockerfile. The main purpose
is to have a more updated image compared with the one that is in libpod
which has not been updated in a while.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
- At the moment we aren't factoring in the kata version on our caches,
so it means that when we bump this just before release, we don't
rebuilt components that pull in the VERSION content, so the release build
ends up with incorrect versions in it's binaries
Fixes: #10092
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The jobs are all executed on ubuntu-22.04 so it's invariant and
can be removed from the matrix (this will shrink the jobs names).
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Created the run-k8s-tests-on-amd64.yaml which is a merge of
run-k8s-tests-on-garm.yaml and run-k8s-tests-with-crio-on-garm.yaml
ps: renamed the job from 'run-k8s-tests' to 'run-k8s-tests-on-amd64' to
it is easier to find on Github UI and be distinguished from s390x,
ppc64le, etc...
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Switch to Github managed runners just like the run-k8s-tests-on-garm
workflow.
See: #9940
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Switched to Github managed runners. The instance_type parameter was
removed and K8S_TEST_HOST_TYPE is set to "all" which combine the
tests of "small" and "normal". This way it will reduze to half of
the jobs.
See: #9940
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
These new "kata-deploy" and "cleanup" actions are equivalent to
"kata-deploy-garm" "cleanup-garm", respectively, and should be
used on the workflows being migrated from GARM to
Github's managed runners.
Eventually "kata-deploy-garm" and "cleanup-garm" won't be used anymore
then we will be able to remove them.
See: #9940
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This PR updates the image that we are using in the kubectl debug command
as part of the exec host function, as the current alpine image does not
allow to create a temporary file for example and creates random kubernetes
failures.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
A recent fix should resolve some the issues seen earlier with clh
with the go runtime. Enabling this test to check if the issue is still
seen.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Our code for handling images being pulled inside the guest relies on a
containerType ("sandbox" or "container") being set as part of the
container annotations, which is done by the CRI Engine being used, and
depending on the used CRI Engine we check for a specfic annotation
related to the image-name, which is then passed to the agent.
However, when running kata-containers without kubernetes, specifically
when using `nerdctl`, none of those annotations are set at all.
One thing that we can do to allow folks to use `nerdctl`, however, is to
take advantage of the `--label` flag, and document on our side that
users must pass `io.kubernetes.cri.image-name=$image_name` as part of
the label.
By doing this, and changing our "fallback" so we can always look for
such annotation, we ensure that nerdctl will work when using the nydus
snapshotter, with kata-containers, to perform image pulling inside the
pod sandbox / guest.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Adding reset_cleanup to cleanup action so that it is done automatically
without the need to run yet another DS just to reset the runtime.
This is now part of the lifecycle hook when issuing kata-deploy.sh
cleanup
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Rather then modifying the kata-depoy scripts let's use Helm and
create a values.yaml that can be used to render the final templates
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
For easier handling of kata-deploy we can leverage a Helm chart to get
rid of all the base and overlays for the various components
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The kata containers hypervisior qemu configuration supports setting
block_device_aio="native", but the kata static build of qemu does
not add the linux aio feature.
The libaio-dev library is a necessary dependency for building qemu
with linux aio.
Fixes: #10130
Signed-off-by: Zhiwei Huang <ai.william@outlook.com>
Provides a test runner that generates a policy and validates it
with canned requests. The initial set of test cases is mostly for
illustration and will be expanded incrementally.
In order to enable both cross-compilation on Ubuntu test runners as well
as native compilation on the Alpine tools builder, it is easiest to
switch to the vendored openssl-src variant. This builds OpenSSL from
source, which depends on Perl at build time.
Adding the test to the Makefile makes it execute in CI, on a variety of
architectures. Building on ppc64le requires a newer version of the
libz-ng-sys crate.
Fixes: #10061
Signed-off-by: Markus Rudy <mr@edgeless.systems>
cargo clippy has two new warnings that need addressing:
- assigning_clones
These were fixed by clippy itself.
- suspicious_open_options
I added truncate(false) because we're opening the file for reading.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
After experimenting a little bit with those tests, they seem to be
passing on all the available TEE machines.
With this in mind, let's just enable them for those machines.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When re-enabling those we'll need a smart way to do so, as this limit of
20 workflows referenced is just ... weird.
However, for now, it's more important to add the jobs related to the new
platforms than keep the ones that are actively disabled.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we have new runners added, let's enable the builders so we can
prevent build failures happening after something gets merged.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit e9710332e7, as there
are now 2 arm64-builders (to be expanded to 4 really soon).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit c5dad991ce, as there
are now 2 arm64-builders (to be expanded to 4 really soon).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This parameter has been deprecated for a long time and QEMU 9.1.0 finally removes it.
Fixes: kata-containers#10112
Signed-off-by: Tom Dohrmann <erbse.13@gmx.de>
- Add --version flag to the genpolicy tool that prints the current
version
- Add version.rs.in template to store the version information
- Update makefile to autogenerate version.rs from version.rs.in
- Add license to Cargo.toml
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
This PR removes duplicated entries (vcpus count, and available memory),
from onednn and openvino results files.
Fixes: #10119
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
The issue is similar to #10011.
The root cause is that tty and stderr are set to true at same time in
containerd: #10031.
Fixes: #10081
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This PR encloses the search string for 'default_vcpus ='
and 'default_memory =' with double quotes in order to
parse the precise values, which are included in the kata
configuration file.
Fixes: #10118
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
1. Use the new value of AllowRequestsFailingPolicy after setting up a
new Policy. Before this change, the only way to enable
AllowRequestsFailingPolicy was to change the default Policy file,
built into the Guest rootfs image.
2. Ignore errors returned by regorus while evaluating Policy rules, if
AllowRequestsFailingPolicy was enabled. For example, trying to
evaluate the UpdateInterfaceRequest rules using a policy that didn't
define any UpdateInterfaceRequest rules results in a "not found"
error from regorus. Allow AllowRequestsFailingPolicy := true to
bypass that error.
3. Add simple CI test for AllowRequestsFailingPolicy.
These changes are restoring functionality that was broken recently by
commmit df23eb09a6.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Let's remove what we commented out, as publish manifest complains:
```
Created manifest list quay.io/kata-containers/kata-deploy-ci:kata-containers-latest
./tools/packaging/release/release.sh: line 146: --amend: command not found
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR updates the memory tests like fast footprint to use grep -F
instead of fgrep as this command has been deprecated.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds missing packages depenencies to install kbs cli in a fresh
new baremetal environment. This will avoid to have a failure when trying
to run install-kbs-client.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
We've done something quite similar for kata-deploy, but I've noticed we
forgot about the kata-manager counterpart.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It's been a reasonable time that we're not able to even build arm64
artefacts.
For now I am removing the builds as it doesn't make sense to keep
running failing builds, and those can be re-enabled once we have arm64
machines plugged in that can be used for building the stuff, and
maintainers for those machines.
The `arm-jetson-xavier-nx-01` is also being removed from the runners.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In #10096, a cleanup step for kata-deploy is removed by mistake.
This leads to a cleanup error in the following `Complete job` step.
This commit restores the removed step to resolve the current CI failure on s390x.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
As suggested in #9934, the following hooks have been introduced for s390x runners:
- ACTIONS_RUNNER_HOOK_JOB_STARTED
- ACTIONS_RUNNER_HOOK_JOB_COMPLETED
These hooks will perfectly replace the existing {pre,post}-action scripts.
This commit wipes out all GHA steps for s390x where the actions are triggered.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Rename k8s-exec-rejected.bats to k8s-policy-hard-coded.bats, getting
ready to test additional hard-coded policies using the same script.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Users of AUTO_GENERATE_POLICY=yes:
- Already tested *auto-generated* policy on any platform.
- Will be able to test *hard-coded* policy too on any platform, after
this change.
CI continues to test hard-coded policies just on the platforms listed
here, but testing those policies locally (outside of CI) on other
platforms can be useful too.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Since we can't find a homogeneous value for the resource/cgroup
management of multiple hypervisors, and we have decoupled the
env vars in the Makefile, we don't need the generic ones.
Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
To avoid overriding env vars when multiple hypervisors are
available, we add per-hypervisor vars for static resource
management and cgroups handling. We reflect that in the
relevant config files as well.
Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
Let's ensure at least 50% of the memory is used for /run, as systemd by
default forces it to be 10%, which is way too small even for very small
workloads.
This is only done for the rootfs-confidential image.
Fixes: kata-containers#6775
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
Signed-off-by: Wang, Arron <arron.wang@intel.com>
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.co
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It's better to check the container's status before
try to send signal to it. Since there's no need
to send signal to it when the container's stopped.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Since stop sandbox would be called in multi path,
thus it's better to set and check the sandbox's state.
Fixes: #10042
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Generate policy that validates each exec command line argument, instead
of joining those args and validating the resulting string. Joining the
args ignored the fact that some of the args might include space
characters.
The older format from genpolicy-settings.json was similar to:
"ExecProcessRequest": {
"commands": [
"sh -c cat /proc/self/status"
],
"regex": []
},
That format will not be supported anymore. genpolicy will detect if its
users are trying to use the older "commands" field and will exit with
a relevant error message in that case.
The new settings format is:
"ExecProcessRequest": {
"allowed_commands": [
[
"sh",
"-c",
"cat /proc/self/status"
]
],
"regex": []
},
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
It's time to delete the kata oci spec implemented just
for kata. As we have already done align OCI Spec with
oci-spec-rs.
Fixes#9766
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit aligns the OCI Spec implementation in runtime-rs
with the OCI Spec definitions and related operations provided
by oci-spec-rs. Key changes as below:
(1) Leveraged oci-spec-rs to align Kata Runtime OCI Spec with
the official OCI Spec.
(2) Introduced runtime-spec to separate OCI Spec definitions
from Kata-specific State data structures.
(3) Preserved the original code logic and implementation as
much as possible.
(4) Made minor code adjustments to adhere to Rust programming
conventions;
Fixes#9766
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Utilized oci-spec-rs to align OCI Spec structures
and data representations in runk with the OCI Spec.
Fixes#9766
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit aligns the OCI Spec used within agent-ctl
with the oci-spec-rs definition and operations. This
enhancement ensures that agent-ctl adheres to the latest
OCI standards and provides a more consistent and reliable
experience for managing container images and configurations.
Fixes#9766
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This commit transitions the data implementation for OCI Spec
from kata-oci-spec to oci-spec-rs. While both libraries adhere
to the OCI Spec standard, significant implementation details
differ. To ensure data exchange through TTRPC services, this
commit reimplements necessary data conversion logic.
This conversion bridges the gap between oci-spec-rs data and
TTRPC data formats, guaranteeing consistent and reliable data
transfer across the system.
Fixes#9766
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Add integration test that creates two bridge networks with nerdctl and
verifies that Kata container is brought up while passing the networks
created.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
For nerdctl and docker runtimes, network is hot-plugged instead of
cold-plugged. While this change was made in the runtime,
we did not have the agent waiting for the device to be ready.
On some systems, the device hotplug could take some time causing
the update_interface rpc call to fail as the interface is not available.
Add a watcher for the network interface based on the pci-path of the
network interface. Note, waiting on the device based on name is really
not reliable especially in case multiple networks are hotplugged.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Share a single test script variable for both:
- Allowing a command to be executed using Policy settings.
- Executing that command using "kubectl exec".
Fixes: #10014
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Hotunplugging memory is not guaranteed or even likely to work.
Nevertheless I'd really like to have this code in for tests and
observation. It shouldn't hurt, from experience so far.
Signed-off-by: Pavel Mores <pmores@redhat.com>
The bulk of this implementation are simple though tedious sanity checks,
alignment computations and logging.
Note that before any hotplugging, we query qemu directly for the current
size of hotplugged memory. This ensures that any request to resize memory
will be properly compared to the actual already available amount and only
necessary amount will be added.
Note also that we borrow checked_next_multiple_of() from CH implementation.
While this might look uncleanly it's just a rather temporary solution since
an equivalent function will apparently be part of std soon, likely the
upcoming 1.75.
Signed-off-by: Pavel Mores <pmores@redhat.com>
The algorithm is rather simple - we query qemu for existing memory devices
to figure out the index of the one we're about to add. Then we add a
backend object and a corresponding frontend device.
Signed-off-by: Pavel Mores <pmores@redhat.com>
As part of aligning the Kata OCI Spec with oci-spec-rs,
the concept of "State" falls outside the scope of the OCI
Spec itself. While we'll retain the existing code for State
management for now, to improve code organizationand clarity,
we propose moving the State-related code from the oci/ dir
to a dedicated directory named runtime-spec/.
This separation will be completed in subsequent commits with
the removal of the oci/ directory.
Fixes#9766
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This PR increases the timeout for running the CoCo tests to avoid random failures.
These failures occur when the action `Run tests` times out after 30 minutes, causing the CI to fail.
Fixes: #10062
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This PR updates the metrics launch times to use grep -F instead of
fgrep as this command has been deprecated.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
For the GPU build we need go/rust and some other helpers
to build the rootfs.
Always use versions.yaml for the correct and working Rust and golang
version
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Initramfs expects /init, create symlink only if ${ROOTFS}/init does not exist
Init may be provided by other packages, e.g. systemd or GPU initrd/rootfs
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Currently, we have found that `assert_logs_contain` does not work on TDX.
We manually located the specific log, but it fails to get the log using `kubectl debug`. The error found in CI is:
```
warning: couldn't attach to pod/node-debugger-984fee00bd70.jf.intel.com-pdgsj,
falling back to streaming logs: error stream protocol error: unknown error
```
Upon debugging the TDX CI machine, we found an error in containerd:
```
Attach container from runtime service failed" err="rpc error: code = InvalidArgument desc = tty and stderr cannot both be true"
containerID="abc8c7a546c5fede4aae53a6ff2f4382ff35da331bfc5fd3843b0c8b231728bf"
```
We believe this is the root cause of the test failures in TDX CI.
Therefore, we need to ensure that tty and stderr are not set to true at same time.
Fixes: #10011
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Signed-off-by: Wang, Arron <arron.wang@intel.com>
With Nvidia DPU or ConnectX network adapter, VF can do VFIO passthrough
to guest VM in `guest-kernel` mode. In the guest kernel, the adapter's
driver is required to claim the VFIO device and create network interface.
Signed-off-by: Lei Huang <leih@nvidia.com>
When create container failed, it should cleanup the container
thus there's no device/resource left.
Fixes: #10044
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
This PR updates the Blogbench reference values for
read and write operations used in the CI check metrics
job.
This is due to the update to version 1.2 of blobench.
Fixes: #10039
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
It is not good practice to call repack_secure_image() from a bats file
because the test code might not consider cases where `qemu-se` is used
as `KATA_HYPERVISOR`.
This commit moves the function call to set_metadata_annotation() if a key
includes `kernel_params` and `KATA_HYPERVISOR` is set to `qemu-se`, allowing
developers to focus on the test scenario itself.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Add policy to pod-secret-env.yaml from k8s-credentials-secrets.bats.
Policy was already auto-generated for the other pod used by the same
test (pod-secret.yaml). pod-secret-env.yaml was inconsistent,
because it was taking advantage of the "allow all" policy built into
the Guest image. Sooner or later, CI Guests for CoCo will not get the
"allow all" policy built in anymore and pod-secret-env.yaml would
have stopped working then.
Note that pod-secret-env.yaml continues to use an "allow all" policy
after these changes. #10033 must be solved before a more restrictive
policy will be generated for pod-secret-env.yaml.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Since #9904 was merged, newly introduced tests for `k8s-guest-pull-image-authenticated.bats`
have been failing on IBM SE (s390x). The agent fails to start because a kernel parameter
cannot pass to the guest VM via annotation. To fix this, the boot image must be rebuilt with
updated parameters.
This commit adds the rebuilding step in create_pod_yaml_with_private_image() for `qemu-se`.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Similar to HotAttach, the HotDetach method signature for network
endoints needs to be changed as well to allow for the method to make
use of device manager to manage the hot unplug of physical network
devices.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Enable physical network interfaces to be hotplugged.
For this, we need to change the signature of the HotAttach method
to make use of Sandbox instead of Hypervisor. Similar approach was
followed for Attach method, but this change was overlooked for
HotAttach.
The signature change is required in order to make use of
device manager and receiver for physical network
enpoints.
Fixes: #8405
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
The current implementation for device binding using driver bind/unbind
and new_id fails in the scenario when the physical device is not bound
to a driver before assigning it to vfio.
There exists and updated mechanism to accomplish the same that does not
have the same issue as above.
The driver_override field for a device allows us to specify the driver for a device
rather than relying on the bound driver to provide a positive match of the
device. It also has other advantages referenced here:
https://patchwork.kernel.org/project/linux-pci/patch/1396372540.476.160.camel@ul30vt.home/
So use the updated driver_override mechanism for binding/unbinding a
physical device/virtual function to vfio-pci.
Signed-off-by: liangxianlong <liang.xianlong@zte.com.cn>
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Move to blogbench 1.2 version from 1.1.
This version includes an important fix for the read_score test
which was reported to be broken in the previous version.
It essentially fixes this issue here:
https://github.com/jedisct1/Blogbench/issues/4
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
1. Use a container image that supports "ps --user 1000 -f".
2. Execute that command using:
sh -c "ps --user 1000 -f"
instead of passing additional arguments to sh:
sh -c ps --user 1000 -f
Fixes: #10019
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Keep track of individual exec args instead of joining them in the
policy text. Verifying each arg results in a more precise policy,
because some of the args might include space characters.
This improved validation applies to commands specified in K8s YAML
files using:
- livenessProbe
- readinessProbe
- startupProbe
- lifecycle.postStart
- lifecycle.preStop
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Add tests for genpolicy's handling of container.exec_commands. These
are commands allowed by the policy and originating from these input
K8s YAML fields:
- livenessProbe
- readinessProbe
- startupProbe
- lifecycle.postStart
- lifecycle.preStop
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
We had a typo in the attestation tests that we've copied around a
lot and Wainer spotted it in the authenticated registry tests, so let's fix it up now
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Add three new test cases for guest pull from an authenticated registry for
the following scenarios:
_**Scenario**: Creating a container from an authenticated image, with correct credentials via KBC works_
**Given** An authenticated container registry *quay.io/kata-containers/confidential-containers-auth*
**And** a version of kata deployed with a guest image that has an agent with `guest_pull`
feature enabled and nydus-snapshotter installed and configured for
[guest-pulling](https://github.com/containerd/nydus-snapshotter/blob/main/misc/snapshotter/config-coco-guest-pulling.toml)
**And** a KBS set up to have the correct auth.json for
registry *quay.io/kata-containers/confidential-containers-auth* embedded in the `"Credential"` section of `its resources file`
**When** I create a pod from the container image *quay.io/kata-containers/confidential-containers-auth:test*
**Then** The pull image works and the pod can start
_**Scenario**: Creating a container from an authenticated image, with incorrect credentials via KBC fails_
**Given** An authenticated container registry *quay.io/kata-containers/confidential-containers-auth*
**And** a version of kata deployed with a guest image that has an agent with `guest_pull`
feature enabled and nydus-snapshotter installed and configured for
[guest-pulling](https://github.com/containerd/nydus-snapshotter/blob/main/misc/snapshotter/config-coco-guest-pulling.toml)
**And** An installed kata CC with the sample_kbs set up to have the auth.json for registry
*quay.io/kata-containers/confidential-containers-auth* embedded in the `"Credential"` resource, but with a dummy user name and password
**When** I create a pod from the container image *quay.io/kata-containers/confidential-containers-auth:test*
**Then** The pull image fails with a message that reflects that the authorisation failed
_**Scenario**: Creating a container from an authenticated image, with no credentials fails_
**Given** An authenticated container registry *quay.io/kata-containers/confidential-containers-auth*
**And** a version of kata deployed with a guest image that has an agent with `guest_pull`
feature enabled and nydus-snapshotter installed and configured for
[guest-pulling](https://github.com/containerd/nydus-snapshotter/blob/main/misc/snapshotter/config-coco-guest-pulling.toml)
**And** An installed kata CC with no credentials section
**When** I create a pod from the container image *quay.io/kata-containers/confidential-containers-auth:test*
**Then** The pull image fails with a message that reflects that the authorisation failed
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
If the agent-config has a value for `image_registry_auth`,
Then pass this to the image-rs client and enable auth mode too
Fixes: #8122
Signed-off-by: Magnus Kulke <magnuskulke@microsoft.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Add optional config for agent.image_registry_auth, to specify
the uri of credentials to be used when pulling images in the guest
from an authenticated registry
Fixes: #8122
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
Signed-off-by: Magnus Kulke <magnuskulke@microsoft.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This PR updates the devmapper docs by updating the url link
for the current containerd devmapper information.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Made run-k8s-tests-on-zvsi.yaml free of warnings by removing:
SC2086:info:1:1: Double quote to prevent globbing and word splitting ...
SC2086:info:2:1: Double quote to prevent globbing and word splitting ...
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Add setting to allow specifying the cpath for a mount source.
cpath is the root path for most files used by a container. For example,
the container rootfs and various files copied from the Host to the
Guest when shared_fs=none are hosted under cpath.
mount_source_cpath is the root of the paths used a storage mount
sources. Depending on Kata settings, mount_source_cpath might have the
same value as cpath - but on TDX for example these two paths are
different: TDX uses "/run/kata-containers" as cpath,
but "/run/kata-containers/shared/containers" as mount_source_cpath.
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
This PR updates the runtime v2 containerd url information at containerd
documentation.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR updates the cri installation guide url link in the containerd
documentation guide as the previous url link does not exists.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The only kernel built for measured rootfs was the kernel-tdx-experimental,
so this test only ran in the qemu-tdx job runs the test.
In commit 6cbdba7 we switched all TEE configurations to use the same kernel-confidential,
so rootfs measured is disabled for qemu-tdx too now.
The VM still fails to boot (because of a different reason...) but the bug
in the assert_logs_contain, fixed in this PR was masking the checks on the logs.
We still have a few open issues related to measured rootfs and generating
the root hash, so let's skip this test that doesn't work until they are looked at
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Add three new tests cases for guest-pull of an encrypted image
for the following scenarios:
_**Scenario: Pull encrypted image on guest with correct key works**_
**Given** I have a version of kata deployed with a guest image that has
an agent with `guest_pull` feature enabled and nydus-snapshotter installed
and configured for guest-pulling
**And** A public encrypted container image *i* with a decryption key *k*
that is configured as a resource the KBS, so that image-rs on the guest can
connect to it
**When** I try and create a pod from *i*
**Then** The pod is successfully created and runs
_**Scenario: Cannot pull encrypted image with no decryption key**_
**Given** I have a version of kata deployed with a guest image that has
an agent with `guest_pull` feature enabled and nydus-snapshotter installed
and configured for guest-pulling
**And** A public encrypted container image *i* with a decryption key *k*,
that is **not** configured in a KBS that image-rs on the guest can connect to
**When** I try and create a pod from *i*
**Then** The pod is not created with an error message that reflects why
_**Scenario: Cannot pull encrypted image with wrong decryption key**_
**Given** I have a version of kata deployed with a guest image that has
an agent with `guest_pull` feature enabled and nydus-snapshotter installed
and configured for guest-pulling
**And** A public encrypted container image *i* with a decryption key *k*
and a different key *k'* that is set as a resource in a KBS, that image-rs
on the guest can connect to
**When** I try and create a pod from *i*
**Then** The pod is not created with an error message that reflects why
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
ocicrypt config is for kata-agent to connect to CDH to request for image
decryption key. This value is specified by an env. We use this
workaround the same as CCv0 branch.
In future, we will consider better ways instead of writting files and
setting envs inside inner logic of kata-agent.
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
- Enable the kata-cc-rustls-tls feature in image-rs, so that it
can get resources from the KBS in order to retrieve the registry
credentials.
- Also bump to the latest image-rs to pick up protobuf fixes
- Add libprotobuf-dev dependency to the agent packaging
as it is needed by the new image-rs feature
- Add extra env in the agent make test as the
new version of the anyhow crate has changed the backtrace capture thus unit
tests of kata-agent that compares a raised error with an expected one
would fail. To fix this, we need only panics to have backtraces, thus
set RUST_BACKTRACE=0 for tests due to document
https://docs.rs/anyhow/latest/anyhow/Fixes#9538
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The pipe needs adding to the grep, otherwise the grep
gets consumed as an argument to `print_node_journal` and
run in the debug pod.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Auto-generate the policy in k8s-security-context.bats - previously
blocked by lacking support for PodSecurityContext.runAsUser.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Add ability to auto-generate policy for SecurityContext.runAsUser and
PodSecurityContext.runAsUser.
Fixes: #8879
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR adds the share fs information for dragonball using kata-ctl
to avoid the failures in runk tests saying that shared_fs is an
unbound variable.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Modify the permissions of containerd.sock just when genpolicy needs
access to this socket, when testing GENPOLICY_PULL_METHOD=containerd.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Explain why the containerd settings on the local machine get set to
containerd's defaults when testing GENPOLICY_PULL_METHOD=containerd.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
oci-distribution is the value used by run-k8s-tests-on-aks.yaml, so
use the same value as default for GENPOLICY_PULL_METHOD in gha-run.sh.
The value of GENPOLICY_PULL_METHOD is currently compared just with
"containerd", but avoid possible future problems due to using a
different default value in gha-run.sh.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Currently, it is not viable to share a writable volume (e.g., emptyDir)
between containers in a single pod for IBM SE.
The following tests are relevant:
- pod-shared-volume.bats
- k8s-empty-dirs.bats
(See: https://github.com/kata-containers/kata-containers/issues/10002)
This commit skips the tests until the issue is resolved.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Sets SharedFS config to NoSharedFS for remote hypervisor in order to start the file watcher which syncs files from the host to the guest VMs.
Signed-off-by: Silenio Quarti <silenio_quarti@ca.ibm.com>
GH-9973 introduced:
* New function get_kata_memory_and_vcpus() in
tests/metrics/lib/common.bash.
* A call to get_kata_memory_and_vcpus() from extract_kata_env(), which
is defined in tests/common.bash.
Because the nydus test only sources tests/common.bash, it can't find
get_kata_memory_and_vcpus() and errors out.
We fix this by moving the get_kata_memory_and_vcpus() call from
tests/common.bash to tests/metrics/lib/json.bash so that it doesn't
impact the nydus test.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This PR increases the timeout to run the CoCo TDX tests in order
to avoid the random failures on TDX saying that
The action 'Run tests' has timed out after 30 minutes and making
the GHA job fail.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
For running a KBS with `se-verifier` in service,
specific credentials need to be configured.
(See https://github.com/confidential-containers/trustee/tree/main/attestation-service/verifier/src/se for details.)
This commit introduces two procedures to support IBM SE attestation:
- Prepare required files and directory structure
- Set necessary environment variables for KBS deployment
- Repackage a secure image once the KBS service address is determined
These changes enable `k8s-confidential-attestation.bats` for s390x.
Fixes: #9933
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Currently, all functions in `build_se_image.sh` are dedicated to
publishing a payload image. However, `build_secure_image()` is now
also used for repackaging a secure image when a kernel parameter
is reconfigured. This reconfiguration is necessary because the KBS
service address is determined after the initial secure image build.
This commit extracts `build_secure_image()` from `build_se_image.sh`
and creates a separate library, which can be loaded by bats-core.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The current KBS deployment creates a file `key.bin` assuming that
`kustomization.yaml` is located in `overlays/`.
However, this does not hold true when the kustomize config is enabled
for multiple architectures. In such cases, the configuration file
should be located in `overlays/$(uname -m)`.
This commit changes the location for file creation.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
As part of the enablement for s390x, KBS should support multi-arch deployment.
This commit updates the version of coco-trustee to a commit where the support
is implemented.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The DAN feature has already been implemented in kata-runtime-rs, and
this commit brings the same capability to the Go kata-runtime.
Fixes: #9758
Signed-off-by: Lei Huang <leih@nvidia.com>
Delete the kata-containers-k8s-tests namespace before resetting the namespace
to ensure that no deployments or services are restarting and creating pods in the default namespace.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Signed-off-by: Wang, Arron <arron.wang@intel.com>
Delete test scripts forcely in `Delete kata-deploy` step before
deleting all kata pods.
Fixes: #9980
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
In dragonball Vfio device passthrough scenarois, the first passthrough
device will be allocated slot 0 which is occupied by root device.
It will cause error, looks like as below:
```
...
6: failed to add VFIO passthrough device: NoResource\n
7: no resource available for VFIO device"): unknown
...
```
To address such problem, we adopt another method with no pre-allocated
guest device id and just let dragonball auto allocate guest device id
and return it to runtime. With this idea, add_device will return value
Result<DeviceType> and apply the change to related code.
Fixes#9813
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This PR removes the common_init function call from the memory
usage script to eliminate duplicate checking that is also done
from the init_env function.
It also eliminates duplicaction of nested conditionals.
Fixes: #9984
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR removes the CI variable from build kernel script which
is not longer supported it as this was part of the jenkins
environment.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR removes the CI variable in kata deploy in docker script
which was supported it in jenkins environment which is not
longer being supported it.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR removes the CI variable from the local build makefile as
this was part of the jenkins environment which is not longer supported
it.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR removes the CI variable in test images script for osbuilder
as this was part of the jenkins environment which is not longer supported
it.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR removes the CI variable in test config osbuilder script
which was supported on the jenkins environment which is not
longer supported it.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
[greg: squash all fixes into a single patch]
Signed-off-by: Greg Kurz <groug@kaod.org>
This PR removes the use_devmapper variable which was part of the jenkins
environment flags which is not longer support it or available for the
cri-containerd tests.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR retrieves the free memory and the vcpus count from
a kata container and includes them to the json results file of
any metric.
Additionally this PR parses the requested vcpus quantity and the
requested amount memory from kata configuration file and includes
this pair of values into the json results file of any metric.
Finally, the file system defined in the kata configuration file
is included in the results template.
Fixes: #9972
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR rephrased the description and usage of certain functions
as such as:
- set_kata_configuration_performance
- set_kata_config_file
- get_current_kata_config_file
- check_if_root
- check_ctr_images
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
run-k8s-tests-on-zvsi runs the coco tests and we've added new
secrets to provide credentials for the authenticated image testing,
so we need to let the zvsi job inherit these from the caller workflow
like the rest of the coco tests
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Fixes: #9532
Instead of call agent.close_stdin in close_io, we call agent.write_stdin
with 0 len data when the stdin pipe ends.
Signed-off-by: Tim Zhang <tim@hyper.sh>
The sealed secret test depends on the KBS to provide
the unsealed value of a vault secret.
This secret is provisioned to an environment variable.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
To test unsealing secrets stored in environment variables,
we create a simple test server that takes the place of
the CDH. We start this server and then use it to
unseal a test secret.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Signed-off-by: Linda Yu <linda.yu@intel.com>
When sealed-secret is enabled, the Kata Agent
intercepts environment variables containing
sealed secrets and uses the CDH to unseal
the value.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Signed-off-by: Linda Yu <linda.yu@intel.com>
This PR removes the CI variable from QAT run script which was used
in the jenkins environment and not longer used.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR removes the CI variable as well as the instructions related
to this as this was part of the jenkins environment which is not
longer supported it.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR removes the CI variable as well as the instructions related
to this variable which was used on the jenkins environment and not
longer supported.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR updates the Intel QAT documentation by removing the CI variable
which is not longer being supported as this was part of the jenkins
CI environment.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR removes the CI variable which is not longer being used or valid
in the kata containers repository. The CI variable was used when we
were using jenkins and scripts setups which are not longer supported.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR fixes the pip installation for nerdctl by removing a flag
which is not longer supported and avoid the failure of
no such option: --break-system-packages.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
- Due to the error we hit with pulling the agnhost
image used in the liveness-probe tests, we want to leave
the console printing to help with debug when we next try
to bump the image-rs version
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Bump the commit of image-rs we are pulling in to 413295415
Note: This is the last commmit before a change to whiteout handling
was introduced that lead to the error `'failed to unpack: convert whiteout"`
when pulling the agnhost:2.21 image
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Let me explain why:
In our previous approach, we implemented guest pull by passing PullImageRequest to the guest.
However, this method resulted in the loss of specifications essential for running the container,
such as commands specified in YAML, during the CreateContainer stage. To address this,
it is necessary to integrate the OCI specifications and process information
from the image’s configuration with the container in guest pull.
The snapshotter method does not care this issue. Nevertheless, a problem arises
when two containers in the same pod attempt to pull the same image, like InitContainer.
This is because the image service searches for the existing configuration,
which resides in the guest. The configuration, associated with <image name, cid>,
is stored in the directory /run/kata-containers/<cid>. Consequently, when the InitContainer finishes
its task and terminates, the directory ceases to exist. As a result, during the creation
of the application container, the OCI spec and process information cannot
be merged due to the absence of the expected configuration file.
Fixes: kata-containers#9665
Fixes: kata-containers#9666
Fixes: kata-containers#9667
Fixes: kata-containers#9668
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Currently, the image is pulled by image-rs in the guest and mounted at
`/run/kata-containers/image/cid/rootfs`. Finally, the agent rebinds
`/run/kata-containers/image/cid/rootfs` to `/run/kata-containers/cid/rootfs` in CreateContainer.
However, this process requires specific cleanup steps for these mount points.
To simplify, we reuse the mount point `/run/kata-containers/cid/rootfs`
and allow image-rs to directly mount the image there, eliminating the need for rebinding.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
- Add a check in setup_bundle to see if the bundle already exists
and if it does then skip the setup.
This commit is cherry-picked from 44ed3ab80e.
The reason that k8s-kill-all-process-in-container.bats failed is that
deletion of the directory `/root/kata-containers/cid/rootfs` failed during removing container
because it was mounted twice (one in image-rs and one in set_bundle ) and only unmounted once in removing container.
Fixes: #9664
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Dave Hay <david_hay@uk.ibm.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
After v5.14 there is no cpu_hotplug_begin function
now cpus_write_lock same for cpu_hotplug_done = cpus_write_unlock
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This PR has better variable definitons as well the use of a variable
which is already defined in the metrics common script for soak parallel
test.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR is for better variable definitions as well as the use of the
CTR_EXE variable which is already defined in the metrics common script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR uses the CTR_EXE which is already defined in the metrics common
script to have uniformity across the multiple stability tests.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
To unseal a secret, the Kata agent will contact the CDH
using ttRPC. Add the proto that describes the sealed
secret service and messages that will be used.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Signed-off-by: Biao Lu <biao.lu@intel.com>
We have not seen instances of the nydus snapshotter hanging on its
deletion that we must patch its finalize.
Let's just drop this line for now.
Signed-Off-By: Adithya Krishnan Kannan <AdithyaKrishnan.Kannan@amd.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add a basic runtime-rs `Hypervisor` trait implementation for
AWS Firecracker
- Add basic hypervisor operations (setup / start / stop / add_device)
- Implement AWS Firecracker API on a separate file `fc_api.rs`
- Add support for running jailed (include all sandbox-related content)
- Add initial device support (limited as hotplug is not supported)
- Add separate config for runtime-rs (FC)
Notes:
- devmapper is the only snapshotter supported
- to account for no sharefs support, we copy files in the sandbox (as
in the GO runtime)
- nerdctl spawn is broken (TODO: #7703)
Fixes: #5268
Signed-off-by: George Pyrros <gpyrros@nubificus.co.uk>
Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
Signed-off-by: Charalampos Mainas <cmainas@nubificus.co.uk>
Signed-off-by: George Ntoutsos <gntouts@nubificus.co.uk>
Some resource names seem to be lingering in Azure limbo but do not map
to any actual resources, so we ignore those.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This PR removes the CI_JOB variable which previously was used but
not longer being supported of the metrics sysbench test.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR removes the jenkins reference from unit testing presentation
as this is not longer supported on the kata containers project.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR updates the container name to put a random name instead
of using a hard coded name. This PR is a general improvement
to avoid random bug failures specially when we are running on
baremetal environments.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Test reports that it is a onednn test when it is openvino; update
description.
Fixes: #9948
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
This commit extends the vfio-ap hotplug test to include the use of `zcrypttest`.
A newly introduced test by the tool consists of several test rounds as follows:
- ioctl_test
- simple_test
- simple_one_thread_test
- simple_multi_threads_test
- multi_thread_stress_test
- hang_after_offline_online_test
A writable root filesystem is required for testing because the reference count
needs to be reset after each test round. The current containerd kata containers
support does not include `--privileged_without_host_devices`, which is necessary
to configure a writable filesystem along with `--privileged`. (Please check out
https://github.com/kata-containers/kata-containers/issues/9791 for details)
So `crictl` is chosen to extend the test.
The commit also includes the removal of old commands previously used for the
tests repository but no longer in use.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This commit copies an internal testing tool `zcrypttest` to the
test image. A base image is changed to `ubuntu:22.04` due to a
library dependency issue.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This commit adds the missing step of passing an attached vfio-ap device
to a container via ociSpec. It instantiates and passes a vfio-ap device
(e.g. a Z crypto device).
A device at `/dev/z90crypt` covers all use cases at the time of writing.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Do not install the packages librados-dev and librbd-dev as they are not needed for building static qemu.
Add machine option cap-ail-mode-3=off while creating the VM to qemu cmdline.
Fixes: #9893
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
The CI is failing with:
```
Invalid workflow file: .github/workflows/cleanup-resources.yaml#L10
The workflow is not valid. .github/workflows/cleanup-resources.yaml (Line: 10, Col: 5): Unexpected value 'secrets'
```
I think this is because `secrets: inherit` is only applicable
when re-using a workflow, not for a standalone job like
we have here.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
For docker-based builds only install Rust when necessary.
Further, execute the detect Rust version check only when
intending to install Rust.
As of today, this is the case when we intend to build the
agent during rootfs build.
Signed-off-by: Manuel Huber <mahuber@microsoft.com>
Per the decision taken in the 6/27 AC meeting, this PR temporarily
disables kata-deploy and GARM tests until we secure further Azure CI
funding.
In the meantime, I'll transition the GARM tests to free runners and
reenable them to regain that coverage without affecting spending (see
#9940). If it turns out the free runners are too slow, we'll switch back
to GARM.
After funding is secured, we'll reenable the kata-deploy tests (see
#9939).
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
As part of archiving the tests repo, we are eliminating the dependency on
`clone_tests_repo()`. The scripts using the function is as follows:
- `ci/install_rust.sh`.
- `ci/setup.sh`
- `ci/lib.sh`
This commit removes or replaces the files, and makes an adjustment accordingly.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
A multi-arch image for `alpine-bash-curl` has been pushed to and available
at `quay.io/kata-containers`.
This commit switches the test image to `quay.io/kata-containers/alpine-bash-curl`.
Fixes: #9935
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The following scripts are not used by the repository any more:
- ci/install_go.sh
- ci/run.sh
- ci/install_vc.sh
Additionally, they rely on the tests repo, which is soon to be archived.
This commit drops the unused scripts.
Fixes: #8507
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
In case we are dealing with multiple interfaces and there exists a
network interface with a conflicting name, we temporarily rename it to
avoid name conflicts.
Before doing this, we need to rename bring the interface down.
Failure to do so results in netlink returning Resource busy errors.
The resource needs to be down for subsequent operation when the name is
swapped back as well.
This solves the issue of passing multiple networks in case of nerdctl
as:
nerdctl run --rm --net foo --net bar docker.io/library/busybox:latest ip a
Fixes: #9900
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This patch re-generates the client code for Cloud Hypervisor v40.0.
Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by openapi-generator.
Fixes: #9929
Signed-off-by: Bo Chen <chen.bo@intel.com>
Observed instability in the API server after deploying kata-deploy caused test failures.
(see: https://github.com/kata-containers/kata-containers/actions/runs/9681494440/job/26743286861)
Specifically, `kubectl_retry logs` failed before the API server could respond properly.
This commit increases the interval and max_tries for kubectl_retry(), allowing sufficient
time to handle this situation.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
In some DMZ'ed or CI systems the repos are not up to date
and multistrap fails to find the ubuntu-keyring package.
Update the repos to fix this;
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This PR increases the timeout to crictl calls on kata monitor
tests to avoid to hit issues every now and avoid random failures.
This PR is very similar to PR #7640.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Updated genpolicy settings to allow 2 empty environment variables that
may be forgotten to specify (AZURE_CLIENT_ID and AZURE_TENANT_ID)
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
This is the first part of adding a job to clean up potentially dangling
Azure resources. This will be based on Jeremi's tool from
https://github.com/jepio/kata-azure-automation.
At first, we'll only clean up AKS clusters, as this is what has been
causing us problems lately, but this could very well be extended to
cleaning up entire resource groups, which is why I left the different
names pretty generic (i.e. "resources" instead of "clusters").
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
- We only want to enable the sample verifier in the KBS for non-TEE
tests, so prevent an edge case where the TEE platform isn't set up
correctly and we might fall back to the sample and get false positives.
To prevent this we add guards around the sample policy enablement and
only run it for non confidential hardware
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Add the `AUTHENTICATED_IMAGE_USER` and
`AUTHENTICATED_IMAGE_PASSWORD` repository secrets as env vars
to the coco tests, so we can use them to pull an images from
and authenticated registry for testing
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Add --layers-cache-file-path flag to allow the user to
specify where the cache file for the container layers
is saved. This allows e.g. to have one cache file
independent of the user's working directory.
Signed-off-by: Leonard Cohnen <lc@edgeless.systems>
While enabling the attestation for IBM SE, it was observed that
a kernel config `CONFIG_S390_UV_UAPI` is missing.
This config is required to present an ultravisor in the guest VM.
Ths commit adds the missing config.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
we do encourage people to set the KATA_RUNTIME, but it is only used by
the webhook. Let's define it in the main `test.sh` and use it in the
smoke test to ensure the user-defined runtime is smoke-tested rather
than hard-coded kata-qemu one.
Related to: #9804
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Now, using vanilla kubernetes, let's re-enable the TDX CI and hope it
becomes more stable than it used to be.
The cleanup-snapshotter is now taking ~4 minutes, and that matches with
the other platforms, mainly considering there's a sum of 210 seconds
sleep in the process.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Move the `sandbox.agent.setPolicy` call out of the remoteHypervisor
if, block, so we can use the policy implementation on peer pods
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
get_vmm_master_tid() currently returns an error with the message "cannot
get qemu pid (though it seems running)" when it finds a valid
QemuInner::qemu_process instance but fails to extract the PID out of it.
This condition however in fact means that a qemu child process was running
(otherwise QemuInner::qemu_process would be None) but isn't anymore (id()
returns None).
Signed-off-by: Pavel Mores <pmores@redhat.com>
Since Hypervisor::stop_vm() is called from the WaitProcess request handling
which appears to be per-container, it can be called multiple times during
kata pod shutdown. Currently the function errors out on any subsequent
call after the initial one since there's no VM to stop anymore. This
commit makes the function tolerate that condition.
While it seems conceivable that sandbox shouldn't be stopped by WaitProcess
handling, and the right fix would then have to happen elsewhere, this
commit at least makes qemu driver's behaviour consistent with other
hypervisor drivers in runtime-rs.
We also slightly improve the error message in case there's no
QemuInner::qemu_process instance.
Signed-off-by: Pavel Mores <pmores@redhat.com>
Since no objections were raised in the linked issue (#9847) this commit
removes the attempt to derive sandbox bundle path from container bundle
path. As described in more detail in the linked issue, this is container
runtime specific and doesn't seem to serve any purpose.
As for implementation, we hoist the only part of
get_shim_info_from_sandbox() that's still useful (getting the socket
address) directly into the caller and remove the function altogether.
Fixes#9847
Signed-off-by: Pavel Mores <pmores@redhat.com>
We've noticed a bunch of issues related to deploying and deleting the
nydus-snapshotter. As we don't see the same issues on other machines
using vanilla kubernetes, let's avoid using k3s for now follow the flow.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR fixes the variables names for the network that was created as well
removes the network that were created for the tests to ensure a clean environment
when running all the tests and avoid failures specially on baremental environments
that network already exists.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Reject CreateContainerRequest field values that are not tested by
Kata CI and that might impact the confidentiality of CoCo Guests.
This change uses a "better safe than sorry" approach to untested
fields. It is very possible that in the future we'll encounter
reasonable use cases that will either:
- Show that some of these fields are benign and don't have to be
verified by Policy, or
- Show that Policy should verify legitimate values of these fields
These are the new CreateContainerRequest Policy rules:
count(input.shared_mounts) == 0
is_null(input.string_user)
i_oci := input.OCI
is_null(i_oci.Hooks)
is_null(i_oci.Linux.Seccomp)
is_null(i_oci.Solaris)
is_null(i_oci.Windows)
i_linux := i_oci.Linux
count(i_linux.GIDMappings) == 0
count(i_linux.MountLabel) == 0
count(i_linux.Resources.Devices) == 0
count(i_linux.RootfsPropagation) == 0
count(i_linux.UIDMappings) == 0
is_null(i_linux.IntelRdt)
is_null(i_linux.Resources.BlockIO)
is_null(i_linux.Resources.Network)
is_null(i_linux.Resources.Pids)
is_null(i_linux.Seccomp)
i_linux.Sysctl == {}
i_process := i_oci.Process
count(i_process.SelinuxLabel) == 0
count(i_process.User.Username) == 0
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR improves the variable definition in memory inside
the container script for metrics. This change declares and assigns
the variables separately to avoid masking return values.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR will avoid the failures when collecting artifacts for the gha.
This will ensure that we collect and archive system's data for the
purpose of debugging.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
For a minimal initrd/image build we may want to leverage busybox.
This is part number two of the NVIDIA initrd/image build
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
TDX CI has been having some issues with the Nydus snapshotter cleanup,
which has been stuck for hours depending every now and then.
With this in mind, let's disable the TDX CI, so we avoid it blocking the
progress of Kata Containers project, and we re-enable it as soon as we
have it solved on Intel's side.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Commit 'ca02c9f5124e' implements the vhost-user-blk reconnection functionality,
However, it has missed assigning VhostUserDeviceReconnect when new the QEMU
HypervisorConfig, resulting in VhostUserDeviceReconnect always set to default value 0.
Real change is this line, most of changes caused by go format,
return vc.HypervisorConfig{
// ...
VhostUserDeviceReconnect: h.VhostUserDeviceReconnect,
}, nil
Fixes: #9848
Signed-off-by: markyangcc <mmdou3@163.com>
This is a counterpart of commit abf52420a4 for the qemu-coco-dev
configuration. By allowing default_vcpu and default_memory annotations
users can fine-tune the VM based on the size of the container
image to avoid issues related with pulling large images in the guest.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Just like the TEE configurations (sev, snp, tdx) we want to have the
qemu-coco-dev using shared_fs=none.
Fixes: #9676
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The following error was observed during the deployment of nydus snapshotter:
```
Error from server (NotFound):
the server could not find the requested resource ( pods/log nydus-snapshotter-5v82v)
'kubectl logs nydus-snapshotter-5v82v -n nydus-system' failed after 3 tries
Error: Process completed with exit code 1.
```
This error can occur when a pod is re-created by a daemonset during the retry interval.
This commit addresses the issue by using `--selector` rather than the pod name
for `kubectl logs/describe`.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The following jobs have failed more than 50% on nightly CI.
run-kata-deploy-tests-on-garm / run-kata-deploy-tests (clh, k0s)
run-kata-deploy-tests-on-garm / run-kata-deploy-tests (clh, rke2)
run-kata-deploy-tests-on-garm / run-kata-deploy-tests (qemu, k0s)
Instead of removing only those jobs, let's skip the kata-deploy-tests
on GARM completely so we can try to fix all the issues (or maybe
drop the jobs altogether).
Issue: #9854
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
To build the build-kata-deploy image, it should be copied ci/install_yq.sh to
tools/packaging/kata-deploy/local-build/dockerbuild as this script will install
yq within the image. Currently, if
tools/packaging/kata-deploy/local-build/dockerbuild/install_yq.sh exists then
make won't copy it again. This can raise problems as, for example, the current
update of yq version (commit c99ba42d) in ci/install_yq.sh won't force the
rebuild of the build-kata-deploy image.
Note: this isn't a problem on a fresh dev or CI environment.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
- Bump tokio to 1.38.0 to fix the security vulnerability
https://rustsec.org/advisories/RUSTSEC-2024-0019
If possible it would be good to add the many runtime-rs creates into the
runtime-rs workspace and provide a centralised version to avoid the updates
in many places.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The job has failed more than 50% on nightly CI. Remove it from the list of
execution until we don't have a fix.
Issue: #9853
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The job has failed more than 50% on nightly CI. Remove it from the list of
execution until we don't have a fix.
Issue: #9852
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The job has failed more than 50% on nightly CI. Remove it from the list of
execution until we don't have a fix.
The clh variation was disabled on commit 5f5274e699 so this change will
actually result on all the VFIO jobs disabled. Instead of delete the entire
entry from this workflow yaml (or comment the entry), I preferred to use
`if: false` which will make the jobs appear on the UI as skipped.
Issue: 9851
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
- Fix the lint error:
```
error: you seem to use `.enumerate()` and immediately discard the index
--> src/device_manager/mod.rs:427:33
|
427 | for (_index, device) in self.virtio_devices.iter().enumerate() {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
```
by removing the unnecessary enumerate
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The ci failed with:
```
error: use of `or_insert_with` to construct default value
--> src/address_space_manager.rs:650:14
|
650 | .or_insert_with(NumaNode::new);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try: `or_default()`
|
```
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
We take advantage of the Inner pattern to enable QemuInner::resize_vcpu()
take `&mut self` which we need to call non-const functions on Qmp.
This runs on Intel architecture but will need to be verified and ported
(if necessary) to other architectures in the future.
Signed-off-by: Pavel Mores <pmores@redhat.com>
The QMP_SOCKET_FILE constant in cmdline_generator.rs is made public to make
it accessible from QemuInner. This is fine for now however if the constant
needs to be accessed from additional places in the future we could consider
moving it to somewhere more visible.
The Debug impl for Qmp is empty since first, we don't actually want it,
it's only forced by Hypervisor trait bounds, and second, it doesn't have
anything to display anyway. If Qmp gets any members in the future that
can be meaningfully displayed they should be handled by Qmp's Debug::fmt().
Signed-off-by: Pavel Mores <pmores@redhat.com>
The constructor handles QMP connection initialisation, too, so there can
be non-functional Qmp instance.
Signed-off-by: Pavel Mores <pmores@redhat.com>
We set the VERSION variable consistently across Makefiles to
'unknown' if the file is empty or not present.
We also use git commands consistently for calculating the COMMIT,
COMMIT_NO variables, not erroring out when building outside of
a git repository.
In create_summary_file we also account for a missing/empty VERSION
file.
This makes e.g. the UVM build process in an environment where we
build outside of git with a minimal/reduced set of files smoother.
Signed-off-by: Manuel Huber <mahuber@microsoft.com>
The following tests are disabled because they fail (alike with dragonball):
- k8s-cpu-ns.bats
- k8s-number-cpus.bats
- k8s-sandbox-vcpus-allocation.bats
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
There is a known issue in qemu 7.2.0 that causes kernel-hashes to fail the verification of the launch binaries for the SEV legacy use case.
Upgraded to qemu 8.2.4.
new available features disabled.
Fixes: #9148
Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
When the total number of files observed is greater than limit, return -1 directly.
runtime has fixed this bug, it should b ported to runtime-rs.
Fixes:#9829
Signed-off-by: gaohuatao <gaohuatao@bytedance.com>
It shouldn't call the initial_size_manager's setup_config
in the load_config since it had been called in the sandbox's
try_init function.
Fixes: #9778
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
For kata container, the container's pid is meaning less to
containerd/crio since the container's pid is belonged to VM,
and containerd/crio couldn't use it. Thus we just return any
tid of kata shim or hypervisor. But since the hypervisor had
been stopped before deleting the container, and it wouldn't
get the hypervisor's tid for some supported hypervisor, thus
we'd better to return the kata shim's pid instead of hypervisor's
tid.
Fixes: #9777
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
This PR uses the nodeport deployment from upstream trustee.
To ensure our deployment is as close to upstream trustee replace
the custom nodeport handling and replace it with nodeport
kustomized flavour from the trustee project.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR makes general improvements like definition of variables and
the use of them to improve the general setup script for kubernetes
tests.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
While running with a remote hypervisor, whenever kata-monitor tries to access
metrics from the shim, the shim does a "panic" and no metric can be gathered.
The function GetVirtioFsPid() is called on metrics gathering, and had a call
to "panic()". Since there is no virtiofs process for remote hypervisor, the
right implementation is to return nil. The caller expects that, and will skip
metrics gathering for virtiofs.
Fixes: #9826
Signed-off-by: Julien Ropé <jrope@redhat.com>
This corrects the warning to point to the \`-j\` flag,
which is the correct flag for the JSON settings file.
Previously, the warning was confusing, as it pointed to
the \`-p\` flag, which specifies to the path for the Rego ruleset.
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
This PR uses the function definition to have uniformity across
all the launch times script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This patch re-generates the client code for Cloud Hypervisor v39.0.
Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by openapi-generator.
Fixes: #8694, #9574
Signed-off-by: Bo Chen <chen.bo@intel.com>
This patch upgrades Cloud Hypervisor to v39.0 from v36.0, which contains
fixes of several security advisories from dependencies. Details can be
found from #9574.
Fixes: #8694, #9574
Signed-off-by: Bo Chen <chen.bo@intel.com>
Start testing the ability of kata-deploy to install and configure
the qemu-runtime-rs runtimeClass.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Allow kata-deploy to install and configure the qemu-runtime-rs runtimeClass
which ties to qemu hypervisor implementation in rust for the runtime-rs.
Fixes: #9804
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
- Update the config parsing logic so that when reading from the
agent-config.toml file any envs are still processed
- Add units tests to formalise that the envs take precedence over values
from the command line and the config file
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
When the total number of files observed is greater than limit, return (-1, err).
When the returned err is not nil, the func countFiles should return -1.
Fixes:#9780
Signed-off-by: gaohuatao <gaohuatao@bytedance.com>
fixes#9810
Add an annotation to the enum values in the agent config that will
deserialize them using a kebab-case conversion, aligning the behaviour
to parsing of params specified via kernel cmdline.
drive-by fix: add config override for guest_component_procs variable
Signed-off-by: Magnus Kulke <magnuskulke@microsoft.com>
Frequent errors have been observed during k8s e2e tests:
- The connection to the server 127.0.0.1:6443 was refused - did you specify the right host or port?
- Error from server (ServiceUnavailable): the server is currently unable to handle the request
- Error from server (NotFound): the server could not find the requested resource
These errors can be resolved by retrying the kubectl command.
This commit introduces a wrapper function in common.sh that runs kubectl up to 3 times
with a 5-second interval. Initially, this change only covers gha-run.sh for Kubernetes.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
It seems I was very lose on disabling some of the tests, and the issues
I faced could be related to other instabilities in the CI.
Let's re-enable this one, following what was done for the SEV, SNP, and
coco-qemu-dev.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It seems I was very lose on disabling some of the tests, and the issues
I faced could be related to other instabilities in the CI.
Let's re-enable this one, following what was done for the SEV, SNP, and
coco-qemu-dev.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It seems I was very lose on disabling some of the tests, and the issues
I faced could be related to other instabilities in the CI.
Let's re-enable this one, following what was done for the SEV, SNP, and
coco-qemu-dev.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It seems I was very lose on disabling some of the tests, and the issues
I faced could be related to other instabilities in the CI.
Let's re-enable this one, following what was done for the SEV, SNP, and
coco-qemu-dev.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It seems I was very lose on disabling some of the tests, and the issues
I faced could be related to other instabilities in the CI.
Let's re-enable this one, following what was done for the SEV, SNP, and
coco-qemu-dev.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's stop skipping the CDH tests for TDX, as know we should have an
environmemnt where it can run and should pass. :-)
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We must ensure we use the host ip to connect to the PCCS running on the
host side, instead of using localhost (which has a different meaning
from inside the KBS pod).
The reason we're using `hostname -i` isntead of the helper functions, is
because the helper functions need the coco-kbs deployed for them to
work, and what we do is before the deployment.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(1) As mis-use of cap.set causing previous Caps lost which
causing assert! failed, just replacing cap.set with cap.add.
(2) It will return error if there's no such name setting when
do update_config_by_annotation {
...
if config.runtime.name.is_empty() {
return Err(io::Error::new(
io::ErrorKind::InvalidData,
"Runtime name is missing in the configuration",
));
}
...
}
Fixes#9783
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Centos8 is EOL and repos are not available anymore. Centos9 contains the
same packages and should do well as a base for testing.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
This is generally useful debug output on test failures,
and specifically this has been useful for nydus-related
issues recently.
Signed-off-by: Chris Porter <porter@ibm.com>
This is required to support SEV-SNP confidential container with kernel-hashes.
Since this ovmf is latest stable version, it is good to upgrade for tdx
and Vanilaa builds too.
Signed-off-by: Niteesh Dubey <niteesh@us.ibm.com>
This is required to provide the hashes of kernel, initrd and cmdline
needed during the attestation of the coco.
Fixes: #9150
Signed-off-by: Niteesh Dubey <niteesh@us.ibm.com>
A new PULL_TYPE environment variable is recognized by the kata-deploy's
install script to allow it to configure CRIO-O for guest-pull image pulling
type.
The tests/integration/kubernetes/gha-run.sh change allows for testing it:
```
export PULL_TYPE=guest-pull
cd tests/integration/kubernetes
./gha-run.sh deploy-k8s
```
Fixes#9474
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This PR replaces the name to use a variable that is already defined
to have a better uniformity across the general script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
In line with the changes for x86_64, the k8s nydus test for s390x should
also use `qemu-coco-dev` for `KATA_HYPERVISOR`.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
It was observed that the `node_start_time` value is sometimes empty,
leading to a test failure.
This commit retries fetching the value up to 3 times.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The Kata blog was recently moved to the project's website. The content
of the blog is stored together with the rest of the website source on
GitHub.
This patch adds a short guide that describes how to submit a new
blog post as a PR, to appear on the project's website.
Signed-off-by: Ildiko Vancsa <ildiko.vancsa@gmail.com>
This test fails when using `shared_fs=none` with the nydus snapshotter.
Issue tracked here: #9666
Skipping for now.
Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
This test is failing due to the initContainers not being properly
handled with the guest image pulling.
Issue tracked here: #9668
Skipping for now.
Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
This test fails when using `shared_fs=none` with the nydus napshotter,
Issue tracked here: #9664
Skipping for now.
Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
To prevent download failures caused by high traffic to the Docker image,
opt for quay.io/prometheus/busybox:latest over docker.io/library/busybox:latest .
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Added configuration file with rules to exclude some self-hosted
runners from the linter warnings.
Related-with: #9646
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The job has failed more than 50% on nightly CI. Remove it from the list of
execution until we don't have a fix.
Issue: 9764
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
These jobs have failed more than 50% on nightly CI. Remove them from the list of
execution until we don't have a fix.
Issue: 9763
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This job has failed more than 50% on nightly CI. Remove it from the list of
execution until we don't have a fix.
Issue: 9761
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
For nerdctl install, add symlinks for runc and slirp4netns in the
binary install path.
runc link comes in handy for running runc containers with nerdctl fir
quick tests.
slirp4netns allows for running containers with user mode networking
useful in case of rootless containers.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
CreateContainerRequest objects can specify devices to be created inside
the guest VM. This change ensures that requested devices have a
corresponding entry in the PodSpec.
Devices that are added to the pod dynamically, for example via the
Device Plugin architecture, can be allowlisted globally by adding their
definition to the settings file.
Fixes: #9651
Signed-off-by: Markus Rudy <mr@edgeless.systems>
This adds structs and fields required to parse PodSpecs with
VolumeDevices and PVCs with non-default VolumeModes.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
If a test is failing during setup, makes no much sense to run the suite.
Let's skip and add some debug messages.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
End of file should not end with --- mark. This will confuse tools like
yq and kubectl that might think this is another object.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
Since yq frequently updates, let's upgrade to a version from February to
bypass potential issues with versions 4.41-4.43 for now. We can always
upgrade to the newest version if necessary.
Fixes#9354
Depends-on:github.com/kata-containers/tests#5818
Signed-off-by: Beraldo Leal <bleal@redhat.com>
golang.mk is not ready to deal with non GOPATH installs. This is
breaking test on s390x.
Since previous steps here are installing go and yq our way, we could
skip this aditional check. A full refactor to golang.mk would be needed
to work with different paths.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
fixes#9748
A configuration option `guest_component_procs` has been introduced that
indicates which guest component processes are supposed to be spawned by
the agent. The default behaviour remains that all of those processes are
actively spawned by the agent. At the moment this is based on presence
of binaries in the rootfs and the guest_component_api_rest option.
The new option is incremental:
none -> attestation-agent -> confidential-data-hub -> api-server-rest
e.g. api-server-rest implies attestation-agent and confidential-data-hub
the `none` option has been removed from guest_component_api_rest, since
this is addresses by the introduced option.
To not change expected behaviour for non-coco guests we still will still
only attempt to spawn the processes if the requested attestation binaries
are present on the rootfs, and issue in warning in those cases.
Signed-off-by: Magnus Kulke <magnuskulke@microsoft.com>
We are currently building Oras from source on ppc64le. Now that they offically release the artefacts
for power, consume them to install Oras.
Fixes: #9213
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
Wainer noticed this is failing for the coco-qemu-dev case, and decided
to skip it, notifying me that he didn't fully understand why it was not
failing on TDX.
Turns out, though, this is also failing on TDX, and we need to skip it
there as well.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add the CLI flag --runtime-class-names, which is used during
policy generation. For resources that can define a
runtimeClassName (e.g., Pods, Deployments, ReplicaSets,...)
the value must have any of the --runtime-class-names as
prefix, otherwise the resource is ignored.
This allows to run genpolicy on larger yaml
files defining many different resources and only generating
a policy for resources which will be deployed in a
confidential context.
Signed-off-by: Leonard Cohnen <lc@edgeless.systems>
Use the variable BATS_TEST_COMPLETED which is defined by the bats framework
when the test finishes. `BATS_TEST_COMPLETED=` (empty) means the test failed,
so the node syslogs will be printed only at that condition.
Fixes: #9750
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This test fails with qemu-coco-dev configuration and guest-pull image pull.
Issue: #9667
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
It's cloning the nydus-snapshotter repo from the version specified in
versions.yaml, however, the deployment files are set to pull in the
latest version of the snapshotter image. With this version we are
pinning the image version too.
This is a temporary fix as it should be better worked out at nydus-snapshotter
project side.
Fixes: #9742
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This test fails with qemu-coco-dev configuration and guest-pull image pull.
Issue: #9668
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This test fails with qemu-coco-dev configuration and guest-pull image pull.
Issue: #9666
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This test fails with qemu-coco-dev configuration and guest-pull image pull.
Issue: #9664
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This test fails with qemu-coco-dev configuration and guest-pull image pull.
Unlike other tests that I've seen failing on this scenario, k8s-seccomp.bats
fails after a couple of consecutive executions, so it's that kind of failure
that happens once in a while.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This will enable the k8s tests to leverage guest pulling when
PULL_TYPE=guest-pull for qemu-coco-dev runtimeclass.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The runtime handler annotation is required for Kubernetes <= 1.28 and
guest-pull pull type. So leverage $PULL_TYPE (which is exported by CI jobs)
to conditionally apply the annotation.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
It creates this line, as the Golang runtime does:
-object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0
Signed-off-by: Emanuel Lima <emlima@redhat.com>
We need to remove the device from the tracking map, a container
restart will increment the bus index and we will get out of root-ports
and crash the machine.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
We need special handling for pod_sandbox, pod_container and
single_container how and when to inject CDI devices
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
In Kubernetes we still do not have proper VM sizing
at sandbox creation level. This KEP tries to mitigates
that: kubernetes/enhancements#4113 but this can take
some time until Kube and containerd or other runtimes
have those changes rolled out.
Before we used a static config of VFIO ports, and we
introduced CDI support which needs a patched contianerd.
We want to eliminate the patched continerd in the GPU case
as well.
Fixes: #8860
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Let's see if it helps with issues like:
```
error: must build at directory: not a valid directory: evalsymlink
failure on
'"/home/runner/actions-runner/_work/kata-containers/kata-containers/tests/functional/kata-deploy/../../..//tools/packaging/kata-deploy/kata-cleanup/overlays/k0s"'
: lstat
/home/runner/actions-runner/_work/kata-containers/kata-containers/tests/functional/kata-deploy/":
no such file or directory
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This takes a few minutes that could be saved, so let's avoid doing this
on all the platforms, but simply do this when it's needed (the baremetal
use case).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Currently only "baremetal" runs all the tests, but we could easily run
"all" locally or using the github provided runners, even when not using
a "baremetal" system.
The reason I'd like to have a differentiation between "all" and
"baremetal" is because "baremetal" may require some cleanup, which "all"
can simply skip if testing against a fresh created VM.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
For now we've only exposed the option to deploy kata-deploy for k3s and
vanilla kubernetes when using containerd.
However, I do need to also deploy k0s and rke2 for an internal CI, and
having those exposed here do not hurt, and allow us to easily expand the
CI at any time in the future.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
k0s was added to kata-deploy, but it's kata-cleanup counterpart was
never added. Let's fix it.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
k0s deployment has been broken since we moved to using `tomlq` in our
scripts. The reason is that before using `tomlq` our script would,
involuntarily, end up creating the file.
Now, in order to fix the situation, we need to explicitly create the
file and let `tomlq` add the needed content.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This option allows to add a debug monitor socket when
`enable_debug = true` to control QEMU within debugging case.
Fixes: #9603
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The `update_env_pci()` function need the PCI address mapping to
translate the host PCI address to guest PCI address in below
environment variables:
- PCIDEVICE_<prefix>_<resource-name>_INFO
- PCIDEVICE_<prefix>_<resource-name>
So collect PCI address mapping for both vfio-pci-gk and
vfio-pci devices.
Fixes#9614
Signed-off-by: Lei Huang <leih@nvidia.com>
The k8s test suite halts on the first failure, i.e., failing-fast. This
isn't the behavior that we used to see when running tests on Jenkins and it
seems that running the entire test suite is still the most productive way. So
this disable fail-fast by default.
However, if you still wish to run on fail-fast mode then just export
K8S_TEST_FAIL_FAST=yes in your environment.
Fixes: #9697
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This test has failed in confidential runtime jobs. Skip it
until we don't have a fix.
Fixes: #9663
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
One of our machines is running CentOS 9 Stream, and we could easily
verify that we can build and install the kbs client there, thus we're
expanding the installation script to also support CentOS 9 Stream.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
`externals.coco-kbs.toolchain` is not defined, get the rust_version from
`externals.coco-trustee.toolchain` instead.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're bumping the version in order to bring in the customisation needed
for setting up a custom pccs, which is needed for the KBS integration
tests with Kata Containers + TDX.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR enables the installation and unistallation of the kbs client
as well as general coco components needed for the TDX GHA CI.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR fixes the minvalue for boot time to avoid the random failures
of the GHA CI.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Replace tab indentation with spaces for the three lines within the ifneq statements, aligning them with the surrounding code.
Fixes:#9692
Signed-off-by: sidneychang <2190206983@qq.com>
The `dial_timeout` works fine for Runtime-go, but is obsoleted in
Runtime-rs.
When the pod cannot connect to the Agent upon starting, we need to adjust
the `reconnect_timeout_ms` to increase the number of connection attempts to
the Agent.
Fixes: #9688
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
For the TD attestation to work the connection to QGS on the host is needed.
By default QGS runs on vsock port 4050, but can be modified by the host owner.
Format of the qemu object follows the SocketAddress structure, so it needs to be provided in the JSON format, as in the example below:
-object '{"qom-type":"tdx-guest","id":"tdx","quote-generation-socket":{"type":"vsock","cid":"2","port":"4050"}}'
Fixes: #9497
Signed-off-by: Jakub Ledworowski <jakub.ledworowski@intel.com>
The new version of sriov-network-device-plugin adds an env
`PCIDEVICE_<prefix>_<resource-name>_INFO`, which has a json
value; kata-agent can't parse it as env
`PCIDEVICE_<prefix>_<resource-name>` which has value in format
"DDDD:BB:SS.F".
This change updates env `PCIDEVICE_<prefix>_<resource-name>_INFO`.
Signed-off-by: Lei Huang <leih@nvidia.com>
Container tags can be a maximum of 128 characters long
so calculate the length of the arch suffix and then restrict
the tag to this length subtracted from 128
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Previously I copied the logic that abbreviated the commit hash
from the versioning, but looking at our versions.yaml the clear pattern
is that when pointing at commits of dependencies we use the full
commit hash, not the abbreviated one, so for consistency I think we should
do the same with the components that we make available
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
As we have multi-arch builds for nearly all components, we want to ensure
that all the cache tags we set have the architecture suffix, not just the
`TARGET_BRANCH` one.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Remove `rand.Seed` call to resolve the following failure:
```
rand.Seed is deprecated: As of Go 1.20 there is no reason to call Seed with a random value.
```
The go rand.Seed docs: https://pkg.go.dev/math/rand@go1.20#Seed
back this up and states:
> If Seed is not called, the generator is seeded randomly at program startup.
so I believe we can just delete the call.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
For now, let's allow the users to set the default_cpu and default_memory
when using TDX, as they may hit issues related to the size of the
container image that must be pulled and unpacked inside the guest,
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
- Make due to us bumping the golang version used in our CI
but `make vendor` fails without the go version in the runtime go.mod
being increased, so update this and run go mod tidy
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This PR adds the use of custom Intel DCAP configuration when
deploying the KBS.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR improves general format like variable definition to have
uniformity across the memory usage script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Now we have updated the release builds to push
artefacts to
our registry for the release, so we can cache the images, we need to
set `secrets: inherit` for all architecture's tarball builds
so that we can log into quay.io and ghcr in those steps
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
VERSION_ID is not guaranteed to be specified in os-release, this
makes kaka-deploy breaks in rolling distros like arch linux and void
linux.
Note that operating system vendors may choose not to provide
version information, for example to accommodate for rolling releases.
In this case, VERSION and VERSION_ID may be unset.
Applications should not rely on these fields to be set.
Signed-off-by: vac <dot.fun@protonmail.com>
Fixes: #9532
Close stdin when write_stdin receives data of length 0.
Stop call notify_term_close() in close_stdin, because it could
discard stdout unexpectedly.
Signed-off-by: Tim Zhang <tim@hyper.sh>
This test doesn't fail with the guest image pulling, but it for sure
should. :-)
We can see in the bats logs, something like:
```
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 31s default-scheduler Successfully assigned kata-containers-k8s-tests/liveness-exec to 984fee00bd70.jf.intel.com
Normal Pulled 23s kubelet Successfully pulled image "quay.io/prometheus/busybox:latest" in 345ms (345ms including waiting)
Normal Started 21s kubelet Started container liveness
Warning Unhealthy 7s (x3 over 13s) kubelet Liveness probe failed: cat: can't open '/tmp/healthy': No such file or directory
Normal Killing 7s kubelet Container liveness failed liveness probe, will be restarted
Normal Pulled 7s kubelet Successfully pulled image "quay.io/prometheus/busybox:latest" in 389ms (389ms including waiting)
Warning Failed 5s kubelet Error: failed to create containerd task: failed to create shim task: the file /bin/sh was not found: unknown
Normal Pulling 5s (x3 over 23s) kubelet Pulling image "quay.io/prometheus/busybox:latest"
Normal Pulled 4s kubelet Successfully pulled image "quay.io/prometheus/busybox:latest" in 342ms (342ms including waiting)
Normal Created 4s (x3 over 23s) kubelet Created container liveness
Warning Failed 3s kubelet Error: failed to create containerd task: failed to create shim task: failed to mount /run/kata-containers/f0ec86fb156a578964007f7773a3ccbdaf60023106634fe030f039e2e154cd11/rootfs to /run/kata-containers/liveness/rootfs, with error: ENOENT: No such file or directory: unknown
Warning BackOff 1s (x3 over 3s) kubelet Back-off restarting failed container liveness in pod liveness-exec_kata-containers-k8s-tests(b1a980bf-a5b3-479d-97c2-ebdb45773eff)
```
Let's skip it for now as we have an issue opened to track it down:
https://github.com/kata-containers/kata-containers/issues/9665
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Similarly to firecracker, which doesn't have support for virtio-fs /
virtio-9p, TDX used with `shared_fs=none` will face the very same
limitations.
The tests affected are:
* k8s-credentials-secrets.bats
* k8s-file-volume.bats
* k8s-inotify.bats
* k8s-nested-configmap-secret.bats
* k8s-projected-volume.bats
* k8s-volume.bats
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure the logs will print the correct annotation and its
value, instead of always mentioning "kernel" and "initrd".
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're doing that as all tests are going to be running with
`shared_fs=none`, meaning that we don't need any specific test for this
case anymore.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function will set the needed annotation for enforcing that the
image pull will be handled by the snapshotter set for the runtime
handler, instead of using the default one.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We shouldn't be using 9p, at all, with TEEs, as off right now we have no
way to ensure the channels are encrypted. The way to work this around
for now is using guest pull, either with containerd + nydus snapshotter
or with CRI-O; or even tardev snapshotter for pulling on the host (which
is the approach used by MSFT).
This is only done for TDX for now, leaving the generic, AMD, and IBM
related stuff for the folks working on those to switch and debug
possible issues on their environment.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In Kubernetes, the following values for namespace are equivalent and all refer to the default namespace:
- ` ` (namespace field missing)
- `namespace: ""` (namespace field is the empty string)
- `namespace: "default"`(namespace field has the explicit value `default`)
Genpolicy currently does not handle the empty string case correctly.
Signed-Off-By: Malte Poll <1780588+malt3@users.noreply.github.com>
When just containerd is installed without installing nerdctl,
cni plugins are missing from the installation.
containerd tarball does not include cni plugin files.
Hence install cni plugins separately for containerd.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
nerdctl requires cni plugins to be installed in /opt/cni/bin
Without bridge plugin installed, it is not possible to run a
container with nerdctl.
The downloaded nerdctl tarball contains cni plugin files, but are
extracted under /usr/local/libexec.
Copy extracted tarball cni files under /usr/local/libexec
to /opt/cni/bin
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Use `eval` to process the `date` command along with its parameters,
thus avoiding misinterpreting the parameters as commands.
Fixes: #9661
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Determine the realpath of kata-shim avoiding the check fails
in case the kata-shim is not a symlink, as was happening prior
to this commit.
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
- The tags have a trailing non-printable character, which results
in our cache tags having a trailing underscore e.g. `ghcr.io/kata-containers/cached-artefacts/agent:ce24e9835_`
For ease of use of these cached components, we should strip off the trailing underscore.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
An e2e test for `vfio-ap` has been conducted internally in IBM
due to the lack of publicly available test machines equipped
with a required crypto device.
The test is performed by the `tests` repository:
(i.e. 772105b560/Makefile (L144))
The community is working to integrate all tests into the `kata-containers`
repository, so the `vfio-ap` test should be part of that effort.
This commit moves a test script and Dockerfile for a test image from
the `tests` repository. We do not rename the script to `gha-run.sh`
because it is not executed by Github Actions' workflow.
You can check the test results from the s390x nightly test with the migrated files here:
https://github.com/kata-containers/kata-containers/actions/runs/9123170010/job/25100026025
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Given that we think the containerd -> snapshotter image cache
problems have been resolved by bumping to nydus-snapshotter v0.3.13
we can try removing the skips to test this out
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- We previously have an expectation for the pause rootfs
to be pull on the host when we did a guest pull. We weren't
really clear why, but it is plausible related to the issues we had
with containerd and nydus caching. Now that is fixed we can begin
to address this with setting shared_fs=none, but let's start with
updating the rootfs host check to be not higher than expected
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This test is called from `tests/integration/run_kuberentes_tests.sh`,
which already ensures that yq is installed.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
On 1423420, I've mistakenly disabled the tests entirely, for both
non-TEEs and TEEs.
This happened as I didn't realise that `confidential_setup` would take
non-TEEs into consideration. :-/
Now, let me follow-up on that and make sure that the tests will be
running on non-TEEs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function is a helper to check whether the KATA_HYPERVISOR being
used is a confidential hardware (TEE) or not, and we can use it to
skip or only run tests on those platforms when needed.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's rename it to `is_confidential_runtime_class`, and adapt all the
places where it's called.
The new name provides a better description, leading to a better
understanding of what the function really does.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Implementation of QemuCmdLine has a fairly uniform and repetitive structure
that's guided by a set of conventions. These conventions have however been
mostly implicit so far, leading to a superfluous and annoying
request/force-push churn during qemu-rs PR reviews.
This commit aims to make things explicit so that contributors can take them
into account before an initial PR submission.
Signed-off-by: Pavel Mores <pmores@redhat.com>
This commit is to append an arch type to the initramfs-cryptsetup image
to prevent a wrong arch image from being pulled on a different arch host.
Fixes: #9654
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
- Container image tags can only contain alphanumeric, period,
hyphen and underscore characters, so convert characters outside
of these to be underscores, to avoid having invalid tag failures
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Recently the extra gpu caching was added, unfortunately when I
rebased I ended up with both the new tagging logic and old logic.
Let's try and integrate them properly to avoid doing the push twice.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
in case the upstream CI fails it's useful to pin-point the PR that
caused the regression. Currently openshift-ci does not allow doing that
from their setup but we can mimic the setup on our infrastructure and
use the available kata-deploy-ci images to find the first failing one.
To help with that add a few helper scripts and a howto.
Fixes: #9228
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Now we have the workflow updated and can test the changes in caching
we've hit an error:
```
line 1180: artefact_tag: unbound variable
```
so we need to fix that up. Sorry for missing this before.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- It appears like the `if` isn't required when setting env as a
conditional
- `inputs.stage` over input.stage
- Swap matrix.component to matrix.asset
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
ResizeMemory for Cloud Hypervisor is missing a check for the new
requested memory being greater than the max hotplug size after
alignment. Add the check, and since an earlier check for this
setsrequested memory to the max hotplug size, do the same in the
post-alignment check.
Fixes#9640
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
All these settings are hardcoded as `false` and result in
no extra options on the QEMU command line, like the go
runtime does. There actually not needed :
- we're never going to ask QEMU to survive a guest shutdown
- we're never going to run QEMU daemonized since it prevents
log collection
- we're never going to ask QEMU to start with the guest stopped
No need to keep this code around then.
Signed-off-by: Greg Kurz <groug@kaod.org>
- CoCo wants to use the agent and coco-guest-components cached artifacts
so tag them with a helpful version, so make these easier to get
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
No commands remaining.
- We don't want to ship certain components (agent, coco-guest-components)
as part of the release, but for other consumers it's useful to be able to pull in the components
from oras, so rather than not building them, just don't upload it as part of the release.
- Also make the archs all consistent on not shipping the agent
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Set RELEASE env to 'yes', or 'no', based on if the stage
passed in was 'release', so we can use it in the build scripts
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
For other projects (e.g. CoCo projects) being able to
access the released versions of components is helpful,
so push these during the release process
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
A length of the result of `git log -1 --pretty=format:%h` could vary
over different CI systems, highly likely messing up their caching
mechanisms.
This commit is to use an option `--abbrev=9` to standardize the length
to 9 characters for CI.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Bump nydus snapshotter to v0.13.13 to fix the gap when switching
different snapshotters in guest pull.
Fixes: #8407
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This will help us to debug issues in the future (and would have helped
in the past as well). :-)
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is needed, as b1710ee2c0 made the
default agent shipped the one with policy support. However, we simply
didn't update the rootfs to reflect that, causing then an issue to start
the agent as shown by the strace below:
```
open("/etc/kata-opa/default-policy.rego", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
futex(0x7f401eba0c28, FUTEX_WAKE_PRIVATE, 1) = 1
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1 RT_2], [], 8) = 0
tkill(553681, SIGABRT) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=553681, si_uid=1000} ---
+++ killed by SIGABRT (core dumped) +++
```
This happens as the default policy **must** be set when the agent is
built with policy support, but the code path that copies that into the
rootfs is only triggered if the rootfs itself is built with
AGENT_POLICY=yes, which we're now doing for both confidential and
non-confidential cases.
Sadly this was not caught by CI till we the cache was not used for
rootfs, which should be solved by the previous commit.
Fixes: #9630, #9631
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is to add an info for files at `tools/packaging/kata-deploy/local-build/*
to a version of the components and ensure that the cached artefacts are not used
when the files of interest are updated.
Fixes: #9630
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
the `tdx_not_supported_warning` function does not exists, the
`tdx_not_supported` should be called instead.
Fixes: #9628
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Using a debugger with the kata runtime is complicated, but it can be done
and can be very useful.
This commits provides a helper script that simplifies it, and updates
the developper's documentation to explain how to use it.
Signed-off-by: Julien Ropé <jrope@redhat.com>
This PR fixes the indentation in gha run k8s common script
to have uniformity across the script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
By default, when a container is created with the `--privileged` flag,
all devices in `/dev` from the host are mounted into the guest. If
there is a block device(e.g. `/dev/dm`) followed by a generic
device(e.g. `/dev/null`)，two identical block devices(`/dev/dm`)
would be requested to the kata agent causing the agent to exit with error:
> Conflicting device updates for /dev/dm-2
As the generic device type does not hit any cases defined in `switch`，
the variable `kataDevice` which is defined outside of the loop is still
the value of the previous block device rather than `nil`. Defining `kataDevice`
in the loop fixes this bug.
Signed-off-by: cncal <flycalvin@qq.com>
kube-router decided to use :8080 for its metrics, and this seems to be a
change that affected k0s 1.30.0+, leading to kube-router pod crashing
all the time and anything can actually be started after that.
Due to this issue, let's simply use a different port (:9999) and move on
with our tests.
Fixes: #9623
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Whenever we count on having the headers tarball, we must unpack the
cached content into the expected directory, otherwise we'd simply fail,
as we've been failing in our CI, at the end of the process where we
generate the tarball from the cached components.
It's weird to me, sincerely, that the headers tarball end up in such
weird place (build/kernel-nvidia-gpu/builddir/), but I'll leave that to
Zvonko to figure out whether something better can be done, as the intuit
of this PR is simply unblock Kata Containers CI.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
New env var so everyone can test the PUSH_TO_REGISTRY feature
export PUSH_TO_REGISTRY=yes
export ARTEFACT_REGISTRY=quay.io
export ARTEFACT_REPOSITORY=my-fancy-kata-containers
export ARTEFACT_REGISTRY_USERNAME=zvonkok
export ARTEFACT_REGISTRY_PASSWORD=<super-secret>
make ...-tarball
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This PR updates the launch times scripts by improving the variable
definition as well as trying to use the same format across all the script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
RTC was being built in a wrong fashion on commit #2bc5e3c6e2ab0145fa9e8be95df0d5086c07a517
RTC was being constructed inside the QemuCmdLine struct,
but it should've been built inside the devices vector.
Signed-off-by: Emanuel Lima <emlima@redhat.com>
This was needed when we were using an old (and not maintained anymore)
host stack. Considering what we have as part of the distros, Today,
this can simply be dropped, as I cannot find any reference of this one
being needed in any up-to-date documentation.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit b7cccfa019.
The `private=on` bit has never made its way upstream, and was removed
from the latest iteration that we're using. With that in mind, let's
revert its usage in the code.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit 582b5b6b19.
The `private=on` bit has never made its way upstream, and was removed
from the latest iteration that we're using. With that in mind, let's
revert its addition, and later on its usage in the code.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Here we're checking the distro's `/etc/os-release` or
`/usr/lib/os-release` in order to get which distro we're deploying the
Kata Containers artefacts to, and then to properly adjust the QEMU and
OVMF with TDX support that's been shipped with the distros.
Together with that, we're also printing the instructions provided by the
distro on how to enable and use TDX.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the PLACEHOLDER_FOR_DISTRO_{QEMU,OVMF}_WITH_TDX_SUPPORT
variables instead of actually setting a path, so we can easily replace
those as part of our deployment scripts.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We'll need to have access to the host os-release file (either under
`/etc/os-release` or under `/usr/lib/os-release`), and the simplest
approach that comes to my mind to do is doing what a debug pod would do,
mounting `/` as `/host` and then allowing us to have access to those
files, and then corectly set the TDX specific QEMU and OVMF (TDVF) paths
for the tdx available configurations.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We haven't been using nor testing with td-shim, as Cloud Hypervisor does
not officially support TDX yet, and TDVF is supposed to be used with
QEMU, instead of td-shim.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's remove everything related to the TDX specific QEMU building /
shipping from our repo, as we'll be relying on the one coming from the
distros.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's remove everything related to the TDVF building / shipping from our
repo, as we'll be relying on the one coming from the distro.
Later on, we may need to re-add TDVF logic, as we're already using
upstream edk2 repo / content, but when that's needed we'll simply revert
this commit.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR fixes the random write value for FIO for qemu by decreasing it
to avoid the random failures of the GHA CI.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's skip those tests on TEEs as we've been facing a reasonable amount
of issues, most likely on the containerd side, related to pulling the
image on the guest.
Once we're able to fix the issues on containerd, we can get back and
re-enable those by reverting this commit.
The decision of disabling the tests for TEEs is because the machines may
end up in a state where human intervention is necessary to get them back
to a functional state, and that's really not optimal for our CI.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is the first step of the work to start relying on the artefacts
coming from the distros (CentOS 9 Stream, and Ubuntu) themselves.
Let's have this first one merged, as this will not run the CI due to the
changes being on the yaml itself, and then follow-up with the changes
needed on other parts of the project (kata-deploy, runtime, etc).
Fixes: #9590 -- part I
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Everytime I create contianer on arm64 machine, containerd/kata logs a redundant warning
as follows:
``` shell
time="2024-05-07" level=warning msg="<nil>" arch=arm64 name=containerd-shim-v2
pid=xxx sandbox=fdd1f05 source=virtcontainers/hypervisor
```
I added an error statement so that the error would be logged when it occurs.
Signed-off-by: cncal <flycalvin@qq.com>
This PR removes oci information from versions file as this is not
longer being used in kata containers repository.
Fixes#9599
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds support to install KBS to k8s TDX GHA workflow in
order to run confidential attestation tests.
Fixes#9451
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds a k8s negative policy test to the confidential attestation
bats test.
Fixes#9437
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
We should init and asign the runtime instance to runtime
handler, otherwise, if the pause container failed to start,
which means the runtime instance failed to start, then the
following delete & shutdown request wouldn't be run, thus
the dead shim would be left.
Fixes: #9597
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
dragonball reserves 2048G of mmio space for the pci root bus by default
on physical addresses greater than 4G. However, for some machines with
smaller physical address widths, such as 39-bit wide physical addresses,
dragonball reserves the mmio space when initializing the memory. It is
less than 2048G, so this commit dynamically calculates and allocates the
mmio size of each pci root bus.
Fixes: #9509
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
The CH v39 upgrade in #9575 is currently blocked because of a bug in the
Mariner host kernel. To address this, we temporarily tweak the Mariner
CI to use an Ubuntu host and the Kata guest kernel, while retaining the
Mariner initrd. This is tracked in #9594.
Importantly, this allows us to preserve CI for genpolicy. We had to
tweak the default rules.rego however, as the OCI version is now
different in the Ubuntu host. This is tracked in #9593.
This change has been tested together with CH v39 in #9588.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
The build message shows that yq was not found when I tried to build
runtime binaries, but I've actually installed yq by yum install.
Signed-off-by: cncal <flycalvin@qq.com>
If device /dev/kvm does not exist, kata-runtime check would fail with
an ambiguous error messae 'no such file or directory'. I added a little
more details to make it understandable and it will belike:
```
ERRO[0000] cannot open kvm device: no such file or directory arch=arm64 check-type=full device=/dev/kvm name=kata-runtime pid=2849085 source=runtime
ERRO[0000] no such file or directory arch=arm64 name=kata-runtime pid=2849085 source=runtime
no such file or directory
```
Signed-off-by: cncal <flycalvin@qq.com>
- The testVersionString logic use regex to check that the ociVersion is
displayed correctly, but with the new go module that version has a
`+` in, so we need to quote this to escape special characters
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Dependabot doesn't follow all our commit format guidelines,
so add a check and skip these if the author is `dependabot[bot]`
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
With the addition of the 'qemu-coco-dev' runtimeClass we no longer need
to run CoCo tests on non-TEE environments with 'qemu'. As a result the
tests also no longer need to set the "io.katacontainers.config.hypervisor.image"
annotation to pods.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Created the runtimeclasses/kata-qemu-coco-dev.yaml file and updated the list
of SHIMS.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Created a new configuration to configure Kata for CoCo without requiring TEE
hardware so to allow developers implement/test/debug platform agnostic code
on their workstations. It will also ease testing of CoCo features on CI with
non-TEE supported VMs.
This is based off qemu configuration. The following differences applied:
- switched to confidential guest image/initrd
- switched to confidential kernel
- switched to 9p shared_fs
Fixes#9487
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Now that the `kata-agent` is being built with policy support, let's stop
building the `kata-opa-agent`, reducing the amount of things we need to
test and maintain.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Now that the OPA binary is not required anymore, let's start shipping
the agent with the policy enabled by default.
The agent *without* policy enabled has 30MB, while it's 34MB *with* the
policy enabled.
This 4MB (~10%) increase is, IMHO, worth it in order to reduce the
amount of components we have to maintain and test, including the
possibility to also reduce the amount of possible rootfs / initrd
images.
Whoever wants to use the agent without policy enabled can simply do that
by building their own agent. :-)
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Since OPA binary was replaced by the regorus crate, we can finally stop
building and shipping the binary.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we have an issue with a golang version for `run-cri-containerd`,
it is required to bump the language.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
iommu_platform support was already added on initial DeviceVhostUserFs
introduction, however it incorrectly enabled iommu_platform also on
non-CCW (e.g. PCI) systems.
Signed-off-by: Pavel Mores <pmores@redhat.com>
iommu_platform is only turned on for CCW systems.
PartialEq is added to VirtioBusType to enable the '==' operator.
Signed-off-by: Pavel Mores <pmores@redhat.com>
The adding itself is done by a new function add_iommu() that conforms with
the add_*() convention. Note though that this function is called
internally, by the QemuCmdLine constructor, simply because there's nothing
to trigger its invocation from QemuInner (unlike the other add_*()
functions so far).
Signed-off-by: Pavel Mores <pmores@redhat.com>
Remove reminder to initialize Policy earlier, because currently there
are no plans to initialize earlier.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
When docker is installed on the host system using script from https://get.docker.com/ it automatically creates a docker group with gid=999.
Then during docker build process of tarball, eg. make qemu-tdx-experimental-tarball docker is also installed inside the image with the same
script, which also automatically adds docker group with gid=999.
Then, the build tries to add a new group docker_on_host with gid=999, which already exists, which breaks the build.
Signed-off-by: Jakub Ledworowski <jakub.ledworowski@intel.com>
Avoid auto-generating Policy on platforms that haven't been tested
yet with auto-generated Policy.
Support for auto-generated Policy on these additional platforms is
coming up in future PRs, so the tests being fixed here were
prematurely enabled.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This should only be done once, and if CRI-O restarts, there's a big
chance kata-deploy will also restart and the user would end up with a
file that looks like:
```
[crio]
log_level = "debug"
[crio]
log_level = "debug"
[crio]
log_level = "debug"
...
```
And that would simply cause CRI-O to not start.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Implement Agent Policy using the regorus crate instead of the OPA
daemon.
The OPA daemon will be removed from the Guest rootfs in a future PR.
Fixes: #9388
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Remove k8s-policy-set-keys.bats in preparation for using the regorus
crate instead of the OPA daemon for evaluating the Agent Policy. This
test depended on sending HTTP requests to OPA.
Fixes: #9388
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Lock anyhow version to 1.0.58 because:
- Versions between 1.0.59 - 1.0.76 have not been tested yet using
Kata CI. However, those versions pass "make test" for the
Kata Agent.
- Versions 1.0.77 or newer fail during "make test" - see
https://github.com/kata-containers/kata-containers/issues/9538.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
In case of block devices using virtio-block, we need to pass the
pci-path as the storage source field to the agent.
Current the virt-path is being passed which works just for mmio block
devices.
In the future when support is added for scsi, block-ccw and pmem
devices, the storage source would need to be handled accordingly.
Fixes: #9034
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This commit is to add a new CI job to run-k8s-tests-on-zvsi.yaml.
Why the job is not configured in run-kata-coco-tests.yaml by having it
integrated with `run-k8s-tests-coco-nontee` is:
- It uses k3s instead of AKS
- It runs on a self-hosted runner
These differences make the integrated job not easy to read and maintain
when it comes to incorporating other platforms in the near future.
Fixes: #9467
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
In do_create_container and do_exec_process, we should create the proc_io first,
in case there's some error occur below, thus we can make sure
the io stream closed when error occur.
Signed-off-by: Tim Zhang <tim@hyper.sh>
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Fixes: #9334
In linux, when a FIFO is opened and there are no writers, the reader
will continuously receive the HUP event. This can be problematic.
To avoid this problem, we open stdin in write mode and keep the stdin-writer
We need to open the stdout/stderr as the read mode and keep the open endpoint
until the process is delete. otherwise,
the process would exit before the containerd side open and read
the stdout fifo, thus runD would write all of the stdout contents into
the stdout fifo and then closed the write endpoint. Then, containerd
open the stdout fifo and try to read, since the write side had closed,
thus containerd would block on the read forever.
Here we keep the stdout/stderr read endpoint File in the common_process,
which would be destroied when containerd send the delete rpc call,
at this time the containerd had waited the stdout read return, thus it
can make sure the contents in the stdout/stderr fifo wouldn't be lost.
Signed-off-by: Tim Zhang <tim@hyper.sh>
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
In scenario passfd-io, we should wait for stdin to close itself
instead of manually intervening in it.
Signed-off-by: Tim Zhang <tim@hyper.sh>
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
This patch adds a default implementation for the use_sandbox_pidns
and updates the structs that implement the K8sResource trait to use
the default.
Fixes: #8960
Signed-off-by: Archana Choudhary <archana1@microsoft.com>
- Provide default implementation for get_sandbox_name in K8sResource trait
- Remove default implementation from structs implementing the trait K8sResource
Fixes: #8960
Signed-off-by: Archana Choudhary <archana1@microsoft.com>
concurrently with itself
Based on 3a1461b0a5186a92afedaaea33ff2bd120d1cea0
Previously the tool would use the layers_cache folder for all instances
and hence delete the cache when it was done, interfereing with other
instances. This change makes it so that each instance of the tool will
have its own temp folder to use.
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
The new run-k8s-tests-coco-nontee job should be the home of attestation
tests.
Changed run-k8s-tests-coco-nontee to get KBS installed and by the time the
KBS variable is exported in the environment then the attestation tests
will kick in (likewise they will skip in run-k8s-tests-on-aks).
Fixes#9455
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This is to update a document `how-to-run-kata-containers-with-SE-VMs`
on using confidential artifacts to build a secure image.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This is to make `boot-image-se-tarball` use confidential kernel and
initrd instead of vanilla version of artifacts.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
updates golang.org/x/net to newer version that closes some reported
vulnerabilities and security issues
Fixes#9486
Signed-off-by: Adil Sadik <sparky.005@gmail.com>
1. EPOLLHUP events also need to be read and will be got len 0.
2. We should kill the connection when EPOLLERR events are received.
Signed-off-by: Tim Zhang <tim@hyper.sh>
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
This is to make a workflow `run-k8s-tests` and `run-cri-containerd`
(s390x and zvsi) run only on the runners labeled by `s390x-large`.
Fixes: #9507
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
- Bump the stalebot action version to v9 as that fixes the
```
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/stale@v8.
```
warning.
Fixes: #9512
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This commit is simply to remove a CI workflow `k8s-cri-containerd-rhel9-e2e-tests`.
Fixes: #9504
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
As documented in https://github.com/actions/stale?tab=readme-ov-file#start-date
> The start date is used to ignore the issues and pull requests created before the start date.
> Particularly useful when you wish to add this stale workflow on an existing repository
> and only wish to stale the new issues and pull requests.
As we don't want need to treat PRs older than May 2023 as a special case, then remove this option.
Fixes: #9502
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
We've discussed this over and over. Let's try to get to an agreement here.
I will use this issue to remove the mandatory Issue - PR dependency.
Fixes: #9500
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Configuration file for qemu with runtime-rs was recently renamed.
Doc contains name for old file. This was somehow not caught in the CI
earlier.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Document describes the steps needed to pass an entire Intel Discrete GPU
as well a GPU SR-IOV interface to a Kata Container.
Fixes: #9083
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Auto-generate the policy and then simulate attacks from the K8s
control plane by modifying the test yaml files. The policy then
detects and blocks those changes.
These test cases are using K8s Pods. Additional policy failures
are injected during CI using other types of K8s resources - e.g.,
using Jobs and Replication Controllers - from separate PRs.
Fixes: #9491
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The commit is to make the OPA build from source working in `ubuntu-rootfs-osbuilder`.
To achieve the goal, the configuration is changed as follows:
- Switch the make target to `ci-build-linux-static` not triggering docker-in-docker build
- Install go in the builder image for s390x and ppc64le
Fixes: #9466
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The new CoCo non-tee job introduced on commit 0d5399ba92 need to read secrets
like AZ_TENANT_ID, so run-kata-coco-tests workflow should inherit the secrets from
the caller workflow.
Fixes#9477
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Fixes: #9472
For initrd and image, the related NVIDIA will not use the default targets and we will pin them to a specific release.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
In order to build a coco {image,initrd}, it is required to
specify its name and version in versions.yaml. This commit
is to add the configuration for them, respectively.
Fixes: #9470
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
`CONFIG_TN3270_TTY` and `CONFIG_S390_AP_IOMMU` are dropped for s390x
in 6.7.x which is used for a confidential kernel.
But they are still used for a vanilla kernel. So we need to add them
to the whitelist.
Fixes: #9465
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This commit expands the VMM matrix for run-cri-containerd,
adding a new item `qemu-runtime-rs` for a test scenario where
the VMM is QEMU and runtime-rs is employed.
This expansion affects the workflows for both x86_64 and s390x platforms.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
To make `qemu-runtime-rs` working for CI, we have to rename a configuration
template file and `CONFIG_FILE_QEMU` in Makefile.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
`qemu-runtime-rs` will be utilized to handle a test scenario where
the VMM is QEMU and runtime-rs is employed.
Note: Some of the tests are skipped. They are going to be reintegrated in
the follow-up PR (Check out #9375).
Fixes: #9371
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Linux kernel generates a panic when the init process exits.
The kernel is booted with panic=1, hence this leads to a
vm reboot.
When used as a service the kata-agent service has an ExecStop
option which does a full sync and shuts down the vm.
This patch mimicks this behavior when kata-agent is used as
the init process.
Fixes: #9429
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
isClhRunning uses signal 0 to test whether the process is
still alive or not. This doesn't work because the process is a
direct child of the shim. Once it is dead the process becomes
zombie.
Since no one waits for it the process lingers until
its parent dies and init reaps it. Hence sending signal 0 in
isClhRunning will always return success whether the process is
dead or not.
This patch calls wait to reap the process, if it succeeds that
means it is our child process, if not we send the signal.
Fixes: #9431
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Auto-generate the policy and then simulate attacks from the K8s
control plane by modifying the test yaml files. The policy then
detects and blocks those changes.
These test cases are using K8s Replication Controllers. Additional
policy failures will be injected using other types of K8s resources
- e.g., using Pods and/or Jobs - in separate PRs.
Fixes: #9463
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
genpolicy is a handy tool to use in CI systems, to prepare workloads
before applying them to the Kubernetes API server. However, many modern
build systems like Bazel or Nix restrict network access, and rightfully
so, so any registry interaction must take place on localhost.
Configuring certificates for localhost is tricky at best, and since
there are no privacy concerns for localhost traffic, genpolicy should
allow to contact some registries insecurely. As this is a runtime
environment detail, not a target environment detail, configuring
insecure registries does not belong into the JSON settings, so it's
implemented as command line flags.
Fixes: #9008
Signed-off-by: Markus Rudy <webmaster@burgerdev.de>
By passing --overwrite-existing to `aks get-credentials` it will stop
asking if I want to overwrite the existing credentials. This is handy
for running the scripts locally.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
When running on non-TEE environments (e.g. KATA_HYPERVISOR=qemu) the tests should
be stressing the CoCo image (/opt/kata/share/kata-containers/kata-containers-confidential.img)
although currently the default image/initrd is built to be able to do guest-pull as well.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Enabled guest-pull tests on non-TEE environment. It know requires the SNAPSHOTTER environment
variable to avoid it running on jobs where nydus-snapshotter is not installed
Fixes: #9410
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Created the new run-k8s-tests-coco-nontee jobs for running CoCo tests on
non-TEE. It currently generates the run-k8s-tests-coco-nontee(qemu, nydus, guest-pull)
job only to run the guest-pull tests.
Fixes: #9410
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This PR adds the function to uninstall kbs client command function
specially when we are running with baremetal devices.
Fixes#9460
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
CRIs don't always use a pause container, but even if they do the
concrete container choice is not specified. Even if the CRI config can
be tweaked, it's not guaranteed that registries in the public internet
can be reached. To be portable across CRI implementations and
configurations, the genpolicy user needs to be able to configure the
container the tool should append to the policy.
Signed-off-by: Markus Rudy <webmaster@burgerdev.de>
Decouple initialization of the Settings struct from creating the
AgentPolicy struct, so that the settings are available for evaluating,
extending or overriding command line arguments.
Signed-off-by: Markus Rudy <webmaster@burgerdev.de>
This patch introduces a one-time cpath to mitigate the cgroup residuals. It
might break the device cgroup merging rules when the cgroup has children.
Fixes: #9456
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This PR defines the PULL_TYPE variable to avoid failures of unbound
variable when this is being test it locally.
Fixes#9453
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Auto-generate the policy and then simulate attacks from the K8s
control plane by modifying the test yaml files. The policy then
detects and blocks those changes.
These test cases are using K8s Jobs. Additional policy failures
will be injected using other types of K8s resources - e.g., using
Pods and/or Replication Controllers - in future PRs.
Fixes: #9406
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR adds onednn test to exercise additional ML benchmarks.
Onednn is an Intel-optimized library for Deep Neural Networks.
Fixes: #9390
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR adds openvino test in order to exercise additional ML
benchmarks.
OpenVino bench used to optimize and deploy deep learning models.
Fixes: #9389
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Include HTTP and HTTPS env variables in the building docker
images because they are required to download packages
such as Phoronix.
Added a restriction that verifies that docker building images
is performed as root.
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Adds a function that receives as a single parameter the name of
a valid Kata configuration file which will be established as
the default kata configuration to start kata containers.
Adds a second function that returns the path to the current
kata configuration file.
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Add an extra function that updates kata config
to use the max num. of vcpus available and
to use the available memory in the system.
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This is actually a first attempt to document our CI, and all this
content was based on the document created by Fabiano Fidencio (kudos to
him). We are just moving the content and discussion from Google Docs to
here.
I used the "poetic license" to add some notes on what I believe our CI
will look like in the future.
Fixes#9006
Co-authored-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Beraldo Leal <bleal@redhat.com>
- Add v1 image test case
- Install protobuf-compiler in build check
- Reset containerd config to default in kubernetes test if we are testing genpolicy
- Update docker_credential crate
- Add test that uses default pull method
- Use GENPOLICY_PULL_METHOD in test
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
This PR improves the kbs_k8s_delete function to verify that the
resources were properly deleted for baremetal environments.
Fixes#9379
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Add optional toggle to use existing containerd installation to pull and manage container images.
This adds support to a wider set of images that are currently not supported by standard pull method,
such as those that use v1 manifest.
Fixes: #9144
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
- Set KBC_PROVIDER and ATTESTER rather than TEE_PLATFORM
to avoid tss build issues for vTPM attester(s)
- There are future plans to make a matching TEE_PLATFORM, so this can be simplified once that is available
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- At the moment we aren't supporting ppc64le or
aarch64 for
CoCo, so filter out these tests from running
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Switch to Ubuntu 20.04 for building guest-components as
The rootfs is based on 20.04, so we need matching GLIBC versions.
See #8955
- Add dependencies needed by TDX verifier as we want to build for all platforms
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Add basic test case to check that a ruuning
pod can use the api-server-rest (and attestation-agent
and confidential-data-hub indirectly) to get a resource
from a remote KBS
Fixes#9057
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Co-authored-by: Linda Yu <linda.yu@intel.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Support different pause images in the guest for guest-pull, such as k8s
pause image (registry.k8s.io/pause) and openshift pause image (quay.io/bpradipt/okd-pause).
Fixes: #9225 -- part III
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This commit is a mess, but I'm not exactly sure what's the best way to
make it less messy, as we're getting QEMU TDX to work while partially
reverting 1e34220c41.
With that said, let me cover the content of this commit.
Firstly, we're reverting all the changes related to
"memory-backend-memfd-private", as that's what was used with the
previous host stack, but it seems it
didn't fly upstream.
Secondly, in order to get QEMU to properly work with TDX, we need to
enforce the 'private=on' knob and use the "memory-backend-ram", and
we're doing so, and also making sure to test the `private=on` newly
added knob.
I'm sorry for the confusion, I understand this is not optimal, I just
don't see an easy path to do changes without leaving the code broken
during those changes.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
`Node.js 19` is deprecated. Bump to a new version based on `Node.js 20`.
This fixes all remaining sites.
Fixes#9245
Signed-off-by: Greg Kurz <groug@kaod.org>
`Node.js 19` is deprecated. Bump to a new version based on `Node.js 20`.
As explained at [1] :
> The contents of an Artifact are uploaded together into an immutable
> archive. They cannot be altered by subsequent jobs. Both of these
> factors help reduce the possibility of accidentally corrupting
> Artifact files.
This means that artifacts cannot have the same name.
Adapt the `run-k8s-tests-on-garm` workflow accordingly by embedding all
the other `${{ vmm.* }}` fields and `${{ inputs.tag }}` in the artifact
names that would otherwise collide.
Fixes#9245
Signed-off-by: Greg Kurz <groug@kaod.org>
`Node.js 19` is deprecated. Bump to a new version based on `Node.js 20`.
As explained at [1] :
> The contents of an Artifact are uploaded together into an immutable
> archive. They cannot be altered by subsequent jobs. Both of these
> factors help reduce the possibility of accidentally corrupting
> Artifact files.
This means that artifacts cannot have the same name.
Adapt all `build-kata-static-tarball` workflows accordingly by
embedding `${{ matrix.asset }}` in the artifact names that would
otherwise collide.
Fixes#9245
Signed-off-by: Greg Kurz <groug@kaod.org>
Occasionally, the removal of GITHUB_WORKSPACE fails for self-hosted runners
because one of the subdirectories is not empty. This is likely due to another
process occupying the directory at the time.
Implementing a secondary cleanup resolves this issue.
This commit focuses on the implementation for the secondary cleanup.
Fixes: #9317
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This reverts commit d1b54ede29.
Conflicts:
src/runtime/virtcontainers/qemu.go
This commit was a hack that was needed in order to get QEMU + TDX to
work atop of the stack our CI was running on. As we're moving to "the
officially supported by distros" host OS, we need to get rid of this.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The private=on|off knob is required in order to properly lauunch a TDX
guest VM.
This is a brand new property that is part of the still in-flight patches
adding TDX support on QEMU.
Please, see:
3fdd8072da
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's update the QEMU to the one that's officially maintained by Intel
till all the TDX patches make their way upstream.
We've had to also update python to explicitly use python3 and add
python3-venv as part of the dependencies.
Fixes: #8810
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR improves the latency test cleanup in order to avoid random
failures of leaving the pods.
Fixes#9418
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
There's an rg name duplication situation that got introduced by #9385
where 2 different test runs might have same rg name.
Add back uniqueness by including the first letter of GENPOLICY_PULL_METHOD to
cluster name.
Fixes: #9412
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
We used to utilize go runtime's "NumCPUs()", which will give the number
of cores available to the Go runtime, which may be a subset of physical
cores if the shim is started from within a cpuset. From the function's
description:
"NumCPU returns the number of logical CPUs usable by the current
process."
As an example, if containerd is run from within a smaller CPUset, the
maximum size of a pod will be dictated by this CPUset, instead of what
will be available on the rest of the system.
Since the shim will be moved into its own cgroup that may have a
different CPUset, let's stick with checking physical cores. This also
aligns with what we have documented for maxVCPU handling.
In the event we fail to read /proc/cpuinfo, let's use the goruntime.
Fixes: #9327
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
This PR defines the GH_PR_NUMBER variable in gha run k8s common
script to avoid failures like unbound variable when running
locally the scripts just like the GHA CI.
Fixes#9408
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
updating the machine config takes even longer than 1200s, use 60m to be
sure everything is updated.
Fixes: #9338
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Try to reduce duplicated code in decrease_attach_count with public
new function do_decrease_count.
Fixes: #8738
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Try to reduce duplicated code in increase_attach_count with public
new function do_increase_count.
Fixes: #8738
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Introduce a dedicated public function do_decrease_count to
reduce duplicated code in drivers' decrease_attach_count.
Fixes: #8738
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Since there are many implementations of reference counting in the
drivers, all of which have the same implementation, we should try
to reduce such duplicated code as much as possible. Therefore, a
new function is introduced to solve the problem of duplicated code.
Fixes: #8738
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Use the "allow all" policy for k8s-sandbox-vcpus-allocation.bats,
instead of relying on the Kata Guest image to use the same policy
as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-nginx-connectivity.bats, instead of
relying on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-volume.bats, instead of relying
on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-sysctls.bats, instead of
relying on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-security-context.bats, instead of
relying on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-seccomp.bats, instead of relying
on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-projected-volume.bats, instead of
relying on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-pod-quota.bats, instead of
relying on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-optional-empty-secret.bats,
instead of relying on the Kata Guest image to use the same policy as
its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-measured-rootfs.bats, instead of
relying on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-liveness-probes.bats, instead of
relying on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-inotify.bats, instead of relying
on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-guest-pull-image.bats, instead of
relying on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-footloose.bats, instead of
relying on the Kata Guest image to use the same policy as its
default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-empty-dirs.bats, instead of
relying on the Kata Guest image to use the same policy as its
default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Check from:
- k8s-exec-rejected.bats
- k8s-policy-set-keys.bats
if policy testing is enabled or not, to reduce the complexity of
run_kubernetes_tests.sh. After these changes, there are no policy
specific commands left in run_kubernetes_tests.sh.
add_allow_all_policy_to_yaml() is moving out of run_kubernetes_tests.sh
too, but it not used yet. It will be used in future commits.
Fixes: #9395
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Add GENPOLICY_PULL_METHOD that will be used to test pulling
container images in genpolicy using the oci-distribution crate
and/or the containerd interface.
GENPOLICY_PULL_METHOD will start being used in a future PR.
Fixes: #9384
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
This PR removes conmon information from versions.yaml as this is not
longer being used in kata containers repository.
Fixes#9396
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Don't add the "allow all" policy to all the test YAML files anymore.
After this change, the k8s tests assume that all the Kata CI Guest
rootfs image files either:
- Don't support Agent Policy at all, or
- Include an "allow all" default policy.
This relience/assumption will be addressed in a future commit.
Fixes: #9395
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Configure the system to mount cgroups-v2 by default during system boot
by the systemd system, We must add systemd.unified_cgroup_hierarchy=1
parameter to kernel cmdline, which will be passed by kernel_params in
configuration.toml.
To enable cgroup-v2, just add systemd.unified_cgroup_hierarchy=true[1]
to kernel_params.
Fixes: #9336
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
When there's a pod with multiple containers, there may be case that
attach point more than 2, we should not return Err in that case when
we are doing attach ops, but just return Ok.
Fixes: #8738
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This reverts commit 1c5693be86.
Avoid apparent infinite loop when ReadStreamRequest is blocked by
policy - for some of the pods.
When running the k8s-limit-range.bats test with Policy enabled,
the Shim + VMM never get terminated on my cluster. Not sure why
the sandbox clean-up works better for other tests, but the
k8s-limit-range test pod gets stuck in an infinite loop:
stdout io stream copy error happens: error = %wrpc error: code =
PermissionDenied desc = \"ReadStreamRequest is blocked by policy
...
policy check: ReadStreamRequest
...
stdout io stream copy error happens: error = %wrpc error: code =
PermissionDenied desc = \"ReadStreamRequest is blocked by policy
...
policy check: ReadStreamRequest
...
Fixes: #9380
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The 3.3.0 release installs the `kata-manager` script with overly restrictive
permissions (see #9373), so add details to help users handle the situation.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This PR removes the runc version information as this is not longer being used
in the kata containers scripts.
Fixes#9364
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Since we're removing the unused service_offload parameter,
don't set it in any of the packaging scripts.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Since we no longer use the service_offload configuration,
remove the ServiceOffload field from the image struct.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
These experimental options were added 2 years ago
in anticipation of features that would be added
in CoCo. These do not match the features that were
eventually added and will soon be ported to main.
Fixes: #8047
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
This is the report from `make check`:
```
error: this loop never actually loops
--> src/signal.rs:147:9
|
147 | / loop {
148 | | select! {
149 | | _ = handle => {
150 | | println!("INFO: task completed");
... |
156 | | }
157 | | }
| |_________^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#never_loop
= note: `#[deny(clippy::never_loop)]` on by default
```
There is only one option: you get something or a timeout. You never retry, so
the report is correct.
Fixes: #9342
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
The lint report is the following:
```
error: `flatten()` will run forever if the iterator repeatedly produces an `Err`
--> src/rpc.rs:1754:10
|
1754 | .flatten()
| ^^^^^^^^^ help: replace with: `map_while(Result::ok)`
|
note: this expression returning a `std::io::Lines` may produce an infinite number of `Err` in case of a read error
--> src/rpc.rs:1752:5
|
1752 | / reader
1753 | | .lines()
| |________________^
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#lines_filter_map_ok
= note: `-D clippy::lines-filter-map-ok` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::lines_filter_map_ok)]`
```
This commit simply applies the suggestion.
Fixes: #9342
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
Running `make check` in the `src/agent` directory gives:
```
error: you seem to use `.enumerate()` and immediately discard the index
--> rustjail/src/mount.rs:572:27
|
572 | for (_index, line) in reader.lines().enumerate() {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unused_enumerate_index
= note: `-D clippy::unused-enumerate-index` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::unused_enumerate_index)]`
help: remove the `.enumerate()` call
|
572 | for line in reader.lines() {
| ~~~~ ~~~~~~~~~~~~~~
Checking tokio-native-tls v0.3.1
Checking hyper-tls v0.5.0
Checking reqwest v0.11.18
error: could not compile `rustjail` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
make: *** [../../utils.mk:177: standard_rust_check] Error 101
```
Fixes: #9342
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
support to configure CreateContainerRequestTimeout in the
configurations.
e.g.:
[runtime]
...
create_container_timeout = 300
Note: The effective timeout is determined by the lesser of two values: runtime-request-timeout from kubelet config
(https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/#:~:text=runtime%2Drequest%2Dtimeout) and create_container_timeout.
In essence, the timeout used for guest pull=runtime-request-timeout<create_container_timeout?runtime-request-timeout:create_container_timeout.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Most of the content of `docs/Stable-Branch-Strategy.md` got de-facto
deprecated by the re-design of the release process described in #9064.
Remove this file and all its references in the repo.
The `## Versioning` section has some useful information though. It is
moved to `docs/Release-Process.md`. The documentation of the `PATCH`
field is adapted according to new workflow.
Fixes#9064 - part VI
Signed-off-by: Greg Kurz <groug@kaod.org>
Support to configure CreateContainerRequestTimeout in the annotations.
e.g.:
annotations:
"io.katacontainers.config.runtime.create_container_timeout": "300"
Note: The effective timeout is determined by the lesser of two values: runtime-request-timeout from kubelet config
(https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/#:~:text=runtime%2Drequest%2Dtimeout) and create_container_timeout.
In essence, the timeout used for guest pull=runtime-request-timeout<create_container_timeout?runtime-request-timeout:create_container_timeout.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
In the situation to pull images in the guest #8484, it’s important to account for pulling large images.
Presently, the image pull process in the guest hinges on `CreateContainerRequest`, which defaults to a 60-second timeout.
However, this duration may prove insufficient for pulling larger images, such as those containing AI models.
Consequently, we must devise a method to extend the timeout period for large image pull.
Fixes: #8141
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This PR updates the journal log name for nerdctl artifacts to make
sure that we have different names in case we add a parallel GHA job.
Fixes#9357
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
the service might not listen on the default port, use the full service
address to ensure we are talking to the right resource.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
this can be used on kcli or other systems where cluster nodes are
accessible from all places where the tests are running.
Fixes: #9272
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
The automated release workflow starts with the creation of the release in
GitHub. This is followed by the build and upload of the various artifacts,
which can be very long (like hours). During this period, the release appears
to be fully available in https://github.com/kata-containers/kata-containers/
even though it lacks all the artifacts. This might be confusing for users
or automation consuming the release.
Create the release as draft and clear the draft flag when all jobs are
done. This ensure that the release will only be tagged and made public
when it is fully usable.
If some job fails because of network timeout or any other transient
error, the correct action is to restart the failed jobs until they
eventually all succeed. This is by far the quicker path to complete
the release process.
If the workflow is *canceled* for some reason, the draft release is left
behind. A new run of the workflow will create a brand new draft release
with the same name (not an issue with GitHub). The draft release from
the previous run should be manually deleted. This step won't be automated
as it looks safer to leave the decision to a human.
[1] https://github.com/kata-containers/kata-containers/releasesFixes#9064 - part VI
Signed-off-by: Greg Kurz <groug@kaod.org>
The remaining code in network.rs was mostly moved to utils.rs which seems
better home for these utility functions anyway (and a closely related
function open_named_tuntap() has already lived there).
ToString implementation for Address was removed after some consideration.
Address should probably ideally implement Display (as per RFC 565) which
would also supply a ToString implementation, however it implements Debug
instead, probably to enable automatic implementation of Debug for anything
that Address is a member of, if for no other reason. Rather than having
two identical functions this commit simply switches to using the Debug
implementation for printing Address on qemu command line.
Fixes#9352
Signed-off-by: Pavel Mores <pmores@redhat.com>
Make file descriptors to be passed to qemu owned by QemuCmdLine. See
commit 52958f17cd for more explanation.
Signed-off-by: Pavel Mores <pmores@redhat.com>
add_network_device() doesn't need to be passed NetworkInfo since it
already has access to the full HypervisorConfig.
Also, one of the goals of QemuCmdLine interface's design is to avoid
coupling between QemuCmdLine and the hypervisor crate's device module,
if at all possible. That's why add_network_device() shouldn't take
device module's NetworkConfig but just parts that are useful in
add_network_device()'s implementation.
Signed-off-by: Pavel Mores <pmores@redhat.com>
is_running_in_vm() is enough to figure out whether to disable_modern but
it's clumsy and verbose to use. should_disable_modern() streamlines the
usage by encapsulating the verbosity.
Signed-off-by: Pavel Mores <pmores@redhat.com>
This commit replaces the existing NetDevice-based implementation with one
using Netdev and DeviceVirtioNet.
Signed-off-by: Pavel Mores <pmores@redhat.com>
In keeping with architecture of QemuCmdLine implementation we split the
functionality into two objects: Netdev to represent and generate the
-netdev part and DeviceVirtioNet for the -device virtio-net-<transport>
part.
This change is a pure refactor, existing functionality does not change.
However, we do remove some stub generalizations and govmm-isms, notably:
- we remove the NetDev enum since the only network interface types that
kata seems to use with qemu are tuntap and macvtap, both of which are
implemented by the same -netdev tap
- enum DeviceDriver is also left out since it doesn't seem reasonable to
try to represent VFIO NICs (which are completely different from
virtio-net ones) with the same struct as virtio-net
- we also remove VirtioTransport because there's no use for it so far, but
with the expectation that it will be added soon.
We also make struct Netdev the owner of any vhost-net and queue file
descriptors so that their lifetime is tied ultimately to the lifetime of
QemuCmdLine automatically, instead of returning the fds to the caller and
forcing it to achieve the equivalent functionality but manually.
Signed-off-by: Pavel Mores <pmores@redhat.com>
generate_netdev_fds() takes NetworkConfig from which it however only needs
a host-side network device name. This commit makes it take the device name
directly, making the function useful to callers who don't have the whole
NetworkConfig but do have the requisite device name.
Signed-off-by: Pavel Mores <pmores@redhat.com>
The idea of this function is to make sure O_CLOEXEC is not set on file
descriptors that should be inherited by a child (=hypervisor) process.
The approach so far is however rather heavy-handed - clearing *all* flags
is unjustifiably aggresive for a low-level function with no knowledge of
context whatsoever.
This commit refactors the function so that it only does what's expected
and renames it accordingly. It also clarifies some of its call sites.
Signed-off-by: Pavel Mores <pmores@redhat.com>
Since v2.2.6 it can detect TDX guests on Azure, so let's bump it even if
Azure peer-pods are not currently used as part of our CI.
Fixes: #9348
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Kata CI has full debug output enabled for the cbl-mariner k8s tests,
and the test AKS node is relatively slow. So debug prints from policy
are expensive during CI.
Fixes: #9296
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The lib.sh script uses the right directory but the wrong path for the
script that installs yq; fix it.
Fixes#9165
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Remove links to tests repo and update with corresponding location in the
current repo.
Fixes#9165
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Change scripts and source that uses files in the tests repo to use the
corresponding file in the current repo.
Fixes#9165
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Now that the version is an invariant for the entire workflow, it
isn't required to obtain it with an environment variable. Just
rely on the content of the `VERSION` file like other actions.
Fixes#9064 - part VI
Signed-off-by: Greg Kurz <groug@kaod.org>
We need some docs about how to build a guest kernel to support
both Upcall and Nvidia GPU Passthrough(hotplug) at the same time.
This patch is to do such thing to help users to build a guest
kernel with support both Upcall and Nvidia GPU hotplug/unlplug.
Fixes: #9140
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Add a document for kata guest image management design.
Related feature: #8484Fixes: #9225 -- part I
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
CONFIG_CRYPTO_ECDSA is not supported in older kernels such as 5.10.x
which may cause building broken problem if we build such kernel with
NVIDIA GPU in version 5.10.x
So this patch is to add CONFIG_CRYPTO_ECDSA into whitelist.conf to
avoid break building guest kernel with NVIDIA GPU.
Fixes: #9140
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Currently, `.lock().await.clone()` results in `Option<ImageService>` being duplicated in memory with each call to `singleton()`.
Consequently, if kata-agent receives numerous image pulling requests simultaneously,
it will lead to the allocation of multiple `Option<ImageService>` instances in memory, thereby consuming additional memory resources.
In image.rs, we introduce two public functions:
`merge_bundle_oci()` and `init_image_service()`. These functions will encapsulate
the operations on `IMAGE_SERVICE`, ensuring that its internal details remain
hidden from external modules such as `rpc.rs`.
Fixes: #9225 -- part II
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
It is observed that virtiofsd exits immediately on s390x
if there is no attached console devices.
This commit resolves the issue by migrating `appendConsole()`
from runtime and being triggered in `start_vm()`.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
For s390x, it requires an additional option `memory-backend` for `-machine`.
Otherwise, virtiofsd exits with HandleRequest(InvalidParam).
This commit is to add a field `memory_backend` to `struct Machine`
and turn it on for s390x.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Like nvdimm for x86_64, a block device for s390x should be
treated differently with `virtio-blk-ccw`.
This is to generate a QEMU command line parameter for a block
device by using `-blockdev` and `-device` if the `vm_rootfs_driver`
is set to `virtio-blk-ccw`.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This version brings in a fix for cleaning up k3s/rke2 environments,
which directly impacts the TDX machine that's part of our CI.
Fixes: #9318
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR fixes the unbound variables error when trying to run
the setup script locally in order to avoid errors.
Fixes#9328
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Add in the full details once cloud-hypervisor/cloud-hypervisor#6103
has been implemented, and the feature is available in a Cloud Hypervisor
release.
Fixes: #8799
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Currently, `*-pci` is used as an argument for the device config.
It is not true for a case where a different type of bus is used.
s390x uses `ccw`.
This commit is to make it flexible to generate the device argument
based on the bus type. A structure `DeviceVhostUserFsPci` and
`VhostVsockPci` is renamed to `DeviceVhostUserFs` and `VhostVsock`
because the structure name is not bound to a certain bus type any more.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This PR updates the journal log names for kubernetes artifacts
in order to make sure that we have different names when we are
running parallel GHA jobs.
Fixes#9308
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR terminates the nydus namespace to avoid the error of
that the flag needs an argument.
Fixes#9264
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This reverts commit 0fa59ff94b, as now
we'll be able to use the `--generate-notes`, hopefully, without blowing
the allowed limit.
Fixes: #9064 - part VI
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Added an announcement message to the `static-checks.sh` script. It runs
platform / architecture specific code so it would be useful to display
details of the platform the checker is running on to help with
debugging.
Fixes: #9258.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This release is a special case, as we've slacked for 6 months and the
release content is way too long ... long enough to exceed the allowed
limit for the release notes.
With this in mind we'll just remove the `--generate-notes` for now, and
then revert this commit as soon as the release is out, as releases
should be happening every month and, ideally, we won't reach this
situation never ever again.
Fixes: #9064 - part V
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It has been observed that the runtime stops running around
`sysinfo::total_memory()` while adjusting a config on s390x.
This is to update the crate to the latest version which happened
to resolve the issue. (No explicit release note for this)
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
MTRR, or Memory-Type Range Registers are a group of x86 MSRs providing a way to control access
and cache ability of physical memory regions.
During our test in runtime-rs + Dragonball, we found out that this register support is a must
for passthrough GPU running CUDA application, GPU needs that information to properly use GPU memory.
fixes: #9310
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Those two architectures are not TEE capable, thus we can just skip
running those tests there.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is needed as the TDX machine is hosted inside Intel and relies on
proxies in order to connect to the external world. Not having those set
causes issues when pulling the image inside the guest.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure those two proxy settings are respected, as those will be
widely used when pulling the image inside the guest on the Confidential
Containers case.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When the https_proxy/no_proxy settings are configured alongside agent-policy enabled, the process of pulling image in the guest will hang.
This issue could stem from the instantiation of `reqwest`’s HTTP client at the time of agent-policy initialization,
potentially impacting the effectiveness of the proxy settings during image guest pulling.
Given that both functionalities use `reqwest`, it is advisable to set https_proxy/no_proxy prior to the initialization of agent-policy.
Fixes: #9212
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
To access the URL, the component to pull image in the guest needs to send a request to the remote.
Therefore, we need to add CA to the rootfs.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Enable to build kata-agent with PULL_TYPE feature.
We build kata-agent with guest-pull feature by default, with PULL_TYPE set to default.
This doesn't affect how kata shares images by virtio-fs. The snapshotter controls the image pulling in the guest.
Only the nydus snapshotter with proxy mode can activate this feature.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
To support handle image-guest-pull block volume from different CRIs, including cri-o and containerd.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
By default the pause image and runtime config will provided
by host side, this may have potential security risks when the
host config a malicious pause image, then we will use the pause
image packaged in the rootfs.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Arron Wang <arron.wang@intel.com>
Co-authored-by: Julien Ropé <jrope@redhat.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Add "guest-pull" feature option to determine that the related dependencies
would be compiled if the feature is enabled.
By default, agent would be built with default-pull feature, which would
support all pull types, including sharing images by virtio-fs and
pulling images in the guest.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
support to pass image information to guest by KataVirtualVolumeImageGuestPullType
in KataVirtualVolume, which will be used to pull image on the guest.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
As we do not employ a forked containerd in confidential-containers, we utilize the KataVirtualVolume
which storing the image information as an integral part of `CreateContainer`.
Within this process, we store the image information in rootfs.storage and pass this image url through `CreateContainerRequest`.
This approach distinguishes itself from the use of `PullImageRequest`, as rootfs.storage is already set and initialized at this stage.
To maintain clarity and avoid any need for modification to the `OverlayfsHandler`,we introduce the `ImagePullHandler`.
This dedicated handler is responsible for orchestrating the image-pulling logic within the guest environment.
This logic encompasses tasks such as calling the image-rs to download and unpack the image into `/run/kata-containers/{container_id}/images`,
followed by a bind mount to `/run/kata-containers/{container_id}`.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Containerd can support set a proxy when downloading images with a environment variable.
For CC stack, image download is offload to the kata agent, we need support similar feature.
Current we add https_proxy and no_proxy, http_proxy is not added since it is insecure.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Arron Wang <arron.wang@intel.com>
With image-rs pull_image API, the downloaded container image layers
will store at IMAGE_RS_WORK_DIR, and generated bundle dir with rootfs
and config.json will be saved under CONTAINER_BASE/cid directory.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Arron Wang <arron.wang@intel.com>
Co-authored-by: Jiang Liu <gerry@linux.alibaba.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: wllenyj <wllenyj@linux.alibaba.com>
This commit updates `filter_k8s_test.sh` to handle skipped tests that
include comments. In addition to the existing parameter expansion,
the following expansions have been added:
- Removal of a comment
- Stripping of trailing spaces
Fixes: #9304
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This is to place a pre-action step for the kata-deploy job in order to
clean up the github workspace directory before checking out the repo.
Fixes: #9301
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Currently we're only running the small instance tests. This adds the
normal instance tests as well.
Fixes: #9298
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Fix the Docker install by removing the second (erroneous) call to
`containerd_installed()` in `handle_docker()`.
Without this fix, installing using Docker (`-D`) will work *iff* you
already have containerd installed. However, if you do not have
containerd installed, the `containerd_installed()` function returns 1,
which exits the script as we're running with `set -e`, leaving a broken
Docker installation.
> **Note:** containerd is installed via Docker's `get-docker.sh` script.
Fixes: #9292.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Now we don't have minor and major releases and
we are now generating a new version
in the release workflow, we can
tidy up the arch specific releases workflows to remove
the extra required inputs
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This drops the documentation of the legacy release scripts and adds
a quick description of the scripts of the new workflow. It also
highlights the bump of the `VERSION` file.
Signed-off-by: Greg Kurz <groug@kaod.org>
All releases are now created in the `main` branch following
the very same workflow. No need to special case pre-releases.
Signed-off-by: Greg Kurz <groug@kaod.org>
This is done in order to avoid having to push a commit to the main
branch, which is against the defined rules on GitHub.
By doing this, we need to educate ourselves to always bump the VERSION
file as soon as a release is cut out.
As a side effect of this change, we can drop the release-major and
release-minor workflows, as those are not needed anymore.
Fixes: #9064 - part IV
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
File descriptors that are passed to QEMU need some special care.
We want them to be closed when the QEMU process is started. But
at the same time, it is required that the associated rust File
structures, either coming from the` std::fs` or the `tokio::fs`
crates, are still in scope when the QEMU process is forked. This
is currently achieved by keeping File structures in variables
at the outer scope of `start_vm()`. This scheme is currently
duplicated, with similar justifications in the corresponding
comments.
Consolidate all this handling in one place with a more generic
explanation.
Fixes#9281
Signed-off-by: Greg Kurz <groug@kaod.org>
The agent now has a number of optional build-time features that can be
enabled.
Add details of these features to the following areas:
- Version output (`kata-agent --version`)
- Announce message (so that the details are always added to the journal
at agent startup).
- The response message returned by the ttRPC `GetGuestDetails()` API.
Fixes: #9285.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Some previous contribution missed to run cargo clippy.
Fix the dependency now so that it doesn't cause noise
in future contributions.
Signed-off-by: Greg Kurz <groug@kaod.org>
Fixes: #9269
From https://github.com/opencontainers/runtime-spec/blob/main/config.md#mounts
type (string, OPTIONAL) The type of the filesystem to be mounted.
bind may be only specified in the oci spec options -> flags update r#type
The agent will ignore bind mounts if they are only specified in the OCI spec options and not in the flags.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
s390x supports a different machine type `s390-ccw-virtio` and it is
not required to configure cpu features by default for the platform.
A hypervisor `dragonball` is not supported on s390x so that `DBCMD`
is not necessary. `vm-rootfs_driver` should be set to `virtio-blk-ccw`.
This commit is to set the architecture-specific flags for Makefile.
Fixes: #9158
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Initialize $GITHUB_ENV to avoid nounset error when running the scripts locally
out of Github Actions.
Fixed commit 9ba5e3d2a8Fixes#9217
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The guest_hook_path item in configuration.toml allows OCI hook scripts
to be executed within Kata's guest environment. Traditionally, these
guest hook programs are pre-built and included in Kata's guest rootfs
image at a fixed location.
While setting guest_hook_path = "/usr/share/oci/hooks" in configuration.toml
works, it lacks flexibility. Not all guest hooks reside in the path
/usr/share/oci/hooks, and users might have custom locations.
To address this, a more flexible and configurable approach is to be proposed
that allows users to specify their desired path. This could include using a
sandbox bind mount path for hooks specific to that particular container.
However, The current implementation of guest hooks and bind mounts in kata-agent
has a reversed order of execution compared to the desired behavior.
To achieve the intended functionality, we simply need to swap the order of their
implementation.
Fixes: #9274
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Fixes: #9267
The doc states we have support for all lifecycle hooks. There are still some missing.
This is the first issue regarding the CreateContainer hook which is run before pivot_root but after prestart and createruntime
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This PR ensures that the self hosted runner is prepared by taking
necesary actions before running the workflow. The script prepare_runner.sh
checks the following:
1. Ensure that containerd/docker is up and running
2. Make sure that the repository workspace is cleaned up and has no conflicts
3. Remove/cleanup any leftover files from the previous runs
Fixes: #9262
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
we are grepping for an expected message but it might contain extra bits
of information fruitful for later debugging. Let's include it in the
output and the full log in case of an error.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
we're hitting this timeout quite often, looks like newer OCP takes
longer to reconfigure. Increase the timeout to 1200.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
This script doesn't serve as a complete cleanup, but it can be used as a
best-effort cleaner between deploying different versions of
kata-containers on the same OCP cluster.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
sometimes we want to test a different than the latest image (eg. when
verifying a PR via ghcr images or when bisecting a failure over older
builds). Let's add a KATA_DEPLOY_IMAGE variable for that while keeping
the latest image by default.
Fixes: #9228
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
previously we only replaced the image when the previously defined one
matched the "old_img". This is good to avoid modifying developers custom
changes, but it might lead to hard-to-debug issues when the image stays
different. Let's ensure we always replace the image with the one we
asked for.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
the latest upstream kata-containers requires the skip_mount_home to be
enabled, which is default on OCP 4.14+ but disabled on OCP 4.13-. Let's
use a "WORKAROUND_9206_CRIO" (called by kata-containers GH issue)
variable to allow users to enable this treatement when needed.
Related to: #9206
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Instead of failing the pipeline let's proceed with an error message that
selinux setup failed so, in case of a later failure, we know what might
have caused it while keeping the coverage in case of a false setup
issue.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
previously we only waited for a random pod to finish the selinux
relabel, which could be error-prone. Let's wait for all of the podst to
contain the expected message.
Increase the timeout to 120s as some pods might take a little bit longer
to finish.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
in case we re-apply the selinux workaround or if user had already
existing similar rule the relabel_selinux was failing. Let's allow it to
modify the existing rules as well to avoid such issues.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
The open_named_tuntap function is designed as a public function to
open a tuntap device with the specified name. However, in order to
reference existing methods in dbs_utils, we still need to keep the
reference "path = "../../../dragonball/src/dbs_utils" in dependencies
and cannot hide it.
Fixes: #8865
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Add network helpers and impl ToQemuParams trait to build
netdev params which are put into cmdline for Qemu VM running.
Fixes: #8865
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
We need ensure the add_network_device happens in netns and
move qemu process into netns which keeps the qemu process
running in this net namespace.
Fixes: #8865
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Add network device handler in start_vm, which is sepcially
for Qemu VM running with added net params to command line.
Fixes: #8865
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
We need add a new netns field in struct QemuInner, and
initialize it with argument passed down in prepare_vm().
Fixes: #8865
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The enter_netns function is designed as a public method to help
VMMs running as a independent process enter a network namespace,
reducing duplicate code.
Fixes: #8865
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
It just move the related code to a public file(utils.rs) and make
it a common method for both vsock and network, or some others.
Fixes: #8865
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Signed-off-by: Pavel Mores <pmores@redhat.com>
In order to better support non-builtin vmm usage of NetnsGuard and
reduce code duplication, we need to move it to a common path that
can be referenced by both hypervisor and resource manager.
In this patch, it just do moving code from network/utils/netns.rs
to kata-sys-utils/src/netns.rs
Fixes: #8865
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The PID needs to be initialized before calling isClhRunning.
waitVMM() uses isClhRunning and is called by launchClh() just
before returning from function.
Fixes: #9230
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Use containerd's default environment for container images that don't
specify the Env field.
Also, re-enable policy env variable verification, now that these
uncommon images are supported too.
Fixes: #9239
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
We're trying to keep just the bare minimum info, as we really would like
to not have the list of commits, and mainly the list of new
contributors, trucated from the release notes.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need to ensure the release type is passed down to workflows,
otherwise we'll fail to get the correct release version for tagging the
daemonset images.
Fixes: #9064 - part III
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Recently confidential-containers/kbs repository was renamed to
confidential-containers/trustee. Github will automatically resolve the
old URL but we better adjust it in code.
The trustee repository will be cloned to $COCO_TRUSTEE_DIR. Adjusted
file paths and pushd/popd's to use $COCO_KBS_DIR
($COCO_TRUSTEE_DIR/kbs).
On versions.yaml changed from `coco-kbs` to `coco-trustee` as in the
future we might need other trustee components, so keeping it generic.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Added the kbs_set_resources_policy() function to set the KBS policy. Also the
kbs_set_allow_all_resources() and kbs_set_deny_all_resources to set the
"allow all" and "deny all" policy, respectively.
Fixes#9056
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Added utility functions to manage resources in KBS:
- kbs_set_resource(), where the resource data is passed via argument
- kbs_set_resource_from_file(), where the resource data is found in a
file
Fixes#9056
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Added kbs_install_cli function to build and install the kbs-client
executable if not present into the system.
Removed the stub from gha-run.sh; now the install kbs-client in the
.github/workflows/run-kata-deploy-tests-on-aks.yaml will effectively
install the executable.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Scripts sourcing ci/lib.sh need to set $GOPATH otherwise it will
fail. This ensure that GOPATH is set to ${HOME}/go unless it is
already exported.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Added functions to return the service host, port or full-qualified
HTTP address, respectively, kbs_k8s_svc_host(), kbs_k8s_svc_port(),
and kbs_k8s_svc_http_addr().
Fixes#9056
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This change updates the module import to use 'util' instead of the deprecated 'io/util'
Fixes: #9166
Signed-off-by: Chungeun Choi <ce.choi@okestro.com>
copyBuffer returns and the streams will be closed when error occurs.
If the error contains "blocked by policy" it means the log output is
disabled by policy with "ReadStreamRequest" and "WriteStreamRequest" set
to false. But at this moment, we want the real stream still working (not
be seen) because we might want to enable logging for debugging purpose,
so we repeat copybuffer in this case to avoid streams being closed.
Fixes: #8797
Signed-off-by: Linda Yu <linda.yu@intel.com>
logging/debugging information might probably be disabled in production
due to security consideration, but we'd better provide an approach for
customer to get logging information during runtime, this PR implement
setpolicy function in kata-runtime tools, although it can set whole policy
other than logging.
setpolicy would evokes remote attestation, which means before setting
policy during runtime, user has to reconfigure new policy hash in KBS/AS.
usage: kata-runtime policy set policy.rego --sandbox-id XXXXXXXX
Fixes: #8797
Signed-off-by: Linda Yu <linda.yu@intel.com>
Disable env variable verification to unblock CI, until container
images that don't specify the Env variables will be handled correctly
(see #9239).
Also, mark the image config Env field as optional, thus allowing
policy generation for these container images.
Fixes: #9240
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Fix the `grep(1)` warning caused by the unnecessary escaping of the
hash/sharp symbol.
Fixes: #9235.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Move the "build from source" and "manual installation" details to the
developer guide. This makes the installation landing page clearer for
users.
Fixes: #9234.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The k3s distribution of k8s uses an embedded version of containerd and
configures it to log to a file, not the journal. Hence, although we
collect the journal as a test artifact, we also need to collect the
actual log files for containerd.
Also collect the k3s containerd config files to help with debugging.
Fixes: #9104.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The jobs running on garm will collect journal information. The data gathered
is based on the time the tests started running. The $start_time is
exported on run_tests() and used in collect_artifacts(). It happens that
run_tests() and collect_artifacts() are called on different steps of the
workflow and the environment variables aren't preserved between them,
i.e, $start_time exported on the first step is not available on the
subsequents.
To solve that issue, let's save $start_time in the file pointed out by
$GITHUB_ENV that Github actions uses to export variables. In case $GITHUB_ENV is
empty then probably it is running locally outside of Github, so it won't
save the start time value.
Fixes#9217
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This PR adds the general tests documentation in main README of the
kata containers repository.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds general README documentation for the tests section
in the kata containers repository.
Fixes#9209
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Add details to the README for `kata-manager` showing how to list
available hypervisor configs (packaged and local), and switch between
the configurations. Also, update the hypervisors page to show a lot more
detail about the hypervisor configurations, including the "short name"
used by `kata-manager` for switching hypervisor config.
> **Note:**
>
> These changes only apply to the current default golang runtime.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add new options to allow the configured hypervisor to be changed:
- `-L`: List available _packaged_ hypervisor config short names.
- `-e`: List available _local_ hypervisor config names.
- `-H <hypervisor>`: Install Kata then switch to the specified hypervisor.
- `-S <hypervisor>`: Switch to the specified hypervisor (by config short name [Errors if Kata not installed]).
For example, to install Kata and configure it to use Cloud Hypervisor
with the golang Kata runtime:
```bash
$ kata-manager.sh -H clh
```
To switch back to the default hypervisor:
```bash
$ kata-manager.sh -S default
```
To show details of the available packaged configs:
```bash
$ kata-manager.sh -L
```
To show details of the local configs:
```bash
$ kata-manager.sh -e
```
> **Notes:**
>
> - This change **only** applies to the current default (golang) Kata runtime.
>
> - Although this is mainly for users wishing to switch hypervisor (by
> changing the Kata config file to another of the packaged config files
> provided for specific hypervisors), strictly it allows users to change
> to _any_ config file. For example, if the user has a config file called
> `/etc/kata-containers/configuration-my-custom-config.toml`, they could
> switch to this by running:
>
> ```bash
> $ kata-manager.sh -S my-custom-config
> ```
>
> - The "config short names" are the hypervisor specific part of the configuration file name.
> For example, the config short name for file `configuration-qemu.toml` is
> `qemu` and the config short name for `configuration-clh.toml` is `clh`.
Fixes: #8305.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The `configure_kata()` function modifies the configuration file to
enable debug. But it was doing this by calling `sed -i` which, by
default, creates a new _file_ from the `configuration.toml` symbolic
link. This defeated the point of the symbolic link which is supposed to
resolve to the local copy of the pristine config file, so we now use
the GNU sed(1) specific `---follow-symlinks` option to retain the
sym-link.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add an info message just before the archive file is checked. This keeps
the user informed about what is happening as it can take a few seconds
to perform the checks on slower systems.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
a) There is some unknown syscalls triggered in new github virt machine
that would break the make test process with SIGSYS after applying
SeccompFilter. In order to fix this, we change the allowlist in this
unit test for seccompfileter into a blocklist to avoid meeting the unknown syscalls.
b) lazy static METRICS is not fully initialize in the unit test and may lead to
unstable result for this UT.
fixes: #9207
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
the mmap region start guest addr hard-code a value and later there
would be check whether the mentioned addr is larger than or equal
to mem_end (default to host_phy_mem >> 1) in order to satisfy the
requirement for DaxMemory. Since github virt machine phy_mem is larger
than previous CI machine we use, the hard-code value could no longer be
worked. To fix this, we change the address to mem_end in unit test to
avoid the influence of host machine change.
fixes: #9207
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
- enable CONFIG_MTRR,CONFIG_X86_PAT on x86_64 for nvidia gpu
- optimize -f of build-kernel.sh, clean old kernel path and config before setup
- add kernel 5.16.x
Fixes: #9143
Signed-off-by: Jimmy-Xu <xjimmyshcn@gmail.com>
PR #8760 tentatively tried to have the shim to run in its own mount
namespace for the sake of improving isolation between the sandbox and
the host. Thus crio storage drivers shouldn't create a PRIVATE
bind mount on their home directory. Otherwise, the container's rootfs
mount wouldn't be propagated to kata runtime's mount namespace, and
kata runtime couldn't access the container's rootfs files.
So, when kata cooperated with crio, crio should set
skip_mount_home=true for its storage overlay.
Fixes: #9028
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
_print_cluster_name() create a string based information like the
pull request number and commit SHA. However, when you are developing the
scripts you might want to use an arbitrary name, so it was introduced
the $AKS_NAME variable that once exported it will overwrite the
generated name.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Until this point the deployed KBS service is only reachable from within
the cluster. This introduces a generic mechanism to apply an Ingress
configuration to expose the service externally.
The first implemened ingress is for AKS. In case the HTTP application
routing isn't enabled in the cluster (this is required for ingress), an
add-on is applied.
It was added the get_cluster_specific_dns_zone() and
enable_cluster_http_application_routing() helper functions
to gha-run-k8s-common.sh.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Introduce the tests/integration/kubernetes/confidential_kbs.sh library
that contains functions to manage the KBS on CI. Initially implemented
the kbs_k8s_deploy() and kbs_k8s_delete() functions to, respectively,
deploy and delete KBS on Kubernetes. Also hooked those functions in the
tests/integration/kubernetes/gha-run.sh script to follow the convention
of running commands from Github Workflows:
$ .tests/integration/kubernetes/gha-run.sh deploy-coco-kbs
$ .tests/integration/kubernetes/gha-run.sh delete-coco-kbs
Fixes#9058
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Kustomize has been used on some of our internal components (e.g.
kata-deploy) to manage k8s deployments. On CI it has been used
the `sed` tool to edit kustomization.yaml files, but `kustomize` is
more suitable for that purpose. So in order to use that tool on CI
scripts in the future, this commit introduces the `install_kustomize()`
function that is going to download and install the binary in
/usr/local/bin in case it's found on $PATH.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This PR adds the collect artifacts function in gha-run script for
the kubernetes tests.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR helps to store the artifacts for different logs for k8s tests
on garm.
Fixes#9103
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This backports the:
9060e930caf2d20f413df07778d3ab497493161c
ci.ocp: Add debug output on HTTP service failure
these logs are vital to analyze a setup failure.
a10a1e2c9cbc21afc1e80f22b0fb8634d27cbd8d
ci.ocp: Improve the service-up detection
waiting for the first response is not sufficient as OCP returns html
page without error even when the route is not yet established describing
the issue (why it doesn't reply with 500?). Waiting for the correct
output should do better.
commits from the kata-containers/tests repo.
Fixes: #8653
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
This fixes a panic on tracing on container exit.
The root cause is that global var needs to be set by "=" instead of
":=".
Fixes: #9102
Signed-off-by: Liu Bo <liub.liubo@gmail.com>
Currently, the checking for kata-deploy is running assume that the
daemonset scheduled at least one pod, however it might not had and the
kubectl wait command fails due to "error: no matching resources found".
On CI I've observed that fail intermittently. I suspect the service
account kata-deploy-sa take a while to show up then no kata-deploy is
scheduled in meanwhile.
Changed the checker logic to use waitForProcess() to keep testing if it is
already running, or hit the timeout (still 10m).
Fixes#9183
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The step to deploy KBS on run-k8s-tests-on-aks workflow should be
increased so that there is enough time for checking the service is
healthy and exposed. Likewise the step that builds the kbs-client
which requires enough time to build the executable.
Fixes#9058
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Kubernetes v1.29 introduced a new `PodReadyToStartContainers` condition
that gets inserted at index 0 in the conditions array. This means that
the expected `PodCompleted` reason can now be either at index 0 with
kubernetes v1.28 and older or at index 1 starting with kubernetes v1.29.
This is fragile at best since the `kubectl wait` doesn't allow to combine
multiple checks. Also, checking the reason is dubious as it doesn't really
tell if the pods have actually completed or not.
Check the pod phase to be `Succeeded` instead, this guarantees that :
> All containers in the Pod have terminated in success, and will not
> be restarted.
Fixes#9178
Signed-off-by: Greg Kurz <groug@kaod.org>
This was introduced by a45988766c, but
didn't follow the correct format for the env declaration.
Fixes: #9064 - part II
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We want all payloads to be built and published, regardless if there's a
new PR merged.
This will help people to easily trace / debug issues.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We were never passing the arguments to add the PAUSE_IMAGE to the
rootfs, leading to it never being present in the confidential image /
initrd.
Fixes: #9032 -- part II
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR implements general fixes to the gha-run script for the
cri-containerd tests.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR skips the cri-containerd in gha-run script for cloud hypervisor
runtime-rs.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the Cloud Hypervisor driver, integrated with the runtime-rs,
as part of the cri-containerd tests.
Fixes#9181
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR updates the code pr advice document to make the proper
references now that we have move the test repository to the kata containers
repository.
Fixes#9171
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This is not needed anymore as we can run the tests from any branch, and
we can patch this locally before doing a test.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
GitHub actions is fun and always willing to play tricks with us. This
nice little kid decided that `echo "FOO=\"bar zaz\"" >> $GITHUB_ENV` is
not valid, and it simply breaks things in a way that is a pain to debug.
But hey, we take this path, and after doing so I realised that the
correct way to export that is `echo "FOO=bar zaz" >> $GITHUB_ENV`.
I know, this looks incorrect, but this fellow never stops surprising us.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
I'm getting here the most relevant parts of what we had as part of the
release-notes.sh script. As the script will not be used anymore, it's
been removed.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
For some reason we need to force its installation in the GOPATH,
otherwise yq is not found.
Ideally we should switch to a packaged version of yq, but that's a topic
for another series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need to adjust the tags as when this workflow ends up being called
from the release side, we'll receive "refs/tags/main" as the
GITHUB_REF, and in that case we must use the release version.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is utterly counter intuitive, but if we change a file during the
GitHub Action, the checkout done for the next workflow won't have that
file updated, but rather the branch on its original state when the
workflow was created.
This makes us safe to always "calculate" the next release version from
the VERSION file at the time the workflow was triggered.
This requires us to have the release type exported for the whole
workflow.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Without those we'll end up running steps in parallel that should
actually wait for a previous step to be completed.
While here, let's also correct some of the "needs" that were waiting fro
the wrong workflow to be finished.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We missed doing this as part of
50011e89a0, but we also need to check for:
* RELEASE_VERSION
* GH_TOKEN
* ARCHITECTURE
* KATA_STATIC_TARBALL
While here, let's fix a ARCHITECURE -> ARCHITECTURE typo.
Fixes: #9064 -- part II
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Changed the "run k8s tests on AKS" workflows to get the CoCo KBS
installed so that we can run attestation tests.
The plan is to run attestation tests only on a subset of non-TEE jobs
initially, so this commit restricts to install KBS only on kata-qemu
configuration. Actually at this point it is added only stubs commands
to tests/integration/kubernetes/gha-run.sh that should be implemented
in a future commit.
Fixes#9058
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Add all agent configuration options to README so that users can more easily understand
what these options do and how to configure them at runtime.
Fixes: #9109
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This commit should be merged as it's now, then we trigger a test
release, fix whatever has to be fixed, and drop it as soon as we know
our workflows are working as expected.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Those are not needed anymore as we're automating our release process
around GitHub actions.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're not doing backports anymore, as we're getting rid of the stable
branches in favour of having a better release cadence from the main
branch.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we're not maintaining a stable branch anymore, let's get rid of the
kata-deploy stable pieces.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Those will allow us to cut a release just by a single click, instead of
the current process we have.
Fixes: #9064 -- part I
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The release workflow is now updated to be a `workflow_call`, and it
includes the steps that had to be manually done in the past, such as
updating the needed files and creating the release itself.
While on this, the kata-deploy multiarch manifest tags have been updated
to match the new release scheme.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function returns the version of the next release (the one about to
be cut), and it'll be used as part of our new workflow that will take
care of the release.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add a function that will be responsible for bumping the project's
version in the VERSION file, and push it to the branch as part of the
release process that will be introduced.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is a helper function that will be used to create a new release as
part of our release process workflow (which will still be modified).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the name of the function says, it's responsible for uploading the
libseccomp source tarballs as par of our release process.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As hinted by the name of the function, this is used to generate and
upload the vendored code we have as its own tarball.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the name says, this function will be used to upload the versions.yaml
file during a given release process of the project.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function, as it names says, will be used to upload the
kata-static.tar.xz tarballs generated during the release process.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function, as it names says, will be used to publish multiarch
manifests for the Kata Containers CI and Kata Containers releases.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
For now this script does nothing, but we're introducing it in order to
redduce the diffs for the next commits in this series.
My intention is to have as much as possible related to the release as
part of this helper script, and it'll be populated function by function
while replacing content that's "hard coded" (and duplicated) on
different GitHub actions.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This refactor the teardown() of tests/integration/nydus/nydus_tests.sh:
* Moved boilerplate code that kill process to a loop;
* Doesn't leave teardown() if a process failed to get killed, so that
other clean up routines are ran;
* Check if the pid exist then attempt to kill the process, so avoid this
misleading message:
```
Usage:
kill [options] <pid> [...]
Options:
<pid> [...] send signal to every <pid> listed
-<signal>, -s, --signal <signal>
specify the <signal> to be sent
-q, --queue <value> integer value to be sent with the signal
-l, --list=[<signal>] list all signal names, or convert one to a name
-L, --table list all signal names in a nice table
-h, --help display this help and exit
-V, --version output version information and exit
For more details see kill(1).
```
Fixes#8948
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
It's recommended to avoid images from docker.io to avoid errors related
with hitting the pull limits that happens mostly on bare-metal machines.
So this replaced the docker.io's busybox with
quay.io/prometheus/busybox.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The "run ps command" test has failed once in a while because it doesn't
wait the sh command to start within the container, consequently `ps`
won't report the amount of lines expected.
Fixes#8975
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Migrated runk tests from pure shell script to bats to be consistent with
other test suites.
The install_dependencies() will install the bats tool locally.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Relax the timeout for calling CLH's CreateVM + BootVM APIs. When
hitting the older 1s timeout, killing a half-booted Guest and
retrying the same boot sequence could have been wasteful and resulting
in unstable CI testing on slower Hosts.
Fixes: #9152
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Add a link in the release notes to the Kata Container survey, to
advertise it, and hopefully encourage users to take the survey.
Fixes: #9074.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add a "twistie" / arrow (`▶`) that the user can click on to see the full
list of commits _if they want to_.
This way, the release notes become easier to read and we can display
information below the shortlog which would (probably) normally not be
seen due to the huge long list of commits.
Fixes: #9075.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
For consistency with the go runtime.
As the shim itself is not using the network (all its communication with
other processes is done with local unix sockets), there is no reason to
keep gathering and reporting shim-specific network metrics.
Actual network usage of the kata containers can be found from the existing
agent network metrics (kata_guest_netdev_stat).
Signed-off-by: Julien Ropé <jrope@redhat.com>
As part of the shim network metrics, the shim is reporting network interfaces
from the host with no namespace isolation - this gives insight in interfaces
not tied to the kata containers, and causes an increase in resource usage for
kata metrics.
As the shim itself is not using the network (all its communication with
other processes is done with local unix sockets), there is no reason to
keep gathering and reporting shim-specific network metrics.
Actual network usage of the kata containers can be found from the existing
hypervisor network metrics (kata_hypervisor_netdev) and from the agent
network metrics (kata_guest_netdev_stat).
Fixes: #5738
Signed-off-by: Julien Ropé <jrope@redhat.com>
CI failed to deploy nydus snapshotter because it was not cleaned up last time.
So we can try to cleanup nydus snapshotter before deploying it.
Fixes: #9121
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need initailize the pci_hotplug_enabled with true before we do GPU
passthrough with runtime-rs/dragonball. Otherwise it fails with error
`InvalidOperation`.
Fixes: #9129
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
teardown() gets executed after each test case, so there is no need to
clean-up before teardown.
Fixes: #9072
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Add set_namespace_to_policy_settings() for changing the pod namespace
in genpolicy settings.
Fixes: #9072
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
When cgroup v2 is in use, a container should only see its part of the
unified hierarchy in `/sys/fs/cgroup`, not the full hierarchy created
at the OS level. Similarly, `/proc/self/cgroup` inside the container
should display `0::/`, rather than a full path such as :
0::/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde291f58_8f20_4d44_aa89_c9e538613d85.slice/crio-9e1823d09627f3c2d42f30d76f0d2933abdbc033a630aab732339c90334fbc5f.scope
What is needed here is isolation from the OS. Do that by running the
container in its own cgroup namespace. This matches what runc and
other non VM based runtimes do.
Fixes#9124
Signed-off-by: Greg Kurz <groug@kaod.org>
Remove specific kernel/initrd/image leftovers in Makefile of
local-build, which is the part of #9026.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This PR adds the error script to display the error message with
much more information to help debugging.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds an enhanced die function in order to dump more information
in a yaml format that will help with the debugging.
Fixes#9105
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
In case an error is encountered while removing a network endpoint during
network cleanup, we cuurently return immediately with the error.
With this change, in case of error we simply log the error and proceed
towards removing the next endpoint. With this, we can cleanup the
network changes made by the shim as much as possible.
This is especially important when multiple interfaces are passed to the
network namespace using a network plugin like multus.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Move the defer for cleaning up network before the call to add network.
This way if any change made by add network is reverted by in case of
failure. This is particulary important for physical network interfaces
as with this step we make sure that driver for the physical interface is
reverted back to the original host driver. Without this the physical
network iterface will remain bound to vfio.
Fixes: #8646
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This has been introduced by 53bc4a432b,
where the condition was changed.
The correct condition is:
* If the list of supported tees does not contain the kata hypervisor
and the list of supported non tees does not contain the kata
hypervisor.
The error is that we were checking whether kata-hypervisor would contain
the list of supported tees, and that would almost always be false
(unless in the case where the list had an one and only one element).
Fixes: #9055 -- part II
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is required to allow creation of SNP coco on host kernel
(e.g. https://github.com/AMDESE/linux ,branch:snp-host-latest)
supporting guest private memory for SNP using gmem.
Note: This qemu does not work if the host kernel does not support
gmem/UPM.
Fixes: #9092
Signed-off-by: Niteesh Dubey <niteesh@us.ibm.com>
A test `vcpu allocation k8s test` exhibits different behavior on s390x
For more details, please refer to issue #9093.
This commit is to make the test skipped until the issue is resolved on
the platform.
Fixes: #9093
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Remove some unnecessary whitespace from a couple of `kata-deploy` files.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
whitespace
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Now we're using a "confidential" image that has support for all of
those.
Fixes: #9010 -- part II
#8982 -- part II
#8978 -- part II
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR pulls the docker image needed for the test as part of the dependencies
in order to avoid failures of timeouts mainly because the image was not
properly download it and it is unable to find it.
Fixes#9089
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the ability to run k8s confidential tests in a
non-TEE environment.
Fixes#9055
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This will ensure no leftovers are in the node, which has been cause the
TDX CI to fail every now and then.
Fixes: #9081
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In the same way that doesn't make sense to ship the pause-image, it also
doesn't make sense to ship the coco-guest-components itself as part an
release artefact.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It doesn't make sense to ship the pause-image itself as an release
artefact.
The reason we build it and cache it is in order to use it inside the
rootfs, and that's it, there's not need to ship it as part of the
release, at all.
Fixes: #9032 -- part II
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR updates the CI link into the README as currently we are
using GHA workflows and they are now part of the kata containers
repository.
Fixes#9078
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Delete the debugger pod created during the test, rather than already
existing debugger pods.
Also, send the output of "kubectl delete" to stderr, just in case it's
useful for debugging.
Fixes: #9069
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Last but not least, all placeholders for argument replacement
should be configured to generate a configuration file when `QEMUCMD`
is defined. This enriches those variables.
Additionally, this involves creating a symbolic link to `configuration-qemu.toml`
if QEMU is defined as the default hypervisor.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Avoid error when building for amd64 using:
USE_CACHE=no AGENT_POLICY=yes DEBUG=1 \
tools/packaging/kata-deploy/local-build/kata-deploy-binaries.sh \
--build=rootfs-image
Fixes: #9067
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
There are some variables newly introduced to runtime-rs, such as:
- runtime.name
- runtime.hypervisor_name
- runtime.agent_name
- vm_rootfs_driver
Additionally some of the placeholders for argument replacement are
made hypervisor-specific based on the changes made for dragonball.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
We can use daemonset to deploy nydus snapshotter, which will decrease
one manual step both for Kata Containers and Confidential Containers CI.
Fixes: #8584
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This PR replaces the add_kernel_initrd_annotations_to_yaml function
more generic so later can be used for other components.
Fixes#9054
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
1. add_requests_to_policy_settings allows one or more ttrpc requests
from the Host to the Guest. Example:
add_requests_to_policy_settings "${policy_settings_dir}" \
"ReadStreamRequest" "WriteStreamRequest"
2. add_copy_from_host_to_policy_settings allows executing on the Guest
the commands initiated behind the scenes by "kubectl cp" from the
Host to the Guest. Example:
add_copy_from_host_to_policy_settings "${policy_settings_dir}"
3. add_copy_from_guest_to_policy_settings allows executing on the Guest
the commands initiated behind the scenes by "kubectl cp" from the
Guest to the Host. Example:
add_copy_from_guest_to_policy_settings "${policy_settings_dir}" \
"/tmp/file.txt"
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
For Confidential containers stack, the pause image is managed by host side,
then it may configure a malicious pause image, we need package
a pause image inside the rootfs and don't the pause image from host.
But the installation of skopeo is not included in 20.04 release, so we
can not directly install skopeo in rootfs and pull pause image.
So I plan to let the task as a static build stuff, which would not be influenced
by the system version in rootfs. And the pause image will be part of the Kata Containers rootfs
that's used by the Confidential Containers usecase. This commit enables the component to be built
both locally and in our CI environment with the command: make pause-image-tarball.
Fixes: #9032
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: Jakob Naucke <jakob.naucke@ibm.com>
For example, Kata CI's k8s-copy-file.bats transfers files between the
Host and the Guest using "kubectl exec", and that results in
CloseStdinRequest being called from the Host.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
1. Remove PullImageRequest because that is not used in the main
branch. It was used in the CCv0 branch.
2. Add default false values for the remaining Kata Agent ttrpc
requests.
These changes don't change the functionality of the auto generated
Policy, but they help with easier understanding the Policy text and
the logging from the Rego rules.
Fixes: #9049
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
1. Rename install_kata_common to install_kata_core.
2. Add TODO for better way to install the Kata tools.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Since cri-o doesn't seem to use address for event publishing as mentioned
in the previous commit it will not send it. However, the exact way of
not sending it is unfortunately different from what is assumed by
runtime-rs. Due to an implementation detail of cri-o which uses containerd
libraries for some low-level tasks, TTRPC_ADDRESS will not be missing from
environment as assumed, instead it will be present with an empty value.
This commit contains a small adjustment to account for that and use
LogForwarder even if TTRPC_ADDRESS is present, but with an empty value.
Fixes#8985
Signed-off-by: Pavel Mores <pmores@redhat.com>
This PR removes the jenkins reference which is not longer being used
in the kernel documentation.
Fixes#9046
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Bump nydus snapshotter version to v0.13.7.
The new release name of nydus snapshotter is `nydus-snapshotter-v0.13.7-linux-amd64.tar.gz`,
which differs from the version used by kata (`nydus-snapshotter-v0.12.0-x86_64.tgz`).
Therefore, we need to update the script to obtain the correct nydus snapshotter name.
Fixes: #9044
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Automatically generate the test policy for k8s-attach-handlers.bats,
if AUTO_GENERATE_POLICY is enabled.
Steps:
- Create a temporary directory for the current test and copy the
common genpolicy settings into this new directory.
- Change genpolicy settings in the temp directory to allow the
"kubectl exec" command that this test needs. (For CoCo, exec is
blocked by the default policy settings)
- Auto-generate the policy for the test YAML file.
- Test as usual, using the YAML file.
- Clean-up the temporary settings described above.
Fixes: #8921
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Enable AUTO_GENERATE_POLICY for one of the Kata CI K8s test platforms.
Additional platforms will be enabled after testing them.
When AUTO_GENERATE_POLICY is enabled, create genpolicy settings that
are common for all tests. Some of the tests will make temporary copies
of these common settings and customize them as needed.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
1. Avoid repeating "kata-containers-k8s-tests".
2. Allow users to specify a different test namespace.
3. Introduce the TEST_CLUSTER_NAMESPACE variable, that will also be
useful when auto-generating the Agent Policy for these tests.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This is needed to fix the bug which is not allowing to create SEV container
on SNP enabled host anymore. This is a regression that was introduced as
part of the following commit:
de39fb7d38Fixes: #9036
Signed-off-by: Niteesh Dubey <niteesh@us.ibm.com>
It makes sense to reuse a configuration template for runtime-golang
as a base. This is simply to copy it into the config directory.
Fixes: #8441
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This PR skips the k8s tests that are not working with cloud hypervisor
runtime-rs with its proper issue.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
It appears that under the shim v2 protocol, a shim has no use of its own
for the -address value, it just passes it back to container runtime's
(mostly containerd or cri-o) event-publishing binary. Since the -address
value only flows through the shim, being passed to the shim by a container
runtime and then essentially passed back by shim to the container runtime,
it seems inappropriate for a shim to validate the value that is fully
owned and only used by the container runtime.
This commit removes such validation from runtime-rs. Doing so, it solves
(part of) an interoperability problem between runtime-rs and cri-o. cri-o
seems to intentionally choose not to implement the event-publishing part
of the shim v2 protocol and thus it has no value it could pass to
runtime-rs for -address. As a result, it sends an empty string which has
been failing the excessive validation performed by runtime-rs so far.
Signed-off-by: Pavel Mores <pmores@redhat.com>
This is required as the tdx-attest-rs crate, which is used as part of
the guest components, has a runtime dependency on libattest-tdx.
Fixes: #9021 -- part II
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The emergent Kata CI tests for Policy use confidential_guest = false
in genpolicy-settings.json. That value is inconsistent with the
following mount settings:
"emptyDir": {
"mount_type": "local",
"mount_source": "^$(cpath)/$(sandbox-id)/local/",
"mount_point": "^$(cpath)/$(sandbox-id)/local/",
"driver": "local",
"source": "local",
"fstype": "local",
"options": [
"mode=0777"
]
},
We need to keep those settings for confidential_guest = true, and
change confidential_guest = false to use:
"emptyDir": {
"mount_type": "local",
"mount_source": "^$(cpath)/$(sandbox-id)/rootfs/local/",
"mount_point": "^$(cpath)/$(sandbox-id)/local/",
"driver": "local",
"source": "local",
"fstype": "local",
"options": [
"mode=0777"
]
},
The value of the mount_source field is different.
This change unblocks testing using Kata CI's pod-empty-dir.yaml:
genpolicy -u -y pod-empty-dir.yaml
kubectl apply -f pod-empty-dir.yaml
k get pod sharevol-kata
NAME READY STATUS RESTARTS AGE
sharevol-kata 1/1 Running 0 53s
Fixes: #8887
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Due to the restrictions on instance provisioning for self-hosted runners, performing
static checks (36 jobs at the time of writing) on them each time a PR is updated could
significantly burden them, consequently slowing down the entire CI system. To address
this, the decision is to trigger these checks only when an 'ok-to-test' label is added.
Meanwhile, the checks for x86_64, which are supported by GitHub-hosted runners, will
remain unchanged.
Fixes: #8998
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The loop that generate test cases for virtio-mem enabled/disabled
doesn't return the integers '1' and '0' as expected. Instead it returns
the strings '{1,' and '0}'.
Fixes#9024
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Let's install the coco-guest-components into the confidential rootfs
image and initrd.
Fixes: #9021
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will make our lives easier to figure out whether the components are
being installed or not.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need to pass this to the container where the rootfs is built, so it
can actually be unpacked inside the rootfs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When moving the webhook we skipped the common.bash as (close-enough)
version is already in `/tests` but we forgot to update the source path,
fixing it here.
Fixes: #8653
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
As DESTDIR was not being passed, we've been installing the final
binaries in a container path that was not exposed to the host, leading
to creating an empty tarball with the guest components.
Now, theoretically, guest-components should respect a PREFIX passed, but
that's not the case and we're manually adding "/usr/local/bin" to the
passed DESTDIR.
Here's the result of the tarball:
```bash
⋊> kata-containers ≡ tar tf build/kata-static-coco-guest-components.tar.xz
./
./usr/
./usr/local/
./usr/local/bin/
./usr/local/bin/confidential-data-hub
./usr/local/bin/attestation-agent
./usr/local/bin/api-server-rest
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR is a split of #8585.
make the changes on the Github workflows, and the skeleton to deploy_snapshotter()
and cleanup_snapshotter() in tests/integration/kubernetes/gha-run.sh in this commit.
After initially merging this patch to trigger CI jobs for CoCo, which will begin executing
the dummy functions deploy_snapshotter() and cleanup_snapshotter(), the implementation details for these functions
remain in #8585. Our subsequent step involves transferring this logic to the PR #8484, enabling the PR to undergo CI testing prior to its merge.
Fixes: #8997
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Implementing Persist API for cloud-hypervisor was done partially with
initial support for cloud-hypervisor. Store and retrieve additional
fields to/from the hypervisor state.
Fixes: #6202
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
The filtering of testing cases installs/uses yq and expects GOPATH to be present. Hence, add it to the workflow.
Fixes: #9018
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
The original handling method does not reach user expectations.
When the ClientSocketAddress method stats the corresponding
path of runtime-rs and has not found it yet, we should return
an error message here that includes the reason for the failure
(which should be an error display indicating that both runtime-go
and runtime-rs were not found). Instead of simply displaying the
corresponding path of runtime-rs as the final error message to
users.
It is also necessary to return the error promptly to the caller
for further error handling.
Fixes: #8999
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Now that we have a confidential image / initrd being built, instead of a
specific one for each TEE, let's use it everywhere possible.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's use a single rootfs image / initrd for confidential workloads,
instead of having those split for different TEEs.
We can easily do this now as the soon-to-be-added guest-components can
be built in a generic way.
Fixes: #8982
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Now that we're using the kernel-confidential, let the rootfs depending
on it, instead of depending on the TEE specific ones.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need to do this in order to ensure that the measure boot will be
taking the latest kernel bits, as needed.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is already done for the TDX kernel, and should have been done also
for the confidential one.
This action requires us to bump the kernel version as the resulting
kernel will be different from the cached one.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we're building a single confidential kernel, we should rely on it
rather than keep using the specific ones for TDX / SEV / SNP.
However, for debugability-sake, let's do this change TEE by TEE.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we're building a single confidential kernel, we should rely on it
rather than keep using the specific ones for TDX / SEV / SNP.
However, for debugability-sake, let's do this change TEE by TEE.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
delete_cluster() has tried to delete the az resources group regardless
if it exists. In some cases the result of that operation is ignored,
i.e., fail to resource group not found, but the log messages get a
little dirty. Let's delete the RG only if it exists then.
Fixes#8989
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
As we're building a single confidential kernel, we should rely on it
rather than keep using the specific ones for TDX / SEV / SNP.
However, for debugability-sake, let's do this change TEE by TEE.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
With this we can properly generate and the the `-confidential` kernel,
which supports SEV / SNP / TDX as part of our configuration files.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
I made this a required argument during the series and ended up
forgetting to add that while calling the function.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This issues was introduced due to a typo not caught during reviews on
e5bca90274.
Fixes: #6415 -- part II
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Until now, runtime-rs couldn't be compiled on s390x.
We need to lift those restrictions in Makefile first.
Fixes: #8446
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This PR adds the Cloud Hypervisor driver, integrated with the runtime-rs,
as part of the kubernetes tests different with devmapper.
Fixes#8995
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This addresses an internal AKS issue that intermittently prevents
clusters from getting created. The fix has been rolled out to eastus but
not yet eastus2, so we unblock the CI by switching. No downsides in
general.
This supersedes #8990.
Fixes: #8989
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
It fails to compile virt_container because Dragonball is only
used in the implementation of the trait method Persist::restore().
As the hypervisor is not compiled on s390x and QEMU implements
the trait method, this commit is to let the method use QEMUi's.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Dragonball and cloud-hypervisor are not supported on s390x.
We need to exclude the plugins for these hypervisors from compilation.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Till now we didn't have a logic to consume the kernel modules cached
tarball. Let's make sure those are consumed as it'll save us a
reasonable amount of build time.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will save us a lot of time, as right now the CI is rebuilding the
kernel for absolutely no reason.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's start doing this for the confidential kernels (and also for SEV,
till it gets removed).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This allows us to add a map, in the format of:
`"tarball1_name:tarball1_path tarball2_name:tarball2_path ..."`
With this we have a base to start doing a better job when caching extra
artefacts, like kernel modules.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Right now this is just being added but not used yet. The idea is to use
this to both cache and later on untar the kernel modules needed for some
of the kernel targets we have (specifically looking at the confidential
one).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This patch can reduce load on systemd process, and
increase the k8s deployment density when using go runtime.
Fixes: #8758
Signed-off-by: Zhigang Wang <wangzhigang17@huawei.com>
Signed-off-by: Liu Wenyuan <liuwenyuan9@huawei.com>
As already done for docker, let's also add support for installing
nerdctl + kata containers.
For now, at least for now, we are explicitly not allowing the
combination of installing both docker and nerdctl in the same
installation in order to reduce the script complexity.
Also, nerdctl installation, for now, is limited to x86_64 and aarch64 as
those are the only architectures that nerdctl releases a "full" package
for.
Fixes: #8358
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR adds workflow for running kubernetes test suite on ppc64le.
It uses scripts to create and delete the cluster using kubeadm as none of the current cluster creation tools are supported on Power.
Fixes: #7950
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
Kata CI's pod-sandbox-vcpus-allocation.yaml ends with "---", so the
empty YAML document following that line should be ignored.
To test this fix:
genpolicy -u -y pod-sandbox-vcpus-allocation.yaml
Fixes: #8895
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Allow users to specify in genpolicy-settings.json a default cluster
namespace other than "default". For example, Kata CI uses as default
namespace: "kata-containers-k8s-tests".
Fixes: #8976
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
There is a race condition in agent HVSOCK_STREAMS hashmap, where a
stream may be taken before it is inserted into the hashmap. This patch
add simple retry logic to the stream consumer to alleviate this issue.
Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
Linux forbids opening an existing socket through /proc/<pid>/fd/<fd>,
making some images relying on the special file /dev/stdout(stderr),
/proc/self/fd/1(2) fail to boot in passfd io mode, where the
stdout/stderr of a container process is a vsock socket.
For back compatibility, a pipe is introduced between the process
and the socket, and its read end is set as stdout/stderr of the
container process instead of the socket. The agent will do the
forwarding between the pipe and the socket.
Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
In passfd io mode, when not using a terminal, the stdout/stderr vsock
streams are directly used as the stdout/stderr of the child process.
These streams are non-blocking by default.
The stdout/stderr of the process should be blocking, otherwise
the process may encounter EAGAIN error when writing to stdout/stderr.
Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
We want the io connection keep connected when the containerd closed
the io pipe, thus it can be attached on the io stream.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Support the hybrid fd passthrough mode with passing pipe fd,
which can specify this connection kept even when the pipe
peer closed, and this connection can be reget wich re-opening
the pipe.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
In linux, when a FIFO is opened and there are no writers, the reader
will continuously receive the HUP event. This can be problematic
when creating containers in detached mode, as the stdin FIFO writer
is closed after the container is created, resulting in this situation.
In passfd io mode, open stdin fifo with O_RDWR|O_NONBLOCK to avoid the
HUP event.
Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
When container exits, the agent should clean up the term master fd,
otherwise the fd will be leaked.
Fixes: kata-containers#6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
When one end of the connection close, the epoll event will be triggered
forever. We should close the connection and kill the connection.
Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
Currently in the kata container, every io read/write operation requires
an RPC request from the runtime to the agent. This process involves
data copying into/from an RPC request/response, which are high overhead.
To solve this issue, this commit utilize the vsock fd passthrough, a
newly introduced feature in the Dragonball hypervisor. This feature
allows other host programs to pass a file descriptor to the Dragonball
process, directly as the backend of an ordinary hybrid vsock connection.
The runtime-rs now utilizes this feature for container process io. It
open the stdin/stdout/stderr fifo from containerd, and pass them to
Dragonball, then don't bother with process io any more, eliminating
the need for an RPC for each io read/write operation.
In passfd io mode, the agent uses the vsock connections as the child
process's stdin/stdout/stderr, eliminating the need for a pipe
to bump data (in non-tty mode).
Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
Two toml options, `use_passfd_io` and `passfd_listener_port` are introduced
to enable and configure dragonball's vsock fd passthrough io feature.
This commit is a preparation for vsock fd passthrough io feature.
Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
Using custom input paths with -i is counter-intuitive. Simplify path handling with explicit flags for rules.rego and genpolicy-settings.json.
Fixes: #8568
Signed-Off-By: Malte Poll <1780588+malt3@users.noreply.github.com>
The necessary files for the agent builder image can be found in
`tools/packaging/static-build/agent`,
`ci/install_libseccomp.sh` and
`tools/packaging/kata-deploy/local-build/kata-deploy-copy-libseccomp-installer.sh`.
Identifying the correct files addresses the previously misreferenced path
used to name the builder image.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Removed the setting of default values for runtime fields. Added explicit checks for missing or empty fields, reporting errors with clear messages.
Fixes: #8838
Signed-off-by: yaoyinnan <35447132+yaoyinnan@users.noreply.github.com>
By running `kustomize edit fix` on those files they have changed
deprecated instructions ('bases' and 'patchesStrategicMerge') as well as
'apiVersion' and 'kind' were added.
Fixes#8268
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
correct the lib file locations after the move from
tests->kata-containers repo and add a minimized version of the
".ci/lib.sh" library into the "ci/openshift-ci" as we don't really
utilize all of the features.
Fixes: #8653
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Move the f15be37d9bef58a0128bcba006f8abb3ea13e8da version of scripts
required for openshift-ci from "kata-containers/tests/.ci/openshift-ci"
into "kata-containers/kata-containers/ci/openshift-ci" and required
webhook+libs into "kata-containers/kata-containers/tools/testing" as is
to simplify verification, the different location handling will be added
in following commit.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
The `noop-method-call` is a rustc lint that has existed since v1.52.0.
This lint has been moved to the warn by default lint level since v1.73.0.
Therefore build is failing with this version and above.
This commit removes the unnecessary call to `<&T as Deref>::deref` on `T: !Deref`.
Fixes: #8586
Signed-off-by: Kvlil <kalil.pelissier@gmail.com>
This PR re-arranged the nerdctl tests to avoid random failures.
In this PR first will run the tests with RunC and then with the kata hypervisor.
This PR tries to avoid the random failures that is happening with cloud-hypervisor
and clh.
Fixes#8963
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The auto-generated Policy already allows these volumes to be mounted,
regardless if they are:
- Present, or
- Missing and optional
Fixes: #8893
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This involves using Ubuntu 20.04 as a build environment for an agent to match with a runtime environment.
Fixes: #8955
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Qemu stderr monitoring runs in its own asynchronous green thread.
For that, `stderr` is taken out of the Child representing the qemu child
process to avoid partial move and make it possible for the main thread
still to call functions on QemuInner::qemu_process (e.g. kill(), id()).
Fixes#8937
Signed-off-by: Pavel Mores <pmores@redhat.com>
We'll want to capture qemu's stderr in parallel with normal runtime-rs
execution. Tokio's primitives make this much easier than std's. This
also makes child process management more consistent across runtime-rs
(i.e. virtiofsd child process is already launched and managed using tokio).
Some changes were necessary due to tokio functions being slightly different
from their std counterparts. Child::kill() is now async and Child::id()
now returns an Option.
Signed-off-by: Pavel Mores <pmores@redhat.com>
Allow Kata CI's pod-nested-configmap-secret.yaml to work with
genpolicy and current cbl-mariner images:
1. Ignore the optional type field of Secret input YAML files.
It's possible that CoCo will need a more sophisticated Policy
for Secrets, but this change at least unblocks CI testing for
already-existing genpolicy features.
2. Adapt the value of the settings field below to fit current CI
images for testing on cbl-mariner Hosts:
"kata_config": {
"confidential_guest": false
},
Switching this value from true to false instructs genpolicy to
expect ConfigMap volume mounts similar to:
"configMap": {
"mount_type": "bind",
"mount_source": "$(sfprefix)",
"mount_point": "^$(cpath)/watchable/$(bundle-id)-[a-z0-9]{16}-",
"driver": "watchable-bind",
"fstype": "bind",
"options": [
"rbind",
"rprivate",
"ro"
]
},
instead of:
"confidential_configMap": {
"mount_type": "bind",
"mount_source": "$(sfprefix)",
"mount_point": "$(sfprefix)",
"driver": "local",
"fstype": "bind",
"options": [
"rbind",
"rprivate",
"ro"
]
}
},
This settings change unblocks CI testing for ConfigMaps.
Simple sanity testing for these changes:
genpolicy -u -y pod-nested-configmap-secret.yaml
kubectl apply -f pod-nested-configmap-secret.yaml
kubectl get pods | grep config
nested-configmap-secret-pod 1/1 Running 0 26s
Fixes: #8892
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR updates the required packages for the TensorFlow ResNet50
Int8 Dockerfile.
Fixes#8950
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Due to the changes done in the CI, we need to set the correct
subscription to be used with the account from now on, otherwise we'd end
up using CoCo subscription.
Fixes: #8946
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This fixes a flaw pointed out in review of PR #8185. Creation of the
directory semantically fits better into VM preparation than VM launch.
Signed-off-by: Pavel Mores <pmores@redhat.com>
We need to actually check that the env var is not empty. :-)
This was introduced by 8307718842.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The Confidential Containers guest-components will, in the very short
future, be part of the Kata Containers rootfs that's used by the
Confidential Containers usecase.
This commit introduces the ability to, standalone, build the component
locally and as part of our CI, and this can be done by calling:
`make coco-guest-components-tarball`
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Linda Yu <linda.yu@intel.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: Jakob Naucke <jakob.naucke@ibm.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
Co-authored-by: zhouliang121 <liang.a.zhou@linux.alibaba.com>
Co-authored-by: Alex Carter <alex.carter@ibm.com>
Co-authored-by: Suraj Deshmukh <suraj.deshmukh@microsoft.com>
Co-authored-by: Xynnn007 <xynnn@linux.alibaba.com>
Validating the node name is currently outside the scope of the CoCo
policy.
This change unblocks testing using Kata CI's test-pod-file-volume.yaml
and pv-pod.yaml.
Fixes: #8888
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Those are not yet being cached for no reason, and they better be as
it'll allow us to save a considerable amount of time building the
rootfs.
Fixes: #8917
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The existing confidential basic test titled `Test unencrypted
confidential container launch success and verify that we are
running in a secure enclave` has been updated to incorporate
IBM Secure Execution (`qemu-se`).
Previously, a secure image was absent from kata-deploy, hindering
the inclusion of IBM SE in the test.
Thanks to the #6755 update, it is now possible to test the TEE.
This modification extends the existing test by introducing
`qemu-se`. The specific changes are outlined below:
- Add an additional test `cc-se-e2e-tests` to s390x nightly
- Expansion of `REMOTE_COMMAND_PER_HYPERVISOR` for `qemu-se`
- Temporary exclusion of two test cases currently incompatible with IBM SE
(`cpu-ns` is a common issue across all TEEs, while `inotify`
will be addressed in a subsequent pull request).
Fixes: #8913
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Add metadata containing the Policy annotation if the user didn't
provide any metadata in the input yaml file.
For a simple sanity test using a Kata CI YAML file:
genpolicy -u -y job.yaml
kubectl apply -f job.yaml
kubectl get pods | grep job
job-pi-test-64dxs 0/1 Completed 0 14s
Fixes: #8891
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Let's start relying on the already cached agent to be deployed inside
the rootfs. By doing this we save a lot of time in our CI, and we have
a better way, for developers, to play with changes in the agent.
Fixes: #8915
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we'll be untarring the agent tarball (and any other component that
may be part of the rootfs) into the rootfs, we have to have xz
installed.
For debian and ubuntu the package is called xz-utils; for centos,
alpine and cbl-mariner the package is called xz.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This env var will serve us to pass the agent tarball to the rootfs
builder, which will then just unpack the content into the rootfs instead
of building the agent again.
AGENT_TARBALL and AGENT_SOURCE_BIN should never be used together.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're moving away from alpine and using ubuntu in order to be able to
build the agent for all the architectures we need.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This has been missed during reviews and is already a problem as we're
trying to build the agent outside of the rootfs for other architectures
than x86_64.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This has been missed during reviews and will become a problem when the
tools start to be built in different architectures.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Several use-cases need a vTPM lets enable it for QEMU, a follow up patch will introduce the runtime config.
Fixes: #8902
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This PR uses a variable that is already defined in the build base
qemu script to have uniformity across the script as this variable
is already used in the script.
Fixes#8903
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Lets remove the GHA target actions first so the the follow-up PR #8874 tests are succeeding.
Fixes: #8900
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Add --show-output-of-passing-tests to the k8s integration tests. The
output of a passing test can be helpful when investigating a failure
of the same test.
Fixes: #8885
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR is to add a document for how to run kata containers under IBM
Secure Execution environment.
Fixes: #7025
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This PR updates the libseccompt version to v2.5.5 which includes
the following changes:
- Update the syscall table for Linux
- Fix minor issues with binary tree testing and with empty binary trees
Fixes#8883
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The versions.yaml has a default for the amd64 binary, but there is no
code to actually build the arm64 binary, which seems an overlook.
Let's simplify the OPA logic by removing the direct link to the binary,
and construct that link as part of the checks we do to decide whether we
need to build OPA or not.
Fixes: #8373
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR updates the packages necessary to build the ResNet50 fp32
Dockerfile to run properly the benchmark.
Fixes#8875
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This is a follow up to the work of minimizing targets, unifying TDX,SNP builds for NVIDIA GPUs
Fixes: #8828
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
SharedVersion fiel add a versiontable property that isn't supported by upstream QEMU.
This is dead code since virtcontainers isn't setting SharedVersions to true.
Fixes: #7720
Signed-off-by: Kvlil <kalil.pelissier@gmail.com>
The changes to install and test genpolicy must come later, after CI
picks up these gha changes.
Fixes: #8856
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Ignore pod DNS settings because policing the network traffic is
currently outside the scope of the Agent Policy.
Example from Kata CI: pod-custom-dns.yaml
Fixes: #8832
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Deploy the framework added by the previous commit to generate qemu
command line and launch the VM.
We now properly store the child process object which allows us to
implement remaining Hypervisor functions necessary for a simple but
successful VM lifecycle, get_vmm_master_tid() and stop_vm().
Fixes#8184
Signed-off-by: Pavel Mores <pmores@redhat.com>
This PR updates the firecracker version to v1.6.0 which includes
the following features
- Added support for per net device metrics. In addition to aggregate metrics net, each individual net device will emit metrics under the label "net_{iface_id}". E.g. the associated metrics for the endpoint "/network-interfaces/eth0" will be available under "net_eth0" in the metrics json object.
- Added support for per block device metrics. In addition to aggregate metrics block, each individual block device will emit metrics under the label "block_{drive_id}". E.g. the associated metrics for the endpoint "/drives/{drive_id}" will be available under "block_drive_id" in the metrics json object.
- Added a new vm-state subcommand to info-vmstate command in the snapshot-editor tool to print MicrovmState of vmstate snapshot file in a readable format. Also made the vcpu-states subcommand available on x86_64.
- Added source-level instrumentation based tracing. See tracing for more details.
- Added developer preview only (NOT for production use) support for vhost-user block devices. Firecracker implements a vhost-user frontend. Users are free to choose from existing open source backend solutions or their own implementation. Known limitation: snapshotting is not currently supported for microVMs containing vhost-user block devices. See the related doc page for details. The device emits metrics under the label "vhost_user_{device}_{drive_id}".
Fixes#8854
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
- test_volume_capacity_stats: verify the file block size against the fetched size via statfs()
- test_reseed_rng: Correct the request codes for RNDADDTOENTCNT and RNDRESEEDCRNG when platform is ppc64le
- test list_routes: Add the route only if destination is not empty
- test_new_fs_manager: skip the test if cgroups v2 is used by default
- skip test cases rpc::tests::test_do_write_stream, sandbox::tests::test_find_process, sandbox::t
ests::test_find_container_process and sandbox::tests::add_and_get_container on ppc64le as they are fl
aky
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
kata-ctl currently fails to build on ppc64le. Skip it for running static checks and the issues will be fixed and tracked in a seperate issue.
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
A few CPU related test cases were failing as the version was being verified against Power8 while the CI machine is Power9.
Fixes: #5531
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
It was observed that a tmporary file `/tmp/kata_hybrid_vsock02.hvsock`
for test_setup_hvsock_failed() is not removed from time to time.
This leads to a test failure for the same test next time due to the
file permission on a self-hosted runner.
This commit is to explicitely delete the file before the check starts.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
At the moment, a project `dragonball` and `runtime-rs` does not support
for s390x. During the enablement, some errors due to the misconfiguration
of Makefile for `make check` and `make vendor` were identified.
This is to skip the build for the affected target of the projects.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Some linting errors were identified during the enablement of `make check`.
These have not been found by the Jenkins CI job because `make test` was
only triggered.
The errors for the `agent` occurs under the s390x specific tests while
the other ones for the `kata-ctl` are the architecture-specific code.
This commit is to fix those errors.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
If `yq_path` is set to `/usr/local/bin/yq`, there could be a situation
where the `yq` cannot be installed without `sudo`.
This commit handles the situation by putting `sudo` in front of `curl`
and `chmod`, respectively.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
As part of the CI migration from Jenkins to GitHub Action, a CI job named
`kata-containers-2.0-ubuntu-s390x-unit-PR` is covered by the static check.
This commit is to enable the check for s390x by incorporating a runner
`s390x` with the corresponding workflow.
Fixes: #8482
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The idea of most of these is just to prevent running into todo!()s where
we can at the moment, while implementing the fundamental functionality of
VM launch.
Signed-off-by: Pavel Mores <pmores@redhat.com>
Install the default genpolicy OPA rules and settings JSON files, in
addition to the genpolicy binary.
Fixes: #8844
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
DirectVolume/Rawblock doesn't work well when device's block driver
is virtio-blk-pci and the storage handler is DRIVER_BLK_PCI_TYPE.
Fixes: #8707
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Temporarily disable the allow_storages() rules, because they are based
on the tarfs snapshotter + container image integrity information that
are not available yet in the main branch - see #8833.
Fixes: #8834
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Adjust genpolicy-settings.json to match the container root path from
the main branch + cbl-mariner Guest VMs.
This configuration might have to be adjusted again when other types of
Guest VMs will be tested during CI using genpolicy, in the future.
Also, improve logging from allow_root_path(), to easier debug these
issues in the future.
Fixes: #8835
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR removes the references to virtiofs from memory average
calculation when the container uses a shared file system other than
virtiofs.
Fixes: #8807
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR removes the iperf3 server protocol as this server definition is
also used for the UDP iperf3 benchmarks to avoid duplication of the
same yaml files.
Fixes#8829
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR uses a specific python version to run tensorflow benchmark
as it needs python 3.8 to run correctly and avoid failures.
Fixes#8791
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Replace the `todo!()` calls with a minimal NOP implementation to return
the CH driver to working order since the `todo!()`'s forcibly crash the
driver at runtime. Full implementations for these APIs will be added on
issues #8800, #8801, and #8802.
Fixes: #8784.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Remove the `todo!()` macro which would cause a runtime crash and replace
with a implementation that returns an error as a stop-gap until #8800 is
implemented.
Fixes: #8785.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Containerd runtime options with wrong setting cause it failed.
Correct it as below:
...
[plugins.cri.containerd.runtimes.${runtime}.options]
ConfigPath= "${KATA_CONFIG_PATH}"
...
Fixes: #8746
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
It is a little bit heavy for the runtime-rs to forwards events via
containerd CLI, contrast to the ttrpc way. Plus, for runtimes that haven't
this mechanism, e.g. CRI-O, we can't get those events anywhere.
This patch introduces two types of forwarders:
- `ContainerdForwarder`: Acquire ttrpc address from environment variables
and forward events via ttrpc connection.
- `LogForwarder`: Write event info into logs.
Fixes: #7881
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This PR updates the runc version to 1.1.11 which includes the
following improvements
- Fix several issues with userns path handling.
- Support memory.peak and memory.swap.peak in cgroups v2. Add
swapOnlyUsage in MemoryStats. This field reports swap-only usage.
For cgroupv1, Usage and Failcnt are set by subtracting memory usage
from memory+swap usage. For cgroupv2, Usage, Limit, and MaxUsage
are set.
- build(deps): bump github.com/cyphar/filepath-securejoin.
Fixes#8795
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
We're using a Kernel based on v6.7, which should include all te
patches needed for SEV / SNP / TDX.
By doing this, later on, we'll be able to stop building the specific
kernel for each one of the targets we have for the TEEs.
Let's note that we've introduced the "confidential" target for the
kernel builder script, while the TEE specific builds are being kept as
they're -- at least for now.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As all the supported architectures are disabling the virtiofsd build,
there's no need to keep the switch statement there.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've not been building QEMU experimental for a very long time, and the
entry there has only been serving the purpose to clutter the
versions.yaml (in the best case scenario) or even confuse new
contributors to the project.
Mind that the machinery to build the QEMU experimental is not touched,
and that's used to build the TEEs capabale artefacts.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR removes the check images function from stressng test as now
it will part of the install dependencies function from gha-run script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
To avoid random failures while trying to build and install the stressng image,
this PR moves that step as part of the install dependencies in order to move
the stability tests and avoid timeouts.
Fixes#8787
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This is not necessarily meant to work, just to stub out unimplemented
functionality while focusing on more fundamental things.
Signed-off-by: Pavel Mores <pmores@redhat.com>
This PR fixes the indentation of the build static stratovirt script
for kata containers.
Fixes#8777
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Log the list of the current pods between tests because these pods
might be related to cluster nodes occasionally running out of memory.
Fixes: #8769
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The agent registers an event fd in `memory.oom_control`. An OOM event is
forwarded to containerd when the event is emitted, regardless of the
content in that file.
I observed content indicating that events should not be forwarded, as shown
below. When `oom_kill` is set to 0, it means no OOM has occurred. Therefore,
it is important to check the content to avoid mistakenly forwarding OOM
events.
```
oom_kill_disable 0
under_oom 0
oom_kill 0
```
Fixes: #8715
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This PR adds the qemu-experimental hypervisor in the function to
kill kata components.
Fixes#8775
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Don't release the lock between is_allowed and set_policy calls,
because the policy might change in between these calls.
Also, move more policy code into policy.rs.
Fixes: #8734
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
- `ttrpc` from `0.7.1` to `0.8`.
- `containerd-shim-protos` from `0.3.0` to `0.6.0`.
Fixes: #8756
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
As the StratoVirt VMM has been added, we can update the docs
and make some intoduction to StratoVirt, thus users can know more
about the hypervisor choices.
Fixes: #8645
Signed-off-by: Zhigang Wang <wangzhigang17@huawei.com>
Signed-off-by: Liu Wenyuan <liuwenyuan9@huawei.com>
In order to avoid rust-vmm upstream change breaks Dragonball
compilation, we introduce Cargo.lock to dbs crates.
fixes: #8770
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
In order to avoid rust-vmm upstream change breaks Dragonball
compilation, we introduce Cargo.lock to dbs crates.
fixes: #8770
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
As reported in #8767, we have found that the root cause is that rust-vmm's vmm-sys-utils
introduce a new release 0.12.1 and dbs-pci rely on rust-vmm's vfio-ioctls which uses >=
to declare vmm-sys-utils so it automatically upgrade vmm-sys-utils to 0.12.1.
That's how two different versions of vmm-sys-utils is introduced and this breaks the compilation.
In order to fix this and also avoid future problems, we introduce Cargo.lock file to dbs crates.
fixes: #8770
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
This PR improves the iperf3 cleanup to ensure all the components are
being deleted properly to avoid the random failures of leaving
the iperf3 clients on the kata metrics CI.
Fixes#8765
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Previously, Dragonball did not support PCI device hot-plugging or
VFIO device passthrough. Therefore, the runtime-rs support for
Dragonball was incomplete. it is time to complete it so that users
can use Dragonball's PCI hot-plugging and VFIO passthrough capabilities.
Fixes: #8748
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
vfio commits introduce quite a lot change in runtime-rs, this commit is
for all the changes related to ci, including compilation errors and so on.
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
add pci add and del guest kernel patch as the extension
in the upcall device manager server side.
also, dump config version to 120 since we need to add config
for dragonball pci in upcall
fixes: #8741
Signed-off-by: Gerry Liu <gerry@linux.alibaba.com>
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Introduce two new vmm action to implement pci hotplug
and pci hot-unplug: PrepareRemoveHostDevice and RemoveHostDevice.
PrepareRemoveHostDevice is to call upcall to unregister the pci device
in the guest kernel.
RemoveHostDevice should be called after PrepareRemoveHostDevice, it is used
to clean the PCI resource in the Dragonball side.
fixes: #8741
Signed-off-by: Gerry Liu <gerry@linux.alibaba.com>
Signed-off-by: Zizheng Bian <zizheng.bian@linux.alibaba.com>
Signed-off-by: Zha Bin <zhabin@linux.alibaba.com>
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Add a pcie_topology field to DeviceManager and initialize
pcie_topology when ResourceManager calls DeviceManager's new()
with TopologyConfigInfo.
Fixes: #7218
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Before calling the device driver to attach a device, register
the device to PCIe topology and allocate a PciPath for it.
However, for some hypervisor such as CLH, the allocation is invalid
when plugging devices to VM, they have the ability to return
DeviceInfo containing PciPath. It'll update the PciPath with the
returned pci path in the PCIe topology for them to prevent the
inferred pcipath from being different from the actual value returned.
But the update will not be executed if the pcipath value doesn't change.
Fixes: #7218
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Introduce helper macros to simplify PCIe device register/unregister
and update, which provides a convenient way to handle devices in
topology.
Fixes: #7218
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Add one more argument with type &mut Option<&mut PCIeTopology>
in attach and detach to inroduce methods within PCIe Topology.
Fixes: #7218
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Implement Trait PCIeDevice register/unregister for pcie/pci
device, such as vfio device which needs set/get device's pci
path for kata agent's device handler.
Fixes: #7218
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Introduce Trait PCIeDevice with register/unregister, which are
used to register or unregister pcie device within the PCIe topology.
Fixes: #7218
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Due to different ways that different VMMs handle PCI devices,
we expect to provide a general PCIe topology processing framework
that is as compatible as possible with VMMs such as dragonball,
qemu, clh(Though it has its own management method, no conflict).
Currently,it's mainly developed for kinds of PCIe/PCI devices in
dragonball/clh which are attached on the pci/pcie root bus directly.
More will be added when Qemu is ready in runtime-rs.
Fixes: #7218
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
A TopologyConfigInfo added to store device config info for PCIe/PCI
devices in the VM from Hypervisor DeviceInfo.
And TopologyConfigInfo::new will be the entry to initialize PCIe
Topology for each VM.
Fixes: #7218
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Bridge the gap between user requirements for direct block device access
and the DirectVolume capabilities provided by Kata runtimes
(kata-runtime/runtime-rs), and facilitate seamless integration with CSI
to improve user experience.
It aims to integrate DirectVolume CSI support into Kata, enabling users
to benefit from its performance and flexibility advantages.
Fixes: #8602
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Otherwise `jq` will complain about:
```sh
jq: error: nydus/0 is not defined at <top-level>, line 1:
.plugins."io.containerd.grpc.v1.cri".containerd.runtimes."kata-clh".snapshotter=nydus
jq: 1 compile error
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This patch introduces a feature of supporting vhost-user-blk device.
This device needs to be defined before the VM instance is started,
which can be done through the dbs-cli tool with --virblks option:
--virblks '{
"drive_id": "8623",
"device_type": "Spdk",
"path_on_host": "spdk:///var/tmp/vhost.sock",
"is_root_device": false,
"is_read_only": false,
"is_direct": false,
"no_drop": false,
"num_queues": 1,
"queue_size": 256
}'
Fixes: #8631
Signed-off-by: Eric Ren <renzhen@linux.alibaba.com>
Signed-off-by: fupan <fupan.lfp@antgroup.com>
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Qinqi Qu <quqinqi@linux.alibaba.com>
In the way the script is without this patch, we're trying to set
```toml
[`$shim`]
snapshotter = $snapshotter
```
However, what we actually want to set is the full runtime table instead
of shim.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We want to check whether the shim is part of the "plain text" shims
passed to the daemonset (meaning, checking against `$SHIMS`). Before
this fix we were checking against `$shims`, which is an array of shims
instead of a string, resulting on a broken check.
Fixes: #8732
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
move `SKIP_GO_VERSION_CHECK` after `make` command to skip
checking golang version.
And also upgrade golang to 1.19.
Fixes: #8728
Signed-off-by: Bin Liu <bin@hyper.sh>
The compiler will give a warning if a developer forget to add an arm for
a new variants defined.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
DAN reads vhost-user-net device from JSON config. It only supports VMM
running as server right now.
Fixes: #8625
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
The changes involve:
- Expose VhostUserConfig struct to runtime-rs.
- Set a default value while num_queues or queue_size are 0.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This commit introduces VhostUserEndpoint and supports relative to
vhost-user-net devices for device manager. For now, Dragonball is able to
attach vhost-user-net devices.
Fixes: #8625
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Config space of network device is shared and accord with virtio 1.1 spec.
It is a good way to abstract the common part into one function.
`set_config_space()` implements this.
Plus, this patch removes `vq_pairs` from vhost-net devices, since there is
a possibility of data inconsistency. For example, some places read that
from `self.vq_pairs`, others read from `queue_sizes.len() / 2`.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Add application that infers K8s user's intentions based on user's
K8s YAML file, and generates a Rego/OPA based policy for that YAML.
Just Pod YAML files are supported as input using this initial source
code. Support for other types of YAML files will come with upcoming
commits.
Fixes: #7673
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Vhost-user-net has a dependency on address space from `MmioV2DeviceState`.
The addition of the address space is introduced in this patch. Plus, it
makes sure all unit tests have the according parameter as well.
Fixes: #8502
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
The default network backend of runtime-rs with Dragonball is vhost-net
after #8609 merged. The tests might be failed if vhost modules are not
loaded.
Fixes: #8717
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
It's important to ensure that these tasks which setup vfio
devices are completed before add_device.
So Moving vfio device setup code to a dedicated method at device
building time which does not affect the behavior of other code.
And this change makes it easier to understand the difference
between create and attach, and also makes the boundaries
clearer.
Fixes: #8665
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Make VhostUserConfig pci_path's type more specific, change it
from Option<String> to Option<PciPath>.
Fixes: #8665
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Since containerd 1.7.0 we can easily set a specific snapshotter to be
used with a runtime handler, and we should take advantage of this,
mostly as it'll help setting up any runtime using devmapper or nydus
snapshotters.
This implementation here has a few caveats:
* The format expected for the SNAPSHOTTER_HANDLER_MAPPING is:
`shim:snapshotter,shim:snapshotter,...`
* It only works with containerd 1.7 or newer
* We **never** change the default containerd snapshotter
* We don't do any check on our side to verify whether the snapshotter
required is properly deployed
* Users will have to add an annotation to their pods, in order to use
the snapshotter set up per runtime handler
* Example:
```
metadata:
...
annotations:
io.containerd.cri.runtime-handler: kata-fc
```
Fixes: #8615
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Switch to use function from Kata repo in common.bash to reduce
dependency on the tests repo.
Fixes#8713
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Add implementations for the following `Hypervisor` trait methods which
simply return the same details as the `get_vmm_master_tid()` method:
- `get_thread_ids()`
- `get_pids()`
Fixes: #6438.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This reverts commit ee5fa08a27.
This is perfectly fine to do as we narrwoed down the issue to be on the
version of `jq` provided by alpine, and we've already updated it in the
previous commit (in this very same series).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
`jq` coming from alpine is in its 1.6 version, and that has a bug that
hits us quite hard, as it changes a float to an int whenever the number
is in the `x.0` format.
One example is:
```bash
/ # jq --version
jq-1.6
/ # echo '{"foo": 1.0}' | jq .foo
1
```
With this in mind, let's switch, at least for now, to using the `jq`
released directly on github, as it does address the issue we've been
hitting.
```bash
⋊> Downloads ./jq-linux-amd64 --version
jq-1.7
⋊> Downloads echo '{"foo": 1.0}' | jq .foo
1.0
```
Fixes: #8678
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
After kata-deploy has installed, check that the worker nodes
are still in Ready state and don't have a containerd://Unknown
container runtime versions, identicating that container isn't working
to ensure that we didn't corrupt the containerd config during kata-deploy's edits
Fixes: #8678
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
To become more resilient against these kinds of errors:
deployment.apps/confidential-unencrypted created
pod/confidential-unencrypted-c5fdd6964-rrb6q condition met
ssh: connect to host 10.42.0.109 port 22: Connection refused
Fixes: #8687
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This is to reintroduce a configuration rule for IBM Z Secure Execution,
where no initrd path should be configured. For the TEE of interest,
only a kernel image should be specified with `confidential_guest=true`.
Fixes: #8692
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
`make SUPPORT_VIRTUALIZATION=1 test` iterates through all subcrates and
does test.
Plus, this patch fixes some issues about unit tests:
- Feed too much parameters to `I8042Device::new()`.
- Virtqueue checks have been introduced since `virtio-queue v0.7.0`.
- GHA might have no access to `/var/tmp` dir on runner.
Fixes: #8690
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This PR fixes the indentation of the confidential common
script for kubernetes tests.
Fixes#8698
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
As a follow-up for #8516, guest_cid and vhost_fd are not necessarily initialised
via new(). Instead, the fields should be initialised later when they are really
used to construct hypervisor's parameters.
This commit is to separate init_config() from new() to initialise guest_cid
and vhost_fd and leave only the assignment of id for the existing function.
Fixes: #8671
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
We now use the static checks script from the main kata containers repo
and not the tests repo; update documentation to reflect this.
Fixes#8681
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Change the two functions in lib.sh to use the static checks script from
the kata containers repo instead of tests. Remove cloning the repo from
these functions since we don't need it anymore. Leave these two
functions because the document checking one may be used locally and the
static checks one is called from the virtcontainers Makefile.
Fixes#8681
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
If a PR contains files from the ignore-paths, these actions do not run
as intended. However, the actions are make as required. And there does
not seem to be a way to mark these as non-required in that case.
As a result a PR containing the files from the ignore-paths remains
stalled.
Hence remove the ignore-paths until github provides a way to mark
actions that are skipped due to ignore-paths as non-required/passed.
Fixes: #8663
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Update the Speed & Density metric tests baseline for StratoVirt
and re-enable them, and skip other metric tests temporarily.
Fixes: #8656
Signed-off-by: Liu Wenyuan <liuwenyuan9@huawei.com>
fix error when running checks and tests:
error: failed to run custom build command for `devicemapper-sys v0.1.5`
fatal error: 'libdevmapper.h' file not found
thread 'main' panicked at 'Could not generate dm.h bindings:
ClangDiagnostic("dm.h:2:10: fatal error: 'libdevmapper.h' file not found\n")',
/home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/devicemapper-sys-0.1.5/build.rs:24:10
stack backtrace:
0: rust_begin_unwind
at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:593:5
1: core::panicking::panic_fmt
at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/panicking.rs:67:14
2: core::result::unwrap_failed
at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/result.rs:1651:5
3: core::result::Result<T,E>::expect
4: build_script_build::main
5: core::ops::function::FnOnce::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
warning: build failed, waiting for other jobs to finish...
make: *** [../../utils.mk:177: standard_rust_check] Error 101
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Fix error when making checks:
```
error: failed to run custom build command for `image-rs v0.1.0
(https://github.com/confidential-containers/guest-components?tag=v0.8.0#e849dc89)`
Caused by:
process didn't exit successfully: `/home/runner/work/kata-containers/kata-containers/src/
agent/target/release/build/image-rs-fd932206d09362b7/build-script-build` (exit status: 101)
--- stdout
cargo:rerun-if-changed=./protos/getresource.proto
cargo:rerun-if-changed=./protos
--- stderr
thread 'main' panicked at 'Could not find `protoc` installation and this build crate cannot proceed without
this knowledge. If `protoc` is installed and this crate had trouble finding
it, you can set the `PROTOC` environment variable with the specific path to your
installed `protoc` binary.If you're on debian, try `apt-get install protobuf-compiler`
or download it from https://github.com/protocolbuffers/protobuf/releases
```
Fixes#8673
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
We use a matching direct-volume path to determine whether an OCI mount
is a DirectVolume. However, we should handle the case where no match is
found appropriately.
This error will be defined as a non-DirectVolume type when judging the
OCI mount but not failed.
Fixes: #8619
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
DirectVolume structure in runtime-rs is different from it in kata-runtime,
which causes they has no unified handling method for DirectVolumeMountInfo
and MountInfo.
We should align the two by simply adding the attribute #[serde(rename="x")
to each field in DirectVolumeMountInfo
Fixes: #8619
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Dragonball sets a default queue config in the case of `None`. The
queue_size and num_queues of vhost-net are set to `Some(0)` by default.
Therefore, we might get an invalid queue config. This patch fixes this
issue.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This patch set vhost-net as default backend of networking. It allows users
to set `disable_vhost_net` to `true` to reenable virtio-net backend.
Plus, which backend to use is a matter of hypervisor, runtime-rs will no
longer need to know that.
Fixes: #8608
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This PR improves the latency network cleanup by removing the pods
even if the test fails.
Fixes#8658
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This is to introduce a pre-action to all the workflows for building artifacts.
The action could take care of tasks such as cleaning up files and reinstalling
packages, which prevents a workflow from getting affected by the environment.
This also includes the removal of the step `Adjust a permission for repo`,
because it could be incorporated into the action.
Fixes: #8648
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
We never have to add a configuration for the "default" case, as we're
already creating the runtime class pointing to what should be the
"default" handler.
This helps to simplify the logic by quite a lot.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The comment adds absolutely nothing to the runtime handler added, and
it'd make our life slightly harder to properly say which VMM is being
used when setting the default `kata` handler.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This save us a lot of trouble on properly sed'ing content that may or
may not be in the containerd configuration file.
Fixes: #8638
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will help us to have an easier time playing with the containerd
configuration, instead of having to sed the **** out of it, which is
super error prone.
`tomlq` is a tool that comes from https://github.com/kislyuk/yq, and
that depends on `jq` to do the toml parsing / editing.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Direct the Makefile to use the static checks script in the tests
directory of the main Kata Containers repo so it is run in GHA.
Fixes#8595
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Some copyright dates were not updated with the most recent changes to
code; update them.
Fixes#8595
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Change directory for running make due to local errors when building with
make -C.
Fixes#8595
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Make the CH (Cloud Hypervisor) `stop_vm()` method check the VM state before
attempting to stop the VM, and update the state once the VM has stopped.
This avoids the method failing if called multiple times which will
happen if the workload exits before the container manager requests that
the container stop.
This change ensures the CH driver finishes cleanly.
Fixes: #8629.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This PR updates the TensorFlow ResNet50 Int8 Dockerfile to use the
proper python version for kata metrics.
Fixes#8643
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Add a `--show-default-config-paths` command line option for parity with
`kata-runtime`.
Note that this requires the `KataCtlCli.command` to be optional so that
the user can run simply:
```bash
$ kata-ctl --show-default-config-paths
```
... without also specifying a (sub-)command.
Fixes: #8640.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Fix paths for yqdir (where the install_yq.sh script currently is) so
that static checks can run without error.
Fixes#8595
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
The layout of packed virtqueue isn't supported by `Endpoint::negotiate()`.
Communication between device and driver will be failed due to the failure
of parsing virtqueue if we don't disable the packed feature. This patch
fixes this issue.
Fixes: #8633
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Right now, cargo fmt check in Dragonball only test with the default
features but not all features. This will cause some code being untested
by the fmt tool.
This PR adds --all option for the Dragonball CI and also fix some code
that forgets to do cargo fmt --all.
fixes: #8598
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
This is to fix a broken usage for `k3s kubectl version` by switching
an option `--short` to `--client=true`.
Fixes: #8621
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Make the URL checker cycle through a list of user agent values until we
hit one the remote server is happy with.
This is required since, unfortunately, we really, really want to check
these URLs, but some sites block clients based on their `User-Agent`
(UA) request header value. And of course, each site is different and can
change its behaviour at any time.
Our strategy therefore is to try various UA's until we find one the
server accepts:
- No explicit UA (use `curl`'s default)
- Explicitly no UA.
- A blank UA.
- Partial UA values for various CLI tools.
- Partial UA values for various console web browsers.
- Partial UA for Emacs's built-in browser.
- The existing UA which is used as a "last ditch" attempt where the UA implies multiple platforms and browser.
> **Notes:**
>
> - The "partial UA" values specify specify the UA "product" but not the
> UA "product version": we specify `foo` and not `foo/1.2.3`). We do
> this since most sites tested appear to not care about the version.
> This is as expected given that the version is strictly optional (see `[*]`).
>
> - We now log all errors and display an error summary if none of the UAs
> worked, in addition to the simple list of the URLs we believe to be
> invalid. This should make future debugging simpler.
`[*]` - https://www.rfc-editor.org/rfc/rfc9110#section-10.1.5Fixes: #8553.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Removed the Azure Portal URL (https://portal.azure.com) since this
causes problems with our static checks script: that URL returns HTTP 403
("Forbidden") when queried using command-line tools like `curl(1)`,
which is used by the static check script.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Split the call to `curl` in the URL checker out into a new
`run_url_check_cmd()` function to make `check_url()` slightly clearer.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
If a wrong configuration.toml file is used by accidentally, runtime-rs
binary could run into panic because of unwrap().
This fixes the panic by returning errors instead of unwrap().
fixes: #8565
Signed-off-by: Liu Bo <liub.liubo@gmail.com>
This is to make GHA secrets inherited for the workflow titled
`build-kata-static-tarball-s390x` to configure an environment
variable `CI_HKD_PATH` for a `build-asset-boot-image-se` step.
Fixes: #8611
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Log-parser-rs was always intended to become a sub-functionality of
kata-ctl, but it was useful to develop it and initaly merge it as a
standalone program, and migrate it to a subcommand later.
Fixes#6797
Signed-off-by: Gabe Venberg <gabevenberg@gmail.com>
A prerequisite for measuring kata network bandwidth is
run Iperf3 tool at a the transport layer provided by a
k8s service for exposing a network where the clients
inside the cluster can use to contact Pods in the service.
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR fixes small issues like:
1. Cleaning up the k8s environment by removing the iperf test
implementation even when the test fails.
2. Checks if the workload returned a result before generating
an empty results json file as it was bein done.
3. Removes the redundancy of calls to functions that process
subtests and should compose the results json file only when
all results are ready and not before.
4. The tcp service manifest was added to the server deployment
which targets TCP port 5201.
Fixes: #8534
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Since cloud-hypervisor is no longer built as an optional feature,
lets mention cloud-hypervisor in the list of hypervisors supported by
runtime-rs.
Fixes: #8587
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
It is to remove the build redundancy of `kernel` and `rootfs-initrd` by making `boot-image-se` built based on them at the second build stage.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This is to make `build_se_image.sh` incorporate the key verification originally supported by `genprotimg`.
It can be achieved by specifying two environment variables called `SIGNING_KEY_CERT_PATH` and `INTERMEDIATE_CA_CERT_PATH`.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This is to make kernel parameters configurable during the secure image build by adding an environment variable SE_KERNEL_PARAMS.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This is to make a base builder image build genprotimg without a package
manager under the cross-compilation environment.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This PR adds the Cloud Hypervisor driver, integrated with the runtime-rs,
as part of the kubernetes tests.
Fixes#8559
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR enables but do not run the nerdctl tests for cloud hypervisor
runtime-rs until we find out how stable they are.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
It is to remove a GITHUB_WORKSPACE directory for self-hosted runners
when a workflow fails due to the merge conflict. This will prevent
the subsequent workflows from getting stuck in the same situation.
Fixes: #8600
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Implement try_from trait function to convert runtime-rs BlockConfig
to cloud-hypervisor DiskConfig. This can allow for code reuse in the
future.
Fixes: #8581
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This PR updates the python version for the TensorFlow ResNet FP32
dockerfile so the benchmark can run without issues.
Fixes#8593
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Fixes: #8508
Create a stable overlay for kata-deploy.yaml so we do not have to maintain two files, only one.
Single source for both. This is also preparation for the helm-overlay
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This patch implements the virtio-fs device used for filesystem sharing
and heavily based on the vhost-user protocol.
This vhost-user-fs device defines 5 parameters:
- path: vhost-user socket path
- tag: mount tag used from the guest to mount the filesystem
- req_num_queues: number of request virtqueues
- queue_size: depth of each virtqueue
- cache_size: cache window size for dax
This device needs to be defined before the VM instance is started,
which can be done through the dbs-cli tool with --fs option:
--fs '{
"sock_path":"/path/to/virtiofs.socket",
"tag":"myfs",
"num_queues":1,
"queue_size":1024,
"cache_size":0,
"thread_pool_size":1,
"cache_policy":"auto",
"writeback_cache":true,
"no_open":true,
"xattr":true,
"drop_sys_resource":false,
"mode":"vhostuser",
"fuse_killpriv_v2":true,
"no_readdir":false,
}'
Fixes: #8428
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
Signed-off-by: Huang Jianan <jnhuang@linux.alibaba.com>
This is required for clh to work with nerdtcl and docker.
This fixes the issues seen with nerdctl while starting a container.
Hoewever, container exit with docker is still broken due to an unrelated
issue.
Fixes: #8579
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This PR updates the config containerd url link in the containerd
kata documentation.
Fixes#8577
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This will make the life easier for dragonball developers to properly
enable the tests once the tests are ready.
Fixes: #8569
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will make the life easier for dragonball developers to properly
enable the tests once the tests are ready.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will make the life easier for dragonball developers to properly
enable the tests once the tests are ready.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
PR #8483 changed the location of the rust runtime config files to
`/etc/kata-containers/runtime-rs/`. However, if you haven't updated your
system to create that directory, attempting to create a container using
the rust runtime was giving the following cryptic message
(formatted for easier reading):
```
failed to handler message try init runtime instance
Caused by:
0: load config
1: load toml config
2: entity not found
```
Now, the message is as follows (again, reformatted for easier reading):
```
failed to handle message try init runtime instance
Caused by:
0: load config
1: load TOML config failed (tried [
\"/etc/kata-containers/runtime-rs/configuration.toml\",
\"/usr/share/defaults/kata-containers/runtime-rs/configuration.toml\",
\"/opt/kata/share/defaults/kata-containers/runtime-rs/configuration.toml\"
])
```
Fixes: #8557.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This will ensure no job will be stuck forever, as we've noticed with a
few jobs already.
Fixes: #8572
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR updates the cri installation url link for the containerd
documentation.
Fixes#8539
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the Cloud Hypervisor driver, integraedwith the runtime-rs,
as part of the stability tests.
Fixes#8462
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The driver enables the userspace interface to communicate with the TDX
module to request the TDX guest details, like the attestation report.
Fixes: #8555
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Bring support for legacy vsock and add Vsock to the ResourceConfig
enum type, and add the processing flow of the Vsock device to the
prepare_before_start_vm function.
Fixes: #8474
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Instroduce prepare_vm_socket_config to VirtSandbox for vm
socket config, including Vsock and Hybrid Vsock.
Use the capabilities() trait of the hypervisor to get the
vm socket supported in VMM.
Fixes: #8474
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Add Cap of HybridVsockSupport for hypervisors CLH and Dragonball
which use hybrid-vsock, default for Qemu, which uses legacy vsock.
Fixes: #8474
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Introduce HybridVsock Cap to judge which kind of vm socket will
be supported by the Hypervisor.
Use `is_hybrid_vsock_supported` to tell if an hypervisor supports
hybrid-vsock, if not, it supports legacy vsock.
Fixes: #8474
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Remove the redundant `kata-ctl` `root` check when running the `env`
command. This check duplicated the `GuestProtection` check, and that
check is now no longer necessary anyway.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
It is no longer necessary to be `root` to query the guest protection
(TDX) on `x86_64` systems, so drop the requirement.
> **Note:**
>
> This change drops the `nix` `Uid` import required for the `root` check.
> But at the same time it adds it for PPC64le since that implementation of
> `available_guest_protection()` needs it and it was previously missing.
Fixes: #8548.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
PR #8311 inadvertently broke the logging since no log messages below the
`Info` level are logged now, regardless of the requested log level.
Resolve the issue by storing the requested log level in the
`RuntimeComponentLevelFilter` and using that level in the `log()`
function, rather than hard-coding `Info` as the default where no entry
is found in the `FILTER_RULE` hashmap.
Fixes: #8546.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Device mapper is the only supported block device driver so far,
which seems limiting. Kata Containers can work well with other
block devices. It is necessary to enhance supporting of multiple
kinds of host block device.
Fixes#4714
Signed-off-by: yuchen.cc <yuchen.cc@alibaba-inc.com>
This reverts commit b0157ad73a.
```
commit b0157ad73a
Refs: 3.3.0-alpha0-124-gb0157ad73
Author: Fabiano Fidêncio <fabiano.fidencio@intel.com>
AuthorDate: Fri Aug 11 14:55:11 2023 +0200
Commit: Fabiano Fidêncio <fabiano.fidencio@intel.com>
CommitDate: Fri Nov 10 12:58:20 2023 +0100
runtime: confidential: Do not set the max_vcpu to cpu
We don't have to do this since we're relying on the
`static_sandbox_resource_mgmt` feature, which gives us the correct
amount of memory and CPUs to be allocated.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
```
This commit was removing a requirement that was made previously, but due
to the SMP issue we're facing with the QEMU used for TDX (see commit
d1b54ede290e95762099fff4e0bcdad10f816126*), QEMU will fail to start due
to:
```
Invalid CPU topology: product of the hierarchy must match maxcpus:
sockets (1) * dies (1) * cores (1) * threads (1) != maxcpus (240)"
```
This has no affect on the SEV / SNP workflow and hopefully we'll be able
to re-revet this soon enough, when this gets solved on te QEMU side.
Last but not least, this is not a "clean" revert as we're using
conf.NumVCPUs() instead of conf.NumVCPUs, to ensure we're dealing with
uint32.
Fixes: #8532
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is a commit that's a pre-req for #6826, as that PR will merge
log-parser-rs into kata-ctl, but that will result in a CI breakage.
So, let's deal with the CI changes here, thanks to GHA and our favourite
`pull_request_target` event, unblocking that PR to be merged.
Fixes: #6797 (not really, but related).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Print more information useful for debugging. Also, use a separate YAML
file for this test, instead of reusing someone else's file.
Fixes: #8270
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
- Update the remote hypervisor code to match the re-genned code for
the ttrpc Hypervisor Service
Fixes: #8519
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
As the configuration for the runtime-rs based drivers are now placed in
a different location than the golang ones, we should adapt this script
accordingly.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
`kata-ctl` is the tool for runtime-rs, and it should be used instead of
`kata-runtime`.
`kata-ctl` requires sudo, and that's the reason it's also been added as
part of the calls.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
`kata-runtime env` is an alias for `kata-runtime kata-env, and calling
it with the `env` paramenter allows us to easily extend the scripts to
use `kata-ctl` instead of `kata-runtime` when dealing with runtime-rs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we've done some changes in the runtime-rs based drivers to install
their configuration into a different location, this should also be
reflected as part of this test.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Updates to scripts for static-checks.sh functionality, including common
functions location, the move of several common functions to the existing
common.bash, adding hadolint and xurls to the versions file, and changes
to static checks for running in the main kata containers repo.
The changes to the vendor check include searching for existing go.mod
files but no other changes to expand the test.
Fixes#8187
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
This reverts commit 44899d4cdf, as we've
decided to keep both golang and rust runtime installable and usable at
the same time.
The decision of having both runtimes installable and usable will help
users to test and easily catch any possible differences between those
runtimes, helping us to get on par with both implementations.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This change for now doesn't do much, apart from making it easier to
expand which runtimes should be linked to the runtime-rs containerd shim
binary.
Also, this matches the logic used for the config files.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Update `kata-deploy` to modify the rust runtime configuration files in
their new `runtime-rs/` directory.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This PR updates the iperf3 network documentation to include
the parallel bandwidth.
Fixes#8523
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Implements the following test case:
Scenario: Check incorrect hash fails
**Given** I have a version of kata installed that has a kernel with the
initramfs built and config with rootfs_verity.scheme=dm-verity
rootfs_verity.hash=<incorrect hash of rootfs> set in the kernel_params
**When** I try and create a container a basic pod
**Then** The pod is doesn't run
**And** Ideally we'd get a helpful message to indicate why
Currently on CI only qemu-tdx is built with measured
rootfs support in the kernel, so the test is restriced to that
runtimeclass.
Fixes#7415
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Bring the setup_common() from CCv0 branch test's
integration/kubernetes/confidential/tests_common.sh. It should be used
to reduce boilerplates on the setup() of the tests.
Unlike the original code, this won't export the `test_start_time` variable
as it wouldn't be accurate to grab logs from the worker nodes due
date/time mismatch between the running tests machine and the worker
node. The function export the `node` variable which holds the name of
a random node which has kata installed. Apart from that, it exports the
`node_start_time` which capture the date/time when the test started,
relative to the `node`.
Tests that should inspect the logs can schedule pods/resources to the `node`
and use `node_start_time` as the value reference to grep the logs.
Fixes#7590
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Bring the assert_logs_contain() from CCv0 branch tests'
integration/kubernetes/confidential/lib.sh.
Introduced the print_node_journal() which uses `kubectl debug` to print
the systemd's journal of a k8s's node.
Fixes#7590
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This new function allow to the annotations to metadata section in a yaml
configuration file.
Co-authored-by: Ryan Savino <ryan.savino@amd.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The new clean-generated-files make target allows for removing the
generated files (including the configuration.toml files).
The tools/packaging/static-build/shim-v2/build.sh script now uses that
target to always force the re-generation of those files.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Copied the new_pod_config() and pod-config.yaml.in from CCv0 branch
tests' integration/kubernetes/confidential/tests_common.sh and fixtures.
Unlike the original version, new_pod_config() now gets the runtimeclass
by parameter as the RUNTIMECLASS environment variable seems not broadly
used on main branch's CI.
The pod-config.yaml.in was changed as the diff shows below. In
particular the imagePullSecrets was removed to avoid it throwing a
warning on the pod's log.
```
--- a/tests/integration/kubernetes/runtimeclass_workloads/pod-config.yaml.in
+++ b/tests/integration/kubernetes/runtimeclass_workloads/pod-config.yaml.in
@@ -5,12 +5,10 @@
apiVersion: v1
kind: Pod
metadata:
- name: busybox-cc
+ name: test-e2e
spec:
runtimeClassName: $RUNTIMECLASS
containers:
- - name: nginx
+ - name: test_container
image: $IMAGE
- imagePullPolicy: Always
- imagePullSecrets:
- - name: cococred
\ No newline at end of file
+ imagePullPolicy: Always
\ No newline at end of file
```
Co-authored-by: Georgina Kinge <georgina.kinge@ibm.com>
Co-authored-by: Megan Wright <Megan.Wright@ibm.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The following functions were copied from CCv0's branch test's
integration/kubernetes/confidential/lib.sh. I did just smalls
refactorings (shortened their names and delinted shellcheck warnings):
- k8s_delete_all_pods_if_any_exists()
- k8s_wait_pod_be_ready()
- k8s_create_pod()
- assert_pod_fail()
Co-authored-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Co-authored-by: Georgina Kinge <georgina.kinge@ibm.com>
Co-authored-by: Jordan Jackson <jordan.jackson@ibm.com>
Co-authored-by: Megan Wright <Megan.Wright@ibm.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
Re-wrote the logic of init.sh to follow the rules:
* the root device MUST exist always because it will be either mounted
or verified (then mounted)
* if rootfs verifier is enabled then the hash device MUST exist. Avoid
the case where dm-verity is set but the hash device does not exist and
so the verification is silently skipped
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
When measured toofs is enabled then the shim-v2 build should find the
guest rootfs hash file, otherwise might (silently) generate configuration
files with empty hash.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The KATA_BUILD_CC variable plus the existence (or not) of the initramfs
were used to determine whether to build the kernel for measured rootfs
or not. Currently the variable MEASURED_ROOTFS has been used
to trigger the feature build and when it is activated it should expect
the initramfs exist. In other words, this changed the kernel build
so that if `MEASURED_ROOTFS=yes` then the initramf file must exist and
be found.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Moved the measure rootfs logic from kata-deploy-binaries.sh to the
shim-v2's builder script so that the former get less bloated with
components's specific code.
Fixes#6674
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Moved the measure rootfs logic from kata-deploy-binaries.sh to the
kernel's builder script so that the former get less bloated with
components's specific code.
Fixes#6674
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
By convention the caller of tools/packaging/kernel/build-kernel.sh changes
the script behavior by passing arguments, whereas, for measured rootfs
it has used an environment variable (MEASURED_ROOTFS). This refactor
the script so that the caller now must pass the "-m" argument to enable
the build of the kernel with measured rootfs support.
Fixes#6674
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
As the configuration files are different, we can safely remove those as
any new installation of the binary should also bring in the new
configurations.
This makes things less error-prone in the future, as we're ensuring that
the rust runtime will only be reading the rust configuration files.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Update the `DEFAULT_RUNTIME_CONFIGURATIONS` list to include a number of
rust runtime specific paths to try to load before checking the
"traditional" (golang) runtime configuration paths.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Install the rust runtime configuration files to a `runtime-rs/`
directory to distinguish them from the golang config files (which may
have a different syntax).
The default values mean that the rust config files are now installed to
`/opt/kata/share/defaults/kata-containers/runtime-rs/` rather than
`/opt/kata/share/defaults/kata-containers/`.
See: https://github.com/kata-containers/kata-containers/issues/6020Fixes: #8444.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
(1) Add enum DirectVolumeType for direct volumes.
(2) Reimplement spdk volume into direct_volume and
do alignment of rawblock volume.
Fixes: #8300
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
The current infra(K8S, CSI, CRI, Containerd) for Kata containers is
unable to properly handle direct volumes, resulting in the need for
workarounds like searching/comparision and then patch up volume type.
In this commit, reimplement of handling method is added to support
raw block volume which backends may be rawdisk or other format file.
Fixes: #8300
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
As vsock device will be used in Qemu or other VMMs, the Vsoock
is reintroduced to DeviceType enum.
Fixes: #8474
Signed-off-by: Pavel Mores <pmores@redhat.com>
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Currently encounters difficulty in utilizing the clone operation
on VsockConfig due to the implicit management of the vhost fd
within the runtime-rs. This responsibility should be delegated to
the VMM(especially QEMU) child process, as it's not runtime-rs core
responsibilities. We'll remove the member vhost_fd from VsockConfig
and make the VsockConfig/VsockDevice Cloneable.
Fixes: #8474
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Introduce a new function generate_vhost_vsock_cid to generate
a guest CID and set guest CID for vsock fd.
Also this commit wouldn't introduce functional change and it's
just splited from the previous VsockDevice::new().
Fixes: #8474
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
(1) rawblock volume is directvol mount type.
(2) block volume is based on the bind mount type.
Fixes: #8300
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
1) Creating storage for all `io.katacontainers.volume=` messages in rootFs.Options,
and then aggregates all storages into `containerStorages`.
2) Creating storage for other data volumes and push them into `volumeStorages`.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
To enhance the construction and administration of `Katavirtualvolume` storages,
this commit expands the 'sharedFile' structure to manage both
rootfs storages(`containerStorages`) including `Katavirtualvolume` and other data volumes storages(`volumeStorages`).
NOTE: `volumeStorages` is intended for future extensions to support Kubernetes data volumes.
Currently, `KataVirtualVolume` is exclusively employed for container rootfs, hence only `containerStorages` is actively utilized.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
The snapshotter will place `KataVirtualVolume` information
into 'rootfs.options' and commence with the prefix 'io.katacontainers.volume='.
The purpose of this commit is to transform the encapsulated KataVirtualVolume data into device information.
Fixes: #8495
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Feng Wang <feng.wang@databricks.com>
Co-authored-by: Samuel Ortiz <sameo@linux.intel.com>
Co-authored-by: Wedson Almeida Filho <walmeida@microsoft.com>
Update cloud hypervisor implementation to allow hybrid vsock device to
be handled.
Fixes#6692
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
In the case of Cloud Hypervisor running on arm64 architecture,
only arm AMBA UART (pl011) is supported as the TTY. Consequently,
when enabling Hypervisor debug mode, it's essential to configure
the console as "ttyAMA0" rather than "ttyS0
Fixes: #8381
Signed-off-by: briwan01 <brian.wang@arm.com>
- statically linked qemu requires slof.bin to run, hence remove it from blacklist
- By default, initrd is used for Power, modify the configuration.toml accordingly
Fixes: #8458
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
The test utils will be used by the upcoming feature tests: vhost-user-net,
vhost-user-blk and vhost-user-fs.
Signed-off-by: Beiyue <beiyue@linux.alibaba.com>
Signed-off-by: Huang Jianan <jnhuang@linux.alibaba.com>
It mainly focus on the two parts:
(1) redesign the ShareFsConfig with ShareFsMountConfig
The device mount operation must depend on the fact that sharefs
device exists, and re-design the structure of SharesFsConfig and
move the ShareFsMountConfig into it with Option type, which is to
describe the relation between ShareFsConfig and ShareFsMountConfig.
(2) move virtiofs into device manager
Currently, virtio-fs is still outside of the device manager.
To do Enhancement of device manager, it will bring virtio-fs
device in device-manager for unified management
Fixes: #7915
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
As a follow-up PR for #8404, this is to set a default value for an environment variable `ALLOWED_HYPERVISOR_ANNOTATIONS`.
This will prevent a pod launching without an explicit configuration for the variable from getting into a `CrashLoop` state.
Fixes: #8477
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
In order to support PCI VFIO functionality in Dragonball, we should
first add PCI bus and PCI device Interrupt information in Dragonball
mptable setup process.
This patch add :
1. pci_legacy_irqs transfered to setup_mptable function.
2. pci bus support in mptable mem
3. pci interrupt support in mptable mem
fixes: #8449
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
This will be used for selecting the correct runtimes and runtimeclasses
to be deployed with kata-deploy.
Fixes: #8475
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add the corresponding data structure in the runtime part according to
kata-containers/kata-containers/pull/7698.
Fixes: #8472
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This PR fixes the result finding for the general throughput for
the tensorflow benchmark.
Fixes#8466
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
It's CCv0 specific for now, and it's needed as the Operator is now
delegating the runtimeclass creation to the kata-deploy daemonset.
Fixes: #7550
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 2df6cb7609)
In order to support different pod VM instance type via
remote hypervisor implementation (cloud-api-adaptor),
we need to pass machine_type, default_vcpus
and default_memory annotations to cloud-api-adaptor.
The cloud-api-adaptor then uses these annotations to spin
up the appropriate cloud instance.
Reference PR for cloud-api-adaptor
https://github.com/confidential-containers/cloud-api-adaptor/pull/1088Fixes: #7140
Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
(based on commit 004f07f076)
This patch updates the template configuration file for
the remote hypervisor to set static_sandbox_resource_mgmt
to be true. The remote hypervisor uses the peer pod config
to determine the sandbox size, so requires this to be set to
true by default.
Fixes: #6616
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
(based on commit 938447803b)
Add the SELinux setting to ensure it is passed through to the remote
hypervisor
Fixes: #5936
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
(based on commit 3ef2fd1784)
This patch adds the support of the remote hypervisor type.
Shim opens a Unix domain socket specified in the config file,
and sends TTPRC requests to a external process to control
sandbox VMs.
Fixes#4482
Co-authored-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
(based on commit f9278f22c3)
This patch adds a protobuf definiton of the remote hypervisor type.
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
(based on commit 150e8aba6d)
This PR fixes k8's configmap/secrets etc update propagation when filesystem sharing is disabled.
The commit introduces below changes with some limitations:
- creates new timestamped directory in guest
- updates the '..data' symlink
- creates user visible symlinks to newly created secrets.
- Limitation: The older timestamped directory and stale user visible symlinks exist in guest
due to missing DELETE api in agent.
Fixes: #7398
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
As part of the CI migration, this PR is to add workflows for containerd and k8s for s390x.
Fixes: #7930
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Improve the `CODEOWNERS` file by specifying more groups.
Since GitHub automatically checks the `CODEOWNERS` file when a PR is
created and adds all matching groups as reviewers for the PR, this may
help reduce the PR backlog since the right people will be alerted and
requested to review the PR. That should improve the quality of reviews
(and thus the quality of the landed code). It may also have a positive
effect on PR velocity.
> **Note:**
>
> This PR combines the other `CODEOWNERS` files so we have
> a single, visible, top-level file.
See: https://github.com/kata-containers/community/issues/253Fixes: #3804.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This commit enables StratoVirt hypervisor to be tested in kata GHA,
incluing k8s, metrics, cri-containerd, nydus and so on.
Meanwhile, adding some unit tests for StratoVirt to make sure it works.
Fixes: #7794
Signed-off-by: Liu Wenyuan <liuwenyuan9@huawei.com>
Allow kata-deploy process to pull StratoVirt from release binaries, and
add them as a part of kata release.
Fixes: #7794
Signed-off-by: Liu Wenyuan <liuwenyuan9@huawei.com>
Add configuration-stratovirt.toml.in to generate the StratoVirt configuration,
and parser to deliver config to StratoVirt.
Fixes: #7794
Signed-off-by: Liu Wenyuan <liuwenyuan9@huawei.com>
Initial support of the MicroVM machine type of StratoVirt
hypervisor for the kata go runtime.
Fixes: #7794
Signed-off-by: Liu Wenyuan <liuwenyuan9@huawei.com>
This PR adds the iperf udp information to the network README
for the kata metrics CI.
Fixes#8452
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
When multiple containers in a kata pod share one direct volume,
it's important to make sure that the corresponding block device
is only mounted once in the guest. This means that there should
be only one mount entry for the device in the mount information.
Fixes: #8328
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
When a direct volume is used by multiple containers in Kata,
Generating many shared paths with cids will cause IO error
as the result of one direct volume mounts more than once.
To correct it, use the device_id instead of cid which
ensures that the guest only mounts the FS once.
Fixes: #8328
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This was added as part of d4d65bed38, but
install_kata has never actually used the passed enable_debug var.
With this in mind, let's just remove it.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
With this change, we give the users the change to try kata-containers
with their own pre-built tarball.
This will become very useful in the CI context, as we won't be
downloading a specific version of kata-containers, but rather installing
whatever was built in previous steps of the CI pipeline.
Fixes: #8438
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The virtio vsock driver has a small window during initialization
where it can silently drop replies to connection requests.
Because no reply is sent, kata waits for 10 seconds and in the
end it generates a connection timeout error in HybridVSockDialer.
Fixes: #8291
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
This patch is to remove vhost-net dependency on virtio-net for
dbs-virtio-devices crate. Then, the feature of vhost-net is able to enable
without enabling virtio-net device, error, etc.
Fixes: #8423
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Introduce the `update_device` trait in Hypervisor to enable
device updates for VMMs.This trait will initially be utilized
for virtiofs Mount operations.
Fixes: #7915
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Two workflows, run-nerdctl-tests-on-garm.yaml and
run-docker-tests-on-garm.yaml, are removed from commit b481d39. However,
they are referenced by CI workflow. It leads to the CI not working
properly. This patch is to remove those files from ci.yaml.
Fixes: #8433
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
There's no need to keep those as separate files, and by having those in
the basic-ci-amd64.yaml file actually helps us to avoid the
undocummented GHA limitation about the number of files imported.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Peng Tao made this move as part of 1280f85343, and here we're
simply adjusting to the move.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
PR #8311 inadvertently broke the runtime-rs / Cloud Hypervisor TDX
handling. It also introduced unrecoverable failure scenarios. Hence,
replace slow, fallible regex matching in logging fast path with single pass
non-failing multi-string log level matching.
Also, added a unit test for `parse_ch_log_level()`.
Fixes: #8418.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
- Remove two panic statements from InsertNetworkDevice test.
- Rename `NUM_QUEUES` to `DEFAULT_NUM_QUEUES`, `QUEUE_SIZE` to
`DEFAULT_QUEUE_SIZE` for vhost-net and virtio-net.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
`test_networkconfig_to_netconfig` from clh depends on `NetworkConfig` which
has some new fields in this PR. Therefore, this commit gives the test
missing fields.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
- Dragonball's vhost-net feature not depends on virtio-net feature.
- Remove `TapError` from dbs-virtio-devices's Error, and add `VirtioNet`
and `VhostNet` two fields.
- Downgrade visiblity of two fields of `VhostNetDeviceMgr` from
`pub(crate)`.
- File an issue to record a todo for network rate limiter.
- Print internal errors with `{0:?}.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
As we've done some changes in the VMM vcpu allocation, let's introduce
basic tests to make sure that we're getting the expected behaviour.
The test consists in checking 3 scenarios:
* default_vcpus = 0 | no limits set
* this should allocate 1 vcpu
* default_vcpus = 0.75 | limits set to 0.25
* this should allocate 1 vcpu
* default_vcpus = 0.75 | limits set to 1.2
* this should allocate 2 vcpus
The tests are very basic, but they do ensure we're rounding things up to
what the new logic is supposed to do.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
With the change done in the last commit, instead of calculating milli
cpus, we're actually converting the CPUs to a fraction number, a float.
Let's update the function name (and associated vars) to represent that
change.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
First of all, this is a controversial piece, and I know that.
In this commit we're trying to make a less greedy approach regards the
amount of vCPUs we allocate for the VMM, which will be advantageous
mainly when using the `static_sandbox_resource_mgmt` feature, which is
used by the confidential guests.
The current approach we have basically does:
* Gets the amount of vCPUs set in the config (an integer)
* Gets the amount of vCPUs set as limit (an integer)
* Sum those up
* Starts / Updates the VMM to use that total amount of vCPUs
The fact we're dealing with integers is logical, as we cannot request
500m vCPUs to the VMMs. However, it leads us to, in several cases, be
wasting one vCPU.
Let's take the example that we know the VMM requires 500m vCPUs to be
running, and the workload sets 250m vCPUs as a resource limit.
In that case, we'd do:
* Gets the amount of vCPUs set in the config: 1
* Gets the amount of vCPUs set as limit: ceil(0.25)
* 1 + ceil(0.25) = 1 + 1 = 2 vCPUs
* Starts / Updates the VMM to use 2 vCPUs
With the logic changed here, what we're doing is considering everything
as float till just before we start / update the VMM. So, the flow
describe above would be:
* Gets the amount of vCPUs set in the config: 0.5
* Gets the amount of vCPUs set as limit: 0.25
* ceil(0.5 + 0.25) = 1 vCPUs
* Starts / Updates the VMM to use 1 vCPUs
In the way I've written this patch we introduce zero regressions, as
the default values set are still the same, and those will only be
changed for the TEE use cases (although I can see firecracker, or any
other user of `static_sandbox_resource_mgmt=true` taking advantage of
this).
There's, though, an implicit assumption in this patch that we'd need to
make explicit, and that's that the default_vcpus / default_memory is the
amount of vcpus / memory required by the VMM, and absolutely nothing
else. Also, the amount set there should be reflected in the
podOverhead for the specific runtime class.
One other possible approach, which I am not that much in favour of
taking as I think it's **less clear**, is that we could actually get the
podOverhead amount, subtract it from the default_vcpus (treating the
result as a float), then sum up what the user set as limit (as a float),
and finally ceil the result. It could work, but IMHO this is **less
clear**, and **less explicit** on what we're actually doing, and how the
default_vcpus / default_memory should be used.
Fixes: #6909
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
We don't have to do this since we're relying on the
`static_sandbox_resource_mgmt` feature, which gives us the correct
amount of memory and CPUs to be allocated.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add a command-line option to list the installed and available versions
of Kata and containerd.
Fixes: #8355.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Rather than creating a container called `test-kata`, prefix with the
script name to make it a bit "more unique" and less likely for users to
have an existing container with the test container name. The new test
container name is `kata-manager-sh-test-kata`.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Update `github_get_latest_release()` to use `sort -V` rather than
sub-sorting on the major, minor and patch level version number elements.
The new approach is safer and more accurate.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add an extra sanity check to ensure that only a single download URL is
found for the specified release version.
Fixes: #8364.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Improve some of the information messages a little by adding
more detail and quoting file names.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
As "/etc/containerd/config.toml" is used from more than one place, let's
just make it a global var.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add support for also installing the Docker CLI, giving users the chance
to try Kata Containers with docker in the same way we provide users the
chance to try Kata Containers with `ctr`.
Fixes: #8357
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The current kata-deploy code has been doing a `sed` to add allowed
hypervisor annotations, so CBL mariner can be tested with their own
kernel and initrd.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Currently the only way one can specify allowed hypervisor annotations is
during build time, which is a big issue for users grabbing kata-deploy
as we provide.
Fixes: #8403
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Right now we're only testing with `ctr` and there's no change in
behaviour with this commit. However, allowing to pass a tool to run the
tests with gives us an easier time when expanding kata-manager to
support, for instance, docker and nerdctl.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
There's no reason to escape the first + on the +k3s[0-9]\+ regex, as
shown here:
```sh
ubuntu@k3s:~$ /usr/local/bin/k3s kubectl version --short 2>/dev/null | \
grep "Client Version" | \
sed \
-e 's/Client Version: //' \
-e 's/+k3s[0-9]\+//'
v1.27.7
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It seems that with the new k3s release, they've bumped their kubectl
version from x.y.z+k3s1 to x.y.z+k3s2.
Let's ensure our regexp is more generic and future proof for such
changes.
Fixes: #8410
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
TestCheckHostIsVMContainerCapable removes sysModuleDir to simulate a
case that the kernel modules are not loaded. However,
checkKernelModules() executes modprobe <module> if a module not
found in that directory. Loading those modules is required to be denied
temporarily.
Fixes: #8390
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
- Disable device cgroup restriction while pod cgroup is not available.
- Remove balcklist-related names and change whitelist-related names to
allowed_all.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
`TestDeviceCgroup` is added to cri-containerd's integration tests. The test
launches two containers. Each container has a block device. It checks the
validity of device cgroup.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
FSManager of systemd cgroup manager is responsible for setting up cgroup
path. The container launching will be failed if the FSManager is in
read-only mode.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
The changes include:
- Change to debug logging level for resources after processed.
- Remove a todo for pod cgroup cleanup.
- Add an anyhow context to `get_paths_and_mounts()`.
- Remove code which denys access to VMROOTFS since it won't take effect. If
blackmode is in use, the VMROOTFS will be denyed as default. Otherwise,
device cgroups won't be updated in whitelist mode.
- Add a unit test for `default_allowed_devices()`.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
The runk is a standard OCI runtime that isnt' aware of concept of sandbox.
Therefore, the `devcg_info` argument of `LinuxContainer::new()` is
unneccessary to be provided.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
The target is to guarantee that containers couldn't escape to access extra
devices, like vm rootfs, etc.
Assume that there is a cgroup, such as `/A/B`. The `B` is container cgroup,
and the `A` is what we called pod cgroup. No matter what permissions are
set for the container (`B`), the `A`'s permission is always `a *:* rwm`. It
leads that containers could acquire permission to access to other devices
in VM that not belongs to themselves.
In order to set devices cgroup properly, the order of setting cgroups is
that the pod cgroup comes first and the container cgroup comes after.
The `Sandbox` has a new field, `devcg_info`, to save cgroup states. To
avoid setting container cgroup too early, an initialization should be done
carefully. `inited`, one of the states, is a boolean to indicate if the pod
cgroup is initialized. If no, the pod cgroup should be created firstly, and
set default permissions. After that, the pause container cgroup is created
and inherits the permissions from the pod cgroup.
If whitelist mode which allows containers to access all devices in VM is
enabled, then device resources from OCI spec are ignored.
This feature not supports systemd cgroup and cgroup v2, since:
- Systemd cgroup implemented on Agent hasn't supported devices subsystem so
far, see: https://github.com/kata-containers/kata-containers/issues/7506.
- Cgroup v2's device controller depends on eBPF programs, which is out of
scope of cgroup.
Fixes: #7507
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Add test to verify kata supports ipvlan networks.
This test can be bit tricky as it requires knowledge about host interfaces
to be used as a master for the ipvlan network.
However, with github actions, we can assume interface called eth0 to be
present on the host and functioning.
Fixes: #8366
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Since moving from network coldplug to hotplug, the only case verified
was veth endpoints. Support for network hotplug for ipvlan and macvlan was
broken/not added. Fix it.
Fixes: #8391
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Remove the redundant `VmConfigError::EmptyVsockSocketPath` error from
the Cloud Hypervisor config crate since this scenario is already handled
by the `VsockConfigError::NoVsockSocketPath` error.
Fixes: #8385.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Remove the redundant `parse_mac()` function: this was never used and we
already have an implementation in `crates/resource/src/network/utils/mod.rs`.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Virtio-net and vhost-net share a common virtio config, and vhost-user-net
uses another config, named `VhostUserConfig`. Thus, the virtio config could
be added into `NetworkConfig` instead of `Backend`.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Moving Dragonball structs convertions out of device drivers to keep driver
neutral. The convertions include `NetworkBackend` to
`DragonballNetworkBackend` and `NetworkConfig` to
`DragonballNetworkConfig`.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Changes include:
- Merge `VhostNetDeviceError` import item.
- Replace if with match in `add_vhost_net_device()`
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Network backends determine the virtio dataplane implementations. Common
protocols include virtio-net, vhost-net and vhost-user-net, etc. Network
config has a new field named `backend` to specify which protocol to use.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
PLEASE NOTE THAT this pull request just implements vhost-net support for
Dragonball, and adaptation for the Runtime-rs. And this pull request
DOESN'T provide an item to config which backend to use. To sum up,
virtio-net as a default backend is only choice for the user so far.
This pull request introduces vhost-net device for the Dragonball. In
addition, this pull request includes changes of Runtime-rs to improve
network configuration abilities.
The Dragonball part implements a vhost-net device and a vhost-net device
manager, named `VhostNetDeviceMgr`, to manage vhost-net device.
`NetworkInterfaceConfig` is introduced as a high-level abstract for network
config. Then, the Dragonball is able to distinguish network backends, e.g.
virtio-net, vhost-net, vhost-user-net(WIP), etc.
The Runtime-rs part adds support of multiple network backends as well.
`NetworkConfig` has a couple of new fields, like `backend`,
`use_shared_irq`, etc. And Dragonball's network config structs are
implmented `From` trait which allow to be converted from the Runtime-rs's
network config conveniently.
Fixes: #7674
Signed-off-by: Eric Ren <renzhen@linux.alibaba.com>
Signed-off-by: Zizheng Bian <zizheng.bian@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Now that most of the test repository got migrated to the main Kata repository,
it is no longer needed to tag the test repository when doing a release.
Update the documentation accordingly by dropping all references to the test
repository and only mention *the* Kata repository.
Fixes#8302
Signed-off-by: Greg Kurz <groug@kaod.org>
The hub tool is deprecated. Releases are now based on the official gh
CLI. A notable improvement : when properly setup (see [1]), gh allows
to directly use HTTPS with one's GitHub credentials, instead of having
to setup proper SSH access for pushes to the repo.
Adjust the documentation accordingly.
Fixes#8302
[1] https://docs.github.com/en/github-cli/github-cli/quickstart#prerequisites
Signed-off-by: Greg Kurz <groug@kaod.org>
TestCheckHostIsVMContainerCapable is failing on AMD machines.
kata-check_amd64_test.go:96 has no AMD modules, also getCPUType is
missing.
Fixes#8384.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
This update includes necessary changes due to the version bump of
containerd and its dependencies. It's part of a broader initiative to
phase out gogo protobuf, which has been deprecated, and to align with
the current supported libraries.
Fixes#7420.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
The +fieldpath option, specific to gogoprotobuf, enabled dynamic field
access in protobuf messages, allowing nested fields to be accessed via
string paths.
This change is part of a larger effort to transition to the official Go
protobuf library for better maintainability and community support.
Upon review, no instances of dynamic field access were found in the
codebase, confirming that the feature is not in use.
By removing this unused feature, we simplify the build process and make
it easier to complete the transition away from gogoprotobuf.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
Those mappings are not used by our .proto files and there is no
difference between .pb.go files generated.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
This PR makes the change to using the SIGKILL signal instead
of SIGTERM to force stop each kata component before start
running any metric test.
Fixes: #8336
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Remove earlier functionality that tries to assign PCI path to vfio
devices from the host assuming pci slots to start from 1.
Get this from the hypervisor instead.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
If PCI path for block device is not empty for a block device, use
that as identifier for agent instead of virt path which is valid only
for mmio devices.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Block(virtio-blk) and vfio devices are currently not handled correctly
by the agent as the agent is not provided with correct PCI paths for
these devices.
The PCI paths for these devices can be inferred from the PCI information
provided by the hypervisor when the device is added.
Hence changing the add_device trait function to return a device copy
with PCI info potentially provided by the hypervisor. This can then be
provided to the agent to correctly detect devices within the VM.
This commit includes implementation for PCI info update for
cloud-hupervisor for virtio-blk devices with stubs provided for other
hypervisors.
Removing Vsock from the DeviceType enum as Vsock currently does not
implement the Device Trait, it has no attach and detach trait functions
among others. Part of the reason is because these functions require Vsock
to implement Clone trait as these functions need cloned copies to be
passed down the hypervisor.
The change introduced for returning a device copy from the add_device
hypervisor trait explicitly requires a device to implement
Copy trait. Hence removing Vsock from the DeviceType enum for now, as
its implementation is incomplete and not currently used.
Note, one of the blockers for adding the Clone trait to Vsock is that it
currently includes a file handle which cannot be cloned. For Clone and
Device Traits to be implemented for Vsock, it requires an implementation
change in the future for it to be cloneable.
Fixes: #8283
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This patch re-generates the client code for Cloud Hypervisor v35.0.
Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by openapi-generator.
Fixes: #8378
Signed-off-by: Bo Chen <chen.bo@intel.com>
kata-manager is a very nice tool, but we shouldn't be trying to take
care of "everything" in "all possible scenarios", and we should focus on
installing Kata Containers dependencies that are supported.
With this in mind, let's limit a little bit the scope of which versions
of containerd can be installed, limitting to "active" and "lts", which
will then install the latest version of those "flavours". The default
value will always be "lts" as that's supposed to be the stable one.
NOTE: This is a breaking change, as it changes the behaviour of what the
script takes in its `-c` parameter. I'm assuming here we're safe to do
so as the majority of the users should / would only be using the full
installation by default.
Fixes: #8356
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will help us to avoid waiting till the next time cron would trigger
the action to test
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is causing the following error:
```
Unable to resolve action action/stale, repository not found
```
Fixes: #8347
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As that's an operation that can easily fail, and it's quite simple /
cheap for us to run it, let's just do it and avoid the failure.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Our goal. as discussed in the Architecture Committee meeting held on
October 31st, 2023, is to take a more aggressive action on issues and
PRs that have been opened for a long time.
This commit is the very first step, and it's **only** targetting
**PRs**. What this action will do is:
* Mark all the PRs that have no activity for more than 180 days,
starting from May 1st, 2023, as stale.
* A message will be added, letting the contributor know that they can
simply comment on the PR in order to make it "not stale".
* If there's no activity on the PR for 7 days, the PR will be
automatically closed.
Fixes: #8347
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Fix: Kata-Manager fails to retrieve the correct Containerd string name
for architectures other than amd64.
Update the 'github_get_release_file_url()' function to make it compatible
with different architecture expressions. eg. aarch64/arm64, or x86_64/amd64,
allowing it to acquire the correct URL addresses
Fixes: #7071
Signed-off-by: briwan.wang <briwan.wang@arm.com>
The Cargo.lock for agent needs to be updated to include
"safe-path" dependency.
Fixes: #8350
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
make check is giving us the following error:
error: this expression creates a reference which is immediately
dereferenced by the compiler.
Fixes#8344
Signed-off-by: Beraldo Leal <bleal@redhat.com>
The hub tool is deprecated. Convert this script to use the
official GitHub CLI gh instead of hub.
A typical gh setup is able to access repos using HTTPS along with
GitHub credentials. It is only needed to patch the remote url when
using SSH.
Signed-off-by: Greg Kurz <groug@kaod.org>
The hub tool is deprecated. Convert this script to use the
official GitHub CLI gh instead of hub.
A couple of adjustments had to be made :
- the notes.md temporary file is moved to ${tmp_dir} in order to silent gh,
otherwise it complains about an untracked file,
- title of a PR no longer goes to the notes.md file since gh requires the
title to be passed with a dedicated --title option.
Fixes#8303
Signed-off-by: Greg Kurz <groug@kaod.org>
We don't want to mess with the official repo when testing a change
in the release scripts. Adapt `update-repository-version.sh` to
be able to use an alternate repo just like `tag_repos.sh` already
does.
This means that the following command :
$ OWNER="$SOME_ORG" ./update-repository-version.sh -p "$NEW_VERSION" "$BRANCH"
will only create a PR in this repo :
http://github.com/$SOME_ORG/kata-containers.git
Signed-off-by: Greg Kurz <groug@kaod.org>
By modifying RuntimeLevelFilter drain to improve logging control,
enabling isolation of change effect of the loggers between components,
tuning clh logs to be logged according to their log levels
given by cloud-hypervisor.
Fixes: #8310
Signed-off-by: Ruoqing He <linuxwatcher@outlook.com>
This PR removes double quotes in jq output to return raw strings
as input of checkmetrics tool.
Fixes: #8331
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR increases the number of attempts to stop kata components
when it is required usually before starting a metrics test.
Fixes: #8307
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR enables the new FIO test based on the containerd client
which is used to track the I/O metrics in the kata-ci environment.
Additionally this PR fixes the parsing of results.
Fixes: #8199
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR updates the values of the fio parameters for iodepth
requests and for the number of jobs, in order to increase the
number of sequential operations.
Additionally, it adds the list of packages needed to parse the
results.
Fixes: #8198
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR adds the tracing capability for dragonball and it depends on the tracing::Subscriber of the upper layer.
Fixes: #7249
Signed-off-by: Songqian Li <mail@lisongqian.cn>
We used the approach of cold-plugging network interface for pre-shimv2
support for docker.Since the hotplug approach was not required,
we never really got to implementing hotplug support for certain network
endpoints, ipvlan and macvlan being among them.
Since moving to shimv2 interface as the default for
runtime, we switched to hotplugging the network interface for supporting
docker and nerdctl. This was done for veth endpoints only.
Implement the hot-attach apis for ipvlan and macvlan as well to support
ipvlan and macvlan networks with docker and nerdctl.
Fixes: #8333
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Legacy device manager adds device metrics to METRICS when a device is created and removes metrics when a device is dropped.
Fixes: #7248
Signed-off-by: Songqian Li <mail@lisongqian.cn>
Contained release files include the version number without a "v" prefix.
However, the tag for the equivalent release does include it so handle
this distinction and also tighten up the Kata check by specifying an
explicit version number in the regex.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
In bare-metal machines the git tree might get on unstable state with the
previous rebase left halfway. So let's attempt to abort any rebase before.
Fixes#8318
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The archive names for x86_64 [Kata releases](https://github.com/kata-containers/kata-containers/releases)
used to include the tag `x86_64`, but that has now been changed to
`amd64`, which unfortunately broke `kata-manager.sh`:
```
kata-static-3.1.3-x86_64.tar.xz
~~~~~~
expected
kata-static-3.2.0-alpha3-x86_64.tar.xz
~~~~~~
expected
kata-static-3.2.0-alpha4-amd64.tar.xz
~~~~~
changed
```
Fixes: #8321.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Improve the code by fixing some lint issues:
- defining variables before using them.
- Using `grep -E` rather than `egrep`.
- Quoting variables.
- Adding a check for invalid CLI arguments.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
In the migration from the tests repo to the kata containers repo we
missed two huspell dictionaries for static checks; add them.
Fixes#8315
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Add the hypervisor security details to the output of the `kata-runtime
env` and `kata-ctl env` commands so the user can see, amongst other
things, the value of `confidential_guest`.
Fixes: #8313.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The config template file for clh is in the new format for runtime-rs.
It is a result of merging the new format file and options supportted by
cloud-hypervisor.
Some config options from the golang runtime are missing as they may not
be currently supported by the rust runtime. An example of this is the
selinux options, rate limiting options as these are not currently
supported or verified with the rust runtime.
Fixes: #8249
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Balloon device manager adds balloon device metrics to METRICS when a device is created and remove metrics when a device is dropped.
Fixes: #7248
Signed-off-by: Songqian Li <mail@lisongqian.cn>
This is to skip a flaky test `create_tmpfs()` on s390x until a root cause is identified and fixed.
Fixes: #4248
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
hub is now deprecated, which has been causing issues with our release
process.
Let's move to the GH cli (https://cli.github.com/manual), and unblock
this release.
**NOTE**: This commit is purposefully not touching anywhere else hub is
used, as that would require more time and investigation to do the
switch, and right now we just want to unblock the release.
Fixes: #8286
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
kata-manager currently adds default config which currently is qemu.
Add config for clh as well to containerd configuration.
This should allow new users to get started with clh using kata-manager.
Also add config related to enabling privileged_without_host_devices.
Always good to have this config enabled when users try to run privileged
containers so that devices from host are not inadverdantly passed to the
guest.
Fixes: #8280
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This is needed in order to properly run the CIs in branches that are not
the main one, as the kata-deploy.yaml file on those branches do not have
the `latest` tag, but rather the latest stable release.
Fixes: #8274
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
- kata-deploy-stable: Switch to using the ubuntu based payload
- libs: protection: Fix typo in TDX output
- ci: k8s: Fix bogus firecracker check in k8s-credentials-secrets.bat
- tests: Enable agent stability test
- docs: Fix paths to build kernel in SNP VMs documentation
- runtime-rs: ch: Add TDX CH features check
- runtime: Validate hypervisor section name in config file
- tests: query data from the OPA service
- release: tag_repos: Stop tagging the `tests` repo
- metrics: fixes common.sh function to always return true
- Memory footprint test removing trailing commas to make json results file valid
- policy: allow access to ReseedRandomDev
- runtime/kata-ctl: update dependencies
- runtime-rs : fix Nydus support for runtime-rs + Dragonball
- metrics: removal of reference in the documentation to the fio dax subtest.
- runtime-rs: ch: Detect Intel TDX version
- runitme-rs: use the same base64 as kata-runtime/direct-volume does
- tests: Enable scability test for stability CI
- runtime-rs: Add support for adding vfio device for cloud-hypervisor
- tests: Enable soak parallel stability test
- dragonball: vcpu metrics change to be recorded per vcpu
- ci: k8s: adapt gha-run.sh to run locally
- metrics: removes kata components and k8s deployment when test finishes
- GHA: fix up referenced yaml exceeding 20 limit problem
- gha: ci: Revert tracing test PR to unbreak CI
- runtime-rs: ch: Enable feature
- gha: ci: Port runk tests over
- ci: gha: Port tracing tests over
- Enable fio test using containerd client
- gha: Add stability tests workflow for gha
- gha: arm64: Ensure the builder is arm64-builder
- kata-deploy: Build kata-agent as we build all the other components
- versions: migrate out of k8s.gcr.io
- doc: Update crictl pod-config
- gha: Fix k0s deployment
- tests: Add stability test for kata CI
- docs: Update url in kata vra document
- gpu: Adding CDI support for cold and hot-plug of VFIO devices
- kata-deploy: build & ship the rust components from src/tools/
- metrics: Add latency value limits for kata CI
- runtime: fix reading cgroup stats of sandboxes
- Upgrade to Cloud Hypervisor v35.0
- ci: Port kata-monitor tests from Jenkins to GHA
- metrics: Fix latency yamls path
- metrics: Fix metrics README
- metrics: Fix C-Ray documentation
- runtime-rs: ch: Enable Intel TDX
- ci: k8s: crio: Follow up patches to have CRI-O also working as part of our CI
- metrics: Enable latency test in gha run script
- local-build: Fix .docker ownership before build-payload
- runtime-rs: Add network support for cloud-hypervisor
- osbuild: Reduce guest components binary size with strip
- gha: Add pandoc as a dependency for static checks
- ci: rootfs-image build-asset is failing
- feat(runtime-rs): introduce huge page mode to select VM RAM's backend
- clh: Direct IO support for block devices
- gha: Install hunspell for static checks
- ci: Trigger payload-after-push on workflow_dispatch
- ci: Actually enable the CRI-O tests
- protocol: remove gogoprotobuff tests
- ci: k8s: Also run tests with CRI-O
- runtime: support kernel params including spaces
- ci: kata-deploy: Fix runner name
- metrics: Enable parallel bandwidth iperf limit
- ci: kata-deploy: Enable all k8s flavours that we support
- ci: Create clusters in individual resource groups
- versions: Bump virtiofsd to v1.8.0
- clh: arm: Use static_sandbox_resource_mgmt=true
- Bump nydus versions and update nydus tests
- runtime/qemu: Rework QMP/HMP support
- clh:arm64: use arm AMBA UART for hypervisor debug
- ci: Use variable size of VMs depending on the tests running
- ci: Rework static checks
- runtime: incorrect handling of non-empty []Endpoint parameter in Remo…
- ci: cache: Check the sha256sum of the components & fix ovmf-sev cache usage
- ci: cache: Use the artefacts stored in ghcr.io/kata-containers/cached-artefacts/${component}
- ci: Run some of the GARM tests in smaller instances
- ci: Reduce the size of the AKS VMs
- ci: cache: Allow pushing our artefacts to an OCI registry
- metrics: Add iperf value for cpu utilization
- ci: cache: Export env vars needed to use ORAS
- gha: vfio: Import test script
- tests: fix kernel and initrd annotations
- metrics: Add iperf bandwidth value for kata metrics
- metrics: Add Cassandra Metrics documentation
- metrics: Remove warning from metrics documentation
- ci: docker: nerdctl: Switch to tcp port 80 ping
- runtime: Naming conflict of network devices
- Remove gogoproto.nullable extension
- metrics: Ensure docker is running in init_env
- metrics: this PR skips the FIO test temprarily to fix issues
- ci: Add a very basic nerdctl sanity test
- runtime-rs: hypervisor: Remove debug kernel options
- versions: Bump rust version
- ci: Add a very basic docker sanity test
- dragonball: fix for non-deterministic builds
- runtime-rs: bring hybrid vsock devices in manager.
- ci: use github.ref_name instead of $GITHUB_REF_NAME
- ci: Add more target-branch related fixes
- ci: Fix target-branch usage
- agent: optimize the code of systemd cgroup manager
- gha: Manually rebase PR atop of the target branch before testing
- Update kernel to the latest LTS release (v6.1.52) and bring in erofs patches needed for the CC work
- kata-deploy: Fix aarch64 image build
- runtime: Fix more virtiofs args
- kata-deploy: Switch to an alpine image
- metrics: Use TensorFlow optimized image
- metrics: fix FIO test initialization
- ci: k8s: Add clean-up-garm argument for gha-run.sh
- ci: k8s: Second round of fix-ups with the devmapper CI
- metrics: re-enable memory-usage initialization step
- Dragonball: optimize the placement of dbs-upcall features
- ci: k8s: Fix typo in run-k8s-tests-on-garm.yaml
- ci: k8s: Add k8s devmapper tests (part 0)
- kata-deploy: Create kata-static.tar with correct ownership
- runtime: run prestart hooks before starting VM for FC
- metrics: Add write 95 percentile FIO value
- runtime: Allow virtio_fs_extra_args annotation
- packaging: do not install docker-compose-plugin for s390x|ppc64le
- runtime-rs: Fix volumes and rootfs cleanup issues
- metrics: Enable iperf benchmark on gha for kata metrics
- CI: switch static-checks-dragonball CI machines to Azure
- metrics: Add README for kata metrics report
- osbuilder: Remove chcon operation for guest SELinux
- kata-sys-util: protection: Update TDX checks
- Improve the way to clean up storage devices for sandbox
- agent: avoid possible leakage of storage device
- tests: add policy to existing tests
- gha: Rebase PR atop of the target branch before testing
- versions: Update alpine to its 3.18 version
- runtime: Fix data race in ioCopy
- metrics: Add grabdata script for metrics report
- Fixes tests on AMD machines
- metrics: Enable FIO limits for kata metrics
- metrics: Add metrics report script
- metrics: Fix memory inside limits for kata metrics
- metrics: fix parsing issue on memory-usage test
- dragonball: vsock add fifo/pipe stream support for passed fd hybridSt…
- tests: Add confidential test
- tdx: Update the components needed for using the 6.2 kernel stack
- tests: delete k8s deployment at the test's end
- tests: use unique test name
- runtime-rs: check peer close in log_forwarder
- gha: Avoid "fail-fast" in tests that are known to be flaky
- Refine storage device management for kata-agent
- metrics: Remove unused variable in tensorflow nhwc script
- kata-deploy: Don't try to remove /opt/kata
- metrics: Add TensorFlow ResNet50 FP32 benchmark
- gha: vfio: Run on Ubuntu 23.04 runner
- kata-agent: use default filemode for block device when it is set to 0
- kata-types: introduce KataVirtualVolume to support nydus, direct volume and image pull
- libs,tests: fix typo disable_guest_seccomp in configuration-anno-1.toml
- local-build: Remove GID before creating group
- kata-deploy: Avoid failing on content removal
- runtime: fix image and initrd assets handling
- metrics: Add disk link to README
- metrics: Fix FIO path
- gha: capture additional kata-deploy output
- metrics: Use function from metrics common in pytorch script
- metrics: Enable kata runtime in K8s for FIO test.
- metrics: Fix README for pytorch
- metrics: Remove unused variable in tensorflow mobilenet script
- rootfs: agent: Policy support with AGENT_INIT=yes
- gha: k8s: kata-deploy: Move kata-deploy specific tests from integration/kubernetes to functional/kata-deploy
- metrics: Fix check results for tensorflow benchmark
- metrics: Add Tensorflow ResNet50 int8 benchmark
- kata-deploy: Properly create default runtime class
- agent: simplify error handling
- metrics: Fix MobileNet help me description
- gha: ci: Start running kata-deploy tests
- runk: Modify kill command's error message for containerd tests
- runtime-rs: add driver option
- gha: cri-containerd: Enable tests
- metrics: Rename tensorflow scripts
- gha: tests: Add kata-deploy functional tests -- Part 1
- agent: runtime: add Agent Policy feature
- runk: Support without pid ns
- metrics: Add Cassandra Kubernetes benchmark for kata metrics
- metrics: Add common functions to the common script
- metrics: fix the loop used to stop kata components
- docs: Remove installation step in virtcontainers doc
- Propogate secrets, config maps etc into guest if sharedFS not available
- kata-deploy: Preliminary k0s support
- gha: static-checks: Move to the Azure instances
- versions: Update firecracker version to 1.4.0
- agent: Allow clippy::redundant_clone in the unit tests
- agent: avoid creating new `Vec` instances when easily avoidable
- metrics: compute tensorflow statistics
- metrics: Add network nginx benchmark
- metrics: install kata once and run multiple checks
- ci: unencrypted-image: Fix build context
- ci: create-confidential-image: Add dependent actions
- Follow up fixes for https://github.com/kata-containers/kata-containers/pull/7596
- tests: Create image that will be used in the unencrypted confidential tests
- kata-deploy: Ensure we cover SHIMS / DEFAULT_SHIM as part of our tests
- tests: upgrade bats version
- Fix mimor bugs and improve coding stype of agent rpc/sandbox/mount
- deps: Bump dependent crate versions
- fix number of queues handling in dragonball share fs device
- runtime-rs: Introduce directly attachable network
- metrics: General improvements to mobilenet tensorflow test
- gha: Add iperf network metrics
- docs: Use control-plane term instead of master
- agent: avoid unnecessary calls to `Arc::clone`
- metrics: Add network latency test
- Image pulling on the host
- Use version 0.10.4 of `fuse-backend-rs`
- kata-deploy: Use host's systemctl
- release: Revert kata-deploy changes after 3.2.0-rc0 release
- metrics: stop kata components before start a metric test.
- runtime-rs: Add block device handling for cloud hypervisor
a93fdb014 kata-deploy-stable: Adapt to what we're using in the stable branch
36109da93 ci: k8s: Fix bogus firecracker check in k8s-credentials-secrets.bat
d01daf749 tests: Adjust timeout for agent stability test
9b14dda14 libs: protection: Fix typo in TDX output
0e0867f15 runtime-rs: ch: Add TDX CH features check
409eadddb runtime-rs: ch: Improve readability of guest protection checks
82a0814fc tests: Enable agent stability test
32be8e3a8 tests: query data from the OPA service
b81c0a669 tests: encode policy file during test
4f9681b41 metrics: fixes common.sh function to always return true
2ef2b2a6d docs: Fix paths to build kernel in SNP VMs documentation
408b59c02 runtime-rs: fix bugs to support Nydus v5
157caea9f Revert "nydus: Temporarily skip tests on dragonball"
678fe3cd3 Dragonball: fix Nydus config serde problem
b6ec62138 policy: allow access to ReseedRandomDev
908519db9 metrics: skips docker restart when it is not installed or is masked.
c2763120a metrics: removing trailing comma characters from json file.
3e8cf6959 runtime: Validate hypervisor section name in config file
ef6388e81 tests: Remove unused function from scability test
fbc8f8f46 scripts: Use install_yq from the `kata-containers` repo
65b1a2d27 release: tag_repos: Stop tagging / updating the `tests` repo
87b760f56 runtime-rs: ch: Detect Intel TDX version
73e81f5e3 runitme-rs: unify base64 encoding for direct-volume
c6463cb5a tests: Fix path for versions yaml for soak parallel test
89c9454fc metrics: removal of reference in the documentation to the dax test.
30ff58904 tests: Enable scability test for stability CI
8d6f7b909 runtime-rs: Add support for handling vfio device for cloud-hypervisor
e786b2b01 gha: Add install dependencies for stability tests
dbfe6512f dragonball: vcpu metrics change to be recorded per vcpu
fa60fbe02 dragonball: METRICS is refactored to RwLock<DragonballMetrics>
500d1c5ce kata-ctl: update rustls-webpki/webpki dependency
d7660d82a runtime: unify gopkg.in/yaml.v3 to v3.0.1
fc9a107e8 runtime: unify swag and testify dependency
79ebb959c runtime: update runc dependency to v1.1.9
7f3e8bd65 runtime: unify golang.org/x/text to v0.7.0
df325ae37 runtime: update golang.org/x/net to v0.7.0
bba34910d metrics: stops kata components and k8s deployment when test finishes
84e3d884e gha: Add general dependencies to stability tests
dec3951ca tests: Add soak parallel stability test
0f04d527d tests: Enable soak parallel test
e669282c2 ci: k8s: set KUBERNETES default value
c30c3ff18 tests: run k8s-volume on a given node
666993da8 tests: run k8s-file-volume on a given node
3a00fc910 tests: exec_host() now gets the node name
61c9c17bf tests: add get_one_kata_node() to tests_common.sh
68f083c4d ci: k8s: set KATA_HYPERVISOR default value
6677a61fe ci: k8s: configurable deploy kata timeout
200e54292 ci: k8s: shellcheck fixes to gha-run.sh
4af78be13 kata-deploy: re-format kata-[deploy|cleanup].yaml
d54e6d9cd ci: k8s: run_tests() for kcli
c2ef1f0fb ci: k8s: add deploy-kata-kcli() to gh-run.sh
d2be8eef1 ci: k8s: add cleanup-kcli() to gha-run.sh
cbb9aa15b ci: k8s: set default image for deploy_kata()
89bef7d03 ci: k8s: create k8s clusters with kcli
954d40cce gha: combine coco jobs into a single yaml
b60e0a9b5 gha: combine basic amd64 jobs into a single yaml
e9bd85211 gha: ci: Revert tracing test PR to unbreak CI
b8a46a4b8 runtime-rs: ch: Enable feature
0f2dc8c67 gha: Add containerd stability tests to ci yaml
da91c9df8 ci: Port runk tests to this repo
7f2377276 ci: Add placeholder for runk tests
9205acc3d ci: Move tracing tests here
85d290a04 gha: Add stability gha run script
54f0c8f88 gha: Add stability tests workflow for gha
3bb2923e5 ci: Add placeholder for tracing tests
2c3bf406d ci: Create a function to install docker
119f03de2 gha: arm64: Ensure the builder is arm64-builder
8c498ef5e metrics: Use jq tool to pretty-print json metrics output
a2159a636 metrics: Enables FIO test for kata containers
70e7ec3e2 gha: Fix k0s deployment
560bbffb5 packaging: tools: Remove `set -x` leftover
18fa483d9 packaging: release: Mention newly added images
ca3b88837 packaging: tools: Fix container image env var name
5ca66795c packaging: Allow passing the TOOLS_CONTAINER_BUILDER
02acef957 gha: Build the kata-agent as part of our workflows
5208386ab packaging: Build the kata-agent
1727487ee agent: Allow specifying DESTDIR and AGENT_POLICY via env vars
45c118883 packaging: Add get_agent_image_name()
0db8fb8f9 versions: migrate out of k8s.gcr.io
a1a054367 doc: Fix spelling
6339605a1 tests: Add general stability fixes
59ae24444 doc: Update crictl pod-config
fd19f4082 tests: Add agent stability test
215577032 tests: Add cassandra stress in stability tests
f2d3ea988 tests: Add stressng dockerfile for stability tests
6493aa309 tests: Add stressor CPU test for stability tests
ef68a3a36 metrics: Add stability test for kata CI
7c934dc7d gpu: Fix cold-plug of VFIO devices
8d66ef518 metrics: Increase qemu jitter value
5600e28b5 metrics: Increase jitter value for clh
a6b1f5e21 ci: Build src/tools components as part of our tests / releases
501a168a8 kata-deploy: Build components from src/tools
6ef42db5e static-build: Add scripts to build content from src/tools
4d08ec29b packaging: Add get_tools_image_name()
98097c96d packaging: Use git abbreviated hash
489caf1ad ci: kata-monitor: Move tests over
a3fb067f1 ci: Add placeholder for kata-monitor tests
57cb4ce20 ci: Make install_kata aware of container engines
de1eeee33 ci: Create a generic install_crio function
64a200085 ci: Add install_cni_plugins helper
8132fe15c ci: Modify containerd default config
8cb7df1be metrics: Add checkmetrics for latency test
e90440ae2 metrics: Add qemu latency value limit
a74a8f8a9 metrics: Add latency value limits for kata CI
d7def8317 metrics: Fix general check static warnings
928553d1b docs: Update url in kata vra document
b0a3293d5 runtime-rs: ch: Enable Intel TDX
523399c32 runtime-rs: ch: Add more consts
dea806581 runtime-rs: ch: Remove unused function
995f2c015 runtime-rs: ch: Only handle particular pending device types
b1b96a5c4 runtime-rs: ch: Remove erroneous "virtio-blk-mmio" check
9ac29b8d3 metrics: Add init_env function to latency test
dfd0c9fa9 runtime: clh: Re-generate the client code
8f9f087e3 versions: Upgrade to Cloud Hypervisor v35.0
81c8babca metrics: Fix latency yamls path
481573682 metrics: Fix C-Ray documentation
ef63d67c4 ci: crio: Trail '\r' from exec_host() output
74c12b292 ci: crio: Enable default capabilities
358dc2f56 kata-deploy: Fix CRI-O detection
ebaa4fa4c ci: crio: Pass `-y` to apt
97e73b223 metrics: Fix spelling warnings
36c8cd6f1 metrics: Fix metrics README
15425a2b8 local-build: Fix .docker ownership before build-payload
13ca7d9f9 gha: Add pandoc as a dependency for static checks
08bc8e4db metrics: Add latency benchmark for gha
6776b55d7 metrics: Enable latency test in gha run script
94e2ccc2d runtime: fix reading cgroup stats of sandboxes
d507d189b fc: Add support for noflush cache option
2ca781518 clh: Direct IO support for block devices
0c95697cc ci: Trigger payload-after-push on workflow_dispatch
28cbc3b51 ci: rootfs-image build-asset is failing Fixes: #802787a861648 gha: Install hunspell for static checks
8c3c50ca8 ci: Actually enable the CRI-O tests
3a6510ad6 osbuild: Reduce guest components binary size with strip
07a6e63a6 ci: k8s: rke2: Use sudo to call systemd
03b82e848 ci: k8s: Add a CRI-O test
d7105cf7a ci: k8s: Add a method to install CRI-O
54c0a471b ci: k8s: k0s: Allow passing parameters to the k0s installer
730ef5169 deps: updating dependencies
3a2c83d69 ci: kata-deploy: Fix runner name
82ff2db46 runtime: support kernel params including spaces
604a9dd67 protocol: remove gogoprotobuff tests
f7fa7f602 ci: Enable kata-deploy tests for all the supported k8s flavours
2c908b598 ci: kata-deploy: Add the ability to deploy rke2
eaf616491 ci: kata-deploy: Add the ability to deploy k0s
001525763 ci: kata-deploy: Add deploy-k8s argument to gha-run.sh
bf2cb0228 ci: kata-deploy: Expland tests to run on k0s / rke2
b12b9e188 ci: kata-deploy: Add placeholder for tests on GARM
9e1fb8a96 ci: kata-deploy: Export KUBERNETES env var
09cc0ed43 ci: Move deploy_k8s() to gha-run-k8s-common.sh
486fe14c9 ci: Properly set K8S_TEST_UNION
d9ef1352a ci: Add first letter of the K8S_TEST_HOST_TYPE to resource group name
68267a399 ci: Create clusters in individual resource groups
9aa8d1c91 metrics: Add parallel bandwidth limit for qemu
44c7c082d versions: Bump virtiofsd to v1.8.0
af59d4bf4 metrics: Enable parallel bandwidth iperf limit
aba36ab18 nydus: Temporarily skip tests on dragonball
b8a8dfcd1 nydus: Use `kata-${KATA_HYPERVISOR}` instead of `kata`
f6df3d6ef static-build: Fix arch error on nydus build
2f9c9e2e6 tests: nydus: Update nydus tests
c9a4e7e46 versions: Bump nydus and nydus-snapshotter to its latest release
b73bde320 gha: nydus: Populate run()
b3904a1a3 gha: nydus: Populate install_dependencies()
d2b3b67f5 gha: nydus: Actually install kata when `install-kata` is called
0ec00ad42 gha: nydus: Get rid of nydus{,-snapshotter} install from nydus_test.sh
568439c77 tests: nydus: Add timeout to the crictl calls
5ac3b76eb tests: nydus: Add uid / namespace to the nydus container / sandbox
376574a16 tests: nydus: Decorate some calls with `sudo`
4290fd4b6 tests: nydus: Adapt "source ..." to GHA
a84efa3e8 tests: nydus: Adapt check to "clh" instead "cloud-hypervisor"
56a14b395 tests: common: Add install_nydus_snapshotter()
b6563783e tests: common: Add install_nydus()
72599f191 clh: arm: Use static_sandbox_resource_mgmt=true
1f16b6627 runtime/qemu: Rework QMP/HMP support
8b1e9b0c7 ci: static-checks: Clean up static-checks job
2c5ca2eaf ci: static-checks: Run tests depending on KVM
509c309ab ci: static-checks: Move "sudo make test" to the new test matrix
4e963cedf ci: static-checks: Move "make test" to the new test matrix
08f2e5ae0 runtime-rs: Ensure static-checks-build is a dep of `make test`
2bc3a616a kata-ctl: Use `loop` instead of `kvm` module in tests
46daddc50 kata-ctl: Ensure GENERATED_CODE is a dep of `make test`
ec826f328 agent: Ensure GENERATED_CODE is a dep of `make test`
1d32410a8 ci: install_libseccomp: Do not depend on the tests repo
bf888b9a5 ci: static-checks: Move "make check" to the new test matrix
473ec8780 kata-ctl: Add `kata-types` to the Cargo.lock file
ea19549a9 kata-ctl: Ensure GENERATED_CODE is a dep of `make check`
e12577586 tests: install_rust: Also install clippy
e2c61a152 ci: static-checks: Move vendor check to its own job
6794d4c84 tests: Move install_rust.sh from the tests repo
e64508c30 tests: install_go: Remove tests repo dependency
11dff731b tests: Move functions from kata_arch script here
75c974c80 ci: static-checks: Move kernel config check to its own job
9c233bb9e test: Add test to verify try_from for clh Netconfig
c69a1e33b ci: Use variable size of VMs depending on the tests running
9049d311d runtime-rs: Add network support for cloud-hypervisor
eecd5bf2a ci: cache: Fix ovmf-sev cache
86c41074b ci: cache: Check the sha256sum of the component
460988c5f ci: cache: Remove the script used to cache artefacts on Jenkins
4533a7a41 ci: cache: Also store the ${component} sha256sum
eccc76df6 ci: cache: Use the cached artefacts from ORAS
7f5e77bcb kernel: enable Arm pl011 support
241c355e0 clh:arm64: use arm AMBA uart for hypervisor debug
094b6b2cf ci: k8s: Temporarily disable tests that require a bigger VM instance
d0c257b3a ci: cache: Push cached artefacts to ghcr.io
108f1b60d kata-deploy: Generate latest_{artefact,image_builder} files
be2eb7b37 ci: cache: Install ORAS in the kata-deploy binaries builder container
fb24fb0dc ci: k8s: devmapper: Use a smaller / cheaper VM instance
1daf02f5d ci: nydus: Use a smaller / cheaper VM instance
e60d81f55 ci: nerdctl: Use a smaller / cheaper VM instance
4db416997 ci: docker: Use a smaller / cheaper VM instance
32841827b ci: cri-containerd: Use a smaller / cheaper VM instance
92fff129f ci: k8s: Don't set cpu limit request for k8s-inotofy test
faf98c062 ci: Reduce the size of the AKS VMs
adc18ecdb ci: cache: For consistency, read all used env vars
c7a851efd ci: cache: Pass the exposed env vars to the kata-deploy binaries in docker
6bd15a85d ci: cache: Export env vars needed to use ORAS
cd4fd1292 metrics: Add iperf cpu utilization limit for qemu
df5cd10ea metrics: Add iperf value for cpu utilization
a96050a7a tests: Apply timeout to 'ctr t kill'
9d9303678 tests/vfio: Bump VM image to Fedora 38
faee59b52 tests/vfio: Accept single device in vfio group for CLH
df3dc1105 tests/vfio: Get rid of sync's
7211c3dcc gha: vfio: Set test timeout to 15m
1b02f89e4 packaging: kernel: Enable VIRTIO_IOMMU on x86_64
3a1db7a86 runtime: clh: Support enabling iommu
9f1a42c6c tests/vfio: Give commands 30s to execute
b46b0ecf8 tests/vfio: Configure a value for 'hot_plug_vfio' for both vmms
bfc93927f runtime: Remove redundant check in checkPCIeConfig
7c4e73b60 runtime: Add test cases for checkPCIeConfig
fc51e4b9e runtime: Check config for supported CLH (cold|hot)_plug_vfio values
509771e6f runtime: clh: Add hot_plug_vfio entry to config
5f6475a28 tests/vfio: Gather debug info and disable tdp_mmu
8fffdc81c tests/vfio: Capture journal from vm
df815087e tests/vfio: Change to get the test working in GHA
a92ddeea1 tests/vfio: Move dependency installation to gha-run.sh
5a551a85b gha: vfio: Import jobs scripts from tests repo
49e2fa189 metrics: Increase jitter value for qemu
49234433a metrics: Increase value limit for jitter in clh
813bfdec0 ci: docker: nerdtl: Use io.containerd.kata-${KATA_HYPERVISOR}.io
46bc0b1c0 ci: nerdctl: Create the containerd config
13968aa7f ci: nerdctl: Switch to tcp port 80 ping
e0c811678 ci: docker: Switch to tcp port 80 ping
1636abbe1 runtime: issue with non-empty []Endpoint in RemoveEndpoints
0aa073967 metrics: Add iperf bandwidth value for qemu
c0ad91476 tests: fix kernel and initrd annotations
615c1cbf1 metrics: Add iperf bandwidth value for kata metrics
d53eb73ee metrics: Ensure docker is running in init_env
ad08321b8 metrics: Add Cassandra Metrics documentation
a58ea6659 metrics: this PR skips the FIO test temprarily to fix issues
f536ef5ce ci: docker: Also run the smoke test with runc
c83f167c5 ci: docker: Run the tests after the kata-static is created
12d833d07 ci: Add a very basic nerdctl sanity test
348b8644d ci: Add a very basic docker sanity test
a75fd5eb8 runk: Fix rust unecessary mut error
a31c14517 kata-ctl: useless-vec warning
c8419fc3b kata-ctl: Resolve non-minimal-cfg warning
3eaf68d95 agent-ctl: Allow clippy lint
1d8b78959 runtime-rs: Fix useless-vec warning
99f3d69e9 runtime-rs: Remove mut
16fbc27b0 dragonball: Allow ambiguous-glob-reexports
bbf191951 dragonball: Resolve non-minimal-cfg warning
75cfdd5d5 agent: config: Allow clippy lint
f3a0fd590 agent: config: Fix useles-vec warning
9e423bd3d libs: Fix clippy unnecesary hashes error
444395050 versions: Bump rust version
a16b0962b chore(cargo): update cargo lock
ca4b6b051 runtime: Naming conflict of network devices
202049f35 feat(runtime-rs): introduce huge page type to select VM RAM's backend
f811b064c ci: use github.ref_name instead of $GITHUB_REF_NAME
6d795c089 ci: Add more target-branch related fixes
8509c3187 ci: Fix target-branch usage
060499dca metrics: Remove warning from metrics documentation
c0f697fcc runtime: Allow kernel_params annotation
b03e49794 dragonball: fix for non-deterministic builds
976d10150 runtime-rs: hypervisor: Remove debug kernel options
fde34610c kernel: Add erofs patches needed for CC related work
dc6a4588a versions: Bump kernel to the latest LTS release (6.1.52)
52f6449b7 kata-manager: Remove initcall_debug kernel option
8b4a0b368 kata-deploy: Remove curl after it's used
139c7f03a kata-deploy: Fix aarch64 image build
470d06541 agent: optimize the code of systemd cgroup manager
bd24afcf7 gha: Manually rebase PR atop of the target branch before testing
72c510d05 runtime/virtiofsd: Drop all references to "--cache=none"
ead724bec protocol: removing gogo.nullable feature
d8e4bb985 protocol: remove unused PROTO_FILE env
5e1106a77 protocol: remove unused import_path
87accaaec protocol: use workdir during build
711a7ed96 protocol: remove mapping definitions
8db84c1bd protocol: force GOPATH to be set
68156d77a protocol: breaking lines to improve readability
670a8e9c7 kata-deploy: Switch to an alpine image
9d74b7ccc k8s: ci: Skip "Pod quota" test with firecracker
f6cd3930c ci: k8s: Remove useless skip statement from tests
3cc20b47a ci: k8s: Also check for "fc" (for firecracker)
b5bad3cb0 ci: k8s: Add clean-up-garm argument for gha-run.sh
aaec5a09f ci: k8s: devmapper tests should be using ubuntu 20.04
27fa7d828 ci: k8s: Add a kata-deploy-garm target
fa62a4c01 ci: k8s: Export KUBERNETES env var
8c9380a79 ci: k8s: Install bats on GARM runners
3de23034f ci: k8s: Wait some time after restarting k3s
adfea55b8 metrics: fix FIO test initialization
2df183fd9 ci: k8s: Append, instead of overwrite, the devmapper config
369a8af8f ci: k8s: Decrease k3s sleep from 4 to 2 minutes
ada65b988 ci: k8s: Use vanilla kubectl with k3s
ad45ab5d3 ci: k8s: Ensure k3s is deploy with --write-kubeconfig-mode=644
028a97e0d ci: k8s: Use the proper command for sleep
3a427795e metrics: Use TensorFlow optimized image
8d99972a8 ci: k8s: Fix typo in run-k8s-tests-on-garm.yaml
deed1b927 Dragonball: optimize the placement of dbs-upcall features
0e8bd50cb ci: k8s: Add k8s devmapper tests (part 0)
b28b54df0 ci: k8s: Add a function to configure devmapper for containerd
54f711721 ci: k8s: Add a function to deploy k3s
81536f21a runtime/qemu: Pass "--xattr" to virtiofsd instead of "-o xattr"
b1dd09a4d runtime: Allow virtio_fs_extra_args annotation
2efda20c7 packaging: do not install docker-compose-plugin for s390x|ppc64le
438fbf966 metrics: Add write 95 percentile for FIO for qemu
024b4d2ff metrics: Add write 95 percentile FIO value
e98e5cdea metrics: Add checkmetrics to gha run script
c1edfe551 metrics: Add checkmetrics value for qemu for iperf
6a79ecedf metrics: Add jitter value for clh
f609a9a75 metrics: Add test selector to iperf metrics
5b8db3042 metrics: Enable iperf benchmark on gha for kata metrics
60f733d30 CI: switch static-checks-dragonball CI machines to Azure
7870b33a2 runtime-rs: bring hybridVsock devices in manager.
18c94ebbe kata-deploy: Create kata-static.tar with correct ownership
57e7bf14a agent: refine StorageDeviceGeneric::cleanup()
53edb1937 agent: implement StorageDeviceGeneric::cleanup()
0c63453e2 types: make StorageDevice::cleanup() return possible error code
3a3d77b3b agent: move StorageDeviceGeneric from kata-types into agent
b151cfd14 metrics: re-enable memory-usage initialization step
f3e1a6a94 osbuilder: alpine: Change mirror
ac612aef5 osbuilder: alpine: Match the version on versions.yaml
9cd706d1c agent: avoid possible leakage of storage device
bf21411e9 tests: add policy to k8s tests
d0e061067 runtime: config: use the SEV initrd for SNP
67fed26f1 runtime: Use TDX image with in the qemu-tdx config
ac939c458 gha: Rebase atop of the target branch
82cd14ba3 versions: Update alpine to its 3.18 version
666882575 metrics: Add grabdata script for metrics report
c290eaed8 kata-sys-util: protection: Update TDX checks
d7a996c68 gha: Update to checkout@v3 action
c2ba29c15 runtime: Fix data race in ioCopy
211de08d9 osbuilder: Remove chcon operation for guest SELinux
9f21fa9b3 metrics: Add report generator link to general documentation
c0ed5ea0a metrics: Add README for kata metrics report
a7b59a5bf metrics: Add limit for 90 percentile for qemu value
99db6568e metrics: Add limit for write 90 percentile value for clh
6e06392c5 metrics: Enable FIO limits for kata metrics
2e4c87472 runtime/vc: runPrestartHooks should ignore GetHypervisorPid failure
21204caf2 runtime: fail early when starting docker container with FC
32fd01371 runtime: run prestart hooks before starting VM for FC
00e7ffd98 tests: check vmx only on Intel machines
c8dd3c073 metrics: Fix memory footprint qemu limit
8877ec62f metrics: Fix memory inside limits for kata metrics
80146f207 tests: Fixes cpuType check on AMD machines
7e364716d metrics: Add test setup details to metrics report
17dc1b976 metrics: Add boot lifecycle times to metrics report
3b0d6538f metrics: Add memory inside container to metrics report
79fbb9d24 metrics: Add scaling system footprint in metrics report
8e6d4e6f3 metrics: Add metrics reportgen
139ffd4f7 metrics: Add report file titles
878d1a2e7 metrics: Generate PNGs alongside the PDF report
fce248797 metrics: Add metrics report R files
08812074d metrics: Add report dockerfile
69781fc02 metrics: Add metrics report script
e286e842c tests: Expand confidential test to support TDX
e31f099be tests: Expand confidential test to support SNP
c3b9d4945 tests: Add confidential test for SEV
538c965c2 metrics: fix parsing issue on memory-usage test
3818bf331 local-build: Remove $HOME/.docker/buildx/activity/default
d1b54ede2 qemu: tdx: Workaround SMP issue with TDX 1.5
1e34220c4 qemu: tdx: Adapt to the TDX 1.5 stack
8115a0522 versions: tdx: Update Kernel to 6.2 + TDX
ec18180f3 versions: tdx: Update TDVF to the "edk2-stable202302"
9803b2428 versions: tdx: Update QEMU to v7.2 + TDX v1.10
dffc16e5b runtime-rs: check peer close in log_forwarder
aaa5ab126 agent: simplify storage device by removing StorageDeviceObject
fb49d5d7c gha: Avoid "fail-fast" in tests that are known to be flaky
183f51d6f tests: use unique test name
6a974679f tests: delete k8s deployment at the test's end
32a778b6d metrics: Remove unused variable in tensorflow nhwc script
d8f3ce649 kata-deploy: Don't try to remove /opt/kata
936e8091a gha: vfio: Run on Ubuntu 23.04 runner
0e7248264 agent: move storage device related code into dedicated files
268e84655 runtime-rs: Fix volumes and rootfs cleanup issues
8f49ee33b agent: refine storage related code a bit
60ca12ccb agent: switch to new storage subsystem
fcbda0b41 kata-types: introduce StorageDevice and StorageHandlerManager
b03b1f613 agent: simplify the way to manage storage object
8392c71bf sys-util: support more mount flags in parse_mount_options()
c00d8f3d4 agent: use create_mount_destination() from kata-sys-util
5e867f053 types: add more mount related constants
880e6c9a7 agent: use function from kata-sys-utils to reduce code
3b881fbc0 local-build: Remove GID before creating group
959ca4944 metrics: Add TensorFlow ResNet50 fp32 Dockerfile
4b7d72c4a metrics: Add TensorFlow ResNet50 FP32 benchmark
5cba38c17 kata-deploy: Avoid failing on content removal
18d42da21 runtime/fc: fix image/initrd annotation handling
9fda7059a runtime/clh: fix image/initrd annotation handling
1a0092d63 runtime/qemu: fix image/initrd annotation handling
22d8f335d libs,tests: fix typo disable_guest_seccomp in configuration-anno-1.toml
8afd158ce metrics: Add disk link to README
40914b25d kata-agent: use default filemode for block device when it is set to 0
eee2ee6ee metrics: Fix FIO path
39bc3488f metrics: Use function from metrics common in pytorch script
400eb8874 gha: capture additional kata-deploy output
4aee3eade kata-types: implement serde methods for KataVirtualVolume
b875e3932 kata-types: validate KataVirtualVolume object
fa2fdc105 kata-types: implement two conversion helpers for KataVirtualVolume
6326af20e kata-types: introduce KataVirtualVolume
c8b43f8b3 metrics: Fix README for pytorch
fb571f8be metrics: Enable kata runtime in K8s for FIO test.
cb056f8cb rootfs: agent: Policy support with AGENT_INIT=yes
85c02828e metrics: Update tensorflow name in gha run script
e8a511934 metrics: Fix check results for tensorflow benchmark
2d896ad12 gha: kata-deploy: Do the runtime class cleanup as part of the cleanup
4ffc2c86f gha: kata-deploy: Add the first kata-deploy test
8616c050a metrics: Remove unused variable in tensorflow mobilenet script
285e616b5 tests: common: Ensure test_type is used as part of the cluster's name
790bd3548 tests: commob: Don't fail if yq is not part of the cache
ce6adecd0 gha: kata-deploy: Add run-kata-deploy-tests.sh
cfc29c11a gha: k8s: Stop running kata-deploy tests as part of the k8s suite
f4dd15286 tests: k8s: Call ensure_yq() in setup.sh
339569b69 kata-deploy: Properly create default runtime class
2a491e9b1 metrics: Fix MobileNet help me description
d19a75e80 gha: ci: Start running kata-deploy tests
d90f7ac68 runtime-rs: add unit test for block driver
e44919f0d runtime-rs: add load_test_config for unit test
7f48a6937 runtime-rs: add driver option
bade6a5c3 docs: Fix TensorFlow word across the document
1a1b20776 docs: Add Tensorflow Resnet50 documentation
24baededc metrics: Add Dockerfile for ResNet50 int8
6d971ba8d metrics: Add Tensorflow ResNet50 int8 benchmark
25d151bd1 runk: Modify kill command's error message for containerd tests
b3592ab25 gha: cri-containerd: Enable tests
84dd02e0f gha: cri-containerd: Add timeout to the crictl calls on testContainerStop
b29782984 gha: cri-containerd: Show pod before deleting it
ae0930824 gha: cri-containerd: Print kata logs in case of error
6c8b2ffa6 gha: cri-containerd: Group containerd logs
9e898701f gha: cri-containerd: Ensure RUNTIME takes KATA_HYPERVISOR into account
76dac8f22 agent: simplify error handling
18a7fd8e4 metrics: Rename tensorflow scripts
e55fa93db tests: kata-deploy: Add placeholder for kata-deploy-tests-on-tdx
d9ee17aae tests: kata-deploy: Add placeholder for kata-deploy-tests-on-aks
ab829d103 agent: runtime: add the Agent Policy feature
831e73ff9 tests: kata-deploy: Add functional/kata-deploy/gha-run.sh placeholder
af1b46bbf tests: Add gha-run-k8s-common.sh
416445e7e docs: Remove installation step in virtcontainers doc
72cbcf040 kata-deploy: Add k0s support
767434d50 metrics: fix the loop used to stop kata components #76295d0f0d43c metrics: Add cassandra statefulset yaml
c1dcc1396 metrics: Add cassandra service yaml
2297a0d1c metrics: Add block loop pvc yaml for cassandra
e3d511946 metrics: Add block loop pv yaml for cassandra test
989027159 metrics: Add block loop pvc for cassandra test
349b89969 metrics: Add Cassandra Kubernetes benchmark for kata metrics
c52d09052 gha: static-checks: Move to the Azure instances
8815ed066 runtime: Remove config warnings
afe1a6ac5 agent: support copying of directories and symlinks
ab13ef87e runtime: propagate configmap/secrets etc changes for remote-hyp
c074ec4df runtime: Copy shared files recursively
fdcd52ff7 metrics: Add check containers are running in tensorflow mobilenet
36337ee14 metrics: Add check containers are up in tensorflow script
f700f9b0b metrics: Remove unused variable in tensorflow script
833cf7a68 metrics: Add check containers are running function
918c78308 metrics: Add check containers are up in tensorflow mobilenet script
9d57a1fab metrics: Use check containers are up in tensorflow script
1c84680d8 metrics: Add check containers are up in common script
d3e57cf45 metrics: Use collect_results function in tensorflow mobilenet test
286de046a metrics: Remove collect results function definition
9879709aa metrics: Add common functions to the common script
4746fa3da docs: Specify supported Firecracker version using `versions.yaml`
cc922be5e versions: Update firecracker version to 1.4.0
39e67b06e dragonball: vsock add fifo/pipe stream support for passed fd hybridStream
473b0d3a3 metrics: compute tensorflow statistics
03d1fa67b ci: unencrypted-image: Fix build context
eb463b38e ci: unencrypted-image: Don't fail to build on s390x
a2d731ad2 ci: create-confidential-image: Add dependent actions
d1a629622 metrics: Add nginx documentation to network README
498f7c054 metrics: Add nginx kubernetes yaml
f8a5255cf metrics: Add network nginx benchmark
43fe5d1b9 ci: k8s: tees: Ensure PR_NUMBER is exported
54f6a7850 ci: {{ pr-number }} should be {{ inputs.pr-number }}
034d7aab8 tests: k8s: Ensure the runtime classes are properly created
fac8ccf5c ci: Add build-and-publish-tee-confidential-unencrypted-image
ab5f603ff ci: k8s: Add the image used for unencrypted confidential tests
1e8fe131b k8s: tests: Take advantage of `SHIMS` and `DEFAULT_SHIM` env vars
729b2dd61 agent: avoid creating new `Vec` instances when easily avoidable
aeaec9dae tests: upgrade bats version
e66496986 metrics: install kata once and run multiple checks
baabfa9f1 agent: refine implementation of mount related code
98ba211a3 agent: fix a bug in update_ephemeral_mounts()
5333618d7 agent: make add_storage() take &[Storage] instead of Vec<Storage>
37f34781d agent: simplify function online_cpu_memory()
d3c542237 agent: refine style of code related to sandbox
71a9f6778 agent: avoid unwrap() in function do_remove_container()
84badd89d agent: avoid clone objects when possible
b23c5ed15 deps: Bump dependent crate versions
863283716 metrics: General improvements to mobilenet tensorflow test
3c319d8d4 metrics: Add iperf to gha run script
5b5caf890 gha: Add iperf network metrics
66db5b535 metrics: Add latency test to network README
c36572418 agent: avoid unnecessary calls to `Arc::clone`
4fbe0a3a5 runtime: bind-mount mounted block device into container
7e1b1949d runtime: add support for kata overlays
6c867d9e8 agent: add io.katacontainers.fs-opt.overlay-rw option
6163c3565 agent: skip mount options that start with "io.katacontainers."
b2ff97aa0 dragonball: use version 0.10.4 of `fuse-backend-rs`
845eeb4d7 agent: Allow clippy::redundant_clone in the unit tests
1163fc9de release: Revert kata-deploy changes after 3.2.0-rc0 release
3958a39d0 runtime-rs: Introduce directly attachable network
1e15369e5 metrics: Improve naming testing containers in launch times test
5dbe88330 metrics: Clean kata components before start a metric test.
3b45060b6 metrics: Add latency server yaml
9bb8451df metrics: Add latency client yaml
64fdb9870 metrics: Add network latency test
a81ad3b58 runtime-rs: Add block device handling in cloud hypervisor
3230dec95 kata-deploy: Use host's systemctl
1b21a4624 docs: Use control-plane term instead of master
28e5e9c86 runtime-rs: fix number of queues handling in dragonball share fs device
f1d8de9be runk: Allow runk to launch a container without pid namespace
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is basically to make sure that folks trying to use the kata-deploy
script from the main branch, to deploy **stable** kata-deploy images, do
not have a hard time.
Fixes: #7194
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Remove the ability to block access to kata agent endpoints by using
agent-config.toml. That functionality is now implemented using the
Agent Policy feature (#7573).
The CCv0 branch relied on blocking endpoints using agent-config.toml
but will set-up an equivalent default policy file instead (#8219).
Fixes: #8228
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Add the missing closing bracket to the output of the TDX details,
so rather than:
```bash
$ sudo kata-ctl env 2>/dev/null | grep available_guest_protection
available_guest_protection = "tdx (major_version: 1, minor_version: 0"
: ^
: Missing ')' !
```
... we now have:
```bash
$ sudo kata-ctl env 2>/dev/null | grep available_guest_protection
available_guest_protection = "tdx (major_version: 1, minor_version: 0)"
: ^
: Aha!
```
Added a unit test for this scenario.
Fixes: #8257.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
If you attempt to create a container (a TD) on a TDX system using a
custom build of Cloud Hypervisor (CH) that was not built with the `tdx`
CH feature, Kata will report the following, somewhat cryptic, CH error:
```
ApiError(VmBoot(InvalidPayload))
```
Newer versions of CH now report their build-time features in the ping
API response message so we now use that, if available, to detect this
scenario and generate a user-friendly error message instead.
This changes improves the readability of `handle_guest_protection()` and
adds a couple of additional tests for that method.
Fixes: #8152.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Improve the way `handle_guest_protection()` is structured by inverting
the logic and checking the value of the `confidential_guest` setting
before checking the guest protection. This makes the code easier to
understand.
> **Notes:**
>
> - This change also unconditionally saves the available guest protection
> (where previously it was only saved when `confidential_guest=true`).
> This explains the minor unit test fix.
>
> - This changes also errors if the CH driver finds an unexpected
> protection (since only Intel TDX is currently tested).
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This PR adds the iperf udp benchmark for bandwdith measurement
for network metrics.
Fixes#8246
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
ACPI PCI device hotplug on qemu virt is not supported. The only way to
hotplug pci device is pcie native way. Thus we need create pcie root
port as default.
Pcie root port number depends on following:
1. reserved one for network device as default;
2. virtio-mem dev;
3. add enough port for vhost user blk dev;
Fixes: #7646
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Add GetEndpointsNum API for Network Interface to get the number of
network endpoints. This is used for caculate the number of pcie root
port for QemuVirt.
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Encode policy file during test - easier to understand than hard-coding
the encoded file contents.
Fixes: #8214
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR corrects the init env() helper function, to make that
systemctl always returns true when enumerating masked services,
and preventing the test from failing
Fixes: #8242
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR fixes the correct path to setup, build and install properly
the kernel for snp.
Fixes#8156
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
1. enable virtio-fs-pro in Dragonball to have the ability to process nydus backend registry
2. change passthrough for rw layer's readonly config to false to have the accurate read write ability.
Fixes:#8013
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Since Nydus snapshotter has been updated in previous commits, there is a
problem that the config passthrough to Dragonball during mount_rafs is
RafsConfig instead of ConfigV2, but Dragonball could only serde ConfigV2
so it will panic.
We need to add the support for RafsConfig
Fixes:#8013
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Allow access to the ReseedRandomDev endpoint by default. Using false
for ReseedRandomDevRequest was unintended.
Fixes: #8225
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
To avoid errors when initializing the test environment, the
kill_processes_before_start() helper function needs to verify that
docker is installed before attempting to stop it.
Fixes: #8218
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR removes trailing commas so that the json results
file is valid.
This PR also changes the way data results are collected by
terating through the array of memory values to calculate
their average.
Fixes: #8204
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Previously, if you accidentally modified the name of the hypervisor
section in the config file, the default golang runtime gives a cryptic
error message ("`VM memory cannot be zero`"). This can be demonstrated
using the `kata-runtime` utility program which uses the same golang
config package as the actual runtime (`containerd-shim-kata-v2`):
```bash
$ kata-runtime env >/dev/null; echo $?
0
$ sudo sed -i 's!^\[hypervisor\.qemu\]!\[hypervisor\.foo\]!g' /etc/kata-containers/configuration.toml
$ kata-runtime env >/dev/null; echo $?
VM memory cannot be zero
1
```
The hypervisor name is now validated so that the behaviour becomes:
```bash
$ kata-runtime env >/dev/null; echo $?
0
$ sudo sed -i 's!^\[hypervisor\.qemu\]!\[hypervisor\.foo\]!g' /etc/kata-containers/configuration.toml
$ ./kata-runtime env >/dev/null; echo $?
/etc/kata-containers/configuration.toml: configuration file contains invalid hypervisor section: "foo"
1
```
Fixes: #8212.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
As the file is already part of the kata-containers repo, and the tests
repo is about to become read-only, we're good to drop the tests
references from here and use everything coming from the
`kata-containers` repo instead.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we've moved all the tests to the `kata-containers` repo, the `tests`
repo will become a read-only repo.
Fixes: #8200
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Improve the `GuestProtection` handling to detect the version of
Intel TDX available.
The TDX version is now logged by the Cloud Hypervisor driver.
Fixes: #8147.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Direct-volume needs to use the same base64 character set as
kata-runtime/direct-volume does.
Fixes: #8175
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This PR removes the reference in the documentation to the DAX
subtest of the FIO benchmark, because this metric is currently
WIP.
Fixes: #8159
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This change adds support for adding and removing vfio devices for
cloud-hypervisor.
Fixes: #6691
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
To pick up fix for the following issue:
A maliciously crafted HTTP/2 stream could cause excessive CPU
consumption in the HPACK decoder, sufficient to cause a denial of
service from a small number of small requests.
Fixes: #8190
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
This PR adds a trap whenever the scrip exits, it deletes the iperf
k8s deployment and k8s services, and deletes the kata components.
This way, when the script finishes, it verifies that there are
indeed no kata components still running.
Fixes: #8126
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
The KUBERNETES variable is mostly used by kata-deploy whether to apply
k3s specific deployments or not. It is used to select the type of
kubernetes to be installed (k3s, k0s, rancher...etc) and it is always
set on CI. Running the script locally we want to set a value by default
to avoid `KUBERNETES: unbound variable` errors.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This test can give false-positive on a multi-node cluster. Changed it to
use the new get_one_kata_node() and the modified exec_host() to run the
setup commands on a given node (that has kata installed) and ensure the
test pod is scheduled at that same node.
Fixes#7619
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This test can give false-positive on a multi-node cluster. Changed it to
use the new get_one_kata_node() and the modified exec_host() to run the
setup commands on a given node (that has kata installed) and ensure the
test pod is scheduled at that same node.
Fixes#7619
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The exec_host() simply fails on cluster with multi-nodes because
`kubectl get node -o name" will return a list o names. Moreover, it will
return control nodes names which usually don't have kata installed.
Fixes#7619
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The introduced get_one_kata_node() returns the first node that
has the kata-runtime=true label, i.e., supposedly a node with
kata installed.
This is useful for tests that should run on a determined worker
node on a multi-nodes cluster.
Fixes#7619
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Let KATA_HYPERVISOR be qemu by default in gh-run.sh as this variable
is required to tweak some configurations of kata-deploy.
Fixes#7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The deploy-kata() of gha-run.sh will wait for 10 minutes for the kata
deploy installation finish. This allow users of the script to overwrite
that value by exporting the KATA_DEPLOY_WAIT_TIMEOUT environment
variable.
Fixes#7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Fixed a couple of warns shellcheck emitted and disabled others:
* SC2154 (var is referenced but not assigned)
* SC2086 (Double quote to prevent globbing and word splitting)
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The .tests/integration/kubernetes/gh-run.sh script run `yq write` a
couple of times to edit the kata-[deploy|cleanup].yaml, resulting
on the file being formatted again. This is annoying because leaves
the git tree dirty.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The only difference to the other platforms is that it needs to
export KUBECONFIG.
Fixes#7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The cleanup-kcli() behaves like other deploy kata for
bare-metal (e.g. sev, tdx...etc) except that KUBECONFIG
should be exported.
Fixes#7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The cleanup-kcli() behaves like other clean up for bare-metal (e.g. sev,
tdx...etc) except that KUBECONFIG should be exported.
Fixes#7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
On CI workflows the variables DOCKER_REGISTRY, DOCKER_REPO and
DOCKER_TAG are exported to match the built image. However, when running
the script outside of CI context, a developer might just use the latest
image which in this case will be
`quay.io/kata-containers/kata-deploy-ci:kata-containers-latest`.
Fixes#7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Adapted the gha-run.sh script to create a Kubernetes cluster locally
using the kcli tool.
Use `./gha-run.sh create-cluster-kcli` to create it, and
`./gha-run.sh delete-cluster-kcli` to delete.
Fixes#7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
GHA has an undocumented limitation that there can be at most 20
referenced yamls in a single yaml file. We workaround it by combining
multiple jobs into a single yaml file.
Fixes: #8161
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Enable the Cloud Hypervisor driver (the `cloud-hypervisor` build feature) for the rust runtime.
Fixes: #6264.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
I'm basically moving the runk tests from the tests repo to this one, and
I'm adding the "Signed-off-by:" of every single contributor the tests.
Fixes: #8116
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Chen Yiyang <cyyzero@qq.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The runk test has been executed as part of the former "ubuntu" jenkins
CI.
We're porting it to GHA and running it against LTS containerd.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The tracing tests are currently running as part of the Jenkins CI with
the following setups:
* Container Engines: containerd
* VMMs: QEMU | Cloud Hypervisor
* Snapshotters: overlayfs | devmapper
We'll be restricting those tests to be running on LTS version of
containerd, without devmapper.
As it's known due to our GHA limitation, this is just a placeholder and
the tests will actually be added in the next interations.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise we'll use any arm64 machine that's added as a runner, and
whenever new machines are added those may end up being only used for
running some specific set of the tests.
Fixes: #8109
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR enables the use of jq pretty-print feature to
improve the formatting of metric results json files.
Fixes: #8081
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
FIO benchmark is enabled to measure IO in Kata
at different latencies using containerd client,
in order to complement the CI metrics testing set.
This PR asl deprecated the previous Fio bench
based on k8s.
Fixes: #8080
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
The tests are failing when setting up k0s, and that happens because we
download a kubectl binary matching the kubernetes version k0s is using,
and we do that by:
```
sudo k0s kubectl version --short 2>/dev/null | ...
```
With kubectl 1.28, which is now the default on k0s, `kubectl version
--short` has been removed, leading us to an empty stringm causing then
the error in the CI.
Fixes: #8105
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've added two new containerd builder images recently, one for the
components under `src/tools` and another one for the Kata Containers
agent.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This follows what we've been doing for all the components we're
building, but was missed as part of #8077.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The kata-agent binary won't be released, just built so it can be used,
later on, as part of our tests and as part of the rootfs build.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the needed functions to start building the kata-agent, with or
without the OPA support.
For now this build is not used as part of the rootfs build, but later on
this will (not as part of this series, though).
Fixes: #8099
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will help to build the agent binary as part of the kata-deploy
localbuild, as we need to pass the DESTDIR to where the agent will be
installed, and also whether we're building the agent with policy support
enabled or not.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The k8s.gcr.io is deprecated for a while now and has been redirected to
registry.k8s.io. However on some bare-metal machines in our testing
pools that redirection is not working, so let's just replace the
registries.
Fixes#8098
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit b2c3bca558c38deff2117d5909d9071c23c05590)
Spell check failed with:
```
[kata-spell-check.sh:275] WARNING: Word 'overcommitment':
did you mean one of the following?: over commitment, over-commitment,
commitment
```
So update this to pass the static checks
Fixes: #
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Ensure that our documented crictl pod config file contents have
uid and namespace fields for compatibility with crictl 1.24+
This avoids a user potentially hitting the error:
```
getting sandbox status of pod "d3af2db414ce8": metadata.Name,
metadata.Namespace or metadata.Uid is not in metadata
"&PodSandboxMetadata{Name:nydus-sandbox,Uid:,Namespace:default,Attempt:1,}"
getting sandbox status of pod "-A": rpc error: code = NotFound desc = an
error occurred when try to find sandbox: not found
```
Fixes: #8092
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
(cherry picked from commit 8f8c2215)
We need to do proper sandbox sizing when we're doing cold-plug introduce CDI,
the de-facto standard for enabling devices in containers. containerd
will pass-through annotations for accumulated CPU,Memory and now CDI
devices. With that information sandbox sizing can be derived correctly.
Fixes: #7331
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Let's add targets and actually enable users and oursevles to build those
components in the same way we build the rest of the project.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we'd like to ship the content from src/tools, we need to build them
in the very same way we build the other components, and the first step
is providing scripts that can build those inside a container.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The kata-monitor tests is currently running as part of the Jenkins CI
with the following setups:
* Container Engines: CRI-O | containerd
* VMMs: QEMU
When using containerd, we're testing it with:
* Snapshotter: overlayfs | devmapper
We will stop running those tests on devmapper / overlayfs as that hardly
would get us a functionality issue.
Also, we're restricting this to run with the LTS version of containerd,
when containerd is used.
As it's known due to our GHA limitation, this is just a placeholder and
the tests will actually be added in the next iterations.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will serve us quite will in the upcoming tests addition, which will
also have to be executed using CRi-O.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will become handy when doing tests with CRI-O, as CRI-O doesn't
install the CNI plugins for us.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's ensure we have runc running with `SystemdCgroups = false`,
otherwise we'll face failures when running tests depending on runc on
Ubuntu 22.04, woth LTS containerd.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Pause and resume task do not currently update the status of the
container to paused or running, so fix this. This is specifically for
pausing the task and not the VM.
Fixes#6434
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Allow Cloud Hypervisor to create a confidential guest (a TD or
"Trust Domain") rather than a VM (Virtual Machine) on Intel systems
that provide TDX functionality.
> **Notes:**
>
> - At least currently, when built with the `tdx` feature, Cloud Hypervisor
> cannot create a standard VM on a TDX capable system: it can only create
> a TD. This implies that on TDX capable systems, the Kata Configuration
> option `confidential_guest=` must be set to `true`. If it is not, Kata
> will detect this and display the following error:
>
> ```
> TDX guest protection available and must be used with Cloud Hypervisor (set 'confidential_guest=true')
> ```
>
> - This change expands the scope of the protection code, changing
> Intel TDX specific booleans to more generic "available guest protection"
> code that could be "none" or "TDX", or some other form of guest
> protection.
Fixes: #6448.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Introduce a few new constants (for PCI segment count and FS queues) and
move the disk queue constants to `convert.rs` to allow them to be used
there too.
> **Note:**
>
> This change gives the `ShareFs` code it's own set of values rather
> than relying on the disk queue constants.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Modify the Cloud Hypervisor `add_device()` method to add `ShareFs` and
`Network` devices to the list of pending devices since only these two
device types need to be cached before VM startup. Full details in the
comments.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Remove the `VIRTIO_BLK_MMIO` check which appears to have been added
erroneously in the first place.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This patch re-generates the client code for Cloud Hypervisor v35.0.
Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by openapi-generator.
Fixes: #8057
Signed-off-by: Bo Chen <chen.bo@intel.com>
We've faced this as part of the CI, only happening with the CRI-O tests:
```
not ok 1 Test readonly volume for pods
# (from function `exec_host' in file tests_common.sh, line 51,
# in test file k8s-file-volume.bats, line 25)
# `exec_host "echo "$file_body" > $tmp_file"' failed with status 127
# [bats-exec-test:38] INFO: k8s configured to use runtimeclass
# bash: line 1: $'\r': command not found
#
# Error from server (NotFound): pods "test-file-volume" not found
```
I must say I didn't dig into figuring out why this is happening, but we
may be safe enough to just trail the '\r', as long as all the tests keep
passing on containerd.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need the default capabilities to be enabled, especially `SYS_CHROOT`,
in order to have tests accessing the host to pass.
A huge thanks to Greg Kurz for spotting this and suggesting the fix.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
Some of the "k8s distros" allow using CRI-O in a non-official way, and
if that's done we cannot simply assume they're on containerd, otherwise
kata-deploy will simply not work.
In order to avoid such issue, let's check for `cri-o` as the container
engine as the first place and only proceed with the checks for the "k8s
distros" after we rule out that CRI-O is not being used.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR fixes the network metrics section at the README by leaving
the current tests that we have in our kata metrics.
Fixes#8017
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The permissions on .docker/buildx/activity/default are regularly broken by us
passing docker.sock + $HOME/.docker to a container running as root and then
using buildx inside. Fixup ownership before executing docker commands.
Fixes: #8027
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
To avoid the failure of not finding pandoc command this PR adds that
package as a dependency for static checks.
Fixes#8041
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The cgroup stats come from resourcecontrol package in the form of pointers
to structs. The sandbox Stat() method incorrectly was expecting structs.
This caused the cpu and memory stats to always be 0, which in turn caused
incorrect pod overhead metrics.
Fixes#8035
Signed-off-by: Peteris Rudzusiks <rye@stripe.com>
Firecracker supports noflush semantic via Unsafe cache type.
There is no support for direct i/o, remove it from config file
Fixes: #7823
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Clh suports direct i/o for disks. It doesn't
offer any support for noflush, removed passing
of option to cloud-hypervisor internal config
Fixes: #7798
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Seems like the static checks are failing due the missing of the hunspell
package this PR fixes that.
Fixes#8019
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The test has been added to the repo, but we have to also add it to the
list of jobs to be executed.
Fixes: #8005
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise we'll face the following error:
```
Failed to enable unit: Interactive authentication required.
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure we'll also be testing k8s using CRI-O.
For now, we'll only be running the CRI-O test with QEMU. Once it
becomes stable we can expand this to other Hypervisors as well.
Fixes: #8005
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Support quoted kernel command line parameters that include space
characters. Example:
dm-mod.create="dm-verity,,,ro,0 736328 verity 1
/dev/vda1 /dev/vda2 4096 4096 92041 0 sha256
f211b9f1921ef726d57a72bf82be23a510076639fa8549ade10f85e214e0ddb4
065c13dfb5b4e0af034685aa5442bddda47b17c182ee44ba55a373835d18a038"
Fixes: #8003
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This is part of a bigger effort to drop gogoprotobuff from our code
base. IIUC, those options are basically used by *pb_test.go, and since
we are dropping gogoprotobuff and those are auto generated tests, let's
just remove it.
Fixes#7978.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
This will be very useful in the near future, when we start testing
kata-deploy with rke2 as well.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will be very useful in the near future, when we start testing
kata-deploy with k0s as well.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We'll be using exactly the same code used for the k8s tests, which are
already deploying k3s on GARM.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We just need to make sure the correct overlay is applied, following what
we already have been doing for k3s.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We'll be testing kata-deploy with different kubernetes flavours as part
of our GARM tests, and this is a place-holder for this.
Once enabled, we'll do nothing, just `return 0`, so we can then properly
add the tests after this commit gets merged.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
So we have a better control on which flavour of kubernetes kata-deploy
is expected to be targetting.
This was also done as part of fa62a4c01b,
for the k8s tests.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Ideally we'd add the instance_type or the full K8S_TEST_HOST_TYPE but
that exceeds the maximum amount of characteres allowed for the cluster
name. With this in mind, let's use the first letter of
K8S_TEST_HOST_TYPE instead.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This makes it so that each AKS cluster is created in its own individual
resource group, rather than using the "kataCI" resource group for all
test clusters.
This is to accommodate a tool that we recently introduced in our Azure
subscription which automatically deletes resource groups after a set
amount of time, in order to keep spending under control.
The tool will automatically delete any resource group, unless it has a
tag SkipAutoDeleteTill = YYYY-MM-DD. When this tag is present, the
resource group will be retained until the specified date.
Note that I tagged all current resource groups in our subscription with
SkipAutoDeleteTill = 2043-01-01 so that we don't lose any existing
resources.
Fixes: #7982
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
We're hitting a specific issue after updating, which will require some
work on dragonball before it can be re-added here.
The issue:
```
...
3: failed to do rafs mount\\n
4: fail to attach rafs \\\"/var/lib/containerd-nydus/snapshots/2/fs/image/image.boot\\\"\\n
5: add share fs mount\\n
6: Mount rafs at
/rafs/197ef3db03c86b91bf3045ff59183ce8b5750941ad1d3484f4a8301a70f5109f/rootfs_lower
error: Failed to Mount backend
...
Caused by:
vmm action error: FsDevice(AttachBackendFailed(\\\"attach/detach a
backend filesystem failed:: missing field `version` at line 1 column
489\\\"))\"): unknown"
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will ensure we're testing with the correct runtime, instead of
using the `default` one.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Fix the arch error when downloading the nydus tarball.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Signed-off-by: Steven Horsman <steven@uk.ibm.com>
To support the v0.12.0 nydus-snapshotter, we need to update the config
files and the commandline to start nydus-snapshotter.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
And with this we finally enable the nydus tests to run as part of our
GHA CI.
Fixes: #6543
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've been simply doing nothing whenever `install-kata` was called, and
that was the intent when we added the placeholder calls.
Now, let's install kata, as expected. :-)
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we've added install_nydus() and install_nydus_snapshotter(), which do
conform with the pattern we're following on GHA, let's rely on them
rather than relying on the bits coming from nydus_test.sh.
Later on we'll have install_nydus() and install_nydus_snapshotter() as
part of the dependencies install in our `gha-run.sh`.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Similarly to what's been done for the cri-containerd tests, as part of
84dd02e0f9, we need to add the timeout
here for the crictl calls.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise we may face errors like:
```
getting sandbox status of pod "d3af2db414ce8": metadata.Name,
metadata.Namespace or metadata.Uid is not in metadata
"&PodSandboxMetadata{Name:nydus-sandbox,Uid:,Namespace:default,Attempt:1,}"
getting sandbox status of pod "-A": rpc error: code = NotFound desc = an
error occurred when try to find sandbox: not found
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise we canoot properly start the nydus snapshotter, nor properly
kill it after it's been started.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The "source ..." we've been doing was not changed since those tests were
part of the Jenkins tests, and we need to adapt them, either setting the
correct path or entirely removing the ones that are not relevant to us
anymore.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function will be used to download and install the
nydus-snapshotter, and it follows the same pattern we already have
introduced for downloading and installing another dependencies from
GitHub.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function will be used to download and install nydus, and it follows
the same pattern we already have introduced for downloading and
installing another dependencies from GitHub.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Users have noticed that this is needed, as CLH does not yet implement a
way to hotplug resources on aarh64.
With this patch, when building for x86_64, I can see the this is the
resulting config:
```
$ ARCH=amd64 make
...
$ cat config/configuration-clh.toml | grep static_sandbox_resource_mgmt
static_sandbox_resource_mgmt=false
```
And when building for aarch64:
```
$ ARCH=arm64 make
...
$ cat config/configuration-clh.toml | grep static_sandbox_resource_mgmt
static_sandbox_resource_mgmt=true
```
Fixes: #7941
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
PR #6146 added the possibility to control QEMU with an extra HMP socket
as an aid for debugging. This is great for development or bug chasing
but this raises some concerns in production.
The HMP monitor allows to temper with the VM state in a variety of ways.
This could be intentionally or mistakenly used to inject subtle bugs in
the VM that would be extremely hard if not even impossible to debug. We
definitely don't want that to be enabled by default.
The feature is currently wired to the `enable_debug` setting in the
`[hypervisor.qemu]` section of the configuration file. This setting has
historically been used to control "debug output" and it is used as such
by some downstream users (e.g. Openshift). Forcing people to have the
extra HMP backdoor at the same time is abusive and dangerous.
A new `extra_monitor_socket` is added to `[hypervisor.qemu]` to give
fine control on whether the HMP socket is wanted or not. This setting
is still gated by `enable_debug = true` to make it clear it is for
debug only. The default is to not have the HMP socket though. This
isn't backward compatible with #6416 but it is for the sake of "better
safe than sorry".
An extra monitor socket makes the QEMU instance untrusted. A warning is
thus logged to the journal when one is requested.
While here, also allow the user to choose between HMP and QMP for the
extra monitor socket. Motivation is that QMP offers way more options to
control or introspect the VM than HMP does. Users can also ask for
pretty json formatting well suited for human reading. This will improve
the debugging experience.
This feature is only made visible in the base and GPU configurations
of QEMU for now.
Fixes#7952
Signed-off-by: Greg Kurz <groug@kaod.org>
Now that the static-checks job only takes care of running the
static-checks, let's clean it up, remove all the unneeded steps, make
sure that we're using the actions in their latest version, and have it
running in a cost free runner.
At some point I'd like to see those tests done in parallel, in the same
way that I've organised the build-checks, but that's something for
someone else, at some other time.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
With this we're removing the dragonball static-checks CI, as the test is
running here now. :-)
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're moving it out of the previous "static-checks" confusing matrix,
and adding it to the matrix that was currently being used for the `make
vendor` and `make check` checks.
This will allow us to have one job per component, and with that we can
easily run those in parallel and on the zero cost runners.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're moving it out of the previous "static-checks" confusing matrix,
and adding it to the matrix that was currently being used for the `make
vendor` and `make check` checks.
This will allow us to have one job per component, and with that we can
easily run those in parallel and on the zero cost runners.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise `make test` will simply fail with:
```
error[E0583]: file not found for module `config`
```
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This makes it pssible to run the tests in the cost free runners, which
are not KVM capable.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise `make test` will simply fail with:
```
error[E0583]: file not found for module `version`
```
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise `make test` will fail with:
```
error[E0583]: file not found for module `version`
```
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're moving it out of the previous "static-checks" confusing matrix,
and adding it to the matrix that was currently being used for the `make
vendor` checks.
This will allow us to have one job per component, and with that we can
easily run those in parallel and on the zero cost runners.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
clippy is used as part our tests, so it's useful to have it installed
while we're already installing rust.
In case of developers, they also better be using it. :-)
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Similarly to the static-check jobs, those jobs can be run on the zero
cost runners.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We'll use it as part of the refactoring we're doing in the static check
tests.
I can see a lot of other uses of this, but changing all of them to this
one is out of the scope for this PR.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We can use this a lot as part of our CI, but right now I'm just moving
those here with the intent to use later on in this series.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It doesn't make sense to run this for all the bits of the matrix,
neither it's demanding enough to require running this in one of our
Azure sponsored runners.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let me start with a fair warning that this commit is hard to split into
different parts that could be easily tested (or not tested, just
ignored) without breaking pieces.
Now, about the commit itself, as we're on the run to reduce costs
related to our sponsorship on Azure, we can split the k8s tests we run
in 2 simple groups:
* Tests that can be run in the smaller Azure instance (D2s_v5)
* Tests that required the normal Azure instance (D4s_v5)
With this in mind, we're now passing to the tests which type of host
we're using, which allows us to select to run either one of the two
types of tests, or even both in case of running the tests on a baremetal
system.
Fixes: #7972
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR adds support for adding a network device before starting the
cloud-hypervisor VM.
Support for adding and removing network devices is not really added to
the resource manager, so supporting this for cloud-hypervisor is not
scoped in this PR.
This also changes "pending_devices" for clh implementation from an
Option of vector to simply a vector. This simplifies the structure a bit
as we can simple iterate over the pending devices instead of having to
check for a "Some" value as this is not really required.
Fixes: #6333
Signed-off-by: Shuaiyi Zhang <zhang_syi@qq.com>
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
We've removed this in the part 2 of this effort, as we were not caching
the sha256sum of the component. Now that this part has been merged,
let's get back to checking it.
Fixes: #7834 -- part 3
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
That's not needed anymore, as we've switched to using ORAS and an OCI
registry to cache the artefacts.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is something that was done by our Jenkins jobs, but that I ended up
missing when writing d0c257b3a7.
Now, let's also add the sha256sum to the cached artefact, and in a
coming up PR (after this one is merged) we will also start checking for
that.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In the previous series related to the artefacts we build, we've
switching from storing the artefacts on Jenkins, to storing those in the
ghcr.io/kata-containers/cached-artefacts/${artefact_name}.
Now, let's take advantage of that and actually use the artefacts coming
from that "package" (as GitHub calls it).
NOTE: One thing that I've noticed that we're missing, is storing and
checking the sha256sum of the artefact. The storing part will be done
in a different commit, and the checking the sha256sum will be done in a
different PR, as we need to ensure those were pushed to the registry
before actually taking the bullet to check for them.
Fixes: #7834 -- part 2
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
cloud hypervisor on arm64 only support arm AMBA UART(pl011) as
tty. So, the console should be set to "ttyAMA0" instead of "ttyS0"
when enable hypervisor debug mode.
Fixes: #5080
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
The list of tests which require a bigger VM instance is:
* k8s-number-cpus.bats -- failing on all CIs
* k8s-parallel.bats -- only failing on the cbl-mariner CI
* k8s-scale-nginx.bats -- only failing on the cbl-mariner CI
We'll keep those disabled while we re-work the logic to **only run
those** in a bigger (and more expensive) VM instance.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's push the artefacts to ghcr.io and stop relying on jenkins for
that.
Fixes: #7834 -- part 1
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Right now this is not used, but it'll be used when we start caching the
artefacts using ORAS.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
ORAS is the tool which will help us to deal with our artefacts being
pushed to and pulled from a container registry.
As both the push to and the pull from will be done inside the
kata-deploy binaries builder container, we need it installed there.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We don't need to run on a D4s_v5. as those tests are not CPU / memory
intense. With this is mind, let's use a smaller version of the
instance, the D2s_v5 one.
Fixes: #7958
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We don't need to run on a D4s_v5. as those tests are not CPU / memory
intense. With this is mind, let's use a smaller version of the
instance, the D2s_v5 one.
Fixes: #7958
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We don't need to run on a D4s_v5. as those tests are not CPU / memory
intense. With this is mind, let's use a smaller version of the
instance, the D2s_v5 one.
Fixes: #7958
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We don't need to run on a D4s_v5. as those tests are not CPU / memory
intense. With this is mind, let's use a smaller version of the
instance, the D2s_v5 one.
Fixes: #7958
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We don't need to run on a D4s_v5. as those tests are not CPU / memory
intense. With this is mind, let's use a smaller version of the
instance, the D2s_v5 one.
Fixes: #7958
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Without setting the cpu limit / request to 1, we can make this test run
in a smaller VM instance without any issue.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the environment variables are now being passed down from the GitHub
Actions, let's make sure they're exposed to the container used to build
the kata-deploy binaries, and during the build process we'll be able to
use those to log in and push the artefacts to the OCI registry, using
ORAS.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We do the build of our artefacts inside a container image, and we need
to expose some env vars to the container so ORAS can be used there to
push the artefacts we want to cache to ghcr.io.
The env vars we're exposing are:
* ARTEFACT_REGISTRY: The registry where we're going to save the
artefacts.
* ARTEFACT_REGISTRY_USERNAME: The username to log in to the registry, as
ORAS does not use the same json file used by docker.
* ARTEFACT_REGISTRY_PASSWORD: The pasword to log in to the the registry,
as the ORAS does not use the same json file used by docker.
* TARGET_BRANCH: The target branch, which will be part of the tag of the
artefact, as we may end up caching the artefacts for both main and
stable branches.
Fixes: #7834 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need a very recent L2 guest kernel to fix all the bugs that occur in nested
virtualization.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
cloud hypervisor does not emulate pcie switches or pci bridges, so we need to
accept a lonely device.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
It is fine to start a VM with the disk image without syncing it as we now run
the test in an ephemeral Azure instance.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Sometimes the test gets stuck running commands in the container - need to
investigate why later.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Cloud Hypervisor exposes a VIRTIO_IOMMU device to the VM when IOMMU support is
enabled. We need to add it to the whitelist because dragonball uses kernel
v5.10 which restricted VIRTIO_IOMMU to ARM64 only.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
by enabling IOMMU on the default PCI segment. For hotplug to work we need a
virtualized iommu and clh exposes one if there is some device or PCI segment
that requests it. I would have preferred to add a separate PCI segment for
hotplugging vfio devices but unfortunately kata assumes there is only one
segment all over the place. See create_pci_root_bus_path(),
split_vfio_pci_option() and grep for '0000'.
Enabling the IOMMU on the default PCI segment requires passing enabling IOMMU on
every device that is attached to it, which is why it is sprinkled all over the
place.
CLH does not support IOMMU for VirtioFs, so I've added a non IOMMU segment for
that device.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
There is no way for this branch to be hit, as port is only set when it is
different than config.NoPort.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
These test cases shows which options are valid for CLH/Qemu, and test that we
correctly catch unsupported combinations.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
The only supported options are hot_plug_vfio=root-port or no-port.
cold_plug_vfio not supported yet.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
hot_plug_vfio needs to be set to root-port, otherwise attaching vfio devices to
CLH VMs fails. Either cold_plug_vfio or hot_plug_vfio is required, and we have
not implemented support for cold_plug_vfio in CLH yet.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
tdp_mmu had some issues up until around Linux v6.3 that make it work
particularly bad when running nested on Hyper-V. Reload the module at the start
of the test and disable the tdp_mmu param.
Gather debug info at the end of the test to make it easier to figure out what
went wrong. This uses github actions group syntax so that each section can be
collapsed.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
- reduce memory and cpu usage to fit in a D4s_v5
- source correct lib
- mount workspace from 9p
- disable cpu mitigations for speed
- drop unused commands and variables
- install containerd
- install kata from built artifacts
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This imports the vfio test scripts github.com/kata-containers/tests. The test
case doesn't work yet but doing the changes in a separate commit will make it
easier to track the changes. The only change in this commit is renaming
vfio_jenkins_job_build.sh -> vfio_fedora_vm_wrapper.sh
Fixes: #6555
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Otherwise we'll fail to configure kata-containers in the `install-kata`
step.
This is mostly needed because the nerdctl-full tarball doesn't provide a
contaienrd configuration, just the binary, as contaienrd does not
actually require a configuration file to run with the default config.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
TIL that the Azure VMs we use are created without an explicit outbund
connectivity defined.
This leads us to issues using `ping ...` as part of our tests, and when
consulting Jeremi Piotrowski about the issue he pointed me out to two
interesting links:
* https://learn.microsoft.com/en-us/azure/virtual-network/ip-services/default-outbound-access
* https://learn.microsoft.com/en-us/archive/blogs/mast/use-port-pings-instead-of-icmp-to-test-azure-vm-connectivity
For your own sanity, do not read the comments, after all this is
internet. :-)
Anyways, the suggestion is to use nping instead, which is provided by
the nmap package, so we can explicitly switch to using the tcp port 80
for the ping. With this in mind, I'm switching the image we use for the
test and using one that provided nping as a possible entry point, and
from now on (this part of) the tests should work.
Fixes: #7910
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
TIL that the Azure VMs we use are created without an explicit outbund
connectivity defined.
This leads us to issues using `ping ...` as part of our tests, and when
consulting Jeremi Piotrowski about the issue he pointed me out to two
interesting links:
* https://learn.microsoft.com/en-us/azure/virtual-network/ip-services/default-outbound-access
* https://learn.microsoft.com/en-us/archive/blogs/mast/use-port-pings-instead-of-icmp-to-test-azure-vm-connectivity
For your own sanity, do not read the comments, after all this is
internet. :-)
Anyways, the suggestion is to use nping instead, which is provided by
the nmap package, so we can explicitly switch to using the tcp port 80
for the ping. With this in mind, I'm switching the image we use for the
test and using one that provided nping as a possible entry point, and
from now on (this part of) the tests should work.
Fixes: #7910
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In the RemoveEndpoints(), when the endpoints paramete isn't empty,
using idx may result in wrong endpoint removals. To improve,
directly passing the endpoint parameter helps
locate the correct elements within n.eps.
Fixes: #7732
Signed-off-by: shixuanqing <1356292400@qq.com>
Fixes: #7732
Signed-off-by: shixuanqing <1356292400@qq.com>
Update src/runtime/virtcontainers/network_linux.go
Co-authored-by: Xuewei Niu <justxuewei@apache.org>
Fix kernel and initrd annotations in the k8s tests on Mariner. These
annotations must be applied to the spec.template for Deployment, Job
and ReplicationController resources.
Fixes: #7764
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR ensures that docker is running as part of the init_env function
in kata metrics to avoid failures like docker is not running and making
the kata metrics CI to fail.
Fixes#7898
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
FIO test is showing ongoing issues when running in k8s.
Working on running FIO on the ctr client which has been
shown to be stable.
Fixes: #7920
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
There's no reason to wait till the payload is created to run the tests,
as we rely on the tarball, not on the kata-deploy payload.
That was a mistake on my side, and that's already fixed for the nerdctl
tests.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add a very basic sanity test to check that we can spawn a
containers using nerdctl + Kata Containers.
This will ensure that, at least, we don't regress to the point where
this feature doesn't work at all.
In the future, we should also test all the VMMs with devmapper, but
that's for a follow-up PR after this test is working as expected.
Fixes: #7911
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add a very basic sanity test to check that we can spawn a
containers using docker + Kata Containers.
This will ensure that, at least, we don't regress to the point where
this feature doesn't work at all.
For now we're running this test against Cloud Hypervisor and QEMU only,
due to an already reported issue with dragonball:
https://github.com/kata-containers/kata-containers/issues/7912
In the future, we should also test all the VMMs with devmapper, but
that's for a follow-up PR after this test is working as expected.
Fixes: #7910
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
- In rust 1.72, clippy warned clippy::non-minimal-cfg
as the cfg has only one condition, so doesn't
need to be wrapped in the any combinator.
Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Allow `clippy::redundant-closure-call`
which has issues with the guard function passed into
the `run_if_auto_values` macro
Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The bindgen generated code is triggering lots of
ambiguous-glob-reexports warnings in rust 1.70+
Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- In rust 1.72, clippy warned clippy::non-minimal-cfg
as the cfg has only one condition, so doesn't
need to be wrapped in the all combinators.
Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Allow `clippy::redundant-closure-call` in `from_cmdline`
which has issues with the guard function passed into
the `parse_cmdline_param` macro
Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
When creating a new endpoint, we check existing endpoint names and automatically adjust the naming of the new endpoint to ensure uniqueness.
Fixes: #7876
Signed-off-by: shixuanqing <1356292400@qq.com>
This commit allows us to specify the huge page backend when enabling huge
page. Currently, we support two backends: thp and hugetlbfs, the default
is hugetlbfs.
To ensure backward compatibility, we introduce another configuration item
"hugepage_type" to select the memory backend, which is available only when
"enable_hugepages" is true. Besides, we add an annotation
"io.katacontainers.config.hypervisor.hugepage_type" to configure huge page
type per pod.
Fixes: #6703
Signed-off-by: Guixiong Wei <weiguixiong@bytedance.com>
Signed-off-by: Yipeng Yin <yinyipeng@bytedance.com>
As, regardless of what's mentioned in the documentation, it seems that
$GITHUB_REF_NAME is passed down as a literal string.
Fixes: #7414
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The ones for the payload-after-push.yamland ci-nightly.yaml are not that
much important right now, but they're needed for when we start running
those on stable branches as well.
The other ones were missed during
bd24afcf73.
Fixes: #7414
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Now that the metrics migration from the tests to kata containers has been completed, this PR removes the warning from the main metrics documentation.
Fixes#7894
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
To support the removal of the `initcall_debug` and `earlyprintk=`
options from the default guest kernel cmdline, add `kernel_params` to the list
of enabled annotations to allow those kernel options (or others) to be
set using `kata-deploy` for either runtime.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Removed the following kernel command line options:
- `earlyprintk=ttyS0`
- `initcall_debug`
Both these options are only useful when debugging a guest kernel failure
which is not a common occurrence.
Further, the `earlyprintk=` option can have a large negative performance
impact (it can increase the VM boot time significantly).
If the user wishes to use either of these options, they can add them to the
`kernel_params=` setting in the Kata configuration file's hypervisor
stanza.
Fixes: #7886.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
All the patches have already been merged upstream and they've just been
cherry-picked to this branch.
Fixes: #7885
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're bumping here in order to make our lives easier backporting EROFS
patches needed for the CC related work.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Removed the addition of the `initcall_debug` kernel option when agent
debugging enabled. This option has nothing to do with the agent.
If the user wishes to use this option, they can add it to the
`kernel_params=` setting in the Kata configuration file's hypervisor
stanza.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Similarly to what's been done for x86_64 -> amd64, we need to do a
aarch64 -> arm64 change in order to be able to download the kubectl
binary.
Fixes: #7861
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
1. Directly support CgroupManager::freeze through systemd API.
2. Avoid always passing unit_name by storing it into DBusClient.
3. Realize CgroupManager::destroy more accurately by killing systemd unit rather than stop it.
4. Ignore no such unit error when destroying systemd unit.
5. Update zbus version and corresponding interface file.
Acknowledgement: error handling for no such systemd unit error refers to
Fixes: #7080, #7142, #7143, #7166
Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
We're changing what's been done as part of ac939c458c, as we've
notcied issues using `github.event.pull_request.merge_commit_sha`.
Basically, whenever a force-push would happen, the reference of
merge_commit_sha wouldn't be updated, leading us to test PRs with the
old code. :-/
In order to get the rebase properly working, we need to ensure we pull
the hash of the commit as part of checkout action, and ensure
fetch-depth is set to 0.
Fixes: #7414
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This syntax belongs to the legacy C virtiofsd implementation that
we don't support anymore since kata-containers 3.1.3 because
of other API breaking changes.
People have been warned to switch from "none" to "never" since
kata-containers 2.5.2. Let's officially do that.
The compat code that would convert "none" to "never" isn't
needed anymore. Just drop it.
Fixes#7864
Signed-off-by: Greg Kurz <groug@kaod.org>
gogo.nullable is the main gogo.protobuf' feature used here. Since we are
trying to remove gogo.protobuf, the first reasonable step seems to be
remove this feature. This is a core update, and it will change how the
structs are defined. I could spot only a few places using those structs,
based on make check/build.
Fixes#7723.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
There is no reference to PROTO_FILE and this is not working. Also we are
not inside a Makefile, so makes sense to adapt the usage to reflect the
script instead of a make command.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
import_path is used as the default package when no input files specify
go_package. However, all the files we are currently building already
have a go_package definition, making this behavior both redundant and
error-prone.
Additionally, one of our files (types.pb.go) resides outside the grpc
directory, indicating that it's indeed ignored but also inconsistent.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
Currently, the script searches for .proto files within $GOPATH/.
Consequently, modifications to a definition file in the current working
directory won't influence the output .pb.go if the directory is outside
of $GOPATH. For developers, it's more intuitive to alter the local
codebase than the version stored in $GOPATH.
With this modification, the generated .pb.go files will be relative to
the current working directory, removing the need to clone this project
under $GOPATH/src/github.com/kata-containers.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
The definitions are already specified in the .proto files using the
go_package option. Centralizing them in one location reduces the
potential for errors and simplifies the script.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
There's absolutely no need to have the skip check as part of the test
itself when it's already done as part of the setup function.
We're only touching the files here that were touched in the previous
commit.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's keep both checks for now, but in the future we'll be able to
remove the check for "firecracker", as the hypervisor name used as part
of the GitHub Actions has to match what's used as part of the
kata-deploy stuff, which is `fc` (as in `kata-fc for the runtime class)
instead of `firecracker`.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
That's what we've been using as part of Jenkins, so let's ensure things
will work as they did before, and only after that consider upgrading the
base OS used for the tests.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've been using the `kata-deploy-tdx` target as that also uses k3s as
base, but it's better to just have a specific garm target.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
So we have a better control on which flavour of kubernetes kata-deploy
is expected to be targetting.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
GARM runners do not come with the whole set of tools we need, or are
used to when it comes to the GHA runners, so we need to manually install
bats on those.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR changes the order in which the FIO test first
cleans the environment and then checks if the environment
is indeed clean.
Fixes: #7869
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
As we were using `tee` without the `-a` (or `--apend`) aptton, the
containerd config would be overwritten, leading to a NotReady state of
the Node.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's download the vanilla kubectl binary into `/usr/bin/`, as we need
to avoid hitting issues like:
```sh
error: open /etc/rancher/k3s/k3s.yaml.lock: permission denied
```
The issue basically happens because k3s links `/usr/local/bin/kubectl`
to `/usr/local/bin/k3s`, and that does extra stuff that vanilla
`kubectl` doesn't do.
Also, in order to properly use the k3s.yaml config with the vanilla
kubectl, we're copying it to ~/.kube/config.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise the /etc/rancher/k3s/k3s.yaml is not readable by other users
than root.
As --write-config-mode is being passed, and that's an option that has to
be passed to the `server`, -s is also added to the command line.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
`wait` waits for a job to complete, not a number of seconds. Not sure
how I got that wrong in the first place, but it's what it's.
Fixes: #6542
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR replaces the ubuntu image for one which has TensorFlow optimized
for kata metrics.
Fixes#7866
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Currently, the dbs-upcall features have 2 problems that are needed to be
fixed :
There are redundant dbs-upcall features that are needed to be removed.
Some place should be controlled by dbs-upcall but not being implemented.
This commit will fix those two problems.
fixes: #6878
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Let's enable the devmapper kubernetes tests to match exactly what's been
tested as part of the Jenkins CI.
Fixes: #6542
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function right now is completely based on what's part of the tests
repo[0], and that's the reason I'm keeping the `Signed-off-by` of all
the contributors to that file.
This is not perfect, though, as it changes the default snapshotter to
devmapper, instead of only doing so for the Kata Containers specific
runtime handlers. OTOH, this is exactly what we've always been doing as
part of the tests.
We'll improve it, soon enough, when we get to also add a way for
kata-deploy to set up different snapshotters for different handlers.
But, for now, this is as good (or as bad) as it's always been.
It's important to note that the devmapper setup doesn't take into
consideration a BM machine, and this is not suitable for that. We're
really only targetting GHA runners which will be thrown away after the
run is over.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Shiming Zhang <wzshiming@foxmail.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
One can use different kubernetes flavours for getting a kubernetes
cluster up and running.
As part of our CI, though, I really would like to avoid contributors
spending time maintaining and updating kubernetes dependencies, as done
with the tests repo, and which has been proven to be really good on
getting things rotten.
With this in mind, I'm taking the bullet and using "k3s" as the way to
deploy kubernetes for the devmapper related tests, and that's the reason
I'm adding a function to do so, and this will be used later on as part
of this series.
It's important to note that the k3s setup doesn't take into
consideration a BM machine, and this is not suitable for that. We're
really only targetting GHA runners which will be thrown away after the
run is over.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Some use cases may just require passing extra arguments to virtiofsd,
and having this disabled by default makes it impossible to set when
using kata-deploy, as changes in the configuration file would be
overwritten by the daemon-set.
With this in mind, let's allow users to pass whatever thet need (and
here I'm specifically looking at `--xattr`) as a virtio_fs_extra_arg.
Fixes: #7853
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR is to skip installing docker-compose-plugin while buiding a `build-kata-deploy` image for s390x|ppc64le.
It is a temporary solution to fix current CI failures for s390x regarding `hash sum mismatch`.
Fixes: #7848
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This PR adds the write 95 percentile for FIO for qemu for
checkmetrics for kata metrics.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the write 95 percentile FIO value for checkmetrics
for kata metrics.
Fixes#7842
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Previously, static-checks-dragonball is using machines from Alibaba
Cloud to run all the CI jobs.
Currently, we are going through an internal process to apply for the new
machines for Dragonball CI. Before the internal process is over, we will
temporarily use Azure VM to run static-checks-dragonball jobs.
fixes: #7838
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Currently, virtio_vsock are still outside of the device
manager. This causes some management issues,such as the
inability to unify PCI address management.
Just do some work for hybrid vsock.
Fixes: #7655
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Pass --owner and --group to the tar invokation to prevent gihtub runner user
from leaking into release artifacts.
Fixes: #7832
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
When a storage device is used by more than one container, the second
and forth instances will cause storage device reference count leakage,
thus cause storage device leakage. The reason is:
add_storages() will increase reference count of existing storage device,
but forget to add the device to the `mount_list` array, thus leak the
reference count.
Fixes: #7820
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Use AGENT_POLICY=yes when building the Guest images, and add a
permissive test policy to the k8s tests for:
- CBL-Mariner
- SEV
- SNP
- TDX
Also, add an example of policy rejecting ExecProcessRequest.
Fixes: #7667
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Let's make sure we use the TDX image as part of the QEMU TDX
configuration, which will help us to have the policies tested here.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We have two scenarios we care about this, `pull_request` and
`pull_request_target` events triggered a job.
`pull_request` event:
When using the checkout action, it'll already provide a "rebased atop of
main" repo for us, nothing else is needed, and that's basically what we
already have as part of the jobs in our CI.
`pull_request_target` event:
This one is a little bit tricky, as the checkout action, unless passing
a spsecific repo, give us the PR checked out rebased atop of the HEAD of
the PR branch. Jeremi Piotrowski nicely pointed out that we could use
github.event.pull_request.merge_commit_sha instead, which is the result
of the PR's branch with the official repo target branch.
Now, the only cases where the contributor's rebase would still be needed
is when the action itself has been changed.
Fixes: #7414
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR adds the grabdata script so it can be used for the metrics report
for kata metrics.
Fixes#7812
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Update the protection checking code to detect newer versions of Intel
TDX (whose userland interface has now stabilised).
> **Note:** that we don't need to retain the existing behaviour since:
>
> - We haven't yet landed the TDX feature (#6448).
> - Systems wishing to use TDX will need to use the latest available
> system components (such as firmware and host kernel).
Also added an explicit TDX unit test.
Fixes: #7384.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
IoCopy is a tricky function (I don't claim to fully understand its contract),
but here is what I see: The goroutine that runs it spawns 3 goroutines - one
for each stream to handle (stdin/stdout/stderr). The goroutine then waits for
the stream goroutines to exit. The idea is that when the process exits and is
closed, the stdout goroutine will be unblocked and close stdin - this should
unblock the stdin goroutine. The stderr goroutine will exit at the same time as
the stdout goroutine. The iocopy routine then closes all tty.io streams.
The problem is that the stdout goroutine decrements the WaitGroup before
closing the stdin stream, which causes the iocopy goroutine to race to close
the streams. Move the wg.Done() of the stdout routine past the close so that
*this* race becomes impossible. I can't guarantee that this doesn't affect some
unspecified behavior.
Fixes: #5031
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
If we are running FC hypervisor, it is not started when prestart hooks
are executed. So we should just ignore such error and just go ahead and
run the hooks.
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
FC does not support network device hotplug. Let's add a check to fail
early when starting containers created by docker.
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Add a new hypervisor capability to tell if it supports device hotplug.
If not, we should run prestart hooks before starting new VMs as nerdctl
is using the prestart hooks to set up netns. To make nerdctl + FC
to work, we need to run the prestart hooks before starting new VMs.
Fixes: #6384
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
When running on amd machines, those tests will fail because there is no
vmx flag. Following other tests that checks for cpuType, let's adapt
them to restrict vmx only on Intel machines.
Fixes#7788.
Related #5066
Signed-off-by: Beraldo Leal <bleal@redhat.com>
This PR fixes the memory inside limit for clh for kata metrics due
to the recent changes that we had in the script which impacted
in the performance measurement.
Fixes#7786
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's expand the confidential test to also support TDX.
The main difference on the test, though, is that we're not grepping for
a string in the `dmesg` output, but rather relying on `cpuid` to detect
a TDX guest.
Fixes: #7184
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add a test case for the launch of unencrypted confidential
container, verifying that we are running inside a TEE.
Right now the test only works with SEV, but it'll be expanded in the
coming commits, as part of this very same series.
Fixes: #7184
Signed-Off-By: Unmesh Deodhar <udeodhar@amd.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR fixes an issues in the parsing results stage,
by collecting just the n-results from the n-running
containers, discarding irrelevant data.
Fixes: #7774
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
The file can be removed between builds without causing any issue, and
leaving it around has been causing us some headache due to:
```
ERROR: open /home/runner/.docker/buildx/activity/default: permission denied
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The log_forwarder task does not check if the peer has closed, causing a
meaningless loop during the period of “kata vm exit”, when the peer
closed, and “ShutdownContainer RPC received” that aborts the log forwarder.
This patch fixes the problem.
Fixes: #7741
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
Otherwise we'll have to re-run all the tests due to a flaky behaviour in
one of the parts.
Fixes: #7757
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
k8s-pid-ns.bats was already using the test name from
k8s-kill-all-process-in-container.bats - probably a copy/paste bug.
Fixes: #7753
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The directory is a host path mount and cannot be removed from within the
container. What we actually want to remove is whatever is inside that
directory.
This may raise errors like:
```
rm: cannot remove '/opt/kata/': Device or resource busy
```
Fixes: #7746
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The vfio test requires nested-nested virtualization:
L0 Azure host
-> L1 Ubuntu VM
-> L2 Fedora VM
-> L3 Kata
This hits a kernel bug on v5.15 but works quite nicely on the v6.2 kernel
included in Ubuntu 23.04. We can switch back to Ubuntu 22.04 when they roll out
v6.2.
Fixes: #6555
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
There are several processes for container exit:
- Non-detach mode: `Wait` request is sent by containerd, then
`wait_process()` will be called eventually.
- Detach mode: `Wait` request is not sent, the `wait_process()` won’t be
called.
- Killed by ctr: For example, a container runs `tail -f /dev/null`, and
is killed by `sudo ctr t kill -a -s SIGTERM <CID>`. Kill request is
sent, then `kill_process()` will be called. User executes `sudo ctr c
rm <CID>`, `Delete` request is sent, then `delete_process()` will be
called.
- Exited on its own: For example, a container runs `sleep 1s`. The
container’s state goes to `Stopped` after 1 second. User executes
the delete command as below.
Where do we do container cleanup things?
- `wait_process()`: No, because it won’t be called in detach mode.
- `delete_process()`: No, because it depends on when the user executes the
delete command.
- `run_io_wait()`: Yes. A container is considered exited once its IO ended.
And this always be called once a container is launched.
Fixes: #7713
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Refine storage related code by:
- remove the STORAGE_HANDLER_LIST
- define type alias
- move code near to its caller
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Introduce StorageDevice and StorageHandlerManager, which will be used
to refine storage device management for kata-agent.
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Simplify the way to manage storage objects, and introduce
StorageStateCommon structures for coming extensions.
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
docker install now creates a group with gid 999 which happens to match what we
need to get docker-in-docker to work. Remove the group first as we don't need
it.
Fixes: #7726
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
We can simply use `rm -f` all over the place and avoid the container
returning any error.
Fixes: #7733
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Right now if we configure an image annotation and have a config file
setting initrd, the initrd config would override the image annotation.
Make sure annotations are preferred over config options in image and initrd
path handling.
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
We should make sure annotations are preferred over
config options in image and initrd path handling.
Fixes: #7705
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Right now if we configure an image annotation and have a config file
setting initrd, the initrd config would override the image annotation.
Add a helper function ImageOrInitrdAssetPath to make sure annotations
are preferred over config options in image and initrd path handling.
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
When the FileMode field for the device is unset (0), use a default value instead
to allow the use of the device from the container.
This behaviour is seen from cri-o typically.
Note: this is what runc is doing, which is why regular containers don't have an
issue. This change makes sure kata behaves the same as runc.
Fixes: #7717
Signed-off-by: Julien Ropé <jrope@redhat.com>
Introduce structure KataVirtualVolume to to encapsulate information
for extra mount options and direct volumes, so we could build a common
infrastructure to handle these cases.
Fixes: #7699
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
This PR configures the corresponding kata runtime in K8s
based on the tested hypervisor.
This PR also enables FIO metrics test in the kata metrics-ci.
Fixes: #7665
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
When building with AGENT_POLICY=yes and AGENT_INIT=yes:
1. Include OPA and the Policy settings in rootfs.
2. Start OPA from the kata agent.
Before these changes, building with both AGENT_POLICY=yes and
AGENT_INIT=yes was unsupported.
Starting OPA from systemd (when AGENT_INIT=no) was already supported.
Fixes: #7615
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR fixes the check results for tensorflow benchmark now
that we change the name of the test.
Fixes#7684
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Instead of doing this as part of the test itself, let's ensure it's done
before running the tests and during the tests cleanup.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This test, at least for now, only checks whether the runtimeclasses
have been properly created.
This is just a migration from a test we had as part of the k8s suite.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
By doing this we can make sure there won't be any clash on the cluster
name created for either the k8s or the kata-deploy tests.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will have the same function as run-k8s-tests.sh has, but for
kata-deploy.
Right now it doesn't have any tests, and the command to actually run the
tests is commented out, but right now this is just a placeholder that
will be populated sooner than later.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In a follow-up series, we'll add a whole suite for the kata-deploy
tests. With this in mind, let's already get rid of this one and avoid
more kata-deploy tests to land here.
Fixes: #7642
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The default `kata` runtime class would get created with the `kata`
handler instead of `kata-$KATA_HYPERVISOR`. This made Kata use the wrong
hypervisor and broke CI.
Fixes: #7663
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Let's add the tests as part of the ci.yaml, so they an be triggered as
part of each PR.
For this PR those tests won't be triggered, courtesy to the
`pull_request_target` event we rely on.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR fixes the TensorFlow word across the document to have uniformity
across all the document.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The error message when the kill command is executed with the container's
state == Stopped should be "container not running" because the containerd
tests expect that OCI runtimes return the error message and compare it.
If the error message is different from the expected one, the tests fail.
Fixes: #7650
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
As the cri-containerd tests have been fully migrated to GHA, let's make
sure we get them running.
Fixes: #6543
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As part of the runners, we're hitting a timeout that I cannot reproduce,
at all, when allocating the same instance and running the tests
manually.
The default timeout to connect to the server is 2s when using `crictl`.
Let's increase this to 20s.
It's fairly important to mention that in the first tests I used a
timeout of 10s, and that helped but we still hit issues every now and
then.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We extend the `Result` and `Option` types with associated types that
allows converting a `Result<T, E>` and `Option<T>` into
`ttrpc::Result<T>`.
This allows the elimination of many `match` statements in favor of
calling the map function plus the `?` operator. This transformation
simplifies the code.
Fixes: #7624
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
This PR renames the tensorflow scripts to include the data format
that is being used as we will have multiple tests with different
data and model formats for tensorflow so this will help us to
distinguish them.
Fixes#7645
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This will not be tested as part of the PR, thanks to the
`pull_request_target` event, but we want it to be added so we can build
atop of that in a coming up series.
Fixes: #7642
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will not be tested as part of the PR, thanks to the
`pull_request_target` event, but we want it to be added so we can build
atop of that in a coming up series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Fixes: #7573
To enable this feature, build your rootfs using AGENT_POLICY=yes. The
default is AGENT_POLICY=no.
Building rootfs using AGENT_POLICY=yes has the following effects:
1. The kata-opa service gets included in the Guest image.
2. The agent gets built using AGENT_POLICY=yes.
After this patch, the shim calls SetPolicy if and only if a Policy
annotation is attached to the sandbox/pod. When creating a sandbox/pod
that doesn't have an attached Policy annotation:
1. If the agent was built using AGENT_POLICY=yes, the new sandbox uses
the default agent settings, that might include a default Policy too.
2. If the agent was built using AGENT_POLICY=no, the new sandbox is
executed the same way as before this patch.
Any SetPolicy calls from the shim to the agent fail if the agent was
built using AGENT_POLICY=no.
If the agent was built using AGENT_POLICY=yes:
1. The agent reads the contents of a default policy file during sandbox
start-up.
2. The agent then connects to the OPA service on localhost and sends
the default policy to OPA.
3. If the shim calls SetPolicy:
a. The agent checks if SetPolicy is allowed by the current
policy (the current policy is typically the default policy
mentioned above).
b. If SetPolicy is allowed, the agent deletes the current policy
from OPA and replaces it with the new policy it received from
the shim.
A typical new policy from the shim doesn't allow any future SetPolicy
calls.
4. For every agent rpc API call, the agent asks OPA if that call
should be allowed. OPA allows or not a call based on the current
policy, the name of the agent API, and the API call's inputs. The
agent rejects any calls that are rejected by OPA.
When building using AGENT_POLICY_DEBUG=yes, additional Policy logging
gets enabled in the agent. In particular, information about the inputs
for agent rpc API calls is logged in /tmp/policy.txt, on the Guest VM.
These inputs can be useful for investigating API calls that might have
been rejected by the Policy. Examples:
1. Load a failing policy file test1.rego on a different machine:
opa run --server --addr 127.0.0.1:8181 test1.rego
2. Collect the API inputs from Guest's /tmp/policy.txt and test on the
machine where the failing policy has been loaded:
curl -X POST http://localhost:8181/v1/data/agent_policy/CreateContainerRequest \
--data-binary @test1-inputs.json
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Right now this file does nothing, as it's not even called by any GHA.
However, it'll be populated later on as part of a different series,
where we'll have kata-deploy specific tests running here.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's split a good portion of `tests/integration/kuberentes/gha-run.sh`
out, and put them in a place where they can be used to the soon-to-come
kata-deploy specific tests.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Remove the installation step in the virtcontainers doc
because the virtcontainers install/uninstall targets have
been removed by 86723b51ae
and they are not used anymore.
Fixes: #7637
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Add k0s support to kata-deploy, in the very same way kata-containers
already supports k3s, and rke2.
k0s support requires v1.27.1, which is noted as part of the kata-deploy
documentation, as it's the way to use dynamic configuration on
containerd CRI runtimes.
This support will only be part of the `main` branch, as it's not a bug
fix that can be backported to the `stable-3.2` branch, and this is also
noted as part of the documentation.
Fixes: #7548
Signed-off-by: Steve Fan <29133953+stevefan1999-personal@users.noreply.github.com>
This PR fixed the loop that stops the kata-shim and the
hypervisors used in metrics checks.
Fixes: #7628
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
The GHA runners are not exactly powerful, which makes the static-checks
take way too long (almost an hour).
Let's give a try and move those to the same size of Azure instances used
as part of our CI, and probably have this time reduced.
Fixes: #7446
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Remove configuration file shared_fs = none warnings
now that there is a solution to updating configMaps, secrets etc
Fixes: #7210
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This patch allows copying of directories and symlinks when
static file copying is used between host and guest. This change is
necessary to support recursive file copying between shim and agent.
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
(cherry picked from commit de232b8030)
For remote hypervisor, the configmap, secrets, downward-api or project-volumes are
copied from host to guest. This patch watches for changes to the host files
and copies the changes to the guest.
Note that configmap updates takes significantly longer than updates via downward-api.
This is similar across runc and Kata runtimes.
Fixes: #7210
Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Signed-off-by: Julien Ropé <jrope@redhat.com>
(cherry picked from commit 3081cd5f8e)
(cherry picked from commit 68ec673bc4d9cd853eee51b21a0e91fcec149aad)
This PR adds check containers are running in tensorflow mobilenet
that is being defined in common script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the check containers are up function from common
in tensorflow script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the check containers are running function the common metrics
script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the check containers are up in the common script
in the tensorflow mobilenet script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR uses the check containers are up from the common script
in the tensorflow script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR uses the collect results function defined in common for
the tensorflow mobilenet test.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR removes the collect results function from tensorflow script
as it is going to be referenced in the common metrics script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Specify the supported version of Firecracker using our `versions.yaml`
to improve the maintainability of the documentation.
Fixes: #7610
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
This patch upgrades Firecracker version from v1.1.0 to v1.4.0.
* Generate swagger models for v1.4.0 (from `firecracker.yaml`)
- The version of go-swagger used is v0.30.0
* The firecracker v1.4.0 includes the following changes.
- Added
* Added support for custom CPU templates allowing users to adjust vCPU features
exposed to the guest via CPUID, MSRs and ARM registers.
* Introduced V1N1 static CPU template for ARM to represent Neoverse V1 CPU
as Neoverse N1.
* Added support for the virtio-rng entropy device. The device is optional. A
single device can be enabled per VM using the /entropy endpoint.
* Added a cpu-template-helper tool for assisting with creating and managing
custom CPU templates.
- Changed
* Set FDP_EXCPTN_ONLY bit (CPUID.7h.0:EBX[6]) and ZERO_FCS_FDS bit
(CPUID.7h.0:EBX[13]) in Intel's CPUID normalization process.
- Fixed
* Fixed feature flags in T2S CPU template on Intel Ice Lake.
* Fixed CPUID leaf 0xb to be exposed to guests running on AMD host.
* Fixed a performance regression in the jailer logic for closing open file
descriptors.
* A race condition that has been identified between the API thread and the VMM
thread due to a misconfiguration of the api_event_fd.
* Fixed CPUID leaf 0x1 to disable perfmon and debug feature on x86 host.
* Fixed passing through cache information from host in CPUID leaf 0x80000006.
* Fixed the T2S CPU template to set the RRSBA bit of the IA32_ARCH_CAPABILITIES
MSR to 1 in accordance with an Intel microcode update.
* Fixed the T2CL CPU template to pass through the RSBA and RRSBA bits of the
IA32_ARCH_CAPABILITIES MSR from the host in accordance with an Intel microcode
update.
* Fixed passing through cache information from host in CPUID leaf 0x80000005.
* Fixed the T2A CPU template to disable SVM (nested virtualization).
* Fixed the T2A CPU template to set EferLmsleUnsupported bit
(CPUID.80000008h:EBX[20]), which indicates that EFER[LMSLE] is not supported.
Fixes: #7610
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Since the passed fd through unix socket would be any
stream fd such as pipe/fifo fd or any other socket
fd, thus we should deal with it as a normal hybrid
stream instead of a unix stream.
Fixes:#7584
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
This PR computes average results for TF bench.
Additionally, it improves the data parsing from
all running containers.
Fixes: #7603
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
The build context should be the folder where the Dockerfile is present,
otherwise the files copied into the image won't be found.
Fixes: #7595
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Following the example on https://github.com/docker/build-push-action,
it's clear that the actions to "Set up QEMU" and "Set up Docker Buildx"
are missing.
Let's add them, and also take the advantage to bump the
build-push-action to its v4, which, by the way, had a typo on its name
(build-and-push-action does **NOT** exist, build-push-action does).
Fixes: #7595
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Right now this is not being used, but it'll as the image generated for
the confidential tests have that as part of their tag.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
One of the joys to rely on the `pull_request_target` is to only be able
to catch those after those are merged.
Fixes: #7595
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
With these 2 simple checks we can ensure that we do not regress on the
behaviour of allowing the runtime classes / default runtime class to be
created by the kata-deploy payload.
Fixes: #7491
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add here the image we'll be using for unencrypted confidential
tests. Later on, we'll make sure to build and use this image as part of
our CI.
The image can easily be built as a multi-arch image, and has `cpuid`
installed in case of `x86_64` build, so it can be used to detect whether
we're running on a TEE guest without having to rely on `dmesg | grep
...`.
Fixes: #7595
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We don't have to do any sed to replace the runtimeclass being used by
the moment we start taking advantage of the `DEFAULT_SHIM` environment
variable exposed merged in the previous commits.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
There are many places where the code currently creates new `Vec`
instances when it's not really needed. The result is a perf hit because
it allocates memory, copies all elements, then frees the memory; in some
cases, copying elements also involves extra allocations (e.g., when
elements are strings, or structs containing strings).
This patch addresses a number of these cases.
Fixes: #7203
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
Instead of using package manager to install bats, building
this from source. This gives us the updated version of bats
which supports functions such as setup_file and
teardown_file.
We can use these functions into our current tests.
Fixes: #7597
Signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
This PR changes the metrics workflow in order to just install
kata once, and run the checks for multiple hypervisor variations.
In this way we save time avoiding installing kata for each
hypervisor to be tested.
Fixes: #7578
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Refine implementation of mount by:
- log message with `path.display()` instead of `{:?}`
- add prefix "_" to unused variables
- pass by reference instead of by value to avoid creating redundant
array
- exactly matching prefix "fsgid=" instead of "fsgid"
- avoid redundant clone() operations
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
There's a bug in function update_ephemeral_mounts() which only handles
the first storage object and ignores all other storage objects.
Fixes: #7551
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Simplify function online_cpu_memory() by on calling update_cpuset_path()
for containers with cpuset configured.
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Refine style of code related to sandbox by:
- remove unnecessary comments for caller to take lock, we have already taken
`&mut self`.
- change "*count < 1 " to "*count == 0", `count` is type of u32.
- make remove_sandbox_storage() to take `&mut self` instead of `&self`.
- group related function to each others
- avoid search the map twice in function find_process()
- avoid unwrap() in function run_oom_event_monitor()
- avoid unwrap() in online_resources()
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Avoid unwrap() in function do_remove_container(), and also make
implmementation symmetric for both timeout and non-timeout cases.
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Optimize agent rpc implementation by:
- avoid clone objects when possible
- avoid unwrap() when possible
- explictly drop object to ensure order
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
This pull request is mainly for updating vm-memory and vmm-sys-util.
The affacted crates include:
- vm-memory: from 0.9.0 to 0.10.0
- vmm-sys-util: from 0.10.0 to 0.11.0
- virtio-queue: from 0.6.0 to 0.7.0
- fuse-backend-rs: from 0.10.4 to 0.10.5
- linux-loader: from 0.6.0 to 0.8.0
- nydus-api: from 0.3.0 to 0.3.1
- nydus-rafs: from 0.3.1 to 0.3.2
- nydus-storage: from 0.6.3 to 0.6.4
Fixes: #0000
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This PR renames the mobilenet tensorflow test to have a more specific
tensorflow name mainly because tensorflow has different configurations
and we will add more tensorflow tests so we want to distinguish each
tensorflow test.
Fixes#7571
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the iperf network metrics to the github actions
for kata metrics.
Fixes#7535
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
These calls cause two extra atomic instructions each time they're used,
one to increment and another one to decrement the refcount.
Since we don't need them because the referred value is guaranteed to
outlive the function, remove the calls.
Fixes: #7190
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
When the mounted block device isn't a layer, we want to mount it into
containers, but since it's already mounted with the correct fs (e.g.,
tar, ext4, etc.) in the pod, we just bind-mount it into the container.
Fixes: #7536
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
When at least one `io.katacontainers.fs-opt.layer` option is added to
the rootfs, it gets inserted into the VM as a layer, and the file system
is mounted as an overlay of all layers using the overlayfs driver.
Additionally, if the `io.katacontainers.fs-opt.block_device=file` option
is present in a layer, it is mounted as a block device backed by a file
on the host.
Fixes: #7536
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
This causes the overlay-fs driver to add the `upperdir` and `workdir`
options to an overlay-fs mount so that the mount becomes writable using
a discardable directory under the container id.
Fixes: #7536
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
This is so that file systems don't fail when we pass kata-specific
options from the snapshotter to kata.
Fixes: #7536
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
Version 0.10.5, which was just released, breaks `nydus-storage`.
This is a workaround to fix the CI which is blocking other PRs.
Fixes: #7541
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
Allow `clippy::redundant_clone` in the agent's unit tests
because rustc>=1.70 shows the errors as false-negatives.
These `clone()` are required because the following codes
refer to the variable, but the clippy analyzes them by mistake,
using the conservative and limited approach.
Ref. https://rust-lang.github.io/rust-clippy/master/index.html#/redundant_cloneFixes: #7534
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
As 3.2.0-rc0 has been released, let's switch the kata-deploy / kata-cleanup
tags back to "latest", and re-add the kata-deploy-stable and the
kata-cleanup-stable files.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Kata containers as VM-based containers are allowed to run in the host
netns. That is, the network is able to isolate in the L2. The network
performance will benefit from this architecture, which eliminates as many
hops as possible. We called it a Directly Attachable Network (DAN for
short).
The network devices are placed at the host netns by the CNI plugins. The
configs are saved at {dan_conf}/{sandbox_id}.json in the format of JSON,
including device name, type, and network info. At the very beginning stage,
the DAN only supports host tap devices. More devices, like the DPDK, will
be supported in later versions.
The format of file looks like as below:
```json
{
"netns": "/path/to/netns",
"devices": [{
"name": "eth0",
"guest_mac": "xx:xx:xx:xx:xx",
"device": {
"type": "vhost-user",
"path": "/tmp/test",
"queue_num": 1,
"queue_size": 1
},
"network_info": {
"interface": {
"ip_addresses": ["192.168.0.1/24"],
"mtu": 1500,
"ntype": "tuntap",
"flags": 0
},
"routes": [{
"dest": "172.18.0.0/16",
"source": "172.18.0.1",
"gateway": "172.18.31.1",
"scope": 0,
"flags": 0
}],
"neighbors": [{
"ip_address": "192.168.0.3/16",
"device": "",
"state": 0,
"flags": 0,
"hardware_addr": "xx:xx:xx:xx:xx"
}]
}
}]
}
```
Fixes: #1922
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This commit provides a new way to name the containers used
in the launch-times-test in this form:
'kata_launch_times_RANDOM_NUMBER', where RANDOM_NUMBER is
in the 0-1000 range.
Fixes: #7529
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
- ci-on-push: Make the CI also run for the stable-* branches
- ci: k8s: Do not fail when gathering info on AKS nodes
- kata-deploy: enable cross build for non-x86
- runtime-rs: add support for gather metrics in runtime-rs
- kata-ctl: add monitor subcommand for runtime-rs
- release: release-note.sh: Fix typos and reference to images
- metrics: Add sysbench performance test
- Simplify implementation of runtime-rs/service
6ad16d497 release: Adapt kata-deploy for 3.2.0-rc0
025596b28 ci-on-push: Make the CI also run for the stable-* branches
7ffc0c122 static-build: enable cross build for qemu
35d6d86ab static-build: enable cross-build for image build
2205fb9d0 static-build: enable cross build for virtiofsd
11631c681 static-build: enable cross build for shim-v2
7923de899 static-build: cross build kernel
e2c31fce2 kata-deploy: enable cross build for kata deploy script
2fc5f0e2e kata-depoly: prepare env for cross build in lib.sh
f5e9985af release: release-note.sh: Fix typos and reference to images
f910c66d6 ci: k8s: Do not fail when gathering info on AKS nodes
632818176 metrics: Add k8s sysbench documentation
b3901c46d runtime-rs: ignore errors during clean up sandbox resources
5a1b5d367 metrics: Add sysbench pod yaml
ad413d164 metrics: Add sysbench dockerfile
151256011 metrics: Add sysbench performance test
62e328ca5 runtime-rs: refine implementation of TaskService
458e1bc71 runtime-rs: make send_message() as an method of ServiceManager
1cc1c81c9 runtime-rs: fix possibe bug in ServiceManager::run()
1a5f90dc3 runtime-rs: simplify implementation of service crate
731e7c763 kata-ctl: add monitor subcommand for runtime-rs The previous kata-monitor in golang could not communicate with runtime-rs to gather metrics due to different sandbox addresses. This PR adds the subcommand monitor in kata-ctl to gather metrics from runtime-rs and monitor itself.
d74639d8c kata-ctl: provide the global TIMEOUT for creating MgmtClient
02cc4fe9d runtime-rs: add support for gather metrics in runtime-rs
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
kata-deploy files must be adapted to a new release. The cases where it
happens are when the release goes from -> to:
* main -> stable:
* kata-deploy-stable / kata-cleanup-stable: are removed
* stable -> stable:
* kata-deploy / kata-cleanup: bump the release to the new one.
There are no changes when doing an alpha release, as the files on the
"main" branch always point to the "latest" and "stable" tags.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
when interacting with systemd. We have occasionally faced issues with
compatibility between the systemctl version used inside the kata-deploy
container and the systemd version on the host. Instead of using a containerized
systemctl with bind mounted sockets, nsenter the host and run systemctl from
there. This provides less coupling between the kata-deploy container and the
host.
Fixes: #7511
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
As we only support one stable branch, it'll be used as part of the
stable-3.2 and onwards.
Fixes: #7518
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Depends on mutiarch feature of ubuntu, we can set up cross build
environment easily and achive as good build performance as native
build.
Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
It's too long a time to cross build agent based on docker buildx, thus
we cross build rootfs based on a container with cross compile toolchain
of gcc and rust with musl libc. Then we get fast build just like native
build.
rootfs initrd cross build is disabled as no cross compile tolchain for
rust with musl lib if found for alpine and based on docker buildx takes
too long a time.
Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Based on messense/rust-musl-cross which offer cross build musl lib
environment to cross compile virtiofsd.
Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
shim-v2 has go and rust code. For rust code, we use messense/rust-musl-cross
to build for speed up as it doesn't depends on qemu emulation. Build go
code based on docker buildx as it doesn't support cross build now.
Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
kata-deploy-binaries-in-docker.sh is the entry to build kata components.
set some environment to facilitate the following cross build work.
Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
We leverage three env, TARGET_ARCH means the buid target tuple;
ARCH nearly the same meaning with TARGET_ARCH but has been widely
used in kata; CROSS_BUILD means if you want to do cross compile.
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
diferent -> different
And also let's make sure we escape the backticks around the kata-deploy
environment variables, otherwise bash will try to interpret those.
Fixes: #7497
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise the VM deletion may not delete, leaving us with several
machines behind.
Fixes: #7509
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This requires the GITHUB_UPLOAD_TOKEN. While we're here, let's also fix
the name of the action and remove the "-tarball" suffix, as it's not
really a tarball.
Fixes: #7497
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
`stage` has been added, but only hooked up to the amd64 logic, leaving
arm64 and s390x behind.
Let's fix this right now, and make sure no error occurs when passing
this down to the yaml files.
Fixes: #7497
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit 7c857d38c1.
I've misunderstood the error given by github action, let's fix this in
the next commit.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise we'll face the following error as part of our GHA:
```
The workflow is not valid.
kata-containers/kata-containers/.github/workflows/release-$foo.yaml
(Line: 13, Col: 14): Invalid input, stage is not defined in the
referenced workflow.
```
Fixes: #7497
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure we can rely on the tests passing down whether they want
to be tested using Node Feataure Discovery or not.
Right now, only the TDX job has this option set to "true", all the other
jobs have this option set to "false".
We can and have to merge this one before merging the NFD related patches
as:
1) It causes no harm in exporting this environment variable, but not
having it used
2) It will allow us to test the NFD after this one is merged, as changes
in the yaml file, in the case of the pull_request_target event, are
not taken into consideration before they're merged
Fixes: #7495
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
If modeVFIO is enabled we need 1st to attach the VFIO control group
device /dev/vfio/vfio an 2nd the actuall device(s) afterwards.Sort the
devices starting with device #1 being the VFIO control group device and
the next the actuall device(s)
/dev/vfio/<group>
Fixes: #7493
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
- tests: Add `k8s-volume` and `k8s-file-volume` tests to GHA CI
- metrics: Update boot time for kata metrics
- metrics: Add FIO report files for kata metrics
- kata-deploy: Allow runtimeclasses to be created by the daemonset
- runtime-rs: change block index to 0
- agent: fix typo in constant
- metrics: Add FIO benchmark for metrics tests
- gha: dragonball: Run only on the dragonball labeled machine
- tests: Fix `k8s-job` test
- agent,libs: Remove unused 'mut' keywords
- runtime-rs: remove unneeded 'mut' keywords
- tests: QoL improvements for running tests locally
- agent: exclude symlinks from recursive ownership change
- cache: kernel: Fix kernel caching
- runk: Add Docker guide to README
- metrics: General improvements to json.bash script
- kata-deploy: Allow shim creation based on what's passed to the daemonset
- gha: ci: Add skeleton of vfio job
- s390x: Fixing device.Bus assignment
- release: Mention the container images used to build the project
- kata-deploy-binaries: kernel_cache: Take module_dir into account
- ci: nydus: Fix typo in "source"
- gha: ci: Add no-op nydus tests to our CI
- Dragonball: migrate dragonball-sandbox crates to Kata
- ci: gha: Add cri-containerd tests (but still do not enable them)
- packaging/tools: Add kata-debug and use it as part of our CI
- cache: kernel: Consider changes in tools/packaging/kernel
- kata-deploy: Properly get the path of the versions.yaml file
- kata-deploy: Add VERSION and versions.yaml to the final tarball
- metrics: Add C-Ray performance test
- metrics: enable TensorFlow benchmark to be run on gha
- metrics: Add function to memory inside container script
- Revert "metrics: Replace backslashes used to escape double quoted key in jq expr"
- versions: Bump virtiofsd to v1.7.0
- metrics: stop hypervirsor and shim at init_env stage
- ci: k8s: Adapt "source ..." to the new location of gha-run.sh
- ci: Move `tests/integration/gha-run.sh` to `tests/integration/kuberentes/` ... and also remove KUBECONFIG from the tdx envs
- versions: Update kernel to version v6.1.x
- agent: Fix exec hang issues with a backgroud process
- agent: Ignore already mounted dev/fs/pseudo-fs
- ci: k8s: Bring TDX tests back
- metrics: Update machine learning documentation
- gha: ci: cri-containerd: Fix KATA_HYPERVSIOR typo
- tests: Add MobileNet Tensorflow performance benchmark
- metrics: replace backslashes used to escape double quoted jq key expr.
- runtime-rs: enhancement of Device Manager for network endpoints.
- feat(Tracing): tracing in Rust runtime
- runtime-rs: ignore unconfigured network interfaces
- metrics: Stop running kata-env before kata is properly installed.
- metrics: use rm -f to remove the oldest continerd config file.
- kernel: Update kernel config name
- kata-deploy: Add a debug option to kata-deploy (and also use it as part of our CI)
- runtime-rs: add parameter for propagation of (u)mount events
- kata-ctl: Move GuestProtection code to kata-sys-util
- tests: Add function before function name in common.bash for metrics
- tests: Add metrics storage documentation
- metrics: Fix metrics ts generator to treat numbers as decimals
- gha: ci: Add cri-containerd tests skeleton -- follow up 1
- dragonball/agent: Add some optimization for Makefile and bugfixes of unit tests on aarch64
- metrics: Enable blogbench test
- tests: Add machine learning performance tests
- tests: gha: ci: Add cri-containerd tests skeleton
- metrics: Enable memory inside container metrics
- tools: Use a consistent target name when building mariner initrd
- gha: ci: Gather info about the node / pods
- runtime-rs: Do not scan network if network model is "none"
- gha: k8s: tdx: Temporarily disable TDX tests
- metrics: Update memory usage script
- gha: Cancel previous jobs if a PR is updated
- gha: nightly: Fix long name of AKS clusters issue and make the CI easier to test
- README: Add badge for our Nightly CI
- gha: Do not run all the tests if only docs are updated
- bugfix: plus default_memory when calculating mem size
- gha: ci: Use github.sha to get the last commit reference
- dragonball: Don't fail if a request asks for more CPUs than allowed
- gha: ci: Fix refernce passed to checkout@v3
- gha: ci: Avoid using env also in the ci-nightly and payload-after-push
- gha: k8s: Ensure cluster doesn't exist before creating it
- gha: ci: More follow up fixes after adding a nightly CI
- tests: Enable running k8s tests on Mariner
- gha: ci: Avoid using env unless it's really needed
- gha: ci: Follow up fixes for the nightly jobs
- tests: Enable memory usage metrics tests
- gha: Add nightly jobs
- metrics: storing metrics workflow artifacts
- gha: k8s: Ensure tests are running on a specific namespace
- metrics: Adds blogbench and webtool metrics tests
- gha: dragonball: Correctly propagate PATH update
- versions: Upgrade to Cloud Hypervisor v33.0
- Convert `is_allowed`, `ttrpc_error` and `sl` to functions
- gha: release: Use a specific release of hub
- metrics: Add checkmetrics to gha-run.sh for metrics CI
- packaging: Fix indentation of build.sh script at ovmf
- doc: Add documentation for the virtualization reference architecture
- gpu: Update kernel building to the latest changes
- runtime: fix PCIe topology for GPUDirect use-case
- metrics: Add memory footprint tests
- runtime: Add "none" as a shared_fs option
- metrics: Uniformity across function names in gha-run.sh
- runtime-rs: support physical endpoint using device manager
- runtime-rs: bugfix for direct volume path's validation.
- metrics: Fix retrieving hypervisor version on metrics
- runtime-rs: fix build error on AArch64
- checkmetrics: Add checkmetrics makefile and documentation
- docs: Add boot time metrics documentation
- runtime-rs: add support spdk/vhost-user based volume.
- static-build: Remove kata-version parameter
- dragonball: avoid obtaining lock twice in create_stdio_console
- metrics: Add checkmetrics for kata metrics CI
- metrics: enable launch-times test on gha-run metrics script
- docs: Add general metrics documentation
- add support vfio device manager
- gha: Don't automatically trigger CI
- kata-ctl: Check for vm capability
- docs: fix spelling of "crate"
- packaging: Fix indentation in init.sh script
- gha: Fix gha actions
- metrics: install kata and launch-times test
- tests: Move tests helper script to this repo
- tests: Add json script for metrics tests
- Cherry pick initramfs caching updates from CCv0
- gha: Fix format for run launchtimes metrics yaml
- tests: Add tests lib common script
- Fix deprecated virtiofsd args (go shim only)
- gha: Add base branch on SHA on pull requst
- gha: ci-on-push: Run metrics tests
- docs: Update Developer Guide
- runtime-rs: Enhance flexibility of virtio-fs config
- versions: Update firecracker version to 1.3.3
- tools: Fix no-op builds
- runtime-rs: update Cargo.lock
- gha: Fix `stage` definition in matrix
- feat(runtime): vcpu resize capability
- packaging: Remove snap package
- gha: Add new build targets for Mariner
- Dragonball: support resize memory
- Port Measured rootfs feature from CCv0 branch to main
- add support direct volume and refactor device manager
- gha: Fix gha-run.sh and unbreak CI
- kata-ctl: Switch to slog logging; add --log-level and --json-logging arguments
- log-parser: Update log parser link at README
- gha: aks: Extract `run` commands to a script
- runtime-rs: handle copy files when share_fs is not available
- agent-ctl: fix the compile error
- agent: fix the issue of exec hang with a backgroud process
- runtime-rs: bugfix: update Cargo.lock
- gha: aks: Use short SHA in cluster name
- README: Display badge for the "Publish Artefacts" job and update the Kata Containers logo
- kata-deploy: Change how we get the Ubuntu k8s key
- gha: aks: Ensure host_os is used everywhere needed
- kubernetes: add agnhost command in pod yaml
- main | release: Standardize kata static file name
- packaging: make BUILDER_REGISTRY configurable
- gha: aks: Add the host_os as part of the aks cluster's name
- kernel: Modify build-kernel.sh to accomodate for changes in version.yaml
- gha: Fix Mariner cluster creation
- gha: Unbreak CI and fix cluster creation step
- Dragonball: support vcpu hotplug on aarch64
- runtime-rs/sandbox_bindmounts: add support for sandbox bindmounts
- runtime-rs/kata-ctl: Enhancement of DirectVolumeMount.
- gha: Create Mariner host as part of k8s tests
- netlink: Fix the issue of update_interface
- gha: Increase timeout for AKS jobs and give more time to start running the tests
- runtime: sending SIGKILL to qemu
- dragonball: convert BlockDeviceMgr and VirtioNetDeviceMgr functions to methods
- dragonball: Remove virtio-net and vsock devices gracefully
- kata-deploy: Improve shim backup / restore
- doc: Update git commands
- kata-deploy: Fix indentation on kata deploy merge script
8353aae41 ci: k8s: Rework get_nodes_and_pods_info()
6ad5d7112 ci: k8s: Do not gather node info before running the tests
5261e3a60 ci: k8s: Group messages to improve readability
9cc6b5f46 ci: k8s: Get logs from kata-deploy
9d285c622 ci: k8s: Let kata-deploy take care of the runtimeclasses
87568ed98 gha: Test split out runtimeclasses are in sync with all-in-one file
39192c608 kata-deploy: Print variables passed to the script
0e157be6f kata-deploy: Allow runtimeclasses to be created by the daemonset
a27433324 kata-deploy: Change default values of DEBUG
69535b808 kata-deploy: runtimeclass: Split out entries
9e1710674 kata-runtimeClasses: Alphabetically sort the enrties
6222bd910 tests: Add k8s-file-volume test
187a72d38 tests: Add k8s-volume test
0c8427035 metrics: Add boot time value for qemu
6520dfee3 metrics: Update boot time for kata metrics
ff2279061 metrics: Update runtime and configuration paths
a5d4e3388 metrics: Add compare virtiofsd dax script
5e937fa62 metrics: Update general FIO tests
b0bea47c5 metrics: Add makefile to report generator
73c57b9a1 metrics: Add FIO report files for kata metrics
c8fcd29d9 runtime-rs: use device manager to handle virtio-pmem
901c19225 runtime-rs: support configure vm_rootfs_driver
5d6199f9b runtime-rs: use device manager to handle vm rootfs
20f1f62a2 runtime-rs: change block index to 0
662f87539 metrics: Add general FIO makefile
c5a87eed2 tests: gha: Add timeout to cluster creation
6daeb08e6 tests: k8s: Clean up node debuggers after running
3aa6c77a0 gha: dragonball: Run only on the dragonball labeled machine
37641a543 metrics: Add example config for fio jobs
314aec73d agent: fix typo in constant
4703434b1 tests: k8s: Allow using custom resource group
350f3f70b tests: Import `common.bash` in `run_kubernetes_tests.sh`
d7f04a64a tests: k8s: Leave `runtimeclass_workloads/` alone
bdde6aa94 tests: k8s: Split deployment and testing commands
91a0b3b40 tests: aks: Simply delete cluster when cleaning up
3c1044d9d metrics: Update FIO paths for k8s runner
6177a0db3 metrics: Add env files for FIO
a45900324 metrics: Add fio exec
ea198fddc metrics: Add FIO runner k8s
8f7ef41c1 metrics: Add FIO vendor code
6293c17bd metrics: Add FIO benchmark for metrics tests
ff4cfcd8a runk: Add Docker guide to README
c8ac56569 cache: kernel: Harmonize commit with fetching side
81775ab1b cache: kernel: Fix SEV kernel caching
717f775f3 gha: ci: Add skeleton of vfio job
b9f100b39 agent,libs: Remove unused 'mut' keywords
a56f96bb2 kata-deploy: Allow shim creation based on what's passed to the daemonset
4a5ab38f1 metrics: General improvements to json.bash script
d4eba3698 kata-deploy-binaries: kernel_cache: Take module_dir into account
b7c9867d6 release: Mention the container images used to build the project
7c4b59781 ci: nydus: Fix typo in "source"
6a680e241 gha: ci: Add placeholder for the nydus tests as part of the CI
fb4f7a002 gha: nydus: Add a no-op GHA for nydus
4a207a16f gha: nydus: Bring tests as they are from the tests repo
2c8f83424 runtime-rs: remove unneeded 'mut' keywords
1fc715bc6 s390x: Add AP Attach/Detach test
e91f5edba ci: cri-containerd: Fix default typo for testContainerStart()
8b8aef09a ci: cri-containerd: Temporarily disable TestContainerSwap
56767001c ci: cri-containerd: Add namespace / uid to the pods
a84773652 ci: cri-containerd: Always use sudo to call crictl
99ba86a1b ci: cri-containerd: Add /usr/local/go/bin to the PATH
7f3b30999 ci: cri-containerd: Add `function` before each function
fde22d6bc ci: cri-containerd: Assume podman is always used
9465a0496 ci: cri-containerd: Adapt "source ..." to this repo
df8d14411 ci: cri-containerd: Remove CI variable
f90570aef ci: cri-containerd: Remove unused runc_runtime_bin
c3637039f ci: cri-containerd: Remove KILL_VMM_TEST env var
bc4919f9b ci: cri-containerd: Always run shim-v2 tests
f9e332c6d ci: cri-containerd: Stop cloning containerd
cfd662fee ci: cri-containerd: Remove ununsed SNAP_CI var
d36c3395c ci: cri-containerd: Update copyright
b5be8a4a8 ci: cri-containerd: Move integration-tests.sh as it was
f2e00c95c ci: cri-containerd: Populate install_dependencies()
897955252 versions: Add "latest" field for cri-tools
1bbcbafa6 ci: Add clone_cri_container()
f66c68a2b ci: Add install_cri_tools()
4dd828414 ci: Add install_cri_containerd()
ad47d1b9f ci: Add download_github_project_tarball()
788c562a9 ci: Add get_latest_patch_release_from_a_github_project()
6742f3a89 ci: Use `function` before each install_go.sh function
5eacecffc ci: Adjust paths for install_go.sh
8ed1595f9 ci: Update copyright for install_go.sh
6123d0db2 ci: Move install_go.sh as it was
8653be71b ci: Do not take cross-build into consideration for kata-arch.sh
6a76bf92c ci: Fix style / identation if kata-arch.sh
72743851c ci: Add `function` before each kata-arch.sh function
9f6d4892c ci: Update copyright for kata-arch.sh
6f73a7283 ci: Move kata-arch.sh as it was
3615d7343 ci: Add get_from_kata_deps()
34779491e gha: kubernetes: Avoid declaring repo_root_dir
f3738beac tests: Use $HOME/go as fallback for $GOPATH
b87ed2741 tests: Move `ensure_yq` to common.bash
124e39033 tests: common: Fix quoting when globbing
db77c9a43 tests: Make install_kata take care of the links
13715db1f tests: Do not call `install_check_metrics` when installing kata
630634c5d ci: k8s: Group logs to make them easier to read
228b30f31 ci: k8s: Gather node info during the cleanup
81f99543e ci: k8s: Cleanup cluster before deleting it
38a7b5325 packaging/tools: Add kata-debug
ae6e8d2b3 kata-deploy: Properly get the path of the versions.yaml file
309e23255 cache: kernel: Consider changes in tools/packaging/kernel
59fdd69b8 kata-deploy: Add VERSION and versions.yaml to the final tarball
5dddd7c5d release: Upload versions.yaml as part of the release
bad3ac84b metrics: Rename C-Ray to cpu performance tests
87d99a71e versions: Remove "kernel-experimental"
545de5042 vfio: Fix tests
62aa6750e vfio: Added better handling of VFIO Control Devices
dd422ccb6 vfio: Remove obsolete HotplugVFIOonRootBus
114542e2b s390x: Fixing device.Bus assignment
371a118ad agent: exclude symlinks from recursive ownership change
e64edf41e metrics: Add tensorflow function in gha-run script
67a6fff4f metrics: Enable tensorflow benchmark on gha
01450deb6 Revert "metrics: Replace backslashes used to escape double quoted key in jq expr."
843006805 metrics: Add function to memory inside container script
bbd3c1b6a Dragonball: migrate dragonball-sandbox crates to Kata
fad801d0f ci: k8s: Adapt "source ..." to the new location of gha-run.sh
55e2f0955 metrics: stop hypervirsor and shim at init_env stage
556e663fc metrics: Add disk link to general metrics README
98c121709 metrics: Add C-Ray README
8e7d9926e metrics: Add C-Ray Dockerfile
e2ee76978 metrics: Add C-Ray performance test
2ee2cd307 ci: k8s: Move gha-run.sh to the kubernetes dir
88eaff533 ci: tdx: Adjust KUBECONFIG
c09e268a1 versions: Downgrade SEV(-SNP) kernel back to v5.19.x
6a7a32365 versions: Bump virtiofsd to v1.7.0
ac5f5353b ci: k8s: Bring TDX tests back
950b89ffa versions: Update kernel to version v6.1.38
8ccc1e5c9 metrics: Update machine learning documentation
f50d2b066 gha: ci: cri-containerd: Fix KATA_HYPERVSIOR typo
620b94597 metrics: Add Tensorflow Mobilenet documentation
6c91af0a2 agent: Fix exec hang issues with a backgroud process
59f4731bb metrics: Stop running kata-env before kata is properly installed.
468f017e2 metrics: Replace backslashes used to escape double quoted key in jq expr.
64f013f3b ci: k8s: Enable debug when running the tests
8f4b1df9c kata-deploy: Give users the ability to run it on DEBUG mode
2c8dfde16 kernel: Update kernel config name
150e54d02 runtime-rs: ignore unconfigured network interfaces
3ae02f920 metrics: use rm -f to remove older continerd config file.
a864d0e34 tests: Add tensorflow mobilenet dockerfile
788d2a254 tests: Add tensorflow mobilenet performance test
3fed61e7a tests: Add storage link to general metrics documentation
b34dda4ca tests: Add storage blogbench metrics documentation
6787c6390 runtime-rs: add parameter for propagation of (u)mount events
6e5679bc4 tests: Add function before function name in common.bash for metrics
62080f83c kata-sys-util: Fix compilation errors
02d99caf6 static-checks: Make cargo clippy pass.
982420682 agent: Make the static checks pass for agent
61e4032b0 kata-ctl: Remove all utility functions to get platform protection
a24dbdc78 kata-sys-util: Move utilities to get platform protection
dacdf7c28 kata-ctl: Remove cpu related functions from kata-ctl
f5d195717 kata-sys-util: Move additional functionality to cpu.rs
304b9d914 kata-sys-util: Move CPU info functions
7319cff77 ci: cri-containerd: Add LTS / Active versions for containerd
2a957d41c ci: cri-containerd: Export GOPATH
75a294b74 ci: cri-containerd: Ensure deps are installed
6924d14df metrics: Fix metrics ts generator to treat numbers as decimals
9e048c8ee checkmetrics: Add blogbench read value for qemu
2935aeb7d checkmetrics: Add blogbench write value for qemu
02031e29a checkmetrics: Add blogbench read value for clh
107fae033 checkmetrics: Add blogbench write value for clh
8c75c2f4b metrics: Update blogbench Dockerfile
49723a9ec metrics: Add double quotes to variables
dc67d902e metrics: Enable blogbench test
438fe3b82 gha: ci: Add cri-containerd tests skeleton
bd08d745f tests: metrics: Move metrics specific function to metrics gha-run.sh
3ffd48bc1 tests: common: Move a few utility functions to common.bash
7f961461b tests: Add machine learning README
bb2ef4ca3 tests: Add `function` before each function
063f7aa7c tests: Add Pytorch Dockerfile
1af03b9b3 tests: Add Pytorch performance test
4cecd6237 tests: Add tensorflow Dockerfile
c4094f62c tests: Add metrics machine learning performance tests
89b622dcb gha: k8s: tdx: Temporarily disable TDX tests
8c9d08e87 gha: ci: Gather info about the node / pods
283f809dd runtime-rs: Enhancing Device Manager for network endpoints.
a65291ad7 agent: rustjail: update test_mknod_dev
46b81dd7d agent: clippy: fix cargo clippy warnings
c4771d9e8 agent: Makefile: enable set SECCOMP dynamically
a88212e2c utils.mk: update BUILD_TYPE argument
883b4db38 dragonball: fix cargo test on aarch64
6822029c8 runtime-rs: Do not scan network if network model is "none"
ce54e43eb metrics: Update memory usage script
fbc2a91ab gha: Cancel previous jobs if a PR is updated
307cfc8f7 tools: Use a consistent target name when building mariner initrd
d780cc08f gha: nightly: Also use `workflow_dispatch` to trigger it
b99ff3026 gha: nightly: Fix name size limit for AKS
aedc586e1 dragonball: Makefile: add coverage target
310e069f7 checkmetrics: Enable checkmetrics for memory inside test
1363fbbf1 README: Add badge for our Nightly CI
1776b18fa gha: Do not run all the tests if only docs are updated
28c29b248 bugfix: plus default_memory when calculating mem size
0c1cbd01d gha: ci: after-push: Use github.sha to get the last commit reference
37a955678 gha: ci: nightly: Use github.sha to get the last commit reference
ed23b47c7 tracing: Add tracing to runtime-rs
96e9374d4 dragonball: Don't fail if a request asks for more CPUs than allowed
38f0aaa51 Revert "gha: k8s: dragonball: Skip k8s-number-cpus"
828a72183 gha: k8s: dragonball: Skip k8s-oom
a79505b66 gha: k8s: dragonball: Skip k8s-number-cpus
275c84e7b Revert "agent: fix the issue of exec hang with a backgroud process"
2be342023 checkmetrics: Add memory usage inside container value for qemu
6ca34f949 checkmetrics: Add memory inside container value for clh
6c6892423 metrics: Enable memory inside container metrics
0ad298895 gha: ci: Fix refernce passed to checkout@v3
86904909a gha: ci: Avoid using env also in the ci-nightly and payload-after-push
f72cb2fc1 agent: Remove shadowed function, add slog-term
1d05b9cc7 gha: ci: Pass down secrets to ci-on-push / ci-nightly
c5b4164cb gha: ci: Fix tarball-suffix passed to the metrics tests
07810bf71 agent: Ignore already mounted dev/fs/pseudo-fs
11e3ccfa4 gha: ci: Avoid using env unless it's really needed
c45f646b9 gha: k8s: Ensure cluster doesn't exist before creating it
1a7bbcd39 gha: ci: Fix typo pull_requesst -> pull_request
ddf4afb96 gha: ci: Fix set-fake-pr-number job
8a0a66655 gha: ci: schedule expects a list, not a map
5c0269dc5 gha: ci: Add pr-number input to the correct job
de83cd9de gha: ci: Use $VAR instead of ${{ env.VAR }}
6acce83e1 metrics: Fix the call to check_metrics function
e067d1833 gha: Add a nightly CI job
7c0de8703 gha: k8s: Ensure tests are running on a specific namespace
106e30571 gha: Create a re-usable `ci.yaml` file
cc3993d86 gha: Pass event specific info from the caller workflow
4e396e728 metrics: Add function keyword to to helper metrics functions
1ca17c2f7 metrics: storing metrics workflow artifacts
5a61065ab checkmetrics: Add checkmetrics value for memory usage in qemu
78086ed1f checkmetrics: Add memory usage value for clh
1c3dbafbf metrics: Fix function of how to retrieve multiple values
18968f428 metrics: Add function to have uniformity
35d096b60 metrics: Adds blogbench and webtool metrics tests
d8f90e89d metrics: Rename function at memory usage script
b9d66e0d5 metrics: Fix double quotes variables in memory usage script
476a11194 tests: Enable memory usage metrics tests
b568c7f7d tests/integration: Provide default value for KATA_HOST_OS
d6e96ea06 tests/integration: Use AzureLinux instead of Mariner
40c46c75e tests/integration: Perform yq install in run_tests()
d8b8f7e94 metrics: Enable launch tests time metrics
72fd562bd gha: release: Use a specific release of hub
0502354b4 checkmetrics: Add checkmetrics json for qemu
b481ef188 makefile: Add -buildvcs=false flag to go build
e94aaed3c ci_worker: Add checkmetrics ci worker for cloud hypervisor
917576e6f metrics: Add double quotes in all variables
cc8f0a24e metrics: Add checkmetrics to gha-run.sh for metrics CI
477856c1e gha: dragonball: Correctly propagate PATH update
1c211cd73 gha: Swap asset/release in build matrix
0152c9aba tools: Introduce `USE_CACHE` environment variable
2b5975689 tests: Build CLH with glibc for Mariner
80c78eadc tests: Use baked-in kernel with Mariner
532755ce3 tests: Build Mariner rootfs initrd
6a21e20c6 runtime: Add "none" as a shared_fs option
5681caad5 versions: Upgrade to Cloud Hypervisor v33.0
b2ce8b4d6 metrics: Add memory footprint tests to the CI
d035955ef doc: Add documentation for the virtualization reference architecture
0f454d0c0 gpu: Fixing typos for PCIe topology changes
6bb2ea819 packaging: Fix indentation of build.sh script at ovmf
0504bd725 agent: convert the `sl` macros to functions
0860fbd41 agent: convert the `ttrpc_error` macro to a function
0e5d6ce6d agent: convert the `is_allowed` macro to a function
f680fc52b agent: change `AGENT_CONFIG`'s lazy type to just `AgentConfig`
beb706368 metrics: Uniformity across function names
1f3e837e4 runtime-rs: fix build error on AArch64
6fd25968c runtime-rs: bugfix for direct volume path's validation.
415578cf3 docs: Add general README
bff4672f7 runtime-rs: support physical endpoint using device manager
32cba7e44 metrics: Fix retrieving hypervisor version on metrics
aa7946de4 checkmetrics: Add general checkmetrics documentation
2fac2b72f checkmetrics: Add checkmetrics makefile
e45899ae0 docs: Add time tests documentation reference
28130d3ce docs: Add boot time metrics documentation
0df2fc270 runtime-rs: add support spdk/vhost-user based volume.
17198089e vendor: Add vendor checkmetrics dependencies
f1dfea6e8 docs: Add metrics documentation reference
8330fb8ee gpu: Update unit tests
859359424 metrics: enable launch-times test on gha-run metrics script
c4ee601bf metrics: Add checkmetrics for kata metrics CI
e0d6475b4 gha: Don't automatically trigger CI
b535c7cbd tests: Enable running k8s tests on Mariner
71071bdb6 docs: Add general metrics documentation
610f7986e check: Relax the unrestricted_guest check when running in a VM
1b406b9d0 kata-ctl:Implement functionality to check host is capable of running VM
adf88eaa8 static-build: Remove kata-version parameter
09720babc docs: fix spelling of "crate"
7185afc50 gha: Fix gha actions
21294b868 packaging: Fix indentation in init.sh script
fad3ac9f5 metrics: install kata and launch-times test
4bbfcfaf1 tests: Move tests helper script to this repo
f152f0e8c metrics: Add launch-times to metrics tests
59510cfee runtime-rs: add support vfio device based volume
1e3b372bb runtime-rs: add support vfio device manager
6b0848930 gha: Fix format for run launchtimes metrics yaml
3cefa43e7 tests: Add json script for metrics tests
6a3710055 initramfs: Build dependencies as part of the Dockerfile
aa2380fdd packaging: Add infra to push the initramfs builder image
1c7fcc6cb packaging: Use existing image to build the initramfs
a43ea24df virtiofsd: Convert legacy `-o` sub-options to their `--` replacement
8e00dc694 virtiofsd: Drop `-o no_posix_lock`
2a15ad978 virtiofsd: Stop using deprecated `-f` option
c3043a6c6 tests: Add tests lib common script
b16e0de73 gha: Add base branch on SHA on pull requst
72f2cb84e gpu: Reset cold or hot plug after overriding
fbacc0964 gpu: PCIe topology, consider vhost-user-block in Virt
bc152b114 gha: ci-on-push: Run metrics tests
dad731d5c docs: Update Developer Guide
b11246c3a gpu: Various fixes for virt machine type
40101ea7d vfio: Added annotation for hot(cold) plug
8f0d4e261 vfio: Cleanup of Cold and Hot Plug
b5c4677e0 vfio: Rearrange the bus assignemnt
b1aa8c8a2 gpu: Moved the PCIe configs to drivers
55a66eb7f gpu: Add config to TOML
da42801c3 gpu: Add config settings tests for hot-plug
de39fb7d3 runtime: Add support for GPUDirect and GPUDirect RDMA PCIe topology
9318e022a gpu: Add CC relates configs
b7932be4b gpu: Add Arm64 Kernel Settings
211b0ab26 gpu: Update Kernel Config
5f103003d gpu: Update kernel building to the latest changes
35e4938e8 tools: Fix no-op builds
347385b4e runtime-rs: Enhance flexibility of virtio-fs config
21d227853 versions: Update firecracker version to 1.3.3
0e2379909 gha: Fix `stage` definition in matrix
ae2cfa826 doc: add vcpu handlint doc for runtime-rs
7b1e67819 fix(clippy): fix clippy error
67972ec48 feat(runtime-rs): calculate initial size
aaa96c749 feat(runtime-rs): modify onlineCpuMemRequest
d66f7572d feat(runtime-rs): clear cpuset in runtime side
a0385e138 feat(runtime-rs): update linux resource when stop_process
a39e1e6cd feat(runtime-rs): merge the update_cgroups in update_linux_resources
fa6dff9f7 feat(runtime-rs): support vcpu resizing on runtime side
8cb4238b4 packaging: Remove snap package
213773998 runtime-rs: update Cargo.lock
56d2ea9b7 kata-ctl: Refactor kernel module check
9f7a45996 gha: Add `rootfs-initrd-mariner` build target
f28a62164 gha: Add `cloud-hypervisor-glibc` build target
8fb7ab751 dragonball: introduce virtio-balloon device
7ed949497 dragonball: introduce virtio-mem device
776a15e09 runtime-rs: add support direct volume.
a8e0f51c5 dragonball: extend DeviceOpContext
abae11404 runtime-rs: refactor device manager implementation
210a15794 dragonball: avoid obtaining lock twice in create_stdio_console
69668ce87 tests: gha-run: Use correct env variable for repo
f487199ed gha: aks: Fix argument in call to gha-run.sh
f6afae9c7 packaging: Add rootfs-image-tdx-tarball target
f62b2670c config: Add root hash value and measure config to kernel params
008058807 kernel: Integrate initramfs into Guest kernel
28b264562 initramfs: Add build script to generate initramfs
5cb02a806 image-build: generate root hash as an separate partition for rootfs
31c0ad207 packaging: Add cryptsetup support in Guest kernel and rootfs
980d084f4 log-parser: Update log parser link at README
410bc1814 agent-ctl: fix the compile error
77519fd12 kata-ctl: Switch to slog logging; add --log-level, --json-logging args
aab603096 gha: aks: Extract `run` commands to a script
e4eb664d2 runtime-rs: update rust to 1.69.0
ed37715e0 runtime-rs: handle copy files when share_fs is not available
5f6fc3ed7 runtime-rs: bugfix: update Cargo.lock
1c6d22c80 gha: aks: Use short SHA in cluster name
3c1f6d36d readme: Update Kata Containers logo
388684113 readme: Add status badge for the "Publish Artefacts" job
26f752038 kata-deploy: Change how we get the Ubuntu k8s key
aebd3b47d gha: aks: Ensure host_os is used everywhere needed
0c8282c22 gha: aks: Add the host_os as part of the aks cluster's name
4b89a6bda release: Standardize kata static file name
9228815ad kernel: Modify build-kernel.sh to accomodate for changes in version.yaml
03027a739 gha: Fix Mariner cluster creation
43e73bdef packaging: make BUILDER_REGISTRY configurable
ffe3157a4 dragonball: add arm64 patches for upcall
560442e6e dragonball: add vcpu_boot_onlined vector
e31772cfe dragonball: add support resize_vcpu on aarch64
64c764c14 dragonball: update dbs-boot to v0.4.0
fd9b41464 dragonball: update comment for init_microvm
af16d3fca gha: Unbreak CI and fix cluster creation step
5ddc4f94c runtime-rs/kata-ctl: Enhancement of DirectVolumeMount.
25d2fb0fd agent: fix the issue of exec hang with a backgroud process
4af4ced1a gha: Create Mariner host as part of k8s tests
eee7aae71 runtime-rs/sandbox_bindmounts: add support for sandbox bindmounts
557b84081 gha: aks: Wait longer to start running the tests
c04c872c4 gha: aks: Increase the timeout time
428041624 kata-deploy: Improve shim backup / restore
14c3f1e9f kata-deploy: Fix indentation on kata deploy merge script
0e47cfc4c runtime: sending SIGKILL to qemu
6a0035e41 doc: Update git commands
433b5add4 kubernetes: add agnhost command in pod yaml
c477ac551 dragonball: Convert VirtioNetDeviceMgr function to method
4659facb7 dragonball: Convert BlockDeviceMgr function to method
ee6deef09 dragonball: Remove virtio-net and vsock devices gracefully
2bda92fac netlink: Fix the issue of update_interface
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The links for pods and kubelets no longer work so update to new links
with relevant info.
Fixes#7487
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Multiple instances of task service may get registered by
ServiceManager::run(), fix it by making operation symmetric.
Fixes: #7479
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
This PR rounds the axelnet and resnet results in order to extract
properly the result.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR will avoid to have the strconv.atoi parsing error when we
are retrieving the results from the json.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR moves the checkmetrics to gha-run script to gathered
tensorflow information.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The previous kata-monitor in golang could not communicate with runtime-rs
to gather metrics due to different sandbox addresses.
This PR adds the subcommand monitor in kata-ctl to gather metrics from
runtime-rs and monitor itself.
Fixes: #5017
Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
Several functions in kata-ctl need to establish a connection with runtime-rs through MgmtClient.
This PR provides a global TIMEOUT to avoid multiple definitions.
Fixes: #5017
Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
1. Implemented metrics collection for runtime-rs shim and dragonball hypervisor.
2. Described the current supported metrics in runtime-rs.(docs/design/kata-metrics-in-runtime-rs.md)
Fixes: #5017
Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
The amount of info we've added seemed unnecessary, and ends up making
our lives even harder when trying to find errors.
Let's just rely on the kata-debug container to collect the needed info
for us.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It's been proven to not be useful, and ends up making things more
confusing due to the amount of logs printed.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure we can debug kata-deploy in case something goes wrong
during its execution.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is needed in order to not lose track of what's been created and
what's been added here and there.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will help folks to debug / understand what's been passed to the
kata-deploy.sh script.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's allow the daemonset to create the runtimeclasses, which will
decrease one manual step a user of kata-deploy should take, and also
help us in the Confidential Containers land as the Operator can just
delegate it to this script.
Fixes: #7409
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This can be easily done as there was no official release with the
previous values.
The reason we're doing so is because when using `yq` to replace the
value, even when forcing `--tag '!!str' "yes"`, the content is placed
without quotes, causing errors in our CI.
While here, we're also removing the fallback value for DEBUG, as it is
**always** set in the kata-deploy.yaml file.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will make things simpler to only create the handlers defined by the
kata-deploy user.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will become handy in the near future, as we want to have separate
enrties for each file, while still keeping this one.
Having the entries sorted will make our lives easier to test those are
always in sync.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This imports the k8s-file-volume test from the tests repo and modifies
it slightly to set up the host volume on the AKS host.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This imports the k8s-volume test from the tests repo and modifies it
slightly to set up the host volume on the AKS host.
Fixes: #6566
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This deletes node debugger pods after execution since their presence may
affect tests that assume only test workloads pods are present.
For example, in `k8s-job` we wait for *any* pod to be in the `Succeeded`
state before proceeding, which causes failures.
Fixes: #7452
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Static checks for dragonball are landing on any of the self-hosted
runners, and the reason for that is because "self-hosted" was the label
selector used.
Let's use "dragonball" instead, as the machine has that label as well.
Fixes: #7464
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This simply allows setting a custom resource group when debugging
locally, so as to prevent name collisions and not pollute the namespace.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Makes it so that `setup.sh` doesn't make changes in
`runtimeclass_workloads/` directly. Instead we treat that as a template
directory and we use the new directory `runtimeclass_workloads_work/` as
a work dir.
This has two advantages:
* Allows rerunning tests without the assumption that `setup.sh` must be
idempotent. E.g. the `set_runtime_class()` step would break.
* Doesn't pollute your git environment with a bunch of changes when
developing.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This splits deploying Kata and running the tests into separate commands
to make it possible to rerun tests locally without having to redeploy
Kata each time.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This PR adds the FIO benchmark scripts and resources for the metrics
tests section.
Fixes#7441
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
kata-deploy-binaries.sh uses the last commit in
tools/packaging/static-build/kernel for its version check, while the cache
generation uses tools/packaging/kernel. Use tools/packaging/static-build/kernel
as $kata_config_version is already part of the version string and covers any
changes to tools/packaging/kernel.
Fixes: #7403
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
The SEV kernel cache calls create_cache_asset() twice, once for the kernel and
once for modules. Both calls need to use the same version string, otherwise the
second call overwrites the "latest" file of the first one and the cache is not
used.
Fixes: #7403
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This job will run on a nested virt capable Azure VM (improving test
concurrency). This is just a placeholder while we adapt the test to GHA.
Fixes: #6555
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Remove unused `mut` because the agent compilation fails
when the rust compiler is >= 1.71. This is related to #7425Fixes: #7438
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Instead of hardcoding shims as part of the script, let's ensure we can
allow them to be created based on environment variables passed to the
daemonset.
This change brings no functionality change as the default values in the
daemonset are exactly what has been used as part of the scripts.
Fixes: #7407
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR adds general improvements like putting function before function
name and consistency in how we declare variables and so on to have
uniformity across the metrics scripts.
Fixes#7429
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
`module_dir` has been passed to the function but was never assigned to a
var, leading to errors when trying to use it.
Fixes: #7416
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We should source from `nydus_dir`, instead of `cri_containerd_dir`, and
that was a leftover from fb4f7a002c.
Fixes: #6543
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will triger the nydus tests, but as they currently are they'll just
return "okay" without actually executing.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This newly added GHA does nothing, is not even triggered, and it's just
a placeholder that we'll grow in the next commits / PRs, so we can
actually start running the nydus tests as part of our CI.
Fixes: #6543
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Now that we have propper AP device support add a
unit test for testing the correct Attach/Detach of AP devices.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Otherwise crictl will fail to remove them with:
```
getting sandbox status of pod "$pod": metadata.Name, metadata.Namespace
or metadata.Uid is not in metadata "..."
```
A huge shout out to Steven Horsman for helping to debug this one.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
For this set of tests, we'll always be using podman in order to avoid
having containerd pulled in by docker.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We don't need the env var, we just need to restrict the test according
to the KATA_HYPERVISOR used, as right now it's very specifict to QEMU.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We only have shim-v2 as the runtime type, so we always need to run tests
using it. :-)
We had to adjust the script in order to properly run the tests with the
current logic.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's move the `integration/containerd/cri/integration-tests.sh` file
from the tests repo to this one.
The file has been moved as it is, it's not used, and in the following
commits we'll clean it up before actually using it.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's install all the dependencies needed for running the
`cri-containerd` tests.
The list of dependencies we have are:
* From the system
- build-essential
- jq
- podman-docker
* From our own repo
- yq
- go
* From GitHub projects
- containerd
- cri-tools
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we don't want to disrupt what we have on the `tests` repo, let's
create a "latest" entry and use that for the GitHub actions tests.
Once we deprecate the `tests` repo we can decide whether we want to
stick to using "latest" or switch back to "version".
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function will simply clone containerd repo, specifically on a tag
we want to use to test.
This can be expanded for different projects, and it will be the case as
soon as we grow the tests. But, for now, let's keep it simple.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function will install cri-tools in the host, and soon enough (as
part of this PR) we'll be using it to install cri-tools as part of the
cri-containerd tests.
I've decided to have this as part of the `common.bash` as other tests
that will be added in the future will require cri-tools to be installed
as well.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function will install cri-containerd in the host, and soon enough
(as part of this PR) we'll be using it to install cri-containerd as part
of the cri-containerd tests.
I've decided to have this as part of the `common.bash` as other tests
that will be added in the future will require cri-containerd to be
installed as well.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function will hel us to get the tarball, from a github project,
that we're going to use as part of our tests.
Right now this is not used anywhere, but it'll soon enough (as part of
this series) be used to download the cri-containerd / cri-tools / cni
tarballs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function will help us to get the latest patch release from a
GitHub project.
The idea behind this function is that we don't have to keep updating
versions.yaml that frequently (or worse, have it outdated as it
currently is), and always test against the latest patch release of a
given project's version that we care about.
Although right now this is not used anywhere, this will be used with the
coming cri-containerd tests, which will be part of this series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's adjust paths for what we source and the scripts we call, after
moving from the tests repo to this one.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's move `.ci/install_go.sh` file from the tests repo to this one.
The file has been moved as it is, it's not used, and in the following
commits we'll clean it up before actually using it.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Right now we'd need to import lib.sh just in order to get cross-build
information for rust, and it seems a little bit premature to do so at
this stage and only for rust.
Let's skip it and keep this transition simple.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's move `.ci/kata-arch.sh` file from the tests repo to this one.
The file has been moved as it is, it's not used, and in the following
commits we'll clean it up before actually using it.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
First of all, I'm 100% aware that I'm duplicating this function here as
I've copied it from the packaging stuff, and I'm not exactly proud of
that.
However, right now it seems a little bit premature to combine that set
of scripts with this set of scripts in a single one and make them used
by both pieces of our project.
Anyways, this functions helps to get information from the
`versions.yaml` file, and it'll be used as part of the cri-containerd
tests and a few others in the future.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is already declared as part of the `common.bash` file, so let's
just make sure we use it from there.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Considering that someone may want to run the tests locally, we shouldn't
rely on having GITHUB_WORKSPACE exported, and fallback to $HOME/go if
needed.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When the glob star is inside quotes, there is only one iteration of the loop
and b holds all matches at once. Move the glob out of the quotes so that we
actually iterate over matched paths.
Fixes: #6543
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
The `install_kata` function was moved from the metrics' `gha-run.sh`
file to the `common.bash` in the commit 3ffd48bc16, but I didn't notice
that it brought with it a call to `install_check_metrics`, which is
totally unrelated to installing Kata Containers.
Let's remove the call so the function is a little bit less specific, and
move the call to install_check_metrics to the metrics `gha-run.sh` file.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will help us to in two fronts:
* catching possible issues related to kata-deploy cleanup
* do more (like, in the future, collect logs) after the tests run
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
kata-debug is a tool that is used as part of the Kata Containers CI to gather
information from the node, in order to help debugging issues with Kata
Containers.
As one can imagine, this can be expanded and used outside of the CI context,
and any contribution back to the script is very much welcome.
The resulting container is stored at the [Kata Containers quay.io
space](https://quay.io/repository/kata-containers/kata-debug) and can
be used as shown below:
```sh
kubectl debug $NODE_NAME -it --image=quay.io/kata-containers/kata-debug:latest
```
Fixes: #7397
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need to correctly get the full path of the versions.yaml file as part
of the merge-builds.sh script, as we do a `pushd` there and that leads
to a fail merging the artefacts as the `versions.yaml` file does not
exists in that path.
Fixes: #7405
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Any change in the script used to build the kernel should invalidate the
cache.
Fixes: #7403
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make things simpler to figure out which version of Kata
Containers has been deployed, and also which artefacts come with it.
This will help us immensely in the future, for the TEEs use case, so we
can easily know whether we can deploy a specific guest kernel for a
specific host kernel.
Fixes: #7394
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Although this file is far away from being a SBOM, it'll help folks to
easily visualise which components are part of a release, and even have
SBOMs generated from that.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've not been using nor shipping this kernel for a very long time.
Regardless, we're leaving behind the logic in the kernel scripts to
build it, in case it becomes necessary in the future.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Removing HotplugVFIOonRootBus which is obsolete with the latest PCI
topology changes, users can set cold_plug_vfio or hot_plug_vfio either
in the configuration.toml or via annotations.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The device.Bus was reset if a specific combination of
configuration parameters were not met. With the new
PCIe topology this should not happen anymore
Fixes: #7381
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
currently when fsGroup is used with direct-assign, kata agent
recursively changes ownership and permission for each file including
symlinks. However the problem with symlinks is, the permission of
the symlink itself may not be same as the underlying file. So while
doing recursive ownership and permission changes we should skip
symlinks.
Fixes: #7364
Signed-off-by: Alakesh Haloi <a_haloi@apple.com>
This PR adds the tensorflow function in gha-run script in order to
be triggered in the gha.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds function before function of the variables at the memory
inside container script in order to have uniformity across the script.
Fixes#7386
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
In order to make it easier for developers to contribute to Dragonball,
we decide to migrate all dragonball-sandbox crates to Kata.
fixes: #7262
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
This PR kills the hypervisor and the kata shim in the
init_env stage prior to launch any metric test.
Additionally this PR adds info messages in the main blocks
of the blogbench test to help in debugging.
Fixes: #7366
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR adds C-Ray performance test in order to be part of the kata
metrics CI.
Fixes#7375
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
We don't need to export KUBECONFIG there. Let's just make sure we have
the server correctly setup and avoid doing that.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
CC-GPU seems to have issues with v6.1, so downgrade the kernels used for
SEV-SNP to a known-working version. It is worth mentioning that TDX is also
still on 5.19.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Now that we have a new TDX machine plugged into our CI, let's re-enable
the TDX tests.
Fixes: #7368
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Kernel v6.1.38 is the current latest LTS version, switch to it. No
patches should be necessary. Some CONFIG options have been removed:
- CONFIG_MEMCG_SWAP is covered by CONFIG_SWAP and CONFIG_MEMCG
- CONFIG_ARCH_RANDOM is unconditionally compiled in
- CONFIG_ARM64_CRYPTO is covered by CONFIG_CRYPTO and ARCH=arm64
Fixes: #6086
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This PR updates the machine learning documentation related with
Tensorflow and Pytorch benchmarks.
Fixes#7359
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the Tensorflow mobilinet documentation for the machine
learning README.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Allow runk to launch a container even though users don't specify the
pid namespace in `config.json` because general container runtimes
such as runc also can launch a container without the namespace.
On the other hand, Kata Containers doesn't allow it due to security issue
so this feature should be enabled in only runk.
Fixes: #7168
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Issue #4747 and pull request #4748 fix exec hang issues where the exec
command hangs when a process's stdout is not closed. However, the PR might
cause the exec command not to work as expected, leading to CI failure. The
PR was reverted in #7042. This PR resolves the exec hang issues and has
undergone 1000 rounds of testing to verify that it would not cause any CI
failures.
Fixes: #4747
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This PR makes kata-env is called only after some metrics have
completed his workload. This fixes a bug that occurs when
kata-env was being called before kata is already installed on the
testing platform.
Fixes: #7348
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR uses squared brackets in a jq expression to access
key values corresponding to metric results in json format.
The values are the data inputs into the checkmetrics tool.
Fixes: #7319
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This will help us to gather more information about Kata Containers in
case of failure.
Fixes: #7343
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The DEBUG env var introduced to the kata-deploy / kata-cleanup yaml file
will be responsible for:
* Setting up the CRI Engine to run with the debug log level set to debug
* The default is usually info
* Setting up Kata Containers to enable:
* debug logs
* debug console
* agent logs
This will help a lot folks trying to debug Kata Containers while using
kata-deploy, and also help us to always run with DEBUG=yes as part of
our CI.
Fixes: #7342
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Fixes: #7294
When installing the kernel config adjust the name like
the vmlinuz and vmlinux files so that any added suffixes
are also reflected in the kernel config name.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
In order to run kata metrics we need to check that the containerd
config file is properly set. When this is not the case, we
need to remove that file, and generate a valid one.
This PR runs rm -f in order to ignore errors in case the
file to delete does not exist.
Fixes: #7336
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR adds the storage metrics documentation for blogbench for kata
metrics.
Fixes#7329
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Add an extra parameter in `bind_mount_unchecked` to specify
the propagation type: "shared" or "slave".
Fixes: #7017
Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
This PR adds function before the function name in common.bash script
in order to have uniformity across all the script.
Fixes#7327
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Since these have been added to kata-sys-util, remove these from
kata-ctl. Change all invocations to get platform protection to make use
of kata-sys-util.
Fixes: #7144
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Remove cpu related functions which have been moved to kata-sys-util.
Change invocations in kata-ctl to make use of functions now moved to
kata-sys-util.
Signed-off-by: Nathan Whyte <nathanwhyte35@gmail.com>
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Make certain imports architecture specific as these are not used on all
architectures.
Move additional constants and functionality to cpu.rs.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Move get_single_cpu_info and get_cpu_flags into kata-sys-util.
Add new functions that get a list of flags and check if a flag
exists in that list.
Fixes#6383
Signed-off-by: Nathan Whyte <nathanwhyte35@gmail.com>
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
As we'll be testing against the LTS and the Active versions of
containers, let's add those entries to the versions.yaml file and make
sure we export what we want to use for the tests as an env var.
The approach taken should not break the current way of getting the
containerd version.
LTS and Active versions of containerd can be found at:
https://containerd.io/releases/#support-horizonFixes: #6543
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure this is exported, as it'll be needed in order to install
`yq`, which will be used to get the versions of the dependencies to be
installed.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure we install the needed dependencies for running the
`cri-containerd` tests.
Right now this commit is basically adding a placeholder, and later on,
when we'll actually be able to test the job, we'll add the logic of
installing the needed dependencies.
The obvious dependencies we've spotted so far are:
* From the OS
* jq
* curl (already present)
* From our repo
* yq (using the install_yq script)
* From GitHub
* cri-containerd
* cri-tools
* cni plugins
We may need a few more packages, but we will only figure this out as
part of the actual work.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Use bc tool to perform math operations even when variables contain
values with leading zero.
Fixes: #7317
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR adds double quotes to variables in the blogbench script to
have uniformity across all the tests.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR builds the foundation for us to start migrating the
cri-containerd tests from Jenkins to GitHub Actions.
Right now the test does nothing and should always finish successfully.
The coming PRs will actually introduce logic to the `gha-run.sh` script
where we'll be able to run the tests and make sure those pass before
having them actually merged.
Fixes: #6543
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Those functions were originally introduced as part of the
`metrics/gha-run.sh` file, but those will be very hand at the time we
start adding more tests.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
TDX tests need to be temporarily disabled as the current machine
allocated for this will be off for some time, and a new machine only
will be added next week.
Fixes: #7307
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Currently, network endpoints are separate from the device manager
and need to be included for proper management. In order to do so,
we need to refactor the implementation of the network endpoints.
The first step is to restructure the NetworkConfig and NetworkDevice
structures.
Next, we will implement the virtio-net driver and add the Network
device to the Device Manager.
Finally, we'll unify entries with do_handle_device for each endpoint.
Fixes: #7215
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
When running cargo test in container, test_mknod_dev may fail sometimes
because of "Operation not permitted". Change the device path to
"/dev/fifo-test" to avoid this case.
Fixes: #7284
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
1. Update memory end assert because address space layout differs between
x86 and arm.
2. Set guest_addr for aarch64 in test_handler_insert_region case.
Fixes: #7284
TODO: #7290
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
This PR updates memory usage script by applying the clean_env_ctr at the main
in order to avoid failures of leaving certain processes not removed.
Fixes#7302
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Currently a mixture of cbl-mariner and mariner is used when creating the
mariner initrd. The kata-static tarball has mariner in the name, but the
jenkins url uses cbl-mariner. This breaks cache usage.
Use mariner as the target name throughout the build, so that caching works.
Fixes: #7292
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This is a very nice suggestion from Steve Horsman, as with that we can
manually trigger the workflow anytime we need to test it, instead of
waiting for a full day for it to be retriggered via the `schedule`
event.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Passing the commit hash as the "pr-number" has shown problematic as it
would make the AKS cluster name longer than what's accepted by AKS.
One easy way to solve this is just passing "nightly" as the PR number,
as that's only used to create the cluster.
Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will help folks to monitor the history of the failing tests, as
we've done in Jenkins with the "Green Effort CI".
Fixes: #7279
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We should not go through the trouble of running all our tests on AKS /
Azure / baremetal machines in case a PR only changes our documentation.
Fixes: #7258
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've noticed this caused regressions with the k8s-oom tests, and then
decided to take a step back and do this in the same way it was done
before 67972ec48a.
Moreover, this step back is also more reasonable in terms of the
controlling logic.
And by doing this we can re-enable the k8s-oom.bats tests, which is done
as part of this PR.
Fixes: #7271
Depends-on: github.com/kata-containers/tests#5705
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
As we need to pass down the commit sha to the jobs that will be
triggered from the `push` event, we must be careful on what exactly
we're using there.
At first we were using ${{ github.ref }}, but this turns out to be the
**branch name**, rather than the commit hash. In order to actually get
the commit hash, Let's use ${{ github.sha }} instead.
Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we need to pass down the commit sha to the jobs that will be
triggered from the `schedule` event, we must be careful on what exactly
we're using there.
At first we were using ${{ github.ref }}, but this turns out to be the
**branch name**, rather than the commit hash. In order to actually get
the commit hash, Let's use ${{ github.sha }} instead, as described by
https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's take the same approach of the go runtime, instead, and allocate
the maximum allowed number of vcpus instead.
Fixes: #7270
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit 25d2fb0fde.
The reason we're reverting the commit is because it to check whether
it's the cause for the regression on devmapper tests.
Fixes: #7253
Depends-on: github.com/kata-containers/tests#5705
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
On cc3993d860 we introduced a regression,
where we started passing inputs.commit-hash, instead of
github.event.pull_request.head.sha. However, we have been setting
commit-hash to github.event.pull_request.sha, meaning that we're mssing
a `.head.` there.
github.event.pull_request.sha is empty for the pull_request_target
event, leading the CI to pull the content from `main` instead of the
content from the PR.
Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The latter workflow is breaking as it doesn't recognise ${GITHUB_REF},
the former would most likely break as well, but it didn't get triggered
yet.
The error we're facing is:
```
Determining the checkout info
/usr/bin/git branch --list --remote origin/${GITHUB_REF}
/usr/bin/git tag --list ${GITHUB_REF}
Error: A branch or tag with the name '${GITHUB_REF}' could not be found
```
Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Remove shadowed get_mounts(), added slog-term as a new crate,
slog can directly log to stdout and we can capture output
in the test-cases that are created in the function to be tested.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
We have to do this, otherwise we cannot log into azure.
This is a regression introduced by
106e305717.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Instead of passing "-${{ inputs.tag }}-amd64", we must only pass
"-${{ inputs.tag }}".
This is a regression introduced by
106e305717.
Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Using an initrd and setting KATA_INIT=yes meaning we're using the kata-agent
as the init process we need to make sure that the agent is not segfaulting
if mounts are already happened. Some workloads need to configure several
things in the initrd before the kata-agent starts which involves having
/proc or /sys already mounted.
Fixes: #6992
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
de83cd9de7 tried to solve an issue, but it
clearly seems that I'm using env wrongly, as what ended up being passed
as input was "$VAR", instead of the content of the VAR variable.
As we can simply avoid using those here, let's do it and save us a
headache.
Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It has to have steps declared, and we need to make it a dependency for
the nightly kata-containers-ci-on-push job.
Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise we'll get the following error from the workflow:
```
The workflow is not valid. .github/workflows/ci-on-push.yaml (Line: 24,
Col: 20): Unrecognized named-value: 'env'. Located at position 1 within
expression: env.COMMIT_HASH .github/workflows/ci-on-push.yaml (Line: 25,
Col: 18): Unrecognized named-value: 'env'. Located at position 1 within
expression: env.PR_NUMBER
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR fixes the call to check_metrics function as KATA_HYPERVISOR
is not needed to be passed.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The idea is to mimic what's been done with Jenkins and the "Green CI"
effort, but now using our GHA and the GHA infrastructure.
Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure we run our tests in a specific namespace, as in case of
any kind of issue, we will just get rid of the namespace itself, which
will take care of cleaning up any leftover from failing tests.
One important thing to mention is why we can get rid of the `namespace:
${namespace}` on the tests that are already using it, and let's do it in
parts:
* namespace: default
We can easily get rid of this as that's the default namespace where
pods are created, so it was a no-op so far.
* namespace: test-quota-ns
My understanding is that we'd need this in order to get a clean
namespace where we'd be setting a quota for. Doing this in the
namespace that's only used for tests should **not** cause any
side-effect on the tests, as we're running those in serial and there's
no other pods running on the `kata-containers-k8s-tests` namespace
Last but not least, we're not dynamically creating namespaces as the
tests are not running in parallel, **never**, not in the case of having
2 tests being ran at same time, neither in the case of having 2 jobs
being scheduled to the same machine.
Fixes: #6864
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is based on the `ci-on-push.yaml` file, and it's called from ther
The reason to split on a new file is that we can easily introduce a
`ci-nightly.yaml` file and re-use the `ci.yaml` file there as well.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's ensure we're not relying, on any of the called workflows, on event
specific information.
Right now, the two information we've been relying on are:
* PR number, coming from github.event.pull_request.number
* Commit hash, coming from github.event.pull_request.head.sha
As we want to, in the future, add nightly jobs, which will be triggered
by a different event (thus, having different fields populated), we
should ensure that those are not used unless it's in the "top action"
that's trigerred by the event.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Use the 'function' keyword to prevent bash aliases from colliding
with other function's name.
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR enables storing metrics workflow artifacts in two
separated flavours: clh and qemu.
Fixes: #7239
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR adds the function name before the function to have uniformity
across all the test.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds blogbench and webtooling metrics checks to this repo.
The function running the test intentionally returns zero, so
the test will be enabled in another PR once the workflow is
green.
Fixes: #7069
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR usses double quotes in all the variables as well as general fixes
to the memory usage script in order to have uniformity.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Non AKS k8s tests (SEV/SNP/TDX) don't currently set KATA_HOST_OS, so provide a
default empty value for the variable so that those tests can run.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
as OSSKU value, to get rid of this warning when creating the AKS cluster:
WARNING: The osSKU "AzureLinux" should be used going forward instead of
"CBLMariner" or "Mariner". The osSKUs "CBLMariner" and "Mariner" will
eventually be deprecated.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
We only need to install in run_tests() so that the yq install is picked up by
kubernets/setup.sh as well. We also need to either use (sudo &&
INSTALL_IN_GOPATH=false) || (INSTALL_IN_GOPATH=true).
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This PR adds the checkmetrics ci worker file for cloud hypervisor in
order to check the boot times limit.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds double quotes in all variables to have uniformity across
all the gha-run.sh script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds checkmetrics installation for gha-run.sh in order to compare
results limits as part of the metrics CI.
Fixes#7198
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
cargo/rust is installed in one step, we need to write the PATH update to
GITHUBENV so that it becomes visible in the next steps.
Fixes: #7221
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This simply displays the asset name first in GH's UI, so that the
release name (always "test") is truncated rather than the asset name.
Makes things slightly easier to read.
e.g.
build-asset (cloud-hypervisor-glibc, te...
instead of
build-asset (test, cloud-hypervisor-gli...
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This allows setting `USE_CACHE=no` to test building e2e during
developmet without having to comment code blocks and so forth.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This enables building CLH with glibc and the mshv feature as required
for Mariner. At test time, it also configures Kata to use that CLH
flavor when running Mariner.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Mariner ships a bleeding-edge kernel that might be ahead of upstream, so
we use that to guarantee compatibility with the host.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
* Adds a new `rootfs-initrd-mariner` build target.
* Sets the custom initrd path via annotation in `setup.sh` at test
time.
* Adapts versions.yaml to specify a `cbl-mariner` initrd variant.
* Introduces env variable `HOST_OS` at deploy time to enable using a
custom initrd.
* Refactors the image builder so that its caller specifies the desired
guest OS.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Currently, even when using devmapper, if the VMM supports virtio-fs /
virtio-9p, that's used to share a few files between the host and the
guest.
This *needed*, as we need to share with the guest contents like secrets,
certificates, and configurations, via Kubernetes objects like configMaps
or secrets, and those are rotated and must be updated into the guest
whenever the rotation happens.
However, there are still use-cases users can live with just copying
those files into the guest at the pod creation time, and for those
there's absolutely no need to have a shared filesystem process running
with no extra obvious benefit, consuming memory and even increasing the
attack surface used by Kata Containers.
For the case mentioned above, we should allow users, making it very
clear which limitations it'll bring, to run Kata Containers with
devmapper without actually having to use a shared file system, which is
already the approach taken when using Firecracker as the VMM.
Fixes: #7207
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR adds memory foot print metrics to tests/metrics/density
folder.
Intentionally, each test exits w/ zero in all test cases to ensure
that tests would be green when added, and will be enabled in a
subsequent PR.
A workflow matrix was added to define hypervisor variation on
each job, in order to run them sequentially.
The launch-times test was updated to make use of the matrix
environment variables.
Fixes: #7066
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
There is nothing in them that requires them to be macros. Converting
them to functions allows for better error messages.
Fixes: #7201
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
There is nothing in it that requires it to be a macro. Converting it to
a function allows for better error messages.
Fixes: #7201
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
Having a function allows for better error messages from the type checker
and it makes it clearer to callers what can happen. For example:
is_allowed!(req);
Gives no indication that it may result in an early return, and no simple
way for callers to modify the behaviour. It also makes it look like
ownership of `req` is being transferred.
On the other hand,
is_allowed(&req)?;
Indicates that `req` is being borrowed (immutably) and may fail. The
question mark indicates that the caller wants an early return on
failure.
Fixes: #7201
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
Since it is never modified, it doesn't really need a lock of any kind.
Removing the `RwLock` wrapper allows us to remove all `.read().await`
calls when accessing it.
Additionally, `AGENT_CONFIG` already has a static lifetime, so there is
no need to wrap it in a ref-counted heap allocation.
Fixes: #5409
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
This PR adds the word function before the function names in order to have
uniformity across the script as some are using this and some are not.
Fixes#7196
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Vfio support introduce build error on AArch64. Remove arch related
annotation can avoid this error.
Fixes: #7187
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
The failure mainly caused by the encoded volume path and
the mount/src. As the src will be validated with stat,but
it's not a full path and encoded, which causes the stat
mount source failed.
Fixes: #7186
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This PR adds link to the unreference docs in the cmd path to make
them more discoverable.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds checkmetrics makefile which is used to process the
metrics json results files.
Fixes#7172
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds time tests documentation reference in the general README
for kata metrics.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Unlike the previous usage which requires creating
/dev/xxx by mknod on the host, the new approach will
fully utilize the DirectVolume-related usage method,
and pass the spdk controller to vmm.
And a user guide about using the spdk volume when run
a kata-containers. it can be found in docs/how-to.
Fixes: #6526
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This PR adds the metrics documentation as a general reference in the
main README for kata containers.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the checkmetrics scripts that will be used for the kata metrics CI.
Fixes#7160
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
We have GH configured so that manual approval is required for CI runs
triggered by outside contributors. However, because CI is triggered by
the `pull_request_target` event, this setting isn't being honored
(see [1]). This means that an attacker could trivially extracts secrets
by submitting a PR.
This change aims to mititgate this issue by preventing PRs from
triggering CI unless the `ok-to-test` label is set.
Note: For further context, we use the `pull_request_target` event and
manually check out the PR branch because it is the only way to both
access secrets and test incoming code changes.
Fixes: #7163
[1]: https://docs.github.com/en/actions/managing-workflow-runs/approving-workflow-runs-from-public-forks
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
When running on a VM, the kernel parameter "unrestricted_guest" for
kernel module "kvm_intel" is not required. So, return success when running
on a VM without checking value of this kernel parameter.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Implement functionality to add to the env output if the host is capable
of running a VM.
Fixes: #6727
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This PR removes an unrecognized value located in one of the yamls for the
gha in order to make it work the CI again.
Fixes#7149
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR replaces single spaces for tabs in order to fix the indentation
in the init.sh script.
Fixes#7147
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR installs kata static tarball on metrics runner
and run launch-times tests.
Fixes: #7049
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
The common.sh script includes helper functions used in
our metrics tests, so we are gradually adding more
metrics used in kata.
Fixes: #7108
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This test measures the duration of a workload that starts, and then
immediately stops the contianer. Also measures the workload period,
the time to quit period, and the time to kernel period.
Fixes: #7049
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
A new choice of using vfio devic based volume for kata-containers.
With the help of kata-ctl direct-volume, users are able to add a
specified device which is BDF or IOMMU group ID.
To help users to use it smoothly, A doc about howto added in
docs/how-to/how-to-run-kata-containers-with-kinds-of-Block-Volumes.
Fixes: #6525
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Limitations:
As no ready rust vmm's vfio manager is ready, it only supports
part of vfio in runtime-rs. And the left part is to call vmm
interfaces related to vfio add/remove.
So when vmm/vfio manager ready, a new PR will be pushed to
narrow the gap.
Fixes: #6525
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This PR fixes the format for the run launchtimes metrics yaml which
is causing to the workflow to fail.
Fixes#7130
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the json script which allow us to save the metrics results
into a json file which will be used in the kata containers metrics.
Fixes#7128
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This will help to not have to build those on every CI run, and rather
take advantage of the cached image.
Fixes: #7084
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit c720869eef)
Let's add the needed infra for only building and pushing the initramfs
builder image to the Kata Containers' quay.io registry.
Fixes: #7084
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 111ad87828)
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder for the initramds.
This will save us some CI time.
Fixes: #7084
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit ebf6c83839)
The `-o` option is the legacy way to configure virtiofsd, inherited
from the C implementation. The rust implementation honours it for
compatibility but it logs deprecation warnings.
Let's use the replacement options in the go shim code. Also drop
references to `-o` from the configuration TOML file.
Fixes#7111
Signed-off-by: Greg Kurz <groug@kaod.org>
The C implementation of virtiofsd had some kind of limited support
for remote POSIX locks that was causing some workflows to fail with
kata. Commit 432f9bea6e hard coded `-o no_posix_lock` in order
to enforce guest local POSIX locks and avoid the issues.
We've switched to the rust implementation of virtiofsd since then,
but it emits a warning about `-o` being deprecated.
According to https://gitlab.com/virtio-fs/virtiofsd/-/issues/53 :
The C implementation of the daemon has limited support for
remote POSIX locks, restricted exclusively to non-blocking
operations. We tried to implement the same level of
functionality in #2, but we finally decided against it because,
in practice most applications will fail if non-blocking
operations aren't supported.
Implementing support for non-blocking isn't trivial and will
probably require extending the kernel interface before we can
even start working on the daemon side.
There is thus no justification to pass `-o no_posix_lock` anymore.
Signed-off-by: Greg Kurz <groug@kaod.org>
The rust implementation of virtiofsd always runs foreground and
spits a deprecation warning when `-f` is passed.
Signed-off-by: Greg Kurz <groug@kaod.org>
This PR adds the test lib common script that is going to be used
for kata containers metrics.
Fixes#7113
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The run-launchtimes-metrics workflow needs to get the commit ID
for the last commit to the head branch of the PR.
Fixes: #7116
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
If we override the cold, hot plug with an annotation
we need to reset the other plugging mechanism to NoPort
otherwise both will be enabled.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
In Virt the vhost-user-block is an PCIe device so
we need to make sure to consider it as well. We're keeping
track of vhost-user-block devices and deduce the correct
amount of PCIe root ports.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This gh-workflow prints a simple msg, but is the base for future
PRs that will gradually add the jobs corresponding to the kata
metrics test.
Fixes: #7100
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR updates the developer guide at the connect to the debug console
section.
Fixes#7094
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Now it is possible to configure the PCIe topology via annotations
and addded a simple test, checking for Invalid and RootPort
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Removed the configuration of PCIeRootPort and PCIeSwitchPort, those
values can be deduced in createPCIeTopology
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Refactor the bus assignment so that the call to GetAllVFIODevicesFromIOMMUGroup
can be used by any module without affecting the topology.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The hypervisor_state file was the wrong location for the PCIe Port
settings, moved everything under device umbrella, where it can be
consumed more easily and we do not get into circular deps.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
For the GPU CC use case we need to set several crypto algorithms.
The driver relies on them in the CC case.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Use now the sev.conf rather then the snp.conf.
Devices can be prestend in two different way in the
container (1) as vfio devices /dev/vfio/<num>
(2) the device is managed by whataever driver in
the VM kernel claims it.
Fixes: #6844
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This fixes the builds of `cloud-hypervisor-glibc` and
`rootfs-initrd-mariner` to properly create the `build/` directory.
Fixes: #7098
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This PR updates the firecracker version to 1.3.3 which includes the following
changes
Fixed passing through cache information from host in CPUID leaf 0x80000006.
A race condition that has been identified between the API thread and the VMM
thread due to a misconfiguration of the api_event_fd.
Fixes#7089
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Kubernetes and Containerd will help calculate the Sandbox Size and pass it to
Kata Containers through annotations.
In order to accommodate this favorable change and be compatible with the past,
we have implemented the handling of the number of vCPUs in runtime-rs. This is
This is slightly different from the original runtime-go design.
This doc introduce how we handle vCPU size in runtime-rs.
Fixes: #5030
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
In this commit, we refactored the logic of static resource management.
We defined the sandbox size calculated from PodSandbox's annotation and
SingleContainer's spec as initial size, which will always be the sandbox
size when booting the VM.
The configuration static_sandbox_resource_mgmt controls whether we will
modify the sandbox size in the following container operation.
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
Some vmms, such as dragonball, will actively help us
perform online cpu operations when doing cpu hotplug.
Under the old onlineCpuMem interface, it is difficult
to adapt to this situation.
So we modify the semantics of nb_cpus in onlineCpuMemRequest.
In the original semantics, nb_cpus represents the number of
newly added CPUs that need to be online. The modified
semantics become that the number of online CPUs in the guest
needs to be guaranteed.
Fixes: #5030
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
The declaration of the cpu number in the cpuset is greater
than the actual number of vcpus, which will cause an error when
updating the cgroup in the guest.
This problem is difficult to solve, so we temporarily clean up
the cpuset in the container spec before passing in the agent.
Fixes: #5030
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
Updating vCPU resources and memory resources of the sandbox and
updating cgroups on the host will always happening together, and
they are all updated based on the linux resources declarations of
all the containers.
So we merge update_cgroups into the update_linux_resources, so we
can better manage the resources allocated to one pod in the host.
Fixes: #5030
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
Support vcpu resizing on runtime side:
1. Calculate vcpu numbers in resource_manager using all the containers'
linux_resources in the spec.
2. Call the hypervisor(vmm) to do the vcpu resize.
3. Call the agent to online vcpus.
Fixes: #5030
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Nobody has volunteered to maintain the (currently broken) snap build, so
remove it.
Fixes: #6769.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
After we support memory resize in Dragonball, we need to update
Cargo.lock in runtime-rs.
Fixes: #6719
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
Adding vhost and vhost-net to the kernel modules. These do not require
any kernel module parameters to be checked. Currently, kernel params is
a required field. Make this as optional. Could make this as <Option>,
but making this a slice instead, as a module could have multiple kernel
params. Refactor the function that checks are for kernel modules into
two with one specifically checking if the module is loaded and other
checking for module parameters.
Refactor some of the tests to take into account these changes.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This adds the glibc flavor of CLH to the list of assets as preparation
for #6839. Mariner Kata is only tested with glibc.
Fixes: #7026
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
We introduce virtio-balloon device to support memory resize.
virtio-balloon device could reclaim memory from guest to host.
Fixes: #6719
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
We introduce virtio-mem device to support memory resize. virtio-mem
device could hot-plug more memory blocks to guest and could also
hot-unplug them from guest.
Fixes: #6719
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
As block/direct volume use similar steps of device adding,
so making full use of block volume code is a better way to
handle direct volume.
the only different point is that direct volume will use
DirectVolume and get_volume_mount_info to parse mountinfo.json
from the direct volume path. That's to say, direct volume needs
the help of `kata-ctl direct-volume ...`.
Details seen at Advanced Topics:
[How to run Kata Containers with kinds of Block Volumes]
docs/how-to/how-to-run-kata-containers-with-kinds-of-Block-Volumes.md
Fixes: #5656
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
In order to support virtio-mem and virtio-balloon devices, we need to
extend DeviceOpContext with VmConfigInfo and InstanceInfo.
Fixes: #6719
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
The key aspects of the DM implementation refactoring as below:
1. reduce duplicated code
Many scenarios have similar steps when adding devices. so to reduce
duplicated code, we should create a common method abstracted and use
it in various scenarios.
do_handle_device:
(1) new_device with DeviceConfig and return device_id;
(2) try_add_device with device_id and do really add device;
(3) return device info of device's info;
2. return full info of Device Trait get_device_info
replace the original type DeviceConfig with full info DeviceType.
3. refactor find_device method.
Fixes: #5656
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
After we have a guest kernel with builtin initramfs which
provide the rootfs measurement capability and Kata rootfs
image with hash device, we need set related root hash value
and measure config to the kernel params in kata configuration file.
Fixes: #6674
Signed-off-by: Wang, Arron <arron.wang@intel.com>
Integrate initramfs into guest kernel as one binary,
which will be measured by the firmware together.
Fixes: #6674
Signed-off-by: Wang, Arron <arron.wang@intel.com>
The init.sh in initramfs will parse the verity scheme,
roothash, root device and setup the root device accordingly.
Fixes: #6674
Signed-off-by: Wang, Arron <arron.wang@intel.com>
Generate rootfs hash data during creating the kata rootfs,
current kata image only have one partition, we add another
partition as hash device to save hash data of rootfs data blocks.
Fixes: #6674
Signed-off-by: Wang, Arron <arron.wang@intel.com>
Add required kernel config for dm-crypt/dm-integrity/dm-verity
and related crypto config.
Add userspace command line tools for disk encryption support
and ext4 file system utilities.
Fixes: #6674
Signed-off-by: Arron Wang <arron.wang@intel.com>
This PR updates the link to the correspondent Developer Guide at the
enabling full containerd debug that we have for kata 2.0 documentation.
Fixes#7034
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
When the version of libc is upgraded to 0.2.145, older getrandom could not adapt
to new API, and this will make agent-ctl fail to compile.
We upgrade the version of `rand`, so the low version of getrandom will no longer
need.
Fixes: #7032
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Fixes: #5401, #6654
- Switch kata-ctl from eprintln!()/println!() to structured logging via
the logging library which uses slog.
- Adds a new create_term_logger() library call which enables printing
log messages to the terminal via a less verbose / more human readable
terminal format with colors.
- Adds --log-level argument to select the minimum log level of printed messages.
- Adds --json-logging argument to switch to logging in JSON format.
Co-authored-by: Byron Marohn <byron.marohn@intel.com>
Co-authored-by: Luke Phillips <lucas.phillips@intel.com>
Signed-off-by: Jayant Singh <jayant.singh@intel.com>
Signed-off-by: Byron Marohn <byron.marohn@intel.com>
Signed-off-by: Luke Phillips <lucas.phillips@intel.com>
Signed-off-by: Kelby Madal-Hellmuth <kelby.madal-hellmuth@intel.com>
Signed-off-by: Liz Lawrens <liz.lawrens@intel.com>
Github Actions reads and runs workflow files from the main branch,
rather than from the PR branch. This means that PRs that modify workflow
files aren't being tested with the updated workflows coming from the PR,
but rather with the old workflows from the main branch. AFAIK, this
behavior isn't avoidable for workflow files (but is for other scripts).
This makes it very hard to reliably test workflow changes before they're
actually merged into main and leads to issues that we have to hotifx
(see #6983, #6995).
This PR aims to mitigate that by extracting the commands used in
workflows to a separate script file. The way our CI is set up, those
script files are read from the PR branch and thus changes would be
reflected in the CI checks.
Fixes: #6971
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
In hypervisors that do not support virtiofs we have to copy files in
the VM sandbox to properly setup the network (resolv.conf, hosts, and hostname).
To do that, we construct the volume as before, with the addition of an extra
variable that designates the path where the file will reside in the sandbox.
In this case, we issue a `copy_file` agent request *and* we patch the spec
to account for this change.
Fixes: #6978
Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
Signed-off-by: George Pyrros <gpyrros@nubificus.co.uk>
When dragonball update dbs-boot crate in commit
64c764c147, the Cargo.lock in runtime-rs
should also be updated.
Fixes: #6969
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
Full SHA is 40 characters, while AKS cluster name has a limit of 63. Trim the
SHA to 12 characters, which is widely considered to be unique enough and is
short enough to be used in the cluster name
Fixes: #7010
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Let's start adding the status of our jobs as part of our main page, so
folks monitoring those can easily check whether they're okay, or if
someone has to be pinged about those.
Fixes: #7008
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We added that to create the cluster name, but I forgot to add that to
the part we get the k8s config file, or to the part where we delete the
AKS cluster.
Fixes: #6999
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need to do so, otherwise we'll create two clusters for testing Cloud
Hypervisor with exactly the same name, one using Ubuntu, and one using
Mariner.
Fixes: #6999
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The string representing the architecture aarch64 and x86_64 need to be changed to arm64 and amd64 for the release.
Fixes: #6986
Signed-off-by: SinghWang <wangxin_0611@126.com>
There were recent changes for the tdx kernel in the version.yaml that are
not currently accounted for in the build-kernel.sh script.
Attempts to setup a tdx kernel to build local changes seemed to not download
the tdx kernel. Instead the mainline kernel is downloaded which has no
tdx-related changes.
The version.yaml has a new entry for tdx kernel. Use that instead for
setting up and downloading the tdx kernel.
Fixes: #6984
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
While the Mariner Kata host is in preview, we need the `aks-preview`
extension to enable the `--workload-runtime KataMshvVmIsolation` flag.
Fixes: #6994
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This PR is to make an environment variable `BUILDER_REGISTRY` configurable
so that those who want to use their own registry for build can set up
the registry.
Fixes: #6988
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The vcpu hotplug/hotunplug feature is implemented with upcall. This commit
add three patches to support the feature on aarch64. Patches:
> 0005: add support of upcall on aarch64
> 0006: skip activate offline cpus' MSI interrupt
> 0007: set the correct boot cpu number
Fixes: #6010
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
This commit implements the vcpu_boot_onlined vector in get_fdt_vm_info.
"boot_enabled" means whether this vcpu should be onlined at first boot.
It will be used by fdt, which write an attribute called boot_enabled,
and will be handled by guest kernel to pass the correct cpu number to
function "bringup_nonboot_cpus".
Fixes: #6010
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
This commit add support of resize_vcpu on aarch64. As kvm will check
whether vgic is initialized when calling KVM_CREATE_VCPU ioctl, all the
vcpu fds should be created before vm is booted.
To support resizing vcpu scenario, we use max_vcpu_count for
create_vcpus and setup_interrupt_controller interfaces. The
SetVmConfiguration API will ensure max_vcpu_count >= boot_vcpu_count.
Fixes: #6010
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
dbs-boot-v0.4.0 refectors the create_fdt interface. It simplifies the
parameters needed to be passed and abstracts them into three structs.
By the way, it also reserves some interfaces for future feature: numa
passthrough and cache passthrough.
Fixes: #6969
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
Rewrite the comment of Vm::init_microvm method for aarch64.
Fixes cargo test warnings on aarch64.
Fixes: #6969
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
Move the get_volume_mount_info to kata-types/src/mount.rs.
If so, it becomes a common method of DirectVolumeMountInfo
and reduces duplicated code.
Fixes: #6701
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
When run a exec process in backgroud without tty, the
exec will hang and didn't terminated.
For example:
crictl -i <container id> sh -c 'nohup tail -f /dev/null &'
Fixes: #4747
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
The current testing setup only supports running Kata on top of an Ubuntu
host. This adds Mariner to the matrix of testable hosts for k8s
tests, with Cloud Hypervisor as a VMM.
As preparation for the upcoming PR that will change only the actual test
code (rather than workflow YAMLs), this also introduces a new file
`setup.sh` that will be used to set host-specific parameters at test
run-time.
Fixes: #6961
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
sandbox_bind_mounts supports kinds of mount patterns, for example:
(1) "/path/to", default readonly mode.
(2) "/path/to:ro", same as (1).
(3) "/path/to:rw", readwrite mode.
Both support configuration and annotation:
(1)[runtime]
sandbox_bind_mounts=["/path/to", "/path/to:rw", "/mnt/to:ro"]
(2) annotation will alse be supported, restricted as below:
io.katacontainers.config.runtime.sandbox_bind_mounts
= "/path/to /path/to:rw /mnt/to:ro"
Fixes: #6597
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
We're still facing issues related to the time taken to deploy the
kata-deplot daemonset and starting to run the tests.
Ideally, we should solve this with a readiness probe, and that's the
approach we want to take in the future. However, for now, let's just
make sure those tests are not on the way of the community.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've seen tests being aborted close to the end of the run due to the
timeout. Let's increase it, avoiding to hit such cases again..
Fixes: #6964
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're currently backing up and restoring all the possible shim files,
but the default one ("containerd-shim-kata-v2").
Let's ensure this is also backed up and restored.
Fixes: #6957
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR fixes the indentation on the kata deploy merge script
that instead of single spaces uses a tap.
Fixes#6925
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
There is a race condition when virtiofsd is killed without finishing all
the clients. Because of that, when a pod is stopped, QEMU detects
virtiofsd is gone, which is legitimate.
Sending a SIGTERM first before killing could introduce some latency
during the shutdown.
Fixes#6757.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
We previously were doing:
* Create a new image on kata-deploy-ci using the commit hash of the
latest tag
* This was used to test on AKS, which is no longer needed as we test
on AKS on every PR
* Create a new image on kata-deploy using the release tag and "latest"
or "stable", by tagging the kata-deploy-ci image accordingly
As part of cfe63527c5, we broke the
workflow described above, as in the first step we would save the PKG_SHA
to be used in the second step, but that part ended up being removed.
Anyways, this back and forth is not needed anymore and we can simplify
the process by doing:
* Create a new image on kata-deploy, using:
- The tag received as ref from the event that triggered this worklow
- "latest" or "stable" tag, depending on whether it's a stable release
or not
Fixes: #6946
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
For some bizarre reason, the login-action will simply fail to
authenticate to docker.io in it's specified as a registry. The way to
proceed, instead, is to *not* specify any registry as it'd be used by
default.
Fixes: #6943
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR updates the nydus version to 2.2.1. This change includes:
nydus-image: fix a underflow issue in get_compressed_size()
backport fix/feature to stable 2.2
[backport] contrib: upgrade runc to v1.1.5
service: add README for nydus-service
nydus: fix a possible panic caused by SubCmdArgs::is_present
Backports two bugfixes from master into stable/v2.2
[backport stable/v2.2] action: upgrade golangci-lint to v1.51.2
[backport] action: fix smoke test for branch pattern
Fixes#6938
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
`docker/login-action@v3` does *not* exist and `docker/login-action@v2`
should be used instead.
Fixes: #6934
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
- Fix cache for OVMF and rootfs-initrd (both x86_64)
- Upgrade to Cloud Hypervisor v32.0
- osbuilder: Bump fedora image version
- local-build: Standardise what's set for the local build scripts
- gha: aks: Wait a little bit more before run the tests
- docs: Update container network model url
- gha: release: Fix s390x worklow
- cache: Fix OVMF caching
- gha: payload-after-push: Pass secrets down
- tools: Fix arch bug
22154e0a3 cache: Fix OVMF tarball name for different flavours
b7341cd96 cache: Use "initrd" as `initrd_type` to build rootfs-initrd
b8ffcd1b9 osbuilder: Bump fedora image version
636539bf0 kata-deploy: Use apt-key.gpg from k8s.io
ae24dc73c local-build: Standardise what's set for the local build scripts
35c3d7b4b runtime: clh: Re-generate the client code
cfee99c57 versions: Upgrade to Cloud Hypervisor v32.0
ad324adf1 gha: aks: Wait a little bit more before run the tests
191b6dd9d gha: release: Fix s390x worklow
cfd8f4ff7 gha: payload-after-push: Pass secrets down
75330ab3f cache: Fix OVMF caching
a89b44aab tools: Fix arch bug
11a34a72e docs: Update container network model url
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
75330ab3f9 tried to fix OVMF caching, but
didn't consider that the "vanilla" OVMF tarball name is not
"kata-static-ovmf-x86_64.tar.xz", but rather "kata-static-ovmf.tar.xz".
The fact we missed that, led to the cache builds of OVMF failing, and
the need to build the component on every single PR.
Fixes: #6917 (hopefully for good this time).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've been defaulting to "", which would lead to a mismatch with the
latest version from the cache, causing a miss, and finally having to
build the rootfs-initrd as part of the tests, every single time.
Fixes: #6917
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're facing some issues to download / use the public key provided by
google for installing kubernetes as part of the kata-deploy image.
```
The following signatures couldn't be verified because the public key is
not available: NO_PUBKEY B53DC80D13EDEF05
Reading package lists... Done
W: GPG error: https://packages.cloud.google.com/apt kubernetes-xenial
InRelease: The following signatures couldn't be verified because the
public key is not available: NO_PUBKEY B53DC80D13EDEF05 E: The
repository 'https://apt.kubernetes.io kubernetes-xenial InRelease' is
not signed.
N: Updating from such a repository can't be done securely, and is
therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user
configuration details.
```
Let's work this around following the suggestion made by @dims, at:
https://github.com/kubernetes/k8s.io/pull/4837#issuecomment-1446426585
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've a discrepancy on what's set along the scripts used to build the
Kata Cotainers artefacts locally.
Some of those were missing a way to easily debug them in case of a
failure happens, but one specific one (build-and-upload-payload.sh)
could actually silently fail.
All of those have been changed as part of this commut.
Fixes: #6908
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This patch re-generates the client code for Cloud Hypervisor v32.0.
Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by openapi-generator.
Fixes: #6632
Signed-off-by: Bo Chen <chen.bo@intel.com>
fa832f4709 increased the timeout, which
helped a lot, mainly in the TEE machines. However, we're still seeing
some failures here and there with the AKS tests.
Let's bump it yet again and, hopefully, those errors to start the tests
will go away.
Fixes: #6905
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
GitHub is warning us that:
"""
The workflow is not valid. In .github/workflows/release.yaml (Line: 21,
Col: 11): Error from called workflow
kata-containers/kata-containers/.github/workflows/release-s390x.yaml@d2e92c9ec993f56537044950a4673e50707369b5
(Line: 14, Col: 12): Job 'kata-deploy' depends on unknown job
'create-kata-tarball'.
"""
This is happening as we need to reference
"build-kata-static-tarball-s390x" instead of "create-kata-tarball".
Fixes: #6903
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The "build-assets-${arch}" jobs need to have access to the secrets in
order to log into the container registry in the cases where
"push-to-registry", which is used to push the builder containers to
quay.io, is set to "yes".
Now that "build-assets-${arch}" pass the secrets down, we need to log
into the container registry in the "build-kata-static-tarball-${arch}"
files, in case "push-to-registry" is set to "yes".
Fixes: #6899
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
OVMF has been cached, but it's not been used from cache as the `version`
set in the cached builds has always been empty.
The reason for that is because we've been trying to look for
`externals.ovmf.ovmf.version`, while we should be actually looking for
`externals.ovmf.x86_64.version`.
Setting `x86_64` as the OVMF_FLAVOUR would cause another bug, as the
expected tarball name would then be `kata-static-x86_64.tar.xz`, instead
of `kata-static-ovmf-x86_64.tar.xz`.
With everything said, let's simplify the OVMF_FLAVOUR usage, by using it
as it's passed, and only adapting the tarball name for the TDVF case,
which is the abnormal one.
Fixes: #6897
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
- runtime: Use static_sandbox_resource_mgmt=true for TEEs
- update tokio dependency
- resource-control: fix setting CPU affinities on Linux
- runtime: use enable_vcpus_pinning from toml
- gha: k8s: Make the tests more reliable
- gha: Enable SEV-SNP tests on main
- gha: tdx: Use the k3s overlay for kata-cleanup
- runtime: Port sev package to main
- gpu: Rename the last bits from `gpu` to `nvidia-gpu`
- deploy: fix shell script error
- ppc64le: switch virtiofsd from C to rust version
- osbuilder: Fix indentation in rootfs.sh
- virtcontainers/qemu_test.go: Improve coverage
- agent: Add context to errors that may occur when AgentConfig file is …
- virtcontainers/pkg/compatoci/: Improved coverage for for Kata 2.0
- kata-manager: Fix '-o' syntax and logic error
- kata-ctl: Add the option to install kata-ctl to a user specified directory
- runtime-rs: fix building instructions to use correct required Rust ve…
- Dragonball: use LinuxBootConfigurator::write_bootparams
- kata-deploy: Add http_proxy as part of the docker build
- kata-deploy: Do not ship the kata tarball
- kata-deploy: Build improvements
- deploy: Fix arch in image tag
- Revert "kata-deploy: Use readinessProbe to ensure everything is ready"
- virtcontainers: Improved test coverage for fc.go from 4.6% to 18.5%
- main | release: Fix multi-arch publishing is not supported
- cache: More fixes to nvidia-gpu kernels caching
- runtime: remove overriding ARCH value by default for ppc64le
- gha: Fix Body Line Length action flagging empty body commit messages
- gha: Fix snap creation workflow
- cache: Fix nvidia-gpu version
- cache: Update the KERNEL_FLAVOUR list to include nvidia-gpu
- packaging: Add SEV-SNP artifacts to main
- docs: Mark snap installation method as unmaintained
- packaging: Add sev artifacts to main
- kata-ctl: add generic kvm check & unit test
- Log-parser-rs
- warning_fix: fix warnings when build with cargo-1.68.0
- cross-compile: Include documentation and configuration for cross-compile
- runtime: Fix virtiofs fd leak
- gpu: cold plug VFIO devices
- pkg/signals: Improved test coverage 60% to 100%
- virtcontainers/persist: Improved test coverage 65% to 87.5%
- virtcontainers/clh_test.go: improve unit test coverage
- virtcontainers/factory: Improved test coverage
- gha: Also run k8s tests on qemu-snp
- gha: sev: fix for kata-deploy error
- gha: Also run k8s tests on qemu-sev
- Implement the "kata-ctl env" command
- runtime-rs: support keep_abnormal in toml config
- gpu: Build and Ship an GPU enabled Kernel
- kata-ctl: checks for kvm, kvm_intel modules loaded
- osbuilder: Fix D-Bus enabling in the dracut case
- snap: fix docker start fail issue
- kata-manager: Fix containerd download
- agent: Fix ut issue caused by fd double closed
- Bump ttrpc to 0.7.2 and protobuf to 3.2.0
- gpu: Add GPU enabled confguration and runtime
- gpu: Do not pass-through PCI (Host) Bridges
- cache-components: Fix caching of TDVF and QEMU for TDX
- gha: tdx: Ensure kata-deploy is removed after the tests run
- versions: Upgrade to Cloud Hypervisor v31.0
- osbuilder: Enable dbus in the dracut case
- runtime: Don't create socket file in /run/kata
- nydus_rootfs/prefetch_files: add prefetch_files for RAFS
- runtime-rs/virtio-fs: add support extra handler for cache mode.
- runtime-rs: enable nerdctl to setup cni plugin
- tdx: Add artefacts from the latest TDX tools release into main
- runtime: support non-root for clh
- gha: ci-on-push: Run k8s tests with dragonball
- rustjail: Use CPUWeight with systemd and CgroupsV2
- gha: k8s-on-aks: {create,delete} AKS must be a coded-in step
- docs: update the rust version from version.yaml
- gha: k8s-on-aks: Set {create,delete}_aks as steps
- gha: k8s-on-aks: Fix cluster name
- gha: Also run k8s tests on AKS with dragonball
- gha: Only push images to registry after merging a PR
- gha: aks: Use D4s_v5 instance
- tools: Avoid building the kernel twice
- rustjail: Fix panic when cgroup manager fails
- runtime: add filter metrics with specific names
- gha: Use ghcr.io for the k8s CI
- GHA |Switch "kubernetes tests" from jenkins to GitHub actions
- docs: Update CNM url in networking document
- kata-ctl: add function to get platform protection.
f6e1b1152 agent: update tokio dependency
4cb83dc21 kata-ctl: update tokio dependency
df615ff25 runk: update tokio dependency
ca6892ddb runtime-rs: update tokio dependency
ca1531fe9 runtime: Use static_sandbox_resource_mgmt=true for TEEs
fa832f470 gha: k8s: Make the tests more reliable
cbb9fe8b8 config: Use standard OVMF with SEV
724437efb kata-deploy: add kata-qemu-sev runtimeclass
521dad2a4 Tests: skip CPU constraints test on SEV and SNP
72308ddb0 gha: ci-on-push: Don't skip tests for SEV
da0f92cef gha: ci-on-push: Don't skip tests for SEV-SNP
12f43bea0 gha: tdx: Use the k3s overlay for kata-cleanup
1a3f8fc1a deploy: fix shell script error
87cb98c01 osbuilder: Fix indentation in rootfs.sh
c5a59caca ppc64le: switch virtiofsd from C to rust version
bfdf0144a versions: Bump virtiofsd to 1.6.1
dd7562522 runtime: pkg/sev: Add kbs utility package for SEV pre-attestation
05de7b260 runtime: Add sev package
3a9d3c72a gpu: Rename the last bits from `gpu` to `nvidia-gpu`
4cde844f7 local-build: Fix kernel-nvidia-gpu target name
593840e07 kata-ctl: Allow INSTALL_PATH= to be specified
bdb75fb21 runtime: use enable_vcpus_pinning from toml
20cb87508 virtcontainers/qemu_test.go: Improve test coverage
b9a1db260 kata-deploy: Add http_proxy as part of the docker build
3e85bf5b1 resource-control: fix setting CPU affinities on Linux
5f3f844a1 runtime-rs: fix building instructions with respect to required Rust version
777c3dc8d kata-deploy: Do not ship the kata tarball
50cc9c582 tests: Improve coverage for virtcontainers/pkg/compatoci/ for Kata 2.0
136e2415d static-build: Download firecracker instead of building it
3bf767cfc static-build: Adjust ARCH for nydus
ac88d34e0 static-build: Use relased binary for CLH (aarch64)
73913c8eb kata-manager: Fix '-o' syntax and logic error
2856d3f23 deploy: Fix arch in image tag
e8f81ee93 Revert "kata-deploy: Use readinessProbe to ensure everything is ready"
cfe63527c release: Fix multi-arch publishing is not supported
197c33651 Dragonball: use LinuxBootConfigurator::write_bootparams to writes the boot parameters into guest memory.
4d17ea4a0 cache: Fix nvidia-snp caching version
a133fadbf cache: Fix nvidia-gpu-tdx-experimental cache URL
b9990c201 cache: Fix nvidia-gpu version
c9bf7808b cache: Update the KERNEL_FLAVOUR list to include nvidia-gpu
3665b4204 gpu: Rename `gpu` targets to `nvidia-gpu`
2c90cac75 local-build: fixup alphabetization
4da6eb588 kata-deploy: Add qemu-snp shim
14dd05375 kata-deploy: add kata-qemu-snp runtimeclass
0bb37bff7 config: Add SNP configuration
af7f2519b versions: update SEV kernel description
dbcc3b5cc local-build: fix default values for OVMF build
b8bbe6325 gha: build OVMF for tests and release
cf0ca265f local-build: Add x86_64 OVMF target
db095ddeb cache: add SNP flavor to comments
f4ee00576 gha: Build and ship QEMU for SNP
7a58a91fa docs: update SNP guide
879333bfc versions: update SNP QEMU version
38ce4a32a local-build: add support to build QEMU for SEV-SNP
5f8008b69 kata-ctl: add unit test for kvm check
a085a6d7b kata-ctl: add generic kvm check
772d4db26 gha: Build and ship SEV initrd
45fa36692 gha: Build and ship SEV OVMF
4770d3064 gha: Build and ship SEV kernel.
fb9c1fc36 runtime: Add qemu-sev config
813e4c576 runtimeClasses: add sev runtime class
af18806a8 static-build: Add caching support to sev ovmf
76ae7a3ab packaging: adding caching capability for kernel
12c5ef902 packaging: add support to build OVMF for SEV
b87820ee8 packaging: add support to build initrd for sev
e1f3b871c docs: Mark snap installation method as unmaintained
022a33de9 agent: Add context to errors when AgentConfig file is missing
b0e6a094b packaging: Add sev kernel build capability
a4c0303d8 virtcontainers: Fixed static checks for improved test coverage for fc.go
8495f830b cross-compile: Include documentation and configuration for cross-compile
13d7f39c7 gpu: Check for VFIO port assignments
6594a9329 tools: made log-parser-rs
03a8cd69c virtcontainers: Improved test coverage for fc.go from 4.6% to 18.5%
9e2b7ff17 gha: sev: fix for kata-deploy error
5c9246db1 gha: Also run k8s tests on qemu-snp
c57a44436 gha: Add the ability to test qemu-snp
406419289 env: Utilize arch specific functionality to get cpu details
fb40c71a2 env: Check for root privileges
1016bc17b config: Add api to fetch config from default config path
b908a780a kata-env: Pass cmd option for file path
b1920198b config: Workaround the way agent and hypervisor configs are fetched
f2b2621de kata-env: Implement the kata-env command.
c849bdb0a gha: Also run k8s tests on qemu-sev
6bf1fc605 virtcontainers/factory: Improved test coverage
0d49ceee0 gha: Fix snap creation workflow warnings
138ada049 gpu: Cold Plug VFIO toml setting
defb64334 runtime: remove overriding ARCH value by default for ppc64le
f7ad75cb1 gpu: Cold-plug extend the api.md
0fec2e698 gpu: Add cold-plug test
f2ebdd81c utils: Get rid of spurious print statement left behind.
9a94f1f14 make: Export VERSION and COMMIT
2f81f48da config: Add file under /opt as another location to look for the config
07f7d17db config: Make the pipe_size field optional
68f635773 config: Make function to get the default conf file public
7565b3356 kata-ctl: Implement Display trait for GuestProtection enum
94a00f934 utils: Make certain constants in utils.rs public
572b338b3 gitignore: Ignore .swp and .swo editor backup files
376884b8a cargo: Update version of clap to 4.1.13
17daeb9dd warning_fix: fix warnings when build with cargo-1.68.0
521519d74 gha: Add the ability to test qemu-sev
205909fbe runtime: Fix virtiofs fd leak
5226f15c8 gha: Fix Body Line Length action flagging empty body commit messages
0f45b0faa virtcontainers/clh_test.go: improve unit test coverage
dded731db gpu: Add OVMF setting for MMIO aperture
2a830177c gpu: Add fwcfg helper function
131f056a1 gpu: Extract VFIO Functions to drivers
c8cf7ed3b gpu: Add ColdPlug of VFIO devices with devManager
e2b5e7f73 gpu: Add Rawdevices to hypervisor
6107c32d7 gpu: Assign default value to cold-plug
377ebc2ad gpu: Add configuration option for cold-plug VFIO
c18ceae10 gpu: Add new struct PCIePort
9c38204f1 virtcontainers/persist: Improved test coverage 65% to 87.5%
1c1ee8057 pkg/signals: Improved test coverage 60% to 100%
cc8ea3232 runtime-rs: support keep_abnormal in toml config
96e8470db kata-manager: Fix containerd download
432d40744 kata-ctl: checks for kvm, kvm_intel modules loaded
b1730e4a6 gpu: Add new kernel build option to usage()
3e7b90226 osbuilder: Fix D-Bus enabling in the dracut case
53c749a9d agent: Fix ut issue caused by fd double closed
2e3f19af9 agent: fix clippy warnings caused by protobuf3
4849c56fa agent: Fix unit test issue cuased by protobuf upgrade
0a582f781 trace-forwarder: remove unused crate protobuf
73253850e kata-ctl: remove unused crate ttrpc
76d2e3054 agent-ctl: Bump ttrpc from 0.6.0 to 0.7.1
eb3d20dcc protocols: Add ut for Serde
59568c79d protocols: add support for Serde
a6b4d92c8 runtime-rs: Bump ttrpc from 0.6.0 to 0.7.1
ac7c63bc6 gpu: Add containerd shim for qemu-gpu
a0cc8a75f gpu: Add a kube runtime class
a81fff706 gpu: Adding a GPU enabled configuration
8af6fc77c agent: Bump ttrpc from 0.6.0 to 0.7.1
009b42dbf protocols: Fix unit test
392732e21 protocols: Bump ttrpc from 0.6.0 to 0.7.1
f4f958d53 gpu: Do not pass-through PCI (Host) Bridges
825e76948 gpu: Add GPU support to default kernel without any TEE
e4ee07f7d gpu: Add GPU TDX experimental kernel
a1272bcf1 gha: tdx: Fix typo overlay -> overlays
3fa0890e5 cache-components: Fix TDVF caching
80e3a2d40 cache-components: Fix TDX QEMU caching
87ea43cd4 gpu: Add configuration fragment
aca6ff728 gpu: Build and Ship an GPU enabled Kernel
dc662333d runtime: Increase the dial_timeout
eb1762e81 osbuilder: Enable dbus in the dracut case
f478b9115 clh: tdx: Update timeouts for confidential guest
3b76abb36 kata-deploy: Ensure node is ready after CRI Engine restart
5ec9ae0f0 kata-deploy: Use readinessProbe to ensure everything is ready
ea386700f kata-deploy: Update podOverhead for TDX
e31efc861 gha: tdx: Use the k3s overlay
542bb0f3f gha: tdx: Set KUBECONFIG env at the job level
d7fdf19e9 gha: tdx: Delete kata-deploy after the tests finish
da35241a9 tests: k8s: Skip k8s-cpu-ns when testing TDX
db2cac34d runtime: Don't create socket file in /run/kata
6d315719f snap: fix docker start fail issue
e4b3b0887 gpu: Add proper CONFIG_LOCALVERSION depending on TEE
69ba2098f runtime-rs: remove network entities and netns
b31f103d1 runtime-rs: enable nerdctl cni plugin
69d7a959c gha: ci-on-push: Run tests on TDX
5a0727ecb kata-deploy: Ship kata-qemu-tdx runtimeClass
98682805b config: Add configuration for QEMU TDX
3e1580019 govmm: Directly pass the firmware using -bios with TDX
3c5ffb0c8 govmm: Set "sept-ve-disable=on"
ed145365e runtime/qemu: Drop "kvm-type=tdx"
25b3cdd38 virtcontainers: Drop check for the `tdx` CPU flag
01bdacb4e virtcontainers: Also check /sys/firmwares/tdx for TDX
9feec533c cache: Add ability to cache OVMF
ce8d98251 gha: Build and ship the OVMF for TDX
39c3fab7b local-build: Add support to build OVMF for TDX
054174d3e versions: Bump OVMF for TDX
800fb49da packaging: Add get_ovmf_image_name() helper
fbf03d7ac cache: Document kernel-tdx-experimental
5d79e9696 cache: Add a space to ease the reading of the kernel flavours
6e4726e45 cache: Fix typos
fc22ed0a8 gha: Build and ship the Kernel for TDX
502844ced local-build: Add support to build Kernel for TDX
b2585eecf local-build: Avoid code duplication building the kernel
f33345c31 versions: Update Kernel TDX version
20ab2c242 versions: Move Kernel TDX to its own experimental entry
3d9ce3982 cache: Allow specifying the QEMU_FLAVOUR
33dc6c65a gha: Build and ship QEMU for TDX
eceaae30a local-build: Add support to build QEMU for TDX
f7b7c187e static-build: Improve qemu-experimental build script
3018c9ad5 versions: Update QEMU TDX version
800ee5cd8 versions: Move QEMU TDX to its own experimental entry
1315bb45f local-build: Add dragonball kernel to the `all` target
73e108136 local-build: Rename non vanilla kernel build functions
1d851b4be local-build: Cosmetic changes in build targets
49ce685eb gha: k8s-on-aks: Always delete the AKS cluster
e2a770df5 gha: ci-on-push: Run k8s tests with dragonball
d1f550bd1 docs: update the rust version from versions.yaml
f3595e48b nydus_rootfs/prefetch_files: add prefetch_files for RAFS
3bfaafbf4 fix: oci hook
c1fbaae8d rustjail: Use CPUWeight with systemd and CgroupsV2
375187e04 versions: Upgrade to Cloud Hypervisor v31.0
79f3047f0 gha: k8s-on-aks: {create,delete} AKS must be a coded-in step
2f35b4d4e gha: ci-on-push: Only run on `main` branch
e7bd2545e Revert "gha: ci-on-push: Depend on Commit Message Check"
0d96d4963 Revert "gha: ci-on-push: Adjust to using workflow_run"
c7ee45f7e Revert "gha: ci-on-push: Adapt chained jobs to workflow_run"
5d4d72064 Revert "gha: k8s-on-aks: Fix cluster name"
13d857a56 gha: k8s-on-aks: Set {create,delete}_aks as steps
dc6569dbb runtime-rs/virtio-fs: add support extra handler for cache mode.
85cc5bb53 gha: k8s-on-aks: Fix cluster name
1688e4f3f gha: aks: Use D4s_v5 instance
108d80a86 gha: Add the ability to also test Dragonball
2550d4462 gha: build-kata-static-tarball: Only push to registry after merge
e81b8b8ee local-build: build-and-upload-payload is not quay.io specific
13929fc61 gha: publish-kata-deploy-payload: Improve registry login
41026f003 gha: payload-after-push: Pass registry / repo as inputs
7855b4306 gha: ci-on-push: Adapt chained jobs to workflow_run
3a760a157 gha: ci-on-push: Adjust to using workflow_run
a159ffdba gha: ci-on-push: Depend on Commit Message Check
8086c75f6 gha: Also run k8s tests on AKS with dragonball
fe86c08a6 tools: Avoid building the kernel twice
3215860a4 gha: Set ci-on-push to run on `pull_request_target`
d17dfe4cd gha: Use ghcr.io for the k8s CI
b661e0cf3 rustjail: Add anyhow context for D-Bus connections
60c62c3b6 gha: Remove kata-deploy-test.yaml
43894e945 gha: Remove kata-deploy-push.yaml
cab9ca043 gha: Add a CI pipeline for Kata Containers
53b526b6b gha: k8s: Add snippet to run k8s tests on aks clusters
c444c24bc gha: aks: Add snippets to create / delete aks clusters
11e0099fb tests: Move k8s tests to this repo
73be4bd3f gha: Update actions for release.yaml
d38d7fbf1 gha: Remove code duplication from release.yaml
56331bd7b gha: Split payload-after-push-*.yaml
a552a1953 docs: Update CNM url in networking document
7796e6ccc rustjail: Fix minor grammatical error in function name
41fdda1d8 rustjail: Do not unwrap potential error with cgroup manager
a914283ce kata-ctl: add function to get platform protection.
0f7351556 runtime: add filter metrics with specific names
cbe6ad903 runtime: support non-root for clh
d3bb25418 utils: Add function to check vhost-vsock
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
If a hypervisor debug console is enabled and sandbox_cgroup_only is set,
the hypervisor can fail to open /dev/ptmx, which prevents the sandbox
from launching.
This is caused by the absence of a device cgroup entry to allow access
to /dev/ptmx. When sandbox_cgroup_only is not set, the hypervisor
inherits the default unrestrcited device cgroup, but with it enabled it
runs into allow / deny list restrictions.
Fix by adding an allowlist entry for /dev/ptmx when debug is enabled,
sandbox_cgroup_only is true, and no /dev/ptmx is already in the list of
devices.
Fixes: #6870
Signed-off-by: Krister Johansen <kjlx@templeofstupid.com>
This PR updates the container network model url that is part of the
virtcontainers documentation.
Fixes#6889
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
When this option is enabled the runtime will attempt to determine the
appropriate sandbox size (memory, CPU) before booting the virtual
machine.
As TEEs do not support memory and CPU hotplug, this approach must be
used.
Fixes: #6818
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We like it or not, every now and then we'll have to deal with flaky
tests, and our tests using GHA are not exempt from that fact.
With this simple commit, we're trying to improve the reliability of the
tests in a few different fronts:
* Giving enough time for the script used by kata-deploy to be executed
* We've hit issues as the kata-deploy pod is considered "Ready" at the
moment it starts running, not when it finishes the needed setup. We
should also be looking on how to solve this on the kata-deploy side
but, for now, let's ensure our tests do not break with the current
kata-deploy behavior.
* Merging the "Deploy kata-deploy" and "Run tests" steps
* We've hit issues re-running tests and seeing even more failures than
the ones we're trying to debug, as a step will simply be taken as
succeeded as part of the re-run, in case it was successful executed
as part of the first run. This causes issues with the kata-deploy
deployment, as the tests would start running before even having the
node set up for running Kata Containers.
Fixes: #6865#6649
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The AmdSev firmware package should be used with
measured direct boot. If the expected hashes are not
injected into the firmware binary by the VMM, the
guest will not boot. This is required for security.
Currently the main branch does not have the extended
shim support for SEV, which tells the VMM to inject
the expected hashes.
We ship the standard OVMF package to use with SNP,
so let's switch SEV to that for now. This will need
to be changed back when shim support for SEV(-ES)
is added to main.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
In order to populate containerd config file with
support for SEV, we need to add the qemu-sev shim
to the kata-deploy script.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Currently Kata does not support memory / CPU hotplug for SEV or
SEV-SNP so we need to skip tests that rely on it.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Now that SEV artifacts are built by GHA, remove
conditional that skips tests when using qemu-sev.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Now that we have SNP artifacts in place and they are built via gha,
remove the condition that skips the tests for SNP.
Fixes: #6809
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
When updating an interface, there's maybe an existed
interface whose name would be the same with the updated
required name, thus it would update failed with interface
name existed error. Thus we should rename the existed interface
with an temporary name and swap it with the previouse interface
name last.
Fixes: #6842
Signed-off-by: fupan <fupan.lfp@antgroup.com>
As the TDX CI runs on k3s, we must ensure the cleanup, as already done
for the deploy, used the k3s overlay.
Fixes: #6857
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR replaces single spaces to tabs in order to fix the
indentation of the rootfs script.
Fixes#6848
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
We have been using the C version of virtiofsd on ppc64le. Now that the issue with
rust virtiofsd have been fixed, let's switch to it.
Fixes: #4259
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
virtiofsd v1.6.1 has been released with the fixes required for running
successfully on ppc64le.
Fixes: #4259
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
Supports both online and offline modes of interaction with simple-kbs
for SEV/SEV-ES confidential guests.
Fixes: #6795
Signed-off-by: Dov Murik <dovmurik@linux.ibm.com>
The sev package provides utilities for launching AMD SEV and SEV-ES
confidential guests.
Fixes: #6795
Signed-off-by: Dov Murik <dovmurik@linux.ibm.com>
Let's specifically name the `gpu` runtime class as `nvidia-gpu`. By
doing this we keep the door open and ease the life of the next vendor
adding GPU support for Kata Containers.
Fixes: #6553
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Update the kata-ctl install rule to allow it to be installed to a given directory
The Makefile was updated to use an INSTALL_PATH variable to track where the
kata-ctl binary should be installed. If the user doesn't specify anything,
then it uses the default path that cargo uses. Otherwise, it will install it
in the directory that the user specified. The README.md file was also updated
to show how to use the new option.
Fixes#5403
Co-authored-by: Cesar Tamayo <cesar.tamayo@intel.com>
Co-authored-by: Kevin Mora Jimenez <kevin.mora.jimenez@intel.com>
Co-authored-by: Narendra Patel <narendra.g.patel@intel.com>
Co-authored-by: Ray Karrenbauer <ray.karrenbauer@intel.com>
Co-authored-by: Srinath Duraisamy <srinath.duraisamy@intel.com>
Signed-off-by: Narendra Patel <narendra.g.patel@intel.com>
Rework TestQemuCreateVM routine to be a table driven test with
various config variations passed to it. After CreateVM a handful
of additional functions are exercised to improve code-coverage.
Also add partial coverage for StartVM routine.
Currently improving from 19.7% to 35.7%
Credit PR to Hackathon Team3
Fixes: #267
Signed-off-by: Tamas K Lengyel <tamas.lengyel@intel.com>
Add http_proxy and https_proxy as part of the docker build arguments
in order to build properly when we are behind a proxy.
Fixes#6834
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
With this fix the vCPU pinning feature chooses the correct
physical cores to pin the vCPU threads on rather than always using core 0.
Fixes#6831
Signed-off-by: Peteris Rudzusiks <rye@stripe.com>
There's absolutely no reason to ship the kata-static tarball as part of
the payload image, as:
* The tarball is already part of the release process
* The payload image already has uncompressed content of the tarball
* The tarball itself is not used anywhere by the kata-deploy scripts
Fixes: #6828
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
There's no reason for us to build firecracker instead of simply
downloading the official released tarball, as tarballs are provided for
the architectures we want to use them.
Fixes: #6770
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When building from aarch64, just use "arm64" as that's what's used in
the name of the released nydus tarballs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
There's no need to build Cloud Hypervisor aarch64 as, for a few releases
already, Cloud Hypervisor provides an official release binary for the
architecture.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Fix the syntax and logic error that is only displayed if the user runs
the script with `-o`. This option requests that "only" Kata Containers
is installed and stops containerd from being installed.
Fixes: #6822.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
`uname -m` produces `x86_64`, but container image convention
is to use `amd64`, so update this in the tag
Fixes: #6820
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This reverts commit 5ec9ae0f04, for two
main reasons:
* The readinessProbe was misintepreted by myself when working on the
original PR
* It's actually causing issues, as the pod ends up marked as not
healthy.
When release is published, kata-deploy payload and kata-static package
can support multi-arch publishing.
Fixes: #6449
Signed-off-by: SinghWang <wangxin_0611@126.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
All the kernel-foo instances, such as "kernel-sev" or "kernel-snp",
should be transformed into "kernel.foo" when looking at the
versions.yaml file.
This was already done for SEV, but missed on the SNP case.
Fixes: #6777
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We were passing "kernel-nvidia-gpu-tdx", missing the "-experimental"
part, leading to a non-valid URL.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
c9bf7808b6 introduced the logic to
properly get the version of nvidia-gpu kernels, but one important part
was dropped during the rebase into main, which is actually getting the
correct version of the kernel.
Fixing this now, and using the old issue as reference.
Fixes: #6777
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need to make sure that, when caching a `-nvidia-gpu` kernel, we still
look at the version of the base kernel used to build the nvidia-gpu
drivers, as the ${vendor}-gpu kernels are based on already existing
entries in the versions.yaml file and do not require a new entry to be
added.
Fixes: #6777
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
A few pieces of the local-build tooling are supposed to be
alphabetized. Fixup a couple minor issues that have accumulated.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Now that we have the SNP components in place, make sure that
kata-deploy knows about the qemu-snp shim so that it will be
added to containerd config.
Fixes: #6575
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Since SEV-SNP has limited hotplug support, increase
the pod overhead to account for fixed resource usage.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
SNP requires many specific configurations, so let's make
a new SNP configuration file that we can use with the
kata-qemu-snp runtime class.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
SNP and SEV will share a (guest) kernel. Update the description
in versions.yaml to mention this.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
The x86_64 package of OVMF is required for deployments
that don't use kernel hashes, which includes SEV-SNP
in the short term. We should keep this in the bundle
in the long term in case someone wants to disable
kernel hashes.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Add targets to build the "plain" x86_64 OVMF.
This will be used by anyone who is using SEV or SNP
without kernel hashes. The SNP QEMU does not yet
support kernel hashes so the OvmfPkg will be used
by default.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
Since we reshuffled versions.yaml, update the guide so that
we can find the SNP QEMU info.
Once runtime support is merged we should overhaul or remove
this guide, but let's keep it for now.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Refactor SNP QEMU entry in versions.yaml to match
qemu-experimental and qemu-tdx-experimental.
Also, update the version of QEMU to what we are using
in CCv0. This is the non-UPM QEMU and it does not
have kernel hashes support.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
Add Make targets and helper functions to build the QEMU
needed for SEV-SNP.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
Check that kvm test fails when run as non-root and when device specified
is not /dev/kvm.
Fixes#5338
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Add kvm check using ioctl macro to create a syscall that checks the kvm
api version and if creation of a vm is successful.
Fixes#5338
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
We have code that builds initrd for SEV.
thus, adding that to the test and release process.
Fixes: #6572
Signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
SEV requires custom kernel arguments when building.
Thus, adding it to the test and release process.
Fixes: #6572
Signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
Adding config file that can be used with qemu-sev runtime class.
Since SEV has limited hotplug support, increase
the pod overhead to account for fixed resource usage.
Fixes: #6572
Signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
SEV requires special OVMF.
Now that we have ability to build this custom OVMF, let's optimize
it by caching so that we don't have to build it for every run.
Fixes: sev: #6572
Signed-Off-By: Unmesh Deodhar <udeodhar@amd.com>
The SEV initrd build requires kernel modules.
So, for SEV case, we need to cache kernel modules tarball in
addition to kernel tarball.
Fixes: #6572
Signed-Off-By: Unmesh Deodhar <udeodhar@amd.com>
SEV requires special OVMF to work with kernel hashes.
Thus, adding changes that builds this custom OVMF for SEV.
Fixes: #6572
Signed-Off-By: Unmesh Deodhar <udeodhar@amd.com>
We need special initrd for SEV. The work on SEV initrd is based on
Ubuntu. Thus, adding another entry in versions.yaml
This binary will have '-sev' suffix to distinguish it from the generic
binary.
Fixes: #6572
Signed-Off-By: Unmesh Deodhar <udeodhar@amd.com>
The snap package is no longer being maintained so update the docs to
warn readers.
We'll remove the snap installation docs in a few weeks.
See: #6769.
Fixes: #6793.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
When the agent config file is missing, the panic message says "no such file or
directory" but doesn't inform the user about which file was missing. Add
context to the parsing (with filename) and to the from_config_file() calls
(with information where the path is coming from).
Fixes: #6771
Depends-on: github.com/kata-containers/tests#5627
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Expanded tests on fc_test.go to cover more lines of code. Coverage went from 4.6% to 18.5%.
Fixed very simple static check fail on line 202.
Fixes: #266
Signed-off-by: Eduardo Berrocal <eduardo.berrocal@intel.com>
`cross` is an open source tool that provides zero-setup cross compile
for rust binaries. Add documentation on this tool for compiling
kata-ctl tool and Cross.toml file that provides required configuration
for installing dependencies for various targets.
This is pretty useful for a developer to make sure code compiles and
passes checks for various architectures.
Fixes: #6765
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Eventual replacement of kata-log-parser, but for now replicates its
functionaility for the new runtime-rs syntax. Takes in log files,
parses, sorts by timestamp, spits them out in json, csv, xml, toml, and
a few others.
Fixes#5350
Signed-off-by: Gabe Venberg <gabevenberg@gmail.com>
Expanded tests on fc_test.go to cover more lines of code. Coverage went from 4.6% to 18.5%.
Fixes: #266
Signed-off-by: Eduardo Berrocal <eduardo.berrocal@intel.com>
With the changes proposed as part of this PR, a qemu-snp cluster
will be created but no tests will be performed.
GitHub Actions will only run the tests using the workflows that are
part of the **target** branch, instead of the using the ones coming
from the PR. No way to work around this for now.
After this commit is merged, the tests (not the yaml files for the
actions) will be altered in order for the checkout action to help in
this case.
Fixes: #6722
Signed-off-by: Ryan Savino <ryan.savino@amd.com>
Have kata-env call architecture specific function to get cpu details
instead of generic function to get cpu details that works only for
certain architectures. The functionality for cpu details has been fully
implemented for x86_64 and arm architectures, but needs to be
implemented for s390 and powerpc.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Add ability to write the environment information to a file
or stdout if file path is absent.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This is essentially a workaround for the issue:
https://github.com/kata-containers/kata-containers/issues/5954
runtime-rs chnages the Kata config format adding agent_name and
hypervisor_name which are then used as keys to fetch the agent and
hypervisor configs. This will not work for older configs.
So use the first entry in the hashmaps to fetch the configs as a
workaround while the config change issue is resolved.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Expanded tests on factory_test.go to cover more lines of code. Coverage went from 34% to 41.5% in the case of user-mode run tests,
and from 77.7% to 84% in the case of priviledge-mode run tests.
Fixes: #260
Signed-off-by: Eduardo Berrocal <eduardo.berrocal@intel.com>
Fix recurring issues of failing to install dependencies due to stale apt cache.
Uprev actions/checkout to v3 to resolve issue "Node.js 12 actions are deprecated."
Fixes: #5659
Signed-off-by: Tamas K Lengyel <tamas.lengyel@intel.com>
Currently, ARCH value is being set to powerpc64le by default.
powerpc64le is only right in context of rust and any operation
which might use this variable for a different purpose would fail on ppc64le.
Fixes: #6741
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
These will be consumed by kata-ctl, so export these so that
they can be used to replace variables available to the rust binary.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Most of kata installation tools use this path for installation, so
add this to the paths to look for the configuration.toml file.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Add the serde default attribute to the field so that parsing
can continue if this field is not present.
The agent assumes a default value for this, so it is not required
by the user to provide a value here.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
With the changes proposed as part of this PR, a qemu-sev cluster will
be created but no tests will be performed.
GitHub Actions will only run the tests using the workflows that are
part of the **target** branch, instead of the using the ones coming
from the PR. No way to work around this for now.
After this commit is merged, the tests (not the yaml files for the
actions) will be altered in order for the checkout action to help in this
case.
Fixes: #6711
Signed-off-by: Ryan Savino <ryan.savino@amd.com>
The kata runtime invokes removeStaleVirtiofsShareMounts after
a container is stopped to clean up the stale virtiofs file caches.
Fixes: #6455
Signed-off-by: Feng Wang <fwang@confluent.io>
Change the Body Line Length workflow to not trigger when the commit
message contains only a message without a body. Other workflows will
flag the missing body sections, and it was confusing to have an error
message that said 'Body line too long (max 150)' when this was not
actually the case.
Fixes: #5561
Co-authored-by: Jayant Singh <jayant.singh@intel.com>
Co-authored-by: Luke Phillips <lucas.phillips@intel.com>
Signed-off-by: Byron Marohn <byron.marohn@intel.com>
Signed-off-by: Jayant Singh <jayant.singh@intel.com>
Signed-off-by: Luke Phillips <lucas.phillips@intel.com>
Signed-off-by: Kelby Madal-Hellmuth <kelby.madal-hellmuth@intel.com>
Signed-off-by: Liz Lawrens <liz.lawrens@intel.com>
Added driver util function for easier handling of VFIO
devices outside of the VFIO module. At the sandbox level
we may need to set options depending if we have a VFIO/PCIe
device, like the fwCfg for confiential guests.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Some functions may be used in other modules then only in
the VFIO module, extract them and make them available to
other layers like sandbox.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
If we have a VFIO device and cold-plug is enabled
we mark each device as ColdPlug=true and let the VFIO
module do the attaching.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
RawDevics are used to get PCIe device info early before the sandbox
is started to make better PCIe topology decisions
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
For the hypervisor to distinguish between PCIe components, adding
a new enum that can be used for hot-plug and cold-plug of PCIe devices
Fixes: #6687
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Expanded tests on signals_test.go to cover more lines of code. 'go test' won't show 100% coverage (only 66.7%), because one test need to spawn a new
process (since it is testing a function that calls os.Exit(1)).
Fixes: #256
Signed-off-by: Eduardo Berrocal <eduardo.berrocal@intel.com>
This patch adds keep_abnormal in runtime config. If keep_abnormal =
true, it means that 1) if the runtime exits abnormally, the cleanup
process will be skipped, and 2) the runtime will not exit even if the
health check fails.
This option is typically used to retain abnormal information for
debugging and should NOT be enabled by default.
Fixes: #6717
Signed-off-by: mengze <mengze@linux.alibaba.com>
Signed-off-by: quanweiZhou <quanweiZhou@linux.alibaba.com>
Newer containerd releases have an additional static package published.
Because of this, download_url contains two urls causing curl to fail.
To resolve this, pick the first url from the containerd releases to
download containerd.
Fixes: #6695
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Ensure that kvm and kvm_intel modules are loaded.
Renames the get_cpu_info() function to read_file_contents()
Fixes#5332
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
- D-Bus enabling now occurs only in setup_rootfs (instead of
prepare_overlay and setup_rootfs)
- Adjust permissions of / so dbus-broker will be able to traverse FS
These changes enables kata-agent to successfully communicate with D-Bus.
Fixes#6677
Signed-off-by: Vladimir <amigo.elite@gmail.com>
Never ever try to close the same fd double times, even in a unit test.
A file descriptor is a number which will be reused, so when you close
the same number twice you may close another file descriptor in the second
time and then there will be an error 'Bad file descriptor (os error 9)'
while the wrongly closed fd is being used.
Fixes: #6679
Signed-off-by: Tim Zhang <tim@hyper.sh>
Last but not least add the continerd shim configuration
pointing to the correct configuration-<shim>.toml
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
We need to set hotplug on pci root port and enable at least one
root port. Also set the guest-hooks-dir to the correct path
Fixes: #6675
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
On some systems a GPU is in a IOMMU group with a PCI Bridge and
PCI Host Bridge. Per default no PCI Bridge needs to be passed-through.
When scanning the IOMMU group, ignore devices with a 0x60 class ID prefix.
Fixes: #6663
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
With each release make sure we ship a GPU and TEE enabled kernel
This adds tdx-experimental kernel support
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The beauty of GHA not allowing us to easily test changes in the yaml
files as part of the PR has hit us again. :-/
The correct path for the k3s deployment is
tools/packaging/kata-deploy/kata-deploy/overlays/k3s instead of
tools/packaging/kata-deploy/kata-deploy/overlay/k3s.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
TDVF caching is not working as the tarball name is incorrect. The result
expected is kata-static-tdvf.tar.xz, but it's looking for
kata-static-tdx.tar.xz.
This happens as a logic to convert tdx -> tdvf has been added as part of
the building scripts, but I missed doing this as part of the caching
scripts.
Fixes: #6669
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
TDX QEMU caching is not working as expected, as we're checking for its
version looking at "assets.hypervisor.${QEMU_FLAVOUR}.version", which is
correct for standard QEMU. However, for TDX QEMU we should be checking
for "assets.hypervisor.${QEMU_FLAVOUR}.tag"
Fixes: #6668
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When testing on AKS, we've been hitting the dial_timeout every now and
then. Let's increase it to 45 seconds (instead of 30) for all the VMMs,
and to 60 seconfs in case of TEEs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The agent now offloads cgroup configuration to systemd when
possible. This requires to enable D-Bus in order to communicate
with systemd.
Fixes#6657
Signed-off-by: Greg Kurz <groug@kaod.org>
Booting up TDX takes more time than booting up a normal VM. Those
values are being already used as part of the CCv0 branch, and we're just
bringing them to the `main` branch as well.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's ensure the node is ready after the CRI Engine restart, otherwise
we may proceed and scripts may simply fail if they try to deploy a pod
while the CRI Engine is not yet restarted (and, consequently, the node
is not Ready).
Related: #6649
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
readinessProbe will help us to only have the kata-deploy pod marked as
Ready when it finishes all the needed configurations in the node.
Related: #6649
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As TEEs cannot hotplug memory / CPU, we *must* consider the default
values for those as part of the podOverhead.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the TDX machine is using k3s, let's make sure we're deploying
kat-deploy using the k3s overlay.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We must ensure that no kata-deploy is left behind after the tests
finish, otherwise it may interfere with the next run.
Fixes: #6647
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The socket file for shim management is created in /run/kata
and it isn't deleted after the container is stopped. After
running and stopping thousands of containers /run folder
will run out of space.
Fixes#6622
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Co-authored-by: Greg Kurz <groug@kaod.org>
If conf_guest is set we need to update the CONFIG_LOCALVERSION
to match the suffix created in install_kata
-nvidia-gpu-{snp|tdx}, the linux headers will be named the very
same if build with make deb-pkg for TDX or SNP.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
1. when we use nerdctl to setup network for kata, no netns is created by
nerdctl, kata need to create netns by its own
2. after start VM, nerdctl will call cni plugin via oci hook, we need to
rescan the netns after the interfaces have been created, and hotplug
the network device into the VM
Fixes:#4693
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Now that we've added a TDX capable external runner, let's make sure we
also run the basic tests using TDX.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure we configure containerd for the kata-qemu-tdx handler
and ship the kata-qemu-tdx runtime class for kubernetes.
Fixes: #6537
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the QEMU configuration for TDX differs quite a lot from the normal
QEMU configuration, let's add a new configuration file for the QEMU TDX.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Since TDX doesn't support readonly memslot, TDVF cannot be mapped as
pflash device and it actually works as RAM. "-bios" option is chosen to
load TDVF.
OVMF is the opensource firmware that implements the TDVF support. Thus
the command line to specify and load TDVF is ``-bios OVMF.fd``
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure we also check /sys/firmwares/tdx for TDX guest
protection, as the location may depend on whether TDX Seam is being used
or not.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the ability to cache OVMF, which right now we're only building
and shipping it for TDX.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's build the OVMF with TDX support as part of our tests, and let's
ship it as part of our releases.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the needed targets and modifications to be able to build
OVMF for TDX as part of the local-build scripts.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's update the OVMF for TDX version to what's the latest tested
release of the Intel TDX tools with Kata Containers.
This change requires a newer version of `nasm` than the one provided by
the container used to build the project. This change will also be
needed for SEV-SNP and was originally done by Alex Carter (thanks!).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
As we'll be using this from different places in the near future, let's
create a helper function as part of the libs.sh.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make users aware of the cache_components_main.sh that they can
also cache the kernel-tdx-experimental builds.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's build the kernel with TDX support as part of our tests, and let's
ship it as part of our releases.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the needed targets and modifications to be able to build
kernel-tdx-experimental as part of the local-build scripts.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's create a `install_kernel_helper()` function, as it was already
done for QEMU, and rely on that when calling `install_kernel` and
`install_kernel_dragonball_experimental`.
This helps us to reduce the code duplication by a fair amount.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's update the Kernel TDX version to what's the latest tested release
of the Intel TDX tools with Kata Containers.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Although we've been providing users a way to build kernel with TDX
support, this must be moved to its own experimental entry instead of how
it currently is.
The reason for that is because the patches are not yet merged into
kernel, and this is still an experimental build of the project.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's do what we already did when caching the kernel, and allow passing
a FLAVOUR of the project to build.
By doing this we can re-use the same function used to cache QEMU to also
cache any kind of experimental QEMU that we may happen to have.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the needed targets and modifications to be able to build
qemu-tdx-experimental as part of the local-build scripts.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure the `qemu_suffix` and `qemu_tarball_name` can be
specified. With this we make it really easy to reuse this script for
any addition flavour of an experimental QEMU that ends up having to be
built (specifically looking at the ones for Confidential Containers
here).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's update the QEMU TDX version to what's the latest tested release of
the Intel TDX tools with Kata Containers.
In order to do such update, we had to relax the checks on the QEMU
version for some of the configuration options, as those were removed
right after the window was open for the 7.1.0 development (thus the
7.0.50 check).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Although we've been providing users a way to build QEMU with TDX
support, this must be moved to its own experimental entry instead of how
it currently is.
The reason for that is because the patches are not yet merged into QEMU,
and this is still an experimental build of the project.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the dragonball kernel is shipped as part of our releases, it must be
added to the `all` target.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In order to make it easier to read, let's just rename the
install_dragonball_experimental_kernel and install_experimental_kernel
to install_kernel_dragonball_experimental and
install_kernel_experimental, respectively.
This allows us to quickly get to those functions when looking for
`install_kernel`.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Now that the infra for running dragonball tests has been enabled, let's
actually make sure to have them running on each PR.
The tests skipped are:
* `k8s-cpu-ns.bats`, as CPU resize doesn't seem to be yet properly
supported on runtime-rs
* https://github.com/kata-containers/kata-containers/issues/6621Fixes: #6605
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
A sandbox annotation used to specify prefetch_files.list
path the container image being used, and runtime will pass
it to Hypervisor to search for corresponding prefetch file:
format looks like:
"io.katacontainers.config.hypervisor.prefetch_files.list"
= /path/to/<uid>/xyz.com/fedora:36/prefetch_file.list
Fixes: #6582
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
1. when do the deserialization for the oci hook, we should use camel
case for createRuntime
2. we should pass the dir of bundle path instead of the path of
config.json
Fixes:#4693
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
I should have seen this coming, but currently the "create" and "delete"
AKS workflows cannot be imported and uses as a job's step, resulting on
an error trying to find the correspondent action.yaml file for those.
Fixes: #6630
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's ensure we're only running this workflow when PRs are opened
against the main branch.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit a159ffdba7.
Unfortunately we have to revert the PRs related to the switch done to
using `workflow_run` instead of `pull_request_target`. The reason for
that being that we can only mark jobs as required if they are targetting
PRs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit 3a760a157a.
Unfortunately we have to revert the PRs related to the switch done to
using `workflow_run` instead of `pull_request_target`. The reason for
that being that we can only mark jobs as required if they are targetting
PRs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit 7855b43062.
Unfortunately we have to revert the PRs related to the switch done to
using `workflow_run` instead of `pull_request_target`. The reason for
that being that we can only mark jobs as required if they are targetting
PRs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit 85cc5bb534.
Unfortunately we have to revert the PRs related to the switch done to
using `workflow_run` instead of `pull_request_target`. The reason for
that being that we can only mark jobs as required if they are targetting
PRs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've been currently using {create,delete}_aks as jobs. However, it
means that if the tests fail we'll end up deleting the AKS cluster (as
expected), but not having a way to recreate the cluster without
re-running all jobs, which is a waste of resources.
Fixes: #6628
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add support for virtiofsd when virtio_fs_extra_args with
"-o cache auto, ..." users specified.
Fixes: #6615
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This was missed from the last series, as GHA will use the "target
branch" yaml file to start the workflow.
Basically we changed the name of the cluster created to stop relying on
the PR number, as that's not easily accessible on `workflow_run`.
Fixes: #6611
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It's been pointed out that D4s_v5 instances are more powerful than the
D4s_v3 ones, and have the very same price. With this in mind, let's
switch to the newer machines.
Fixes: #6606
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
With the changes proposed as part of this PR, an AKS cluster will be
created but no tests will be performed.
The reason we have to do this is because GitHub Actions will only run
the tests using the workflows that are part of the **target** branch,
instead of the using the ones coming from the PR, and we didn't find yet
a way to work this around.
Once this commit is in, we'll actually change the tests themselves (not
the yaml files for the actions), as those will be the ones we want as
the checkout action helps us on this case.
Fixes: #6583
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
56331bd7bc oversaw the fact that we
mistakenly tried to push the build containers to the registry for a PR,
rather than doing so only when the code is merged.
As the workflow is now shared between different actions, let's introduce
an input variable to specify which are the cases we actually need to
perform a push to the registry.
Fixes: #6592
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's just print "to the registry" instead of printing "to quay.io", as
the registry used is not tied to quay.io.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We made registry / repo mandatory, but we only adapted that to the amd64
job. Let's fix it now and make sure this is also passed to the arm64
and s390x jobs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we're using the `workflow_run` event, the checkout action would
pull the **current target branch** instead of the PR one.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The way previously used to get the PR's commit sha can only be used with
`pull_request*` kind of events.
Let's adapt it to the `workflow_run` now that we're using it.
With this change we ended up dropping the PR number from the tarball
suffix, as that's not straightforward to get and, to be honest, not a
unique differentiator that would justify the effort.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make this workflow dependent of the commit message check, and only
start it if the commit message check one passes.
As a side effect, this allows us to run this specific workflow using
secrets, without having to rely on `pull_request_target`.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As already done for Cloud Hypervisor and QEMU, let's make sure we can
run the AKS tests using dragonball.
Fixes: #6583
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Two different kernel build targets (build,install) have both instructions to
build the kernel, hence it was executed twice. Install should only do
install and build should only do build.
Fixes: #6588
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This is less secure than running the PR on `pull_request`, and will
require using an additional `ok-to-test` label to make sure someone
deliverately ran the actions coming from a forked repo.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's switch to using the `ghcr.io` registry for the k8s CI, as this
will save us some troubles on running the CI with PRs coming from forked
repos.
Fixes: #6587
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In cases where the D-Bus connection fails, add a little additional context about
the origin of the error.
Fixes: 6561
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
Suggested-by: Archana Shinde <archana.m.shinde@intel.com>
Spell-checked-by: Greg Kurz <gkurz@redhat.com>
This workflow becomes redundant as we're already testing kubernetes
using kata-deploy, and also testing it on AKS.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is the very first step to replacing the Jenkins CI, and I've
decided to start with an x86_64 approach only (although easily
expansible for other arches as soon as they're ready to switch), and to
start running our kubernetes tests (now running on AKS).
Fixes: #6541
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will be shortly used as part of a newly created GitHub action which
will replace our Jenkins CI.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Those will be shortly used as part of a newly added GitHub action for
testing k8s tests on Azure.
They've been created using the secrets we already have exposed as part
of our GitHub, and they follow a similar way to authenticate to Azure /
create an AKS cluster as done in the `/test-kata-deploy` action.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The first part of simplifying things to have all our tests using GitHub
actions is moving the k8s tests to this repo, as those will be the first
vict^W targets to be migrated to GitHub actions.
Those tests have been slightly adapted, mainly related to what they load
/ import, so they are more self-contained and do not require us bringing
a lot of scripts from the tests repo here.
A few scripts were also dropped along the way, as we no longer plan to
deploy kubernetes as part of every single run, but rather assume there
will always be k8s running whenever we land to run those tests.
It's important to mention that a few tests were not added here:
* k8s-block-volume:
* k8s-file-volume:
* k8s-volume:
* k8s-ro-volume:
These tests depend on some sort of volume being created on the
kubernetes node where the test will run, and this won't fly as the
tests will run from a GitHub runner, targetting a different machine
where kubernetes will be running.
* https://github.com/kata-containers/kata-containers/issues/6566
* k8s-hugepages: This test depends a whole lot on the host where it
lands and right now we cannot assume anything about that anymore, as
the tests will run from a GitHub runner, targetting a different
machine where kubernetes will be running.
* https://github.com/kata-containers/kata-containers/issues/6567
* k8s-expose-ip: This is simply hanging when running on AKS and has to
be debugged in order to figure out the root cause of that, and then
adapted to also work on AKS.
* https://github.com/kata-containers/kata-containers/issues/6578
Till those issues are solved, we'll keep running a jenkins job with
hose tests to avoid any possible regression.
Last but not least, I've decided to **not** keep the history when
bringing those tests here, otherwise we'd end up polluting a lot the
history of this repo, without any clear benefit on doing so.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We can easily re-use the newly added build-kata-static-tarball-*.yaml as
part of the release.yaml file.
By doing this we consolidate on how we build the components accross our
actions.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's split those actions into two different ones:
* Build the kata-static tarball
* Publish the kata-deploy payload
We're doing this as, later in this series we'll start taking advantage
of both pieces.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR updates the url for the Container Network Model
in the network document.
Fixes#6563
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
There can be an error while connecting to the cgroups managager, for
example a `ENOENT` if a file is not found. Make sure that this is
reported through the proper channels instead of causing a `panic()`
that does not provide much information.
Fixes: #6561
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
Reported-by: Greg Kurz <gkurz@redhat.com>
The kata monitor metrics API returns a huge size response,
if containers or sandboxs are a large number,
focus on what we need will be harder.
Fixes: #6500
Signed-off-by: Miao Xia <xia.miao1@zte.com.cn>
- nydus: upgrad to v2.2.0
- osbuilder: Add support for CBL-Mariner
- kata-deploy: Fix bash semantics error
- make only_kata work without -f
- runtime-rs: ch: Implement confidential guest handling
- qemu/arm64: disable image nvdimm once no firmware offered
- static checks workflow improvements
- A couple of kata-deploy fixes
- agent: Bring in VFIO-AP device handling again
- bugfix: set hostname in CreateSandboxRequest
- packaging / kata-deploy builds: Add the ability to cache and consume cached components
- versions: Update firecracker version
- dependency: update cgroups-rs
- Built-in Sandbox: add more unit tests for dragonball. Part 6
- runtime: add support for Hyper-V
- runtime-rs: update load_config comment
- Add support for ephemeral mounts to occupy entire sandbox's memory
- runtime-rs: fix default kernel location and add more default config paths
- Implement direct-volume commands handler for shim-mgmt
- bugfix: modify tty_win info in runtime when handling ResizePtyRequest
- bugfix: add get_ns_path API for Hypervisor
- runtime-rs: add the missing default trait
- packaging: Simplify get_last_modification()
- utils: Make kata-manager.sh runs checks
- dragonball: support pmu on aarch64
- docs: fix typo in key filename in AWS installation guide
- backport rustjail systemd cgroup fix#6331 to 3.1
- main | kata-deploy: Fix kata deploy arm64 image build error
- workflows: Yet more fixes for publishing the kata-deploy payload after every PR merged
- rustjail: fix cgroup handling in agent-init mode
- runtime/Makefile: Fix install-containerd-shim-v2 dependency
- fix wrong notes for func GetSandboxesStoragePathRust()
- fix(runtime-rs): add exited state to ensure cleanup
- runtime-rs: add oci hook support
- utils: Remove kata-manager.sh cgroups v2 check
- workflows: Fixes for the `payload-after-push` action
- Dragonball: update dependencies
- workflows: Do not install docker
- workflows: Publish kata-deploy payload after a merge
- src: Fixed typo mod.rs
- actions: Use `git-diff` to get changes in kernel dir
- agent: don't set permission of existing directory in copy_file
- runtime: use filepath.Clean() to clean the mount path
- Upgrade to Cloud Hypervisor v30.0
- feat(runtime): make static resource management consistent with 2.0
- osbuilder: Include minimal set of device nodes in ubuntu initrd
- kata-ctl/exec: add new command exec to enter guest VM.
- kernel: Add CONFIG_SEV_GUEST to SEV kernel config
- runtime-rs: Improve Cloud Hypervisor config handling
- virtiofsd: update to a valid path on ppc64le
- runtime-rs: cleanup kata host share path
- osbuilder: fix default build target in makefile
- devguide: Add link to the contribution guidelines
- kata-deploy: Ensure go binaries can run on Ubuntu 20.04
- dragonball: config_manager: preserve device when update
- Revert "workflows: Push the builder image to quay.io"
- Remove all remaining unsafe impl
- kata-deploy: Fix building the kata static firecracker arm64 package occurred an error
- shim-v2: Bump Ubuntu container image to 22.04
- packaging: Cache the container used to build the kata-deploy artefacts
- utils: always check some dependencies.
- versions: Use ubuntu as the default distro for the rootfs-image
- github-action: Replace deprecated command with environment file
- docs: Change the order of release step
- runtime-rs: remove unnecessary Send/Sync trait implement
- runtime-rs: Don't build on Power, don't break on Power.
- runtime-rs: handle sys_dir bind volume
- sandbox: set the dns for the sandbox
- packaging/shim-v2: Only change the config if the file exists
- runtime-rs: Add basic CH implementation
- release: Revert kata-deploy changes after 3.1.0-rc0 release
8b008fc743 kata-deploy: fix bash semantics error
74ec38cf02 osbuilder: Add support for CBL-Mariner
ac58588682 runtime-rs: ch: Generate Cloud Hypervisor config for confidential guests
96555186b3 runtime-rs: ch: Honour debug setting
e3c2d727ba runtime-rs: ch: clippy fix
ece5edc641 qemu/arm64: disable image nvdimm if no firmware offered
dd23f452ab utils: renamed only_kata to skip_containerd
59c81ed2bb utils: informed pre-check about only_kata
4f0887ce42 kata-deploy: fix install failing to chmod runtime-rs/bin/*
09c4828ac3 workflows: add missing artifacts on payload-after-push
fbf891fdff packaging: Adapt `get_last_modification()`
82a04dbce1 local-build: Use cached VirtioFS when possible
3b99004897 local-build: Use cached shim v2 when possible
1b8c5474da local-build: Use cached RootFS when possible
09ce4ab893 local-build: Use cached QEMU when possible
1e1c843b8b local-build: Use cached Nydus when possible
64832ab65b local-build: Use cached Kernel when possible
04fb52f6c9 local-build: Use cached Firecracker when possible
8a40f6f234 local-build: Use cached Cloud Hypervisor when possible
194d5dc8a6 tools: Add support for caching VirtioFS artefacts
a34272cf20 tools: Add support for caching shim v2 artefacts
7898db5f79 tools: Add support for caching RootFS artefacts
e90891059b tools: Add support for caching QEMU artefacts
7aed8f8c80 tools: Add support for caching Nydus artefacts
cb4cbe2958 tools: Add support for caching Kernel artefacts
762f9f4c3e tools: Add support for caching Firecracker artefacts
6b1b424fc7 tools: Add support for caching Cloud Hypervisor artefacts
08fe49f708 versions: Adjust kernel names to match kata-deploy build targets
99505c0f4f versions: Update firecracker version
f4938c0d90 bugfix: set hostname
96baa83895 agent: Bring in VFIO-AP device handling again
f666f8e2df agent: Add VFIO-AP device handling
b546eca26f runtime: Generalize VFIO devices
4c527d00c7 agent: Rename VFIO handling to VFIO PCI handling
db89c88f4f agent: Use cfg-if for s390x CCW
68a586e52c agent: Use a constant for CCW root bus path
a8b55bf874 dependency: update cgroups-rs
97cdba97ea runtime-rs: update load_config comment
974a5c22f0 runtime: add support for Hyper-V
40f4eef535 build: Use the correct kernel name
a6c67a161e runtime: add support for ephemeral mounts to occupy entire sandbox memory
844bf053b2 runtime-rs: add the missing default trait
e7bca62c32 bugfix: modify tty_win info in runtime when handling ResizePtyRequest
30e235f0a1 runtime-rs: impl volume-resize trait for sandbox
e029988bc2 bugfix: add get_ns_path API for Hypervisor
42b8867148 runtime-rs: impl volume-stats trait for sandbox
462d4a1af2 workflows: static-checks: Free disk space before running checks
e68186d9af workflows: static-checks: Set GOPATH only once
439ff9d4c4 tools/osbuilder/tests: Remove TRAVIS variable
43ce3f7588 packaging: Simplify get_last_modification()
33c5c49719 packaging: Move repo_root_dir to lib.sh
16e2c3cc55 agent: implement update_ephemeral_mounts api
3896c7a22b protocol: add updateEphemeralMounts proto
23488312f5 agent: always use cgroupfs when running as init
8546387348 agent: determine value of use_systemd_cgroup before LinuxContainer::new()
736aae47a4 rustjail: print type of cgroup manager
dbae281924 workflows: Properly set the kata-tarball architecture
76b4591e2b tools: Adjust the build-and-upload-payload.sh script
cd2aaeda2a kata-deploy: Switch to using an ubuntu image
2d43e13102 docs: fix typo in AWS installation guide
760f78137d dragonball: support pmu on aarch64
9bc7bef3d6 kata-deploy: Fix path to the Dockerfile
78ba363f8e kata-deploy: Use different images for s390x and aarch64
6267909501 kata-deploy: Allow passing BASE_IMAGE_{NAME,TAG}
3443f558a6 nydus: upgrad nydus to v2.2.0
395645e1ce runtime: hybrid-mode cause error in the latest nydusd
f8e44172f6 utils: Make kata-manager.sh runs checks
f31c79d210 workflows: static-checks: Remove TRAVIS_XXX variables
8030e469b2 fix(runtime-rs): add exited state to ensure cleanup
7d292d7fc3 workflows: Fix the path of imported workflows
e07162e79d workflows: Fix action name
dd2713521e Dragonball: update dependencies
bd1ed26c8d workflows: Publish kata-deploy payload after a merge
fea7e8816f runtime-rs: Fixed typo mod.rs
a9e2fc8678 runtime/Makefile: Fix install-containerd-shim-v2 dependency
b6880c60d3 logging: Correct the code notes
12cfad4858 runtime-rs: modify the transfer to oci::Hooks
828d467222 workflows: Do not install docker
4b8a5a1a3d utils: Remove kata-manager.sh cgroups v2 check
2c4428ee02 runtime-rs: move pre-start hooks to sandbox_start
e80c9f7b74 runtime-rs: add StartContainer hook
977f281c5c runtime-rs: add CreateContainer hook support
875f2db528 runtime-rs: add oci hook support
ecac3a9e10 docs: add design doc for Hooks
3ac6f29e95 runtime: clh: Re-generate the client code
262daaa2ef versions: Upgrade to Cloud Hypervisor v30.0
192df84588 agent: always use cgroupfs when running as init
b0691806f1 agent: determine value of use_systemd_cgroup before LinuxContainer::new()
dc86d6dac3 runtime: use filepath.Clean() to clean the mount path
c4ef5fd325 agent: don't set permission of existing directory
3483272bbd runtime-rs: ch: Enable initrd usage
fbee6c820e runtime-rs: Improve Cloud Hypervisor config handling
1bff1ca30a kernel: Add CONFIG_SEV_GUEST to SEV kernel config Adding kernel config to sev case since it is needed for SNP and SNP will use the SEV kernel. Incrementing kernel config version to reflect changes
ad8968c8d9 rustjail: print type of cgroup manager
b4a1527aa6 kata-deploy: Fix static shim-v2 build on arm64
2c4f8077fd Revert "shim-v2: Bump Ubuntu container image to 22.04"
afaccf924d Revert "workflows: Push the builder image to quay.io"
4c39c4ef9f devguide: Add link to the contribution guidelines
76e926453a osbuilder: Include minimal set of device nodes in ubuntu initrd
697ec8e578 kata-deploy: Fix kata static firecracker arm64 package build error
ced3c99895 dragonball: config_manager: preserve device when update
da8a6417aa runtime-rs: remove all remaining unsafe impl
0301194851 dragonball: use crossbeam_channel in VmmService instead of mpsc::channel
9d78bf9086 shim-v2: Bump Ubuntu container image to 22.04
3cfce5a709 utils: improved unsupported distro message.
919d19f415 feat(runtime): make static resource management consistent with 2.0
b835c40bbd workflows: Push the builder image to quay.io
781ed2986a packaging: Allow passing a container builder to the scripts
45668fae15 packaging: Use existing image to build td-shim
e8c6bfbdeb packaging: Use existing image to build td-shim
3fa24f7acc packaging: Add infra to push the OVMF builder image
f076fa4c77 packaging: Use existing image to build OVMF
c7f515172d packaging: Add infra to push the QEMU builder image
fb7b86b8e0 packaging: Use existing image to build QEMU
d0181bb262 packaging: Add infra to push the virtiofsd builder image
7c93428a18 packaging: Use existing image to build virtiofsd
8c227e2471 virtiofsd: Pass the expected toolchain to the build container
7ee00d8e57 packaging: Add infra to push the shim-v2 builder image
24767d82aa packaging: Use existing image to build the shim-v2
e84af6a620 virtiofsd: update to a valid path on ppc64le
6c3c771a52 packaging: Add infra to push the kernel builder image
b9b23112bf packaging: Use existing image to build the kernel
869827d77f packaging: Add push_to_registry()
e69a6f5749 packaging: Add get_last_modification()
6c05e5c67a packaging: Add and export BUILDER_REGISTRY
1047840cf8 utils: always check some dependencies.
95e3364493 runtime-rs: remove unnecessary Send/Sync trait implement
a96ba99239 actions: Use `git-diff` to get changes in kernel dir
619ef54452 docs: Change the order of release step
a161d11920 versions: Use ubuntu as the default distro for the rootfs-image
be40683bc5 runtime-rs: Add a generic powerpc64le-options.mk
47c058599a packaging/shim-v2: Install the target depending on the arch/libc
b582c0db86 kata-ctl/exec: add new command exec to enter guest VM.
07802a19dc runtime-rs: handle sys_dir bind volume
04e930073c sandbox: set the dns for the sandbox
32ebe1895b agent: fix the issue of creating the dns file
44aaec9020 github-action: Replace deprecated command with environment file
a68c5004f8 packaging/shim-v2: Only change the config if the file exists
ee76b398b3 release: Revert kata-deploy changes after 3.1.0-rc0 release
bbc733d6c8 docs: runtime-rs: Add CH status details
37b594c0d2 runtime-rs: Add basic CH implementation
545151829d kata-types: Add Cloud Hypervisor (CH) definitions
2dd2421ad0 runtime-rs: cleanup kata host share path
0a21ad78b1 osbuilder: fix default build target in makefile
9a01d4e446 dragonball: add more unit test for virtio-blk device.
Signed-off-by: Greg Kurz <groug@kaod.org>
Our CI and release process are currently taking advantage of the
kata-deploy local build scripts to build the artefacts.
Having snap doing the same is the next logical step, and it will also
help to reduce, by a lot, the CI time as we only build the components
that a PR is touching (otherwise we just pull the cached component).
Fixes: #6514
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add osbuilder support to build a rootfs and image
based on the CBL-Mariner Linux distro
Fixes: #6462
Signed-off-by: Dallas Delaney <dadelan@microsoft.com>
This change provides a preliminary implementation for the Cloud Hypervisor (CH) feature ([currently
disabled](https://github.com/kata-containers/kata-containers/pull/6201))
to allow it to generate the CH configuration for handling confidential guests.
This change also introduces concrete errors using the `thiserror` crate
(see `src/runtime-rs/crates/hypervisor/ch-config/src/errors.rs`) and a
lot of unit tests for the conversion code that generates the CH
configuration from the generic Hypervisor configuration.
Fixes: #6430.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Enable Cloud Hypervisor debug based on the specified configuration
rather than hard-coding debug to be disabled.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
For now, image nvdimm on qemu/arm64 depends on UEFI/ACPI, so if there
is no firmware offered, it should be disabled.
Fixes: #6468
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
passed the only_kata variable through to pre_check, only_kata does not
abort the install when containerd is already installed.
fixes#6385
Signed-off-by: Gabe Venberg <gabevenberg@gmail.com>
The kata-deploy install method tried to `chmod +x /opt/kata/runtime-rs/bin/*` but it isn't
always true that /opt/kata/runtime-rs/bin/ exists. For example, the
s390x payload does not build the kernel-dragonball-experimental
artifacts. So let's ensure the dir exist before issuing the command.
Fixes#6494
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The kata-deploy-ci payloads for amd64 and arm64 were missing the shim-v2
and kernel-dragonball-experimental artifacts.
Fixes#6493
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The function is returning "" when called from the script used to cache
the artefacts and one difference noted between this version and the
already working one from the CCv0 is that we make sure to `pushd
${repo_root_dir}` in the CCv0 version.
Let's give it a try here and see if it solves the issue.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add support for caching VirtioFS artefacts that are generated using
the kata-deploy local-build scripts.
Right now those are not used, but we'll switch to using them very soon
as part of upcoming changes of how we build the components we test in
our CI.
Fixes: #6480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's add support for caching shim v2 artefacts that are generated using
the kata-deploy local-build scripts.
Right now those are not used, but we'll switch to using them very soon
as part of upcoming changes of how we build the components we test in
our CI.
Fixes: #6480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's add support for caching RootFS artefacts that are generated using
the kata-deploy local-build scripts.
Right now those are not used, but we'll switch to using them very soon
as part of upcoming changes of how we build the components we test in
our CI.
Fixes: #6480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's add support for caching QEMU artefacts that are generated using
the kata-deploy local-build scripts.
Right now those are not used, but we'll switch to using them very soon
as part of upcoming changes of how we build the components we test in
our CI.
Fixes: #6480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's add support for caching Nydus artefacts that are generated using
the kata-deploy local-build scripts.
Right now those are not used, but we'll switch to using them very soon
as part of upcoming changes of how we build the components we test in
our CI.
Fixes: #6480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's add support for caching Kernel artefacts that are generated using
the kata-deploy local-build scripts.
Right now those are not used, but we'll switch to using them very soon
as part of upcoming changes of how we build the components we test in
our CI.
Fixes: #6480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's add support for caching Firecracker artefacts that are generated
using the kata-deploy local-build scripts.
Right now those are not used, but we'll switch to using them very soon
as part of upcoming changes of how we build the components we test in
our CI.
Fixes: #6480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's add support for caching Cloud Hypervisor artefacts that are
generated using the kata-deploy local-build scripts.
Right now those are not used, but we'll switch to using them very soon
as part of upcoming changes of how we build the components we test in
our CI.
Fixes: #6480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's adjust the kernel names in versions.yaml so those can match the
names used as part of the kata-deploy local build scripts.
Right now this doesn't bring any benefit nor drawback, but it'll make
our life easier later on in this same series.
Depends-on: github.com/kata-containers/tests#5534
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR updates the firecracker version being used in kata containers
versions.yaml
The changes in version 1.3.1 are
Added
Introduced T2CL (Intel) and T2A (AMD) CPU templates to provide
instruction set feature parity between Intel and AMD CPUs when using
these templates.
Added Graviton3 support (c7g instance type).
Changed
Improved error message when invalid network backend provided.
Improved TCP throughput by between 5% and 15% (depending on CPU) by using
scatter-gather I/O in the net device's TX path.
Upgraded Rust toolchain from 1.64.0 to 1.66.0.
Made seccompiler output bit-reproducible.
Fixed
Fixed feature flags in T2 CPU template on Intel Ice Lake.
Fixes#6482
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR is a continuing work for (kata-containers#3679).
This generalizes the previous VFIO device handling which only
focuses on PCI to include AP (IBM Z specific).
Fixes: kata-containers#3678
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Initial VFIO-AP support (#578) was simple, but somewhat hacky; a
different code path would be chosen for performing the hotplug, and
agent-side device handling was bound to knowing the assigned queue
numbers (APQNs) through some other means; plus the code for awaiting
them was written for the Go agent and never released. This code also
artificially increased the hotplug timeout to wait for the (relatively
expensive, thus limited to 5 seconds at the quickest) AP rescan, which
is impractical for e.g. common k8s timeouts.
Since then, the general handling logic was improved (#1190), but it
assumed PCI in several places.
In the runtime, introduce and parse AP devices. Annotate them as such
when passing to the agent, and include information about the associated
APQNs.
The agent awaits the passed APQNs through uevents and triggers a
rescan directly.
Fixes: #3678
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Generalize VFIO devices to allow for adding AP in the next patch.
The logic for VFIOPciDeviceMediatedType() has been changed and IsAPVFIOMediatedDevice() has been removed.
The rationale for the revomal is:
- VFIODeviceMediatedType is divided into 2 subtypes for AP and PCI
- Logic of checking a subtype of mediated device is included in GetVFIODeviceType()
- VFIOPciDeviceMediatedType() can simply fulfill the device addition based
on a type categorized by GetVFIODeviceType()
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
e.g., split_vfio_option is PCI-specific and should instead be named
split_vfio_pci_option. This mutually affects the runtime, most notably
how the labels are named for the agent.
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Since shimv2 create task option is already implemented, we need to update the
corresponding comments.
Also, the ordering is also updated to fit with the code.
fixes: #3961
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
This adds /dev/mshv to the list of sandbox devices so that VMMs can
create Hyper-V VMs.
In our testing, this also doesn't error out in case /dev/mshv isn't
present.
Fixes#6454.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
When calling `MAKE_KERNEL_NAME` we're considering the default kernel
name will be `vmlinux.container` or `vmlinuz.container`, which is not
the case as the runtime-rs, when used with dragonball, relies on the
`vmlinu[zx]-dragonball-experimental.container` kernel.
Other hypervisors will have to introduce a similar
`MAKE_KERNEL_NAME_${HYPERVISOR}` to adapt this to the kernel they want
to use, similarly to what's already done for the go runtime.
By doing this we also ensure that no changes in the configuration file
will be required to run runtime-rs, with dragonball, as part of our CI
or as part of kata-deploy.
Fixes: #6290
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
On hotplug of memory as containers are started, remount all ephemeral mounts with size option set to the total sandbox memory
Fixes: #6417
Signed-off-by: Sidhartha Mani <sidhartha_mani@apple.com>
Some structs in the runtime-rs don't implement Default trait.
This commit adds the missing Default.
Fixes: #5463
Signed-off-by: Li Hongyu <lihongyu1999@bupt.edu.cn>
Currently, we only create the new exec process in runtime, this will cause error
when the following requests needing to be handled:
- Task: exec process
- Task: resize process pty
- ...
The agent do not do_exec_process when we handle ExecProcess, thus we can not find
any process information in the guest when we handle ResizeProcessPty. This will
report an error.
In this commit, the handling process is modified to the:
* Modify process tty_win information in runtime
* If the exec process is not running, we just return. And the truly pty_resize will
happen when start_process
Fixes: #6248
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Implements resize-volume handlers in shim-mgmt,
trait for sandbox and add RPC calls to agent.
Note the actual rpc handler for the resize request is currently not
implemented, refer to issue #3694.
Fixes#5369
Signed-off-by: Tingzhou Yuan <tzyuan15@bu.edu>
For external hypervisors(qemu, cloud-hypervisor, ...), the ns they launch vm in
is different from internal hypervisor(dragonball). And when we doing CreateContainer
hook, we will rely on the netns path. So we add a get_ns_path API.
Fixes: #6442
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Implements get-volume-stats trait for sandbox,
handler for shim-mgmt and add RPC calls to
agent. Also added type conversions in trans.rs
Fixes#5369
Signed-off-by: Tingzhou Yuan <tzyuan15@bu.edu>
We've been seeing the 'sudo make test' job occasionally run out of space in
/tmp, which is part of the root filesystem. Removing dotnet and
`AGENT_TOOLSDIRECTORY` frees around 10GB of space and in my tests the job still
has 13GB of space left after running.
Fixes: #6401
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
{{ runner.workspace }}/kata-containers and {{ github.workspace }} resolve to
the same value, but they're being used multiple times in the workflow. Remove
multiple definitions and define the GOPATH var at job level once.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
The last remaining user of the TRAVIS variable in this repo is
tools/osbuilder/tests and it is only used to skip spinning up VMs. Travis
didn't support virtualization and the same is true for github actions hosted
runners. Replace the variable with KVM_MISSING and determine availability of
/dev/kvm at runtime.
TRAVIS is also used by '.ci/setup.sh' in kata-containers/tests to reduce the
set of dependencies that gets installed, but this is also in the process of
being removed.
Fixes: #3544
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
There's no need to pass repo_root_dir to get_last_modification() as the
variable used everywhere is exported from that very same file.
Fixes: #6431
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is used in several parts of the code, and can have a single
declaration as part of the `lib.sh` file, which is already imported by
all the places where it's used.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
- implement update_ephemeral_mounts rpc
- for each mountpoint passed in, remount it with new options
Signed-off-by: Sidhartha Mani <sidhartha_mani@apple.com>
- adds a new rpc call to the agent service named `updateEphemeralMounts`
- this call takes a list of grpc.Storage objects
Signed-off-by: Sidhartha Mani <sidhartha_mani@apple.com>
The logic to decide which cgroup driver is used is currently based on the
cgroup path that the host provides. This requires host and guest to use the
same cgroup driver. If the guest uses kata-agent as init, then systemd can't be
used as the cgroup driver. If the host requests a systemd cgroup, this
currently results in a rustjail panic:
thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: I/O error: No such file or directory (os error 2)
Caused by:
No such file or directory (os error 2)', rustjail/src/cgroups/systemd/manager.rs:44:51
stack backtrace:
0: 0x7ff0fe77a793 - std::backtrace_rs::backtrace::libunwind::trace::h8c197fa9a679d134
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
1: 0x7ff0fe77a793 - std::backtrace_rs::backtrace::trace_unsynchronized::h9ee19d58b6d5934a
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
2: 0x7ff0fe77a793 - std::sys_common::backtrace::_print_fmt::h4badc450600fc417
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:65:5
3: 0x7ff0fe77a793 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::had334ddb529a2169
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:44:22
4: 0x7ff0fdce815e - core::fmt::write::h1aa7694f03e44db2
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/fmt/mod.rs:1209:17
5: 0x7ff0fe74e0c4 - std::io::Write::write_fmt::h61b2bdc565be41b5
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/io/mod.rs:1682:15
6: 0x7ff0fe77cd3f - std::sys_common::backtrace::_print::h4ec69798b72ff254
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:47:5
7: 0x7ff0fe77cd3f - std::sys_common::backtrace::print::h0e6c02048dec3c77
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:34:9
8: 0x7ff0fe77c93f - std::panicking::default_hook::{{closure}}::hcdb7e705dc37ea6e
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:267:22
9: 0x7ff0fe77d9b8 - std::panicking::default_hook::he03a933a0f01790f
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:286:9
10: 0x7ff0fe77d9b8 - std::panicking::rust_panic_with_hook::he26b680bfd953008
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:688:13
11: 0x7ff0fe77d482 - std::panicking::begin_panic_handler::{{closure}}::h559120d2dd1c6180
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:579:13
12: 0x7ff0fe77d3ec - std::sys_common::backtrace::__rust_end_short_backtrace::h36db621fc93b005a
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:137:18
13: 0x7ff0fe77d3c1 - rust_begin_unwind
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:575:5
14: 0x7ff0fda52ee2 - core::panicking::panic_fmt::he7679b415d25c5f4
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/panicking.rs:65:14
15: 0x7ff0fda53182 - core::result::unwrap_failed::hb71caff146724b6b
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/result.rs:1791:5
16: 0x7ff0fe5bd738 - <rustjail::cgroups::systemd::manager::Manager as rustjail::cgroups::Manager>::apply::hd46958d9d807d2ca
17: 0x7ff0fe606d80 - <rustjail::container::LinuxContainer as rustjail::container::BaseContainer>::start::{{closure}}::h1de806d91fcb878f
18: 0x7ff0fe604a76 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h1749c148adcc235f
19: 0x7ff0fdc0c992 - kata_agent::rpc::AgentService::do_create_container::{{closure}}::{{closure}}::hc1b87a15dfdf2f64
20: 0x7ff0fdb80ae4 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h846a8c9e4fb67707
21: 0x7ff0fe3bb816 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h53de16ff66ed3972
22: 0x7ff0fdb519cb - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h1cbece980286c0f4
23: 0x7ff0fdf4019c - <tokio::future::poll_fn::PollFn<F> as core::future::future::Future>::poll::hc8e72d155feb8d1f
24: 0x7ff0fdfa5fd8 - tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut::h0a407ffe2559449a
25: 0x7ff0fdf033a1 - tokio::runtime::task::raw::poll::h1045d9f1db9742de
26: 0x7ff0fe7a8ce2 - tokio::runtime::scheduler::multi_thread::worker::Context::run_task::h4924ae3464af7fbd
27: 0x7ff0fe7afb85 - tokio::runtime::task::raw::poll::h5c843be39646b833
28: 0x7ff0fe7a05ee - std::sys_common::backtrace::__rust_begin_short_backtrace::ha7777c55b98a9bd1
29: 0x7ff0fe7a9bdb - core::ops::function::FnOnce::call_once{{vtable.shim}}::h27ec83c953360cdd
30: 0x7ff0fe7801d5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hed812350c5aef7a8
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/alloc/src/boxed.rs:1987:9
31: 0x7ff0fe7801d5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hc7df8e435a658960
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/alloc/src/boxed.rs:1987:9
32: 0x7ff0fe7801d5 - std::sys::unix::thread::Thread::new::thread_start::h575491a8a17dbb33
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys/unix/thread.rs:108:17
Forward the value of "init_mode" to AgentService, so that we can force cgroupfs
when systemd is unavailable.
Fixes: #5779
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Right now LinuxContainer::new() gets passed a CreateOpts struct, but then
modifies the use_systemd_cgroup field inside that struct. Pull the cgroups path
parsing logic into do_create_container, so that CreateOpts can be immutable in
LinuxContainer::new. This is just moving things around, there should be no
functional changes.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Since the cgroup manager is wrapped in a dyn now, the print in
LinuxContainer::new has been useless and just says "CgroupManager". Extend the
Debug trait for 'dyn Manager' to print the type of the cgroup manager so that
it's easier to debug issues.
Fixes: #5779
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Let's make sure the kata-tarball architecture upload / downloaded / used
is exactly the same one that we need as part of the architecture we're
using to generate the image.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Now that we've switched the base container image to using Ubuntu instead
of CentOS, we don't need any kind of extra logic to correctly build the
image for different architectures, as Ubuntu is a multi-arch image that
supports all the architectures we're targetting.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure we use a multi-arch image for building kata-deploy.
A few changes were also added in order to get systemd working inside the
kata-deploy image, due to the switch from CentOS to Ubuntu.
Fixes: #6358
Signed-off-by: SinghWang <wangxin_0611@126.com>
This commit adds support for pmu virtualization on aarch64. The
initialization of pmu is in the following order:
1. Receive pmu parameter(vpmu_feature) from runtime-rs to determine the
VpmuFeatureLevel.
2. Judge whether to initialize pmu devices and add pmu device node into
fdt on aarch64, according to VpmuFeatureLevel.
Fixes: #6168
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
As part of bd1ed26c8d, we've pointed to
the Dockerfile that's used in the CC branch, which is wrong.
For what we're doing on main, we should be pointing to the one under the
`kata-deploy` folder, and not the one under the non-existent
`kata-deploy-cc` one.
Fixes: #6343
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the image provided as part of registry.centos.org is not a multi-arch
one, at least not for CentOS 7, we need to expand the script used to
build the image to pass images that are known to work for s390x (ClefOS)
and aarch64 (CentOS, but coming from dockerhub).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's break the IMAGE build parameter into BASE_IMAGE_NAME and
BASE_IMAGE_TAG, as it makes it easier to replace the default CentOS
image by something else.
Spoiler alert, the default CentOS image is **not** multi-arch, and we do
want to support at least aarch64 and s390x in the near term future.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When update the nydusd to 2.2, the argument "--hybrid-mode" cause
the following error:
thread 'main' panicked at 'ArgAction::SetTrue / ArgAction::SetFalse is defaulted'
Maybe we should remove it to upgrad nydusd
Fixes: #6407
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Updated the `kata-manager.sh` script to make it run all the checks on
the host system before attempting to create a container. If any checks
fail, they will indicate to the user what the problem is in a clearer
manner than those reported by the container manager.
Fixes: #6281.
Signed-off-by: tg5788re <jfokugas@gmail.com>
These variables are unused since we don't use travis CI. This also allows to
remove two steps:
- 'Setup GOPATH' only printed variables
- 'Setup travis reference' modified some shell local variables that don't have
any influence on the rest of the steps
The TRAVIS var is still used by tools/osbuilder/tests to determine if
virtualization is available.
Fixes: #3544
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Set process status to exited at end of io wait, which indicate process
exited only, but stop process has not been finished. Otherwise, the
cleanup_container will be skipped.
Fixes: #6393
Signed-off-by: Yipeng Yin <yinyipeng@bytedance.com>
In `payload-after-push.yaml` we ended up mentioning cc-*.yaml workflows,
which are non existent in the main branch.
Let's adapt the name to the correct ones.
Fixes: #6343
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We have a few actions in the `payload-after-push.*.yaml` that are
referring to Confidential Containers, but they should be referring to
Kata Containers instead.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Since rust-vmm and dragonball-sandbox has introduced several updates
such as vPMU support for aarch64, we also need to update Dragonball
dependencies to include those changes.
Update:
virtio-queue to v0.6.0
kvm-ioctls to v0.12.0
dbs-upcall to v0.2.0
dbs-virtio-devices to v0.2.0
kvm-bindings to v0.6.0
Also, several aarch64 features are updated because of dependencies
changes:
1. update vcpu hotplug API.
2. update vpmu related API.
3. adjust unit test cases for aarch64 Dragonball.
fixes: #6268
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
For the architectures we know that `make kata-tarball` works as
expected, let's start publishing the kata-deploy payload after each
merge.
This will help to:
* Easily test the content of current `main` or `stable-*` branch
* Easily bisect issues
* Start providing some sort of CI/CD content pipeline for those who
need that
This is a forward-port work from the `CCv0` and groups together patches
that I've worked on, with the work that Choi did in order to support
different architectures.
Fixes: #6343
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
$ make install
make: *** No rule to make target 'containerd-shim-kata-v2', needed by 'install-containerd-shim-v2'. Stop.
Spotted when building kata-runtime with a different name for
SHIMV2_OUTPUT. For instance, trying to keep different runtime binaries
installed at the same time, one from master and another from lets say,
the CCv0 branch, with the following small change applied.
diff --git a/src/runtime/Makefile b/src/runtime/Makefile
index 95efaff78..2bab9eb75 100644
--- a/src/runtime/Makefile
+++ b/src/runtime/Makefile
@@ -231,7 +231,7 @@ SED = sed
CLI_DIR = cmd
SHIMV2 = containerd-shim-kata-v2
-SHIMV2_OUTPUT = $(bCURDIR)/$(SHIMV2)
+SHIMV2_OUTPUT = $(CURDIR)/$(SHIMV2)-ccv0
SHIMV2_DIR = $(CLI_DIR)/$(SHIMV2)
MONITOR = kata-monitor
Fixes: #6398
Signed-off-by: Eduardo Lima (Etrunko) <etrunko@redhat.com>
In this commit, we have done:
* modify the tranfer process from grpc::Hooks to oci::Hooks, so the code
can be more clean
* add more tests for create_runtime, create_container, start_container hooks
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
The latest ubuntu runners already have docker installed and trying to
install it manually will cause the following issue:
```
Run curl -fsSL https://test.docker.com/ -o test-docker.sh
Warning: the "docker" command appears to already exist on this system.
If you already have Docker installed, this script can cause trouble, which is
why we're displaying this warning and provide the opportunity to cancel the
installation.
If you installed the current Docker package using this script and are using it
again to update Docker, you can safely ignore this message.
You may press Ctrl+C now to abort this script.
+ sleep 20
+ sudo -E sh -c apt-get update -qq >/dev/null
E: The repository 'https://packages.microsoft.com/ubuntu/22.04/prod jammy Release' is no longer signed.
```
Fixes: #6390
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Removed the part in the `kata-manager.sh` script that checks if the host system only runs cgroups v2.
Fixes: #6259.
Signed-off-by: Alec Pemberton <pembek1901@gmail.com>
In some cases, network endpoints will be configured through Prestart
Hook. So network endpoints may need to be added(hotpluged) after vm
is started and also Prestart Hook is executed.
We move pre-start hook functions' execution to sandbox_start to allow
hooks running between vm_start and netns_scan easily, so that the
lifecycle API can be cleaner.
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
StartContainer will be execute in guest container namespace in Kata.
The Hook Path of this kind of hook is also in guest container namespace.
StartContainer is executed after start operation is called, and it
should be executed before user-specific command is executed.
Fixes: #5787
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
CreateContainer hook is one kind of OCI hook. In kata, it will be
executed after VM is started, before container is created, and after
CreateRuntime is executed.
The hook path of CreateContainer hook is in host runtime namespace, but
it will be executed in host vmm namespace.
Fixes: #5787
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
According to the runtime OCI Spec, there can be some hook
operations in the lifecycle of the container. In these hook
operations, the runtime can execute some commands. There are different
points in time in the container lifecycle and different hook types
can be executed.
In this commit, we are now supporting 4 types of hooks(same in
runtime-go): Prestart hook, CreateRuntime hook, Poststart hook and
Poststop hook.
Fixes: #5787
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
This patch re-generates the client code for Cloud Hypervisor v30.0.
Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by openapi-generator.
Fixes: #6375
Signed-off-by: Bo Chen <chen.bo@intel.com>
The logic to decide which cgroup driver is used is currently based on the
cgroup path that the host provides. This requires host and guest to use the
same cgroup driver. If the guest uses kata-agent as init, then systemd can't be
used as the cgroup driver. If the host requests a systemd cgroup, this
currently results in a rustjail panic:
thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: I/O error: No such file or directory (os error 2)
Caused by:
No such file or directory (os error 2)', rustjail/src/cgroups/systemd/manager.rs:44:51
stack backtrace:
0: 0x7ff0fe77a793 - std::backtrace_rs::backtrace::libunwind::trace::h8c197fa9a679d134
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
1: 0x7ff0fe77a793 - std::backtrace_rs::backtrace::trace_unsynchronized::h9ee19d58b6d5934a
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
2: 0x7ff0fe77a793 - std::sys_common::backtrace::_print_fmt::h4badc450600fc417
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:65:5
3: 0x7ff0fe77a793 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::had334ddb529a2169
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:44:22
4: 0x7ff0fdce815e - core::fmt::write::h1aa7694f03e44db2
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/fmt/mod.rs:1209:17
5: 0x7ff0fe74e0c4 - std::io::Write::write_fmt::h61b2bdc565be41b5
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/io/mod.rs:1682:15
6: 0x7ff0fe77cd3f - std::sys_common::backtrace::_print::h4ec69798b72ff254
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:47:5
7: 0x7ff0fe77cd3f - std::sys_common::backtrace::print::h0e6c02048dec3c77
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:34:9
8: 0x7ff0fe77c93f - std::panicking::default_hook::{{closure}}::hcdb7e705dc37ea6e
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:267:22
9: 0x7ff0fe77d9b8 - std::panicking::default_hook::he03a933a0f01790f
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:286:9
10: 0x7ff0fe77d9b8 - std::panicking::rust_panic_with_hook::he26b680bfd953008
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:688:13
11: 0x7ff0fe77d482 - std::panicking::begin_panic_handler::{{closure}}::h559120d2dd1c6180
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:579:13
12: 0x7ff0fe77d3ec - std::sys_common::backtrace::__rust_end_short_backtrace::h36db621fc93b005a
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:137:18
13: 0x7ff0fe77d3c1 - rust_begin_unwind
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:575:5
14: 0x7ff0fda52ee2 - core::panicking::panic_fmt::he7679b415d25c5f4
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/panicking.rs:65:14
15: 0x7ff0fda53182 - core::result::unwrap_failed::hb71caff146724b6b
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/result.rs:1791:5
16: 0x7ff0fe5bd738 - <rustjail::cgroups::systemd::manager::Manager as rustjail::cgroups::Manager>::apply::hd46958d9d807d2ca
17: 0x7ff0fe606d80 - <rustjail::container::LinuxContainer as rustjail::container::BaseContainer>::start::{{closure}}::h1de806d91fcb878f
18: 0x7ff0fe604a76 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h1749c148adcc235f
19: 0x7ff0fdc0c992 - kata_agent::rpc::AgentService::do_create_container::{{closure}}::{{closure}}::hc1b87a15dfdf2f64
20: 0x7ff0fdb80ae4 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h846a8c9e4fb67707
21: 0x7ff0fe3bb816 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h53de16ff66ed3972
22: 0x7ff0fdb519cb - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h1cbece980286c0f4
23: 0x7ff0fdf4019c - <tokio::future::poll_fn::PollFn<F> as core::future::future::Future>::poll::hc8e72d155feb8d1f
24: 0x7ff0fdfa5fd8 - tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut::h0a407ffe2559449a
25: 0x7ff0fdf033a1 - tokio::runtime::task::raw::poll::h1045d9f1db9742de
26: 0x7ff0fe7a8ce2 - tokio::runtime::scheduler::multi_thread::worker::Context::run_task::h4924ae3464af7fbd
27: 0x7ff0fe7afb85 - tokio::runtime::task::raw::poll::h5c843be39646b833
28: 0x7ff0fe7a05ee - std::sys_common::backtrace::__rust_begin_short_backtrace::ha7777c55b98a9bd1
29: 0x7ff0fe7a9bdb - core::ops::function::FnOnce::call_once{{vtable.shim}}::h27ec83c953360cdd
30: 0x7ff0fe7801d5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hed812350c5aef7a8
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/alloc/src/boxed.rs:1987:9
31: 0x7ff0fe7801d5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hc7df8e435a658960
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/alloc/src/boxed.rs:1987:9
32: 0x7ff0fe7801d5 - std::sys::unix::thread::Thread::new::thread_start::h575491a8a17dbb33
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys/unix/thread.rs:108:17
Forward the value of "init_mode" to AgentService, so that we can force cgroupfs
when systemd is unavailable.
Fixes: #5779
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Right now LinuxContainer::new() gets passed a CreateOpts struct, but then
modifies the use_systemd_cgroup field inside that struct. Pull the cgroups path
parsing logic into do_create_container, so that CreateOpts can be immutable in
LinuxContainer::new. This is just moving things around, there should be no
functional changes.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Fix path check bypassed issuse introduced by #6082,
use filepath.Clean() to clean path before check
Fixes: #6082
Signed-off-by: XDTG <click1799@163.com>
This patch fixes the issue that do_copy_file changes
the directory permission of the parent directory of
a target file, even when the parent directory already
exists.
Fixes#6367
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
This change enables to run cloud-hypervisor VMM using a non-root user
when rootless flag is set true in the configuration
Fixes: #2567
Signed-off-by: Feng Wang <fwang@confluent.io>
Allow an initrd/initramfs image to be used with Cloud Hypervisor, which
is handled differently to the default rootfs image type.
Fixes: #6335.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Replace `cloud_hypervisor_vm_create_cfg()` with a set of `TryFrom` trait
implementations in the new CH specific `convert.rs` to allow the generic
`Hypervisor` configuration to be converted into the CH specific
`VmConfig` type.
Note that device configuration is not currently handled in `convert.rs`
(it's handled in `inner_device.rs`).
This change removes the old hard-coded CH specific configuration.
Fixes: #6203.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Adding kernel config to sev case since it is needed for SNP and SNP will use the SEV kernel.
Incrementing kernel config version to reflect changes
Fixes: #6123
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
Since the cgroup manager is wrapped in a dyn now, the print in
LinuxContainer::new has been useless and just says "CgroupManager". Extend the
Debug trait for 'dyn Manager' to print the type of the cgroup manager so that
it's easier to debug issues.
Fixes: #5779
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Following Jong Wu suggestion, let's link /usr/bin/musl-gcc to
/usr/bin/aarch64-linux-musl-gcc.
Fixes: #6320
Signed-off-by: SinghWang <wangxin_0611@126.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit 9d78bf9086.
Golang binaries are built statically by default, unless linking against
CGO, which we do. In this case we dynamically link against glibc,
causing us troubles when running a binary built with Ubuntu 22.04 on
Ubuntu 20.04 (which will still be supported for the next few years ...)
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
New developers are often confused by some of our requirements, notably porting
labels. While our CONTRIBUTING.md file points to the solution, the developer's
guide does not. Add a link there.
Fixes: #6329
Signed-off-by: Christophe de Dinechin <christophe@dinechin.org>
When starting an initrd the kernel expects to find /dev/console in the initrd,
so that it can connect it as stdin/stdout/stderr to the /init process. If the
device node is missing the kernel will complain that it was unable to open an
initial console. If kata-agent is the initrd init process, it will also result
in log messages not being logged to console and thus not forwarded to host
syslog.
Add a set of standard device nodes for completeness, so that console logging
works. To do that we install the makedev packge which provides a MAKEDEV helper
that knows the major/minor numbers. Unfortunately the debian package tries to
create devnodes from postinst, which can be suppressed if systemd-detect-virt
is present. That's why we create a small dummy script that matches what
systemd-detect-virt would output (anything is enough to suppress mknod).
Fixes: #6261
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
DeviceConfigInfo contains config and device, so when we want to do
update we could simply update config part of the info, and device would
not be changed during update.
Fixes: #6324
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
Because crossbeam_channel has more features and better performance than
mpsc::channel and finally rust replace its channel implementation with
crossbeam_channel on version 1.67
Signed-off-by: Tim Zhang <tim@hyper.sh>
Let's bump the base container image to use the 22.04 version of Ubuntu,
as it does bring up-to-date package dependencies that we need to
statically build the runtime-rs on aarch64.
Fixes: #6320
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
previously, if installing on unkown distro, script would tell user that
their distro was unsupported. Changed error message prompting user to
install dependecies manually, then retry.
Signed-off-by: Gabe Venberg <gabevenberg@gmail.com>
Let's push the builder images to a registry, so we can take advantage of
those on each step of our building process.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This, combined with the effort of caching builder images *and* only
performing the build itself inside the builder images, is the very first
step for reproducible builds for the project.
Reproducible builds are quite important when we talk about Confidential
Containers, as users may want to verify the content used / provided by
the CSPs, and this is the first step towards that direction.
Fixes: #5517
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the td-shim.
This will save us some CI time.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the td-shim.
This will save us some CI time.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the needed infra for building and pushing the OVMF builder
image to the Kata Containers' quay.io registry.
Fixes: #5477
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's first try to pull a pre-existing image, instead of buildinf our
own, to be used as a builder image for OVMF.
This will save us some CI time.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the needed infra for only building and pushing the QEMU
builder image to the Kata Containers' quay.io registry.
Fixes: #5481
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's first try to pull a pre-existsing image, instead of building our
own, to be used as a builder image for QEMU.
This will save us some CI time.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the needed infra for only building and pushing the virtiofsd
builder image to the Kata Containers' quay.io registry.
Fixes: #5480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the virtiofsd.
This will save us some CI time.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's ensure we're building virtiofsd with a specific toolchain that's
known to not cause any issues, instead of always using the latest one.
On each bump of the virtiofsd, we'll make sure to adjust this according
to what's been used by the virtiofsd community.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the needed infra for only building and pushing the shim-v2
builder image to the Kata Containers' quay.io registry.
Fixes: #5478
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's try to pull a pre-existing image, instead of building our own, to
be used as a builder for the shim-v2.
This will save us some CI time.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Currently the symbolic link for virtiofsd which is used as
a valid path is not updated on every CI run. Fix it by
using the actual path of installation.
Fixes: #6311
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
Let's add the needed infra for only building and pushing the kernel
builder image to the Kata Containers' quay.io registry.
Fixes: #5476
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the kernel.
This will save us some CI time.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function will push a specific tag to a registry, whenever the
PUSH_TO_REGISTRY environment variable is set, otherwise it's a no-op.
This will be used in the future to avoid replicating that logic in every
builder used by the kata-deploy scripts.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add a function to get the hash of the last commit modifying a
specific file.
This will help to avoid writing `git rev-list ...` into every single
build script used by the kata-deploy.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
BUILD_REGISTRY, which points to quay.io/kata-containers/builder, will be
used for storing the builder images used to build the artefacts via the
kata-deploy scripts.
The plan is to tag, whenever it's possible and makes sense, images like:
* ${BUILDER_REGISTRY}:${component}-${unique_identifier}
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Every dependency in check_deps is used inside the script (apart from
git, which may be a historical artifact), and therefore should be
checked even when the -f option is passed to the script. Simply changed
at what point check_deps is called in order to always run it.
Fixes#6302.
Signed-off-by: Gabe Venberg <gabevenberg@gmail.com>
Send and Sync are automatically derived traits,
if a type is composed entirely of Send or Sync types, then it is Send or Sync.
Almost all primitives are Send and Sync,
so we don't need to implement them manually most of the time.
Fixes: #6307
Signed-off-by: Tim Zhang <tim@hyper.sh>
Use `git-diff` instead of legacy `git-whatchanged` to get
differences in the packaging/kernel directory. This also fixes
a bug by grepping for the kernel directory in the output of the
git command.
Fixes: #6210
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
When a new stable branch is created, it is necessary to change the
references in the tests repo from main to the new stable branch.
However this step needs to be performed after the repos have been tagged
as the `tags_repos.sh` script is the one that creates the new branch.
Clarify this in the documentation and move the step to change branch
references in test repo after repos have been tagged.
Fixes: #1824
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Currently ubuntu is already the default distro for all the architectures
but x86_64, which uses clearlinux. However, our CI does *not* test the
clearlinux image we ship.
Taking a look at our CI code [0], we've been using ubuntu as base for
the tests for a few years already, if not forever.
The minimum we can do is to switch to distributing ubuntu, as the tested
rootfs-image, and then decide later on whether we should switch back to
clearlinux (once we switch our CI to using that, and make sure all tests
will be green), or if we move to slimmer distro, such as alpine.
[0]: 0a39dd1a01/.ci/install_kata_image.sh (L44)Fixes: #6303
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
There's a check in the runtime-rs Makefile that basically checks whether
the `arch/$arch-options.mk` exists or not and, if it doesn't, the build
is just aborted.
With this in mind, let's create a generic powerpc64le-options.mk file
and not bail when building for this architecture.
Fixes: #6142
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In the `install_go_rust.sh` file we're adding a
x86_64-unknown-linux-musl target unconditionally. That should be,
instead, based in the ARCH of the host and the appropriate LIBC to be
used with that host.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The patchset will help users to easily enter guest VM by debug
console sock.
In order to enter guest VM smoothly, users needs to do some
configuration, options as below:
(1) Set debug_console_enabled = true with default vport 1026.
(2) Or add agent.debug_console agent.debug_console_vport=<PORT>
into kernel_params, and the vport is <PORT> you set.
The detail of usage:
$ kata-ctl exec -h
kata-ctl-exec
Enter into guest VM by debug console
USAGE:
kata-ctl exec [OPTIONS] <SANDBOX_ID>
ARGS:
<SANDBOX_ID> pod sandbox ID
Fixes: #5340
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
For some cases, users will mount system directories as bind volume.
We should not bind mount these kind of directories in the host as it does
not make sense.
Fixes: #6299
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
The rust agent had supported to set the guest dns
server in start sandbox request, thus add the dns
in the runtime side.
Fixes:#6286
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
We should make sure the dns's source file's parent
directory exist, otherwise, it would failed to create
the file directly.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
In workflow, `set-output` command is deprecated and will be disabled soon.
This commit replaces the deprecated `set-output` command with putting a
value in the environment file `$GITHUB_OUTPUT`.
Fixes#6266
Signed-off-by: jongwooo <jongwooo.han@gmail.com>
Let's not try to sed a file that doesn't exist, which may be the case
depending on the architecture we're building the shim-v2 for.
This is a partial-forward port of
f24c47ea47.
Fixes: #6293
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Removed the `` around containerd, because when you execute this as a
script it runs the containerd command within the script, which it should
not do.
Fixes#4217
Signed-off-by: Willem Dendauw <willem.dendauw@hotmail.com>
As 3.1.0-rc0 has been released, let's switch the kata-deploy / kata-cleanup
tags back to "latest", and re-add the kata-deploy-stable and the
kata-cleanup-stable files.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
- kata-deploy: Install protobuf-compiler explicitly in shim-v2 Dockerfile
- runtime: tracing: Fix missing ctx return
- runtime: add reconnect timeout for vhost user block
- SEV: Update ReducedPhysBits
- shim-v2/build.sh: Only build runtime-rs for the supported arches
- kata-ctl: Expand unit tests for CPU check
- runtime: support cgroup v2 metrics marshal guest metrics
- Typo: change tabs in comment to spaces
- rootfs: support EROFS filesystem
- versions: Update runc version
- runtime: Improve documentation of appendFDs
- Minor cleanups in make file
- main | docs: Fix missing critical steps in how-to-hotplug-memory-arm64.md
- Action check kernel config version
- clh: Enforce API timeout only for vm.boot request
- virtiofsd: change cache mod to const
- runtime-rs: ignor "no such process" error when delete cgroup for a thread to let it go
- kernel: Add console kernel config for s390
- runtime: remove not used shim configurations
- improvement: Fix naming conventions for span name and log subsystem
- Dragonball: add cpu resize ability
- arm64/CI: fix unit test failure on arm64
- CI: Make docker version stick to v20.10 in ubuntu:20.04 for s390x|ppc64le
- virtiofsd: fix the build on ppc64le
- runtime:fix stat uds path
- cni: Update cni plugins version to 1.2.0
- Built-in Sandbox: add more unit tests for dragonball. Part 5
- runtime: Drop QEMU log file support
- docs: Add documentation for building agent with seccomp support.
- Add kernel-dragonball-experimental to kata-deploy, kata-deploy-test, and the release
- runtime-rs: add missing config section for share-fs
- runtime: Add hmp for qemu
- upcall: add document for upcall
- runtime: Start QEMU undaemonized and get logs
- docs: Update url link in QAT documentation
- versions: update cni plugins version
- versions: Upgrade to Cloud Hypervisor v29.0
- runtime: Use consts in `kata-runtime check`
- versions: Bump QEMU to v7.2.0
- agent: Eliminate unnecessary metrics
- runtime:all APIs are hang in the service.mu
- Utility functions for kata-env
- versions: Update conmon version
- runtime: paas enablevhostuserstore annotation to hypervisor config
- runk: Upgrade liboci-cli to v0.0.4
- runtime: use system pagesize for hugepage test
- dependency: update cgroups-rs
- runtime: Use git rev-parse for the kata-monitor tag
- virtcontainers: split out linux-specific bits for mount, factory
- Add darwin skeletons
- vendor: revendor netlink to get latest
- Address issues with the initial vCPU pinning functionality
- virtcontainers: Fix misspelling in error message
- runtime: add test generated file to .gitignore
- runtime: fix up disable_netns handling
- docs: add hint of probing loop module
- tools: add --locked option for cargo install
- runtime-rs: add Single Container support
- virtcontainers: tests: Ensure Linux specific tests are just run on Linux
- Change cache mode from none to never
- tools: Fix indentation for setup aks script
- virtcontainers: fs_share: Add Darwin skeleton
- virtcontainers: Add a Virtualization.framework skeleton
- kata-ctl: remove get_kata_version_by_url function
- kata-ctl: fix build error on s390x
- virtcontainers: Introduce hypervisor_darwin
- runtime: Define Darwin handled signals list
- nydus: net-ns handling needs to be only executed on Linux hosts
- clh: Ensure it works with Docker / Moby
- agent: refactor guest hooks
- fix moby prestart hook handling
- schedcore: Make buildable on !linux
- Built-in Sandbox: add more unit tests for dragonball. Part 4
- runtime-rs: cleanup the run dir of hypervisor when shut down
- Feat: implementation of kata-ctl direct-volume operations
- Runtime: Clarify mutability of global var
- kata-runtime: add rust runtime path for kata-runtime exec
- versions: Upgrade to Cloud Hypervisor v28.1
- runtime-rs: add dbs-upcall feature
- runtime/Makefile: Get some bits happy on darwin
- docs: remove old and misleading instructions for minikube
- packaging: fix indents in build-kernel.sh
- kernel: adding kmod to do docker env
- versions: Update the rust toolchain to 1.66.0
- kata-ctl: skip test if access GitHub.com fail
- agent: unset `CC` for cross-build
- runtime-rs: enable hugepage
- runtime-rs: Clean up mount points shared to guest
- kata-ctl: fix checkcpu bug in non-x86 arches
d144ded12 release: Adapt kata-deploy for 3.1.0-rc0
8e3863cec kata-deploy: Install protobuf-compiler explicitly in shim-v2 Dockerfile
c45391991 runtime: tracing: Fix missing ctx return
4139d68d5 runtime-rs: Include target install in conditional branch
ca02c9f51 runtime: add reconnect timeout for vhost user block
2f5bc0f40 kata-ctl: Expand unit tests for CPU check
67b8f0773 SEV: Update ReducedPhysBits
bdf20b5d2 rootfs: support EROFS filesystem
fff0e50a7 versions: Update runc version
ed02c8a05 docs: add guide for building rootfs with EROFS
01765e173 runtime: support cgroup v2 metrics marshal guest metrics
49326fe4e fix(clippy): fix hypervisor clippy checks
94b1d9814 cargo: Update Cargo.lock files
f1855594a make: Get rid of verbose output while creating tar
c3836010a make: clean up obsolete targets
ac64b021a clh: Enforce API timeout only for vm.boot request
56071c6e7 virtiofsd: change cache mod to const
5d37d31ac cgroups: upgrade cgroupfs to 0.3.1
ab59a65c9 runtime-rs: neglect a certain error when delete cgroup
390916b33 runtime: remove not used shim configurations
9794c52c6 improvement: Fix naming conventions for span name and log subsystem
f49b89b63 CI: Set docker version to v20.10 in ubuntu:20.04 for s390x|ppc64le
3c24e2340 README: Update Readme under packaging/kernel
d73f3a8a2 github-action: Add step to verify kernel config version id updated
59f104c02 runtime: skip unit test that fail regularly on aarch64
b7dd97cac kata-ctl: fix permission deny issue in test_add_remove
57c5e5629 Dragonball: add cpu resize ability
3c48f2202 runtime: Improve documentation of appendFDs
856ab6687 virtiofsd: fix the build on ppc64le
f83115a83 docs: Fix missing critical steps in how-to-hotplug-memory-arm64.md
e071d9251 Typo: change tabs in comment to spaces
56f0a27fe kernel: Add console kernel config for s390
334c4b8bd runtime: Drop QEMU log file support
3a63e3c1f cni: Update cni plugins version to 1.2.0
510798155 dragonball: Improve test cases
dc90c6e30 dragonball: add more unit test for vm
c07135535 runtime-rs: Improve s390x error message
4e2db96ef runtime-rs: Don't try to build on Power
8e8c720d5 kata-deploy-push: Ensure we build Dragonball specific kernel
1e531b44d runtime:fix stat uds path
9092c23a2 runtime: Add hmp for qemu
b7f4e96ff kata-deploy-test: Ensure we build dragonball specific kernel
063dec37c release: Add the dragonball-experimental kernel
0b3c91d2a kata-deploy: Add kernel-dragonball-experimental target
00dcd900f docs: Add documentation for building agent with seccomp support.
2b779cba0 docs: Update url link in QAT documentation
39fe4a4b6 runtime: Collect QEMU's stderr
a5319c6be runtime: Start QEMU undaemonized
bf4e3a618 runtime: Launch QEMU with cmd.Start()
8a1723a5c runtime: Pre-establish the QMP connection
8a4f08cb0 govmm: Optionally pass QMP listener to QEMU
219bb8e7d govmm: Optionally start QMP with a pre-configured connection
a85d0e465 versions: update cni plugins version
676d02850 versions: Bump QEMU to v7.2.0
861c38b6a versions: Upgrade to Cloud Hypervisor v29.0
ba87e0afe runtime: Use consts in `kata-runtime check`
9f490d16f upcall: add document for upcall
596037e20 versions: Update conmon version
095e8fdef runk: Use the original Kill command instead of the customed it.
0f9e23a3d runk: Upgrade liboci-cli to v0.0.4
69fc8de71 runtime:all APIs are hang in the service.mu
8d4c2cf1b kata-ctl: Allow certain constants to go unused
64c11a66f kata-ctl: Have function to get cpu details to run on specific arch
923cd3fda virtcontainers: split out Linux parts from mount
cf1bae352 runtime: paas enablevhostuserstore annotation to hypervisor config
1592a385e dependency: update cgroups-rs
60ff230d8 virtcontainers: Split the factory package into Linux and Darwin bits
76437a972 runtime: Use git rev-parse for the kata-monitor tag
a9626682a virtcontainers: resourcecontrol: Add skeleton for Darwin
ea06fe3af virtcontainers: Add a Network API skeleton for Darwin
6ee550e9a runtime: vCPUs pinning is sandbox specific, not hypervisor
6199b6917 runtime-rs: change cache mode
a33a22ccd runtime-rs: add missing config section for share-fs
e3d3b72fa virtcontainers: use resource control for setting CPU affinity
f137048be resource-control: add helper function for setting CPU affinity
73216a810 vendor: revendor netlink to get latest
fc17d7cc4 virtcontainers: Fix misspelling in error message
12fd6ffc1 runtime: fix up disable_netns handling
64c9114a3 tools: add --locked option for cargo install
7eb43cec1 runtime: add test generated file to .gitignore
8551853cf runtime: use system pagesize for hugepage test
86a82cace runtime: change cache mode from none to never
82c59efd6 runtime-rs: change cache mode from none to never
7b309b578 kata-types: change cache mode from none to never
fee4e7c7c docs: change cache mode from none to never
594b57d08 utils: Add utility functions to get cpu and distro details.
d33e34361 check: Move PROC_CPUINFO from architecture specific files
f8a93a1de tools: Fix indentation for setup aks script
03de5f41b kata-ctl: remove get_kata_version_by_url function
464d4c94d runtime-rs: process single_container
5f9c892e4 kata-types: add single_container support
fa9ae9362 virtcontainers: Add a Virtualization.framework skeleton
d48b22bb1 virtcontainers: fs_share: add Darwin skeleton
fafc7a8b1 virtcontainers: tests: Ensure Linux specific tests are just run on Linux
efa4fc0b2 clh: Add hotplug support for network devices
1074d2c1d clh: Make vmAddNetPutRequest capable of doing hotplugs
9ec8a1398 virtcontainers: introduce hypervisor_darwin
8bb68a9f2 vc/network: skip existing endpoints when scanning for new ones
c21a8d5ff kata-ctl: fix build error on s390x
3b4420eb8 runtime: Define Darwin handled signals list
24b05a99b schedcore: Make buildable on !linux
3886aad19 nydus: net-ns handling needs to be only executed on Linux hosts
e256903af runtime-rs: cleanup the run dir of hypervisor when shut down
937a41346 kata-ctl: add unit tests for volume ops
8451db7c0 kata-ctl: direct-volume: add Add and Remove handlers
2d4b2cf72 runtime-rs: add POST method to shim-client
cae78a685 kata-ctl: add constants for direct-volume commands
652021ad9 versions: Upgrade to Cloud Hypervisor v28.1
d08538912 vc: fix up UT for CreateSandbox API change
578a9c25f vc: rescan network endpoints after running prestart hooks
cb84b0fb0 katautils: run prestart hooks after starting VM
079462d2e runk: Fix needless_borrow warning
2c24fcf34 runtime-rs: Fix clippy::bool-to-int-with-if warnings
025e78341 runtime-rs: Fix needless_borrow warnings
4fb163d57 runtime-rs: Allow clippy:box_default warnings
20121fcda runtime-rs: Fix unnecessary_cast warnings
b95364a14 dragonball: Allow question_mark warning in allocate_device_resources()
0b2f060bf dragonball: Fix unnecessary_cast warnings
a545a6593 agent: Allow clippy::question_mark warning in Namespace{}
9ced34dd2 agent: Fix explicit_auto_deref warnings
f77220490 agent: Fix needless_borrow warnings
7bcdc9049 rustjail: Fix unnecessary_cast warnings
41d7dbaae rustjail: Fix needless_borrow warnings
2a73e057d kata-types: Fix unnecessary_cast warnings
cf9ef1833 kata-types: Fix needless_borrow warnings
126187e81 safe-path: Fix needless_borrow warnings
bb78d35db kata-sys-util: Fix "match-like-matches-macro" warning
668e65240 kata-sys-util: Fix unnecessary_cast warnings
c1a8d89a7 kata-sys-util: Fix needless_borrow warnings
c9c38e6d0 logging: Allow clippy::type-complexity warning
ffd6fbb6b logging: Fix needless_borrow warnings
60df30015 protocols: Fix unnecessary_cast warnings
56e7b5d0f runtime/Makefile: Get some bits happy on darwin
0bbeb34b4 protocols: Fix needless_borrow warnings
dfea6c7d2 versions: Update the rust toolchain to 1.66.0
86ee24b33 Runtime: Clarify mutability of global var
dae667062 kata-runtime: add rust runtime path for kata-runtime exec
a2e3715e0 upcall: remove upcall client when stopping vm
31591d791 dragonball: fix unit test failure case about Kvm.
2b02e0a9b dragonball: add more unit test for vcpu manager
85f9094f1 agent: refactor guest hooks
360506225 runtime-rs: add dbs-upcall feature
03a0c9d78 kata-ctl: skip test if access GitHub.com fail
1dcbda3f0 kata-ctl: update Cargo.lock
b4b5d8150 docs: remove old and misleading instructions for minikube
0fe24e08b packaging: fix indents in build-kernel.sh
3480780bd kata-ctl: add check framework support for non-x86
1bd533f10 kata-ctl: let check framework arch-agnostic
fd77eebd4 runtime-rs: fix the issues mentioned in the code review
0e6920790 runtime-rs: Clean up mount points shared to guest
ecb28e2b1 kernel: adding kmod to do docker env
087515a46 agent: unset `CC` for cross-build
bf8848f92 agent: Eliminate unnecessary metrics
f8a48ab41 docs: add hint of probing loop module
afaf17f42 runtime-rs: enable container hugepage
fc4a67eec runtime-rs: enable vm hugepage
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
kata-deploy files must be adapted to a new release. The cases where it
happens are when the release goes from -> to:
* main -> stable:
* kata-deploy-stable / kata-cleanup-stable: are removed
* stable -> stable:
* kata-deploy / kata-cleanup: bump the release to the new one.
There are no changes when doing an alpha release, as the files on the
"main" branch always point to the "latest" and "stable" tags.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Add a few details about the current state of the Cloud Hypervisor (CH)
runtime-rs external hypervisor implementation with pointers to the
appropriate issues.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add a basic runtime-rs `Hypervisor` trait implementation for Cloud
Hypervisor (CH).
> **Notes:**
>
> - This only supports a default Kata configuration for CH currently.
>
> - Since this feature is still under development, `cargo` features have
> been added to enable the feature optionally. The default is to not enable
> currently since the code is not ready for general use.
>
> To enable the feature for testing and development, enable the
> `cloud-hypervisor` feature in the `virt_container` crate and enable the
> `cloud-hypervisor` feature for its `hypervisor` dependency.
Fixes: #5242.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This is to install a missing binary protoc in shim-v2 Dockerfile.
Fixes: #6244
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
(cherry picked from commit 10603e3def)
Normally we return the context when creating a trace span so that the
ordering of spans w.r.t. calls is maintained in tracing output. Add
missing context for StartVM() for Cloud Hypervisor.
Fixes#6271
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Change unit tests for CPU check to table-driven tests and expand test
cases including temp files for cpuinfo.
Fixes#5919
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Updating this field, as `cpuid` provides host level data, which is not
what a guest would expect for Reduced Phsycial Bits. In almost all
cases, we should be using `1` for the value here.
Amend: Adding unit test change.
Fixes: #5006
Signed-off-by: Larry Dewey <larry.dewey@amd.com>
For kata containers, rootfs is used in the read-only way.
EROFS can noticably decrease metadata overhead.
On the basis of supporting the EROFS file system, it supports using the config parameter to switch the file system used by rootfs.
Fixes: #6063
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Signed-off-by: yaoyinnan <yaoyinnan@foxmail.com>
This PR updates the runc version. This new version include
changes in:
- Fix mounting via wrong proc fd. When the user and mount namespaces are
used, and the bind mount is followed by the cgroup mount in the spec,
the cgroup was mounted using the bind mount's mount fd.
- Switch kill() in libcontainer/nsenter to sane_kill().
- Fix "permission denied" error from runc run on noexec fs.
- Fix failed exec after systemctl daemon-reload. Due to a regression
in v1.1.3, the DeviceAllow=char-pts rwm rule was no longer added and
was causing an error open /dev/pts/0: operation not permitted: unknown when systemd was reloaded.
Fixes#6251
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
We already have verbose output while merging the builds from various
build targets. Getting rid of verbose output to speed up.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Cleanup targets that have been removed in the past when the
makefile for kata-deploy was included.
Instead, add targets from the makefile under local-build kata-deploy.
Fixes: #6165
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
launchClh already has a timeout of 10seconds for launching clh, e.g.
if launchClh or setupVirtiofsDaemon takes a few seconds the context's
deadline will already be expired by the time it reaches bootVM
Fixes#6240
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Change cache mod from literal to const and place them in one place.
Also set default cache mode from `none` to `never` in
`pkg/katautils/config-settings.go.in`.
Fixes: #6151
Signed-off-by: Bin Liu <bin@hyper.sh>
The .dracut_rootfs.done file is accidentally being picked up as the default
target, regardless of BUILD_METHOD. Move the 'all' target definition up, so
that it's the default (=first) target in the makefile. Additionally make the
.dracut_rootfs.done target conditional on the right BUILD_METHOD being
selected, as building it doesn't make sense with BUILD_METHOD=distro.
Fixes: #6235
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Trait method cause for std::error::Error is deprecated thus need replace
it with source method for cgroups-fs::error::ErrorKind.
Fixes: #6192
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Delete cgroup for a thread which may exit can lead to panic. Just
neglect that error is harmless also avoid this failure.
Fixes: #6192
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Normally, the span name should be the same as the function name, and the log subsystem should not contain spaces.
Fixes#6153
Signed-off-by: joannejchen <chenjjoanne@gmail.com>
This is to make a docker version to v20.10 in docker upstream image ubuntu:20.04 for s390x and ppc64le.
Fixes: #6211
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Update Readme to instruct users to increment the kata config version
for any changes made to configs or patches under packaging/kernel.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
The version mentioned in the `kata_config_version` needs to be
updated for any kernel config change or changed to the patches applied.
Without this, CI would not test with the latest kernel changes.
We use to enforce this earlier as part of CI when `packaging` was
a standalone repo.
Add back this check as part of a github action so that the check is
performed early on instead of a CI job.
Fixes: #6210
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
There are lots of unit test cases fails regularly on aarch64, including
TestIOCopy, create_tmpfs. Temporarily skip it for now and enable it
after them get fixed.
Fixes: #6194
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
test_add_remove and test_get_sandbox_id_for_volume need root user, but
test_drop_privs can temporarily change the user to "nobody" that can
lead to the failure of these tests.
Serialise these three tests can fix it.
Fixes: #6055
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Add cpu resize ability upon upcall communication channel. Runtime could
use ResizeVcpu VmmAction and pass the desired vCPU number to the
Dragonball hypervisor.
Dragonball will trigger the device manager service in guest kernel's
upcall server to do cpu resize.
Fixes: #6008
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
The cmd.ExtraFiles feature that is used to implement appendFDs takes an
array of arbitray file descriptors and internally renumbers them to be
consecutive starting from 3, using dup2().
This isn't especially obvious : document it for the sake of clarity.
Fixes#6199
Signed-off-by: Greg Kurz <groug@kaod.org>
link-self-contained is not supported on ppc64le rust target.
Hence, do not pass it while building virtiofsd.
Fixes: #6195
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
The key steps in how-to-hotplug-memory-arm64.md are missing, resulting in the kata qemu pod not being created successfully.
Fixes: #6105
Signed-off-by: SinghWang <wangxin_0611@126.com>
The QEMU log file is essentially about fine grain tracing of QEMU
internals and mostly useful for developpers, not production. Notably,
the log file isn't limited in size, nor rotated in any way. It means
that a container running in the VM could possibly flood the log file
with a guest triggerable trace. For example, on openshift, the log
file is supposed to reside on a per-VM 14 GiB tmpfs mount. This means
that each pod running with the kata runtime could potentially consume
this amount of host RAM which is not acceptable.
Error messages are best collected from QEMU's stderr as kata is doing
now since PR #5736 was merged. Drop support for the QEMU log file
because it doesn't bring any value but can certainly do harm.
Fixes#6173
Signed-off-by: Greg Kurz <groug@kaod.org>
As the dragonball specific kernel is now part of the release, let's make
sure we build it as part of the kata-deploy-push action.
Fixes: #5859
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
os.Stat("unix:///run/vc/sbs/sid/shim-monitor.sock") will fail,
should be os.Stat("/run/vc/sbs/sid/shim-monitor.sock")
Fixes:#6148
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
As the dragonball specific kernel is now part of the release, let's make
sure we build it as part of the kata-deploy-test action.
Fixes: #5859
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the dragonball specific kernel, which takes advantage of
upcall, as part of the release tarball, so it can be used from the
release tarball / kata-deploy.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As Chao Wu added the support for building the dragonball kernel as a new
experimental kernel, let's make sure we reflect that as part of the
kata-deploy build scripts.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The default for the agent today is building with seccomp support.
However, additional steps need to be taken for building against
musl such as installing the static seccomp library for musl.
Add documentation to explain this.
Fixes#6136
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
LaunchQemu now connects a pipe to QEMU's stderr and makes it
usable by callers through a Go io.ReadCloser object. As
explained in [0], all messages should be read from the pipe
before calling cmd.Wait : introduce a LogAndWait helper to handle
that.
Fixes#5780
Signed-off-by: Greg Kurz <groug@kaod.org>
QEMU has always been started daemonized since the beginning. I
could not find any justification for that though, but it certainly
introduces a problem : QEMU stops logging errors when started this
way, which isn't accaptable from a support standpoint. The QEMU
community discourages the use of -daemonize ; mostly because
libvirt, QEMU's primary consummer, doesn't use this option and
prefers getting errors from QEMU's stderr through a pipe in order
to enforce rollover.
Now that virtcontainers knows how to start QEMU with a pre-
established QMP connection, let's start QEMU without -daemonize.
This requires to handle the reaping of QEMU when it terminates.
Since cmd.Wait() is blocking, call it from a goroutine.
Signed-off-by: Greg Kurz <groug@kaod.org>
LaunchCustomQemu() currently starts QEMU with cmd.Run() which is
supposed to block until the child process terminates. This assumes
that QEMU daemonizes itself, otherwise LaunchCustomQemu() would
block forever. The virtcontainers package indeed enables the
Daemonize knob in the configuration but having such an implicit
dependency on a supposedly configurable setting is ugly and fragile.
cmd.Run() is :
func (c *Cmd) Run() error {
if err := c.Start(); err != nil {
return err
}
return c.Wait()
}
Let's open-code this : govmm calls cmd.Start() and returns the
cmd to virtcontainers which calls cmd.Wait().
If QEMU doesn't start, e.g. missing binary, there won't be any
errors to collect from QEMU output. Just drop these lines in govmm.
Similarily there won't be any log file to read from in virtcontainers.
Drop that as well.
Signed-off-by: Greg Kurz <groug@kaod.org>
Running QEMU daemonized ensures that the QMP socket is ready to
accept connections when LaunchQemu() returns. In order to be
able to run QEMU undaemonized, let's handle that part upfront.
Create a listener socket and connect to it. Pass the listener
to QEMU and pass the connected socket to QMP : this ensures
that we cannot fail to establish QMP connection and that we
can detect if QEMU exits before accepting the connection.
This is basically what libvirt does.
Signed-off-by: Greg Kurz <groug@kaod.org>
QEMU's -qmp option can be passed the file descriptor of a socket that
is already in listening mode. This is done with by passing `fd=XXX`
to `-qmp` instead of a path. Note that these two options are mutually
exclusive : QEMU errors out if both are passed, so we check that as
well in the validation function.
While here add the `path=` stanza in the path based case for clarity.
Signed-off-by: Greg Kurz <groug@kaod.org>
When QEMU is launched daemonized, we have the guarantee that the
QMP socket is available. In order to launch a non-daemonized QEMU,
the QMP connection should be created before QEMU is started in order
to avoid a race. Introduce a variant of QMPStart() that can use such
an existing connection.
Signed-off-by: Greg Kurz <groug@kaod.org>
As QEMU released its v7.2.0 version in December last year, last do the
bump on our side.
A few configuration options have been removed between the v6.2.0 (the
version we currently use) and v7.2.0, so those have also been dropped
from our configure-hypervison.sh script (for this specific version).
Also, we're explicitly setting --disable-virtiofsd for the platforms
that we're testing using the rust version.
See: a8d6abe129/docs/about/deprecated.rst (virtiofsd)Fixes: #6102
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Fixes: #6095
We're already importing the virtcontainers package so might as well
use the constants for the hypervisor types we're checking against instead
of typing the names out in the switch cases.
Signed-off-by: Danny Canter <danny@dcantah.dev>
In order for users to get better understand of upcall features, we add
this document for upcall to illustrate what is upcall and how to enable
upcall.
fixes: #6054
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
When the vmm process exits abnormally, a goroutine sets s.monitor
to null in the 'watchSandbox' function without getting service.mu,
This will cause another goroutine to block when sending a message
to s.monitor, and it holds service.mu, which leads to a deadlock.
For example, the wait function in the file
.../pkg/containerd-shim-v2/wait.go will send a message to s.monitor
after obtaining service.mu, but s.monitor may be null at this time
Fixes: #6059
Signed-off-by: ls <335814617@qq.com>
The generic constants for cpu vendor and model may be superseded
by architecture specific constants. Allow these to be marked as
dead code to ignore warnings on architectures where they are overrided.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This function relies on get_single_cpu function which has configured
to compile on amd64 and s390x.
Making the function get_generic_cpu_details to compile on these
architectures until we resolve the compilation for functions defined
in check.rs. This is a temporary solution until we cleanup check.rs to
make it build on all architectures.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Mount handling is often unique in Linux. Let's ensure that the common
parts remain in mount.go, while Linux speific parts are within a linux
file.
Fixes: #6049
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
The .git-commit can be a multiple line file, potentially confusing
the Darwin linker for example.
Fixes: #6046
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Cgroups do not exist on Darwin, so use an empty implementation for
resourcecontrol for the time being. In the process, ensure that the
utilized cgroup handling (ie, isSystemdCgroup) is kept in general file,
since we use this to help assess/constrain the container spec we pass to
the guest.
Fixes: #6051
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
This PR fixes a misspelling in the error message when it tries to run
a system without Confidential computing support.
Fixes#6042
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
With `disable_netns=true`, we should never scan the sandbox netns which
is the host netns in such case.
Fixes: #6021
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
There is a broken release of cgroup-rs, but cargo install will not use
the version in Cargo.lock, so add the `--locked` option to use the version
specified in the Cargo.toml
Fixes: #5376
Signed-off-by: Bin Liu <bin@hyper.sh>
In TestHandleHugepages it will do a mount operation with different pagesizes,
but some systems only support 2M pagesize, test for a 1g pagesize will fail.
This commit try to fix by only mount pagesizes under `/sys/kernel/mm/hugepages`, which are
supported to mount by the OS.
Fixes: #6029
Signed-off-by: Bin Liu <bin@hyper.sh>
Move PROC_CPUINFO into check.rs. This file is used accross
architectures and does not need to be in arch-specific files.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
In `src/tools/kata-ctl/src/check.rs`, there is a function
`get_kata_version_by_url` in the tests mod,
indeed we can use the `get_kata_all_releases_by_url` in the main mod
to replace it.
Fixes: #5981
Signed-off-by: Bin Liu <bin@hyper.sh>
Process single_container like pod_sandbox when create container but like
pod_container when get the size info of memory/cpu from oci/spec.
Fixes: #6006
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
For now, only pod_sandbox and pod_container are supported. It doesn't cover
the case that container started by ctr which is a single_container defined
in kata 2.0. port the single_container kata type from kata 2.0 to kata 3.0.
Fixes: #6006
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Fixes: #6004
A Virtualization.framework based Hypervisor implementation.
This is just stubs for now to eventually get this building.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Signed-off-by: Danny Canter <danny@dcantah.dev>
Fixes: #6002
As a first pass for testing, let's add a skeleton for filesystem
sharing support on Darwin..
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Signed-off-by: Danny Canter <danny@dcantah.dev>
Fixes: #5993
Several tests utilize linux'isms like Mounts, bindmounts, vsock etc.
Let's ensure that these are still tested on Linux, but that we also skip
these tests when on other operating systems (Darwin). This commit just
moves tests; there shouldn't be any functional test changes. While the
tests still won't be runnable on Darwin/other hosts yet, this is a necessary
step forward.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Signed-off-by: Danny Canter <danny@dcantah.dev>
This is needed in order to have Moby / Docker working properly with
Cloud Hypervisor, as Moby / Docker relies on hotplugging a network
device to the VM as a preStartHook.
Fixes: #5997
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
THe only bit needed for having the vmAddNetPutRequest() capable of
dealing with hotplugs, instead of only coldplugs, is making sure it
doesn't error out in case a `200` response is returned.
The 200 response means:
"""
The new device was successfully added to the VM instance.
"""
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Fixes: #5995
Placeholder skeleton at this point - implementation will be added after
basic build refactoring lands.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Signed-off-by: Danny Canter <danny@dcantah.dev>
Fixes: #5990
Some signals may not be defined on non Linux host OSes, like
SIGSTKFLT for example. It's also not defined on certain architectures,
but irrelevant for this.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Signed-off-by: Danny Canter <danny@dcantah.dev>
Fixes: #5983
sched-core only makes sense on Linux hosts. Let's add stub/error for
other platforms.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Signed-off-by: Danny Canter <danny@dcantah.dev>
Fixes: #5985
With nydus not being its own pkg, it is challenging to implement cleanly
in a virtcontainers package that isn't necesarily Linux-only. The
existing code utilizes network namespace code in order to ensure nydus
is launched in the host netns. This is very Linux specific - so let's
make sure we only carry this out in a linux specific file.
In the Darwin case, to allow for compilation at least, let's add a stub
for doNetNS. Ideally the nydus and vc code can be refactored /
decoupled.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Signed-off-by: Danny Canter <danny@dcantah.dev>
Added table driven unit tests and
funcitionality test for functions in volume_ops.
`join_path` relies on safe_path::scoped_join
to validate the unsafe part of the input.
Testcase also takes into account the possibility of specially
constructed string that would get b64-encoded into path-like string.
Fixes#5341
Signed-off-by: Tingzhou Yuan <tzyuan15@bu.edu>
This commit adds direct-volume command handlers for kata-ctl,
including add, remove, stats and resize. Stats and resize
makes HTTP over UDS calls to runtime-rs while add and remove
runs locally on the host.
Fixes#5341
Signed-off-by: Tingzhou Yuan <tzyuan15@bu.edu>
kata-ctl: direct-volume: add Add and Remove handlers
This commit adds direct-volume command handlers for kata-ctl,
including add, remove, stats and resize. Stats and resize
makes HTTP over UDS calls to runtime-rs while add and remove
runs locally on the host.
Fixes#5341
Signed-off-by: Tingzhou Yuan <tzyuan15@bu.edu>
partly refactored shim-client to reuse code, added POST method
support, and made path string constants public for client imports.
Fixes#5341
Signed-off-by: Tingzhou Yuan <tzyuan15@bu.edu>
Moby relies on the prestart hooks to configure network endpoints. We
should rescan the netns after running them so that the newly added
endpoints can be found and plugged to the guest.
Fixes: #5941
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Substitution in the yq install script doesn't like zsh, and additionally
the version of yq we're using doesn't have a darwin/arm64 build so grab
the amd64 version and let rosetta work its magic.
Additionally swap to abspath from readlink -m for the printing of what binaries
to install, as the -m flag doesn't exist on the BSD variant, and this
should be the same behavior.
Fixes: #5970
Signed-off-by: Danny Canter <danny@dcantah.dev>
Was about to change `urandomdev` to a constant when I realized it's
intentionally mutable so it can be mocked in tests. There's other
comments to the same effect so clarify here as well.
Fixes: #5965
Signed-off-by: Danny Canter <danny@dcantah.dev>
In order to avoid resource leak, we need to remove upcall client in vm
and vcpu manager when stopping vm.
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
We have to execute some hooks both in host and guest. And in
/libs/kata-sys-util/src/hooks.rs, the coomon operations are implemented.
In this commit, we are going to refactor the code of guest hooks using
code in /libs/kata-sys-util/src/hooks.rs. At the same time, we move
function valid_env to kata-sys-util to make it usable by both agent and
runtime.
Fixes: #5857
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
This commit will call `error_for_status` after `send`, this call
will generate errors if status code between 400-499 and 500-599.
And sometime access github.com will fail, in this case we can
skip the test to prevent the CI failing.
Fixes: #5948
Signed-off-by: Bin Liu <bin@hyper.sh>
kata-ctl depends on runtime-rs, and this commit:
fbf294da3f
added a new dependency named shim-interface, this Cargo.lock should be updated too.
Signed-off-by: Bin Liu <bin@hyper.sh>
The current check framwork is specific for x86. Refactor the code
to let it arch-agnostic.
Fixes: #5923
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
In order to avoid cloning, changed the signature of
`ShareFsMount::share_rootfs`, `ShareFsMount::share_volume`, and
`ShareFsMount::umount_rootfs` to receive a reference to a config.
Fixes: #5898
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Fixed issues where shared volumes couldn't umount correctly.
The rootfs of each container is cleaned up after the container is killed, except
for `NydusRootfs`. `ShareFsRootfs::cleanup()` calls
`VirtiofsShareMount::umount_rootfs()` to umount mount points shared to the
guest, and umounts the bundle rootfs.
Fixes: #5898
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
- tools: Add some new gitignore items
- shim: return hypervisor's pid not shim's pid
- Dragonball: introduce upcall
- refactor(shim-mgmt): move client side to libs
- kata-ctl: Add --list option
- kata-ctl: check: only-list-releases and include-all-releases options
- basic framework for QEMU support in runtime-rs
- tools: Fix indentation on build kernel script
- runtime-rs: fix standalone share fs
- runtime-rs: fix sandbox_pidns calculation and oci spec amending
- runtime,agent: Add SELinux support for containers inside the guest
- kata-sys-util: fix issues where umount2 couldn't get the correct path
- agent: Drop the Option for LinuxContainer.cgroup_manager
- dragonball: enable kata3.0/dragonball CI on Arm
- fix kata deploy error after node reboot.
- tools: Fix indentation for ovmf script
- runtime: prevent waiting 50 ms minimum for a process exit
- runtime-rs: fix high cpu
- agent: remove `sysinfo` dependency
- runtime-rs: bind mount volumes in sandbox level
- docs: Update the rust version in the installation documentation
- runtime-rs: fix some variable names and typos
- kata-ctl: add host check for aarch64
- kata-ctl: fix dependency version conflict
- workflow: fix cargo-deny-runner.yaml syntax error
- runtime: Add identification in version for runtime-rs
- workflow: call cargo in user's $PATH
- runtime-rs: remove the version number from the commit display message
- runk: Re-implement start operation using the agent codes
- build: update golang version to 1.19.3
- snap: Fix snapcraft setup (unbreak snap releases)
- fix(agent): fix iptables binary path in guest
- runtime-rs: moving only vCPU threads into sandbox controller
- tools: Remove extra tab spaces from kata deploy binaries script
- ci: let static checks don't depend on build
- actions: use matrix to refactor static checks
- agent: support systemd cgroup for kata agent.
- actions: skip some jobs using "paths-ignore" filter
- runtime: go fix code for 1.19
- doc: update runtime-rs "Build and Install"
- runtime: don't fail mkdir if the folder is already created by another process
- kernel: add CONFIG_X86_SGX into whitelist
- runtime-rs: block on the current thread when setup the network to avoid be take over by other task
- Refactor(runtime-rs): add conditional compile for virt-sandbox persist
- runtime: add log record to the qemu config method `appendDevices` for…
- runtime: Use containerd v1.6.8
- tools: Fix indentation of build static firecracker script
- package: add nydus to release artifacts
- agent: check if command exist before do ip_tables test
- runtime: Support virtiofs queue size for qemu and make it configurable
- docs: change mount-info.json to mountInfo.json
- docs: update doc "NVIDIA GPU passthrough"
- runtime-rs: support vhost-vsock
- utils: Add utility function to fetch the kernel version.
- versions: update nydusd version
- runtime-rs: support nydus v5 and v6 rootfs
- Upgrade to Cloud Hypervisor v28.0
- docs: update doc "Setup swap device in guest kernel"
- Rust fixes + Golang bump
- clh: avoid race condition when stopping clh
- tools: Fix indentation of build static virtiofsd script
- docs: Fix configuration path
- runtime-rs : fix the shim source in the documentation test is ambiguous
- versions: update vmm-sys-util and related crates to v0.11.0
- runtime-rs: delete all cargo patches
- feat(shim-mgmt): iptables handler
- tools: Remove empty spaces from build kernel script
- Built-in Sandbox: add more unit tests for dragonball. Part 3
- Dragonball: enable mem_file_path config into hugetlbfs process
- runtime-rs:add hypervisor interface capabilities
- cloud-hypervisor: Fix GetThreadIDs function
- github: Parallelise static checks
- runtime-rs: blanks filled & fixes made to virtiofsd launch
- vCPUs pinning support for Kata Containers
- runtime-rs: fix shared volume permission issue
- runk: Ignore an error when calling kill cmd with --all option
- runk: Upgrade libseccomp crate to v0.3.0 in Cargo.lock
- snap: Unbreak docker install
- add EnterNetNS in virtcontainers
- tools: Fix indentation of build static clh script
- virtiofsd: Not use "link-self-contained=yes" on s390x
- Kata ctl drop privs
- versions: bump golangci-lint version
- runtime-rs: generate config files with the default target
- docs: Fix volumeMounts in SGX usage example
- versions: Update Cloud Hypervisor to b4e39427080
- docs: update rust runtime installation guide
- rustjail: Upgrade libseccomp crate to v0.3.0
- makefile: remove sudo when create symbolic link
- agent: remove redundant checks
- shim: Ensure pagesize is set when reporting hugetlb stats
- kata-ctl: Re-enable network tests on s390x (fixes 5438)
- agent: use NLM_F_REPLACE replace NLM_F_EXCL in rtnetlink
- fix readme content error at doc directory
- agent: validate hugepage size is supported
- Makefile: fix an typo in runtime-rs makefile
- qemu: Re-work static-build Dockerfile
- Modify agent-url return value in runtime-rs
- runtime-rs: regulate the comment in runtime-rs makefile
- doc: Update how-to-run-kata-containers-with-SNP-VMs.md
- kata-ctl: Disable network check on s390x
- virtiofsd: Build inside a container
- Dragonball: remove redundant comments in event manager
- versions: Update TDX QEMU
- runtime-rs: fix typo get_contaier_type to get_container_type
- kata-ctl: improve command descriptions for consistency
- runtime-rs: force shutdown shim process in it can't exit
- versions: Update TDX kernel
- ci: skip s390x for dragonball.
- Dragonball: delete redundant comments in blk_dev_mgr
- kata-ctl: Move development to main branch
- runtime-rs: support ephemeral storage for emptydir
- docs: fix a typo in rust-runtime-installation-guide
- Built-in Sandbox: add more unit tests for dragonball
- readme: remove libraries mentioning
b5cfd0958 kata-ctl: Fixed format for check release options
fbf294da3 refactor(shim-mgmt): move client side to libs
ae0dcacd4 tools: Add some new gitignore items
99485d871 shim: return hypervisor's pid not shim's pid
1f28ff683 runtime-rs: add binary to exercise shim proper w/o containerd dependencies
eb8c9d38f runtime-rs: add launch of a simple qemu process to start_vm()
2f6d0d408 runtime-rs: support qemu in VirtContainer
1413dfe91 runtime-rs: add basic empty boilerplate for qemu driver
a81ced0e3 upcall: add upcall into kernel build script
f5c34ed08 Dragonball: introduce upcall
8dbfc3dc8 kata-ctl: Fixed format for check release options
f3091a9da kata-ctl: Add kata-ctl check release options
a577df8b7 tools: Fix indentation on build kernel script
b087667ac kata-deploy: Fix the pod of kata deploy starts to occur an error
79cf38e6e runtime-rs: clear OCI spec namespace path
62f4603e8 runtime-rs: reset rdma cgroup
5b6596f54 runtime-rs: CreateContainerRequest has Default
e9e82ce28 runtime-rs: fix is_pid_namespace_enabled check
8079a9732 kata-sys-util: fix issues where umount2 couldn't get the correct path
4661ea8d3 runtime-rs: fix standalone share fs
c5abc5ed4 config: speed up rng init when kernel boot for arm64
3e6114b2e tools: Fix indentation for ovmf script
7fdbbcda8 agent: Drop the Option for LinuxContainer.cgroup_manager
d04d45ea0 runtime: use pidfd to wait for processes on Linux
e9ba0c11d runtime: use exponential backoff for process wait
748f22e7d agent: remove sysinfo dependency
0019d653d runtime-rs: fix high cpu
46b38458a docs: Update the rust version in the installation documentation
71491a69c runtime: move process wait logic to another function
92ebe61fe runtime: reap force killed processes
fdf0a7bb1 runtime-rs: fix the issues mentioned in the code review
1d823c4f6 runtime-rs: umount and permission controls in sandbox level
527b87141 runtime-rs: bind mount volumes in sandbox level
9ccf2ebe8 agent: add signal value to log
fb2c142f1 runtime-rs: fix some variable names and typos
737420469 kata-ctl: fix dependency version conflict
89574f03f workflow: call cargo in user's $PATH
d4321ab48 runtime: Add identification in version for runtime-rs
f7fc436be workflow: fix cargo-deny-runner.yaml syntax error
78532154d docs: Add description for guest SELinux support
c617bbe70 runtime: Pass SELinux policy for containers to the agent
935476928 agent: Add SELinux support for containers
a75f99d20 osbuilder: Create guest image for SELinux
a9c746f28 kernel: Add kernel configs for SELinux
86cb05883 snap: Fix snapcraft setup (unbreak snap releases)
f443b7853 build: update golang version to 1.19.3
e12db92e4 runk: Re-implement start operation using the agent codes
e723bad0a ci: let static checks don't depend on build
69aae0227 actions: use matrix to refactor static checks
a5e4cad4b kata-ctl: add host check for aarch64
2edbe389d runtime-rs: moving only vCPU threads into sandbox controller
340e24f17 actions: skip some job using "paths-ignore" filter
2426ea9bd doc: update runtime-rs "Build and Install"
67fe703ff runtime-rs: remove the version number from the commit display message
1d93a9346 fix(agent): fix iptables binary path in guest
1dfd845f5 runtime: go fix code for 1.19
cd85a44a0 tools: Remove extra tab spaces from kata deploy binaries script
cb199e0ec kernel: add CONFIG_X86_SGX into whitelist
4b45e1386 runtime: don't fail mkdir if the folder is already created
b987bbc57 runtime-rs: block on the current thread when setup the network
abb9ebeec package: add nydus to release artifacts
30a7ebf43 runtime: Log invalid devices in QEMU config
2539f3186 runtime: Use containerd v1.6.8
993d05a42 docs: change mount-info.json to mountInfo.json
d808adef9 runtime-rs: support vhost-vsock
6b2ef66f0 runtime-rs: add conditional compile for virt-sandbox persist
6c1e153a6 docs: update doc "NVIDIA GPU passthrough"
b53171b60 agent: check command before do test_ip_tables
a636d426d versions: update nydusd version
3bb145c63 runtime: Support virtiofs queue size for qemu and make it configurable
e80a9f09f utils: Add utility function to fetch the kernel version.
36545aa81 runtime: clh: Re-generate the client code
f4b02c224 versions: Upgrade to Cloud Hypervisor v28.0
e4a6fbadf docs: update doc "Setup swap device in guest kernel"
2f5f575a4 log-parser: Simplify check
d94718fb3 runtime: Fix gofmt issues
16b837509 golang: Stop using io/ioutils
66aa330d0 versions: Update golangci-lint
b3a4a1629 versions: bump containerd version
eab8d6be1 build: update golang version to 1.19.2
e80dbc15d runtime-rs: workaround Dragonball compilation problem
c3f1922df fix(fmt): fix cargo fmt to pass static check
a4099dab8 tools: Fix indentation of build static firecracker script
c46814b26 runtime-rs:support nydus v5 and v6
a04afab74 qemu: early exit from Check if the process was stopped
7e481f217 qemu: set stopped only if StopVM is successful
0e3ac66e7 clh: return faster with dead clh process from isClhRunning
9ef68e0c7 clh: fast exit from isClhRunning if the process was stopped
2631b08ff clh: don't try to stop clh multiple times
f45fe4f90 versions: update vmm-sys-util and related crates to v0.11.0
8be081730 tools: Fix indentation of build static virtiofsd script
f8f97c1e2 feat(shim-mgmt): iptables handler
29c75cf12 runtime-rs: delete all cargo patches
9f70a6949 tools: Remove empty spaces from build kernel script
57336835d dragonball: add more unit test for device manager
233370023 dragonball: add test utils.
3e9c3f12c docs: Fix configuration path
2adb1c182 Dragonball: enable mem_file_path config into hugetlbfs process
daeee26a1 cloud-hypervisor: Fix GetThreadIDs function
40d514aa2 github: Parallelise static checks
2508d39b7 runtime: added vcpus pinning logics Core VCPU threads pinning logics for issue 4476. Also provided docs.
fef8e92af runtime-rs:add hypervisor interface capabilities
27b191358 runtime-rs: blanks filled & fixes made to virtiofsd launch
990e6359b snap: Unbreak docker install
ca69a9ad6 snap: Use metadata for dependencies
df092185e runk: Upgrade libseccomp crate to v0.3.0 in Cargo.lock
16dca4ecd runk: Ignore an error when calling kill cmd with --all option
b74c18024 runtime-rs: fix shared volume permission issue
936fe35ac runtime-rs : fix shim source is ambiguous
0ed7da30d tools: Fix indentation of build static clh script
43fcb8fd0 virtiofsd: Not use "link-self-contained=yes" on s390x The compile option link-self-contained=yes asks rustc to use C library startup object files that come with the compiler, which are not available on the target s390x-unknown-linux-gnu. A build does not contain any startup files leading to a broken executable entry point (causing segmentation fault).
219919e9f docs: Fix volumeMounts in SGX usage example
c0f5bc81b cargo: Add Cargo.lock to version control
474927ec9 gitignore: Add gitignore file
699f821e1 utils: Add function to drop priveleges
a6fb4e2a6 versions: bump golangci-lint version
b015f34af runtime-rs: generate config files with the default target
d7bb4b551 agent: support systemd cgroup for kata agent
144efd1a7 docs: update rust runtime installation guide
abf4f9b29 docs: kata 3.0 Architecture fix readme content error
44d8de892 agent: remove redundant checks
9d286af7b versions: Update Cloud Hypervisor to b4e39427080
081ee4871 agent: use NLM_F_REPLACE replace NLM_F_EXCL in rtnetlink
e95089b71 kata-ctl: add basic cpu check for s390x
871d2cf2c kata-ctl: Limit running tests to x86 and use native-tls on s390x
cbd84c3f5 rustjail: Upgrade libseccomp crate to v0.3.0
748be0fe3 makefile: remove sudo when create symbolic link
227e717d2 qemu: Re-work static-build Dockerfile
72738dc11 agent: validate hugepage size is supported
f74e328ff Makefile: fix an typo in runtime-rs makefile
f205472b0 Makefile: regulate the comment style for the runtime-rs comments
9f2c7e47c Revert "kata-ctl: Disable network check on s390x"
ac403cfa5 doc: Update how-to-run-kata-containers-with-SNP-VMs.md
00981b3c0 kata-ctl: Disable network check on s390x
39363ffbf runtime: remove same function
c322d1d12 kata-ctl: arch: Improve check call
0bc5baafb snap: Build virtiofsd using the kata-deploy scripts
cb4ef4734 snap: Create a task for installing docker
7e5941c57 virtiofsd: Build inside a container
35d52d30f versions: Update TDX QEMU
4d9dd8790 runtime-rs: fix typo get_contaier_type to get_container_type
70676d4a9 kata-ctl: improve command descriptions for consistency
9eb73d543 versions: Update TDX kernel
00a42f69c kata-ctl: cargo: 2021 -> 2018
fb6327474 kata-ctl: rustfmt + clippy fixes
1f1901e05 dragonball: fix clippy warning for aarch64
a343c570e dragonball: enhance dragonball ci
6a64fb0eb ci: skip s390x for dragonball.
a743e37da Dragonball: delete redundant comments in blk_dev_mgr
2b345ba29 build: Add kata-ctl to tools list
f7010b806 kata-ctl: docs: Write basic documentation
862eaef86 docs: fix a typo in rust-runtime-installation-guide
26c043dee ci: Add dragonball test
781e604c3 docs: Reference kata-ctl README
15c343cbf kata-ctl: Don't rely on system ssl libs
c23584994 kata-ctl: clippy: Resolve warnings and reformat
133690434 kata-ctl: implement CLI argument --check-version-only
eb5423cb7 kata-ctl: switch to use clap derive for CLI handling
018aa899c kata-ctl: Add cpu check
7c9f9a5a1 kata-ctl: Make arch test run at compile time
b63ba66dc kata-ctl: Formatting tweaks
cca7e32b5 kata-ctl: Lint fixes to allow the branch to be built
8e7bb8521 kata-ctl: add code for framework for arch
303fc8b11 kata-ctl: Add unit tests cases
d0b33e9a3 versions: Add kata-ctl version entry
002b18054 kata-ctl: Add initial rust code for kata-ctl
b62b18bf1 dragonball: fix clippy warning
2ddc948d3 Makefile: add dragonball components.
3fe81fe4a dragonball-ut: use skip_if_not_root to skip root case
72259f101 dragonball: add more unit test for vmm actions
9717dc3f7 Dragonball: remove redundant comments in event manager
9c1ac3d45 runtime-rs: return port on agent-url req
89e62d4ed shim: Ensure pagesize is set when reporting hugetbl stats
8d4ced3c8 runtime-rs: support ephemeral storage for emptydir
046ddc646 readme: remove libraries mentioning
86ad832e3 runtime-rs: force shutdown shim process in it can't exit
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
If the serial path is given, legacy_manager should create socket console
based on that path. Or the console should be created based on stdio.
Fixes: #5914
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
enable start container from bundle in this way
$ ls ./bundle
config.json rootfs
$ sudo ctr run -d --runtime io.containerd.kata.v2 --config bundle/config.json test_kata
Fixes:#5872
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
The client side is moved to libs. This is to solve the problem
that including clients will bring about messy dependencies.
Fixes: #5874
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
Add some new ignore items to avoid local builds that cause git to track a lot of files
Fixes: #5900
Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
When `HOST_ARCH` != `ARCH` unset `CC`
Specifying a foreign CC is incompatible with building libgit2. Thus after the RUSTFLAGS linker
has been set we can safely unset CC to avoid passing this value through the build.
Fixes: #5890
Signed-off-by: James Tumber <james.tumber@ibm.com>
After building the binary as usual with `cargo build` run it as follows.
It needs a configuration.toml in which only qemu keys `path`, `kernel`
and `initrd` will initially need to be set. Point them to respective
files e.g. from a kata distribution tarball.
It also needs to be launched from an exported container bundle
directory. One can be created by running
mkdir rootfs
podman export $(podman create busybox) | tar -C ./rootfs -xvf -
runc spec -b .
in a suitable directory.
Then launch the program like this:
KATA_CONF_FILE=/path/to/configuration-qemu.toml /path/to/shim-ctl
Fixes: #5817
Signed-off-by: Pavel Mores <pmores@redhat.com>
This does almost literally nothing so far apart from getting and setting
HypervisorConfig. It's mostly copied from/inspired by dragonball.
Signed-off-by: Pavel Mores <pmores@redhat.com>
DEFAULT_REGISTRY pre-registers many metrics that we don't need or have duplicated.
This PR uses a custom register for metrics without interference and ensures that
the registration process is executed only once when the program is running.
Fixes: #5255
Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
In order to let upcall being used by Kata Container, we need to add
those patches into kernel build script.
Currently, only when experimental (-e) and hypervisor type dragonball
(-t dragonball) are both enabled, that the upcall patches will be
applied to build a 5.10 guest kernel.
example commands: sh ./build-kernel.sh -e -t dragonball -d setup
fixes: #5642
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Upcall is a direct communication tool between VMM and guest developed
upon vsock. The server side of the upcall is a driver in guest kernel
(kernel patches are needed for this feature) and it'll start to serve
the requests after the kernel starts. And the client side is in
Dragonball VMM , it'll be a thread that communicates with vsock through
uds.
We want to keep the lightweight of the VM through the implementation of
the upcall, through which we could achieve vCPU hotplug, virtio-mmio
hotplug without implementing complex and heavy virtualization features
such as ACPI virtualization.
fixes: #5642
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
If `loop` module is not probed, it causes error like "losetup: cannot find an unused loop device".
Fixes: #5887
Signed-off-by: Guoqiang Ding <dgq8211@gmail.com>
If a pod of kata is deployed on a machine, after the machine restarts, the pod status of kata-deploy will be CrashLoopBackOff.
Fixes: #5868
Signed-off-by: SinghWang <wangxin_0611@126.com>
None of the host namespace paths make sense in the guest. Let's clear
them all before sending the spec to the agent.
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
We should test is_pid_namespace_enabled before amending the container
spec, where the pid namespace path is cleared and resulting
sandbox_pidns to always being false.
Fixes: #5881
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Strings in Rust don't have \0 at the end, but C does, which leads to `umount2`
in the libc can't get the correct path. Besides, calling `nix::mount::umount2`
to avoid using an unsafe block is a robust solution.
Fixes: #5871
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Standalone share fs should add virtiofs device in setup_device_before_start_vm
and return the storages to mount the directory in guest. And it uses
hypervisor's jailer root directly instead of jail config.
Besides, we tweaked the parameter, so it adapts to rust version virtiofsd
now. And its cache policy which forbids caching is "never" now, instead of
"none". Hence, we change the default cache mode.
Fixes: #5655
Signed-off-by: Yipeng Yin <yinyipeng@bytedance.com>
For now, rng init is too slow for kata3.0/dragonball. Enable
random_trust_cpu can speed up rng init when kernel boot.
Fixes: #5870
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Script to execute to build virtiofsd has been changed in #5426 but not in the doc. This commit update the developer guide.
Fixes: #5860
Signed-off-by: Mathias Flagey <mathiasflagey1201@gmail.com>
Cgroup manager for a container will always be created.
Thus, dropping the option for LinuxContainer.cgroup_manager
is feasible and could simplify the code.
Fixes: #5778
Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
Use pidfd_open and poll on newer versions of Linux to wait
for the process to exit. For older versions use existing wait logic
Fixes: #5617
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Fixed the issue when using nonblocking, the `tokio::io::copy()` needing
to handle EAGAIN, resulting in high CPU usage.
Fixes: #5740
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
Removed the `Debug` trait for the `ShareFs` and etc. Renamed
`ShareFsMount::upgrade()` and `ShareFsMount::downgrade()` to
`upgrade_to_rw()` and `downgrade_to_ro()`. Protected `mounted_info_set`
with a mutex to avoid race conditions.
Fixes: #5588
Signed-off-by: Xuewei Niu <justxuewei@apache.org>
This commit implemented umonut controls and permission controls. When a volume
is no longer referenced, it will be umounted immediately. When a volume mounted
with readonly permission and a new coming container needs readwrite permission,
the volume should be upgraded to readwrite permission. On the contrary, if a
volume with readwrite permission and no container needs readwrite, then the
volume should be downgraded.
Fixes: #5588
Signed-off-by: Xuewei Niu <justxuewei@apache.org>
Implemented bind mount related managment on the sandbox side, involving bind
mount a volume if it's not mounted before, upgrade permission to readwrite if
there is a new container needs.
Fixes: #5588
Signed-off-by: Xuewei Niu <justxuewei@apache.org>
Also added crate `runtime-rs/crates/runtimes` as dependency as it's
immediately depended upon by the `direct-volume` feature, see issue
5341 and PR 5467.
Fixes#5810
Signed-off-by: Tingzhou Yuan <tzyuan15@bu.edu>
Call cargo in root's HOME may lead to permission error, should
call cargo installed in user's HOME/PATH.
Fixes: #5813
Signed-off-by: Bin Liu <bin@hyper.sh>
Now we are supporting two runtime/shim, the go version,
and the rust version, for debug purposes, we can
add an identification in the version info
to tell us which runtime/shim is used.
Fixes: #5806
Signed-off-by: Bin Liu <bin@hyper.sh>
Add the description about how to enable SELinux for containers
running inside the guest.
Fixes: #4812
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Pass SELinux policy for containers to the agent if `disable_guest_selinux`
is set to `false` in the runtime configuration. The `container_t` type
is applied to the container process inside the guest by default.
Users can also set a custom SELinux policy to the container process using
`guest_selinux_label` in the runtime configuration. This will be an
alternative configuration of Kubernetes' security context for SELinux
because users cannot specify the policy in Kata through Kubernetes's security
context. To apply SELinux policy to the container, the guest rootfs must
be CentOS that is created and built with `SELINUX=yes`.
Fixes: #4812
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
The kata-agent supports SELinux for containers inside the guest
to comply with the OCI runtime specification.
Fixes: #4812
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Create a guest image to support SELinux for containers inside the guest
if `SELINUX=yes` is specified. This works only if the guest rootfs is
CentOS and the init service is systemd, not the agent init. To enable
labeling the guest image on the host, selinuxfs must be mounted on the
host. The kata-agent will be labeled as `container_runtime_exec_t` type.
Fixes: #4812
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Add kernel configs related to SELinux in order to add the
support for containers running inside the guest.
Fixes: #4812
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Setup the snapcraft environment manually as the action we had been using
for this does not appear to be actively maintained currently.
Related to this, switch to specifying the snapcraft store credentials
using the `SNAPCRAFT_STORE_CREDENTIALS` secret. This unbreaks
`snapcraft upload`, which Canonical appear to have broken by removing
the previous facility.
Fixes: #5772.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This commit re-implements `start` operation by leveraging the agent codes.
Currently, `runk` has own `start` mechanism even if the agent already
has the feature to handle starting a container. This worsen the maintainability
and `runk` cannot keep up with the changes on the agent side easily.
Hence, `runk` replaces own implementations with agent's ones.
Fixes: #5648
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
For now, we can check if host support running kata by check if "/dev/kvm"
exist on aarch64.
Fixes: #5768
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
When using source code to compile runtime-rs,make the
documentation point out the detailed environment build
and compilation methods to avoid errors caused by related
dependent packages.
Fixes:#5757
Signed-off-by: Chen Taotao <chentt10@chinatelecom.cn>
The displayed commit message and version message are partially duplicated.
Remove the version number from the commit display message.
Fixes:#5735
Signed-off-by: Chen Taotao <chentt10@chinatelecom.cn>
Some rootfs put iptables-save and iptables-restore
under /usr/sbin instead of /sbin. This pr checks both
and returns the one exist.
Fixes: #5608
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
We have starting to use golang 1.19, some features are
not supported later, so run `go fix` to fix them.
Fixes: #5750
Signed-off-by: Bin Liu <bin@hyper.sh>
CONFIG_X86_SGX is introduced after kernel 5.11, and that config is a
default x86_64 config for Kata build-kernel.sh script.
But if we use -v to specify any kernel version below 5.11 will cause an
inevitable error because CONFIG_X86_SGX is not supported in older
kernels and that may cause problem for the situation if we need kernel
version below 5.11.
So I propose to put CONFIG_X86_SGX into whitelist.conf to avoid break
building guest kernel below 5.11.
fixes: #5741
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Use MkdirAll instead of Mkdir so it doesn't generate an
error when the folder is created by another process
Fixes#5713
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
As the increase of the I/O intensive tasks, two issues could be caused:
1. When the future is blocked, the current thread (which is in the network namespace)
might be take over by other tasks. After the future is finished, the thread take over
the current task might not be in the pod network namespace
2. When finish setting up the network, the current thread will be set back to the host namsapce.
But the task which be taken over would still stay in the pod network namespace
To avoid that, we need to block the future on the current thread.
Fixes:#5728
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
When the user tried to add new devices to the VM, there is no error info for the invalid
device. This PR adds a log record to the `appendDevices` for the invalid device of the
qemu config.
Fixes: #5719
Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
Let's follow the binary bump used in the CI and also bump the vendored
version of containerd to v1.6.8.
Fixes: #5722
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Rename old VsockConfig to HybridVsockConfig. And add VsockConfig to
support vhost-vsock. We follow kata's old way to try random vhost fd
for 50 times to generate uniqe fd.
Fixes: #5654
Signed-off-by: Yipeng Yin <yinyipeng@bytedance.com>
test_ip_tables test depends on iptables tools. But we can't
ensure these tools are exist. it's better to skip the test
if there is no such tools.
Fixes: #5697
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
The default vhost-user-fs queue-size of qemu is 128 now. Set it to 1024
by default which is same as clh. Also make this value configurable.
Fixes: #5694
Signed-off-by: liyuxuan.darfux <liyuxuan.darfux@bytedance.com>
Add functionality to get kernel version and related unit tests.
This is intended to be used in the kata-env command going forward.
Fixes: #5688
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This patch re-generates the client code for Cloud Hypervisor v28.0.
Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by openapi-generator.
Fixes: #5683
Signed-off-by: Bo Chen <chen.bo@intel.com>
```
14:13:15 parse.go:306:5: S1009: should omit nil check; len() for github.com/kata-containers/kata-containers/src/tools/log-parser.kvPairs is defined as zero (gosimple)
14:13:15 if pairs == nil || len(pairs) == 0 {
14:13:15 ^
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It seems that bumping the version of golang and golangci-lint new format
changes are required.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The package has been deprecated as part of 1.16 and the same
functionality is now provided by either the io or the os package.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's bump the golangci-lint in order to fix issues that popped up after
updating Golang to its 1.19.2 version.
Depends-on: github.com/kata-containers/tests#5257
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
v1.5.2 cannot be built from source by newer golang. Let's bump
containerd version to 1.6.8. The GO runtime dependency has
been moved to v1.6.6 for some time already.
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
So that we get the latest language fixes.
There is little use to maitain compiler backward compatibility.
Let's just set the default golang version to the latest 1.19.2.
Fixes: #5494
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Since the upstream rust-vmm is changing its dependency style towards
caret requirements in these days (more information:
rust-vmm/vm-memory#199) and it breaks Dragonball compilation frequently.
rust-vmm is expected to finish the changes this week and in order to not
break Kata CI due to Dragonball's compilation error, we will add
Cargo.lock file into /src/dragonball first and remove it later when
rust-vmm is stable.
fixes: #5657
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Through proactively checking if Cloud Hypervisor process is dead,
this patch provides a faster path for isClhRunning
Fixes: #5623
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Use atomic operations instead of acquiring a mutex in isClhRunning.
This stops isClhRunning from generating a deadlock by trying to
reacquire an already-acquired lock when called via StopVM->terminate.
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Avoid executing StopVM concurrently when virtiofs dies as a result of clh
being stopped in StopVM.
Fixes: #5622
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Since the upstream of vmm-sys-utils upgraded to 0.11.0, some crates
automatically upgrade to v0.11.0, and some stay at v0.10.0 ( depending
on how they write version dependency in Cargo toml` which causes the
compile error in runtime-rs.
In order to fix this problem, we need to upgrade all vmm-sys-util
dependencies in runtime-rs to v0.11.0.
fixes: #5636
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Support the handlers in runtime, which are used by kata-ctl iptables series of commands in runtime.
Fixes: #5370
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
The cargo patch in the cargo.toml seems to cause the whole runtime-rs
building time longer and also makes it harder to build runtime-rs in an
environment without the network
We should delete all patches from the cargo.toml file and publish all
the crates that was once patched.
fixes: #5614#5527#5526#5449
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
On install you generate a configuration-fc.toml
file when building the kata-runtime and
copy it to either /etc/kata-containers/configuration-fc.toml
or /usr/share/defaults/kata-containers/configuration-fc.toml.
To reflect that the path must be one of the above,
we can fix the path in doc.
Fixes: #5589
Signed-off-by: Mathis Joffre <mariusjoffre@gmail.com>
In the current Dragonball code, mem_file_path config is not used when
hugetlbfs is enabled.
In this commit we add mem_file_path into hugetlbfs enable process.
fixes: #5566
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Although introducing an awful amount of code duplication, let's
parallelise the static checks in order to reduce its time and the space
used in the VMs running those.
While I understand there may be ways to make the whole setup less
repetitive and error prone, I'm taking the approach of:
* Make it work
* Make it right
* Make it fast
So, it's clear that I'm only attempting to make it work, and I'd
appreciate community help in order to improve the situation here. But,
for now, this is a stopgap solution.
JFYI, the time needed for run the tests on the `main` branch went down
from ~110 minutes to ~60 minutes. Plus, we're not running those on a
single VM anymore, which decreases the change to hit the space limit.
Reference: https://github.com/kata-containers/kata-containers/actions/runs/3393468605/jobs/5640842041
Ideally, each one of the following tests should be also split into
smaller tests, each test for one component, for instance.
* static-checks
* compiler-checks
* unit-tests
* unit-tests-as-root
Fixes: #5585
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
1. be able to check does hypervisor support use block device, block
device hotplug, multi-queue, and share file
2. be able to set the hypervisor capability of using block device, block
device hotplug, multi-queue, and share file
Fixes: #5569
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
The 'config' argument to ShareVirtioFsStandalone::new() is now actually
used, taking care of an explicit TODO.
If a shared path doesn't exist in ShareVirtioFsStandalone::virtiofsd_args()
it is now created instead of returning an error, thus following
ShareVirtioFsInline's suit.
The '-o vhost_user_socket=...' command line argument doesn't seem to be
supported by newer versions of virtiofsd so we replace it with
'--socket-path' which should be functionally equivalent according to docs.
Fixes#5572
Signed-off-by: Pavel Mores <pmores@redhat.com>
It appears that _either_ the GitHub workflow runners have changed their
environment, or the Ubuntu archive has changed package dependencies,
resulting in the following error when building the snap:
```
Installing build dependencies: bc bison build-essential cpio curl docker.io ...
:
The following packages have unmet dependencies:
docker.io : Depends: containerd (>= 1.2.6-0ubuntu1~)
E: Unable to correct problems, you have held broken packages.
```
This PR uses the simplest solution: install the `containerd` and `runc`
packages. However, we might want to investigate alternative solutions in
the future given that the docker and containerd packages seem to have
gone wild in the Ubuntu GitHub workflow runner environment. If you
include the official docker repo (which the snap uses), a _subset_ of
the related packages is now:
- `containerd`
- `containerd.io`
- `docker-ce`
- `docker.io`
- `moby-containerd`
- `moby-engine`
- `moby-runc`
- `runc`
Fixes: #5545.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Rather than hard-coding the package manager into the docker part,
use the `build-packages` section to specify the parts package
dependencies in a distro agnostic manner.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The libseccomp crate was upgraded to v0.3.0 by 4696ead,
but `Cargo.lock` of runk wasn't updated by mistake.
So, this commit updates `Cargo.lock` of runk to the latest dependencies.
Fixes: #5487
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Ignore an error handling that is triggered when the kill command is called
with `--all option` to the stopped container.
High-level container runtimes such as containerd call the kill command with
`--all` option in order to terminate all processes inside the container
even if the container already is stopped. Hence, a low-level runtime
should allow `kill --all` regardless of the container state like runc.
This commit reverts to the previous behavior.
Fixes: #5555
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Fix the issue where share volumes always have readwrite permission even if
readonly permission is enough.
Fixes: #5549
Signed-off-by: Xuewei Niu <justxuewei@apache.org>
In the documentation test, the name shim has multiple potential
sources of import, now give it a clear source.
Fixes: #5535
Signed-off-by: Chen TaoTao <chentt10@chinatelecom.cn>
The compile option link-self-contained=yes asks rustc to use
C library startup object files that come with the compiler,
which are not available on the target s390x-unknown-linux-gnu.
A build does not contain any startup files leading to a
broken executable entry point (causing segmentation fault).
Fixes: #5522
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The /dev/sgx is not mounted and the enclave is not available,
causing the demo job to report an error in the logs. Add volumeMounts to
container in order to have the device available in the container.
Fixes: #5514
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
This function is meant to be used before operations
such as accessing network to make sure those operations
are not performed as a privilged user.
Fixes: #5331
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
There is little point to maintain backward compatiblity for
golangci-lint. Let's just use a unified version of it.
Fixes: #5512
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
1. Implemented a rust module for operating cgroups through systemd with the help of zbus (src/agent/rustjail/src/cgroups/systemd).
2. Add support for optional cgroup configuration through fs and systemd at agent (src/agent/rustjail/src/container.rs).
3. Described the usage and supported properties of the agent systemd cgroup (docs/design/agent-systemd-cgroup.md).
Fixes: #4336
Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
An API change, done a long time ago, has been exposed on Cloud
Hypervisor and we should update it on the Kata Containers side to ensure
it doesn't affect Cloud Hypervisor CI and because the change is needed
for an upcoming work to get QAT working with Cloud Hypervisor.
Fixes: #5492
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add a basic s390x cpu check for the "sie" feature to be present.
Also re-enable cpu check testing.
Fixes: #5438
Signed-off-by: Hendrik Brueckner <brueckner@linux.ibm.com>
For s390x, use native-tls for reqwest because the rustls-tls/ring
dependency is not available for s390x.
Also exclude s390x, powerpc64le, and aarch64 from running the cpu
check due to the lack of the arch-specific implementation. In this
case, rust complains about unused functions in src/check.rs (both
normal and test context).
Fixes: #5438
Co-authored-by: James O. D. Hunt <james.o.hunt@intel.com>
Signed-off-by: Hendrik Brueckner <brueckner@linux.ibm.com>
Differently than every single other bit that's part of our repo, QEMU
has been using a single Dockerfile that prepares an environment where
the project can be built, but *also* building the project as part of
that very same Dockerfile.
This is a problem, for several different reasons, including:
* It's very hard to have a reproducible build if you don't have an
archived image of the builder
* One cannot cache / ipload the image of the builder, as that contains
already a specific version of QEMU
* Every single CI run we end up building the builder image, which
includes building dependencies (such as liburing)
Let's split the logic into a new build script, and pass the build script
to be executed inside the builder image, which will be only responsible
for providing an environment where QEMU can be built.
Fixes: #5464
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
before setting a limit, otherwise paths may not be found.
guest supporting different hugepage size is more likely with peer-pods where
podvm may use different flavor.
Fixes: #5191
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
In runtime-rs makefile, we use
```
```
to let make help print out help information for variables and targets,
but later commits forgot this rule.
So we need to follow the previous rule and change the current comments.
fixes: #5413
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
If the needed libraries (for virtfs) are installed on the host,
QEMU will pick it up and enable it. If not installed and you
do not enable the flag, QEMU will just ignore it, and you end
up without 9p support. Enabling it explicitly will fail if the
needed libs are not installed so this way we can be sure that
it gets build.
Fixes: #5418
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
s390x apparently does not support rust-tls, which is required by the
network check (due to the `reqwest` crate dependency).
Disable the network check on s390x until we can find a solution to the
problem.
> **Note:**
>
> This fix is assumed to be a temporary one until we find a solution.
> Hence, I have not moved the network check code (which should be entirely
> generic) into an architecture specific module.
Fixes: #5435.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Rework the architecture-specific `check()` call by moving all the
conditional logic out of the function.
Fixes: #5402.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Let's build virtiofsd using the kata-deploy build scripts, which
simplifies and unifies the way we build our components.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's have the docker installation / configuration as part of its own
task, which can be set as a dependency of other tasks whcih may or may
not depend on docker.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When moving to building the CI artefacts using the kata-deploy scripts,
we've noticed that the build would fail on any machine where the tarball
wasn't officially provided.
This happens as rust is missing from the 1st layer container. However,
it's a very common practice to leave the 1st layer container with the
minimum possible dependencies and install whatever is needed for
building a specific component in a 2nd layer container, which virtiofsd
never had.
In this commit we introduce the second layer containers (yes,
comtainers), one for building virtiofsd using musl, and one for building
virtiofsd using glibc. The reason for taking this approach was to
actually simplify the scripts and avoid building the dependencies
(libseccomp, libcap-ng) using musl libc.
Fixes: #5425
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The previously used repo will be removed by Intel, as done with the one
used for TDX kernel. The TDX team has already worked on providing the
patches that were hosted atop of the QEMU commit with the following hash
4c127fdbe81d66e7cafed90908d0fd1f6f2a6cd0 as a tarball in the
https://github.com/intel/tdx-tools repo, see
https://github.com/intel/tdx-tools/pull/162.
On the Kata Containers side, in order to simplify the process and to
avoid adding hundreds of patches to our repo, we've revived the
https://github.com/kata-containers/qemu repo, and created a branch and a
tag with those hundreds of patches atop of the QEMU commit hash
4c127fdbe81d66e7cafed90908d0fd1f6f2a6cd0. The branch is called
4c127fdbe81d66e7cafed90908d0fd1f6f2a6cd0-plus-TDX-v3.1 and the tag is
called TDX-v3.1.
Knowing the whole background, let's switch the repo we're getting the
TDX QEMU from.
Fixes: #5419
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This change improves the command descriptions for kata-ctl and can avoid certain confusions in command functionality.
Fixes#5411
Signed-off-by: Tingzhou Yuan <tzyuan15@bu.edu>
The previously used repo has been removed by Intel. As this happened,
the TDX team worked on providing the patches that were hosted atop of
the v5.15 kernel as a tarball present in the
https://github.com/intel/tdx-tools repos, see
https://github.com/intel/tdx-tools/pull/161.
On the Kata Containers side, in order to simplify the process and to
avoid adding ~1400 kernel patches to our repo, we've revived the
https://github.com/kata-containers/linux repo, and created a branch and
a tag with those ~1400 patches atop of the v5.15. The branch is called
v5.15-plus-TDX, and the tag is called 5.15-plus-TDX (in order to avoid
having to change how the kernel builder script deals with versioning).
Knowing the whole background, let's switch the repo we're getting the
TDX kernel from.
Fixes: #5326
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Make this file conform to the standard rust layout conventions and
simplify the code as recommended by `clippy`.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Build using the rust TLS implementation rather than the system ones.
This resolves the `reqwest` crate build failure: it doesn't appear to
build against the native libssl libraries due to Kata defaulting to
using the musl libc.
Fixes: #5387.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This kata-ctl argument returns the latest stable Kata
release by hitting github.com.
Adds check-version unit tests.
Fixes: #11
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Add architecture-specific code for x86_64 and generic calls handling
checks for CPU flags and attributes.
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Changed the `panic!()` call to a `compile_error!()` one to ensure it
fires at compile time rather than runtime.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add framework for different architectures for check. In the existing
kata-runtime check, the network checks do not appear to be
architecture-specific while the kernel module, cpu, and kvm checks do
have separate implementations for different architectures.
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
As we're switching to using the rust version of the kata-ctl, lets
provide with its own entry in the kata-ctl command line.
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Commit-edited-by: James O. D. Hunt <james.o.hunt@intel.com>
handle_events for EventManager doesn't take max_events as arguments, so
we need to update the comments for it.
p.s. max_events is defined when initializing the EventManager.
fixes: #5382
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
- libs/kata-types: adjust default_vcpus correctly
- runtime-rs: delete duplicated PASSTHROUGH_FS_DIR const
- Enable ACRN hypervisor support for Kata 2.x release
- agent: reduce reference count for failed mount
- agent: don't exit early if signal fails due to ESRCH
- kata-sys-util: delete duplicated get_bundle_path
- packaging: Mount $HOME/.docker in the 1st layer container
- Upgrade to Cloud Hypervisor v27.0
- microvm: Remove kernel_irqchip=on option
- kata-sys-util: fix typo `unknow`
- dragonball: update ut for kernel config
- versions: Update gperf url to avoid libseccomp random failures
- versions: Update oci version
- dragonball: fix no "as_str" error on Arm
- tools: release: fix bogus version check
- runtime-rs: update Cargo.lock
- refactor(runtime-rs): Use RwLock in runtime-agent
- runtime-rs: fix shim close_io call to support kubectl cp
- runtime-rs: add comments for runtime-rs shared directory
- workflow: trigger test-kata-deploy with pull_request and fix workflow_dispatch
- Dragonball: update linux_loader to 0.6.0
- modify virtio_net_dev_mgr.rs wrong code comments
- docs: Update urls in runk documentation
- runtime-rs: support watchable mount
- runtime-rs: debug console support in runtime
- kata-deploy: ship the rustified runtime binary
- runtime-rs: define VFIO unbind path as a const
- runtime-rs: set agent timeout to 0 for stream RPCs
- Added SNP-Support for Kata-Containers
- packaging: fix typo in configure-hypervisor.sh
- runtime/runtime-rs: update dependency
- release: Revert kata-deploy changes after 3.0.0-rc0 release
- runtime-rs: add test for StaticResource
- runtime-rs: remove hardcoded string
- docs: add README for runtime-rs hypervisor crate
- runtime-rs: use Path.is_file to check regular files
- osbuilder: Export directory variables for libseccomp
- runtime-rs: add unit tests for network resource
- runtime-rs/resource: use macro to reduce duplicated code
- runtime-rs: fix incorrect comments
- kernel: Add crypto kernel config for s390
- Non-root hypervisor uid reuse bug
- Build-in Sandbox: update dragonball-sandbox dependencies
- docs: Update url in virtualization document
- dragonball: Fix problem that stdio console cannot connect to stdout
- runtime-rs: call TomlConfig's validate function after load
- feat(Shimmgmt): Shim management server and client
53f209af4 libs/kata-types: adjust default_vcpus correctly
ef5a2dc3b agent: don't exit early if signal fails due to ESRCH
435c8f181 acrn: Enable ACRN hypervisor support for Kata 2.x release
c31cf7269 agent: reduce reference count for failed mount
4da743f90 packaging: Mount $HOME/.docker in the 1st layer container
067e2b1e3 runtime: clh: Use the new API to boot with TDX firmware (td-shim)
5d63fcf34 runtime: clh: Re-generate the client code
fe6107042 versions: Upgrade to Cloud Hypervisor v27.0
17de94e11 microvm: Remove kernel_irqchip=on option
3aeaa6459 runtime-rs: delete duplicated PASSTHROUGH_FS_DIR const
43ae97233 kata-sys-util: delete duplicated get_bundle_path
ac0483122 kata-sys-util: fix typo `unknow`
a24127659 versions: Update gperf url to avoid libseccomp random failures
a617a6348 versions: Update oci version
6d585d591 dragonball: fix no "as_str" error on Arm
421729f99 tools: release: fix bogus version check
457b0beaf runtime-rs: update Cargo.lock
f89ada2de dragonball: update ut for kernel config
0e899669e runtime-rs: fix shim close_io call to support kubectl cp
96cf21fad runtime-rs: add comments for runtime-rs shared directory
9bd941098 docs: Update urls in runk documentation
90ecc015e Dragonball: update linux_loader to 0.6.0
4a763925e runtime-rs: support watchable mount
abc26b00b dragonball: modify wrong code comments modify virtio_net_dev_mgr.rs wrong code comments
20bcaf0e3 runtime-rs: set agent timeout to 0 for stream RPCs
274de024c docs: add README for runtime-rs hypervisor crate
a4a23457c osbuilder: Export directory variables for libseccomp
d663f110d kata-deploy: get the config path from cri options
c6b3dcb67 kata-deploy: support kata-deploy for runtime-rs
46965739a runtime-rs: remove hardcoded string
a394761a5 kata-deploy: add installation for runtime-rs
50299a329 refactor(runtime-rs): Use RwLock in runtime agent
9628c7df0 runtime: update runc dependency
7fbc88387 runtime-rs: drop dependency on rustc-serialize
bf2be0cf7 release: Revert kata-deploy changes after 3.0.0-rc0 release
e23bfd615 runtime-rs: make function name more understandable
426a43678 runtime-rs: add unit test and eliminate raw string
87959cb72 runtime-rs: debug console support in runtime
d55cf9ab7 docs: Update url in virtualization document
0399da677 runtime-rs: update dependencies
f6f19917a dragonball: update dragonball-sandbox dependencies
2caee1f38 runtime-rs: define VFIO unbind path as a const
3f65ff2d0 runtime-rs: fix incorrect comments
9670a3caa runtime-rs: use Path.is_file to check regular files
d9e6eb11a docs: Guide to use SNP-VMs with Kata-Containers
ded60173d runtime: Enable choice between AMD SEV and SNP
22bda0838 runtime: Support for AMD SEV-SNP VMs
a2bbd2942 kernel: Introduce SNP kernel
0e69405e1 docs: Developer-Guide updated
105eda5b9 runtime: Initrd path option added to config
a8a8a28a3 runtime-rs/resource: use macro to reduce duplicated code
7622452f4 Dragonball: Fix the problem about stdio console
208233288 runtime-rs: add test for StaticResource
adb33a412 packaging: fix typo in configure-hypervisor.sh
f91431987 runtime: store the user name in hypervisor config
86a02c5f6 kernel: Add crypto kernel config for s390
5cafe2177 runtime: make StopVM thread-safe
c3015927a runtime: add more debug logs for non-root user operation
5add50aea runtime-rs: timeout for shim management client
9f13496e1 runtime-rs: shim management client
aaf6d6908 runtime-rs: call TomlConfig's validate function after load
e891295e1 runtime-rs: shim management - agent-url
59aeb776b runtime-rs: shim management
a828292b4 runtime-rs: add unit tests for network resource
7676cde0c workflow: trigger test-kata-deploy with pull_request
f10827357 workflow: require PR num input on test-kata-deploy workflow_dispatch
428d6dc80 workflow: Revert "workflow: trigger test-kata-deploy with pull_request"
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
This reverts commit 7676cde0c5.
It turns out that when triggerred from a PR, the docker login command is
failing with
```
Error: Cannot perform an interactive login from a non TTY device
```
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
With default_maxvcpus = 0 and default_vcpus = 1 settings, the
default_vcpus will be set to 0 and leads to starting fail.
The default_maxvcpus is not set correctly when it is set to 0,
and the default_vcpus is set to 0.
The correct action is setting default_maxvcpus to the max number
of CPUs or MAX_DRAGONBALL_VCPUS, and the default_vcpus should be
set to the desired value if the valuse is between 0 and
default_maxvcpus.
Fixes: #5110
Signed-off-by: Bin Liu <bin@hyper.sh>
ESRCH usually means the process has exited. In this case,
the execution should continue to kill remaining container processes.
Fixes: #5366
Signed-off-by: Feng Wang <feng.wang@databricks.com>
[Fix up cargo updates]
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Currently ACRN hypervisor support in Kata2.x releases is broken.
This commit re-enables ACRN hypervisor support and also refactors
the code so as to remove dependency on Sandbox.
Fixes#3027
Signed-off-by: Vijay Dhanraj <vijay.dhanraj@intel.com>
The kata agent adds a reference for each storage object before mount
and skip mount again if the storage object is known. We need to
remove the object reference if mount fails.
Fixes: #5364
Signed-off-by: Feng Wang <feng.wang@databricks.com>
In order to ensure that the proxy configuration is passed to the 2nd
layer container, let's ensure the $HOME/.docker/config.json file is
exposed inside the 1st layer container.
For some reason which I still don't fully understand exporting
https_proxy / http_proxy / no_proxy was not enough to get those
variables exported to the 2nd layer container.
In this commit we're creating a "$HOME/.docker" directory, and removing
it after the build, in case it doesn't exist yet. The reason we do this
is to avoid docker not running in case "$HOME/.docker" doesn't exist.
This was not tested with podman, but if there's an issue with podman,
the issue was already there beforehand and should be treated as a
different problem than the one addressed in this commit.
Fixes: #5077
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The containerd stats method and metrics API are broken with Kata 2.5.x, the stats fail to load and the metrics API responds with status code 500
This seems to be down to the conversion from the stats reported by the agent RPC `StatsContainer` where the field `Pagesize` is not
completed by the `setHugetlbStats` method. In the case where multiple sized tables stats are reported, this causes containerd to register two metrics
with the same label set, rather than each being partitioned by the `page` label.
Fixes: #5316
Signed-off-by: Champ-Goblem <cameron@northflank.com>
The new way to boot from TDX firmware (e.g. td-shim) is using the
combination of '--platform tdx=on' with '--firmware tdshim'.
Fixes: #5309
Signed-off-by: Bo Chen <chen.bo@intel.com>
This release has been tracked in our new [roadmap project ](https://github.com/orgs/cloud-hypervisor/projects/6) as iteration v27.0.
**Community Engagement**
A new mailing list has been created to support broader community discussions.
Please consider [subscribing](https://lists.cloudhypervisor.org/g/dev/); an announcement of a regular meeting will be
announced via this list shortly.
**Prebuilt Packages**
Prebuilt packages are now available. Please see this [document](https://github.com/cloud-hypervisor/obs-packaging/blob/main/README.md)
on how to install. These packages also include packages for the different
firmware options available.
**Network Device MTU Exposed to Guest**
The MTU for the TAP device associated with a virtio-net device is now exposed
to the guest. If the user provides a MTU with --net mtu=.. then that MTU is
applied to created TAP interfaces. This functionality is also exposed for
vhost-user-net devices including those created with the reference backend.
**Boot Tracing**
Support for generating a trace report for the boot time has been added
including a script for generating an SVG from that trace.
**Simplified Build Feature Flags**
The set of feature flags, for e.g. experimental features, have been simplified:
* msvh and kvm features provide support for those specific hypervisors
(with kvm enabled by default),
* tdx provides support for Intel TDX; and although there is no MSHV support
now it is now possible to compile with the mshv feature,
* tracing adds support for boot tracing,
* guest_debug now covers both support for gdbing a guest (formerly gdb
feature) and dumping guest memory.
The following feature flags were removed as the functionality was enabled by
default: amx, fwdebug, cmos and common.
**Asynchronous Kernel Loading**
AArch64 has gained support for loading the guest kernel asynchronously like
x86-64.
**GDB Support for AArch64**
GDB stub support (accessed through --gdb under guest_debug feature) is now
available on AArch64 as well as as x86-64.
**Notable Bug Fixes**
* This version incorporates a version of virtio-queue that addresses an issue
where a rogue guest can potentially DoS the VMM,
* Improvements around PTY handling for virtio-console and serial devices,
* Improved error handling in virtio devices.
**Deprecations**
Deprecated features will be removed in a subsequent release and users should
plan to use alternatives.
* Booting legacy firmware (compiled without a PVH header) has been deprecated.
All the firmware options (Cloud Hypervisor OVMF and Rust Hypervisor Firmware)
support booting with PVH so support for loading firmware in a legacy mode is no
longer needed. This functionality will be removed in the next release.
Details can be found: https://github.com/cloud-hypervisor/cloud-hypervisor/releases/tag/v27.0
Note: To have the new API of loading firmware for booting (e.g. boot
from td-shim), a specific commit revision after the v27.0 release is
used as the Cloud Hypervisor version from the 'versions.yaml'.
Fixes: #5309
Signed-off-by: Bo Chen <chen.bo@intel.com>
`kernel_irqchip` option doesn't seem to bring any benefits and, on the
contrary, its usage cause issues when using the microvm machine type.
With this in mind, let's remove it.
Fixes: #1984, #4386
Signed-off-by: norbjd <norbjd@users.noreply.github.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add support for ephemeral storage and k8s emptydir.
Depends-on:github.com/kata-containers/tests#5161
Fixes: #4730
Signed-off-by: Bin Liu <bin@hyper.sh>
This PR updates the gperf url to avoid random failures when installing
libseccomp as it seems that the mirrror url produces network random
failures in multiple CIs.
Fixes#5294
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Cmdline struct update in the latest linux-loader lib and its as_str
method is changed to as_cstring, thus we need fix it according whereas
the old as_str method is used.
Fixes: #5287
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Shell expands `*"rc"*` to the top-level `src` directory. This results
in comparing a version with a directory name. This doesn't make sense
and causes the script to choose the wrong branch of the `if`.
The intent of the check is actually to detect `rc` in the version.
Fixes: #5283
Signed-off-by: Greg Kurz <groug@kaod.org>
There are two duplicated mentioning of the rust libraries in README.md.
Let's just remove them all as the section is intended to list out core
Kata components rather than general libraries.
Fixes: #5275
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Since linux loader is updated in the Dragonball and the api for Cmdline
has been changed ( as_str() changed to as_cstring() ), we need to update
unit test in Dragonball.
fixes: #5277
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Add close_io to shim and call agent's close_stdin in close_io.
Depends-on:github.com/kata-containers/tests#5155
Fixes: #4729
Signed-off-by: Bin Liu <bin@hyper.sh>
Since linux-loader 0.4.0 and 0.5.0 is yanked due to null terminator bug,
we need to update linux-loader to 0.6.0.
And as_str() function should also be changed.
fixes: #5253
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
For stream RPCs:
- write_stdin
- read_stdout
- read_stderr
there should be no timeout (by setting it to 0).
Fixes: #5249
Signed-off-by: Bin Liu <bin@hyper.sh>
The qmp command of hotplug cpu failed error was hidden. It didn't friendly for
the user tracing the hotplug cpu error. The PR help us to improve the hotplug
cpu error log. Add real qemu command error log for `failed to hot add vCPUs`.
Through the error message, we can get the reason of the failed qmp command
for hotplug cpu operation.
Fixes: #5234
Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
To avoid the random failures when we are building the rootfs as it seems
that it does not find the value for the libseccomp and gperf directory,
this PR export these variables.
Fixes#5232
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
As 3.0.0-rc0 has been released, let's switch the kata-deploy / kata-cleanup
tags back to "latest", and re-add the kata-deploy-stable and the
kata-cleanup-stable files.
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
This PR updates the url for the cloud hypervisor in the virtualization
document.
Fixes#5203
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
In src/runtime-rs/crates/hypervisor/src/device/vfio.rs,
the path of new_id is defined as a const, but unbind is used
as a local variable, they should be unified to const.
Fixes: #5189
Signed-off-by: Bin Liu <bin@hyper.sh>
The guide describes how to set Kata-Containers up so that AMD SEV-SNP
encrypted VMs are used when deploying confidential containers.
Signed-off-by: Joana Pecholt <joana.pecholt@aisec.fraunhofer.de>
This is based on a patch from @niteeshkd that adds a config
parameter to choose between AMD SEV and SEV-SNP VMs as the
confidential guest type in case both types are supported. SEV is
the default.
Signed-off-by: Joana Pecholt <joana.pecholt@aisec.fraunhofer.de>
This commit adds AMD SEV-SNP as a confidential guest option to the
runtime. Information on required components such as OVMF, QEMU and
a kernel supporting SEV-SNP are defined in the versions file and
corresponding configs are added.
Note: The CPU model 'host' provided by the current SNP-QEMU does
not support all SNP capabilities yet, which is why this option is
changed to EPYC-v4.
Note: The guest's physical address space reduction specified with
ReducedPhysBits is 1. Details are can be found in Section 15.34.6
here https://www.amd.com/system/files/TechDocs/24593.pdfFixes#4437
Signed-off-by: Joana Pecholt <joana.pecholt@aisec.fraunhofer.de>
Developer-Guide.md is updated to work using current golang versions.
Related Readmes are also updated.
Signed-off-by: Joana Pecholt <joana.pecholt@aisec.fraunhofer.de>
Adds initrd configuration option to the configuration.toml that is
generated for the setup using QEMU.
Signed-off-by: Joana Pecholt <joana.pecholt@aisec.fraunhofer.de>
- runtime-rs: delete some allow(dead_code) attributes
- kata-types: don't check virtio_fs_daemon for inline-virtio-fs
- kata-types: change return type of getting CPU period/quota function
- runtime-rs: fix host device check pattern
- runtime-rs: remove meaningless comment
- runtime-rs: update rust runtime roadmap
- runk: Enable seccomp support by default
- config: add "inline-virtio-fs" as a "shared_fs" type
- runtime-rs: add README.md
- runk: Refactor container builder
- kernel: fix kernel tarball name for SEV
- libs/kata-types: replace tabs by spaces in comments
- gperf: point URL to mirror site
be242a3c3 release: Adapt kata-deploy for 3.0.0-rc0
156e1c324 runtime-rs: delete some allow(dead_code) attributes
62cf6e6fc runtime-rs: remove meaningless comment
bcf6bf843 runk: Enable seccomp support by default
2b1d05857 runtime-rs: fix host device check pattern
85b49cee0 runtime-rs: add README.md
36d805fab config: add "inline-virtio-fs" as a "shared_fs" type
b948a8ffe kernel: fix kernel tarball name for SEV
50f912615 libs/kata-types: replace tabs by spaces in comments
96c8be715 libs/kata-types: change return type of getting CPU period/quota
fc9c6f87a kata-types: don't check virtio_fs_daemon for inline-virtio-fs
968c2f6e8 runk: Refactor container builder
84268f871 runtime-rs: update rust runtime roadmap
566656b08 gperf: point URL to mirror site
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
kata-deploy files must be adapted to a new release. The cases where it
happens are when the release goes from -> to:
* main -> stable:
* kata-deploy-stable / kata-cleanup-stable: are removed
* stable -> stable:
* kata-deploy / kata-cleanup: bump the release to the new one.
There are no changes when doing an alpha release, as the files on the
"main" branch always point to the "latest" and "stable" tags.
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Some device types have the same definition, they can be implemented
by macro to reduce code.
And this commit also deleted the `peer_name` field of the structs that
is never been used.
Fixes: #5170
Signed-off-by: Bin Liu <bin@hyper.sh>
The user name will be used to delete the user instead of relying on
uid lookup because uid can be reused.
Fixes: #5155
Signed-off-by: Feng Wang <feng.wang@databricks.com>
Enable seccomp support in `runk` by default.
Due to this, `runk` is built with `gnu libc` by default
because the building `runk` with statically linked the `libseccomp`
and `musl` requires additional configurations.
Also, general container runtimes are built with `gnu libc` as
dynamically linked binaries by default.
The user can disable seccomp by `make SECCOMP=no`.
Fixes: #4896
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Let client side support timeout if the timeout value is set.
If timeout not set, execute directly.
Fixes: #5114
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
"inline-virtio-fs" is newly supported by kata 3.0 as a "shared_fs" type,
it should be described in configuration file.
"inline-virtio-fs" is the same as "virtio-fs", but it is running in
the same process of shim, does not need an external virtiofsd process.
Fixes: #5102
Signed-off-by: Bin Liu <bin@hyper.sh>
Add client side function(public), to establish http connections (PUT,
POST, GET) to the long standing shim mgmt server.
Fixes: #5114
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
Add agent-url to its handler. The general framework of registering URL
handlers is done.
Fixes: #5114
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
Add shim management http server and boot it as a light-weight thread
when the sandbox is created.
Fixes: #5114
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
period should have a type of u64, and quota should be i64, the
function of getting CPU period and quota from annotations should
use the same data type as function return type.
Fixes: #5100
Signed-off-by: Bin Liu <bin@hyper.sh>
Kata 3.0 introduced 3 new configurations under runtime section:
name="virt_container"
hypervisor_name="dragonball"
agent_name="kata"
Blank values will lead to starting to fail.
Adding default values will make user easy to migrate to kata 3.0.
Fixes: #5098
Signed-off-by: Bin Liu <bin@hyper.sh>
Refactor the container builder code (`InitContainer` and `ActivatedContainer`)
to make it easier to understand and to maintain.
The details:
1. Separate the existing `builder.rs` into an `init_builder.rs` and
`activated_builder.rs` to make them easy to read and maintain.
2. Move the `create_linux_container` function from the `builder.rs` to
`container.rs` because it is shared by the both files.
3. Some validation functions such as `validate_spec` from `builder.rs`
to `utils.rs` because they will be also used by other components as
utilities in the future.
Fixes: #5033
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
In the commit 54d6d01754 we ended up
removing the BUILD_SUFFIX argument passed to QEMU as it only seemed to
be used to generate the HYPERVISOR_NAME and PKGVERSION, which were added
as arguments to the dockerfile.
However, it turns out BUILD_SUFFIX is used by the `qemu-build-post.sh`
script, so it can rename the QEMU binary accordingly.
Let's just bring it back.
Fixes: #5078
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In some case the call of cleanup from shim to service manager will fail,
and the shim process will continue to running, that will make process leak.
This commit will force shutdown the shim process in case of any errors in
service crate.
Fixes: #5087
Signed-off-by: Bin Liu <bin@hyper.sh>
Following the instructions in guidance doc will result in the ECONNREFUSED,
thus we need to keep the unix socket address in the two commands consistent.
Fixes: #5085
Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
Dockerfile cannot decipher multiple conditional statements in the main RUN call.
Cannot segregate statements in Dockerfile with '{}' braces without wrapping entire statement in 'bash -c' statement.
Dockerfile does not support setting variables by bash command.
Must set HYPERVISOR_NAME and PKGVERSION from parent script: build-base-qemu.sh
Fixes: #5078
Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
amend_spec do two works:
- modify the spec
- check if the pid namespace is enabled
This make it confusable. So split it into two functions.
Fixes: #5062
Signed-off-by: Bin Liu <bin@hyper.sh>
Augment the mock hypervisor so that we can validate that ACPI memory hotplug
is carried out as expected.
We'll augment the number of memory slots in the hypervisor config each
time the memory of the hypervisor is changed. In this way we can ensure
that large memory hotplugs are broken up into appropriately sized
pieces in the unit test.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
If we're using ACPI hotplug for memory, there's a limitation on the
amount of memory which can be hotplugged at a single time.
During hotplug, we'll allocate memory for the memmap for each page,
resulting in a 64 byte per 4KiB page allocation. As an example, hotplugging 12GiB
of memory requires ~192 MiB of *free* memory, which is about the limit
we should expect for an idle 256 MiB guest (conservative heuristic of 75%
of provided memory).
From experimentation, at pod creation time we can reliably add 48 times
what is provided to the guest. (a factor of 48 results in using 75% of
provided memory for hotplug). Using prior example of a guest with 256Mi
RAM, 256 Mi * 48 = 12 Gi; 12GiB is upper end of what we should expect
can be hotplugged successfully into the guest.
Note: It isn't expected that we'll need to hotplug large amounts of RAM
after workloads have already started -- container additions are expected
to occur first in pod lifecycle. Based on this, we expect that provided
memory should be freely available for hotplug.
If virtio-mem is being utilized, there isn't such a limitation - we can
hotplug the max allowed memory at a single time.
Fixes: #4847
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
gperf download fails intermittently.
Changing to mirror site will hopefully increase download reliability.
Fixes: #5057
Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
00aadfe20a introduced a regression on
`make cc-tdx-kernel-tarball` as we stopped passing all the needed
information to the `build-kernel.sh` script, leading to requiring `yq`
installed in the container used to build the kernel.
This commit partially reverts the faulty one, rewritting it in a way the
old behaviour is brought back, without changing the behaviour that was
added by the faulty commit.
Fixes: #5043
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It'll be useful to get the total memory provided to the guest
(hotplugged + coldplugged). We'll use this information when calcualting
how much memory we can add at a time when utilizing ACPI hotplug.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
- Initrd fixes for ubuntu systemd
- kernel: Add CONFIG_CGROUP_HUGETLB=y as part of the cgroup fragments
- Fix kata-deploy to work on CI context
- github-actions: Auto-backporting
- runtime-rs: add support for core scheduling
- ci: Use versions.yaml for the libseccomp
- runk: Add cli message for init command
- agent: add some logs for mount operation
- Use iouring for qemu block devices
- logging: Replace nix::Error::EINVAL with more descriptive msgs
- kata-deploy: fix threading conflicts
- kernel: Ignore CONFIG_SPECULATION_MITIGATIONS for older kernels
- runtime-rs: support loading kernel modules in guest vm
- TDX: Get TDX working again with Cloud Hypervisor + a minor change on QEMU's code
- runk: Move delete logic to libcontainer
- runtime: cri-o annotations have been moved to podman
- Fix depbot reported rust crates dependency security issues
- UT: test_load_kernel_module needs root
- enable vmx for vm factory
- runk: add pause/resume commands
- kernel: upgrade guest kernel support to 5.19
- Drop-in cfg files support in runtime-rs
- agent: do some rollback works if case of do_create_container failed
- network: Fix error message for setting hardware address on TAP interface
- Upgrade to Cloud Hypervisor v26.0
- runtime: tracing: End root span at end of trace
- ci: Update libseccomp version
- dep: update nix dependency
- Updated the link target of CRI-O
- libs/test-utils: share test code by create a new crate
dc32c4622 osbuilder: fix ubuntu initrd /dev/ttyS0 hang
cc5f91dac osbuilder: add systemd symlinks for kata-agent
c08a8631e agent: add some logs for mount operation
0a6f0174f kernel: Ignore CONFIG_SPECULATION_MITIGATIONS for older kernels
6cf16c4f7 agent-ctl: fix clippy error
4b57c04c3 runtime-rs: support loading kernel modules in guest vm
dc90eae17 qemu: Drop unnecessary `tdx_guest` kernel parameter
d4b67613f clh: Use HVC console with TDX
c0cb3cd4d clh: Avoid crashing when memory hotplug is not allowed
9f0a57c0e clh: Increase API and SandboxStop timeouts for TDX
b535bac9c runk: Add cli message for init command
c142fa254 clh: Lift the sharedFS restriction used with TDX
bdf8a57bd runk: Move delete logic to libcontainer
a06d819b2 runtime: cri-o annotations have been moved to podman
ffd1c1ff4 agent-ctl/trace-forwarder: udpate thread_local dependency
69080d76d agent/runk: update regex dependency
e0ec09039 runtime-rs: update async-std dependency
763ceeb7b logging: Replace nix::Error::EINVAL with more descriptive msgs
4ee2b99e1 kata-deploy: fix threading conflicts
731d39df4 kernel: Add CONFIG_CGROUP_HUGETLB=y as part of the cgroup fragments
96d903734 github-actions: Auto-backporting
a6fbaac1b runk: add pause/resume commands
8e201501e kernel: fix for set_kmem_limit error
00aadfe20 kernel: SEV guest kernel upgrade to 5.19.2
0d9d8d63e kernel: upgrade guest kernel support to 5.19.2
57bd3f42d runtime-rs: plug drop-in decoding into config-loading code
87b97b699 runtime-rs: add filesystem-related part of drop-in handling
cf785a1a2 runtime-rs: add core toml::Value tree merging
92f7d6bf8 ci: Use versions.yaml for the libseccomp
f508c2909 runtime: constify splitIrqChipMachineOptions
2b0587db9 runtime: VMX is migratible in vm factory case
fa09f0ec8 runtime: remove qemuPaths
326f1cc77 agent: enrich some error code path
4f53e010b agent: skip test_load_kernel_module if non-root
3a597c274 runtime: clh: Use the new 'payload' interface
16baecc5b runtime: clh: Re-generate the client code
50ea07183 versions: Upgrade to Cloud Hypervisor v26.0
f7d41e98c kata-deploy: export CI in the build container
4f90e3c87 kata-deploy: add dockerbuild/install_yq.sh to gitignore
8ff5c10ac network: Fix error message for setting hardware address on TAP interface
338c28295 dep: update nix dependency
78231a36e ci: Update libseccomp version
34746496b libs/test-utils: share test code by create a new crate
3829ab809 docs: Update CRI-O target link
fcc1e0c61 runtime: tracing: End root span at end of trace
c1e3b8f40 govmm: Refactor qmp functions for adding block device
598884f37 govmm: Refactor code to get rid of redundant code
00860a7e4 qmp: Pass aio backend while adding block device
e1b49d758 config: Add block aio as a supported annotation
ed0f1d0b3 config: Add "block_device_aio" as a config option for qemu
b6cd2348f govmm: Add io_uring as AIO type
81cdaf077 govmm: Correct documentation for Linux aio.
a355812e0 runtime-rs: fixed bug on core-sched error handling
591dfa4fe runtime-rs: add support for core scheduling
09672eb2d agent: do some rollback works if case of do_create_container failed
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Updates versions of crossbeam-channel because 0.52.0 is a yanked package
(creators mark version as not for release except as a dependency for
another package)
Updates chrono to use >0.42.0 to avoid:
https://rustsec.org/advisories/RUSTSEC-2020-0159
Updates lz4-sys.
Signed-off-by: Derek Lee <derlee@redhat.com>
Adds oci under the src/libs workplace.
oci shares a Cargo.lock file with the rest of src/libs but was not
listed as a member of the workspace.
There is no clear reason why it is not included in the workspace, so
adding it so cargo-deny stop complaining
Signed-off-by: Derek Lee <derlee@redhat.com>
One of the checks done by cargo-deny is ensuring all crates have a valid
license. As the rust programs import each other, cargo.toml files
without licenses trigger the check. While I could disable this check
this would be bad practice.
This adds an Apache-2.0 license in the Cargo.toml files.
Some of these files already had a header comment saying it is an Apache
license. As the entire project itself is under an Apache-2.0 license, I
assumed all individual components would also be covered under that
license.
Signed-off-by: Derek Lee <derlee@redhat.com>
Adds cargo-deny to scan for vulnerabilities and license issues regarding
rust crates.
GitHub Actions does not have an obvious way to loop over each of the
Cargo.toml files. To avoid hardcoding it, I worked around the problem
using a composite action that first generates the cargo-deny action by
finding all Cargo.toml files before calling this new generated action in
the master workflow.
Uses recommended deny.toml from cargo-deny repo with the following
modifications:
ignore = ["RUSTSEC-2020-0071"]
because chrono is dependent on the version of time with the
vulnerability and there is no simple workaround
multiple-versions = "allow"
Because of the above error and other packages, there are instances
where some crates require different versions of a crate.
unknown-git = "allow"
I don't see a particular issue with allowing crates from other repos.
An alternative would be the manually set each repo we want in an
allow-git list, but I see this as more of a nuisance that its worth.
We could leave this as a warning (default), but to avoid clutter I'm
going to allow it.
If deny.toml needs to be edited in the future, here's the guide:
https://embarkstudios.github.io/cargo-deny/index.htmlFixes#3359
Signed-off-by: Derek Lee <derlee@redhat.com>
This PR updates the cni plugins version that is being used in the kata CI.
Fixes#5039
Depends-on: github.com/kata-containers/tests#5088
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This removes two options that are not needed (any longer). These
are not set for any kernel so they do not need to be ignored either.
Fixes#5035
Signed-off-by: Joana Pecholt <joana.pecholt@aisec.fraunhofer.de>
Guest log is showing a hang on systemd getty start.
Adding symlink for /dev/ttyS0 resolves issue.
Fixes: #4932
Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
Somewhere is lack of log info, add more details about
the storage and log when error will help understand
what happened.
Fixes: #4962
Signed-off-by: Bin Liu <bin@hyper.sh>
TDX kernel is based on a kernel version which doesn't have the
CONFIG_SPECULATION_MITIGATIONS option.
Having this in the allow list for missing configs avoids a breakage in
the TDX CI.
Fixes: #4998
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Users can specify the kernel module to be loaded through the agent
configuration in kata configuration file or in pod anotation file.
And information of those modules will be sent to kata agent when
sandbox is created.
Fixes: #4894
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
With the current TDX kernel used with Kata Containers, `tdx_guest` is
not needed, as TDX_GUEST is now a kernel configuration.
With this in mind, let's just drop the kernel parameter.
Fixes: #4981
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As right now the TDX guest kernel doesn't support "serial" console,
let's switch to using HVC in this case.
Fixes: #4980
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The runtime will crash when trying to resize memory when memory hotplug
is not allowed.
This happens because we cannot simply set the hotplug amount to zero,
leading is to not set memory hotplug at all, and later then trying to
access the value of a nil pointer.
Fixes: #4979
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
While doing tests using `ctr`, I've noticed that I've been hitting those
timeouts more frequently than expected.
Till we find the root cause of the issue (which is *not* in the Kata
Containers), let's increase the timeouts when dealing with a
Confidential Guest.
Fixes: #4978
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add cli message for init command to tell the user
not to run this command directly.
Fixes: #4367
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
When booting the TDX kernel with `tdx_disable_filter`, as it's been done
for QEMU, VirtioFS can work without any issues.
Whether this will be part of the upstream kernel or not is a different
story, but it easily could make it there as Cloud Hypervisor relies on
the VIRTIO_F_IOMMU_PLATFORM feature, which forces the guest to use the
DMA API, making these devices compatible with TDX.
See Sebastien Boeuf's explanation of this in the
3c973fa7ce208e7113f69424b7574b83f584885d commit:
"""
By using DMA API, the guest triggers the TDX codepath to share some of
the guest memory, in particular the virtqueues and associated buffers so
that the VMM and vhost-user backends/processes can access this memory.
"""
Fixes: #4977
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Move delete logic to `libcontainer` crate to make the code clean
like other commands.
Fixes: #4975
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Let's swith to depending on podman which also simplies indirect
dependency on kubernetes components. And it helps to avoid cri-o
security issues like CVE-2022-1708 as well.
Fixes: #4972
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
So that we bump several indirect dependencies like crossbeam-channel,
crossbeam-utils to bring in fixes to known security issues like CVE-2020-15254.
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Replaces instances of anyhow!(nix::Error::EINVAL) with other messages to
make it easier to debug.
Fixes#954
Signed-off-by: Derek Lee <derlee@redhat.com>
Fix threading conflicts when kata-deploy 'make kata-tarball' is called.
Force the creation of rootfs tarballs to happen serially instead of in parallel.
Fixes: #4787
Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
Kata guest os cgroup is not work properly kata guest kernel config option
CONFIG_CGROUP_HUGETLB is not set, leading to:
root@clr-b08d402cc29d44719bb582392b7b3466 ls /sys/fs/cgroup/hugetlb/
ls: cannot access '/sys/fs/cgroup/hugetlb/': No such file or directory
Fixes: #4953
Signed-off-by: Miao Xia <xia.miao1@zte.com.cn>
An implementation of semi-automating the backporting
process.
This implementation has two steps:
1. Checking whether any associated issues are marked as bugs
If they do, mark with `auto-backport` label
2. On a successful merge, if there is a `auto-backport` label and there
are any tags of `backport-to-BRANCHNAME`, it calls an action that
cherry-picks the commits in the PR and automatically creates a PR to
those branches.
This action uses https://github.com/sqren/backport-github-actionFixes#3618
Signed-off-by: Derek Lee <derlee@redhat.com>
To make cgroup v1 and v2 works well, I use `cgroups::cgroup` in
`Container` to manager cgroup now. `CgroupManager` in rustjail has some
drawbacks. Frist, methods in Manager traits are not visiable. So we need
to modify rustjail and make them public. Second, CgrupManager.cgroup is
private too, and it can't be serialized. We can't load/save it in
status file. One solution is adding getter/setter in rustjail, then
create `cgroup` and set it when loading status. In order to keep the
modifications to a minimum in rustjail, I use `cgroups::cgroup`
directly. Now it can work on cgroup v1 or v2, since cgroup-rs do this
stuff.
Fixes: #4364#4821
Signed-off-by: Chen Yiyang <cyyzero@qq.com>
kernel: Update SEV guest kernel to 5.19.2
Kernel 5.19.2 has all the needed patches for running SEV, thus let's update it and stop using the version coming from confidential-containers.
Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
To plug drop-in support into existing config-loading code in a robust
way, more specifically to create a single point where this needs to be
handled, load_from_file() and load_raw_from_file() were refactored.
Seeing as the original implemenations of both functions were identical
apart from adjust_config() calls in load_from_file(), load_from_file()
was reimplemented in terms of load_raw_from_file().
Fixes #4771
Signed-off-by: Pavel Mores <pmores@redhat.com>
The central function being added here is load() which takes a path to a
base config file and uses it to load the base config file itself, find
the corresponding drop-in directory (get_dropin_dir_path()), iterate
through its contents (update_from_dropins()) and load each drop-in in
turn and merge its contents with the base file (update_from_dropin()).
Also added is a test of load() which mirrors the corresponding test in
the golang runtime (TestLoadDropInConfiguration() in config_test.go).
Signed-off-by: Pavel Mores <pmores@redhat.com>
This is the core functionality of merging config file fragments into the
base config file. Our TOML parser crate doesn't seem to allow working
at the level of TomlConfig instances like BurntSushi, used in the Golang
runtime, does so we implement the required functionality at the level of
toml::Value trees.
Tests to verify basic requirements are included. Values set by a base
config file and not touched by a subsequent drop-in should be preserved.
Drop-in config file fragments should be able to change values set by the
base config file and add settings not present in the base. Conversion
of a merged tree into a mock TomlConfig-style structure is tested as
well.
Signed-off-by: Pavel Mores <pmores@redhat.com>
It would be nice to use `versions.yaml` for the maintainability.
Previously, we have been specified the `libseccomp` and the `gperf` version
directly in this script without using the `versions.yaml` because the current
snap workflow is incomplete and fails.
This is because snap CI environment does not have kata-cotnainers repository
under ${GOPATH}. To avoid the failure, the `rootfs.sh` extracts the libseccomp
version and url in advance and pass them to the `install_libseccomp.sh` as
environment variables.
Fixes: #4941
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Highlights from the Cloud Hypervisor release v26.0:
**SMBIOS Improvements via `--platform`**
`--platform` and the appropriate API structure has gained support for supplying
OEM strings (primarily used to communicate metadata to systemd in the guest)
**Unified Binary MSHV and KVM Support**
Support for both the MSHV and KVM hypervisors can be compiled into the same
binary with the detection of the hypervisor to use made at runtime.
**Notable Bug Fixes**
* The prefetchable flag is preserved on BARs for VFIO devices
* PCI Express capabilties for functionality we do not support are now filtered
out
* GDB breakpoint support is more reliable
* SIGINT and SIGTERM signals are now handled before the VM has booted
* Multiple API event loop handling bug fixes
* Incorrect assumptions in virtio queue numbering were addressed, allowing
thevirtio-fs driver in OVMF to be used
* VHDX file format header fix
* The same VFIO device cannot be added twice
* SMBIOS tables were being incorrectly generated
**Deprecations**
Deprecated features will be removed in a subsequent release and users should
plan to use alternatives.
The top-level `kernel` and `initramfs` members on the `VmConfig` have been
moved inside a `PayloadConfig` as the `payload` member. The OpenAPI document
has been updated to reflect the change and the old API members continue to
function and are mapped to the new version. The expectation is that these old
versions will be removed in the v28.0 release.
**Removals**
The following functionality has been removed:
The unused poll_queue parameter has been removed from --disk and
equivalent. This was residual from the removal of the vhost-user-block
spawning feature.
Details can be found: https://github.com/cloud-hypervisor/cloud-hypervisor/releases/tag/v26.0Fixes: #4952
Signed-off-by: Bo Chen <chen.bo@intel.com>
The clone_tests_repo() in ci/lib.sh relies on CI variable to decide
whether to checkout the tests repository or not. So it is required to
pass that variable down to the build container of kata-deploy, otherwise
it can fail on some scenarios.
Fixes#4949
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The install_yq.sh is copied to tools/packaging/kata-deploy/local-build/dockerbuild
so that it is added in the kata-deploy build image. Let's tell git to
ignore that file.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
More and more Rust code is introduced, the test utils original in agent
should be made easy to share, move it into a new crate will make it
easy to share between different crates.
Fixes: #4925
Signed-off-by: Bin Liu <bin@hyper.sh>
vergen is a build dependency, but it is not being used.
we are processing ver/commit hash by make command, but not by vergen.
Fixes: #4920
Signed-off-by: Bin Liu <bin@hyper.sh>
- runtime-rs: fix design doc's typo
- docs: use curl as default downloader for runtime-rs
- runtime-rs: update Cargo.lock
- Fix some GitHub actions workflow issues
- versions: Update libseccomp version
- runtime-rs:merge runtime rs to main
- nydus: wait nydusd API server ready before mounting share fs
- versions: Update TD-shim due to build breakage
- agent-ctl: Add an empty [workspace]
- packaging: Create no_patches.txt for the SPR-BKC-PC-v9.6.x
- docs: Improve SGX documentation
- runtime: explicitly mark the source of the log is from qemu.log
- runtime: add unlock before return in sendReq
- docs: add back host network limitation
- runk: add ps sub-command
- Depends-on:github.com/kata-containers/tests#4986
- runtime-rs:update rtnetlink version
- runtime-rs:skip the build process when the arch is s390x
- docs: Improve SGX documentation
- agent: Use rtnetlink's neighbours API to add neighbors
- Bump TDX dependencies (QEMU and Kernel)
- OVMF / td-shim: Adjust final tarball location
- libs: fix CI error for protocols
- runtime-rs: merge main to runtime-rs
- packaging: Add support for building TDVF
- versions: Track and add support for building TD-shim
- versions: Upgrade rust version
- Merge Main into runtime-rs branch
- agent: log RPC calls for debugging
- runtime-rs: fix stop failed in azure
- Add support AmdSev build of OVMF
- runtime: Support for host cgroupv2
- versions: Update runc version
- qemu: Add liburing to qemu build
- runtime-rs: fix set share sandbox pid namespace
- Docs: fix tables format error
- versions: Update Firecracker version to v1.1.0
- agent: Fix stream fd's double close
- container: kill all of the processes in a container when it terminated
- fix network failed for kata ci
- runtime-rs: handle default_vcpus greator than default_maxvcpu
- agent: fix fd-double-close problem in ut test_do_write_stream
- runtime-rs: add functionalities support for macvlan and vlan endpoints
- Docs: add rust environment setup for kata 3.0
- rustjail: check result to let it return early
- upgrade nydus version
- support disable_guest_seccomp
- cgroups: remove unnecessary get_paths()
- versions: Update firecracker version
- kata-monitor: fix can't monitor /run/vc/sbs
- runtime-rs: fix sandbox_cgroup_only=false panic
- runtime-rs: fix ctr exit failed
- docs: add installation guide for kata 3.0
- runtime-rs: support functionalities of ipvlan endpoint
- runtime-rs: remove the value of hypervisor path in DB config
- kata-sys-util: upgrade nix version
- runtime-rs: fix some bugs to make runtime-rs on aarch64
- runk: Support `exec` sub-command
- runtime-rs: hypervisor part
- clh: Don't crash if no network device is set by the upper layer
- packaging: Rework how ${BUILD_SUFFIX} is used with the QEMU builder scripts
- versions: Update Cloud Hypervisor to v25.0
- Runtime-rs merge main
- kernel: Deduplicate code used for building TEE kernels
- runtime-rs: Dragonball-sandbox - add virtio device feature support for aarch64
- packaging: Simplify config path handling
- build: save lines for repository_owner check
- kata 3.0 Architecture
- Fix clh tarball build
- runtime-rs: built-in Dragonball sandbox part III - virtio-blk, virtio-fs, virtio-net and VMM API support
- runtime: Fix DisableSelinux config
- docs: Update URL links for containerd documentation
- docs: delete CRI containerd plugin statement
- release: Revert kata-deploy changes after 2.5.0-rc0 release
- tools/snap: simplify nproc
- action: revert commit message limit to 150 bytes
- runtime-rs: Dragonball sandbox - add Vcpu::configure() function for aarch64
- runtime-rs: makefile for dragonball
- runtime-rs:refactor network model with netlink
- runtime-rs: Merge Main into runtime-rs branch
- runtime-rs: built-in Dragonball sandbox part II - vCPU manager
- runtime-rs: runtime-rs merge main
- runtime-rs: built-in Dragonball sandbox part I - resource and device managers
caada34f1 runtime-rs: fix design doc's typo
b61dda40b docs: use curl as default downloader for runtime-rs
ca9d16e5e runtime-rs: update Cargo.lock
99a7b4f3e workflow: Revert "static-checks: Allow Merge commit to be >75 chars"
d14e80e9f workflow: Revert "docs: modify move-issues-to-in-progress.yaml"
1f4b6e646 versions: Update libseccomp version
8a4e69008 versions: Update TD-shim due to build breakage
065305f4a agent-ctl: Add an empty [workspace]
1444d7ce4 packaging: Create no_patches.txt for the SPR-BKC-PC-v9.6.x
2ae807fd2 nydus: wait nydusd API server ready before mounting share fs
c8d4ea84e docs: Improve SGX documentation
d8ad16a34 runtime: add unlock before return in sendReq
8bbffc42c runtime-rs:update rtnetlink version
c5452faec docs: Improve SGX documentation
389ae9702 runtime-rs:skip the test when the arch is s390x
945e02227 runtime-rs:skip the build process when the arch is s390x
8d1cb1d51 td-shim: Adjust final tarball location
62f05d4b4 ovmf: Adjust final tarball location
9972487f6 versions: Bump Kernel TDX version
c9358155a kernel: Sort the TDX configs alphabetically
dd397ff1b versions: Bump QEMU TDX version
230a22905 runk: add ps sub-command
889557ecb docs: add back host network limitation
c9b5bde30 versions: Track and build TDVF
e6a5a5106 packaging: Generate a tarball as OVMF build result
42eaf19b4 packaging: Simplify OVMF repo clone
4d33b0541 packaging: Don't hardcode "edk2" as the cloned repo's dir.
7247575fa runtime-rs:fix cargo clippy
b06bc8228 versions: Track and add support for building TD-shim
86ac653ba libs: fix CI error for protocols
81fe51ab0 agent: fix unittests for arp neighbors
845c1c03c agent: use rtnetlink's neighbours API to add neighbors
9b1940e93 versions: update rust version
638c2c416 static-build: Add AmdSev option for OVMF builder Introduces new build of firmware needed for SEV
f0b58e38d static-build: Add build script for OVMF
fa0b11fc5 runtime-rs: fix stdin hang in azure
5c3155f7e runtime: Support for host cgroup v2
4ab45e5c9 docs: Update support for host cgroupv2
326eb2f91 versions: Update runc version
f5aa6ae46 agent: Fix stream fd's double close problem
6e149b43f Docs: fix tables format error
85f4e7caf runtime: explicitly mark the source of the log is from qemu.log
56d49b507 versions: Update Firecracker version to v1.1.0
b3147411e runtime-rs:add unit test for set share pid ns
1ef3f8eac runtime-rs: set share sandbox pid namespace
57c556a80 runtime-rs: fix stop failed in azure
0e24f47a4 agent: log RPC calls for debugging
c825065b2 runtime-rs: fix tc filter setup failed
e0194dcb5 runtime-rs: update route destination with prefix
fa85fd584 docs: add rust environment setup for kata 3.0
896478c92 runtime-rs: add functionalities support for macvlan and vlan endpoints
df79c8fe1 versions: Update firecracker version
912641509 agent: fix fd-double-close problem in ut test_do_write_stream
43045be8d runtime-rs: handle default_vcpus greator than default_maxvcpu
0d7cb7eb1 agent: delete agent-type property in announce
eec9ac81e rustjail: check result to let it return early.
402bfa0ce nydus: upgrade nydus/nydus-snapshotter version
54f53d57e runtime-rs: support disable_guest_seccomp
4331ef80d Runtime-rs: add installation guide for rust-runtime
72dbd1fcb kata-monitor: fix can't monitor /run/vc/sbs.
e9988f0c6 runtime-rs: fix sandbox_cgroup_only=false panic
cebbebbe8 runtime-rs: fix ctr exit failed
62182db64 runtime-rs: add unit test for ipvlan endpoint
99654ce69 runtime-rs: update dbs-xxx dependencies
f4c3adf59 runtime-rs: Add compile option file
545ae3f0e runtime-rs: fix warning
19eca71cd runtime-rs: remove the value of hypervisor path in DB config
d8920b00c runtime-rs: support functionalities of ipvlan endpoint
2b01e9ba4 dragonball: fix warning
996a6b80b kata-sys-util: upgrade nix version
f690b0aad qemu: Add liburing to qemu build
d93e4b939 container: kill all of the processes in this container
3c989521b dragonball: update for review
274598ae5 kata-runtime: add dragonball config check support.
1befbe673 runtime-rs: Cargo lock for fix version problem
3d6156f6e runtime-rs: support dragonball and runtime-binary
3f6123b4d libs: update configuration and annotations
9ae2a45b3 cgroups: remove unnecessary get_paths()
be31207f6 clh: Don't crash if no network device is set by the upper layer
051181249 packaging: Add a "-" in the dir name if $BUILD_DIR is available
dc3b6f659 versions: Update Cloud Hypervisor to v25.0
201ff223f packaging: Use the $BUILD_SUFFIX when renaming the qemu binary
1a25afcdf kernel: Allow passing the URL to download the tarball
80c68b80a kernel: Deduplicate code used for building TEE kernels
d2584991e dragonball: fix dependency unused warning
458f6f42f dragonball: use const string for legacy device type
939959e72 docs: add Dragonball to hypervisors
f6f96b8fe dragonball: add legacy device support for aarch64
7a4183980 dragonball: add device info support for aarch64
f7ccf92dc kata-deploy: Rely on the configured config path
386a523a0 kata-deploy: Pass the config path to CRI-O
13df57c39 build: save lines for repository_owner check
57c2d8b74 docs: Update URL links for containerd documentation
e57a1c831 build: Mark git repos as safe for build
2551924bd docs: delete CRI containerd plugin statement
9cee52153 fmt: do cargo fmt and add a dependency for blk_dev
47a4142e0 fs: change vhostuser and virtio into const
e14e98bbe cpu_topo: add handle_cpu_topology function
5d3b53ee7 downtime: add downtime support
6a1fe85f1 vfio: add vfio as TODO
5ea35ddcd refractor: remove redundant by_id
b646d7cb3 config: remove ht_enabled
cb54ac6c6 memory: remove reserve_memory_bytes
bde6609b9 hotplug: add room for other hotplug solution
d88b1bf01 dragonball: update vsock dependency
dd003ebe0 Dragonball: change error name and fix compile error
38957fe00 UT: fix compile error in unit tests
11b3f9514 dragonball: add virtio-fs device support
948381bdb dragonball: add virtio-net device support
3d20387a2 dragonball: add virtio-blk device support
87d38ae49 Doc: add document for Dragonball API
2bb1eeaec docs: further questions related to upcall
026aaeecc docs: add FAQ to the report
fffcb8165 docs: update the content of the report
42ea854eb docs: kata 3.0 Architecture
efdb92366 build: Fix clh source build as normal user
0e40ecf38 tools/snap: simplify nproc
f59939a31 runk: Support `exec` sub-command
4d89476c9 runtime: Fix DisableSelinux config
090de2dae dragonball: fix the clippy errors.
a1593322b dragonball: add vsock api to api server
89b9ba860 dragonball: add set_vm_configuration api
95fa0c70c dragonball: add start microvm support
5c1ccc376 dragonball: add Vmm struct
4d234f574 dragonball: refactor code layout
cfd5dae47 dragonball: add vm struct
527b73a8e dragonball: remove unused feature in AddressSpaceMgr
3bafafec5 action: extend commit message line limit to 150 bytes
5010c643c release: Revert kata-deploy changes after 2.5.0-rc0 release
7120afe4e dragonball: add vcpu test function for aarch64
648d285a2 dragonball: add vcpu support for aarch64
7dad7c89f dragonball: update dbs-xxx dependency
07231b2f3 runtime-rs:refactor network model with netlink
c8a905206 build: format files
242992e3d build: put install methods in utils.mk
8a697268d build: makefile for dragonball config
9c526292e runtime-rs:refactor network model with netlink
71db2dd5b hotplug: add room for future acpi hotplug mechanism
8bb00a3dc dragonball: fix a bug when generating kernel boot args
2aedd4d12 doc: add document for vCPU, api and device
bec22ad01 dragonball: add api module
07f44c3e0 dragonball: add vcpu manager
78c971875 dragonball: add upcall support
7d1953b52 dragonball: add vcpu
468c73b3c dragonball: add kvm context
e89e6507a dragonball: add signal handler
b6cb2c4ae dragonball: add metrics system
e80e0c464 dragonball: add io manager wrapper
d5ee3fc85 safe-path: fix clippy warning
93c10dfd8 runtime-rs: add crosvm license in Dragonball
dfe6de771 dragonball: add dragonball into kata README
39ff85d61 dragonball: green ci
71f24d827 dragonball: add Makefile.
a1df6d096 Doc: Update Dragonball Readme and add document for device
8619f2b3d dragonball: add virtio vsock device manager.
52d42af63 dragonball: add device manager.
c1c1e5152 dragonball: add kernel config.
6850ef99a dragonball: add configuration manager.
0bcb422fc dragonball: add legacy devices manager
3c45c0715 dragonball: add console manager.
3d38bb300 dragonball: add address space manager.
aff604055 dragonball: add resource manager support.
8835db6b0 dragonball: initial commit
9cb15ab4c agent: add the FSGroup support
ff7874bc2 protobuf: upgrade the protobuf version to 2.27.0
06f398a34 runtime-rs: use withContext to evaluate lazily
fd4c26f9c runtime-rs: support network resource
4be7185aa runtime-rs: runtime part implement
10343b1f3 runtime-rs: enhance runtimes
9887272db libs: enhance kata-sys-util and kata-types
3ff0db05a runtime-rs: support rootfs volume for resource
234d7bca0 runtime-rs: support cgroup resource
75e282b4c runtime-rs: hypervisor base define
bdfee005f runtime-rs: service and runtime framework
4296e3069 runtime-rs: agent implements
d3da156ee runtime-rs: uint FsType for s390x
e705ee07c runtime-rs: update containerd-shim-protos to 0.2.0
8c0a60e19 runtime-rs: modify the review suggestion
278f843f9 runtime-rs: shim implements for runtime-rs
641b73610 libs: enhance kata-sys-util
69ba1ae9e trans: fix the issue of wrong swapness type
d2a9bc667 agent: agent-protocol support async
aee9633ce libs/sys-util: provide functions to execute hooks
8509de0ae libs/sys-util: add function to detect and update K8s emptyDir volume
6d59e8e19 libs/sys-util: introduce function to get device id
5300ea23a libs/sys-util: implement reflink_copy()
1d5c898d7 libs/sys-util: add utilities to parse NUMA information
87887026f libs/sys-util: add utilities to manipulate cgroup
ccd03e2ca libs/sys-util: add wrappers for mount and fs
45a00b4f0 libs/sys-util: add kata-sys-util crate under src/libs
48c201a1a libs/types: make the variable name easier to understand
b9b6d70aa libs/types: modify implementation details
05ad026fc libs/types: fix implementation details
d96716b4d libs/types:fix styles and implementation details
6cffd943b libs/types:return Result to handle parse error
6ae87d9d6 libs/types: use contains to make code more readable
45e5780e7 libs/types: fixed spelling and grammer error
2599a06a5 libs/types:use include_str! in test file
8ffff40af libs/types:Option type to handle empty tomlconfig
626828696 libs/types: add license for test-config.rs
97d8c6c0f docs: modify move-issues-to-in-progress.yaml
8cdd70f6c libs/types: change method to update config by annotation
e19d04719 libs/types: implement KataConfig to wrap TomlConfig
387ffa914 libs/types: support load Kata agent configuration from file
69f10afb7 libs/types: support load Kata hypervisor configuration from file
21cc02d72 libs/types: support load Kata runtime configuration from file
5b89c1df2 libs/types: add kata-types crate under src/libs
4f62a7618 libs/logging: fix clippy warnings
6f8acb94c libs: refine Makefile rules
7cdee4980 libs/logging: introduce a wrapper writer for logging
426f38de9 libs/logging: implement rotator for log files
392f1ecdf libs: convert to a cargo workspace
575df4dc4 static-checks: Allow Merge commit to be >75 chars
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Static resource management should be default to false. If default to be
true, later update sandbox operation, e.g. resize, will not work.
Fixes: #4742
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
The root span should exist the duration of the trace. Defer ending span
until the end of the trace instead of end of function. Add the span to
the service struct to do so.
Fixes#4902
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
As route model is used for specific internal scenario, and it's not for
the general requirement.
Fixes:#4838
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
This PR updates the libseccomp version at the versions.yaml that is
being used in the kata CI.
Fixes#4858
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
"We need a newer nightly 1.62 rust to deal with the change
rust-lang/libc@576f778 on crate libc which breaks the compilation."
This comes from the a pull-request raised on TD-shim repo,
https://github.com/confidential-containers/td-shim/pull/354, which fixes
the issues with the commit being used with Kata Containers.
Let's bump to a newer commit of TD-shim and to a newer version of the
nightly toolchain as part of our versions file.
Fixes: #4840
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
"An empty [workspace] can be used with a package to conveniently create a
workspace with the package and all of its path dependencies", according
to the https://doc.rust-lang.org/cargo/reference/workspaces.html
This is also matches with the suggestion provided by the Cargo itself,
due to the errors faced with the Cloud Hypervisor CI:
```
10:46:23 this may be fixable by adding `go/src/github.com/kata-containers/kata-containers/src/tools/agent-ctl` to the `workspace.members` array of the manifest located at: /tmp/jenkins/workspace/kata-containers-2-clh-PR/Cargo.toml
10:46:23 Alternatively, to keep it out of the workspace, add the package to the `workspace.exclude` array, or add an empty `[workspace]` table to the package's manifest.
```
Fixes: #4843
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The file was added as part of the commit that tested this changes in the
CCv0 branch, but forgotten when re-writing it to the `main` branch.
Fixes: #4841
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
If the API server is not ready, the mount call will fail, so before
mounting share fs, we should wait the nydusd is started and
the API server is ready.
Fixes: #4710
Signed-off-by: liubin <liubin0329@gmail.com>
Signed-off-by: Bin Liu <bin@hyper.sh>
Instead of passing a bunch of arguments to qmp functions for
adding block devices, use govmm BlockDevice structure to reduce these.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Get rid of redundant return values from function.
args and blockdevArgs used to return different values to maintain
compatilibity between qemu versions. These are exactly the same now.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This configuration will allow users to choose between different
I/O backends for qemu, with the default being io_uring.
This will allow users to fallback to a different I/O mechanism while
running on kernels olders than 5.1.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Remove line about annotations support in CRI-O and containerd since it
has been supported for a couple years.
Fixes#4819
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
To keep runtime-rs up to date, we will merge main into runtime-rs every
week.
Fixes:kata-containers#4822
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Update documentation with details regarding
intel-device-plugins-for-kubernetes setup and dependencies.
Fixes#4819
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
github.com/kata-containers/tests#4986.To avoid returning an error when
running the ci, we just skip the test if the arch is s390x
Fixes: #4816
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
github.com/kata-containers/tests#4986.To avoid returning an error when running the ci, we just skip the build
process if the arch is s390x
Fixes: #4816
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
io_uring was introduced as a new kernel IO interface in kernel 5.1.
It is designed for higher performance than the older Linux AIO API.
This feature was added in qemu 5.0.
Fixes#4645
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Let's create the td-shim tarball in the directory where the script was
called from, instead of doing it in the $DESTDIR.
This aligns with the logic being used for creating / extracting the
tarball content, which is already in use by the kata-deploy local build
scripts.
Fixes: #4809
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's create the OVMF tarball in the directory where the script was
called from, instead of doing it in the $DESTDIR.
This aligns with the logic being used for creating / extracting the
tarball content, which is already in use by the kata-deploy local build
scripts.
Fixes: #4808
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The latest kernel with TDX support should be pulled from a different
repo (https://github.com/intel/linux-kernel-dcp, instead of
https://github.com/intel/tdx), and the latest version to be used is
SPR-BKC-PC-v9.6.
With the new version being used, let's make sure we enable the
INTEL_TDX_ATTESTATION config option, and all the dependencies needed to
do so.
Fixes: #4803
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's just re-order the TDX configs alphabetically. No new config has
been added or removed, thus no need to bump the kernel version.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
ps command supprot two formats, `json` and `table`. `json` format just
outputs pids in the container. `table` format will use `ps` utilty in
the host, search and output all processes in the container. Add a struct
`container` to represent a spawned container. Move the `kill`
implemention from kill.rs as a method of `container`.
Fixes: #4361
Signed-off-by: Chen Yiyang <cyyzero@qq.com>
Linux 5.14 supports core scheduling to have better security control
for SMT siblings. This PR supports that.
Fixes: #4429
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
Kata Containers doesn't support host network namespace,
it's a common issue for new users. The limitation
is deleted, this commit will add them back.
Also, Docker has support to run containers using
Kata Containers, delete Docker from not support list.
This commit reverts parts of #3710Fixes: #4794
Signed-off-by: Bin Liu <bin@hyper.sh>
TDVF is the firmware used by QEMU to start TDX capable VMs. Let's start
tracking it as it'll become part of the Confidential Containers sooner
or later.
TDVF lives in the public https://github.com/tianocore/edk2-staging repo
and we're using as its version tags that are consumed internally at
Intel.
Fixes: #4624
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Instead of having as a result the directory where OVMF artefacts where
installed, let's follow what we do with the other components and have a
tarball as a result of the OVMF build.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Instead of cloning the repo, and then switching to a specific branch,
let's take advantage of `--branch` and directly clone the specific
branch / tag.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As TDVF comes from a different repo, the edk2-staging one, we cannot
simply hardcode the name. Instead, let's get the name of the directory
from name of the git repo.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
To keep runtime-rs up to date, we will merge main into runtime-rs every
week.
Fixes: #4790
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
TD-shim is a simplified TDX virtual firmware, used by Cloud Hypervisor,
in order to create a TDX capable VM.
TD-shim is heavily under development, and is hosted as part of the
Confidential Containers project:
https://github.com/confidential-containers/td-shim
The version chosen for this commit, is a version that's being tested
inside Intel, but we, most likely, will need to change it before we have
it officially packaged as part of an official release.
Fixes: #4779
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Bump rtnetlink version from 0.8.0 to 0.11.0. Use rtnetlinks's API to
add neighbors and fix issues to adapt new verson of rtnetlink.
Fixes: #4607
Signed-off-by: Xuewei Niu <justxuewei@apache.org>
To keep runtime-rs up to date, we will merge main into runtime-rs every
week.
Fixes:#4776
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Fixes#4764
versions: update rust version to fix ccv0 attestation-agent build error
static-checks: kata tools, libs, and agent fixes
Signed-Off-By: Ryan Savino <ryan.savino@amd.com>
In some cases do_create_container may return an error, mostly due to
`container.start(process)` call. This commit will do some rollback
works if this function failed.
Fixes: #4749
Signed-off-by: Bin Liu <bin@hyper.sh>
In qemu.StopVM(), if debug is enabled, the shim will dump logs
from qemu.log, but users don't know which logs are from qemu.log
and shim itself. Adding some additional messages will
help users to distinguish these logs.
Fixes: #4745
Signed-off-by: Bin Liu <bin@hyper.sh>
We can log all RPC calls to the agent for debugging purposes
to check which RPC is called, which can help us to understand
the container lifespan.
Fixes: #4738
Signed-off-by: liubin <liubin0329@gmail.com>
when the default_vcpus is greater than the default_maxvcpus, the default
vcpu number should be set equal to the default_maxvcpus.
Fixes: #4712
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
When run with configuration `sandbox_cgroup_only=false`, we will call
`gen_overhead_path()` as the overhead path. The `cgroup-rs` will push
the path with the subsystem prefix by `PathBuf::push()`. When the path
has prefix “/” it will act as root path, such as
```
let mut path = PathBuf::from("/tmp");
path.push("/etc");
assert_eq!(path, PathBuf::from("/etc"));
```
So we shoud not set overhead path with prefix "/".
Fixes: #4687
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
During use, there will be cases where the container is in the stop state
and get another stop. In this case, the second stop needs to be ignored.
Fixes: #4683
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
Update dbs-xxx commit ID for aarch64 in runtime-rs/Cargo.toml file to add
dependencies for aarch64.
Fixes: #4676
Signed-off-by: xuejun-xj <jiyunxue@alibaba.linux.com>
Module anyhow::anyhow is only used on x86_64 architecture in
crates/hypervisor/src/device/vfio.rs file.
Fixes: #4676
Signed-off-by: xuejun-xj <jiyunxue@alibaba.linux.com>
As a built in VMM, Path, jailer path, ctlpath are not needed for
Dragonball. So we don't generate those value in Makefile.
Fixes: #4677
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
New nix is supporting UMOUNT_NOFOLLOW, upgrade nix
version to use this flag instead of the self-defined flag.
Fixes: #4670
Signed-off-by: liubin <liubin0329@gmail.com>
io_uring is a Linux API for asynchronous I/O introduced in qemu 5.0.
It is designed to better performance than older aio API.
We could leverage this in order to get better storage performance.
We should be adding liburing-dev to qemu build to leverage this feature.
However liburing-dev package is not available in ubuntu 20.04,
it is avaiable in 22.04.
Upgrading the ubuntu version in the dockerfile to 22.04 is causing
issues in the static qemu build related to libpmem.
So instead we are building liburing from source until those build issues
are solved.
Fixes: #4645
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
When a container terminated, we should make sure there's no processes
left after destroying the container.
Before this commit, kata-agent depended on the kernel's pidns
to destroy all of the process in a container after the 1 process
exit in a container. This is true for those container using a
separated pidns, but for the case of shared pidns within the
sandbox, the container exit wouldn't trigger the pidns terminated,
and there would be some daemon process left in this container, this
wasn't expected.
Fixes: #4663
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
1. support annotation for runtime.name, hypervisor_name, agent_name.
2. fix parse memory from annotation
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Change get_mounts to get paths from a borrowed argument rather than
calling get_paths a second time.
Fixes#3768
Signed-off-by: Derek Lee <derlee@redhat.com>
Currently $BUILD_DIR will be used to create a directory as:
/opt/kata/share/kata-qemu${BUILD_DIR}
It means that when passing a BUILD_DIR, like "foo", a name would be
built like /opt/kata/share/kata-qemufoo
We should, instead, be building it as /opt/kata/share/kata-qemu-foo.
Fixes: #4638
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Instead of always naming the binary as "-experimental", let's take
advantage of the $BUILD_SUFFIX that's already passed and correctly name
the binary according to it.
Fixes: #4638
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Passing the URL to be used to download the kernel tarball is useful in
various scenarios, mainly when doing a downstream build, thus let's add
this new option.
This new option also works around a known issue of the Dockerfile used
to build the kernel not having `yq` installed.
Fixes: #4629
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
There's no need to have the entire function for building SEV / TDX
duplicated.
Let's remove those functions and create a `get_tee_kernel()` which takes
the TEE as the argument.
Fixes: #4627
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Fix the warning "unused import: `dbs_arch::gic::Error as GICError`" and
"unused import: `dbs_arch::gic::GICDevice`" in file src/vm/mod.rs when
compiling.
Fixes: #4544
Signed-off-by: xuejun-xj <jiyunxue@alibaba.linux.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
As string "com1", "com2" and "rtc" are used in two files
(device_manager/mod.rs and device_manager/legacy.rs), we use public
const variables COM1, COM2 and RTC to replace them respectively.
Fixes: #4544
Signed-off-by: xuejun-xj <jiyunxue@alibaba.linux.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Implement generate_virtio_device_info() and
get_virtio_mmio_device_info() functions su support the mmio_device_info
member, which is used by FDT.
Fixes: #4544
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
Signed-off-by: jingshan <jingshan@linux.alibaba.com>
Instead of passing a `KATA_CONF_FILE` environament variable, let's rely
on the configured (in the container engine) config path, as both
containerd and CRI-O support it, and we're using this for both of them.
Fixes: #4608
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we're already doing for containerd, let's also pass the configuration
path to CRI-O, as all the supported CRI-O versions do support this
configuration option.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
repository_owner check in docs-url-alive-check.yaml now is specified for each step, it can be in job level to save lines.
Fixes: #4611
Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
This is not an issue when the build is run as non-privilged user.
Marking these as safe in case where the build may be run as root
or some other user.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
There is no independent CRI containerd plugin for new containerd,
the related documentation should be updated too.
Fixes: #4605
Signed-off-by: liubin <liubin0329@gmail.com>
remove redundant by_id in get_vm_by_id_mut and get_vm_by_id. They are
optimized to get_vm_mut and get_vm.
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Since cpu topology could tell whether hyper thread is enabled or not, we
removed ht_enabled config from VmConfigInfo
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Change error name from `StartMicrovm` to `StartMicroVm`,
`StartMicrovmError` to `StartMicroVmError`.
Besides, we fix a compile error in config_manager.
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
1. Explain why the current situation is a problem.
2. We are beyond a simple introduction now, it's a real proposal.
3. Explain why you think it is solid, and fix a grammatical error.
4. The Rust rationale does not really belong to the initial paragraph.
Also, I rephrased it to highlight the contrast with Go and the Kata community's
past experience switching to Rust for the agent.
Fixes:#4193
Signed-off-by: Christophe de Dinechin <christophe@dinechin.org>
While running make as non-privileged user, the make errors out with
the following message:
"INFO: Build cloud-hypervisor enabling the following features: tdx
Got permission denied while trying to connect to the Docker daemon
socket at unix:///var/run/docker.sock: Post
"http://%2Fvar%2Frun%2Fdocker.sock/v1.24/images/create?fromImage=cloudhypervisor%2Fdev&tag=20220524-0":
dial unix /var/run/docker.sock: connect: permission denied"
Even though the user may be part of docker group, the clh build from
source does a docker in docker build. It is necessary for the user of
the nested container to be part of docker build for the build to
succeed.
Fixes#4594
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Replaces calls of nproc with nproc with
nproc ${CI:+--ignore 1}
to run nproc with one less processing unit than the maximum to prevent
DOS-ing the local machine.
If process is being run in a container (determined via whether $CI is
null), all processing units avaliable will be used.
Fixes#3967
Signed-off-by: Derek Lee <derlee@redhat.com>
`exec` will execute a command inside a container which exists and is not
frozon or stopped. *Inside* means that the new process share namespaces
and cgroup with the container init process. Command can be specified by
`--process` parameter to read from a file, or from other parameters such
as arg, env, etc. In order to be compatible with `create`/`run`
commands, I refactor libcontainer. `Container` in builder.rs is divided
into `InitContainer` and `ActivatedContainer`. `InitContainer` is used
for `create`/`run` command. It will load spec from given bundle path.
`ActivatedContainer` is used by `exec` command, and will read the
container's status file, which stores the spec and `CreateOpt` for
creating the rustjail::LinuxContainer. Adapt the spec by replacing the
process with given options and updating the namesapces with some paths
to join the container. I also rename the `ContainerContext` as
`ContainerLauncher`, which is only used to spawn process now. It uses
the `LinuxContaier` in rustjail as the runner. For `create`/`run`, the
`launch` method will create a new container and run the first process.
For `exec`, the `launch` method will spawn a process which joins a
container.
Fixes#4363
Signed-off-by: Chen Yiyang <cyyzero@qq.com>
Enable Kata runtime to handle `disable_selinux` flag properly in order
to be able to change the status by the runtime configuration whether the
runtime applies the SELinux label to VMM process.
Fixes: #4599
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
As 2.5.0-rc0 has been released, let's switch the kata-deploy / kata-cleanup
tags back to "latest", and re-add the kata-deploy-stable and the
kata-cleanup-stable files.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
- Drop in cfg files support
- agent: enhance get handled signal
- oci: fix serde skip serializing condition
- agent: Run OCI poststart hooks after a container is launched
- agent: Replace some libc functions with nix ones
- runtime: overwrite mount type to bind for bind mounts
- build: Set safe.directory for runtime repo
- ci/cd: update check-commit-message
- Set safe.directory against tests repository
- runtime: delete Console from Cmd type
- Add `default_maxmemory` config option
- shim: set a non-zero return code if the wait process call failed.
- Refactor how hypervisor config validation is handled
- packaging: Remove unused kata docker configure script
- kata-with-k8s: Add cgroupDriver for containerd
- shim: support shim v2 logging plugin
- device package cleanup/refactor
- versions: Update kernel to latest LTS version 5.15.48
- agent: Allow BUILD_TYPE=debug
- Fix clippy warnings and update agent's vendored code
- block: Leverage multiqueue for virtio-block
- kernel: Add CONFIG_EFI=y as part of the TDX fragments
- runtime: Add heuristic to get the right value(s) for mem-reserve
- runtime: enable sandbox feature on qemu
- snap: fix snap build on ppc64le
- packaging: Remove unused publish kata image script
- rootfs: Fix chronyd.service failing on boot
- tracing: Remove whitespace from root span
- workflow: Removing man-db, workflow kept failing
- docs: Update outdated URLs and keep them available
- runtime: fix error when trying to parse sandbox sizing annotations
- snap: Fix debug cli option
- deps: Resolve dependabot bumps of containerd, crossbeam-utils, regex
- Allow Cloud Hypervisor to run under the `container_kvm_t`
- docs: Update containerd url link
- agent: refactor reading file timing for debugging
- safe-path: fix clippy warning
- kernel building: efi_secret module
- runtime: Switch to using the rust version of virtiofsd (all arches but powerpc)
- shim: change the log level for GetOOMEvent call failures
- docs: Add more kata monitor details
- Allow io.katacontainers.config.hypervisor.enable_iommu annotation by …
- versions: Bump virtiofsd to v1.3.0
- docs: Add storage limits to arch doc
- docs: Update source for cri-tools
- tools: Enable extra detail on error
- docs: Add agent-ctl examples section
f4eea832a release: Adapt kata-deploy for 2.5.0-rc0
0ddb34a38 oci: fix serde skip serializing condition
fbb2e9bce agent: Replace some libc functions with nix ones
acd3302be agent: Run OCI poststart hooks after a container is launched
1f363a386 runtime: overwrite mount type to bind for bind mounts
4e48509ed build: Set safe.directory for runtime repo
48ccd4233 ci: Set safe.directory against tests repository
2a4fbd6d8 agent: enhance get handled signal
433816cca ci/cd: update check-commit-message
a5a25ed13 runtime: delete Console from Cmd type
96553e8bd runtime: Add documentation of drop-in config file fragments
c656457e9 runtime: Add tests of drop-in config file decoding
99f5ca80f runtime: Plug drop-in decoding into decodeConfig()
0f9856c46 runtime: Scan drop-in directory, read files and decode them
2c1efcc69 runtime: Add helpers to copy fields between tomlConfig instances
20f11877b runtime: Add framework to manipulate config structs via reflection
ab5f1c956 shim: set a non-zero return code if the wait process call failed.
e5be5cb08 runtime: device: cleanup outdated comments
5f936f268 virtcontainers: config validation is host specific
323271403 virtcontainers: Remove unused function
0939f5181 config: Expose default_maxmemory
58ff2bd5c clh,qemu: Adapt to using default_maxmemory
1a78c3df2 packaging: Remove unused kata docker configure script
afdc96042 hypervisor: Add default_maxmemory configuration
4e30e11b3 shim: support shim v2 logging plugin
bdf5e5229 virtcontainers: validate hypervisor config outside of hypervisor itself
469e09854 katautils: don't do validation when loading hypervisor config
e32bf5331 device: deduplicate state structures
f97d9b45c runtime: device/persist: drop persist dependency from device pkgs
f9e96c650 runtime: device: move to top level package
3880e0c07 agent: refactor reading file timing for debugging
c70d3a2c3 agent: Update the dependencies
612fd79ba random: Fix "nonminimal-bool" clippy warning
d4417f210 netlink: Fix "or-fun-call" clippy warnings
93874cb3b packaging: Restrict kernel patches applied to top-level dir
07b1367c2 versions: Update kernel to latest LTS version 5.15.48
1b7d36fdb agent: Allow BUILD_TYPE=debug
9ff10c083 kernel: Add CONFIG_EFI=y as part of the TDX fragments
e227b4c40 block: Leverage multiqueue for virtio-block
e7e7dc9df runtime: Add heuristic to get the right value(s) for mem-reserve
c7dd10e5e packaging: Remove unused publish kata image script
0bbbe7068 snap: fix snap build on ppc64le
ef925d40c runtime: enable sandbox feature on qemu
28995301b tracing: Remove whitespace from root span
9941588c0 workflow: Removing man-db, workflow kept failing
90a7763ac snap: Fix debug cli option
a305bafee docs: Update outdated URLs and keep them available
bee770343 docs: Update containerd url link
ac5dbd859 clh: Improve logging related to the net dev addition
0b75522e1 network: Set queues to 1 to ensure we get the network fds
93b61e0f0 network: Add FFI_NO_PI to the netlink flags
bf3ddc125 clh: Pass the tuntap fds down to Cloud Hypervisor
55ed32e92 clh: Take care of the VmAdNetdPut request ourselves
01fe09a4e clh: Hotplug the network devices
2e0753833 clh: Expose VmAddNetPut
1ef0b7ded runtime: Switch to using the rust version of virtiofsd (all but power)
bb26bd73b safe-path: fix clippy warning
1a5ba31cb agent: refactor reading file timing for debugging
721ca72a6 runtime: fix error when trying to parse sandbox sizing annotations
9773838c0 virtiofsd: export env vars needed for building it
b0e090f40 versions: Bump virtiofsd to v1.3.0
db5048d52 kernel: build efi_secret module for SEV
1b845978f docs: Add storage limits to arch doc
412441308 docs: Add more kata monitor details
eff4e1017 shim: change the log level for GetOOMEvent call failures
5d7fb7b7b build(deps): bump github.com/containerd/containerd in /src/runtime
d0ca2fcbb build(deps): bump crossbeam-utils in /src/tools/trace-forwarder
a60dcff4d build(deps): bump regex from 1.5.4 to 1.5.6 in /src/tools/agent-ctl
dbf50672e build(deps): bump crossbeam-utils in /src/tools/agent-ctl
8e2847bd5 build(deps): bump crossbeam-utils from 0.8.6 to 0.8.8 in /src/libs
e9ada165f build(deps): bump regex from 1.5.4 to 1.5.5 in /src/agent
adad9cef1 build(deps): bump crossbeam-utils from 0.8.5 to 0.8.8 in /src/agent
34bcef884 docs: Add agent-ctl examples section
815157bf0 docs: Remove erroneous whitespace
f5099620f tools: Enable extra detail on error
8f10e13e0 config: Allow enable_iommu pod annotation by default
7ae11cad6 docs: Update source for cri-tools
0e2459d13 docs: Add cgroupDriver for containerd
1b7fd19ac rootfs: Fix chronyd.service failing on boot
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
kata-deploy files must be adapted to a new release. The cases where it
happens are when the release goes from -> to:
* main -> stable:
* kata-deploy-stable / kata-cleanup-stable: are removed
* stable -> stable:
* kata-deploy / kata-cleanup: bump the release to the new one.
There are no changes when doing an alpha release, as the files on the
"main" branch always point to the "latest" and "stable" tags.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Replace `libc::setgroups()`, `libc::fchown()`, and `libc::sethostname()`
functions with nix crate ones for safety and maintainability.
Fixes: #4579
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Run the OCI `poststart` hooks must be called after the
user-specified process is executed but before the `start`
operation returns in accordance with OCI runtime spec.
Fixes: #4575
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Some clients like nerdctl may pass mount type of none for volumes/bind mounts,
this will lead to container start fails.
Referring to runc, it overwrites the mount type to bind and ignores the input value.
Fixes: #4548
Signed-off-by: liubin <liubin0329@gmail.com>
While doing a docker build for shim-v2, we see this:
```
fatal: unsafe repository
('/home/${user}/go/src/github.com/kata-containers/kata-containers' is
owned by someone else)
To add an exception for this directory, call:
git config --global --add safe.directory
/home/${user}/go/src/github.com/kata-containers/kata-containers
```
This is because the docker container build is run as root while the
runtime repo is checked out as normal user.
Unlike this error causing the rootfs build to error out, the error here
does not really cause `make shim-v2-tarball` to fail.
However its good to get rid of this error message showing during the
make process.
Fixes: #4572
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Set `safe.directory` against `kata-containers/tests` repository
before checkout because the user in the docker container is root,
but the `tests` repository on the host machine is usually owned
by the normal user.
This works when we already have the `tests` repository which is
not owned by root on the host machine and try to create a rootfs
using Docker (`USE_DOCKER=true`).
Fixes: #4561
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
For runC, send the signal to the init process directly.
For kata, we try to send `SIGKILL` instead of `SIGTERM` when the process
has not installed the handler for `SIGTERM`.
The `is_signal_handled` function determine which signal the container
process has been handled. But currently `is_signal_handled` is only
catching (SigCgt). While the container process is ignoring (SigIgn) or
blocking (SigBlk) also should not be converted from the `SIGTERM` to
`SIGKILL`. For example, when using terminationGracePeriodSeconds the k8s
will send SIGTERM first and then send `SIGKILL`, in this case, the
container ignores the `SIGTERM`, so we should send the `SIGTERM` not the
`SIGKILL` to the container.
Fixes: #4478
Signed-off-by: quanweiZhou <quanweiZhou@linux.alibaba.com>
Recently added check-commit-message to the tests repository. Minor
changes were also made to action. For consistency's sake, copied changes
over to here as well.
tests - https://github.com/kata-containers/tests/pull/4878
Minor Changes:
1. Body length check is now 75 and consistent with guidelines
2. Lines without spaces are not counted in body length check
Fixes#4559
Signed-off-by: Derek Lee <derlee@redhat.com>
The tests ensure that interactions between drop-ins and the base
configuration.toml and among drop-ins themselves work as intended,
basically that files are evaluated in the correct order (base file
first, then drop-ins in alphabetical order) and the last one to set
a specific key wins.
Signed-off-by: Pavel Mores <pmores@redhat.com>
updateFromDropIn() uses the infrastructure built by previous commits to
ensure no contents of 'tomlConfig' are lost during decoding. To do
this, we preserve the current contents of our tomlConfig in a clone and
decode a drop-in into the original. At this point, the original
instance is updated but its Agent and/or Hypervisor fields are
potentially damaged.
To merge, we update the clone's Agent/Hypervisor from the original
instance. Now the clone has the desired Agent/Hypervisor and the
original instance has the rest, so to finish, we just need to move the
clone's Agent/Hypervisor to the original.
Signed-off-by: Pavel Mores <pmores@redhat.com>
These functions take a TOML key - an array of individual components,
e.g. ["agent" "kata" "enable_tracing"], as returned by BurntSushi - and
two 'tomlConfig' instances. They copy the value of the struct field
identified by the key from the source instance to the target one if
necessary.
This is only done if the TOML key points to structures stored in
maps by 'tomlConfig', i.e. 'hypervisor' and 'agent'. Nothing needs to
be done in other cases.
Signed-off-by: Pavel Mores <pmores@redhat.com>
For 'tomlConfig' substructures stored in Golang maps - 'hypervisor' and
'agent' - BurntSushi doesn't preserve their previous contents as it does
for substructures stored directly (e.g. 'runtime'). We use reflection
to work around this.
This commit adds three primitive operations to work with struct fields
identified by their `toml:"..."` tags - one to get a field value, one to
set a field value and one to assign a source struct field value to the
corresponding field of a target.
Signed-off-by: Pavel Mores <pmores@redhat.com>
Return code is an int32 type, so if an error occurred, the default value
may be zero, this value will be created as a normal exit code.
Set return code to 255 will let the caller(for example Kubernetes) know
that there are some problems with the pod/container.
Fixes: #4419
Signed-off-by: liubin <liubin0329@gmail.com>
Prior device config move didn't update the comments. Let's address this,
and make sure comments match the new path...
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Ideally this config validation would be in a seperate package
(katautils?), but that would introduce circular dependency since we'd
call it from vc, and it depends on vc types (which, shouldn't be vc, but
probably a hypervisor package instead).
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
While working on the previous commits, some of the functions become
non-used. Let's simply remove them.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Expose the newly added `default_maxmemory` to the project's Makefile and
to the configuration files.
Fixes: #4516
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's adapt Cloud Hypervisor's and QEMU's code to properly behave to the
newly added `default_maxmemory` config.
While implementing this, a change of behaviour (or a bug fix, depending
on how you see it) has been introduced as if a pod requests more memory
than the amount avaiable in the host, instead of failing to start the
pod, we simply hotplug the maximum amount of memory available, mimicing
better the runc behaviour.
Fixes: #4516
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR removes an unused kata configure docker script which was used
in packaging for kata 1.x but not longer being used in kata 2.x
Fixes#4546
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's add a `default_maxmemory` configuration, which allows the admins
to set the maximum amount of memory to be used by a VM, considering the
initial amount + whatever ends up being hotplugged via the pod limits.
By default this value is 0 (zero), and it means that the whole physical
RAM is the limit.
Fixes: #4516
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Now kata shim only supports stdout/stderr of fifo from
containerd/CRI-O, but shim v2 supports logging plugins,
and nerdctl default will use the binary schema for logs.
This commit will add the others type of log plugins:
- file
- binary
In case of binary, kata shim will receive a stdout/stderr like:
binary:///nerdctl?_NERDCTL_INTERNAL_LOGGING=/var/lib/nerdctl/1935db59
That means the nerdctl process will handle the logs(stdout/stderr)
Fixes: #4420
Signed-off-by: Bin Liu <bin@hyper.sh>
Depending on the user of it, the hypervisor from hypervisor interface
could have differing view on what is valid or not. To help decouple,
let's instead check the hypervisor config validity as part of the
sandbox creation, rather than as part of the CreateVM call within the
hypervisor interface implementation.
Fixes: #4251
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Policy for whats valid/invalid within the config varies by VMM, host,
and by silicon architecture. Let's keep katautils simple for just
translating a toml to the hypervisor config structure, and leave
validation to virtcontainers.
Without this change, we're doing duplicate validation.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
In order to support ACPI hotplug in the future with the cooperative work
from the Kata community, we add ACPI feature and dbs-upcall feature to
add room for ACPI hotplug.
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Before, we maintained almost identical structures between our persist
API and what we keep for our devices, with the persist API being a
slight subset of device structures.
Let's deduplicate this, now that persist is importing device package.
Json unmarshal of prior persist structure will work fine, since it was
an exact subset of fields.
Fixes: #4468
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Rather than have device package depend on persist, let's define the
(almost duplicate) structures within device itself, and have the Kata
Container's persist pkg import these.
This'll help avoid unecessary dependencies within our core packages.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
In the original code, reads mountstats file and return
the content in the error, but at this time the file maybe
changed, we should return the file content that parsed
line by line to check why there is not a fstype option.
Fixes: #4246
Signed-off-by: Bin Liu <bin@hyper.sh>
Let's run a `cargo update` and ensure the deps are up-to-date before we
cut the "-rc0" release.
Fixes: #4525
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The error shown below was caught during a dependency bump in the CCv0
branch, but we better fix it here first.
```
error: this boolean expression can be simplified
--> src/random.rs:85:21
|
85 | assert!(!ret.is_ok());
| ^^^^^^^^^^^^ help: try: `ret.is_err()`
|
= note: `-D clippy::nonminimal-bool` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#nonminimal_bool
error: this boolean expression can be simplified
--> src/random.rs:93:17
|
93 | assert!(!ret.is_ok());
| ^^^^^^^^^^^^ help: try: `ret.is_err()`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#nonminimal_bool
```
Fixes: #4523
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The error shown below was caught during a dependency bump in the CCv0
branch, but we better fix it here first.
```
error: use of `ok_or` followed by a function call
--> src/netlink.rs:526:14
|
526 | .ok_or(anyhow!(nix::Error::EINVAL))?;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try this: `ok_or_else(|| anyhow!(nix::Error::EINVAL))`
|
= note: `-D clippy::or-fun-call` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#or_fun_call
error: use of `ok_or` followed by a function call
--> src/netlink.rs:615:49
|
615 | let v = u8::from_str_radix(split.next().ok_or(anyhow!(nix::Error::EINVAL))?, 16)?;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try this: `ok_or_else(|| anyhow!(nix::Error::EINVAL))`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#or_fun_call
```
Fixes: #4523
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The apply_patches.sh script applies all patches in the patches
directory, as well as subdirectories. This means if there is a sub-dir
called "experimental" under a major kernel version directory,
experimental patches would be applied to the default kernel supported by
Kata.
We did not come accross this issue earlier as typically the experimental
kernel version was different from the default kernel.
With both the default kernel and the arm-experimental kernel having the
same major kernel version (5.15.x) at this time, trying to update the
kernel patch version revealed that arm-experimental patches were being
applied to the default kernel.
Restricting the patches to be applied to the top level directory will
solve the issue. The apply_patches script should ignore any
sub-directories meant for experimental patches.
Fixes#4520
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This brings in a few security fixes.
Removing arm patches related to virtio-mem that are no longer required
as they have been merged.
Fixes#4438
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
The cargo command creates debug build binaries, when the --release
option is not specified. Specifying --debug option causes an error.
This patch specifies --release option when BUILD_TYPE=release,
and does not specify any build type option when BUILD_TYPE=debug.
Fixes#4504
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
Otherwise `./build-kernel.sh -x tdx setup` will fail with the following
error:
```
$ ./build-kernel.sh -x tdx setup
INFO: Config version: 92
INFO: Kernel version: tdx-guest-v5.15-4
INFO: kernel path does not exist, will download kernel
INFO: Apply patches from
/home/ffidenci/go/src/github.com/kata-containers/kata-containers/tools/packaging/kernel/patches/tdx-guest-v5.15-4.x
INFO: Found 0 patches
INFO: Enabling config for 'tdx' confidential guest protection
INFO: Constructing config from fragments:
/home/ffidenci/go/src/github.com/kata-containers/kata-containers/tools/packaging/kernel/configs/fragments/x86_64/.config
WARNING: unmet direct dependencies detected for UNACCEPTED_MEMORY
Depends on [n]: EFI [=n] && EFI_STUB [=n]
Selected by [y]:
- INTEL_TDX_GUEST [=y] && HYPERVISOR_GUEST [=y] && X86_64 [=y] &&
CPU_SUP_INTEL [=y] && PARAVIRT [=y] && SECURITY [=y] &&
X86_X2APIC[=y]
INFO: Some CONFIG elements failed to make the final .config:
INFO: Value requested for CONFIG_EFI_STUB not in final .config
INFO: Generated config file can be found in
/home/ffidenci/go/src/github.com/kata-containers/kata-containers/tools/packaging/kernel/configs/fragments/x86_64/.config
ERROR: Failed to construct requested .config file
ERROR: failed to find default config
```
Fixes: #4510
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Similar to network, we can use multiple queues for virtio-block
devices. This can help improve storage performance.
This commit changes the number of queues for block devices to
the number of cpus for cloud-hypervisor and qemu.
Today the default number of cpus a VM starts with is 1.
Hence the queues used will be 1. This change will help
improve performance when the default cold-plugged cpus is greater
than one by changing this in the config file. This may also help
when we use the sandboxing feature with k8s that passes down
the sum of the resources required down to Kata.
Fixes#4502
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This PR removes unused the publish kata image script which
was used on kata 1.x when we had OBS packages which are not
longer used on kata 2.x
Fixes#4496
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Enable "-sandbox on" in qemu can introduce another protect layer
on the host, to make the secure container more secure.
The default option is disable because this feature may introduce some
performance cost, even though user can enable
/proc/sys/net/core/bpf_jit_enable to reduce the impact.
Fixes: #2266
Signed-off-by: Feng Wang <feng.wang@databricks.com>
Remove space from root span name to follow camel casing of other tracing
span names in the runtime and to make parsing easier in testing.
Fixes#4483
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
`snap`/`snapcraft` seems to have changed recently. Since `snap`
auto-updates all `snap` packages and since we use the `snapcraft` `snap`
for building snaps, this is impacting all our CI jobs which now show:
```
Installing Snapcraft for Linux…
snapcraft 7.0.4 from Canonical* installed
Run snapcraft -d snap --destructive-mode
Usage: snapcraft [options] command [args]...
Try 'snapcraft pack -h' for help.
Error: unrecognized arguments: -d
Error: Process completed with exit code 1.
```
Move the debug option to make it a sub-command (long) option to resolve
this issue.
Fixes: #4457.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
By comparing the content of the old url and the new url,
ensure that their content is consistent and does not contain ambiguities
Fixes: #4454
Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
Let's improve the log so we make it clear that we're only *actually*
adding the net device to the Cloud Hypervisor configuration when calling
our own version of VmAddNetPut().
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We want to have the file descriptors of the opened tuntap device to pass
them down to the VMMs, so the VMMs don't have to explicitly open a new
tuntap device themselves, as the `container_kvm_t` label does not allow
such a thing.
With this change we ensure that what's currently done when using QEMU as
the hypervisor, can be easily replicated with other VMMs, even if they
don't support multiqueue.
As a side effect of this, we need to close the received file descriptors
in the code of the VMMs which are not going to use them.
Fixes: #3533
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Adding FFI_NO_PI to the netlink flags causes no harm to the supported
and tested hypervisors as when opening the device by its name Cloud
Hypervisor[0], Firecracker[1], and QEMU[2] do set the flag already.
However, when receiving the file descriptor of an opened tutap device
Cloud Hypervisor is not able to set the flag, leaving the guest without
connectivity.
To avoid such an issue, let's simply add the FFI_NO_PI flag to the
netlink flags and ensure, from our side, that the VMMs don't have to set
it on their side when dealing with an already opened tuntap device.
Note that there's a PR opened[3] just for testing that this change
doesn't cause any breakage.
[0]: e52175c2ab/net_util/src/tap.rs (L129)
[1]: b6d6f71213/src/devices/src/virtio/net/tap.rs (L126)
[2]: 3757b0d08b/net/tap-linux.c (L54)
[3]: https://github.com/kata-containers/kata-containers/pull/4292
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is basically a no-op right now, as:
* netPair.TapInterface.VMFds is nil
* the tap name is still passed to Cloud Hypervisor, which is the Cloud
Hypervisor's first choice when opening a tap device.
In the very near future we'll stop passing the tap name to Cloud
Hypervisor, and start passing the file descriptors of the opened tap
instead.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Knowing that VmAddNetPut works as expected, let's switch to manually
building the request and writing it to the appropriate socket.
By doing this it gives us more flexibility to, later on, pass the file
descriptor of the tuntap device to Cloud Hypervisor, as openAPI doesn't
support such operation (it has no notion of SCM Rights).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Instead of creating the VM with the network device already plugged in,
let's actually add the network device *after* the VM is created, but
*before* the Vm is actually booted.
Although it looks like it doesn't make any functional difference between
what's done in the past and what this commit introduces, this will be
used to workaround a limitation on OpenAPI when it comes to passing down
the network device's file descriptor to Cloud Hypervisor, so Cloud
Hypervisor can use it instead of opening the device by its name on the
VMM side.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
VmAddNetPut is the API provided by the Cloud Hypervisor client (auto
generated) code to hotplug a new network device to the VM.
Let's expose it now as it'll be used as part this series, mostly to
guide the reviewer through the process of what we have to do, as later
on, spoiler alert, it'll end up being removed.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
So far this has been done for x86_64. Now that the support for building
and testing has been added for all arches, let's do the second part of
the switch.
We're still not done yet for powerpc, as some a virtifosd crash on the
rust version has been found by the maintainer.
Fixes: #4258, #4260
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In the original code, reads mountstats file and return
the content in the error, but at this time the file maybe
changed, we should return the file content that parsed
line by line to check why there is not a fstype option.
Fixes: #4246
Signed-off-by: Bin Liu <bin@hyper.sh>
Changed bitsize for parsing functions to 64-bit in order to avoid
parsing errors.
Fixes#4435
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Revert this patch, after dragonball-sandbox is ready. And all
subsequent implementations are submitted.
Fixes: #4257
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Update Dragonball Readme to fix style problem and add github issue for
TODOs.
Add document for devices in dragonball. This is the document for the
current dragonball device status and we'll keep updating it when we
introduce more devices in later pull requets.
Fixes: #4257
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Device manager to manage IO devices for a virtual machine. And added
DeviceManagerTx to provide operation transaction for device management,
added DeviceManagerContext to operation context for device management.
Fixes: #4257
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Console manager to manage frontend and backend console devcies.
A virtual console are composed up of two parts: frontend in virtual
machine and backend in host OS. A frontend may be serial port,
virtio-console etc, a backend may be stdio or Unix domain socket. The
manager connects the frontend with the backend.
Fixes: #4257
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
1. support oom event
2. use ContainerProcess to store container_id and exec_id
3. support stats
Fixes: #3785
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
1. service: Responsible for processing services, such as task service, image service
2. Responsible for implementing different runtimes, such as Virt-container,
Linux-container, Wasm-container
Fixes: #3785
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
1. support async.
2. update ttrpc and protobuf
update ttrpc to 0.6.0
update protobuf to 2.23.0
3. support trans from oci
Fixes: #3746
Signed-off-by: Quanwei Zhou <quanweiZhou@linux.alibaba.com>
Implement reflink_copy() to copy file by reflink, and fallback to normal
file copy.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
Changes since v1.2.0:
!123 Update rust-vmm dependencies (main) ← (update-deps)
!121 implement std::error::Error trait (main) ← (fix-impl-error)
!120 Show the nofile hard limit value in the warning me... (main) ← (fix-rlimit-warn)
!119 Do not create tmpdir and bind mount /proc/self/fd ... (main) ← (remove-tmp-dir-for-proc)
!116 Disable killpriv_v2 by default (main) ← (no-killpriv-default)
The one that affected Kata Containers the most was !119, as virtiofsd
would get denied when SELinux was set to run on enforcing mode.
Fixes: #4433
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The kata-sys-util crate is a collection of modules that provides helpers
and utilities used by multiple Kata Containers components.
Fixes: #3305
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
1. modify default values for hypervisor
2. change the variable name
3. check the min memory limit
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
1. Some Nit problems are fixed
2. Make the code more readable
3. Modify some implementation details
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
If there is a parse error when we are trying to get the annotations, we
will return Result<Option<type>> to handle that.
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
loading from empty string is only used to identity that the config is
not initialized yet, so Option<TomlConfig> is a better option
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Some annotations are used to override hypervisor configurations, and you
know it's dangerous. We must be careful when overriding hypervisor configuration
by annotations, to avoid security flaws.
There are two existing mechanisms to prevent attacks by annotations:
1) config.hypervisor.enable_annotations defines the allowed annotation
keys for config.hypervisor.
2) config.hyperisor.xxxx_paths defines allowd values for specific keys.
The access methods for config.hypervisor.xxx enforces the permisstion
checks for above rules.
To update conifg, traverse the annotation hashmap,check if the key is enabled in hypervisor or not.
If it is enabled. For path related annotation, check whether it is valid or not
before updating conifg. For cpu and memory related annotation, check whether it
is more than or less than the limitation for DB and qemu beforing updating config.
If it is not enabled, there will be three possibilities, agent related
annotation, runtime related annotation and hypervisor related annotation
but not enabled. The function will handle agent and runtime annotation
first, then the option left will be the invlaid hypervisor, err message
will be returned.
add more edge cases tests for updating config
clean up unused functions, delete unused files and fix warnings
Fixes: #3523
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
The TomlConfig structure is a parsed form of Kata configuration file,
but it's a little inconveneient to access those configuration
information directly. So introduce a wrapper KataConfig to easily
access those configuration information.
Two singletons of KataConfig is provided:
- KATA_DEFAULT_CONFIG: the original version directly loaded from Kata
configuration file.
- KATA_ACTIVE_CONFIG: the active version is the KATA_DEFAULT_CONFIG
patched by annotations.
So the recommended to way to use these two singletons:
- Load TomlConfig from configuration file and set it as the default one.
- Clone the default one and patch it with values from annotations.
- Use the default one for permission checks, such as to check for
allowed annotation keys/values.
- The patched version may be set as the active one or passed to clients.
- The clients directly accesses information from the active/passed one,
and do not need to check annotation for override.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Add structures to load Kata agent configuration from configuration files.
Also define a mechanism for vendor to extend the Kata configuration
structure.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Add structures to load Kata hypevisor configuration from configuration
files. Also define a mechanisms to:
1) for hypervisors to handle the configuration info.
2) for vendor to extend the Kata configuration structure.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Add structures to load Kata runtime configuration from configuration
files. Also define a mechanism for vendor to extend the Kata
configuration structure.
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Add FileRotator to rotate log files.
The FileRotator structure may be used as writer for create_logger()
and limits the storage space occupied by log files.
Fixes: #3304
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Wei Yang <wei.yang1@linux.alibaba.com>
Signed-off-by: yanlei <yl.on.the.way@gmail.com>
Convert libs into a Cargo workspace, so all libraries could share the
build infrastructure.
Fixes#3282
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Some generated merge commit messages are >75 chars
Allow these to not trigger the subject line length failure
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Add kernel fork for sev to kernel builder with efi_secret. Additionally, install efi_secret module for sev.
Fixes: #4179
Signed-off-by: Alex Carter <alex.carter@ibm.com>
this will require to set a PR number when triggering the test-kata-deploy workflow manually
also make sure user variables are set correctly when workflow_dispatch is used
Fixes: #4349
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
Updated the architecture document to explain that if you wish to
constrain the amount of disk space a container uses, you need to use an
existing facility such as `quota(1)`s or device mapper limits.
Fixes: #4430.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add more detail to the `kata-monitor` doc to allow an admin to make a
more informed decision about where and how to run the daemon.
Fixes: #4416.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
GetOOMEvent is a blocking call that will fail if
the container exit, in this case, it's not an error or warning.
Changing the log level for logs in case of GetOOMEvent call fails
will reduce log noise in a large cluster that has pods
creating/deleting frequently.
Fixes: #4376
Signed-off-by: Bin Liu <bin@hyper.sh>
- docs: Update storage documentation link
- rustjail: get home dir using nix crate
- runk: Support `list` sub-command
- docs: Update vGPU use-case
- runtime: ignore ESRCH error from stop container
- docs: Update configuration reference for snap documentation
- workflows: add workflow_dispatch triggering to test-kata-deploy
- snap: Use helper script and cleanup
- feature: add ability to interact with IPTables within the guest
- agent: return mount file content if parse mountinfo failed
- docs: Update Intel QAT documentation links
- osbuilder: add iptables package
- runk: Return error when tty is used without console socket
- runk: Add Podman guide in README
- agent: Pass standard I/O to container launched by runk
- agent, runk: Enable test for the agent built with standard-oci-runtime feature
- runk: Handle rootfs path in config.json properly
- Update containerd docs
- clh: Update to v24.0
- snap: Build and package rust version of virtiofsd
- runk: merge oci-kata-agent into runk
- virtiofsd: static build virtiofsd from rust code for non-x86
- Fix issues with direct-volume stats feature
- runtime: fix incorrect Action function for direct-volume stats
- runtime: Adding the correct detection of mediated PCIe devices
- runtime: remove duplicate 'types' import
- runtime: sync docstrings with function names
- qemu: allow using legacy serial device for the console
- docs: Remove clear containers reference in README
- runtime: do not check for EOF error in console watcher
- kernel: Remove nemu.conf from packaging
- tools: delete unused param from get_from_kata_deps callers
- agent: Fix is_signal_handled failing parsing str to u64
- Improve Go unit test script
- packaging: Add kernel config option for SGX in Gramine
- ci: Don't run Docs URL Alive Check workflow on forks
- tools: Add QEMU patches for SGX numa support
- docs: Update runc containerd runtime
- Build and distribute the rust version of virtiofsd
- doc: Update log parser link
- Move the kata-log-parser from the tests repo
- versions: Upgrade to Cloud Hypervisor v23.1
- agent: Add a macro to skip a loop easier
- runk: use custom Kill command to support --all option
- agent: add test coverage for functions find_process and online_resources
fe3c1d9cd docs: Update storage documentation link
9d27c1fce agent: ignore ESRCH error when destroying containers
9726f56fd runtime: force stop container after the container process exits
168f325c4 docs: Update configuration reference for snap documentation
38a318820 runk: Support `list` sub-command
b9fc24ff3 docs: update release process github token instructions
c1476a174 docs: update release process with latest workflow triggering
002f2cd10 snap: Use helper script and cleanup
2e04833fb docs: Update Intel QAT documentation links
8b57bf97a workflows: add workflow_dispatch triggering to test-kata-deploy
6d0ff901a docs: Update vGPU use-case
9b108d993 docs: Improve snap formatting
894f661cc docs: Add warning to snap build
d759f6c3e snap: Fix CH architecture check
590381574 agent: Pass standard I/O to container launched by runk
af2ef3f7a agent-ctl: introduce handle for iptables get/set
65f0cef16 kata-runtime: add iptables CLI to test http endpoint
3201ad083 shim-client: ensure we check resp status for Put/Post
0706fb28a kata-runtime: shmgmt: make url usage consistent
2a09378dd shim-client: add support for DoPut
640173cfc shim-mgmt: Add endpoint handler for interacting with iptables
0136be22c virtcontainers: plumb iptable set/get from sandbox to agent
bd50d463b agent: iptables: get/set handling for iptables
7c4049aab osbuilder: add iptables package
03176a9e0 proto: update generated code based on proto update
38ebbc705 proto: update to add set/get iptables
78d45b434 agent: return mount file content if parse mountinfo failed
c7b3941c9 runk: Enable test for the agent built with standard-oci-runtime feature
6dbce7c3d agent: Remove unused import in console test
6ecea84bc rustjail: get home dir using nix crate
648b8d0ae runk: Return error when tty is used without console socket
5205efd9b runk: Add Podman guide in README
d862ca059 runk: Handle rootfs path in config.json properly
56591804b docs: Improve snap build instructions
cb2b30970 snap: Build using destructive mode
60823abb9 docs: Move snap README
fff832874 clh: Update to v24.0
49361749e snap: Build and package rust version of virtiofsd
27d903b76 snap: Put the yq binary in the staging bin directory
d7b4ce049 snap: Remove unused variable
43de5440e snap: Fix unbound variable error
c9b291509 snap: Fix whitespace
122a85e22 agent: remove bin oci-kata-agent
35619b45a runk: merge oci-kata-agent into runk
10c13d719 qemu: remove virtiofsd option in qemu config
d20bc5a4d virtiofsd: build rust based virtiofsd from source for non-x86_64
c95ba63c0 docs: Remove information related to Kata 1.x
34b80382b docs: Get rid of note related to networking.
dfad5728a docs: Mention --cni flag while invoking ctr
8e7c5975c agent: fix direct-assigned volume stats
4428ceae1 runtime: direct-volume stats use correct name
ffdc065b4 runtime: direct-volume stats update to use GET parameter
f29595318 runtime: fix incorrect Action function for direct-volume stats
7a5ccd126 runtime: sync docstrings with function names
ce2e521a0 runtime: remove duplicate 'types' import
834f93ce8 docs: fix annotations example
f4994e486 runtime: allow annotation configuration to use_legacy_serial
24a2b0f6a docs: Remove clear containers reference in README
abad33eba kernel: Remove nemu.conf from packaging
e87eb13c4 tools: delete unused param from get_from_kata_deps callers
8052fe62f runtime: do not check for EOF error in console watcher
c67b9d297 qemu: allow using legacy serial device for the console
44814dce1 qemu: treat console kernel params within appendConsole
4f586d2a9 packaging: Add kernel config option for SGX in Gramine
4b437d91f agent: Fix is_signal_handled failing parsing str to u64
88fb9b72e docs: Update runc containerd runtime
d1f2852d8 tools: Stop building virtiofsd with qemu (for x86_64)
c39852e83 runtime: Use ${LIBEXEC}/virtiofsd as the default virtiofsd path
b4b9068cb tools: Add QEMU patches for SGX numa support
a475956ab workflows: Add support for building virtiofsd
71f59f3a7 local-build: Add support for building virtiofsd
c7ac55b6d dockerbuild: Install unzip
8e2042d05 tools: add script to pull virtiofsd
dbedea508 versions: Add virtiofsd entry
e73b70baf runtime: Don't run unit tests verbose by default
f24a6e761 runtime: Consolidate flags setting in unit tests script
cf465feb0 runtime: Don't change test behaviour based on $CI or $KATA_DEV_MODE
34c4ac599 runtime: Remove redundant subcommands from go-test.sh
0aff5aaa3 runtime: Simplify package listing in go-test.sh
557c4cfd0 runtime: Don't chmod coverage files in Go tests
04c8b52e0 runtime: Remove HTML coverage option from go-test.sh
7f7691442 runtime: Add coverage.txt.tmp to gitignore
13c257700 runtime: Move go testing script locally
421064680 doc: Update log parser link
271933fec log-parser: fix some of the documentation
c7dacb121 log-parser: move the kata-log-parser from the tests repo
82ea01828 versions: Upgrade to Cloud Hypervisor v23.1
2a1d39414 runtime: Adding the correct detection of mediated PCIe devices
7bc4ab68c ci: Don't run Docs URL Alive Check workflow on forks
475e3bf38 agent: add test coverage for functions find_process and online_resources
383be2203 agent: Add a macro to skip a loop easier
97d7b1845 runk: use custom Kill command to support --all option
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
Add a new `Examples` section to the `agent-ctl` docs giving some
examples of how to use the tool with QEMU and stand-alone.
Fixes: #4414.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The `agent-ctl` and `trace-forwarder` tools make use of
`anyhow::Context` to provide additional call site information on error.
However, previously neither tool was using the "alternate debug" format
to display the error, meaning full error output was not displayed.
Fixes: #4411.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This PR updates the storage documentation link for the devicemapper
snapshotter.
Fixes#4398
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Since #902 the `io.katacontainers.config.hypervisor` pod annotations
have only been permitted if explicitly allowed in the global
configuration. The default global configuration allows no such
annotations. That's important because several of those annotations
would cause Kata to execute arbitrary binaries, and so were wildly
unsafe.
However, this is inconvenient for the
`io.katacontainers.config.hypervisor.enable_iommu` annotation
specifically, which controls whether the sandbox VM includes a vIOMMU.
A guest side vIOMMU is necessary to implement VFIO passthrough devices
with `vfio_mode = vfio`, so enabling that mode of operation currently
requires a global configuration change, and can't just be enabled
per-pod.
Unlike some of the other hypervisor annotations, the `enable_iommu`
annotation is quite safe. By default the vIOMMU is not present, so
allowing a user to override it for a pod only improves their
facilities for isolation. Even if the global default were changed to
enable the vIOMMU, that doesn't compel the guest kernel to use it, so
allowing a user to disable the vIOMMU doesn't materially affect
isolation either.
Therefore, allow the io.katacontainers.config.hypervisor.enable_iommu
annotation to work in the default configurations.
fixes#4330
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
destroy() method should ignore the ESRCH error from signal::kill
and continue the operation as ESRCH is often considered harmless.
Fixes: #4359
Signed-off-by: Feng Wang <feng.wang@databricks.com>
Set thestop container force flag to true so that the container state is always set to
“StateStopped” after the container wait goroutine is finished. This is necessary for
the following delete container step to succeed.
Fixes: #4359
Signed-off-by: Feng Wang <feng.wang@databricks.com>
This PR updates the url link for the kata containers configuration
for the general snap documentation.
Fixes#4341
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Support list sub-command. It will traverse the root directory, parse
status file and print basic information of containers. Behavior and
print format consistent with runc. To handle race with runk delete
or system user modify, the loop will continue to traverse when errors
are encountered.
Fixes: #4362
Signed-off-by: Chen Yiyang <cyyzero@qq.com>
Move the common shell code to a helper script that is sourced by all
parts.
Add extra quoting to some variables in the snap config file
and simplify.
Fixes: #4304.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Now that #4213 is merged we need updated documentation for vGPU time-sliced or vGPU MIG-backed.
Fixes: #4343
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Improve the snap docs by using more consistent formatting and proper
shell code in the shell example.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Since we must build with `--destructive-mode`, add a warning that the
host environment could change the behaviour of the build, depending on
the packages installed on the system.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The `kata-agent` passes its standard I/O file descriptors
through to the container process that will be launched
by `runk` without manipulation or modification in order to
allow the container process can handle its I/O operations.
Fixes: #4327
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
In linux 5.14 and hopefully some backports, core scheduling allows processes to
be co scheduled within the same domain on SMT enabled systems.
Containerd impl sets the core sched domain when launching a shim. This
allows a clean way for each shim(container/pod) to be in its own domain and any
additional containers, (v2 pods) be be launched with the same domain as well as
any exec'd process added to the container.
kernel docs: https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/core-scheduling.html
For Kata specifically, we will look for SCHED_CORE environment variable
to be set to indicate we shuold create a new schedule core domain.
This is equivalent to the containerd shim's PR: e48bbe8394Fixes: #4309
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Signed-off-by: Michael Crosby <michael@thepasture.io>
While end users can connect directly to the shim, let's provide a way to
easily get/set iptables from kata-runtime itself.
Fixes: #4080
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Without this, potential errors are silently dropped. Let's ensure we
return the error code as well as potenial data from the response.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Before, we had a mix of slash, etc. Unfortunately, when cleaning URL
paths, serve mux seems to mangle the request method, resulting in each
request being a GET (instead of PUT or POST).
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Add two endpoints: ip6tables, iptables.
Each url handler supports GET and PUT operations. PUT expects
the requests' data to be []bytes, and to contain iptable information in
format to be consumed by iptables-restore.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Introduce get/set iptable handling. We add a sandbox API for getting and
setting the IPTables within the guest. This routes it from sandbox
interface, through kata-agent, ultimately making requests to the guest
agent.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Since we are introducing an agent API for interacting with guest
iptables, let's ensure that our example rootfs' have iptables-save/restore
installed.
Fixes: #4356
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Update the agent protocol definition to introduce support for setting
and getting iptables from the guest.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
This enables tests for the kata-agent for runk that is built
with standard-oci-runtime feature in CI.
Fixes: #4351
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Get user's home dir using `nix::unistd` crate instead of `utils` crate,
and remove useless code from agent.
Fixes: #4209
Signed-off-by: Xuewei Niu <justxuewei@apache.org>
runk always launches containers with detached mode,
so users have to use a console socket with run or
create operation when a terminal is used.
If users set `terminal` to `true` in `config.json` and
try to launch a container without specifying a console
socket, runk returns an error with a message early.
Fixes: #4324
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
This commit enables runk to handle `root.path` in `config.json`
properly even if the path is specified by a relative path that
includes the single (`.`) or the double (`..`) dots.
For example, with a bundle at `/to/bundle` and a rootfs directly
under `/to/bundle` such as `/to/bundle/{bin,dev,etc,home,...}`,
the `root.path` value can be either `/to/bundle` or just `.`.
This behavior conforms to OCI runtime spec.
Accordingly, a bundle path managed by runk's status file
(`status.json`) always is statically stored as a canonical path.
Previously, a bundle path has been got by `oci_state()` of rustjail's
API that returns the path as the parent directory path of a rootfs
(`root.path`). In case of the kata-agent, this works properly because
the kata containers assume that the rootfs path is always
`/to/bundle/rootfs`. However in case of standard OCI runtimes,
a rootfs can be placed anywhere under a bundle, so the rootfs path
doesn't always have to be at a `/to/bundle/rootfs`.
Fixes: #4334
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Destructive mode is required to build the Kata Containers snap. See:
```
.github/workflows/snap-release.yaml
.github/workflows/snap.yaml
```
Hence, update the last file that we forgot to update with
`--destructive-mode`.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Move the snap README to a subdirectory to resolve the warning given by
`snapcraft` (folded and reformatted slightly for clarity):
```
The 'snap' directory is meant specifically for snapcraft,
but it contains the following non-snapcraft-related paths,
which is unsupported and will cause unexpected behavior:
- README.md
If you must store these files within the 'snap' directory,
move them to 'snap/local', which is ignored by snapcraft.
```
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This release has been tracked through the v24.0 project.
virtio-iommu specification describes how a device can be attached by default
to a bypass domain. This feature is particularly helpful for booting a VM with
guest software which doesn't support virtio-iommu but still need to access
the device. Now that Cloud Hypervisor supports this feature, it can boot a VM
with Rust Hypervisor Firmware or OVMF even if the virtio-block device exposing
the disk image is placed behind a virtual IOMMU.
Multiple checks have been added to the code to prevent devices with identical
identifiers from being created, and therefore avoid unexpected behaviors at boot
or whenever a device was hot plugged into the VM.
Sparse mmap support has been added to both VFIO and vfio-user devices. This
allows the device regions that are not fully mappable to be partially mapped.
And the more a device region can be mapped into the guest address space, the
fewer VM exits will be generated when this device is accessed. This directly
impacts the performance related to this device.
A new serial_number option has been added to --platform, allowing a user to
set a specific serial number for the platform. This number is exposed to the
guest through the SMBIOS.
* Fix loading RAW firmware (#4072)
* Reject compressed QCOW images (#4055)
* Reject virtio-mem resize if device is not activated (#4003)
* Fix potential mmap leaks from VFIO/vfio-user MMIO regions (#4069)
* Fix algorithm finding HOB memory resources (#3983)
* Refactor interrupt handling (#4083)
* Load kernel asynchronously (#4022)
* Only create ACPI memory manager DSDT when resizable (#4013)
Deprecated features will be removed in a subsequent release and users should
plan to use alternatives
* The mergeable option from the virtio-pmem support has been deprecated
(#3968)
* The dax option from the virtio-fs support has been deprecated (#3889)
Fixes: #4317
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Update the snap config file to build the rust version of `virtiofsd` for
x86_64, but build QEMU's C version for other platforms.
Fixes: #4261.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Rather than putting the `yq` binary in the staging directory itself,
put it in the `bin/` sub-directory.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Remove the unused `kata_url` variable and use the value in the `website`
YAML metadata instead.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Based on @fidencio's opoinon,
On Arm: static build virtiofsd using musl lib;
on ppc64 & s390: static build virtiofsd using gnu lib;
Fixes: #4258
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Since Kata 2.x does not support runtime cli, remove information
related to it. Update the configuration snippet accordingly.
Fixes#3870
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
One may want to use standalone containerd without k8s
and still have network enabled for the container.
Getting rid of note due to inaccuracy.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Specify that the `--cni` flag needs to be passed to the `ctr` tool
while starting a container in order to have networking enabled for the
container. This flag allows containerd to call into the configured
network plugin which in turn creates a network interface for the
container.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
The current implementation of walking the
disks to match with the requested volume path
in agent doesn't work because the volume path
provided by the shim to the agent is the mount
path within the guest and not the device name.
The current logic is trying to match the
device name to the volume path which will never
match.
This change will simplify the
get_volume_capacity_stats and
get_volume_inode_stats to just call statfs and
get the bytes and inodes usage of the volume
path directly.
Fixes: #4297
Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
Today the shim does a translation when doing
direct-volume stats where it takes the source and
returns the mount path within the guest.
The source for a direct-assigned volume is actually
the device path on the host and not the publish
volume path.
This change will perform a lookup of the mount info
during direct-volume stats to ensure that the
device path is provided to the shim for querying
the volume stats.
Fixes: #4297
Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
The go default http mux AFAIK doesn’t support pattern
routing so right now client is padding the url
for direct-volume stats with a subpath of the volume
path and this will always result in 404 not found returned
by the shim.
This change will update the shim to take the volume
path as a GET query parameter instead of a subpath.
If the parameter is missing or empty, then return
400 BadRequest to the client.
Fixes: #4297
Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
The action function expects a function that returns error
but the current direct-volume stats Action returns
(string, error) which is invalid.
This change fixes the format and print out the stats from
the command instead.
Fixes: #4293
Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
This PR removes the clear containers reference as this is not longer
being used and is deprecated at the rootfs builder README.
Fixes#4278
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR removes the nemu.conf as we are not longer using NEMU from
the kernel configurations.
Fixes#4272
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The documentation of the bufio package explicitly says
"Err returns the first non-EOF error that was encountered by the
Scanner."
When io.EOF happens, `Err()` will return `nil` and `Scan()` will return
`false`.
Fixes#4079
Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
For the Gramine Shielded Containers guest kernel, CONFIG_NUMA must be
enabled.
Fixes #4266
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
In the is_signal_handled function, when parsing the hex string returned
from `/proc/<pid>/status` the space/tab character after the colon
is not removed.
This patch trims the result of SigCgt so that
all whitespace characters are removed. It also extends the existing
test cases to check for this scenario.
Fixes: #4250
Signed-off-by: Champ-Goblem <cameron@northflank.com>
As we are using a containerd version > 1.4 we need to update
the runc containerd runtime.
Fixes#4263
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This commit updates the "Run Kata Containers with Kubernetes" to include
cgroupDriver configuration via "KubeletConfiguration". Without this
setting kubeadm defaults to systemd cgroupDriver. Containerd with Kata
cannot spawn conntainers with systemd cgroup driver.
Fixes: #4262
Signed-off-by: Suraj Deshmukh <suraj.deshmukh@microsoft.com>
As we finally can move to using the rust virtiofs daemon, let's stop
bulding and packaging the C version of the virtiofsd for x86_64.
Fixes: #4249
Depends-on: github.com/kata-containers/tests#4785
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As now we build and ship the rust version of virtiofsd, which is not
tied to QEMU, we need to update its default location to match with where
we're installing this binary.
Fixes: #4249
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
There are a few patches for SGX numa support in QEMU added after the
6.2.0 release. Add them for SGX support in Kata.
Fixes#4254
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
As already done for the other assets we rely on, let's build (well, pull
in this very specific case) the virtiofsd binary, as we're relying on
its standlone rust version from now on.
Fixes: #4234
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As done for the other binaries we release, let's add support for
"building" (or pulling down) the static binary we ship as part of the
kata-containers static tarball (the same one used by kata-deploy).
Right now the virtiofsd is installed in /opt/kata/libexec/virtiofsd, a
different path than the virtiofsd that comes with QEMU.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As virtiofsd comes in the `zip` format, let's install unzip in the
containers and then be able to access the virtiofsd binary.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Right now this is very much x86_64 specific, but I'd like to count on
the maintainers of the other architectures to expand it.
Also, the name as it's now may be misleading, as we're actually only
pulling the binary that's statically built using `musl` and released as
part of virtiofsd official releases. But we'll need to build it for the
other architectures, thus I'm following the naming of the scripts used
by the other components.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we're switching to using the rust version of the virtiofsd, let's
give it its own entry in the versions.yaml file, as it's no longer part
of QEMU.
It's important to mention that GitLab doesn't provide a well formed URL
for the releases. Instead, it adds there a hash, leading us to have to
add the specific link for the tarball.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
go-test.sh by default adds the -v option to 'go test' meaning that output
will be printed from all the passing tests as well as any failing ones.
This results in a lot of output in which it's often difficult to locate the
failing tests you're interested in.
So, remove -v from the default flags.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
One of the responsibilities of the go-test.sh script is setting up the
default flags for 'go test'. This is constructed across several different
places in the script using several unneeded intermediate variables though.
Consolidate all the flag construction into one place.
fixes#4190
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
go-test.sh changes behaviour based on both the $CI and $KATA_DEV_MODE
variables, but not in a way that makes a lot of sense.
If either one is set it uses the test_coverage path, instead of the
test_local path. That collects coverage information, as the name
suggests, but it also means it runs the tests twice as root and
non-root, which is very non-obvious.
It's not clear what use case the test_local path is for at all.
Developer local builds will typically have $KATA_DEV_MODE set and CI
builds will have $CI set. There's essentially no downside to running
coverage all the time - it has little impact on the test runtime.
In addition, if *both* $CI and $KATA_DEV_MODE are set, the script
refuses to run things as root, considering it "unsafe". While having
both set might be unwise in a general sense, there's not really any
way running sudo can be any more unsafe than it is with either one
set.
So, simplify everything by just always running the test_coverage path.
This leaves the test_local path unused, so we can remove it entirely.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
go-test.sh accepts subcommands, however invoking it in the usual way via
the Makefile doesn't use them. In fact the only remaining subcommand is
"help" and we already have another way of getting the usage information
(-h or --help). We don't need a second way, so just drop subcommand
handling.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
go-test.sh defaults to testing all the packages listed by go list, except
for a number filtered out. It turns out that none of those filters are
necessary any more:
* We've long required a Go newer than 1.9 which means the vendor filter
isn't needed
* The agent filter doesn't do anything now that we've moved to the Kata
2.x unified repo
* The tests filters don't hit anything on the list of modules in
src/runtime (which is the only user of the script)
But since we don't need to filter anything out any more, we don't even need
to iterate through a list ourselves. We can simply pass "./..." directly
to go test and it will iterate through all the sub-packages itself.
Interestingly this more than doubles the speed of "make test" for me - I
suspect because go test's internal paralellism works better over a larger
pool of tests.
This also lets us remove handling of non-existent coverage files from
test_go_package(), since with default options we will no longer test packages without tests
by default. If the user explicitly requests testing of a package with no
tests, then failing makes sense.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The go-test.sh script has an explicit chmod command, run as root, to
set the mode of the temporary coverage files to 0644. AFAICT the
point of this is specifically the 004 bit allowing world read access,
so that we can then merge the temporary coverage file into the main
coverage file.
That's a convoluted way of doing things. Instead we can just run the tail
command which reads the temporary file as the same user that generated it.
In addition, go-test.sh became root to remove that temporary coverage
file. This is not necessary, since deleting a regular file just requires
write access to the directory, not the file itself.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The html-coverage option to this script doesn't really alter behaviour
it just does the same thing as normal coverage, then converts the
report to HTML. That conversion is a single command, plus a chmod to
make the final output mode 0644. That overrides any umask the user
has set, which doesn't seem like a policy decision this script should
be making.
Nothing in the kata-containers or tests repository uses this, so it doesn't
really make sense to keep this logic inside this script.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
In addition to coverage.txt, the go-test.sh script creates
coverage.txt.tmp files while running. These are temporary and
certainly shouldn't be committed, so add them to the gitignore file.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The go unit tests for the runtime are invoked by the helper script
ci/go-test.sh. Which calls the run_go_test() function in ci/lib.sh. Which
calls into .ci/go-test.sh from the tests repository.
But.. the runtime is the only user of this script, and generally stuff for
unit tests (rather than functional or integration tests) lives in the main
repository, not the tests repository.
So, just move the actual script into src/runtime. A change to remove it
from the tests repo will follow.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
to the kata-containers repo under the src/tools/log-parser folder
and vendor the modules
Fixes: #4100
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
The following issues have been addressed from the latest bug fix release
v23.1 of Cloud Hypervisor: 1) Add some missing seccomp rules; 2) Remove
virtio-fs filesystem entries from config on removal; 3) Do not delete
API socket on API server start; 4) Reject virtio-mem resize if the guest
doesn't activate the device; 5) Fix OpenAPI naming of I/O throttling
knobs;
Fixes: #4222
Signed-off-by: Bo Chen <chen.bo@intel.com>
This workflow is a scheduled job that runs at 23:00
every Sunday, it should only run the main repo
but not the forked ones.
Fixes: #4219
Signed-off-by: Bin Liu <bin@hyper.sh>
- agent watchers: ensure uid/gid is preserved on copy/mkdir
- clh: Rely on Cloud Hypervisor for generating the device ID
- agent: add tests for create_logger_task function
- runk: set BinaryName for runk for containerd
- tools: Add a Rust-based standard OCI container runtime based on Kata agent
- rustjail: add tests for parse_mount_table
- Virtcontainers: Enable hot plugging vhost-user-blk device on ARM
- docs: repropose direct-assigned volume
- versions: change qemu tdx url and tag
- doc: Update for NVIDIA GPUs
- agent-ctl: Fix abstract socket connections
- Implement network and disk rate limiter for Cloud Hypervisor
- kata-deploy: Add support to RKE2
- docs: Update containerd link to installation guide
- docs: remove pc machine type supports
- Agent: Unit tests for random.rs
- rustjail: Add tests for mount_grpc_to_oci
- packaging: Fix broken path in `build-static-clh.sh`
- Fix Go unit tests to clean up /tmp after themselves
- rustjail: add tests for mount_from function
- rustjail: Add tests for hooks_grpc_to_oci
- agent: modify the type of swappiness to u64
- libs/safe-path: add crate to safely resolve fs paths
- agent: move assert_result macro to test_utils file
- rustjail: Add tests for root_grpc_to_oci
- agent: add tests for mount_to_rootfs function
- agent: add tests for update_container_namespaces
- agent: add tests for is_signal_handled function
- Upgrade to Cloud Hypervisor v23.0
- agent: best-effort removing mount point
- test: Fix golangci-lint error for s390x
- fsGroup support for direct-assigned volume
- kata-monitor: add the README file
- kata-monitor: update the hrefs in the debug/pprof index page
- runtime: Base64 encode the direct volume mountInfo path
- runtime: no need to write virtiofsd error to log
- kata-monitor: add some links when generating pages for browsers
- agent: Avoid agent panic when reading empty stats
- docs: Update link to contributions guide
- agent: add tests for mount_storage
- agent: add test coverage for parse_mount_flags_and_options function
- agent: add tests for do_write_stream function
- runtime: delete debug option in virtiofsd
- rustjail: add test coverage for process_grpc_to_oci function
- agent: Allow the agent to be rebuilt with the change of Cargo features
- protocols: add src/csi.rs to .gitignore
- kata-runtime enable hugepage support
- docs: Add a firecracker installation guide
- runtime: Allow and require no initrd for SE
- test: use `T.TempDir` to create temporary test directory
- clh: Expose service offload configuration
33a8b705 clh: Rely on Cloud Hypervisor for generating the device ID
70eda2fa agent: watchers: ensure uid/gid is preserved on copy/mkdir
7772f7dd runk: set BinaryName for runk for containerd
7ffe5a16 docs: Direct-assigned volume design
081f6de8 versions: change qemu tdx url and tag
666aee54 docs: Add VSOCK localhost example for agent-ctl
86d348e0 docs: Use VM term in agent-ctl doc
4b9b62bb agent-ctl: Fix abstract socket connections
b6467ddd clh: Expose disk rate limiter config
7580bb5a clh: Expose net rate limiter config
a88adaba clh: Cloud Hypervisor has a built-in Rate Limiter
63c4da03 clh: Implement the Disk RateLimiter logic
511f7f82 config: Add DiskRateLimiter* to Cloud Hypervisor
5b18575d hypervisor: Add disk bandwidth and operations rate limiters
1cf94692 clh: Implement the Network RateLimiter logic
00a5b1bd utils: Define DefaultRateLimiterRefillTimeMilliSecs
be1bb7e3 utils: Move FC's function to revert bytes to utils
c9f6496d config: Add NetRateLimiter* to Cloud Hypervisor
2d35e606 hypervisor: Add network bandwidth and operations rate limiters
b0e439cb rustjail: add tests for parse_mount_table
ccb01839 kata-deploy: Add support to RKE2
9d39362e kata-deploy: Reestructure the installing section
18d27f79 kata-deploy: Add a missing `$` prefix in the README
6948b4b3 docs: Update containerd link to installation guide
b221a259 tools: Add runk
2c218a07 agent: Modify Kata agent for runk
dd4bd7f4 doc: Added initial doc update for NV GPUs
832c33d5 docs: remove pc machine type supports
b658dccc tools: fix typo in clh directory name
afbd60da packaging: Fix clh build from source fall-back
4b9e78b8 rustjail: Add tests for mount_grpc_to_oci
81f6b486 agent: add tests for create_logger_task function
96bc3ec2 rustjail: Add tests for hooks_grpc_to_oci
02395027 agent: modify the type of swappiness to u64
1b931f42 runtime: Allock mockfs storage to be placed in any directory
ef6d54a7 runtime: Let MockFSInit create a mock fs driver at any path
5d8438e9 runtime: Move mockfs control global into mockfs.go
963d03ea runtime: Export StoragePathSuffix
1719a8b4 runtime: Don't abuse MockStorageRootPath() for factory tests
bec59f9e runtime: Make bind mount tests better clean up after themselves
f7ba21c8 runtime: Clean up mock hook logs in tests
90b2f5b7 runtime: Make SetupOCIConfigFile clean up after itself
2eeb5dc2 runtime: Don't use fixed /tmp/mountPoint path
0ad89ebd safe-path: add more unit test cases
b63774ec libs/safe-path: add crate to safely resolve fs paths
f385b21b rustjail: add tests for mount_from function
0e7f1a5e agent: move assert_result macro to test_utils file
2256bcb6 rustjail: Add tests for root_grpc_to_oci
7b2ff026 kata-monitor: add a README file
29e569aa virtcontainers: clh: Re-generate the client code
6012c197 versions: Upgrade to Cloud Hypervisor v23.0
aabcebbf agent: best-effort removing mount point
d136c9c2 test: Fix golangci-lint error for s390x
86977ff7 kata-monitor: update the hrefs in the debug/pprof index page
78f30c33 agent: Avoid agent panic when reading empty stats
6e79042a runtime: no need to write virtiofsd error to log
9b6f24b2 agent: add tests for mount_to_rootfs function
c3776b17 agent: add tests for is_signal_handled function
9c22d955 agent: add tests for update_container_namespaces
92c00c7e agent: fsGroup support for direct-assigned volume
6e9e4e8c docs: Update link to contributions guide
532d5397 runtime: fsGroup support for direct-assigned volume
6a47b82c proto: fsGroup support for direct-assigned volume
9d5e7ee0 agent: add tests for mount_storage
f8cc5d1a kata-monitor: add some links when generating pages for browsers
c31cd0e8 rustjail: add test coverage for process_grpc_to_oci function
1118a3d2 agent: add test coverage for parse_mount_flags_and_options function
9d5b03a1 runtime: delete debug option in virtiofsd
eff7c7e0 agent: Allow the agent to be rebuilt with the change of Cargo features
b975f2e8 Virtcontainers: Enable hot plugging vhost-user-blk device on ARM
962d05ec protocols: add src/csi.rs to .gitignore
354cd3b9 runtime: Base64 encode the direct volume mountInfo path
485aeabb agent: add tests for do_write_stream function
4405b188 docs: Add a firecracker installation guide
98750d79 clh: Expose service offload configuration
59c7165e test: use `T.TempDir` to create temporary test directory
ff17c756 runtime: Allow and require no initrd for SE
1cad3a46 agent/random: Ensure data.len > 0
33c953ac agent: Add test_ressed_rng_not_root
39a35b69 agent: Add test to random::reseed_rng()
d8f39fb2 agent/random: Rename RNDRESEEDRNG to RNDRESEEDCRNG
a2f5c176 runtime/virtcontainers: Pass the hugepages resources to agent
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're currently hitting a race condition on the Cloud Hypervisor's
driver code when quickly removing and adding a block device.
This happens because the device removal is an asynchronous operation,
and we currently do *not* monitor events coming from Cloud Hypervisor to
know when the device was actually removed. Together with this, the
sandbox code doesn't know about that and when a new device is attached
it'll quickly assign what may be the very same ID to the new device,
leading to the Cloud Hypervisor's driver trying to hotplug a device with
the very same ID of the device that was not yet removed.
This is, in a nutshell, why the tests with Cloud Hypervisor and
devmapper have been failing every now and then.
The workaround taken to solve the issue is basically *not* passing down
the device ID to Cloud Hypervisor and simply letting Cloud Hypervisor
itself generate those, as Cloud Hypervisor does it in a manner that
avoids such conflicts. With this addition we have then to keep a map of
the device ID and the Cloud Hypervisor's generated ID, so we can
properly remove the device.
This workaround will probably stay for a while, at least till someone
has enough cycles to implement a way to watch the device removal event
and then properly act on that. Spoiler alert, this will be a complex
change that may not even be worth it considering the race can be avoided
with this commit.
Fixes: #4176
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add test coverage for the functions find_process and online_resources in src/sandbox.rs.
Fixes#4085Fixes#4136
Signed-off-by: Jack Hance <jack.hance@ndsu.edu>
Today in agent watchers, when we copy files/symlinks
or create directories, the ownership of the source path
is not preserved which can lead to permission issues.
In copy, ensure that we do a chown of the source path
uid/gid to the destination file/symlink after copy to
ensure that ownership matches the source ownership.
fs::copy() takes care of setting the permissions.
For directory creation, ensure that we set the
permissions of the created directory to the source
directory permissions and also perform a chown of the
source path uid/gid to ensure directory ownership
and permissions matches to the source.
Fixes: #4188
Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
runk uses liboci-cli crate to parse command line options,
but liboci-cli does not support --all option for kill command,
though this is the runtime spec behavior.
But crictl will issue kill --all command when stopping containers,
as a workaround, we use a custom kill command instead of the one
provided by liboci-cli.
Fixes: #4182
Signed-off-by: Bin Liu <bin@hyper.sh>
In at least kata versions 2.3.3 and 2.4.0 it was noticed that the guest
operating system's clock would drift out of sync slowly over time
whilst the pod was running.
This had previously been raised and fixed in the old reposity via [1].
In essence kvm_ptp and chrony were paired together in order to
keep the system clock up to date with the host.
In the recent versions of kata metioned above,
the chronyd.service fails upon boot with status `266/NAMESPACE`
which seems to be due to the fact that the `/var/lib/chrony`
directory no longer exists.
This change sets the `/var/lib/chrony` directory for the `ReadWritePaths`
to be ignored when the directory does not exist, as per [2].
[1] https://github.com/kata-containers/runtime/issues/1279
[2] https://www.freedesktop.org/software/systemd
/man/systemd.exec.html#ReadWritePaths=
Fixes: #4167
Signed-off-by: Champ-Goblem <cameron_mcdermott@yahoo.co.uk>
The default runtime for io.containerd.runc.v2 is runc,
to use runk, the containerd configuration should set the
default runtime to runk or add BinaryName options for the
runtime.
Fixes: #4177
Signed-off-by: Bin Liu <bin@hyper.sh>
Update the `agent-ctl` docs to show how to use a VSOCK local address
when running the agent and the tool in the same environment. This is an
alternative to using a Unix socket.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Unbreak the `agent-ctl` tool connecting to the agent with a Unix domain
socket.
It appears that [1] changed the behaviour of connecting to the agent
using a local Unix socket (which is not used by Kata under normal
operation).
The change can be seen by reverting to commit
72b8144b56 (the one before [1]) and
running the agent manually as:
```bash
$ sudo KATA_AGENT_SERVER_ADDR=unix:///tmp/foo.socket target/x86_64-unknown-linux-musl/release/kata-agent
```
Before [1], in another terminal we see this:
```bash
$ sudo lsof -U 2>/dev/null |grep foo|awk '{print $9}'
@/tmp/foo.socket@
```
But now, we see the following:
```bash
$ sudo lsof -U 2>/dev/null |grep foo|awk '{print $9}'
@/tmp/foo.socket
```
Note the last byte which represents a nul (`\0`) value.
The `agent-ctl` tool used to add that trailing nul but now it seems to not
be needed, so this change removes it, restoring functionality. No
external changes are necessary so the `agent-ctl` tool can connect to
the agent as below like this:
```bash
$ cargo run -- -l debug connect --server-address "unix://@/tmp/foo.socket" --bundle-dir "$bundle_dir" -c Check -c GetGuestDetails
```
[1] - https://github.com/kata-containers/kata-containers/issues/3124Fixes: #4164.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
With everything implemented, let's now expose the disk rate limiter
configuration options in the Cloud Hypervisor configuration file.
Fixes: #4139
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
With everything implemented, let's now expose the net rate limiter
configuration options in the Cloud Hypervisor configuration file.
Fixes: #4017
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The notion of "built-in rate limiter" was added as part of
bd8658e362, and that commit considered
that only Firecracker had a built-in rate limiter, which I think was the
case when that was introduced (mid 2020).
Nowadays, however, Cloud Hypervisor takes advantage of the very same crate
used by Firecraker to do I/O throttling.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's take advantage of the newly added DiskRateLimiter* options and
apply those to the network device configuration.
The logic here is identical to the one already present in the Network
part of Cloud Hypervisor's driver.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the newly added disk rate limiter configurations to the Cloud
Hypervisor's hypervisor configuration.
Right now those are not used anywhere, and there's absolutely no way the
users can set those up. That's coming later in this very same series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is the disk counterpart of the what was introduced for the network
as part of the previous commits in this series.
The newly added fields are:
* DiskRateLimiterBwMaxRate, defined in bits per second, which is used to
control the network I/O bandwidth at the VM level.
* DiskRateLimiterBwOneTimeBurst, also defined in bits per second, which
is used to define an *initial* max rate, which doesn't replenish.
* DiskRateLimiterOpsMaxRate, the operations per second equivalent of the
DiskRateLimiterBwMaxRate.
* DiskRateLimiterOpsOneTimeBurst, the operations per second equivalent of
the DiskRateLimiterBwOneTimeBurst.
For now those extra fields have only been added to the hypervisor's
configuration and they'll be used in the coming patches of this very
same series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's take advantage of the newly added NetRateLimiter* options and
apply those to the network device configuration.
The logic here is quite similar to the one already present in the
Firecracker's driver, with the main difference being the single Inbound
/ Outbound MaxRate and the presence of both Bandwidth and Operations
rate limiter.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Firecracker's driver doesn't expose the RefillTime option of the rate
limiter to the user. Instead, it uses a contant value of 1000
miliseconds (1 second).
As we're following Firecracker's driver implementation, let's expose
create a new constant, use it as part of the Firecracker's driver, and
later on re-use it as part of the Cloud Hypervisor's driver.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Firecracker's revertBytes function, now called "RevertBytes", can be
exposed as part of the virtcontainers' utils file, as this function will
be reused by Cloud Hypervisor, when adding the rate limiter logic there.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the newly added network rate limiter configurations to the
Cloud Hypervisor's hypervisor configuration.
Right now those are not used anywhere, and there's absolutely no way the
users can set those up. That's coming later in this very same series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In a similar way to what's already exposed as RxRateLimiterMaxRate and
TxRateLimiterMaxRate, let's add four new fields to the Hypervisor's
configuration.
The values added are related to bandwidth and operations rate limiters,
which have to be added so we can expose I/O throttling configurations to
users using Cloud Hypervisor as their preferred VMM.
The reason we cannot simply re-use {Rx,Tx}RateLimiterMaxRate is because
Cloud Hypervisor exposes a single MaxRate to be used for both inbound
and outbound queues.
The newly added fields are:
* NetRateLimiterBwMaxRate, defined in bits per second, which is used to
control the network I/O bandwidth at the VM level.
* NetRateLimiterBwOneTimeBurst, also defined in bits per second, which
is used to define an *initial* max rate, which doesn't replenish.
* NetRateLimiterOpsMaxRate, the operations per second equivalent of the
NetRateLimiterBwMaxRate.
* NetRateLimiterOpsOneTimeBurst, the operations per second equivalent of
the NetRateLimiterBwOneTimeBurst.
For now those extra fields have only been added to the hypervisor's
configuration and they'll be used in the coming patches of this very
same series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add tests for parse_mount_table function in rustjail/src/mount.rs.
Includes some minor refactoring improve the testability of the
function and improve its error values.
Fixes: #4082
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
"RKE2 - Rancher's Next Generation Kuberentes Distribution" can easily be
supported by kata-deploy with some simple adjustments to what we've been
relying on for "k3s".
The main differences between k3s and RKE2 are, basically:
1. The location where the containerd configuration is stored
- k3s: /var/lib/rancher/k3s/agent/etc/containerd/
- rke2: /var/lib/rancher/rke2/agent/etc/containerd/
2. The name of the systemd services used:
- k3s: k3s.service or k3s-agent.service
- rke2: rke2-server.service or rke2-agent.service
Knowing this, let's add a new overlay for RKE2, adapt the kata-deploy
and the kata-cleanup scripts, and that's it.
Fixes: #4160
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's move the specific installation instructions, such as for k3s,
upper in the document.
This helps reading (and also skipping) according to what the user
is looking for.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Add a Rust-based standard OCI container runtime based on
Kata agent.
You can build and install runk as follows:
```sh
$ cd src/tools/runk
$ make
$ sudo make install
$ runk --help
```
Fixes: #2784
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Generate an oci-kata-agent which is a customized agent to be
called from runk which is a Rust-based standard OCI container
runtime based on Kata agent.
Fixes: #2784
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Currently the 'pc' machine type is no longer supported in kata configuration,
so remove it in the design docs.
Fixes: #4155
Signed-off-by: Jason Zhang <zhanghj.lc@inspur.com>
If we fail to download the clh binary, we fall-back to build from source.
Unfortunately, `pull_clh_released_binary()` leaves a `cloud_hypervisor`
directory behind, which causes `build_clh_from_source()` not to clone
the git repo:
[ -d "${repo_dir}" ] || git clone "${cloud_hypervisor_repo}"
When building from a kata-containers git repo, the subsequent calls
to `git` in this function thus apply to the kata-containers repo and
eventually fail, e.g.:
+ git checkout v23.0
error: pathspec 'v23.0' did not match any file(s) known to git
It doesn't quite make sense actually to keep an existing directory the
content of which is arbitrary when we want to it to contain a specific
version of clh. Just remove it instead.
Fixes: #4151
Signed-off-by: Greg Kurz <groug@kaod.org>
The type of MemorySwappiness in runtime is uint64, and the type of swappiness in agent is int64,
if we set max uint64 in runtime and pass it to agent, the value will be equal to -1. We should
modify the type of swappiness to u64
Fixes: #4123
Signed-off-by: holyfei <yangfeiyu20092010@163.com>
Currently EnableMockTesting() takes no arguments and will always place the
mock storage in the fixed location /tmp/vc/mockfs. This means that one
test run can interfere with the next one if anything isn't cleaned up
(and there are other bugs which means that happens). If if those were
fixed this would allow developers testing on the same machine to interfere
with each other.
So, allow the mockfs to be placed at an arbitrary place given as a
parameter to EnableMockTesting(). In TestMain() we place it under our
existing temporary directory, so we don't need any additional cleanup just
for the mockfs.
fixes#4140
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently MockFSInit always creates the mockfs at the fixed path
/tmp/vc/mockfs. This change allows it to be initialized at any path
given as a parameter. This allows the tests in fs_test.go to be
simplified, because the by using a temporary directory from
t.TempDir(), which is automatically cleaned up, we don't need to
manually trigger initTestDir() (which is misnamed, it's actually a
cleanup function).
For now we still use the fixed path when auto-creating the mockfs in
MockAutoInit(), but we'll change that later.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
virtcontainers/persist/fs/mockfs.go defines a mock filesystem type for
testing. A global variable in virtcontainers/persist/manager.go is used to
force use of the mock fs rather than a normal one.
This patch moves the global, and the EnableMockTesting() function which
sets it into mockfs.go. This is slightly cleaner to begin with, and will
allow some further enhancements.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
storagePathSuffix defines the file path suffix - "vc" - used for
Kata's persistent storage information, as a private constant. We
duplicate this information in fc.go which also needs it.
Export it from fs.go instead, so it can be used in fc.go.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
A number of unit tests under virtcontainers/factory use
MockStorageRootPath() as a general purpose temporary directory. This
doesn't make sense: the mockfs driver isn't even in use here since we only
call EnableMockTesting for the pase virtcontainers package, not the
subpackages.
Instead use t.TempDir() which is for exactly this purpose. As a bonus it
also handles the cleanup, so we don't need MockStorageDestroy any more.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
There are several tests in mount_test.go which perform a sample bind
mount. These need a corresponding unmount to clean up afterwards or
attempting to delete the temporary files will fail due to the existing
mountpoint. Most of them had such an unmount, but
TestBindMountInvalidPgtypes was missing one.
In addition, the existing unmounts where done inconsistently - one was
simply inline (so wouldn't be executed if the test fails too early) and one
is a defer. Change them all to use the t.Cleanup mechanism.
For the dummy mountpoint files, rather than cleaning them up after the
test, the tests were removing them at the beginning of the test. That
stops the test being messed up by a previous run, but messily. Since
these are created in a private temporary directory anyway, if there's
something already there, that indicates a problem we shouldn't ignore.
In fact we don't need to explicitly remove these at all - they'll be
removed along with the rest of the private temporary directory.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The tests in hook_test.go run a mock hook binary, which does some debug
logging to /tmp/mock_hook.log. Currently we don't clean up those logs
when the tests are done. Use a test cleanup function to do this.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
SetupOCIConfigFile creates a temporary directory with os.MkDirTemp(). This
means the callers need to register a deferred function to remove it again.
At least one of them was commented out meaning that a /temp/katatest-
directory was leftover after the unit tests ran.
Change to using t.TempDir() which as well as better matching other parts of
the tests means the testing framework will handle cleaning it up.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Several tests in kata_agent_test.go create /tmp/mountPoint as a dummy
directory to mount. This is not cleaned up after the test. Although it
is in /tmp, that's still a little messy and can be confusing to a user.
In addition, because it uses the same name every time, it allows for one
run of the test to interfere with the next.
Use the built in t.TempDir() to use an automatically named and deleted
temporary directory instead.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
There are always path(symlink) based attacks, so the `safe-path` crate
tries to provde some mechanisms to harden path resolution related code.
Fixes: #3451
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Move the assert_result macro to the shared test_utils file
so that it is not duplicated in individual files.
Fixes: #4093
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
During container exit, the agent tries to remove all the mount point directories,
which can fail if it's a readonly filesytem (e.g. device mapper). This commit ignores
the removal failure and logs a warning message.
Fixes: #4043
Signed-off-by: Feng Wang <feng.wang@databricks.com>
This is to fix a test failure for the
kata-containers-2.0-ubuntu-20.04-s390x-main-baseline jenkins job
Fixes: #4088
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
kata-monitor allows to get data profiles from the kata shim
instances running on the same node by acting as a proxy
(e.g., http://$NODE_ADDRESS:8090/debug/pprof/?sandbox=$MYSANDBOXID).
In order to proxy the requests and the responses to the right shim,
kata-monitor requires to pass the sandbox id via a query string in the
url.
The profiling index page proxied by kata-monitor contains the link to all
the data profiles available. All the links anyway do not contain the
sandbox id included in the request: the links result then broken when
accessed through kata-monitor.
This happens because the profiling index page comes from the kata shim,
which will not include the query string provided in the http request.
Let's add on-the-fly the sandbox id in each href tag returned by the kata
shim index page before providing the proxied page.
Fixes: #4054
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
The scanner reads nothing from viriofsd stderr pipe, because param
'--syslog' rediercts stderr to syslog. So there is no need to write
scanner.Text() to kata log
Fixes: #4063
Signed-off-by: Zhuoyu Tie <tiezhuoyu@outlook.com>
Add test coverage for mount_to_rootfs function in src/mount.rs.
Includes minor refactoring to make function more easily testable.
Fixes#4073
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
Add test coverage for is_signal_handled function in rpc.rs. Includes
refactors to make the function testable and handle additional cases.
Fixes#3939
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
Add test coverage for update_container_namespaces function
in src/rpc.rs. Includes minor refactor to make function easier
to test.
Fixes#4034
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
Adding two functions set_ownership and
recursive_ownership_change to support changing group id
ownership for a mounted volume.
The set_ownership will be called in common_storage_handler
after mount_storage performs the mount for the volume.
set_ownership will be a noop if the FSGroup field in the
Storage struct is not set which indicates no chown will be
performed. If FSGroup field is specified, then it will
perform the recursive walk of the mounted volume path to
change ownership of all files and directories to the
desired group id. It will also configure the SetGid bit
so that files created the directory will have group
following parent directory group.
If the fsGroupChangePolicy is on root mismatch,
then the group ownership will be skipped if the root
directory group id alreasy matches the desired group
id and if the SetGid bit is also set on the root directory.
This is the same behavior as what
Kubelet does today when performing the recursive walk
to change ownership.
Fixes#4018
Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
This PR updates the url link to the contributions guide
at the Limitations document.
Fixes#4070
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The fsGroup will be specified by the fsGroup key in
the direct-assign mountinfo metadate field.
This will be set when invoking the kata-runtime
binary and providing the key, value pair in the metadata
field. Similarly, the fsGroupChangePolicy will also
be provided in the mountinfo metadate field.
Adding an extra fields FsGroup and FSGroupChangePolicy
in the Mount construct for container mount which will
be populated when creating block devices by parsing
out the mountInfo.json.
And in handleDeviceBlockVolume of the kata-agent client,
it checks if the mount FSGroup is not nil, which
indicates that fsGroup change is required in the guest,
and will provide the FSGroup field in the protobuf to
pass the value to the agent.
Fixes#4018
Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
This change adds two fields to the Storage pb
FSGroup which is a group id that the runtime
specifies to indicate to the agent to perform a
chown of the mounted volume to the specified
group id after mounting is complete in the guest.
FSGroupChangePolicy which is a policy to indicate
whether to always perform the group id ownership
change or only if the root directory group id
does not match with the desired group id.
These two fields will allow CSI plugins to indicate
to Kata that after the block device is mounted in
the guest, group id ownership change should be performed
on that volume.
Fixes#4018
Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
Add some links to rendered webpages for better user experience,
let users can jump to pages only by clicking links in browsers.
Fixes: #4061
Signed-off-by: bin <bin@hyper.sh>
Add test coverage for the parse_mount_flags_and_options function
in src/mount.rs.
Fixes#4056
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
virtiofsd's debug will be enabled if hypervisor's debug has been
enabled, this will generate too many noisy logs from virtiofsd.
Unbind the relationship of log level between virtiofsd and
hypervisor, if users want to see debug log of virtiofsd,
can set it by:
virtio_fs_extra_args = ["-o", "log_level=debug"]
Fixes: #3303
Signed-off-by: bin <bin@hyper.sh>
This allows the kata-agent to be rebuilt when Cargo "features" is
changed. The Makefile for the agent do not need to specify the
sources for prerequisites by having Cargo check for the sources
changes.
Fixes: #4052
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
The vhost-user-blk can be hotplugged on the PCI bridge successfully on
X86, but failed on Arm. However, hotplugging it on Root Port as a PCIe
device can work well on ARM.
Open the "pcie_root_port" in configuration.toml is needed.
Fixes: #4019
Signed-off-by: Jaylyn Ren <jaylyn.ren@arm.com>
After running make in src/agent, the git working area will be changed:
Untracked files:
(use "git add <file>..." to include in what will be committed)
src/libs/protocols/src/csi.rs
The generated file by `build.rs` should be ignored in git.
Fixes: #3959
Signed-off-by: bin <bin@hyper.sh>
Add test coverage for do_write_stream function of AgentService
in src/rpc.rs. Includes minor refactoring to make function more
easily testable.
Fixes#3984
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
This configuration option is valid for all the hypervisor that are going
to be used with the confidential containers effort, thus exposing the
configuration option for Cloud Hypervisor as well.
Fixes: #4022
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
- agent: fix container stop error with signal SIGRTMIN+3
- doc: Improve kata-deploy README.md by changing sh blocks to bash blocks
- docs: Remove kata-proxy reference
- kata-monitor: fix duplicated output when printing usage
- Stop getting OOM events from agent for "ttrpc closed" error
- tools/packaging: Fix error path in `kata-deploy-binaries.sh -s`
- kata-deploy: fix version bump from -rc to stable
- release: Include all the rust vendored code into the vendored tarball
- docs: Remove VPP documentation
- runtime: Remove the explicit VirtioMem set and fix the comment
- tools/packaging/kata-deploy: Copy install_yq.sh before starting parallel builds
- docs: Remove kata-proxy references in documentation
- agent: Signal the whole process group
- osbuilder/qat: don't pull kata sources if exist
- docs: fix markdown issues in how-to-run-docker-with-kata.md
- osbuilder/qat: use centos as base OS
- docs: Update vcpu handling document
- Agent: fix unneeded late initialization lint
- static-build,clh: Add the ability to build from a PR
- Don't use a globally installed mock hook for hook tests
- ci: Weekly check whether the docs url is alive
- Multistrap Ubuntu & enable cross-building guest
- device: using const strings for block-driver option instead of hard coding
- doc: update Intel SGX use cases document
- tools: update QEMU to 6.2
- action: Update link for format patch documentation
- runtime: properly handle ESRCH error when signaling container
- docs: Update k8s documentation
- rustjail: optimization, merged several writelns into one
- doc: fix kata-deploy README typo
- versions: Upgrade to Cloud Hypervisor v22.1
- Add debug and self-test control options to Kata Manager
- scripts: Change here document delimiters
- agent: add tests for get_memory_info function
- CI: Update GHA secret name
- tools: release: Do not consider release candidates as stable releases
- kernel: fix cve-2022-0847
- docs: Update contact link in runtime README
- Improve error checking of hugepage allocation
- CI: Create GHA to add PR sizing label
- release: Revert kata-deploy changes after 2.4.0-rc0 release
2b91dcfe docs: Remove kata-proxy reference
0d765bd0 agent: fix container stop error with signal SIGRTMIN+3
a63bbf97 kata-monitor: fix duplicated output when printing usage
9e4ca0c4 doc: Improve kata-deploy README.md by changing sh blocks to bash blocks
a779e19b tools/packaging: Fix error path in 'kata-deploy-binaries.sh -s'
0baebd2b tools/packaging: Fix usage of kata-deploy-binaries.sh
3606923a workflows,release: Ship *all* the rust vendored code
2eb07455 tools: Add a generate_vendor.sh script
5e1c30d4 runtime: add logs around sandbox monitor
fb8be961 runtime: stop getting OOM events when ttrpc: closed error
93d03cc0 kata-deploy: fix version bump from -rc to stable
a9314023 docs: Remove kata-proxy references in documentation
66f05c5b runtime: Remove the explicit VirtioMem set and fix the comment
0928eb9f agent: Kill the all the container processes of the same cgroup
c2796327 osbuilder/qat: don't pull kata sources if exist
154c8b03 tools/packaging/kata-deploy: Copy install_yq.sh in a dedicated script
1ed7da8f packaging: Eliminate TTY_OPT and NO_TTY variables in kata-deploy
bad859d2 tools/packaging/kata-deploy/local-build: Add build to gitignore
19f372b5 runtime: Add more debug logs for container io stream copy
459f4bfe osbuilder/qat: use centos as base OS
9a5b4770 docs: Update vcpu handling document
ecf71d6d docs: Remove VPP documentation
c77e34de runtime: Move mock hook source
86723b51 virtcontainers: Remove unused install/uninstall targets
0e83c95f virtcontainers: Run mock hook from build tree rather than system bin dir
77434864 docs: fix markdown issues in how-to-run-docker-with-kata.md
32131cb8 Agent: fix unneeded late initialization lint
e65db838 virtcontainers: Remove VC_BIN_DIR
c20ad283 virtcontainers: Remove unused Makefile defines
c776bdf4 virtcontainers: Remove unused parameter from go-test.sh
ebec6903 static-build,clh: Add the ability to build from a PR
24b29310 doc: update Intel SGX use cases document
18d4d7fb tools: update QEMU to 6.2
62351637 action: Update link for format patch documentation
aa5ae6b1 runtime: Properly handle ESRCH error when signaling container
efa19c41 device: use const strings for block-driver option instead of hard coding
dacf6e39 doc: fix filename typo
92ce5e2d rustjail: optimization, merged several writelns into one
7a18e32f versions: Upgrade to Cloud Hypervisor v22.1
5c434270 docs: Update k8s documentation
5d6d39be scripts: Change here document delimiters
be12baf3 manager: Change here documents to use standard delimiter
9576a7da manager: Add options to change self test behaviour
d4d65bed manager: Add option to enable component debug
019da91d manager: Whitespace fix
d234cb76 manager: Create containerd link
c088a3f3 agent: add tests for get_memory_info function
4b1e2f52 CI: Update GHA secret name
ffdf961a docs: Update contact link in runtime README
5ec7592d kernel: fix cve-2022-0847
6a850899 CI: Create GHA to add PR sizing label
2b41d275 release: Revert kata-deploy changes after 2.4.0-rc0 release
4adf93ef tools: release: Do not consider release candidates as stable releases
72f7e9e3 osbuilder: Multistrap Ubuntu
df511bf1 packaging: Enable cross-building agent
0a313eda osbuilder: Fix use of LIBC in rootfs.sh
2c86b956 osbuilder: Simplify Rust installation
0072cc2b osbuilder: Remove musl installations
5c3e5536 osbuilder: apk add --no-cache
42e35505 agent: Verify that we allocated as many hugepages as we need
608e003a agent: Don't attempt to create directories for hugepage configuration
168fadf1 ci: Weekly check whether the docs url is alive
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
This PR removes the kata-proxy reference from this document as it is
not longer a component in kata 2.0
Fixes#4013
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This release changes Docker images repository from DockerHub to Amazon
ECR. This resolves the `You have reached your pull rate limit` error
when building the firecracker tarball.
Fixes#4001
Signed-off-by: Greg Kurz <groug@kaod.org>
The nix::sys::signal::Signal package api cannot deal with SIGRTMIN+3,
directly use libc function to send the signal.
Fixes: #3990
Signed-off-by: Wang Xingxing <stellarwxx@163.com>
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.
This commit also updates the unit test advice to use `T.TempDir` to
create temporary directory in tests.
Fixes: #3924
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
(default: "/run/containerd/containerd.sock") is duplicated when
printing kata-monitor usage:
[root@kubernetes ~]# kata-monitor --help
Usage of kata-monitor:
-listen-address string
The address to listen on for HTTP requests. (default ":8090")
-log-level string
Log level of logrus(trace/debug/info/warn/error/fatal/panic). (default "info")
-runtime-endpoint string
Endpoint of CRI container runtime service. (default: "/run/containerd/containerd.sock") (default "/run/containerd/containerd.sock")
the golang flag package takes care of adding the defaults when printing
usage. Remove the explicit print of the value so that it would not be
printed on screen twice.
Fixes: #3998
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
The idea is to pass this README file to kata-doc-to-script.sh script and
then execute the result.
Added comments with a file name on top of each YAML snippet.
This helps in assigning a file name when we cat the YAML to a file.
Fixes: #3943
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
`make kata-tarball` relies on `kata-deploy-binaries.sh -s` which
silently ignores errors, and you may end up with an incomplete
tarball without noticing it because `make`'s exit status is 0.
`kata-deploy-binaries.sh` does set the `errexit` option and all the
code in the script seems to assume that since it doesn't do error
checking. Unfortunately, bash automatically disables `errexit` when
calling a function from a conditional pipeline, like done in the `-s`
case:
if [ "${silent}" == true ]; then
if ! handle_build "${t}" &>"$log_file"; then
^^^^^^
this disables `errexit`
and `handle_build` ends with a `tar tvf` that always succeeds.
Adding error checking all over the place isn't really an option
as it would seriously obfuscate the code. Drop the conditional
pipeline instead and print the final error message from a `trap`
handler on the special ERR signal. This requires the `errtrace`
option as `trap`s aren't propagated to functions by default.
Since all outputs of `handle_build` are redirected to the build
log file, some file descriptor duplication magic is needed for
the handler to be able to write to the orignal stdout and stderr.
Fixes#3757
Signed-off-by: Greg Kurz <groug@kaod.org>
Instead of only vendoring the code needed by the agent, let's ensure we
vendor all the needed rust code, and let's do it using the newly
introduced enerate_vendor.sh script.
Fixes: #3973
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This script is responsible for generating a tarball with all the rust
vendored code that is needed for fully building kata-containers on a
disconnected environment.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
getOOMEvents is a long-waiting call, it will retry when failed.
For cases of agent shutdown, the retry should stop.
When the agent hasn't detected agent has died, we can also check
whether the error is "ttrpc: closed".
Fixes: #3815
Signed-off-by: bin <bin@hyper.sh>
This PR removes the kata-proxy references in VSocks documentation,
as this is not a component in kata 2.0 and all the examples that
were used belonged to kata 1.x.
Fixes#3980
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Modify the 2Mib in the comment to 4Mib.
VirtioMem is set by configuration file or annotation. And setupVirtioMem is called only when VirtioMem is true.
Fixes: #3750
Signed-off-by: yaoyinnan <yaoyinnan@foxmail.com>
For the library `procfs`, the unit of values in `CpuTime` is ticks,
and we do not know how many ticks per second from metrics because the
`tps` in `CpuTime` is private.
But there are some implements in `CpuTime` for getting these values,
e.g., `user_ms()` for `user`, and `nice_ms()` for `nice`. With these
values, accurate time can be obtained.
Fixes: #3979
Acked-by: zhaojizhuang <571130360@qq.com>
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
Previously, it was not permitted to have neither an initrd nor an image.
However, this is the exact config to use for Secure Execution, where the
initrd is part of the image to be specified as `-kernel`. Require the
configuration of no initrd for Secure Execution.
Also
- remove redundant code for image/initrd checking -- no need to check in
`newQemuHypervisorConfig` (calling) when it is also checked in
`getInitrdAndImage` (called)
- use `QemuCCWVirtio` constant when possible
Fixes: #3922
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
'make kata-tarball' sometimes fails early with:
cp: cannot create regular file '[...]/tools/packaging/kata-deploy/local-build/dockerbuild/install_yq.sh': File exists
This happens because all assets are built in parallel using the same
`kata-deploy-binaries-in-docker.sh` script, and thus all try to copy
the `install_yq.sh` script to the same location with the `cp` command.
This is a well known race condition that cannot be avoided without
serialization of `cp` invocations.
Move the copying of `install_yq.sh` to a separate script and ensure
it is called *before* parallel builds. Make the presence of the copy
a prerequisite for each sub-build so that they still can be triggered
individually. Update the GH release workflow to also call this script
before calling `kata-deploy-binaries-in-docker.sh`.
Fixes#3756
Signed-off-by: Greg Kurz <groug@kaod.org>
NO_TTY configured whether to add the -t option to docker run. It makes no
sense for the caller to configure this, since whether you need it depends
on the commands you're running. Since the point here is to run
non-interactive build scripts, we don't need -t, or -i either.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Greg Kurz <groug@kaod.org>
This directory consists entirely of files built during a make kata-tarball,
so it should not be committed to the tree. A symbolic link to this directory
might be created during 'make tarball', ignore it as well.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
[greg: - rearranged the subject to make the subsystem checker happy
- also ignore the symbolic link created by
`kata-deploy-binaries-in-docker.sh`]
Signed-off-by: Greg Kurz <groug@kaod.org>
This PR updates the vcpu handling document by removing docker information
which is not longer being used in kata 2.x and leaving only k8s information.
Fixes#3950
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR is removing VPP documentation as it is not longer valid with
kata 2.x, all the instructions were used for kata 1.x
Fixes#3946
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
src/runtime/virtcontainers/hook/mock contains a simple example hook in Go.
The only thing this is used for is for some tests in
src/runtime/pkg/katautils/hook_test.go. It doesn't really have anything
to do with the rest of the virtcontainers package.
So, move it next to the test code that uses it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We've now removed the need to install the mock hook binary for unit tests.
However, it turns out that managing that was the *only* thing that the
install and uninstall targets in the virtcontainers Makefile handled.
So, remove them.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Running unit tests should generally have minimal dependencies on
things outside the build tree. It *definitely* shouldn't modify
system wide things outside the build tree. Currently the runtime
"make test" target does so, though.
Several of the tests in src/runtime/pkg/katautils/hook_test.go require a
sample hook binary. They expect this hook in
/usr/bin/virtcontainers/bin/test/hook, so the makefile, as root, installs
the test binary to that location.
Go tests automatically run within the package's directory though, so
there's no need to use a system wide path. We can use a relative path to
the binary build within the tree just as easily.
fixes#3941
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The VC_BIN_DIR variable in the virtcontainers Makefile is almost unused.
It's used to generate TEST_BIN_DIR, and it's created in the install target.
However, we also create TEST_BIN_DIR, which is a subdirectory of VC_BIN_DIR
with mkdir -p, so it will necessarily create VC_BIN_DIR along the way.
So we can drop the unnecessary mkdir and expand the definition of
VC_BIN_DIR in the definition of TEST_BIN_DIR.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The INSTALL_EXEC and UNINSTALL_EXEC definitions from the virtcontainers
Makefile (unlike those from the runtime Makefile in the parent directory)
are entirely unused. Remove them.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The check-go-test target passes the path to the mock hook test binary to
go-test.sh when it invokes it. But go-test.sh just calls run_go_test from
ci/lib.sh, which invokes a script from the tests repo *without* any
parameters.
That is, this parameter is ignored anyway, so remove it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Right now it doesn't do much for us, as we're always building from a
specific version. However, this opens the possibility for us to add a
CI, similar to the one we have for CRI-O, for testing against each
cloud-hypervisor PR, on the cloud-hypervisor branch.
Fixes: #3908
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Installation section is not longer needed because of the latest
default kata kernel supports Intel SGX.
Include QEMU to the list of supported hypervisors.
fixes#3911
Signed-off-by: Julio Montes <julio.montes@intel.com>
bring Intel SGX support
Changes tha may impact in Kata Containers
Arm:
The 'virt' machine now supports an emulated ITS
The 'virt' machine now supports more than 123 CPUs in TCG emulation mode
The pl031 real-time clock device now supports sending RTC_CHANGE QMP events
PowerPC:
Improved POWER10 support for the 'powernv' machine
Initial support for POWER10 DD2.0 CPU added
Added support for FORM2 PAPR NUMA descriptions in the "pseries" machine
type
s390x:
Improved storage key emulation (e.g. fixed address handling, lazy
storage key enablement for TCG, ...)
New gen16 CPU features are now enabled automatically in the latest
machine type
KVM:
Support for SGX in the virtual machine, using the /dev/sgx_vepc device
on the host and the "memory-backend-epc" backend in QEMU.
New "hv-apicv" CPU property (aliased to "hv-avic") sets the
HV_DEPRECATING_AEOI_RECOMMENDED bit in CPUID[0x40000004].EAX.
virtio-mem:
QEMU now fully supports guest memory dumps with virtio-mem.
QEMU now cleanly supports precopy migration, postcopy migration and
background snapshots with virtio-mem.
fixes#3902
Signed-off-by: Julio Montes <julio.montes@intel.com>
This PR updates the link for the format patch documentation for the
commit message check.
Fixes#3900
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Currently kata shim v2 doesn't translate ESRCH signal, causing container
fail to stop and shim leak.
Fixes: #3874
Signed-off-by: Feng Wang <feng.wang@databricks.com>
Currently, the block driver option is specifed by hard coding, maybe it
is better to use const string variables instead of hard coded strings.
Another modification is to remove duplicate consts for virtio driver in
manager.go.
Fixes: #3321
Signed-off-by: Jason Zhang <zhanghj.lc@inspur.com>
Update documentation with missing step to untaint node to enable
scheduling and update the example to run a pod using the kata runtime
class instead of untrusted workloads, which applies to versions of CRI-O
prior to v1.12.
Fixes#3863
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
All scripts should use `EOF` as the shell here document delimiter as
this is checked by the static checker.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Added new `kata-manager` options to control the self-test behaviour. By
default, after installation the manager will run a test to ensure a Kata
Containers container can be created. New options allow:
- The self test to be disabled.
- Only the self test to be run (no installation).
These features allow changes to be made to the installed system before
the self test is run.
Fixes: #3851.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Make the `kata-manager` create a `containerd` link to ensure the
downloaded containerd systemd service file can find the daemon when
using the GitHub packaged version of containerd.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add test coverage for get_memory_info function in src/rpc.rs. Includes
some minor refactoring of the function.
Fixes#3837
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
Change the secret used by the GitHub Action that adds the PR size
label to one with the correct set of privileges.
Fixes: #3856.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
As 2.4.0-rc0 has been released, let's switch the kata-deploy / kata-cleanup
tags back to "latest", and re-add the kata-deploy-stable and the
kata-cleanup-stable files.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
- Enhancement: fix comments/logs and delete not used function
- storage: make k8s emptyDir volume creation location configurable
- Implement direct-assigned volume
- Bump containerd to 1.6.1
- experimentally enable vcpu hotplug and virtio-mem on arm64 in kernel part
- versions: Upgrade to Cloud Hypervisor v22.0
- katatestutils: remove distro constraints
- Minor fixes for the `disable_block_device_use` comments
- clh: stop virtofsd if clh fails to boot up the vm
- clh: tdx: Don't use sharedFS with Confidential Guests
- runtime: Build golang components with extra security options
- snap: Use git clone depth 1 for QEMU and dependencies
- snap: Don't build cloud-hypevisor on ppc64le
- build: always reset ARCH after getting it
- virtcontainers: remove temp dir created for vsock in test code
- docs: Add unit testing presentation
- virtcontainers: Use available s390x hugepages
- Update QEMU >= 6.1.0 in configure-hypervisor.sh
- Fix monitor listen address
- snap: clh: Re-use kata-deploy script here
- osbuilder: Add CentOS Stream rootfs
- runtime: Gofmt fixes
- Update `confidential_guest` comments
- cleanup runtime pkgs for Darwin build, add basic Darwin build/unit test
- docs: Update Readme document
- runtime: use Cmd.StdoutPipe instead of self-created pipe
- docs: Developer-Guide build a custom Kata agent with musl
- kata-agent: Fix mismatching error of cgroup and mountinfo.
- runtime, config: make selinux configurable
- Fix unbound variable / typo on error mesage
- clh: Add TDX support
- virtcontainers: Do not add a virtio-rng-ccw device
- kata-monitor: fix collecting metrics for sandboxes not started through CRI
- runtime: fix package declaration for ppc64le
- Make the hypervisor framework not Linux specific
- kata-deploy: Simplify Dockerfile and support s390x
- Support nerdctl OCI hooks
- shim: log events for CRI-O
- docs: Update contributing link
- kata-deploy: Use (kata with) qemu as the default shim-v2 binary
- kata-monitor: simplify sandbox cache management and attach kubernetes POD metadata to metrics
- nydus: add lazyload support for kata with clh
- kernel: remove SYS_SUPPORTS_HUGETLBFS from powerpc fragments
- packaging: Use `patch` for applying patches
- virtcontainers: Remove duplicated assert messages in utils test code
- versions: add nydus-snapshotter
- docs: Update limitations document
- packaging: support qemu-tdx
- Kata manager fix install
- versions: Linux 5.15.x
- trace-forwarder/agent-ctl: run cargo fmt/clippy in make check
- docs: Improve top-level README
- runtime: use github.com/mdlayher/vsock@v1.1.0
- tools: Build cloud-hypervisor with "--features tdx"
- virtiofsd: Use "-o announce_submounts"
- feature: hugepages support
- tools: clh: Allow to set when to build from sources and the build flags passed down to cargo
- docs: Remove docker run and shared memory from limitations
- versions: Udpate Cloud Hypervisor to 55479a64d237
- kernel: add missing config fragment for TDx
- runtime: The index variable is initialized multiple times in for
- scripts: fix a typo while to check build_type
- versions: bump CRI-O to its 1.23 release
- feature(nydusd): add nydusd support to introduce lazyload ability
- docs: Fix relative links in Markdown
- kernel: support TDx
- device: Actually update PCIDEVICE_ environment variables for the guest
- docs: Update link to EFK stack docs
- runtime: support QEMU SGX
- snap: update qemu version to 6.1.0 for arm
- Release process related fixes
- openshift-ci: switch to CentOS Stream
- virtcontainers: Split the rootless package into OS specific parts
- runtime: suppport split firmware
- kata-deploy: for testing, make sure we use the PR branch
- docs: Remove Zun documentation with kata containers
- agent: Fix execute_hook() args error
- workflows: stop checking revert commit
84dff440 release: Adapt kata-deploy for 2.4.0-rc0
b257e0e5 rustjail: delete function signal in BaseContainer
d647b28b agent: delete meaningless FIXME comment
1b34494b runtime: fix invalid comments for pkg/resourcecontrol
afc567a9 storage: make k8s emptyDir creation configurable
e76519af runtime: small refactor to improve readability
7e5f11a5 vendor: Update containerd to 1.6.1
42771fa7 runtime: don't set socket and thread for arm/virt
8828ef41 kernel: add arm experimental kernel build support
8a9007fe config: remove 2 config as they are removed in 5.15
1b6f7401 kernel: add arm experimental patches to support vcpu hotplug and virtio-mem
f905161b runtime: mount direct-assigned block device fs only once
27fb4902 agent: add get volume stats handler in agent
ea51ef1c runtime: forward the stat and resize requests from shimv2 to kata agent
c39281ad runtime: update container creation to work with direct assigned volumes
4e00c237 agent: add grpc interface for stat and resize operations
e9b5a255 runtime: add stat and resize APIs to containerd-shim-v2
6e0090ab runtime: persist direct volume mount info
fa326b4e runtime: augment kata-runtime CLI to support direct-assigned volume
b8844fb8 versions: Upgrade to Cloud Hypervisor v22.0
af804734 clh: stop virtofsd if clh fails to boot up the vm
97951a2d clh: Don't use SharedFS with Confidential Guests
c30b3a9f clh: Adding a volume is not supported without SharedFS
f889f1f9 clh: introduce supportsSharedFS()
54d27ed7 clh: introduce loadVirtiofsDaemon()
ae2221ea clh: introduce stopVirtiofsDaemon()
e8bc26f9 clh: introduce setupVirtiofsDaemon()
413b3b47 clh: introduce createVirtiofsDaemon()
55cd0c89 runtime: Build golang components with extra security options
76e4f6a2 Revert "hypervisors: Confidential Guests do not support Device hotplug"
fa8b9392 config: qemu: Fix disable_block_device_use comments
9615c8bc config: fc: Don't expose disable_block_device_use
c1fb4bb7 snap: Don't build cloud-hypevisor on ppc64le
58913694 snap: Use git clone depth 1 for QEMU and dependencies
b27c7f40 docs: Add unit testing presentation
e64c54a2 monitor: Listen to localhost only by default
e6350d3d monitor: Fix build options
a67b93bb snap: clh: Re-use kata-deploy script here
f31125fe version: Bump cloud-hypervisor to b0324f85571c441f
54d0a672 subsystem: build
edf20766 docs: Update Readme document
eda8ea15 runtime: Gofmt fixes
4afb278f ci: add github action to exercise darwin build, unit tests
e355a718 container: file is not linux specific
b31876ee device-manager: move linux-only test to a linux-only file
6a5c6344 resourcecontrol: SystemdCgroup check is not necessarily linux specific
cc58cf69 resourcecontrol: convert stats dev_t to unit64types
5be188cc utils: Add darwin stub
ad044919 virtcontainers: Convert stats dev_t to uint64
56751089 katautils: Use a syscall wrapper for the hook JSON state
7d64ae7a runtime: Add a syscall wrapper package
abc681ca katautils: Add Darwin stub for the netNS API
de574662 config: Expand confidential_guest comments
641d475f config: clh: Use "Intel TDX" instead of just "TDX"
0bafa2de config: clh: Mention supported TEEs
81ed269e runtime: use Cmd.StdoutPipe instead of self-created pipe
8edca8bb kata-agent: Fix mismatching error of cgroup and mountinfo.
a9ba7c13 clh: Fix typo on HotplugRemoveDevice
827ab82a tools: clh: Fix unbound variable
082d538c runtime: make selinux configurable
1103f5a4 virtcontainers: Use FilesystemSharer for sharing the containers files
533c1c0e virtcontainers: Keep all filesystem sharing prep code to sandbox.go
61590bbd virtcontainers: Add a Linux implementation for the FilesystemSharer
03fc1cbd virtcontainers: Add a filesystem sharing interface
72434333 clh: Add TDX support
a13b4d5a clh: Add firmware to the config file
a8827e0c hypervisors: Confidential Guests do not support NVDIMM
f50ff9f7 hypervisors: Confidential Guests do not support Memory hotplug
df8ffecd hypervisors: Confidential Guests do not support Device hotplug
28c4c044 hypervisors: Confidential Guests do not support VCPUs hotplug
29ee870d clh: Add confidential_guest to the config file
9621c596 clh: refactor image / initrd configuration set
dcdc412e clh: use common kernel params from the hypervisor code
4c164afb versions: Update Cloud Hypervisor to 5343e09e7b8db
b2a65f90 virtcontainers: Use available s390x hugepages
cb4230e6 runtime: fix package declaration for ppc64le
fec26f8e kata-monitor: trivial: rename symbols & labels
9fd4e551 runtime: Move the resourcecontrol package one layer up
823faee8 virtcontainers: Rename the cgroups package
0d1a7da6 virtcontainers: Rename and clean the cgroup interface
ad10e201 virtcontainers: cgroups: Move non Linux routine to utils.go
d49d0b6f virtcontainers: cgroups: Define a cgroup interface
3ac52e81 kata-monitor: fix updating sandbox cache at startup
160bb621 kata-monitor: bump version to 0.3.0
1a3381b0 docs: Developer-Guide build a custom Kata agent with musl
f6fc1621 shim: log events for CRI-O
1d68a08f docs: Update contributing link
9123fc09 kata-deploy: Simplify Dockerfile and support s390x
11220f05 kata-deploy: Use (kata with) qemu as the default shim-v2 binary
3175aad5 virtiofs-nydus: add lazyload support for kata with clh
94b831eb virtcontainers: remove temp dir created for vsock in test code
8cc1b186 kernel: remove SYS_SUPPORTS_HUGETLBFS from powerpc fragments
5c9d2b41 packaging: Use `patch` for applying patches
5b3fb6f8 kernel: Build SGX as part of the vanilla kernel
2c35d8cb workflows: Stop building the experimental kernel
32e7845d snap: Build vanilla kernel for all arches
27de212f runtime: Always add network endpoints from the pod netns
1cee0a94 virtcontainers: Remove duplicated assert messages in utils test code
6c1d149a docs: Update limitations document
7c4ee6ec packaging/qemu: create no_patches file for qemu-tdx
d47c488b versions: add qemu tdx section
77c29bfd container: Remove VFIO lazy attach handling
7241d618 versions: add nydus-snapshotter
26b3f001 virtcontainers: Split hypervisor into Linux and OS agnostic bits
fa0e9dc6 virtcontainers: Make all Linux VMMs only build on Linux
c91035d0 virtcontainers: Move non QEMU specific constants to hypervisor.go
10ae0591 virtcontainers: Move guest protection definitions to hypervisor.go
b28d0274 virtcontainers: Make max vCPU config less QEMU specific
a5f6df6a govmm: Define the number of supported vCPUs per architecture
a6b40151 tools: clh: Remove unused variables
5816c132 tools: Build cloud-hypervisor with "--features tdx"
e6060cb7 versions: Linux 5.15.x
9818cf71 docs: Improve top-level and runtime README
36c3fc12 agent: support hugepages for containers
81a8baa5 runtime: add hugepages support
7df677c0 runtime: Update calculateSandboxMemory to include Hugepages Limit
948a2b09 tools: clh: Ensure the download binary is executable
72bf5496 agent: handle hook process result
80e8dbf1 agent: valid envs for hooks
4f96e3ea katautils: Pass the nerdctl netns annotation to the OCI hooks
a871a33b katautils: Run the createRuntime hooks
d9dfce14 katautils: Run the preStart hook in the host namespace
6be6d0a3 katautils: Pass the OCI annotations back to the called OCI hooks
493ebc8c utils: Update kata manager docs
34b2e67d utils: Added more kata manager cli options
714c9f56 utils: Improve containerd configuration
c464f326 utils: kata-manager: Force containerd sym link creation
4755d004 utils: Fix unused parameter
601be4e6 utils: Fix containerd installation
ae21fcc7 utils: Fix Kata tar archive check
f4d1e45c utils: Add kata-manager CLI options for kata and containerd
395cff48 docs: Remove docker run and shared memory from limitations
e07545a2 tools: clh: Allow passing down a build flag
55cdef22 tools: clh: Add the possibility to always build from sources
3f87835a utils: Switch kata manager to use getopts
4bd945b6 virtiofsd: Use "-o announce_submounts"
37df1678 build: always reset ARCH after getting it
3a641b56 katatestutils: remove distro constraints
90fd625d versions: Udpate Cloud Hypervisor to 55479a64d237
573a37b3 osbuilder: Add CentOS Stream rootfs
f10642c8 osbuilder: Source .cargo/env before checking Rust
955d359f kernel: add missing config fragment for TDx
734b618c agent-ctl: run cargo fmt/clippy in make check
12c37faf trace-forwarder: add make check for Rust
c1ce67d9 runtime: use github.com/mdlayher/vsock@v1.1.0
42a878e6 runtime: The index variable is initialized multiple times in for
1797b3eb packaging/kernel: build TDX guest kernel
98752529 versions: add url and tag for tdx kernel
bc8464e0 packaging/kernel: add option -s option
2d9f89ae feature(nydusd): add nydusd support to introduse lazyload ability
b19b6938 docs: Fix relative links in Markdown
9590874d device: Update PCIDEVICE_ environment variables for the guest
7b7f426a device: Keep host to VM PCI mapping persistently
0b2bd641 device: Rework update_spec_pci() to update_env_pci()
982f14fa runtime: support QEMU SGX
40aa43f4 docs: Update link to EFK stack docs
54e1faec scripts: fix a typo while to check build_type
07b9d93f virtcontainer: Simplify the sandbox network creation flow
2c7087ff virtcontainers: Make all endpoints Linux only
49d2cde1 virtcontainers: Split network tests into generic and OS specific parts
0269077e virtcontainers: Remove the netlink package dependency from network.go
7fca5792 virtcontainers: Unify Network endpoints management interface
c67109a2 virtcontainers: Remove the Network PostAdd method
e0b26443 virtcontainers: Define a Network interface
5e119e90 virtcontainers: Rename the Network structure fields and methods
b858d0de virtcontainers: Make all Network fields private
49eee79f virtcontainers: Remove the NetworkNamespace structure
844eb619 virtcontainers: Have CreateVM use a Network reference
d7b67a7d virtcontainers: Network API cleanups and simplifications
2edea883 virtcontainers: Make the Network structure manage endpoints
8f48e283 virtcontainers: Expand the Network structure
5ef522f7 runtime: check kvm module `sev` correctly
419d8134 snap: update qemu version to 6.1.0 for arm
00722187 docs: update Release-Process.md
496bc10d tools: check for yq before using it
88a70d32 Revert "workflows: Ensure a label change re-triggers the actions"
a9bebb31 openshift-ci: switch to CentOS Stream
89047901 kata-deploy-push: only run if PR modifying tools path
7ffe9e51 virtcontainers: Do not add a virtio-rng-ccw device
1f29478b runtime: suppport split firmware
24796d2f kata-deploy: for testing, make sure we use the PR branch
1cc1c8d0 docs: Remove images from Zun documentation
5861e52f docs: Remove Zun documentation with kata containers
903a6a45 versions: Bump critools to its 1.23 release
63eb1158 versions: bump CRI-O to its 1.23 release
5083ae65 workflows: stop checking revert commit
14e7f52a virtcontainers: Split the rootless package into OS specific parts
ab447285 kata-monitor: add kubernetes pod metadata labels to metrics
834e199e kata-monitor: drop unused functions
7516a8c5 kata-monitor: rework the sandbox cache sync with the container manager
e78d80ea kata-monitor: silently ignore CHMOD events on the sandboxes fs
e9eb34ce kata-monitor: improve debug logging
4fc4c76b agent: Fix execute_hook() args error
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
kata-deploy files must be adapted to a new release. The cases where it
happens are when the release goes from -> to:
* main -> stable:
* kata-deploy-stable / kata-cleanup-stable: are removed
* stable -> stable:
* kata-deploy / kata-cleanup: bump the release to the new one.
There are no changes when doing an alpha release, as the files on the
"main" branch always point to the "latest" and "stable" tags.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
During the release of 2.4.0-rc0 @egernst noticed an incositency in the
way we handle release tags, as release candidates are being taken as
"stable" releases, while both the kata-deploy tests and the release
action consider this as "latest".
Ideally we should have our own tag for "release candidate", but that's
something that could and should be discussed more extensively outside of
the scope of this quick fix.
For now, let's align the code generating the PR for bumping the release
with what we already do as part of the release action and kata-deploy
test, and tag "-rc" as latest, regardless of which branch it's coming
from.
Fixes: #3847
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Use `multistrap` for building Ubuntu rootfs. Adds support for building
for foreign architectures using the `ARCH` environment variable.
In the process, the Ubuntu rootfs workflow is vastly simplified.
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Requires setting ARCH and CC.
- Add CC linker option for building agent.
- Set host for building libseccomp.
Fixes: #3681
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
- Add a doc comment
- Pass to build container, e.g. to build x86_64 with glibc (would
always use musl)
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Remove a lot of cruft of musl installations -- we needed those for the
Go agent, but Rustup just takes care of everything. aarch64 on
Debian-based & Alpine is an exception -- create a symlink
`aarch64-linux-musl-gcc` to `musl-tools`'s `musl-gcc` or `gcc` on
Alpine. This is unified -- arch-specific Dockerfiles are removed.
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Hadolint DL3019. If you're wondering why this is in this PR, that's
because I touch the file later, and we're only triggering the lints for
changed files.
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
This change introduces the `disable_guest_empty_dir` config option,
which allows the user to change whether a Kubernetes emptyDir volume is
created on the guest (the default, for performance reasons), or the host
(necessary if you want to pass data from the host to a guest via an
emptyDir).
Fixes#2053
Signed-off-by: Evan Foster <efoster@adobe.com>
Let's bring in the latest release of Containerd, 1.6.1, released on
March 2nd, 2022.
With this, we take the opportunity to remove containerd/api reference as
we shouldn't need a separate module only for the API.
Here's the list of changes needed in the code due to the bump:
* stop using `grpc.WithInsecure()` as it's been deprecated
- use `grpc.WithTransportCredentials(insecure.NewCredentials())`
instead
Fixes: #3820
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As this is just a initial vcpu hotplug support, thread and socket has
not been supported. So, don't set socket and thread when hotadd cpu for
arm/virt.
Fixes: #3280
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Add a new entry of arm-kernel-experimental and let the kernel build
script support to build it.
Fixes: #3280
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
I'm sure that it is correct to remove CONFIG_ARM64_UAO and
CONFIG_MANDATORY_FILE_LOCKING and . Both are gone in 5.15. Maintain a
specific config files for a kernel version is a little ugly. If someone
needs them, shout at me.
Fixes: #3280
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
As the support for vcpu hotplug is on the road, I pick them up here as
experimental to let user try cpu hotplug and virtio-mem on arm64.
Fixes: #3280
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Mount the direct-assigned block device fs only once and keep a refcount
in the guest. Also use the ro flag inside the options field to determine
whether the block device and filesystem should be mounted as ro
Fixes: #3454
Signed-off-by: Feng Wang <feng.wang@databricks.com>
Translate the volume path from host-known path to guest-known path
and forward the request to kata agent.
Fixes: #3454
Signed-off-by: Feng Wang <feng.wang@databricks.com>
During the container creation, it will parse the mount info file
of the direct assigned volumes and update the in memory mount object.
Fixes: #3454
Signed-off-by: Feng Wang <feng.wang@databricks.com>
Add GetVolumeStats and ResizeVolume APIs for the runtime to query stat
and resize fs in the guest.
Fixes: #3454
Signed-off-by: Feng Wang <feng.wang@databricks.com>
To query fs stats and resize fs, the requests need to be passed to
kata agent through containerd-shim-v2. So we're adding to rest APIs
on the shim management endpoint.
Also refactor shim management client to its own go file.
Fixes: #3454
Signed-off-by: Feng Wang <feng.wang@databricks.com>
In the direct assigned volume scenario, Kata Containers persists
the information required for managing the volume inside the guest
on host filesystem.
Fixes: #3454
Signed-off-by: Feng Wang <feng.wang@databricks.com>
Add commands to add, remove, resize and get stats of a direct-assigned volume.
These commands are expected to be consumed by CSI.
Fixes: #3454
Signed-off-by: Feng Wang <feng.wang@databricks.com>
Highlights from the Cloud Hypervisor release v22.0: 1) GDB Debug Stub
Support; 2) `virtio-iommu` Backed Segments (to facilitate hotplug
devices that require being behind an IOMMU, e.g. QAT); 3) Before Boot
Configuration Changes; 4) `virtio-balloon` Free Page Reporting; 5)
Support for Direct Kernel Booting with TDX; 6) PMU Support for AArch64;
7) Documentation Under CC-BY-4.0 License; 8) Deprecation of "Classic"
virtiofsd (rust-based virtiofsd now is recommended); 9) Bug fixes on
`virtio-balloon`, `virtio-net` with multiple TAP fd support, REST APIs,
seccomp filters, migration with `vhost-user`, etc;
Details can be found: https://github.com/cloud-hypervisor/cloud-hypervisor/releases/tag/v22.0Fixes: #3825
Signed-off-by: Bo Chen <chen.bo@intel.com>
If, for some reason, we're able to launch cloud hypervisor but not able
to boot the VM up, the virtiofsd process would be left behind.
Let's ensure, via defer, that we stop virtiofsd in case of errors.
Fixes: #3819
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
kata-containers/pulls#3771 added TDX support for Cloud Hypervisor, but
two big things got overlooked while doing that.
1. virtio-fs, as of now, cannot be part of the trust boundary, so the
Confidential Guest will not be using it.
2. virtio-block hotplug should be enabled in order to use virtio-block
for the rootfs (used with the devmapper plugin).
When trying to use cloud-hypervisor with TDX using virtio-fs, we're
facing the following error on the guest kernel:
```
virtiofs virtio2: device must provide VIRTIO_F_ACCESS_PLATFORM
```
After checking and double-checking with virtiofs and cloud-hypervisor
developers, it happens as confidential containers might put some
limitations on the device, so it can't access all of the guests' memory
and that's where this restriction seems to be coming from. Vivek
mentioned that virtiofsd do not support VIRTIO_F_ACCESS_PLATFORM (aka
VIRTIO_F_IOMMU_PLATFORM) yet, and that for ecrypted guests virtiofs may
not be the best solution at the moment.
@sboeuf put this in a very nice way: "if the virtio-fs driver doesn't
support VIRTIO_F_ACCESS_PLATFORM, then the pages corresponding to the
virtqueues and the buffers won't be marked as SHARED, meaning the VMM
won't have access to it".
Interestingly enough, it works with QEMU, and it may be due to some
change done on the patched QEMU that @devimc is packaging, but we won't
take the path to figure out what was the change and patch
cloud-hypervisor on the same way, because of 1.
Fixes: #3810
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As mounting volumes into the guest requires SharedFS setup, let's ensure
we error out if trying to do so in a situation where SharedFS is not
supported.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
supportsSharedFS() is a new method to be used to ensure that no SharedFS
specifics are called when, for a reason or another, Cloud Hypervisor is
in a mode where SharedFSs are not supported.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Similarly to the `createVirtiofsDaemon` and `stopVirtiofsDaemon` methos,
let's introduce and use loadVirtiofsDaemon, at it'll also be handy later
in this series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Similary to the `createVirtiofsDaemon` method, let's introduce and use
its counterpart, as it'll also be handy later in this series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Similarly to what's been done with the `createVirtiofsDaemon`, let's
create a `setupVirtiofsDaemon` one.
It will also become handy later in this series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's introduce and use a new `createVirtiofsDaemon` method. Its name
says it all, and it'll be handy later in this series when, spoiler
alert, SharedFS cannot be used (in such cases as in Confidential
Guests).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit df8ffecde0, as device
hotplug *is* supported and, more than that, is very much needed when
using virtio-blk instead of virtio-fs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
allocate_hugepages() writes to the kernel sysfs file to allocate hugepages
in the Kata VM. However, even if the write succeeds, it's not certain that
the kernel will actually be able to allocate as many hugepages as we
requested.
This patch reads back the file after writing it to check if we were able to
allocate all the required hugepages.
fixes#3816
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
allocate_hugepages() constructs the path for the sysfs directory containing
hugepage configuration, then attempts to create this directory if it does
not exist.
This doesn't make sense: sysfs is a view into kernel configuration, if the
kernel has support for the hugepage size, the directory will already be
there, if it doesn't, trying to create it won't help.
For the same reason, attempting to create the "nr_hugepages" file
itself is pointless, so there's no reason to call
OpenOptions::create(true).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
virtio-fs, instead of virtio-9p, is the default shared file system type
in case virtio-blk is not used.
Fixes: #3813
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Relying on virtio-block is the *only* way to use Firecracker with Kata
Containers, as shared FS (virtio-{fs,fs-nydus,9p}) is not supported by
Firecracker.
As configuration doesn't make sense to be exposed, we hardcode the
`false` value in the Firecracker configuration structure.
Fixes: #3813
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
snapcraft build is failing due to:
``
utils.mk:130: "WARNING: powerpc64le-unknown-linux-musl target is unavailable"
```
It seems to happen as powerpc64-unknown-linux-musl is a target that
although there's support for it, it's not exactly built or
automatically tested, at least according to:
https://doc.rust-lang.org/rustc/platform-support.htmlFixes: #3803
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Use `git clone --depth 1 ...` for QEMU and its dependencies
to speed up checkouts.
Fixes: #3799.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add the Kata Containers unit testing presentation I gave to the Kata
outreach students as this may be of some use to others.
Fixes: #3781
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Change `kata-monitor` to listen to port `8090` on the local interface
only by default.
> **Note:**
>
> This is a breaking change as previously it listened on all interfaces.
Fixes: #3795.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Removed redundant and duplicated build options to build
`kata-monitor` the same way as the other components:
- `CGO_ENABLED=0` is not necessary.
- `-buildmode=exe` is not necessary since `BUILDFLAGS` already sets the
build mode.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The current snap build for clh is broken as it's not aware of how to
build the binary from sources.
Instead of fixing it here, let's take advantage of the kata-deploy
script, which is capable of building from sources, and re-use it here.
Fixes: #3693
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This bump brings a fix on the build script, for ARM, so we can use the
very same build script everywhere.
The commit of our interest is b0324f85571c441f840e9bdeb25410514a00bb74:
```
scripts: Fix musl build on aarch64
Adding the missing TARGET_CC environment variable to get the build to
complete correctly.
Fixes#3776
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
With the ACPI PCI hotplug changes introduced in 2.3, QEMU >= 6.1 is required.
Remove unnecessary qemu version check in build script.
Fixes#3547
Signed-off-by: goodluckbot <tangbo_gl@hotmail.com>
This PR updates the README document by using the proper link for
the contributing guide as well as a misspelling.
Fixes#3791
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
- Mostly blank lines after `+build` -- see
https://pkg.go.dev/go/build@go1.14.15 -- this is, to date, enforced by
`gofmt`.
- 1.17-style go:build directives are also added.
- Spaces in govmm/vmm_s390x.go
Fixes: #3769
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
There are a few outstanding changes required to build the runtime on
Darwin.
Let's add a GitHub action to exercise build and unit tests of the
packages which we do expect to work. Eventually this should be dropped
and we can run any Darwin specific tests, or just add MacOS to the
matrix for our static check OSes.
Fixes: #3778
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
This utility function is also used to check the spec that will run in
the guest - no need for this to be linux specific.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Their types may differ on various host OSes, but
unix.Major|Minor always takes a uint64
Depends-on: github.com/kata-containers/tests#4516
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Add a stub for utils_darwin to facilitate building this package on
Darwin. We can probably drop this empty stub if we have better
abstraction for the various parts of virtcontainers that call it
today...
Fixes:# 3777
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
We need to convert them to uint64 as their types may differ on various
host OSes, but unix.Major|Minor takes a uint64 regardless.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Let's clarify that an error will be reported in case confidential_guest
is enabled, but the hardware where Kata Containers is running doesn't
provide the required feature set.
Fixes: #3787
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's use "Intel TDX" rather than just "TDX", as it can ease the
understanding of the terminology.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's mention the supported TEEs to be used with confidential guests.
Right now, Cloud Hyperisor supports only Intel TDX, used together with
TD Shim.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Nydusd uses a bufio.Scanner to check if nydusd process has
existed, but stderr/stdout passed to Cmd is self-created pipe,
this pipe will not be closed if the process start failing.
Use standard Cmd.StdoutPipe can close the stdout and kata shim
will detect the existence of the nydusd process, then call cmd.Wait to
reap the process' resources.
Fixes: #3783
Signed-off-by: bin <bin@hyper.sh>
The content about systemd in "/proc/self/cgroup" is as:
1:name=systemd:/kubepods/pod1815643d-3789-4e4e-aaf4-00de024912e1/0e15a65bd5f7b30a0b818d90706212354d8b3f0998a1495473c3be9a24706ccf
and in "/prol/self/mountinfo" is as:
30 29 0:26 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime shared:6 - cgroup cgroup rw,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd
The keys extracted from the two files are the same as "name=systemd". So no need to rename the key to "systemd".
Fixes: #3385
Signed-off-by: sailorvii <challengingway@hotmail.com>
A copy and paste mistake was made and the error on HotplugRemoveDevice()
should be about removal and not about addition.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
4c164afbac renamed extra_build_args to
features, but did it only in one place, leading to:
```
21:15:28 /home/jenkins/workspace/kata-containers-2.0-ubuntu-ARM-PR/go/src/github.com/kata-containers/kata-containers/tools/packaging/static-build/cloud-hypervisor/build-static-clh.sh: line 55: features: unbound variable
21:15:29 make[1]: *** [tools/packaging/kata-deploy/local-build/Makefile:30: cloud-hypervisor-tarball-build] Error 1
```
Fixes: #3775
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
removes --tags selinux handling in the makefile (part of it introduced here: d78ffd6)
and makes selinux configurable via configuration.toml
Fixes: #3631
Signed-off-by: Tanweer Noor <tnoor@apple.com>
Switching to the generic FilesystemSharer brings 2 majors improvements:
1. Remove container and sandbox specific code from kata_agent.go
2. Allow for non Linux implementations to provide ways to share
container files and root filesystems with the Kata Linux guest.
Fixes#3622
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
With the Linux implementation of the FilesystemSharer interface, we can
now remove all host filesystem sharing code from kata_agent and keep it
where it belongs: sandbox.go.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
This gathers the current kata agent and container filesystem sharing
code into a FilesystemSharer implementation.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Filesystem sharing here means the ability to share some parts of the
host filesystem with the guest. It's mostly about sharing files and
container bundle root filesystems.
In order to allow for different file and rootfs sharing implementations,
we define a FilesystemSharer interface.
This interface provides a preparation step, where concrete
implementations will be able to e.g. prepare the host filesysstem.
Then it provides 2 methods, one for sharing any file (regular file or a
directory) and another one for sharing a container root filesystem
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Let's enable TDX support for Cloud Hypervisor, using td-shim as its
desired firmware.
Fixes: #3632
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
"firmware" option was already present for a while, but it's never been
exposed to the configuration file before.
Let's do it now as it can be used, in combination with the newly added
confidential_guest option, to boot a guest VM using the so called
`td-shim`[0] with Cloud Hypervisor.
[0]: https://github.com/confidential-containers/td-shim
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
NVDIMM is also not supported with Confidential Guests and Virtio Block
devices should be used instead.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Similarly to VCPUs and Device hotplug, Confidential Guests also do not
support Memory hotplug.
Let's make it clear in the documentation and guard the code on both QEMU
and Cloud Hypervisor side to ensure we don't advertise Memory hotplug as
being supported when running Confidential Guests.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Similarly to VCPUs hotplug, Confidential Guests also do not support
Device hotplug.
Let's make it clear in the documentation and guard the code on both QEMU
and Cloud Hypervisor side to ensure we don't advertise Device hotplug as
being supported when running Confidential Guests.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As confidential guests do not support VCPUs hotplug, let's set the
"DefaultMaxVCPUs" value to "NumVCPUs".
The reason to do this is to ensure that guests will be started with the
correct amount of VCPUs, without giving to the guest with all the
possible VCPUs the host could provide.
One clear side effect of this limitation is that workloads that would
require more VCPUs on their yaml definition will not run on this
scenario.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
ConfidentialGuest is an option already present and exposed for QEMU,
which is used for using Kata Containers together with different sorts of
Guest Protections, such as TDX and SEV for x86_64, PEF for ppc64le, and
SE for s390x.
Right now we error out in case confidential_guest is enabled, as we will
be implementing the needed blocks for this as part of this series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is a small code refactor removing a deadcode based the checks
already done in the generic hypervisor abstraction.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The hypervisor code already defines 3 common kernel root params for the
following cases:
* NVDIMM
* NVDIMM without DAX support
* Virtio Block
As parameters used for cloud-hypervisor have an overlap with the ones
provided by the NVDIMM case, let's take advantage of that.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's bump the Cloud Hypervisor version to 5343e09e7b8db, as that brings
a few fixes we're interested in, such as:
* hypervisor, vmm: Handle TDX hypercalls with INVALID_OPERAND
- https://github.com/cloud-hypervisor/cloud-hypervisor/pull/3723
- This is needed for the TDX support on the cloud hypervisor driver,
which is part of this very same series.
* openapi: Update the PciBdf types
- https://github.com/cloud-hypervisor/cloud-hypervisor/pull/3748
- This is needed due to a change in a DeviceNode field, which would
cause a marshalling / demarshalling error when running with a
version of cloud-hypervisor that includes the TDX fixes mentioned
above.
* scripts: dev_cli: Don't quote $features_build
* scripts: dev_cli: Add --features option
- https://github.com/cloud-hypervisor/cloud-hypervisor/pull/3773
- This is needed due to changes in the scripts used to build Cloud
Hypervisor, which are used as part of Kata Containers CIs and
github actions.
Due to this change, we're also adapting the build scripts as part
of this very same commit.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
in TestHandleHugepages. On s390x, hugepage sizes must be set at boot, so
test with any that are present (default is 1M).
Depends-on: github.com/kata-containers/kata-containers#3770
Fixes: #3763
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
We introduced collection of sandboxes metadata from the CRI that will be
attached to the sandbox metrics: this will allow to immediately match
sandboxes metrics with CRI workloads.
Rename the symbols from *Kube* to *CRI* as the metadata will be there
every time pods are created through CRI, also if kubernetes is not
installed (e.g., 'crictl runp').
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
To resourcecontrol, and make it consistent with the fact that cgroups
are a Linux implementation of the ResourceController interface.
Fixes: #3601
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
We call it a ResourceController, and we make it not so Linux specific.
Now the Linux implementations is the cgroups one.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
We now rely on fs events only to update the sandbox cache. This is not
true anyway for sandboxes already present at kata-monitor startup: we
just retrieve the list and add them in the cache only when we get their
CRI metadata. If CRI metadata is not available we will never add them to
the sandbox cache.
Fix this by immediately adding the sandboxes we find at startup time to
the sandbox cache.
Fixes: #3705
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
Since kata-monitor now:
- relies on fs events *only* to update the sandbox cache
- adds CRI meta-data as labels (CRI pod name, namespace and uid)
it deserves a version bump.
Note that while we could let kata-monitor match the runtime version,
kata-monitor will usually work flawlessy with different kata shim
releases: so it makes sense to keep kata-monitor version separated.
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
The Developer-Guide.md build a custom kata agent with `x86_64-unknown-linux-musl`.
The `musl` should be changed by the system arch. The system arch is aarch64,
ppc64le and s390x, the musl should be changed. When the arch is ppc64le or s390x,
the musl should be replaced by the gnu.
Fixes: #3731
Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
CRI-O start shim process without setting TTRPC_ADDRESS,
that the forwarding events goroutine will get errors.
For CRI-O runtime, we can log the events to log file.
Fixes: #3733
Signed-off-by: bin <bin@hyper.sh>
This PR updates the contributing documentation link to the
one that is using kata 2.0
Fixes#3740
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The kata-deploy Dockerfile is based on CentOS 7, which has no s390x
support. Add an `IMAGE` argument to specify the registry, which still
defaults to CentOS, but e.g. ClefOS can be selected instead.
Other x86_64 assumptions are also removed. Other general simplicifations
are made.
This does not address the more general issue of #3723 -- what we're
doing here does not seem to be working with systemd >= something between
235-237.
Fixes: #3722
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
When using kata-deploy, no `containerd-shim-kata-v2` binary is deployed,
but we do deploy a `kata` runtime class, which seems very much
incosistent.
As the default configuration for kata-containers points to QEMU, let's
also use kata with QEMU as the default shim-v2 binary.
Fixes: #3228, #3734
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As kata with qemu has supported lazyload, so this pr aims to
bring lazyload ability to kata with clh.
Fixes#3654
Signed-off-by: luodaowen.backend <luodaowen.backend@bytedance.com>
The name of SYS_SUPPORTS_HUGETLBFS has been changed to
ARCH_SUPPORTS_HUGETLBFS which is being selected on default
by another kernel config.
More info- 855f9a8e87
Change applicable from v5.13.
Fixes: #3720
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
`tools/packaging/scripts/apply_patches.sh` uses `git apply $patch`, but
this will not apply to subdirectories. If one wanted to apply with
`git apply`, they'd have to run it with `--directory=...`
_relative to the Git tree's root_ (absolute will not work!). I suggest
we just use `patch`, which will do what we expected `git apply` would
do.
`patch` is also added to build containers that require it.
Fixes: #3690
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Let's take advantage of the fact that we've bumped to our kernel version
ot the 5.15 LTS and enable SGX by default, as it's present there.
Fixes: #3692
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's stop building the experimental kernel as, currently, we have
all the needed contents as part of the vanilla kernel.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
There's no need to build an experimental kernel for x86_64 as all the
bits which were part of the experimental one (SGX only, really) are now
part of the vanilla one.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the container runtime, we're never inspecting, adding or configuring
host networking endpoints.
Make sure we're always do that by wrapping addSingleEndpoint calls into
the pod network namespace.
Fixes#3661
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
This PR updates the limitations document by removing the docker
references belonged to kata 1.x and add as a limitation the
docker and podman support for kata 2.0
Fixes#3709
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Keep all the OS agnostic bits in the hypervisor.go and
hypervisor_ARCH.go files.
Fixes#3614
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Some of them (e.g. QEMU) can run on other OSes (e.g. Darwin) but the
current virtcontainers implementation is Linux specific.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Even though it's still actually defined as the QEMU upper bound,
it's now abstracted away through govmm.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Right now we're getting the info for the Cloud Hypervisor repo and
version, but we don't do anything with them, as those are not passed
down to the build script.
Morever, the build script itself gets the info from exactly the same
place when those are not passed, making those redundant.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Right now TDx support on Cloud Hypervisor is gated behind a "--features
tdx" flag. However, having TDx support enabled should not and does not
impact on the general usability of cloud-hypervisor.
As sooner than later we'll need kata-deploy binaries to be tested on a
CI that's TDx capable, for the confidential containers effort, let's
take the bullet and already enable it by default.
By the way, touching kata-deploy-binaries.sh as it's ensure the change
will be used in the following workflows:
* kata-deploy-push
* kata-deploy-test
* release
Fixes: #3688
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Various improvements to the top-level README file:
- Moved the following sections from the runtime's README to the
top-level README:
- License
- Platform support / Hardware requirements
- Added the following sections to the top-level README:
- Configuration
- Hypervisors
- Improved formatting of the Documentation section in the top-level
README.
- Removed some unused named links from the top-level README.
Also improvements to the runtime README:
- Removed confusing mention of the old 1.x runtime name.
- Clarify the binary name for the 2.x runtime and the utility program.
> **Note:**
>
> We cannot currently link to the AMD website as that site's
> configuration causes the CI static checks to fail. See
> https://github.com/kata-containers/tests/issues/4401Fixes: #3557.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Mount hugepage directories and configure the requested number of hugepages
dynamically by writing to sysfs files
Port from:
78b307b5bdFixes: #3342
Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Signed-off-by: bin <bin@hyper.sh>
We're downloading the released cloud-hypervisor binary from GitHub, but
we should also ensure we set the binary as executable.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Current hook process is handled by just calling
unwrap() on it, sometime it will cause panic.
By handling all Result type and check the error can
avoid panic.
Fixes: #3649
Signed-off-by: bin <bin@hyper.sh>
Envs contain null-byte will cause running hooks to panic,
this commit will filter envs and only pass valid envs to hooks.
Fixes: #3667
Signed-off-by: bin <bin@hyper.sh>
The OCI spec is very specific about it:
"The prestart hooks MUST be executed in the runtime namespace."
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
That allows us to amend those annotations with information that could be
used when running those hooks.
For example nerdctl will use those annotations to resolve the networking
namespace path in where to run the CNI plugin, i.e. the created pod
networking namespace.
Fixes#3629
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Update the `kata-manager.sh` README to recommend users view the
available options before running the script.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Added CLI options to the `kata-manager.sh` script to:
- Force installation
- Disable cleanup (retain downloaded files)
- Only install Kata (don't consider containerd).
> **Note:**
>
> This change introduces a subtle behaviour difference:
>
> - Previously, the script would error if containerd was already installed.
>
> - Now, the script will detect the existing installation and skip
> trying to install containerd.
>
> This new behaviour makes more sense for most users but if you wish
> to use the old behaviour, you (now) need to run the script specifying
> the `-f` (force) option.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
`kata-manager.sh` improvements for containerd:
- Fixed containerd default branch (which is now `main`).
- Only install service file if it doesn't already exist.
- Enable the containerd service to ensure it can be started.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
For consistency with the rest of the script force the creation of a
symbolic link for containerd in `kata-manager.sh`.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Actually make use of the `requested_version` parameter in
`kata-manager.sh` and added a comment.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Fix bug introduced inadvertently on #3330 which fixes the Kata
installation, but unfortunately breaks installing containerd.
The new approach is to check that the download URL matches a
project-specific regular expression.
Also improves the architecture test to handle the containerd
architecture name (`amd64` rather than `x86_64`).
Fixes: #3674.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The static tar archive published on GitHub (now) contains `./` which is
being being flagged as an "unknown path" and resulting in the
`kata-manager.sh` script failing.
Partially fixes: #3674.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This PR removes the docker run and shared memory segment from the
limitations document as for kata 2.0 we do not support docker
and this is not longer valid.
Fixes#3676
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's allow passing down a build flag to cargo, when building Cloud
Hypervisor.
By doing this we allow calling this script with:
```
extra_build_flags="--features tdx" ./build-static-clh.sh
```
Fixes: #3671
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The current code will always pull the release binaries in case the
version requested by Kata Containers matches with a released version.
This, however, has a limitation of preventing users / CIs to build
cloud-hypervisor from source for a reason or another, such as passing a
specific build flag to cloud-hypervisor.
This is a pre-req to solving
https://github.com/kata-containers/kata-containers/issues/3671.
While here, a small changes were needed in order to improve readability
and debugability of why we're building something from the sources rather
than simply downloading and using a pre-built binary.
Fixes: #3672
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
German Maglione, one of the current virtio-fs developers, has brought to
our attention that using "announce-submounts" could help us to prevent
inode number collisions.
This feature was introduced a year ago or so by Hanna Reitz as part of
the 08dce386e77eb9ab044cb118e5391dc9ae11c5a8, and as we already mandate
QEMU >= 6.1.0, let's take advantage of that.
Fixes: #3507
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When building with `ARCH=x86_64`, the previous `Makefile` will use it
without checking and cause:
Makefile:319: *** "ERROR: No hypervisors known for architecture x86_64 (looked for: acrn firecracker qemu cloud-hypervisor)". Stop.
This commit fix the above issue by checking `ARCH` no matter where it
is assigned.
Fixes: #3444
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
The distro constraint parses os release files, which may not contain
distro version(VERSION_ID field), for example rolling release distributions
like Debian testing, archlinux.
These distro constraints are not used anyway, so removing them instead
of fixing the complex version detection.
Fixes: #1864
Signed-off-by: Shengjing Zhu <zhsj@debian.org>
Let's update cloud-hypervisor to a version that exposes the TDx support
via the OpenAPI's auto-generated code.
Fixes: #3663
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
to cover a Red Hat (adjacent) rootfs with great cross-platform compatibility
and a workable release cadence. The previous CentOS & Fedora workflows are
simplified.
Also remove unnecessary `/usr/share` files as on Ubuntu and mark Alpine
as unuspported on ppc64le (due to musl, for a while already).
Fixes: #3340
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
We install Rust in the build containers, but we also install Rust in
`rootfs.sh` if it is missing. It makes sense to install Rust in the build
containers so it does not have to be installed every time, but for that check
to work on non-login shells, we should source `.cargo/env` before running it.
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Change the variables `mountTypeFieldIdx := 8`, `mntDestIdx := 4` and `netNsMountType := "nsfs"` to const.
And unify the variable naming style, modify `mntDestIdx` to `mountDestIdx`.
Fixes: #3646
Signed-off-by: yaoyinnan <yaoyinnan@foxmail.com>
Add support for building TDX kernel from github.com/intel/tdx
To build a guest kernel that supports Intel TDx run:
```
./build-kernel.sh -s -x tdx -d setup
./build-kernel.sh -s -x tdx -d install
```
fixes#3650
Signed-off-by: Julio Montes <julio.montes@intel.com>
Pulling image is the most time-consuming step in the container lifecycle. This PR
introduse nydus to kata container, it can lazily pull image when container start. So it
can speed up kata container create and start.
Fixes#2724
Signed-off-by: luodaowen.backend <luodaowen.backend@bytedance.com>
Relative links within this repository allow for easier navigation to
the corresponding file / directory in the current commit / for the
selected version.
Link text was slightly changed / fixed in
- docs/Unit-Test-Advice.md
- docs/how-to/how-to-run-docker-with-kata.md
Fixes#3045
Signed-off-by: Daniel Höxtermann <daniel@hxtm.dev>
In commit 78dff468bf1 we introduced logic to rewrite PCIDEVICE_ environment
variables for the container so that they contain correct addresses for the
Kata VM rather than for the host. Unfortunately, we never actually invoked
the function to do this.
It turns out we need to do this not only at container creation time, but
also for environment variables supplied to processes exec-ed into the
container after creation (e.g. with crictl exec). Add calls to make both
those updates.
fixes#3634
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
add_devices() generates a mapping of host to guest PCI addresses which is
used to update some environment variables for the workload. Currently it
just does this locally, but it turns out we're going to need the same map
again in order to correct environment variables for processes exec-ed into
the existing container.
Move the map to the sandbox structure so we can keep it around for those
later uses.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This function updates PCIDEVICE_ environment variables (such as those
supplied by the Kubernetes SR-IOV plugin) in the OCI spec to be correct
for the Kata VM, rather than for the host.
We neglected to actually call this function, however, and it turns out that
when we do, we need to do things slightly different. We actually need to
adjust envionment variables both in the OCI spec when creating a container
and also in the variables supplied for exec-ing a new process within an
existing container.
Adjust the function so that it can be used for both these cases.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We don't need to call NewNetwork() twice, and we can have the VM factory
case return immediatly. That makes the code more readable.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Move the netlink dependent code into network_linux.go.
Other OSes will have to provide the same functions.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
And only have AddEndpoints/RemoveEndpoints for all cases (single
endpoint vs all of them, hotplug or not).
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
We are converting the Network structure into an interface, so that
different host OSes can have different networking implementations for
Kata.
One step into that direction is to rename all the Network structure
fields and methods to something that is less Linux networking namespace
specific. This will make the Network interface naming consistent.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
We are replacing the NetworkingNamespace structure with the Network
one, so we should have the hypervisor interface switching to it as well.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Remove unused parameters.
Reduce the number of parameters by deriving some of them (e.g. a
networking config) from their outer structure (e.g. a Sandbox
reference).
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Endpoints creations, attachement and hotplug are bound to the networking
namespace described through the Network structure.
Making them Network methods is natural and simplifies the code.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
For simplicity sake, there should only be one networking structure per
sandbox, as opposed to two (Network and NetworkingNamespace) currently.
This commit start expanding the Network structure in order to eventually
make it the single representation of a virtcontainers sandbox
networking.
Signed-off-by: Samuel Ortiz <s.ortiz@apple.com>
Runtime now accepts both `1` and `Y` as valid values for
kvm_amd module parameter kvm_amd.sev.
Fixes#3273
Signed-off-by: Pierre Kohler <pierre.kohler@cysec.systems>
Update qemu version of snap for arm to 6.1.0 thus the arch specific qemu
version for arm needs clean up.
Fixes: #3627
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
This reverts commit 7a879164bd, as it's
been proved that re-triggering the checks at every single change is more
painful than having to close / re-open a PR in case we ever use the
`force-skip-ci` label again.
Fixes: #2804
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The build root container is switched from CentOS 8 to Stream 8 as
the former reached EOL.
Fixes#3605
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Since we are using this to exercise any changes to osbuilder or
packaging scripts, let's make sure that we only run the test in that
case.
Similarly, don't run for every single push. Just run this workflow for
pull requests.
Fixes: #3594
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
On s390x, skip adding a virtio-rng device. The on-chip CPACF provides
entropy instead. For Confidential Containers, when using Secure
Execution, entropy attacks on virtio-rng are mitigated.
Fixes: #3598
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
firmware can be split into FIRMWARE_VARS.fd (UEFI variables as
configuration) and FIRMWARE_CODE.fd (UEFI program image). UEFI
variables can be customized per each user while UEFI code is kept same.
fixes#3583
Signed-off-by: Julio Montes <julio.montes@intel.com>
Since we are already checking that only an admin is triggering the job,
let's go ahead and make sure we are testing against the PR itself. This
will ensure that we are exercising changes to kata-deploy tooling, which
is important for this test.
While at it, cleanup and simplify some of the tarball creation.
Fixes: #3586
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
This PR removes the images belonged to the Zun documentation at
the use cases directory.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR removes the zun documentation use case with kata containers mainly
because is not longer valid as it is using as a reference docker with
clear containers 2.0 which are not longer being supported and it is also
using docker to test kata with openstack zun and docker is also not supported.
Fixes#3581
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
critools v1.23.0 has been released a few days ago. As we're already
bumping kubernetes, and CRI-O, let's also update critools.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As done for kubernetes, CRI-O should also be bumped to its 1.23 release
so those are in sync.
Fixes: #3481
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
- virtcontainers: Enable initrd for Cloud Hypervisor
- versions: update Rust to 1.58.1
- Sandbox sizing feature
- kata-deploy: Fix the tag replacement logic
- docs: Update networking details in the architecture doc
- Fix and re-enable s390x GoVMM tests
- runtime: fix handling container spec's memory limit
- ci: Pass function arguments in static-checks.sh
- docs: Remove docker run and sysctl limitation
- runtime: update runc and image-spec dependencies
- agent: resolve unused variables in tests
- Upgrade to Cloud Hypervisor v21.0
- runtime: rectify passing empty options to -ldflags
- osbuilder: Remove libseccomp from Dockerfile
- agent: fix the issue of creating new namespaces for agent
- docs: Remove kata-pkgsync reference
- docs: Redirect glossary to the wiki
- workflows: Use base instead of head ref for kata-deploy-test
- govmm: Use it from our own repo
- tools: Fix groupname if it differs from username
- workflows: Fix typo in kata-deploy-push action
- release: Escape backticks in Libseccomp Notices
- packaging: Remove kata-pkgsync tool
- govmm: Bring the project in
- version: bump to kubernetes 1.23
- vendor: update govmm
- workflows: Ensure force-skip-ci skips all actions
- runtime: -Wl,--s390-pgste for s390x
- workflows: Use the correct branch ref on test kata-deploy
- update apiVersion
- scripts: Use shebang /usr/bin/env bash
- packaging: Make kernel config accessible to guest
- docs: fix a typo in host-cgroups.md doc
- qemu: add support for SGX
- experimentally enable the vcpu-hotplug for arm in qemu side
- Remove all the non-tested rootfs
- docs: Remove ccloudvm reference
- runtime: Provide protection for shared data
- kata-deploy: validate conf file can be created
- runtime: it should rollback when failed in Sandbox AddInterface
- libs: add some generated files to .gitignore
- runtime: close span before return from function in case of error
- packaging: Remove ccloudvm instructions and script
- docs: Default machine type is q35 meanwhile
- CI: Revert "CI: Switch to a mirror as gnu.org is down"
- agent: fix the broken protobuf generation code
- packaging: Remove obs packages testing for kata 2.0
- runtime: Remove docker comments for kata 2.0 configuration.tomls
- docs: fix agent proto file path
- qemu: update readonly flag for block devices
- qemu: only set wait parameter for server mode socket based char device
- qemu: Fix 32 bit int overflow in test file
- qemu: Add support for legacy serial device
- qemu: Remove -realtime in favor of -overcommit
- Add clean shutdown support
- govmm/qemu: Let IO/memory reservations be specified for bridge devices
- QMP: Add ExecuteBlockdevAddWithDriverCache
- qemu: Fix iommu_platform for CCW
- qemu: Add credentials to qemu Cmd
- Don't use deprecated 'props' argument to QMP 'object-add'
- Use 'host_device' driver for blockdev backends
- add support for "sandbox" feature to qemu
- qemu: support read-only nvdimm
- Support golang 1.16
- qemu: Consistent parameter building
- qemu: Allow hot-plugging memory devices on PCI bridges
- qemu: Add support for PEF
- qemu: Add support for Secure Execution
- qemu: VhostUserDevice CCW device numbers
- qmp: remove chatty log
- Fix qemu commandline issue with empty romfile
- qemu: add support for tdx-guest object
- qemu: Append memory backend for non-DIMM setups
- qemu: add support for device loaders
- qemu: support QEMU 6
- qmp: Add ro argument for block-device hotplug funcs
- qemu: add arm64 to support list of dimm
- qemu: enable "-pflash"
- qemu: add pvpanic and dump guest memory support
- Add serial ID to blk device
- Make fw_cfg a slice
- contributors: remove CONTRIBUTORS.md file
- misc: Update for new GitHub organisation name
- qemu: add fw_cfg flag to config
- Add qom-get function
- typo fix
- Add support for hot-plugging IBM Adjunct Processor (AP) devices
- github: enable github actions
- travis: Run coveralls after success
- qemu: add iommu_platform knob for qemuParams
- qemu: Add NoReboot config Knob for qemuParams
- Add multidevs option to fsdev
- qemu/qmp: use boolean type for the vhost
- qemu: add IOMMU Device
- Enable Numa support for Power (ppc64le) architecture
- qemu: Add max_ports option to virtio-serial device
- Add rt clock definition for rtc clock in qemu
- qemu: Add microvm machine type support
- qemu: add pmem flag to memory-backend-file
- Refactor code to support multiple virtio transports at runtime
- qemu: Don't set ".cache-size=" when CacheSize is 0
- qemu: Add pcie-root-port device support.
- qmp: Add ExecMemdevAdd and ExecQomSet API
- qmp: add ExecutePCIVhostUserDevAdd and ExecuteChardevDel to hotplug vhost-user device
- s390x: add s390x travis support
- virtio-blk: Add support for share-rw flag
- s390x: dimm not supported
- improve qemu interaction
- qmp: support command 'query-qmp-schema'
- qmp: add checks for the CPU toplogy
- qemu: support x86 SMP die
- Support x-pci-vendor-id and x-pci-device-id pass to qemu
- Support for virtio-blk-ccw
- Allow sharing of memory backend file
- qemu: add migration incoming defer support
- qmp: add virtio-blk multiqueue
- qemu: fix the issue of wrong driver for VirtioBlock
- qemu: use MiB instead of Gib for virtio-fs cache size
- qemu/qmp: re-implement mainLoop
- qemu/qmp: fix readLoop() reuse scanner.Bytes() underlying array problem
- govmm: add VhostUserFS vhost-user device type
- qmp: Conditionally pass threadID and socketID when CPU device add
- Fix travis
- qmp: Add nvdimm support
- qemu: Allow disable-modern option from QMP
- qmp: Output error detail when execute QMP command failed
- Run tests for the s390x build
- Contributors: Add Clare Chen to CONTRIBUTORS.md
- Verify govmm builds on s390x
- Contributors: Add my name
- qemu: Add s390x support
- Update file headers , CONTRIBUTING.md and add CONTRIBUTORS.md
- qmp: fix mem-path properties for hotplug memory.
- qemu: change Context ID for Vsock to uint64
- qemu/qmp: preparation for s390x support
- qemu/qmp: add new function ExecuteBlockdevAddWithCache
- qemu: add support for pidfile option
- qemu: Fix virtio-net-pci QMP command
- qemu: Add support for romfile option
- Update guidelines on security issue reporting
- qemu: Add virtio-balloon device suppport.
- qemu: Show full path to qemu binary at launch time
- qemu: Fix the support of PCIe bridge
- qmp: add ExecuteQueryMigration
- qemu: skip setting system memory if it is set via dimm device
- qmp: add "query-cpus" support
- qemu/qmp: add vfio mediated device support on root bus
- qemu/image: Reduce permissions of .iso creation dir
- qemu/qmp: nic can works without vhost
- qemu: Add rng device .
- qemu/qmp: support query-memory-devices qmp command.
- govmm: modify govmm to be compatible with qemu 2.8
- qemu/qmp: support hotplug a nic whose qdisc is mq
- qmp: Remind users that you must first call ExecuteQMPCapabilities()
- qemu/qmp: Add netdev_add with chardev support
- Add some negative test cases
- qemu: Use the supplied context.Context for launching
- disk: Add --share-rw option for hotplugging disks
- qemu/qmp: add vfio mediated device support
- qemu: Do not try and generate invalid RTC parameters
- qemu/qmp: add addr and bus to hotplug vsock devices
- qemu/qmp: add function for hotplug network by fds
- qemu/qmp: implement functions to hotplug chardevs and serial ports
- qemu: add vhostfd and disable-modern to vsock hotplug
- Add two additional static analysis tools to the travis builds
- qemu/qmp: implement function for hotplug network
- qemu: add vhostfd and disable-modern to vhost-vsock-pci
- qemu/qmp: implement function to hotplug vsock-pci
- Add APIs to enable vm templating
- qemu: Add qemu parameter for PCI address for a bridge.
- Add ability to associate a SCSI controller device with an iothread
- qemu: add initrd support
- qemu: add DisableModern to SCSIController
- qemu: add extra options for the machine type
- scsi: Add function to send device_add qmp command for a scsi device
- Compute coverage statistics for unit tests in Travis builds
- scsi: Add a scsi controller device
- qemu: Add VSOCK support
- Vhost-user: add block device support
- qemu: Add maxcpus attribute to -smp
- Add badges to the README.md file
- Enable Travis builds
- qemu: introduce vhost-user handling
bcce1a19 versions: update Rust to 1.58.1
7c956e0d virtcontainers: Enable initrd for Cloud Hypervisor
aa3fae13 kata-deploy: Fix the tag replacement logic
8cde5413 runtime: introduce static sandbox resource management
13eb1f81 docs: describe vCPU handling when hotplug is unavailable
c3e97a0a config: updates to configuration clh, fc toml template
75ae5361 docs: Update networking details in the architecture doc
fc0e0951 runtime: fix handling container spec's memory limit
7af40fbc docs: Remove docker run, sysctl and docker daemon limitations
17211979 ci: Pass function arguments in static-checks.sh
5643c6dc runtime: update runc and image-spec dependencies
2f37165f govmm: Unite VirtioNet tests
4a428fd1 govmm: readonly=on in s390x blkdev test
79ecebb2 govmm: TestAppendPCIBridgeDevice et al. on !s390x
dc285ab1 govmm: Remove unnecessary comma in iommu_platform
d23f2eb0 govmm: Revert "govmm: s390x: Skip broken tests"
f52ce302 runtime: rectify passing empty options to -ldflags
2d799cbf virtcontainers: clh: Re-generate the client code
7e15e99d versions: Upgrade to Cloud Hypervisor v21.0
9c2f1de1 docs: Remove kata-pkgsync reference
df6ae1e7 osbuilder: Remove libseccomp from Dockerfile
0338fc65 docs: Redirect glossary to the wiki
3924470c workflows: Use base instead of head ref for kata-deploy-test
5ce9011a govmm: s390x: Skip broken tests
8bcaed0b govmm: Adapt license headers to kata-containers
6dd65779 govmm: Ignore govet checks, at least for now
de678a3a govmm: Remove non-relevant top files
ec6655af govmm: Use govmm from our own pkg
8cc088b5 packaging: Remove kata-pkgsync tool
a8b66de5 release: Escape backticks in Libseccomp Notices
c3785f66 workflows: Fix typo in kata-deploy-push action
f4a4c3c7 version: bump to kubernetes 1.23
49223e67 runtime: remove enable_swap option
7a879164 workflows: Ensure a label change re-triggers the actions
d87ab14f workflows: Ensure force-skip-ci skips all actions
5285ac2b runtime: -Wl,--s390-pgste for s390x
fc646434 workflows: Use the correct branch ref on test kata-deploy
e347694f tools: Fix groupname if it differs from username
41e0c414 vendor: update govmm
a5829a29 docs: fix a typo in host-cgroups.md doc
92773170 agent: resolve unused variables in tests
8939b0f8 qemu: add support for SGX
2d0ec00a Qemu: Enable the vcpu-hotplug for arm
e22a4e2a packaging: Make kernel config accessible to guest
adffd3f8 scripts: Use shebang /usr/bin/env bash
e4b7a12b qat: Add Debian to the distro examples
6979d5be osbuilder: Remove gentoo rootfs-builder
22c1a093 osbuilder: Remove suse rootfs-builder
85dd5873 osbuilder: Remove fedora rootfs-builder
06fae29f osbuilder: Remove centos rootfs-builder
01005c5a docs: Remove ccloudvm reference
878ab93c runtime: Provide protection for shared data
ac7acbf8 kata-deploy: validate conf file can be created
7e2bc4d7 packaging: Remove ccloudvm instructions and script
85f5ae19 runtime: close span before return from function in case of error
106df33f libs: add some generated files to .gitignore
b133a236 runtime: it should rollback when failed in Sandbox AddInterface
7f546748 CI: Revert "CI: Switch to a mirror as gnu.org is down"
c486c2ca agent: fix the broken protobuf generation code
f6cdf464 docs: Default machine type is q35 meanwhile
b48322d4 packaging: Remove obs packages testing for kata 2.0
ad16d75c runtime: Remove docker comments for kata 2.0 configuration.tomls
905e124b docs: fix agent proto file path
ea1a1738 agent: fix the issue of creating new namespaces for agent
b17f0739 qemu: update readonly flag for block devices
b5b9de1d kata-deploy: Update API Version of RuntimeClass to v1
f971801b qemu: only set wait parameter for server mode socket based char device
82cc01d2 qemu: Fix 32 bit int overflow in test file
1d1a2313 qemu: Add support for legacy serial device
9a2bbeda qemu: Remove -realtime in favor of -overcommit
fe83c208 qemu: Add support for --no-shutdown Knob
1ed52714 qmp: wait for POWERDOWN event in ExecuteSystemPowerdown()
de039da2 govmm/qemu: Let IO/memory reservations be specified for bridge devices
5c7998db QMP: Add ExecuteBlockdevAddWithDriverCache
3a9a6749 qemu: Add credentials to qemu Cmd
d27256f8 qmp: Don't use deprecated 'props' field for object-add
d8cdf9aa qemu: Drop support for versions older than 5.0
18352c36 qemu: Fix iommu_platform for vhost user CCW
1b021929 Use 'host_device' driver for blockdev backends
9518675e add support for "sandbox" feature to qemu
335fa816 qemu: fix golangci-lint errors
61b63787 .github/workflows: reimplement github actions CI
9d6e7970 go: support go modules
0d21263a qemu: support read-only nvdimm
ff34d283 qemu: Consistent parameter building
0e19ffb6 qemu: Allow hot-plugging memory devices on PCI bridges
c135681d qemu: Add support for PEF
03b55ea5 qemu: Add support for Secure Execution
7a367dc0 qemu: Simplify (Object).Valid()
a6cec2d3 qemu: add support for SevGuest object
abd3c7ea qemu: VhostUserDevice CCW device numbers
3eaeda7f qemu: Refactor vhostuserDev.QemuParams
511cf58b Fix qemu commandline issue with empty romfile
b3eac95b qmp: remove frequent, chatty log
31418940 qemu: add support for tdx-guest object
4b136f3f qemu: Append memory backend for non-DIMM setups
6213dea4 qemu: support QEMU 6
0d47025d qemu: add support for device loaders
e2eb549f qmp: Add ro argument for block-device hotplug funcs
0592c825 qemu: add arm64 to support list of dimm
2079c15c qemu: enable "-pflash"
b8cd7059 qmp: add dump-guest-memory support
d7836877 qemu: add pvpanic device to get GUEST_PANICKED event
43d774d2 Add serial to blk device
8cb8b24c Make fw_cfg a slice
cb0d3391 contributors: remove CONTRIBUTORS.md file
29ba5a90 qemu: add fw_cfg flag to config
9f309c2a misc: Update for new GitHub organisation name
3d46d08a Add qom-get function
39c372a2 Add support for hot-plugging IBM VFIO-AP devices
f5bdd53c travis: disable amd64 jobs
1af1c0d7 github: enable github actions
4831c6e0 travis: Run coveralls after success
cf0f05d2 qemu: add iommu_platform knob for qemuParams
6645baf2 qemu: Add NoReboot config Knob for qemuParams
abca6f3c Add multidevs option to fsdev
cc538766 qemu/qmp: use boolean type for the vhost
e57e86e2 qemu: add IOMMU Device
b2aa0225 Enable Numa support for Power (ppc64le) architecture
29529a5d Add rt clock definition for rtc clock in qemu
0e98b613 qemu: Add max_ports option to virtio-serial device
787c86b7 qemu: Add microvm machine type support
5378725f qemu: add pmem flag to memory-backend-file
3700c55d qemu: add block device readonly support
88a25a2d Refactor code to support multiple virtio transports at runtime
2ee53b00 qemu: Don't set ".cache-size=" when CacheSize is 0
f1252f6e qemu: Add pcie-root-port device support.
6667f4e9 qmp_test: Add TestExecMemdevAdd and TestExecQomSet
201fd0ae qmp: Add ExecMemdevAdd and ExecQomSet API
e04be2cc qmp: add ExecutePCIVhostUserDevAdd API
13aeba09 qmp: support command 'chardev-remove'
6d6b2d88 s390x: add s390x travis support
175ac499 typo fix
cb9f640b virtio-blk: Add support for share-rw flag
9463486d s390x: dimm not supported
164bd8cd test/fmt: drop extra newlines
73555a40 qmp: add query-status API
234e0edf qemu: fix memory prealloc handling
30bfcaaa qemu: add debug logfile
79e0d533 qmp: support command 'query-qmp-schema'
68cdf64f test: add cpu topology tests
e0cf9d5c qmp: add checks for the CPU toplogy
a5c11908 qemu: support x86 SMP die
8fd28e23 Support x-pci-vendor-id and x-pci-device-id pass to qemu
713d0d94 s390x: add virtio-blk-ccw type
65cc343f test: add devno in the tests for s390x
9cf98da0 s390x: add devno support
0c900f59 Allow sharing of memory backend file
f695ddf8 qemu: add migration incoming defer support
f0f18dd0 qmp: add virtio-blk multiqueue
7d3deea4 qemu: Add a virtio-blk-pci device driver support
058cda06 qemu: use MiB instead of Gib for virtio-fs cache size
694a7b1c qemu/qmp: re-implement mainLoop
5712b119 qemu/qmp: fix readLoop() reuse scanner.Bytes() underlying array problem
3c84b1da govmm: add VhostUserFS vhost-user device type
4692f6b9 qmp: Conditionally pass threadID and socketID when CPU device add
1f51b438 Update the versions of Go used to build GoVMM
ad310f9f Fix staticcheck S1023
932fdc7f Fix staticcheck S1023
cb2ce933 Fix staticcheck S1008
f0172cd2 Fix staticcheck (S1002)
5f2e630b Fix staticcheck (S1025)
4beea513 Fix staticcheck (ST1005) errors
97fc3435 contributors: add my name
c891f5f8 qmp: Add nvdimm support
f9b31c0f qemu: Allow disable-modern option from QMP
d6173077 Run tests for the s390x build
b36b5a8f Contributors: Add Clare Chen to CONTRIBUTORS.md
b41939c6 Contributors: Add my name
dab4cf1d qmp: Add tests
5ea6da14 Verify govmm builds on s390x
ee75813a contributors: add my name
c80fc3b1 qemu: Add s390x support
ca477a18 Update source file headers
e68e0056 Update the CONTRIBUTING.md
2b7db547 Add the CONTRIBUTORS.md file
b3b765cb qemu: test Valid for Vsock for Context ID
3becff5f qemu: change of ContextID from uint32 to uint64
f30fd135 qmp: Output error detail when execute QMP command failed
7da6a4c7 qmp: fix mem-path properties for hotplug memory.
e4892e33 qemu/qmp: preparation for s390x support
110d2fa0 qemu/qmp: add new function ExecuteBlockdevAddWithCache
a0b0c86e qmp_test: Change QMP version from 2.6 to 2.9
10c36a13 qemu: add support for pidfile option
9c819db5 qemu: Fix virtio-net-pci QMP command
7fdfc6a4 qemu: Add support for romfile option
e74de3c7 Update guidelines on security issue reporting
ec83abe6 qemu: Add virtio-balloon device suppport.
46970781 qemu: Show full path to qemu binary at launch time
ef725050 qemu: Fix the support of PCIe bridge
56f645ea qmp: add ExecuteQueryMigration
a429677a govmm: fix memory prealloc
1130aab8 qmp: add "query-cpus" support
de5d2788 qemu/qmp: add vfio mediated device support on root bus
de00d7a6 qemu/image: Reduce permissions of .iso creation dir
1a1fee75 qemu/qmp: nic can works without vhost
6c3d84ea qemu: Add virtio RNG device.
b16291cf qemu/qmp: support query-memory-devices qmp command.
ce070d11 govmm: modify govmm to be compatible with qemu 2.8
0286ff9e qemu/qmp: support hotplug a nic whose qdisc is mq
8515ae48 qmp: Remind users that you must first call ExecuteQMPCapabilities()
21504d31 qemu/qmp: Add netdev_add with chardev support
ed34f616 Add some negative test cases for qmp.go
17cacc72 Add negative test cases for qemu.go
2706a07b qemu: Use the supplied context.Context for launching
e46092e0 qemu: Do not try and generate invalid RTC parameters
fcaf61dc qemu/qmp: add vfio mediated device support
4461c459 disk: Add --share-rw option for hotplugging disks
68519998 qemu/qmp: add addr and bus to hotplug vsock devices
10efa841 qemu/qmp: add function for hotplug network by fds
80ed88ed qemu/qmp: implement function to hotplug serial ports
ca46f21f qemu/qmp: implement function to hotplug character devices
03f1a1c3 qemu/qmp: implement getfd
84b212f1 qemu: add vhostfd and disable-modern to vsock hotplug
12dfa872 qemu/qmp: implement function for hotplug network
3830b441 qemu: add vhostfd and disable-modern to vhost-vsock-pci
f700a97b qemu/qmp: implement function to hotplug vsock-pci
4ca232ec qmp_test: Fix Warning and Error level logs
430e72c6 qemu,qmp: Enable gas security checker
ffc06e6b qemu,qmp: Add staticcheck to travis and fix errors
54caf781 qmp: add hotplug memory
e66a9b48 qemu: add appendMemoryKnobs helper
8aeca153 qmp: add migrate set arguments
a03d4968 qmp: add set migration capabilities
0ace4176 qemu: allow to set migration incoming
723bc5f3 qemu: allow to create a stopped guest
283d7df9 qemu: add file backed memory device support
30aeacb8 qemu: Add qemu parameter for PCI address for a bridge.
9130f375 scsi: Allow scsi controller to associate with an IO thread.
a54de183 iothread: Add ability to configure iothreads
0c0ec8f3 qemu: add initrd support
68f30718 qemu: add DisableModern to SCSIController
693d9548 qemu: add options for the machine type
3273aafd scsi: Add function to send device_add qmp command for a scsi device
6d198b8a Compute coverage statistics for unit tests in Travis builds
3a31da32 scsi: Add a scsi controller device
5316779d qemu: Add VSOCK support
f5655366 vhost-user: add blk device support
e9e27673 vhost-user: updating comments for accuracy, rename device field
8fe57236 qemu: Add maxcpus attribute to -smp
3baa7765 Add badges to the README.md file
d74e3b66 Fix errcheck failures in the unit tests
db60e32f Enable Travis builds
9cb47fc0 Add .gitignore file.
a8aaf534 Add project documentation
57aafb56 Remove all references to and dependencies on ciao
27709fce Move files to the qemu folder
48feb29f qemu: introduce vhost-user handling
b8ddd244 qemu: Add function to list hotpluggable CPUs
8c428ed7 qemu: Add function to hotplug CPUs
24b14059 qemu: Add functions to process QMP response
e39da6ca qmp: Add support for hot plugging VFIO devices on PCI(E) bridges
bc030d13 qemu: Add a SysProcAttr parameter to CreateCloudInitISO
11977072 qemu: Add a SysProcAttr parameter to LaunchCustomQemu
b639da45 qemu: Add function to hotplug vfio device
7e5614b8 Networking: Add vhost fd support
14316ce0 qemu/qmp: Implement function to hot plug PCI devices
83485dc9 qemu: Implement Bridge struct
cfa8a995 Networking: Add support for handling macvtap interfaces
83126d3e bios: add support for custom bios
3da2ef9d QEMU: Knobs: Huge Page Support: Add support for huge pages
9bfa7927 vfio: Add ability to pass VFIO devices to qemu
a70ffd19 Build: Fix the build after repo move.
0c206170 Knobs: Modify the behaviour of the Mlock knob.
ddee41d5 QEMU: Enable realtime options
4ecb9de5 qemu: Add support for memory pre-allocation
1fbe6c5d qmp: Update block device deletion for newer versions of qemu
e74aeef1 qemu: Add disable-modern option for virtio devices
8d617ff5 qemu: Update virtio-net-pci command line
25a2dc8f qemu: Update blockdev-add qmp command to support newer qemu versions
d4f77103 misc: Remove some of the code flagged by unused linter
a1600dc1 misc: Remove unused fields identified by structcheck
58a835e6 misc: Remove unused variables identified by varcheck
d48b5b5f qemu: Add PCI option to the NetDevice
a84228ae qemu: Document how cancelling works.
1e7202a5 qemu: Fix spelling error in qmp_test.go
c6f33453 qemu: Fix command cancelling.
a8a798b0 qemu, ciao-launcher: Move ConfigDrive ISO creation code to qemu
30cf1163 Add missing bus parameter for a CharDevice
2aa5f5a3 qemu: Add support for serial port addition
6fe338d6 qemu: Support creating multiple QMP sockets
992b861e qemu: Add the daemonize qemu option to the Knobs structure
997cb233 qemu: Remove dead code
e555f565 qemu: Add support for socket based consoles
eae8fae0 qemu: Fix security model typo
db067857 qemu: Make Config's FDs field private
12f6ebe3 qemu: Embed the qemu parameters into the Config structure
e193a77b qemu: Add support for block devices
3908185c qemu: Add MACVTAP support
6d7dfa04 qemu: Get rid of the Driver structure
cc9cb33a qemu: Add QMPSocket specific type
2d736d71 qemu: Add RTC specific types
e543c338 qemu: Probe each qemu device with a driver
eda8607c qemu: Add netdev options to the Device structure
4780e237 qemu: Add multi-queue and vhost definitions to NetDevice
137e7c72 qemu: Add a NetDevice slice to the Config structure
c0e2aaca qemu: Add one unit test for the Config strings
5ba8ef79 qemu: Add QMP socket unit tests
7b2f7eb5 qemu: Add Memory and SMP unit tests
2ea9b9a3 qemu: Add a Kernel unit test
8e495f6e qemu: Add a Knobs unit test
8aeb3d45 qemu: Add an Object unit test
38e041dc qemu: Add Device unit tests
54d32c24 qemu: Add parameters adding unit tests
ebfa382d qemu: Add a Knobs field to the Config structure
fe1bdcd2 qemu: Remove the extra parameters field from the Config structure
15bce61a qemu: Group all machine configurations into one structure
d94b5af8 qemu: Add a VGA parameter field to the Config structure
4892d041 qemu: Add a Global parameter field to the Config structure
612a5a9e qemu: Add a RTC field to the Config structure
c63ec096 qemu: Add a SMP field to the Config structure
7cf386a8 qemu: Add a Memory field to the Config structure
b198bc67 qemu: Add a UUID field to the Config structure
6239e846 qemu: Add a Character Devices slice field to the Config structure
73e2d53c qemu: Add a Filesystem Devices slice field to the Config structure
518ba627 qemu: Add a Kernel field to the Config structure
b973bc59 qemu: Add an Object slice field to the Config structure
8744dfe8 qemu: Add a Device slice field to the Config structure
5458de70 qemu: Add a QMP socket field to the Config structure
17118270 qemu: Add qemu's name to the Config structure
37a1f500 qemu: Add configuration structure to simplify LaunchQemu
5ccbaf2b ciao-launcher, qemu: Upgrade to new context package.
f5720198 qemu: Use null QMP logger when the logger parameter is nil
7d4199a4 qemu: Fix ineffassign error
7f50a415 qemu: Fix a silly bug in LaunchQemu
fc6bf8cf qemu: Add package documentation
306f54a9 ciao-launcher, qemu: Move launchQemu to qemu
344aa22b qemu: Add the qemu package
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
The commit message of a revert commit usually generated by
`git revert`, we should consider this as legal.
Consider the commit as the merge commit if the subject
starts with 'Reject "'
Follow the pr kata-containers/tests/#3938, the suttle diffrence
is we skip all commit checks for revert commit including fixes checking
and subsystem checking. Because the commit was reverted must have passed
the check so the revert-commit should have the Fixes and Subsystem.
Fixes: #3568Fixes: kata-containers/tests#3934
Signed-off-by: Tim Zhang <tim@hyper.sh>
When building a non-stable release, the tag is **always** "latest¨,
instead of the version. The same magic done for setting the correct
tags up should be done for replacing the tag on the kata-deploy and
kata-cleanup yaml files, as part of the kata-deploy test.
Fixes: #3559
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
There are software and hardware architectures which do not support
dynamically adjusting the CPU and memory resources associated with a
sandbox. For these, today, they rely on "default CPU" and "default
memory" configuration options for the runtime, either set by annotation
or by the configuration toml on disk.
In the case of a single container (launched by ctr, or something like
"docker run"), we could allow for sizing the VM correctly, since all of
the information is already available to us at creation time.
In the sandbox / pod container case, it is possible for the upper layer
container runtime (ie, containerd or crio) could send a specific
annotation indicating the total workload resource requirements
associated with the sandbox creation request.
In the case of sizing information not being provided, we will follow
same behavior as today: start the VM with (just) the default CPU/memory.
If this information is provided, we'll track this as Workload specific
resources, and track default sizing information as Base resources. We
will update the hypervisor configuration to utilize Base+Workload
resources, thus starting the VM with the appropriate amount of CPU and
memory.
In this scenario (we start the VM with the "right" amount of
CPU/Memory), we do not want to update the VM resources when containers
are added, or adjusted in size.
This functionality is introduced behind a configuration flag,
`static_sandbox_resource_mgmt`. This is defaulted to false for all
configurations except Firecracker, which is set to true.
This'll greatly improve UX for folks who are utilizing
Kata with a VMM or hardware architecture that doesn't support hotplug.
Note, users will still be unable to do in place vertical pod autoscaling
or other dynamic container/pod sizing with this enabled.
Fixes: #3264
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Describe the static_sandbox_resource_mgmt flag, and how this applies to
configurations that do not utilize hotplug.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Add the POD metadata we get from the container manager to the metrics by
adding more labels.
Fixes: #3551
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
Kata-monitor detects started and terminated kata pods by monitoring the
vc/sbs fs (this makes sense since we will have to access that path to
access the sockets there to get the metrics from the shim).
While kata-monitor updates its sandbox cache based on the sbs fs events,
it will schedule also a sync with the container manager via the CRI in
order to sync the list of sandboxes there.
The container manager will be the ultimate source of truth, so we will
stick with the response from the container manager, removing the
sandboxes not reported from the container manager.
May happen anyway that when we check the container manager, the new kata
pod is not reported yet, and we will remove it from the kata-monitor pod
cache. If we don't get any new kata pod added or removed, we will not
check with the container manager again, missing reporting metrics about
that kata pod.
Let's stick with the sbs fs as the source of truth: we will update the
cache just following what happens on the sbs fs.
At this point we may have also decided to drop the container manager
connection... better instead to keep it in order to get the kube pod
metadata from it, i.e., the kube UID, Name and Namespace associated with
the sandbox.
Every time we get a new sandbox from the sbs fs we will try to retrieve the
pod metadata associated with it.
Right now we just attach the container manager sandbox id as a label to
the exposed metrics, making hard to link the metrics to the running pod
in the kubernetes cluster.
With kubernetes pod metadata we will be able to add them as labels to map
explicitly the metrics to the kubernetes workloads.
Fixes: #3550
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
We currently WARN about unexpected fs events, which includes CHMOD
operations (which should be actually expected...).
Just ignore all the fs events we don't care about without any warn.
We dump all the events with debug log in any case.
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
Improve debug log formatting of the sandbox cache update process.
Move raw and tracing logs from the DEBUG to the TRACE log level.
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
Updated the doc to clarify certain networking details and
external links to some of the networking terms used.
Fixes#3308
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
The OCI container spec specifies a limit of -1 signifies
unlimited memory. Update the sandbox memory calculator
to reflect this part of the spec.
Fixes: #3512
Signed-off-by: Braden Rayhorn <bradenrayhorn@fastmail.com>
This PR removes the docker run and sysctl limitation reference
for kata 2.0 as well as docker daemon limitation as currently
for kata we are not supporting docker and this reference belonged
to kata 1.0
Fixes#3545
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
1. The hook.args[0] is the hook binary name which shouldn't be included
in the Command.args.
2. Add new unit tests
Fixes: #2610
Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
no explicit PCI test, just switch path depending on architecture
(CCW for s390x, PCI for others). Also fixes an unknown variable error.
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
When no options are passed to -ldflags, it passes
incorrect values(in this case, $BUILDFLAGS) to it.
Fix passing empty values by passing $KATA_LDFLAGS
in quotes.
Fixes: #3521
Signed-off-by: Amulya Meka <amulmek1@in.ibm.com>
Now that kata-pkgsync has been removed, this PR removes the reference
in the documentation.
Fixes#3513
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Remove the libseccomp package from Dockerfile of `alpine` and `clearlinux`
because the libseccomp library is installed by the `ci/install_libseccomp.sh`
script when building the kata-agent.
Fixes: #3508
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Weekly check(at 23:00 every Sunday) whether the docs url is ALIVE, so that
we can find the failed url in time
Fixes#815
Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
Although I've done tests on my own fork using `head_ref` and those
worked, it seems those only worked as the PR was coming from exactly the
same repository as the target one.
Let's switch to base_ref, instead, which we for sure have as part of our
repo.
The downside of this is that we run the test with the last merged PR,
rather than with the "to-be-approved" PR, but that's a limitation we've
always had.
Fixes: #3482
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
For now a bunch of tests are simply not working.
Let's skip them all, and re-enable them once
kata-containers/kata-containers/issues/3500 gets fixed.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Both projects follow the same license, Apache-2.0, but the header saying
that comes from govmm is different from the one expected for the tests
present on the kata-containers repo.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
govet checks have been ignored on govmm repo, but those are enabled on
kata-containers one. So, in order to avoid failing our CIs let's just
keep ignoring the checks for the govmm structs and have an issue opened
for fixing it whenever someone has cycles to do it.
The important bit here is, we're not making anything worse that it
already is. :-)
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
govmm, from now on, should follow the same guidelines from contributing,
copying, and etc as kata-containers does.
The go.mod is not needed anymore as the project lives inside the
runtime.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's stop using govmm from kata-containers/govmm and let's start using
it from our own repo.
Fixes: #3495
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR removes the kata-pkgsync tool that is mainly used for OBS
packages, currently for kata 2.0 we do not have OBS packages and
this tool is not being used for kata 2.0
Fixes#3493
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Escape (with backslash) backticks (`) to prevent them from being
evaluated by the shell.
Fixes: #3487
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Current latest release is 1.23.1. Let's update to this version for our
integration testing.
Fixes: #3477
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
`enable_swap` option was added long time ago to add
`-realtime mlock=off` to the QEMU's command line.
Kata now supports QEMU 6, `-realtime` option has been deprecated and
`mlock=on` is causing unexpected behaviors in kata.
This patch removes support for `enable_swap`, `-realtime` and `mlock=`
since they are causing bugs in kata.
Signed-off-by: Julio Montes <julio.montes@intel.com>
This is needed in order to ensure that, for instance, if `force-skip-ci`
label is either added or removed later, the jobs related to the actions
will be restarted and accordingly checked.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Before this change it was only applied to the static-checks, but if
we're already taking the extreme path of skipping the CI, we better
ensure we skip all the actions and not just a few of them.
Fixes: #3471
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The action used for testing kata-deploy is entirely based on the action
used to build the kata-deploy tarball, but while the latter is able to
use the correct branch, the former always uses `main`.
This happens as the `issue_comment`, from GitHub actions, passed the
"default branch" as the GITHUB_REF.
As we're not the first ones to face such a issue, I've decided to take
one of the approaches suggested at one of the checkout's issues,
https://github.com/actions/checkout/issues/331, and take advantage of a
new action provided by the community, which will get the PR where the
comment was made, give us that ref, and that then can be used with the
checkout action, resulting on what we originally wanted.
Fixes: #3443
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The script `tools/packaging/static-build/qemu/build-base-qemu.sh`
previously failed on systems where the user's groupname differs from the
username
Fixes: #3461
Signed-off-by: Sebastian Hasler <sebastian.hasler@stuvus.uni-stuttgart.de>
bring SGX support and other fixes
shortlog:
8939b0f qemu: add support for SGX
b17f073 qemu: update readonly flag for block devices
f971801 qemu: only set wait parameter for server mode socket based
char device
82cc01d qemu: Fix 32 bit int overflow in test file
1d1a231 qemu: Add support for legacy serial device
9a2bbed qemu: Remove -realtime in favor of -overcommit
fe83c20 qemu: Add support for --no-shutdown Knob
1ed5271 qmp: wait for POWERDOWN event in ExecuteSystemPowerdown()
fixes#3080
Signed-off-by: Julio Montes <julio.montes@intel.com>
Provide the `/proc/config.gz` file in guest kernels that allow the guest
to determine the kernel configuration used to build the running kernel.
Note that since `gunzip` expects to rename the gzip'ed file it operates
on, to use this feature you need to run something like the following in
the container environment:
```bash
# cat /proc/config.gz|gunzip -c
```
Fixes: #3445.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Debian is a supported rootfs that uses systemd as init, thus, it should
be mentioned in the QAT README document.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the gentoo rootfs is not tested in our CI, we can't guarantee it
actually works as expected.
Whenever we have someone willing to maintain this rootfs we can have it
added back, and also add a CI job to test it altogether, avoiding then
any possible regression.
Fixes: #2144
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the suse rootfs is not tested in our CI, we can't guarantee it
actually works as expected.
Whenver we have someone willing to maintain this rootfs we can have it
added back, and also add a CI job to test it altogether, avoiding then
any possible regression.
Fixes: #2145
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the fedora rootfs is not tested in our CI, we can't guarantee it
actually works as expected.
Whenever we have someone willing to maintain the rootfs we can have it
added back, and also add a CI job to test it altogether, avoiding then
any possible regression.
Fixes: #2143
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the centos rootfs is not tested in our CI, we can't guarantee it
actually works as expected.
Whenever we have someone willing to maintain the rootfs we can have it
added back, and also add a CI job to test it altogether, avoiding then
any possible regression.
Fixes: #2140
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR removes the ccloudvm reference at the README document as the
setup of scripts of ccloudvm were removed.
Fixes#3448
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR removes ccloudvm for kata 2.0, ccloudvm was used in kata 1.x
and we are not longer using it for kata 2.0.
Fixes#3427
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Generated protocols files should not be inclued in Git repo.
And also add Cargo.lock in oci/protocols directory to .gitignore.
Fixes: #3422
Signed-off-by: bin <bin@hyper.sh>
When Sandbox AddInterface() is called, it may fail after endpoint.HotAttach,
we'd better rollback and call save() in the end.
Fixes: #3419
Signed-off-by: yangfeiyu <yangfeiyu20102011@163.com>
This reverts commit 321995b7df.
Now that gnu.org is back online, we don't need to use a mirror.
Fixes: #3313.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
After the protocols are moved to upper libs (PR3355),
the runtime protocol generation is broken. This fixes it.
Fixes: #3414
Signed-off-by: Feng Wang <feng.wang@databricks.com>
This PR removes the scripts and the dockerfiles that were used in kata 1.x
to test the different kata components for different distributions in OBS.
Currently for kata 2.0 we are not generating packages in OBS so these scripts
are not longer being used.
Fixes#3404
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR removes the reference of how to use disable_new_netns
configuration with docker as for kata 2.0 we are not supporting docker
and this information was used for kata 1.x
Fixes#3400
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
- kata-deploy: fix tar command in dockerfile
- vendor: update to containerd v1.6.0-beta.4
- versions: Upgrade to Cloud Hypervisor v20.2
- vc: remove swagger binary
- agent: Refactor command line parsing to use a framework
- move the oci and protocols crates from agent to upper libs
- docs: Remove word duplication
- osbuilder: Restore Debian as a rootfs
- runtime: fix a typo in kata-collect-data.sh
- agent: return detail error message for RPC calls from shim
- use-cases: clarify SPDK vhost-user-nvme target status in using-spdk-v…
- Delint dockerfiles
- Makefile: update `make go-test` call
- docs: add how-to on DinD in Kata
- agent: Ignore unknown seccomp system calls
- agent: mount: Remove unneeded mount_point local variable
- docs: Fix outdated links
- docs: Fix kernel configs README spelling errors
- security: Update rust crate versions
- kata-manager: Retrieve static tarball
- osbuilder: avoid to copy versions.txt which already deprecated
- qemu: Disable libudev for QEMU 5.2 and newer
- osbuilder: Add protoc to the alpine container
- docs: Clarify where to run agent API generation commands
- packaging/qemu: partial git clone
- docs: Fix arch doc formatting
- CI: Switch to a mirror as gnu.org is down
- Split architecture doc into separate files
- docs: Update the stable branch strategy
- tracing: Add span name to logging error
- docs: Update code PR advice document
- agent: Add config file option to cli
- update container type handling
- docs: Update architecture document
- runtime: update golang to 1.16 and remove ioutil package
- kata-deploy: Deal with empty containerd conf file
- src: reorg source code directory
- osbuilder: show usage if no options/arguments specified
- Upgrade to Cloud Hypervisor v20.1
- image_build: add help info for '-f' option and 'BLOCK_SIZE' env.
- osbuilder: be runtime consistent with podman build
- osbuilder: Revert to using apk.static for Alpine
- runtime/template: Handling new attributes for hypervisor config
- docs: fix check-markdown test
- runtime: correct span name for stopSandbox function
- runtime: only call stopVirtiofsd when shared_fs is virtio-fs
- snap: read initrd and image distros from version.yaml
- versions: Use Ubuntu initrd for non-musl archs
- packaging: Fix missing commit message in building kata-runtime
- virtcontainers: clh: Upgrade to openapi-generator v5.3.0
- agent: user container ID as watchable storage key for hashmap
- runtime: enable vhost-net for rootless hypervisor
- packaging: add help information for '-f' option in install_go.sh
- Cleanup some unused variables, definitions
- Upgrade to Cloud Hypervisor v20.0
- docs: Update limitation document regarding docker swarm
- runtime: Enable FUSE_DAX kernel config for DAX
- agent: copy empty directories for watchable-bind mounts
- runtime: Update comments for virtcontainers to use kata 2.0
- Update rust crate versions
- osbuilder: Remove debian as a rootfs
e2c1e65e kata-deploy: fix tar command in dockerfile
615224e9 agent: move the protocols to upper libs
330e3dcc agent: move the oci crate to upper libs
7b03d78f vendor: update to containerd v1.6.0-beta.4
1f581a04 versions: Upgrade to Cloud Hypervisor v20.2
623d8f08 docs: Remove word duplication
1c4edb96 agent: Refactor arg parsing to use clap
3093f93a osbuilder: Restore Debian as a rootfs
073a3459 use-cases: clarify vhost-user-nvme status in using-spdk-vhost-user
2254fa86 runtime: fix a typo in kata-collect-data.sh
2d0f9d2d vc: remove swagger binary
cf91307c agent: return detail error message for rpc calls from shim
137e217b docs: Fix outdated k8s link
55bac67a docs: Fix kernel configs README spelling errors
205420d2 docs: Replicate branch rename on runtime-spec
91abebf9 agent: mount: Remove unneeded mount_point local variable
b1f4e945 security: Update rust crate versions
d79268ac tools/packaging: add copyright to kata-monitor's Dockerfile
428cf0a6 packaging: delint tests dockerfiles
1ea9b703 packaging: delint kata-deploy dockerfiles
3669e1b6 ci/openshift-ci: delint dockerfiles
aeb2b673 osbuilder: delint dockerfiles
bc120289 packaging: delint kata-monitor dockerfiles
bc71dd58 packaging: delint static-build dockerfiles
99ef52a3 osbuilder: Add protoc to the alpine container
c2578cd9 docs: Clarify where to run agent API generation commands
321995b7 CI: Switch to a mirror as gnu.org is down
fb1989b2 docs: Fix arch doc formatting
2938bb7f packaging/qemu: Use QEMU script to update submodules
5d49ccd6 packaging/qemu: Use partial git clone
87a219a1 docs: Update the stable branch strategy
d1bc409d osbuilder: avoid to copy versions.txt which already deprecated
1653dd4a tracing: Add span name to logging error
12c8e41c qemu: Disable libudev for QEMU 5.2 and newer
233015a6 docs: Split guest assets details out of arch doc
db411c23 docs: Split k8s info out of arch doc
7ac619b2 docs: Split networking out of arch doc
5df0cb64 docs: Split storage out of arch doc
7229b7a6 docs: Split background and example out of arch doc
283d7d52 docs: Split history out of arch doc
6f9efb40 docs: Move arch doc to separate directory
02608e13 docs: Update code PR advice document
cb5c948a kata-manager: Retrieve static tarball
51bf9807 docs: Update architecture document
f3a97e94 docs: add how-to on Docker in Kata
7a989a83 runtime: api-test: fixup
52f79aef utils: update container type handling
5b002f3c docs: change io/ioutil to io/os packages
03546f75 runtime: change io/ioutil to io/os packages
24a530ce versions: bump minimum golang version to 1.16.10
7c4263b3 src: reorg source directories
1a34fbcd agent: Add config file option to cli
bbfb10e1 versions: Upgrade to Cloud Hypervisor v20.1
84571506 kata-deploy: Deal with empty containerd conf file
3f7cf7ae osbuilder: show usage if no options/arguments specified
2ebaaac7 osbuilder: be runtime consistent also with podman build
f3103696 docs: fix check-markdown test
2204ecac versions: Upgrade Alpine, using minor version
dfd0732f osbuilder: Revert to using apk.static for Alpine
6b3e4c21 image_build: add help info for '-f' option and 'BLOCK_SIZE' env.
b92babf9 runtime/template: Handling new attributes for hypervisor config
40bd34ca runtime: only call stopVirtiofsd when shared_fs is virtio-fs
33f343ee runtime: correct span name for stopSandbox function
d7cc952c versions: Use Ubuntu initrd for non-musl archs
ff929fc0 snap: read initrd and image distros from version.yaml
8fae2631 packaging: Fix missing commit message in building kata-runtime
99530026 virtcontainers: clh: Upgrade to openapi-generator v5.3.0
b3bcb7b2 runtime: enable vhost-net for rootless hypervisor
7cb7b9d5 agent: remove unused field in mount handling
f6ae1582 agent: drop unused fields from network
4756a04b virtcontainers: clh: Re-generate the client code
0bf4d257 versions: Upgrade to Cloud Hypervisor v20.0
647082b2 docs: Update limitation document regarding docker swarm
39b35d00 agent: user container ID as watchable storage key for hashmap
1e6f58e5 packaging: add help information for '-f' option in install_go.sh
2af95bc5 agent: create directories for watchable-bind mounts
6105e3ee runtime: enable FUSE_DAX kernel config for DAX
591d4af1 runtime: Update comments for virtcontainers to use kata 2.0
923e098d osbuilder: Remove debian as a rootfs
afb96c00 agent: Wrap remaining nix errors with anyhow
aba572e0 rustjail: Wrap remaining nix errors with anyhow
30d60078 uevent: Fix clippy issue in test code
4a2be13c agent: Upgrade nix version for security fix
256d5008 agent: Update crate versions
13257986 agent-ctl: Update rust lockfile
4ebdd424 forwarder: Update rust lockfile
6007322d agent: Fixed invalid error message
7b356151 agent: Log unknown seccomp system calls
7304e52a Makefile: update `make go-test` call
c66b5668 agent: Ignore unknown seccomp system calls
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
move the protocols to upper libs thus it can
be shared between agent and other rust runtime.
Depends-on: github.com/kata-containers/tests#4306
Fixes: #3348
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Move the oci crate to upper libs thus it can be
shared between agent and other rust runtimes.
Fixes: #3348
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Update our containerd vendoring. In particular, we're interested in
grabbing the updated annotation definitions for defining sandbox sizing.
- go get github.com/containerd/containerd@v1.6.0-beta.4
- edit go.mod to remove containerd v1.5.8 replacement directive
- go mod vendor
- go mod tidy
Fixes: #3276
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
This is a bug release from Cloud Hypervisor addressing the following
issues: 1) Don't error out when setting up the SIGWINCH handler (for
console resize) when this fails due to older kernel; 2) Seccomp rules
were refined to remove syscalls that are now unused; 3) Fix reboot on
older host kernels when SIGWINCH handler was not initialised; 4) Fix
virtio-vsock blocking issue.
Details can be found: https://github.com/cloud-hypervisor/cloud-hypervisor/releases/tag/v20.2Fixes: #3383
Signed-off-by: Bo Chen <chen.bo@intel.com>
Restore Debian as a rootfs.
1. revert of #3154, but some change
2. update debian version to 10.11
3. update `libstdc++-6-dev` to `libstdc++-8-dev`
4. changes discarded in QAT are not restored
Fixes: #3372
Signed-off-by: zhaojizhuang <571130360@qq.com>
SPDK vhost-user-nvme target is removed from SPDK 21.07 release since
upstreamed QEMU version does not support. Fixes this usage.
Fixes#3371
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
The tokio's spawn will only create an future async task
instead of a new real thread, thus executing unshare to
create a new namespace in tokio's async task would make
the agent process to join in the new created namespace,
which isn't expected.
Thus, we'd better to to the unshare in a real thread to
prevent moving the agent process into a new namespace.
Fixes: #3369
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
For calls from shim to agent, the return error will be processed like this:
match self.do_start_container(req).await {
Err(e) => Err(ttrpc_error(ttrpc::Code::INTERNAL, e.to_string())),
Ok(_) => Ok(Empty::new()),
}
The e.to_string() return only a part of the error(for example set by context()),
this may lead lack of information.
The `format!("{:?}", err)` will return more info.
Fixes: #3353
Signed-off-by: bin <bin@hyper.sh>
We already have a `mount_path` local Path variable which holds the mount
point.
Use it instead of creating a new `mount_point` variable with identical
type and content.
Fixes: #3332
Signed-off-by: Dov Murik <dovmurik@linux.ibm.com>
The kata-monitor's Dockerfile was added by Eric Ernst on commit 2f1cb7995f
but for some reason the static checker did not catch the file misses the copyright statement
at the time it was added. But it is now complaining about it. So this assign the copyright to
him to make the static-checker happy.
Fixes#3329
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Removed all errors/warnings pointed out by hadolint version 2.7.0, except for the following
ignored rules:
- "DL3008 warning: Pin versions in apt get install"
- "DL3041 warning: Specify version with `dnf install -y <package>-<version>`"
- "DL3033 warning: Specify version with `yum install -y <package>-<version>`"
- "DL3048 style: Invalid label key"
- "DL3003 warning: Use WORKDIR to switch to a directory"
- "DL3018 warning: Pin versions in apk add. Instead of apk add <package> use apk add <package>=<version>"
- "DL3037 warning: Specify version with zypper install -y <package>[=]<version>"
Fixes#3107
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Removed all errors/warnings pointed out by hadolint version 2.7.0, except for the following
ignored rules:
- "DL3008 warning: Pin versions in apt get install"
- "DL3041 warning: Specify version with `dnf install -y <package>-<version>`"
- "DL3033 warning: Specify version with `yum install -y <package>-<version>`"
- "DL3048 style: Invalid label key"
- "DL3003 warning: Use WORKDIR to switch to a directory"
- "DL3018 warning: Pin versions in apk add. Instead of apk add <package> use apk add <package>=<version>"
- "DL3037 warning: Specify version with zypper install -y <package>[=]<version>"
Fixes#3107
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Removed all errors/warnings pointed out by hadolint version 2.7.0, except for the following
ignored rules:
- "DL3008 warning: Pin versions in apt get install"
- "DL3041 warning: Specify version with `dnf install -y <package>-<version>`"
- "DL3033 warning: Specify version with `yum install -y <package>-<version>`"
- "DL3048 style: Invalid label key"
- "DL3003 warning: Use WORKDIR to switch to a directory"
- "DL3018 warning: Pin versions in apk add. Instead of apk add <package> use apk add <package>=<version>"
- "DL3037 warning: Specify version with zypper install -y <package>[=]<version>"
Fixes#3107
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Removed all errors/warnings pointed out by hadolint version 2.7.0, except for the following
ignored rules:
- "DL3008 warning: Pin versions in apt get install"
- "DL3041 warning: Specify version with `dnf install -y <package>-<version>`"
- "DL3033 warning: Specify version with `yum install -y <package>-<version>`"
- "DL3048 style: Invalid label key"
- "DL3003 warning: Use WORKDIR to switch to a directory"
- "DL3018 warning: Pin versions in apk add. Instead of apk add <package> use apk add <package>=<version>"
- "DL3037 warning: Specify version with zypper install -y <package>[=]<version>"
Fixes#3107
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Removed all errors/warnings pointed out by hadolint version 2.7.0, except for the following
ignored rules:
- "DL3008 warning: Pin versions in apt get install"
- "DL3041 warning: Specify version with `dnf install -y <package>-<version>`"
- "DL3033 warning: Specify version with `yum install -y <package>-<version>`"
- "DL3048 style: Invalid label key"
- "DL3003 warning: Use WORKDIR to switch to a directory"
- "DL3018 warning: Pin versions in apk add. Instead of apk add <package> use apk add <package>=<version>"
- "DL3037 warning: Specify version with zypper install -y <package>[=]<version>"
Fixes#3107
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Removed all errors/warnings pointed out by hadolint version 2.7.0, except for the following
ignored rules:
- "DL3008 warning: Pin versions in apt get install"
- "DL3041 warning: Specify version with `dnf install -y <package>-<version>`"
- "DL3033 warning: Specify version with `yum install -y <package>-<version>`"
- "DL3048 style: Invalid label key"
- "DL3003 warning: Use WORKDIR to switch to a directory"
- "DL3018 warning: Pin versions in apk add. Instead of apk add <package> use apk add <package>=<version>"
- "DL3037 warning: Specify version with zypper install -y <package>[=]<version>"
Fixes#3107
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
It seems the lack of protoc in the alpine containers is causing issues
with some of our CIs, such as the VFIO one.
Fixes: #3323
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Make it clear when reading the table in the agent's "Change the agent
API" documentation that the commands in the "Generation method" column
should be run in the agent repo.
Fixes: #3317.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
All CI jobs are failing as www.gnu.org is down, so switch to a mirror
for the time being.
Fixes: #3314.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
PR #3298 failed to move the named link for the debug console to the
`guest-assets.md` meaning the debug console cells in the "User
accessible" column in the table in the "Root filesystem image" section
do not work as a link.
Fixes: #3311.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Currently QEMU's submodules are git cloned but there is the scripts/git-submodule.sh
which is meant for that. Let's use that script.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The static build of QEMU takes a good amount of time on cloning the
source tree because we do a full git clone. In order to speed up that
operation this changed the Dockerfile so that it is carried out a
partial clone by using --depth=1 argument.
Fixes#3291
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
On the last architecture committee meeting, the one held on December
14th 2021, we reached the agreement that minor releases will be cut once
every 16 weeks (instead of 12), and that patch releases will be cut
every 4 weeks (instead of 3)
Fixes: #3298
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Currently the versions.txt in rootfs-builder dir is already removed,
so avoid to copy it in list of helper files.
Fixes: #3267
Signed-off-by: zhanghj <zhanghj.lc@inspur.com>
Add span name to logging error to help with debugging when the context
is not set before the span is created.
Fixes#3289
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Commit 112ea25859 disabled libudev for static builds because it was
breaking snap. It turns out that the only users of libudev in QEMU are
qemu-pr-helper and USB. Kata already disables USB and doesn't use
qemu-pr-helper. Disable libudev for all builds if QEMU supports it, i.e.
version 5.2 or newer.
Fixes#3078
Signed-off-by: Greg Kurz <groug@kaod.org>
Move the guest assets details out of the architecture doc and into a
separate file.
Fixes: #3246.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Move the Kubernetes information out of the architecture doc and into a
separate file.
Partially fixes: #3246.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Move the networking details out of the architecture doc and into a
separate file.
Partially fixes: #3246.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Move the background and example command details out of the architecture
doc and into separate files.
Partially fixes: #3246.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Move the historical details out of the architecture doc
and into a separate file.
Partially fixes: #3246.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Move the architecture document into a new `docs/design/architecture/` directory
in preparation for splitting it into more manageable pieces.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Allow using `expect()` for `Mutex.lock()` because it is almost
unrecoverable if failed in the lock acquisition
Fixes: #3285
Signed-off-by: Zack <zmlcc@linux.alibaba.com>
In `utils/kata-manager.sh`, we download the first asset listed for the
release, which used to be the static x86_64 tarball. If that happened to
not match the system architecture, we would abort. Besides that logic
being invalid for !x86_64 (despite not distributing other tarballs at
the moment), the first asset listed is also not the static tarball any
more, it is the vendored source tarball. Retrieve all _static_ tarballs
and select the appropriate one depending on architecture.
Fixes: #3254
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Refresh the content and formatting of the architecture document.
Out of scope of these changes:
- Diagram updates.
- Updates to the Networking section.
Fixes: #3190.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
not clear why this was commented out before -- ensure that we set
approprate annotation on the sandbox container's annotations to indicate
this is a sandbox.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Today we assume that if the CRI/upper layer doesn't provide a container
type annotation, it should be treated as a sandbox. Up to this point, a
sandbox with a pause container in CRI context and a single container
(ala ctr run) are treated the same.
For VM sizing and container constraining, it'll be useful to know if
this is a sandbox or if this is a single container.
In updating this, we cleanup the type handling tests and we update the
containerd annotations vendoring.
Fixes: #2926
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Change io/ioutil to io/os packages because io/ioutil package
is deprecated from 1.16:
TempDir => os.MkdirTemp
Details: https://go.dev/doc/go1.16#ioutilFixes: #3265
Signed-off-by: bin <bin@hyper.sh>
since qemu 6.0, readonly flag for block devices must be enable or
disable with `on` or `off` respectively.
Signed-off-by: Julio Montes <julio.montes@intel.com>
According to https://endoflife.date/go golang 1.11.10 is not supported
anymore, 1.16.10 is the minimum supported version.
Fixes: #3265
Signed-off-by: bin <bin@hyper.sh>
This is a bug release from Cloud Hypervisor addressing the following
issues: 1) Networking performance regression with virtio-net; 2) Limit
file descriptors sent in vfio-user support; 3) Fully advertise PCI MMIO
config regions in ACPI tables; 4) Set the TSS and KVM identity maps so
they don't overlap with firmware RAM; 5) Correctly update the DeviceTree
on restore.
Details can be found: https://github.com/cloud-hypervisor/cloud-hypervisor/releases/tag/v20.1Fixes: #3262
Signed-off-by: Bo Chen <chen.bo@intel.com>
As containerd can properly run without having a existent
`/etc/containerd/config.toml` file (it'd run using the default
cobnfiguration), let's explicitly create the file in those cases.
This will avoid issues on ammending runtime classes to a non-existent
file.
Fixes: #3229
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Tested-by: Jakob Naucke <jakob.naucke@ibm.com>
Now if no options/arguments specified, the shell scripts will return an error:
ERROR: Invalid rootfs directory: ''
This commit will show usage if no options/arguments specified.
Fixes: #3256
Signed-off-by: bin <bin@hyper.sh>
Use the same runtime used for podman run also for the podman build cmd
Additionally remove "docker" from the docker_run_args variable
Fixes: #3239
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
Unit-Test-Advice.md was moved to kata-containers repo but URLs pointing
to that document were not updated. This patch updates these URLs.
Depends-on: github.com/kata-containers/tests#4273
fixes#3240
Signed-off-by: Julio Montes <julio.montes@intel.com>
- Upgrade Alpine guest rootfs to 3.15
- Specify a minor version rather than patch level as the Alpine
repositories use that.
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
#2399 partially reverted #418, missing on returning to bootstrapping a
rootfs with `apk.static` instead of copying the entire root, which can
result in drastically larger (more than 10x) images. Revert this as well
(requires some updates to URL building).
Fixes: #3216
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
The help information of '-f' option is missing, and same issue
with 'BLOCK_SIZE' env variables, fix it in usage() function.
Fixes: #3231
Signed-off-by: zhanghj <zhanghj.lc@inspur.com>
Some new attributes are added to hypervisor config:
- VMStorePath
- RunStorePath
- SharedPath
These attributes should be handled in two places:
- reset when check the new hypervisor's config is suitable
to the base config.
- copy from new hypervisor's config when create new VM
Fixes: #3193
Signed-off-by: bin <bin@hyper.sh>
ppc64le & s390x have no (well supported) musl target for Rust,
therefore, the agent must use glibc and cannot use Alpine. Specify
Ubuntu as the distribution to be used for initrd.
Fixes: #3212
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
vhost-net is disabled in the rootless kata runtime feature, which has been abandoned since kata 2.0.
I reused the rootless flag for nonroot hypervisor and would like to enable vhost-net.
Fixes#3182
Signed-off-by: Feng Wang <feng.wang@databricks.com>
This PR removes the information about docker swarm and docker compose
as currently for kata 2.0 we have not support for docker swarm and docker
compose and the links and references that the document is referring are
currently not part of kata 1.0
Fixes#3174
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
In function `update_target`, if the updated source is a directory,
we should create the corresponding directory.
Fixes: #3140
Signed-off-by: bin <bin@hyper.sh>
This PR updates the comments in the configuration.toml to point to
the current kata containers repository instead of the kata 1.x.
Fixes#3163
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Currently we do not have debian as part of the kata CI as we
do not have a mantainer, this PR removes debian as a supported
rootfs in order to have only the distros that we are supporting
and mantainining.
Fixes#3153
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Wrap `nix` `Error`'s in an `anyhow` error for consistency with the way
`rustjail` handles errors.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Replace `Result` values that use a "bare" `nix` `Error` like this:
```rust
return Err(nix::Error::EINVAL.into());
```
... to the following which wraps the nix` error in an `anyhow` call for
consistency with the other errors returned by `rustjail`:
```rust
return Err(anyhow!(nix::Error::EINVAL));
```
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Remove a bare `return` from a test function. This looks wrong but isn't
because the callers are all tests that just wait for a state change
caused by this test function.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Running `cargo audit` showed that the `nix` package for the agent and
the `rustjail` and `vsock-exporter` local crates need to be updated to
resolve rust security issue
[RUSTSEC-2021-0119](https://rustsec.org/advisories/RUSTSEC-2021-0119).
Hence, bumped `nix` to the latest version (which required changes to
work with the new, simpler `errno` handling).
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Run `cargo update` to update to the latest crate dependency versions.
The agent is an application so this includes expanding the partially
specified semvers to full semver values for the following crates,
which makes those crates consistent with the other agent dependencies:
- `futures`
- `regex`
- `scan_fmt`
- `tokio`
Fixes: #3124.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Remove the format specifier in the `"failed to get VFIO group"` error
returned by `vfio_device_handler()`.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
- osbuilder: fix missing cpio package when building rootfs-initrd image
- osbuilder: add coreutils to guest rootfs
- workflows: only allow org members to run `/test_kata_deploy`
- agent: use temp directory for test containers
- tools/osbuilder: build QAT kernel in fedora 34
- agent: refactor find_process function and add test cases
- Hypervisor cleanup, refactoring
- agent: clear cargo test warnings
- docs: Add a code PR advice document
- tools: Automatically revert kata-deploy changes
- runtime: delete netmon
- agent: Remove some unwrap and expect calls
- agent: fixed the `make optimize` bug
- docs: make kata-deploy more visible
- workflows: Add back the checks for running test-kata-deploy
- kata-deploy: Ensure we test HEAD with `/test_kata_deploy`
- docs: update using-SPDK-vhostuser-and-kata.md
- Update k8s SR-IOV plugin environment variables to work properly with Kata
- watchers: don't dereference symlinks when copying files
- kata-deploy: Add back stable & latest tags
- agent: fix the issue of missing create a new session for container
- runtime: Update containerd to 1.5.8
- qemu: fix snap build on ppc64le
- virtcontainers: fix failing template test on ppc64le
- agent: Update README
- Remove cruft, do some simple non-functional cleanup in the runtime
- macvlan: drop bridged part of name
- clh: Fix race condition that prevent start pods
- Update CRI-O documentation
- cgroups: Fix systemd cgroup support
- runtime: merge virtcontainers/pkg/types into virtcontainers/types
- workflows: Remove non-used main.yaml
- agent/src: improve unit test coverage for src/namespace.rs
- doc: update kata metrics documentation
- runtime: delete not used codes
- versions: bump golang to 1.17.x
- release: Use ${GOPATH}/bin/yq for upload-libseccomp-tarball action
- agent-ctl: Allow API specification in JSON format
- virtcontainers: Lint protection types
- agent: check environment variables if empty or invalid
- runtime: Revert "runtime: use containerd package instead of cri-containerd"
- rustjail: Fix created time of container
- agent: Remove dynamic tracing APIs
- kernel: add VFIO kernel dependencies for ppc64le
- logging: Always run crate tests
8ee67aae osbuilder: fix missing cpio package when building rootfs-initrd image
f59d3ff6 osbuilder: add coreutils to guest rootfs
5e7c1a29 workflows: only allow org members to run `/test_kata_deploy`
857501d8 tools/osbuilder: build QAT kernel in fedora 34
a32e02a1 agent: use temp directory as root of test containers
f0734f52 docs: Remove extraneous whitespace
aff32756 docs: Add a code PR advice document
d41c375c docs: Add more advice to the UT advice doc
baf4f76d docs: More detail on running tests as different users
fcf45b0c docs: Use more idiomatic rust string check
9fed7d0b docs: Mention anyhow for error handling in UT doc
318b3f18 docs: No present continuous in UT advice doc
e8bb6b26 docs: Correct repo name usage
c1111a1d docs: Use leading caps for lang names in UT advice doc
597b239e docs: Remove TOC in UT advice doc
cf360fad docs: Move unit test advice doc from tests repo
bc955814 docs: Move doc requirements section higher
6a0b7165 agent: refactor find_process function and add test cases
5ba2f52c tools: Quote functions arguments in the update repos script
5dbd752f tools: Remove the check for the VERSION file
85eb743f tools: Make hub usage slightly less fragile
76540dbd tools: Automatically revert kata-deploy changes
36d73c96 tools: Do the kata-deploy changes on its own commit
c8e22daf tools: Use vars for the registry in the update repo script
ac958a30 tools: Use vars for the yaml files used in the update repo script
edca8292 tools: Rewrite the logic around kata-deploy changes
31f6c2c2 tools: Update comments about the kata-deploy yaml changes
75bb3401 shimv2/service: fix defer funtions never run with os.Exit()
bd3217da agent: Remove redundant returns
adab6434 agent: Remove some unwrap and expect calls
351cef7b agent: Remove unwrap from verify_cid()
a7d1c70c agent: Improve baremount
09abcd4d agent-ctl: Remove some unwrap and expect calls
35db75ba agent-ctl: Remove redundant returns
46e45958 agent-ctl: Simplify main
c7349d0b agent-ctl: Simplify error handling
ddc68131 runtime: delete netmon
705687dc docs: Add kata-deploy as part of the install docs
acece849 docs: Use the default notation for "Note" on install README
143fb278 kata-deploy: Use the default notation for "Note"
45d76407 kata-deploy: Don't mention arch specific binaries in the README
0c6c0735 agent: fixed the `make optimize` bug
a7c08aa4 workflows: Add back the checks for running test-kata-deploy
ce0693d6 agent: clear cargo test warnings
ce92cadc vc: hypervisor: remove setSandbox
2227c46c vc: hypervisor: use our own logger
4c2883f7 vc: hypervisor: remove dependency on persist API
34f23de5 vc: hypervisor: Remove need to get shared address from sandbox
c28e5a78 acrn: remove dependency on sandbox, persistapi datatypes
a0e0e186 hypervisors: introduce pkg to unbreak vc/persist dependency
b5dfcf26 watcher: tests: ensure there is 20ms delay between fs writes
78dff468 agent/device: Adjust PCIDEVICE_* container environment variables for VM
4530e7df agent/device: Use simpler structure in update_spec_devices()
b6062278 agent/device: Correct misleading comment on test case
89ff7000 agent/device: Remove unnecessary check for empty container_path
c855a312 agent/device: Make DevIndex local to update_spec_devices()
084538d3 agent/device: Change update_spec_device to handle multiple devices at once
d6a3ebc4 agent/device: Obtain guest major/minor numbers when creating DevNumUpdate
f4982130 agent/device: Check for conflicting device updates
f10e8c81 agent/device: Batch changes to the OCI specification
46a4020e agent/device: Types to represent update for a device in the OCI spec
e7beed54 agent/device: Remove unneeded clone() from several device handlers
2029eeeb agent/device: Improve update_spec_device() final_path handling
57541315 agent/device: Correct misleading parameter name in update_spec_device()
0c51da3d agent/device: Correct misleading error message in update_spec_device()
94b7936f agent/device: Use nix::sys::stat::{major,minor} instead of libc::*
296e76f8 watchers: handle symlinked directories, dir removal
2b6dfe41 watchers: don't dereference symlinks when copying files
3c9ae7fb kata-deploy: Ensure we test HEAD with `/test_kata_deploy`
0380b9bd runtime: Update containerd to 1.5.8
112ea258 qemu: fix snap build by disabling libudev
d5a18173 virtcontainers: fix failing template test on ppc64le
6955d144 kata-deploy: Add back stable & latest tags
bbaf57ad agent: fix the issue of missing create a new session for container
46fd5069 docs: update using-SPDK-vhostuser-and-kata.md
7e6f2b8d vc-utils: don't export unused function
860f3088 virtcontainers: move oci, uuid packages top level
8acb3a32 virtcontainers: remove unused package nsenter
4788cb82 vc-network: remove unused functions
b6ebddd7 oci: remove unused function GetContainerType
599bc0c2 agent: Update README
1e7cb4bc macvlan: drop bridged part of name
55412044 monitor: Fix monitor race condition doing hypervisor.check()
eb11d053 cri-o: Update deployment documentation
92e3a140 cri-o: Update links for the CRI-O github page
0a19340a cri-o: Remove outdated documentation
a3b3c85e workflows: Remove non-used main.yaml
09f7962f runtime: merge virtcontainers/pkg/types into virtcontainers/types
6acedc25 runtime: delete not used codes
395638c4 versions: bump golang to 1.17.x
570915a8 docs: update kata 2.0 metrics documentation
bcf181b7 cgroups: Fix systemd cgroup support
34307235 release: Use ${GOPATH}/bin/yq for upload-libseccomp-tarball action
6339fdd1 docs: update kata metrics architecture image
57bb7ffa agent: check environment variables if empty or invalid
8ab90e10 agent-ctl: Allow API specification in JSON format
eacfcdec runtime: Revert "runtime: use containerd package instead of cri-containerd"
e7856ff1 rustjail: Fix created time of container
b7b89905 virtcontainers: Lint protection types
7566b736 kernel: add VFIO kernel dependencies for ppc64le
87f67606 agent: Remove dynamic tracing APIs
b09dd7a8 docs: Fix typo
d47484e7 logging: Always run crate tests
5c9c0b6e build: Fix default target
b34ed403 cgroups: pass vhost-vsock device to cgroup
7362e1e8 runtime: remove prefix when cgroups are managed by systemd
1b1790fd agent/src: improve unit test coverage for src/namespace.rs
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
So that the debug console is more useful. In the meantime, remove
iptables as it is not used by kata-agent any more.
Fixes: #3138
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Let's take advantage of the "is-organization-member" action and only
allow members who are part of the `kata-containers` organization to
trigger `/test_kata_deploy`.
One caveat with this approach is that for the user to be considered as
part of an organization, they **must** have their "Organization
Visibility" configured as Public (and I think the default is Private).
This was found out and suggested by @jcvenegas!
Fixes: #3130
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
kernel compiled in fedora 35 (latest) is not working, following error
is reported:
```
qemu-system-x86_64: Error loading uncompressed kernel without PVH ELF
Note
```
Build QAT kernel in fedora 34 container to fix it
fixes#3135
Signed-off-by: Julio Montes <julio.montes@intel.com>
Some tests in sandbox.rs need root user to run, because they need create
directories under /run/agent directories, actually this is a limit
that shouldn't be there. By using a temp directory for test containers
will not need run tests as root user.
Fixes: #3122
Signed-off-by: bin <bin@hyper.sh>
Kata agent logs unknown system calls given by seccomp profiles
in advance before the log file descriptor closes.
Fixes: #2957
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Rather than comparing a string to a literal in the rust example,
use `.is_empty()` as that approach is more idiomatic and preferred.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add a comment stating that `anyhow` and `thiserror` should be used in
real rust code, rather than the unwieldy default `Result` handling
shown in the example.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Change some headings to avoid using the present continuous tense which
should not be used for headings.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Use a capital letter when referring to Golang and Rust (and remove
unnecessary backticks for Rust).
> **Note:**
>
> We continue refer to "Go" as "Golang" since it's a common alias,
> but, crucially, familiarity with this name makes searching for
> information using this term possible: "Go" is too generic a word.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Unit tests necessarily need to be maintained with the code they test so
it makes sense to keep the Unit Test Advice document into the main repo
since that is where the majority of unit tests reside.
Note: The
[`Unit-Test-Advice.md` file](https://github.com/kata-containers/tests/blob/main/Unit-Test-Advice.md)
was copied from the `tests` repo when it's `HEAD` was
38855f1f40.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Move the documentation requirements document link up so that it appears
immediately below the "How to Contribute" section.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
`grep`ing by a specific output, in a specific language, is quite fragile
and could easily break `hub`. For now, let's work this around following
James' suggestion of setting `LC_ALL=C LANG=C` when calling `hub`.
> **Note**: I don't think we should invest much time on fixing `hub`
> usage, as it'll be soon replaced by `gh`, see:
> https://github.com/kata-containers/kata-containers/issues/3083
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When branching the "stable-x.y" branch, we need to do some quite
specific changes to kata-deploy / kata-cleanup files, such as:
* changing the tags from "latest" to "stable-x.y".
* removing the kata-deploy / kata-cleanup stable files.
However, after the branching is done, we need to get the `main` repo to
its original state, with the kata-deploy / kata-cleanup using the
"latest" tag, and with the stable files present there, and this commit
ensures that, during the release process, a new PR is automatically
created with these changes.
Fixes: #3069
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Rather than doing the kata-deploy changes as part of the release bump
commit, let's split those on its own changes, as it will both make the
life of the reviewer less confusing and also allows us to start
preparing the field for a possible automated revert of these changes,
whenever it becomes needed.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Similarly to what was done for the yaml files, let's use a var for
representing the registry where our images will be pushed to and avoid
repetition and too long lines.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Instead of always writing the full path of some files, let's just create
some vars and avoid both repetition (which is quite error prone) and too
long lines (which makes the file not so easy to read).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We can simplify the code a little bit, as at least now we group common
operationr together. Hopefully this will improve the maintainability
and the readability of the code.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The comments were mentioning kata-deploy-base files while it really
should mention kata-deploy-stable files.
While here, I've also added a missing '"' to one of the tags.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
os.Exit() will terminate program immediately, the defer functions
won't be executed, so we add defer functions again before os.Exit().
Refer to https://pkg.go.dev/os#ExitFixes: #3059
Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
Replace some `unwrap()` and `expect()` calls with code to return the
error to the caller.
Fixes: #3011.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Improved the `verify_cid()` function that validates container ID's by
removing the need for an `unwrap()`.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Change `baremount()` to accept `Path` values rather than string values
since:
- `Path` is more natural given the function deals with paths.
- This minimises the caller having to convert between string and `Path`
types, which simplifies the surrounding code.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Although the binary name of the shipped binary is `qemu-system-x86_64`,
and we only ship kata-deploy for `x86_64`, we better leaving the
architecture specific name out of our README file.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The unrecognized option: 'deny-warnings' args caused `make optimize` failed.
Fixed the Makefile of the agent project, make sure the `make optimize` command
execute correctly. This PR modify the rustc args from '--deny-warnings' to
'--deny warnings'.
Fixes: #3104
Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
Commit 3c9ae7f made /test_kata_deploy run
against HEAD, but it also mistakenly removed all the checks that ensure
/test_kata_deploy only runs when explicitly called.
Mea culpa on this, and let's add the tests back.
Fixes: #3101
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Function parameters in test config is not used. This
commit will add under score before variable name
in test config.
Fixes: #3091
Signed-off-by: bin <bin@hyper.sh>
1. use ci/go-test.sh to replace the direct call to go test
2. fix data race test
3. install hook whether it is root or not
Fixes#1494
Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
This'll end up moving to hypervisors pkg, but let's stop using virtLog,
instead introduce hvLogger.
Fixes: #2884
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Today the hypervisor code in vc relies on persist pkg for two things:
1. To get the VM/run store path on the host filesystem,
2. For type definition of the Load/Save functions of the hypervisor
interface.
For (1), we can simply remove the store interface from the hypervisor
config and replace it with just the path, since this is all we really
need. When we create a NewHypervisor structure, outside of the
hypervisor, we can populate this path.
For (2), rather than have the persist pkg define the structure, let's
let the hypervisor code (soon to be pkg) define the structure. persist
API already needs to call into hypervisor anyway; let's allow us to
define the structure.
We'll probably want to look at following similar pattern for other parts
of vc that we want to make independent of the persist API.
In doing this, we started an initial hypervisors pkg, to hold these
types (avoid a circular dependency between virtcontainers and persist
pkg). Next step will be to remove all other dependencies and move the
hypervisor specific code into this pkg, and out of virtcontaienrs.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Today, acrn relies on sandbox level information, as well as a store
provided by common parts of the hypervisor. As we cleanup the
abstractions within our runtime, we need to ensure that there aren't
cross dependencies between the sandbox, the persistence logic and the
hypervisor.
Ensure that ACRN still compiles, but remove the setSandbox usage as
well as persist driver setup.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
We noticed s390x test failures on several of the watcher unit tests.
Discovered that on s390 in particular, if we update a file in quick
sucecssion, the time stampe on the file would not be unique between the
writes. Through testing, we observe that a 20 millisecond delay is very
reliable for being able to observe the timestamp update. Let's ensure we
have this delay between writes for our tests so our tests are more
reliable.
In "the real world" we'll be polling for changes every 2 seconds, and
frequency of filesystem updates will be on order of minutes and days,
rather that microseconds.
Fixes: #2946
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
The k8s SR-IOV plugin, when it assigns a VFIO device to a container, adds
an variable of the form PCIDEVICE_<identifier> to the container's
environment, so that the payload knows which device is which. The contents
of the variable gives the PCI address of the device to use.
Kata allows VFIO devices to be passed in to a Kata container, however it
runs within a VM which has a different PCI topology. In order for the
payload to find the right device, the environment variables therefore need
to be converted to list the guest PCI addresses instead of the host PCI
addresses.
fixes#2897
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
update_spec_devices() takes a bunch of updates for the device entries in
the OCI spec and applies them, adjusting things in both the linux.devices
and linux.resources.devices sections of the spec.
It's important that each entry in the spec only be updated once. Currently
we ensure this by first creating an index of where the entries are, then
consulting that as we apply each update, so that earlier updates don't
cause us to incorrectly detect an entry as being relevant to a later
update. This method works, but it's quite awkward.
This inverts the loop structure in update_spec_devices() to make this
clearer. Instead of stepping through each update and finding the relevant
entries in the spec to change, we step through each entry in the spec and
find the relevant update. This makes it structurally clear that we're only
updating each entry once.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We have a test case commented as testing the case where linux.devices is
empty in the OCI spec. While it's true that linux.devices is empth in this
example, the reason it fails isn't specifically because it's empty but
because it doesn't contain a device for the update we're trying to apply.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
update_spec_devices() explicitly checks for being called with an empty
container path and fails. We have a unit test to verify this behaviour.
But while an empty container_path probably does mean something has gone
wrong elsewhere, that's also true of any number of other bad paths. Having
an empty string here doesn't prevent what we're doing in this function
making sense - we can compare it to the strings in the OCI spec perfectly
well (though more likely we simply won't find it there).
So, there's no real reason to check this one particular odd case.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The DevIndex data structure keeps track of devices in the OCI
specification. We used to carry it around to quite a lot of
functions, but it's now used only within update_spec_devices(). That
means we can simplify things a bit by just open coding the maps we
need, rather than declaring a special type.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
update_spec_device() adjusts the OCI spec for device differences
between the host and guest. It is called repeatedly for each device
we need to alter. These calls are now all in a single loop in
add_devices(), so it makes more sense to move the loop into a renamed
update_spec_devices() and process all the fixups in one call.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently the DevNumUpdate structure is created with a path to a
device node in the VM, which is then used by update_spec_device().
However the only piece of information that update_spec_device()
actually needs is the VM side major and minor numbers for the device.
We can determine those when we create the DevNumUpdate structure.
This means we detect errors earlier and as a bonus we don't need to
make a copy of the vm path string.
Since that change requires updating 2 of the log statements, we take the
opportunity to update all the log statements to structured style.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
For each device in the OCI spec we need to update it to reflect the guest
rather than the host. We do this with additional device information
provided by the runtime. There should only be one update for each device
though, if there are multiple, something has gone horribly wrong.
Detect and report this situation, for safety.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
As we process container devices in the agent, we repeatedly call
update_spec_device() to adjust the OCI spec as necessary for differences
between the host and the VM. This means that for the whole of a pretty
complex call graph, the spec is in a partially-updated state - neither
fully as it was on the host, not fully as it will be for the container
within the VM.
Worse, it's not discernable from the contents itself which parts of the
spec have already been updated and which have not. We used to have real
bugs because of this, until the DevIndex structure was introduced, but that
means a whole, fairly complex, parallel data structure needs to be passed
around this call graph just to keep track of the state we're in.
Start simplifying this by having the device handler functions not directly
update the spec, but instead return an update structure describing the
change they need. Once all the devices are added, add_devices() will
process all the updates as a batch.
Note that collecting the updates in a HashMap, rather than a simple Vec
doesn't make a lot of sense in the current code, but will reduce churn
in future changes which make use of it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently update_spec_device() takes parameters 'vm_path' and 'final_path'
to give it the information it needs to update a single device in the OCI
spec for the guest. This bundles these parameters into a single structure
type describing the updates to a single device. This doesn't accomplish
much immediately, but will allow a number of further cleanups.
At the same time we change the representation of vm_path from a Unicode
string to a std::path::Path, which is a bit more natural since we are
performing file operations on it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
virtio_blk_device_handler(), virtio_blk_ccw_device_handler() and
virtio_scsi_device_handler() all take a clone of their 'device' parameter.
They appear to do this in order to get a mutable copy in which they can
update the vm_path field.
However, the copy is dropped at the end of the function, so the only thing
that's used in it is the vm_path field passed to update_spec_device()
afterwards.
We can avoid the clone by just using a local variable for the vm_path.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
update_spec_device() takes a 'final_path' parameter which gives the
name the device should be given in the "inner" OCI spec. We need this
for VFIO devices where the name the payload sees needs to match the
VM's IOMMU groups. However, in all other cases (for now, and maybe
forever), this is the same as the original 'container_path' given in
the input OCI spec. To make this clearer and simplify callers, make
this parameter an Option, and only update the device name if it is
non-None.
Additionally, update_spec_device() needs to call to_string() on
update_path to get an owned version. Rust convention[0] is to let the
caller decide whether it should copy, or just give an existing owned
version to the function. Change from &str to String to allow that; it
doesn't buy us anything right now, but will make some things a little
nicer in future.
[0] https://rust-lang.github.io/api-guidelines/flexibility.html?highlight=clone#caller-decides-where-to-copy-and-place-data-c-caller-control
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
update_spec_device() takes a 'host_path' parameter which it uses to locate
the device to correct in the OCI spec. Although this will usually be the
path of the device on the host, it doesn't have to be - a traditional
runtime like runc would create a device node of that name in the container
with the given (host) major and minor numbers. To clarify that, rename it
to 'container_path'.
We also update the block comment to explain the distinctions more
carefully. Finally we update some variable names in tests to match.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This error is returned if we have information for a device from the
runtime, but a matching device does not appear in the OCI spec. However,
the name for the device we print is the name from the VM, rather than the
name from the container which is what we actually expect in the spec.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
update_spec_devices() includes an unsafe block, in order to call the libc
functions to get the major and minor numbers from a device ID. However,
the nix crate already has a safe wrapper for this function, which we use in
other places in the file.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
- Even a directory could be a symlink - check for this. This is very
common when using configmaps/secrets
- Add unit test to better mimic a configmap, configmap update
- We would never remove directories before. Let's ensure that these are
added to the watched_list, and verify in unit tests
- Update unit tests which exercise maximum number of files per entry. There's a change
in behavior now that we consider directories/symlinks watchable as well.
For these tests, it means we support one less file in a watchable mount.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
The current implementation just copies the file, dereferencing any
simlinks in the process. This results in symlinks no being preserved,
and a change in layout relative to the mount that we are making
watchable.
What we want is something like "cp -d"
This isn't available in a crate, so let's go ahead and introduce a copy
function which will create a symlink with same relative path if the
source file is a symlink. Regular files are handled with the standard
fs::copy.
Introduce a unit test to verify symlinks are now handled appropriately.
Fixes: #2950
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Is the past few releases we ended up hitting issues that could be easily
avoided if `/test_kata_deploy` would use HEAD instead of a specific
tarball.
By the end of the day, we want to ensure kata-deploy works, but before
we cut a release we also want to ensure that the binaries used in that
release are in a good shape. If we don't do that we end up either
having to roll a release back, or to cut a second release in a really
short time (and that's time consuming).
Note: there's code duplication here that could and should be avoided,b
but I sincerely would prefer treating it in a different PR.
Fixes: #3001
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
While building snap, static qemu is considered. Disable libudev
as it doesn't have static libraries on most of the distros of all
archs.
Fixes: #3002
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
If a file/directory doesn't exist, os.Stat() returns an
error. Assert the returned value with os.IsNotExist() to
prevent it from failing.
Fixes: #2920
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
stable-2.3 was the first time we branched the repo since
43a72d76e2 was merged. One bit that I
didn't notice while working on this, regardless of being warned by
@amshinde (sorry!), was that the change would happen on `main` branch,
rather than on the branched `stable-2.3` one.
In my mind, the workflow was:
* we branch.
* we do the changes, including removing the files.
* we tag a release.
However, the workflow actually is:
* we do the changes, including removing the files.
* we branch.
* we tag a release.
A better way to deal with this has to be figured out before 2.4.0 is
out, but for now let's just re-add the files back.
Fixes: #3067
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Now the `wait` is passed to qmp command, even at non-server mode. This
will cause qemu return this error:
'wait' option is incompatible with socket in client connect mode
Fixes: #205
Signed-off-by: bin liu <liubin0329@gmail.com>
When the container didn't had a tty console, it would be in a same
process group with the kata-agent, which wasn't expected. Thus,
create a new session for the container process.
Fixes: #3063
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Many of these functions are just used on one place throughout the rest
of the code base. If we create hypervisor package, newtork package, etc, we may want to
parse this out.
Fixes: #3049
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
This will be useful at runtime level; no need for oci or uuid to be subpkg of
virtcontainers.
While at it, ensure we run gofmt on the changed files.
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Update the agent README by removing the historical details about the
conversion from golang to rust which (occurred at the start of Kata 2.x
development) and replacing it with information that developers and
testers should find more useful.
Fixes: #3056.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The fact that we need to "bridge" the endpoint is a bit irrelevant. To
be consistent with the rest of the endpoints, let's just call this
"macvlan"
Fixes: #3050
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
The thread monitor will check if the agent and the VMM are alive every
second in a blocking thread. The Cloud hypervisor API server is
single-threaded, if the monitor does a `check()`, while a slow request
is still in progress, the monitor check() method will timeout. The
monitor thread will stop all the shim-v2 execution.
This commit modifies the monitor thread to make it check the status of
the hypervisor after 5 seconds. Additionally, the `check()` method from
cloud-hypervisor will use the method `clh.isClhRunning(timeout)` with a
10 seconds timeout. The monitor function does no timeout, so even if
`hypervisor.check()` takes more 10 seconds, the isClhRunning method
handles errors doing a VmmPing and retry in case of errors until the
timeout is reached.
Reduce the time to the next check to 5 should not affect any functionality,
but it will reduce the overhead polling the hypervisor.
Fixes: #2777
Signed-off-by: Carlos Venegas <jose.carlos.venegas.munoz@intel.com>
CRI-O deployment documentation was quite outdated, giving info from the
`1.x` era. Let's update this to reflect what we currently have.
Fixes: #2498
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The links are either pointing to the not-used-anymore `master` branch,
or to the kubernetes-incubator page.
Let's always point to the CRI-O github page, using the `main`branch.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Although the documentation removed is correct, it's not relevant to the
current supported versions of CRI-O.
Related: #2498
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The main.yaml workflow was created and used only on 1.x. We inherited
it, but we didn't remove it after deprecating the 1.x repos.
While here, let's also update the reference to the `main.yaml` file,
and point to `release.yaml` (the file that's actually used for 2.x).
Fixes: #3033
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
There are two types packages under virtcontainers, and the
virtcontainers/pkg/types has a few codes, merging them into
one can make it easy for outstanding and using types package.
Fixes: #3031
Signed-off-by: bin <bin@hyper.sh>
According to https://endoflife.date/go golang 1.15 is not supported
anymore. Let's remove it from out tests, add 1.17.x, and bump the
newest version known to work when building kata to 1.17.3.
Fixes: #3016
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We now support any container engine CRI compliant in kata-monitor.
Update documentation to reflect it.
Fixes: #980
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
As github.com/containerd/cgroups doesn't support scope
units which are essential in some cases lets create
the cgroups manually and load it trough the cgroups
api
This is currently done only when there's single sandbox
cgroup (sandbox_cgroup_only=true), otherwise we set it
as static cgroup path as it used to be (until a proper
soultion for overhead cgroup under systemd will be
suggested)
Fixes: #2868
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
We need to explicitly call `${GOPATH}/bin/yq` that is installed by
`ci/install_yq.sh`.
Fixes: #3014
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
We now support any CRI container engine in kata-monitor, notably CRI-O.
Add both containerd and CRI-O in the kata metrics architecture image.
Signed-off-by: Francesco Giudici <fgiudici@redhat.com>
Update the `agent-ctl` tool to allow API fields to be specified in JSON
format, either directly on the command-line, or via a file URI.
This feature is made possible by enabling `serde` support in the agent
`protocols` crate. Careful use of the `serde` macros allows the
`agent-ctl` tool to accept _partially_ specified API objects in JSON
format; fields that are not specified are set to the default value for
their respective types.
`build.rs` changes based on work by Fupan.
Fixes: #2978.
Contributions-by: Fupan Li <lifupan@gmail.com>
Contributions-by: Bin Liu <bin@hyper.sh>
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This reverts commit 76f16fd1a7 to bring
back cri-containerd crioptions parsing so that kata works with older
containerd versions like v1.3.9 and v1.4.6.
Fixes: #2999
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Protection types like tdxProtection or seProtection were marked nolint,
remove this. As a side effect, ARM needs dummy tests for these.
Fixes: #2801
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
If Kata agent cannot resolve the system calls given by seccomp profiles,
the agent ignores the system calls and continues to run without an error.
Fixes: #2957
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
- Add support for legacy serial device
- Additionally add support for the file backend for chardev
Legacy serial plus char backend file will allow us to support
capture early boot messages.
Signed-off-by: Manohar Castelino <mcastelino@apple.com>
Ensure the tests in the local `logging` crate are run for all consumers
of it.
Additionally, add a new test which checks that output is generated by a
range of different log level `slog` macros. This is designed to ensure
debug level output is always available for the consumers of the
`logging` crate.
Fixes: #2969.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Fixed the top-level build which was broken: the kata deploy
Makefile was being sourced, but it was defining the first target, which
became the default.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
as `-realtime` has been removed in QEMU 6. `-overcommit` has been
supported since at least QEMU 3.1.
Fixes: #189
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Add support for --no-shutdown Knob. This allows us to
shutdown the VM without quitting QEMU.
Note: Also fix the comment around --no-reboot to be
more accurate.
Signed-off-by: Manohar Castelino <mcastelino@apple.com>
ExecuteSystemPowerdown issues `system_powerdown` and waits
for `SHUTDOWN`. The event emitted is `POWERDOWN` per spec.
Without this we get an error even though the VM has shutdown
gracefully.
Per QEMU spec:
```
POWERDOWN (Event)
Emitted when the virtual machine is powered down through the power
control system, such as via ACPI.
Since
0.12
Example
<- { "event": "POWERDOWN",
"timestamp": { "seconds": 1267040730, "microseconds": 682951 } }
SHUTDOWN (Event)
Emitted when the virtual machine has shut down, indicating that qemu is
about to exit.
Arguments
guest: boolean
If true, the shutdown was triggered by a guest request (such as a
guest-initiated ACPI shutdown request or other hardware-specific action)
rather than a host request (such as sending qemu a SIGINT). (since 2.10)
reason: ShutdownCause
The ShutdownCause which resulted in the SHUTDOWN. (since 4.0)
Note
If the command-line option “-no-shutdown” has been specified, qemu will
not exit, and a STOP event will eventually follow the SHUTDOWN event
Since
0.12
Example
<- { "event": "SHUTDOWN", "data": { "guest": true },
"timestamp": { "seconds": 1267040730, "microseconds": 682951 } }
```
Signed-off-by: Manohar Castelino <mcastelino@apple.com>
This adds fields to BridgeDevice struct to allow qemu's io-reserve,
mem-reserve and pref64-reserve properties to be set for PCI bridges.
This is needed for Kata's upcoming change to ACPI hotplug.
fixes#200
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
ExecuteBlockdevAddWithDriverCache has three one parameter driver
than ExecuteBlockdevAddWithCache.
Parameter driver can set the driver of block device.
Fixes: #198
Signed-off-by: Hui Zhu <teawater@antfin.com>
Use of the 'props' argument to 'object-add' has been deprecated since QEMU
5.0 (commit 5f07c4d60d09) in favor of flattening the properties directly
into the 'object-add' arguments. Support for 'props' is removed entirely
in qemu 6.0 (commit 50243407457a).
fixes#193
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Kata requires version 5.2 (or 5.1 on ARM) anyway. Simplify code by
dropping support for older versions. In any case explicit checks against
version number aren't necessarily reliable for patched qemu versions.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
ExecuteBlockdevAdd() and ExecuteBlockdevAddWithCache() both appear to be
intended to create block devices in the guest which backend onto a block
device in the host. That seems to be the way that Kata always uses it.
However blockdevAddBaseArgs(), used by both those functions always uses the
"file" driver, which is only intended for use with regular file backends.
Use of the "file" driver for host block devices was deprecated in qemu-3.0,
and has been removed entirely in qemu-6.0 (commit 8d17adf34f5). We should
be using the "host_device" driver instead.
fixes#191
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Update the govmm code in order to support "sandbox" feature on qemu,
which can introduce another protect layer on the host,
to make the secure container more secure.
Fixes: #185
Signed-off-by: Liang Zhou <zhoul110@chinatelecom.cn>
* Remove golang 1.13 and 1.14, add golang 1.16
* gometalinter has been deprecated, use golangci-lint instead
Signed-off-by: Julio Montes <julio.montes@intel.com>
Append `readonly=on` to a `memory-backend-file` object and
`unarmed=on` to a `nvdimm` device when `ReadOnly` is set to `true`
Signed-off-by: Julio Montes <julio.montes@intel.com>
Always join by ",", do not put commas in the parameter slices. Always
use the variable name `deviceParams`.
Fixes: #180
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Currently virtio-mem-pci devices can be hotplugged only on the root bus.
This doesn't work for PCIe machines like q35.
Extend the API to optionally support hotplugging on PCI bridges.
Fixes: #176
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
Adding the support for Protected Execution Facility(PEF) is
which is the confidential computing technology on ppc64le.
Fixes: #174
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
Secure Execution, also known as Protected Virtualization in QEMU, is a
confidential computing technology for s390x (IBM Z & LinuxONE). Allow
the respective object.
Fixes: #172
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Add CCW (s390x) device numbers to VhostUserDevices, as is with other
device types. Add them to VhostUserFS devices (the only type currently
supported on s390x) when building QEMU parameters.
Fixes: #170
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
by splitting out the respective functionality to QemuNetParams,
QemuSCSIParams, QemuBlkParams, and QemuFSParams. This allows adding
functionality to these functions without going beyond the cyclomatic
complexity of 15 mandated by the lint checks.
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Currently, if romfile field is empty, the commandline will
shows like below:
-device driver=virtio-net-pci,...,mq=on,vectors=4,romfile=
This does not make sense, just remove this field in commandline
Add unittest support.
Signed-off-by: Michael Qiu <qiudayu@huayun.com>
In Kata, we are getting a *lot* of logs at runtime from QMP, in particular `read from QMP: xxxx`
Ideally we'd set this to only be visible for trace, but I did not see this working when adding a
V(7) check around these prints. To avoid filling journal with info that isn't useful, let's drop.
Fixes: #165
Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
Some architectures and setups do not support DIMM/NUMA. However, they
can still use memory backends, provided a memory backend of the same ID
is specified under -machine. This was introduced in QEMU 5.0. Enable
this functionality in appendMemoryKnobs.
Fixes: #160
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Listening to the events channel from QEMU and a guest
panic event issued, then we can get the event and do some
work for the special event.
Fixes: #152
Signed-off-by: bin liu <bin@hyper.sh>
Remove CONTRIBUTORS.md file since, this repo is now part of the
kata-containers organization, the other repos don't have this file
and we are not willing to maintain (update) it.
Signed-off-by: Julio Montes <julio.montes@intel.com>
`govmm` is now part of the `kata-containers` GitHub organisation, so
update to reflect this.
Fixes: #145.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add ExecuteAPVFIOMediatedDeviceAdd to qmp.go, which executes a hotplug
for an IBM Adjunct processor (AP) VFIO device (see also
https://www.kernel.org/doc/html/latest/s390/vfio-ap.html )
Also includes the respective unittest and adds the VfioAP DeviceDriver
constant to qemu.go.
Pushing again due to incidental CI failure
Fixes: #133
Signed-off-by: Jakob-Naucke <jakob.naucke@ibm.com>
Reviewed-by: alicefr <afrosi@redhat.com>
Use github actions to run unit tests.
Github actions service looks more stable and reliable than travis.
fixes#136
Signed-off-by: Julio Montes <julio.montes@intel.com>
Fix the following error:
```
Bad response status from coveralls: 422
{"message":"service_job_id (717167073) must be unique for Travis Jobs
not supplying a Coveralls Repo Token","error":true}
The command "$GOPATH/bin/goveralls -v -service=travis-ci" exited with 1.
```
fixes#135
Signed-off-by: Julio Montes <julio.montes@intel.com>
The Kata architecture does not support rebooting VMs (the lifecycle
being start/exec/kill) and if a VM is killed (e.g. using sysrq-trigger),
the VM does not exit fully and other layers do not notice the state change.
Kata needs a way to tell QEMU to run with the '--no-reboot' option
so that the guest VM exits and does not attempt to reboot.
Add a NoReboot boolean Knob so when Knobs.NoReboot is set, the '--no-reboot'
command-line option will be passed to QEMU on startup.
Signed-off-by: Liam Merwick <liam.merwick@oracle.com>
multidevs specifies how to deal with multiple devices being shared with a 9p
export. `multidevs=remap` fixes the following warning:
```
9p: Multiple devices detected in same VirtFS export, which might lead to file
ID collisions and severe misbehaviours on guest!
You should either use a separate export for each device shared from host or
use virtfs option 'multidevs=remap'!
```
Signed-off-by: Julio Montes <julio.montes@intel.com>
vhost is a Netdev Tap Option used to configure a host TAP network interface
backend, according to the QMP API documentation the type for such option must
be a boolean. Use boolean type for vhost option to fix the following
error on recent versions of QEMU:
```
Invalid parameter type for 'vhost', expected: boolean
```
Signed-off-by: Julio Montes <julio.montes@intel.com>
The following options can be provided
Intremap: activates interrupt remapping
DeviceIotlb: enables device IOTLB support for the vIOMMU
CachingMode: enables Cahing Mode
See: https://wiki.qemu.org/Features/VT-d
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
There are three different types for the RTC clock: host, rt and vm.
Add `rt` to the list of RTC clocks.
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
Allow API consumers to change the maximum number of ports in the virtio-serial
devices, setting a lower number of ports can improve the boot time and
reduce the attack surface.
fixes#120
Signed-off-by: Julio Montes <julio.montes@intel.com>
Following on from #111 which added support for multiple virtio transports,
add code to use virtio-mmio as the transport when booting a guest with
the microvm machine type and add a microvm case when checking for
NUMA support. Also add a test case for machine string parsing.
Signed-off-by: Liam Merwick <liam.merwick@oracle.com>
According to QEMU's nvdimm documentation: When 'pmem' is 'on' and QEMU is
built with libpmem support, QEMU will take necessary operations to guarantee
the persistence of its own writes to the vNVDIMM backend.
Signed-off-by: Julio Montes <julio.montes@intel.com>
Currently, virtio transports for each device are determined with
architecture dependent build time conditionals. This isn't the ideal
solution, as virtio transports aren't exactly tied to the host's
architecture.
For example, aarch64 VMs do support both PCI and MMIO devices, and
after the recent introduction of the microvm machine type, that's also
the case for x86_64.
This patch extends each device that supports multiple transports with
a VirtioTransport field, so users of the library can manually specify
a transport for each device. To avoid breaking the compatibility, if
VirtioTransport is empty a behavior equivalent to the legacy one is
achieved by checking runtime.GOARCH and Config.Machine.Type.
Keeping support for isVirtioPCI/isVirtioCCW in qmp.go is a bit
tricky. Eventually, the hot-plug API should be extended so callers
must manually specify the transport for the device.
Signed-off-by: Sergio Lopez <slp@redhat.com>
As there's no guarantee that ".cache-size" is a supported QEMU property,
let's not add it to the QEMU command line when the user explicitly set
virtio_fs_cache_size to zero.
By not always setting ".cache-size" property we avoid errors like:
```
$ sudo podman --runtime=/usr/bin/kata-runtime run --security-opt label=disable -it fedora:31 /bin/bash
Error: failed to launch qemu: exit status 1, error messages from qemu log: qemu-kvm: -device vhost-user-fs-pci,chardev=char-88c350403e95d3db,tag=kataShared,cache-size=0M: Property '.cache-size' not found: OCI runtime error
```
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
Caller can hotplug vhost-user device via qmp.
The Qemu vhost-user device, like vhost-user-blk-pci and
vhost-user-scsi-pci can be hotplugged by qmp API:
ExecuteCharDevUnixSocketAdd() together with
ExecutePCIVhostUserDevAdd()
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Memory preallocation is just a property of different memory backends.
We should treat it similar to memory sharing property. Also rename
FileBackedMemShared to MemShared as it is just another memory backend
property that works with different memory backends not just file backed
memory.
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
The upper hyervisor manager application maybe need to wait some
QMP event to control boot sequence, but the event we wanted maybe
not exist in some older version, so we need query all QMP ABI and
check the event is supported or not.
related: kata-containers/runtime#1918
Signed-off-by: Ning Bo <ning.bo9@zte.com.cn>
Support for function isSocketIDSupported, isThreadIDSupported and isDieIDSupported.
The functions check if the cpu driver and the qemu version support the
id parameter.
Fixes: #102
Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
In QEMU 4.1 the CPU topology for x86 will change to:
`socket > die > core > thread`.
Add `die-id` field to `CPUProperties` and include it in CPU hotplugging
Signed-off-by: Julio Montes <julio.montes@intel.com>
since some vendor id like 1ded can not be identified by virtio-pci
driver, so upper level need to pass a specified vendor id to qemu.
the upper level will change unavailable id and pass it to qemu.
Signed-off-by: Ace-Tang <aceapril@126.com>
Hotplugged memory could be backed by a file on the host with sharing
turned on. This change allows qmp to pass that option to a govmm.
Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
QEMU supports finer-grained units than GiB. Change the cache size to
MiB so users have more control over the cache size.
Note that changing the semantics of the CacheSize field is fine because
there are no users of this API yet. kata-runtime will be the first
users and prefers MiB instead of GiB.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
In newer versions of QEMU, like 4.0-rc2, QMP events can be thrown even before
the QMP-version response, one example of this behaviour is when a virtio serial
is closed and a VSERPORT_CHANGE event is thrown.
Re-implement mainLoop to check the data received from the VM channel, since
it's not a guarantee that the first data read from the VM channel is the
QMP version.
fixes https://github.com/kata-containers/runtime/issues/1474
Signed-off-by: Julio Montes <julio.montes@intel.com>
Since []byte channel type transfer slice info(include slice underlying array pointer, len, cap)
between channel sender and receiver. scanner.Bytes() function returned slice's underlying array
may point to data that will be overwritten by a subsequent call to Scan(reference from:
https://golang.org/pkg/bufio/#Scanner.Bytes), which may make consecutive scan() call write the
read data into the same underlying array which causes receiver read mixed data,so we need to
copy line to new allocated space and then send to channel receiver to solve this problem.
Fixes: #88
Signed-off-by: jiangpengfei <jiangpengfei9@huawei.com>
The QEMU vhost-user-fs-pci device provides virtio-fs host<->guest file
system sharing (https://virtio-fs.gitlab.io/). The device is
instantiated like this:
$ qemu -chardev socket,path=/tmp/vhost-fs.sock,id=chr0
-device vhost-user-fs-pci,tag=myfs,chardev=chr0,cache-size=4G,versiontable=/dev/shm/fuse_shared_versions
This patch adds the VhostUserFS DeviceDriver and command-line generation
for this QEMU device.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
For vCPU hotplug to work on ppc64le, we need not
pass threadID and socketID. So conditionally pass
arguments when executing CPU device add.
Fixes: #83
Signed-off-by: Nitesh Konkar niteshkonkar@in.ibm.com
The .travis file was building GoVMM with some old of date versions of
Go that seem to be incompatible with the latest versions of gometalinter.
This commit updates the .travis file so that we build against 1.10 and
1.11.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
Static check was complaining about code that looked like
_ = <-ch
when it wants to see simply
<-ch
There was only one instance of this in govmm and this commit fixes
that instance.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
static check was complaining about code that looked like
if x == "" {
return false
}
return true
when what it wants to see is return x != "". This commit fixes the issue.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
staticcheck was complaining about code that looked like
if x == true {
}
rather than
if x {
}
This commit fixes the issue.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
staticcheck was complaining as there were quite a lot of
fmt.Sprintf("%s",d) in the code where d was either a string or
had string as its underlying type.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
staticcheck was complaining as some of the error messages returned by
govmm began with a capital letter. This commit fixes the issue.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
ExecuteNVDIMMDeviceAdd can add a nvdimm disk to qemu.
Not implement NVDIMM device delete function because qemu doesn't support it.
Signed-off-by: Hui Zhu <teawater@hyper.sh>
For devices that actually support the option disable-modern, this
current commit provides a proper flag to the caller. This will allow
for better support when used in nested environment as virtio-pci
devices should rely on virtio 0.9 instead of 1.0 due to a bug in
KVM.
Fixes#80
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
It turns out it is possible to run the unit tests for the s390x build
on travis by renaming the s390x specific files, so that their
inclusion in the build is determined only by tags and not by filename,
and by introducing a new tag s390x_test that we can use to force
their inclusion into a build by using this tag. The .travis file is
then updated to include the line
go test --tags s390x_test ./...
This creates a build on travis that includes the s390x specific
files and runs the unit tests.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
This commit adds a single command to the travis script that checks
that the s390x build works. We can't run the unit tests but at
least we can check that everything builds on this architecture.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
The PR adds the s390x support. It sets the CCW devices and sets to false
all the devices in the mapping isVirtioPCI. It reimplements the functions
QemuNetdevParam and QemuDeviceParam to print an error message if the vhost-user
devices are used. It introduces a new function ExecuteNetCCWDeviceAdd for qmp
for the CCW devices.
Fixes: #37
Co-authored-by: Yash D Jain <ydjainopensource@gmail.com>
Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
This commit updates the headers in the Go source files to adhere
to the new guidelines in the CONTRIBUTING.md file.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
The CONTRIBUTING.md file is updated to provide a template for new
source files and to invite contributors to add themselves to the
CONTRIBUTORS.md file.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
This file is a partial list of contributors to the Virtual Machine
Manager for Go project. To see the full list of contributors,
see the revision history in source control.
Contributors who wish to be recognized in this file should add
themselves (or their employer, as appropriate).
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
Only get 'QMP command failed' error message now when execute QMP
command by 'executeCommandWithResponse' failed. This patch will
output more error detail.
Signed-off-by: NingBo <ning.bo9@zte.com.cn>
This PR prepares for the s390x support. It introduces:
- a generalization of ccw and pci devices. The variables for the pci devices
have been renamed by removing the Pci suffix. They have been moved to the
qemu_arch_base.go
- the mapping isVirtioPCI has been move to qemu_arch_base.go because in
this way a different mapping can be added for other architecture (e.g
s390x)
- the functions QemuNetdevParam and QemuDeviceParam have been moved to
qemu_arch_base.go. In this way, they could be reimplemented for other
architecture for the case VHOSTUSER
- a function disableModern has been introduced to check if the device is
a pci device and then returns the right parameters. In the case of ccw
devices, they don't have the disable-modern flag
- a function mqParameter has been introduced to return the right
parameters for the mq case. The virtio-net-ccw device doesn't have the
vectors flag
- in qemu_arch_base_test.go contains the test and strings that can be
overwritten for other architectures (e.g s390). The devices names and
the flags for the devices can be overwritten.
- the string for the romfile has been replaced by a variable romfile
that could be left empty if the devices doesn't support a romfile as
for the ccw devices for s390.
- clean-up: the disable-modern=on/off options have been changed to
disable-modern=true/false. In the code there was a mixture of on/true
off/false
Fixes: #61
Co-authored-by: Yash D Jain <ydjainopensource@gmail.com>
Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
ExecuteBlockdevAddWithCache has two more parameters direct and noFlush
than ExecuteBlockdevAdd.
They are cache-related options for block devices that are described in
https://github.com/qemu/qemu/blob/master/qapi/block-core.json.
direct denotes whether use of O_DIRECT (bypass the host page cache)
is enabled. noFlush denotes whether flush requests for the device are
ignored.
Signed-off-by: Hui Zhu <teawater@hyper.sh>
Also change TestQMPXBlockdevDel to TestQMPBlockdevDel because QMP verion
2.9 and older use blockdev-del but not x-blockdev-del.
Signed-off-by: Hui Zhu <teawater@hyper.sh>
Add input for -pidfile option of qemu, so that we can get pid of
qemu main process, and apply resource limitations to it.
Fixes#62
Signed-off-by: l00397676 <lujingxiao@huawei.com>
This patch fixes the wrong behavior of specifying a netdev, MAC
address or PCI address entry when those were empty. Instead, it
does not provide those entries if the content is empty.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Any device inheriting from virtio-pci can specify a ROM file. This
option is provisioned by default with "efi-virtio.rom", but most
of the time, firmwares such as OVMF or seabios will already support
what is provided by this ROM file.
In order to reduce the "forced" dependency on such ROM file, govmm
should provide an empty path if the consumer of the library does not
provide one.
This patch reorganizes the list of devices, so that it gets easier to
list which devices inherit from virtio-pci, and then adds the romfile
option to every single device that support this option.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Add support for virtio-balloon.
- Add test
- Support disable-modern
- Support deflate-on-oom
Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
Rather than show the generic "qemu", log the full path to the
particular qemu binary being used.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
In case the type of bridge is PCIEBridge, which we expect as ending
up using pcie-pci-bridge device from Qemu, the properties chassis_nr
and shpc don't exist.
This commit simply fixes this use case by removing those parameters
from the command line.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The memory-backend-ram should also be set to a numa node instead of
being inserted as a new device. Otherwise it becomes additional memory
and requires explicit online to be available, instead of just being a
backend of the memory specified by -m option.
Signed-off-by: Peng Tao <bergwolf@gmail.com>
In addition to supporting hotplug for VFIO mediated device on PCI bridge,
this patch adds hotplug functionality on root bus.
When parameter bus and addr are set to be empty, the system will pick up
an empty slot on root bus.
Signed-off-by: Zhao Xinda <xinda.zhao@intel.com>
The contents of .iso used to bootstrap VMs with cloudinit are
initialised using a precreated, short-lived directory. The
permissions on this directory were too lenient. This commit
restricts access to this directory to the user and his/her group.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
govmm has ExecuteBlockdevAdd() function and ExecuteBlockdevDel() function
doesn't compatible with qemu 2.8,because blockdev-add and x-blockdev-del usages
are different between qemu 2.7 and qemu 2.8
Follow the qemu 2.7 and qemu 2.8 qmp-commands.txt documents to modify ExecuteBlockdevAdd()
function and ExecuteBlockdevDel() function to be compatible with qemu 2.8
Signed-off-by: flyflypeng <jiangpengfei9@huawei.com>
If we hotplug a nic with args mq=on, its qdisc will be mq by default.
This aligns with cold plug nics.
Signed-off-by: Ruidong Cao <caoruidong@huawei.com>
Before calling any other command it is necessary to call
ExecuteQMPCapabilities() otherwise QEMU will not process the subsequent QMP
commands.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
In order to be able to hotplug network devices such as vhost user
net, we need to be able to define a previously declared chardev as
a parameter of this new network device.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This commit adds a couple of negative test cases for qmp.go, one
which checks that failed commands return errors and the other
checks that QMPStart exits gracefully when passed an invalid
socket path.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
This commit adds some negative test cases for the append functions
in qemu.go that build up the qemu command line.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
This will kill the process when the context is cancelled. As using a nil
context is not permitted it is necessary to substitute with a real
context if it is not initialised in the Config struct.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
If no RTC is specified in the config then do not generate any RTC command line
options. RTC command line options are optional for QEMU so make Valid() return
false when presented with the empty version of the RTC struct containing empty
strings.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
In addition to normal VFIO device, this patch adds VFIO mediated device
as a supplement to do hot plug on PCI(E) bridges.
Signed-off-by: Zhao Xinda <xinda.zhao@intel.com>
With qemu 2.10, a write lock was added for qcow images that
prevents the same image to be passed more than once.
This can be over-ridden using the --share-rw option which is
desired for raw images.
This solves an issue with running Kata with devicemapper
using the privileged mode as in this case all devices on the host
are passed to the container using the block device associated
with the rootfs, causing it to be passed twice to qemu.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
For machines types based on PCIe like q35, device addr and bus must be specified.
For machines types based on PCI like pc, device addr must be specified and bus
is optional since devices can be hot plugged directly on the root bus.
Signed-off-by: Julio Montes <julio.montes@intel.com>
Implement function to hotplug a network device to QEMU by fds.
Macvtap can only be hotplug by this way.
Signed-off-by: Ruidong Cao <caoruidong@huawei.com>
Implement function to hotplug virtio serial ports, the serial ports
are visible in the guest at the directory /dev/virtio-ports.
Signed-off-by: Julio Montes <julio.montes@intel.com>
implement function to hotplug character devices using as backend
unix sockets, binding a character device with a serial port allows
the communnication between processes running in the guest with
processes running in the host.
Signed-off-by: Julio Montes <julio.montes@intel.com>
`getfd` receives a file descriptor via SCM rights and assign it a name,
this command is useful to send file descriptors from the host, and then
hot plug devices that needs file descriptors like vhost-vsock-pci devices.
Signed-off-by: Julio Montes <julio.montes@intel.com>
`vhostfd` is used to specify the vhost-vsock device fd, and it holds
the context ID previously opened.
`disable-modern` is to disable the use of "modern" devices, by using virtio 0.9
instead of virtio 1.0. Particularly, this is useful when running the VM in a
nested environment.
Signed-off-by: Julio Montes <julio.montes@intel.com>
`vhostfd` is the vhost file descriptor that holds the socket context ID
`disable-modern` prevents qemu from relying on fast MMIO
Signed-off-by: Julio Montes <julio.montes@intel.com>
Implement function to hotplug vsocks, vsocks are needed
to communicate processes are running inside the VM
with processes are running on the host.
Signed-off-by: Julio Montes <julio.montes@intel.com>
This commit fixes an issue with the log handlers defined by qmp_test.
The issue was picked up by the latest version of go vet on go tip.
qemu/qmp_test.go:56::error: missing ... in args forwarded to printf-like function (vet)
qemu/qmp_test.go:60::error: missing ... in args forwarded to printf-like function (vet)
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
This commit enables the gas security checker on govmm builds. The
security checker has signalled 4 issues all of which I've checked
and have determined to be non issues. These issues are disabled
by this commit.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
This commit enables staticcheck in the travis builds and fixes the existing
errors detected by staticcheck. There was one type of error repeated in
qemu.go in which the type of some constants was not explicitly specified.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
It is useful when we want to specify migration incoming source.
Supported source are fd and exec right now.
Signed-off-by: Peng Tao <bergwolf@gmail.com>
It allows a caller to use a local file as the memory backend of the
guest, and it also allows the file backed memory device to be set shared
or not.
Signed-off-by: Peng Tao <bergwolf@gmail.com>
We need to be able to specify the PCI slot for a bridge while
adding it.
Add test to verify bridge is correctly added.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This enable data-plane for scsi. All drives attached to the
scsi controller will have their IO processed in a single separate
IO thread instead of qemu's main event loop.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
IOthreads also known as x-data-plane allow IO to
be processed in a separate thread rather than the main event
loop. This produces much better IO throughput and latency.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
certain machines types need to have options to enable or disable features
For example the machine type virt in certain hosts must have the gic version
(gic-version=3 or gic-version=host) to start without problems
Signed-off-by: Julio Montes <julio.montes@intel.com>
device_add qmp command for scsi devices accepts additional parameters like
scsi-id and lun. Implement function to add scsi devices. Devices
with drivers "scsi-hd", "scsi-cd" and "scsi-disk" are accepted.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This commit enables unit test coverage computation in Travis CI builds.
Going forward, builds that decrease the unit test coverage by more than
1.0% will fail.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
VSOCK sockets are added through a vhost PCI device.
It takes a device ID and a context ID, the latter being
the endpoint value to be reached from the host.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Introduce basic vhost-user-blk-pci support.
In adding this, cleaned up the QemuParams function to use a more
appropriate switch statement. Similarly, cleanup up the Valid() logic.
We still need to look into parameterization of the block parameter
fields as well as introducing multiqueue support for the vhost-user devices.
Signed-off-by: Eric Ernst <eric.ernst@intel.com>
Some comments were network specific for vhost-user devices, which is
incorect. Fixed these.
Renamed the HWAddress field to be Address, so that it could potentially
be used more generically for non-network based vhost-user types.
Signed-off-by: Eric Ernst <eric.ernst@intel.com>
maxcpus is used to specify how many cpus a VM can have.
This attribute must be specified to enable the hotplugging CPUs capability,
otherwise the maximum number of CPU will be defined by the number of CPU
in -smp.
Signed-off-by: Julio Montes <julio.montes@intel.com>
There were some unchecked errors in some of the unit files relating to
the closure and removal of temporary files. As the closure and removal
of these files is not really important to whether the next passes or
fails we ignore the errors.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
This commit adds a .travis file which enables Travis builds for
govmm. The script builds the source and runs the unit tests
and gometalinter enabling
- misspell
- vet
- ineffassign
- gofmt
- gocyclo 15
- golint
- errcheck
- deadcode
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
This commit adds three documents:
- CONTRIBUTING.md ( a files describing how to contribute to the project )`
- COPYING ( the Apache 2.0 license )
- README.md ( a brief description of the project)
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
This commit removes all the references to the ciao project. It also removes
some of the dependencies that the unit tests were pulling in.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
Add ability to add a vhostuser device to the
QEMU commandline. We expect two different types of devices
to be connected through a vhostuser socket: SCSI and network.
Signed-off-by: Eric Ernst <eric.ernst@intel.com>
Some QMP commands like ```query-hotpluggable-cpus``` returns a
response that needs to be processed and returned to the client as
a struct. This patch adds the function ```executeCommandWithResponse```
that returns the response of a QMP command.
Signed-off-by: Julio Montes <julio.montes@intel.com>
This patch adds a new function to hot plug VFIO devices on PCI(E) bridges,
This change allows to hot plug N VFIO devices in Qemu PC and Q35
Signed-off-by: Julio Montes <julio.montes@intel.com>
This change adds an additional parameter to CreateCloudInitISO that
allows users more control over the newly created xorriso process.
They can for instance specify the user under which the new qemu process
should run and which capabilities should be retained in the child
xorriso process.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
This change adds an additional parameter to LaunchCustomQemu that
allows users more control over the newly created process. They can
for instance specify the user under which the new qemu process should
run and which capabilities should be retained in the child qemu
process.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
ExecutePCIDeviceAdd is a function that can be used to hot plug
devices directly on pci(e).0 or pci(e) bridges. ExecutePCIDeviceAdd
is PCI specific because unlike ExecuteDeviceAdd, it includes an
extra parameter to specify the device address on its parent bus.
Signed-off-by: Julio Montes <julio.montes@intel.com>
Bridge struct represent pci bridges(pci-bridge) or
pcie bridges(pcie-pci-bridges), bridges can be used to
hot plug devices in pc and q35 machines
Signed-off-by: Julio Montes <julio.montes@intel.com>
Add support for macvtap interfaces. This also brings in support
for generic multiqueue support in virt containers.
Signed-off-by: Manohar Castelino <manohar.r.castelino@intel.com>
Add support to launch virtual machines where the RAM is
allocated using huge pages. This is useful for running
with a user mode networking stack, and for custom setups
which require high performance and low latency.
Signed-off-by: Manohar Castelino <manohar.r.castelino@intel.com>
VFIO is meant for exposing exposing direct device access
to the virtual machine.
Add ability to append VFIO devices to qemu command line.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Ciao has recently moved from github.com/01org/ciao to
github.com/ciao-project/ciao. This moves requires us to update our
import paths to build successfully.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
The Mlock knob is unfortunately tied to realtime.
Allow Mlock knob to implicitly enable realtime to get the
desired swapping behavior when swapping is desired.
Note: Realtime as implemented today can only be used to enable
swap, and as such does not really control realtime behaviour.
The knob is redundant but retained here just to ensure that
when more capabilities are added in future QEMU iterations
we can take advantage of the same.
Signed-off-by: Manohar Castelino <manohar.r.castelino@intel.com>
Enable realtime options in QEMU. Also add support to control memory
locking. Turning realtime on with memory locking disabled allows
memory to be swapped out, potentially increasing density of VMs.
Signed-off-by: Manohar Castelino <manohar.r.castelino@intel.com>
Add support for pre-allocating all of the RAM.
This increases the memory footprint of QEMU and should be used
only when needed.
Signed-off-by: Manohar Castelino <manohar.r.castelino@intel.com>
blockdev-del command has been added in qemu 2.9 to replace
x-blockdev-del command used earlier for deleting block devices.
Update ExecuteXBlockdevDel() to use this updated qmp command.
Rename ExecuteXBlockdevDel to ExecuteBlockdevDel as this no longer
executes x-block-del command for qemu>=2.9.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
For some cases, we have to disable the fast MMIO support, by disabling
virtio 1.0. The reason for this is that we want to be able to nest our
qemu VM inside a VM run by an hypervisor with no support for fast MMIO.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In case of a network device, and specifically virtio-net-pci, we have
to update to what is expected by qemu. In this case, the driver name
should be prefixed with "driver=".
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
With qemu 2.9, the qmp block-dev command was updated from:
{ "execute": "blockdev-add", "arguments": { "options": { ... } } }
to:
{ "execute": "blockdev-add", "arguments": { ... } }
Also, instead of id, blockdev-add now requires a node-name for the
root node(https://wiki.qemu.org/index.php/ChangeLog/2.9)
Store the version information with QMPStart and use that to issue
qmp command for adding block devices in the correct format.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Unfortunately the ununused linter is overzealous with some of the fields
that it things are unused as gophercloud relies on their values. So go
ahead with the most straightforward removals but do not enable unused on
travis builds.
ciao-image/datastore/datastore_test.go:28:5⚠️ var metaDsTables is unused (U1000) (unused)
ciao-controller/api/api_test.go:39:6⚠️ func myHostname is unused (U1000) (unused)
ciao-cli/identity.go:58:3⚠️ field Description is unused (U1000) (unused)
ciao-cli/identity.go:59:3⚠️ field DomainID is unused (U1000) (unused)
ciao-cli/identity.go:60:3⚠️ field Enabled is unused (U1000) (unused)
ciao-cli/identity.go:62:3⚠️ field ParentID is unused (U1000) (unused)
ciao-cli/identity.go:63:3⚠️ field Links is unused (U1000) (unused)
ciao-cli/identity.go:70:3⚠️ field Self is unused (U1000) (unused)
ciao-cli/identity.go:71:3⚠️ field Previous is unused (U1000) (unused)
ciao-cli/identity.go:72:3⚠️ field Next is unused (U1000) (unused)
ciao-cli/identity.go:207:3⚠️ field Next is unused (U1000) (unused)
ciao-cli/identity.go:208:3⚠️ field Previous is unused (U1000) (unused)
ciao-cli/identity.go:209:3⚠️ field Self is unused (U1000) (unused)
ciao-cli/identity.go:213:3⚠️ field Description is unused (U1000) (unused)
ciao-cli/identity.go:214:3⚠️ field DomainID is unused (U1000) (unused)
ciao-cli/identity.go:215:3⚠️ field Enabled is unused (U1000) (unused)
ciao-cli/identity.go:217:3⚠️ field Links is unused (U1000) (unused)
ciao-cli/identity.go:221:3⚠️ field ParentID is unused (U1000) (unused)
ciao-cli/main.go:105:6⚠️ type action is unused (U1000) (unused)
ciao-cli/volume.go:37:6⚠️ type customVolumeExt is unused (U1000) (unused)
ciao-cli/volume.go:39:2⚠️ field customVolumeExt is unused (U1000) (unused)
networking/ciao-cnci-agent/network.go:98:8⚠️ const maxKey is unused (U1000) (unused)
networking/libsnnet/tests/parallel/parallel_test.go:371:6⚠️ func dockerNetList is unused (U1000) (unused)
networking/libsnnet/tests/parallel/parallel_test.go:379:6⚠️ func dockerNetInfo is unused (U1000) (unused)
openstack/compute/api.go:308:2⚠️ const limit is unused (U1000) (unused)
openstack/compute/api.go:309:2⚠️ const marker is unused (U1000) (unused)
openstack/compute/api.go:312:6⚠️ type pager is unused (U1000) (unused)
openstack/compute/api.go:313:2⚠️ func pager.filter is unused (U1000) (unused)
openstack/compute/api.go:314:2⚠️ func pager.nextPage is unused (U1000) (unused)
openstack/compute/api_test.go:34:6⚠️ func myHostname is unused (U1000) (unused)
ciao-controller/api.go:72:2⚠️ const statusFilter is unused (U1000) (unused)
ciao-controller/api.go:75:6⚠️ type pager is unused (U1000) (unused)
ciao-controller/api.go:76:2⚠️ func pager.filter is unused (U1000) (unused)
ciao-controller/api.go:77:2⚠️ func pager.nextPage is unused (U1000) (unused)
ciao-controller/api.go:136:25⚠️ func (*nodePager).filter is unused (U1000) (unused)
ciao-controller/api.go:198:31⚠️ func (*nodeServerPager).filter is unused (U1000) (unused)
ciao-controller/controller_test.go:107:6⚠️ func addTestTenantNoCNCI is unused (U1000) (unused)
ciao-controller/controller_test.go:1104:6⚠️ func startTestWorkload is unused (U1000) (unused)
ciao-controller/controller_test.go:1123:6⚠️ func testStartWorkloadLaunchCNCI is unused (U1000) (unused)
ciao-controller/openstack_compute.go:552:5⚠️ field Links is unused (U1000) (unused)
qemu/qmp_test.go:493:3⚠️ const seconds is unused (U1000) (unused)
qemu/qmp_test.go:494:3⚠️ const microsecondsEv1 is unused (U1000) (unused)
qemu/qmp_test.go:495:3⚠️ const device is unused (U1000) (unused)
qemu/qmp_test.go:496:3⚠️ const path is unused (U1000) (unused)
templateutils/example_test.go:53:3⚠️ field hidden is unused (U1000) (unused)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Add structcheck to the list of linters used on travis runs.
ciao-cli/event.go:109:2⚠️ unused struct field github.com/01org/ciao/ciao-cli.eventDeleteCommand.all (structcheck)
ciao-cli/event.go:110:2⚠️ unused struct field github.com/01org/ciao/ciao-cli.eventDeleteCommand.tenant (structcheck)
ciao-cli/external_ips.go:636:2⚠️ unused struct field github.com/01org/ciao/ciao-cli.poolAddCommand.ips (structcheck)
ciao-cli/node.go:43:2⚠️ unused struct field github.com/01org/ciao/ciao-cli.nodeListCommand.nodeID (structcheck)
ciao-controller/client_wrapper_test.go:29:2⚠️ unused struct field github.com/01org/ciao/ciao-controller.ssntpClientWrapper.ctl (structcheck)
qemu/qmp.go:111:2⚠️ unused struct field github.com/01org/ciao/qemu.qmpResult.data (structcheck)
ssntp/ssntp_test.go:193:2⚠️ unused struct field github.com/01org/ciao/ssntp_test.ssntpClient.evtTracedChannel (structcheck)
ssntp/ssntp_test.go:192:2⚠️ unused struct field github.com/01org/ciao/ssntp_test.ssntpClient.staTracedChannel (structcheck)
ssntp/ssntp_test.go:194:2⚠️ unused struct field github.com/01org/ciao/ssntp_test.ssntpClient.errTracedChannel (structcheck)
ssntp/server.go:75:2⚠️ unused struct field github.com/01org/ciao/ssntp.Server.roleVerify (structcheck)
networking/ciao-cnci-agent/client.go:97:2⚠️ unused struct field github.com/01org/ciao/networking/ciao-cnci-agent.agentClient.netCh (structcheck)
testutil/agent.go:37:2⚠️ unused struct field github.com/01org/ciao/testutil.SsntpTestClient.ticker (structcheck)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
And add varcheck to the list of linters used on travis runs (with an
increased deadline.)
ciao-launcher/qemu_test.go:31:5⚠️ unused variable or constant imageInfoTestGood (varcheck)
ciao-launcher/qemu_test.go:44:5⚠️ unused variable or constant imageInfoTestMissingBytes (varcheck)
ciao-launcher/qemu_test.go:57:5⚠️ unused variable or constant imageInfoTestMissingLine (varcheck)
ciao-launcher/qemu_test.go:69:5⚠️ unused variable or constant imageInfoTooBig (varcheck)
ciao-launcher/qemu_test.go:82:5⚠️ unused variable or constant imageInfoBadBytes (varcheck)
configuration/configuration_test.go:35:7⚠️ unused variable or constant glanceURL (varcheck)
ciao-controller/controller_test.go:1918:5⚠️ unused variable or constant testClients (varcheck)
qemu/qmp_test.go:44:2⚠️ unused variable or constant qmpSuccess (varcheck)
qemu/qmp_test.go:45:2⚠️ unused variable or constant qmpFailure (varcheck)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The existing NetDevice relies on virtio-net driver, but there is a
useful PCI variant which was not available: virtio-net-pci.
This patch adds this new driver and adds two parameters specific to
this: "bus" and "addr".
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The code that handles the serialization and cancelling of QMP commands
is a little complex and it took me some time to remember how it actually
works and why it works in this particular way. For this reason I've
added some comments which will hopefully make the next bug fix in this
area a little less painful.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
There was a bug with the cancelling of commands that meant that when
an attempt was made to cancel a command and then to issue a second
command, the first, cancelled command was re-issued. This commit
fixes the issue and adds a new test case to check that cancelling
of commands does indeed work. There was also an issue with the
test harness which meant that tests that issued more than one command
were not actually testing the second and third commands.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
Launcher's ConfigDrive ISO creation function, createCloudInitISO has
been moved to the qemu package so that it can be re-used by ciao-down.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
When creating a CharDevice, we need to add a "bus" parameter
so that it can match the serial pci device previously created.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
We add a new device driver, and also a name to the CharDev structure
this is needed for qemu to actually create the serial port on
the guest.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
The QMP socket implementation does not support multiple clients sending
and receiving QMP commands. As a consequence we need to be able to
create multiple QMP sockets from the qemu package, so that at least we
can support a fixed number of QMP clients.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
When we get no virtual console to plug into, we may want qemu to create
a socket where we can asynchronously connect to.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
All file descriptors will come from specific devices configurations, so
this patch:
1) Make the Config FDs file private
2) Provide an appendFDs() method for Config, that takes a slice of
os.File pointers and
a) Adds them to the Config private fd slice
b) Return a slice of ints that represent the file descriptors for
these device specific files, as seen by the qemu process.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
It is a private field now, and all append*() routines are now
Config methods instead of private qemu functions.
Since we will have to carry a kernelParams private field as well,
this change will keep all built parameters internal and make things
consistent.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
By adding QemuParams() to the Device interface, we can get rid of the
driver structure and simplify further the appendDevices() routine.
With that implementation we can generate the following qemu parameters:
"-device virtio-9p-pci,fsdev=foo,mount_tag=rootfs -fsdev local,id=foo,path=/bar/foo,security-model=none"
from these single structures:
fsdev := FSDevice{
Driver: Virtio9P
FSDriver: Local,
ID: "foo",
Path: "/bar/foo",
MountTag: "rootfs",
SecurityModel: None,
}
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Instead of open coding the RTC fields, we now have specific types for
it.
We also have a RTC unit test now.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Having separate structures for the qemu driver definitions
and each possible device definitions is confusing and error prone as one
needs to be very careful using matching IDs and names in both
structures.
As the driver parameter can be derived from the device
ones, this patch changes the Device and Driver structures to be linked
together, i.e. each driver needs to have its corresponding device.
For example this allows us to build the following 9pfs qemu parameters:
"-fsdev local,id=foo,path=/bar/foo,security-model=none -device virtio-9p-pci,fsdev=foo,mount_tag=rootfs"
from these structures:
fsdev := FSDevice{
Driver: Local,
ID: "foo",
Path: "/bar/foo",
MountTag: "rootfs",
SecurityModel: None,
}
driver := Driver{
Driver: Virtio9P,
Device: fsdev,
}
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
With the NetDev and MACAddress strings, we can now create networking
device drivers.
We also add a unit test for netdev Device creation.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
We can now specify if we want vhost to be enabled and wich fds we should
use for multiqueue support.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
The NetDevice structure represents a network device to be emulated by
qemu.
We also add the corresponding unit test.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
The Knobs structure groups all qemu isolated boolean settings.
For now this is -no-user-config, -no-defaults and -nographic.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
The extraParams is confusing and can conflict with the rest of the
Config structure definitions.
We remove it and will add new fields to that structure as needed.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Here we group the machine type and acceleration together as they are
defined through the same qemu parameter (-machine).
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
The Global string represents the set of default Device driver properties
we want qemu to use. This is mostly useful for automatically created
devices.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
The SMP structure defines the amount of virtual CPUs, sockets, and
threads per CPU that is made available to the guest.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
The Memory field holds the guest memory configuration.
It is used to define the current and maximum RAM is made available to
the guest and how this amount of RAM is splitted into several slots.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Qemu character devices typically allow for sending traffic from the
guest to the host by emulating a console, a tty, a serial device for
example.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Each Filesystem device should have a corresponding "virtio-9p-pci"
Device driver. They represent a filesystem to be exported through 9pfs.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
The Kernel structure holds the guest kernel configuration: its path and
its parameters. This is the kernel qemu will boot the VM from.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
The Object slice tells qemu which specific object to create.
Qemu objects can represent memory backend files, random number
generators, TLS credentials, etc...
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
We may need to support a large range of devices in the qemu created VM
and the Device slice allows us to define which drivers are needed.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
QMP sockets are used to send qemu specific commands to the running qemu
process.
The QMPSocket structure allows us to define the socket type we want,
along with its name.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
LaunchQemu() now takes a Config structure that contains some more
descriptive fields than raw qemu parameter strings.
LaunchQemu is now simpler to call and more extensible as supporting more
qemu parameters would mean expanding Config instead of changing the API.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Ciao will use the new standard library context package from now on.
This will allow us to use some of the new standard library functions
such as DialContext.
Partial fix for issue #541
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
Fix ciao/qemu/qmp.go:349:3: ineffectual assignment to ok.
Strictly speaking this is a bug in ineffassign but it's easier
to change the ciao code.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
There's no point in setting cmd.ExtraFiles if the fds array is an
empty slice. This won't do any harm but is essentially a no-op.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
This commit adds some package documentation to the qemu package,
including an overview of the package and an example of its use.
Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
# Consider using larger runners or machines with greater resources for possible analysis time improvements.
runs-on:ubuntu-24.04
permissions:
# required for all workflows
security-events:write
# required to fetch internal or private CodeQL packs
packages:read
# only required for workflows in private repositories
actions:read
contents:read
strategy:
fail-fast:false
matrix:
include:
- language:go
build-mode:manual
- language:python
build-mode:none
# CodeQL supports the following values keywords for 'language': 'c-cpp', 'csharp', 'go', 'java-kotlin', 'javascript-typescript', 'python', 'ruby', 'swift'
# Use `c-cpp` to analyze code written in C, C++ or both
# Use 'java-kotlin' to analyze code written in Java, Kotlin or both
# Use 'javascript-typescript' to analyze code written in JavaScript, TypeScript or both
# To learn more about changing the languages that are analyzed or customizing the build mode for your analysis,
# see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning.
# If you are analyzing a compiled language, you can modify the 'build-mode' for that language to customize how
# your codebase is analyzed, see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages
# Add any setup steps before running the `github/codeql-action/init` action.
# This includes steps like installing compilers or runtimes (`actions/setup-node`
# or others). This is typically only required for manual builds.
# - name: Setup runtime (example)
# uses: actions/setup-example@v1
# Initializes the CodeQL tools for scanning.
- name:Initialize CodeQL
uses:github/codeql-action/init@v3
with:
languages:${{ matrix.language }}
build-mode:${{ matrix.build-mode }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# For more details on CodeQL's query packs, refer to: https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
# queries: security-extended,security-and-quality
# If the analyze step fails for one of the languages you are analyzing with
# "We were unable to automatically build your code", modify the matrix above
# to set the build mode to "manual" for that language. Then modify this step
# to build your code.
# ℹ️ Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
## This repo is part of [Kata Containers](https://katacontainers.io)
For details on how to contribute to the Kata Containers project, please see the main [contributing document](https://github.com/kata-containers/community/blob/master/CONTRIBUTING.md).
For details on how to contribute to the Kata Containers project, please see the main [contributing document](https://github.com/kata-containers/community/blob/main/CONTRIBUTING.md).
a method used in cloud computing, whereby the amount of computational resources in a server farm, typically measured in terms of the number of active servers, which vary automatically based on the load on the farm.
## B
## C
### Container Security Solutions
The process of implementing security tools and policies that will give you the assurance that everything in your container is running as intended, and only as intended.
### Container Software
A standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.
### Container Runtime Interface
A plugin interface which enables Kubelet to use a wide variety of container runtimes, without the need to recompile.
### Container Virtualization
A container is a virtual runtime environment that runs on top of a single operating system (OS) kernel and emulates an operating system rather than the underlying hardware.
## D
## E
## F
## G
## H
## I
### Infrastructure Architecture
A structured and modern approach for supporting an organization and facilitating innovation within an enterprise.
## J
## K
### Kata Containers
Kata containers is an open source project delivering increased container security and Workload isolation through an implementation of lightweight virtual machines.
## L
## M
## N
## O
## P
### Pod Containers
A Group of one or more containers , with shared storage/network, and a specification for how to run the containers.
### Private Cloud
A computing model that offers a proprietary environment dedicated to a single business entity.
### Public Cloud
Computing services offered by third-party providers over the public Internet, making them available to anyone who wants to use or purchase them.
## Q
## R
## S
### Serverless Containers
An architecture in which code is executed on-demand. Serverless workloads are typically in the cloud, but on-premises serverless platforms exist, too.
## T
## U
## V
### Virtual Machine Monitor
Computer software, firmware or hardware that creates and runs virtual machines.
### Virtual Machine Software
A software program or operating system that not only exhibits the behavior of a separate computer, but is also capable of performing tasks such as running applications and programs like a separate computer.
## W
## X
## Y
## Z
See the [project glossary hosted in the wiki](https://github.com/kata-containers/kata-containers/wiki/Glossary).
which contains a number of sections for various parts of the Kata
Containers system including the [runtime](src/runtime), the
[agent](src/agent) and the [hypervisor](#hypervisors).
## Hypervisors
See the [hypervisors document](docs/hypervisors.md) and the
[Hypervisor specific configuration details](src/runtime/README.md#hypervisor-specific-configuration).
## Community
@@ -48,6 +109,8 @@ Please raise an issue
## Developers
See the [developer guide](docs/Developer-Guide.md).
### Components
### Main components
@@ -57,9 +120,11 @@ The table below lists the core parts of the project:
| Component | Type | Description |
|-|-|-|
| [runtime](src/runtime) | core | Main component run by a container manager and providing a containerd shimv2 runtime implementation. |
| [runtime-rs](src/runtime-rs) | core | The Rust version runtime. |
| [agent](src/agent) | core | Management process running inside the virtual machine / POD that sets up the container environment. |
| [`dragonball`](src/dragonball) | core | An optional built-in VMM brings out-of-the-box Kata Containers experience with optimizations on container workloads |
| [documentation](docs) | documentation | Documentation common to all components (such as design and install documentation). |
| [tests](https://github.com/kata-containers/tests) | tests | Excludes unit tests which live with the main code. |
| [tests](tests) | tests | Excludes unit tests which live with the main code. |
### Additional components
@@ -70,22 +135,29 @@ The table below lists the remaining parts of the project:
| [packaging](tools/packaging) | infrastructure | Scripts and metadata for producing packaged binaries<br/>(components, hypervisors, kernel and rootfs). |
| [kernel](https://www.kernel.org) | kernel | Linux kernel used by the hypervisor to boot the guest image. Patches are stored [here](tools/packaging/kernel). |
| [osbuilder](tools/osbuilder) | infrastructure | Tool to create "mini O/S" rootfs and initrd images and kernel for the hypervisor. |
| [`agent-ctl`](tools/agent-ctl) | utility | Tool that provides low-level access for testing the agent. |
| [`ci`](https://github.com/kata-containers/ci) | CI | Continuous Integration configuration files and scripts. |
| [kata-debug](tools/packaging/kata-debug/README.md) | infrastructure | Utility tool to gather Kata Containers debug information from Kubernetes clusters. |
| [`agent-ctl`](src/tools/agent-ctl) | utility | Tool that provides low-level access for testing the agent. |
| [`kata-ctl`](src/tools/kata-ctl) | utility | Tool that provides advanced commands and debug facilities. |
| [`runk`](src/tools/runk) | utility | Standard OCI container runtime based on the agent. |
| [`ci`](.github/workflows) | CI | Continuous Integration configuration files and scripts. |
| [`ocp-ci`](ci/openshift-ci/README.md) | CI | Continuous Integration configuration for the OpenShift pipelines. |
| [`katacontainers.io`](https://github.com/kata-containers/www.katacontainers.io) | Source for the [`katacontainers.io`](https://www.katacontainers.io) site. |
| [`Webhook`](tools/testing/kata-webhook/README.md) | utility | Example of a simple admission controller webhook to annotate pods with the Kata runtime class |
### Packaging and releases
Kata Containers is now
[available natively for most distributions](docs/install/README.md#packaged-installation-methods).
However, packaging scripts and metadata are still used to generate snap and GitHub releases. See
the [components](#components) section for further details.
## General tests
See the [tests documentation](tests/README.md).
## Metrics tests
See the [metrics documentation](tests/metrics/README.md).
## Glossary of Terms
See the [glossary of terms](Glossary.md) related to Kata Containers.
> While this project's CI has several areas for improvement, it is constantly
> evolving. This document attempts to describe its current state, but due to
> ongoing changes, you may notice some outdated information here. Feel free to
> modify/improve this document as you use the CI and notice anything odd. The
> community appreciates it!
## Introduction
The Kata Containers CI relies on [GitHub Actions][gh-actions], where the actions
themselves can be found in the `.github/workflows` directory, and they may call
helper scripts, which are located under the `tests` directory, to actually
perform the tasks required for each test case.
## The different workflows
There are a few different sets of workflows that are running as part of our CI,
and here we're going to cover the ones that are less likely to get rotten. With
this said, it's fair to advise that if the reader finds something that got
rotten, opening an issue to the project pointing to the problem is a nice way to
help, and providing a fix for the issue is a very encouraging way to help.
### Jobs that run automatically when a PR is raised
These are a bunch of tests that will automatically run as soon as a PR is
opened, they're mostly running on "cost free" runners, and they do some
pre-checks to evaluate that your PR may be okay to start getting reviewed.
Mind, though, that the community expects the contributors to, at least, build
their code before submitting a PR, which the community sees as a very fair
request.
Without getting into the weeds with details on this, those jobs are the ones
responsible for ensuring that:
- The commit message is in the expected format
- There's no missing Developer's Certificate of Origin
- Static checks are passing
### Jobs that require a maintainer's approval to run
There are some tests, and our so-called "CI". These require a
maintainer's approval to run as parts of those jobs will be running on "paid
runners", which are currently using Azure infrastructure.
Once a maintainer of the project gives "the green light" (currently by adding an
`ok-to-test` label to the PR, soon to be changed to commenting "/test" as part
of a PR review), the following tests will be executed:
- Build all the components (runs on free cost runners, or bare-metal depending on the architecture)
- Create a tarball with all the components (runs on free cost runners, or bare-metal depending on the architecture)
- Create a kata-deploy payload with the tarball generated in the previous step (runs on free costs runner, or bare-metal depending on the architecture)
- Run the following tests:
- Tests depending on the generated tarball
- Metrics (runs on bare-metal)
-`docker` (runs on cost free runners)
-`nerdctl` (runs on cost free runners)
-`kata-monitor` (runs on cost free runners)
-`cri-containerd` (runs on cost free runners)
-`nydus` (runs on cost free runners)
-`vfio` (runs on cost free runners)
- Tests depending on the generated kata-deploy payload
- kata-deploy (runs on cost free runners)
- Tests are performed using different "Kubernetes flavors", such as k0s, k3s, rke2, and Azure Kubernetes Service (AKS).
- Kubernetes (runs in Azure small and medium instances depending on what's required by each test, and on TEE bare-metal machines)
- Tests are performed with different runtime engines, such as CRI-O and containerd.
- Tests are performed with different snapshotters for containerd, namely OverlayFS and devmapper.
- Tests are performed with all the supported hypervisors, which are Cloud Hypervisor, Dragonball, Firecracker, and QEMU.
For all the tests relying on Azure instances, real money is being spent, so the
community asks for the maintainers to be mindful about those, and avoid abusing
them to merely debug issues.
## The different runners
In the previous section we've mentioned using different runners, now in this section we'll go through each type of runner used.
- Cost free runners: Those are the runners provided by GitHub itself, and
those are fairly small machines with virtualization capabilities enabled.
- Azure small instances: Those are runners which have virtualization
capabilities enabled, 2 CPUs, and 8GB of RAM. These runners have a "-smaller"
suffix to their name.
- Azure normal instances: Those are runners which have virtualization
capabilities enabled, 4 CPUs, and 16GB of RAM. These runners are usually
`garm` ones with no "-smaller" suffix.
- Bare-metal runners: Those are runners provided by community contributors,
and they may vary in architecture, size and virtualization capabilities.
Builder runners don't actually require any virtualization capabilities, while
runners which will be actually performing the tests must have virtualization
capabilities and a reasonable amount for CPU and RAM available (at least
matching the Azure normal instances).
## Adding new tests
Before someone decides to add a new test, we strongly recommend them to go
through [GitHub Actions Documentation][gh-actions],
which will provide you a very sensible background on how to read and understand
current tests we have, and also become familiar with how to write a new test.
On the Kata Containers land, there are basically two sets of tests: "standalone"
and "part of something bigger".
The "standalone" tests, for example the commit message check, won't be covered
here as they're better covered by the GitHub Actions documentation pasted above.
The "part of something bigger" is the more complicated one and not so
straightforward to add, so we'll be focusing our efforts on describing the
addition of those.
> [!NOTE]
> TODO: Currently, this document refers to "tests" when it actually means the
> jobs (or workflows) of GitHub. In an ideal world, except in some specific cases,
> new tests should be added without the need to add new workflows. In the
> not-too-distant future (hopefully), we will improve the workflows to support
> this.
### Adding a new test that's "part of something bigger"
The first important thing here is to align expectations, and we must say that
the community strongly prefers receiving tests that already come with:
- Instructions how to run them
- A proven run where it's passing
There are several ways to achieve those two requirements, and an example of that
can be seen in PR #8115.
With the expectations aligned, adding a test consists in:
- Adding a new yaml file for your test, and ensure it's called from the
"bigger" yaml. See the [Kata Monitor test example][monitor-ex01].
- Adding the helper scripts needed for your test to run. Again, use the [Kata Monitor script as example][monitor-ex02].
Following those examples, the community advice during the review, and even
asking the community directly on Slack are the best ways to get your test
accepted.
## Required tests
In our CI we have two categories of jobs - required and non-required:
- Required jobs need to all pass for a PR to be merged normally and
should cover all the core features on Kata Containers that we want to
ensure don't have regressions.
- The non-required jobs are for unstable tests, or for features that
are experimental and not-fully supported. We'd like those tests to also
pass on all PRs ideally, but don't block merging if they don't as it's
not necessarily an indication of the PR code causing regressions.
### Transitioning between required and non-required status
Required jobs that fail block merging of PRs, so we want to ensure that
jobs are stable and maintained before we make them required.
The [Kata Containers CI Dashboard](https://kata-containers.github.io/)
is a useful resource to check when collecting evidence of job stability.
At time of writing it reports the last ten days of Kata CI nightly test
results for each job. This isn't perfect as it doesn't currently capture
results on PRs, but is a good guideline for stability.
> [!NOTE]
> Below are general guidelines about jobs being marked as
> required/non-required, but they are subject to change and the Kata
> Architecture Committee may overrule these guidelines at their
> discretion.
#### Initial marking as required
For new jobs, or jobs that haven't been marked as required recently,
the criteria to be initially marked as required is ten days
of passing tests, with no relevant PR failures reported in that time.
Required jobs also need one or more nominated maintainers that are
responsible for the stability of their jobs. Maintainers can be registered
in [`maintainers.yml`](https://github.com/kata-containers/kata-containers.github.io/blob/main/maintainers.yml)
and will then show on the CI Dashboard.
To add transparency to making jobs required/non-required and to keep the
GitHub UI in sync with the [Gatekeeper job](../tools/testing/gatekeeper),
the process to update a job's required state is as follows:
1. Create a PR to update `maintainers.yml`, if new maintainers are being
pipelines to monitor selected functional tests on OpenShift.
There are 2 pipelines, history and logs can be accessed here:
* [main - currently supported OCP](https://prow.ci.openshift.org/job-history/gs/origin-ci-test/logs/periodic-ci-kata-containers-kata-containers-main-e2e-tests)
* [next - currently under development OCP](https://prow.ci.openshift.org/job-history/gs/origin-ci-test/logs/periodic-ci-kata-containers-kata-containers-main-next-e2e-tests)
Running openshift-tests on OCP with kata-containers manually
Following steps require the [openshift tests](https://github.com/openshift/origin)
being cloned and built in the current directory:
```bash
#!/bin/bash -e
# Define tests to be skipped (see the pipeline config for the current version)
TEST_SKIPS="\[sig-node\] Security Context should support seccomp runtime/default\|\[sig-node\] Variable Expansion should allow substituting values in a volume subpath\|\[k8s.io\] Probing container should be restarted with a docker exec liveness probe with timeout\|\[sig-node\] Pods Extended Pod Container lifecycle evicted pods should be terminal\|\[sig-node\] PodOSRejection \[NodeConformance\] Kubelet should reject pod when the node OS doesn't match pod's OS\|\[sig-network\].*for evicted pods\|\[sig-network\].*HAProxy router should override the route\|\[sig-network\].*HAProxy router should serve a route\|\[sig-network\].*HAProxy router should serve the correct\|\[sig-network\].*HAProxy router should run\|\[sig-network\].*when FIPS.*the HAProxy router\|\[sig-network\].*bond\|\[sig-network\].*all sysctl on whitelist\|\[sig-network\].*sysctls should not affect\|\[sig-network\] pods should successfully create sandboxes by adding pod to network"
# Get the list of tests to be executed
TESTS="$(./openshift-tests run --dry-run --provider "${TEST_PROVIDER}" "${TEST_SUITE}")"
pushd"${katacontainers_repo_dir}/tools/packaging/kata-deploy"||{echo"Failed to push to ${katacontainers_repo_dir}/tools/packaging/kata-deploy";exit 125;}
oc delete -f kata-deploy/base/kata-deploy.yaml
oc -n kube-system wait --timeout=10m --for=delete -l name=kata-deploy pod
oc apply -f kata-cleanup/base/kata-cleanup.yaml
echo"Wait for all related pods to be gone"
(repeats=1;for _ in $(seq 1 600);do
oc get pods -l name="kubelet-kata-cleanup" --no-headers=true -n kube-system 2>&1| grep "No resources found" -q &&((repeats++))||repeats=1
# Instruct the daemonset installer to configure Kata Containers to use the
# host kernel.
#
apiVersion:v1
kind:ConfigMap
metadata:
name:ci.kata.installer.kernel
data:
host_kernel:"yes"
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
