Commit Graph

1022 Commits

Author SHA1 Message Date
Wang, Arron
5cb02a8067 image-build: generate root hash as an separate partition for rootfs
Generate rootfs hash data during creating the kata rootfs,
current kata image only have one partition, we add another
partition as hash device to save hash data of rootfs data blocks.

Fixes: #6674

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2023-06-06 12:31:14 +02:00
Arron Wang
31c0ad2076 packaging: Add cryptsetup support in Guest kernel and rootfs
Add required kernel config for dm-crypt/dm-integrity/dm-verity
and related crypto config.

Add userspace command line tools for disk encryption support
and ext4 file system utilities.

Fixes: #6674

Signed-off-by: Arron Wang <arron.wang@intel.com>
2023-06-06 12:30:07 +02:00
Fabiano Fidêncio
26f7520387 kata-deploy: Change how we get the Ubuntu k8s key
The current method has been failing every now and then, and was reported
on https://github.com/kubernetes/release/issues/2862.

Ding poked me and suggested to do this change here, so here we go. :-)

Fixes: #7006

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-06-01 12:10:30 +02:00
Fabiano Fidêncio
1ed917a079 Merge pull request #6989 from BbolroC/configurable-build-registry
packaging: make BUILDER_REGISTRY configurable
2023-05-31 15:18:51 +02:00
Archana Shinde
9228815ad2 kernel: Modify build-kernel.sh to accomodate for changes in version.yaml
There were recent changes for the tdx kernel in the version.yaml that are
not currently accounted for in the build-kernel.sh script.
Attempts to setup a tdx kernel to build local changes seemed to not download
the tdx kernel. Instead the mainline kernel is downloaded which has no
tdx-related changes.

The version.yaml has a new entry for tdx kernel. Use that instead for
setting up and downloading the tdx kernel.

Fixes: #6984

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2023-05-30 13:44:58 -07:00
Hyounggyu Choi
43e73bdef7 packaging: make BUILDER_REGISTRY configurable
This PR is to make an environment variable `BUILDER_REGISTRY` configurable
so that those who want to use their own registry for build can set up
the registry.

Fixes: #6988
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2023-05-30 14:40:02 +02:00
xuejun-xj
ffe3157a46 dragonball: add arm64 patches for upcall
The vcpu hotplug/hotunplug feature is implemented with upcall. This commit
add three patches to support the feature on aarch64. Patches:
> 0005: add support of upcall on aarch64
> 0006: skip activate offline cpus' MSI interrupt
> 0007: set the correct boot cpu number

Fixes: #6010

Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
2023-05-30 15:51:08 +08:00
Fabiano Fidêncio
3d5d6eb361 Merge pull request #6958 from fidencio/topic/kata-deploy-improve-backup-restore
kata-deploy: Improve shim backup / restore
2023-05-25 10:54:06 +02:00
Fabiano Fidêncio
3f0735a7e8 Merge pull request #6952 from stevenhorsman/git-clone-doc-fix
doc: Update git commands
2023-05-25 10:36:08 +02:00
Fabiano Fidêncio
428041624a kata-deploy: Improve shim backup / restore
We're currently backing up and restoring all the possible shim files,
but the default one ("containerd-shim-kata-v2").

Let's ensure this is also backed up and restored.

Fixes: #6957

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-24 18:39:27 +02:00
Gabriela Cervantes
14c3f1e9f5 kata-deploy: Fix indentation on kata deploy merge script
This PR fixes the indentation on the kata deploy merge script
that instead of single spaces uses a tap.

Fixes #6925

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-05-24 16:01:10 +00:00
stevenhorsman
6a0035e419 doc: Update git commands
Fix bad migrations from `go get` to `git clone` and update the cloned
directory path

Fixes: #6951
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-05-24 13:16:48 +01:00
Fabiano Fidêncio
22154e0a3b cache: Fix OVMF tarball name for different flavours
75330ab3f9 tried to fix OVMF caching, but
didn't consider that the "vanilla" OVMF tarball name is not
"kata-static-ovmf-x86_64.tar.xz", but rather "kata-static-ovmf.tar.xz".

The fact we missed that, led to the cache builds of OVMF failing, and
the need to build the component on every single PR.

Fixes: #6917 (hopefully for good this time).

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-22 18:12:30 +02:00
Fabiano Fidêncio
b7341cd968 cache: Use "initrd" as initrd_type to build rootfs-initrd
We've been defaulting to "", which would lead to a mismatch with the
latest version from the cache, causing a miss, and finally having to
build the rootfs-initrd as part of the tests, every single time.

Fixes: #6917

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-22 18:12:30 +02:00
Fabiano Fidêncio
a28cefd538 Merge pull request #6924 from stevenhorsman/fedora-bump
osbuilder: Bump fedora image version
2023-05-22 18:10:57 +02:00
stevenhorsman
b8ffcd1b9b osbuilder: Bump fedora image version
- Swap out an EoL fedora image for the latest

Fixes: #6923
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-05-22 13:48:00 +01:00
Fabiano Fidêncio
636539bf0c kata-deploy: Use apt-key.gpg from k8s.io
We're facing some issues to download / use the public key provided by
google for installing kubernetes as part of the kata-deploy image.
```
The following signatures couldn't be verified because the public key is
not available: NO_PUBKEY B53DC80D13EDEF05
Reading package lists... Done
W: GPG error: https://packages.cloud.google.com/apt kubernetes-xenial
   InRelease: The following signatures couldn't be verified because the
   public key is not available: NO_PUBKEY B53DC80D13EDEF05 E: The
   repository 'https://apt.kubernetes.io kubernetes-xenial InRelease' is
   not signed.
N: Updating from such a repository can't be done securely, and is
   therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user
   configuration details.
```

Let's work this around following the suggestion made by @dims, at:
https://github.com/kubernetes/k8s.io/pull/4837#issuecomment-1446426585

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-22 11:06:01 +02:00
Fabiano Fidêncio
ae24dc73c1 local-build: Standardise what's set for the local build scripts
We've a discrepancy on what's set along the scripts used to build the
Kata Cotainers artefacts locally.

Some of those were missing a way to easily debug them in case of a
failure happens, but one specific one (build-and-upload-payload.sh)
could actually silently fail.

All of those have been changed as part of this commut.

Fixes: #6908

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-22 08:36:01 +02:00
Steve Horsman
d5355dee20 Merge pull request #6898 from fidencio/topic/fix-ovmf-caching
cache: Fix OVMF caching
2023-05-19 18:24:51 +01:00
Fabiano Fidêncio
75330ab3f9 cache: Fix OVMF caching
OVMF has been cached, but it's not been used from cache as the `version`
set in the cached builds has always been empty.

The reason for that is because we've been trying to look for
`externals.ovmf.ovmf.version`, while we should be actually looking for
`externals.ovmf.x86_64.version`.

Setting `x86_64` as the OVMF_FLAVOUR would cause another bug, as the
expected tarball name would then be `kata-static-x86_64.tar.xz`, instead
of `kata-static-ovmf-x86_64.tar.xz`.

With everything said, let's simplify the OVMF_FLAVOUR usage, by using it
as it's passed, and only adapting the tarball name for the TDVF case,
which is the abnormal one.

Fixes: #6897

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-19 14:00:39 +02:00
stevenhorsman
a89b44aabf tools: Fix arch bug
Fix mismatched case of `arch`

Fixes: #6895
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-05-19 09:28:22 +01:00
Tobin Feldman-Fitzthum
724437efb3 kata-deploy: add kata-qemu-sev runtimeclass
In order to populate containerd config file with
support for SEV, we need to add the qemu-sev shim
to the kata-deploy script.

Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
2023-05-17 11:36:02 +02:00
Fabiano Fidêncio
9630c13ac0 Merge pull request #6845 from fidencio/topic/yet-more-nvidia-gpu-naming-fixes
gpu: Rename the last bits from `gpu` to `nvidia-gpu`
2023-05-17 09:05:12 +02:00
Steve Horsman
e4a458035c Merge pull request #6852 from stevenhorsman/container-image-arch-consistency
deploy: fix shell script error
2023-05-17 08:01:39 +01:00
Amulya Meka
3ccc29030d Merge pull request #6780 from Amulyam24/rust-virtfs
ppc64le: switch virtiofsd from C to rust version
2023-05-17 09:36:28 +05:30
GabyCT
e0e46de12d Merge pull request #6849 from GabyCT/topic/fixtabs
osbuilder: Fix indentation in rootfs.sh
2023-05-16 16:47:09 -06:00
stevenhorsman
1a3f8fc1a2 deploy: fix shell script error
- Remove local introduced by bad copy-paste

Fixes: #6814
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-05-16 19:30:32 +01:00
Gabriela Cervantes
87cb98c01d osbuilder: Fix indentation in rootfs.sh
This PR replaces single spaces to tabs in order to fix the
indentation of the rootfs script.

Fixes #6848

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-05-16 15:30:50 +00:00
Amulyam24
c5a59caca1 ppc64le: switch virtiofsd from C to rust version
We have been using the C version of virtiofsd on ppc64le. Now that the issue with
rust virtiofsd have been fixed, let's switch to it.

Fixes: #4259

Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
2023-05-16 14:46:19 +02:00
Fabiano Fidêncio
3a9d3c72aa gpu: Rename the last bits from gpu to nvidia-gpu
Let's specifically name the `gpu` runtime class as `nvidia-gpu`.  By
doing this we keep the door open and ease the life of the next vendor
adding GPU support for Kata Containers.

Fixes: #6553

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-16 13:47:52 +02:00
Fabiano Fidêncio
4cde844f70 local-build: Fix kernel-nvidia-gpu target name
It must have `-tarball` as part of its name.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-16 13:34:52 +02:00
GabyCT
3d8185863d Merge pull request #6835 from GabyCT/topic/buildkataproxy
kata-deploy: Add http_proxy as part of the docker build
2023-05-15 16:15:27 -06:00
Fabiano Fidêncio
da877a603d Merge pull request #6829 from fidencio/topic/kata-deploy-remove-tarball-from-payload-image
kata-deploy: Do not ship the kata tarball
2023-05-15 19:01:14 +02:00
Gabriela Cervantes
b9a1db2601 kata-deploy: Add http_proxy as part of the docker build
Add http_proxy and https_proxy as part of the docker build arguments
in order to build properly when we are behind a proxy.

Fixes #6834

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2023-05-15 15:57:29 +00:00
Fabiano Fidêncio
9e83795fca Merge pull request #6825 from fidencio/topic/kata-deploy-build-improvements
kata-deploy: Build improvements
2023-05-15 13:49:15 +02:00
Fabiano Fidêncio
802cd2f673 Merge pull request #6821 from stevenhorsman/container-image-arch-consistency
deploy: Fix arch in image tag
2023-05-15 11:16:01 +02:00
Fabiano Fidêncio
777c3dc8d2 kata-deploy: Do not ship the kata tarball
There's absolutely no reason to ship the kata-static tarball as part of
the payload image, as:
* The tarball is already part of the release process
* The payload image already has uncompressed content of the tarball
* The tarball itself is not used anywhere by the kata-deploy scripts

Fixes: #6828

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-15 09:22:39 +02:00
Fabiano Fidêncio
136e2415da static-build: Download firecracker instead of building it
There's no reason for us to build firecracker instead of simply
downloading the official released tarball, as tarballs are provided for
the architectures we want to use them.

Fixes: #6770

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-12 22:05:33 +02:00
Fabiano Fidêncio
3bf767cfcd static-build: Adjust ARCH for nydus
When building from aarch64, just use "arm64" as that's what's used in
the name of the released nydus tarballs.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-12 22:05:33 +02:00
Fabiano Fidêncio
ac88d34e0c static-build: Use relased binary for CLH (aarch64)
There's no need to build Cloud Hypervisor aarch64 as, for a few releases
already, Cloud Hypervisor provides an official release binary for the
architecture.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-12 22:05:01 +02:00
stevenhorsman
2856d3f23d deploy: Fix arch in image tag
`uname -m` produces `x86_64`, but container image convention
is to use `amd64`, so update this in the tag

Fixes: #6820
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-05-12 16:14:19 +01:00
Fabiano Fidêncio
e8f81ee93d Revert "kata-deploy: Use readinessProbe to ensure everything is ready"
This reverts commit 5ec9ae0f04, for two
main reasons:
* The readinessProbe was misintepreted by myself when working on the
  original PR
* It's actually causing issues, as the pod ends up marked as not
  healthy.
2023-05-12 14:28:23 +02:00
Fabiano Fidêncio
4d17ea4a01 cache: Fix nvidia-snp caching version
All the kernel-foo instances, such as "kernel-sev" or "kernel-snp",
should be transformed into "kernel.foo" when looking at the
versions.yaml file.

This was already done for SEV, but missed on the SNP case.

Fixes: #6777

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-11 21:26:58 +02:00
Fabiano Fidêncio
a133fadbfa cache: Fix nvidia-gpu-tdx-experimental cache URL
We were passing "kernel-nvidia-gpu-tdx", missing the "-experimental"
part, leading to a non-valid URL.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-11 21:20:06 +02:00
Fabiano Fidêncio
b9990c2017 cache: Fix nvidia-gpu version
c9bf7808b6 introduced the logic to
properly get the version of nvidia-gpu kernels, but one important part
was dropped during the rebase into main, which is actually getting the
correct version of the kernel.

Fixing this now, and using the old issue as reference.

Fixes: #6777

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-11 13:55:14 +02:00
Fabiano Fidêncio
c9bf7808b6 cache: Update the KERNEL_FLAVOUR list to include nvidia-gpu
We need to make sure that, when caching a `-nvidia-gpu` kernel, we still
look at the version of the base kernel used to build the nvidia-gpu
drivers, as the ${vendor}-gpu kernels are based on already existing
entries in the versions.yaml file and do not require a new entry to be
added.

Fixes: #6777

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-11 10:56:13 +02:00
Fabiano Fidêncio
3665b42045 gpu: Rename gpu targets to nvidia-gpu
This will make it easier for other GPU vendors to add the needed bits in
the future.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-05-11 10:55:55 +02:00
Tobin Feldman-Fitzthum
2c90cac751 local-build: fixup alphabetization
A few pieces of the local-build tooling are supposed to be
alphabetized. Fixup a couple minor issues that have accumulated.

Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
2023-05-10 21:23:38 +00:00
Tobin Feldman-Fitzthum
4da6eb588d kata-deploy: Add qemu-snp shim
Now that we have the SNP components in place, make sure that
kata-deploy knows about the qemu-snp shim so that it will be
added to containerd config.

Fixes: #6575

Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
2023-05-10 20:55:36 +00:00
Tobin Feldman-Fitzthum
14dd053758 kata-deploy: add kata-qemu-snp runtimeclass
Since SEV-SNP has limited hotplug support, increase
the pod overhead to account for fixed resource usage.

Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
2023-05-10 20:55:36 +00:00