Commit Graph

1152 Commits

Author SHA1 Message Date
Fabiano Fidêncio
0520b272a3 Merge pull request #8987 from fidencio/topic/fix-cache-for-confidential-kernel
packaging: cache: Fix caching kernels which rely on extra modules
2024-02-02 09:10:52 +01:00
Fabiano Fidêncio
5d2906c36a packaging: Bump the kata config kernel version
Just to make sure we won't use cached components.

Fixes: #6415

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-01 16:57:15 +01:00
Fabiano Fidêncio
d2ea11dbff packaging: Use the cached kernel modules
Till now we didn't have a logic to consume the kernel modules cached
tarball.  Let's make sure those are consumed as it'll save us a
reasonable amount of build time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-01 16:57:15 +01:00
Fabiano Fidêncio
e5bca90274 packaging: Cache the kernel modules
This will save us a lot of time, as right now the CI is rebuilding the
kernel for absolutely no reason.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-01 16:55:21 +01:00
Fabiano Fidêncio
f481f58659 packaging: Create the tarball for the kernel modules
Let's start doing this for the confidential kernels (and also for SEV,
till it gets removed).

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-01 16:55:20 +01:00
Fabiano Fidêncio
a58caca723 packaging: Take extra tarballs in install_cached_tarball_component()
This allows us to add a map, in the format of:
`"tarball1_name:tarball1_path tarball2_name:tarball2_path ..."`

With this we have a base to start doing a better job when caching extra
artefacts, like kernel modules.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-01 16:55:20 +01:00
Fabiano Fidêncio
33ac5468fe packaging: Add function to get the kernel modules directory
Right now this is just being added but not used yet.  The idea is to use
this to both cache and later on untar the kernel modules needed for some
of the kernel targets we have (specifically looking at the confidential
one).

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-02-01 16:55:20 +01:00
Fabiano Fidêncio
39a64d1447 Merge pull request #8269 from wainersm/kata-deploy_deprecated
kata-deploy: fix deprecations on kustomization files
2024-01-31 20:02:01 +01:00
Hyounggyu Choi
2e1d770fcf packaging: Track files correctly when naming builder image for agent
The necessary files for the agent builder image can be found in
`tools/packaging/static-build/agent`,
`ci/install_libseccomp.sh` and
`tools/packaging/kata-deploy/local-build/kata-deploy-copy-libseccomp-installer.sh`.
Identifying the correct files addresses the previously misreferenced path
used to name the builder image.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-01-31 10:49:20 +01:00
Wainer dos Santos Moschetta
abc2fcd88f kata-deploy: fix deprecations on kustomization files
By running `kustomize edit fix` on those files they have changed
deprecated instructions ('bases' and 'patchesStrategicMerge') as well as
'apiVersion' and 'kind' were added.

Fixes #8268
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
2024-01-30 18:41:03 -03:00
Hyounggyu Choi
f3bc6e4155 packaging: Use Ubuntu 20.04 for building an agent
This involves using Ubuntu 20.04 as a build environment for an agent to match with a runtime environment.

Fixes: #8955

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2024-01-30 10:22:14 +01:00
Greg Kurz
d687b601f1 Merge pull request #8933 from fidencio/topic/package-coco-guest-components
packaging: Build coco-guest-components
2024-01-29 16:34:06 +01:00
Zvonko Kaiser
a9348fa35b Merge pull request #8375 from zvonkok/opa-binary-fix
arm64: agent_policy build always pulls amd64 opa binary
2024-01-29 15:10:10 +01:00
Fabiano Fidêncio
98dc2d4c52 rootfs: agent: Initialise AGENT_SOURCE_BIN & AGENT_TARBALL
Otherwise those would be unbound if not passed.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-26 19:58:41 +01:00
Fabiano Fidêncio
5e57e0235e rootfs: agent: Fix build with AGENT_SOURCE_BIN
We need to actually check that the env var is not empty. :-)
This was introduced by 8307718842.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-26 19:58:20 +01:00
Fabiano Fidêncio
fbfc880eb6 rootfs: Add COCO_GUEST_COMPONENTS_TARBALL env var
This env ver will serve us to pass the Confidential Containers
guest-components tarball to the rootfs builder, which will then just
unpack the content into the rootfs.

Fixes: #8848 -- part I

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Linda Yu <linda.yu@intel.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: Jakob Naucke <jakob.naucke@ibm.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
Co-authored-by: zhouliang121 <liang.a.zhou@linux.alibaba.com>
Co-authored-by: Alex Carter <alex.carter@ibm.com>
Co-authored-by: Suraj Deshmukh <suraj.deshmukh@microsoft.com>
Co-authored-by: Xynnn007 <xynnn@linux.alibaba.com>
2024-01-26 19:58:19 +01:00
Fabiano Fidêncio
644abde35c packaging: coco-guest-components: Allow building the project
The Confidential Containers guest-components will, in the very short
future, be part of the Kata Containers rootfs that's used by the
Confidential Containers usecase.

This commit introduces the ability to, standalone, build the component
locally and as part of our CI, and this can be done by calling:
`make coco-guest-components-tarball`

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Linda Yu <linda.yu@intel.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: Jakob Naucke <jakob.naucke@ibm.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
Co-authored-by: zhouliang121 <liang.a.zhou@linux.alibaba.com>
Co-authored-by: Alex Carter <alex.carter@ibm.com>
Co-authored-by: Suraj Deshmukh <suraj.deshmukh@microsoft.com>
Co-authored-by: Xynnn007 <xynnn@linux.alibaba.com>
2024-01-26 19:36:01 +01:00
Fabiano Fidêncio
a7c68225aa Merge pull request #8916 from fidencio/topic/packaging-reuse-already-built-agent
packaging:  Don't always build the kata-agent
2024-01-26 12:00:55 +01:00
Fabiano Fidêncio
95c569b0a6 packaging: Add safe.directory to the git config
Otherwise building as root will not work, as demonstrated by the arm64
CI.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-26 09:44:43 +01:00
GabyCT
c13a63c8ba Merge pull request #8905 from zvonkok/enable-tpm
qemu: enable TPM
2024-01-25 14:52:00 -06:00
GabyCT
aa958adf90 Merge pull request #8904 from GabyCT/topic/buildbq
tools: Use defined variable in build base qemu script
2024-01-25 13:51:44 -06:00
Fabiano Fidêncio
dd49479829 packaging: Don't build the agent if not needed
Let's start relying on the already cached agent to be deployed inside
the rootfs.  By doing this we save a lot of time in our CI, and we have
a better way, for developers, to play with changes in the agent.

Fixes: #8915

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-25 19:41:33 +01:00
Fabiano Fidêncio
21fd7e6dfd packaging: Fail in case oras can't find an artefact
It just means the component is not cached, and that it must be built in
the usual way.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-25 19:41:32 +01:00
Fabiano Fidêncio
eb7a33ee71 rootfs: Always strip the agent binary
Let's always do this, regardless of where the agent is coming from.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-25 19:41:32 +01:00
Fabiano Fidêncio
f23451de01 rootfs: Add xz as a dep
As we'll be untarring the agent tarball (and any other component that
may be part of the rootfs) into the rootfs, we have to have xz
installed.

For debian and ubuntu the package is called xz-utils; for centos,
alpine and cbl-mariner the package is called xz.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-25 19:41:32 +01:00
Fabiano Fidêncio
8307718842 rootfs: Add AGENT_TARBALL env var
This env var will serve us to pass the agent tarball to the rootfs
builder, which will then just unpack the content into the rootfs instead
of building the agent again.

AGENT_TARBALL and AGENT_SOURCE_BIN should never be used together.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-25 19:41:32 +01:00
Fabiano Fidêncio
5b0d0687e5 packaging: agent: Allow building in all arches
We're moving away from alpine and using ubuntu in order to be able to
build the agent for all the architectures we need.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-25 19:41:32 +01:00
Fabiano Fidêncio
1039641ab8 packaging: agent: Add the arch to the builder container
This has been missed during reviews and is already a problem as we're
trying to build the agent outside of the rootfs for other architectures
than x86_64.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-25 14:11:14 +01:00
Fabiano Fidêncio
58874f9c3e packaging: tools: Add the arch to the builder container
This has been missed during reviews and will become a problem when the
tools start to be built in different architectures.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-25 14:10:22 +01:00
Dan Mihai
723c76d945 tools: allow all users to execute genpolicy
This tool can be useful for any users.

Fixes: #8907

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-01-25 00:40:53 +00:00
Zvonko Kaiser
19ecdbca3b qemu: enable TPM
Several use-cases need a vTPM lets enable it for QEMU, a follow up patch will introduce the runtime config.

Fixes: #8902

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-01-24 17:49:08 +00:00
Gabriela Cervantes
98b5a19b3a tools: Use defined variable in build base qemu script
This PR uses a variable that is already defined in the build base
qemu script to have uniformity across the script as this variable
is already used in the script.

Fixes #8903

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-01-24 17:05:17 +00:00
Zvonko Kaiser
ab597a4d5b opa: Improve the download logic
The versions.yaml has a default for the amd64 binary, but there is no
code to actually build the arm64 binary, which seems an overlook.

Let's simplify the OPA logic by removing the direct link to the binary,
and construct that link as part of the checks we do to decide whether we
need to build OPA or not.

Fixes: #8373

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-23 09:16:16 +00:00
Zvonko Kaiser
4fc34323ae gpu: Add NVIDIA GPU Confidential kernel target
This is a follow up to the work of minimizing targets, unifying TDX,SNP builds for NVIDIA GPUs

Fixes: #8828

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2024-01-22 14:58:57 +00:00
Dan Mihai
febabef08c tools: install genpolicy settings files
Install the default genpolicy OPA rules and settings JSON files, in
addition to the genpolicy binary.

Fixes: #8844

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-01-16 23:59:59 +00:00
Dan Mihai
201eec628a tools: genpolicy static checks
Package genpolicy and enable static checks for it.

Fixes: #8813

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-01-15 16:49:58 +00:00
Fabiano Fidêncio
c3f6eaa267 build-kernel: Fix typo 'terball' -> 'tarball'
SSIA. :-)

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-09 14:35:45 -03:00
Fabiano Fidêncio
8b2f43a2c2 build: Add "confidential" kernel
We're using a Kernel based on v6.7, which should include all te
patches needed for SEV / SNP / TDX.

By doing this, later on, we'll be able to stop building the specific
kernel for each one of the targets we have for the TEEs.

Let's note that we've introduced the "confidential" target for the
kernel builder script, while the TEE specific builds are being kept as
they're -- at least for now.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-09 14:35:45 -03:00
Jianyong Wu
379e2f3da2 kernel: update some configs based on kernel 6.5 and 6.6
There are lots of configs removed from latest kernel. Update them here
for convenience of next kernel upgrade.

Remove CONFIG_SECURITY_SELINUX_CHECKREQPROT_VALUE [1]
Remove CONFIG_IP_NF_TARGET_CLUSTERIP [2]
Remove CONFIG_NET_SCH_CBQ [3]
Remove CONFIG_AUTOFS4_FS [4]
Remove CONFIG_EMBEDDED [5]

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v6.6&id=a7e4676e8e2cb158a4d24123de778087955e1b36
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v6.6&id=9db5d918e2c07fa09fab18bc7addf3408da0c76f
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v6.6&id=051d442098421c28c7951625652f61b1e15c4bd5
[4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v6.6&id=1f2190d6b7112d22d3f8dfeca16a2f6a2f51444e
[5] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v6.6&id=ef815d2cba782e96b9aad9483523d474ed41c62a

Fixes: #8408
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2024-01-09 14:35:45 -03:00
Fabiano Fidêncio
cf4835e3ae packaging: qemu: Simplify "--disable-virtiofsd" logic
As all the supported architectures are disabling the virtiofsd build,
there's no need to keep the switch statement there.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-09 14:35:45 -03:00
Fabiano Fidêncio
bfc6fc7a85 build: Get rid of QEMU experimental
We've not been building QEMU experimental for a very long time, and the
entry there has only been serving the purpose to clutter the
versions.yaml (in the best case scenario) or even confuse new
contributors to the project.

Mind that the machinery to build the QEMU experimental is not touched,
and that's used to build the TEEs capabale artefacts.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2024-01-09 14:35:45 -03:00
Gabriela Cervantes
7d41c97f60 packaging: Fix indentation of build static stratovirt
This PR fixes the indentation of the build static stratovirt script
for kata containers.

Fixes #8777

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-01-05 18:06:08 +00:00
Zhigang Wang
44b5b88f4c docs: Update docs for new StratoVirt VMM introduction
As the StratoVirt VMM has been added, we can update the docs
and make some intoduction to StratoVirt, thus users can know more
about the hypervisor choices.

Fixes: #8645

Signed-off-by: Zhigang Wang <wangzhigang17@huawei.com>
Signed-off-by: Liu Wenyuan <liuwenyuan9@huawei.com>
2024-01-04 14:26:48 +08:00
Chao Wu
f9e0a4bd7e upcall: introduce pci device add & del kernel patch
add pci add and del guest kernel patch as the extension
in the upcall device manager server side.

also, dump config version to 120 since we need to add config
for dragonball pci in upcall

fixes: #8741

Signed-off-by: Gerry Liu <gerry@linux.alibaba.com>
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2023-12-28 16:21:30 +08:00
Fabiano Fidêncio
6ee7fb5402 kata-deploy: Double quote the snapshotter name
Otherwise `jq` will complain about:
```sh
jq: error: nydus/0 is not defined at <top-level>, line 1:
.plugins."io.containerd.grpc.v1.cri".containerd.runtimes."kata-clh".snapshotter=nydus
jq: 1 compile error
```

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-12-26 09:14:36 -03:00
Fabiano Fidêncio
8332f3c684 kata-deploy: Fix the snapshotter config placement
In the way the script is without this patch, we're trying to set
```toml
[`$shim`]
snapshotter = $snapshotter
```

However, what we actually want to set is the full runtime table instead
of shim.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-12-26 08:26:38 -03:00
Fabiano Fidêncio
907f1ddb9e kata-deploy: Fix shim check for snapshotter configuration
We want to check whether the shim is part of the "plain text" shims
passed to the daemonset (meaning, checking against `$SHIMS`).  Before
this fix we were checking against `$shims`, which is an array of shims
instead of a string, resulting on a broken check.

Fixes: #8732

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-12-26 07:42:36 -03:00
Bin Liu
23eb3042c7 kata-monitor: fix Dockerfile to build image
move `SKIP_GO_VERSION_CHECK` after `make` command to skip
checking golang version.

And also upgrade golang to 1.19.

Fixes: #8728

Signed-off-by: Bin Liu <bin@hyper.sh>
2023-12-26 15:11:13 +08:00
stevenhorsman
4a95c0d07f kata-deploy: snapshotter typo fixes
- Add spaces so that the if statements are valid

Fixes: #8720
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-12-22 16:32:02 +00:00
Fabiano Fidêncio
6cc6ca5a7f kata-deploy: Allow setting up snapshotters per runtime handler
Since containerd 1.7.0 we can easily set a specific snapshotter to be
used with a runtime handler, and we should take advantage of this,
mostly as it'll help setting up any runtime using devmapper or nydus
snapshotters.

This implementation here has a few caveats:
* The format expected for the SNAPSHOTTER_HANDLER_MAPPING is:
  `shim:snapshotter,shim:snapshotter,...`
* It only works with containerd 1.7 or newer
* We **never** change the default containerd snapshotter
* We don't do any check on our side to verify whether the snapshotter
  required is properly deployed
* Users will have to add an annotation to their pods, in order to use
  the snapshotter set up per runtime handler
  * Example:
    ```
    metadata:
      ...
      annotations:
        io.containerd.cri.runtime-handler: kata-fc
    ```

Fixes: #8615

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-12-21 07:20:10 -03:00