Commit Graph

9971 Commits

Author SHA1 Message Date
Fabiano Fidêncio
6cd021ce86 Merge branch 'CCv0' into CCv0 2022-10-26 01:20:54 +02:00
Fabiano Fidêncio
712177a337
Merge pull request #5482 from fidencio/topic/CC-cache-images-used-to-build-the-artefacts
CC | Avoid rebuilding the containers used for building the artefacts
2022-10-25 17:55:13 +02:00
Wang, Arron
25a961f5f3 initramfs: Add build script to generate initramfs
The init.sh in initramfs will parse the verity scheme,
roothash, root device and setup the root device accordingly.

Fixes: #5135

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2022-10-25 14:23:09 +01:00
Fabiano Fidêncio
c916c98ab5 actions: Push the builder images as part of the payload generation
Let's take advantge of an existing action that publishes the payload
after each pull request, to also publish the "builder images" used to
build each one of the artefacts.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
111ad87828 packaging: Add infra to push the initramfs builder image
Let's add the needed infra for only building and pushing the initramfs
builder image to the Kata Containers' quay.io registry.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
ebf6c83839 packaging: Use exissting image to build the initramfs
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder for the initramds.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
94807e73e7 packaging: Don't remove QEMU image
Now that the QEMU builder image provides only the environment used for
building QEMU, let's ensure it doesn't get removed.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
d4db7ed3c8 packaging: Add infra to push the QEMU builder image
Let's add the needed infra for only building and pushing the QEMU
builder image to the Kata Containers' quay.io registry.

Fixes: #5481

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
9e1df04e66 packaging: Use existing image to build QEMU
Let's first try to pull a pre-existsing image, instead of building our
own, to be used as a builder image for QEMU.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
b26cd250c8 qemu: Re-work static-build Dockerfile
Differently than every single other bit that's part of our repo, QEMU
has been using a single Dockerfile that prepares an environment where
the project can be built, but *also* building the project as part of
that very same Dockerfile.

This is a problem, for several different reasons, including:
* It's very hard to have a reproducible build if you don't have an
  archived image of the builder
* One cannot cache / ipload the image of the builder, as that contains
  already a specific version of QEMU
* Every single CI run we end up building the builder image, which
  includes building dependencies (such as liburing)

Let's split the logic into a new build script, and pass the build script
to be executed inside the builder image, which will be only responsible
for providing an environment where QEMU can be built.

Fixes: #5464
Backports: #5465

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
a036584ed9 packaging: Add infra to push the virtiofsd builder image
Let's add the needed infra for only building and pushing the virtiofsd
builder image to the Kata Containers' quay.io registry.

Fixes: #5480

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
29f64d6181 packaging: Use existing image to build virtiofsd
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the virtiofsd.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
9ba01f36de virtiofsd: Pass the expected toolchain to the build container
Let's ensure we're building virtiofsd with a specific toolchain that's
known to not cause any issues, instead of always using the latest one.

On each bump of the virtiofsd, we'll make sure to adjust this according
to what's been used by the virtiofsd community.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
42fd229f26 packaging: Add infra to push the td-shim builder image
Let's add the needed infra for only building and pushing the td-shim
builder image to the Kata Containers' quay.io registry.

Fixes: #5479

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
55cdd92b57 packaging: Use existing image to build td-shim
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the td-shim.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
ca8abc6cae packaging: Add infra to push the shim-v2 builder image
Let's add the needed infra for only building and pushing the shim-v2
builder image to the Kata Containers' quay.io registry.

Fixes: #5478

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
1c1034255a packaging: Use existing image to build the shim-v2
Let's try to pull a pre-existing image, instead of building our own, to
be used as a builder for the shim-v2.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
92d5dbb20c packaging: Add infra to push the OVMF builder image
Let's add the needed infra for building and pushing the OVMF builder
image to the Kata Containers' quay.io registry.

Fixes: #5477

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
5cef4d9837 packaging: Use existing image to build OVMF
Let's first try to pull a pre-existing image, instead of buildinf our
own, to be used as a builder image for OVMF.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
31a13e8081 packaging: Add infra to push the kernel builder image
Let's add the needed infra for only building and pushing the kernel
builder image to the Kata Containers' quay.io registry.

Fixes: #5476

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
3cd900da6d packaging: Use existing image to build the kernel
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the kernel.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
fe8b246ae4 packaging: Add infra to push the kata-deploy builder image
Let's add the needed infra for only building and pushing the image used
to build the kata-deploy artefacts to the Kata Containers' quay.io
registry.

Fixes: #5475

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
c1aac0cdea packaging: Use existing image for the kata-deploy-build
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the kata-deploy artefacts.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
a6c0bf8823 packaging: Add push_to_registry()
This function will push a specific tag to a registry, whenever the
PUSH_TO_REGISTRY environment variable is set, otherwise it's a no-op.

This will be used in the future to avoid replicating that logic in every
builder used by the kata-deploy scripts.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
b1454dbcaa packaging: Add get_last_modification()
Let's add a function to get the hash of the last commit modifying a
specific file.

This will help to avoid writing `git rev-list ...` into every single
build script used by the kata-deploy.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:16:59 +02:00
Fabiano Fidêncio
fa1bf8f75c packaging: Add and export CC_BUILDER_REGISTRY
CC_BUILD_REGISTRY, which points to quay.io/kata-containers/cc-builder,
will be used for storing the builder images used to build the artefacts
via the kata-deploy scripts.

The plan is to tag, whenever it's possible and makes sense, images like:
* ${CC_BUILDER_REGISTRY}:kernel-${sha}
* ${CC_BUILDER_REGISTRY}:qemu-${sha}
* ${CC_BUILDER_REGISTRY}:ovmf-${sha}
* ${CC_BUILDER_REGISTRY}:shim-v2-${go-toolchain}-{rust-toolchain}-${sha}
* ${CC_BUILDER_REGISTRY}:td-shim-${toolchain}-${sha}
* ${CC_BUILDER_REGISTRY}:virtiofsd-${toolchain}-${sha}

Where ${sha} is the sha of the last commit modifying the Dockerfile used
by the builder.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:03:06 +02:00
Fabiano Fidêncio
fb89a83c89
Merge pull request #5136 from arronwy/initramfs
initramfs: Add build script to generate initramfs
2022-10-25 15:01:22 +02:00
snir911
18283fd65a
Merge pull request #5192 from kata-containers/CCv0-validate-hp-size
CCv0: agent: validate hugepage size is supported
2022-10-23 08:15:16 +03:00
Fabiano Fidêncio
5ddbce0746
Merge pull request #5483 from fidencio/topic/CC-do-not-build-runtime-rs
CC | packaging: Don't build runtime-rs if no RUST_VERSION is provided
2022-10-21 12:06:16 +02:00
Wang, Arron
a5dd0cd3ab initramfs: Add build script to generate initramfs
The init.sh in initramfs will parse the verity scheme,
roothash, root device and setup the root device accordingly.

Fixes: #5135

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2022-10-21 17:38:54 +08:00
Fabiano Fidêncio
53bcaf0547
Merge pull request #5314 from anakrish/anakrish-ccv0-optimize-integrity-no-wipe
CCv0: Optimize integrity device creation by avoiding a full device clear
2022-10-21 09:37:44 +02:00
Fabiano Fidêncio
299829aec0 packaging: Don't build runtime-rs if no RUST_VERSION is provided
As the CCv0 effort is not using the runtime-rs, let's add a mechanism to
avoid building it.

The easiest way to do so, is to simply *not* build the runtime-rs if the
RUST_VERSION is not provided, and then not providing the RUST_VERSION as
part of the cc-shim-v2-tarball target.

Fixes: #5462

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-20 21:54:45 +02:00
Fabiano Fidêncio
2ca6319f18
Merge pull request #5466 from fidencio/topic/CC-fix-runtime-payload-ci-registry-address
CC | actions: Fix runtime-payload-ci registry address
2022-10-20 07:29:50 +02:00
Fabiano Fidêncio
323fb9cfe8
Merge pull request #5459 from fidencio/topic/add-github-action-to-automate-CC-release
CC | action: Automate CC payload release
2022-10-19 23:37:40 +02:00
Fabiano Fidêncio
b3bd4e432c actions: Fix runtime-payload-ci registry address
There was a typo in the registry name, which should be
quay.io/confidential-containers/runtime-payload-ci instead of
quay.io/repository/confidential-containers/runtime-payload-ci

Fixes: #5469

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-19 22:41:04 +02:00
Fabiano Fidêncio
65de96e774
Merge pull request #5461 from fidencio/topic/generate-a-CI-tarball-after-every-push
CC | actions: Publish a payload on every CCv0 push
2022-10-19 21:49:12 +02:00
Fabiano Fidêncio
74b2ab001d action: Automate CC payload release
Let's create a GitHub action to automate the Kata Containers payload
generation for the Confidential Containers project.

This GitHub action builds the artefacts (in parallel), merges them into
a single tarball, generates the payload with the resulting tarball, and
uploads the payload to the Confidential Containers quay.io.

It expects the tags to be used to be in the `CC-x.y.z` format, with x,
y, and z being numbers.

Fixes: #5330

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-19 19:40:30 +02:00
Fabiano Fidêncio
4648d8bec7 actions: Publish a payload on every CCv0 push
Let's have a GitHub action to publish the Kata Containers payload, after
every push to the CCv0 branch, to the Confidential Containers
`runtime-payload-ci` registry.

The intention of this action is to allow developers to test new
features, and easily bisect breakages that could've happened during the
development process.  Ideally we'd have a CI/CD pipeline where every
single change would be tested with the operator, but we're not yet
there.  In any case, this work would still be needed. :-)

It's very important to mention that this should be carefully considered
on whether it should or should not be merged back to `main`, as the flow
of PRs there is way higher than what we currently have as part of the
CCv0 branch.

Fixes: #5460

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-19 19:09:49 +02:00
Fabiano Fidêncio
c57f8ff669 packaging: Expand the vars on build-and-upload-payload.sh
Just for the sake of avoiding issues in the future.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-19 13:58:08 +02:00
Fabiano Fidêncio
f4437980b4 packaging: Allow passing an extra tag to build-and-upload-payload.sh
Let's modify the script so we allow passing an extra tag, which will be
used as part of the Kata Containers pyload for Confidential Containers
CI GitHub action.

With this we can pass a `latest` tag, which will make things easier for
the integration on the operator side.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-19 13:58:05 +02:00
Fabiano Fidêncio
54544dd617 packaging: Allow passing registry to build-and-upload-payload.sh
Let's make the registry an optional argument to be passed to the
`kata-deploy-build-and-upload-payload.sh` script, defaulting to the
official Confidential Containers payload registry.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-19 12:59:38 +02:00
Megan Wright
a06c6dd861 CCv0: Merge main into CCv0 branch
Merge remote-tracking branch 'upstream/main' into CCv0

Fixes: #5457
Signed-off-by: Megan Wright <megan.wright@ibm.com>
2022-10-19 11:01:13 +01:00
Bin Liu
faf363db75
Merge pull request #5414 from openanolis/chao/regulate_runtime_rs_makefile_comments
runtime-rs: regulate the comment in runtime-rs makefile
2022-10-19 15:36:00 +08:00
Chao Wu
f205472b01 Makefile: regulate the comment style for the runtime-rs comments
In runtime-rs makefile, we use
```
```
to let make help print out help information for variables and targets,
but later commits forgot this rule.
So we need to follow the previous rule and change the current comments.

fixes: #5413
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2022-10-19 12:12:50 +08:00
Fabiano Fidêncio
c97b7b18e7
Merge pull request #5416 from zvonkok/patch-1
doc: Update how-to-run-kata-containers-with-SNP-VMs.md
2022-10-18 22:45:05 +02:00
Fabiano Fidêncio
f087380180
Merge pull request #5437 from fidencio/topic/backport-virtiofsd-build-inside-a-container
CC | backport | virtiofsd: Build inside a container
2022-10-18 19:57:17 +02:00
Fabiano Fidêncio
411888796b packaging: Fix kata-deploy-binaries-in-docker.sh with podman
Instad of bailing out whenever the docker group doesn't exist, just
consider podman is being used, and set the docker_gid to the user's gid.

Also, let's ensure to pass `--privileged` to the container, so
`/run/podman/podman.socket` (which is what `/var/run/docker.sock` points
to) can be passed to the container.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-18 15:11:20 +02:00
Fabiano Fidêncio
07e76c7392 packaging: Pass RUST_VERSION when building shim-v2
We need to pass the RUST_VERSION, in the same way done for GO_VERSION,
as nowadays both the go and the rust runtime are built.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-18 12:23:26 +02:00
James O. D. Hunt
dd60a0298d
Merge pull request #5439 from jodh-intel/kata-ctl-s390x-disable-tls
kata-ctl: Disable network check on s390x
2022-10-18 09:58:09 +01:00
Zvonko Kaiser
ac403cfa5a doc: Update how-to-run-kata-containers-with-SNP-VMs.md
If the needed libraries (for virtfs) are installed on the host,
 QEMU will pick it up and enable it. If not installed and you
do not enable the flag, QEMU will just ignore it, and you end
up without 9p support. Enabling it explicitly will fail if the
needed libs are not installed so this way we can be sure that
it gets build.

Fixes: #5418

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2022-10-17 05:56:19 -07:00