Commit Graph

10152 Commits

Author SHA1 Message Date
Fabiano Fidêncio
c3cb65d0bb Revert "packaging: Use existing image for the kata-deploy-build"
This reverts commit c1aac0cdea.

The reason this has to be reverted is because we cannot cache an image
that has a specific user, uid, gid, docker_host_id, and expect that to
work equally on different machines.  Unfortunately, this is one of the
images that cannot be cached at all.
2022-10-25 20:24:38 +02:00
Fabiano Fidêncio
959dc3226b Revert "packaging: Add infra to push the kata-deploy builder image"
This reverts commit fe8b246ae4.

The reason this has to be reverted is because we cannot cache an image
that has a specific user, uid, gid, docker_host_id, and expect that to
work equally on different machines.  Unfortunately, this is one of the
images that cannot be cached at all.
2022-10-25 20:23:28 +02:00
Fabiano Fidêncio
44414e1708 lib.sh: Fix get_last_modification()
The ${file} path is an absolute path, as /home/fidencio/..., while the
result of the `git status --porcelain` is a path relative to the
${repo_root_dir}.  Because of this, the logic to adding `-dirty` to the
image name would never work.

Let's fix this by removing the ${repo_root_dir} from the ${file} when
grepping for it.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 20:21:03 +02:00
Fabiano Fidêncio
712177a337
Merge pull request #5482 from fidencio/topic/CC-cache-images-used-to-build-the-artefacts
CC | Avoid rebuilding the containers used for building the artefacts
2022-10-25 17:55:13 +02:00
Wang, Arron
25a961f5f3 initramfs: Add build script to generate initramfs
The init.sh in initramfs will parse the verity scheme,
roothash, root device and setup the root device accordingly.

Fixes: #5135

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2022-10-25 14:23:09 +01:00
Fabiano Fidêncio
c916c98ab5 actions: Push the builder images as part of the payload generation
Let's take advantge of an existing action that publishes the payload
after each pull request, to also publish the "builder images" used to
build each one of the artefacts.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
111ad87828 packaging: Add infra to push the initramfs builder image
Let's add the needed infra for only building and pushing the initramfs
builder image to the Kata Containers' quay.io registry.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
ebf6c83839 packaging: Use exissting image to build the initramfs
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder for the initramds.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
94807e73e7 packaging: Don't remove QEMU image
Now that the QEMU builder image provides only the environment used for
building QEMU, let's ensure it doesn't get removed.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
d4db7ed3c8 packaging: Add infra to push the QEMU builder image
Let's add the needed infra for only building and pushing the QEMU
builder image to the Kata Containers' quay.io registry.

Fixes: #5481

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
9e1df04e66 packaging: Use existing image to build QEMU
Let's first try to pull a pre-existsing image, instead of building our
own, to be used as a builder image for QEMU.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
b26cd250c8 qemu: Re-work static-build Dockerfile
Differently than every single other bit that's part of our repo, QEMU
has been using a single Dockerfile that prepares an environment where
the project can be built, but *also* building the project as part of
that very same Dockerfile.

This is a problem, for several different reasons, including:
* It's very hard to have a reproducible build if you don't have an
  archived image of the builder
* One cannot cache / ipload the image of the builder, as that contains
  already a specific version of QEMU
* Every single CI run we end up building the builder image, which
  includes building dependencies (such as liburing)

Let's split the logic into a new build script, and pass the build script
to be executed inside the builder image, which will be only responsible
for providing an environment where QEMU can be built.

Fixes: #5464
Backports: #5465

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
a036584ed9 packaging: Add infra to push the virtiofsd builder image
Let's add the needed infra for only building and pushing the virtiofsd
builder image to the Kata Containers' quay.io registry.

Fixes: #5480

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
29f64d6181 packaging: Use existing image to build virtiofsd
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the virtiofsd.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
9ba01f36de virtiofsd: Pass the expected toolchain to the build container
Let's ensure we're building virtiofsd with a specific toolchain that's
known to not cause any issues, instead of always using the latest one.

On each bump of the virtiofsd, we'll make sure to adjust this according
to what's been used by the virtiofsd community.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
42fd229f26 packaging: Add infra to push the td-shim builder image
Let's add the needed infra for only building and pushing the td-shim
builder image to the Kata Containers' quay.io registry.

Fixes: #5479

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
55cdd92b57 packaging: Use existing image to build td-shim
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the td-shim.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
ca8abc6cae packaging: Add infra to push the shim-v2 builder image
Let's add the needed infra for only building and pushing the shim-v2
builder image to the Kata Containers' quay.io registry.

Fixes: #5478

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
1c1034255a packaging: Use existing image to build the shim-v2
Let's try to pull a pre-existing image, instead of building our own, to
be used as a builder for the shim-v2.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
92d5dbb20c packaging: Add infra to push the OVMF builder image
Let's add the needed infra for building and pushing the OVMF builder
image to the Kata Containers' quay.io registry.

Fixes: #5477

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
5cef4d9837 packaging: Use existing image to build OVMF
Let's first try to pull a pre-existing image, instead of buildinf our
own, to be used as a builder image for OVMF.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
31a13e8081 packaging: Add infra to push the kernel builder image
Let's add the needed infra for only building and pushing the kernel
builder image to the Kata Containers' quay.io registry.

Fixes: #5476

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
3cd900da6d packaging: Use existing image to build the kernel
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the kernel.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
fe8b246ae4 packaging: Add infra to push the kata-deploy builder image
Let's add the needed infra for only building and pushing the image used
to build the kata-deploy artefacts to the Kata Containers' quay.io
registry.

Fixes: #5475

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
c1aac0cdea packaging: Use existing image for the kata-deploy-build
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the kata-deploy artefacts.

This will save us some CI time.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
a6c0bf8823 packaging: Add push_to_registry()
This function will push a specific tag to a registry, whenever the
PUSH_TO_REGISTRY environment variable is set, otherwise it's a no-op.

This will be used in the future to avoid replicating that logic in every
builder used by the kata-deploy scripts.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:17:03 +02:00
Fabiano Fidêncio
b1454dbcaa packaging: Add get_last_modification()
Let's add a function to get the hash of the last commit modifying a
specific file.

This will help to avoid writing `git rev-list ...` into every single
build script used by the kata-deploy.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:16:59 +02:00
Fabiano Fidêncio
fa1bf8f75c packaging: Add and export CC_BUILDER_REGISTRY
CC_BUILD_REGISTRY, which points to quay.io/kata-containers/cc-builder,
will be used for storing the builder images used to build the artefacts
via the kata-deploy scripts.

The plan is to tag, whenever it's possible and makes sense, images like:
* ${CC_BUILDER_REGISTRY}:kernel-${sha}
* ${CC_BUILDER_REGISTRY}:qemu-${sha}
* ${CC_BUILDER_REGISTRY}:ovmf-${sha}
* ${CC_BUILDER_REGISTRY}:shim-v2-${go-toolchain}-{rust-toolchain}-${sha}
* ${CC_BUILDER_REGISTRY}:td-shim-${toolchain}-${sha}
* ${CC_BUILDER_REGISTRY}:virtiofsd-${toolchain}-${sha}

Where ${sha} is the sha of the last commit modifying the Dockerfile used
by the builder.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-25 15:03:06 +02:00
Fabiano Fidêncio
fb89a83c89
Merge pull request #5136 from arronwy/initramfs
initramfs: Add build script to generate initramfs
2022-10-25 15:01:22 +02:00
Bo Chen
a151d8ee50
Merge pull request #5493 from fidencio/topic/update-clh
versions: Update Cloud Hypervisor to b4e39427080
2022-10-24 07:54:02 -07:00
Bin Liu
0f7088a4b1
Merge pull request #5501 from openanolis/update_install_guide
docs: update rust runtime installation guide
2022-10-24 17:49:34 +08:00
Bin Liu
4696eadfeb
Merge pull request #5488 from ManaSugi/fix/update-libseccomp-crate
rustjail: Upgrade libseccomp crate to v0.3.0
2022-10-24 17:03:30 +08:00
Bin Liu
badb2600b3
Merge pull request #5474 from openanolis/makefile
makefile: remove sudo when create symbolic link
2022-10-24 17:03:20 +08:00
Bin Liu
ab5f97759d
Merge pull request #5497 from Rouzip/remove-redundant
agent: remove redundant checks
2022-10-24 16:41:49 +08:00
Fabiano Fidêncio
190e623c40
Merge pull request #5317 from Champ-Goblem/fix-containerd-stats
shim: Ensure pagesize is set when reporting hugetlb stats
2022-10-24 10:24:49 +02:00
Fabiano Fidêncio
7248cf51c5
Merge pull request #5447 from hbrueckner/fix-5438
kata-ctl: Re-enable network tests on s390x (fixes 5438)
2022-10-24 10:23:35 +02:00
Zhongtao Hu
144efd1a7a docs: update rust runtime installation guide
As kata-deploy support rust runtime, we need to update the installation docs

Fixes:#5500
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
2022-10-24 15:55:30 +08:00
James O. D. Hunt
65ef2a0a0b
Merge pull request #5089 from liubin/fix/4895-ignore-exit-error
agent: use NLM_F_REPLACE replace NLM_F_EXCL in rtnetlink
2022-10-24 08:46:54 +01:00
Zhongtao Hu
164ecca3f0
Merge pull request #5499 from zhaoxuat/main
fix readme content error at doc directory
2022-10-24 14:15:52 +08:00
zhaoxu
abf4f9b299 docs: kata 3.0 Architecture
fix readme content error

Fixes: #5498
Signed-off-by: zhaoxu <zhaoxu@megvii.com>
2022-10-24 11:07:34 +08:00
snir911
18283fd65a
Merge pull request #5192 from kata-containers/CCv0-validate-hp-size
CCv0: agent: validate hugepage size is supported
2022-10-23 08:15:16 +03:00
snir911
ee189d2ebe
Merge pull request #5455 from kata-containers/main-validate-hp-size
agent: validate hugepage size is supported
2022-10-23 08:15:05 +03:00
Rouzip
44d8de8923
agent: remove redundant checks
Remove redundant checks for executable files.

FIXes: #3730

Signed-off-by: Rouzip <1226015390@qq.com>
2022-10-22 23:31:18 +08:00
Fabiano Fidêncio
9d286af7b4 versions: Update Cloud Hypervisor to b4e39427080
An API change, done a long time ago, has been exposed on Cloud
Hypervisor and we should update it on the Kata Containers side to ensure
it doesn't affect Cloud Hypervisor CI and because the change is needed
for an upcoming work to get QAT working with Cloud Hypervisor.

Fixes: #5492

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-10-21 20:52:54 +02:00
Bin Liu
081ee48713 agent: use NLM_F_REPLACE replace NLM_F_EXCL in rtnetlink
Sometimes we will face EEXIST error when adding arp neighbour.
Using NLM_F_REPLACE replace NLM_F_EXCL will avoid fail if the
entry exists.

See https://man7.org/linux/man-pages/man7/netlink.7.html

Fixes: #4895

Signed-off-by: Bin Liu <bin@hyper.sh>
2022-10-21 21:19:14 +08:00
Hendrik Brueckner
e95089b716 kata-ctl: add basic cpu check for s390x
Add a basic s390x cpu check for the "sie" feature to be present.
Also re-enable cpu check testing.

Fixes: #5438

Signed-off-by: Hendrik Brueckner <brueckner@linux.ibm.com>
2022-10-21 12:04:28 +00:00
Hendrik Brueckner
871d2cf2c0 kata-ctl: Limit running tests to x86 and use native-tls on s390x
For s390x, use native-tls for reqwest because the rustls-tls/ring
dependency is not available for s390x.

Also exclude s390x, powerpc64le, and aarch64 from running the cpu
check due to the lack of the arch-specific implementation. In this
case, rust complains about unused functions in src/check.rs (both
normal and test context).

Fixes: #5438

Co-authored-by: James O. D. Hunt <james.o.hunt@intel.com>
Signed-off-by: Hendrik Brueckner <brueckner@linux.ibm.com>
2022-10-21 11:54:26 +00:00
Fabiano Fidêncio
5ddbce0746
Merge pull request #5483 from fidencio/topic/CC-do-not-build-runtime-rs
CC | packaging: Don't build runtime-rs if no RUST_VERSION is provided
2022-10-21 12:06:16 +02:00
Wang, Arron
a5dd0cd3ab initramfs: Add build script to generate initramfs
The init.sh in initramfs will parse the verity scheme,
roothash, root device and setup the root device accordingly.

Fixes: #5135

Signed-off-by: Wang, Arron <arron.wang@intel.com>
2022-10-21 17:38:54 +08:00
Fabiano Fidêncio
53bcaf0547
Merge pull request #5314 from anakrish/anakrish-ccv0-optimize-integrity-no-wipe
CCv0: Optimize integrity device creation by avoiding a full device clear
2022-10-21 09:37:44 +02:00