kairos/luet
1
0
Fork 0
mirror of https://github.com/mudler/luet.git synced 2025-06-27 07:50:18 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,174 Commits 12 Branches 164 Tags 38 MiB
06fa8b1c87
Commit Graph

257 Commits

Author SHA1 Message Date
Ettore Di Giacinto
be7b56bae3 Split ImageDefinitionToTar test 
ImageDefinitionToTar it is not actually used by compiler code, but can
be handy from an API perspective, so we keep it.
 2021-01-24 12:56:25 +01:00
Ettore Di Giacinto
eae2382764 unpackDelta needs a rootfs where to extract files from 2021-01-24 12:41:56 +01:00
Ettore Di Giacinto
7d11df3225 Simplify delta generation, and avoid two-pass with img backend 
This changeset also drops --keep-exported-images, which is quite unused
and can be replaced with a plugin, or either by manually exporting the
resulting images.
 2021-01-24 12:27:07 +01:00
Ettore Di Giacinto
b9f0ef1c55 Implement ImageExists in the img backend 2021-01-23 22:01:29 +01:00
Ettore Di Giacinto
d5f70aea26 Add test for Artifact GetUncompressedName() 2021-01-22 19:15:54 +01:00
Ettore Di Giacinto
a6e7a3059c Respect artifact extension when populating cache 
This caused cache to not hit correctly
 2021-01-22 16:55:51 +01:00
Ettore Di Giacinto
91ea2ed99f Make docker image repositories actually working 
Several changes are included:
- Expose ensureDir in helpers, and call it in the Docker client. In
  other implementations that was handled by CopyFile behind the scenes,
  but that's not the case here
- Create accessor in Artifact to create Artifact objects from files.
  This is handy when we have to carry over downloaded package content
  into caches when artifacts are already verified
- Fix various issues around the imagePush flag, so now trees are pushed
  forcefully each time
- Take into consideration the real artifact name when pushing single
  files in the docker image. This behavior should be changed eventually,
  because single files which aren't repository packages now are in its
  own docker image, but we should have just one that brings the required
  metadata alltogether.
 2021-01-22 16:54:19 +01:00
Ettore Di Giacinto
b27b146b45 Refactor artifact Verify() 2021-01-22 16:54:19 +01:00
Ettore Di Giacinto
dbd37afced Add docker client #169 2021-01-22 16:54:19 +01:00
Ettore Di Giacinto
322fe72ef2 Generate repository metadata and packages for docker repository type 
Drop image-repository on create-repo. In case of a docker repository, --output is the image reference to use.
Also restore default output build dir.

See also: #169
 2021-01-22 16:53:52 +01:00
Ettore Di Giacinto
88b5576611 Expect full image name to GenerateFinalImage 
We will re-use this method also when generating repository metadata
 2021-01-18 12:26:22 +01:00
Ettore Di Giacinto
a1f4c28973 Add GenerateFinalImage to package artifacts 
GenerateFinalImage generates a docker image from scratch with the
artifact content.

Related to #169
 2021-01-18 12:08:47 +01:00
Ettore Di Giacinto
43b0b11028 Define a build context for backends 2021-01-18 11:06:54 +01:00
Ettore Di Giacinto
ecae2873d6 zstd extension suffix is zst, not zstd 
Fixes #163
 2021-01-12 15:52:34 +01:00
Ettore Di Giacinto
43f5b69c18 Let the build fail when depending on virtuals 
This is currently not a valid use case. Virtuals are empty packages and
if the `build.yaml` is completely empty, nothing could depend on them.

Let's try to not be too smart and build the package image if a source
image is supplied, and fail hardly when we depend on a virtual in build
time.
 2021-01-03 23:03:01 +01:00
Ettore Di Giacinto
457acd0d8a Add virtual packages support 2021-01-03 20:08:04 +01:00
Niklas Engvall
bfbcb81210
Set import name for zstd 2020-12-29 22:22:53 +01:00
Ettore Di Giacinto
2b6fe2baa1 Add luet build --wait 
It allows to wait for intermediate images to be available instead of
building all of them
 2020-12-18 23:19:18 +01:00
Ettore Di Giacinto
f0fae82ad9 Add experimental zstd support 
Closes #97
 2020-12-18 21:24:01 +01:00
Ettore Di Giacinto
8e1a457bf1 Check if we have to pull images before generating delta 
As we might skip building entirely, it's possible that the image is not
there yet, so we check if have to pull it or not
 2020-12-15 17:01:56 +01:00
Ettore Di Giacinto
70f05f41e8 Check only if package image exists 
We don't need to look after the builder image as its optional. In this
way we can also reduce the compiler options, as we don't require a
--clean flag anymore. --only-target-package is sufficient to determine
what we can skip and how.
 2020-12-14 18:41:39 +01:00
Ettore Di Giacinto
ef034d87b0 Detect if images are available if we don't have to generate a Package 
While building, if we aren't doing a clean build, we scan now to see if
images are available and we skip, in case we don't find a metadata
already.
 2020-12-14 18:32:32 +01:00
Ettore Di Giacinto
265e2371b4 Add ArtifactNode to test, now we get the gen Dockerfile in the diff 2020-12-12 16:04:54 +01:00
Ettore Di Giacinto
d97e606a31 Adapt fixtures and tests 2020-12-12 12:10:24 +01:00
Ettore Di Giacinto
95da20e366 Context files are immutable 2020-12-12 11:55:25 +01:00
Ettore Di Giacinto
797a34ba49 Reuse same dockerfile gen logic between prelude and steps 
As now we build only when necessary, we need to make sure the images are
built similarly. The discrepancies between the two are less now, and
they can share the same logic.

This fixes a regresion where when no prelude is defined, the build
context isn't copied over
 2020-12-12 11:16:34 +01:00
Ettore Di Giacinto
a1453b7242 Fixup error messages 2020-12-11 23:03:56 +01:00
Ettore Di Giacinto
b5990b5333 Generate changes from CompilerBackendOptions and pass by image name so img can unpack images 2020-12-09 00:27:37 +01:00
Ettore Di Giacinto
9a34296be0 Build step is always required for tagging images 2020-12-07 19:39:56 +01:00
Ettore Di Giacinto
ebd18ae22c Set builderTagged image afterwards 2020-12-07 18:58:14 +01:00
Ettore Di Giacinto
6bf7368993 Don't replace buildertaggedImage if there aren't build steps 2020-12-07 18:39:15 +01:00
Ettore Di Giacinto
338f310d67 Tag and push an image when virtual is supplied, to have a track of it in the image graph tree 2020-12-07 17:59:30 +01:00
Ettore Di Giacinto
86c256a062 Generate empty tar 2020-12-07 17:20:32 +01:00
Ettore Di Giacinto
3c0dd2b71d Adapt test 2020-12-07 00:07:57 +01:00
Ettore Di Giacinto
e9b4d66a3e Retrieve should be rendered also for step images 2020-12-07 00:00:32 +01:00
Ettore Di Giacinto
5047316b70 Try to build only when strictly necessary 2020-12-06 23:50:51 +01:00
Ettore Di Giacinto
6dbf19f085 Use single image to build packages 2020-12-02 21:18:12 +01:00
Ettore Di Giacinto
9423b7c1e3 Add image build events, and add luet replace 
Enhance also some commands descriptions
 2020-12-02 18:24:35 +01:00
Ettore Di Giacinto
9784d6192a Don't hide error on pulling image 2020-11-28 18:03:43 +01:00
Ettore Di Giacinto
0fe30ddcfd Add ability to interpolate during build 
Now build takes a --values argument, which is a yaml file that can be
used to interpolate the specs that are going to be compiled.
 2020-11-28 15:47:29 +01:00
Ettore Di Giacinto
44d33eceba Set workdir also on step image 
Otherwise with DOCKER_SQUASH=true it wouldn't be coherent on where to
find the package files
 2020-11-28 12:07:07 +01:00
Ettore Di Giacinto
8ce135fe12 Add DOCKER_SQUASH 2020-11-27 23:38:31 +01:00
Ettore Di Giacinto
ac149e9336 Use candidate for search, as doesn't have a selector 2020-11-15 11:47:32 +01:00
Ettore Di Giacinto
b9c8e50e42 Allow to define multiple templated packages with collections 
Collections, similarly to packages, have a `build.yaml` and
a `finalize.yaml` that are templated for each package.
They have a `collection.yaml` containing a list of
packages that are part of the tree.
 2020-11-15 00:13:46 +01:00
Ettore Di Giacinto
0e46e763d5 Move bus implementation to a separate repo, hook to events in luet 2020-11-13 18:25:44 +01:00
Ettore Di Giacinto
a8624fe451 Move image removal in compileWithImage and further cleanup 2020-11-10 18:48:39 +01:00
Ettore Di Giacinto
14c1d6ef24 Refactor and optimize build process 2020-11-10 18:14:18 +01:00
Ettore Di Giacinto
36c58307e2 Don't export unless needed 2020-11-10 16:57:24 +01:00
Ettore Di Giacinto
a765147c1d Add templated finalizers 2020-11-08 21:14:19 +01:00
Ettore Di Giacinto
b1316b50b4 Add excludes tests 2020-11-08 16:02:11 +01:00
Ettore Di Giacinto
d92ee9e1d9 Add preliminar support for excludes 2020-11-08 15:35:24 +01:00
Daniele Rondina
d219a2e0fb Run travis task with/without buildkit 2020-11-07 11:41:44 +01:00
Daniele Rondina
6819a28f07 Add support to DOCKER_BUILDKIT on test 2020-11-07 11:37:58 +01:00
Daniele Rondina
24eb6eaef5 Fix test with docker buildkit 2020-11-07 11:37:58 +01:00
Daniele Rondina
c72565e019 Integrate tests for config protects with uninstall 2020-11-06 23:30:37 +01:00
Daniele Rondina
b24d335538 GetProtectFiles() is used also for tree tarball without specs 2020-11-06 23:00:37 +01:00
Daniele Rondina
51417ecb5d pkg/compiler/artifact.go: permit to support config protect with only annotation 2020-11-06 20:23:46 +01:00
Daniele Rondina
130eb8de1a Integrate config protection on uninstall too 2020-11-06 20:14:25 +01:00
Ettore Di Giacinto
91dfb8ce3a Enhance CLI output 2020-10-30 19:15:04 +01:00
Ettore Di Giacinto
95e640c9d0 Make solver type switchable 2020-10-29 16:32:13 +01:00
Daniele Rondina
d67cf2fa33 Revert "Do image export only if we have to generate the package" 
This reverts commit 0857e53b03.
 2020-10-07 10:14:06 +02:00
Ettore Di Giacinto
0857e53b03 Do image export only if we have to generate the package 2020-10-06 19:01:25 +02:00
Ettore Di Giacinto
1c1bdca343 Add only-target-package option to luet build 2020-10-06 17:57:57 +02:00
Ettore Di Giacinto
74246780d4 Support templated packages 2020-10-04 19:33:15 +02:00
Daniele Rondina
097ea37c97 compiler/artefact: remove debug println 2020-10-02 22:37:38 +02:00
Daniele Rondina
c64660b8d1 Permit to ignore config protect rules 
* Added command line option --skip-config-protect

* Added config option config_protect_skip
 2020-10-02 22:25:21 +02:00
Daniele Rondina
93187182e5 pkg/compiler: Fix typo on error message 2020-08-19 19:24:46 +02:00
Ettore Di Giacinto
a076613f66
Fixup import path 2020-08-07 19:30:08 +02:00
Ettore Di Giacinto
95b125cb91
Pull images before executing diff tests 2020-08-06 18:03:00 +02:00
Ettore Di Giacinto
66513955c7
Compute image diffs internally 
Is it more faster in this way as we already have all the needed folders
to the comparison extracted. In this way we don't repeat I/O operation
twice by calling container-diff.

Do not depend on container-diff anymore
 2020-08-05 19:09:45 +02:00
Ettore Di Giacinto
07a1058ac1
Add cli option to skip packages if only metadata is present (without checking the image) 2020-07-17 22:42:03 +02:00
Ettore Di Giacinto
5dcf77c987
Add package buildtimestamp and luet upgrade --sync 
Annotate the package build time when compiling, and use that from the
client to force upgrade of packages that changed the artifact, but
didn't changed any version.

The client can trigger this behavior with `luet upgrade --sync`
 2020-07-12 15:29:38 +02:00
Ettore Di Giacinto
16e9d7b20c
Use packageImage as builder image fingerprint 
This allows to have an unique identifier for the builder image id against
different depgraphs combinations. The package fingerprint is not enough,
as an atom could have a difference deptree depending on the requires
constraints.

TODO: Don't use the full image name, but only the hash as a salt
(currently the salt contains ALSO a reference of the image-repository,
as such it doesn't allow to port a tree in a different docker registry)
 2020-06-23 18:59:18 +02:00
Ettore Di Giacinto
7b51e83902
Merge pull request #119 from mudler/config-protect 
Integrate feature config-protect like Gentoo/Funtoo do
 2020-06-07 11:33:55 +02:00
Daniele Rondina
3a365c709b Integrate config-protect from package spec 2020-06-06 13:12:54 +02:00
Ettore Di Giacinto
264e1e9652
Print inspect output as string 2020-06-06 11:32:47 +02:00
Daniele Rondina
c87db16d31 tarModifierWrapperFunc: Now use strings.HasPrefix for match path 2020-06-06 10:38:11 +02:00
Ettore Di Giacinto
90a25406a0
Add --full option to build 
Don't compile dependencies when computing all the compilation specs from
a tree if are among the target deps

Fixes #41
 2020-06-06 08:58:18 +02:00
Daniele Rondina
16f717f04b tarModifierWrapperFunc: Fix config protect filename 2020-06-05 23:12:19 +02:00
Ettore Di Giacinto
9e0e1199df
Check if image exists before skipping compilation 2020-06-03 21:00:30 +02:00
Daniele Rondina
8f0c528c08 Create helpers.UntarProtect for handle protected files 
Currently, it's used the archive.ReplaceFileTarWrapper
that requite a []byte of the files replaced. This is not
a good idea if files are big and instead could be better
in the near future reimplement ReplaceFileTarWrapper with
a callback that return io.Reader instead of []byte.

If a protected file is already present on target rootfs
it is created a file with the same prefix used in Gentoo:

._cfgXXXX_<filename>
 2020-06-02 11:08:37 +02:00
Ettore Di Giacinto
9971fe9f45
Unique hashes for packages without deps 2020-05-18 19:57:01 +02:00
Ettore Di Giacinto
51711dafba
Add package_dir to pack a spec dir as the main artifact 2020-05-16 21:34:27 +02:00
Daniele Rondina
11944873ea Integrate tmpdir_base params and tmpdirs cleanup 2020-04-30 20:29:28 +02:00
Ettore Di Giacinto
528f481a7b
Annotate package files in metadata 
Fixes #87
 2020-04-13 10:52:41 +02:00
Daniele Rondina
69a82a1ca5 simpledocker: Use debug option for print container-diff results 2020-04-10 09:32:12 +02:00
Daniele Rondina
02c33896d5 simpledocker: Move warning before return 2020-04-10 09:31:08 +02:00
Daniele Rondina
c2e9176ab2 solver.Order now return error 2020-04-09 17:07:57 +02:00
Ettore Di Giacinto
0d2668e452
Print warning if container-diffs return errors to stderr 2020-04-08 18:31:40 +02:00
Ettore Di Giacinto
5e31d940f0
Introduce Packages for []Package 2020-04-04 14:29:08 +02:00
Ettore Di Giacinto
fab2d21ac1
Drop unused function 2020-03-19 22:39:22 +01:00
Ettore Di Giacinto
487239334f
Treat base case with same algorithm of dependencies 2020-03-19 22:37:39 +01:00
Ettore Di Giacinto
69477a0c36
Make uniform build of seeds and packages 2020-03-19 22:34:26 +01:00
Ettore Di Giacinto
67362a3bc1
Drop unused condition 2020-03-19 22:29:34 +01:00
Ettore Di Giacinto
302d18e749
Tag the target image with the computed hash 2020-03-19 22:28:33 +01:00
Ettore Di Giacinto
e3ebfd6bfe
Respect ImageRepository with build --nodeps 2020-02-26 19:00:22 +01:00
Ettore Di Giacinto
07610bc216
Add build option to remove backend artifacts 
Add keep-exported-images to build to instruct to remove backend artifacts (docker images)
 2020-02-21 22:33:18 +01:00
Ettore Di Giacinto
7e16e0cdb6
Add HashFingerprint() to Package and consume it while generating image names 
In this way we don't depend on the package fingerpint characters while generating images, allowing
to support completely semver (e.g. + character is not allowed in docker images tag).

This change is not backward compatible with images currently already built and will invalidate the cache
 2020-02-19 18:28:29 +01:00
Ettore Di Giacinto
2ce427d601
Add Nodeps, Onlydeps and Force options to Compiler 
See #41
 2020-02-18 18:36:44 +01:00
Ettore Di Giacinto
26b94888c3
Skip image building if we pulled them successfully 2020-02-15 22:53:00 +01:00