Commit Graph

707 Commits

Author SHA1 Message Date
Ettore Di Giacinto
06fa8b1c87 fixup: jointag index 2021-05-24 22:13:27 +02:00
Ettore Di Giacinto
ff153f367f fixup: enhance output of copy tag when cycling over packages 2021-05-24 21:35:22 +02:00
Ettore Di Giacinto
459676397c Enhance multi-stage output 2021-05-24 19:40:39 +02:00
Ettore Di Giacinto
93057fbf6d Respect PackageTargetOnly during multi-stage copy image generation 2021-05-24 19:40:17 +02:00
Ettore Di Giacinto
5e1a7c50df fixup: display hash of join image 2021-05-23 18:22:14 +02:00
Ettore Di Giacinto
0ceaf09615 fixup: propagate outputpath 2021-05-23 17:36:09 +02:00
Ettore Di Giacinto
27c2e3c51f fixup: make sure the outputdir exists 2021-05-21 22:25:18 +02:00
Ettore Di Giacinto
6344e47eb3 fixup: resolve multistage and join also when building deps 2021-05-21 19:47:23 +02:00
Ettore Di Giacinto
c277ac0f94 Add join keyword to generate parent image from final artifacts
A new keyword `join` is introduced to generate the parent image. It
takes precedence over a `requires` or a `image` already defined in a
spec.

It will generate all the artifacts from the packages listed and join
them in a single image which will be used as parent for the package
build process.

This is a change which invalidates priorly generated hashes.

Fixes #173
2021-05-21 14:52:48 +02:00
Ettore Di Giacinto
85a7968ecc Check if we have the image locally and extract directly it 2021-05-20 10:29:41 +02:00
Ettore Di Giacinto
1ef18ed2c5 Add salted method for assertion hashing
- Add the spec Hash as salt for image hashes
- Add tests and adapt existing ones
- Use a signature to build a spec hash

Fixes: #207
2021-05-19 17:34:01 +02:00
Ettore Di Giacinto
4b1b711a5c Add single suite tests for spec and artifact
They went missing when the refactoring of the compiler occurred
2021-05-19 17:34:01 +02:00
Ettore Di Giacinto
7f047e4fc2 Multi-stage builds from hash images
Implement multi-stage copy from images part of the build cache of a
package.

Note, this is not the final images where are we copying files from, but
the underlying build container.

Skipping the test on img backend because it fails when pulling external
images during multi-stage build...

Fixes #190
2021-05-18 12:16:35 +02:00
Ettore Di Giacinto
9d2ee1b760 Add hashtree and extract hash logic from compiler
Add unit tests, consume imagehashtree in compiler and cleanup

Fixes: #203
2021-05-17 15:22:08 +02:00
Ettore Di Giacinto
fd12227d53 plugins: Make execution fail if loaded plugins are erroring, print debug output on emitted responses
Now plugins failing to answer make execution fail, adapt tests
2021-05-17 15:17:03 +02:00
Ettore Di Giacinto
1e617b0c67 Add copy field
Initial implementation that just works with standard docker image
references

Related: #190
2021-05-17 15:16:15 +02:00
Ettore Di Giacinto
4c3532e3c6 fixup: unchecked err causes invalid read in certain cases 2021-05-16 23:12:54 +02:00
Ettore Di Giacinto
7193ea03f9 Use hasher to work out also big files 2021-05-16 17:21:19 +02:00
Ettore Di Giacinto
beeb0dcaaa Don't write protect file if it is same
Add checksum check to config protect files
Add test file

Fixes #214
2021-05-16 11:54:08 +02:00
Ettore Di Giacinto
0379855592 Drop docker squash support
It can be easily implemented as a plugin

Fixes: #206
2021-05-04 11:17:10 +02:00
Ettore Di Giacinto
958b8c32e1 Add DeepCopyFile for copies with additional permission bits
It's not needed for most of all the copying we do, except when we
generate the deltas

See also #204
2021-05-04 11:14:16 +02:00
Ettore Di Giacinto
f85891e362 Ensure we carry permissions from dirs while we create packages by delta 2021-05-01 00:14:01 +02:00
Ettore Di Giacinto
2cbd97ff3a Respect replace options 2021-04-24 21:09:25 +02:00
Ettore Di Giacinto
a4d77f8f99 Fixup clean path uninstall
While uninstalling, we weren't checking if we left any empty dir behind.
Now we walk the full path to the file in the artifact, and check each
subdir if it's empty. If it is, we delete it as it is claimed by the
package
2021-04-24 19:21:15 +02:00
Ettore Di Giacinto
adcb459fd2 Inplace upgrades 2021-04-24 18:48:57 +02:00
Ettore Di Giacinto
55ae67be0f Pass by options to compute functions in install 2021-04-24 14:26:26 +02:00
Ettore Di Giacinto
7bfff97f57 Fixup race when updating compiler options
Instead of updating the compiler options that can be accessed by each
worker, update the compilespec BuildOptions directly
2021-04-23 12:02:41 +02:00
Ettore Di Giacinto
0288eedbc3 Always resolve buildhash image, add --rebuild to build 2021-04-23 12:02:41 +02:00
Ettore Di Giacinto
b27237b7ff Allow to pull images from multiple-repo while generating all artifacts
before, without --only-target-package, we were forcing to build the
images locally. Now we extend it also to that use-case.

Also revisit how we pass by the builder image hash, so it's easier to
read.

This change also re-enables tagging and bulding builder images all the
times.
Previously, we regressed into not tagging build images coming out-of-the
tree, with the unpleasant side-effect of not be able to re-build the
same artifacts for those leaf packages.
2021-04-23 12:02:41 +02:00
Ettore Di Giacinto
c9aed37fa7 Fixup on values interpolation and metadata retrieve
- Fixup search path on metadata spec load. Previously we were reading
  the package being passed, and not the one resolved (it failed against
selectors)
- Do inherit first pushrepositories, so they take precedence over pull
- Add test cases to cover build values interpolation by remote
  repositories
- Enhance test cases to check image cache repository inheritance when
  --from-repositories is passed
- Fix race condition when inheriting buildspec options: Instead of consuming the compiler one, annotate the updates in the
package BuildOption spec which is passed by
- Update vendor
2021-04-23 12:02:19 +02:00
Ettore Di Giacinto
ef92f23221 Inherit pullimages as pushimages while parsing compilespec metadata
Fixes #200
2021-04-19 17:18:37 +02:00
Ettore Di Giacinto
562fcc2421 Strip invalid chars from docker images also for metadata files
Fixes #199
2021-04-19 17:18:37 +02:00
Ettore Di Giacinto
ecc41ce370 Check if there are upgrades before attempting install 2021-04-16 23:38:10 +02:00
Ettore Di Giacinto
dd6501a642 Check if there are upgrades before attempting install 2021-04-16 22:28:30 +02:00
Ettore Di Giacinto
9f3e7fd0b2 PackageIndex is in repository metadata file only
This was a regression in those changeset, don't replicate the info, and
at the same time, write the artifact path containing the name of the
file only (excluding the path)
2021-04-16 15:12:51 +02:00
Ettore Di Giacinto
612477718e Add values interpolation inheritance test 2021-04-16 14:01:23 +02:00
Ettore Di Giacinto
c27d4d258e Allow create-repo to source trees from remote repositories
This makes possible to create the repositorie from external ones,
It also required to address #26.

Fixes #26
2021-04-16 14:01:13 +02:00
Ettore Di Giacinto
9202bcbbbe Convert raw value to templatedata before merging 2021-04-16 14:00:11 +02:00
Ettore Di Giacinto
fadd5a10a3 Load trees separately while reconstructing spectrees 2021-04-16 14:00:11 +02:00
Ettore Di Giacinto
7ba7add2a8 Allow to pull specfiles from published repositories
- Interpolates values from the repositories compilespec if present
- Automatically merge cache images coming from specified repository when
  necessary

Fixes #194
2021-04-16 13:58:51 +02:00
Ettore Di Giacinto
57c769b4a5 Refactor compiler and annotate buildoptions into compiler metadata
This allows to later pick up values used during build of each package
2021-04-16 13:57:54 +02:00
Ettore Di Giacinto
44cae094e8 Allow multiple build values files
Fixes #198
2021-04-16 13:56:51 +02:00
Ettore Di Giacinto
c022c75239 Write BuildTree as an additional repository file
Also split Docker repository generation a bit more for readability
2021-04-16 13:56:51 +02:00
Ettore Di Giacinto
3250d63072 Don't merge install and compilation DB. Use a temporary DB for generation 2021-04-16 13:56:51 +02:00
Ettore Di Giacinto
b8961be793 Add accessor to embed custom files to a repository while writing
Split up into methods so it's easier to add custom files
2021-04-16 13:56:45 +02:00
Ettore Di Giacinto
036b5c08c6 Split up repository generators 2021-04-08 15:24:32 +02:00
Ettore Di Giacinto
c7b79bf630 Compiler recipe now saves the entire tree 2021-04-08 15:22:00 +02:00
Ettore Di Giacinto
83cb6a2804 Check target with PathSeparator in finalizer.go 2021-04-08 10:22:24 +02:00
Ettore Di Giacinto
ec19b34ca8 Allow to pull from multiple repositories
Adds a new cli flag to luet build `--pull-repository` which allows to
pass-by a list of docker image references which are used to pull the
cache from

Fixes #185
Fixes #184
Closes #161
2021-04-08 09:17:54 +02:00
Ettore Di Giacinto
b8352a81a2 Use Lchown when copying bits, lower message to warn 2021-03-18 11:23:42 +01:00