Commit Graph

25 Commits

Author SHA1 Message Date
Ettore Di Giacinto
097e310d3a Drop join from compilation specs
Adapt also hash and integration tests
2021-10-10 00:28:54 +02:00
Ettore Di Giacinto
a348fd4835 Replace Yaml with YAML in function, add debug output when extracting runtime data 2021-09-17 14:46:32 +02:00
Ettore Di Giacinto
fc45eae80a create-repo: annotate runtime definition in artifacts 2021-09-17 14:30:06 +02:00
Ettore Di Giacinto
33b1c63815 Allow to have shared templates across packages
This changeset allows to have shared templates in a static folder
"templates" present in each luet tree. If the directory is present, it
gets scanned and templated accordingly on top of each package. This
allows to use such folder to store custom blocks to share between
packages.

This is still experimental and subject to change, this is just a first
pass version to provide the feature. It needs to be refined still as it
would be more elegant to use the helm engine properly and map our
structure to the engine instead of adapting it roughly.

Fixes #224
2021-08-04 16:35:28 +02:00
Ettore Di Giacinto
9d1594c036 Check if the system-target path supplied is absolute
docker.Untar
(https://github.com/mudler/luet/blob/master/vendor/github.com/docker/docker/pkg/archive/archive.go#L942) requires absolute paths.
We didn't do any input validation before, assuming the path passed by
were absolute since they were coming from YAML configuration files, now
that this is not the truth anymore we need to sanitize the input.

With this change we check if the given path is absolute or relative, if
it's relative we calculate the absolute path and use it in place.
2021-08-03 16:22:59 +02:00
Ettore Di Giacinto
75906c4198 Add missing check for error 2021-08-03 13:36:38 +02:00
Ettore Di Giacinto
cb032dc714 Allow to manipulate requires/conflicts/provides
This allows to manipulate requires, conflicts and provides with
templating in build time.

Signed-off-by: Ettore Di Giacinto <mudler@sabayon.org>
2021-08-03 13:27:44 +02:00
Ettore Di Giacinto
76b70ebeb4 Add dirhash to package signatures
While this breaks current hashing, it ties also the spec content to the
hash, in this way if we change something in the spec folder, it breaks
the hashing for the package.

Signed-off-by: Ettore Di Giacinto <mudler@sabayon.org>
2021-07-09 12:37:38 +02:00
Ettore Di Giacinto
2efb17a06c Add specific field to resolve final images
This commit starts deprecation of `join` keyword in favor of
`requires_final_images` as boolean in the compilation spec.

The change is driven by two reasons: syntax and guarantee unique hashes.

- the hashtree when computing a hash it analizes the requires field of
  each spec, ignoring the join field
- the join field doesn't add much value. Having it separate suggests
  that a spec can contain both `requires` and `join`, but that's not
  actually true. We just act differently on the same list.

Signed-off-by: Ettore Di Giacinto <mudler@sabayon.org>
2021-07-09 12:37:38 +02:00
Daniele Rondina
b6b91cfd7a
compiler: Avoid generation of delta if there are only prelude steps (#232) 2021-07-09 12:23:32 +02:00
Itxaka
1bd4d520a4
Add 2 new events for image unpacking (#226)
* Reduce possibility of circular dependency

Just by adding an import for bus to anything in the helper dir, we would
run into a circular dependency due to how things are structured. That
means that we cannot set any events for unpacking or docker helper
pulling an image.

This commit tries to work around this by doing several things.
 - Remove full imports of the helper module by segmentating some modules
   into their own submodule, like docker or match so just using a small match
   function doesnt bring the whole module
 - Removing a simple function to check if a dir exists from importing
   the full helper module and instead write the function (5 lines)
 - Using logrus in the bus module instead of logger, which avoids a
   circular dependency

Signed-off-by: Itxaka <igarcia@suse.com>

* Add two new events for unpacking an image

Both pre and post unpacking an image

Signed-off-by: Itxaka <igarcia@suse.com>
2021-06-01 16:43:31 +02:00
Ettore Di Giacinto
c277ac0f94 Add join keyword to generate parent image from final artifacts
A new keyword `join` is introduced to generate the parent image. It
takes precedence over a `requires` or a `image` already defined in a
spec.

It will generate all the artifacts from the packages listed and join
them in a single image which will be used as parent for the package
build process.

This is a change which invalidates priorly generated hashes.

Fixes #173
2021-05-21 14:52:48 +02:00
Ettore Di Giacinto
1ef18ed2c5 Add salted method for assertion hashing
- Add the spec Hash as salt for image hashes
- Add tests and adapt existing ones
- Use a signature to build a spec hash

Fixes: #207
2021-05-19 17:34:01 +02:00
Ettore Di Giacinto
4b1b711a5c Add single suite tests for spec and artifact
They went missing when the refactoring of the compiler occurred
2021-05-19 17:34:01 +02:00
Ettore Di Giacinto
7f047e4fc2 Multi-stage builds from hash images
Implement multi-stage copy from images part of the build cache of a
package.

Note, this is not the final images where are we copying files from, but
the underlying build container.

Skipping the test on img backend because it fails when pulling external
images during multi-stage build...

Fixes #190
2021-05-18 12:16:35 +02:00
Ettore Di Giacinto
1e617b0c67 Add copy field
Initial implementation that just works with standard docker image
references

Related: #190
2021-05-17 15:16:15 +02:00
Ettore Di Giacinto
4c3532e3c6 fixup: unchecked err causes invalid read in certain cases 2021-05-16 23:12:54 +02:00
Ettore Di Giacinto
7193ea03f9 Use hasher to work out also big files 2021-05-16 17:21:19 +02:00
Ettore Di Giacinto
beeb0dcaaa Don't write protect file if it is same
Add checksum check to config protect files
Add test file

Fixes #214
2021-05-16 11:54:08 +02:00
Ettore Di Giacinto
958b8c32e1 Add DeepCopyFile for copies with additional permission bits
It's not needed for most of all the copying we do, except when we
generate the deltas

See also #204
2021-05-04 11:14:16 +02:00
Ettore Di Giacinto
0288eedbc3 Always resolve buildhash image, add --rebuild to build 2021-04-23 12:02:41 +02:00
Ettore Di Giacinto
c9aed37fa7 Fixup on values interpolation and metadata retrieve
- Fixup search path on metadata spec load. Previously we were reading
  the package being passed, and not the one resolved (it failed against
selectors)
- Do inherit first pushrepositories, so they take precedence over pull
- Add test cases to cover build values interpolation by remote
  repositories
- Enhance test cases to check image cache repository inheritance when
  --from-repositories is passed
- Fix race condition when inheriting buildspec options: Instead of consuming the compiler one, annotate the updates in the
package BuildOption spec which is passed by
- Update vendor
2021-04-23 12:02:19 +02:00
Ettore Di Giacinto
c27d4d258e Allow create-repo to source trees from remote repositories
This makes possible to create the repositorie from external ones,
It also required to address #26.

Fixes #26
2021-04-16 14:01:13 +02:00
Ettore Di Giacinto
7ba7add2a8 Allow to pull specfiles from published repositories
- Interpolates values from the repositories compilespec if present
- Automatically merge cache images coming from specified repository when
  necessary

Fixes #194
2021-04-16 13:58:51 +02:00
Ettore Di Giacinto
57c769b4a5 Refactor compiler and annotate buildoptions into compiler metadata
This allows to later pick up values used during build of each package
2021-04-16 13:57:54 +02:00