Commit Graph

47 Commits

Author SHA1 Message Date
Ettore Di Giacinto
7818d41d71
ci: do not run flaky tests 2024-08-01 09:15:39 +02:00
Ettore Di Giacinto
2fa1defd87 ⚙️ Add support for subpackages during buildtime 2022-04-27 23:33:49 +02:00
Ettore Di Giacinto
c363c916d6 ⚙️ Small fixups and enhancements 2022-04-27 23:33:49 +02:00
Ettore Di Giacinto
e70a543f42 ⚙️ Add ability to build from Dockerfiles directly 2022-04-27 21:39:55 +02:00
mudler
77c7eab1ee ⚙️ Fixup runtime tree detection during join images
We need to hook up the runtime DB in order to query the correct part of
the tree
2022-04-13 23:28:44 +02:00
Ettore Di Giacinto
c6170fabd6
⚙️ Enhancements to final images building
This commit is multi-fold and impacts several areas:

- Don't re-generate final artifact locally if already present while
building with `requires_final_images`.
- Expose to CLI a way to build final images without pushing them.
- The packages listed with `requires_final_images` now are evaluated by
  the solver so the full deptree is took into account

Fixes: https://github.com/mudler/luet/issues/294
2022-04-13 18:04:50 +02:00
Ettore Di Giacinto
ba0d625b5f 🎨 Port package to types
This refactors DefaultPackage into types.Package and gets rid of the
interface. This is a preceeding for a follow up where accessors will be
removed from the code.
It also does several cleanup, so we get rid also of some unneeded
dependencies.
2022-01-08 00:05:20 +01:00
Ettore Di Giacinto
519058f13c 🔧 Do not store file list of hidden packages 2022-01-07 00:02:22 +01:00
Ettore Di Giacinto
9bd6730aeb 🤖 Adapt makefile/scripts to ginkgo changes 2022-01-04 18:47:21 +01:00
Ettore Di Giacinto
c98f427156 🎨 Introduce contextualized logging
This commit is multi-fold as it also refactors internally context and logger
as interfaces so it is easier to plug luet as a library externally.

Introduces a garbage collector (related to #227) but doesn't handle yet
parallelism.

Closes #265
2021-12-21 21:54:14 +01:00
Ettore Di Giacinto
702a9f17db Drop code which is called already by containerd
Drop also direct xattrs handling
2021-11-10 16:28:22 +01:00
Ettore Di Giacinto
1e78570c50 Allow to push final images while compiling
Add --push-final-images, --push-final-images-repository and
--push-final-images-force to luet build.

--push-final-images enables pushing final artifact during build. Each
package built will be packed and pushed to the final repository
specified with --push-final-images-repository. By default if no
final-repository is specified and pushing final images is enabled will
default to the cache repository.

--push-final-images-force allows to force-push final images even if
there are already available on the specified registry
2021-11-05 23:03:29 +01:00
Ettore Di Giacinto
c220eac061 Move bus to api/core 2021-10-24 19:07:41 +02:00
Ettore Di Giacinto
c897bffdfc Drop untar 2021-10-24 19:07:41 +02:00
Ettore Di Giacinto
a1c669d3ae Define context for scoped operation across core types
It holds necessary state plus additional information relative to the
context which we are being run to (e.g. if we are in a terminal or not).
Besides in the future we can use it also as a contextual logger to
provide more smart logging capabilities.

This also replace the general global configuration instance that
previously was share between the core components.
2021-10-21 23:58:24 +02:00
Ettore Di Giacinto
e764b1cd29 Add CLI helpers and BuildTree heuristic
The BuildTree implementation was in parallel-build-tools, which actually makes
sense to share in the compiler as can be used as another way to trigger
builds.

The CLI helpers were split in several CIs
implementations. This moves a new layout where we expose several
packages, as we will refactor things slowly.
2021-10-15 21:14:48 +02:00
Itxaka
1bd4d520a4
Add 2 new events for image unpacking (#226)
* Reduce possibility of circular dependency

Just by adding an import for bus to anything in the helper dir, we would
run into a circular dependency due to how things are structured. That
means that we cannot set any events for unpacking or docker helper
pulling an image.

This commit tries to work around this by doing several things.
 - Remove full imports of the helper module by segmentating some modules
   into their own submodule, like docker or match so just using a small match
   function doesnt bring the whole module
 - Removing a simple function to check if a dir exists from importing
   the full helper module and instead write the function (5 lines)
 - Using logrus in the bus module instead of logger, which avoids a
   circular dependency

Signed-off-by: Itxaka <igarcia@suse.com>

* Add two new events for unpacking an image

Both pre and post unpacking an image

Signed-off-by: Itxaka <igarcia@suse.com>
2021-06-01 16:43:31 +02:00
Ettore Di Giacinto
c277ac0f94 Add join keyword to generate parent image from final artifacts
A new keyword `join` is introduced to generate the parent image. It
takes precedence over a `requires` or a `image` already defined in a
spec.

It will generate all the artifacts from the packages listed and join
them in a single image which will be used as parent for the package
build process.

This is a change which invalidates priorly generated hashes.

Fixes #173
2021-05-21 14:52:48 +02:00
Ettore Di Giacinto
57c769b4a5 Refactor compiler and annotate buildoptions into compiler metadata
This allows to later pick up values used during build of each package
2021-04-16 13:57:54 +02:00
Ettore Di Giacinto
d97e606a31 Adapt fixtures and tests 2020-12-12 12:10:24 +01:00
Ettore Di Giacinto
b1316b50b4 Add excludes tests 2020-11-08 16:02:11 +01:00
Ettore Di Giacinto
95e640c9d0 Make solver type switchable 2020-10-29 16:32:13 +01:00
Ettore Di Giacinto
74246780d4 Support templated packages 2020-10-04 19:33:15 +02:00
Ettore Di Giacinto
90a25406a0
Add --full option to build
Don't compile dependencies when computing all the compilation specs from
a tree if are among the target deps

Fixes #41
2020-06-06 08:58:18 +02:00
Ettore Di Giacinto
51711dafba
Add package_dir to pack a spec dir as the main artifact 2020-05-16 21:34:27 +02:00
Ettore Di Giacinto
528f481a7b
Annotate package files in metadata
Fixes #87
2020-04-13 10:52:41 +02:00
Ettore Di Giacinto
bcc6ce19ea
Move compiler options to its own struct
Also add Clean attribute, to handle a future build clean method
2020-01-05 14:32:26 +01:00
Ettore Di Giacinto
3d5b723668
Add compression tests
Refers to #33
2019-12-30 16:35:35 +01:00
Ettore Di Giacinto
475b63be95
Consume concurrency from compiler
Refers to #33
2019-12-30 16:35:18 +01:00
Ettore Di Giacinto
d941c9755b Relax best match constraint rule
Or we could return unsat
2019-12-18 19:24:44 +01:00
Ettore Di Giacinto
15250bd991 Add support for Package provides
Add "provides" field in packages (which affect both runtime and buildtime deps).
It replaces all the occurences in the deptree before solving, actually
allowing to swap packages and provide virtuals. Along with a mechanism
for package rename #25.
2019-12-17 19:32:31 +01:00
Ettore Di Giacinto
097f2ec827
Support includes also on packages coming from unpacked images 2019-12-03 18:15:07 +01:00
Ettore Di Giacinto
fa46601638
Adapt compiler and installer to Tree removal 2019-11-29 19:01:49 +01:00
Ettore Di Giacinto
df78308e98
Annotate artifact metadata after compiling 2019-11-22 21:01:29 +01:00
Ettore Di Giacinto
8ed2badefc
Add test for shared layer unpack 2019-11-17 15:59:52 +01:00
Ettore Di Giacinto
3499c0c580
Add test for compilation with version selection 2019-11-16 16:05:51 +01:00
Ettore Di Giacinto
3e850e9607
Adapt tests and fixup races in spinner
Signed-off-by: Ettore Di Giacinto <mudler@gentoo.org>
2019-11-16 13:26:33 +01:00
Ettore Di Giacinto
b2060c82e3
Make DB Switchable
Fixes races conditions and make the DB Switchable. Also prepare inside
the CompilationSpec the tree of the deps to be built, and parallelize
only the building jobs.

Closes #7

Signed-off-by: Ettore Di Giacinto <mudler@gentoo.org>
2019-11-16 13:26:33 +01:00
Ettore Di Giacinto
c95e0ed91d
Add revdep calculation
Add CompilationSpecs type to handle slices of CompilationSpec, to
perform operation such as Unique() and Remove().

Add also dependencies and the spec associated to the Artifact, to track
how the artifact was generated.

Add revdeps compilation wrapper, and unit tests
2019-11-15 18:11:26 +01:00
Ettore Di Giacinto
cd73d950cc
Add fixture and test for includes and layered packages 2019-11-14 20:28:21 +01:00
Ettore Di Giacinto
d752c0572b
Add unpack field to specify a package used as layer
Closes #9
2019-11-13 09:43:54 +01:00
Ettore Di Giacinto
a2c32aaaa1
Propagate reverse dep calculation
Whenever we update the tree with new informations, all the sibilings
needs to be updated as well

Also switch to inmemory db for recipe - even if this should be a switch
2019-11-12 17:30:06 +01:00
Ettore Di Giacinto
4c1c7451e7
Resolve deps before compiling 2019-11-12 08:48:07 +01:00
Ettore Di Giacinto
e7a328bca7
Fix compilation and hash image calculation. Tests fails on spec (installing d) 2019-11-12 00:13:03 +01:00
Ettore Di Giacinto
af49696aff
WIP Compiler for images by solving the deptree 2019-11-11 19:19:13 +01:00
Ettore Di Giacinto
ebd4cfc318
Make Compile work in parallel
Also make the spinner thread-safe
2019-11-11 10:22:55 +01:00
Ettore Di Giacinto
f634493dc0
Add tests for package compilation with images 2019-11-10 10:49:24 +01:00