Commit Graph

106 Commits

Author SHA1 Message Date
Ettore Di Giacinto
e64f68d36b Introduce install --relaxed
It introduces a relaxed way to install packages with loose deps. Default
installation now will by default prefer up-to-date packages during
selection.

Also:
- Upgrade now it's used in install so it have to return the full system view also when there is nothing to upgrade
- Avoid checking upgrade upfront if relaxed is on
2021-10-10 00:27:58 +02:00
Ettore Di Giacinto
fc45eae80a create-repo: annotate runtime definition in artifacts 2021-09-17 14:30:06 +02:00
Ettore Di Giacinto
e4fff77d43 Source packages metadata to update repository tree
This allows to generate trees with `create-repo` by just having
the metadata files, making tree(s) an optional requirement.
2021-09-16 10:56:09 +02:00
Ettore Di Giacinto
10ae872a3e Simplify revdep code
Isolate common code into a function and also fix a sublte bug hiding in
that code.

We need to stab a copy of the package inside our PackageMap to avoid
having symlinks pointing at the same values when iterating over
requires. ( e.g. happened in this case:
https://github.com/rancher-sandbox/cOS-toolkit/pull/467#issuecomment-895060115
)
2021-08-09 12:47:27 +02:00
Itxaka
1bd4d520a4
Add 2 new events for image unpacking (#226)
* Reduce possibility of circular dependency

Just by adding an import for bus to anything in the helper dir, we would
run into a circular dependency due to how things are structured. That
means that we cannot set any events for unpacking or docker helper
pulling an image.

This commit tries to work around this by doing several things.
 - Remove full imports of the helper module by segmentating some modules
   into their own submodule, like docker or match so just using a small match
   function doesnt bring the whole module
 - Removing a simple function to check if a dir exists from importing
   the full helper module and instead write the function (5 lines)
 - Using logrus in the bus module instead of logger, which avoids a
   circular dependency

Signed-off-by: Itxaka <igarcia@suse.com>

* Add two new events for unpacking an image

Both pre and post unpacking an image

Signed-off-by: Itxaka <igarcia@suse.com>
2021-06-01 16:43:31 +02:00
Ettore Di Giacinto
c277ac0f94 Add join keyword to generate parent image from final artifacts
A new keyword `join` is introduced to generate the parent image. It
takes precedence over a `requires` or a `image` already defined in a
spec.

It will generate all the artifacts from the packages listed and join
them in a single image which will be used as parent for the package
build process.

This is a change which invalidates priorly generated hashes.

Fixes #173
2021-05-21 14:52:48 +02:00
Ettore Di Giacinto
1ef18ed2c5 Add salted method for assertion hashing
- Add the spec Hash as salt for image hashes
- Add tests and adapt existing ones
- Use a signature to build a spec hash

Fixes: #207
2021-05-19 17:34:01 +02:00
Ettore Di Giacinto
55ae67be0f Pass by options to compute functions in install 2021-04-24 14:26:26 +02:00
Ettore Di Giacinto
562fcc2421 Strip invalid chars from docker images also for metadata files
Fixes #199
2021-04-19 17:18:37 +02:00
Ettore Di Giacinto
612477718e Add values interpolation inheritance test 2021-04-16 14:01:23 +02:00
Ettore Di Giacinto
7ba7add2a8 Allow to pull specfiles from published repositories
- Interpolates values from the repositories compilespec if present
- Automatically merge cache images coming from specified repository when
  necessary

Fixes #194
2021-04-16 13:58:51 +02:00
Ettore Di Giacinto
9eda81667b Use common ImageID() when computing final images
It also adds tests and strip invalid "+" character which is not
supported in docker image tags
2021-03-07 11:01:08 +01:00
Ettore Di Giacinto
d9286a1a1e Download repository metadata with client DownloadFile, uniform downloads for Docker repositories 2021-01-22 16:54:19 +01:00
Ettore Di Giacinto
1f807f369a Move revdeps computation to db 2020-11-20 17:23:21 +01:00
Ettore Di Giacinto
b9c8e50e42 Allow to define multiple templated packages with collections
Collections, similarly to packages, have a `build.yaml` and
a `finalize.yaml` that are templated for each package.
They have a `collection.yaml` containing a list of
packages that are part of the tree.
2020-11-15 00:13:46 +01:00
Ettore Di Giacinto
f6a4b634c1 Don't always walk all World() packages
With this change the solver during install now considers only the part
of the tree which is required to calculate the solution, it doesn't
consider anymore World() as the search space.

The search space now is narrowed down to the packages that related to
the one which we are considering.

In this subset of changes we are also optimizing the Parallel solver
avoiding an useless loop.

This change boost overall performance on large datasets which don't
necessarly have relations touching the whole tree.
2020-10-30 19:12:12 +01:00
Ettore Di Giacinto
ffea4d8cf9 Fix priority constraint formula
The parallel solver made the issue more visible, the constraints needed
to be less relaxed and needed to be exclusive so our candidate is looked
up at it first
2020-10-29 16:32:13 +01:00
Daniele Rondina
045d25bb28 pkg/package: Add method SetVersion to DefaultPackage 2020-08-30 08:53:37 +02:00
Ettore Di Giacinto
052a551c0c
Add "hidden" field to packages
Also drop residual of IsSet which isn't actually used

Related to #26
2020-08-02 11:31:23 +02:00
Ettore Di Giacinto
5dcf77c987
Add package buildtimestamp and luet upgrade --sync
Annotate the package build time when compiling, and use that from the
client to force upgrade of packages that changed the artifact, but
didn't changed any version.

The client can trigger this behavior with `luet upgrade --sync`
2020-07-12 15:29:38 +02:00
Ettore Di Giacinto
16e9d7b20c
Use packageImage as builder image fingerprint
This allows to have an unique identifier for the builder image id against
different depgraphs combinations. The package fingerprint is not enough,
as an atom could have a difference deptree depending on the requires
constraints.

TODO: Don't use the full image name, but only the hash as a salt
(currently the salt contains ALSO a reference of the image-repository,
as such it doesn't allow to port a tree in a different docker registry)
2020-06-23 18:59:18 +02:00
Daniele Rondina
44d68a9583 Add annotations option to package spec 2020-05-23 09:27:38 +02:00
geaaru
916b2a8927
Merge pull request #112 from mudler/package-sanitized
Package sanitized
2020-05-20 11:42:47 +02:00
Daniele Rondina
e38a4b3d9b Use DefaultPackageSanitized struct for write specs 2020-05-20 09:59:48 +02:00
Ettore Di Giacinto
9971fe9f45
Unique hashes for packages without deps 2020-05-18 19:57:01 +02:00
Daniele Rondina
11759f98e0 package/NewPackage: init Labels on AddLabel 2020-05-18 19:51:47 +02:00
Daniele Rondina
cb2ac15de8 package: Fix typo on labels key 2020-05-17 19:17:14 +02:00
Ettore Di Giacinto
6acc5fc97e
Make BuildFormula support dependency cycles 2020-04-30 18:56:35 +02:00
Ettore Di Giacinto
6a4557a3b3
Make RequireContains support dependency cycles 2020-04-30 18:56:06 +02:00
Ettore Di Giacinto
a457c53824
Fixups to ExpandedRevdeps 2020-04-24 00:15:18 +02:00
Ettore Di Giacinto
8305d01e76
Walk requires in ExpandedRevdeps
In this way we annotate the visited and we avoid cycles that can be
generated by revdeps
2020-04-23 23:44:42 +02:00
Ettore Di Giacinto
4c2d38be59
Add package ExpandedRevdeps 2020-04-19 11:55:13 +02:00
Ettore Di Giacinto
77ba4193aa
Add ValidateSelector to versioner interface and consume it
We can refactor furthermore by dropping the package methods, as now we
can consume a versioner in all places that requires it
2020-04-05 15:52:16 +02:00
Ettore Di Giacinto
5a5e7f1dfa
Move version logic inside versioner
WIP, it needs yet to be under the interface implementation
2020-04-05 11:16:14 +02:00
Ettore Di Giacinto
84625be9ac
Adapt package.Best to take a Versioner interface 2020-04-04 15:33:14 +02:00
Ettore Di Giacinto
5e31d940f0
Introduce Packages for []Package 2020-04-04 14:29:08 +02:00
Daniele Rondina
129ca8da55 Add version sanitized to Best() 2020-04-03 18:32:26 +02:00
Daniele Rondina
6bd8fe6789 Add omitempty to DefaultPackage fields 2020-03-22 22:23:11 +01:00
Daniele Rondina
c7652c8a70 Added labels to package definition
* cmd/search: Add support for search of the packages
  with a specific label.

* review Search method of Repositories for permit
  different search modes.

* labels are k/v attributes and could be matched
  through label key (with HasLabel method) or through
  regex that use "$key" + "=" + "$value"
2020-03-21 19:24:27 +01:00
Daniele Rondina
fc40c770ab Add 'tree bump' command 2020-03-16 22:18:48 +01:00
Ettore Di Giacinto
4357ee45e9
Drop unused functions in package 2020-03-16 18:08:46 +01:00
Ettore Di Giacinto
5bcc8d112a
Enforce solver constraints
- Don't sign installed packages during finalizer execution
- Enforce solver constraints: build ALO and AMO rules taking into account
  that the current package might not be selected at all.
- Force uninstalls on upgrade
- Enable option to tell uninstall to ignore conflict with the analized system state,
  as we don't want any conflict with the installed to raise during the upgrade.
  In this way we both force uninstalls and we avoid to check with conflicts
  against the current system state which is pending to deletion.
  This is due to the fact that now the solver enforces the constraints
  and explictly denies two packages of the same version installed.
- Adapt test as now we generate more constraints, which makes the solver more
  noisy on the package that are explictly selected or not
2020-02-27 18:38:31 +01:00
Ettore Di Giacinto
25f2abf103
Do conflict with same package versions while building formula 2020-02-26 23:01:08 +01:00
Ettore Di Giacinto
7e16e0cdb6
Add HashFingerprint() to Package and consume it while generating image names
In this way we don't depend on the package fingerpint characters while generating images, allowing
to support completely semver (e.g. + character is not allowed in docker images tag).

This change is not backward compatible with images currently already built and will invalidate the cache
2020-02-19 18:28:29 +01:00
Ettore Di Giacinto
b01c017507
Add HumanReadableString() to Package 2020-02-18 18:20:45 +01:00
Ettore Di Giacinto
7e0ea34b81
Switch back to gophersat 2020-02-11 14:58:17 +01:00
Ettore Di Giacinto
6f6e2bf15f
Pin to gophersat version
Having the same var in the and block seems to make gophersat crash. Even if might be unoptimal,
we need this to tighten the conditions between packages.

Switch to gophersat fork until this fix is merged upstream:

https://github.com/crillab/gophersat/pull/17
2020-02-11 09:06:57 +01:00
Ettore Di Giacinto
33b442a832
Add accessor to decode from package String() 2020-02-10 17:14:46 +01:00
Ettore Di Giacinto
4dc4205868
Drop json skip from id. It prevents storm to track entities #55 2020-02-08 18:18:05 +01:00
Ettore Di Giacinto
0e30e6a1ad
Treat single conflict cases as well when building conflict formula 2020-01-31 20:50:55 +01:00