Commit Graph

154 Commits

Author SHA1 Message Date
Ettore Di Giacinto
a1c669d3ae Define context for scoped operation across core types
It holds necessary state plus additional information relative to the
context which we are being run to (e.g. if we are in a terminal or not).
Besides in the future we can use it also as a contextual logger to
provide more smart logging capabilities.

This also replace the general global configuration instance that
previously was share between the core components.
2021-10-21 23:58:24 +02:00
Ettore Di Giacinto
fe14d56afe Massive UX rewrite
- Ditch multiple libraries for progressbar, spinner, colors and replace
  with pterm
- Detect when running on terminal and disable automatically spinner
- Add support for multiple progress bars
- Huge rewrite of the configuration part. No more crazy stuff with viper
  CLI commands now correctly overrides default config file as expected
- Limit banner to be displayed on relevant parts

Fixes #211
Fixes #105
Fixes #247
Fixes #233
2021-10-21 23:58:00 +02:00
Ettore Di Giacinto
e64f68d36b Introduce install --relaxed
It introduces a relaxed way to install packages with loose deps. Default
installation now will by default prefer up-to-date packages during
selection.

Also:
- Upgrade now it's used in install so it have to return the full system view also when there is nothing to upgrade
- Avoid checking upgrade upfront if relaxed is on
2021-10-10 00:27:58 +02:00
Ettore Di Giacinto
fc45eae80a create-repo: annotate runtime definition in artifacts 2021-09-17 14:30:06 +02:00
Ettore Di Giacinto
e4fff77d43 Source packages metadata to update repository tree
This allows to generate trees with `create-repo` by just having
the metadata files, making tree(s) an optional requirement.
2021-09-16 10:56:09 +02:00
Ettore Di Giacinto
ccf83b0d5f Avoid deadlocking by FindPackages calls 2021-08-09 15:13:53 +02:00
Ettore Di Giacinto
57cefc7d6b Fixup data race when updating revdeps 2021-08-09 14:09:10 +02:00
Ettore Di Giacinto
97ff647f07 Turn tmpdir in abs path if required
Fixes #239
2021-08-09 13:17:16 +02:00
Ettore Di Giacinto
10ae872a3e Simplify revdep code
Isolate common code into a function and also fix a sublte bug hiding in
that code.

We need to stab a copy of the package inside our PackageMap to avoid
having symlinks pointing at the same values when iterating over
requires. ( e.g. happened in this case:
https://github.com/rancher-sandbox/cOS-toolkit/pull/467#issuecomment-895060115
)
2021-08-09 12:47:27 +02:00
Itxaka
1bd4d520a4
Add 2 new events for image unpacking (#226)
* Reduce possibility of circular dependency

Just by adding an import for bus to anything in the helper dir, we would
run into a circular dependency due to how things are structured. That
means that we cannot set any events for unpacking or docker helper
pulling an image.

This commit tries to work around this by doing several things.
 - Remove full imports of the helper module by segmentating some modules
   into their own submodule, like docker or match so just using a small match
   function doesnt bring the whole module
 - Removing a simple function to check if a dir exists from importing
   the full helper module and instead write the function (5 lines)
 - Using logrus in the bus module instead of logger, which avoids a
   circular dependency

Signed-off-by: Itxaka <igarcia@suse.com>

* Add two new events for unpacking an image

Both pre and post unpacking an image

Signed-off-by: Itxaka <igarcia@suse.com>
2021-06-01 16:43:31 +02:00
Ettore Di Giacinto
c277ac0f94 Add join keyword to generate parent image from final artifacts
A new keyword `join` is introduced to generate the parent image. It
takes precedence over a `requires` or a `image` already defined in a
spec.

It will generate all the artifacts from the packages listed and join
them in a single image which will be used as parent for the package
build process.

This is a change which invalidates priorly generated hashes.

Fixes #173
2021-05-21 14:52:48 +02:00
Ettore Di Giacinto
1ef18ed2c5 Add salted method for assertion hashing
- Add the spec Hash as salt for image hashes
- Add tests and adapt existing ones
- Use a signature to build a spec hash

Fixes: #207
2021-05-19 17:34:01 +02:00
Ettore Di Giacinto
55ae67be0f Pass by options to compute functions in install 2021-04-24 14:26:26 +02:00
Ettore Di Giacinto
562fcc2421 Strip invalid chars from docker images also for metadata files
Fixes #199
2021-04-19 17:18:37 +02:00
Ettore Di Giacinto
612477718e Add values interpolation inheritance test 2021-04-16 14:01:23 +02:00
Ettore Di Giacinto
7ba7add2a8 Allow to pull specfiles from published repositories
- Interpolates values from the repositories compilespec if present
- Automatically merge cache images coming from specified repository when
  necessary

Fixes #194
2021-04-16 13:58:51 +02:00
Ettore Di Giacinto
9eda81667b Use common ImageID() when computing final images
It also adds tests and strip invalid "+" character which is not
supported in docker image tags
2021-03-07 11:01:08 +01:00
Ettore Di Giacinto
f07cf6c245 Add tests for file search 2021-03-06 18:49:14 +01:00
Ettore Di Giacinto
233429bbeb Allow to search by file
Also make possible to retrieve the artifact when searching for matches
between repositories list. This made possible to show the package list
when calling `luet search`.
2021-02-28 18:42:58 +01:00
Ettore Di Giacinto
92e73051a0 Initialize cached in singleton 2021-02-17 09:34:54 +01:00
Ettore Di Giacinto
2448f3175e Initialize cache db if empty 2021-02-15 18:53:37 +01:00
Ettore Di Giacinto
d9286a1a1e Download repository metadata with client DownloadFile, uniform downloads for Docker repositories 2021-01-22 16:54:19 +01:00
Ettore Di Giacinto
bd3e483f0f Be sure to cache packages with same fingerprint only once
this makes sure that we are fast to process also invalid trees
2021-01-12 10:30:31 +01:00
Ettore Di Giacinto
0652fce55e Update revdeps table while populating Cache
When we cycle, we don't necessarly have all the packages into the DB
yet.

With this change, luet annotates the reverse dependency without any version, and we try to
update revdeps table when new items gets added, by checking the version
required in the selector.

Thanks to @joostruis for noticing the issue
2020-12-30 01:12:35 +01:00
Ettore Di Giacinto
38c9540a1d Use DB copy in GetRevdeps in BoltDB 2020-12-30 01:12:09 +01:00
Ettore Di Giacinto
736c9470cf Add db copy and clone 2020-12-19 17:45:50 +01:00
Ettore Di Giacinto
f4e2f32aff Return candidate not found when appropriate 2020-12-19 14:57:42 +01:00
Ettore Di Giacinto
39c8895f80 Return provided if selector isn't a range in boltdb 2020-12-17 23:17:40 +01:00
Ettore Di Giacinto
ce169f49af If provided isn't a selector, it means we don't have to return a range
Add also more tests about provides
2020-12-16 22:17:34 +01:00
Ettore Di Giacinto
1f807f369a Move revdeps computation to db 2020-11-20 17:23:21 +01:00
Ettore Di Giacinto
b9c8e50e42 Allow to define multiple templated packages with collections
Collections, similarly to packages, have a `build.yaml` and
a `finalize.yaml` that are templated for each package.
They have a `collection.yaml` containing a list of
packages that are part of the tree.
2020-11-15 00:13:46 +01:00
Ettore Di Giacinto
984366d3a5 Consider provides during upgrades 2020-11-05 20:52:02 +01:00
Ettore Di Giacinto
f6a4b634c1 Don't always walk all World() packages
With this change the solver during install now considers only the part
of the tree which is required to calculate the solution, it doesn't
consider anymore World() as the search space.

The search space now is narrowed down to the packages that related to
the one which we are considering.

In this subset of changes we are also optimizing the Parallel solver
avoiding an useless loop.

This change boost overall performance on large datasets which don't
necessarly have relations touching the whole tree.
2020-10-30 19:12:12 +01:00
Ettore Di Giacinto
39bc74fc73 Add boltDB test and fixup range over interface cast 2020-10-29 16:32:13 +01:00
Ettore Di Giacinto
99c59643a1 Add benchmarks tests 2020-10-29 16:32:13 +01:00
Ettore Di Giacinto
ffea4d8cf9 Fix priority constraint formula
The parallel solver made the issue more visible, the constraints needed
to be less relaxed and needed to be exclusive so our candidate is looked
up at it first
2020-10-29 16:32:13 +01:00
Ettore Di Giacinto
e459ddf470 Optimize BoltDB World() call 2020-10-29 16:32:13 +01:00
Daniele Rondina
045d25bb28 pkg/package: Add method SetVersion to DefaultPackage 2020-08-30 08:53:37 +02:00
Ettore Di Giacinto
e30bb056d5
Drop IsFlagged from tests 2020-08-02 12:22:43 +02:00
Ettore Di Giacinto
052a551c0c
Add "hidden" field to packages
Also drop residual of IsSet which isn't actually used

Related to #26
2020-08-02 11:31:23 +02:00
Ettore Di Giacinto
5dcf77c987
Add package buildtimestamp and luet upgrade --sync
Annotate the package build time when compiling, and use that from the
client to force upgrade of packages that changed the artifact, but
didn't changed any version.

The client can trigger this behavior with `luet upgrade --sync`
2020-07-12 15:29:38 +02:00
Ettore Di Giacinto
16e9d7b20c
Use packageImage as builder image fingerprint
This allows to have an unique identifier for the builder image id against
different depgraphs combinations. The package fingerprint is not enough,
as an atom could have a difference deptree depending on the requires
constraints.

TODO: Don't use the full image name, but only the hash as a salt
(currently the salt contains ALSO a reference of the image-repository,
as such it doesn't allow to port a tree in a different docker registry)
2020-06-23 18:59:18 +02:00
Daniele Rondina
3a365c709b Integrate config-protect from package spec 2020-06-06 13:12:54 +02:00
Daniele Rondina
44d68a9583 Add annotations option to package spec 2020-05-23 09:27:38 +02:00
geaaru
916b2a8927
Merge pull request #112 from mudler/package-sanitized
Package sanitized
2020-05-20 11:42:47 +02:00
Daniele Rondina
e38a4b3d9b Use DefaultPackageSanitized struct for write specs 2020-05-20 09:59:48 +02:00
Ettore Di Giacinto
9971fe9f45
Unique hashes for packages without deps 2020-05-18 19:57:01 +02:00
Daniele Rondina
11759f98e0 package/NewPackage: init Labels on AddLabel 2020-05-18 19:51:47 +02:00
Daniele Rondina
cb2ac15de8 package: Fix typo on labels key 2020-05-17 19:17:14 +02:00
Ettore Di Giacinto
9cb290b484
Improve database errors 2020-04-30 20:44:34 +02:00