Commit Graph

767 Commits

Author SHA1 Message Date
Ettore Di Giacinto
6b45b1d61c ux: rework displaying of success messages 2021-10-22 17:55:38 +02:00
Ettore Di Giacinto
5eb5a42bf7 Generate snapshot and push it along
Fixes #260
Fixes #223
2021-10-22 17:55:08 +02:00
Ettore Di Giacinto
0bacdc75f2 Allow clients to pick a specific referenceID
Related to #260
2021-10-22 12:39:36 +02:00
Ettore Di Giacinto
917d0935ad Show progressbar only if terminal is big enough
Fixes #259

The bug is caused by
4c725e56bf/progressbar_printer.go (L190)
which is not taking into account when barCurrentLength is <=0.

As a workaround display it only if the terminal width is big enough.

Now the context drives this kind of runtime state, so we keep everything
tight and we inject only that into the workers
2021-10-22 12:39:07 +02:00
Ettore Di Giacinto
8d34a6ebb1 Use the cache to store temporary download files 2021-10-21 23:58:45 +02:00
Ettore Di Giacinto
d58a563d52 uninstall: limit cleanup to s.Target
This was specific to when s.Target differs from /. The helpers creating
the path were considering the full traversal, instead they should stop
at target.

E.g. our rootfs is in /foo/bar, and while we build the list of paths to
prune for /foo/bar/baz/bar, we should stop looking at /foo/bar.
2021-10-21 23:58:45 +02:00
Ettore Di Giacinto
a1c669d3ae Define context for scoped operation across core types
It holds necessary state plus additional information relative to the
context which we are being run to (e.g. if we are in a terminal or not).
Besides in the future we can use it also as a contextual logger to
provide more smart logging capabilities.

This also replace the general global configuration instance that
previously was share between the core components.
2021-10-21 23:58:24 +02:00
Ettore Di Giacinto
fe14d56afe Massive UX rewrite
- Ditch multiple libraries for progressbar, spinner, colors and replace
  with pterm
- Detect when running on terminal and disable automatically spinner
- Add support for multiple progress bars
- Huge rewrite of the configuration part. No more crazy stuff with viper
  CLI commands now correctly overrides default config file as expected
- Limit banner to be displayed on relevant parts

Fixes #211
Fixes #105
Fixes #247
Fixes #233
2021-10-21 23:58:00 +02:00
Ettore Di Giacinto
d4edaa9de8 Adapt unit tests 2021-10-19 21:25:01 +02:00
Ettore Di Giacinto
5b4e930fc3 Do implement a real cache
Instead of merely storing files into disk use a real cache.

This also makes possible finally to reference artifacts in the cache with the
package checksum, which solves the cache hit checksum failures we had
previously.
2021-10-19 21:25:01 +02:00
Ettore Di Giacinto
2eeb464946 Make repositories arch-aware
Introduce an arch field that can be used to filter repositories based on
the local architecture.

If arch is provided and matching with the current GOARCH then the
repository is enabled
2021-10-19 14:19:53 +02:00
Ettore Di Giacinto
e764b1cd29 Add CLI helpers and BuildTree heuristic
The BuildTree implementation was in parallel-build-tools, which actually makes
sense to share in the compiler as can be used as another way to trigger
builds.

The CLI helpers were split in several CIs
implementations. This moves a new layout where we expose several
packages, as we will refactor things slowly.
2021-10-15 21:14:48 +02:00
Ettore Di Giacinto
6ea05e59ea Drop imgworker client
Cleanup code which allows to extract images in a privileged way. the
containerd way supersedes it.
2021-10-14 22:06:01 +02:00
Ettore Di Giacinto
d5819bb4cb Fixup benchmark tests 2021-10-10 19:07:46 +02:00
Ettore Di Giacinto
6ba028f0ea Make sure we do compute the best fit
While install calls upgrade which in turns calls a relaxed install on
its results, this doesn't make sure that the new results are at the best
available version. We do iterate here over the results to compute the
best set.

It also expands computeUpgrade with the possibility to selectively
choose which packages to upgrade and which not.
2021-10-10 19:04:55 +02:00
Ettore Di Giacinto
9de6b22764 Make singlecore default solver 2021-10-10 00:29:13 +02:00
Ettore Di Giacinto
097e310d3a Drop join from compilation specs
Adapt also hash and integration tests
2021-10-10 00:28:54 +02:00
Ettore Di Giacinto
5a63bbd0d2 Deprecate parallel solver 2021-10-10 00:28:54 +02:00
Ettore Di Giacinto
e64f68d36b Introduce install --relaxed
It introduces a relaxed way to install packages with loose deps. Default
installation now will by default prefer up-to-date packages during
selection.

Also:
- Upgrade now it's used in install so it have to return the full system view also when there is nothing to upgrade
- Avoid checking upgrade upfront if relaxed is on
2021-10-10 00:27:58 +02:00
Ettore Di Giacinto
ff46bc7641 Don't use semver library for ordering by default
Use debian to parse versions and selections.
This covers ordering of versions like -0,-1 in sequence plus many others
not available with semver. It is backward compatible as we do support
the same featureset as before as tests are passing (adding more actually
to cover interesting cases)
2021-09-21 14:16:39 +02:00
Ettore Di Giacinto
a348fd4835 Replace Yaml with YAML in function, add debug output when extracting runtime data 2021-09-17 14:46:32 +02:00
Ettore Di Giacinto
fc45eae80a create-repo: annotate runtime definition in artifacts 2021-09-17 14:30:06 +02:00
Ettore Di Giacinto
b73ac21004 create-repo: don't inherit build requirements in runtime 2021-09-17 12:06:39 +02:00
Ettore Di Giacinto
14914f3c8e Update tests for including packages in tree from metadata
Also switches GenerateRepository to functional interface to allow
more parametrization
2021-09-16 12:59:54 +02:00
Ettore Di Giacinto
e4fff77d43 Source packages metadata to update repository tree
This allows to generate trees with `create-repo` by just having
the metadata files, making tree(s) an optional requirement.
2021-09-16 10:56:09 +02:00
Itxaka
aba89db204 installer: Take into consideration provides names
When installing a package that has a provides values, the installer only
checks the original package against the resolved package. This leads to
the requires keyword not working as expected as the solved final package
is never gonna match the name of the old/superseeded package.

This patch checks the asked-to-install package name against the list of
names that the matched name provides.

Signed-off-by: Itxaka <igarcia@suse.com>
2021-09-06 14:58:50 +02:00
Itxaka
178690842f
Show the package name on uninstall error (#251)
* Show the package name on uninstall error

Currently there is a generic message of error when uninstalling if one
of the provided packages is not found in the system. This is quite
obnoxious if you are providing luet a big list to uninstall as you dont
know which package is not installed and whats preventing you from
running the uninstall command.

This pathc is a very simple change that prints the packag name along
with the error to provide more info to the end user

Signed-off-by: Itxaka <igarcia@suse.com>

* Update pkg/installer/installer.go

Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>

Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
2021-08-26 14:45:19 +02:00
Daniele Rondina
0f545952cd
cmd/config: simplify code (#246) 2021-08-11 16:26:34 +02:00
Daniele Rondina
0cc8930708
Finalizer envs (#242)
* Allow to define envs for finalizer

Fixes: #241

* tests: Add integration test for finalizer with envs
2021-08-11 11:18:16 +02:00
Ettore Di Giacinto
35eb63a31c Create dest-dir if doesn't exist while unpacking without snapshotter
Signed-off-by: Ettore Di Giacinto <mudler@sabayon.org>
2021-08-11 10:36:41 +02:00
Ettore Di Giacinto
16bb93e165 Don't check file conflicts while running installops
We already check at beginning during install/upgrade. Also, better
to not fail while upgrade has already started for a check that we already did.

Signed-off-by: Ettore Di Giacinto <mudler@sabayon.org>
2021-08-11 10:36:41 +02:00
Daniele Rondina
540e8151ad
compiler: Speedup & troubleshooting debug messages (#240)
* compiler: Speedup stripFromRootfs and add debug log

* compiler: on stripFromRootfs show files removed with debug
2021-08-11 10:06:52 +02:00
Ettore Di Giacinto
ccf83b0d5f Avoid deadlocking by FindPackages calls 2021-08-09 15:13:53 +02:00
Ettore Di Giacinto
57cefc7d6b Fixup data race when updating revdeps 2021-08-09 14:09:10 +02:00
Ettore Di Giacinto
97ff647f07 Turn tmpdir in abs path if required
Fixes #239
2021-08-09 13:17:16 +02:00
Ettore Di Giacinto
ff092db97d Increase http timeout to 120s by default 2021-08-09 12:58:01 +02:00
Ettore Di Giacinto
10ae872a3e Simplify revdep code
Isolate common code into a function and also fix a sublte bug hiding in
that code.

We need to stab a copy of the package inside our PackageMap to avoid
having symlinks pointing at the same values when iterating over
requires. ( e.g. happened in this case:
https://github.com/rancher-sandbox/cOS-toolkit/pull/467#issuecomment-895060115
)
2021-08-09 12:47:27 +02:00
Ettore Di Giacinto
4d6cccb2fa Give explaination when formulas are unsat
Fixes #168

Signed-off-by: Ettore Di Giacinto <mudler@sabayon.org>
2021-08-07 14:46:05 +02:00
Ettore Di Giacinto
33b1c63815 Allow to have shared templates across packages
This changeset allows to have shared templates in a static folder
"templates" present in each luet tree. If the directory is present, it
gets scanned and templated accordingly on top of each package. This
allows to use such folder to store custom blocks to share between
packages.

This is still experimental and subject to change, this is just a first
pass version to provide the feature. It needs to be refined still as it
would be more elegant to use the helm engine properly and map our
structure to the engine instead of adapting it roughly.

Fixes #224
2021-08-04 16:35:28 +02:00
Ettore Di Giacinto
658612fcf3 Check file conflicts also on packages that are going to be swapped out
Fixes #237
2021-08-04 10:43:12 +02:00
Ettore Di Giacinto
9d1594c036 Check if the system-target path supplied is absolute
docker.Untar
(https://github.com/mudler/luet/blob/master/vendor/github.com/docker/docker/pkg/archive/archive.go#L942) requires absolute paths.
We didn't do any input validation before, assuming the path passed by
were absolute since they were coming from YAML configuration files, now
that this is not the truth anymore we need to sanitize the input.

With this change we check if the given path is absolute or relative, if
it's relative we calculate the absolute path and use it in place.
2021-08-03 16:22:59 +02:00
Ettore Di Giacinto
75906c4198 Add missing check for error 2021-08-03 13:36:38 +02:00
Ettore Di Giacinto
cb032dc714 Allow to manipulate requires/conflicts/provides
This allows to manipulate requires, conflicts and provides with
templating in build time.

Signed-off-by: Ettore Di Giacinto <mudler@sabayon.org>
2021-08-03 13:27:44 +02:00
Ettore Di Giacinto
2c7e495fa1 Add http timeout to grab
grab.NewClient() doesn't set a specific timeout
https://github.com/cavaliercoder/grab/blob/v2.0.0/client.go#L37 even if
the project advertize "default sane settings".

We default to 30, and allow to set it up with HTTP_TIMEOUT

Signed-off-by: Ettore Di Giacinto <mudler@sabayon.org>
2021-08-03 11:57:44 +02:00
Ettore Di Giacinto
5eb586ddb0 Speedup conflict check by caching in memory files 2021-07-30 12:40:55 +02:00
Ettore Di Giacinto
becac7d853 Add tests 2021-07-29 11:03:02 +02:00
Ettore Di Giacinto
9aa3159787 Check for file conflicts before install
Fixes #88
2021-07-29 11:03:02 +02:00
Ettore Di Giacinto
c8573f9535 Adapt hash tests
Signed-off-by: Ettore Di Giacinto <mudler@sabayon.org>
2021-07-09 17:15:17 +02:00
Ettore Di Giacinto
76b70ebeb4 Add dirhash to package signatures
While this breaks current hashing, it ties also the spec content to the
hash, in this way if we change something in the spec folder, it breaks
the hashing for the package.

Signed-off-by: Ettore Di Giacinto <mudler@sabayon.org>
2021-07-09 12:37:38 +02:00
Ettore Di Giacinto
2efb17a06c Add specific field to resolve final images
This commit starts deprecation of `join` keyword in favor of
`requires_final_images` as boolean in the compilation spec.

The change is driven by two reasons: syntax and guarantee unique hashes.

- the hashtree when computing a hash it analizes the requires field of
  each spec, ignoring the join field
- the join field doesn't add much value. Having it separate suggests
  that a spec can contain both `requires` and `join`, but that's not
  actually true. We just act differently on the same list.

Signed-off-by: Ettore Di Giacinto <mudler@sabayon.org>
2021-07-09 12:37:38 +02:00