Commit Graph

799 Commits

Author SHA1 Message Date
Itxaka
1b35a674ea Print plugin success messages + print plugin location on load (#267)
* report plugin state if succeed

We havbe a state field in the plugin response that its not being used
for anything. This patch makes luet print the state reported from the
plugin if its not empty as a way for plugins to report data on success
to users. If the field is empty it will be ignored.

Signed-off-by: Itxaka <igarcia@suse.com>

* Print plugin path

This patch adds the plugin location to the printed plugin list for a
more rich view of the loaded plugins

Signed-off-by: Itxaka <igarcia@suse.com>
2021-10-28 11:42:56 +02:00
Ettore Di Giacinto
b5def989ac Drop unused code 2021-10-27 11:22:06 +02:00
Ettore Di Giacinto
37cc186c0b delta: trim path when computing src files set
The path contains an ending "/" which wouldn't match when we walk dst as
it's not there.

That had the unpleasant effect of creating empty folders in the
destinations
2021-10-27 11:00:10 +02:00
Ettore Di Giacinto
9799b7c94b Add Image reference by pipe, refactor 2021-10-26 16:56:49 +02:00
Ettore Di Giacinto
262d09dfbc Lower message levels 2021-10-26 16:56:49 +02:00
Ettore Di Giacinto
b974f44095 Add cache to avoid RAM consumption
When we have huge file lists we can burst too much into RAM which would
cause OOMs in certain devices. Use instead a smart cache which
automatically drops to disk when necessary.
2021-10-26 16:56:49 +02:00
Ettore Di Giacinto
35fcd868ee Switch to ondisk also when unpacking FS
From benchmarks it seems to be still faster. Add a note for a future
improvement
2021-10-26 16:56:49 +02:00
Ettore Di Giacinto
aea3cdff8d Use ondisk reference for deltas 2021-10-26 16:56:49 +02:00
Ettore Di Giacinto
daa9eb98d2 Walk destination only once when computing delta
Avoid the double pass by constructing the list on the fly
2021-10-26 16:56:49 +02:00
Itxaka
1f0324c452 Log debug before failing (#263)
If a plugin failed, we were skipping the debug info which is kind of
useful :=)

Signed-off-by: Itxaka <igarcia@suse.com>
2021-10-26 11:18:56 +02:00
Itxaka
7cd455fff4 Set proper error message on plugin failure
Currently we are setting the error message in a no-space full sentence
which is pretty ugly:

| FATA[0000] Pluginluet-cosignat/usr/local/bin/luet-cosignErrorerror while executing plugin: exit status 1

Signed-off-by: Itxaka <igarcia@suse.com>
2021-10-26 00:28:30 +02:00
Ettore Di Giacinto
144c409908 Disable buffer on docker remote
This causes to load otherwise the full tarball into memory
2021-10-25 23:57:09 +02:00
Ettore Di Giacinto
f6bb7a9405 Make sure to pull images before generating artifacts
Fixes #262
2021-10-25 23:56:38 +02:00
Ettore Di Giacinto
1b1ab6225c Use table lookup for checking addition files 2021-10-24 21:55:42 +02:00
Ettore Di Giacinto
c220eac061 Move bus to api/core 2021-10-24 19:07:41 +02:00
Ettore Di Giacinto
67a07e7c5a Drop link to moby fork 2021-10-24 19:07:41 +02:00
Ettore Di Giacinto
c897bffdfc Drop untar 2021-10-24 19:07:41 +02:00
Ettore Di Giacinto
52ad2b5cfa Fixup config protect 2021-10-24 18:26:30 +02:00
Ettore Di Giacinto
6ff22d923c Make default build dir over context temp 2021-10-24 18:26:30 +02:00
Ettore Di Giacinto
37a9a3ef55 use containerd to uncompress 2021-10-24 18:26:30 +02:00
Ettore Di Giacinto
4a45b5410d Introduce lock for installation
It is used to ensure integrity and that we do install one package at
once. This is to ensure that we extract correctly, and that we are not
too much I/O intensive depending on CPU
2021-10-24 18:26:30 +02:00
Ettore Di Giacinto
819271b9bd Fixup tests 2021-10-24 18:26:30 +02:00
Ettore Di Giacinto
063f704057 update vendor 2021-10-24 18:26:30 +02:00
Ettore Di Giacinto
ebbb3aad27 Use API also when pulling from helpers used in client 2021-10-24 18:26:30 +02:00
Ettore Di Giacinto
ad489c2157 tests: pull image before running 2021-10-24 18:26:30 +02:00
Ettore Di Giacinto
454a560f4c Take count of os separator in extraction 2021-10-24 18:26:30 +02:00
Ettore Di Giacinto
acd685b927 Extract with new image API 2021-10-24 18:26:25 +02:00
Ettore Di Giacinto
6a9f19941a Add crane-based methods for extraction
- create a new api package to encapsulate image manipulation
- use new api method to calculate delta

Fixes #258
Fixes #204
Fixes #90
2021-10-24 18:26:08 +02:00
Ettore Di Giacinto
d44befe9ff tests: add context unit tests 2021-10-23 15:26:45 +02:00
Ettore Di Giacinto
65892f9bfc ux: Display only success on green 2021-10-23 12:40:55 +02:00
Ettore Di Giacinto
315bfb5a54 Move http timeout to the general configuration
Fixes https://github.com/mudler/luet/issues/250 as now it is documented
in the cli --help too.
2021-10-23 11:41:15 +02:00
Ettore Di Giacinto
57c8236184 fixup: cache miss with docker client 2021-10-23 01:50:01 +02:00
Ettore Di Giacinto
6b45b1d61c ux: rework displaying of success messages 2021-10-22 17:55:38 +02:00
Ettore Di Giacinto
5eb5a42bf7 Generate snapshot and push it along
Fixes #260
Fixes #223
2021-10-22 17:55:08 +02:00
Ettore Di Giacinto
0bacdc75f2 Allow clients to pick a specific referenceID
Related to #260
2021-10-22 12:39:36 +02:00
Ettore Di Giacinto
917d0935ad Show progressbar only if terminal is big enough
Fixes #259

The bug is caused by
4c725e56bf/progressbar_printer.go (L190)
which is not taking into account when barCurrentLength is <=0.

As a workaround display it only if the terminal width is big enough.

Now the context drives this kind of runtime state, so we keep everything
tight and we inject only that into the workers
2021-10-22 12:39:07 +02:00
Ettore Di Giacinto
8d34a6ebb1 Use the cache to store temporary download files 2021-10-21 23:58:45 +02:00
Ettore Di Giacinto
d58a563d52 uninstall: limit cleanup to s.Target
This was specific to when s.Target differs from /. The helpers creating
the path were considering the full traversal, instead they should stop
at target.

E.g. our rootfs is in /foo/bar, and while we build the list of paths to
prune for /foo/bar/baz/bar, we should stop looking at /foo/bar.
2021-10-21 23:58:45 +02:00
Ettore Di Giacinto
a1c669d3ae Define context for scoped operation across core types
It holds necessary state plus additional information relative to the
context which we are being run to (e.g. if we are in a terminal or not).
Besides in the future we can use it also as a contextual logger to
provide more smart logging capabilities.

This also replace the general global configuration instance that
previously was share between the core components.
2021-10-21 23:58:24 +02:00
Ettore Di Giacinto
fe14d56afe Massive UX rewrite
- Ditch multiple libraries for progressbar, spinner, colors and replace
  with pterm
- Detect when running on terminal and disable automatically spinner
- Add support for multiple progress bars
- Huge rewrite of the configuration part. No more crazy stuff with viper
  CLI commands now correctly overrides default config file as expected
- Limit banner to be displayed on relevant parts

Fixes #211
Fixes #105
Fixes #247
Fixes #233
2021-10-21 23:58:00 +02:00
Ettore Di Giacinto
d4edaa9de8 Adapt unit tests 2021-10-19 21:25:01 +02:00
Ettore Di Giacinto
5b4e930fc3 Do implement a real cache
Instead of merely storing files into disk use a real cache.

This also makes possible finally to reference artifacts in the cache with the
package checksum, which solves the cache hit checksum failures we had
previously.
2021-10-19 21:25:01 +02:00
Ettore Di Giacinto
2eeb464946 Make repositories arch-aware
Introduce an arch field that can be used to filter repositories based on
the local architecture.

If arch is provided and matching with the current GOARCH then the
repository is enabled
2021-10-19 14:19:53 +02:00
Ettore Di Giacinto
e764b1cd29 Add CLI helpers and BuildTree heuristic
The BuildTree implementation was in parallel-build-tools, which actually makes
sense to share in the compiler as can be used as another way to trigger
builds.

The CLI helpers were split in several CIs
implementations. This moves a new layout where we expose several
packages, as we will refactor things slowly.
2021-10-15 21:14:48 +02:00
Ettore Di Giacinto
6ea05e59ea Drop imgworker client
Cleanup code which allows to extract images in a privileged way. the
containerd way supersedes it.
2021-10-14 22:06:01 +02:00
Ettore Di Giacinto
d5819bb4cb Fixup benchmark tests 2021-10-10 19:07:46 +02:00
Ettore Di Giacinto
6ba028f0ea Make sure we do compute the best fit
While install calls upgrade which in turns calls a relaxed install on
its results, this doesn't make sure that the new results are at the best
available version. We do iterate here over the results to compute the
best set.

It also expands computeUpgrade with the possibility to selectively
choose which packages to upgrade and which not.
2021-10-10 19:04:55 +02:00
Ettore Di Giacinto
9de6b22764 Make singlecore default solver 2021-10-10 00:29:13 +02:00
Ettore Di Giacinto
097e310d3a Drop join from compilation specs
Adapt also hash and integration tests
2021-10-10 00:28:54 +02:00
Ettore Di Giacinto
5a63bbd0d2 Deprecate parallel solver 2021-10-10 00:28:54 +02:00