Commit Graph

56 Commits

Author SHA1 Message Date
Ettore Di Giacinto
196cdc5cfc 🔧 Extract common func into api function, also set sane defaults 2021-12-28 18:55:59 +01:00
Ettore Di Giacinto
c98f427156 🎨 Introduce contextualized logging
This commit is multi-fold as it also refactors internally context and logger
as interfaces so it is easier to plug luet as a library externally.

Introduces a garbage collector (related to #227) but doesn't handle yet
parallelism.

Closes #265
2021-12-21 21:54:14 +01:00
Ettore Di Giacinto
c7f9708f90 Add CreateTar to image API
Add api call which uses go-container registry to create OCI images from
standard tar archives.

Consume new API when generating final images instead of docker building them
and adapts/add tests as necessary.

This change now allows to carry over xattrs to final images.

Fixes #266
2021-10-29 10:35:03 +02:00
Ettore Di Giacinto
9799b7c94b Add Image reference by pipe, refactor 2021-10-26 16:56:49 +02:00
Ettore Di Giacinto
35fcd868ee Switch to ondisk also when unpacking FS
From benchmarks it seems to be still faster. Add a note for a future
improvement
2021-10-26 16:56:49 +02:00
Ettore Di Giacinto
aea3cdff8d Use ondisk reference for deltas 2021-10-26 16:56:49 +02:00
Ettore Di Giacinto
144c409908 Disable buffer on docker remote
This causes to load otherwise the full tarball into memory
2021-10-25 23:57:09 +02:00
Ettore Di Giacinto
c220eac061 Move bus to api/core 2021-10-24 19:07:41 +02:00
Ettore Di Giacinto
ebbb3aad27 Use API also when pulling from helpers used in client 2021-10-24 18:26:30 +02:00
Ettore Di Giacinto
6a9f19941a Add crane-based methods for extraction
- create a new api package to encapsulate image manipulation
- use new api method to calculate delta

Fixes #258
Fixes #204
Fixes #90
2021-10-24 18:26:08 +02:00
Ettore Di Giacinto
6b45b1d61c ux: rework displaying of success messages 2021-10-22 17:55:38 +02:00
Ettore Di Giacinto
a1c669d3ae Define context for scoped operation across core types
It holds necessary state plus additional information relative to the
context which we are being run to (e.g. if we are in a terminal or not).
Besides in the future we can use it also as a contextual logger to
provide more smart logging capabilities.

This also replace the general global configuration instance that
previously was share between the core components.
2021-10-21 23:58:24 +02:00
Itxaka
1bd4d520a4
Add 2 new events for image unpacking (#226)
* Reduce possibility of circular dependency

Just by adding an import for bus to anything in the helper dir, we would
run into a circular dependency due to how things are structured. That
means that we cannot set any events for unpacking or docker helper
pulling an image.

This commit tries to work around this by doing several things.
 - Remove full imports of the helper module by segmentating some modules
   into their own submodule, like docker or match so just using a small match
   function doesnt bring the whole module
 - Removing a simple function to check if a dir exists from importing
   the full helper module and instead write the function (5 lines)
 - Using logrus in the bus module instead of logger, which avoids a
   circular dependency

Signed-off-by: Itxaka <igarcia@suse.com>

* Add two new events for unpacking an image

Both pre and post unpacking an image

Signed-off-by: Itxaka <igarcia@suse.com>
2021-06-01 16:43:31 +02:00
Ettore Di Giacinto
76695b2fc8 docker: check during extraction if image exists and pull if needed 2021-05-25 11:04:06 +02:00
Ettore Di Giacinto
0379855592 Drop docker squash support
It can be easily implemented as a plugin

Fixes: #206
2021-05-04 11:17:10 +02:00
Ettore Di Giacinto
57c769b4a5 Refactor compiler and annotate buildoptions into compiler metadata
This allows to later pick up values used during build of each package
2021-04-16 13:57:54 +02:00
Ettore Di Giacinto
4f1e4c0b41 Switch to containerd when unpacking layers 2021-03-18 10:47:58 +01:00
Ettore Di Giacinto
662742851a Generate backend bus events in the backends 2021-03-16 14:46:28 +01:00
Ettore Di Giacinto
749a4cb615 Add --backend-args
Allow to add arguments to the backend build arguments

Fixes #146
2021-02-22 13:49:29 +01:00
Ettore Di Giacinto
21bd76af9c Uncomplicate runCommand and return command output
Fixes #189
2021-02-22 11:44:46 +01:00
Ettore Di Giacinto
b3e3abec8f Fixup spinner data race
Add spinner lock
2021-02-17 13:01:32 +01:00
Daniele Rondina
c22adb3a47 compiler: Move spinner at the low level 2021-02-13 09:28:54 +01:00
Daniele Rondina
c1fe3278fa backend: Add realtime output on building phase
The realtime output could be configured through
LUET_GENERAL__SHOW_BUILD_OUTPUT environment
variable or related config option or through
`--live-output` option.
2021-02-02 12:58:34 +01:00
Ettore Di Giacinto
85b5c96bdd Promote to info building image messages
The user wants to know whats going on in this case. Image builds can
take up also long time
2021-01-24 19:09:09 +01:00
Ettore Di Giacinto
6f5f400765 Don't bail out if image doesn't exist locally
The backend will figure out if we have the image or not, otherwise will
atempt to pull if not there.

Skip retrieve integration test with img as its not supported.
2021-01-24 19:05:21 +01:00
Ettore Di Giacinto
7d11df3225 Simplify delta generation, and avoid two-pass with img backend
This changeset also drops --keep-exported-images, which is quite unused
and can be replaced with a plugin, or either by manually exporting the
resulting images.
2021-01-24 12:27:07 +01:00
Ettore Di Giacinto
dbd37afced Add docker client #169 2021-01-22 16:54:19 +01:00
Ettore Di Giacinto
a1f4c28973 Add GenerateFinalImage to package artifacts
GenerateFinalImage generates a docker image from scratch with the
artifact content.

Related to #169
2021-01-18 12:08:47 +01:00
Ettore Di Giacinto
43b0b11028 Define a build context for backends 2021-01-18 11:06:54 +01:00
Ettore Di Giacinto
ef034d87b0 Detect if images are available if we don't have to generate a Package
While building, if we aren't doing a clean build, we scan now to see if
images are available and we skip, in case we don't find a metadata
already.
2020-12-14 18:32:32 +01:00
Ettore Di Giacinto
b5990b5333 Generate changes from CompilerBackendOptions and pass by image name so img can unpack images 2020-12-09 00:27:37 +01:00
Ettore Di Giacinto
9784d6192a Don't hide error on pulling image 2020-11-28 18:03:43 +01:00
Ettore Di Giacinto
8ce135fe12 Add DOCKER_SQUASH 2020-11-27 23:38:31 +01:00
Ettore Di Giacinto
14c1d6ef24 Refactor and optimize build process 2020-11-10 18:14:18 +01:00
Ettore Di Giacinto
66513955c7
Compute image diffs internally
Is it more faster in this way as we already have all the needed folders
to the comparison extracted. In this way we don't repeat I/O operation
twice by calling container-diff.

Do not depend on container-diff anymore
2020-08-05 19:09:45 +02:00
Ettore Di Giacinto
264e1e9652
Print inspect output as string 2020-06-06 11:32:47 +02:00
Ettore Di Giacinto
9e0e1199df
Check if image exists before skipping compilation 2020-06-03 21:00:30 +02:00
Daniele Rondina
11944873ea Integrate tmpdir_base params and tmpdirs cleanup 2020-04-30 20:29:28 +02:00
Daniele Rondina
69a82a1ca5 simpledocker: Use debug option for print container-diff results 2020-04-10 09:32:12 +02:00
Daniele Rondina
02c33896d5 simpledocker: Move warning before return 2020-04-10 09:31:08 +02:00
Ettore Di Giacinto
0d2668e452
Print warning if container-diffs return errors to stderr 2020-04-08 18:31:40 +02:00
Ettore Di Giacinto
fcec6c5699
Add Push to hub feature
Add to the backends the Push capabilities to push images that were correctly built.
Also expose them to the CLI, along with KeepImg which was hidden.
2020-02-15 14:45:05 +01:00
Daniele Rondina
07633dc307 Add container-diff summary 2020-01-05 18:05:56 +01:00
Daniele Rondina
358b39b5dd
simpledocker: Support show_build_output option 2020-01-01 13:43:22 +01:00
Ettore Di Giacinto
f4fff415f4
Disable extra output from container-diff 2019-11-30 11:51:37 +01:00
Ettore Di Giacinto
6d25e5d881
Check if a layer is provided in the manifest 2019-11-26 20:22:33 +01:00
Ettore Di Giacinto
393164da36
Extract all layers of the docker image 2019-11-26 20:12:06 +01:00
Ettore Di Giacinto
602f5c68d5
Use emojis for icons 2019-11-16 14:16:59 +01:00
Ettore Di Giacinto
588d06430c
Do not spin in the backend
Signed-off-by: Ettore Di Giacinto <mudler@gentoo.org>
2019-11-16 13:26:34 +01:00
Ettore Di Giacinto
321edd1676
Cleans container-diff cache on each run
It generates false positives if run with the same image paths
2019-11-14 22:50:39 +01:00