Commit Graph

84 Commits

Author SHA1 Message Date
Ettore Di Giacinto
ba0d625b5f 🎨 Port package to types
This refactors DefaultPackage into types.Package and gets rid of the
interface. This is a preceeding for a follow up where accessors will be
removed from the code.
It also does several cleanup, so we get rid also of some unneeded
dependencies.
2022-01-08 00:05:20 +01:00
Ettore Di Giacinto
9bd6730aeb 🤖 Adapt makefile/scripts to ginkgo changes 2022-01-04 18:47:21 +01:00
Ettore Di Giacinto
196cdc5cfc 🔧 Extract common func into api function, also set sane defaults 2021-12-28 18:55:59 +01:00
Ettore Di Giacinto
7e825400e2 🔧 Use crane.Insecure while checking image availability
As those checks are not consuming any digest, we just use them to assess
if we need to build or not certain packages. The backend will refuse the
image if not configured appropriately
2021-12-28 14:54:11 +01:00
Ettore Di Giacinto
c98f427156 🎨 Introduce contextualized logging
This commit is multi-fold as it also refactors internally context and logger
as interfaces so it is easier to plug luet as a library externally.

Introduces a garbage collector (related to #227) but doesn't handle yet
parallelism.

Closes #265
2021-12-21 21:54:14 +01:00
Ettore Di Giacinto
c7f9708f90 Add CreateTar to image API
Add api call which uses go-container registry to create OCI images from
standard tar archives.

Consume new API when generating final images instead of docker building them
and adapts/add tests as necessary.

This change now allows to carry over xattrs to final images.

Fixes #266
2021-10-29 10:35:03 +02:00
Ettore Di Giacinto
9799b7c94b Add Image reference by pipe, refactor 2021-10-26 16:56:49 +02:00
Ettore Di Giacinto
35fcd868ee Switch to ondisk also when unpacking FS
From benchmarks it seems to be still faster. Add a note for a future
improvement
2021-10-26 16:56:49 +02:00
Ettore Di Giacinto
aea3cdff8d Use ondisk reference for deltas 2021-10-26 16:56:49 +02:00
Ettore Di Giacinto
144c409908 Disable buffer on docker remote
This causes to load otherwise the full tarball into memory
2021-10-25 23:57:09 +02:00
Ettore Di Giacinto
c220eac061 Move bus to api/core 2021-10-24 19:07:41 +02:00
Ettore Di Giacinto
ebbb3aad27 Use API also when pulling from helpers used in client 2021-10-24 18:26:30 +02:00
Ettore Di Giacinto
6a9f19941a Add crane-based methods for extraction
- create a new api package to encapsulate image manipulation
- use new api method to calculate delta

Fixes #258
Fixes #204
Fixes #90
2021-10-24 18:26:08 +02:00
Ettore Di Giacinto
6b45b1d61c ux: rework displaying of success messages 2021-10-22 17:55:38 +02:00
Ettore Di Giacinto
a1c669d3ae Define context for scoped operation across core types
It holds necessary state plus additional information relative to the
context which we are being run to (e.g. if we are in a terminal or not).
Besides in the future we can use it also as a contextual logger to
provide more smart logging capabilities.

This also replace the general global configuration instance that
previously was share between the core components.
2021-10-21 23:58:24 +02:00
Ettore Di Giacinto
fe14d56afe Massive UX rewrite
- Ditch multiple libraries for progressbar, spinner, colors and replace
  with pterm
- Detect when running on terminal and disable automatically spinner
- Add support for multiple progress bars
- Huge rewrite of the configuration part. No more crazy stuff with viper
  CLI commands now correctly overrides default config file as expected
- Limit banner to be displayed on relevant parts

Fixes #211
Fixes #105
Fixes #247
Fixes #233
2021-10-21 23:58:00 +02:00
Ettore Di Giacinto
5b4e930fc3 Do implement a real cache
Instead of merely storing files into disk use a real cache.

This also makes possible finally to reference artifacts in the cache with the
package checksum, which solves the cache hit checksum failures we had
previously.
2021-10-19 21:25:01 +02:00
Itxaka
1bd4d520a4
Add 2 new events for image unpacking (#226)
* Reduce possibility of circular dependency

Just by adding an import for bus to anything in the helper dir, we would
run into a circular dependency due to how things are structured. That
means that we cannot set any events for unpacking or docker helper
pulling an image.

This commit tries to work around this by doing several things.
 - Remove full imports of the helper module by segmentating some modules
   into their own submodule, like docker or match so just using a small match
   function doesnt bring the whole module
 - Removing a simple function to check if a dir exists from importing
   the full helper module and instead write the function (5 lines)
 - Using logrus in the bus module instead of logger, which avoids a
   circular dependency

Signed-off-by: Itxaka <igarcia@suse.com>

* Add two new events for unpacking an image

Both pre and post unpacking an image

Signed-off-by: Itxaka <igarcia@suse.com>
2021-06-01 16:43:31 +02:00
Ettore Di Giacinto
76695b2fc8 docker: check during extraction if image exists and pull if needed 2021-05-25 11:04:06 +02:00
Ettore Di Giacinto
0379855592 Drop docker squash support
It can be easily implemented as a plugin

Fixes: #206
2021-05-04 11:17:10 +02:00
Ettore Di Giacinto
57c769b4a5 Refactor compiler and annotate buildoptions into compiler metadata
This allows to later pick up values used during build of each package
2021-04-16 13:57:54 +02:00
Ettore Di Giacinto
4f1e4c0b41 Switch to containerd when unpacking layers 2021-03-18 10:47:58 +01:00
Ettore Di Giacinto
662742851a Generate backend bus events in the backends 2021-03-16 14:46:28 +01:00
Ettore Di Giacinto
749a4cb615 Add --backend-args
Allow to add arguments to the backend build arguments

Fixes #146
2021-02-22 13:49:29 +01:00
Ettore Di Giacinto
21bd76af9c Uncomplicate runCommand and return command output
Fixes #189
2021-02-22 11:44:46 +01:00
Ettore Di Giacinto
b3e3abec8f Fixup spinner data race
Add spinner lock
2021-02-17 13:01:32 +01:00
Daniele Rondina
c22adb3a47 compiler: Move spinner at the low level 2021-02-13 09:28:54 +01:00
Daniele Rondina
c1fe3278fa backend: Add realtime output on building phase
The realtime output could be configured through
LUET_GENERAL__SHOW_BUILD_OUTPUT environment
variable or related config option or through
`--live-output` option.
2021-02-02 12:58:34 +01:00
Ettore Di Giacinto
85b5c96bdd Promote to info building image messages
The user wants to know whats going on in this case. Image builds can
take up also long time
2021-01-24 19:09:09 +01:00
Ettore Di Giacinto
6f5f400765 Don't bail out if image doesn't exist locally
The backend will figure out if we have the image or not, otherwise will
atempt to pull if not there.

Skip retrieve integration test with img as its not supported.
2021-01-24 19:05:21 +01:00
Ettore Di Giacinto
be87861657 img: pull image if not locally present while extracting 2021-01-24 13:17:11 +01:00
Ettore Di Giacinto
be7b56bae3 Split ImageDefinitionToTar test
ImageDefinitionToTar it is not actually used by compiler code, but can
be handy from an API perspective, so we keep it.
2021-01-24 12:56:25 +01:00
Ettore Di Giacinto
7d11df3225 Simplify delta generation, and avoid two-pass with img backend
This changeset also drops --keep-exported-images, which is quite unused
and can be replaced with a plugin, or either by manually exporting the
resulting images.
2021-01-24 12:27:07 +01:00
Ettore Di Giacinto
b9f0ef1c55 Implement ImageExists in the img backend 2021-01-23 22:01:29 +01:00
Ettore Di Giacinto
dbd37afced Add docker client #169 2021-01-22 16:54:19 +01:00
Ettore Di Giacinto
322fe72ef2 Generate repository metadata and packages for docker repository type
Drop image-repository on create-repo. In case of a docker repository, --output is the image reference to use.
Also restore default output build dir.

See also: #169
2021-01-22 16:53:52 +01:00
Ettore Di Giacinto
a1f4c28973 Add GenerateFinalImage to package artifacts
GenerateFinalImage generates a docker image from scratch with the
artifact content.

Related to #169
2021-01-18 12:08:47 +01:00
Ettore Di Giacinto
43b0b11028 Define a build context for backends 2021-01-18 11:06:54 +01:00
Ettore Di Giacinto
ef034d87b0 Detect if images are available if we don't have to generate a Package
While building, if we aren't doing a clean build, we scan now to see if
images are available and we skip, in case we don't find a metadata
already.
2020-12-14 18:32:32 +01:00
Ettore Di Giacinto
265e2371b4 Add ArtifactNode to test, now we get the gen Dockerfile in the diff 2020-12-12 16:04:54 +01:00
Ettore Di Giacinto
797a34ba49 Reuse same dockerfile gen logic between prelude and steps
As now we build only when necessary, we need to make sure the images are
built similarly. The discrepancies between the two are less now, and
they can share the same logic.

This fixes a regresion where when no prelude is defined, the build
context isn't copied over
2020-12-12 11:16:34 +01:00
Ettore Di Giacinto
a1453b7242 Fixup error messages 2020-12-11 23:03:56 +01:00
Ettore Di Giacinto
b5990b5333 Generate changes from CompilerBackendOptions and pass by image name so img can unpack images 2020-12-09 00:27:37 +01:00
Ettore Di Giacinto
9784d6192a Don't hide error on pulling image 2020-11-28 18:03:43 +01:00
Ettore Di Giacinto
44d33eceba Set workdir also on step image
Otherwise with DOCKER_SQUASH=true it wouldn't be coherent on where to
find the package files
2020-11-28 12:07:07 +01:00
Ettore Di Giacinto
8ce135fe12 Add DOCKER_SQUASH 2020-11-27 23:38:31 +01:00
Ettore Di Giacinto
14c1d6ef24 Refactor and optimize build process 2020-11-10 18:14:18 +01:00
Daniele Rondina
d219a2e0fb Run travis task with/without buildkit 2020-11-07 11:41:44 +01:00
Daniele Rondina
6819a28f07 Add support to DOCKER_BUILDKIT on test 2020-11-07 11:37:58 +01:00
Ettore Di Giacinto
95e640c9d0 Make solver type switchable 2020-10-29 16:32:13 +01:00