Commit Graph

48 Commits

Author SHA1 Message Date
Ettore Di Giacinto
917d0935ad Show progressbar only if terminal is big enough
Fixes #259

The bug is caused by
4c725e56bf/progressbar_printer.go (L190)
which is not taking into account when barCurrentLength is <=0.

As a workaround display it only if the terminal width is big enough.

Now the context drives this kind of runtime state, so we keep everything
tight and we inject only that into the workers
2021-10-22 12:39:07 +02:00
Ettore Di Giacinto
8d34a6ebb1 Use the cache to store temporary download files 2021-10-21 23:58:45 +02:00
Ettore Di Giacinto
a1c669d3ae Define context for scoped operation across core types
It holds necessary state plus additional information relative to the
context which we are being run to (e.g. if we are in a terminal or not).
Besides in the future we can use it also as a contextual logger to
provide more smart logging capabilities.

This also replace the general global configuration instance that
previously was share between the core components.
2021-10-21 23:58:24 +02:00
Ettore Di Giacinto
fe14d56afe Massive UX rewrite
- Ditch multiple libraries for progressbar, spinner, colors and replace
  with pterm
- Detect when running on terminal and disable automatically spinner
- Add support for multiple progress bars
- Huge rewrite of the configuration part. No more crazy stuff with viper
  CLI commands now correctly overrides default config file as expected
- Limit banner to be displayed on relevant parts

Fixes #211
Fixes #105
Fixes #247
Fixes #233
2021-10-21 23:58:00 +02:00
Ettore Di Giacinto
5b4e930fc3 Do implement a real cache
Instead of merely storing files into disk use a real cache.

This also makes possible finally to reference artifacts in the cache with the
package checksum, which solves the cache hit checksum failures we had
previously.
2021-10-19 21:25:01 +02:00
Ettore Di Giacinto
6ea05e59ea Drop imgworker client
Cleanup code which allows to extract images in a privileged way. the
containerd way supersedes it.
2021-10-14 22:06:01 +02:00
Ettore Di Giacinto
ff092db97d Increase http timeout to 120s by default 2021-08-09 12:58:01 +02:00
Ettore Di Giacinto
2c7e495fa1 Add http timeout to grab
grab.NewClient() doesn't set a specific timeout
https://github.com/cavaliercoder/grab/blob/v2.0.0/client.go#L37 even if
the project advertize "default sane settings".

We default to 30, and allow to set it up with HTTP_TIMEOUT

Signed-off-by: Ettore Di Giacinto <mudler@sabayon.org>
2021-08-03 11:57:44 +02:00
Ettore Di Giacinto
92e18d5782 Support priv/unpriv image extraction
Optionally add back privileged extraction which can be enabled with
LUET_PRIVILEGED_EXTRACT=true

Signed-off-by: Ettore Di Giacinto <mudler@sabayon.org>
2021-06-16 23:30:28 +02:00
Ettore Di Giacinto
c3b552103f Return a specific struct for the image
And drop imgworker
2021-06-04 11:09:41 +02:00
Itxaka
1bd4d520a4
Add 2 new events for image unpacking (#226)
* Reduce possibility of circular dependency

Just by adding an import for bus to anything in the helper dir, we would
run into a circular dependency due to how things are structured. That
means that we cannot set any events for unpacking or docker helper
pulling an image.

This commit tries to work around this by doing several things.
 - Remove full imports of the helper module by segmentating some modules
   into their own submodule, like docker or match so just using a small match
   function doesnt bring the whole module
 - Removing a simple function to check if a dir exists from importing
   the full helper module and instead write the function (5 lines)
 - Using logrus in the bus module instead of logger, which avoids a
   circular dependency

Signed-off-by: Itxaka <igarcia@suse.com>

* Add two new events for unpacking an image

Both pre and post unpacking an image

Signed-off-by: Itxaka <igarcia@suse.com>
2021-06-01 16:43:31 +02:00
Ettore Di Giacinto
c9aed37fa7 Fixup on values interpolation and metadata retrieve
- Fixup search path on metadata spec load. Previously we were reading
  the package being passed, and not the one resolved (it failed against
selectors)
- Do inherit first pushrepositories, so they take precedence over pull
- Add test cases to cover build values interpolation by remote
  repositories
- Enhance test cases to check image cache repository inheritance when
  --from-repositories is passed
- Fix race condition when inheriting buildspec options: Instead of consuming the compiler one, annotate the updates in the
package BuildOption spec which is passed by
- Update vendor
2021-04-23 12:02:19 +02:00
Ettore Di Giacinto
562fcc2421 Strip invalid chars from docker images also for metadata files
Fixes #199
2021-04-19 17:18:37 +02:00
Ettore Di Giacinto
57c769b4a5 Refactor compiler and annotate buildoptions into compiler metadata
This allows to later pick up values used during build of each package
2021-04-16 13:57:54 +02:00
Ettore Di Giacinto
f8ef1c0889 Print error when failing downloading docker image 2021-03-16 11:34:36 +01:00
Ettore Di Giacinto
0028dd3a92 Support content trust images and pull with authentication
Contact the notary server if ```--verify``` is specified (or `verify:
true` is enabled on the repo config) and verify if the image is signed,
use the returned value to pull the verified image.
2021-03-11 17:57:59 +01:00
Ettore Di Giacinto
675170939d Expose DownloadAndExtractDockerImage as a util
Create a util sub cmd to add all utils that are handy for development
and already present in the luet codebase. We expose in this case `luet
util unpack` to unpack a docker image without a docker daemon running.
2021-03-09 09:22:37 +01:00
Ettore Di Giacinto
9eda81667b Use common ImageID() when computing final images
It also adds tests and strip invalid "+" character which is not
supported in docker image tags
2021-03-07 11:01:08 +01:00
Ettore Di Giacinto
57e19c61e7 Start spinner before pulling docker artifacts 2021-02-22 11:54:09 +01:00
Ettore Di Giacinto
5ee1e28b9c Display informative message when pulling docker artifacts 2021-02-22 11:50:52 +01:00
Ettore Di Giacinto
a037bc545b Add comment on required permission to unpack 2021-01-22 16:55:51 +01:00
Ettore Di Giacinto
d2e6409451 Add docker client unit test 2021-01-22 16:55:51 +01:00
Ettore Di Giacinto
a6e7a3059c Respect artifact extension when populating cache
This caused cache to not hit correctly
2021-01-22 16:55:51 +01:00
Ettore Di Giacinto
163f93067c Temporarly rework genuinetools/img code for pull/unpack without Docker 2021-01-22 16:55:51 +01:00
Ettore Di Giacinto
91ea2ed99f Make docker image repositories actually working
Several changes are included:
- Expose ensureDir in helpers, and call it in the Docker client. In
  other implementations that was handled by CopyFile behind the scenes,
  but that's not the case here
- Create accessor in Artifact to create Artifact objects from files.
  This is handy when we have to carry over downloaded package content
  into caches when artifacts are already verified
- Fix various issues around the imagePush flag, so now trees are pushed
  forcefully each time
- Take into consideration the real artifact name when pushing single
  files in the docker image. This behavior should be changed eventually,
  because single files which aren't repository packages now are in its
  own docker image, but we should have just one that brings the required
  metadata alltogether.
2021-01-22 16:54:19 +01:00
Ettore Di Giacinto
dbd37afced Add docker client #169 2021-01-22 16:54:19 +01:00
Ettore Di Giacinto
b81ce66914 Reduce download verbosity 2020-12-27 20:21:05 +01:00
Daniele Rondina
60d5c9dfd5 Add download progress bar 2020-11-20 18:12:23 +01:00
Daniele Rondina
f9a7113ab9 client/http: Add experimental download info 2020-11-19 00:56:28 +01:00
Daniele Rondina
9a1787ddaf client/local: Handle config_from_host on DownloadFile 2020-11-08 17:06:05 +01:00
Daniele Rondina
e70cdbaaf7 Respect rootfs on repositories urls 2020-11-08 00:00:15 +01:00
Daniele Rondina
11944873ea Integrate tmpdir_base params and tmpdirs cleanup 2020-04-30 20:29:28 +02:00
Ettore Di Giacinto
d330fedcc4
fmt 2020-02-12 11:05:13 +01:00
Ettore Di Giacinto
dfb6dab9dc
Move repository helpers under config
They are generated after the system config, let the structure provide such information
2020-02-12 10:20:07 +01:00
Daniele Rondina
c353ab4978 client/http: Add support for authentication Basic/Token 2020-02-03 00:58:55 +01:00
Daniele Rondina
298de447b8 Fix installer/client test suite 2020-02-01 21:18:32 +01:00
Daniele Rondina
78b5963a4f Add support for local packages cache 2020-02-01 19:35:30 +01:00
Ettore Di Giacinto
0147b2cf99
Don't cleanup the whole cache download dir, just the downloaded artifact
Otherwise whatever we download cannot be consumed (e.g. tree).
Instead whoever is calling the client should care to cleanup the returned artifact
2020-01-30 18:03:25 +01:00
Daniele Rondina
c9b684523f installer: Add support for cached repository 2020-01-12 23:35:58 +01:00
Daniele Rondina
01e66ee0b4
Review install phase 2020-01-01 13:44:39 +01:00
Daniele Rondina
3b266fd600
installer: Repositoris now support multiple uris 2020-01-01 13:43:23 +01:00
Daniele Rondina
0cb49a40c0
Support logging level and spinner customization 2020-01-01 13:43:22 +01:00
Ettore Di Giacinto
58eb483e32
Do not create new artifact on client
Otherwise we loose artifact metadata - as checksum and compressiontype

Refers to #33
2019-12-30 16:35:33 +01:00
Ettore Di Giacinto
5915688830
Add client tests 2019-11-26 18:06:23 +01:00
Ettore Di Giacinto
a54040dcd3
Add http client for installer #3 2019-11-26 18:05:48 +01:00
Ettore Di Giacinto
e9c01b46a7
Use well defined structure for serializing, fixups to make test green 2019-11-23 15:42:53 +01:00
Ettore Di Giacinto
e664f4f2cf
Return client from repository and install packages in the workers 2019-11-22 23:13:33 +01:00
Ettore Di Giacinto
a2f23d3bf5
WIP 2019-11-22 21:01:38 +01:00