Commit Graph

245 Commits

Author SHA1 Message Date
Ettore Di Giacinto
268239a561 Propagate verify when serializing 2021-03-15 11:37:56 +01:00
Ettore Di Giacinto
0028dd3a92 Support content trust images and pull with authentication
Contact the notary server if ```--verify``` is specified (or `verify:
true` is enabled on the repo config) and verify if the image is signed,
use the returned value to pull the verified image.
2021-03-11 17:57:59 +01:00
Ettore Di Giacinto
675170939d Expose DownloadAndExtractDockerImage as a util
Create a util sub cmd to add all utils that are handy for development
and already present in the luet codebase. We expose in this case `luet
util unpack` to unpack a docker image without a docker daemon running.
2021-03-09 09:22:37 +01:00
Ettore Di Giacinto
42f5210764 Add DownloadOnly option
It allows to download only the packages, without
installing/upgrading/replacing them

Fixes #179
2021-03-07 11:39:59 +01:00
Ettore Di Giacinto
9eda81667b Use common ImageID() when computing final images
It also adds tests and strip invalid "+" character which is not
supported in docker image tags
2021-03-07 11:01:08 +01:00
Ettore Di Giacinto
f07cf6c245 Add tests for file search 2021-03-06 18:49:14 +01:00
Ettore Di Giacinto
233429bbeb Allow to search by file
Also make possible to retrieve the artifact when searching for matches
between repositories list. This made possible to show the package list
when calling `luet search`.
2021-02-28 18:42:58 +01:00
Ettore Di Giacinto
57e19c61e7 Start spinner before pulling docker artifacts 2021-02-22 11:54:09 +01:00
Ettore Di Giacinto
5ee1e28b9c Display informative message when pulling docker artifacts 2021-02-22 11:50:52 +01:00
Ettore Di Giacinto
544895e051 Handle empty packages when pushing final images
We used to create dockerfiles blindly assuming there is content, but
that's not the case for virtual packages.

Due to https://github.com/moby/moby/issues/38039 we are forced for a
"unpleasant" workaround, as we can't create empty FROM scratch images
and export them.
2021-02-09 18:27:53 +01:00
Ettore Di Giacinto
715ee1db08 Add unit test for creating docker repositories 2021-01-22 19:59:26 +01:00
Ettore Di Giacinto
a037bc545b Add comment on required permission to unpack 2021-01-22 16:55:51 +01:00
Ettore Di Giacinto
d2e6409451 Add docker client unit test 2021-01-22 16:55:51 +01:00
Ettore Di Giacinto
a2df02e1bf Add comments on repository files #159 2021-01-22 16:55:51 +01:00
Ettore Di Giacinto
928c305ff1 Respect metadata fields about the tree filename
This caused to always ignore what was explictly asked during repo
creation
2021-01-22 16:55:51 +01:00
Ettore Di Giacinto
a6e7a3059c Respect artifact extension when populating cache
This caused cache to not hit correctly
2021-01-22 16:55:51 +01:00
Ettore Di Giacinto
9b2b877a53 Avoid to build images if already present 2021-01-22 16:55:51 +01:00
Ettore Di Giacinto
75fad993f3 Fixup repository revision bump 2021-01-22 16:55:51 +01:00
Ettore Di Giacinto
4b4c3a2e14 Adapt tests to new constructor changes 2021-01-22 16:55:51 +01:00
Ettore Di Giacinto
163f93067c Temporarly rework genuinetools/img code for pull/unpack without Docker 2021-01-22 16:55:51 +01:00
Ettore Di Giacinto
91ea2ed99f Make docker image repositories actually working
Several changes are included:
- Expose ensureDir in helpers, and call it in the Docker client. In
  other implementations that was handled by CopyFile behind the scenes,
  but that's not the case here
- Create accessor in Artifact to create Artifact objects from files.
  This is handy when we have to carry over downloaded package content
  into caches when artifacts are already verified
- Fix various issues around the imagePush flag, so now trees are pushed
  forcefully each time
- Take into consideration the real artifact name when pushing single
  files in the docker image. This behavior should be changed eventually,
  because single files which aren't repository packages now are in its
  own docker image, but we should have just one that brings the required
  metadata alltogether.
2021-01-22 16:54:19 +01:00
Ettore Di Giacinto
dbd37afced Add docker client #169 2021-01-22 16:54:19 +01:00
Ettore Di Giacinto
2f459c0469 Use only one docker image reference to push repository.
Instead of generating different images, which are harder to track and
clean, we generate a single image with various tags, corresponding to
the packages available in the repositories.

Tagging, and pushing separate images will be possible with the plugin
mechanism
2021-01-22 16:54:19 +01:00
Ettore Di Giacinto
ad455edafc Allow to push images in create-repo
Add also the --force flag to allow image overwrite

Related to #169
2021-01-22 16:54:19 +01:00
Ettore Di Giacinto
d9286a1a1e Download repository metadata with client DownloadFile, uniform downloads for Docker repositories 2021-01-22 16:54:19 +01:00
Ettore Di Giacinto
322fe72ef2 Generate repository metadata and packages for docker repository type
Drop image-repository on create-repo. In case of a docker repository, --output is the image reference to use.
Also restore default output build dir.

See also: #169
2021-01-22 16:53:52 +01:00
Ettore Di Giacinto
7390623e40 Don't warn user of accepting license in case of uninstall
Closes #160
2021-01-07 10:36:57 +01:00
Ettore Di Giacinto
bd0d2765aa Mark executed finalizers at beginning
don't retry failing finalizers, but mark as executed right away
2021-01-04 17:04:20 +01:00
Ettore Di Giacinto
a81d0bc3a3 Build assertions when swapping
When we are swapping packages, we do not run the solver to gather things
to install, but we trust the given list when calling computeInstall. In this case, the assertion
returned by computeInstall is empty, as we force l.Options.NoDeps.

This change generates the assertion list while calling computeSwap so
it's available later when we call ExecuteFinalizer.
2021-01-02 21:28:54 +01:00
Ettore Di Giacinto
55ab1894e9 Add unit test for Uninstall in Installer 2020-12-29 22:58:03 +01:00
Ettore Di Giacinto
b81ce66914 Reduce download verbosity 2020-12-27 20:21:05 +01:00
Ettore Di Giacinto
f871111e50 Collect errors from finalizer runs
Instead of failing and depend on the --force flag, always execute
finalizer and collect errors to determine if install was successfull or
not
2020-12-25 10:35:09 +01:00
Ettore Di Giacinto
736c9470cf Add db copy and clone 2020-12-19 17:45:50 +01:00
Ettore Di Giacinto
96e877fc0b Allow uninstall to take multiple packages
And treat those as a list, instead of each single of them
2020-12-19 17:16:58 +01:00
Ettore Di Giacinto
525bfb5ebf Respect --nodeps when calling Swap from the public interface 2020-12-19 15:26:18 +01:00
Ettore Di Giacinto
f4e2f32aff Return candidate not found when appropriate 2020-12-19 14:57:42 +01:00
Ettore Di Giacinto
7cf650a8f6 Break Swap in computeSwap() and display uninstall dialog only when asked 2020-12-19 14:55:59 +01:00
Ettore Di Giacinto
1c9b821058 Drop unneeded if 2020-12-18 00:50:20 +01:00
Ettore Di Giacinto
0e21548bc0 Lookup uninstall and Install in installer.Swap
In this way we resolve selectors from user inputs
2020-12-18 00:49:51 +01:00
Ettore Di Giacinto
d97e606a31 Adapt fixtures and tests 2020-12-12 12:10:24 +01:00
Ettore Di Giacinto
9eef7e5c6d Clean up if condition 2020-12-09 22:58:33 +01:00
Ettore Di Giacinto
332824fd42 Fail in the downloader goroutine and don't skip errors with force 2020-12-09 22:56:55 +01:00
Ettore Di Giacinto
737fbdbdc1 Don't make artifact checksum skippable 2020-12-09 21:31:07 +01:00
Ettore Di Giacinto
59d78c3f5c While upgrading always use nodeps while computing uninstall 2020-12-07 17:20:55 +01:00
Ettore Di Giacinto
d479ada402 Don't consider deps while uninstalling during package Swap
Beside being forced, it also doesn't need to look deep into the deps, as
we already have precalculated those
2020-12-06 22:48:48 +01:00
Ettore Di Giacinto
7b800c9a20 Pre-compute swap step
Otherwise, while upgrading, it could happen that package dependencies
aren't downloaded before, and they would just be installed in the middle
of installation, after removal already happened.
2020-12-06 22:11:17 +01:00
Ettore Di Giacinto
67c43eb936 Don't bail out if package is installed and we have a list 2020-12-03 20:03:37 +01:00
Ettore Di Giacinto
cf80e5fc09 Resolvers might omit packages 2020-12-03 18:53:57 +01:00
Ettore Di Giacinto
d668d8344b Accept selectors on uninstall and fixup failure logic 2020-12-03 18:32:24 +01:00
Ettore Di Giacinto
b17ac447f1 Display matched packages only, and check if they are available 2020-12-03 17:25:29 +01:00
Ettore Di Giacinto
9423b7c1e3 Add image build events, and add luet replace
Enhance also some commands descriptions
2020-12-02 18:24:35 +01:00
Ettore Di Giacinto
0fe30ddcfd Add ability to interpolate during build
Now build takes a --values argument, which is a yaml file that can be
used to interpolate the specs that are going to be compiled.
2020-11-28 15:47:29 +01:00
Ettore Di Giacinto
18d9366bca Minor fixes 2020-11-24 18:27:49 +01:00
Ettore Di Giacinto
db1b190fb5 Minor fixup and cleanups around the new prompt feature 2020-11-23 18:20:30 +01:00
Ettore Di Giacinto
b349665ff2 Add user prompts
Fixes #106
2020-11-22 23:43:29 +01:00
Ettore Di Giacinto
53ab0e0dd2
Merge pull request #151 from mudler/download-progress-bar
Download progress bar
2020-11-20 19:00:25 +01:00
Daniele Rondina
60d5c9dfd5 Add download progress bar 2020-11-20 18:12:23 +01:00
Ettore Di Giacinto
f2df3faee5 Now Uninstall takes multiple packages 2020-11-19 18:05:27 +01:00
Daniele Rondina
f9a7113ab9 client/http: Add experimental download info 2020-11-19 00:56:28 +01:00
Ettore Di Giacinto
0e46e763d5 Move bus implementation to a separate repo, hook to events in luet 2020-11-13 18:25:44 +01:00
Ettore Di Giacinto
a765147c1d Add templated finalizers 2020-11-08 21:14:19 +01:00
Ettore Di Giacinto
cead09fb9f
Merge pull request #148 from mudler/respect_rootfs4conf
Respect rootfs path for configs and url
2020-11-08 18:25:29 +01:00
Daniele Rondina
9a1787ddaf client/local: Handle config_from_host on DownloadFile 2020-11-08 17:06:05 +01:00
Ettore Di Giacinto
6a1b64acea Order files before uninstall
Fixes #149
2020-11-08 12:36:41 +01:00
Ettore Di Giacinto
459eb01a59
Don't write err to stdout if not present 2020-11-08 10:02:00 +01:00
Daniele Rondina
e70cdbaaf7 Respect rootfs on repositories urls 2020-11-08 00:00:15 +01:00
Daniele Rondina
513f441bb3 Add option config_from_host 2020-11-07 18:56:25 +01:00
Daniele Rondina
ebe7466fdc Respect rootfs path for load config 2020-11-07 18:28:23 +01:00
Ettore Di Giacinto
46ed6423ad
Merge pull request #147 from mudler/fix-protect-uninstall
Fix protect uninstall
2020-11-07 12:28:24 +01:00
Daniele Rondina
d5df40512b installer: Improve message for protected files 2020-11-07 12:27:18 +01:00
Ettore Di Giacinto
99bf9e291d Use LStat and attempt removing before bailing out on first failure 2020-11-06 21:34:56 +01:00
Daniele Rondina
130eb8de1a Integrate config protection on uninstall too 2020-11-06 20:14:25 +01:00
Ettore Di Giacinto
c3ef549673 Warn user only when required when uninstalling directories 2020-10-31 11:56:03 +01:00
Ettore Di Giacinto
0e764e525e Filter packages to install instead of looping solver result 2020-10-31 01:25:18 +01:00
Ettore Di Giacinto
91dfb8ce3a Enhance CLI output 2020-10-30 19:15:04 +01:00
Ettore Di Giacinto
eb2c240e84 Adapt installer tests 2020-10-29 16:32:13 +01:00
Ettore Di Giacinto
95e640c9d0 Make solver type switchable 2020-10-29 16:32:13 +01:00
Daniele Rondina
6806103b3e installer: Start spinner of upgrade calculation 2020-10-25 14:01:45 +01:00
Daniele Rondina
07dff7f197 installer: log packages ignored on create-repo 2020-10-12 08:42:19 +02:00
Ettore Di Giacinto
5dcf77c987
Add package buildtimestamp and luet upgrade --sync
Annotate the package build time when compiling, and use that from the
client to force upgrade of packages that changed the artifact, but
didn't changed any version.

The client can trigger this behavior with `luet upgrade --sync`
2020-07-12 15:29:38 +02:00
Daniele Rondina
8f0c528c08 Create helpers.UntarProtect for handle protected files
Currently, it's used the archive.ReplaceFileTarWrapper
that requite a []byte of the files replaced. This is not
a good idea if files are big and instead could be better
in the near future reimplement ReplaceFileTarWrapper with
a callback that return io.Reader instead of []byte.

If a protected file is already present on target rootfs
it is created a file with the same prefix used in Gentoo:

._cfgXXXX_<filename>
2020-06-02 11:08:37 +02:00
Ettore Di Giacinto
cc21e6fa5e
Respect user-defined repository naming 2020-05-24 12:16:02 +02:00
Ettore Di Giacinto
4197d7af61
Add upgrade by using only the SAT core
- Adds upgrade --universe and upgrade --universe --clean. It will
  attempt to bring the system as much close as the content available in
  the repositories. It differs from a standard upgrade which checks
  directly that what is pulled in doesn't conflict with the system. In
  this new way, we just query the SAT solver to decide that on our
  behalf.
- Add uninstall --full-clean. It uses only the SAT solver to uninstall
  the package and it will drop as many packages as required (including
  revdeps of packages too.
2020-05-22 21:20:58 +02:00
Ettore Di Giacinto
bfde9afc7f
Add Nodeps and Full options to upgrade 2020-05-22 21:20:57 +02:00
Ettore Di Giacinto
3237423dde
Enhance upgrade output 2020-05-22 21:20:57 +02:00
Daniele Rondina
a7d1381cb5 cmd/create-repo: Add support for multiple trees 2020-05-10 20:18:10 +02:00
Daniele Rondina
13df161fc6 logging: permit to disable color and emoji
Now it's possible disable color and emoji on standard
output with:

$> luet <cmd> --color=false --emoji=false

Hereinafter, the list of changes:

* Added logging option logging.color (default true)
* Added logging option logging.enable_emoji (default true)
* Added persistent flag --color
* Added persistent flag --emoji
2020-05-09 10:08:21 +02:00
Ettore Di Giacinto
46014fb9c1
Enhance install output 2020-05-03 13:22:06 +02:00
Ettore Di Giacinto
ada9d886fa
Enhance uninstall output 2020-05-03 13:22:00 +02:00
Ettore Di Giacinto
a1d8ef1422
Allow to partially uninstall a package graph, make uninstall --full optional 2020-05-03 13:04:34 +02:00
Ettore Di Giacinto
3befbfa915
Optimize uninstall computation 2020-05-02 15:43:57 +02:00
Ettore Di Giacinto
d4c25d74f5
Remove focus from test 2020-05-02 13:27:15 +02:00
Ettore Di Giacinto
2ed9781c88
Don't try to match against repo already installed packages 2020-05-02 13:24:46 +02:00
Ettore Di Giacinto
9d6d6bc0c8
Add more upgrade and reclaim tests scenarios 2020-05-02 12:18:19 +02:00
Ettore Di Giacinto
f8b2837741
Enhance reclaim output 2020-05-02 12:17:54 +02:00
geaaru
878e6d7b9c
Merge pull request #103 from mudler/tmpdir-cleanup
Tmpdir cleanup
2020-05-02 09:20:01 +02:00
Ettore Di Giacinto
4b63c9eaf9
Add test to be sure we don't index packages not in the tree 2020-05-01 11:48:05 +02:00
Ettore Di Giacinto
286d0fba2c
Fix typo 2020-05-01 11:07:14 +02:00
Ettore Di Giacinto
624518bf77
Don't add package to the repository which aren't referenced by the tree 2020-05-01 10:52:40 +02:00
Daniele Rondina
11944873ea Integrate tmpdir_base params and tmpdirs cleanup 2020-04-30 20:29:28 +02:00