kairos/luet
1
0
Fork 0
mirror of https://github.com/mudler/luet.git synced 2025-07-31 06:50:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,083 Commits 12 Branches 164 Tags 38 MiB
0028dd3a92
Commit Graph

651 Commits

Author SHA1 Message Date
Daniele Rondina
c8c53644f3 Avoid segfault if tree path doesn't exist 2020-09-17 08:05:05 +02:00
Daniele Rondina
045d25bb28 pkg/package: Add method SetVersion to DefaultPackage 2020-08-30 08:53:37 +02:00
Daniele Rondina
93187182e5 pkg/compiler: Fix typo on error message 2020-08-19 19:24:46 +02:00
Ettore Di Giacinto
a076613f66
Fixup import path 2020-08-07 19:30:08 +02:00
Ettore Di Giacinto
95b125cb91
Pull images before executing diff tests 2020-08-06 18:03:00 +02:00
Ettore Di Giacinto
66513955c7
Compute image diffs internally 
Is it more faster in this way as we already have all the needed folders
to the comparison extracted. In this way we don't repeat I/O operation
twice by calling container-diff.

Do not depend on container-diff anymore
 2020-08-05 19:09:45 +02:00
Ettore Di Giacinto
e30bb056d5
Drop IsFlagged from tests 2020-08-02 12:22:43 +02:00
Ettore Di Giacinto
052a551c0c
Add "hidden" field to packages 
Also drop residual of IsSet which isn't actually used

Related to #26
 2020-08-02 11:31:23 +02:00
Ettore Di Giacinto
07a1058ac1
Add cli option to skip packages if only metadata is present (without checking the image) 2020-07-17 22:42:03 +02:00
Ettore Di Giacinto
5dcf77c987
Add package buildtimestamp and luet upgrade --sync 
Annotate the package build time when compiling, and use that from the
client to force upgrade of packages that changed the artifact, but
didn't changed any version.

The client can trigger this behavior with `luet upgrade --sync`
 2020-07-12 15:29:38 +02:00
Daniele Rondina
364b5648b4 repository loader now support .yaml extension 2020-07-04 20:07:32 +02:00
Daniele Rondina
e2260b6956 Add --no-spinner option 2020-06-27 16:45:49 +02:00
Ettore Di Giacinto
16e9d7b20c
Use packageImage as builder image fingerprint 
This allows to have an unique identifier for the builder image id against
different depgraphs combinations. The package fingerprint is not enough,
as an atom could have a difference deptree depending on the requires
constraints.

TODO: Don't use the full image name, but only the hash as a salt
(currently the salt contains ALSO a reference of the image-repository,
as such it doesn't allow to port a tree in a different docker registry)
 2020-06-23 18:59:18 +02:00
Ettore Di Giacinto
7b51e83902
Merge pull request #119 from mudler/config-protect 
Integrate feature config-protect like Gentoo/Funtoo do
 2020-06-07 11:33:55 +02:00
Daniele Rondina
3a365c709b Integrate config-protect from package spec 2020-06-06 13:12:54 +02:00
Daniele Rondina
0917c2703e helpers/archive: Refactor and add support for config protect when sameOwner is false 2020-06-06 12:34:44 +02:00
Ettore Di Giacinto
264e1e9652
Print inspect output as string 2020-06-06 11:32:47 +02:00
Daniele Rondina
c87db16d31 tarModifierWrapperFunc: Now use strings.HasPrefix for match path 2020-06-06 10:38:11 +02:00
Daniele Rondina
837eeb04ec config/config_protect: Fix load of configuration files 2020-06-06 10:35:38 +02:00
Ettore Di Giacinto
90a25406a0
Add --full option to build 
Don't compile dependencies when computing all the compilation specs from
a tree if are among the target deps

Fixes #41
 2020-06-06 08:58:18 +02:00
Daniele Rondina
d946e39a15 pkg/helpers/archive_test: Fix test 2020-06-05 23:53:33 +02:00
Daniele Rondina
415b1dab9a Add test of modifier 2020-06-05 23:12:35 +02:00
Daniele Rondina
16f717f04b tarModifierWrapperFunc: Fix config protect filename 2020-06-05 23:12:19 +02:00
Ettore Di Giacinto
9e0e1199df
Check if image exists before skipping compilation 2020-06-03 21:00:30 +02:00
Daniele Rondina
8f0c528c08 Create helpers.UntarProtect for handle protected files 
Currently, it's used the archive.ReplaceFileTarWrapper
that requite a []byte of the files replaced. This is not
a good idea if files are big and instead could be better
in the near future reimplement ReplaceFileTarWrapper with
a callback that return io.Reader instead of []byte.

If a protected file is already present on target rootfs
it is created a file with the same prefix used in Gentoo:

._cfgXXXX_<filename>
 2020-06-02 11:08:37 +02:00
Daniele Rondina
a2231749ab Begin implementation for catch config protect files 2020-06-02 09:04:40 +02:00
Ettore Di Giacinto
990a5405cf
Merge pull request #116 from mudler/review-config-path 
Review configuration file parsing logic
 2020-05-31 11:05:55 +02:00
Daniele Rondina
1d8a6174bb Drop duplicate code for general.same_owner 2020-05-30 16:51:10 +02:00
Daniele Rondina
9e7c7e69f8 Review configuration file parsing logic 
Luet support now these priorities on read configuration file:
- command line option (if available)
- $PWD/.luet.yaml
- $HOME/.luet.yaml
- /etc/luet/luet.yaml
 2020-05-30 16:46:29 +02:00
Ettore Di Giacinto
cc21e6fa5e
Respect user-defined repository naming 2020-05-24 12:16:02 +02:00
geaaru
8c4f5b2911
Merge pull request #113 from mudler/annotations 
Add Annotations to package spec
 2020-05-23 10:52:32 +02:00
Daniele Rondina
44d68a9583 Add annotations option to package spec 2020-05-23 09:27:38 +02:00
Daniele Rondina
dba6c361c2 Move helpers/cli to cmd/helpers 2020-05-23 08:51:33 +02:00
Ettore Di Giacinto
4197d7af61
Add upgrade by using only the SAT core 
- Adds upgrade --universe and upgrade --universe --clean. It will
  attempt to bring the system as much close as the content available in
  the repositories. It differs from a standard upgrade which checks
  directly that what is pulled in doesn't conflict with the system. In
  this new way, we just query the SAT solver to decide that on our
  behalf.
- Add uninstall --full-clean. It uses only the SAT solver to uninstall
  the package and it will drop as many packages as required (including
  revdeps of packages too.
 2020-05-22 21:20:58 +02:00
Ettore Di Giacinto
bfde9afc7f
Add Nodeps and Full options to upgrade 2020-05-22 21:20:57 +02:00
Ettore Di Giacinto
3237423dde
Enhance upgrade output 2020-05-22 21:20:57 +02:00
Ettore Di Giacinto
ab179db96a
Don't drop packages that would be re-installed during upgrade 
Check for packages that are marked for deletion. If the ones that are
marked for install are depending on it, don't remove them at all
 2020-05-22 21:20:52 +02:00
geaaru
916b2a8927
Merge pull request #112 from mudler/package-sanitized 
Package sanitized
 2020-05-20 11:42:47 +02:00
Daniele Rondina
a16bdddeb2 Add spectooling test suite 2020-05-20 10:26:30 +02:00
Daniele Rondina
e38a4b3d9b Use DefaultPackageSanitized struct for write specs 2020-05-20 09:59:48 +02:00
Ettore Di Giacinto
9971fe9f45
Unique hashes for packages without deps 2020-05-18 19:57:01 +02:00
Daniele Rondina
11759f98e0 package/NewPackage: init Labels on AddLabel 2020-05-18 19:51:47 +02:00
Daniele Rondina
cb2ac15de8 package: Fix typo on labels key 2020-05-17 19:17:14 +02:00
Ettore Di Giacinto
51711dafba
Add package_dir to pack a spec dir as the main artifact 2020-05-16 21:34:27 +02:00
Ettore Di Giacinto
2803430515
Add solver test scenario 2020-05-16 11:11:17 +02:00
Daniele Rondina
a7d1381cb5 cmd/create-repo: Add support for multiple trees 2020-05-10 20:18:10 +02:00
Daniele Rondina
13df161fc6 logging: permit to disable color and emoji 
Now it's possible disable color and emoji on standard
output with:

$> luet <cmd> --color=false --emoji=false

Hereinafter, the list of changes:

* Added logging option logging.color (default true)
* Added logging option logging.enable_emoji (default true)
* Added persistent flag --color
* Added persistent flag --emoji
 2020-05-09 10:08:21 +02:00
Daniele Rondina
fe5ab9246f Added option for enable log to file. 
Now it's possible logging to file is handled by the enable_logfile
option and by the path.

From cli is now possible:
* enable log to file with the option --enable-logfile
* modify the logfile path with the option --logfile/-l <path>
 2020-05-09 10:05:34 +02:00
Ettore Di Giacinto
20cb96e0cc
Simplify ordering check 
we don't need a map[string]map[string]interface{}, as we don't need to
keep the data around
 2020-05-04 17:34:29 +02:00
Daniele Rondina
b68634b58a solver: skip same packages in the order and avoid loop 2020-05-04 12:21:49 +02:00
Ettore Di Giacinto
46014fb9c1
Enhance install output 2020-05-03 13:22:06 +02:00
Ettore Di Giacinto
ada9d886fa
Enhance uninstall output 2020-05-03 13:22:00 +02:00
Ettore Di Giacinto
a1d8ef1422
Allow to partially uninstall a package graph, make uninstall --full optional 2020-05-03 13:04:34 +02:00
Ettore Di Giacinto
7b6e4a2176
Add to the Solver the capability to check conflicts with revdeps 2020-05-03 10:34:18 +02:00
Ettore Di Giacinto
3befbfa915
Optimize uninstall computation 2020-05-02 15:43:57 +02:00
Ettore Di Giacinto
d4c25d74f5
Remove focus from test 2020-05-02 13:27:15 +02:00
Ettore Di Giacinto
2ed9781c88
Don't try to match against repo already installed packages 2020-05-02 13:24:46 +02:00
Ettore Di Giacinto
9d6d6bc0c8
Add more upgrade and reclaim tests scenarios 2020-05-02 12:18:19 +02:00
Ettore Di Giacinto
f8b2837741
Enhance reclaim output 2020-05-02 12:17:54 +02:00
geaaru
878e6d7b9c
Merge pull request #103 from mudler/tmpdir-cleanup 
Tmpdir cleanup
 2020-05-02 09:20:01 +02:00
Daniele Rondina
a89c0af2f8 config/config_test: Fix typo 2020-05-01 15:34:17 +02:00
Daniele Rondina
1f99fde1c5 Add config test suite 2020-05-01 15:26:53 +02:00
Ettore Di Giacinto
4b63c9eaf9
Add test to be sure we don't index packages not in the tree 2020-05-01 11:48:05 +02:00
Ettore Di Giacinto
286d0fba2c
Fix typo 2020-05-01 11:07:14 +02:00
Ettore Di Giacinto
624518bf77
Don't add package to the repository which aren't referenced by the tree 2020-05-01 10:52:40 +02:00
Daniele Rondina
51a4037b1b config: Initialize luet tmp basedir if doesn't exist 2020-05-01 08:18:18 +02:00
Ettore Di Giacinto
9cb290b484
Improve database errors 2020-04-30 20:44:34 +02:00
Daniele Rondina
11944873ea Integrate tmpdir_base params and tmpdirs cleanup 2020-04-30 20:29:28 +02:00
Ettore Di Giacinto
322ac99f17
Annotate package runtime definition when reclaiming 
This was an issue as we were copying the buildspec instead
 2020-04-30 18:56:50 +02:00
Ettore Di Giacinto
6acc5fc97e
Make BuildFormula support dependency cycles 2020-04-30 18:56:35 +02:00
Ettore Di Giacinto
6a4557a3b3
Make RequireContains support dependency cycles 2020-04-30 18:56:06 +02:00
Ettore Di Giacinto
a457c53824
Fixups to ExpandedRevdeps 2020-04-24 00:15:18 +02:00
Ettore Di Giacinto
8305d01e76
Walk requires in ExpandedRevdeps 
In this way we annotate the visited and we avoid cycles that can be
generated by revdeps
 2020-04-23 23:44:42 +02:00
Ettore Di Giacinto
9321a310a5
Support mount with target folder in box exec 
Also mount bind before pivotroot, this fixes bind mounting in general
 2020-04-21 17:30:27 +02:00
Ettore Di Giacinto
4cee27da7f
Fixup box CLI params and consume new box options 2020-04-20 23:03:03 +02:00
Ettore Di Giacinto
98876c8f20
Add support for env and hostmounts in box 2020-04-20 23:01:49 +02:00
Ettore Di Giacinto
4c2d38be59
Add package ExpandedRevdeps 2020-04-19 11:55:13 +02:00
Ettore Di Giacinto
8e91c255a3
Make FindPackageMatch match packages HumanReadableString 2020-04-19 10:47:55 +02:00
Ettore Di Giacinto
ac871cb0a3
Add --output option to search 
In such way it can be parsed by scripts more easily.
It also disable the spinner based on loglevel

Fixes #92
 2020-04-18 23:22:00 +02:00
Ettore Di Giacinto
4d466d0330
Sync the repository data with user configured data 
Fixup a regression, in this way we respect the user configuration in
regards to a repository to be consumed (priority and repository type)
regardless of the one advertized remotely.

Also add informative message about prio and type.
 2020-04-18 17:33:46 +02:00
Ettore Di Giacinto
84407e5ae7
Add SetPriority to repository 2020-04-18 17:33:34 +02:00
Ettore Di Giacinto
64bac0823c
Consume our moby fork 
It handles #91
 2020-04-18 11:41:34 +02:00
Ettore Di Giacinto
ee0fe1a86a
Allow to finalizer to specify entrypoint command 
In this way, finalizer in strict environment can override the default
shell used to run commands.

The shell keyword is a list, as it needs to contain the full command +
args.
 2020-04-14 17:46:39 +02:00
Ettore Di Giacinto
538ba8f5df
Add luet reclaim 
Reclaim allows to migrate between different system layouts. This
is a experimental feature (yet) and might be revisited in the future.

This change:

- Adds Reclaim(system) to Installer
- Adds unit tests

Relates to #86
 2020-04-13 19:32:51 +02:00
Ettore Di Giacinto
eb56956c65
Add Touch file helper 2020-04-13 19:32:50 +02:00
Ettore Di Giacinto
4e2d42e397
Drop downloadOnly bool from installer.Install 2020-04-13 19:32:50 +02:00
Ettore Di Giacinto
528f481a7b
Annotate package files in metadata 
Fixes #87
 2020-04-13 10:52:41 +02:00
Ettore Di Giacinto
f987ee9fa0
Handle errors from os/user 
We use it to generate defaults, if error is found, we imply preserve
permissions
 2020-04-11 20:05:56 +02:00
Daniele Rondina
69a82a1ca5 simpledocker: Use debug option for print container-diff results 2020-04-10 09:32:12 +02:00
Daniele Rondina
02c33896d5 simpledocker: Move warning before return 2020-04-10 09:31:08 +02:00
Daniele Rondina
c2e9176ab2 solver.Order now return error 2020-04-09 17:07:57 +02:00
Ettore Di Giacinto
0d2668e452
Print warning if container-diffs return errors to stderr 2020-04-08 18:31:40 +02:00
Ettore Di Giacinto
77ba4193aa
Add ValidateSelector to versioner interface and consume it 
We can refactor furthermore by dropping the package methods, as now we
can consume a versioner in all places that requires it
 2020-04-05 15:52:16 +02:00
Ettore Di Giacinto
5a5e7f1dfa
Move version logic inside versioner 
WIP, it needs yet to be under the interface implementation
 2020-04-05 11:16:14 +02:00
Ettore Di Giacinto
cf8091a7fd
Re-enable debian versioning test, add more cases 2020-04-04 16:49:41 +02:00
Ettore Di Giacinto
3caaa01eb8
Add semver detection to versioner 
In this way we compare only all version that are compliant to semver,
otherwise we fallback to debian sorting.
 2020-04-04 16:48:48 +02:00
Ettore Di Giacinto
4eae0a851c
Add versioner unit tests 2020-04-04 16:33:08 +02:00
Ettore Di Giacinto
626419f955
Fix original version mapping in versioner 2020-04-04 16:23:43 +02:00
Ettore Di Giacinto
84625be9ac
Adapt package.Best to take a Versioner interface 2020-04-04 15:33:14 +02:00
Ettore Di Giacinto
04f9a88a5f
Add versioner interface to wrap versioning operations 
This allows to scope responsabilities and enables switching to different
ordering algorithms
 2020-04-04 15:29:20 +02:00
Ettore Di Giacinto
9d58b0a0cc
Adapt tests to refactor 2020-04-04 14:41:58 +02:00
Ettore Di Giacinto
5e31d940f0
Introduce Packages for []Package 2020-04-04 14:29:08 +02:00
Daniele Rondina
129ca8da55 Add version sanitized to Best() 2020-04-03 18:32:26 +02:00
Daniele Rondina
ee66839bd3 pkg/package/version: Better search for last + 2020-03-29 09:10:52 +02:00
Ettore Di Giacinto
a0113dcd13
Drop verbose output 2020-03-28 17:05:23 +01:00
Ettore Di Giacinto
49d7b4e2bf
Add box structure to handle containerized executions 2020-03-28 16:58:46 +01:00
Ettore Di Giacinto
f15ed3fda1
Move finalizer in its own struct 
Also distinguish when we run in a root target dir ("/") or when we want
to run the commands in a separate folder
 2020-03-28 16:57:40 +01:00
Ettore Di Giacinto
2ad16fa875
Add ListDir helper 2020-03-28 16:55:16 +01:00
Ettore Di Giacinto
416be23a46
Don't always remove unpacked repository content 2020-03-28 12:07:40 +01:00
Daniele Rondina
50091b2a4b pkg/installer: Add test for uncompress tree and minor cleanup 2020-03-25 00:19:52 +01:00
Ettore Di Giacinto
117554792d
Don't attempt to upgrade packages already in system 2020-03-24 20:30:59 +01:00
Daniele Rondina
454e9d934e Use filename instead of name on repo specs 2020-03-24 19:40:11 +01:00
Ettore Di Giacinto
af7f1de9f1
Split download and installer worker 
Also, drop downloadOnly from install
 2020-03-24 18:19:09 +01:00
Daniele Rondina
a0f9222068 Add check of repository metafile archive in tests 2020-03-24 14:01:22 +01:00
Daniele Rondina
520768d0ca pkg/installer: Align tests with new default 2020-03-24 12:52:45 +01:00
Daniele Rondina
4f002ab40f pkg/installer/repository: The check for repo_files is not needed for parsing normal repo 2020-03-24 12:52:24 +01:00
Daniele Rondina
60635a03eb Fix propagation of repo file name and add test for meta compressed 2020-03-24 00:31:41 +01:00
Daniele Rondina
202ed2651a 💥 Refactor and split repository.yaml file 2020-03-24 00:05:16 +01:00
Daniele Rondina
6bd8fe6789 Add omitempty to DefaultPackage fields 2020-03-22 22:23:11 +01:00
Daniele Rondina
40830ecf42 installer/repository.go: Use switch with search options 2020-03-22 10:46:51 +01:00
Daniele Rondina
c7652c8a70 Added labels to package definition 
* cmd/search: Add support for search of the packages
  with a specific label.

* review Search method of Repositories for permit
  different search modes.

* labels are k/v attributes and could be matched
  through label key (with HasLabel method) or through
  regex that use "$key" + "=" + "$value"
 2020-03-21 19:24:27 +01:00
Ettore Di Giacinto
fab2d21ac1
Drop unused function 2020-03-19 22:39:22 +01:00
Ettore Di Giacinto
487239334f
Treat base case with same algorithm of dependencies 2020-03-19 22:37:39 +01:00
Ettore Di Giacinto
69477a0c36
Make uniform build of seeds and packages 2020-03-19 22:34:26 +01:00
Ettore Di Giacinto
67362a3bc1
Drop unused condition 2020-03-19 22:29:34 +01:00
Ettore Di Giacinto
302d18e749
Tag the target image with the computed hash 2020-03-19 22:28:33 +01:00
Daniele Rondina
0249c0fa4a version: Add temporary workaround for handle build string with _ 2020-03-16 22:18:48 +01:00
Daniele Rondina
fc40c770ab Add 'tree bump' command 2020-03-16 22:18:48 +01:00
Ettore Di Giacinto
4357ee45e9
Drop unused functions in package 2020-03-16 18:08:46 +01:00
Ettore Di Giacinto
6f138811dd Make recipes idempotent, allowing to load multiple trees 2020-03-15 13:31:12 +01:00
Daniele Rondina
bc5c0fa0cf cmd/tree/pkglist: Cleanup code 2020-03-14 11:14:14 +01:00
Ettore Di Giacinto
dc64dbff75
Don't try to solve finalizers with --nodeps 2020-03-13 19:32:00 +01:00
Daniele Rondina
93f8f0dd0f solver: Add category to tests 2020-03-12 00:26:53 +01:00
Daniele Rondina
f4a5a97cff tree/compiler_recipe: Return right error message 2020-03-08 16:28:12 +01:00
Daniele Rondina
8c972403a3 tree/compiler_recipe: Handle invalid tree path 
If instead of abs path is used bash annotation
now return an error related to wrong directory
supply.
 2020-03-08 15:47:40 +01:00
Daniele Rondina
17238d187a Change 'Skip dir' message to debug 2020-03-08 00:07:20 +01:00
Ettore Di Giacinto
765261f233
Add Swap() to Installer. Refactor code in Upgrade() to consume it 2020-02-27 23:48:14 +01:00
Ettore Di Giacinto
6c7e24fadf
Sync repositories once (and at beginning) during upgrade 2020-02-27 23:25:29 +01:00
Ettore Di Giacinto
de23e0d5b1
Download dependencies before installing 
This enables smoother upgrades, and uses cache to pre-download
packages before uninstall/install

Allows also from cli to download deps only
 2020-02-27 23:15:31 +01:00
Ettore Di Giacinto
8572aa5222
Preserve cache data from deletion during uninstall 2020-02-27 23:14:36 +01:00
Ettore Di Giacinto
5bcc8d112a
Enforce solver constraints 
- Don't sign installed packages during finalizer execution
- Enforce solver constraints: build ALO and AMO rules taking into account
  that the current package might not be selected at all.
- Force uninstalls on upgrade
- Enable option to tell uninstall to ignore conflict with the analized system state,
  as we don't want any conflict with the installed to raise during the upgrade.
  In this way we both force uninstalls and we avoid to check with conflicts
  against the current system state which is pending to deletion.
  This is due to the fact that now the solver enforces the constraints
  and explictly denies two packages of the same version installed.
- Adapt test as now we generate more constraints, which makes the solver more
  noisy on the package that are explictly selected or not
 2020-02-27 18:38:31 +01:00
Ettore Di Giacinto
a0618107a8
Adapt tests 2020-02-26 23:56:21 +01:00
Ettore Di Giacinto
25f2abf103
Do conflict with same package versions while building formula 2020-02-26 23:01:08 +01:00
Ettore Di Giacinto
e3ebfd6bfe
Respect ImageRepository with build --nodeps 2020-02-26 19:00:22 +01:00
Ettore Di Giacinto
07610bc216
Add build option to remove backend artifacts 
Add keep-exported-images to build to instruct to remove backend artifacts (docker images)
 2020-02-21 22:33:18 +01:00
Daniele Rondina
a04dadf100 Add support for parsing version with build 2020-02-21 21:56:32 +01:00
Daniele Rondina
def04724d4 Add support for build identifier on version 2020-02-20 17:46:47 +01:00
Ettore Di Giacinto
38296bc5d7
Add test for HashFingerprint() 2020-02-19 18:33:14 +01:00
Ettore Di Giacinto
7e16e0cdb6
Add HashFingerprint() to Package and consume it while generating image names 
In this way we don't depend on the package fingerpint characters while generating images, allowing
to support completely semver (e.g. + character is not allowed in docker images tag).

This change is not backward compatible with images currently already built and will invalidate the cache
 2020-02-19 18:28:29 +01:00
Ettore Di Giacinto
647ea35983
Add a way to extend the CLI with external binaries 
This lets luet to be extended like git does (just by having luet-subcommand binaries under $PATH)
following the UNIX filosophy. It has the downside that we silence the errors from cobra and we handle
them by ourselves.

Also the Exec syscall is not portable, and it should have implementation for each platform (hence why in helpers)
 2020-02-18 23:08:14 +01:00
Ettore Di Giacinto
4648dedb55
Add --nodeps, --onlydeps and --force to Installer 
This allows also to use it on upgrade and uninstall as it exposes the Installer options to the CLI.

See #48, #49
 2020-02-18 18:37:56 +01:00
Ettore Di Giacinto
2ce427d601
Add Nodeps, Onlydeps and Force options to Compiler 
See #41
 2020-02-18 18:36:44 +01:00
Ettore Di Giacinto
b01c017507
Add HumanReadableString() to Package 2020-02-18 18:20:45 +01:00
Ettore Di Giacinto
dd00d491b9
Skip repository if no candidate is found 
FindPackageCandidate is idempotent and returns the same definition if no new is found.
This prevents installs from multiple-repos
 2020-02-17 17:21:31 +01:00
Ettore Di Giacinto
26b94888c3
Skip image building if we pulled them successfully 2020-02-15 22:53:00 +01:00
Ettore Di Giacinto
fcec6c5699
Add Push to hub feature 
Add to the backends the Push capabilities to push images that were correctly built.
Also expose them to the CLI, along with KeepImg which was hidden.
 2020-02-15 14:45:05 +01:00
Ettore Di Giacinto
c7253ac8ad
Adapt test to spec changes 2020-02-14 08:04:26 +01:00
Ettore Di Giacinto
f329e1d5e0
Add retrieve to compilation spec 
It allows to copy build artifact during buildtime. In this way
artifacts between different seed images can be shared

```
retrieve:
- a-test-1.0.package.*
- https://...
```

They will be available under WORKDIR
 2020-02-13 14:15:43 +01:00
Ettore Di Giacinto
d6f7c47eae
Add compact string notation for SolverOptions 2020-02-12 12:23:37 +01:00
Ettore Di Giacinto
b2a5de9222
Adapt installer test to constructor changes 2020-02-12 12:05:28 +01:00
Ettore Di Giacinto
7e388c6fed
Cleanup resolver from logger 
It creates cycle and we don't want to output anything from the computation process.
We should handle output in different stages

Also create constructor for solver to be able to consume resolvers.
 2020-02-12 11:22:56 +01:00
Ettore Di Giacinto
07a154474b
Consume SolverOptions in compiler 2020-02-12 11:21:55 +01:00
Ettore Di Giacinto
6aa353edb2
Consume SolverOptions in installer 2020-02-12 11:21:30 +01:00
Ettore Di Giacinto
8951203165
Add LuetSolverOptions to tweak solver parameter in Config 2020-02-12 11:21:08 +01:00
Ettore Di Giacinto
d330fedcc4
fmt 2020-02-12 11:05:13 +01:00
Ettore Di Giacinto
dfb6dab9dc
Move repository helpers under config 
They are generated after the system config, let the structure provide such information
 2020-02-12 10:20:07 +01:00
Ettore Di Giacinto
4f33eca263
Consume internal attempts in QLearn so resolver can be re-used 2020-02-12 09:22:10 +01:00
Ettore Di Giacinto
54b0dce54b
Respect argument passed to Untar in helpers 2020-02-11 16:55:00 +01:00
Ettore Di Giacinto
ea2a60a853
Cleanup, drop hardcoded values and use constructors 2020-02-11 15:58:28 +01:00
Ettore Di Giacinto
c8f4ba0a47
Add Factorial helper 2020-02-11 15:00:28 +01:00
Ettore Di Giacinto
7e0ea34b81
Switch back to gophersat 2020-02-11 14:58:17 +01:00
Ettore Di Giacinto
711c039296
Reward by observedDelta 
Keep a record of the observed delta and maximize reward for it.
Also add Noop actions which is turned off by default.

Let finish the execution also when no solution is found, as we will take the
minimum observed delta as result.

This is done on purpose to avoid guessing "when" is a good time to stop the agent,
as it could be in the middle of picking up a new action which is not the final
(but we need limits, we can't let it run forever).
 2020-02-11 14:52:24 +01:00
Ettore Di Giacinto
ac6554c291
Support Add/Removal actions 
Also keep the list of the wanted targets as we walk it. We will let the agent to try different solutions
 2020-02-11 09:51:52 +01:00
Ettore Di Giacinto
d4255b086b
Uncomment failing test 
Add test to support that normally we need to pass explictly the installable ones only.
 2020-02-11 09:50:26 +01:00
Ettore Di Giacinto
1b90407475
Don't preclude action space 2020-02-11 09:21:25 +01:00
Ettore Di Giacinto
6f6e2bf15f
Pin to gophersat version 
Having the same var in the and block seems to make gophersat crash. Even if might be unoptimal,
we need this to tighten the conditions between packages.

Switch to gophersat fork until this fix is merged upstream:

https://github.com/crillab/gophersat/pull/17
 2020-02-11 09:06:57 +01:00
Ettore Di Giacinto
6d450d3af0
Add simple QLearning solver support 
Still experimental, it covers very small subset of action domains (just removal from target).

Added a pending test that currently fails
 2020-02-10 17:18:18 +01:00
Ettore Di Giacinto
33b442a832
Add accessor to decode from package String() 2020-02-10 17:14:46 +01:00
Ettore Di Giacinto
f068bfdb9b
Add PackageResolver to add heuristics on unsat solutions 2020-02-10 09:41:09 +01:00
Ettore Di Giacinto
4dc4205868
Drop json skip from id. It prevents storm to track entities #55 2020-02-08 18:18:05 +01:00
Ettore Di Giacinto
f813370501
Resolve selectors if present before starting install 2020-02-04 20:15:59 +01:00
Daniele Rondina
c353ab4978 client/http: Add support for authentication Basic/Token 2020-02-03 00:58:55 +01:00
Daniele Rondina
298de447b8 Fix installer/client test suite 2020-02-01 21:18:32 +01:00
Daniele Rondina
40687c3072 Use dynamic pkgs cache dir for test suite 2020-02-01 20:05:19 +01:00
Daniele Rondina
78b5963a4f Add support for local packages cache 2020-02-01 19:35:30 +01:00
Daniele Rondina
524bbf990e Add support for same-owner config option 2020-02-01 19:01:15 +01:00
Ettore Di Giacinto
0e30e6a1ad
Treat single conflict cases as well when building conflict formula 2020-01-31 20:50:55 +01:00
Ettore Di Giacinto
0147b2cf99
Don't cleanup the whole cache download dir, just the downloaded artifact 
Otherwise whatever we download cannot be consumed (e.g. tree).
Instead whoever is calling the client should care to cleanup the returned artifact
 2020-01-30 18:03:25 +01:00
Ettore Di Giacinto
c6fe34b059
Support compression and checksum for trees 
Fixes #33 #34 #35
 2020-01-28 17:46:32 +01:00
Daniele Rondina
c9b684523f installer: Add support for cached repository 2020-01-12 23:35:58 +01:00
Daniele Rondina
aeea0cc5fe Add Cached option to LuetRepository 2020-01-12 23:33:19 +01:00
Daniele Rondina
091e51e426 logger: Add support for custom color text 2020-01-12 23:33:19 +01:00
Ettore Di Giacinto
22bc53ba13
Merge branch 'master' into develop 2020-01-06 19:14:17 +01:00
Ettore Di Giacinto
f5e7c2ad92
Allow more matches, we select the best one anyway 2020-01-06 11:33:35 +01:00
Daniele Rondina
07633dc307 Add container-diff summary 2020-01-05 18:05:56 +01:00
Ettore Di Giacinto
d5fd14bceb
Merge branch 'build_clean' into develop 2020-01-05 16:27:31 +01:00
Ettore Di Giacinto
d2abaa9cc1
Add message asserting skip. Also return pointer to allow to edit Options 2020-01-05 16:26:42 +01:00
Ettore Di Giacinto
d23e1dee78
Default to Clean true in CompilerOptions 2020-01-05 16:08:40 +01:00
Ettore Di Giacinto
ee055e08b1
Set abs path when returning artifact from yaml 2020-01-05 16:08:39 +01:00
Ettore Di Giacinto
02c37c7451
Fix tests after contructor change 2020-01-05 16:08:39 +01:00
Ettore Di Giacinto
1d1efad18b
Skip building if artifact already exists 2020-01-05 15:30:16 +01:00
Ettore Di Giacinto
bcc6ce19ea
Move compiler options to its own struct 
Also add Clean attribute, to handle a future build clean method
 2020-01-05 14:32:26 +01:00
Daniele Rondina
9b6f4a094d Use const for repository.yaml and tree.tar 2020-01-05 00:40:28 +01:00
Daniele Rondina
e751b989e0 Fix installer_tests 2020-01-04 16:15:56 +01:00
Daniele Rondina
6012e0081e Add support of incremental revision for repos 
* on repository creation now if the repository.yaml
  is already present, the current revision is used.

* add --reset-revision option for force reset revision
  of a specific repository
 2020-01-04 00:31:11 +01:00
Daniele Rondina
d3bd78d618 Repository now support Revision and LastUpdate 2020-01-03 20:09:29 +01:00
Daniele Rondina
6af62b5851 Define default values for loaded repositories 2020-01-03 19:37:51 +01:00
Daniele Rondina
7ec36da059 Add --fatal option 
Close #8
 2020-01-03 15:41:45 +01:00
Daniele Rondina
c284d3e4bf Add support for logging to file 2020-01-03 15:22:55 +01:00
Daniele Rondina
f28e8deb96 logger: Add support for json format 2020-01-03 15:20:42 +01:00
Daniele Rondina
716d404307 Align logic of cache repositories in all commands 2020-01-02 18:31:25 +01:00
Daniele Rondina
b12410edb7 Errors on create database paths are now fatal 2020-01-01 22:58:53 +01:00
Daniele Rondina
db7301f7bf Align selector logic for explicit version with PkgSelectorConditionFromInt 2020-01-01 22:58:19 +01:00
Daniele Rondina
ebcf6075d0 database_mem: Add test for specific candidate 2020-01-01 22:55:19 +01:00
Daniele Rondina
ce888a2f40
Fix marshal of repository and use of cache repos 2020-01-01 13:44:39 +01:00
Daniele Rondina
01e66ee0b4
Review install phase 2020-01-01 13:44:39 +01:00
Daniele Rondina
a71e1a6f1d
Integrate config.LuetRepository with installer 
* installer.LuetRepository is now installer.LuetSystemRepository,
  a struct that extend config.LuetRepository

* config: system_repositories option is now "repositories".
  This resolve viper issue.

* config: cache_repositories option is now "repetitors"
  This resolve viper issue.

* cmd/*: Now use new config.LuetRepository

* cmd/search: now create local luet repository if database_engine
  is equal to "boltdb"
 2020-01-01 13:43:23 +01:00
Daniele Rondina
3b266fd600
installer: Repositoris now support multiple uris 2020-01-01 13:43:23 +01:00
Daniele Rondina
0d02eccc6c
installer: Fix tests 2020-01-01 13:43:23 +01:00
Daniele Rondina
7f160a7a89
config: Add system section 2020-01-01 13:43:23 +01:00
Daniele Rondina
358b39b5dd
simpledocker: Support show_build_output option 2020-01-01 13:43:22 +01:00
Daniele Rondina
da11a84d23
Rename repository type local to "disk" 
See reference #4
 2020-01-01 13:43:22 +01:00
Daniele Rondina
0cb49a40c0
Support logging level and spinner customization 2020-01-01 13:43:22 +01:00
Daniele Rondina
bbc9574745
Review luet configuration file 2020-01-01 13:43:22 +01:00
Daniele Rondina
4dffc658db
database_boltdb: Align review of version comparision logic 2020-01-01 13:43:21 +01:00
Daniele Rondina
91d05b071d
Review version comparision logic 2020-01-01 13:43:21 +01:00
Ettore Di Giacinto
5e8a29caf5
Detect already installed packages when calling install 
We wasn't checking this previously, which was drawing weird errors on the CLI
 2020-01-01 11:58:33 +01:00
Ettore Di Giacinto
efdfe72568
Treat CompressionType none as default 
To provide backward compatibility with repos that didn't declares it explictly
 2019-12-31 12:29:53 +01:00
Ettore Di Giacinto
c193e4d320
Enhance output 2019-12-30 16:35:38 +01:00
Ettore Di Giacinto
3d5b723668
Add compression tests 
Refers to #33
 2019-12-30 16:35:35 +01:00
Ettore Di Giacinto
58eb483e32
Do not create new artifact on client 
Otherwise we loose artifact metadata - as checksum and compressiontype

Refers to #33
 2019-12-30 16:35:33 +01:00
Ettore Di Giacinto
4f65d46d56
Drop CompressedPath, or we don't have a way to compare checksums 
Refers to #33
 2019-12-30 16:35:30 +01:00
Ettore Di Giacinto
d48f510f14
Propagate Checksum and CompressionType when building ArtifactIndex 
Refers to #33
 2019-12-30 16:35:28 +01:00
Ettore Di Giacinto
ea27ada6c0
Do not return errors after we uncompress successfully 
Refers to #33
 2019-12-30 16:35:25 +01:00
Ettore Di Giacinto
f71c9937c4
Add compression to build CLI 
Also handle how concurrency is set now.

Adds also an accessor to compiler to set the desired compression type

Refers to #33
 2019-12-30 16:35:21 +01:00
Ettore Di Giacinto
475b63be95
Consume concurrency from compiler 
Refers to #33
 2019-12-30 16:35:18 +01:00
Ettore Di Giacinto
a40ecaea40
Use a separate attribute to handle the compressed artifact 
Refers to #33
 2019-12-30 16:35:15 +01:00
Ettore Di Giacinto
5155681513
Fixup tests 
Refers to #33
 2019-12-30 16:35:12 +01:00
Ettore Di Giacinto
d2d72c3fc4
Add package compression type 
TODO: Handle the path substitution in a separate field
Adds GZip support and allows the compiler to switch compression type.

It also adds Concurrency as a compiler attribute (not consumed yet)

Refers to #33
 2019-12-30 16:34:41 +01:00
Ettore Di Giacinto
bb98259a48
Add sanity check test 
To verify that we are actually comparing with some data

Closes #28
 2019-12-29 14:14:06 +01:00
Ettore Di Giacinto
fea6061f89
Add hash test to artifact_test 
Refers to #28
 2019-12-29 14:14:03 +01:00
Ettore Di Giacinto
cb98a49917
Create new Checksum struct for Artifact 
Refers to #28
 2019-12-29 14:13:51 +01:00
Ettore Di Giacinto
2693ec2f8c
Consume artifact verify mechanism 
Refers to #28
 2019-12-29 14:00:03 +01:00
Ettore Di Giacinto
eeb6719529
Add accessors to Hash and Verify artifacts 
Refers to #28
 2019-12-29 13:59:58 +01:00
Ettore Di Giacinto
17982e9527
Add package to calculate and compare artifact checksums 
Refers to #28
 2019-12-29 13:59:47 +01:00
Ettore Di Giacinto
2fa9c754ae
Move archive helpers to artifact 
This allow in the future to swap and provide archive/compression methods without hijacking the code.

Refers to #33
 2019-12-28 16:48:05 +01:00
Ettore Di Giacinto
d941c9755b Relax best match constraint rule 
Or we could return unsat
 2019-12-18 19:24:44 +01:00
Ettore Di Giacinto
15250bd991 Add support for Package provides 
Add "provides" field in packages (which affect both runtime and buildtime deps).
It replaces all the occurences in the deptree before solving, actually
allowing to swap packages and provide virtuals. Along with a mechanism
for package rename #25.
 2019-12-17 19:32:31 +01:00
Ettore Di Giacinto
0627b03121 Add IsSelector() to Package 
It indicates if the package is a selector, by checking if versions
contains constraints signs (<>=)
 2019-12-17 18:00:27 +01:00
Daniele Rondina
82c9795dc3 Add pkg description, url and license 
* Move to mvdan.cc/sh/v3
* Improve RDEPEND parsing
 2019-12-16 23:56:58 +01:00