kairos/luet
1
0
Fork 0
mirror of https://github.com/mudler/luet.git synced 2025-08-01 15:28:20 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,083 Commits 12 Branches 164 Tags 38 MiB
0028dd3a92
Commit Graph

651 Commits

Author SHA1 Message Date
Ettore Di Giacinto
46014fb9c1
Enhance install output 2020-05-03 13:22:06 +02:00
Ettore Di Giacinto
ada9d886fa
Enhance uninstall output 2020-05-03 13:22:00 +02:00
Ettore Di Giacinto
a1d8ef1422
Allow to partially uninstall a package graph, make uninstall --full optional 2020-05-03 13:04:34 +02:00
Ettore Di Giacinto
7b6e4a2176
Add to the Solver the capability to check conflicts with revdeps 2020-05-03 10:34:18 +02:00
Ettore Di Giacinto
3befbfa915
Optimize uninstall computation 2020-05-02 15:43:57 +02:00
Ettore Di Giacinto
d4c25d74f5
Remove focus from test 2020-05-02 13:27:15 +02:00
Ettore Di Giacinto
2ed9781c88
Don't try to match against repo already installed packages 2020-05-02 13:24:46 +02:00
Ettore Di Giacinto
9d6d6bc0c8
Add more upgrade and reclaim tests scenarios 2020-05-02 12:18:19 +02:00
Ettore Di Giacinto
f8b2837741
Enhance reclaim output 2020-05-02 12:17:54 +02:00
geaaru
878e6d7b9c
Merge pull request #103 from mudler/tmpdir-cleanup 
Tmpdir cleanup
 2020-05-02 09:20:01 +02:00
Daniele Rondina
a89c0af2f8 config/config_test: Fix typo 2020-05-01 15:34:17 +02:00
Daniele Rondina
1f99fde1c5 Add config test suite 2020-05-01 15:26:53 +02:00
Ettore Di Giacinto
4b63c9eaf9
Add test to be sure we don't index packages not in the tree 2020-05-01 11:48:05 +02:00
Ettore Di Giacinto
286d0fba2c
Fix typo 2020-05-01 11:07:14 +02:00
Ettore Di Giacinto
624518bf77
Don't add package to the repository which aren't referenced by the tree 2020-05-01 10:52:40 +02:00
Daniele Rondina
51a4037b1b config: Initialize luet tmp basedir if doesn't exist 2020-05-01 08:18:18 +02:00
Ettore Di Giacinto
9cb290b484
Improve database errors 2020-04-30 20:44:34 +02:00
Daniele Rondina
11944873ea Integrate tmpdir_base params and tmpdirs cleanup 2020-04-30 20:29:28 +02:00
Ettore Di Giacinto
322ac99f17
Annotate package runtime definition when reclaiming 
This was an issue as we were copying the buildspec instead
 2020-04-30 18:56:50 +02:00
Ettore Di Giacinto
6acc5fc97e
Make BuildFormula support dependency cycles 2020-04-30 18:56:35 +02:00
Ettore Di Giacinto
6a4557a3b3
Make RequireContains support dependency cycles 2020-04-30 18:56:06 +02:00
Ettore Di Giacinto
a457c53824
Fixups to ExpandedRevdeps 2020-04-24 00:15:18 +02:00
Ettore Di Giacinto
8305d01e76
Walk requires in ExpandedRevdeps 
In this way we annotate the visited and we avoid cycles that can be
generated by revdeps
 2020-04-23 23:44:42 +02:00
Ettore Di Giacinto
9321a310a5
Support mount with target folder in box exec 
Also mount bind before pivotroot, this fixes bind mounting in general
 2020-04-21 17:30:27 +02:00
Ettore Di Giacinto
4cee27da7f
Fixup box CLI params and consume new box options 2020-04-20 23:03:03 +02:00
Ettore Di Giacinto
98876c8f20
Add support for env and hostmounts in box 2020-04-20 23:01:49 +02:00
Ettore Di Giacinto
4c2d38be59
Add package ExpandedRevdeps 2020-04-19 11:55:13 +02:00
Ettore Di Giacinto
8e91c255a3
Make FindPackageMatch match packages HumanReadableString 2020-04-19 10:47:55 +02:00
Ettore Di Giacinto
ac871cb0a3
Add --output option to search 
In such way it can be parsed by scripts more easily.
It also disable the spinner based on loglevel

Fixes #92
 2020-04-18 23:22:00 +02:00
Ettore Di Giacinto
4d466d0330
Sync the repository data with user configured data 
Fixup a regression, in this way we respect the user configuration in
regards to a repository to be consumed (priority and repository type)
regardless of the one advertized remotely.

Also add informative message about prio and type.
 2020-04-18 17:33:46 +02:00
Ettore Di Giacinto
84407e5ae7
Add SetPriority to repository 2020-04-18 17:33:34 +02:00
Ettore Di Giacinto
64bac0823c
Consume our moby fork 
It handles #91
 2020-04-18 11:41:34 +02:00
Ettore Di Giacinto
ee0fe1a86a
Allow to finalizer to specify entrypoint command 
In this way, finalizer in strict environment can override the default
shell used to run commands.

The shell keyword is a list, as it needs to contain the full command +
args.
 2020-04-14 17:46:39 +02:00
Ettore Di Giacinto
538ba8f5df
Add luet reclaim 
Reclaim allows to migrate between different system layouts. This
is a experimental feature (yet) and might be revisited in the future.

This change:

- Adds Reclaim(system) to Installer
- Adds unit tests

Relates to #86
 2020-04-13 19:32:51 +02:00
Ettore Di Giacinto
eb56956c65
Add Touch file helper 2020-04-13 19:32:50 +02:00
Ettore Di Giacinto
4e2d42e397
Drop downloadOnly bool from installer.Install 2020-04-13 19:32:50 +02:00
Ettore Di Giacinto
528f481a7b
Annotate package files in metadata 
Fixes #87
 2020-04-13 10:52:41 +02:00
Ettore Di Giacinto
f987ee9fa0
Handle errors from os/user 
We use it to generate defaults, if error is found, we imply preserve
permissions
 2020-04-11 20:05:56 +02:00
Daniele Rondina
69a82a1ca5 simpledocker: Use debug option for print container-diff results 2020-04-10 09:32:12 +02:00
Daniele Rondina
02c33896d5 simpledocker: Move warning before return 2020-04-10 09:31:08 +02:00
Daniele Rondina
c2e9176ab2 solver.Order now return error 2020-04-09 17:07:57 +02:00
Ettore Di Giacinto
0d2668e452
Print warning if container-diffs return errors to stderr 2020-04-08 18:31:40 +02:00
Ettore Di Giacinto
77ba4193aa
Add ValidateSelector to versioner interface and consume it 
We can refactor furthermore by dropping the package methods, as now we
can consume a versioner in all places that requires it
 2020-04-05 15:52:16 +02:00
Ettore Di Giacinto
5a5e7f1dfa
Move version logic inside versioner 
WIP, it needs yet to be under the interface implementation
 2020-04-05 11:16:14 +02:00
Ettore Di Giacinto
cf8091a7fd
Re-enable debian versioning test, add more cases 2020-04-04 16:49:41 +02:00
Ettore Di Giacinto
3caaa01eb8
Add semver detection to versioner 
In this way we compare only all version that are compliant to semver,
otherwise we fallback to debian sorting.
 2020-04-04 16:48:48 +02:00
Ettore Di Giacinto
4eae0a851c
Add versioner unit tests 2020-04-04 16:33:08 +02:00
Ettore Di Giacinto
626419f955
Fix original version mapping in versioner 2020-04-04 16:23:43 +02:00
Ettore Di Giacinto
84625be9ac
Adapt package.Best to take a Versioner interface 2020-04-04 15:33:14 +02:00
Ettore Di Giacinto
04f9a88a5f
Add versioner interface to wrap versioning operations 
This allows to scope responsabilities and enables switching to different
ordering algorithms
 2020-04-04 15:29:20 +02:00
Ettore Di Giacinto
9d58b0a0cc
Adapt tests to refactor 2020-04-04 14:41:58 +02:00
Ettore Di Giacinto
5e31d940f0
Introduce Packages for []Package 2020-04-04 14:29:08 +02:00
Daniele Rondina
129ca8da55 Add version sanitized to Best() 2020-04-03 18:32:26 +02:00
Daniele Rondina
ee66839bd3 pkg/package/version: Better search for last + 2020-03-29 09:10:52 +02:00
Ettore Di Giacinto
a0113dcd13
Drop verbose output 2020-03-28 17:05:23 +01:00
Ettore Di Giacinto
49d7b4e2bf
Add box structure to handle containerized executions 2020-03-28 16:58:46 +01:00
Ettore Di Giacinto
f15ed3fda1
Move finalizer in its own struct 
Also distinguish when we run in a root target dir ("/") or when we want
to run the commands in a separate folder
 2020-03-28 16:57:40 +01:00
Ettore Di Giacinto
2ad16fa875
Add ListDir helper 2020-03-28 16:55:16 +01:00
Ettore Di Giacinto
416be23a46
Don't always remove unpacked repository content 2020-03-28 12:07:40 +01:00
Daniele Rondina
50091b2a4b pkg/installer: Add test for uncompress tree and minor cleanup 2020-03-25 00:19:52 +01:00
Ettore Di Giacinto
117554792d
Don't attempt to upgrade packages already in system 2020-03-24 20:30:59 +01:00
Daniele Rondina
454e9d934e Use filename instead of name on repo specs 2020-03-24 19:40:11 +01:00
Ettore Di Giacinto
af7f1de9f1
Split download and installer worker 
Also, drop downloadOnly from install
 2020-03-24 18:19:09 +01:00
Daniele Rondina
a0f9222068 Add check of repository metafile archive in tests 2020-03-24 14:01:22 +01:00
Daniele Rondina
520768d0ca pkg/installer: Align tests with new default 2020-03-24 12:52:45 +01:00
Daniele Rondina
4f002ab40f pkg/installer/repository: The check for repo_files is not needed for parsing normal repo 2020-03-24 12:52:24 +01:00
Daniele Rondina
60635a03eb Fix propagation of repo file name and add test for meta compressed 2020-03-24 00:31:41 +01:00
Daniele Rondina
202ed2651a 💥 Refactor and split repository.yaml file 2020-03-24 00:05:16 +01:00
Daniele Rondina
6bd8fe6789 Add omitempty to DefaultPackage fields 2020-03-22 22:23:11 +01:00
Daniele Rondina
40830ecf42 installer/repository.go: Use switch with search options 2020-03-22 10:46:51 +01:00
Daniele Rondina
c7652c8a70 Added labels to package definition 
* cmd/search: Add support for search of the packages
  with a specific label.

* review Search method of Repositories for permit
  different search modes.

* labels are k/v attributes and could be matched
  through label key (with HasLabel method) or through
  regex that use "$key" + "=" + "$value"
 2020-03-21 19:24:27 +01:00
Ettore Di Giacinto
fab2d21ac1
Drop unused function 2020-03-19 22:39:22 +01:00
Ettore Di Giacinto
487239334f
Treat base case with same algorithm of dependencies 2020-03-19 22:37:39 +01:00
Ettore Di Giacinto
69477a0c36
Make uniform build of seeds and packages 2020-03-19 22:34:26 +01:00
Ettore Di Giacinto
67362a3bc1
Drop unused condition 2020-03-19 22:29:34 +01:00
Ettore Di Giacinto
302d18e749
Tag the target image with the computed hash 2020-03-19 22:28:33 +01:00
Daniele Rondina
0249c0fa4a version: Add temporary workaround for handle build string with _ 2020-03-16 22:18:48 +01:00
Daniele Rondina
fc40c770ab Add 'tree bump' command 2020-03-16 22:18:48 +01:00
Ettore Di Giacinto
4357ee45e9
Drop unused functions in package 2020-03-16 18:08:46 +01:00
Ettore Di Giacinto
6f138811dd Make recipes idempotent, allowing to load multiple trees 2020-03-15 13:31:12 +01:00
Daniele Rondina
bc5c0fa0cf cmd/tree/pkglist: Cleanup code 2020-03-14 11:14:14 +01:00
Ettore Di Giacinto
dc64dbff75
Don't try to solve finalizers with --nodeps 2020-03-13 19:32:00 +01:00
Daniele Rondina
93f8f0dd0f solver: Add category to tests 2020-03-12 00:26:53 +01:00
Daniele Rondina
f4a5a97cff tree/compiler_recipe: Return right error message 2020-03-08 16:28:12 +01:00
Daniele Rondina
8c972403a3 tree/compiler_recipe: Handle invalid tree path 
If instead of abs path is used bash annotation
now return an error related to wrong directory
supply.
 2020-03-08 15:47:40 +01:00
Daniele Rondina
17238d187a Change 'Skip dir' message to debug 2020-03-08 00:07:20 +01:00
Ettore Di Giacinto
765261f233
Add Swap() to Installer. Refactor code in Upgrade() to consume it 2020-02-27 23:48:14 +01:00
Ettore Di Giacinto
6c7e24fadf
Sync repositories once (and at beginning) during upgrade 2020-02-27 23:25:29 +01:00
Ettore Di Giacinto
de23e0d5b1
Download dependencies before installing 
This enables smoother upgrades, and uses cache to pre-download
packages before uninstall/install

Allows also from cli to download deps only
 2020-02-27 23:15:31 +01:00
Ettore Di Giacinto
8572aa5222
Preserve cache data from deletion during uninstall 2020-02-27 23:14:36 +01:00
Ettore Di Giacinto
5bcc8d112a
Enforce solver constraints 
- Don't sign installed packages during finalizer execution
- Enforce solver constraints: build ALO and AMO rules taking into account
  that the current package might not be selected at all.
- Force uninstalls on upgrade
- Enable option to tell uninstall to ignore conflict with the analized system state,
  as we don't want any conflict with the installed to raise during the upgrade.
  In this way we both force uninstalls and we avoid to check with conflicts
  against the current system state which is pending to deletion.
  This is due to the fact that now the solver enforces the constraints
  and explictly denies two packages of the same version installed.
- Adapt test as now we generate more constraints, which makes the solver more
  noisy on the package that are explictly selected or not
 2020-02-27 18:38:31 +01:00
Ettore Di Giacinto
a0618107a8
Adapt tests 2020-02-26 23:56:21 +01:00
Ettore Di Giacinto
25f2abf103
Do conflict with same package versions while building formula 2020-02-26 23:01:08 +01:00
Ettore Di Giacinto
e3ebfd6bfe
Respect ImageRepository with build --nodeps 2020-02-26 19:00:22 +01:00
Ettore Di Giacinto
07610bc216
Add build option to remove backend artifacts 
Add keep-exported-images to build to instruct to remove backend artifacts (docker images)
 2020-02-21 22:33:18 +01:00
Daniele Rondina
a04dadf100 Add support for parsing version with build 2020-02-21 21:56:32 +01:00
Daniele Rondina
def04724d4 Add support for build identifier on version 2020-02-20 17:46:47 +01:00
Ettore Di Giacinto
38296bc5d7
Add test for HashFingerprint() 2020-02-19 18:33:14 +01:00
Ettore Di Giacinto
7e16e0cdb6
Add HashFingerprint() to Package and consume it while generating image names 
In this way we don't depend on the package fingerpint characters while generating images, allowing
to support completely semver (e.g. + character is not allowed in docker images tag).

This change is not backward compatible with images currently already built and will invalidate the cache
 2020-02-19 18:28:29 +01:00
Ettore Di Giacinto
647ea35983
Add a way to extend the CLI with external binaries 
This lets luet to be extended like git does (just by having luet-subcommand binaries under $PATH)
following the UNIX filosophy. It has the downside that we silence the errors from cobra and we handle
them by ourselves.

Also the Exec syscall is not portable, and it should have implementation for each platform (hence why in helpers)
 2020-02-18 23:08:14 +01:00
Ettore Di Giacinto
4648dedb55
Add --nodeps, --onlydeps and --force to Installer 
This allows also to use it on upgrade and uninstall as it exposes the Installer options to the CLI.

See #48, #49
 2020-02-18 18:37:56 +01:00
Ettore Di Giacinto
2ce427d601
Add Nodeps, Onlydeps and Force options to Compiler 
See #41
 2020-02-18 18:36:44 +01:00
Ettore Di Giacinto
b01c017507
Add HumanReadableString() to Package 2020-02-18 18:20:45 +01:00
Ettore Di Giacinto
dd00d491b9
Skip repository if no candidate is found 
FindPackageCandidate is idempotent and returns the same definition if no new is found.
This prevents installs from multiple-repos
 2020-02-17 17:21:31 +01:00
Ettore Di Giacinto
26b94888c3
Skip image building if we pulled them successfully 2020-02-15 22:53:00 +01:00
Ettore Di Giacinto
fcec6c5699
Add Push to hub feature 
Add to the backends the Push capabilities to push images that were correctly built.
Also expose them to the CLI, along with KeepImg which was hidden.
 2020-02-15 14:45:05 +01:00
Ettore Di Giacinto
c7253ac8ad
Adapt test to spec changes 2020-02-14 08:04:26 +01:00
Ettore Di Giacinto
f329e1d5e0
Add retrieve to compilation spec 
It allows to copy build artifact during buildtime. In this way
artifacts between different seed images can be shared

```
retrieve:
- a-test-1.0.package.*
- https://...
```

They will be available under WORKDIR
 2020-02-13 14:15:43 +01:00
Ettore Di Giacinto
d6f7c47eae
Add compact string notation for SolverOptions 2020-02-12 12:23:37 +01:00
Ettore Di Giacinto
b2a5de9222
Adapt installer test to constructor changes 2020-02-12 12:05:28 +01:00
Ettore Di Giacinto
7e388c6fed
Cleanup resolver from logger 
It creates cycle and we don't want to output anything from the computation process.
We should handle output in different stages

Also create constructor for solver to be able to consume resolvers.
 2020-02-12 11:22:56 +01:00
Ettore Di Giacinto
07a154474b
Consume SolverOptions in compiler 2020-02-12 11:21:55 +01:00
Ettore Di Giacinto
6aa353edb2
Consume SolverOptions in installer 2020-02-12 11:21:30 +01:00
Ettore Di Giacinto
8951203165
Add LuetSolverOptions to tweak solver parameter in Config 2020-02-12 11:21:08 +01:00
Ettore Di Giacinto
d330fedcc4
fmt 2020-02-12 11:05:13 +01:00
Ettore Di Giacinto
dfb6dab9dc
Move repository helpers under config 
They are generated after the system config, let the structure provide such information
 2020-02-12 10:20:07 +01:00
Ettore Di Giacinto
4f33eca263
Consume internal attempts in QLearn so resolver can be re-used 2020-02-12 09:22:10 +01:00
Ettore Di Giacinto
54b0dce54b
Respect argument passed to Untar in helpers 2020-02-11 16:55:00 +01:00
Ettore Di Giacinto
ea2a60a853
Cleanup, drop hardcoded values and use constructors 2020-02-11 15:58:28 +01:00
Ettore Di Giacinto
c8f4ba0a47
Add Factorial helper 2020-02-11 15:00:28 +01:00
Ettore Di Giacinto
7e0ea34b81
Switch back to gophersat 2020-02-11 14:58:17 +01:00
Ettore Di Giacinto
711c039296
Reward by observedDelta 
Keep a record of the observed delta and maximize reward for it.
Also add Noop actions which is turned off by default.

Let finish the execution also when no solution is found, as we will take the
minimum observed delta as result.

This is done on purpose to avoid guessing "when" is a good time to stop the agent,
as it could be in the middle of picking up a new action which is not the final
(but we need limits, we can't let it run forever).
 2020-02-11 14:52:24 +01:00
Ettore Di Giacinto
ac6554c291
Support Add/Removal actions 
Also keep the list of the wanted targets as we walk it. We will let the agent to try different solutions
 2020-02-11 09:51:52 +01:00
Ettore Di Giacinto
d4255b086b
Uncomment failing test 
Add test to support that normally we need to pass explictly the installable ones only.
 2020-02-11 09:50:26 +01:00
Ettore Di Giacinto
1b90407475
Don't preclude action space 2020-02-11 09:21:25 +01:00
Ettore Di Giacinto
6f6e2bf15f
Pin to gophersat version 
Having the same var in the and block seems to make gophersat crash. Even if might be unoptimal,
we need this to tighten the conditions between packages.

Switch to gophersat fork until this fix is merged upstream:

https://github.com/crillab/gophersat/pull/17
 2020-02-11 09:06:57 +01:00
Ettore Di Giacinto
6d450d3af0
Add simple QLearning solver support 
Still experimental, it covers very small subset of action domains (just removal from target).

Added a pending test that currently fails
 2020-02-10 17:18:18 +01:00
Ettore Di Giacinto
33b442a832
Add accessor to decode from package String() 2020-02-10 17:14:46 +01:00
Ettore Di Giacinto
f068bfdb9b
Add PackageResolver to add heuristics on unsat solutions 2020-02-10 09:41:09 +01:00
Ettore Di Giacinto
4dc4205868
Drop json skip from id. It prevents storm to track entities #55 2020-02-08 18:18:05 +01:00
Ettore Di Giacinto
f813370501
Resolve selectors if present before starting install 2020-02-04 20:15:59 +01:00
Daniele Rondina
c353ab4978 client/http: Add support for authentication Basic/Token 2020-02-03 00:58:55 +01:00
Daniele Rondina
298de447b8 Fix installer/client test suite 2020-02-01 21:18:32 +01:00
Daniele Rondina
40687c3072 Use dynamic pkgs cache dir for test suite 2020-02-01 20:05:19 +01:00
Daniele Rondina
78b5963a4f Add support for local packages cache 2020-02-01 19:35:30 +01:00
Daniele Rondina
524bbf990e Add support for same-owner config option 2020-02-01 19:01:15 +01:00
Ettore Di Giacinto
0e30e6a1ad
Treat single conflict cases as well when building conflict formula 2020-01-31 20:50:55 +01:00
Ettore Di Giacinto
0147b2cf99
Don't cleanup the whole cache download dir, just the downloaded artifact 
Otherwise whatever we download cannot be consumed (e.g. tree).
Instead whoever is calling the client should care to cleanup the returned artifact
 2020-01-30 18:03:25 +01:00
Ettore Di Giacinto
c6fe34b059
Support compression and checksum for trees 
Fixes #33 #34 #35
 2020-01-28 17:46:32 +01:00
Daniele Rondina
c9b684523f installer: Add support for cached repository 2020-01-12 23:35:58 +01:00
Daniele Rondina
aeea0cc5fe Add Cached option to LuetRepository 2020-01-12 23:33:19 +01:00
Daniele Rondina
091e51e426 logger: Add support for custom color text 2020-01-12 23:33:19 +01:00
Ettore Di Giacinto
22bc53ba13
Merge branch 'master' into develop 2020-01-06 19:14:17 +01:00
Ettore Di Giacinto
f5e7c2ad92
Allow more matches, we select the best one anyway 2020-01-06 11:33:35 +01:00
Daniele Rondina
07633dc307 Add container-diff summary 2020-01-05 18:05:56 +01:00
Ettore Di Giacinto
d5fd14bceb
Merge branch 'build_clean' into develop 2020-01-05 16:27:31 +01:00
Ettore Di Giacinto
d2abaa9cc1
Add message asserting skip. Also return pointer to allow to edit Options 2020-01-05 16:26:42 +01:00
Ettore Di Giacinto
d23e1dee78
Default to Clean true in CompilerOptions 2020-01-05 16:08:40 +01:00
Ettore Di Giacinto
ee055e08b1
Set abs path when returning artifact from yaml 2020-01-05 16:08:39 +01:00
Ettore Di Giacinto
02c37c7451
Fix tests after contructor change 2020-01-05 16:08:39 +01:00
Ettore Di Giacinto
1d1efad18b
Skip building if artifact already exists 2020-01-05 15:30:16 +01:00
Ettore Di Giacinto
bcc6ce19ea
Move compiler options to its own struct 
Also add Clean attribute, to handle a future build clean method
 2020-01-05 14:32:26 +01:00
Daniele Rondina
9b6f4a094d Use const for repository.yaml and tree.tar 2020-01-05 00:40:28 +01:00
Daniele Rondina
e751b989e0 Fix installer_tests 2020-01-04 16:15:56 +01:00
Daniele Rondina
6012e0081e Add support of incremental revision for repos 
* on repository creation now if the repository.yaml
  is already present, the current revision is used.

* add --reset-revision option for force reset revision
  of a specific repository
 2020-01-04 00:31:11 +01:00
Daniele Rondina
d3bd78d618 Repository now support Revision and LastUpdate 2020-01-03 20:09:29 +01:00
Daniele Rondina
6af62b5851 Define default values for loaded repositories 2020-01-03 19:37:51 +01:00
Daniele Rondina
7ec36da059 Add --fatal option 
Close #8
 2020-01-03 15:41:45 +01:00
Daniele Rondina
c284d3e4bf Add support for logging to file 2020-01-03 15:22:55 +01:00
Daniele Rondina
f28e8deb96 logger: Add support for json format 2020-01-03 15:20:42 +01:00
Daniele Rondina
716d404307 Align logic of cache repositories in all commands 2020-01-02 18:31:25 +01:00
Daniele Rondina
b12410edb7 Errors on create database paths are now fatal 2020-01-01 22:58:53 +01:00
Daniele Rondina
db7301f7bf Align selector logic for explicit version with PkgSelectorConditionFromInt 2020-01-01 22:58:19 +01:00
Daniele Rondina
ebcf6075d0 database_mem: Add test for specific candidate 2020-01-01 22:55:19 +01:00
Daniele Rondina
ce888a2f40
Fix marshal of repository and use of cache repos 2020-01-01 13:44:39 +01:00
Daniele Rondina
01e66ee0b4
Review install phase 2020-01-01 13:44:39 +01:00
Daniele Rondina
a71e1a6f1d
Integrate config.LuetRepository with installer 
* installer.LuetRepository is now installer.LuetSystemRepository,
  a struct that extend config.LuetRepository

* config: system_repositories option is now "repositories".
  This resolve viper issue.

* config: cache_repositories option is now "repetitors"
  This resolve viper issue.

* cmd/*: Now use new config.LuetRepository

* cmd/search: now create local luet repository if database_engine
  is equal to "boltdb"
 2020-01-01 13:43:23 +01:00
Daniele Rondina
3b266fd600
installer: Repositoris now support multiple uris 2020-01-01 13:43:23 +01:00
Daniele Rondina
0d02eccc6c
installer: Fix tests 2020-01-01 13:43:23 +01:00
Daniele Rondina
7f160a7a89
config: Add system section 2020-01-01 13:43:23 +01:00
Daniele Rondina
358b39b5dd
simpledocker: Support show_build_output option 2020-01-01 13:43:22 +01:00
Daniele Rondina
da11a84d23
Rename repository type local to "disk" 
See reference #4
 2020-01-01 13:43:22 +01:00
Daniele Rondina
0cb49a40c0
Support logging level and spinner customization 2020-01-01 13:43:22 +01:00
Daniele Rondina
bbc9574745
Review luet configuration file 2020-01-01 13:43:22 +01:00
Daniele Rondina
4dffc658db
database_boltdb: Align review of version comparision logic 2020-01-01 13:43:21 +01:00
Daniele Rondina
91d05b071d
Review version comparision logic 2020-01-01 13:43:21 +01:00
Ettore Di Giacinto
5e8a29caf5
Detect already installed packages when calling install 
We wasn't checking this previously, which was drawing weird errors on the CLI
 2020-01-01 11:58:33 +01:00
Ettore Di Giacinto
efdfe72568
Treat CompressionType none as default 
To provide backward compatibility with repos that didn't declares it explictly
 2019-12-31 12:29:53 +01:00
Ettore Di Giacinto
c193e4d320
Enhance output 2019-12-30 16:35:38 +01:00
Ettore Di Giacinto
3d5b723668
Add compression tests 
Refers to #33
 2019-12-30 16:35:35 +01:00
Ettore Di Giacinto
58eb483e32
Do not create new artifact on client 
Otherwise we loose artifact metadata - as checksum and compressiontype

Refers to #33
 2019-12-30 16:35:33 +01:00
Ettore Di Giacinto
4f65d46d56
Drop CompressedPath, or we don't have a way to compare checksums 
Refers to #33
 2019-12-30 16:35:30 +01:00
Ettore Di Giacinto
d48f510f14
Propagate Checksum and CompressionType when building ArtifactIndex 
Refers to #33
 2019-12-30 16:35:28 +01:00
Ettore Di Giacinto
ea27ada6c0
Do not return errors after we uncompress successfully 
Refers to #33
 2019-12-30 16:35:25 +01:00
Ettore Di Giacinto
f71c9937c4
Add compression to build CLI 
Also handle how concurrency is set now.

Adds also an accessor to compiler to set the desired compression type

Refers to #33
 2019-12-30 16:35:21 +01:00
Ettore Di Giacinto
475b63be95
Consume concurrency from compiler 
Refers to #33
 2019-12-30 16:35:18 +01:00
Ettore Di Giacinto
a40ecaea40
Use a separate attribute to handle the compressed artifact 
Refers to #33
 2019-12-30 16:35:15 +01:00
Ettore Di Giacinto
5155681513
Fixup tests 
Refers to #33
 2019-12-30 16:35:12 +01:00
Ettore Di Giacinto
d2d72c3fc4
Add package compression type 
TODO: Handle the path substitution in a separate field
Adds GZip support and allows the compiler to switch compression type.

It also adds Concurrency as a compiler attribute (not consumed yet)

Refers to #33
 2019-12-30 16:34:41 +01:00
Ettore Di Giacinto
bb98259a48
Add sanity check test 
To verify that we are actually comparing with some data

Closes #28
 2019-12-29 14:14:06 +01:00
Ettore Di Giacinto
fea6061f89
Add hash test to artifact_test 
Refers to #28
 2019-12-29 14:14:03 +01:00
Ettore Di Giacinto
cb98a49917
Create new Checksum struct for Artifact 
Refers to #28
 2019-12-29 14:13:51 +01:00
Ettore Di Giacinto
2693ec2f8c
Consume artifact verify mechanism 
Refers to #28
 2019-12-29 14:00:03 +01:00
Ettore Di Giacinto
eeb6719529
Add accessors to Hash and Verify artifacts 
Refers to #28
 2019-12-29 13:59:58 +01:00
Ettore Di Giacinto
17982e9527
Add package to calculate and compare artifact checksums 
Refers to #28
 2019-12-29 13:59:47 +01:00
Ettore Di Giacinto
2fa9c754ae
Move archive helpers to artifact 
This allow in the future to swap and provide archive/compression methods without hijacking the code.

Refers to #33
 2019-12-28 16:48:05 +01:00
Ettore Di Giacinto
d941c9755b Relax best match constraint rule 
Or we could return unsat
 2019-12-18 19:24:44 +01:00
Ettore Di Giacinto
15250bd991 Add support for Package provides 
Add "provides" field in packages (which affect both runtime and buildtime deps).
It replaces all the occurences in the deptree before solving, actually
allowing to swap packages and provide virtuals. Along with a mechanism
for package rename #25.
 2019-12-17 19:32:31 +01:00
Ettore Di Giacinto
0627b03121 Add IsSelector() to Package 
It indicates if the package is a selector, by checking if versions
contains constraints signs (<>=)
 2019-12-17 18:00:27 +01:00
Daniele Rondina
82c9795dc3 Add pkg description, url and license 
* Move to mvdan.cc/sh/v3
* Improve RDEPEND parsing
 2019-12-16 23:56:58 +01:00
Ettore Di Giacinto
d583fa8bf5 Force the solver to look at the best match first 
Add support clauses to force the solver to look after the best match
first.

Closes #29
 2019-12-16 17:56:52 +01:00
Ettore Di Giacinto
fe608469d8
Do not call solver multiple times, reuse same assertion 
Introduce Cut(), it allows to filter in the assertion to achieve the same hashes
 2019-12-14 15:00:16 +01:00
Ettore Di Giacinto
c4b83605ef
Add SearchByName and avoid false positive due to greedy search 2019-12-14 09:12:49 +01:00
Ettore Di Giacinto
fb68f98b15
Enforce requirements 
Make explicit that at least one must be selected
 2019-12-14 01:02:12 +01:00
Ettore Di Giacinto
fa79afd6ba
Various fixes 
We can't route the message to the spinner, we would hide deps tree
 2019-12-14 00:31:21 +01:00
Ettore Di Giacinto
94f65f3c55
Drop superflous call 2019-12-14 00:07:46 +01:00
Ettore Di Giacinto
396c090bc7
Drop unused import 2019-12-13 23:46:04 +01:00
Ettore Di Giacinto
66f2115cd1
Decoder must search in assertions only 2019-12-13 23:45:02 +01:00
Ettore Di Giacinto
b8c62c3e85
Move selection logic to package BuildFormula() 2019-12-13 23:37:44 +01:00
Ettore Di Giacinto
926f636dff
Revert "Do not allow multiple versions in the results" 
This reverts commit 15534ce253.
 2019-12-13 23:05:09 +01:00
Ettore Di Giacinto
15534ce253
Do not allow multiple versions in the results 
This happens because we select the best instead of selecting the best match in Package BuildFormula()

- Extend the selection test
- Select from databases when ordering
- Relax assertions search
- Split compile step in test
- Adapt tests

Note: This is temporarly until we treat this case in BuildFormula() inside Package
we need to build the constraints between all the requires when expanding
and then create a new constraint that selects the best match
 2019-12-13 22:10:12 +01:00
Ettore Di Giacinto
adcd8df49e
Add Bigger/Lower to Package to allow version comparison easier 2019-12-13 17:18:51 +01:00
Ettore Di Giacinto
69be9a2dd1
Add FindPackageVersions to DB 2019-12-13 17:18:26 +01:00
Ettore Di Giacinto
adda44a752
Send text to spinner if active 2019-12-12 23:49:15 +01:00
Ettore Di Giacinto
1f60d591d5
Stringify finalizers output 2019-12-12 23:48:29 +01:00
Ettore Di Giacinto
8cab02ad88 Adapt test which checks for db ID 2019-12-06 16:30:20 +01:00
Ettore Di Giacinto
57181d7cbf Add FindPackages() 
The version inmemory is optimized, while the boltdb implementation is
not.

It returns a list of the same package but with all the versions present
in the db.
 2019-12-06 16:29:15 +01:00
Ettore Di Giacinto
556668fcc4 Make Expand and Revdeps consume a database 2019-12-06 16:28:42 +01:00
Ettore Di Giacinto
595c10f357
Use fingerprint as ids in inmemorydb 2019-12-03 23:38:25 +01:00
Ettore Di Giacinto
097f2ec827
Support includes also on packages coming from unpacked images 2019-12-03 18:15:07 +01:00
Ettore Di Giacinto
2f9d12a248
Add KeepImg and PullFirst options 
They allow to keep a cache,define an endpoint and pull it first.

Also slim logic in favor of more freedom on backend side
 2019-12-02 16:36:55 +01:00
Ettore Di Giacinto
1a584d0bfa
Fix img backend calls 
Options have to go first always.

Note: it needs runc when unpacking
 2019-12-02 16:36:29 +01:00
Ettore Di Giacinto
58169770e1
Add test for FindPackageCandidate 
Adapt tests for env
 2019-12-01 23:24:42 +01:00
Ettore Di Giacinto
ebf818ff08
Add Environment to compilespec #24 
Now compilespecs can define envs and have PACKAGE_NAME, PACKAGE_VERSION and PACKAGE_CATEGORY injected
 2019-12-01 21:20:16 +01:00
Ettore Di Giacinto
55fa7265e5
Move selection logic to a db accessor #19 
In this way the compiler consume such accessor, and make it possible to expand whenever used
 2019-12-01 19:11:31 +01:00
Ettore Di Giacinto
ddfb4fb8e8
Drop explicit directory copy 2019-12-01 19:10:40 +01:00
Ettore Di Giacinto
e23e38b571
Do not treat copy errors as fatal for now 2019-11-30 11:51:50 +01:00
Ettore Di Giacinto
f4fff415f4
Disable extra output from container-diff 2019-11-30 11:51:37 +01:00
Ettore Di Giacinto
c7e1803540
Fix path deletion on artifacts 2019-11-29 19:01:59 +01:00
Ettore Di Giacinto
30a7312911
Compute upgrades 
Make solver compute simply upgrades and the installer execute them.

Add simple upgrade test
 2019-11-29 19:01:56 +01:00
Ettore Di Giacinto
ff3b322ea2
Expand also when selecting candidates to install/uninstall 2019-11-29 19:01:55 +01:00
Ettore Di Giacinto
e31f8820e2
Adapt tests 
Adapt docker and package tests

Fix tree test
 2019-11-29 19:01:54 +01:00
Ettore Di Giacinto
2c7391fcc9
Adapt installer to new changes 2019-11-29 19:01:52 +01:00
Ettore Di Giacinto
d0c84d24a0
Expand also when ordering assertions 2019-11-29 19:01:52 +01:00
Ettore Di Giacinto
91b3daf180
Drop Best from solver and add cases for solver with expansion 2019-11-29 19:01:51 +01:00
Ettore Di Giacinto
4031a4ae81
Marshal JSON of package without escapes 
Otherwise >, < get escaped as JSON does that in favour of browsers
 2019-11-29 19:01:50 +01:00
Ettore Di Giacinto
fa46601638
Adapt compiler and installer to Tree removal 2019-11-29 19:01:49 +01:00
Ettore Di Giacinto
f71cc5281e
Offer World from database 
Compute here world instead of scattering it in different structures.

Also move Best() in package and make expansion here #20
 2019-11-29 19:01:48 +01:00
Ettore Di Giacinto
c9d93454b8
Drop IsFlagged() from what the solver considers 2019-11-29 19:01:47 +01:00
Ettore Di Giacinto
66cd6d1027
Drop ResolveDeps, and the Tree structure #20 2019-11-29 19:01:46 +01:00
Ettore Di Giacinto
b5381e0248
Make solver consume databases instead of world lists 
first step - it is slower in the implementation for now, but all cases
seems to be sorted out.
Drop Flagged() and IsSet() from solver. the solver wont care, and only
the assertion value does matter (exception for uninstall)
 2019-11-29 19:01:41 +01:00
Ettore Di Giacinto
da9e14fb45
Distinguish solverdb 2019-11-29 19:01:12 +01:00
Ettore Di Giacinto
6d25e5d881
Check if a layer is provided in the manifest 2019-11-26 20:22:33 +01:00
Ettore Di Giacinto
393164da36
Extract all layers of the docker image 2019-11-26 20:12:06 +01:00
Ettore Di Giacinto
1cf5f2eb74
Enhance compile output 2019-11-26 20:11:51 +01:00
Ettore Di Giacinto
caca3fbf8c
Return immediately in case of deps failures 2019-11-26 20:11:27 +01:00
Ettore Di Giacinto
5915688830
Add client tests 2019-11-26 18:06:23 +01:00
Ettore Di Giacinto
a54040dcd3
Add http client for installer #3 2019-11-26 18:05:48 +01:00
Ettore Di Giacinto
08897b5105
Allow one instance of the application to run 2019-11-25 20:28:08 +01:00
Ettore Di Giacinto
62af9c81d5
BoltDB fixups 
Unmarshalling needs well-defined structs, also swap pointers in few places where needed.

There are few more TODOS pending (like getting UpdatePackage fixed properly)
 2019-11-25 20:03:37 +01:00
Ettore Di Giacinto
7bbd6d5e68
Add Search() in Repositories 2019-11-25 20:03:17 +01:00
Ettore Di Giacinto
820a157698
Add SetUri to Repository 2019-11-25 20:02:59 +01:00
Ettore Di Giacinto
f3ba014593
Add test for persistent database with bolt 2019-11-25 20:02:43 +01:00
Ettore Di Giacinto
c9c803a819
Return errors more consistently 2019-11-25 20:02:18 +01:00
Ettore Di Giacinto
b751b02830
Do not register viper bindings in init 
Otherwise they get overlapped. Use PreRun instead
 2019-11-25 19:55:30 +01:00
Ettore Di Giacinto
a8b350fd8e
Implement uninstall 
Also add RemovePackage in database implementation (boltdb is not tested yet)

Add test case for uninstall
 2019-11-24 00:16:12 +01:00
Ettore Di Giacinto
0d7b2cf448
Track installed files 2019-11-23 22:41:51 +01:00
Ettore Di Giacinto
a0d2f9cc12
Use a smarter copy 2019-11-23 19:13:32 +01:00
Ettore Di Giacinto
3a7cfae9ae
Adapt solver test to structure change 2019-11-23 19:01:26 +01:00
Ettore Di Giacinto
7d99d57ad5
Check if package was inserted in systemdb in tests 2019-11-23 18:58:34 +01:00
Ettore Di Giacinto
d4bb0e346e
Execute finalizers if present 2019-11-23 18:58:09 +01:00
Ettore Di Giacinto
3b9337a03b
Keep the path around after writing the yaml, otherwise compile specs couldn't be found 2019-11-23 18:41:15 +01:00
Ettore Di Giacinto
8f6393e157
Add repository test file 2019-11-23 15:43:14 +01:00
Ettore Di Giacinto
e9c01b46a7
Use well defined structure for serializing, fixups to make test green 2019-11-23 15:42:53 +01:00
Ettore Di Giacinto
a879411c54
Add test case 2019-11-23 12:19:05 +01:00
Ettore Di Giacinto
542d45a646
Do not use interface in struct used for serialization 2019-11-23 00:29:44 +01:00
Ettore Di Giacinto
43ab851cb9
Load tree when generating repo 2019-11-22 23:24:22 +01:00
Ettore Di Giacinto
e664f4f2cf
Return client from repository and install packages in the workers 2019-11-22 23:13:33 +01:00
Ettore Di Giacinto
75c1b66ae3
Add finalizers 2019-11-22 22:23:05 +01:00
Ettore Di Giacinto
dd0face86d
WIP 2019-11-22 21:01:53 +01:00
Ettore Di Giacinto
a2f23d3bf5
WIP 2019-11-22 21:01:38 +01:00
Ettore Di Giacinto
df78308e98
Annotate artifact metadata after compiling 2019-11-22 21:01:29 +01:00
Ettore Di Giacinto
d134690560
Expand conflicts #14 2019-11-21 14:50:15 +01:00
Ettore Di Giacinto
f107f47242
Prepare for 0.1 tag 2019-11-17 19:45:20 +01:00
Ettore Di Giacinto
6e662c75ce
Reuse docker backend in some Img functions 2019-11-17 18:38:57 +01:00
Ettore Di Giacinto
8ed2badefc
Add test for shared layer unpack 2019-11-17 15:59:52 +01:00
Ettore Di Giacinto
a39f596e8f
Do not compute diff when package is unpacked 2019-11-17 15:59:52 +01:00
Ettore Di Giacinto
d0b128000d
Clean build dirs for each package after build 2019-11-17 15:59:52 +01:00
Ettore Di Giacinto
dea8ea7f9a
Respect unpack also when compiling images #15 2019-11-17 15:59:48 +01:00
Geaaru
4ef86170d4 package: Fix typo 2019-11-16 17:29:58 +01:00
Geaaru
cf9bcf4726 tree: Fix repices_test 2019-11-16 17:29:58 +01:00
Geaaru
39e658dab6 gentoo/simpleparser: Use now GentooPackage methods for PV, P, PN, PVR 2019-11-16 17:29:58 +01:00
Geaaru
d96464cae4 gentoo/simpleparser: Integrate support of P* envs and avoid duplicate 2019-11-16 17:29:58 +01:00
Geaaru
8989d3f8d7 Add String implementation 2019-11-16 17:29:58 +01:00
Geaaru
758c820c5d logger: Permit to disable spinner at compile time 2019-11-16 17:29:58 +01:00
Geaaru
0e3952cbd8 gentoo/simpleparser: Fix parsing of RDEPEND on multiple level 2019-11-16 17:17:20 +01:00
Geaaru
aa829acf80 Integrate parsing of IUSE 2019-11-16 17:17:20 +01:00
Geaaru
d6d90edd67 Fix build of pkgstr 2019-11-16 17:17:20 +01:00
Geaaru
7ad4cb64e5 Integrate pkgs-checker pkg parser and review RDEPEND parsing 2019-11-16 17:17:20 +01:00
Ettore Di Giacinto
3499c0c580
Add test for compilation with version selection 2019-11-16 16:05:51 +01:00
Ettore Di Giacinto
6face9c980
Add package expansion during resolve deps 
Closes #2
 2019-11-16 14:58:50 +01:00
Ettore Di Giacinto
32dea6f7fd
Add Best() to solver for package selection based on version 2019-11-16 14:40:58 +01:00
Ettore Di Giacinto
602f5c68d5
Use emojis for icons 2019-11-16 14:16:59 +01:00
Ettore Di Giacinto
b59256337a
Drop Fit on recipe tests 
Signed-off-by: Ettore Di Giacinto <mudler@gentoo.org>
 2019-11-16 13:26:34 +01:00
Ettore Di Giacinto
588d06430c
Do not spin in the backend 
Signed-off-by: Ettore Di Giacinto <mudler@gentoo.org>
 2019-11-16 13:26:34 +01:00
Ettore Di Giacinto
3e850e9607
Adapt tests and fixup races in spinner 
Signed-off-by: Ettore Di Giacinto <mudler@gentoo.org>
 2019-11-16 13:26:33 +01:00
Ettore Di Giacinto
b2060c82e3
Make DB Switchable 
Fixes races conditions and make the DB Switchable. Also prepare inside
the CompilationSpec the tree of the deps to be built, and parallelize
only the building jobs.

Closes #7

Signed-off-by: Ettore Di Giacinto <mudler@gentoo.org>
 2019-11-16 13:26:33 +01:00
Ettore Di Giacinto
c95e0ed91d
Add revdep calculation 
Add CompilationSpecs type to handle slices of CompilationSpec, to
perform operation such as Unique() and Remove().

Add also dependencies and the spec associated to the Artifact, to track
how the artifact was generated.

Add revdeps compilation wrapper, and unit tests
 2019-11-15 18:11:26 +01:00
Ettore Di Giacinto
83fb1d1219
Add Matches() to pkg.Package 2019-11-15 18:04:46 +01:00
Ettore Di Giacinto
891daf4b71
Add package Revdeps expansion #12 2019-11-14 23:46:52 +01:00