Compare commits

..

39 Commits

Author SHA1 Message Date
Ettore Di Giacinto
a7b4ae67c9 Tag 0.16.6 2021-06-04 16:23:51 +02:00
Ettore Di Giacinto
68edfd58e7 Keep emitting plugin events 2021-06-04 12:08:57 +02:00
Ettore Di Giacinto
0658020c60 Update go.mod and vendor 2021-06-04 11:15:10 +02:00
Ettore Di Giacinto
c3b552103f Return a specific struct for the image
And drop imgworker
2021-06-04 11:09:41 +02:00
Darren Shepherd
796967cc9d Allow luet install to work inside a non-privileged container
This switches from using the containerd snapshotter to go-containerregistry
library which requires no additional privileges beyond root file system
access.

Signed-off-by: Ettore Di Giacinto <mudler@sabayon.org>
2021-06-04 10:29:57 +02:00
Ettore Di Giacinto
5cccc34f32 Tag 0.16.5 2021-06-01 22:54:10 +02:00
Ettore Di Giacinto
5ef1d04055 Create CONTRIBUTING.md 2021-06-01 17:13:23 +02:00
Itxaka
1bd4d520a4 Add 2 new events for image unpacking (#226)
* Reduce possibility of circular dependency

Just by adding an import for bus to anything in the helper dir, we would
run into a circular dependency due to how things are structured. That
means that we cannot set any events for unpacking or docker helper
pulling an image.

This commit tries to work around this by doing several things.
 - Remove full imports of the helper module by segmentating some modules
   into their own submodule, like docker or match so just using a small match
   function doesnt bring the whole module
 - Removing a simple function to check if a dir exists from importing
   the full helper module and instead write the function (5 lines)
 - Using logrus in the bus module instead of logger, which avoids a
   circular dependency

Signed-off-by: Itxaka <igarcia@suse.com>

* Add two new events for unpacking an image

Both pre and post unpacking an image

Signed-off-by: Itxaka <igarcia@suse.com>
2021-06-01 16:43:31 +02:00
Daniele Rondina
b12c7678d4 get_luet_root.sh: Fix installation on alpine/busybox env (#218)
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
2021-05-27 17:17:49 +02:00
Ettore Di Giacinto
32a99a4a49 Update readme with badges and a small logo 2021-05-26 17:57:41 +02:00
Ettore Di Giacinto
56e9c6f82e Tag 0.16.4 2021-05-25 14:55:55 +02:00
Ettore Di Giacinto
92ea69a2b9 Don't always generate artifact
Distinguish rather if we should generate artifact of the deps of a
package or not, and ensure that we just force artifact generation for
targets when computing the join image.

Fixes #217
2021-05-25 13:28:14 +02:00
Ettore Di Giacinto
838899aa83 Respect create-repo --force-push with metadata files
Don't force push package metadata with docker repository.

We were pushing regardless .metadata.yaml files even if packages were
not, also when calling create-repo without --force-push.
2021-05-25 12:13:23 +02:00
Ettore Di Giacinto
76695b2fc8 docker: check during extraction if image exists and pull if needed 2021-05-25 11:04:06 +02:00
Ettore Di Giacinto
5c84e5b0a7 Tag 0.16.3 2021-05-24 22:23:57 +02:00
Ettore Di Giacinto
06fa8b1c87 fixup: jointag index 2021-05-24 22:13:27 +02:00
Ettore Di Giacinto
ff153f367f fixup: enhance output of copy tag when cycling over packages 2021-05-24 21:35:22 +02:00
Ettore Di Giacinto
459676397c Enhance multi-stage output 2021-05-24 19:40:39 +02:00
Ettore Di Giacinto
93057fbf6d Respect PackageTargetOnly during multi-stage copy image generation 2021-05-24 19:40:17 +02:00
Ettore Di Giacinto
5e1a7c50df fixup: display hash of join image 2021-05-23 18:22:14 +02:00
Ettore Di Giacinto
0ceaf09615 fixup: propagate outputpath 2021-05-23 17:36:09 +02:00
Ettore Di Giacinto
0dc78ebe41 Tag 0.16.2 2021-05-22 14:49:58 +02:00
Ettore Di Giacinto
27c2e3c51f fixup: make sure the outputdir exists 2021-05-21 22:25:18 +02:00
Ettore Di Giacinto
e83f600ed3 Tag 0.16.1 2021-05-21 21:28:33 +02:00
Ettore Di Giacinto
6344e47eb3 fixup: resolve multistage and join also when building deps 2021-05-21 19:47:23 +02:00
Ettore Di Giacinto
8b1c5558b2 Tag 0.16.0 2021-05-21 16:10:28 +02:00
Ettore Di Giacinto
c277ac0f94 Add join keyword to generate parent image from final artifacts
A new keyword `join` is introduced to generate the parent image. It
takes precedence over a `requires` or a `image` already defined in a
spec.

It will generate all the artifacts from the packages listed and join
them in a single image which will be used as parent for the package
build process.

This is a change which invalidates priorly generated hashes.

Fixes #173
2021-05-21 14:52:48 +02:00
Ettore Di Giacinto
d8c8c2194f Tag 0.15.0 2021-05-20 13:53:53 +02:00
Ettore Di Giacinto
4494385f5b Fixup test rewording match 2021-05-20 11:17:39 +02:00
Ettore Di Giacinto
85a7968ecc Check if we have the image locally and extract directly it 2021-05-20 10:29:41 +02:00
Ettore Di Giacinto
1ba987b0f1 cmd/tree: consume Hashtree to display image hashes 2021-05-19 17:43:14 +02:00
Ettore Di Giacinto
c72b5be364 Update vendor 2021-05-19 17:34:54 +02:00
Ettore Di Giacinto
1ef18ed2c5 Add salted method for assertion hashing
- Add the spec Hash as salt for image hashes
- Add tests and adapt existing ones
- Use a signature to build a spec hash

Fixes: #207
2021-05-19 17:34:01 +02:00
Ettore Di Giacinto
4b1b711a5c Add single suite tests for spec and artifact
They went missing when the refactoring of the compiler occurred
2021-05-19 17:34:01 +02:00
Ettore Di Giacinto
7f047e4fc2 Multi-stage builds from hash images
Implement multi-stage copy from images part of the build cache of a
package.

Note, this is not the final images where are we copying files from, but
the underlying build container.

Skipping the test on img backend because it fails when pulling external
images during multi-stage build...

Fixes #190
2021-05-18 12:16:35 +02:00
Ettore Di Giacinto
356350f724 Tag 0.14.7 2021-05-17 19:11:44 +02:00
Ettore Di Giacinto
9d2ee1b760 Add hashtree and extract hash logic from compiler
Add unit tests, consume imagehashtree in compiler and cleanup

Fixes: #203
2021-05-17 15:22:08 +02:00
Ettore Di Giacinto
fd12227d53 plugins: Make execution fail if loaded plugins are erroring, print debug output on emitted responses
Now plugins failing to answer make execution fail, adapt tests
2021-05-17 15:17:03 +02:00
Ettore Di Giacinto
1e617b0c67 Add copy field
Initial implementation that just works with standard docker image
references

Related: #190
2021-05-17 15:16:15 +02:00
749 changed files with 2134 additions and 157124 deletions

53
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,53 @@
We love your input! We want to make contributing to this project as easy and transparent as possible, whether it's:
- Reporting a bug
- Discussing the current state of the code
- Submitting a fix
- Proposing new features
- Becoming a maintainer
## We Develop with Github
We use github to host code, to track issues and feature requests, as well as accept pull requests.
## Stay in touch
Join us in [slack](https://luet.slack.com/join/shared_invite/enQtOTQxMjcyNDQ0MDUxLWQ5ODVlNTI1MTYzNDRkYzkyYmM1YWE5YjM0NTliNDEzNmQwMTkxNDRhNDIzM2Y5NDBlOTZjZTYxYWQyNDE4YzY#/) and hang out with the community! It will be much easier to get started and do your first steps in contributing to the project.
## All Code Changes Happen Through Pull Requests
Pull requests are the best way to propose changes to the codebase. We actively welcome your pull requests:
1. Fork the repo you want to contribute to and create your branch from `develop`.
2. If you've added code that should be tested, add tests.
3. If you've changed APIs, update the [documentation](https://github.com/Luet-lab/docs).
4. Ensure the test suite passes.
5. Make sure your code lints.
6. Issue that pull request!
## Any contributions you make will be under the Software License of the repository
In short, when you submit code changes, your submissions are understood to be under the same License that covers the project. Feel free to contact the maintainers if that's a concern.
## Report bugs using Github's [issues](https://github.com/mudler/luet/issues)
We use GitHub issues to track public bugs. Report a bug by [opening a new issue](https://github.com/mudler/luet/issues/new); it's that easy!
## Write bug reports with detail, background, and sample code
Try to be as more descriptive as possible. When opening a new issue you will be prompted to choose between a bug or a feature request, with a small template to fill details with. Be specific!
**Great Bug Reports** tend to have:
- A quick summary and/or background
- Steps to reproduce
- Be specific!
- Give sample code if you can.
- What you expected would happen
- What actually happens
- Notes (possibly including why you think this might be happening, or stuff you tried that didn't work)
People *love* thorough bug reports.
## License
By contributing, you agree that your contributions will be licensed under the project Licenses.
## References
This document was adapted from the open-source contribution guidelines from https://gist.github.com/briandk/3d2e8b3ec8daf5a27a62

View File

@@ -1,19 +1,24 @@
<p align="center">
<img width=150 height=150 src="https://user-images.githubusercontent.com/2420543/119691600-0293d700-be4b-11eb-827f-49ff1174a07a.png">
</p>
# luet - Container-based Package manager
[![Docker Repository on Quay](https://quay.io/repository/luet/base/status "Docker Repository on Quay")](https://quay.io/repository/luet/base)
[![Go Report Card](https://goreportcard.com/badge/github.com/mudler/luet)](https://goreportcard.com/report/github.com/mudler/luet)
[![Build Status](https://travis-ci.org/mudler/luet.svg?branch=master)](https://travis-ci.org/mudler/luet)
[![Build and release on push](https://github.com/mudler/luet/actions/workflows/release.yml/badge.svg)](https://github.com/mudler/luet/actions/workflows/release.yml)
[![GoDoc](https://godoc.org/github.com/mudler/luet?status.svg)](https://godoc.org/github.com/mudler/luet)
[![codecov](https://codecov.io/gh/mudler/luet/branch/master/graph/badge.svg)](https://codecov.io/gh/mudler/luet)
[![asciicast](https://asciinema.org/a/388348.svg)](https://asciinema.org/a/388348)
Luet is a multi-platform Package Manager based off from containers - it uses Docker (and others) to build packages. It has zero dependencies and it is well suitable for "from scratch" environments. It can also version entire rootfs and enables delivery of OTA-alike updates, making it a perfect fit for the Edge computing era and IoT embedded devices.
It offers a simple [specfile format](https://luet-lab.github.io/docs/docs/concepts/packages/specfile/) in YAML notation to define both [packages](https://luet-lab.github.io/docs/docs/concepts/packages/) and [rootfs](https://luet-lab.github.io/docs/docs/concepts/packages/#package-layers). As it is based on containers, it can be also used to build stages for Linux From Scratch installations and it can build and track updates for those systems.
It is written entirely in Golang and where used as package manager, it can run in from scratch environment, with zero dependencies.
[![asciicast](https://asciinema.org/a/388348.svg)](https://asciinema.org/a/388348)
## In a glance
- Luet can reuse Gentoo's portage tree hierarchy, and it is heavily inspired from it.

View File

@@ -247,11 +247,12 @@ Build packages specifying multiple definition trees:
}
for _, sp := range toCalculate {
packs, err := luetCompiler.ComputeDepTree(sp)
ht := compiler.NewHashTree(generalRecipe.GetDatabase())
hashTree, err := ht.Query(luetCompiler, sp)
if err != nil {
errs = append(errs, err)
}
for _, p := range packs {
for _, p := range hashTree.Dependencies {
results.Packages = append(results.Packages,
PackageResult{
Name: p.Package.GetName(),

View File

@@ -22,7 +22,7 @@ import (
. "github.com/mudler/luet/pkg/config"
config "github.com/mudler/luet/pkg/config"
"github.com/mudler/luet/pkg/helpers"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
. "github.com/mudler/luet/pkg/logger"
"github.com/spf13/cobra"
@@ -47,7 +47,7 @@ var cleanupCmd = &cobra.Command{
LuetCfg.System.DatabasePath = dbpath
LuetCfg.System.Rootfs = rootfs
// Check if cache dir exists
if helpers.Exists(LuetCfg.GetSystem().GetSystemPkgsCacheDirPath()) {
if fileHelper.Exists(LuetCfg.GetSystem().GetSystemPkgsCacheDirPath()) {
files, err := ioutil.ReadDir(LuetCfg.GetSystem().GetSystemPkgsCacheDirPath())
if err != nil {

View File

@@ -25,6 +25,7 @@ import (
"github.com/marcsauter/single"
bus "github.com/mudler/luet/pkg/bus"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
extensions "github.com/mudler/cobra-extensions"
config "github.com/mudler/luet/pkg/config"
@@ -40,7 +41,7 @@ var Verbose bool
var LockedCommands = []string{"install", "uninstall", "upgrade"}
const (
LuetCLIVersion = "0.14.6"
LuetCLIVersion = "0.16.6"
LuetEnvPrefix = "LUET"
)
@@ -97,7 +98,7 @@ To build a package, from a tree definition:
plugin := viper.GetStringSlice("plugin")
bus.Manager.Load(plugin...).Register()
bus.Manager.Initialize(plugin...)
if len(bus.Manager.Plugins) != 0 {
Info(":lollipop:Enabled plugins:")
for _, p := range bus.Manager.Plugins {
@@ -252,7 +253,7 @@ func initConfig() {
}
homeDir := helpers.GetHomeDir()
if helpers.Exists(filepath.Join(pwdDir, ".luet.yaml")) || (homeDir != "" && helpers.Exists(filepath.Join(homeDir, ".luet.yaml"))) {
if fileHelper.Exists(filepath.Join(pwdDir, ".luet.yaml")) || (homeDir != "" && fileHelper.Exists(filepath.Join(homeDir, ".luet.yaml"))) {
viper.AddConfigPath(".")
if homeDir != "" {
viper.AddConfigPath(homeDir)

View File

@@ -96,9 +96,14 @@ func NewTreeImageCommand() *cobra.Command {
if err != nil {
Fatal("Error: " + err.Error())
}
asserts, err := luetCompiler.ComputeDepTree(spec)
for _, assertion := range asserts { //highly dependent on the order
ht := compiler.NewHashTree(reciper.GetDatabase())
hashtree, err := ht.Query(luetCompiler, spec)
if err != nil {
Fatal("Error: " + err.Error())
}
for _, assertion := range hashtree.Solution { //highly dependent on the order
//buildImageHash := imageRepository + ":" + assertion.Hash.BuildHash
currentPackageImageHash := imageRepository + ":" + assertion.Hash.PackageHash

View File

@@ -23,7 +23,7 @@ import (
"github.com/docker/docker/api/types"
"github.com/docker/go-units"
"github.com/mudler/luet/pkg/config"
"github.com/mudler/luet/pkg/helpers"
"github.com/mudler/luet/pkg/helpers/docker"
. "github.com/mudler/luet/pkg/logger"
"github.com/spf13/cobra"
@@ -77,7 +77,7 @@ func NewUnpackCommand() *cobra.Command {
RegistryToken: registryToken,
}
info, err := helpers.DownloadAndExtractDockerImage(temp, image, destination, auth, verify)
info, err := docker.DownloadAndExtractDockerImage(temp, image, destination, auth, verify)
if err != nil {
Error(err.Error())
os.Exit(1)

View File

@@ -7,7 +7,7 @@ fi
set -ex
export LUET_NOLOCK=true
LUET_VERSION=$(curl -s https://api.github.com/repos/mudler/luet/releases/latest | ( grep -oP '"tag_name": "\K(.*)(?=")' || echo "0.9.24" ))
LUET_VERSION=$(curl -s https://api.github.com/repos/mudler/luet/releases/latest | grep tag_name | awk '{ print $2 }' | sed -e 's/\"//g' -e 's/,//g' || echo "0.9.24" )
LUET_ROOTFS=${LUET_ROOTFS:-/}
LUET_DATABASE_PATH=${LUET_DATABASE_PATH:-/var/luet/db}
LUET_DATABASE_ENGINE=${LUET_DATABASE_ENGINE:-boltdb}

16
go.mod
View File

@@ -9,14 +9,17 @@ require (
github.com/asdine/storm v0.0.0-20190418133842-e0f77eada154
github.com/briandowns/spinner v1.12.1-0.20201220203425-e201aaea0a31
github.com/cavaliercoder/grab v1.0.1-0.20201108051000-98a5bfe305ec
github.com/containerd/cgroups v0.0.0-20200217135630-d732e370d46d // indirect
github.com/containerd/containerd v1.4.1-0.20201117152358-0edc412565dc
github.com/containerd/ttrpc v0.0.0-20200121165050-0be804eadb15 // indirect
github.com/containerd/typeurl v0.0.0-20200205145503-b45ef1f1f737 // indirect
github.com/crillab/gophersat v1.3.2-0.20201023142334-3fc2ac466765
github.com/docker/cli v0.0.0-20200227165822-2298e6a3fe24
github.com/docker/distribution v2.7.1+incompatible
github.com/docker/docker v20.10.0-beta1.0.20201110211921-af34b94a78a1+incompatible
github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c // indirect
github.com/docker/go-units v0.4.0
github.com/ecooper/qlearning v0.0.0-20160612200101-3075011a69fd
github.com/genuinetools/img v0.5.11
github.com/ghodss/yaml v1.0.0
github.com/google/go-containerregistry v0.2.1
github.com/google/renameio v1.0.0
@@ -32,7 +35,7 @@ require (
github.com/kyokomi/emoji v2.1.0+incompatible
github.com/logrusorgru/aurora v0.0.0-20190417123914-21d75270181e
github.com/marcsauter/single v0.0.0-20181104081128-f8bf46f26ec0
github.com/moby/buildkit v0.7.2
github.com/mitchellh/hashstructure/v2 v2.0.1
github.com/moby/sys/mount v0.2.0 // indirect
github.com/mudler/cobra-extensions v0.0.0-20200612154940-31a47105fe3d
github.com/mudler/docker-companion v0.4.6-0.20200418093252-41846f112d87
@@ -42,6 +45,7 @@ require (
github.com/onsi/gomega v1.10.3
github.com/opencontainers/go-digest v1.0.0
github.com/opencontainers/image-spec v1.0.1
github.com/opencontainers/runc v1.0.0-rc9.0.20200221051241-688cf6d43cc4 // indirect
github.com/otiai10/copy v1.2.1-0.20200916181228-26f84a0b1578
github.com/philopon/go-toposort v0.0.0-20170620085441-9be86dbd762f
github.com/pkg/errors v0.9.1
@@ -54,8 +58,8 @@ require (
go.uber.org/atomic v1.5.1 // indirect
go.uber.org/multierr v1.4.0
go.uber.org/zap v1.13.0
google.golang.org/grpc v1.29.1
gopkg.in/yaml.v2 v2.3.0
gotest.tools/v3 v3.0.2 // indirect
helm.sh/helm/v3 v3.3.4
)
@@ -63,9 +67,3 @@ require (
replace github.com/docker/docker => github.com/Luet-lab/moby v17.12.0-ce-rc1.0.20200605210607-749178b8f80d+incompatible
replace github.com/containerd/containerd => github.com/containerd/containerd v1.3.1-0.20200227195959-4d242818bf55
replace github.com/hashicorp/go-immutable-radix => github.com/tonistiigi/go-immutable-radix v0.0.0-20170803185627-826af9ccf0fe
replace github.com/jaguilar/vt100 => github.com/tonistiigi/vt100 v0.0.0-20190402012908-ad4c4a574305
replace github.com/opencontainers/runc => github.com/opencontainers/runc v1.0.0-rc9.0.20200221051241-688cf6d43cc4

113
go.sum
View File

@@ -23,8 +23,6 @@ cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiy
cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=
cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
github.com/AkihiroSuda/containerd-fuse-overlayfs v0.0.0-20200220082720-bb896865146c h1:2pWkaq3X2yFR5o5OI7QP0CYNNKtfE2ZCK3hMRaTkhmc=
github.com/AkihiroSuda/containerd-fuse-overlayfs v0.0.0-20200220082720-bb896865146c/go.mod h1:K4kx7xAA5JimeQCnN+dbeLlfaBxzZLaLiDD8lusFI8w=
github.com/Azure/azure-sdk-for-go v16.2.1+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
github.com/Azure/azure-sdk-for-go v35.0.0+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
github.com/Azure/azure-sdk-for-go v38.0.0+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
@@ -73,18 +71,15 @@ github.com/Masterminds/sprig/v3 v3.1.0 h1:j7GpgZ7PdFqNsmncycTHsLmVPf5/3wJtlgW9TN
github.com/Masterminds/sprig/v3 v3.1.0/go.mod h1:ONGMf7UfYGAbMXCZmQLy8x3lCDIPrEZE/rU8pmrbihA=
github.com/Masterminds/squirrel v1.4.0/go.mod h1:yaPeOnPG5ZRwL9oKdTsO/prlkPbXWZlRVMQ/gGlzIuA=
github.com/Masterminds/vcs v1.13.1/go.mod h1:N09YCmOQr6RLxC6UNHzuVwAdodYbbnycGHSmwVJjcKA=
github.com/Microsoft/go-winio v0.4.11/go.mod h1:VhR8bwka0BXejwEJY73c50VrPtXAaKcyvVC4A4RozmA=
github.com/Microsoft/go-winio v0.4.14/go.mod h1:qXqCSQ3Xa7+6tgxaGTIe4Kpcdsi+P8jBhyzoq1bpyYA=
github.com/Microsoft/go-winio v0.4.15-0.20190919025122-fc70bd9a86b5/go.mod h1:tTuCMEN+UleMWgg9dVx4Hu52b1bJo+59jBh3ajtinzw=
github.com/Microsoft/go-winio v0.4.15-0.20200113171025-3fe6c5262873 h1:93nQ7k53GjoMQ07HVP8g6Zj1fQZDDj7Xy2VkNNtvX8o=
github.com/Microsoft/go-winio v0.4.15-0.20200113171025-3fe6c5262873/go.mod h1:tTuCMEN+UleMWgg9dVx4Hu52b1bJo+59jBh3ajtinzw=
github.com/Microsoft/hcsshim v0.8.6/go.mod h1:Op3hHsoHPAvb6lceZHDtd9OkTew38wNoXnJs8iY7rUg=
github.com/Microsoft/hcsshim v0.8.7 h1:ptnOoufxGSzauVTsdE+wMYnCWA301PdoN4xg5oRdZpg=
github.com/Microsoft/hcsshim v0.8.7/go.mod h1:OHd7sQqRFrYd3RmSgbgji+ctCwkbq2wbEYNSzOYtcBQ=
github.com/MottainaiCI/simplestreams-builder v0.1.0 h1:A8KJN22Xkx7NUKC9/zWmd1UhIqRn3bdHo0wv/HsAHx8=
github.com/MottainaiCI/simplestreams-builder v0.1.0/go.mod h1:+Gbv6dg6TPHWq4oDjZY1vn978PLCEZ2hOu8kvn+S7t4=
github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ=
github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5/go.mod h1:lmUJ/7eu/Q8D7ML55dXQrVaamCz2vxCfdQBasLZfHKk=
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/PuerkitoBio/purell v1.0.0/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0=
github.com/PuerkitoBio/purell v1.1.0/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0=
@@ -108,7 +103,6 @@ github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuy
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/apache/thrift v0.0.0-20161221203622-b2a4d4ae21c7/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ=
github.com/apache/thrift v0.12.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ=
github.com/apache/thrift v0.13.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ=
github.com/apex/log v1.1.1 h1:BwhRZ0qbjYtTob0I+2M+smavV0kOC8XgcnGZcyL9liA=
@@ -186,46 +180,29 @@ github.com/cloudflare/cfssl v0.0.0-20180223231731-4e2dcbde5004 h1:lkAMpLVBDaj17e
github.com/cloudflare/cfssl v0.0.0-20180223231731-4e2dcbde5004/go.mod h1:yMWuSON2oQp+43nFtAV/uvKQIFpSPerB57DCt9t8sSA=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa/go.mod h1:zn76sxSg3SzpJ0PPJaLDCu+Bu0Lg3sKTORVIj19EIF8=
github.com/codahale/hdrhistogram v0.0.0-20160425231609-f8ad88b59a58/go.mod h1:sE/e/2PUdi/liOCUjSTXgM1o87ZssimdTWN964YiIeI=
github.com/codahale/hdrhistogram v0.0.0-20161010025455-3a0bb77429bd/go.mod h1:sE/e/2PUdi/liOCUjSTXgM1o87ZssimdTWN964YiIeI=
github.com/containerd/cgroups v0.0.0-20190919134610-bf292b21730f h1:tSNMc+rJDfmYntojat8lljbt1mgKNpTxUZJsSzJ9Y1s=
github.com/containerd/cgroups v0.0.0-20190919134610-bf292b21730f/go.mod h1:OApqhQ4XNSNC13gXIwDjhOQxjWa/NxkwZXJ1EvqT0ko=
github.com/containerd/cgroups v0.0.0-20200217135630-d732e370d46d h1:UKAt78F1OvM4ceTn1VvXuYuatXohsFU1eSI2IBtTw9g=
github.com/containerd/cgroups v0.0.0-20200217135630-d732e370d46d/go.mod h1:CStdkl05lBnJej94BPFoJ7vB8cELKXwViS+dgfW0/M8=
github.com/containerd/console v0.0.0-20180822173158-c12b1e7919c1/go.mod h1:Tj/on1eG8kiEhd0+fhSDzsPAFESxzBBvdyEgyryXffw=
github.com/containerd/console v0.0.0-20191206165004-02ecf6a7291e/go.mod h1:8Pf4gM6VEbTNRIT26AyyU7hxdQU3MvAvxVI0sc00XBE=
github.com/containerd/console v0.0.0-20191219165238-8375c3424e4d h1:VuiIRfgJ2M3vYEU0F6E5lg3+V0l9YpbGQr3jpZor5fo=
github.com/containerd/console v0.0.0-20191219165238-8375c3424e4d/go.mod h1:8Pf4gM6VEbTNRIT26AyyU7hxdQU3MvAvxVI0sc00XBE=
github.com/containerd/containerd v1.3.1-0.20200227195959-4d242818bf55 h1:FGO0nwSBESgoGCakj+w3OQXyrMLsz2omdo9b2UfG/BQ=
github.com/containerd/containerd v1.3.1-0.20200227195959-4d242818bf55/go.mod h1:bC6axHOhabU15QhwfG7w5PipXdVtMXFTttgp+kVtyUA=
github.com/containerd/continuity v0.0.0-20180921161001-7f53d412b9eb/go.mod h1:GL3xCUCBDV3CZiTSEKksMWbLE66hEyuu9qyDOOqM47Y=
github.com/containerd/continuity v0.0.0-20181001140422-bd77b46c8352/go.mod h1:GL3xCUCBDV3CZiTSEKksMWbLE66hEyuu9qyDOOqM47Y=
github.com/containerd/continuity v0.0.0-20190426062206-aaeac12a7ffc/go.mod h1:GL3xCUCBDV3CZiTSEKksMWbLE66hEyuu9qyDOOqM47Y=
github.com/containerd/continuity v0.0.0-20200107194136-26c1120b8d41/go.mod h1:Dq467ZllaHgAtVp4p1xUQWBrFXR9s/wyoTpG8zOJGkY=
github.com/containerd/continuity v0.0.0-20200228182428-0f16d7a0959c h1:8ahmSVELW1wghbjerVAyuEYD5+Dio66RYvSS0iGfL1M=
github.com/containerd/continuity v0.0.0-20200228182428-0f16d7a0959c/go.mod h1:Dq467ZllaHgAtVp4p1xUQWBrFXR9s/wyoTpG8zOJGkY=
github.com/containerd/fifo v0.0.0-20190226154929-a9fb20d87448/go.mod h1:ODA38xgv3Kuk8dQz2ZQXpnv/UZZUHUCL7pnLehbXgQI=
github.com/containerd/fifo v0.0.0-20191213151349-ff969a566b00 h1:lsjC5ENBl+Zgf38+B0ymougXFp0BaubeIVETltYZTQw=
github.com/containerd/fifo v0.0.0-20191213151349-ff969a566b00/go.mod h1:jPQ2IAeZRCYxpS/Cm1495vGFww6ecHmMk1YJH2Q5ln0=
github.com/containerd/go-cni v0.0.0-20200107172653-c154a49e2c75 h1:5Q5C6jDObSVpjeX8CuZ5yac8d/KIYuPzUHbUzdL+NFw=
github.com/containerd/go-cni v0.0.0-20200107172653-c154a49e2c75/go.mod h1:0mg8r6FCdbxvLDqCXwAx2rO+KA37QICjKL8+wHOG5OE=
github.com/containerd/go-runc v0.0.0-20180907222934-5a6d9f37cfa3/go.mod h1:IV7qH3hrUgRmyYrtgEeGWJfWbgcHL9CSRruz2Vqcph0=
github.com/containerd/go-runc v0.0.0-20200220073739-7016d3ce2328 h1:PRTagVMbJcCezLcHXe8UJvR1oBzp2lG3CEumeFOLOds=
github.com/containerd/go-runc v0.0.0-20200220073739-7016d3ce2328/go.mod h1:PpyHrqVs8FTi9vpyHwPwiNEGaACDxT/N/pLcvMSRA9g=
github.com/containerd/ttrpc v0.0.0-20190828154514-0e0f228740de h1:dlfGmNcE3jDAecLqwKPMNX6nk2qh1c1Vg1/YTzpOOF4=
github.com/containerd/ttrpc v0.0.0-20190828154514-0e0f228740de/go.mod h1:PvCDdDGpgqzQIzDW1TphrGLssLDZp2GuS+X5DkEJB8o=
github.com/containerd/ttrpc v0.0.0-20191028202541-4f1b8fe65a5c/go.mod h1:LPm1u0xBw8r8NOKoOdNMeVHSawSsltak+Ihv+etqsE8=
github.com/containerd/ttrpc v0.0.0-20200121165050-0be804eadb15 h1:+jgiLE5QylzgADj0Yldb4id1NQNRrDOROj7KDvY9PEc=
github.com/containerd/ttrpc v0.0.0-20200121165050-0be804eadb15/go.mod h1:UAxOpgT9ziI0gJrmKvgcZivgxOp8iFPSk8httJEt98Y=
github.com/containerd/typeurl v0.0.0-20180627222232-a93fcdb778cd h1:JNn81o/xG+8NEo3bC/vx9pbi/g2WI8mtP2/nXzu297Y=
github.com/containerd/typeurl v0.0.0-20180627222232-a93fcdb778cd/go.mod h1:Cm3kwCdlkCfMSHURc+r6fwoGH6/F1hH3S4sg0rLFWPc=
github.com/containerd/typeurl v0.0.0-20190911142611-5eb25027c9fd/go.mod h1:GeKYzf2pQcqv7tJ0AoCuuhtnqhva5LNU3U+OyKxxJpk=
github.com/containerd/typeurl v0.0.0-20200205145503-b45ef1f1f737 h1:HovfQDS/K3Mr7eyS0QJLxE1CbVUhjZCl6g3OhFJgP1o=
github.com/containerd/typeurl v0.0.0-20200205145503-b45ef1f1f737/go.mod h1:TB1hUtrpaiO88KEK56ijojHS1+NeF0izUACaJW2mdXg=
github.com/containernetworking/cni v0.7.1 h1:fE3r16wpSEyaqY4Z4oFrLMmIGfBYIKpPrHK31EJ9FzE=
github.com/containernetworking/cni v0.7.1/go.mod h1:LGwApLUm2FpoOfxTDEeq8T9ipbpZ61X79hmU3w8FmsY=
github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk=
github.com/coreos/clair v0.0.0-20180919182544-44ae4bc9590a/go.mod h1:uXhHPWAoRqw0jJc2f8RrPCwRhIo9otQ8OEWUFtpCiwA=
github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
github.com/coreos/etcd v3.3.13+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=
@@ -266,39 +243,27 @@ github.com/dgrijalva/jwt-go v0.0.0-20170104182250-a601269ab70c/go.mod h1:E3ru+11
github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no=
github.com/dnaeon/go-vcr v1.0.1/go.mod h1:aBB1+wY4s93YsC3HHjMBMrwTj2R9FHDzUr9KyGc8n1E=
github.com/docker/cli v0.0.0-20180920165730-54c19e67f69c/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/cli v0.0.0-20191017083524-a8ff7f821017/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/cli v0.0.0-20200130152716-5d0cf8839492 h1:FwssHbCDJD025h+BchanCwE1Q8fyMgqDr2mOQAWOLGw=
github.com/docker/cli v0.0.0-20200130152716-5d0cf8839492/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/cli v0.0.0-20200227165822-2298e6a3fe24 h1:bjsfAvm8BVtvQFxV7TYznmKa35J8+fmgrRJWvcS3yJo=
github.com/docker/cli v0.0.0-20200227165822-2298e6a3fe24/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/distribution v0.0.0-20180920194744-16128bbac47f/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/distribution v0.0.0-20191216044856-a8371794149d/go.mod h1:0+TTO4EOBfRPhZXAeF1Vu+W3hHZ8eLp8PgKVZlcvtFY=
github.com/docker/distribution v0.0.0-20200223014041-6b972e50feee/go.mod h1:xgJxuOjyp98AvnpRTR1+lGOqQ493ylRnRPmewD5GWtc=
github.com/docker/distribution v2.7.1-0.20190205005809-0d3efadf0154+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/distribution v2.7.1+incompatible h1:a5mlkVzth6W5A4fOsS3D2EO5BUmsJpcB+cRlLU7cSug=
github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker-ce v0.0.0-20180924210327-f53bd8bb8e43/go.mod h1:l1FUGRYBvbjnZ8MS6A2xOji4aZFlY/Qmgz7p4oXH7ac=
github.com/docker/docker-credential-helpers v0.6.0/go.mod h1:WRaJzqw3CTB9bk10avuGsjVBZsD05qeibJ1/TYlvc0Y=
github.com/docker/docker-credential-helpers v0.6.1/go.mod h1:WRaJzqw3CTB9bk10avuGsjVBZsD05qeibJ1/TYlvc0Y=
github.com/docker/docker-credential-helpers v0.6.3 h1:zI2p9+1NQYdnG6sMU26EX4aVGlqbInSQxQXLvzJ4RPQ=
github.com/docker/docker-credential-helpers v0.6.3/go.mod h1:WRaJzqw3CTB9bk10avuGsjVBZsD05qeibJ1/TYlvc0Y=
github.com/docker/go v1.5.1-1.0.20160303222718-d30aec9fd63c h1:lzqkGL9b3znc+ZUgi7FlLnqjQhcXxkNM/quxIjBVMD0=
github.com/docker/go v1.5.1-1.0.20160303222718-d30aec9fd63c/go.mod h1:CADgU4DSXK5QUlFslkQu2yW2TKzFZcXq/leZfM0UH5Q=
github.com/docker/go-connections v0.0.0-20180821093606-97c2040d34df/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec=
github.com/docker/go-connections v0.3.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec=
github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ=
github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec=
github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c h1:+pKlWGMw7gf6bQ+oDZB4KHQFypsfjYlq/C4rfL7D3g8=
github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c/go.mod h1:Uw6UezgYA44ePAFQYUehOuCzmy5zmg/+nl2ZfMWGkpA=
github.com/docker/go-metrics v0.0.0-20180209012529-399ea8c73916 h1:yWHOI+vFjEsAakUTSrtqc/SAHrhSkmn48pqjidZX3QA=
github.com/docker/go-metrics v0.0.0-20180209012529-399ea8c73916/go.mod h1:/u0gXw0Gay3ceNrsHubL3BtdOL2fHf93USgMTe0W5dI=
github.com/docker/go-units v0.3.1/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/docker/go-units v0.3.3/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/docker/go-units v0.4.0 h1:3uh0PgVws3nIA0Q+MwDC8yjEPf9zjRfZZWXZYDct3Tw=
github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/docker/libnetwork v0.8.0-dev.2.0.20200226230617-d8334ccdb9be h1:GJzljYRqZapOwyfeRyExF2/5qfv8f1feNDMiz0hmRPY=
github.com/docker/libnetwork v0.8.0-dev.2.0.20200226230617-d8334ccdb9be/go.mod h1:93m0aTqz6z+g32wla4l4WxTrdtvBRmVzYRkYvasA5Z8=
github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1/go.mod h1:cyGadeNEkKy96OOhEzfZl+yxihPEzKnqJwvfuSUqbZE=
github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7 h1:UhxFibDNY/bfvqU5CAUmr9zpesgbU6SWc8/B4mflAE4=
github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7/go.mod h1:cyGadeNEkKy96OOhEzfZl+yxihPEzKnqJwvfuSUqbZE=
@@ -327,7 +292,6 @@ github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d/go.mod h1:ZZM
github.com/fatih/camelcase v1.0.0/go.mod h1:yN2Sb0lFhZJUdVvtELVWefmrXpuZESvPmqwoZc+/fpc=
github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/fernet/fernet-go v0.0.0-20180830025343-9eac43b88a5e/go.mod h1:2H9hjfbpSMHwY503FclkV/lZTBh2YlOmLLSda12uL8c=
github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k=
github.com/franela/goblin v0.0.0-20200105215937-c9ffbefa60db/go.mod h1:7dvUGVsVBjqR7JHJk0brhHOZYGmfBYOrK0ZhYMEtBr4=
github.com/franela/goreq v0.0.0-20171204163338-bcd34c9993f8/go.mod h1:ZhphrRTfi2rbfLwlschooIH4+wKKDR4Pdxhh+TRoA20=
@@ -338,10 +302,6 @@ github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4
github.com/fsouza/go-dockerclient v1.6.4 h1:B+L+1lz1LUrNgEUUh8PSG76s70EYC49ssv2xvTefTMM=
github.com/fsouza/go-dockerclient v1.6.4/go.mod h1:GOdftxWLWIbIWKbIMDroKFJzPdg6Iw7r+jX1DDZdVsA=
github.com/garyburd/redigo v0.0.0-20150301180006-535138d7bcd7/go.mod h1:NR3MbYisc3/PwhQ00EMzDiPmrwpPxAn5GI05/YaO1SY=
github.com/genuinetools/img v0.5.11 h1:qIP3nVgRYbXmtSNMjmcdx+TBzWjZS03AdqTNBf6QtGY=
github.com/genuinetools/img v0.5.11/go.mod h1:m58lsi0AC97XSTXBTKpqZE1zCZpMCg9nLjikvgLCo0A=
github.com/genuinetools/pkg v0.0.0-20180910213200-1c141f661797/go.mod h1:XTcrCYlXPxnxL2UpnwuRn7tcaTn9HAhxFoFJucootk8=
github.com/genuinetools/reg v0.16.0/go.mod h1:12Fe9EIvK3dG/qWhNk5e9O96I8SGmCKLsJ8GsXUbk+Y=
github.com/ghodss/yaml v0.0.0-20150909031657-73d445a93680/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
@@ -424,19 +384,15 @@ github.com/godbus/dbus v0.0.0-20190422162347-ade71ed3457e/go.mod h1:bBOAhwG1umN6
github.com/godbus/dbus/v5 v5.0.3 h1:ZqHaoEF7TBzh4jzPmqVhE/5A1z9of6orkAe5uHoAeME=
github.com/godbus/dbus/v5 v5.0.3/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
github.com/godror/godror v0.13.3/go.mod h1:2ouUT4kdhUBk7TAkHWD4SN0CdI0pgEQbo8FVHhbSKWg=
github.com/gofrs/flock v0.7.0/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU=
github.com/gofrs/flock v0.7.1 h1:DP+LD/t0njgoPBvT5MJLeliUIVQR03hiKR6vezdwHlc=
github.com/gofrs/flock v0.7.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU=
github.com/gogo/googleapis v1.1.0/go.mod h1:gf4bu3Q80BeJ6H1S1vYPm8/ELATdvryBaNFGgqEef3s=
github.com/gogo/googleapis v1.3.2 h1:kX1es4djPJrsDhY7aZKJy7aZasdcB5oSOEphMjSB53c=
github.com/gogo/googleapis v1.3.2/go.mod h1:5YRNX2z1oM5gXdAkurHa942MDgEJyk02w4OecKY87+c=
github.com/gogo/protobuf v1.0.0/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.0/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.1 h1:/s5zKNz0uPFCZ5hddgPdo2TK2TVrUNMn0OOX8/aZMTE=
github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
github.com/gogo/protobuf v1.2.2-0.20190723190241-65acae22fc9d/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=
github.com/gogo/protobuf v1.3.0/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=
github.com/gogo/protobuf v1.3.1 h1:DqDEcV5aeaTmdFBePNpYsp3FlcVH/2ISVVM9Qf8PSls=
github.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=
github.com/golang-sql/civil v0.0.0-20190719163853-cb61b32ac6fe h1:lXe2qZdvpiX5WZkZR4hgp4KJVfY3nMkvmwbVkpv1rVY=
@@ -450,7 +406,6 @@ github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4er
github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e h1:1r7pUrabqp18hOBcwBwiTsbnFeTZHV9eER/QT5JVZxY=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/lint v0.0.0-20180702182130-06c8688daad7/go.mod h1:tluoj9z5200jBnyusfRPU2LqT6J+DAorxEvtC7LHB+E=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=
@@ -504,8 +459,6 @@ github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hf
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/renameio v1.0.0 h1:xhp2CnJmgQmpJU4RY8chagahUq5mbPPAbiSQstKpVMA=
github.com/google/renameio v1.0.0/go.mod h1:t/HQoYBZSsWSNK35C6CO/TpPLDVWvxOHboWUAweKUpk=
github.com/google/shlex v0.0.0-20150127133951-6f45313302b9 h1:JM174NTeGNJ2m/oLH3UOWOvWQQKd+BoL3hcSCUWFLt0=
github.com/google/shlex v0.0.0-20150127133951-6f45313302b9/go.mod h1:RpwtwJQFrIEPstU94h88MWPXP2ektJZ8cZ0YntAmXiE=
github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.1.1 h1:Gkbcsh/GbpXz7lPftLA3P6TYMwjCLYm83jiFQZF/3gY=
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
@@ -531,16 +484,12 @@ github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c/go.mod h1:E7qHFY
github.com/gorilla/websocket v1.4.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/gosuri/uitable v0.0.4/go.mod h1:tKR86bXuXPZazfOTG1FIzvjIdXzd0mo4Vtn16vt0PJo=
github.com/gotestyourself/gotestyourself v2.2.0+incompatible/go.mod h1:zZKM6oeNM8k+FRljX1mnzVYeS8wiGgQyvST1/GafPbY=
github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA=
github.com/grpc-ecosystem/go-grpc-middleware v1.0.0/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs=
github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs=
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk=
github.com/grpc-ecosystem/grpc-gateway v1.5.0/go.mod h1:RSKVYQBd5MCa4OVpNdGskqpgL2+G+NZTnrVHpWWfpdw=
github.com/grpc-ecosystem/grpc-gateway v1.9.0/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
github.com/grpc-ecosystem/grpc-gateway v1.9.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
github.com/grpc-ecosystem/grpc-opentracing v0.0.0-20180507213350-8e809c8a8645 h1:MJG/KsmcqMwFAkh8mTnAwhyKoB+sTAnY4CACC110tbU=
github.com/grpc-ecosystem/grpc-opentracing v0.0.0-20180507213350-8e809c8a8645/go.mod h1:6iZfnjpejD4L/4DwD7NryNaJyCQdzwWwH2MWhCA90Kw=
github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed h1:5upAirOpQc1Q53c0bnx2ufif5kANL7bfZWcc6VJWJd8=
github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed/go.mod h1:tMWxXQ9wFIaZeTI9F+hmhFiGpFmhOHzyShyFUhRm0H4=
github.com/hashicorp/consul/api v1.1.0/go.mod h1:VmuI/Lkw1nC05EYQWNKwWGbkg+FbDBtguAZLlVdkD9Q=
@@ -551,6 +500,7 @@ github.com/hashicorp/errwrap v0.0.0-20141028054710-7554cd9344ce/go.mod h1:YH+1FK
github.com/hashicorp/errwrap v1.0.0 h1:hLrqtEDnRye3+sgx6z4qVLNuviH3MR5aQ0ykNJa/UYA=
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60=
github.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM=
github.com/hashicorp/go-multierror v0.0.0-20161216184304-ed905158d874/go.mod h1:JMRHfdO9jKNzS/+BTlxCjKNQHg/jZAft8U7LloJvN7I=
github.com/hashicorp/go-multierror v1.0.0 h1:iVjPR7a6H0tWELX5NxNe7bYopibicUzc7uPribsnS6o=
@@ -558,6 +508,7 @@ github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHh
github.com/hashicorp/go-rootcerts v1.0.0/go.mod h1:K6zTfqpRlCUIjkwsN4Z+hiSfzSTQa6eBIzfwKfwNnHU=
github.com/hashicorp/go-sockaddr v1.0.0/go.mod h1:7Xibr9yA9JjQq1JpNB2Vw7kxv8xerXegt+ozgdvDeDU=
github.com/hashicorp/go-syslog v1.0.0/go.mod h1:qPfqrKkXGihmCqbJM2mZgkZGvKG1dFdvsLplgctolz4=
github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hashicorp/go-version v1.2.0 h1:3vNe/fWF5CBgRIguda1meWhsZHy3m8gCJ5wx+dIzX/E=
github.com/hashicorp/go-version v1.2.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
@@ -575,8 +526,6 @@ github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO
github.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ=
github.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2pPBoIllUwCN7I=
github.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc=
github.com/hashicorp/uuid v0.0.0-20160311170451-ebb0a03e909c h1:nQcv325vxv2fFHJsOt53eSRf1eINt6vOdYUFfXs4rgk=
github.com/hashicorp/uuid v0.0.0-20160311170451-ebb0a03e909c/go.mod h1:fHzc09UnyJyqyW+bFuq864eh+wC7dj65aXmXLRe5to0=
github.com/heroku/docker-registry-client v0.0.0-20181004091502-47ecf50fd8d4 h1:44WMsEqwiYnpHA3E4Rg1K379MH5iZllp2sO5nzXARI0=
github.com/heroku/docker-registry-client v0.0.0-20181004091502-47ecf50fd8d4/go.mod h1:ceV82AfTGFCOL/b0cdpP54uKVSL1Gef0TBSTGFDuqyY=
github.com/hpcloud/tail v1.0.0 h1:nfCOvKYfkgYP8hkirhJocXT2+zOD8yUNjXaWfTlyFKI=
@@ -586,14 +535,11 @@ github.com/huandu/xstrings v1.3.1/go.mod h1:y5/lhBue+AyNmUVz9RLU9xbLR0o4KIIExikq
github.com/hudl/fargo v1.3.0/go.mod h1:y3CKSmjA+wD2gak7sUSXTAoopbhU08POFhmITJgmKTg=
github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/imdario/mergo v0.3.5/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
github.com/imdario/mergo v0.3.7/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
github.com/imdario/mergo v0.3.8 h1:CGgOkSJeqMRmt0D9XLWExdT4m4F1vd3FV3VPt+0VxkQ=
github.com/imdario/mergo v0.3.8/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
github.com/influxdata/influxdb1-client v0.0.0-20191209144304-8bf82d3c094d/go.mod h1:qj24IKcXYK6Iy9ceXlo3Tc+vtHo9lIhSX5JddghvEPo=
github.com/ishidawataru/sctp v0.0.0-20191218070446-00ab2ac2db07 h1:rw3IAne6CDuVFlZbPOkA7bhxlqawFh7RJJ+CejfMaxE=
github.com/ishidawataru/sctp v0.0.0-20191218070446-00ab2ac2db07/go.mod h1:co9pwDoBCm1kGxawmb4sPq0cSIOOWNPT4KnHotMP1Zg=
github.com/jedib0t/go-pretty v4.3.0+incompatible h1:CGs8AVhEKg/n9YbUenWmNStRW2PHJzaeDodcfvRAbIo=
github.com/jedib0t/go-pretty v4.3.0+incompatible/go.mod h1:XemHduiw8R651AF9Pt4FwCTKeG3oo7hrHJAoznj9nag=
github.com/jedib0t/go-pretty/v6 v6.0.5 h1:oOo0/jSb3NEYKT6l1hhFXoX2UZnkanMuCE2DVT1mqnE=
@@ -725,9 +671,8 @@ github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrk
github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI=
github.com/mitchellh/go-wordwrap v1.0.0/go.mod h1:ZXFpozHsX6DPmq2I0TCekCxypsnAUbP2oI0UX1GXzOo=
github.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS42BGNg=
github.com/mitchellh/hashstructure v0.0.0-20170609045927-2bca23e0e452/go.mod h1:QjSHrPWS+BGUVBYkbTZWEnOh3G1DutKwClXU/ABz6AQ=
github.com/mitchellh/hashstructure v1.0.0 h1:ZkRJX1CyOoTkar7p/mLS5TZU4nJ1Rn/F8u9dGS02Q3Y=
github.com/mitchellh/hashstructure v1.0.0/go.mod h1:QjSHrPWS+BGUVBYkbTZWEnOh3G1DutKwClXU/ABz6AQ=
github.com/mitchellh/hashstructure/v2 v2.0.1 h1:L60q1+q7cXE4JeEJJKMnh2brFIe3rZxCihYAB61ypAY=
github.com/mitchellh/hashstructure/v2 v2.0.1/go.mod h1:MG3aRVU/N29oo/V/IhBX8GR/zz4kQkprJgF2EVszyDE=
github.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY=
github.com/mitchellh/mapstructure v0.0.0-20150613213606-2caf8efc9366/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
@@ -736,8 +681,6 @@ github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh
github.com/mitchellh/osext v0.0.0-20151018003038-5e2d6d41470f/go.mod h1:OkQIRizQZAeMln+1tSwduZz7+Af5oFlKirV/MSYes2A=
github.com/mitchellh/reflectwalk v1.0.0 h1:9D+8oIskB4VJBN5SFlmc27fSlIBZaov1Wpk/IfikLNY=
github.com/mitchellh/reflectwalk v1.0.0/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw=
github.com/moby/buildkit v0.7.2 h1:wp4R0QMXSqwjTJKhhWlJNOCSQ/OVPnsCf3N8rs09+vQ=
github.com/moby/buildkit v0.7.2/go.mod h1:D3DN/Nl4DyMH1LkwpRUJuoghqdigdXd1A6HXt5aZS40=
github.com/moby/sys/mount v0.2.0 h1:WhCW5B355jtxndN5ovugJlMFJawbUODuW8fSnEH6SSM=
github.com/moby/sys/mount v0.2.0/go.mod h1:aAivFE2LB3W4bACsUXChRHQ0qKWsetY4Y9V7sxOougM=
github.com/moby/sys/mountinfo v0.4.0 h1:1KInV3Huv18akCu58V7lzNlt+jFmqlu1EaErnEHE/VM=
@@ -751,7 +694,6 @@ github.com/modern-go/reflect2 v1.0.1 h1:9f412s+6RmYXLWZSEzVVgPGK7C2PphHj5RJrvfx9
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826 h1:RWengNIwukTxcDr9M+97sNutRR1RKhG96O6jWumTTnw=
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826/go.mod h1:TaXosZuwdSHYgviHp1DAtfrULt5eUgsSMsZf+YrPgl8=
github.com/morikuni/aec v0.0.0-20170113033406-39771216ff4c/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/mudler/cobra-extensions v0.0.0-20200612154940-31a47105fe3d h1:fKh+rvwZQCA+TPzK0EMwwbqhjvRHaQ6H8AsVU1Wt+NQ=
@@ -790,7 +732,6 @@ github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+W
github.com/onsi/ginkgo v1.8.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.10.1 h1:q/mM8GF/n0shIN8SaAZ0V+jnLPzen6WIVZdiwrRlMlo=
github.com/onsi/ginkgo v1.10.1/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.10.3/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.11.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.12.0/go.mod h1:oUhWkIvk5aDxtKvDDuw8gItl8pKl42LzjC9KZE0HfGg=
github.com/onsi/ginkgo v1.12.1 h1:mFwc4LvZ0xpSvDZ3E+k8Yte0hLOMxXUlP+yXtJqkYfQ=
@@ -798,7 +739,6 @@ github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108
github.com/onsi/ginkgo v1.14.2 h1:8mVmC9kjFFmA8H4pKMUhcblgifdkOIXPvbhN1T36q1M=
github.com/onsi/ginkgo v1.14.2/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9klQyY=
github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA=
github.com/onsi/gomega v1.4.2/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/onsi/gomega v1.5.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/onsi/gomega v1.7.0 h1:XPnZz8VVBHjVsy1vzJmRwIcSwiUO+JFfrv/xGiigmME=
@@ -822,20 +762,16 @@ github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3I
github.com/opencontainers/image-spec v1.0.0/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
github.com/opencontainers/image-spec v1.0.1 h1:JMemWkRwHx4Zj+fVxWoMCFm/8sYGGrUVojFA6h/TRcI=
github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
github.com/opencontainers/runc v0.0.0-20190115041553-12f6a991201f/go.mod h1:qT5XzbpPznkRYVz/mWwUaVBUv2rmF59PVA73FjuZG0U=
github.com/opencontainers/runc v0.1.1/go.mod h1:qT5XzbpPznkRYVz/mWwUaVBUv2rmF59PVA73FjuZG0U=
github.com/opencontainers/runc v1.0.0-rc9.0.20200221051241-688cf6d43cc4 h1:JhRvjyrjq24YPSDS0MQo9KJHQh95naK5fYl9IT+dzPM=
github.com/opencontainers/runc v1.0.0-rc9.0.20200221051241-688cf6d43cc4/go.mod h1:qT5XzbpPznkRYVz/mWwUaVBUv2rmF59PVA73FjuZG0U=
github.com/opencontainers/runtime-spec v0.1.2-0.20190507144316-5b71a03e2700/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-spec v1.0.1 h1:wY4pOY8fBdSIvs9+IDHC55thBuEulhzfSgKeC1yFvzQ=
github.com/opencontainers/runtime-spec v1.0.1/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-tools v0.0.0-20181011054405-1d69bd0f9c39/go.mod h1:r3f7wjNzSs2extwzU3Y+6pKfobzPh+kKFJ3ofN+3nfs=
github.com/opencontainers/selinux v1.3.2 h1:DR4lL9SYVjgcTZKEZIncvDU06fKSc/eygjmNGOA3E1s=
github.com/opencontainers/selinux v1.3.2/go.mod h1:yTcKuYAh6R95iDpefGLQaPaRwJFwyzAJufJyiTt7s0g=
github.com/opentracing-contrib/go-observer v0.0.0-20170622124052-a52f23424492/go.mod h1:Ngi6UdF0k5OKD5t5wlmGhe/EDKPoUM3BXZSSfIuJbis=
github.com/opentracing-contrib/go-stdlib v0.0.0-20171029140428-b1a47cfbdd75/go.mod h1:PLldrQSroqzH70Xl+1DQcGnefIbqsKR7UDaiux3zV+w=
github.com/opentracing-contrib/go-stdlib v0.0.0-20180702182724-07a764486eb1 h1:gmB1XmLjI0RXG8rJCP0PK6g8rwhX8COSGFTiOgJ4Wx4=
github.com/opentracing-contrib/go-stdlib v0.0.0-20180702182724-07a764486eb1/go.mod h1:PLldrQSroqzH70Xl+1DQcGnefIbqsKR7UDaiux3zV+w=
github.com/opentracing/basictracer-go v1.0.0/go.mod h1:QfBfYuafItcjQuMwinw9GhYKwFXS9KnPs5lxoYwgW74=
github.com/opentracing/opentracing-go v0.0.0-20171003133519-1361b9cd60be/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=
github.com/opentracing/opentracing-go v1.0.2/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=
github.com/opentracing/opentracing-go v1.1.0 h1:pWlfV3Bxv7k65HYwkikxat0+s3pV4bsqf19k25Ur8rU=
github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=
@@ -860,7 +796,6 @@ github.com/pelletier/go-toml v1.2.0 h1:T5zMGML61Wp+FlcbWjRDT7yAxhJNAiPPLOFECq181
github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
github.com/performancecopilot/speed v3.0.0+incompatible/go.mod h1:/CLtqpZ5gBg1M9iaPbIdPPGyKcA8hKdoy6hAWba7Yac=
github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU=
github.com/peterhellberg/link v1.0.0/go.mod h1:gtSlOT4jmkY8P47hbTc8PTgiDDWpdPbFYl75keYyBB8=
github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2/go.mod h1:iIss55rKnNBTvrwdmkUpLnDpZoAHvWaiq5+iMmen4AE=
github.com/philopon/go-toposort v0.0.0-20170620085441-9be86dbd762f h1:WyCn68lTiytVSkk7W1K9nBiSGTSRlUOdyTnSjwrIlok=
github.com/philopon/go-toposort v0.0.0-20170620085441-9be86dbd762f/go.mod h1:/iRjX3DdSK956SzsUdV55J+wIsQ+2IBWmBrB4RvZfk4=
@@ -879,7 +814,6 @@ github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZN
github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI=
github.com/pquerna/cachecontrol v0.0.0-20171018203845-0dec1b30a021/go.mod h1:prYjPmNq4d1NPVmpShWobRqXY3q7Vp+80DqgxxUrUIA=
github.com/prometheus/client_golang v0.0.0-20180209125602-c332b6f63c06/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v0.0.0-20180924113449-f69c853d21c1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v0.9.0-pre1.0.20180209125602-c332b6f63c06/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v0.9.3-0.20190127221311-3c4408c8b829/go.mod h1:p2iRAGwDERtqlqzRXnrOVns+ignqQo//hLXqYxZYVNs=
@@ -896,7 +830,6 @@ github.com/prometheus/client_model v0.1.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6T
github.com/prometheus/client_model v0.2.0 h1:uq5h0d+GuxiXLJLNABMgp2qUWDPiLvgCzz2dUR+/W/M=
github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/common v0.0.0-20180110214958-89604d197083/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro=
github.com/prometheus/common v0.0.0-20180801064454-c7de2306084e/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro=
github.com/prometheus/common v0.0.0-20181113130724-41aa239b4cce/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro=
github.com/prometheus/common v0.2.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
@@ -904,7 +837,6 @@ github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y8
github.com/prometheus/common v0.7.0 h1:L+1lyG48J1zAQXA3RBX/nG/B3gjlHq0zTt2tlbJLyCY=
github.com/prometheus/common v0.7.0/go.mod h1:DjGbpBbp5NYNiECxcL/VnbXCCaQpKd3tt26CguLLsqA=
github.com/prometheus/procfs v0.0.0-20180125133057-cb4147076ac7/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.0-20180920065004-418d78d0b9a7/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.0-20190117184657-bf6a532e95b1/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
@@ -940,11 +872,8 @@ github.com/schollz/progressbar/v3 v3.7.1/go.mod h1:CG/f0JmacksUc6TkZToO7tVq4t03z
github.com/sclevine/spec v1.2.0/go.mod h1:W4J29eT/Kzv7/b9IWLB055Z+qvVC9vt0Arko24q7p+U=
github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc=
github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
github.com/serialx/hashring v0.0.0-20190422032157-8b2912629002/go.mod h1:/yeG0My1xr/u+HZrFQ1tOQQQQrOawfyMUH13ai5brBc=
github.com/shurcooL/httpfs v0.0.0-20171119174359-809beceb2371/go.mod h1:ZY1cvUeJuFPAdZ/B6v7RHavJWZn2YPVFQ1OSXhCGOkg=
github.com/shurcooL/sanitized_anchor_name v1.0.0 h1:PdmoCO6wvbs+7yrJyMORt4/BmY5IYyJwS/kOiWx8mHo=
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/sirupsen/logrus v1.0.3/go.mod h1:pMByvHTf9Beacp5x1UXfOR9xyW/9antXMhjMPG0dEzc=
github.com/sirupsen/logrus v1.0.4-0.20170822132746-89742aefa4b2/go.mod h1:pMByvHTf9Beacp5x1UXfOR9xyW/9antXMhjMPG0dEzc=
github.com/sirupsen/logrus v1.0.6/go.mod h1:pMByvHTf9Beacp5x1UXfOR9xyW/9antXMhjMPG0dEzc=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
@@ -1007,13 +936,11 @@ github.com/spf13/viper v1.7.1/go.mod h1:8WkrPz2fc9jxqZNCJI/76HCieCp4Q8HaLFoCha5q
github.com/streadway/amqp v0.0.0-20190404075320-75d898a42a94/go.mod h1:AZpEONHx3DKn8O/DFsRAY58/XVQiIPMTMB1SddzLXVw=
github.com/streadway/amqp v0.0.0-20190827072141-edfb9018d271/go.mod h1:AZpEONHx3DKn8O/DFsRAY58/XVQiIPMTMB1SddzLXVw=
github.com/streadway/handy v0.0.0-20190108123426-d5acb3125c2a/go.mod h1:qNTQ5P5JnDBl6z3cMAg/SywNDC5ABu5ApDIw6lUbRmI=
github.com/stretchr/objx v0.0.0-20180129172003-8a3f7159479f/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.2.0 h1:Hbg2NidpLE8veEBkEZTL3CvlkUIVzuU9jDplZO54c48=
github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE=
github.com/stretchr/testify v0.0.0-20151208002404-e3a8ff8ce365/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v0.0.0-20180303142811-b89eecf5ca5d/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
@@ -1024,8 +951,6 @@ github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
github.com/subosito/gotenv v1.2.0 h1:Slr1R9HxAlEKefgq5jn9U+DnETlIUa6HfgEzj0g5d7s=
github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=
github.com/syndtr/gocapability v0.0.0-20170704070218-db04d3cc01c8/go.mod h1:hkRG7XYTFWNJGYcbNJQlaLq0fg1yr4J4t/NcTQtrfww=
github.com/syndtr/gocapability v0.0.0-20180916011248-d98352740cb2 h1:b6uOv7YOFK0TYG7HtkIgExQo+2RdLuwRft63jn2HWj8=
github.com/syndtr/gocapability v0.0.0-20180916011248-d98352740cb2/go.mod h1:hkRG7XYTFWNJGYcbNJQlaLq0fg1yr4J4t/NcTQtrfww=
github.com/theupdateframework/notary v0.7.0 h1:QyagRZ7wlSpjT5N2qQAh/pN+DVqgekv4DzbAiAiEL3c=
github.com/theupdateframework/notary v0.7.0/go.mod h1:c9DRxcmhHmVLDay4/2fUYdISnHqbFDGRSlXPO0AhYWw=
github.com/tidwall/pretty v1.0.0 h1:HsD+QiTn7sK6flMKIvNmpqz1qrpP3Ps6jOKIKMooyg4=
@@ -1036,14 +961,6 @@ github.com/tj/go-kinesis v0.0.0-20171128231115-08b17f58cb1b/go.mod h1:/yhzCV0xPf
github.com/tj/go-spin v1.1.0/go.mod h1:Mg1mzmePZm4dva8Qz60H2lHwmJ2loum4VIrLgVnKwh4=
github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
github.com/tonistiigi/fsutil v0.0.0-20200326231323-c2c7d7b0e144 h1:6RY1EKxCnPQShPM46xFDHta2JSOd+YKCgHyyBHtKuo8=
github.com/tonistiigi/fsutil v0.0.0-20200326231323-c2c7d7b0e144/go.mod h1:0G1sLZ/0ttFf09xvh7GR4AEECnjifHRNJN/sYbLianU=
github.com/tonistiigi/go-immutable-radix v0.0.0-20170803185627-826af9ccf0fe h1:pd7hrFSqUPxYS9IB+UMG1AB/8EXGXo17ssx0bSQ5L6Y=
github.com/tonistiigi/go-immutable-radix v0.0.0-20170803185627-826af9ccf0fe/go.mod h1:/+MCh11CJf2oz0BXmlmqyopK/ad1rKkcOXPoYuPCJYU=
github.com/tonistiigi/units v0.0.0-20180711220420-6950e57a87ea/go.mod h1:WPnis/6cRcDZSUvVmezrxJPkiO87ThFYsoUiMwWNDJk=
github.com/tonistiigi/vt100 v0.0.0-20190402012908-ad4c4a574305/go.mod h1:gXOLibKqQTRAVuVZ9gX7G9Ykky8ll8yb4slxsEMoY0c=
github.com/uber/jaeger-client-go v0.0.0-20180103221425-e02c85f9069e/go.mod h1:WVhlPFC8FDjOFMMWRy2pZqQJSXxYSwNYOkTr/Z6d3Kk=
github.com/uber/jaeger-lib v1.2.1/go.mod h1:ComeNDZlWwrWnDv8aPp0Ba6+uUTzImX/AauajbLI56U=
github.com/ugorji/go v1.1.4/go.mod h1:uQMGLiO92mf5W77hV/PUCpI3pbzQx3CRekS0kk+RGrc=
github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0=
github.com/urfave/cli v0.0.0-20171014202726-7bc6a0acffa5/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA=
@@ -1056,10 +973,6 @@ github.com/vbatts/go-mtree v0.4.4 h1:+CncqETnSpxBCCUhRnNQBvxhsjWXNuc+ExZsLSNaj5o
github.com/vbatts/go-mtree v0.4.4/go.mod h1:3sazBqLG4bZYmgRTgdh9X3iKTzwBpp5CrREJDzrNSXY=
github.com/vdemeester/k8s-pkg-credentialprovider v1.18.1-0.20201019120933-f1d16962a4db/go.mod h1:grWy0bkr1XO6hqbaaCKaPXqkBVlMGHYG6PGykktwbJc=
github.com/vektah/gqlparser v1.1.2/go.mod h1:1ycwN7Ij5njmMkPPAOaRFY4rET2Enx7IkVv3vaXspKw=
github.com/vishvananda/netlink v1.0.0 h1:bqNY2lgheFIu1meHUFSH3d7vG93AFyqg3oGbJCOJgSM=
github.com/vishvananda/netlink v1.0.0/go.mod h1:+SR5DhBJrl6ZM7CoCKvpw5BKroDKQ+PJqOg65H/2ktk=
github.com/vishvananda/netns v0.0.0-20180720170159-13995c7128cc h1:R83G5ikgLMxrBvLh22JhdfI8K6YXEPHx5P03Uu3DRs4=
github.com/vishvananda/netns v0.0.0-20180720170159-13995c7128cc/go.mod h1:ZjcWmFBXmLKZu9Nxj3WKYEafiSqer2rnvPr0en9UNpI=
github.com/vmihailenco/msgpack v4.0.1+incompatible h1:RMF1enSPeKTlXrXdOcqjFUElywVZjjC6pqse21bKbEU=
github.com/vmihailenco/msgpack v4.0.1+incompatible/go.mod h1:fy3FlTQTDXWkZ7Bh6AcGMlsjHatGryHQYUTf1ShIgkk=
github.com/vmware/govmomi v0.20.3/go.mod h1:URlwyTFZX72RmxtxuaFL2Uj3fD1JTvZdx59bHWk6aFU=
@@ -1114,7 +1027,6 @@ go.uber.org/zap v1.13.0 h1:nR6NoDBgAf67s68NhaXbsojM+2gxp3S1hWkHDl27pVU=
go.uber.org/zap v1.13.0/go.mod h1:zwrFLgMcdUuIBviXEYEH1YKNaOBnKXsx2IPda5bBwHM=
golang.org/x/crypto v0.0.0-20171113213409-9f005a07e0d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20180910181607-0e37d006457b/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190211182817-74369b46fc67/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
@@ -1134,7 +1046,6 @@ golang.org/x/crypto v0.0.0-20191206172530-e9b2fee46413/go.mod h1:LzIPMQfyMNhhGPh
golang.org/x/crypto v0.0.0-20200128174031-69ecbb4d6d5d/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975 h1:/Tl7pH94bvbAAHBdZJT947M/+gp0+CqQXDtMRC0fseo=
golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200221231518-2aa609cf4a9d/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200302210943-78000ba7a073/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200414173820-0848c9571904/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9 h1:psW17arqaxU48Z5kZ0CQnkZWQJsqcURM6tKiBApRjXI=
@@ -1158,7 +1069,6 @@ golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EH
golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/lint v0.0.0-20180702182130-06c8688daad7/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
@@ -1184,7 +1094,6 @@ golang.org/x/net v0.0.0-20170114055629-f2499483f923/go.mod h1:mL1N/T3taQHkDXs73r
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180925072008-f04abc6bdfa7/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181005035420-146acd28ed58/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -1243,7 +1152,6 @@ golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5h
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180925112736-b09afc3d579e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -1260,7 +1168,6 @@ golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190514135907-3a4b5fb9f71f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190515120540-06a5c4944438/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190522044717-8097e1b27ff5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190602015325-4c4f7f33c9ed/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190616124812-15dcb6c0061f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -1276,10 +1183,8 @@ golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191022100944-742c48ecaeb7/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191115151921-52ab43148777/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191210023423-ac6580df4449/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191220142924-d4481acd189f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -1405,12 +1310,10 @@ google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCID
google.golang.org/cloud v0.0.0-20151119220103-975617b05ea8/go.mod h1:0H1ncTHf11KCFhTc/+EFRbzSCOZx+VUbRMk55Yv5MYk=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8 h1:Nw54tB0rB7hY/N0NQvRW8DG4Yk3Q6T9cu9RcFQDu1tc=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20180924164928-221a8d4f7494/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190522204451-c2c4e71fbf69/go.mod h1:z3L6/3dTEVtUr6QSP8miRzeRqwQOioJ9I66odjN4I7s=
google.golang.org/genproto v0.0.0-20190530194941-fb225487d101/go.mod h1:z3L6/3dTEVtUr6QSP8miRzeRqwQOioJ9I66odjN4I7s=
google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55 h1:gSJIx1SDwno+2ElGhA4+qG2zF97qiUzTM+rQ0klBOcE=
@@ -1426,14 +1329,12 @@ google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvx
google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA=
google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200227132054-3f1135a288c9/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
google.golang.org/genproto v0.0.0-20200527145253-8367513e4ece h1:1YM0uhfumvoDu9sx8+RyWwTI63zoCQvI23IYFRlvte0=
google.golang.org/genproto v0.0.0-20200527145253-8367513e4ece/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA=
google.golang.org/grpc v0.0.0-20160317175043-d3ddb4469d5a/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw=
google.golang.org/grpc v1.0.5/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw=
google.golang.org/grpc v1.15.0/go.mod h1:0JHn/cJsOMiMfNA9+DeHDlAU7KAAB5GDlYFpa9MZMio=
google.golang.org/grpc v1.17.0/go.mod h1:6QZJwpn2B+Zp71q/5VxRsJ6NXXVCE5NRUHRo+f3cWCs=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.0/go.mod h1:chYK+tFQF0nDUGJgXMSgLCQk3phJEuONr2DCgLDdAQM=
@@ -1444,7 +1345,6 @@ google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ij
google.golang.org/grpc v1.22.1/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.23.1/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.24.0/go.mod h1:XDChyiUovWa60DnaeDeZmSW86xtLtjtZbwvSiRnRtcA=
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
@@ -1507,7 +1407,6 @@ gopkg.in/yaml.v2 v2.3.0 h1:clyUAQHOM3G0M3f5vQj7LuJrETvjVot3Z5el9nffUtU=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gotest.tools v2.1.0+incompatible/go.mod h1:DsYFclhRJ6vuDpmuTbkuFWG+y2sxOXAzmJt81HFBacw=
gotest.tools v2.2.0+incompatible h1:VsBPFP1AI068pPrMxtb/S8Zkgf9xEmTLJjfM+P5UIEo=
gotest.tools v2.2.0+incompatible/go.mod h1:DsYFclhRJ6vuDpmuTbkuFWG+y2sxOXAzmJt81HFBacw=
gotest.tools/v3 v3.0.2 h1:kG1BFyqVHuQoVQiR1bWGnfz/fmHvvuiSPIV7rvl360E=

View File

@@ -23,9 +23,9 @@ import (
"strings"
"syscall"
"github.com/pkg/errors"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
helpers "github.com/mudler/luet/pkg/helpers"
"github.com/pkg/errors"
)
type Box interface {
@@ -107,7 +107,7 @@ func (b *DefaultBox) Exec() error {
func (b *DefaultBox) Run() error {
if !helpers.Exists(b.Root) {
if !fileHelper.Exists(b.Root) {
return errors.New(b.Root + " does not exist")
}

View File

@@ -2,6 +2,7 @@ package bus
import (
"github.com/mudler/go-pluggable"
"github.com/sirupsen/logrus"
)
var (
@@ -44,24 +45,59 @@ var (
EventRepositoryPreBuild pluggable.EventType = "repository.pre.build"
// EventRepositoryPostBuild is the event fired after a repository was built
EventRepositoryPostBuild pluggable.EventType = "repository.post.build"
// Image unpack
// EventImagePreUnPack is the event fired before unpacking an image to a local dir
EventImagePreUnPack pluggable.EventType = "image.pre.unpack"
// EventImagePostUnPack is the event fired after unpacking an image to a local dir
EventImagePostUnPack pluggable.EventType = "image.post.unpack"
)
// Manager is the bus instance manager, which subscribes plugins to events emitted by Luet
var Manager *pluggable.Manager = pluggable.NewManager(
[]pluggable.EventType{
EventPackageInstall,
EventPackageUnInstall,
EventPackagePreBuild,
EventPackagePreBuildArtifact,
EventPackagePostBuildArtifact,
EventPackagePostBuild,
EventRepositoryPreBuild,
EventRepositoryPostBuild,
EventImagePreBuild,
EventImagePrePull,
EventImagePrePush,
EventImagePostBuild,
EventImagePostPull,
EventImagePostPush,
},
)
var Manager *Bus = &Bus{
Manager: pluggable.NewManager(
[]pluggable.EventType{
EventPackageInstall,
EventPackageUnInstall,
EventPackagePreBuild,
EventPackagePreBuildArtifact,
EventPackagePostBuildArtifact,
EventPackagePostBuild,
EventRepositoryPreBuild,
EventRepositoryPostBuild,
EventImagePreBuild,
EventImagePrePull,
EventImagePrePush,
EventImagePostBuild,
EventImagePostPull,
EventImagePostPush,
EventImagePreUnPack,
EventImagePostUnPack,
},
),
}
type Bus struct {
*pluggable.Manager
}
func (b *Bus) Initialize(plugin ...string) {
b.Manager.Load(plugin...).Register()
for _, e := range b.Manager.Events {
b.Manager.Response(e, func(p *pluggable.Plugin, r *pluggable.EventResponse) {
if r.Errored() {
logrus.Fatal("Plugin", p.Name, "at", p.Executable, "Error", r.Error)
}
logrus.Debug(
"plugin_event",
"received from",
p.Name,
"at",
p.Executable,
r,
)
})
}
}

View File

@@ -24,6 +24,7 @@ import (
"strings"
bus "github.com/mudler/luet/pkg/bus"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
capi "github.com/mudler/docker-companion/api"
@@ -181,6 +182,12 @@ func (b *SimpleDocker) ExtractRootfs(opts Options, keepPerms bool) error {
name := opts.ImageName
dst := opts.Destination
if !b.ImageExists(name) {
if err := b.DownloadImage(opts); err != nil {
return errors.Wrap(err, "failed pulling image "+name+" during extraction")
}
}
tempexport, err := ioutil.TempDir(dst, "tmprootfs")
if err != nil {
return errors.Wrap(err, "Error met while creating tempdir for rootfs")
@@ -231,7 +238,7 @@ func (b *SimpleDocker) ExtractRootfs(opts Options, keepPerms bool) error {
return errors.Wrap(err, "Error met while unpacking rootfs")
}
manifest, err := helpers.Read(filepath.Join(rootfs, "manifest.json"))
manifest, err := fileHelper.Read(filepath.Join(rootfs, "manifest.json"))
if err != nil {
return errors.Wrap(err, "Error met while reading image manifest")
}

View File

@@ -21,12 +21,12 @@ import (
"github.com/mudler/luet/pkg/compiler/backend"
. "github.com/mudler/luet/pkg/compiler/backend"
"github.com/mudler/luet/pkg/compiler/types/artifact"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
"io/ioutil"
"os"
"path/filepath"
helpers "github.com/mudler/luet/pkg/helpers"
pkg "github.com/mudler/luet/pkg/package"
"github.com/mudler/luet/pkg/tree"
. "github.com/onsi/ginkgo"
@@ -60,7 +60,7 @@ var _ = Describe("Docker backend", func() {
err = lspec.WriteBuildImageDefinition(filepath.Join(tmpdir, "Dockerfile"))
Expect(err).ToNot(HaveOccurred())
dockerfile, err := helpers.Read(filepath.Join(tmpdir, "Dockerfile"))
dockerfile, err := fileHelper.Read(filepath.Join(tmpdir, "Dockerfile"))
Expect(err).ToNot(HaveOccurred())
Expect(dockerfile).To(Equal(`
FROM alpine
@@ -79,11 +79,11 @@ ENV PACKAGE_CATEGORY=app-admin`))
Expect(b.BuildImage(opts)).ToNot(HaveOccurred())
Expect(b.ExportImage(opts)).ToNot(HaveOccurred())
Expect(helpers.Exists(filepath.Join(tmpdir2, "output1.tar"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(tmpdir2, "output1.tar"))).To(BeTrue())
err = lspec.WriteStepImageDefinition(lspec.Image, filepath.Join(tmpdir, "LuetDockerfile"))
Expect(err).ToNot(HaveOccurred())
dockerfile, err = helpers.Read(filepath.Join(tmpdir, "LuetDockerfile"))
dockerfile, err = fileHelper.Read(filepath.Join(tmpdir, "LuetDockerfile"))
Expect(err).ToNot(HaveOccurred())
Expect(dockerfile).To(Equal(`
FROM luet/base
@@ -103,7 +103,7 @@ RUN echo bar > /test2`))
Expect(b.BuildImage(opts2)).ToNot(HaveOccurred())
Expect(b.ExportImage(opts2)).ToNot(HaveOccurred())
Expect(helpers.Exists(filepath.Join(tmpdir, "output2.tar"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(tmpdir, "output2.tar"))).To(BeTrue())
artifacts := []artifact.ArtifactNode{{
Name: "/luetbuild/LuetDockerfile",
@@ -132,7 +132,7 @@ RUN echo bar > /test2`))
}
Expect(b.ImageDefinitionToTar(opts2)).ToNot(HaveOccurred())
Expect(helpers.Exists(filepath.Join(tmpdir, "output3.tar"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(tmpdir, "output3.tar"))).To(BeTrue())
Expect(b.ImageExists(opts2.ImageName)).To(BeFalse())
})

View File

@@ -16,7 +16,9 @@
package compiler
import (
"crypto/md5"
"fmt"
"io"
"io/ioutil"
"os"
"path"
@@ -34,6 +36,7 @@ import (
"github.com/mudler/luet/pkg/compiler/types/options"
compilerspec "github.com/mudler/luet/pkg/compiler/types/spec"
"github.com/mudler/luet/pkg/helpers"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
. "github.com/mudler/luet/pkg/logger"
pkg "github.com/mudler/luet/pkg/package"
"github.com/mudler/luet/pkg/solver"
@@ -89,7 +92,7 @@ func (cs *LuetCompiler) compilerWorker(i int, wg *sync.WaitGroup, cspecs chan *c
defer wg.Done()
for s := range cspecs {
ar, err := cs.compile(concurrency, keepPermissions, s)
ar, err := cs.compile(concurrency, keepPermissions, nil, nil, s)
if err != nil {
errors <- err
}
@@ -146,11 +149,6 @@ func (cs *LuetCompiler) CompileParallel(keepPermissions bool, ps *compilerspec.L
}
for _, p := range ps.All() {
asserts, err := cs.ComputeDepTree(p)
if err != nil {
panic(err)
}
p.SetSourceAssertion(asserts)
all <- p
}
@@ -295,17 +293,6 @@ func (cs *LuetCompiler) unpackDelta(concurrency int, keepPermissions bool, p *co
return artifact, nil
}
func (cs *LuetCompiler) genBuilderImageTag(p *compilerspec.LuetCompilationSpec, packageImage string) string {
// Use packageImage as salt into the fp being used
// so the hash is unique also in cases where
// some package deps does have completely different
// depgraphs
// TODO: We should use the image tag, or pass by the package assertion hash which is unique
// and identifies the deptree of the package.
return fmt.Sprintf("builder-%s", p.GetPackage().HashFingerprint(helpers.StripRegistryFromImage(packageImage)))
}
func (cs *LuetCompiler) buildPackageImage(image, buildertaggedImage, packageImage string,
concurrency int, keepPermissions bool,
p *compilerspec.LuetCompilationSpec) (backend.Options, backend.Options, error) {
@@ -333,7 +320,7 @@ func (cs *LuetCompiler) buildPackageImage(image, buildertaggedImage, packageImag
defer os.RemoveAll(buildDir) // clean up
// First we copy the source definitions into the output - we create a copy which the builds will need (we need to cache this phase somehow)
err = helpers.CopyDir(p.GetPackage().GetPath(), buildDir)
err = fileHelper.CopyDir(p.GetPackage().GetPath(), buildDir)
if err != nil {
return builderOpts, runnerOpts, errors.Wrap(err, "Could not copy package sources")
}
@@ -522,7 +509,7 @@ func oneOfImagesAvailable(images []string, b CompilerBackend) (bool, string) {
return false, ""
}
func (cs *LuetCompiler) resolveExistingImageHash(imageHash string, p *compilerspec.LuetCompilationSpec) string {
func (cs *LuetCompiler) findImageHash(imageHash string, p *compilerspec.LuetCompilationSpec) string {
var resolvedImage string
Debug("Resolving image hash for", p.Package.HumanReadableString(), "hash", imageHash, "Pull repositories", p.BuildOptions.PullImageRepository)
toChecklist := append([]string{fmt.Sprintf("%s:%s", cs.Options.PushImageRepository, imageHash)},
@@ -535,6 +522,11 @@ func (cs *LuetCompiler) resolveExistingImageHash(imageHash string, p *compilersp
resolvedImage = which
}
}
return resolvedImage
}
func (cs *LuetCompiler) resolveExistingImageHash(imageHash string, p *compilerspec.LuetCompilationSpec) string {
resolvedImage := cs.findImageHash(imageHash, p)
if resolvedImage == "" {
resolvedImage = fmt.Sprintf("%s:%s", cs.Options.PushImageRepository, imageHash)
@@ -619,12 +611,24 @@ func (cs *LuetCompiler) compileWithImage(image, builderHash string, packageTagHa
Debug("Checking if an image is already available")
// FIXUP here. If packageimage hash exists and pull is true, generate package
resolved := cs.resolveExistingImageHash(packageTagHash, p)
Debug("Resolved: " + resolved)
Debug("Expected remote: " + resolved)
Debug("Package image: " + packageImage)
Debug("Resolved builder image: " + builderResolved)
//
if resolved != packageImage && remoteBuildertaggedImage != builderResolved { // an image is there already
Debug("Images available for", p.Package.HumanReadableString(), "generating artifact from remote images:", resolved)
// a remote image is there already
remoteImageAvailable := resolved != packageImage && remoteBuildertaggedImage != builderResolved
// or a local one is available
localImageAvailable := cs.Backend.ImageExists(remoteBuildertaggedImage) && cs.Backend.ImageExists(packageImage)
switch {
case remoteImageAvailable:
Debug("Images available remotely for", p.Package.HumanReadableString(), "generating artifact from remote images:", resolved)
return cs.genArtifact(p, backend.Options{ImageName: builderResolved}, backend.Options{ImageName: resolved}, concurrency, keepPermissions)
} else {
case localImageAvailable:
Debug("Images locally available for", p.Package.HumanReadableString(), "generating artifact from image:", resolved)
return cs.genArtifact(p, backend.Options{ImageName: remoteBuildertaggedImage}, backend.Options{ImageName: packageImage}, concurrency, keepPermissions)
default:
Debug("Images not available for", p.Package.HumanReadableString())
}
}
@@ -680,33 +684,7 @@ func (cs *LuetCompiler) FromDatabase(db pkg.PackageDatabase, minimum bool, dst s
}
}
// ComputeMinimumCompilableSet strips specs that are eventually compiled by leafs
func (cs *LuetCompiler) ComputeMinimumCompilableSet(p ...*compilerspec.LuetCompilationSpec) ([]*compilerspec.LuetCompilationSpec, error) {
// Generate a set with all the deps of the provided specs
// we will use that set to remove the deps from the list of provided compilation specs
allDependencies := solver.PackagesAssertions{} // Get all packages that will be in deps
result := []*compilerspec.LuetCompilationSpec{}
for _, spec := range p {
ass, err := cs.ComputeDepTree(spec)
if err != nil {
return result, errors.Wrap(err, "computin specs deptree")
}
allDependencies = append(allDependencies, ass.Drop(spec.GetPackage())...)
}
for _, spec := range p {
if found := allDependencies.Search(spec.GetPackage().GetFingerPrint()); found == nil {
result = append(result, spec)
}
}
return result, nil
}
// ComputeDepTree computes the dependency tree of a compilation spec and returns solver assertions
// in order to be able to compile the spec.
func (cs *LuetCompiler) ComputeDepTree(p *compilerspec.LuetCompilationSpec) (solver.PackagesAssertions, error) {
s := solver.NewResolver(cs.Options.SolverOptions.Options, pkg.NewInMemoryDatabase(false), cs.Database, pkg.NewInMemoryDatabase(false), cs.Options.SolverOptions.Resolver())
solution, err := s.Install(pkg.Packages{p.GetPackage()})
@@ -718,32 +696,35 @@ func (cs *LuetCompiler) ComputeDepTree(p *compilerspec.LuetCompilationSpec) (sol
if err != nil {
return nil, errors.Wrap(err, "While order a solution for "+p.GetPackage().HumanReadableString())
}
return dependencies, nil
}
assertions := solver.PackagesAssertions{}
for _, assertion := range dependencies { //highly dependent on the order
if assertion.Value {
nthsolution := dependencies.Cut(assertion.Package)
assertion.Hash = solver.PackageHash{
BuildHash: nthsolution.HashFrom(assertion.Package),
PackageHash: nthsolution.AssertionHash(),
}
assertion.Package.SetTreeDir(p.Package.GetTreeDir())
assertions = append(assertions, assertion)
// ComputeMinimumCompilableSet strips specs that are eventually compiled by leafs
func (cs *LuetCompiler) ComputeMinimumCompilableSet(p ...*compilerspec.LuetCompilationSpec) ([]*compilerspec.LuetCompilationSpec, error) {
// Generate a set with all the deps of the provided specs
// we will use that set to remove the deps from the list of provided compilation specs
allDependencies := solver.PackagesAssertions{} // Get all packages that will be in deps
result := []*compilerspec.LuetCompilationSpec{}
for _, spec := range p {
sol, err := cs.ComputeDepTree(spec)
if err != nil {
return nil, errors.Wrap(err, "failed querying hashtree")
}
allDependencies = append(allDependencies, sol.Drop(spec.GetPackage())...)
}
for _, spec := range p {
if found := allDependencies.Search(spec.GetPackage().GetFingerPrint()); found == nil {
result = append(result, spec)
}
}
p.SetSourceAssertion(assertions)
return assertions, nil
return result, nil
}
// Compile is a non-parallel version of CompileParallel. It builds the compilation specs and generates
// an artifact
func (cs *LuetCompiler) Compile(keepPermissions bool, p *compilerspec.LuetCompilationSpec) (*artifact.PackageArtifact, error) {
asserts, err := cs.ComputeDepTree(p)
if err != nil {
return nil, err
}
p.SetSourceAssertion(asserts)
return cs.compile(cs.Options.Concurrency, keepPermissions, p)
return cs.compile(cs.Options.Concurrency, keepPermissions, nil, nil, p)
}
func genImageList(refs []string, hash string) []string {
@@ -772,14 +753,185 @@ func (cs *LuetCompiler) inheritSpecBuildOptions(p *compilerspec.LuetCompilationS
Debug(p.GetPackage().HumanReadableString(), "Build options after inherit", p.BuildOptions)
}
func (cs *LuetCompiler) compile(concurrency int, keepPermissions bool, p *compilerspec.LuetCompilationSpec) (*artifact.PackageArtifact, error) {
// TODO: Racy, remove it
// Inherit build options from compilation specs metadata
// orig := cs.Options.PullImageRepository
// defer func() { cs.Options.PullImageRepository = orig }()
func (cs *LuetCompiler) getSpecHash(pkgs pkg.DefaultPackages, salt string) (string, error) {
ht := NewHashTree(cs.Database)
overallFp := ""
for _, p := range pkgs {
compileSpec, err := cs.FromPackage(p)
if err != nil {
return "", errors.Wrap(err, "Error while generating compilespec for "+p.GetName())
}
packageHashTree, err := ht.Query(cs, compileSpec)
if err != nil {
return "nil", errors.Wrap(err, "failed querying hashtree")
}
overallFp = overallFp + packageHashTree.Target.Hash.PackageHash + p.GetFingerPrint()
}
h := md5.New()
io.WriteString(h, fmt.Sprintf("%s-%s", overallFp, salt))
return fmt.Sprintf("%x", h.Sum(nil)), nil
}
func (cs *LuetCompiler) resolveJoinImages(concurrency int, keepPermissions bool, p *compilerspec.LuetCompilationSpec) error {
joinTag := ">:loop: join<"
if len(p.Join) != 0 {
Info(joinTag, "Generating a joint parent image from final packages")
} else {
return nil
}
// First compute a hash and check if image is available. if it is, then directly consume that
overallFp, err := cs.getSpecHash(p.Join, "join")
if err != nil {
return errors.Wrap(err, "could not generate image hash")
}
Info(joinTag, "Searching existing image with hash ", overallFp)
image := cs.findImageHash(overallFp, p)
if image != "" {
Info("Image already found", image)
p.SetImage(image)
return nil
}
Info(joinTag, "Image not found. Generating image join with hash ", overallFp)
// Make sure there is an output path
if err := os.MkdirAll(p.GetOutputPath(), os.ModePerm); err != nil {
return errors.Wrap(err, "while creating output path")
}
// otherwise, generate it and push it aside
joinDir, err := ioutil.TempDir(p.GetOutputPath(), "join")
if err != nil {
return errors.Wrap(err, "could not create tempdir for joining images")
}
defer os.RemoveAll(joinDir) // clean up
for _, p := range p.Join { //highly dependent on the order
Info(joinTag, ":arrow_right_hook:", p.HumanReadableString(), ":leaves:")
}
current := 0
for _, c := range p.Join {
current++
if c != nil && c.Name != "" && c.Version != "" {
joinTag2 := fmt.Sprintf("%s %d/%d ⤑ :hammer: build %s", joinTag, current, len(p.Join), c.HumanReadableString())
Info(joinTag2, "compilation starts")
spec, err := cs.FromPackage(c)
if err != nil {
return errors.Wrap(err, "while generating images to join from")
}
wantsArtifact := true
genDepsArtifact := !cs.Options.PackageTargetOnly
spec.SetOutputPath(p.GetOutputPath())
artifact, err := cs.compile(concurrency, keepPermissions, &wantsArtifact, &genDepsArtifact, spec)
if err != nil {
return errors.Wrap(err, "failed building join image")
}
err = artifact.Unpack(joinDir, keepPermissions)
if err != nil {
return errors.Wrap(err, "failed building join image")
}
Info(joinTag2, ":white_check_mark: Done")
}
}
artifactDir, err := ioutil.TempDir(p.GetOutputPath(), "artifact")
if err != nil {
return errors.Wrap(err, "could not create tempdir for final artifact")
}
defer os.RemoveAll(joinDir) // clean up
Info(joinTag, ":droplet: generating artifact for source image of", p.GetPackage().HumanReadableString())
// After unpack, create a new artifact and a new final image from it.
// no need to compress, as we are going to toss it away.
a := artifact.NewPackageArtifact(filepath.Join(artifactDir, p.GetPackage().GetFingerPrint()+".join.tar"))
if err := a.Compress(joinDir, concurrency); err != nil {
return errors.Wrap(err, "error met while creating package archive")
}
joinImageName := fmt.Sprintf("%s:%s", cs.Options.PushImageRepository, overallFp)
Info(joinTag, ":droplet: generating image from artifact", joinImageName)
opts, err := a.GenerateFinalImage(joinImageName, cs.Backend, keepPermissions)
if err != nil {
return errors.Wrap(err, "could not create final image")
}
if cs.Options.Push {
Info(joinTag, ":droplet: pushing image from artifact", joinImageName)
if err = cs.Backend.Push(opts); err != nil {
return errors.Wrapf(err, "Could not push image: %s %s", image, opts.DockerFileName)
}
}
Info(joinTag, ":droplet: Consuming image", joinImageName)
p.SetImage(joinImageName)
return nil
}
func (cs *LuetCompiler) resolveMultiStageImages(concurrency int, keepPermissions bool, p *compilerspec.LuetCompilationSpec) error {
resolvedCopyFields := []compilerspec.CopyField{}
copyTag := ">:droplet: copy<"
if len(p.Copy) != 0 {
Info(copyTag, "Package has multi-stage copy, generating required images")
}
current := 0
// TODO: we should run this only if we are going to build the image
for _, c := range p.Copy {
current++
if c.Package != nil && c.Package.Name != "" && c.Package.Version != "" {
copyTag2 := fmt.Sprintf("%s %d/%d ⤑ :hammer: build %s", copyTag, current, len(p.Copy), c.Package.HumanReadableString())
Info(copyTag2, "generating multi-stage images for", c.Package.HumanReadableString())
spec, err := cs.FromPackage(c.Package)
if err != nil {
return errors.Wrap(err, "while generating images to copy from")
}
// If we specify --only-target package, we don't want any artifact, otherwise we do
genArtifact := !cs.Options.PackageTargetOnly
spec.SetOutputPath(p.GetOutputPath())
artifact, err := cs.compile(concurrency, keepPermissions, &genArtifact, &genArtifact, spec)
if err != nil {
return errors.Wrap(err, "failed building multi-stage image")
}
resolvedCopyFields = append(resolvedCopyFields, compilerspec.CopyField{
Image: cs.resolveExistingImageHash(artifact.PackageCacheImage, spec),
Source: c.Source,
Destination: c.Destination,
})
Info(copyTag2, ":white_check_mark: Done")
} else {
resolvedCopyFields = append(resolvedCopyFields, c)
}
}
p.Copy = resolvedCopyFields
return nil
}
func (cs *LuetCompiler) compile(concurrency int, keepPermissions bool, generateFinalArtifact *bool, generateDependenciesFinalArtifact *bool, p *compilerspec.LuetCompilationSpec) (*artifact.PackageArtifact, error) {
Info(":package: Compiling", p.GetPackage().HumanReadableString(), ".... :coffee:")
//Before multistage : join - same as multistage, but keep artifacts, join them, create a new one and generate a final image.
// When the image is there, use it as a source here, in place of GetImage().
if err := cs.resolveJoinImages(concurrency, keepPermissions, p); err != nil {
return nil, errors.Wrap(err, "while resolving join images")
}
if err := cs.resolveMultiStageImages(concurrency, keepPermissions, p); err != nil {
return nil, errors.Wrap(err, "while resolving multi-stage images")
}
Debug(fmt.Sprintf("%s: has images %t, empty package: %t", p.GetPackage().HumanReadableString(), p.HasImageSource(), p.EmptyPackage()))
if !p.HasImageSource() && !p.EmptyPackage() {
return nil,
@@ -789,40 +941,62 @@ func (cs *LuetCompiler) compile(concurrency int, keepPermissions bool, p *compil
)
}
targetAssertion := p.GetSourceAssertion().Search(p.GetPackage().GetFingerPrint())
ht := NewHashTree(cs.Database)
packageHashTree, err := ht.Query(cs, p)
if err != nil {
return nil, errors.Wrap(err, "failed querying hashtree")
}
// This is in order to have the metadata in the yaml
p.SetSourceAssertion(packageHashTree.Solution)
targetAssertion := packageHashTree.Target
bus.Manager.Publish(bus.EventPackagePreBuild, struct {
CompileSpec *compilerspec.LuetCompilationSpec
Assert solver.PackageAssert
CompileSpec *compilerspec.LuetCompilationSpec
Assert solver.PackageAssert
PackageHashTree *PackageImageHashTree
}{
CompileSpec: p,
Assert: *targetAssertion,
CompileSpec: p,
Assert: *targetAssertion,
PackageHashTree: packageHashTree,
})
// Update compilespec build options - it will be then serialized into the compilation metadata file
//p.SetBuildOptions(cs.Options)
p.BuildOptions.PushImageRepository = cs.Options.PushImageRepository
//p.BuildOptions.BuildValues = cs.Options.BuildValues
//p.BuildOptions.BuildValuesFile = cs.Options.BuildValuesFile
// - If image is set we just generate a plain dockerfile
// Treat last case (easier) first. The image is provided and we just compute a plain dockerfile with the images listed as above
if p.GetImage() != "" {
return cs.compileWithImage(p.GetImage(), cs.genBuilderImageTag(p, targetAssertion.Hash.PackageHash), targetAssertion.Hash.PackageHash, concurrency, keepPermissions, cs.Options.KeepImg, p, true)
localGenerateArtifact := true
if generateFinalArtifact != nil {
localGenerateArtifact = *generateFinalArtifact
}
a, err := cs.compileWithImage(p.GetImage(), packageHashTree.BuilderImageHash, targetAssertion.Hash.PackageHash, concurrency, keepPermissions, cs.Options.KeepImg, p, localGenerateArtifact)
if err != nil {
return nil, errors.Wrap(err, "building direct image")
}
a.SourceAssertion = p.GetSourceAssertion()
a.PackageCacheImage = targetAssertion.Hash.PackageHash
return a, nil
}
// - If image is not set, we read a base_image. Then we will build one image from it to kick-off our build based
// on how we compute the resolvable tree.
// This means to recursively build all the build-images needed to reach that tree part.
// - We later on compute an hash used to identify the image, so each similar deptree keeps the same build image.
dependencies := p.GetSourceAssertion().Drop(p.GetPackage()) // at this point we should have a flattened list of deps to build, including all of them (with all constraints propagated already)
departifacts := []*artifact.PackageArtifact{} // TODO: Return this somehow
var lastHash string
dependencies := packageHashTree.Dependencies // at this point we should have a flattened list of deps to build, including all of them (with all constraints propagated already)
departifacts := []*artifact.PackageArtifact{} // TODO: Return this somehow
depsN := 0
currentN := 0
packageDeps := !cs.Options.PackageTargetOnly
if generateDependenciesFinalArtifact != nil {
packageDeps = *generateDependenciesFinalArtifact
}
buildDeps := !cs.Options.NoDeps
buildTarget := !cs.Options.OnlyDeps
@@ -845,8 +1019,6 @@ func (cs *LuetCompiler) compile(concurrency int, keepPermissions bool, p *compil
Debug("PullImage repos:", compileSpec.BuildOptions.PullImageRepository)
compileSpec.SetOutputPath(p.GetOutputPath())
Debug(pkgTag, " :arrow_right_hook: :whale: Builder image from hash", assertion.Hash.BuildHash)
Debug(pkgTag, " :arrow_right_hook: :whale: Package image from hash", assertion.Hash.PackageHash)
bus.Manager.Publish(bus.EventPackagePreBuild, struct {
CompileSpec *compilerspec.LuetCompilationSpec
@@ -856,29 +1028,53 @@ func (cs *LuetCompiler) compile(concurrency int, keepPermissions bool, p *compil
Assert: assertion,
})
lastHash = assertion.Hash.PackageHash
// for the source instead, pick an image and a buildertaggedImage from hashes if they exists.
// otherways fallback to the pushed repo
// Resolve images from the hashtree
resolvedBuildImage := cs.resolveExistingImageHash(assertion.Hash.BuildHash, compileSpec)
if compileSpec.GetImage() != "" {
Debug(pkgTag, " :wrench: Compiling "+compileSpec.GetPackage().HumanReadableString()+" from image")
a, err := cs.compileWithImage(compileSpec.GetImage(), assertion.Hash.BuildHash, assertion.Hash.PackageHash, concurrency, keepPermissions, cs.Options.KeepImg, compileSpec, packageDeps)
if err != nil {
return nil, errors.Wrap(err, "Failed compiling "+compileSpec.GetPackage().HumanReadableString())
}
departifacts = append(departifacts, a)
Info(pkgTag, ":white_check_mark: Done")
continue
if err := cs.resolveJoinImages(concurrency, keepPermissions, compileSpec); err != nil {
return nil, errors.Wrap(err, "while resolving join images")
}
Debug(pkgTag, " :wrench: Compiling "+compileSpec.GetPackage().HumanReadableString()+" from tree")
a, err := cs.compileWithImage(resolvedBuildImage, cs.genBuilderImageTag(compileSpec, targetAssertion.Hash.PackageHash), assertion.Hash.PackageHash, concurrency, keepPermissions, cs.Options.KeepImg, compileSpec, packageDeps)
if err := cs.resolveMultiStageImages(concurrency, keepPermissions, compileSpec); err != nil {
return nil, errors.Wrap(err, "while resolving multi-stage images")
}
buildHash, err := packageHashTree.DependencyBuildImage(assertion.Package)
if err != nil {
return nil, errors.Wrap(err, "failed looking for dependency in hashtree")
}
Debug(pkgTag, " :arrow_right_hook: :whale: Builder image from hash", assertion.Hash.BuildHash)
Debug(pkgTag, " :arrow_right_hook: :whale: Package image from hash", assertion.Hash.PackageHash)
var sourceImage string
if compileSpec.GetImage() != "" {
Debug(pkgTag, " :wrench: Compiling "+compileSpec.GetPackage().HumanReadableString()+" from image")
sourceImage = compileSpec.GetImage()
} else {
// for the source instead, pick an image and a buildertaggedImage from hashes if they exists.
// otherways fallback to the pushed repo
// Resolve images from the hashtree
sourceImage = cs.resolveExistingImageHash(assertion.Hash.BuildHash, compileSpec)
Debug(pkgTag, " :wrench: Compiling "+compileSpec.GetPackage().HumanReadableString()+" from tree")
}
a, err := cs.compileWithImage(
sourceImage,
buildHash,
assertion.Hash.PackageHash,
concurrency,
keepPermissions,
cs.Options.KeepImg,
compileSpec,
packageDeps,
)
if err != nil {
return nil, errors.Wrap(err, "Failed compiling "+compileSpec.GetPackage().HumanReadableString())
}
a.PackageCacheImage = assertion.Hash.PackageHash
Info(pkgTag, ":white_check_mark: Done")
bus.Manager.Publish(bus.EventPackagePostBuild, struct {
CompileSpec *compilerspec.LuetCompilationSpec
Artifact *artifact.PackageArtifact
@@ -888,24 +1084,24 @@ func (cs *LuetCompiler) compile(concurrency int, keepPermissions bool, p *compil
})
departifacts = append(departifacts, a)
Info(pkgTag, ":white_check_mark: Done")
}
} else if len(dependencies) > 0 {
lastHash = dependencies[len(dependencies)-1].Hash.PackageHash
}
if buildTarget {
resolvedBuildImage := cs.resolveExistingImageHash(lastHash, p)
localGenerateArtifact := true
if generateFinalArtifact != nil {
localGenerateArtifact = *generateFinalArtifact
}
resolvedSourceImage := cs.resolveExistingImageHash(packageHashTree.SourceHash, p)
Info(":rocket: All dependencies are satisfied, building package requested by the user", p.GetPackage().HumanReadableString())
Info(":package:", p.GetPackage().HumanReadableString(), " Using image: ", resolvedBuildImage)
a, err := cs.compileWithImage(resolvedBuildImage, cs.genBuilderImageTag(p, targetAssertion.Hash.PackageHash), targetAssertion.Hash.PackageHash, concurrency, keepPermissions, cs.Options.KeepImg, p, true)
Info(":package:", p.GetPackage().HumanReadableString(), " Using image: ", resolvedSourceImage)
a, err := cs.compileWithImage(resolvedSourceImage, packageHashTree.BuilderImageHash, targetAssertion.Hash.PackageHash, concurrency, keepPermissions, cs.Options.KeepImg, p, localGenerateArtifact)
if err != nil {
return a, err
}
a.Dependencies = departifacts
a.SourceAssertion = p.GetSourceAssertion()
a.PackageCacheImage = targetAssertion.Hash.PackageHash
bus.Manager.Publish(bus.EventPackagePostBuild, struct {
CompileSpec *compilerspec.LuetCompilationSpec
Artifact *artifact.PackageArtifact
@@ -978,7 +1174,7 @@ func (cs *LuetCompiler) templatePackage(vals []map[string]interface{}, pack pkg.
if err != nil {
return nil, errors.Wrap(err, "while marshalling values file")
}
f := filepath.Join(valuesdir, helpers.RandStringRunes(20))
f := filepath.Join(valuesdir, fileHelper.RandStringRunes(20))
if err := ioutil.WriteFile(f, out, os.ModePerm); err != nil {
return nil, errors.Wrap(err, "while writing temporary values file")
}

View File

@@ -25,6 +25,7 @@ import (
"github.com/mudler/luet/pkg/compiler/types/options"
compilerspec "github.com/mudler/luet/pkg/compiler/types/spec"
helpers "github.com/mudler/luet/pkg/helpers"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
pkg "github.com/mudler/luet/pkg/package"
"github.com/mudler/luet/pkg/tree"
. "github.com/onsi/ginkgo"
@@ -59,15 +60,15 @@ var _ = Describe("Compiler", func() {
artifact, err := compiler.Compile(false, spec)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test6"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test6"))).To(BeTrue())
content1, err := helpers.Read(spec.Rel("test5"))
content1, err := fileHelper.Read(spec.Rel("test5"))
Expect(err).ToNot(HaveOccurred())
content2, err := helpers.Read(spec.Rel("test6"))
content2, err := fileHelper.Read(spec.Rel("test6"))
Expect(err).ToNot(HaveOccurred())
Expect(content1).To(Equal("artifact5\n"))
Expect(content2).To(Equal("artifact6\n"))
@@ -75,6 +76,68 @@ var _ = Describe("Compiler", func() {
})
})
Context("Copy and Join", func() {
It("Compiles it correctly with Copy", func() {
generalRecipe := tree.NewCompilerRecipe(pkg.NewInMemoryDatabase(false))
err := generalRecipe.Load("../../tests/fixtures/copy")
Expect(err).ToNot(HaveOccurred())
Expect(len(generalRecipe.GetDatabase().GetPackages())).To(Equal(3))
compiler := NewLuetCompiler(sd.NewSimpleDockerBackend(), generalRecipe.GetDatabase(), options.Concurrency(2))
spec, err := compiler.FromPackage(&pkg.DefaultPackage{Name: "c", Category: "test", Version: "1.2"})
Expect(err).ToNot(HaveOccurred())
Expect(spec.GetPackage().GetPath()).ToNot(Equal(""))
tmpdir, err := ioutil.TempDir("", "tree")
Expect(err).ToNot(HaveOccurred())
defer os.RemoveAll(tmpdir) // clean up
spec.SetOutputPath(tmpdir)
artifact, err := compiler.Compile(false, spec)
Expect(err).ToNot(HaveOccurred())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
Expect(fileHelper.Exists(spec.Rel("result"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("bina/busybox"))).To(BeTrue())
})
It("Compiles it correctly with Join", func() {
generalRecipe := tree.NewCompilerRecipe(pkg.NewInMemoryDatabase(false))
err := generalRecipe.Load("../../tests/fixtures/join")
Expect(err).ToNot(HaveOccurred())
Expect(len(generalRecipe.GetDatabase().GetPackages())).To(Equal(3))
compiler := NewLuetCompiler(sd.NewSimpleDockerBackend(), generalRecipe.GetDatabase(), options.Concurrency(2))
spec, err := compiler.FromPackage(&pkg.DefaultPackage{Name: "c", Category: "test", Version: "1.2"})
Expect(err).ToNot(HaveOccurred())
Expect(spec.GetPackage().GetPath()).ToNot(Equal(""))
tmpdir, err := ioutil.TempDir("", "tree")
Expect(err).ToNot(HaveOccurred())
defer os.RemoveAll(tmpdir) // clean up
spec.SetOutputPath(tmpdir)
artifact, err := compiler.Compile(false, spec)
Expect(err).ToNot(HaveOccurred())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
Expect(fileHelper.Exists(spec.Rel("newc"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test4"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test3"))).To(BeTrue())
})
})
Context("Simple package build definition", func() {
It("Compiles it in parallel", func() {
generalRecipe := tree.NewCompilerRecipe(pkg.NewInMemoryDatabase(false))
@@ -102,7 +165,7 @@ var _ = Describe("Compiler", func() {
artifacts, errs := compiler.CompileParallel(false, compilerspec.NewLuetCompilationspecs(spec, spec2))
Expect(errs).To(BeNil())
for _, artifact := range artifacts {
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
}
@@ -165,23 +228,23 @@ var _ = Describe("Compiler", func() {
Expect(len(artifacts)).To(Equal(3))
for _, artifact := range artifacts {
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
}
Expect(helpers.Exists(spec.Rel("test3"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test4"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test3"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test4"))).To(BeTrue())
content1, err := helpers.Read(spec.Rel("c"))
content1, err := fileHelper.Read(spec.Rel("c"))
Expect(err).ToNot(HaveOccurred())
content2, err := helpers.Read(spec.Rel("cd"))
content2, err := fileHelper.Read(spec.Rel("cd"))
Expect(err).ToNot(HaveOccurred())
Expect(content1).To(Equal("c\n"))
Expect(content2).To(Equal("c\n"))
content1, err = helpers.Read(spec.Rel("d"))
content1, err = fileHelper.Read(spec.Rel("d"))
Expect(err).ToNot(HaveOccurred())
content2, err = helpers.Read(spec.Rel("dd"))
content2, err = fileHelper.Read(spec.Rel("dd"))
Expect(err).ToNot(HaveOccurred())
Expect(content1).To(Equal("s\n"))
Expect(content2).To(Equal("dd\n"))
@@ -215,17 +278,17 @@ var _ = Describe("Compiler", func() {
Expect(len(artifacts2)).To(Equal(1))
for _, artifact := range artifacts {
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
}
for _, artifact := range artifacts2 {
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
}
Expect(helpers.Exists(spec.Rel("etc/hosts"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test1"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("etc/hosts"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test1"))).To(BeTrue())
})
It("Compiles and includes ony wanted files", func() {
@@ -254,12 +317,12 @@ var _ = Describe("Compiler", func() {
Expect(len(artifacts)).To(Equal(1))
for _, artifact := range artifacts {
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
}
Expect(helpers.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("marvin"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test6"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("marvin"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test6"))).ToNot(BeTrue())
})
It("Compiles and excludes files", func() {
@@ -288,13 +351,13 @@ var _ = Describe("Compiler", func() {
Expect(len(artifacts)).To(Equal(1))
for _, artifact := range artifacts {
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
}
Expect(helpers.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("marvin"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("marvot"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel("test6"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("marvin"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("marvot"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test6"))).To(BeTrue())
})
It("Compiles includes and excludes files", func() {
@@ -323,13 +386,13 @@ var _ = Describe("Compiler", func() {
Expect(len(artifacts)).To(Equal(1))
for _, artifact := range artifacts {
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
}
Expect(helpers.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("marvin"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("marvot"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel("test6"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("marvin"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("marvot"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test6"))).ToNot(BeTrue())
})
It("Compiles and excludes ony wanted files also from unpacked packages", func() {
@@ -357,12 +420,12 @@ var _ = Describe("Compiler", func() {
Expect(len(artifacts)).To(Equal(1))
for _, artifact := range artifacts {
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
}
Expect(helpers.Exists(spec.Rel("marvin"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test6"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("marvin"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test6"))).To(BeTrue())
})
It("Compiles includes and excludes ony wanted files also from unpacked packages", func() {
@@ -390,12 +453,12 @@ var _ = Describe("Compiler", func() {
Expect(len(artifacts)).To(Equal(1))
for _, artifact := range artifacts {
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
}
Expect(helpers.Exists(spec.Rel("marvin"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test6"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("marvin"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test6"))).To(BeTrue())
})
It("Compiles and includes ony wanted files also from unpacked packages", func() {
@@ -423,16 +486,16 @@ var _ = Describe("Compiler", func() {
Expect(len(artifacts)).To(Equal(1))
for _, artifact := range artifacts {
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
}
Expect(helpers.Exists(spec.Rel("var/lib/udhcpd"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("marvin"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test5"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel("test6"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel("test"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel("test2"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel("lib/firmware"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("var/lib/udhcpd"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("marvin"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test5"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test6"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test2"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("lib/firmware"))).ToNot(BeTrue())
})
It("Compiles a more complex tree", func() {
@@ -461,18 +524,18 @@ var _ = Describe("Compiler", func() {
Expect(len(artifacts)).To(Equal(1))
for _, artifact := range artifacts {
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
}
Expect(helpers.Untar(spec.Rel("extra-layer-0.1.package.tar"), tmpdir, false)).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("extra-layer"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("extra-layer"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("usr/bin/pkgs-checker"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("base-layer-0.1.package.tar"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("base-layer-0.1.metadata.yaml"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("extra-layer-0.1.metadata.yaml"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("extra-layer-0.1.package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("usr/bin/pkgs-checker"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("base-layer-0.1.package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("base-layer-0.1.metadata.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("extra-layer-0.1.metadata.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("extra-layer-0.1.package.tar"))).To(BeTrue())
})
It("Compiles with provides support", func() {
@@ -502,19 +565,19 @@ var _ = Describe("Compiler", func() {
Expect(len(artifacts[0].Dependencies)).To(Equal(1))
for _, artifact := range artifacts {
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
}
Expect(helpers.Untar(spec.Rel("c-test-1.0.package.tar"), tmpdir, false)).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("d"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("dd"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("c"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("cd"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("d"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("dd"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("c"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("cd"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("d-test-1.0.metadata.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("d-test-1.0.metadata.yaml"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("c-test-1.0.metadata.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("c-test-1.0.metadata.yaml"))).To(BeTrue())
})
It("Compiles with provides and selectors support", func() {
@@ -545,19 +608,19 @@ var _ = Describe("Compiler", func() {
Expect(len(artifacts[0].Dependencies)).To(Equal(1))
for _, artifact := range artifacts {
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
}
Expect(helpers.Untar(spec.Rel("c-test-1.0.package.tar"), tmpdir, false)).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("d"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("dd"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("c"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("cd"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("d"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("dd"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("c"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("cd"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("d-test-1.0.metadata.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("d-test-1.0.metadata.yaml"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("c-test-1.0.metadata.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("c-test-1.0.metadata.yaml"))).To(BeTrue())
})
It("Compiles revdeps", func() {
generalRecipe := tree.NewCompilerRecipe(pkg.NewInMemoryDatabase(false))
@@ -585,16 +648,16 @@ var _ = Describe("Compiler", func() {
Expect(len(artifacts)).To(Equal(2))
for _, artifact := range artifacts {
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
}
Expect(helpers.Untar(spec.Rel("extra-layer-0.1.package.tar"), tmpdir, false)).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("extra-layer"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("extra-layer"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("usr/bin/pkgs-checker"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("base-layer-0.1.package.tar"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("extra-layer-0.1.package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("usr/bin/pkgs-checker"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("base-layer-0.1.package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("extra-layer-0.1.package.tar"))).To(BeTrue())
})
It("Compiles complex dependencies trees with best matches", func() {
@@ -623,17 +686,17 @@ var _ = Describe("Compiler", func() {
Expect(len(artifacts)).To(Equal(1))
Expect(len(artifacts[0].Dependencies)).To(Equal(6))
for _, artifact := range artifacts {
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
}
Expect(helpers.Untar(spec.Rel("vhba-sys-fs-5.4.2-20190410.package.tar"), tmpdir, false)).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("sabayon-build-portage-layer-0.20191126.package.tar"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("build-layer-0.1.package.tar"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("build-sabayon-overlay-layer-0.20191212.package.tar"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("build-sabayon-overlays-layer-0.1.package.tar"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("linux-sabayon-sys-kernel-5.4.2.package.tar"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("sabayon-sources-sys-kernel-5.4.2.package.tar"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("vhba"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("sabayon-build-portage-layer-0.20191126.package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("build-layer-0.1.package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("build-sabayon-overlay-layer-0.20191212.package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("build-sabayon-overlays-layer-0.1.package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("linux-sabayon-sys-kernel-5.4.2.package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("sabayon-sources-sys-kernel-5.4.2.package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("vhba"))).To(BeTrue())
})
It("Compiles revdeps with seeds", func() {
@@ -658,31 +721,31 @@ var _ = Describe("Compiler", func() {
Expect(len(artifacts)).To(Equal(4))
for _, artifact := range artifacts {
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
}
// A deps on B, so A artifacts are here:
Expect(helpers.Exists(spec.Rel("test3"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test4"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test3"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test4"))).To(BeTrue())
// B
Expect(helpers.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test6"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("artifact42"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test6"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("artifact42"))).To(BeTrue())
// C depends on B, so B is here
content1, err := helpers.Read(spec.Rel("c"))
content1, err := fileHelper.Read(spec.Rel("c"))
Expect(err).ToNot(HaveOccurred())
content2, err := helpers.Read(spec.Rel("cd"))
content2, err := fileHelper.Read(spec.Rel("cd"))
Expect(err).ToNot(HaveOccurred())
Expect(content1).To(Equal("c\n"))
Expect(content2).To(Equal("c\n"))
// D is here as it requires C, and C was recompiled
content1, err = helpers.Read(spec.Rel("d"))
content1, err = fileHelper.Read(spec.Rel("d"))
Expect(err).ToNot(HaveOccurred())
content2, err = helpers.Read(spec.Rel("dd"))
content2, err = fileHelper.Read(spec.Rel("dd"))
Expect(err).ToNot(HaveOccurred())
Expect(content1).To(Equal("s\n"))
Expect(content2).To(Equal("dd\n"))
@@ -715,19 +778,19 @@ var _ = Describe("Compiler", func() {
artifacts, errs := compiler.CompileParallel(false, compilerspec.NewLuetCompilationspecs(spec))
Expect(errs).To(BeNil())
for _, artifact := range artifacts {
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
for _, d := range artifact.Dependencies {
Expect(helpers.Exists(d.Path)).To(BeTrue())
Expect(fileHelper.Exists(d.Path)).To(BeTrue())
Expect(helpers.Untar(d.Path, tmpdir, false)).ToNot(HaveOccurred())
}
}
Expect(helpers.Exists(spec.Rel("test3"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test4"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test6"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test3"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test4"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test6"))).To(BeTrue())
})
})
@@ -759,8 +822,8 @@ var _ = Describe("Compiler", func() {
Expect(len(artifacts)).To(Equal(1))
Expect(len(artifacts[0].Dependencies)).To(Equal(1))
Expect(helpers.Untar(spec.Rel("runtime-layer-0.1.package.tar"), tmpdir, false)).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("bin/busybox"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("var"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("bin/busybox"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("var"))).ToNot(BeTrue())
})
})
@@ -807,13 +870,13 @@ var _ = Describe("Compiler", func() {
Expect(len(artifacts[0].Dependencies)).To(Equal(0))
Expect(helpers.Untar(spec.Rel("dironly-test-1.0.package.tar"), tmpdir, false)).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("test1"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test2"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test1"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test2"))).To(BeTrue())
Expect(helpers.Untar(spec2.Rel("dironly_filter-test-1.0.package.tar"), tmpdir2, false)).ToNot(HaveOccurred())
Expect(helpers.Exists(spec2.Rel("test5"))).To(BeTrue())
Expect(helpers.Exists(spec2.Rel("test6"))).ToNot(BeTrue())
Expect(helpers.Exists(spec2.Rel("artifact42"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec2.Rel("test5"))).To(BeTrue())
Expect(fileHelper.Exists(spec2.Rel("test6"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec2.Rel("artifact42"))).ToNot(BeTrue())
})
})
@@ -843,12 +906,12 @@ var _ = Describe("Compiler", func() {
Expect(errs).To(BeNil())
Expect(len(artifacts)).To(Equal(1))
Expect(len(artifacts[0].Dependencies)).To(Equal(1))
Expect(helpers.Exists(spec.Rel("runtime-layer-0.1.package.tar.gz"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("runtime-layer-0.1.package.tar"))).To(BeFalse())
Expect(fileHelper.Exists(spec.Rel("runtime-layer-0.1.package.tar.gz"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("runtime-layer-0.1.package.tar"))).To(BeFalse())
Expect(artifacts[0].Unpack(tmpdir, false)).ToNot(HaveOccurred())
// Expect(helpers.Untar(spec.Rel("runtime-layer-0.1.package.tar"), tmpdir, false)).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("bin/busybox"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("var"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("bin/busybox"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("var"))).ToNot(BeTrue())
})
})
@@ -899,7 +962,7 @@ var _ = Describe("Compiler", func() {
Expect(len(artifacts[0].Dependencies)).To(Equal(1))
Expect(artifacts[0].Files).To(ContainElement("bin/busybox"))
Expect(helpers.Exists(spec.Rel("runtime-layer-0.1.metadata.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("runtime-layer-0.1.metadata.yaml"))).To(BeTrue())
art, err := LoadArtifactFromYaml(spec)
Expect(err).ToNot(HaveOccurred())

View File

@@ -0,0 +1,161 @@
// Copyright © 2021 Ettore Di Giacinto <mudler@mocaccino.org>
//
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License along
// with this program; if not, see <http://www.gnu.org/licenses/>.
package compiler
import (
"fmt"
compilerspec "github.com/mudler/luet/pkg/compiler/types/spec"
"github.com/mudler/luet/pkg/config"
pkg "github.com/mudler/luet/pkg/package"
"github.com/mudler/luet/pkg/solver"
"github.com/pkg/errors"
)
// ImageHashTree is holding the Database
// and the options to resolve PackageImageHashTrees
// for a given specfile
// It is responsible of returning a concrete result
// which identifies a Package in a HashTree
type ImageHashTree struct {
Database pkg.PackageDatabase
SolverOptions config.LuetSolverOptions
}
// PackageImageHashTree represent the Package into a given image hash tree
// The hash tree is constructed by a set of images representing
// the package during its build stage. A Hash is assigned to each image
// from the package fingerprint, plus the SAT solver assertion result (which is hashed as well)
// and the specfile signatures. This guarantees that each image of the build stage
// is unique and can be identified later on.
type PackageImageHashTree struct {
Target *solver.PackageAssert
Dependencies solver.PackagesAssertions
Solution solver.PackagesAssertions
dependencyBuilderImageHashes map[string]string
SourceHash string
BuilderImageHash string
}
func NewHashTree(db pkg.PackageDatabase) *ImageHashTree {
return &ImageHashTree{
Database: db,
}
}
func (ht *PackageImageHashTree) DependencyBuildImage(p pkg.Package) (string, error) {
found, ok := ht.dependencyBuilderImageHashes[p.GetFingerPrint()]
if !ok {
return "", errors.New("package hash not found")
}
return found, nil
}
func (ht *PackageImageHashTree) String() string {
return fmt.Sprintf(
"Target buildhash: %s\nTarget packagehash: %s\nBuilder Imagehash: %s\nSource Imagehash: %s\n",
ht.Target.Hash.BuildHash,
ht.Target.Hash.PackageHash,
ht.BuilderImageHash,
ht.SourceHash,
)
}
// Query takes a compiler and a compilation spec and returns a PackageImageHashTree tied to it.
// PackageImageHashTree contains all the informations to resolve the spec build images in order to
// reproducibly re-build images from packages
func (ht *ImageHashTree) Query(cs *LuetCompiler, p *compilerspec.LuetCompilationSpec) (*PackageImageHashTree, error) {
assertions, err := ht.resolve(cs, p)
if err != nil {
return nil, err
}
targetAssertion := assertions.Search(p.GetPackage().GetFingerPrint())
dependencies := assertions.Drop(p.GetPackage())
var sourceHash string
imageHashes := map[string]string{}
for _, assertion := range dependencies {
var depbuildImageTag string
compileSpec, err := cs.FromPackage(assertion.Package)
if err != nil {
return nil, errors.Wrap(err, "Error while generating compilespec for "+assertion.Package.GetName())
}
if compileSpec.GetImage() != "" {
depbuildImageTag = assertion.Hash.BuildHash
} else {
depbuildImageTag = ht.genBuilderImageTag(compileSpec, targetAssertion.Hash.PackageHash)
}
imageHashes[assertion.Package.GetFingerPrint()] = depbuildImageTag
sourceHash = assertion.Hash.PackageHash
}
return &PackageImageHashTree{
Dependencies: dependencies,
Target: targetAssertion,
SourceHash: sourceHash,
BuilderImageHash: ht.genBuilderImageTag(p, targetAssertion.Hash.PackageHash),
dependencyBuilderImageHashes: imageHashes,
Solution: assertions,
}, nil
}
func (ht *ImageHashTree) genBuilderImageTag(p *compilerspec.LuetCompilationSpec, packageImage string) string {
// Use packageImage as salt into the fp being used
// so the hash is unique also in cases where
// some package deps does have completely different
// depgraphs
return fmt.Sprintf("builder-%s", p.GetPackage().HashFingerprint(packageImage))
}
// resolve computes the dependency tree of a compilation spec and returns solver assertions
// in order to be able to compile the spec.
func (ht *ImageHashTree) resolve(cs *LuetCompiler, p *compilerspec.LuetCompilationSpec) (solver.PackagesAssertions, error) {
dependencies, err := cs.ComputeDepTree(p)
if err != nil {
return nil, errors.Wrap(err, "While computing a solution for "+p.GetPackage().HumanReadableString())
}
// Get hash from buildpsecs
salts := map[string]string{}
for _, assertion := range dependencies { //highly dependent on the order
if assertion.Value {
spec, err := cs.FromPackage(assertion.Package)
if err != nil {
return nil, errors.Wrap(err, "while computing hash buildspecs")
}
hash, err := spec.Hash()
if err != nil {
return nil, errors.Wrap(err, "failed computing hash")
}
salts[assertion.Package.GetFingerPrint()] = hash
}
}
assertions := solver.PackagesAssertions{}
for _, assertion := range dependencies { //highly dependent on the order
if assertion.Value {
nthsolution := dependencies.Cut(assertion.Package)
assertion.Hash = solver.PackageHash{
BuildHash: nthsolution.SaltedHashFrom(assertion.Package, salts),
PackageHash: nthsolution.SaltedAssertionHash(salts),
}
assertion.Package.SetTreeDir(p.Package.GetTreeDir())
assertions = append(assertions, assertion)
}
}
return assertions, nil
}

View File

@@ -0,0 +1,145 @@
// Copyright © 2021 Ettore Di Giacinto <mudler@mocaccino.org>
//
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License along
// with this program; if not, see <http://www.gnu.org/licenses/>.
package compiler_test
import (
. "github.com/mudler/luet/pkg/compiler"
sd "github.com/mudler/luet/pkg/compiler/backend"
"github.com/mudler/luet/pkg/compiler/types/options"
pkg "github.com/mudler/luet/pkg/package"
"github.com/mudler/luet/pkg/tree"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
)
var _ = Describe("ImageHashTree", func() {
generalRecipe := tree.NewCompilerRecipe(pkg.NewInMemoryDatabase(false))
compiler := NewLuetCompiler(sd.NewSimpleDockerBackend(), generalRecipe.GetDatabase(), options.Concurrency(2))
hashtree := NewHashTree(generalRecipe.GetDatabase())
Context("Simple package definition", func() {
BeforeEach(func() {
generalRecipe = tree.NewCompilerRecipe(pkg.NewInMemoryDatabase(false))
err := generalRecipe.Load("../../tests/fixtures/buildable")
Expect(err).ToNot(HaveOccurred())
compiler = NewLuetCompiler(sd.NewSimpleDockerBackend(), generalRecipe.GetDatabase(), options.Concurrency(2))
hashtree = NewHashTree(generalRecipe.GetDatabase())
})
It("Calculates the hash correctly", func() {
spec, err := compiler.FromPackage(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})
Expect(err).ToNot(HaveOccurred())
packageHash, err := hashtree.Query(compiler, spec)
Expect(err).ToNot(HaveOccurred())
Expect(packageHash.Target.Hash.BuildHash).To(Equal("4db24406e8db30a3310a1cf8c4d4e19597745e6d41b189dc51a73ac4a50cc9e6"))
Expect(packageHash.Target.Hash.PackageHash).To(Equal("4c867c9bab6c71d9420df75806e7a2f171dbc08487852ab4e2487bab04066cf2"))
Expect(packageHash.BuilderImageHash).To(Equal("builder-e6f9c5552a67c463215b0a9e4f7c7fc8"))
})
})
expectedPackageHash := "15811d83d0f8360318c54d91dcae3714f8efb39bf872572294834880f00ee7a8"
Context("complex package definition", func() {
BeforeEach(func() {
generalRecipe = tree.NewCompilerRecipe(pkg.NewInMemoryDatabase(false))
err := generalRecipe.Load("../../tests/fixtures/upgrade_old_repo_revision")
Expect(err).ToNot(HaveOccurred())
compiler = NewLuetCompiler(sd.NewSimpleDockerBackend(), generalRecipe.GetDatabase(), options.Concurrency(2))
hashtree = NewHashTree(generalRecipe.GetDatabase())
})
It("Calculates the hash correctly", func() {
spec, err := compiler.FromPackage(&pkg.DefaultPackage{Name: "c", Category: "test", Version: "1.0"})
Expect(err).ToNot(HaveOccurred())
packageHash, err := hashtree.Query(compiler, spec)
Expect(err).ToNot(HaveOccurred())
Expect(packageHash.Dependencies[len(packageHash.Dependencies)-1].Hash.PackageHash).To(Equal(expectedPackageHash))
Expect(packageHash.SourceHash).To(Equal(expectedPackageHash))
Expect(packageHash.BuilderImageHash).To(Equal("builder-3d739cab442aec15a6da238481df73c5"))
//Expect(packageHash.Target.Hash.BuildHash).To(Equal("79d7107d13d578b362e6a7bf10ec850efce26316405b8d732ce8f9e004d64281"))
Expect(packageHash.Target.Hash.PackageHash).To(Equal("99c4ebb4bc4754985fcc28677badf90f525aa231b1db0fe75659f11b86dc20e8"))
a := &pkg.DefaultPackage{Name: "a", Category: "test", Version: "1.1"}
hash, err := packageHash.DependencyBuildImage(a)
Expect(err).ToNot(HaveOccurred())
Expect(hash).To(Equal("f48f28ab62f1379a4247ec763681ccede68ea1e5c25aae8fb72459c0b2f8742e"))
assertionA := packageHash.Dependencies.Search(a.GetFingerPrint())
Expect(assertionA.Hash.PackageHash).To(Equal(expectedPackageHash))
b := &pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"}
assertionB := packageHash.Dependencies.Search(b.GetFingerPrint())
Expect(assertionB.Hash.PackageHash).To(Equal("f48f28ab62f1379a4247ec763681ccede68ea1e5c25aae8fb72459c0b2f8742e"))
hashB, err := packageHash.DependencyBuildImage(b)
Expect(err).ToNot(HaveOccurred())
Expect(hashB).To(Equal("2668e418eab6861404834ad617713e39b8e58f68016a1fbfcc9384efdd037376"))
})
})
Context("complex package definition, with small change in build.yaml", func() {
BeforeEach(func() {
generalRecipe = tree.NewCompilerRecipe(pkg.NewInMemoryDatabase(false))
//Definition of A here is slightly changed in the steps build.yaml file (1 character only)
err := generalRecipe.Load("../../tests/fixtures/upgrade_old_repo_revision_content_changed")
Expect(err).ToNot(HaveOccurred())
compiler = NewLuetCompiler(sd.NewSimpleDockerBackend(), generalRecipe.GetDatabase(), options.Concurrency(2))
hashtree = NewHashTree(generalRecipe.GetDatabase())
})
It("Calculates the hash correctly", func() {
spec, err := compiler.FromPackage(&pkg.DefaultPackage{Name: "c", Category: "test", Version: "1.0"})
Expect(err).ToNot(HaveOccurred())
packageHash, err := hashtree.Query(compiler, spec)
Expect(err).ToNot(HaveOccurred())
Expect(packageHash.Dependencies[len(packageHash.Dependencies)-1].Hash.PackageHash).ToNot(Equal(expectedPackageHash))
sourceHash := "1d91b13d0246fa000085a1071c63397d21546300b17f69493f22315a64b717d4"
Expect(packageHash.Dependencies[len(packageHash.Dependencies)-1].Hash.PackageHash).To(Equal(sourceHash))
Expect(packageHash.SourceHash).To(Equal(sourceHash))
Expect(packageHash.SourceHash).ToNot(Equal(expectedPackageHash))
Expect(packageHash.BuilderImageHash).To(Equal("builder-03ee108a7c56b17ee568ace0800dd16d"))
//Expect(packageHash.Target.Hash.BuildHash).To(Equal("79d7107d13d578b362e6a7bf10ec850efce26316405b8d732ce8f9e004d64281"))
Expect(packageHash.Target.Hash.PackageHash).To(Equal("7677da23b2cc866c2d07aa4a58fbf703340f2f78c0efbb1ba9faf8979f250c87"))
a := &pkg.DefaultPackage{Name: "a", Category: "test", Version: "1.1"}
hash, err := packageHash.DependencyBuildImage(a)
Expect(err).ToNot(HaveOccurred())
Expect(hash).To(Equal("f48f28ab62f1379a4247ec763681ccede68ea1e5c25aae8fb72459c0b2f8742e"))
assertionA := packageHash.Dependencies.Search(a.GetFingerPrint())
Expect(assertionA.Hash.PackageHash).To(Equal("1d91b13d0246fa000085a1071c63397d21546300b17f69493f22315a64b717d4"))
Expect(assertionA.Hash.PackageHash).ToNot(Equal(expectedPackageHash))
b := &pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"}
assertionB := packageHash.Dependencies.Search(b.GetFingerPrint())
Expect(assertionB.Hash.PackageHash).To(Equal("f48f28ab62f1379a4247ec763681ccede68ea1e5c25aae8fb72459c0b2f8742e"))
hashB, err := packageHash.DependencyBuildImage(b)
Expect(err).ToNot(HaveOccurred())
Expect(hashB).To(Equal("2668e418eab6861404834ad617713e39b8e58f68016a1fbfcc9384efdd037376"))
})
})
})

View File

@@ -42,6 +42,7 @@ import (
compilerspec "github.com/mudler/luet/pkg/compiler/types/spec"
. "github.com/mudler/luet/pkg/config"
"github.com/mudler/luet/pkg/helpers"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
. "github.com/mudler/luet/pkg/logger"
pkg "github.com/mudler/luet/pkg/package"
"github.com/mudler/luet/pkg/solver"
@@ -55,12 +56,13 @@ import (
type PackageArtifact struct {
Path string `json:"path"`
Dependencies []*PackageArtifact `json:"dependencies"`
CompileSpec *compilerspec.LuetCompilationSpec `json:"compilationspec"`
Checksums Checksums `json:"checksums"`
SourceAssertion solver.PackagesAssertions `json:"-"`
CompressionType compression.Implementation `json:"compressiontype"`
Files []string `json:"files"`
Dependencies []*PackageArtifact `json:"dependencies"`
CompileSpec *compilerspec.LuetCompilationSpec `json:"compilationspec"`
Checksums Checksums `json:"checksums"`
SourceAssertion solver.PackagesAssertions `json:"-"`
CompressionType compression.Implementation `json:"compressiontype"`
Files []string `json:"files"`
PackageCacheImage string `json:"package_cacheimage"`
}
func (p *PackageArtifact) ShallowCopy() *PackageArtifact {
@@ -167,7 +169,7 @@ func CreateArtifactForFile(s string, opts ...func(*PackageArtifact)) (*PackageAr
}
defer os.RemoveAll(archive) // clean up
dst := filepath.Join(archive, fileName)
if err := helpers.CopyFile(s, dst); err != nil {
if err := fileHelper.CopyFile(s, dst); err != nil {
return nil, errors.Wrapf(err, "error while copying %s to %s", s, dst)
}
@@ -208,7 +210,7 @@ func (a *PackageArtifact) GenerateFinalImage(imageName string, b ImageBuilder, k
return builderOpts, errors.Wrap(err, "error met while uncompressing artifact "+a.Path)
}
empty, err := helpers.DirectoryIsEmpty(uncompressedFiles)
empty, err := fileHelper.DirectoryIsEmpty(uncompressedFiles)
if err != nil {
return builderOpts, errors.Wrap(err, "error met while checking if directory is empty "+uncompressedFiles)
}
@@ -217,7 +219,7 @@ func (a *PackageArtifact) GenerateFinalImage(imageName string, b ImageBuilder, k
// We can't generate FROM scratch empty images. Docker will refuse to export them
// workaround: Inject a .virtual empty file
if empty {
helpers.Touch(filepath.Join(uncompressedFiles, ".virtual"))
fileHelper.Touch(filepath.Join(uncompressedFiles, ".virtual"))
}
data := a.genDockerfile()
@@ -400,12 +402,12 @@ func tarModifierWrapperFunc(dst, path string, header *tar.Header, content io.Rea
// We want to protect file only if the hash of the files are differing OR the file size are
differs := (existingHash != "" && existingHash != tarHash) || (err != nil && f != nil && header.Size != f.Size())
// Check if exists
if helpers.Exists(destPath) && differs {
if fileHelper.Exists(destPath) && differs {
for i := 1; i < 1000; i++ {
name := filepath.Join(filepath.Join(filepath.Dir(path),
fmt.Sprintf("._cfg%04d_%s", i, filepath.Base(path))))
if helpers.Exists(name) {
if fileHelper.Exists(name) {
continue
}
Info(fmt.Sprintf("Found protected file %s. Creating %s.", destPath,
@@ -627,7 +629,7 @@ func worker(i int, wg *sync.WaitGroup, s <-chan CopyJob) {
_, err := os.Lstat(job.Dst)
if err != nil {
Debug("Copying ", job.Src)
if err := helpers.DeepCopyFile(job.Src, job.Dst); err != nil {
if err := fileHelper.DeepCopyFile(job.Src, job.Dst); err != nil {
Warning("Error copying", job, err)
}
}

View File

@@ -0,0 +1,32 @@
// Copyright © 2019 Ettore Di Giacinto <mudler@gentoo.org>
//
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License along
// with this program; if not, see <http://www.gnu.org/licenses/>.
package artifact_test
import (
"testing"
. "github.com/mudler/luet/cmd"
config "github.com/mudler/luet/pkg/config"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
)
func TestArtifact(t *testing.T) {
RegisterFailHandler(Fail)
LoadConfig(config.LuetCfg)
RunSpecs(t, "Artifact Suite")
}

View File

@@ -29,6 +29,7 @@ import (
. "github.com/mudler/luet/pkg/compiler"
helpers "github.com/mudler/luet/pkg/helpers"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
pkg "github.com/mudler/luet/pkg/package"
"github.com/mudler/luet/pkg/tree"
. "github.com/onsi/ginkgo"
@@ -41,7 +42,7 @@ var _ = Describe("Artifact", func() {
generalRecipe := tree.NewGeneralRecipe(pkg.NewInMemoryDatabase(false))
err := generalRecipe.Load("../../tests/fixtures/buildtree")
err := generalRecipe.Load("../../../../tests/fixtures/buildtree")
Expect(err).ToNot(HaveOccurred())
Expect(len(generalRecipe.GetDatabase().GetPackages())).To(Equal(1))
@@ -71,7 +72,7 @@ var _ = Describe("Artifact", func() {
err = lspec.WriteBuildImageDefinition(filepath.Join(tmpdir, "Dockerfile"))
Expect(err).ToNot(HaveOccurred())
dockerfile, err := helpers.Read(filepath.Join(tmpdir, "Dockerfile"))
dockerfile, err := fileHelper.Read(filepath.Join(tmpdir, "Dockerfile"))
Expect(err).ToNot(HaveOccurred())
Expect(dockerfile).To(Equal(`
FROM alpine
@@ -89,12 +90,12 @@ ENV PACKAGE_CATEGORY=app-admin`))
}
Expect(b.BuildImage(opts)).ToNot(HaveOccurred())
Expect(b.ExportImage(opts)).ToNot(HaveOccurred())
Expect(helpers.Exists(filepath.Join(tmpdir2, "output1.tar"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(tmpdir2, "output1.tar"))).To(BeTrue())
Expect(b.BuildImage(opts)).ToNot(HaveOccurred())
err = lspec.WriteStepImageDefinition(lspec.Image, filepath.Join(tmpdir, "LuetDockerfile"))
Expect(err).ToNot(HaveOccurred())
dockerfile, err = helpers.Read(filepath.Join(tmpdir, "LuetDockerfile"))
dockerfile, err = fileHelper.Read(filepath.Join(tmpdir, "LuetDockerfile"))
Expect(err).ToNot(HaveOccurred())
Expect(dockerfile).To(Equal(`
FROM luet/base
@@ -113,7 +114,7 @@ RUN echo bar > /test2`))
}
Expect(b.BuildImage(opts2)).ToNot(HaveOccurred())
Expect(b.ExportImage(opts2)).ToNot(HaveOccurred())
Expect(helpers.Exists(filepath.Join(tmpdir, "output2.tar"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(tmpdir, "output2.tar"))).To(BeTrue())
diffs, err := compiler.GenerateChanges(b, opts, opts2)
Expect(err).ToNot(HaveOccurred())
@@ -140,15 +141,15 @@ RUN echo bar > /test2`))
a, err := ExtractArtifactFromDelta(rootfs, filepath.Join(tmpdir, "package.tar"), diffs, 2, false, []string{}, []string{}, compression.None)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(filepath.Join(tmpdir, "package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(tmpdir, "package.tar"))).To(BeTrue())
err = helpers.Untar(a.Path, unpacked, false)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(filepath.Join(unpacked, "test"))).To(BeTrue())
Expect(helpers.Exists(filepath.Join(unpacked, "test2"))).To(BeTrue())
content1, err := helpers.Read(filepath.Join(unpacked, "test"))
Expect(fileHelper.Exists(filepath.Join(unpacked, "test"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(unpacked, "test2"))).To(BeTrue())
content1, err := fileHelper.Read(filepath.Join(unpacked, "test"))
Expect(err).ToNot(HaveOccurred())
Expect(content1).To(Equal("foo\n"))
content2, err := helpers.Read(filepath.Join(unpacked, "test2"))
content2, err := fileHelper.Read(filepath.Join(unpacked, "test2"))
Expect(err).ToNot(HaveOccurred())
Expect(content2).To(Equal("bar\n"))
@@ -156,7 +157,7 @@ RUN echo bar > /test2`))
Expect(err).ToNot(HaveOccurred())
err = a.Verify()
Expect(err).ToNot(HaveOccurred())
Expect(helpers.CopyFile(filepath.Join(tmpdir, "output2.tar"), filepath.Join(tmpdir, "package.tar"))).ToNot(HaveOccurred())
Expect(fileHelper.CopyFile(filepath.Join(tmpdir, "output2.tar"), filepath.Join(tmpdir, "package.tar"))).ToNot(HaveOccurred())
err = a.Verify()
Expect(err).To(HaveOccurred())
@@ -244,7 +245,7 @@ RUN echo bar > /test2`))
err = b.ExtractRootfs(backend.Options{ImageName: resultingImage, Destination: result}, false)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.DirectoryIsEmpty(result)).To(BeFalse())
Expect(fileHelper.DirectoryIsEmpty(result)).To(BeFalse())
content, err := ioutil.ReadFile(filepath.Join(result, ".virtual"))
Expect(err).ToNot(HaveOccurred())

View File

@@ -40,13 +40,13 @@ var _ = Describe("Checksum", func() {
Expect(len(definitionsum)).To(Equal(0))
Expect(len(definitionsum2)).To(Equal(0))
err = buildsum.Generate(NewPackageArtifact("../../tests/fixtures/layers/alpine/build.yaml"))
err = buildsum.Generate(NewPackageArtifact("../../../../tests/fixtures/layers/alpine/build.yaml"))
Expect(err).ToNot(HaveOccurred())
err = definitionsum.Generate(NewPackageArtifact("../../tests/fixtures/layers/alpine/definition.yaml"))
err = definitionsum.Generate(NewPackageArtifact("../../../../tests/fixtures/layers/alpine/definition.yaml"))
Expect(err).ToNot(HaveOccurred())
err = definitionsum2.Generate(NewPackageArtifact("../../tests/fixtures/layers/alpine/definition.yaml"))
err = definitionsum2.Generate(NewPackageArtifact("../../../../tests/fixtures/layers/alpine/definition.yaml"))
Expect(err).ToNot(HaveOccurred())
Expect(len(buildsum)).To(Equal(1))

View File

@@ -16,9 +16,11 @@
package compilerspec
import (
"fmt"
"io/ioutil"
"path/filepath"
"github.com/mitchellh/hashstructure/v2"
options "github.com/mudler/luet/pkg/compiler/types/options"
pkg "github.com/mudler/luet/pkg/package"
@@ -85,6 +87,13 @@ func (specs *LuetCompilationspecs) Unique() *LuetCompilationspecs {
return &newSpecs
}
type CopyField struct {
Package *pkg.DefaultPackage `json:"package"`
Image string `json:"image"`
Source string `json:"source"`
Destination string `json:"destination"`
}
type LuetCompilationSpec struct {
Steps []string `json:"steps"` // Are run inside a container and the result layer diff is saved
Env []string `json:"env"`
@@ -103,6 +112,43 @@ type LuetCompilationSpec struct {
Excludes []string `json:"excludes"`
BuildOptions *options.Compiler `json:"build_options"`
Copy []CopyField `json:"copy"`
Join pkg.DefaultPackages `json:"join"`
}
// Signature is a portion of the spec that yields a signature for the hash
type Signature struct {
Image string
Steps []string
PackageDir string
Prelude []string
Seed string
Env []string
Retrieve []string
Unpack bool
Includes []string
Excludes []string
Copy []CopyField
Join pkg.DefaultPackages
}
func (cs *LuetCompilationSpec) signature() Signature {
return Signature{
Image: cs.Image,
Steps: cs.Steps,
PackageDir: cs.PackageDir,
Prelude: cs.Prelude,
Seed: cs.Seed,
Env: cs.Env,
Retrieve: cs.Retrieve,
Unpack: cs.Unpack,
Includes: cs.Includes,
Excludes: cs.Excludes,
Copy: cs.Copy,
Join: cs.Join,
}
}
func NewLuetCompilationSpec(b []byte, p pkg.Package) (*LuetCompilationSpec, error) {
@@ -213,7 +259,14 @@ func (cs *LuetCompilationSpec) UnpackedPackage() bool {
// a compilation spec has an image source when it depends on other packages or have a source image
// explictly supplied
func (cs *LuetCompilationSpec) HasImageSource() bool {
return (cs.Package != nil && len(cs.GetPackage().GetRequires()) != 0) || cs.GetImage() != ""
return (cs.Package != nil && len(cs.GetPackage().GetRequires()) != 0) || cs.GetImage() != "" || len(cs.Join) != 0
}
func (cs *LuetCompilationSpec) Hash() (string, error) {
// build a signature, we want to be part of the hash only the fields that are relevant for build purposes
signature := cs.signature()
h, err := hashstructure.Hash(signature, hashstructure.FormatV2, nil)
return fmt.Sprint(h), err
}
func (cs *LuetCompilationSpec) CopyRetrieves(dest string) error {
@@ -256,6 +309,13 @@ ADD ` + s + ` /luetbuild/`
}
}
for _, c := range cs.Copy {
if c.Image != "" {
copyLine := fmt.Sprintf("\nCOPY --from=%s %s %s\n", c.Image, c.Source, c.Destination)
spec = spec + copyLine
}
}
for _, s := range cs.Env {
spec = spec + `
ENV ` + s

View File

@@ -0,0 +1,32 @@
// Copyright © 2019 Ettore Di Giacinto <mudler@gentoo.org>
//
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License along
// with this program; if not, see <http://www.gnu.org/licenses/>.
package compilerspec_test
import (
"testing"
. "github.com/mudler/luet/cmd"
config "github.com/mudler/luet/pkg/config"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
)
func TestSpec(t *testing.T) {
RegisterFailHandler(Fail)
LoadConfig(config.LuetCfg)
RunSpecs(t, "Spec Suite")
}

View File

@@ -20,10 +20,11 @@ import (
"os"
"path/filepath"
options "github.com/mudler/luet/pkg/compiler/types/options"
compilerspec "github.com/mudler/luet/pkg/compiler/types/spec"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
. "github.com/mudler/luet/pkg/compiler"
helpers "github.com/mudler/luet/pkg/helpers"
pkg "github.com/mudler/luet/pkg/package"
"github.com/mudler/luet/pkg/tree"
. "github.com/onsi/ginkgo"
@@ -74,11 +75,67 @@ var _ = Describe("Spec", func() {
})
})
Context("Image hashing", func() {
It("is stable", func() {
spec1 := &compilerspec.LuetCompilationSpec{
Image: "foo",
BuildOptions: &options.Compiler{BuildValues: []map[string]interface{}{{"foo": "bar", "baz": true}}},
Package: &pkg.DefaultPackage{
Name: "foo",
Category: "Bar",
Labels: map[string]string{
"foo": "bar",
"baz": "foo",
},
},
}
spec2 := &compilerspec.LuetCompilationSpec{
Image: "foo",
BuildOptions: &options.Compiler{BuildValues: []map[string]interface{}{{"foo": "bar", "baz": true}}},
Package: &pkg.DefaultPackage{
Name: "foo",
Category: "Bar",
Labels: map[string]string{
"foo": "bar",
"baz": "foo",
},
},
}
spec3 := &compilerspec.LuetCompilationSpec{
Image: "foo",
Steps: []string{"foo"},
Package: &pkg.DefaultPackage{
Name: "foo",
Category: "Bar",
Labels: map[string]string{
"foo": "bar",
"baz": "foo",
},
},
}
hash, err := spec1.Hash()
Expect(err).ToNot(HaveOccurred())
hash2, err := spec2.Hash()
Expect(err).ToNot(HaveOccurred())
hash3, err := spec3.Hash()
Expect(err).ToNot(HaveOccurred())
Expect(hash).To(Equal(hash2))
hashagain, err := spec2.Hash()
Expect(err).ToNot(HaveOccurred())
Expect(hash).ToNot(Equal(hash3))
Expect(hash).To(Equal(hashagain))
})
})
Context("Simple package build definition", func() {
It("Loads it correctly", func() {
generalRecipe := tree.NewGeneralRecipe(pkg.NewInMemoryDatabase(false))
err := generalRecipe.Load("../../tests/fixtures/buildtree")
err := generalRecipe.Load("../../../../tests/fixtures/buildtree")
Expect(err).ToNot(HaveOccurred())
Expect(len(generalRecipe.GetDatabase().GetPackages())).To(Equal(1))
@@ -97,7 +154,7 @@ var _ = Describe("Spec", func() {
lspec.Env = []string{"test=1"}
err = lspec.WriteBuildImageDefinition(filepath.Join(tmpdir, "Dockerfile"))
Expect(err).ToNot(HaveOccurred())
dockerfile, err := helpers.Read(filepath.Join(tmpdir, "Dockerfile"))
dockerfile, err := fileHelper.Read(filepath.Join(tmpdir, "Dockerfile"))
Expect(err).ToNot(HaveOccurred())
Expect(dockerfile).To(Equal(`
FROM alpine
@@ -110,7 +167,7 @@ ENV test=1`))
err = lspec.WriteStepImageDefinition(lspec.Image, filepath.Join(tmpdir, "Dockerfile"))
Expect(err).ToNot(HaveOccurred())
dockerfile, err = helpers.Read(filepath.Join(tmpdir, "Dockerfile"))
dockerfile, err = fileHelper.Read(filepath.Join(tmpdir, "Dockerfile"))
Expect(err).ToNot(HaveOccurred())
Expect(dockerfile).To(Equal(`
FROM luet/base
@@ -130,7 +187,7 @@ RUN echo bar > /test2`))
It("Renders retrieve and env fields", func() {
generalRecipe := tree.NewGeneralRecipe(pkg.NewInMemoryDatabase(false))
err := generalRecipe.Load("../../tests/fixtures/retrieve")
err := generalRecipe.Load("../../../../tests/fixtures/retrieve")
Expect(err).ToNot(HaveOccurred())
Expect(len(generalRecipe.GetDatabase().GetPackages())).To(Equal(1))
@@ -148,7 +205,7 @@ RUN echo bar > /test2`))
err = lspec.WriteBuildImageDefinition(filepath.Join(tmpdir, "Dockerfile"))
Expect(err).ToNot(HaveOccurred())
dockerfile, err := helpers.Read(filepath.Join(tmpdir, "Dockerfile"))
dockerfile, err := fileHelper.Read(filepath.Join(tmpdir, "Dockerfile"))
Expect(err).ToNot(HaveOccurred())
Expect(dockerfile).To(Equal(`
FROM alpine
@@ -165,7 +222,7 @@ ENV test=1`))
err = lspec.WriteBuildImageDefinition(filepath.Join(tmpdir, "Dockerfile"))
Expect(err).ToNot(HaveOccurred())
dockerfile, err = helpers.Read(filepath.Join(tmpdir, "Dockerfile"))
dockerfile, err = fileHelper.Read(filepath.Join(tmpdir, "Dockerfile"))
Expect(err).ToNot(HaveOccurred())
Expect(dockerfile).To(Equal(`
FROM alpine
@@ -180,7 +237,7 @@ ENV test=1`))
err = lspec.WriteStepImageDefinition(lspec.Image, filepath.Join(tmpdir, "Dockerfile"))
Expect(err).ToNot(HaveOccurred())
dockerfile, err = helpers.Read(filepath.Join(tmpdir, "Dockerfile"))
dockerfile, err = fileHelper.Read(filepath.Join(tmpdir, "Dockerfile"))
Expect(err).ToNot(HaveOccurred())
Expect(dockerfile).To(Equal(`

View File

@@ -27,7 +27,6 @@ import (
"strings"
"time"
"github.com/mudler/luet/pkg/helpers"
pkg "github.com/mudler/luet/pkg/package"
solver "github.com/mudler/luet/pkg/solver"
@@ -406,8 +405,13 @@ system:
}
func (c *LuetSystemConfig) InitTmpDir() error {
if !helpers.Exists(c.TmpDirBase) {
return os.MkdirAll(c.TmpDirBase, os.ModePerm)
if _, err := os.Stat(c.TmpDirBase); err != nil {
if os.IsNotExist(err) {
err = os.MkdirAll(c.TmpDirBase, os.ModePerm)
if err != nil {
return err
}
}
}
return nil
}

View File

@@ -22,7 +22,7 @@ import (
"strings"
config "github.com/mudler/luet/pkg/config"
"github.com/mudler/luet/pkg/helpers"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
@@ -38,7 +38,7 @@ var _ = Describe("Config", func() {
tmpDir, err := config.LuetCfg.GetSystem().TempDir("test1")
Expect(err).ToNot(HaveOccurred())
Expect(strings.HasPrefix(tmpDir, filepath.Join(os.TempDir(), "tmpluet"))).To(BeTrue())
Expect(helpers.Exists(tmpDir)).To(BeTrue())
Expect(fileHelper.Exists(tmpDir)).To(BeTrue())
defer os.RemoveAll(tmpDir)
})
@@ -49,7 +49,7 @@ var _ = Describe("Config", func() {
tmpFile, err := config.LuetCfg.GetSystem().TempFile("testfile1")
Expect(err).ToNot(HaveOccurred())
Expect(strings.HasPrefix(tmpFile.Name(), filepath.Join(os.TempDir(), "tmpluet"))).To(BeTrue())
Expect(helpers.Exists(tmpFile.Name())).To(BeTrue())
Expect(fileHelper.Exists(tmpFile.Name())).To(BeTrue())
defer os.Remove(tmpFile.Name())
})

View File

@@ -25,6 +25,8 @@ import (
"os"
"path/filepath"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
"github.com/docker/docker/pkg/archive"
. "github.com/mudler/luet/pkg/helpers"
. "github.com/onsi/ginkgo"
@@ -128,7 +130,7 @@ var _ = Describe("Helpers Archive", func() {
err = archive.Untar(replacerArchive, targetDir, opts)
Expect(err).ToNot(HaveOccurred())
Expect(Exists(filepath.Join(targetDir, "._cfg0001_file-0"))).Should(Equal(true))
Expect(fileHelper.Exists(filepath.Join(targetDir, "._cfg0001_file-0"))).Should(Equal(true))
})
})
})

View File

@@ -13,20 +13,27 @@
// You should have received a copy of the GNU General Public License along
// with this program; if not, see <http://www.gnu.org/licenses/>.
package helpers
package docker
import (
"context"
"encoding/hex"
"os"
"strings"
"time"
"github.com/containerd/containerd/archive"
"github.com/docker/cli/cli/trust"
"github.com/docker/distribution/reference"
"github.com/docker/docker/api/types"
"github.com/docker/docker/registry"
"github.com/mudler/luet/pkg/helpers/imgworker"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/mutate"
"github.com/google/go-containerregistry/pkg/v1/remote"
"github.com/mudler/luet/pkg/bus"
"github.com/opencontainers/go-digest"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"github.com/theupdateframework/notary/tuf/data"
)
@@ -94,10 +101,56 @@ func trustedResolveDigest(ctx context.Context, ref reference.NamedTagged, authCo
return reference.WithDigest(ref, dgst)
}
// DownloadAndExtractDockerImage is a re-adaption
// from genuinetools/img https://github.com/genuinetools/img/blob/54d0ca981c1260546d43961a538550eef55c87cf/pull.go
func DownloadAndExtractDockerImage(temp, image, dest string, auth *types.AuthConfig, verify bool) (*imgworker.ListedImage, error) {
type staticAuth struct {
auth *types.AuthConfig
}
func (s staticAuth) Authorization() (*authn.AuthConfig, error) {
if s.auth == nil {
return nil, nil
}
return &authn.AuthConfig{
Username: s.auth.Username,
Password: s.auth.Password,
Auth: s.auth.Auth,
IdentityToken: s.auth.IdentityToken,
RegistryToken: s.auth.RegistryToken,
}, nil
}
type Image struct {
// Name of the image.
//
// To be pulled, it must be a reference compatible with resolvers.
//
// This field is required.
Name string
// Labels provide runtime decoration for the image record.
//
// There is no default behavior for how these labels are propagated. They
// only decorate the static metadata object.
//
// This field is optional.
Labels map[string]string
// Target describes the root content for this image. Typically, this is
// a manifest, index or manifest list.
Target specs.Descriptor
CreatedAt, UpdatedAt time.Time
ContentSize int64
}
// UnpackEventData is the data structure to pass for the bus events
type UnpackEventData struct {
Image string
Dest string
}
// DownloadAndExtractDockerImage is a re-adaption
func DownloadAndExtractDockerImage(temp, image, dest string, auth *types.AuthConfig, verify bool) (*Image, error) {
if verify {
img, err := verifyImage(image, auth)
if err != nil {
@@ -106,22 +159,61 @@ func DownloadAndExtractDockerImage(temp, image, dest string, auth *types.AuthCon
image = img
}
defer os.RemoveAll(temp)
c, err := imgworker.New(temp, auth)
ref, err := name.ParseReference(image)
if err != nil {
return nil, errors.Wrapf(err, "failed creating client")
}
defer c.Close()
listedImage, err := c.Pull(image)
if err != nil {
return nil, errors.Wrapf(err, "failed listing images")
return nil, err
}
img, err := remote.Image(ref, remote.WithAuth(staticAuth{auth}))
if err != nil {
return nil, err
}
m, err := img.Manifest()
if err != nil {
return nil, err
}
mt, err := img.MediaType()
if err != nil {
return nil, err
}
d, err := img.Digest()
if err != nil {
return nil, err
}
reader := mutate.Extract(img)
defer reader.Close()
os.RemoveAll(temp)
os.RemoveAll(dest)
err = c.Unpack(image, dest)
return listedImage, err
if err := os.MkdirAll(dest, 0700); err != nil {
return nil, err
}
bus.Manager.Publish(bus.EventImagePreUnPack, UnpackEventData{Image: image, Dest: dest})
c, err := archive.Apply(context.TODO(), dest, reader)
if err != nil {
return nil, err
}
bus.Manager.Publish(bus.EventImagePostUnPack, UnpackEventData{Image: image, Dest: dest})
return &Image{
Name: image,
Labels: m.Annotations,
Target: specs.Descriptor{
MediaType: string(mt),
Digest: digest.Digest(d.String()),
Size: c,
},
ContentSize: c,
}, nil
}
func StripInvalidStringsFromImage(s string) string {

View File

@@ -13,10 +13,10 @@
// You should have received a copy of the GNU General Public License along
// with this program; if not, see <http://www.gnu.org/licenses/>.
package helpers_test
package docker_test
import (
. "github.com/mudler/luet/pkg/helpers"
"github.com/mudler/luet/pkg/helpers/docker"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
)
@@ -24,7 +24,7 @@ import (
var _ = Describe("StripInvalidStringsFromImage", func() {
Context("Image names", func() {
It("strips invalid chars", func() {
Expect(StripInvalidStringsFromImage("foo+bar")).To(Equal("foo-bar"))
Expect(docker.StripInvalidStringsFromImage("foo+bar")).To(Equal("foo-bar"))
})
})
})

View File

@@ -13,7 +13,7 @@
// You should have received a copy of the GNU General Public License along
// with this program; if not, see <http://www.gnu.org/licenses/>.
package helpers
package file
import (
"fmt"

View File

@@ -13,14 +13,15 @@
// You should have received a copy of the GNU General Public License along
// with this program; if not, see <http://www.gnu.org/licenses/>.
package helpers_test
package file_test
import (
"io/ioutil"
"os"
"path/filepath"
. "github.com/mudler/luet/pkg/helpers"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
)
@@ -28,8 +29,8 @@ import (
var _ = Describe("Helpers", func() {
Context("Exists", func() {
It("Detect existing and not-existing files", func() {
Expect(Exists("../../tests/fixtures/buildtree/app-admin/enman/1.4.0/build.yaml")).To(BeTrue())
Expect(Exists("../../tests/fixtures/buildtree/app-admin/enman/1.4.0/build.yaml.not.exists")).To(BeFalse())
Expect(fileHelper.Exists("../../tests/fixtures/buildtree/app-admin/enman/1.4.0/build.yaml")).To(BeTrue())
Expect(fileHelper.Exists("../../tests/fixtures/buildtree/app-admin/enman/1.4.0/build.yaml.not.exists")).To(BeFalse())
})
})
@@ -38,15 +39,15 @@ var _ = Describe("Helpers", func() {
testDir, err := ioutil.TempDir(os.TempDir(), "test")
Expect(err).ToNot(HaveOccurred())
defer os.RemoveAll(testDir)
Expect(DirectoryIsEmpty(testDir)).To(BeTrue())
Expect(fileHelper.DirectoryIsEmpty(testDir)).To(BeTrue())
})
It("Detects directory with files", func() {
testDir, err := ioutil.TempDir(os.TempDir(), "test")
Expect(err).ToNot(HaveOccurred())
defer os.RemoveAll(testDir)
err = Touch(filepath.Join(testDir, "foo"))
err = fileHelper.Touch(filepath.Join(testDir, "foo"))
Expect(err).ToNot(HaveOccurred())
Expect(DirectoryIsEmpty(testDir)).To(BeFalse())
Expect(fileHelper.DirectoryIsEmpty(testDir)).To(BeFalse())
})
})
@@ -72,7 +73,7 @@ var _ = Describe("Helpers", func() {
err = ioutil.WriteFile(filepath.Join(testDir, "baz2", "foo"), []byte("test\n"), 0644)
Expect(err).ToNot(HaveOccurred())
ordered, notExisting := OrderFiles(testDir, []string{"bar", "baz", "bar/foo", "baz2", "foo", "baz2/foo", "notexisting"})
ordered, notExisting := fileHelper.OrderFiles(testDir, []string{"bar", "baz", "bar/foo", "baz2", "foo", "baz2/foo", "notexisting"})
Expect(ordered).To(Equal([]string{"baz", "bar/foo", "foo", "baz2/foo", "bar", "baz2"}))
Expect(notExisting).To(Equal([]string{"notexisting"}))
@@ -96,7 +97,7 @@ var _ = Describe("Helpers", func() {
err = os.MkdirAll(filepath.Join(testDir, "foo", "baz", "fa"), os.ModePerm)
Expect(err).ToNot(HaveOccurred())
ordered, _ := OrderFiles(testDir, []string{"foo", "foo/bar", "bar", "foo/baz/fa", "foo/baz"})
ordered, _ := fileHelper.OrderFiles(testDir, []string{"foo", "foo/bar", "bar", "foo/baz/fa", "foo/baz"})
Expect(ordered).To(Equal([]string{"foo/baz/fa", "foo/bar", "foo/baz", "foo", "bar"}))
})
})

View File

@@ -3,6 +3,8 @@ package helpers
import (
"io/ioutil"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
"github.com/imdario/mergo"
"github.com/pkg/errors"
"gopkg.in/yaml.v2"
@@ -75,7 +77,7 @@ func RenderFiles(toTemplate, valuesFile string, defaultFile ...string) (string,
return "", errors.Wrap(err, "reading file "+toTemplate)
}
if !Exists(valuesFile) {
if !fileHelper.Exists(valuesFile) {
return "", errors.Wrap(err, "file not existing "+valuesFile)
}
val, err := ioutil.ReadFile(valuesFile)

View File

@@ -1,36 +0,0 @@
package imgworker
import (
"context"
"github.com/docker/docker/api/types"
"github.com/moby/buildkit/session"
"github.com/moby/buildkit/session/auth"
"google.golang.org/grpc"
)
func NewDockerAuthProvider(auth *types.AuthConfig) session.Attachable {
return &authProvider{
config: auth,
}
}
type authProvider struct {
config *types.AuthConfig
}
func (ap *authProvider) Register(server *grpc.Server) {
// no-op
}
func (ap *authProvider) Credentials(ctx context.Context, req *auth.CredentialsRequest) (*auth.CredentialsResponse, error) {
res := &auth.CredentialsResponse{}
if ap.config.IdentityToken != "" {
res.Secret = ap.config.IdentityToken
} else {
res.Username = ap.config.Username
res.Secret = ap.config.Password
}
return res, nil
}

View File

@@ -1,81 +0,0 @@
package imgworker
// FROM Slightly adapted from genuinetools/img worker
import (
"context"
"os"
"path/filepath"
"github.com/containerd/containerd/namespaces"
dockertypes "github.com/docker/docker/api/types"
"github.com/genuinetools/img/types"
"github.com/moby/buildkit/control"
"github.com/moby/buildkit/session"
"github.com/moby/buildkit/util/appcontext"
"github.com/moby/buildkit/worker/base"
"github.com/pkg/errors"
)
// Client holds the information for the client we will use for communicating
// with the buildkit controller.
type Client struct {
backend string
localDirs map[string]string
root string
sessionManager *session.Manager
controller *control.Controller
opts *base.WorkerOpt
sess *session.Session
ctx context.Context
auth *dockertypes.AuthConfig
}
// New returns a new client for communicating with the buildkit controller.
func New(root string, auth *dockertypes.AuthConfig) (*Client, error) {
// Native backend is fine, our images have just one layer. No need to depend on anything
backend := types.NativeBackend
// Create the root/
root = filepath.Join(root, "runc", backend)
if err := os.MkdirAll(root, 0700); err != nil {
return nil, err
}
c := &Client{
backend: types.NativeBackend,
root: root,
localDirs: nil,
auth: auth,
}
if err := c.prepare(); err != nil {
return nil, errors.Wrapf(err, "failed preparing client")
}
// Create the start of the client.
return c, nil
}
func (c *Client) Close() {
c.sess.Close()
}
func (c *Client) prepare() error {
ctx := appcontext.Context()
sess, sessDialer, err := c.Session(ctx)
if err != nil {
return errors.Wrapf(err, "failed creating Session")
}
ctx = session.NewContext(ctx, sess.ID())
ctx = namespaces.WithNamespace(ctx, "buildkit")
c.ctx = ctx
c.sess = sess
go func() {
sess.Run(ctx, sessDialer)
}()
return nil
}

View File

@@ -1,129 +0,0 @@
package imgworker
// FROM Slightly adapted from genuinetools/img worker
import (
"fmt"
"github.com/containerd/containerd/images"
"github.com/containerd/containerd/platforms"
"github.com/docker/distribution/reference"
"github.com/moby/buildkit/cache"
"github.com/moby/buildkit/exporter"
imageexporter "github.com/moby/buildkit/exporter/containerimage"
"github.com/moby/buildkit/source"
"github.com/moby/buildkit/source/containerimage"
)
// ListedImage represents an image structure returuned from ListImages.
// It extends containerd/images.Image with extra fields.
type ListedImage struct {
images.Image
ContentSize int64
}
// Pull retrieves an image from a remote registry.
func (c *Client) Pull(image string) (*ListedImage, error) {
ctx := c.ctx
sm, err := c.getSessionManager()
if err != nil {
return nil, err
}
// Parse the image name and tag.
named, err := reference.ParseNormalizedNamed(image)
if err != nil {
return nil, fmt.Errorf("parsing image name %q failed: %v", image, err)
}
// Add the latest lag if they did not provide one.
named = reference.TagNameOnly(named)
image = named.String()
// Get the identifier for the image.
identifier, err := source.NewImageIdentifier(image)
if err != nil {
return nil, err
}
// Create the worker opts.
opt, err := c.createWorkerOpt()
if err != nil {
return nil, fmt.Errorf("creating worker opt failed: %v", err)
}
cm, err := cache.NewManager(cache.ManagerOpt{
Snapshotter: opt.Snapshotter,
MetadataStore: opt.MetadataStore,
ContentStore: opt.ContentStore,
LeaseManager: opt.LeaseManager,
GarbageCollect: opt.GarbageCollect,
Applier: opt.Applier,
})
if err != nil {
return nil, err
}
// Create the source for the pull.
srcOpt := containerimage.SourceOpt{
Snapshotter: opt.Snapshotter,
ContentStore: opt.ContentStore,
Applier: opt.Applier,
CacheAccessor: cm,
ImageStore: opt.ImageStore,
RegistryHosts: opt.RegistryHosts,
LeaseManager: opt.LeaseManager,
}
src, err := containerimage.NewSource(srcOpt)
if err != nil {
return nil, err
}
s, err := src.Resolve(ctx, identifier, sm)
if err != nil {
return nil, err
}
ref, err := s.Snapshot(ctx)
if err != nil {
return nil, err
}
// Create the exporter for the pull.
iw, err := imageexporter.NewImageWriter(imageexporter.WriterOpt{
Snapshotter: opt.Snapshotter,
ContentStore: opt.ContentStore,
Differ: opt.Differ,
})
if err != nil {
return nil, err
}
expOpt := imageexporter.Opt{
SessionManager: sm,
ImageWriter: iw,
Images: opt.ImageStore,
RegistryHosts: opt.RegistryHosts,
LeaseManager: opt.LeaseManager,
}
exp, err := imageexporter.New(expOpt)
if err != nil {
return nil, err
}
e, err := exp.Resolve(ctx, map[string]string{"name": image})
if err != nil {
return nil, err
}
if _, err := e.Export(ctx, exporter.Source{Ref: ref}); err != nil {
return nil, err
}
// Get the image.
img, err := opt.ImageStore.Get(ctx, image)
if err != nil {
return nil, fmt.Errorf("getting image %s from image store failed: %v", image, err)
}
size, err := img.Size(ctx, opt.ContentStore, platforms.Default())
if err != nil {
return nil, fmt.Errorf("calculating size of image %s failed: %v", img.Name, err)
}
return &ListedImage{Image: img, ContentSize: size}, nil
}

View File

@@ -1,49 +0,0 @@
package imgworker
// FROM Slightly adapted from genuinetools/img worker
import (
"context"
"github.com/moby/buildkit/session"
"github.com/moby/buildkit/session/filesync"
"github.com/moby/buildkit/session/testutil"
"github.com/pkg/errors"
)
func (c *Client) getSessionManager() (*session.Manager, error) {
if c.sessionManager == nil {
var err error
c.sessionManager, err = session.NewManager()
if err != nil {
return nil, err
}
}
return c.sessionManager, nil
}
// Session creates the session manager and returns the session and it's
// dialer.
func (c *Client) Session(ctx context.Context) (*session.Session, session.Dialer, error) {
m, err := c.getSessionManager()
if err != nil {
return nil, nil, errors.Wrap(err, "failed to create session manager")
}
sessionName := "luet"
s, err := session.NewSession(ctx, sessionName, "")
if err != nil {
return nil, nil, errors.Wrap(err, "failed to create session")
}
syncedDirs := make([]filesync.SyncedDir, 0, len(c.localDirs))
for name, d := range c.localDirs {
syncedDirs = append(syncedDirs, filesync.SyncedDir{Name: name, Dir: d})
}
s.Allow(filesync.NewFSSyncProvider(syncedDirs))
s.Allow(NewDockerAuthProvider(c.auth))
return s, sessionDialer(s, m), err
}
func sessionDialer(s *session.Session, m *session.Manager) session.Dialer {
// FIXME: rename testutil
return session.Dialer(testutil.TestStream(testutil.Handler(m.HandleConn)))
}

View File

@@ -1,82 +0,0 @@
package imgworker
// FROM Slightly adapted from genuinetools/img worker
import (
"errors"
"fmt"
"os"
"github.com/containerd/containerd/content"
"github.com/containerd/containerd/images"
"github.com/containerd/containerd/platforms"
"github.com/docker/distribution/reference"
"github.com/docker/docker/pkg/archive"
"github.com/sirupsen/logrus"
)
// TODO: this requires root permissions to mount/unmount layers, althrought it shouldn't be required.
// See how backends are unpacking images without asking for root permissions.
// Unpack exports an image to a rootfs destination directory.
func (c *Client) Unpack(image, dest string) error {
ctx := c.ctx
if len(dest) < 1 {
return errors.New("destination directory for rootfs cannot be empty")
}
if _, err := os.Stat(dest); err == nil {
return fmt.Errorf("destination directory already exists: %s", dest)
}
// Parse the image name and tag.
named, err := reference.ParseNormalizedNamed(image)
if err != nil {
return fmt.Errorf("parsing image name %q failed: %v", image, err)
}
// Add the latest lag if they did not provide one.
named = reference.TagNameOnly(named)
image = named.String()
// Create the worker opts.
opt, err := c.createWorkerOpt()
if err != nil {
return fmt.Errorf("creating worker opt failed: %v", err)
}
if opt.ImageStore == nil {
return errors.New("image store is nil")
}
img, err := opt.ImageStore.Get(ctx, image)
if err != nil {
return fmt.Errorf("getting image %s from image store failed: %v", image, err)
}
manifest, err := images.Manifest(ctx, opt.ContentStore, img.Target, platforms.Default())
if err != nil {
return fmt.Errorf("getting image manifest failed: %v", err)
}
for _, desc := range manifest.Layers {
logrus.Debugf("Unpacking layer %s", desc.Digest.String())
// Read the blob from the content store.
layer, err := opt.ContentStore.ReaderAt(ctx, desc)
if err != nil {
return fmt.Errorf("getting reader for digest %s failed: %v", desc.Digest.String(), err)
}
// Unpack the tarfile to the rootfs path.
// FROM: https://godoc.org/github.com/moby/moby/pkg/archive#TarOptions
if err := archive.Untar(content.NewReader(layer), dest, &archive.TarOptions{
NoLchown: false,
ExcludePatterns: []string{"dev/"}, // prevent 'operation not permitted'
}); err != nil {
return fmt.Errorf("extracting tar for %s to directory %s failed: %v", desc.Digest.String(), dest, err)
}
}
return nil
}

View File

@@ -1,106 +0,0 @@
package imgworker
// FROM Slightly adapted from genuinetools/img worker
import (
"context"
"fmt"
"path/filepath"
"github.com/containerd/containerd/content/local"
"github.com/containerd/containerd/diff/apply"
"github.com/containerd/containerd/diff/walking"
ctdmetadata "github.com/containerd/containerd/metadata"
"github.com/containerd/containerd/platforms"
"github.com/containerd/containerd/remotes/docker"
ctdsnapshot "github.com/containerd/containerd/snapshots"
"github.com/containerd/containerd/snapshots/native"
"github.com/moby/buildkit/cache/metadata"
containerdsnapshot "github.com/moby/buildkit/snapshot/containerd"
"github.com/moby/buildkit/util/binfmt_misc"
"github.com/moby/buildkit/util/leaseutil"
"github.com/moby/buildkit/worker/base"
specs "github.com/opencontainers/image-spec/specs-go/v1"
bolt "go.etcd.io/bbolt"
)
// createWorkerOpt creates a base.WorkerOpt to be used for a new worker.
func (c *Client) createWorkerOpt() (opt base.WorkerOpt, err error) {
if c.opts != nil {
return *c.opts, nil
}
// Create the metadata store.
md, err := metadata.NewStore(filepath.Join(c.root, "metadata.db"))
if err != nil {
return opt, err
}
snapshotRoot := filepath.Join(c.root, "snapshots")
s, err := native.NewSnapshotter(snapshotRoot)
if err != nil {
return opt, fmt.Errorf("creating %s snapshotter failed: %v", c.backend, err)
}
// Create the content store locally.
contentStore, err := local.NewStore(filepath.Join(c.root, "content"))
if err != nil {
return opt, err
}
// Open the bolt database for metadata.
db, err := bolt.Open(filepath.Join(c.root, "containerdmeta.db"), 0644, nil)
if err != nil {
return opt, err
}
// Create the new database for metadata.
mdb := ctdmetadata.NewDB(db, contentStore, map[string]ctdsnapshot.Snapshotter{
c.backend: s,
})
if err := mdb.Init(context.TODO()); err != nil {
return opt, err
}
// Create the image store.
imageStore := ctdmetadata.NewImageStore(mdb)
contentStore = containerdsnapshot.NewContentStore(mdb.ContentStore(), "buildkit")
id, err := base.ID(c.root)
if err != nil {
return opt, err
}
xlabels := base.Labels("oci", c.backend)
var supportedPlatforms []specs.Platform
for _, p := range binfmt_misc.SupportedPlatforms(false) {
parsed, err := platforms.Parse(p)
if err != nil {
return opt, err
}
supportedPlatforms = append(supportedPlatforms, platforms.Normalize(parsed))
}
opt = base.WorkerOpt{
ID: id,
Labels: xlabels,
MetadataStore: md,
Snapshotter: containerdsnapshot.NewSnapshotter(c.backend, mdb.Snapshotter(c.backend), "buildkit", nil),
ContentStore: contentStore,
Applier: apply.NewFileSystemApplier(contentStore),
Differ: walking.NewWalkingDiff(contentStore),
ImageStore: imageStore,
Platforms: supportedPlatforms,
RegistryHosts: docker.ConfigureDefaultRegistries(),
LeaseManager: leaseutil.WithNamespace(ctdmetadata.NewLeaseManager(mdb), "buildkit"),
GarbageCollect: mdb.GarbageCollect,
}
c.opts = &opt
return opt, err
}

View File

@@ -14,7 +14,7 @@
// You should have received a copy of the GNU General Public License along
// with this program; if not, see <http://www.gnu.org/licenses/>.
package helpers
package match
import (
"reflect"

View File

@@ -28,8 +28,8 @@ import (
"github.com/mudler/luet/pkg/compiler/types/artifact"
"github.com/mudler/luet/pkg/config"
"github.com/mudler/luet/pkg/helpers"
"github.com/mudler/luet/pkg/helpers/imgworker"
"github.com/mudler/luet/pkg/helpers/docker"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
. "github.com/mudler/luet/pkg/logger"
)
@@ -64,7 +64,7 @@ func (c *DockerClient) DownloadArtifact(a *artifact.PackageArtifact) (*artifact.
artifactName := path.Base(a.Path)
cacheFile := filepath.Join(config.LuetCfg.GetSystem().GetSystemPkgsCacheDirPath(), artifactName)
Debug("Cache file", cacheFile)
if err := helpers.EnsureDir(cacheFile); err != nil {
if err := fileHelper.EnsureDir(cacheFile); err != nil {
return nil, errors.Wrapf(err, "could not create cache folder %s for %s", config.LuetCfg.GetSystem().GetSystemPkgsCacheDirPath(), cacheFile)
}
ok := false
@@ -77,7 +77,7 @@ func (c *DockerClient) DownloadArtifact(a *artifact.PackageArtifact) (*artifact.
// is done in such cases (see repository.go)
// Check if file is already in cache
if helpers.Exists(cacheFile) {
if fileHelper.Exists(cacheFile) {
Debug("Cache hit for artifact", artifactName)
resultingArtifact = a
resultingArtifact.Path = cacheFile
@@ -102,7 +102,7 @@ func (c *DockerClient) DownloadArtifact(a *artifact.PackageArtifact) (*artifact.
}
// imageName := fmt.Sprintf("%s/%s", uri, artifact.GetCompileSpec().GetPackage().GetPackageImageName())
info, err := helpers.DownloadAndExtractDockerImage(contentstore, imageName, temp, c.auth, c.RepoData.Verify)
info, err := docker.DownloadAndExtractDockerImage(contentstore, imageName, temp, c.auth, c.RepoData.Verify)
if err != nil {
Warning(fmt.Sprintf(errImageDownloadMsg, imageName, err.Error()))
continue
@@ -140,7 +140,7 @@ func (c *DockerClient) DownloadFile(name string) (string, error) {
var file *os.File = nil
var err error
var temp, contentstore string
var info *imgworker.ListedImage
var info *docker.Image
// Files should be in URI/repository:<file>
ok := false
@@ -161,10 +161,10 @@ func (c *DockerClient) DownloadFile(name string) (string, error) {
continue
}
imageName := fmt.Sprintf("%s:%s", uri, helpers.StripInvalidStringsFromImage(name))
imageName := fmt.Sprintf("%s:%s", uri, docker.StripInvalidStringsFromImage(name))
Info("Downloading", imageName)
info, err = helpers.DownloadAndExtractDockerImage(contentstore, imageName, temp, c.auth, c.RepoData.Verify)
info, err = docker.DownloadAndExtractDockerImage(contentstore, imageName, temp, c.auth, c.RepoData.Verify)
if err != nil {
Warning(fmt.Sprintf(errImageDownloadMsg, imageName, err.Error()))
continue
@@ -174,7 +174,7 @@ func (c *DockerClient) DownloadFile(name string) (string, error) {
Info(fmt.Sprintf("Size: %s", units.BytesSize(float64(info.ContentSize))))
Debug("\nCopying file ", filepath.Join(temp, name), "to", file.Name())
err = helpers.CopyFile(filepath.Join(temp, name), file.Name())
err = fileHelper.CopyFile(filepath.Join(temp, name), file.Name())
if err != nil {
continue
}

View File

@@ -22,8 +22,8 @@ import (
"github.com/mudler/luet/pkg/compiler/types/artifact"
compilerspec "github.com/mudler/luet/pkg/compiler/types/spec"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
helpers "github.com/mudler/luet/pkg/helpers"
pkg "github.com/mudler/luet/pkg/package"
. "github.com/mudler/luet/pkg/installer/client"
@@ -32,7 +32,7 @@ import (
)
// This test expect that the repository defined in UNIT_TEST_DOCKER_IMAGE is in zstd format.
// the repository is built by the 01_simple_docker.sh integration test file.
// the repository is built by the 01_simple_docker.sh integration test fileHelper.
// This test also require root. At the moment, unpacking docker images with 'img' requires root permission to
// mount/unmount layers.
var _ = Describe("Docker client", func() {
@@ -51,7 +51,7 @@ var _ = Describe("Docker client", func() {
It("Downloads single files", func() {
f, err := c.DownloadFile("repository.yaml")
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Read(f)).To(ContainSubstring("Test Repo"))
Expect(fileHelper.Read(f)).To(ContainSubstring("Test Repo"))
os.RemoveAll(f)
})
@@ -71,8 +71,8 @@ var _ = Describe("Docker client", func() {
Expect(err).ToNot(HaveOccurred())
defer os.RemoveAll(tmpdir) // clean up
Expect(f.Unpack(tmpdir, false)).ToNot(HaveOccurred())
Expect(helpers.Read(filepath.Join(tmpdir, "c"))).To(Equal("c\n"))
Expect(helpers.Read(filepath.Join(tmpdir, "cd"))).To(Equal("c\n"))
Expect(fileHelper.Read(filepath.Join(tmpdir, "c"))).To(Equal("c\n"))
Expect(fileHelper.Read(filepath.Join(tmpdir, "cd"))).To(Equal("c\n"))
os.RemoveAll(f.Path)
})
})

View File

@@ -25,12 +25,11 @@ import (
"time"
"github.com/mudler/luet/pkg/compiler/types/artifact"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
. "github.com/mudler/luet/pkg/logger"
"github.com/mudler/luet/pkg/config"
"github.com/mudler/luet/pkg/helpers"
"github.com/cavaliercoder/grab"
"github.com/mudler/luet/pkg/config"
"github.com/schollz/progressbar/v3"
)
@@ -77,7 +76,7 @@ func (c *HttpClient) DownloadArtifact(a *artifact.PackageArtifact) (*artifact.Pa
ok := false
// Check if file is already in cache
if helpers.Exists(cacheFile) {
if fileHelper.Exists(cacheFile) {
Debug("Use artifact", artifactName, "from cache.")
} else {
@@ -156,7 +155,7 @@ func (c *HttpClient) DownloadArtifact(a *artifact.PackageArtifact) (*artifact.Pa
fmt.Sprintf("%.2f", (float64(resp.BytesPerSecond())/1024)/1024), "MiB/s )")
Debug("\nCopying file ", filepath.Join(temp, artifactName), "to", cacheFile)
err = helpers.CopyFile(filepath.Join(temp, artifactName), cacheFile)
err = fileHelper.CopyFile(filepath.Join(temp, artifactName), cacheFile)
bar.Finish()
ok = true
@@ -218,7 +217,7 @@ func (c *HttpClient) DownloadFile(name string) (string, error) {
fmt.Sprintf("%.2f", (float64(resp.BytesComplete())/1000)/1000), "MB (",
fmt.Sprintf("%.2f", (float64(resp.BytesPerSecond())/1024)/1024), "MiB/s )")
err = helpers.CopyFile(filepath.Join(temp, name), file.Name())
err = fileHelper.CopyFile(filepath.Join(temp, name), file.Name())
if err != nil {
continue
}

View File

@@ -23,8 +23,7 @@ import (
"path/filepath"
"github.com/mudler/luet/pkg/compiler/types/artifact"
helpers "github.com/mudler/luet/pkg/helpers"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
. "github.com/mudler/luet/pkg/installer/client"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
@@ -47,7 +46,7 @@ var _ = Describe("Http client", func() {
c := NewHttpClient(RepoData{Urls: []string{ts.URL}})
path, err := c.DownloadFile("test.txt")
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Read(path)).To(Equal("test"))
Expect(fileHelper.Read(path)).To(Equal("test"))
os.RemoveAll(path)
})
@@ -65,7 +64,7 @@ var _ = Describe("Http client", func() {
c := NewHttpClient(RepoData{Urls: []string{ts.URL}})
path, err := c.DownloadArtifact(&artifact.PackageArtifact{Path: "test.txt"})
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Read(path.Path)).To(Equal("test"))
Expect(fileHelper.Read(path.Path)).To(Equal("test"))
os.RemoveAll(path.Path)
})

View File

@@ -22,9 +22,8 @@ import (
"github.com/mudler/luet/pkg/compiler/types/artifact"
"github.com/mudler/luet/pkg/config"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
. "github.com/mudler/luet/pkg/logger"
"github.com/mudler/luet/pkg/helpers"
)
type LocalClient struct {
@@ -50,7 +49,7 @@ func (c *LocalClient) DownloadArtifact(a *artifact.PackageArtifact) (*artifact.P
}
// Check if file is already in cache
if helpers.Exists(cacheFile) {
if fileHelper.Exists(cacheFile) {
Debug("Use artifact", artifactName, "from cache.")
} else {
ok := false
@@ -61,7 +60,7 @@ func (c *LocalClient) DownloadArtifact(a *artifact.PackageArtifact) (*artifact.P
Info("Downloading artifact", artifactName, "from", uri)
//defer os.Remove(file.Name())
err = helpers.CopyFile(filepath.Join(uri, artifactName), cacheFile)
err = fileHelper.CopyFile(filepath.Join(uri, artifactName), cacheFile)
if err != nil {
continue
}
@@ -104,7 +103,7 @@ func (c *LocalClient) DownloadFile(name string) (string, error) {
}
//defer os.Remove(file.Name())
err = helpers.CopyFile(filepath.Join(uri, name), file.Name())
err = fileHelper.CopyFile(filepath.Join(uri, name), file.Name())
if err != nil {
continue
}

View File

@@ -21,8 +21,7 @@ import (
"path/filepath"
"github.com/mudler/luet/pkg/compiler/types/artifact"
helpers "github.com/mudler/luet/pkg/helpers"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
. "github.com/mudler/luet/pkg/installer/client"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
@@ -42,7 +41,7 @@ var _ = Describe("Local client", func() {
c := NewLocalClient(RepoData{Urls: []string{tmpdir}})
path, err := c.DownloadFile("test.txt")
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Read(path)).To(Equal("test"))
Expect(fileHelper.Read(path)).To(Equal("test"))
os.RemoveAll(path)
})
@@ -58,7 +57,7 @@ var _ = Describe("Local client", func() {
c := NewLocalClient(RepoData{Urls: []string{tmpdir}})
path, err := c.DownloadArtifact(&artifact.PackageArtifact{Path: "test.txt"})
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Read(path.Path)).To(Equal("test"))
Expect(fileHelper.Read(path.Path)).To(Equal("test"))
os.RemoveAll(path.Path)
})

View File

@@ -24,16 +24,16 @@ import (
"strings"
"sync"
artifact "github.com/mudler/luet/pkg/compiler/types/artifact"
. "github.com/logrusorgru/aurora"
"github.com/mudler/luet/pkg/bus"
artifact "github.com/mudler/luet/pkg/compiler/types/artifact"
"github.com/mudler/luet/pkg/config"
"github.com/mudler/luet/pkg/helpers"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
"github.com/mudler/luet/pkg/helpers/match"
. "github.com/mudler/luet/pkg/logger"
pkg "github.com/mudler/luet/pkg/package"
"github.com/mudler/luet/pkg/solver"
. "github.com/logrusorgru/aurora"
"github.com/pkg/errors"
)
@@ -597,7 +597,7 @@ func (l *LuetInstaller) Reclaim(s *System) error {
"from", repo.GetName(), "is installed")
FILES:
for _, f := range artefact.Files {
if helpers.Exists(filepath.Join(s.Target, f)) {
if fileHelper.Exists(filepath.Join(s.Target, f)) {
p, err := repo.GetTree().GetDatabase().FindPackage(artefact.CompileSpec.GetPackage())
if err != nil {
return err
@@ -915,7 +915,7 @@ func pruneEmptyFilePath(path string) {
currentPath = filepath.Join(currentPath, p)
allPaths = append(allPaths, currentPath)
}
helpers.ReverseAny(allPaths)
match.ReverseAny(allPaths)
for _, p := range allPaths {
checkAndPrunePath(p)
}
@@ -943,7 +943,7 @@ func (l *LuetInstaller) uninstall(p pkg.Package, s *System) error {
cp.Map(files)
}
toRemove, notPresent := helpers.OrderFiles(s.Target, files)
toRemove, notPresent := fileHelper.OrderFiles(s.Target, files)
// Remove from target
for _, f := range toRemove {

View File

@@ -27,6 +27,7 @@ import (
"github.com/mudler/luet/pkg/compiler/types/options"
compilerspec "github.com/mudler/luet/pkg/compiler/types/spec"
"github.com/mudler/luet/pkg/helpers"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
. "github.com/mudler/luet/pkg/installer"
pkg "github.com/mudler/luet/pkg/package"
@@ -84,34 +85,34 @@ var _ = Describe("Installer", func() {
a, err := c.Compile(false, spec)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(a.Path)).To(BeTrue())
Expect(fileHelper.Exists(a.Path)).To(BeTrue())
Expect(helpers.Untar(a.Path, tmpdir, false)).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test6"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test6"))).To(BeTrue())
content1, err := helpers.Read(spec.Rel("test5"))
content1, err := fileHelper.Read(spec.Rel("test5"))
Expect(err).ToNot(HaveOccurred())
content2, err := helpers.Read(spec.Rel("test6"))
content2, err := fileHelper.Read(spec.Rel("test6"))
Expect(err).ToNot(HaveOccurred())
Expect(content1).To(Equal("artifact5\n"))
Expect(content2).To(Equal("artifact6\n"))
Expect(helpers.Exists(spec.Rel("b-test-1.0.package.tar"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("b-test-1.0.metadata.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("b-test-1.0.package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("b-test-1.0.metadata.yaml"))).To(BeTrue())
repo, err := stubRepo(tmpdir, "../../tests/fixtures/buildable")
Expect(err).ToNot(HaveOccurred())
Expect(repo.GetName()).To(Equal("test"))
Expect(helpers.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
err = repo.Write(tmpdir, false, false)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("repository.yaml"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("repository.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(repo.GetUrls()[0]).To(Equal(tmpdir))
Expect(repo.GetType()).To(Equal("disk"))
@@ -136,8 +137,8 @@ urls:
err = inst.Install([]pkg.Package{&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"}}, system)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(filepath.Join(fakeroot, "test5"))).To(BeTrue())
Expect(helpers.Exists(filepath.Join(fakeroot, "test6"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test5"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test6"))).To(BeTrue())
_, err = systemDB.FindPackage(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})
Expect(err).ToNot(HaveOccurred())
@@ -154,8 +155,8 @@ urls:
Expect(err).ToNot(HaveOccurred())
// Nothing should be there anymore (files, packagedb entry)
Expect(helpers.Exists(filepath.Join(fakeroot, "test5"))).ToNot(BeTrue())
Expect(helpers.Exists(filepath.Join(fakeroot, "test6"))).ToNot(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test5"))).ToNot(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test6"))).ToNot(BeTrue())
_, err = systemDB.FindPackage(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})
Expect(err).To(HaveOccurred())
@@ -199,21 +200,21 @@ urls:
artifact, err := c.Compile(false, spec)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test6"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test6"))).To(BeTrue())
content1, err := helpers.Read(spec.Rel("test5"))
content1, err := fileHelper.Read(spec.Rel("test5"))
Expect(err).ToNot(HaveOccurred())
content2, err := helpers.Read(spec.Rel("test6"))
content2, err := fileHelper.Read(spec.Rel("test6"))
Expect(err).ToNot(HaveOccurred())
Expect(content1).To(Equal("artifact5\n"))
Expect(content2).To(Equal("artifact6\n"))
Expect(helpers.Exists(spec.Rel("b-test-1.0.package.tar"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("b-test-1.0.metadata.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("b-test-1.0.package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("b-test-1.0.metadata.yaml"))).To(BeTrue())
repo, err := stubRepo(tmpdir, "../../tests/fixtures/buildable")
Expect(err).ToNot(HaveOccurred())
@@ -223,15 +224,15 @@ urls:
repo.SetRepositoryFile(REPOFILE_TREE_KEY, treeFile)
Expect(err).ToNot(HaveOccurred())
Expect(repo.GetName()).To(Equal("test"))
Expect(helpers.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
err = repo.Write(tmpdir, false, false)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("repository.yaml"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("repository.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(repo.GetUrls()[0]).To(Equal(tmpdir))
Expect(repo.GetType()).To(Equal("disk"))
@@ -256,8 +257,8 @@ urls:
err = inst.Install([]pkg.Package{&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"}}, system)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(filepath.Join(fakeroot, "test5"))).To(BeTrue())
Expect(helpers.Exists(filepath.Join(fakeroot, "test6"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test5"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test6"))).To(BeTrue())
_, err = systemDB.FindPackage(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})
Expect(err).ToNot(HaveOccurred())
@@ -274,8 +275,8 @@ urls:
Expect(err).ToNot(HaveOccurred())
// Nothing should be there anymore (files, packagedb entry)
Expect(helpers.Exists(filepath.Join(fakeroot, "test5"))).ToNot(BeTrue())
Expect(helpers.Exists(filepath.Join(fakeroot, "test6"))).ToNot(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test5"))).ToNot(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test6"))).ToNot(BeTrue())
_, err = systemDB.FindPackage(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})
Expect(err).To(HaveOccurred())
@@ -319,21 +320,21 @@ urls:
artifact, err := c.Compile(false, spec)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test6"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test6"))).To(BeTrue())
content1, err := helpers.Read(spec.Rel("test5"))
content1, err := fileHelper.Read(spec.Rel("test5"))
Expect(err).ToNot(HaveOccurred())
content2, err := helpers.Read(spec.Rel("test6"))
content2, err := fileHelper.Read(spec.Rel("test6"))
Expect(err).ToNot(HaveOccurred())
Expect(content1).To(Equal("artifact5\n"))
Expect(content2).To(Equal("artifact6\n"))
Expect(helpers.Exists(spec.Rel("b-test-1.0.package.tar"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("b-test-1.0.metadata.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("b-test-1.0.package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("b-test-1.0.metadata.yaml"))).To(BeTrue())
repo, err := GenerateRepository(
"test",
@@ -345,15 +346,15 @@ urls:
pkg.NewInMemoryDatabase(false), nil, "", false, false, false, nil)
Expect(err).ToNot(HaveOccurred())
Expect(repo.GetName()).To(Equal("test"))
Expect(helpers.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
err = repo.Write(tmpdir, false, false)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("repository.yaml"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("repository.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(repo.GetUrls()[0]).To(Equal(tmpdir))
Expect(repo.GetType()).To(Equal("disk"))
@@ -383,8 +384,8 @@ urls:
err = inst.Install([]pkg.Package{&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"}}, system)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(filepath.Join(fakeroot, "test5"))).To(BeTrue())
Expect(helpers.Exists(filepath.Join(fakeroot, "test6"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test5"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test6"))).To(BeTrue())
_, err = systemDB.FindPackage(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})
Expect(err).ToNot(HaveOccurred())
@@ -401,8 +402,8 @@ urls:
Expect(err).ToNot(HaveOccurred())
// Nothing should be there anymore (files, packagedb entry)
Expect(helpers.Exists(filepath.Join(fakeroot, "test5"))).ToNot(BeTrue())
Expect(helpers.Exists(filepath.Join(fakeroot, "test6"))).ToNot(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test5"))).ToNot(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test6"))).ToNot(BeTrue())
_, err = system.Database.GetPackageFiles(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})
Expect(err).To(HaveOccurred())
@@ -444,21 +445,21 @@ urls:
artifact, err := c.Compile(false, spec)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test6"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test6"))).To(BeTrue())
content1, err := helpers.Read(spec.Rel("test5"))
content1, err := fileHelper.Read(spec.Rel("test5"))
Expect(err).ToNot(HaveOccurred())
content2, err := helpers.Read(spec.Rel("test6"))
content2, err := fileHelper.Read(spec.Rel("test6"))
Expect(err).ToNot(HaveOccurred())
Expect(content1).To(Equal("artifact5\n"))
Expect(content2).To(Equal("artifact6\n"))
Expect(helpers.Exists(spec.Rel("b-test-1.0.package.tar"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("b-test-1.0.metadata.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("b-test-1.0.package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("b-test-1.0.metadata.yaml"))).To(BeTrue())
repo, err := GenerateRepository(
"test",
@@ -471,15 +472,15 @@ urls:
pkg.NewInMemoryDatabase(false), nil, "", false, false, false, nil)
Expect(err).ToNot(HaveOccurred())
Expect(repo.GetName()).To(Equal("test"))
Expect(helpers.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
err = repo.Write(tmpdir, false, false)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("repository.yaml"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("repository.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(repo.GetUrls()[0]).To(Equal(tmpdir))
Expect(repo.GetType()).To(Equal("disk"))
@@ -531,7 +532,7 @@ urls:
artifact, err = c.Compile(false, spec)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
repo, err = stubRepo(tmpdir2, "../../tests/fixtures/alpine")
Expect(err).ToNot(HaveOccurred())
@@ -604,15 +605,15 @@ urls:
repo, err := stubRepo(tmpdir, "../../tests/fixtures/upgrade")
Expect(err).ToNot(HaveOccurred())
Expect(repo.GetName()).To(Equal("test"))
Expect(helpers.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
err = repo.Write(tmpdir, false, false)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("repository.yaml"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("repository.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(repo.GetUrls()[0]).To(Equal(tmpdir))
Expect(repo.GetType()).To(Equal("disk"))
@@ -642,8 +643,8 @@ urls:
err = inst.Install([]pkg.Package{&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"}}, system)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(filepath.Join(fakeroot, "test5"))).To(BeTrue())
Expect(helpers.Exists(filepath.Join(fakeroot, "test6"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test5"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test6"))).To(BeTrue())
_, err = systemDB.FindPackage(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})
Expect(err).ToNot(HaveOccurred())
@@ -660,11 +661,11 @@ urls:
Expect(err).ToNot(HaveOccurred())
// Nothing should be there anymore (files, packagedb entry)
Expect(helpers.Exists(filepath.Join(fakeroot, "test5"))).ToNot(BeTrue())
Expect(helpers.Exists(filepath.Join(fakeroot, "test6"))).ToNot(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test5"))).ToNot(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test6"))).ToNot(BeTrue())
// New version - new files
Expect(helpers.Exists(filepath.Join(fakeroot, "newc"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "newc"))).To(BeTrue())
_, err = system.Database.GetPackageFiles(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})
Expect(err).To(HaveOccurred())
_, err = system.Database.FindPackage(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})
@@ -729,9 +730,9 @@ urls:
repo, err := stubRepo(tmpdir, "../../tests/fixtures/upgrade_old_repo")
Expect(err).ToNot(HaveOccurred())
Expect(repo.GetName()).To(Equal("test"))
Expect(helpers.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
err = repo.Write(tmpdir, false, false)
Expect(err).ToNot(HaveOccurred())
@@ -774,8 +775,8 @@ urls:
err = inst.Install([]pkg.Package{&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"}}, system)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(filepath.Join(fakeroot, "test5"))).To(BeTrue())
Expect(helpers.Exists(filepath.Join(fakeroot, "test6"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test5"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test6"))).To(BeTrue())
_, err = systemDB.FindPackage(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})
Expect(err).ToNot(HaveOccurred())
@@ -793,11 +794,11 @@ urls:
Expect(err).ToNot(HaveOccurred())
// Nothing should be there anymore (files, packagedb entry)
Expect(helpers.Exists(filepath.Join(fakeroot, "test5"))).ToNot(BeTrue())
Expect(helpers.Exists(filepath.Join(fakeroot, "test6"))).ToNot(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test5"))).ToNot(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test6"))).ToNot(BeTrue())
// New version - new files
Expect(helpers.Exists(filepath.Join(fakeroot, "newc"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "newc"))).To(BeTrue())
_, err = system.Database.GetPackageFiles(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})
Expect(err).To(HaveOccurred())
_, err = system.Database.FindPackage(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})
@@ -856,17 +857,17 @@ urls:
repo, err := stubRepo(tmpdir, "../../tests/fixtures/upgrade")
Expect(err).ToNot(HaveOccurred())
Expect(repo.GetName()).To(Equal("test"))
Expect(helpers.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
err = repo.Write(tmpdir, false, false)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("b-test-1.1.package.tar.gz"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("b-test-1.1.package.tar"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("b-test-1.1.package.tar.gz"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("b-test-1.1.package.tar"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel("repository.yaml"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("repository.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(repo.GetUrls()[0]).To(Equal(tmpdir))
Expect(repo.GetType()).To(Equal("disk"))
@@ -896,8 +897,8 @@ urls:
err = inst.Install([]pkg.Package{&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"}}, system)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(filepath.Join(fakeroot, "test5"))).To(BeTrue())
Expect(helpers.Exists(filepath.Join(fakeroot, "test6"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test5"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test6"))).To(BeTrue())
_, err = systemDB.FindPackage(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})
Expect(err).ToNot(HaveOccurred())
@@ -914,11 +915,11 @@ urls:
Expect(err).ToNot(HaveOccurred())
// Nothing should be there anymore (files, packagedb entry)
Expect(helpers.Exists(filepath.Join(fakeroot, "test5"))).ToNot(BeTrue())
Expect(helpers.Exists(filepath.Join(fakeroot, "test6"))).ToNot(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test5"))).ToNot(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test6"))).ToNot(BeTrue())
// New version - new files
Expect(helpers.Exists(filepath.Join(fakeroot, "newc"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "newc"))).To(BeTrue())
_, err = system.Database.GetPackageFiles(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})
Expect(err).To(HaveOccurred())
_, err = system.Database.FindPackage(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})
@@ -1014,17 +1015,17 @@ urls:
repo, err := stubRepo(tmpdir, "../../tests/fixtures/upgrade")
Expect(err).ToNot(HaveOccurred())
Expect(repo.GetName()).To(Equal("test"))
Expect(helpers.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
err = repo.Write(tmpdir, false, false)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("b-test-1.1.package.tar.gz"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("b-test-1.1.package.tar"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("b-test-1.1.package.tar.gz"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("b-test-1.1.package.tar"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel("repository.yaml"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("repository.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(repo.GetUrls()[0]).To(Equal(tmpdir))
Expect(repo.GetType()).To(Equal("disk"))
@@ -1059,13 +1060,13 @@ urls:
Expect(err).To(HaveOccurred())
Expect(len(system.Database.World())).To(Equal(0))
Expect(helpers.Exists(filepath.Join(fakeroot, "test5"))).To(BeFalse())
Expect(helpers.Exists(filepath.Join(fakeroot, "test6"))).To(BeFalse())
Expect(helpers.Exists(filepath.Join(fakeroot, "c"))).To(BeFalse())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test5"))).To(BeFalse())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test6"))).To(BeFalse())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "c"))).To(BeFalse())
Expect(helpers.Touch(filepath.Join(fakeroot, "test5"))).ToNot(HaveOccurred())
Expect(helpers.Touch(filepath.Join(fakeroot, "test6"))).ToNot(HaveOccurred())
Expect(helpers.Touch(filepath.Join(fakeroot, "c"))).ToNot(HaveOccurred())
Expect(fileHelper.Touch(filepath.Join(fakeroot, "test5"))).ToNot(HaveOccurred())
Expect(fileHelper.Touch(filepath.Join(fakeroot, "test6"))).ToNot(HaveOccurred())
Expect(fileHelper.Touch(filepath.Join(fakeroot, "c"))).ToNot(HaveOccurred())
err = inst.Reclaim(system)
Expect(err).ToNot(HaveOccurred())
@@ -1115,15 +1116,15 @@ urls:
repo, err := stubRepo(tmpdir, "../../tests/fixtures/upgrade_old_repo")
Expect(err).ToNot(HaveOccurred())
Expect(repo.GetName()).To(Equal("test"))
Expect(helpers.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel("repository.yaml"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
err = repo.Write(tmpdir, false, false)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("repository.yaml"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("repository.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(repo.GetUrls()[0]).To(Equal(tmpdir))
Expect(repo.GetType()).To(Equal("disk"))
@@ -1158,13 +1159,13 @@ urls:
Expect(err).To(HaveOccurred())
Expect(len(system.Database.World())).To(Equal(0))
Expect(helpers.Exists(filepath.Join(fakeroot, "test5"))).To(BeFalse())
Expect(helpers.Exists(filepath.Join(fakeroot, "test6"))).To(BeFalse())
Expect(helpers.Exists(filepath.Join(fakeroot, "c"))).To(BeFalse())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test5"))).To(BeFalse())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test6"))).To(BeFalse())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "c"))).To(BeFalse())
Expect(helpers.Touch(filepath.Join(fakeroot, "test5"))).ToNot(HaveOccurred())
Expect(helpers.Touch(filepath.Join(fakeroot, "test6"))).ToNot(HaveOccurred())
Expect(helpers.Touch(filepath.Join(fakeroot, "c"))).ToNot(HaveOccurred())
Expect(fileHelper.Touch(filepath.Join(fakeroot, "test5"))).ToNot(HaveOccurred())
Expect(fileHelper.Touch(filepath.Join(fakeroot, "test6"))).ToNot(HaveOccurred())
Expect(fileHelper.Touch(filepath.Join(fakeroot, "c"))).ToNot(HaveOccurred())
err = inst.Reclaim(system)
Expect(err).ToNot(HaveOccurred())
@@ -1218,11 +1219,11 @@ urls:
Expect(err).ToNot(HaveOccurred())
// Nothing should be there anymore (files, packagedb entry)
Expect(helpers.Exists(filepath.Join(fakeroot, "test5"))).ToNot(BeTrue())
Expect(helpers.Exists(filepath.Join(fakeroot, "test6"))).ToNot(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test5"))).ToNot(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "test6"))).ToNot(BeTrue())
// New version - new files
Expect(helpers.Exists(filepath.Join(fakeroot, "newc"))).To(BeTrue())
Expect(fileHelper.Exists(filepath.Join(fakeroot, "newc"))).To(BeTrue())
_, err = system.Database.GetPackageFiles(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})
Expect(err).To(HaveOccurred())
_, err = system.Database.FindPackage(&pkg.DefaultPackage{Name: "b", Category: "test", Version: "1.0"})

View File

@@ -28,11 +28,11 @@ import (
artifact "github.com/mudler/luet/pkg/compiler/types/artifact"
compression "github.com/mudler/luet/pkg/compiler/types/compression"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
"go.uber.org/multierr"
"github.com/mudler/luet/pkg/compiler"
"github.com/mudler/luet/pkg/config"
"github.com/mudler/luet/pkg/helpers"
"github.com/mudler/luet/pkg/installer/client"
. "github.com/mudler/luet/pkg/logger"
pkg "github.com/mudler/luet/pkg/package"
@@ -723,7 +723,7 @@ func (r *LuetSystemRepository) SyncBuildMetadata(path string) error {
if err != nil {
return errors.Wrapf(err, "while downloading metadata for %s", ai.HumanReadableString())
}
if err := helpers.Move(file, filepath.Join(path, ai.GetMetadataFilePath())); err != nil {
if err := fileHelper.Move(file, filepath.Join(path, ai.GetMetadataFilePath())); err != nil {
return err
}
}
@@ -810,7 +810,7 @@ func (r *LuetSystemRepository) Sync(force bool) (*LuetSystemRepository, error) {
if r.Cached {
// Copy updated repository.yaml file to repo dir now that the tree is synced.
err = helpers.CopyFile(file, filepath.Join(repobasedir, REPOSITORY_SPECFILE))
err = fileHelper.CopyFile(file, filepath.Join(repobasedir, REPOSITORY_SPECFILE))
if err != nil {
return nil, errors.Wrap(err, "Error on update "+REPOSITORY_SPECFILE)
}

View File

@@ -24,16 +24,16 @@ import (
"strings"
"time"
artifact "github.com/mudler/luet/pkg/compiler/types/artifact"
"github.com/mudler/luet/pkg/bus"
compiler "github.com/mudler/luet/pkg/compiler"
"github.com/mudler/luet/pkg/compiler/backend"
artifact "github.com/mudler/luet/pkg/compiler/types/artifact"
"github.com/mudler/luet/pkg/config"
"github.com/mudler/luet/pkg/helpers"
"github.com/mudler/luet/pkg/helpers/docker"
. "github.com/mudler/luet/pkg/logger"
pkg "github.com/mudler/luet/pkg/package"
"github.com/mudler/luet/pkg/bus"
compiler "github.com/mudler/luet/pkg/compiler"
"github.com/mudler/luet/pkg/config"
"github.com/mudler/luet/pkg/helpers"
"github.com/pkg/errors"
)
@@ -61,7 +61,7 @@ func (l *dockerRepositoryGenerator) Initialize(path string, db pkg.PackageDataba
return nil
}
if err := l.pushImageFromArtifact(artifact.NewPackageArtifact(currentpath), l.b); err != nil {
if err := l.pushImageFromArtifact(artifact.NewPackageArtifact(currentpath), l.b, true); err != nil {
return errors.Wrap(err, "while pushing metadata file associated to the artifact")
}
@@ -159,16 +159,20 @@ func (d *dockerRepositoryGenerator) pushRepoMetadata(repospec string, r *LuetSys
return nil
}
func (d *dockerRepositoryGenerator) pushImageFromArtifact(a *artifact.PackageArtifact, b compiler.CompilerBackend) error {
func (d *dockerRepositoryGenerator) pushImageFromArtifact(a *artifact.PackageArtifact, b compiler.CompilerBackend, checkIfExists bool) error {
// we generate a new archive containing the required compressed file.
// TODO: Bundle all the extra files in 1 docker image only, instead of an image for each file
treeArchive, err := artifact.CreateArtifactForFile(a.Path)
if err != nil {
return errors.Wrap(err, "failed generating checksums for tree")
}
imageTree := fmt.Sprintf("%s:%s", d.imagePrefix, helpers.StripInvalidStringsFromImage(a.GetFileName()))
return d.pushFileFromArtifact(treeArchive, imageTree)
imageTree := fmt.Sprintf("%s:%s", d.imagePrefix, docker.StripInvalidStringsFromImage(a.GetFileName()))
if checkIfExists && d.imagePush && d.b.ImageAvailable(imageTree) && !d.force {
Info("Image", imageTree, "already present, skipping. use --force-push to override")
return nil
} else {
return d.pushFileFromArtifact(treeArchive, imageTree)
}
}
// Generate creates a Docker luet repository
@@ -225,7 +229,7 @@ func (d *dockerRepositoryGenerator) Generate(r *LuetSystemRepository, imagePrefi
// we generate a new archive containing the required compressed file.
// TODO: Bundle all the extra files in 1 docker image only, instead of an image for each file
if err := d.pushImageFromArtifact(a, d.b); err != nil {
if err := d.pushImageFromArtifact(a, d.b, false); err != nil {
return errors.Wrap(err, "error met while pushing runtime tree")
}
@@ -235,7 +239,7 @@ func (d *dockerRepositoryGenerator) Generate(r *LuetSystemRepository, imagePrefi
}
// we generate a new archive containing the required compressed file.
// TODO: Bundle all the extra files in 1 docker image only, instead of an image for each file
if err := d.pushImageFromArtifact(a, d.b); err != nil {
if err := d.pushImageFromArtifact(a, d.b, false); err != nil {
return errors.Wrap(err, "error met while pushing compiler tree")
}
@@ -251,7 +255,7 @@ func (d *dockerRepositoryGenerator) Generate(r *LuetSystemRepository, imagePrefi
return errors.Wrap(err, "failed adding Metadata file to repository")
}
if err := d.pushImageFromArtifact(a, d.b); err != nil {
if err := d.pushImageFromArtifact(a, d.b, false); err != nil {
return errors.Wrap(err, "error met while pushing docker image from artifact")
}

View File

@@ -26,14 +26,15 @@ import (
"github.com/mudler/luet/pkg/compiler"
backend "github.com/mudler/luet/pkg/compiler/backend"
compilerspec "github.com/mudler/luet/pkg/compiler/types/spec"
artifact "github.com/mudler/luet/pkg/compiler/types/artifact"
compilerspec "github.com/mudler/luet/pkg/compiler/types/spec"
config "github.com/mudler/luet/pkg/config"
"github.com/mudler/luet/pkg/helpers"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
. "github.com/mudler/luet/pkg/installer"
pkg "github.com/mudler/luet/pkg/package"
"github.com/mudler/luet/pkg/tree"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
)
@@ -83,34 +84,34 @@ var _ = Describe("Repository", func() {
artifact, err := compiler.Compile(false, spec)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test6"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test6"))).To(BeTrue())
content1, err := helpers.Read(spec.Rel("test5"))
content1, err := fileHelper.Read(spec.Rel("test5"))
Expect(err).ToNot(HaveOccurred())
content2, err := helpers.Read(spec.Rel("test6"))
content2, err := fileHelper.Read(spec.Rel("test6"))
Expect(err).ToNot(HaveOccurred())
Expect(content1).To(Equal("artifact5\n"))
Expect(content2).To(Equal("artifact6\n"))
Expect(helpers.Exists(spec.Rel("b-test-1.0.package.tar"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("b-test-1.0.metadata.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("b-test-1.0.package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("b-test-1.0.metadata.yaml"))).To(BeTrue())
repo, err := stubRepo(tmpdir, "../../tests/fixtures/buildable")
Expect(err).ToNot(HaveOccurred())
Expect(repo.GetName()).To(Equal("test"))
Expect(helpers.Exists(spec.Rel(REPOSITORY_SPECFILE))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_SPECFILE))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
err = repo.Write(tmpdir, false, true)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel(REPOSITORY_SPECFILE))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_SPECFILE))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
})
It("Generate repository metadata of files ONLY referenced in a tree", func() {
@@ -156,41 +157,41 @@ var _ = Describe("Repository", func() {
artifact, err := compiler.Compile(false, spec)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(artifact.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact.Path)).To(BeTrue())
Expect(helpers.Untar(artifact.Path, tmpdir, false)).ToNot(HaveOccurred())
artifact2, err := compiler2.Compile(false, spec2)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(artifact2.Path)).To(BeTrue())
Expect(fileHelper.Exists(artifact2.Path)).To(BeTrue())
Expect(helpers.Untar(artifact2.Path, tmpdir, false)).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test6"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test6"))).To(BeTrue())
content1, err := helpers.Read(spec.Rel("test5"))
content1, err := fileHelper.Read(spec.Rel("test5"))
Expect(err).ToNot(HaveOccurred())
content2, err := helpers.Read(spec.Rel("test6"))
content2, err := fileHelper.Read(spec.Rel("test6"))
Expect(err).ToNot(HaveOccurred())
Expect(content1).To(Equal("artifact5\n"))
Expect(content2).To(Equal("artifact6\n"))
Expect(helpers.Exists(spec.Rel("b-test-1.0.package.tar"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("b-test-1.0.metadata.yaml"))).To(BeTrue())
Expect(helpers.Exists(spec2.Rel("alpine-seed-1.0.package.tar"))).To(BeTrue())
Expect(helpers.Exists(spec2.Rel("alpine-seed-1.0.metadata.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("b-test-1.0.package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("b-test-1.0.metadata.yaml"))).To(BeTrue())
Expect(fileHelper.Exists(spec2.Rel("alpine-seed-1.0.package.tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec2.Rel("alpine-seed-1.0.metadata.yaml"))).To(BeTrue())
repo, err := stubRepo(tmpdir, "../../tests/fixtures/buildable")
Expect(err).ToNot(HaveOccurred())
Expect(repo.GetName()).To(Equal("test"))
Expect(helpers.Exists(spec.Rel(REPOSITORY_SPECFILE))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_SPECFILE))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
err = repo.Write(tmpdir, false, true)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel(REPOSITORY_SPECFILE))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_SPECFILE))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).To(BeTrue())
// We check now that the artifact not referenced in the tree
// (spec2) is not indexed in the repository
@@ -267,18 +268,18 @@ urls:
a, err := localcompiler.Compile(false, spec)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(a.Path)).To(BeTrue())
Expect(fileHelper.Exists(a.Path)).To(BeTrue())
Expect(helpers.Untar(a.Path, tmpdir, false)).ToNot(HaveOccurred())
Expect(helpers.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(helpers.Exists(spec.Rel("test6"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test5"))).To(BeTrue())
Expect(fileHelper.Exists(spec.Rel("test6"))).To(BeTrue())
repo, err := dockerStubRepo(tmpdir, "../../tests/fixtures/buildable", repoImage, true, true)
Expect(err).ToNot(HaveOccurred())
Expect(repo.GetName()).To(Equal("test"))
Expect(helpers.Exists(spec.Rel(REPOSITORY_SPECFILE))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_SPECFILE))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
err = repo.Write(repoImage, false, true)
Expect(err).ToNot(HaveOccurred())
@@ -295,7 +296,7 @@ urls:
f, err := c.DownloadFile("repository.yaml")
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Read(f)).To(ContainSubstring("name: test"))
Expect(fileHelper.Read(f)).To(ContainSubstring("name: test"))
a, err = c.DownloadArtifact(&artifact.PackageArtifact{
Path: "test.tar",
@@ -310,7 +311,7 @@ urls:
Expect(err).ToNot(HaveOccurred())
Expect(a.Unpack(extracted, false)).ToNot(HaveOccurred())
Expect(helpers.Read(filepath.Join(extracted, "test6"))).To(Equal("artifact6\n"))
Expect(fileHelper.Read(filepath.Join(extracted, "test6"))).To(Equal("artifact6\n"))
})
It("generates images of virtual packages", func() {
@@ -341,15 +342,15 @@ urls:
a, err := localcompiler.Compile(false, spec)
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Exists(a.Path)).To(BeTrue())
Expect(fileHelper.Exists(a.Path)).To(BeTrue())
Expect(helpers.Untar(a.Path, tmpdir, false)).ToNot(HaveOccurred())
repo, err := dockerStubRepo(tmpdir, "../../tests/fixtures/virtuals", repoImage, true, true)
Expect(err).ToNot(HaveOccurred())
Expect(repo.GetName()).To(Equal("test"))
Expect(helpers.Exists(spec.Rel(REPOSITORY_SPECFILE))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(helpers.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_SPECFILE))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(TREE_TARBALL + ".gz"))).ToNot(BeTrue())
Expect(fileHelper.Exists(spec.Rel(REPOSITORY_METAFILE + ".tar"))).ToNot(BeTrue())
err = repo.Write(repoImage, false, true)
Expect(err).ToNot(HaveOccurred())
@@ -366,7 +367,7 @@ urls:
f, err := c.DownloadFile("repository.yaml")
Expect(err).ToNot(HaveOccurred())
Expect(helpers.Read(f)).To(ContainSubstring("name: test"))
Expect(fileHelper.Read(f)).To(ContainSubstring("name: test"))
a, err = c.DownloadArtifact(&artifact.PackageArtifact{
Path: "test.tar",
@@ -382,7 +383,7 @@ urls:
Expect(a.Unpack(extracted, false)).ToNot(HaveOccurred())
Expect(helpers.DirectoryIsEmpty(extracted)).To(BeFalse())
Expect(fileHelper.DirectoryIsEmpty(extracted)).To(BeFalse())
content, err := ioutil.ReadFile(filepath.Join(extracted, ".virtual"))
Expect(err).ToNot(HaveOccurred())

View File

@@ -3,6 +3,7 @@ package installer
import (
"github.com/hashicorp/go-multierror"
"github.com/mudler/luet/pkg/helpers"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
. "github.com/mudler/luet/pkg/logger"
pkg "github.com/mudler/luet/pkg/package"
"github.com/mudler/luet/pkg/tree"
@@ -23,7 +24,7 @@ func (s *System) ExecuteFinalizers(packs []pkg.Package) error {
var errs error
executedFinalizer := map[string]bool{}
for _, p := range packs {
if helpers.Exists(p.Rel(tree.FinalizerFile)) {
if fileHelper.Exists(p.Rel(tree.FinalizerFile)) {
out, err := helpers.RenderFiles(p.Rel(tree.FinalizerFile), p.Rel(tree.DefinitionFile))
if err != nil {
Warning("Failed rendering finalizer for ", p.HumanReadableString(), err.Error())

View File

@@ -26,7 +26,8 @@ import (
"strconv"
"strings"
"github.com/mudler/luet/pkg/helpers"
"github.com/mudler/luet/pkg/helpers/docker"
"github.com/mudler/luet/pkg/helpers/match"
version "github.com/mudler/luet/pkg/versioner"
gentoo "github.com/Sabayon/pkgs-checker/pkg/gentoo"
@@ -118,6 +119,8 @@ type Package interface {
SetTreeDir(s string)
GetTreeDir() string
Mark() Package
JSON() ([]byte, error)
}
@@ -131,6 +134,19 @@ type Tree interface {
type Packages []Package
type DefaultPackages []*DefaultPackage
func (d DefaultPackages) Hash(salt string) string {
overallFp := ""
for _, c := range d {
overallFp = overallFp + c.HashFingerprint("join")
}
h := md5.New()
io.WriteString(h, fmt.Sprintf("%s-%s", overallFp, salt))
return fmt.Sprintf("%x", h.Sum(nil))
}
// >> Unmarshallers
// DefaultPackageFromYaml decodes a package from yaml bytes
func DefaultPackageFromYaml(yml []byte) (DefaultPackage, error) {
@@ -309,7 +325,7 @@ func (p *DefaultPackage) GetPackageName() string {
}
func (p *DefaultPackage) ImageID() string {
return helpers.StripInvalidStringsFromImage(p.GetFingerPrint())
return docker.StripInvalidStringsFromImage(p.GetFingerPrint())
}
// GetBuildTimestamp returns the package build timestamp
@@ -344,19 +360,19 @@ func (p *DefaultPackage) IsHidden() bool {
}
func (p *DefaultPackage) HasLabel(label string) bool {
return helpers.MapHasKey(&p.Labels, label)
return match.MapHasKey(&p.Labels, label)
}
func (p *DefaultPackage) MatchLabel(r *regexp.Regexp) bool {
return helpers.MapMatchRegex(&p.Labels, r)
return match.MapMatchRegex(&p.Labels, r)
}
func (p *DefaultPackage) HasAnnotation(label string) bool {
return helpers.MapHasKey(&p.Annotations, label)
return match.MapHasKey(&p.Annotations, label)
}
func (p *DefaultPackage) MatchAnnotation(r *regexp.Regexp) bool {
return helpers.MapMatchRegex(&p.Annotations, r)
return match.MapMatchRegex(&p.Annotations, r)
}
// AddUse adds a use to a package
@@ -492,6 +508,12 @@ func (p *DefaultPackage) Matches(m Package) bool {
return false
}
func (p *DefaultPackage) Mark() Package {
marked := p.Clone()
marked.SetName("@@" + marked.GetName())
return marked
}
func (p *DefaultPackage) Expand(definitiondb PackageDatabase) (Packages, error) {
var versionsInWorld Packages

View File

@@ -260,24 +260,42 @@ func (a PackagesAssertions) TrueLen() int {
// and checks it's not the only one. if it's unique it marks it specially - so the hash
// which is generated is unique for the selected package
func (assertions PackagesAssertions) HashFrom(p pkg.Package) string {
return assertions.SaltedHashFrom(p, map[string]string{})
}
func (assertions PackagesAssertions) AssertionHash() string {
return assertions.SaltedAssertionHash(map[string]string{})
}
func (assertions PackagesAssertions) SaltedHashFrom(p pkg.Package, salts map[string]string) string {
var assertionhash string
// When we don't have any solution to hash for, we need to generate an UUID by ourselves
latestsolution := assertions.Drop(p)
if latestsolution.TrueLen() == 0 {
assertionhash = assertions.Mark(p).AssertionHash()
// Preserve the hash if supplied of marked packages
marked := p.Mark()
if markedHash, exists := salts[p.GetFingerPrint()]; exists {
salts[marked.GetFingerPrint()] = markedHash
}
assertionhash = assertions.Mark(p).SaltedAssertionHash(salts)
} else {
assertionhash = latestsolution.AssertionHash()
assertionhash = latestsolution.SaltedAssertionHash(salts)
}
return assertionhash
}
func (assertions PackagesAssertions) AssertionHash() string {
func (assertions PackagesAssertions) SaltedAssertionHash(salts map[string]string) string {
var fingerprint string
for _, assertion := range assertions { // Note: Always order them first!
if assertion.Value { // Tke into account only dependencies installed (get fingerprint of subgraph)
fingerprint += assertion.ToString() + "\n"
salt, exists := salts[assertion.Package.GetFingerPrint()]
if exists {
fingerprint += assertion.ToString() + salt + "\n"
} else {
fingerprint += assertion.ToString() + "\n"
}
}
}
hash := sha256.Sum256([]byte(fingerprint))
@@ -316,8 +334,7 @@ func (assertions PackagesAssertions) Mark(p pkg.Package) PackagesAssertions {
for _, a := range assertions {
if a.Package.Matches(p) {
marked := a.Package.Clone()
marked.SetName("@@" + marked.GetName())
marked := a.Package.Mark()
a = PackageAssert{Package: marked.(*pkg.DefaultPackage), Value: a.Value, Hash: a.Hash}
}
ass = append(ass, a)

View File

@@ -382,6 +382,9 @@ var _ = Describe("Decoder", func() {
Expect(solution.HashFrom(X)).ToNot(Equal(solution2.HashFrom(F)))
Expect(solution3.HashFrom(D)).To(Equal(solution.HashFrom(X)))
Expect(solution3.SaltedHashFrom(D, map[string]string{D.GetFingerPrint(): "foo"})).ToNot(Equal(solution3.HashFrom(D)))
Expect(solution4.SaltedHashFrom(Y, map[string]string{X.GetFingerPrint(): "foo"})).ToNot(Equal(solution4.HashFrom(Y)))
Expect(empty.AssertionHash()).ToNot(Equal(solution3.HashFrom(D)))
Expect(empty.AssertionHash()).ToNot(Equal(solution2.HashFrom(F)))

View File

@@ -26,6 +26,7 @@ import (
"path/filepath"
"github.com/mudler/luet/pkg/helpers"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
pkg "github.com/mudler/luet/pkg/package"
"github.com/pkg/errors"
)
@@ -61,7 +62,7 @@ type CompilerRecipe struct {
// and the build context required for reproducible builds
func (r *CompilerRecipe) Save(path string) error {
for _, p := range r.SourcePath {
if err := helpers.CopyDir(p, filepath.Join(path, filepath.Base(p))); err != nil {
if err := fileHelper.CopyDir(p, filepath.Join(path, filepath.Base(p))); err != nil {
return errors.Wrap(err, "while copying source tree")
}
}
@@ -102,7 +103,7 @@ func (r *CompilerRecipe) Load(path string) error {
// Instead of rdeps, have a different tree for build deps.
compileDefPath := pack.Rel(CompilerDefinitionFile)
if helpers.Exists(compileDefPath) {
if fileHelper.Exists(compileDefPath) {
dat, err := helpers.RenderFiles(compileDefPath, currentpath)
if err != nil {
@@ -149,7 +150,7 @@ func (r *CompilerRecipe) Load(path string) error {
// Instead of rdeps, have a different tree for build deps.
compileDefPath := pack.Rel(CompilerDefinitionFile)
if helpers.Exists(compileDefPath) {
if fileHelper.Exists(compileDefPath) {
raw := packsRaw.Find(pack.GetName(), pack.GetCategory(), pack.GetVersion())
buildyaml, err := ioutil.ReadFile(compileDefPath)

View File

@@ -26,8 +26,9 @@ import (
"os"
"path/filepath"
"github.com/mudler/luet/pkg/helpers"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
pkg "github.com/mudler/luet/pkg/package"
"github.com/pkg/errors"
)
@@ -61,8 +62,8 @@ func (r *InstallerRecipe) Save(path string) error {
}
// Instead of rdeps, have a different tree for build deps.
finalizerPath := p.Rel(FinalizerFile)
if helpers.Exists(finalizerPath) { // copy finalizer file from the source tree
helpers.CopyFile(finalizerPath, filepath.Join(dir, FinalizerFile))
if fileHelper.Exists(finalizerPath) { // copy finalizer file from the source tree
fileHelper.CopyFile(finalizerPath, filepath.Join(dir, FinalizerFile))
}
}
@@ -71,7 +72,7 @@ func (r *InstallerRecipe) Save(path string) error {
func (r *InstallerRecipe) Load(path string) error {
if !helpers.Exists(path) {
if !fileHelper.Exists(path) {
return errors.New(fmt.Sprintf(
"Path %s doesn't exit.", path,
))

View File

@@ -26,9 +26,10 @@ import (
"os"
"path/filepath"
helpers "github.com/mudler/luet/pkg/helpers"
fileHelper "github.com/mudler/luet/pkg/helpers/file"
pkg "github.com/mudler/luet/pkg/package"
spectooling "github.com/mudler/luet/pkg/spectooling"
"github.com/pkg/errors"
)
@@ -77,7 +78,7 @@ func (r *Recipe) Load(path string) error {
// if err != nil {
// return err
// }
if !helpers.Exists(path) {
if !fileHelper.Exists(path) {
return errors.New(fmt.Sprintf(
"Path %s doesn't exit.", path,
))

17
tests/fixtures/copy/c/a/build.yaml vendored Normal file
View File

@@ -0,0 +1,17 @@
image: "alpine"
copy:
- package:
name: "a"
category: "test"
version: ">=0"
source: /test3
destination: /test3
- image: "busybox"
source: /bin/busybox
destination: /busybox
steps:
- mkdir /bina
- cp /test3 /result
- cp -rf /busybox /bina/busybox

View File

@@ -0,0 +1,3 @@
category: "test"
name: "c"
version: "1.2"

View File

@@ -0,0 +1,7 @@
image: "alpine"
prelude:
- echo foo > /test
- echo bar > /test2
steps:
- echo artifact3 > /test3
- echo artifact4 > /test4

View File

@@ -0,0 +1,3 @@
category: "test"
name: "a"
version: "1.2"

View File

@@ -0,0 +1,13 @@
requires:
- category: "test"
name: "a"
version: ">=0"
prelude:
- echo foo > /test
- echo bar > /test2
steps:
- echo artifact5 > /newc
- echo artifact6 > /newnewc
- chmod +x generate.sh
- ./generate.sh

View File

@@ -0,0 +1,3 @@
category: "test"
name: "b"
version: "1.1"

View File

@@ -0,0 +1 @@
echo generated > /sonewc

8
tests/fixtures/join/c/a/build.yaml vendored Normal file
View File

@@ -0,0 +1,8 @@
join:
- name: "a"
category: "test"
version: ">=0"
- name: "b"
category: "test"
version: ">=0"
unpack: true

View File

@@ -0,0 +1,3 @@
category: "test"
name: "c"
version: "1.2"

View File

@@ -0,0 +1,7 @@
image: "alpine"
prelude:
- echo foo > /test
- echo bar > /test2
steps:
- echo artifact3 > /test3
- echo artifact4 > /test4

View File

@@ -0,0 +1,3 @@
category: "test"
name: "a"
version: "1.2"

View File

@@ -0,0 +1,13 @@
requires:
- category: "test"
name: "a"
version: ">=0"
prelude:
- echo foo > /test
- echo bar > /test2
steps:
- echo artifact5 > /newc
- echo artifact6 > /newnewc
- chmod +x generate.sh
- ./generate.sh

View File

@@ -0,0 +1,3 @@
category: "test"
name: "b"
version: "1.1"

View File

@@ -0,0 +1 @@
echo generated > /sonewc

View File

@@ -0,0 +1,7 @@
join:
- name: "a"
category: "test"
version: ">=0"
- name: "b"
category: "test"
version: ">=0"

View File

@@ -0,0 +1,3 @@
category: "test"
name: "c"
version: "1.2"

View File

@@ -0,0 +1,8 @@
join:
- name: "a"
category: "test"
version: ">=0"
- name: "b"
category: "test"
version: ">=0"
unpack: true

View File

@@ -0,0 +1,3 @@
category: "test"
name: "f"
version: "1.2"

View File

@@ -0,0 +1,2 @@
image: "alpine"
unpack: true

View File

@@ -0,0 +1,3 @@
category: "test"
name: "a"
version: "1.2"

View File

@@ -0,0 +1,13 @@
requires:
- category: "test"
name: "a"
version: ">=0"
prelude:
- echo foo > /test
- echo bar > /test2
steps:
- echo artifact5 > /newc
- echo artifact6 > /newnewc
- chmod +x generate.sh
- ./generate.sh

View File

@@ -0,0 +1,3 @@
category: "test"
name: "b"
version: "1.1"

View File

@@ -0,0 +1 @@
echo generated > /sonewc

View File

@@ -0,0 +1,10 @@
join:
- category: "test"
name: "f"
version: ">=0"
# this is actually a virtual, no files shipped in c
- category: "test"
name: "c"
version: ">=0"
steps:
- mv /newnewc /x

View File

@@ -0,0 +1,3 @@
name: "x"
category: "test"
version: "0.1"

View File

@@ -0,0 +1,6 @@
requires:
- category: "test"
name: "c"
version: ">=0"
steps:
- mv /newnewc /z

View File

@@ -0,0 +1,3 @@
name: "z"
category: "test"
version: "0.1"

View File

@@ -1,3 +1,5 @@
#!/bin/bash
echo "$1" >> $EVENT_FILE
echo "$2" >> $PAYLOAD_FILE
echo "$2" >> $PAYLOAD_FILE
echo "{}"

View File

@@ -0,0 +1,10 @@
prelude:
- echo foo > /test
- echo bar > /test2
steps:
- echo c > /c
- echo c > /cd
requires:
- category: "test"
name: "a"
version: ">=1.0"

View File

@@ -0,0 +1,3 @@
category: "test"
name: "c"
version: "1.0"

View File

@@ -0,0 +1,11 @@
image: "alpine"
prelude:
- echo fozo > /test
- echo bar > /test2
steps:
- echo artifact3 > /test3
- echo artifact4 > /test4
requires:
- category: "test"
name: "b"
version: "1.0"

View File

@@ -0,0 +1,8 @@
category: "test"
name: "a"
version: "1.1"
requires:
- category: "test2"
name: "b"
version: "1.0"

View File

@@ -0,0 +1,9 @@
image: "alpine"
prelude:
- echo foo > /test
- echo bar > /test2
steps:
- echo artifact5 > /newc
- echo artifact6 > /newnewc
- chmod +x generate.sh
- ./generate.sh

View File

@@ -0,0 +1,3 @@
category: "test"
name: "b"
version: "1.0"

View File

@@ -0,0 +1 @@
echo generated > /sonewc

View File

@@ -51,9 +51,9 @@ testBuild() {
assertTrue 'create package' "[ -e '$tmpdir/testbuild/c-test-1.0.package.tar.zst' ]"
assertTrue 'create package Z' "[ -e '$tmpdir/testbuild/z-test-1.0+2.package.tar.zst' ]"
assertTrue 'create package interpolated' "[ -e '$tmpdir/testbuild/interpolated-test-1.0+2.package.tar.zst' ]"
assertContains 'Does use the upstream cache without specifying it test/c' "$build_output" "Images available for test/c-1.0 generating artifact from remote images: quay.io/mocaccinoos/integration-test-cache:d620e573c81eab36a9dc5cc314e80fd7b6e04aeff26127de4225bf24fe1f8e71"
assertContains 'Does use the upstream cache without specifying it test/z' "$build_output" "Images available for test/z-1.0+2 generating artifact from remote images: quay.io/mocaccinoos/integration-test-cache:b0f34b0d2d271f0f2619324476b2857b3b39ca895bddc2474a741f3c8c1acbbc"
assertContains 'Does use the upstream cache without specifying it test/interpolated' "$build_output" "Images available for test/interpolated-1.0+2 generating artifact from remote images: quay.io/mocaccinoos/integration-test-cache:c1f11f48113cd71d8795a06c7b49e1558bd7211d2aa88f5d79a3334f0393c64d"
assertContains 'Does use the upstream cache without specifying it test/c' "$build_output" "Images available remotely for test/c-1.0 generating artifact from remote images: quay.io/mocaccinoos/integration-test-cache:ac34beb3fc2752bab54549db095dd6994d7531b88e1ff7f902d01ae80fdd030d"
assertContains 'Does use the upstream cache without specifying it test/z' "$build_output" "Images available remotely for test/z-1.0+2 generating artifact from remote images: quay.io/mocaccinoos/integration-test-cache:d8b8881d50ae83646728bb28ad678fc14e4e003e4b0d0a66f8e6167c6116e024"
assertContains 'Does use the upstream cache without specifying it test/interpolated' "$build_output" "Images available remotely for test/interpolated-1.0+2 generating artifact from remote images: quay.io/mocaccinoos/integration-test-cache:2aaccd929ebe5683f95c70c6ec8d68f240ceea8633f3350b94904ad73da5fd47"
}
testRepo() {

View File

@@ -45,7 +45,7 @@ EOF
mkdir $tmpdir/testbuild
mkdir $tmpdir/empty
# With --rebuild, the package gets ignored
# Without --rebuild, the package gets ignored
build_output=$(luet build --pull --tree "$tmpdir/empty" \
--config $tmpdir/luet.yaml --values $tmpdir/default.yaml --concurrency 1 \
--from-repositories --destination $tmpdir/testbuild --compression zstd test/c@1.0 test/z test/interpolated)
@@ -56,7 +56,8 @@ EOF
assertTrue 'create package' "[ -e '$tmpdir/testbuild/c-test-1.0.package.tar.zst' ]"
assertTrue 'create package Z' "[ -e '$tmpdir/testbuild/z-test-1.0+2.package.tar.zst' ]"
assertTrue 'create package interpolated' "[ -e '$tmpdir/testbuild/interpolated-test-1.0+2.package.tar.zst' ]"
assertContains 'Does use the upstream cache without specifying it' "$build_output" "Images available for test/interpolated-1.0+2 generating artifact from remote images: quay.io/mocaccinoos/integration-test-cache:c1f11f48113cd71d8795a06c7b49e1558bd7211d2aa88f5d79a3334f0393c64d"
assertNotContains 'Does NOT use the upstream cache without specifying it' "$build_output" "Images available remotely for test/interpolated-1.0+2 generating artifact from remote images: quay.io/mocaccinoos/integration-test-cache:2aaccd929ebe5683f95c70c6ec8d68f240ceea8633f3350b94904ad73da5fd47"
assertContains 'Does generate a new hash as values changed build.yaml for test/interpolated-1.0+2 package image' "$build_output" "Building image luet/cache:c34b533cf76886c332fe9b2d3208f04265360a465a90c996cb4fcdaf959dee36 done"
}
testRepo() {
@@ -113,7 +114,7 @@ testInstall() {
assertTrue 'package installed' "[ -e '$tmpdir/testrootfs/c' ]"
assertTrue 'package Z installed' "[ -e '$tmpdir/testrootfs/z' ]"
ls -liah $tmpdir/testrootfs/
assertTrue 'package interpolated installed' "[ -e '$tmpdir/testrootfs/interpolated-baz-bar' ]"
assertTrue 'package interpolated installed' "[ -e '$tmpdir/testrootfs/interpolated-baz-an' ]"
}
testReInstall() {

View File

@@ -56,7 +56,7 @@ EOF
assertTrue 'create package' "[ -e '$tmpdir/testbuild/c-test-1.0.package.tar.zst' ]"
assertTrue 'create package Z' "[ -e '$tmpdir/testbuild/z-test-1.0+2.package.tar.zst' ]"
assertTrue 'create package interpolated' "[ -e '$tmpdir/testbuild/interpolated-test-1.0+2.package.tar.zst' ]"
assertContains 'Does use the upstream cache without specifying it' "$build_output" "Downloading image quay.io/mocaccinoos/integration-test-cache:6490e800fe443b99328fc363529aee74bda513930fb27ce6ab814d692bba068e"
assertContains 'Does use the upstream cache without specifying it' "$build_output" "Downloading image quay.io/mocaccinoos/integration-test-cache:4db24406e8db30a3310a1cf8c4d4e19597745e6d41b189dc51a73ac4a50cc9e6"
}
testRepo() {

38
tests/integration/30_copy.sh Executable file
View File

@@ -0,0 +1,38 @@
#!/bin/bash
export LUET_NOLOCK=true
oneTimeSetUp() {
export tmpdir="$(mktemp -d)"
docker images --filter='reference=luet/cache' --format='{{.Repository}}:{{.Tag}}' | xargs -r docker rmi
}
oneTimeTearDown() {
rm -rf "$tmpdir"
docker images --filter='reference=luet/cache' --format='{{.Repository}}:{{.Tag}}' | xargs -r docker rmi
}
testBuild() {
[ "$LUET_BACKEND" == "img" ] && startSkipping
cat <<EOF > $tmpdir/default.yaml
extra: "bar"
foo: "baz"
EOF
mkdir $tmpdir/testbuild
luet build --tree "$ROOT_DIR/tests/fixtures/copy" \
--destination $tmpdir/testbuild --concurrency 1 \
--compression gzip --values $tmpdir/default.yaml \
test/c
buildst=$?
assertEquals 'builds successfully' "$buildst" "0"
assertTrue 'create package c' "[ -e '$tmpdir/testbuild/c-test-1.2.package.tar.gz' ]"
mkdir $tmpdir/extract
tar -xvf $tmpdir/testbuild/c-test-1.2.package.tar.gz -C $tmpdir/extract
assertTrue 'create result in package c' "[ -e '$tmpdir/extract/result' ]"
assertTrue 'create busybox in package c' "[ -f '$tmpdir/extract/bina/busybox' ]"
}
# Load shUnit2.
. "$ROOT_DIR/tests/integration/shunit2"/shunit2

39
tests/integration/31_join.sh Executable file
View File

@@ -0,0 +1,39 @@
#!/bin/bash
export LUET_NOLOCK=true
oneTimeSetUp() {
export tmpdir="$(mktemp -d)"
docker images --filter='reference=luet/cache' --format='{{.Repository}}:{{.Tag}}' | xargs -r docker rmi
}
oneTimeTearDown() {
rm -rf "$tmpdir"
docker images --filter='reference=luet/cache' --format='{{.Repository}}:{{.Tag}}' | xargs -r docker rmi
}
testBuild() {
[ "$LUET_BACKEND" == "img" ] && startSkipping
cat <<EOF > $tmpdir/default.yaml
extra: "bar"
foo: "baz"
EOF
mkdir $tmpdir/testbuild
luet build --tree "$ROOT_DIR/tests/fixtures/join" \
--destination $tmpdir/testbuild --concurrency 1 \
--compression gzip --values $tmpdir/default.yaml \
test/c
buildst=$?
assertEquals 'builds successfully' "$buildst" "0"
assertTrue 'create package c' "[ -e '$tmpdir/testbuild/c-test-1.2.package.tar.gz' ]"
mkdir $tmpdir/extract
tar -xvf $tmpdir/testbuild/c-test-1.2.package.tar.gz -C $tmpdir/extract
assertTrue 'create result from join' "[ -e '$tmpdir/extract/test3' ]"
assertTrue 'create result from join' "[ -f '$tmpdir/extract/newc' ]"
assertTrue 'create result from join' "[ -e '$tmpdir/extract/test4' ]"
}
# Load shUnit2.
. "$ROOT_DIR/tests/integration/shunit2"/shunit2

View File

@@ -0,0 +1,37 @@
#!/bin/bash
export LUET_NOLOCK=true
oneTimeSetUp() {
export tmpdir="$(mktemp -d)"
docker images --filter='reference=luet/cache' --format='{{.Repository}}:{{.Tag}}' | xargs -r docker rmi
}
oneTimeTearDown() {
rm -rf "$tmpdir"
docker images --filter='reference=luet/cache' --format='{{.Repository}}:{{.Tag}}' | xargs -r docker rmi
}
testBuild() {
[ "$LUET_BACKEND" == "img" ] && startSkipping
mkdir $tmpdir/testbuild
luet build --tree "$ROOT_DIR/tests/fixtures/join_complex" \
--destination $tmpdir/testbuild --concurrency 1 \
--compression gzip \
test/z test/x
buildst=$?
assertEquals 'builds successfully' "$buildst" "0"
assertTrue 'create package z' "[ -e '$tmpdir/testbuild/z-test-0.1.package.tar.gz' ]"
assertTrue 'create package z' "[ -e '$tmpdir/testbuild/x-test-0.1.package.tar.gz' ]"
mkdir $tmpdir/extract
tar -xvf $tmpdir/testbuild/x-test-0.1.package.tar.gz -C $tmpdir/extract
tar -xvf $tmpdir/testbuild/z-test-0.1.package.tar.gz -C $tmpdir/extract
assertTrue 'create result from a package that requires a join' "[ -e '$tmpdir/extract/z' ]"
assertTrue 'create result from join of a join' "[ -e '$tmpdir/extract/x' ]"
}
# Load shUnit2.
. "$ROOT_DIR/tests/integration/shunit2"/shunit2

View File

@@ -1,6 +1,6 @@
#!/bin/bash
set -e
export LUET_NO_SPINNER=true
export LUET_YES=true
export ROOT_DIR="$(git rev-parse --show-toplevel)"

Some files were not shown because too many files have changed in this diff Show More