Move import-boss to k/k, retool to not use gengo

* Moved code and tests out of gengo -> code_generator
* Reworked it to use packages.Load
* Reworked tests (still not comprehensive but pretty good?)
* Dropped test support from gengo (support for tests in
  x/tools/go/packages is pretty hostile to gengo, and nobody used it)
This commit is contained in:
Tim Hockin 2024-01-09 16:11:03 -08:00
parent 08ce6a0f14
commit e78dc86288
No known key found for this signature in database
53 changed files with 1172 additions and 81 deletions

View File

@ -25,9 +25,9 @@ import (
_ "github.com/onsi/ginkgo/v2/ginkgo"
_ "k8s.io/code-generator/cmd/go-to-protobuf"
_ "k8s.io/code-generator/cmd/go-to-protobuf/protoc-gen-gogo"
_ "k8s.io/code-generator/cmd/import-boss"
_ "k8s.io/gengo/v2/examples/deepcopy-gen/generators"
_ "k8s.io/gengo/v2/examples/defaulter-gen/generators"
_ "k8s.io/gengo/v2/examples/import-boss/generators"
_ "k8s.io/kube-openapi/cmd/openapi-gen"
// submodule test dependencies

View File

@ -27,16 +27,14 @@ KUBE_ROOT=$(dirname "${BASH_SOURCE[0]}")/..
source "${KUBE_ROOT}/hack/lib/init.sh"
kube::golang::setup_env
kube::util::require-jq
GOPROXY=off go install k8s.io/code-generator/cmd/import-boss
# Doing it this way is MUCH faster than simply saying "all", and there doesn't
# seem to be a simpler way to express "this whole workspace".
packages=()
kube::util::read-array packages < <(
go work edit -json | jq -r '.Use[].DiskPath + "/..."'
)
$(kube::util::find-binary "import-boss") \
-v "${KUBE_VERBOSE:-0}" \
--include-test-files \
--input-dirs "./pkg/..." \
--input-dirs "./cmd/..." \
--input-dirs "./plugin/..." \
--input-dirs "./test/e2e_node/..." \
--input-dirs "./test/e2e/framework/..." \
--input-dirs "./test/integration/..." \
--input-dirs "./staging/src/..."
GOPROXY=off \
go run k8s.io/code-generator/cmd/import-boss -v "${KUBE_VERBOSE:-0}" "${packages[@]}"

View File

@ -3,6 +3,10 @@ inverseRules:
- selectorRegexp: k8s[.]io/apiextensions-apiserver
allowedPrefixes:
- ''
# Allow use from within e2e tests.
- selectorRegexp: k8s[.]io/kubernetes/test
allowedPrefixes:
- k8s.io/kubernetes/test/e2e/apimachinery
# Forbid use of this package in other k8s.io packages.
- selectorRegexp: k8s[.]io
forbiddenPrefixes:

View File

@ -1,97 +1,104 @@
## Purpose
- `import-boss` enforces import restrictions against all pull requests submitted to the [k/k](https://github.com/kubernetes/kubernetes) repository. There are a number of `.import-restrictions` files that in the [k/k](https://github.com/kubernetes/kubernetes) repository, all of which are defined in `YAML` (or `JSON`) format.
`import-boss` enforces optional import restrictions between packages. This is
useful to manage the dependency graph within a large repository, such as
[kubernetes](https://github.com/kubernetes/kubernetes).
## How does it work?
- When a directory is verified, `import-boss` looks for a file called `.import-restrictions`. If this file is not found, `import-boss` will go up to the parent directory until it finds this `.import-restrictions` file.
When a package is verified, `import-boss` looks for a file called
`.import-restrictions` in the same directory and all parent directories, up to
the module root (defined by the presence of a go.mod file). These files
contain rules which are evaluated against each dependency of the package in
question.
- Adding `.import-restrictions` files does not add them to CI runs. They need to be explicitly added to `hack/verify-import-boss.sh`. Once an `.import-restrictions` file is added, all of the sub-packages of this file's directory are added as well.
Evaluation starts with the rules file closest to the package. If that file
makes a determination to allow or forbid the import, evaluation is done. If
the import does not match any rule, the next-closest rules file is consulted,
and so forth. If the rules files are exhausted and no determination has been
made, the import will be flagged as an error.
### What are rules files?
A rules file is a JSON or YAML document with two top-level keys, both optional:
* `Rules`
* `InverseRules`
### What are Rules?
- If an `.import-restrictions` file is found, then all imports of the package are checked against each `rule` in the file. A `rule` consists of three parts:
A `rule` defines a policy to be enforced on packages which are depended on by
the package in question. It consists of three parts:
- A `SelectorRegexp`, to select the import paths that the rule applies to.
- A list of `AllowedPrefixes`
- A list of `ForbiddenPrefixes`
- An import is allowed if it matches at least one allowed prefix and does not match any forbidden prefixes. An example `.import-restrictions` file looks like this:
An import is allowed if it matches at least one allowed prefix and does not
match any forbidden prefixes.
Rules also have a boolean `Transitive` option. When this option is true, the
rule is applied to transitive imports.
Example:
```json
{
"Rules": [
{
"SelectorRegexp": "k8s[.]io",
"SelectorRegexp": "example[.]com",
"AllowedPrefixes": [
"k8s.io/gengo/v2/examples",
"k8s.io/kubernetes/third_party"
"example.com/project/package",
"example.com/other/package"
],
"ForbiddenPrefixes": [
"k8s.io/kubernetes/pkg/third_party/deprecated"
"example.com/legacy/package"
]
},
{
"SelectorRegexp": "^unsafe$",
"AllowedPrefixes": [
],
"ForbiddenPrefixes": [
""
]
"AllowedPrefixes": [],
"ForbiddenPrefixes": [ "" ],
"Transitive": true
}
]
}
```
- Take note of `"SelectorRegexp": "k8s[.]io"` in the first block. This specifies that we are applying these rules to the `"k8s.io"` import path.
- The second block explicitly matches the "unsafe" package, and forbids it ("" is a prefix of everything).
### What are Inverse Rules?
The `SelectorRegexp` specifies that this rule applies only to imports which
match that regex.
- In contrast to non-inverse rules, which are defined in importing packages, inverse rules are defined in imported packages.
Note: an empty list (`[]`) matches nothing, and an empty string (`""`) is a
prefix of everything.
- Inverse rules allow for fine-grained import restrictions for "private packages" where we don't want to spread use inside of [kubernetes/kubernetes](https://github.com/kubernetes/kubernetes).
### What are InverseRules?
- If an `.import-restrictions` file is found, then all imports of the package are checked against each `inverse rule` in the file. This check will continue, climbing up the directory tree, until a match is found and accepted.
In contrast to rules, which are defined in terms of "things this package
depends on", inverse rules are defined in terms of "things which import this
package". This allows for fine-grained import restrictions for "semi-private
packages" which are more sophisticated than Go's `internal` convention.
- Inverse rules also have a boolean `transitive` option. When this option is true, the import rule is also applied to `transitive` imports.
- `transitive` imports are dependencies not directly depended on by the code, but are needed to run the application. Use this option if you want to apply restrictions to those indirect dependencies.
If inverse rules are found, then all known imports of the package are checked
against each such rule, in the same fashion as regular rules. Note that this
can only handle known imports, which is defined as any package which is also
being considered by this `import-boss` run. For most repositories, `./...` will
suffice.
Example:
```yaml
rules:
- selectorRegexp: k8s[.]io
allowedPrefixes:
- k8s.io/gengo/v2/examples
- k8s.io/kubernetes/third_party
forbiddenPrefixes:
- k8s.io/kubernetes/pkg/third_party/deprecated
- selectorRegexp: ^unsafe$
forbiddenPrefixes:
- ""
inverseRules:
- selectorRegexp: k8s[.]io
- selectorRegexp: example[.]com
allowedPrefixes:
- k8s.io/same-repo
- k8s.io/kubernetes/pkg/legacy
- example.com/this-same-repo
- example.com/close-friend/legacy
forbiddenPrefixes:
- k8s.io/kubernetes/pkg/legacy/subpkg
- selectorRegexp: k8s[.]io
- example.com/other-project
- selectorRegexp: example[.]com
transitive: true
forbiddenPrefixes:
- k8s.io/kubernetes/cmd/kubelet
- k8s.io/kubernetes/cmd/kubectl
- example.com/other-team
```
## How do I run import-boss within the k/k repo?
## How do I run import-boss?
- In order to include _test.go files, make sure to pass in the `include-test-files` flag:
```sh
hack/verify-import-boss.sh --include-test-files=true
```
- To include other directories, pass in a directory or directories using the `input-dirs` flag:
```sh
hack/verify-import-boss.sh --input-dirs="k8s.io/kubernetes/test/e2e/framework/..."
```
## Reference
- [import-boss](https://github.com/kubernetes/gengo/tree/master/examples/import-boss)
For most scenarios, simply running `import-boss ./...` will work. For projects
which use Go workspaces, this can even span multiple modules.

View File

@ -21,32 +21,565 @@ import (
"flag"
"os"
"github.com/spf13/pflag"
"k8s.io/gengo/v2/args"
"k8s.io/gengo/v2/examples/import-boss/generators"
"errors"
"fmt"
"path/filepath"
"regexp"
"sort"
"strings"
"time"
"github.com/spf13/pflag"
"golang.org/x/tools/go/packages"
"k8s.io/klog/v2"
"sigs.k8s.io/yaml"
)
const (
rulesFileName = ".import-restrictions"
goModFile = "go.mod"
)
func main() {
klog.InitFlags(nil)
arguments := args.Default()
pflag.CommandLine.BoolVar(&arguments.IncludeTestFiles, "include-test-files", false, "If true, include *_test.go files.")
// Collect and parse flags.
arguments.AddFlags(pflag.CommandLine)
pflag.CommandLine.AddGoFlagSet(flag.CommandLine)
pflag.Parse()
if err := arguments.Execute(
generators.NameSystems(),
generators.DefaultNameSystem(),
generators.GetTargets,
"",
); err != nil {
klog.Errorf("Error: %v", err)
pkgs, err := loadPkgs(pflag.Args()...)
if err != nil {
klog.Errorf("failed to load packages: %v", err)
}
pkgs = massage(pkgs)
boss := newBoss(pkgs)
var allErrs []error
for _, pkg := range pkgs {
if pkgErrs := boss.Verify(pkg); pkgErrs != nil {
allErrs = append(allErrs, pkgErrs...)
}
}
fail := false
for _, err := range allErrs {
if lister, ok := err.(interface{ Unwrap() []error }); ok {
for _, err := range lister.Unwrap() {
fmt.Printf("ERROR: %v\n", err)
}
} else {
fmt.Printf("ERROR: %v\n", err)
}
fail = true
}
if fail {
os.Exit(1)
}
klog.V(2).Info("Completed successfully.")
}
func loadPkgs(patterns ...string) ([]*packages.Package, error) {
cfg := packages.Config{
Mode: packages.NeedName | packages.NeedFiles | packages.NeedImports |
packages.NeedDeps | packages.NeedModule,
Tests: true,
}
klog.V(1).Infof("loading: %v", patterns)
tBefore := time.Now()
pkgs, err := packages.Load(&cfg, patterns...)
if err != nil {
return nil, err
}
klog.V(2).Infof("loaded %d pkg(s) in %v", len(pkgs), time.Since(tBefore))
var allErrs []error
for _, pkg := range pkgs {
var errs []error
for _, e := range pkg.Errors {
if e.Kind == packages.ListError || e.Kind == packages.ParseError {
errs = append(errs, e)
}
}
if len(errs) > 0 {
allErrs = append(allErrs, fmt.Errorf("error(s) in %q: %v", pkg.PkgPath, errors.Join(errs...)))
}
}
if len(allErrs) > 0 {
return nil, errors.Join(allErrs...)
}
return pkgs, nil
}
func massage(in []*packages.Package) []*packages.Package {
out := []*packages.Package{}
for _, pkg := range in {
klog.V(2).Infof("considering pkg: %q", pkg.PkgPath)
// Discard packages which represent the <pkg>.test result. They don't seem
// to hold any interesting source info.
if strings.HasSuffix(pkg.PkgPath, ".test") {
klog.V(3).Infof("ignoring testbin pkg: %q", pkg.PkgPath)
continue
}
// Packages which end in "_test" have tests which use the special "_test"
// package suffix. Packages which have test files must be tests. Don't
// ask me, this is what packages.Load produces.
if strings.HasSuffix(pkg.PkgPath, "_test") || hasTestFiles(pkg.GoFiles) {
// NOTE: This syntax can be undone with unmassage().
pkg.PkgPath = strings.TrimSuffix(pkg.PkgPath, "_test") + " ((tests:" + pkg.Name + "))"
klog.V(3).Infof("renamed to: %q", pkg.PkgPath)
}
out = append(out, pkg)
}
return out
}
func unmassage(str string) string {
idx := strings.LastIndex(str, " ((")
if idx == -1 {
return str
}
return str[0:idx]
}
type ImportBoss struct {
// incomingImports holds all the packages importing the key.
incomingImports map[string][]string
// transitiveIncomingImports holds the transitive closure of
// incomingImports.
transitiveIncomingImports map[string][]string
}
func newBoss(pkgs []*packages.Package) *ImportBoss {
boss := &ImportBoss{
incomingImports: map[string][]string{},
transitiveIncomingImports: map[string][]string{},
}
for _, pkg := range pkgs {
// Accumulate imports
for imp := range pkg.Imports {
boss.incomingImports[imp] = append(boss.incomingImports[imp], pkg.PkgPath)
}
}
boss.transitiveIncomingImports = transitiveClosure(boss.incomingImports)
return boss
}
func hasTestFiles(files []string) bool {
for _, f := range files {
if strings.HasSuffix(f, "_test.go") {
return true
}
}
return false
}
func (boss *ImportBoss) Verify(pkg *packages.Package) []error {
pkgDir := packageDir(pkg)
if pkgDir == "" {
// This Package has no usable files, e.g. only tests, which are modelled in
// a distinct Package.
return nil
}
restrictionFiles, err := recursiveRead(filepath.Join(pkgDir, rulesFileName))
if err != nil {
return []error{fmt.Errorf("error finding rules file: %v", err)}
}
if len(restrictionFiles) == 0 {
return nil
}
klog.V(2).Infof("verifying pkg %q (%s)", pkg.PkgPath, pkgDir)
var errs []error
errs = append(errs, boss.verifyRules(pkg, restrictionFiles)...)
errs = append(errs, boss.verifyInverseRules(pkg, restrictionFiles)...)
return errs
}
// packageDir tries to figure out the directory of the specified package.
func packageDir(pkg *packages.Package) string {
if len(pkg.GoFiles) > 0 {
return filepath.Dir(pkg.GoFiles[0])
}
if len(pkg.IgnoredFiles) > 0 {
return filepath.Dir(pkg.IgnoredFiles[0])
}
return ""
}
type FileFormat struct {
Rules []Rule
InverseRules []Rule
path string
}
// A single import restriction rule.
type Rule struct {
// All import paths that match this regexp...
SelectorRegexp string
// ... must have one of these prefixes ...
AllowedPrefixes []string
// ... and must not have one of these prefixes.
ForbiddenPrefixes []string
// True if the rule is to be applied to transitive imports.
Transitive bool
}
// Disposition represents a decision or non-decision.
type Disposition int
const (
// DepForbidden means the dependency was explicitly forbidden by a rule.
DepForbidden Disposition = iota
// DepAllowed means the dependency was explicitly allowed by a rule.
DepAllowed
// DepAllowed means the dependency did not match any rule.
DepUnknown
)
// Evaluate considers this rule and decides if this dependency is allowed.
func (r Rule) Evaluate(imp string) Disposition {
// To pass, an import muct be allowed and not forbidden.
// Check forbidden first.
for _, forbidden := range r.ForbiddenPrefixes {
klog.V(5).Infof("checking %q against forbidden prefix %q", imp, forbidden)
if hasPathPrefix(imp, forbidden) {
klog.V(5).Infof("this import of %q is forbidden", imp)
return DepForbidden
}
}
for _, allowed := range r.AllowedPrefixes {
klog.V(5).Infof("checking %q against allowed prefix %q", imp, allowed)
if hasPathPrefix(imp, allowed) {
klog.V(5).Infof("this import of %q is allowed", imp)
return DepAllowed
}
}
return DepUnknown
}
// recursiveRead collects all '.import-restriction' files, between the current directory,
// and the module root.
func recursiveRead(path string) ([]*FileFormat, error) {
restrictionFiles := make([]*FileFormat, 0)
for {
if _, err := os.Stat(path); err == nil {
rules, err := readFile(path)
if err != nil {
return nil, err
}
restrictionFiles = append(restrictionFiles, rules)
}
nextPath, removedDir := removeLastDir(path)
if nextPath == path || isGoModRoot(path) || removedDir == "src" {
break
}
path = nextPath
}
return restrictionFiles, nil
}
func readFile(path string) (*FileFormat, error) {
currentBytes, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("couldn't read %v: %v", path, err)
}
var current FileFormat
err = yaml.Unmarshal(currentBytes, &current)
if err != nil {
return nil, fmt.Errorf("couldn't unmarshal %v: %v", path, err)
}
current.path = path
return &current, nil
}
// isGoModRoot checks if a directory is the root directory for a package
// by checking for the existence of a 'go.mod' file in that directory.
func isGoModRoot(path string) bool {
_, err := os.Stat(filepath.Join(filepath.Dir(path), goModFile))
return err == nil
}
// removeLastDir removes the last directory, but leaves the file name
// unchanged. It returns the new path and the removed directory. So:
// "a/b/c/file" -> ("a/b/file", "c")
func removeLastDir(path string) (newPath, removedDir string) {
dir, file := filepath.Split(path)
dir = strings.TrimSuffix(dir, string(filepath.Separator))
return filepath.Join(filepath.Dir(dir), file), filepath.Base(dir)
}
func (boss *ImportBoss) verifyRules(pkg *packages.Package, restrictionFiles []*FileFormat) []error {
klog.V(3).Infof("verifying pkg %q rules", pkg.PkgPath)
// compile all Selector regex in all restriction files
selectors := make([][]*regexp.Regexp, len(restrictionFiles))
for i, restrictionFile := range restrictionFiles {
for _, r := range restrictionFile.Rules {
re, err := regexp.Compile(r.SelectorRegexp)
if err != nil {
return []error{
fmt.Errorf("regexp `%s` in file %q doesn't compile: %w", r.SelectorRegexp, restrictionFile.path, err),
}
}
selectors[i] = append(selectors[i], re)
}
}
realPkgPath := unmassage(pkg.PkgPath)
direct, indirect := transitiveImports(pkg)
isDirect := map[string]bool{}
for _, imp := range direct {
isDirect[imp] = true
}
relate := func(imp string) string {
if isDirect[imp] {
return "->"
}
return "-->"
}
var errs []error
for _, imp := range uniq(direct, indirect) {
if unmassage(imp) == realPkgPath {
// Tests in package "foo_test" depend on the test package for
// "foo" (if both exist in a giver directory).
continue
}
klog.V(4).Infof("considering import %q %s %q", pkg.PkgPath, relate(imp), imp)
matched := false
decided := false
for i, file := range restrictionFiles {
klog.V(4).Infof("rules file %s", file.path)
for j, rule := range file.Rules {
if !rule.Transitive && !isDirect[imp] {
continue
}
matching := selectors[i][j].MatchString(imp)
if !matching {
continue
}
matched = true
klog.V(6).Infof("selector %v matches %q", rule.SelectorRegexp, imp)
disp := rule.Evaluate(imp)
if disp == DepAllowed {
decided = true
break // no further rules, next file
} else if disp == DepForbidden {
errs = append(errs, fmt.Errorf("%q %s %q is forbidden by %s", pkg.PkgPath, relate(imp), imp, file.path))
decided = true
break // no further rules, next file
}
}
if decided {
break // no further files, next import
}
}
if matched && !decided {
klog.V(5).Infof("%q %s %q did not match any rule", pkg, relate(imp), imp)
errs = append(errs, fmt.Errorf("%q %s %q did not match any rule", pkg.PkgPath, relate(imp), imp))
}
}
if len(errs) > 0 {
return errs
}
return nil
}
func uniq(slices ...[]string) []string {
m := map[string]bool{}
for _, sl := range slices {
for _, str := range sl {
m[str] = true
}
}
ret := []string{}
for str := range m {
ret = append(ret, str)
}
sort.Strings(ret)
return ret
}
func hasPathPrefix(path, prefix string) bool {
if prefix == "" || path == prefix {
return true
}
if !strings.HasSuffix(path, string(filepath.Separator)) {
prefix = prefix + string(filepath.Separator)
}
return strings.HasPrefix(path, prefix)
}
func transitiveImports(pkg *packages.Package) ([]string, []string) {
direct := []string{}
indirect := []string{}
seen := map[string]bool{}
for _, imp := range pkg.Imports {
direct = append(direct, imp.PkgPath)
dfsImports(&indirect, seen, imp)
}
return direct, indirect
}
func dfsImports(dest *[]string, seen map[string]bool, p *packages.Package) {
for _, p2 := range p.Imports {
if seen[p2.PkgPath] {
continue
}
seen[p2.PkgPath] = true
*dest = append(*dest, p2.PkgPath)
dfsImports(dest, seen, p2)
}
}
// verifyInverseRules checks that all packages that import a package are allowed to import it.
func (boss *ImportBoss) verifyInverseRules(pkg *packages.Package, restrictionFiles []*FileFormat) []error {
klog.V(3).Infof("verifying pkg %q inverse-rules", pkg.PkgPath)
// compile all Selector regex in all restriction files
selectors := make([][]*regexp.Regexp, len(restrictionFiles))
for i, restrictionFile := range restrictionFiles {
for _, r := range restrictionFile.InverseRules {
re, err := regexp.Compile(r.SelectorRegexp)
if err != nil {
return []error{
fmt.Errorf("regexp `%s` in file %q doesn't compile: %w", r.SelectorRegexp, restrictionFile.path, err),
}
}
selectors[i] = append(selectors[i], re)
}
}
realPkgPath := unmassage(pkg.PkgPath)
isDirect := map[string]bool{}
for _, imp := range boss.incomingImports[pkg.PkgPath] {
isDirect[imp] = true
}
relate := func(imp string) string {
if isDirect[imp] {
return "<-"
}
return "<--"
}
var errs []error
for _, imp := range boss.transitiveIncomingImports[pkg.PkgPath] {
if unmassage(imp) == realPkgPath {
// Tests in package "foo_test" depend on the test package for
// "foo" (if both exist in a giver directory).
continue
}
klog.V(4).Infof("considering import %q %s %q", pkg.PkgPath, relate(imp), imp)
matched := false
decided := false
for i, file := range restrictionFiles {
klog.V(4).Infof("rules file %s", file.path)
for j, rule := range file.InverseRules {
if !rule.Transitive && !isDirect[imp] {
continue
}
matching := selectors[i][j].MatchString(imp)
if !matching {
continue
}
matched = true
klog.V(6).Infof("selector %v matches %q", rule.SelectorRegexp, imp)
disp := rule.Evaluate(imp)
if disp == DepAllowed {
decided = true
break // no further rules, next file
} else if disp == DepForbidden {
errs = append(errs, fmt.Errorf("%q %s %q is forbidden by %s", pkg.PkgPath, relate(imp), imp, file.path))
decided = true
break // no further rules, next file
}
}
if decided {
break // no further files, next import
}
}
if matched && !decided {
klog.V(5).Infof("%q %s %q did not match any rule", pkg.PkgPath, relate(imp), imp)
errs = append(errs, fmt.Errorf("%q %s %q did not match any rule", pkg.PkgPath, relate(imp), imp))
}
}
if len(errs) > 0 {
return errs
}
return nil
}
func transitiveClosure(in map[string][]string) map[string][]string {
type edge struct {
from string
to string
}
adj := make(map[edge]bool)
imports := make(map[string]struct{})
for from, tos := range in {
for _, to := range tos {
adj[edge{from, to}] = true
imports[to] = struct{}{}
}
}
// Warshal's algorithm
for k := range in {
for i := range in {
if !adj[edge{i, k}] {
continue
}
for j := range imports {
if adj[edge{i, j}] {
continue
}
if adj[edge{k, j}] {
adj[edge{i, j}] = true
}
}
}
}
out := make(map[string][]string, len(in))
for i := range in {
for j := range imports {
if adj[edge{i, j}] {
out[i] = append(out[i], j)
}
}
sort.Strings(out[i])
}
return out
}

View File

@ -0,0 +1,322 @@
/*
Copyright 2024 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"path/filepath"
"reflect"
"strings"
"testing"
"golang.org/x/tools/go/packages"
)
func TestRemoveLastDir(t *testing.T) {
table := map[string]struct{ newPath, removedDir string }{
"a/b/c": {"a/c", "b"},
}
for slashInput, expect := range table {
input := filepath.FromSlash(slashInput)
gotPath, gotRemoved := removeLastDir(input)
if e, a := filepath.FromSlash(expect.newPath), gotPath; e != a {
t.Errorf("%v: wanted %v, got %v", input, e, a)
}
if e, a := filepath.FromSlash(expect.removedDir), gotRemoved; e != a {
t.Errorf("%v: wanted %v, got %v", input, e, a)
}
}
}
func TestTransitiveClosure(t *testing.T) {
cases := []struct {
name string
in map[string][]string
expected map[string][]string
}{
{
name: "no transition",
in: map[string][]string{
"a": {"b"},
"c": {"d"},
},
expected: map[string][]string{
"a": {"b"},
"c": {"d"},
},
},
{
name: "simple",
in: map[string][]string{
"a": {"b"},
"b": {"c"},
"c": {"d"},
},
expected: map[string][]string{
"a": {"b", "c", "d"},
"b": {"c", "d"},
"c": {"d"},
},
},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
out := transitiveClosure(c.in)
if !reflect.DeepEqual(c.expected, out) {
t.Errorf("expected: %#v, got %#v", c.expected, out)
}
})
}
}
func TestHasTestFiles(t *testing.T) {
cases := []struct {
input []string
expect bool
}{{
input: nil,
expect: false,
}, {
input: []string{},
expect: false,
}, {
input: []string{"foo.go"},
expect: false,
}, {
input: []string{"foo.go", "bar.go"},
expect: false,
}, {
input: []string{"foo_test.go"},
expect: true,
}, {
input: []string{"foo.go", "foo_test.go"},
expect: true,
}, {
input: []string{"foo.go", "foo_test.go", "bar.go", "bar_test.go"},
expect: true,
}}
for _, tc := range cases {
ret := hasTestFiles(tc.input)
if ret != tc.expect {
t.Errorf("expected %v, got %v: %q", tc.expect, ret, tc.input)
}
}
}
func TestPackageDir(t *testing.T) {
cases := []struct {
input *packages.Package
expect string
}{{
input: &packages.Package{
PkgPath: "example.com/foo/bar/qux",
GoFiles: []string{"/src/prj/file.go"},
IgnoredFiles: []string{"/otherdir/file.go"},
},
expect: "/src/prj",
}, {
input: &packages.Package{
PkgPath: "example.com/foo/bar/qux",
IgnoredFiles: []string{"/src/prj/file.go"},
},
expect: "/src/prj",
}, {
input: &packages.Package{
PkgPath: "example.com/foo/bar/qux",
},
expect: "",
}}
for i, tc := range cases {
ret := packageDir(tc.input)
if ret != tc.expect {
t.Errorf("[%d] expected %v, got %v: %q", i, tc.expect, ret, tc.input)
}
}
}
func TestHasPathPrefix(t *testing.T) {
cases := []struct {
base string
pfx string
expect bool
}{{
base: "",
pfx: "",
expect: true,
}, {
base: "/foo/bar",
pfx: "",
expect: true,
}, {
base: "",
pfx: "/foo",
expect: false,
}, {
base: "/foo",
pfx: "/foo",
expect: true,
}, {
base: "/foo/bar",
pfx: "/foo",
expect: true,
}, {
base: "/foobar/qux",
pfx: "/foo",
expect: false,
}, {
base: "/foo/bar/bat/qux/zrb",
pfx: "/foo/bar/bat",
expect: true,
}}
for _, tc := range cases {
ret := hasPathPrefix(tc.base, tc.pfx)
if ret != tc.expect {
t.Errorf("expected %v, got %v: (%q, %q)", tc.expect, ret, tc.base, tc.pfx)
}
}
}
func checkAllErrorStrings(t *testing.T, errs []error, expect []string) {
t.Helper()
if len(errs) != len(expect) {
t.Fatalf("expected %d errors, got %d: %q", len(expect), len(errs), errs)
}
for _, str := range expect {
found := false
for _, err := range errs {
if strings.HasPrefix(err.Error(), str) {
found = true
break
}
}
if !found {
t.Errorf("did not find error %q", str)
t.Logf("\tseek: %s\n\t in:", str)
for _, err := range errs {
t.Logf("\t %s", err.Error())
}
}
}
}
func TestSimpleForward(t *testing.T) {
pkgs, err := loadPkgs("./testdata/simple-fwd/aaa")
if err != nil {
t.Fatalf("unexpected failure: %v", err)
}
if len(pkgs) != 1 {
t.Fatalf("expected 1 pkg result, got %d", len(pkgs))
}
if pkgs[0].PkgPath != "k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/aaa" {
t.Fatalf("wrong PkgPath: %q", pkgs[0].PkgPath)
}
boss := newBoss(pkgs)
errs := boss.Verify(pkgs[0])
expect := []string{
`"k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/aaa" -> "k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/forbidden" is forbidden`,
`"k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/aaa" -> "k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/forbidden/f1" is forbidden`,
`"k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/aaa" -> "k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/neither" did not match any rule`,
`"k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/aaa" -> "k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/neither/n1" did not match any rule`,
}
checkAllErrorStrings(t, errs, expect)
}
func TestNestedForward(t *testing.T) {
pkgs, err := loadPkgs("./testdata/nested-fwd/aaa")
if err != nil {
t.Fatalf("unexpected failure: %v", err)
}
if len(pkgs) != 1 {
t.Fatalf("expected 1 pkg result, got %d", len(pkgs))
}
if pkgs[0].PkgPath != "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/aaa" {
t.Fatalf("wrong PkgPath: %q", pkgs[0].PkgPath)
}
boss := newBoss(pkgs)
errs := boss.Verify(pkgs[0])
expect := []string{
`"k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/aaa" -> "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/forbidden-by-both" is forbidden`,
`"k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/aaa" -> "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/forbidden-by-root" is forbidden`,
`"k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/aaa" -> "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/forbidden-by-sub" is forbidden`,
`"k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/aaa" -> "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/neither/n1" did not match any rule`,
}
checkAllErrorStrings(t, errs, expect)
}
func TestInverse(t *testing.T) {
pkgs, err := loadPkgs("./testdata/inverse/...")
if err != nil {
t.Fatalf("unexpected failure: %v", err)
}
if len(pkgs) != 10 {
t.Fatalf("expected 10 pkg results, got %d", len(pkgs))
}
boss := newBoss(pkgs)
var errs []error
for _, pkg := range pkgs {
errs = append(errs, boss.Verify(pkg)...)
}
expect := []string{
`"k8s.io/code-generator/cmd/import-boss/testdata/inverse/forbidden" <- "k8s.io/code-generator/cmd/import-boss/testdata/inverse/aaa" is forbidden`,
`"k8s.io/code-generator/cmd/import-boss/testdata/inverse/forbidden/f1" <- "k8s.io/code-generator/cmd/import-boss/testdata/inverse/aaa" is forbidden`,
`"k8s.io/code-generator/cmd/import-boss/testdata/inverse/allowed/a2" <- "k8s.io/code-generator/cmd/import-boss/testdata/inverse/allowed" did not match any rule`,
`"k8s.io/code-generator/cmd/import-boss/testdata/inverse/forbidden/f2" <- "k8s.io/code-generator/cmd/import-boss/testdata/inverse/allowed" did not match any rule`,
}
checkAllErrorStrings(t, errs, expect)
}
func TestTransitive(t *testing.T) {
pkgs, err := loadPkgs("./testdata/transitive/...")
if err != nil {
t.Fatalf("unexpected failure: %v", err)
}
if len(pkgs) != 10 {
t.Fatalf("expected 10 pkg results, got %d", len(pkgs))
}
boss := newBoss(pkgs)
var errs []error
for _, pkg := range pkgs {
errs = append(errs, boss.Verify(pkg)...)
}
expect := []string{
`"k8s.io/code-generator/cmd/import-boss/testdata/transitive/forbidden" <- "k8s.io/code-generator/cmd/import-boss/testdata/transitive/aaa" is forbidden`,
`"k8s.io/code-generator/cmd/import-boss/testdata/transitive/forbidden/f1" <- "k8s.io/code-generator/cmd/import-boss/testdata/transitive/aaa" is forbidden`,
`"k8s.io/code-generator/cmd/import-boss/testdata/transitive/forbidden/f2" <-- "k8s.io/code-generator/cmd/import-boss/testdata/transitive/aaa" is forbidden`,
`"k8s.io/code-generator/cmd/import-boss/testdata/transitive/allowed/a2" <- "k8s.io/code-generator/cmd/import-boss/testdata/transitive/allowed" did not match any rule`,
`"k8s.io/code-generator/cmd/import-boss/testdata/transitive/forbidden/f2" <- "k8s.io/code-generator/cmd/import-boss/testdata/transitive/allowed" did not match any rule`,
}
checkAllErrorStrings(t, errs, expect)
}

View File

@ -0,0 +1,12 @@
package aaa
import (
_ "k8s.io/code-generator/cmd/import-boss/testdata/inverse/allowed"
_ "k8s.io/code-generator/cmd/import-boss/testdata/inverse/allowed/a1"
_ "k8s.io/code-generator/cmd/import-boss/testdata/inverse/forbidden"
_ "k8s.io/code-generator/cmd/import-boss/testdata/inverse/forbidden/f1"
_ "k8s.io/code-generator/cmd/import-boss/testdata/inverse/neither"
_ "k8s.io/code-generator/cmd/import-boss/testdata/inverse/neither/n1"
)
var X = "aaa"

View File

@ -0,0 +1,4 @@
inverseRules:
- selectorRegexp: k8s[.]io
allowedPrefixes:
- k8s.io/code-generator/cmd/import-boss/testdata/inverse/aaa

View File

@ -0,0 +1,3 @@
package a1
var X = "a1"

View File

@ -0,0 +1,3 @@
package a2
var X = "a2"

View File

@ -0,0 +1,9 @@
package allowed
import (
_ "k8s.io/code-generator/cmd/import-boss/testdata/inverse/allowed/a2"
_ "k8s.io/code-generator/cmd/import-boss/testdata/inverse/forbidden/f2"
_ "k8s.io/code-generator/cmd/import-boss/testdata/inverse/neither/n2"
)
var X = "allowed"

View File

@ -0,0 +1,4 @@
inverseRules:
- selectorRegexp: k8s[.]io
forbiddenPrefixes:
- k8s.io/code-generator/cmd/import-boss/testdata/inverse/aaa

View File

@ -0,0 +1,3 @@
package f1
var X = "f1"

View File

@ -0,0 +1,3 @@
package f2
var X = "f2"

View File

@ -0,0 +1,3 @@
package forbidden
var X = "forbidden"

View File

@ -0,0 +1,3 @@
package neither
var X = "neither"

View File

@ -0,0 +1,3 @@
package n1
var X = "n1"

View File

@ -0,0 +1,3 @@
package n2
var X = "n2"

View File

@ -0,0 +1,8 @@
rules:
- selectorRegexp: k8s[.]io
allowedPrefixes:
- k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/allowed-by-root
- k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/allowed-by-both
forbiddenPrefixes:
- k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/forbidden-by-root
- k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/forbidden-by-both

View File

@ -0,0 +1,9 @@
rules:
- selectorRegexp: k8s[.]io
allowedPrefixes:
- k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/bbb
- k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/allowed-by-sub
- k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/allowed-by-both
forbiddenPrefixes:
- k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/forbidden-by-sub
- k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/forbidden-by-both

View File

@ -0,0 +1,14 @@
package aaa
import (
_ "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/allowed-by-both"
_ "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/allowed-by-root"
_ "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/allowed-by-sub"
_ "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/bbb"
_ "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/forbidden-by-both"
_ "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/forbidden-by-root"
_ "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/forbidden-by-sub"
_ "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/neither/n1"
)
var X = "aaa"

View File

@ -0,0 +1,3 @@
package allowedbyboth
var X = "allowedbyboth"

View File

@ -0,0 +1,3 @@
package allowedbyroot
var X = "allowedbyroot"

View File

@ -0,0 +1,3 @@
package allowedbysub
var X = "allowedbysub"

View File

@ -0,0 +1,13 @@
package bbb
import (
_ "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/allowed-by-both"
_ "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/allowed-by-root"
_ "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/allowed-by-sub"
_ "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/forbidden-by-both"
_ "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/forbidden-by-root"
_ "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/forbidden-by-sub"
_ "k8s.io/code-generator/cmd/import-boss/testdata/nested-fwd/neither/n2"
)
var X = "bbb"

View File

@ -0,0 +1,3 @@
package forbiddenbyboth
var X = "forbiddenbyboth"

View File

@ -0,0 +1,3 @@
package forbiddenbyroot
var X = "forbiddenbyroot"

View File

@ -0,0 +1,3 @@
package forbiddenbysub
var X = "forbiddenbysub"

View File

@ -0,0 +1,3 @@
package n1
var X = "n1"

View File

@ -0,0 +1,3 @@
package n2
var X = "n2"

View File

@ -0,0 +1,6 @@
rules:
- selectorRegexp: k8s[.]io
allowedPrefixes:
- k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/allowed
forbiddenPrefixes:
- k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/forbidden

View File

@ -0,0 +1,12 @@
package aaa
import (
_ "k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/allowed"
_ "k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/allowed/a1"
_ "k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/forbidden"
_ "k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/forbidden/f1"
_ "k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/neither"
_ "k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/neither/n1"
)
var X = "aaa"

View File

@ -0,0 +1,3 @@
package a1
var X = "a1"

View File

@ -0,0 +1,3 @@
package a2
var X = "a2"

View File

@ -0,0 +1,9 @@
package allowed
import (
_ "k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/allowed/a2"
_ "k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/forbidden/f2"
_ "k8s.io/code-generator/cmd/import-boss/testdata/simple-fwd/neither/n2"
)
var X = "allowed"

View File

@ -0,0 +1,3 @@
package f1
var X = "f1"

View File

@ -0,0 +1,3 @@
package f2
var X = "f2"

View File

@ -0,0 +1,3 @@
package forbidden
var X = "forbidden"

View File

@ -0,0 +1,3 @@
package neither
var X = "neither"

View File

@ -0,0 +1,3 @@
package n1
var X = "n1"

View File

@ -0,0 +1,3 @@
package n2
var X = "n2"

View File

@ -0,0 +1,12 @@
package aaa
import (
_ "k8s.io/code-generator/cmd/import-boss/testdata/transitive/allowed"
_ "k8s.io/code-generator/cmd/import-boss/testdata/transitive/allowed/a1"
_ "k8s.io/code-generator/cmd/import-boss/testdata/transitive/forbidden"
_ "k8s.io/code-generator/cmd/import-boss/testdata/transitive/forbidden/f1"
_ "k8s.io/code-generator/cmd/import-boss/testdata/transitive/neither"
_ "k8s.io/code-generator/cmd/import-boss/testdata/transitive/neither/n1"
)
var X = "aaa"

View File

@ -0,0 +1,5 @@
inverseRules:
- selectorRegexp: k8s[.]io
allowedPrefixes:
- k8s.io/code-generator/cmd/import-boss/testdata/transitive/aaa
transitive: true

View File

@ -0,0 +1,3 @@
package a1
var X = "a1"

View File

@ -0,0 +1,3 @@
package a2
var X = "a2"

View File

@ -0,0 +1,9 @@
package allowed
import (
_ "k8s.io/code-generator/cmd/import-boss/testdata/transitive/allowed/a2"
_ "k8s.io/code-generator/cmd/import-boss/testdata/transitive/forbidden/f2"
_ "k8s.io/code-generator/cmd/import-boss/testdata/transitive/neither/n2"
)
var X = "allowed"

View File

@ -0,0 +1,5 @@
inverseRules:
- selectorRegexp: k8s[.]io
forbiddenPrefixes:
- k8s.io/code-generator/cmd/import-boss/testdata/transitive/aaa
transitive: true

View File

@ -0,0 +1,3 @@
package f1
var X = "f1"

View File

@ -0,0 +1,3 @@
package f2
var X = "f2"

View File

@ -0,0 +1,3 @@
package forbidden
var X = "forbidden"

View File

@ -0,0 +1,3 @@
package neither
var X = "neither"

View File

@ -0,0 +1,3 @@
package n1
var X = "n1"

View File

@ -0,0 +1,3 @@
package n2
var X = "n2"