Commit Graph

121397 Commits

Author SHA1 Message Date
Tim Hockin
5ddd6772a4
Fix applyconfiguration-gen wrt gengo/v2 2024-02-29 22:06:33 -08:00
Tim Hockin
9c62235266
Make update-codegen informer work on gengo/v2 2024-02-29 22:06:32 -08:00
Tim Hockin
0aa98ee8e3
Fix informer-gen wrt gengo/v2 2024-02-29 22:06:31 -08:00
Tim Hockin
46665fa76a
Make update-codegen lister work on gengo/v2 2024-02-29 22:06:31 -08:00
Tim Hockin
b82527b6fb
Fix lister-gen wrt gengo/v2 2024-02-29 22:06:30 -08:00
Tim Hockin
1965f56f59
Make update-codegen openapi work on gengo/v2 2024-02-29 22:06:29 -08:00
Tim Hockin
5475797f43
Make update-codegen conversion work on gengo/v2 2024-02-29 22:06:28 -08:00
Tim Hockin
ed35118664
Fix conversion-gen wrt gengo/v2 2024-02-29 22:06:27 -08:00
Tim Hockin
1e96f4e50a
Make update-codegen prerelease work on gengo/v2 2024-02-29 22:06:26 -08:00
Tim Hockin
fd2ca0a344
Fix prerelease-lifecycle-gen wrt gengo/v2 2024-02-29 22:06:25 -08:00
Tim Hockin
b5b8c9d869
Make update-codegen swagger work on gengo/v2 2024-02-29 22:06:25 -08:00
Tim Hockin
8420278c7f
Make update-codegen defaulter work on gengo/v2 2024-02-29 22:06:24 -08:00
Tim Hockin
1fb4e824ba
Make update-codegen deepcopy work on gengo/v2 2024-02-29 22:06:23 -08:00
Tim Hockin
d5e8e9c29c
Change update-codegen.sh to use new setup_env
This breaks all the generators, which will be fixed subsequently.
2024-02-29 22:06:22 -08:00
Tim Hockin
db590ecd89
Fix scripts to avoid testdata 2024-02-29 22:06:21 -08:00
Tim Hockin
514dd0e1f6
Convert all gengo references to gengo/v2
TO REPEAT:

    git ls-files ':!:vendor/k8s.io/gengo' ':!:gengo-v2' ':(glob)**/*.go' \
        | xargs grep -l 'k8s.io/gengo/' \
        | while read F; do \
            sed -i 's|k8s.io/gengo/\([^v]\)|k8s.io/gengo/v2/\1|' $F
        done
2024-02-29 22:06:20 -08:00
Tim Hockin
6f2f3735e0
Re-vendor latest kube-openapi and gengo/v2
./hack/pin-dependency.sh k8s.io/kube-openapi latest
./hack/pin-dependency.sh k8s.io/gengo/v2 latest
./hack/update-vendor.sh
2024-02-29 22:06:19 -08:00
Tim Hockin
b3dd724174
Remove dep on set-gen: gone in gengo/v2 2024-02-29 22:06:18 -08:00
Tim Hockin
487cd02711
Change verify-generated to new setup_env 2024-02-29 22:06:18 -08:00
Tim Hockin
c79e5abec4
Change test-e2e-node to use new setup_env 2024-02-29 22:06:17 -08:00
Tim Hockin
7a7dff1eeb
Change verify-imports to use new setup_env
Also clean up the tool a bit and drop vendor as an explicit thing to
be handled.
2024-02-29 22:06:16 -08:00
Tim Hockin
3be358ecc1
Make verify-conformance-requirements use new setup_env 2024-02-29 22:06:15 -08:00
Tim Hockin
08622aa685
Make verify-no-vendor-cycles use new setup_env 2024-02-29 22:06:14 -08:00
Tim Hockin
854037d5b5
Change update-generated-api-compatibility-data test to new setup_env 2024-02-29 22:06:13 -08:00
Tim Hockin
25b97dae82
Change ipamperf test to new setup_env 2024-02-29 22:06:12 -08:00
Tim Hockin
238b33de3b
Change test/instrumentation/update-* to new setup_env 2024-02-29 22:06:12 -08:00
Tim Hockin
cbc887a1d6
Change verify-govulncheck to new setup_env 2024-02-29 22:06:11 -08:00
Tim Hockin
024f4e9c09
Change verify-generated-docs to new setup_env 2024-02-29 22:06:10 -08:00
Tim Hockin
35aa5b3c69
Change verify-fieldname-docs to new setup_env 2024-02-29 22:06:09 -08:00
Tim Hockin
0d30f918fa
Change verify-description to new setup_env 2024-02-29 22:06:08 -08:00
Tim Hockin
bad67dc973
Fix golangci-lint for workspaces 2024-02-29 22:06:07 -08:00
Tim Hockin
3163fc4f69
Change update-golangci-lint-config to new setup_env 2024-02-29 22:06:06 -08:00
Tim Hockin
714f8a0855
Change verify-yamlfmt to new setup_env 2024-02-29 22:06:06 -08:00
Tim Hockin
3dc447708b
Change verify-gofmt to new setup_env 2024-02-29 22:06:05 -08:00
Tim Hockin
d785f13c05
Change verify-licenses to new setup_env 2024-02-29 22:06:04 -08:00
Tim Hockin
974276e4a0
Change update-openapi-spec to new setup_env 2024-02-29 22:06:03 -08:00
Tim Hockin
2ee339b023
Simplify verify-cli-conventions; use new setup_env 2024-02-29 22:06:02 -08:00
Tim Hockin
2924b90d75
Change verify-api-groups.sh to s/vendor/staging/
I'm really not sure what this particular file check is doing, but that's
for tomorrow.
2024-02-29 22:06:01 -08:00
Tim Hockin
b725fd20c2
Fix update/verify-mocks.sh
There appears to be a bug in `go generate` for workspaces which will be
fixed in the 1.22.1 release.
2024-02-29 22:06:00 -08:00
Tim Hockin
58ab5eea89
Remove some references to GOPATH 2024-02-29 22:05:59 -08:00
Tim Hockin
5f490b4fe8
Clarify, document KUBE_OUTPUT_* variables 2024-02-29 22:05:42 -08:00
Tim Hockin
f47e0cb779
Rename KUBE_OUTPUT_BINPATH -> KUBE_OUTPUT_BIN
"...PATH" doesn't add anything and confuses it with KUBE_OUTPUT_SUBPATH.
2024-02-29 00:22:14 -08:00
Tim Hockin
aaf6fc07d8
Make build scripts use Go import paths
This helps 2 things:
1) output is cleaner
2) whether a build is static or non-static depends on string matching,
   which is easier after this

If you specify a `...` pattern, this will preserve it, which breaks the
static/non-static matching but really, can we just agree that `make` is
for binaries?  It works for other things but so does `go build`.

Example:

```
$ make WHAT="cmd/kubectl/ ./staging/src/k8s.io/api/... ./test/e2e/e2e.test ./test/e2e/e2e.test ./cmd/kubectl/ "
go version go1.21.4 linux/amd64
+++ [1208 22:24:43] Building go targets for linux/amd64
    k8s.io/api/... (non-static)
    k8s.io/kubernetes/cmd/kubectl (static)
    k8s.io/kubernetes/test/e2e/e2e.test (test)
```
2024-02-29 00:22:13 -08:00
Tim Hockin
f3c8e92def
Clean up test.sh and get rid of GOPATH
Result: `make test` works:

```
$ #### BUILD ####

$ make WHAT="./cmd/kubectl/ ./staging/src/k8s.io/api"
+++ [1211 11:20:46] Building go targets for linux/amd64
    ./cmd/kubectl/ (non-static)
    ./staging/src/k8s.io/api (non-static)

$ #### TEST ####

$ make test WHAT=./pkg/proxy/iptables/
+++ [1211 11:22:31] Set GOMAXPROCS automatically to 6
+++ [1211 11:22:31] Running tests without code coverage and with -race
ok  	k8s.io/kubernetes/pkg/proxy/iptables	9.517s

$ make test WHAT=pkg/proxy/iptables/
+++ [1211 11:22:53] Set GOMAXPROCS automatically to 6
+++ [1211 11:22:53] Running tests without code coverage and with -race
ok  	k8s.io/kubernetes/pkg/proxy/iptables	(cached)

$ make test WHAT=k8s.io/kubernetes/pkg/proxy/iptables/
+++ [1211 11:23:09] Set GOMAXPROCS automatically to 6
+++ [1211 11:23:09] Running tests without code coverage and with -race
ok  	k8s.io/kubernetes/pkg/proxy/iptables	(cached)

$ make test WHAT=./staging/src/k8s.io/api
+++ [1211 11:23:24] Set GOMAXPROCS automatically to 6
+++ [1211 11:23:24] Running tests without code coverage and with -race
ok  	k8s.io/api	21.981s

$ make test WHAT=staging/src/k8s.io/api
+++ [1211 11:23:54] Set GOMAXPROCS automatically to 6
+++ [1211 11:23:54] Running tests without code coverage and with -race
ok  	k8s.io/api	(cached)

$ make test WHAT=k8s.io/api
+++ [1211 11:24:06] Set GOMAXPROCS automatically to 6
+++ [1211 11:24:06] Running tests without code coverage and with -race
ok  	k8s.io/api	(cached)
```
2024-02-29 00:22:13 -08:00
Tim Hockin
8cb8535d9c
Excise KUBE_GOPATH from coverage build 2024-02-29 00:22:12 -08:00
Tim Hockin
0af3106442
Dedup binaries when building 2024-02-29 00:22:11 -08:00
Tim Hockin
9256031a87
Workspaces: excise vendor from golang.sh
```
$ make
go version go1.20.4 linux/amd64
+++ [0506 13:21:33] Building go targets for linux/amd64
    ./cmd/kube-proxy (static)
    ./cmd/kube-apiserver (static)
    ./cmd/kube-controller-manager (static)
    ./cmd/kubelet (non-static)
    ./cmd/kubeadm (static)
    ./cmd/kube-scheduler (static)
    ./staging/src/k8s.io/component-base/logs/kube-log-runner (static)
    ./staging/src/k8s.io/kube-aggregator (static)
    ./staging/src/k8s.io/apiextensions-apiserver (static)
    ./cluster/gce/gci/mounter (non-static)
    ./cmd/kubectl (static)
    ./cmd/kubectl-convert (static)
    github.com/onsi/ginkgo/v2/ginkgo (non-static)
    ./test/e2e/e2e.test (test)
    ./test/conformance/image/go-runner (non-static)
    ./cmd/kubemark (static)
    github.com/onsi/ginkgo/v2/ginkgo (non-static)
    ./test/e2e_node/e2e_node.test (test)
```

before:
real	4m31.100s
user	21m1.466s
sys	2m23.546s

after:
real	3m46.591s
user	16m34.665s
sys	2m1.553s
2024-02-29 00:22:10 -08:00
Tim Hockin
df42d56947
Remove vendor from ldflags 2024-02-29 00:22:09 -08:00
Tim Hockin
81d932906a
Make golang::binaries_from_targets smarter
and use it from test

Result: `make` works, but `make test` still broken

```
$ make kubectl
+++ [1211 11:15:46] Building go targets for linux/amd64
    ./cmd/kubectl (static)

$ make WHAT=./cmd/kubectl/
+++ [1211 11:15:53] Building go targets for linux/amd64
    ./cmd/kubectl/ (non-static)

$ make WHAT=cmd/kubectl/
+++ [1211 11:16:02] Building go targets for linux/amd64
    ./cmd/kubectl/ (non-static)

$ make WHAT=k8s.io/kubernetes/cmd/kubectl
+++ [1211 11:16:16] Building go targets for linux/amd64
    k8s.io/kubernetes/cmd/kubectl (static)

$ make WHAT=./staging/src/k8s.io/api
+++ [1211 11:16:31] Building go targets for linux/amd64
    ./staging/src/k8s.io/api (non-static)

$ make WHAT=staging/src/k8s.io/api
+++ [1211 11:16:43] Building go targets for linux/amd64
    ./staging/src/k8s.io/api (non-static)

$ make WHAT=k8s.io/api
+++ [1211 11:16:50] Building go targets for linux/amd64
    k8s.io/api (non-static)
```

```
$ make test WHAT=./cmd/kubectl
+++ [1211 11:17:23] Set GOMAXPROCS automatically to 6
+++ [1211 11:17:23] Running tests without code coverage and with -race
cmd/kubectl/kubectl.go:25:2: cannot find package "k8s.io/client-go/plugin/pkg/client/auth" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/client-go/plugin/pkg/client/auth (vendor tree)
	/home/thockin/sdk/gotip/src/k8s.io/client-go/plugin/pkg/client/auth (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/client-go/plugin/pkg/client/auth (from $GOPATH)
cmd/kubectl/kubectl.go:20:2: cannot find package "k8s.io/component-base/cli" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/component-base/cli (vendor tree)
	/home/thockin/sdk/gotip/src/k8s.io/component-base/cli (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/component-base/cli (from $GOPATH)
cmd/kubectl/kubectl.go:21:2: cannot find package "k8s.io/kubectl/pkg/cmd" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd (vendor tree)
	/home/thockin/sdk/gotip/src/k8s.io/kubectl/pkg/cmd (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubectl/pkg/cmd (from $GOPATH)
cmd/kubectl/kubectl.go:22:2: cannot find package "k8s.io/kubectl/pkg/cmd/util" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util (vendor tree)
	/home/thockin/sdk/gotip/src/k8s.io/kubectl/pkg/cmd/util (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubectl/pkg/cmd/util (from $GOPATH)
make: *** [Makefile:191: test] Error 1
```
2024-02-29 00:22:08 -08:00
Tim Hockin
8b579b2347
Change the build_binaries path to use modules
This makes "new" and "old" setup_env functions. In subsequent commits,
all callers of the "old" form will be fixed, and the "new" will be
renamed back.

The old and new functions diff:

```diff
--- /tmp/a	2023-12-14 09:02:57.804092696 -0800
+++ /tmp/b	2023-12-14 09:03:09.679999585 -0800
@@ -1,4 +1,4 @@
-kube::golang::old::setup_env() {
+kube::golang:🆕:setup_env() {
   kube::golang::verify_go_version

   # Set up GOPATH.  We have tools which depend on being in a GOPATH (see
@@ -7,9 +7,9 @@
   # Even in module mode, we need to set GOPATH for `go build` and `go install`
   # to work.  We build various tools (usually via `go install`) from a lot of
   # scripts.
-  #   * We can't set GOBIN because that does not work on cross-compiles.
-  #   * We could use `go build -o <something>`, but it's subtle when it comes
-  #     to cross-compiles and whether the <something> is a file or a directory,
+  #   * We can't just set GOBIN because that does not work on cross-compiles.
+  #   * We could always use `go build -o <something>`, but it's subtle wrt
+  #     cross-compiles and whether the <something> is a file or a directory,
   #     and EVERY caller has to get it *just* right.
   #   * We could leave GOPATH alone and let `go install` write binaries
   #     wherever the user's GOPATH says (or doesn't say).
@@ -20,16 +20,6 @@
   #
   # Eventually, when we no longer rely on run-in-gopath.sh we may be able to
   # simplify this some.
-  local go_pkg_dir="${KUBE_GOPATH}/src/${KUBE_GO_PACKAGE}"
-  local go_pkg_basedir
-  go_pkg_basedir=$(dirname "${go_pkg_dir}")
-
-  mkdir -p "${go_pkg_basedir}"
-
-  # TODO: This symlink should be relative.
-  if [[ ! -e "${go_pkg_dir}" || "$(readlink "${go_pkg_dir}")" != "${KUBE_ROOT}" ]]; then
-    ln -snf "${KUBE_ROOT}" "${go_pkg_dir}"
-  fi
   export GOPATH="${KUBE_GOPATH}"

   # If these are not set, set them now.  This ensures that any subsequent
@@ -40,24 +30,10 @@
   # Make sure our own Go binaries are in PATH.
   export PATH="${KUBE_GOPATH}/bin:${PATH}"

-  # Change directories so that we are within the GOPATH.  Some tools get really
-  # upset if this is not true.  We use a whole fake GOPATH here to collect the
-  # resultant binaries.
-  local subdir
-  subdir=$(kube::realpath . | sed "s|${KUBE_ROOT}||")
-  cd "${KUBE_GOPATH}/src/${KUBE_GO_PACKAGE}/${subdir}" || return 1
-
-  # Set GOROOT so binaries that parse code can work properly.
-  GOROOT=$(go env GOROOT)
-  export GOROOT
-
   # Unset GOBIN in case it already exists in the current session.
   # Cross-compiles will not work with it set.
   unset GOBIN

-  # This seems to matter to some tools
-  export GO15VENDOREXPERIMENT=1
-
-  # Disable workspaces
-  export GOWORK=off
+  # Explicitly turn on modules.
+  export GO111MODULE=on
 }
```

Result: `make` works for k/k:

```
$ make kubectl
+++ [1211 11:07:31] Building go targets for linux/amd64
    k8s.io/kubernetes/cmd/kubectl (static)

$ make WHAT=./cmd/kubectl/
+++ [1211 11:08:19] Building go targets for linux/amd64
    k8s.io/kubernetes/./cmd/kubectl/ (non-static)

$ make WHAT=k8s.io/kubernetes/cmd/kubectl
+++ [1211 11:08:52] Building go targets for linux/amd64
    k8s.io/kubernetes/cmd/kubectl (static)
```

Result: `make` works for staging by package:

```
$ make WHAT=k8s.io/api
+++ [1211 11:11:37] Building go targets for linux/amd64
    k8s.io/api (non-static)
```

Result: `make` fails for staging by path:

```
$ make WHAT=./staging/src/k8s.io/api
+++ [1211 11:12:44] Building go targets for linux/amd64
    k8s.io/kubernetes/./staging/src/k8s.io/api (non-static)
cannot find module providing package k8s.io/kubernetes/staging/src/k8s.io/api: import lookup disabled by -mod=vendor
	(Go version in go.work is at least 1.14 and vendor directory exists.)
!!! [1211 11:12:44] Call tree:
!!! [1211 11:12:44]  1: /home/thockin/src/kubernetes/hack/lib/golang.sh:850 kube::golang::build_some_binaries(...)
!!! [1211 11:12:44]  2: /home/thockin/src/kubernetes/hack/lib/golang.sh:1012 kube::golang::build_binaries_for_platform(...)
!!! [1211 11:12:44]  3: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
!!! [1211 11:12:44] Call tree:
!!! [1211 11:12:44]  1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
!!! [1211 11:12:44] Call tree:
!!! [1211 11:12:44]  1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
make: *** [Makefile:96: all] Error 1
```

Result: `make test` fails:

```
$ make test WHAT=./cmd/kubectl
+++ [1211 11:13:38] Set GOMAXPROCS automatically to 6
+++ [1211 11:13:38] Running tests without code coverage and with -race
cmd/kubectl/kubectl.go:25:2: cannot find package "k8s.io/client-go/plugin/pkg/client/auth" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/client-go/plugin/pkg/client/auth (vendor tree)
	/home/thockin/sdk/gotip/src/k8s.io/client-go/plugin/pkg/client/auth (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/client-go/plugin/pkg/client/auth (from $GOPATH)
cmd/kubectl/kubectl.go:20:2: cannot find package "k8s.io/component-base/cli" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/component-base/cli (vendor tree)
	/home/thockin/sdk/gotip/src/k8s.io/component-base/cli (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/component-base/cli (from $GOPATH)
cmd/kubectl/kubectl.go:21:2: cannot find package "k8s.io/kubectl/pkg/cmd" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd (vendor tree)
	/home/thockin/sdk/gotip/src/k8s.io/kubectl/pkg/cmd (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubectl/pkg/cmd (from $GOPATH)
cmd/kubectl/kubectl.go:22:2: cannot find package "k8s.io/kubectl/pkg/cmd/util" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util (vendor tree)
	/home/thockin/sdk/gotip/src/k8s.io/kubectl/pkg/cmd/util (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubectl/pkg/cmd/util (from $GOPATH)
make: *** [Makefile:191: test] Error 1
```
2024-02-29 00:22:07 -08:00